tkp.testutil – helper functions for writing tests

tkp.testutil.db_queries

A collection of back end db query subroutines used for unittesting

tkp.testutil.db_queries.convert_to_cartesian(conn, ra, decl)

Returns tuple (x,y,z)

tkp.testutil.db_queries.get_assoc_entries(db, runcat_id)

Return the full history of variability indices for a runcat entry, ordered by time.

tkp.testutil.db_subs

class tkp.testutil.db_subs.ExtractedSourceTuple(ra, dec, ra_fit_err, dec_fit_err, peak, peak_err, flux, flux_err, sigma, beam_maj, beam_min, beam_angle, ew_sys_err, ns_sys_err, error_radius, fit_type, chisq, reduced_chisq)

beam_angle

Alias for field number 11

beam_maj

Alias for field number 9

beam_min

Alias for field number 10

chisq

Alias for field number 16

dec

Alias for field number 1

dec_fit_err

Alias for field number 3

error_radius

Alias for field number 14

ew_sys_err

Alias for field number 12

fit_type

Alias for field number 15

flux

Alias for field number 6

flux_err

Alias for field number 7

ns_sys_err

Alias for field number 13

peak

Alias for field number 4

peak_err

Alias for field number 5

ra

Alias for field number 0

ra_fit_err

Alias for field number 2

reduced_chisq

Alias for field number 17

sigma

Alias for field number 8

class tkp.testutil.db_subs.MockSource(template_extractedsource, lightcurve)

Defines a MockSource for generating mock source lists.

(These can be used to test the database routines.)

The lightcurve-dict entries define the times of non-zero flux (we do not support time-ranges here, discretely defined datapoints are sufficiently complex for the current unit-test suite). In this case, any undefined datetimes requested will produce a zero-flux measurement. A defaultdict may be supplied to simulate a steady-flux source.

Parameters:

• template_extractedsource (ExtractedSourceTuple) – This defines everything except the flux and significance of the extraction (i.e. position, fit error, beam properties, etc.).
• lightcurve (dict) –

  A dict mapping datetime -> flux value [Jy]. Any undefined datetimes will produce a zero-flux measurement. A defaultdict with constant-valued default may be supplied to represent a steady source, e.g.

  >>>MockSource(base_source, defaultdict(lambda:steady_flux_val))

simulate_extraction(db_image, extraction_type, rms_attribute='rms_min')

Simulate extraction process, returns extracted source or none.

Uses the database image properties (extraction region, rms values) to determine if this source would be extracted in the given image, and return an extraction or None accordingly.

Parameters:

• db_image (int) – Database Image object.
• extraction_type – Valid values are ‘blind’, ‘ff_nd’. If ‘blind’ then we only return an extracted source if the flux is above rms_value * detection_threshold.
• rms_attribute (str) – Valid values are ‘rms_min’, ‘rms_max’. Determines which rms value we use when deciding if this source will be seen in a blind extraction.

Returns:

ExtractedSourceTuple or None.

value_at_dtime(dtime, image_rms)

Returns an extractedsource for a given datetime.

If lightcurve is defined but does not contain the requested datetime, then peak, flux, sigma are all set to zero.

tkp.testutil.db_subs.deRuiter_radius(src1, src2)

Calculates the De Ruiter radius for two sources

tkp.testutil.db_subs.delete_test_database(database)

Use with caution!

NB. Not the same as a freshly initialised database.

All the sequence counters are offset.

tkp.testutil.db_subs.example_dbimage_data_dict(**kwargs)

Defines the canonical default image-data for unit-testing the database.

By defining this in one place we make it simple to make changes. A subset of the default values may be overridden by passing the keys as keyword-args.

Note that while RA, Dec and extraction radius are arbitrary, they should (usually) be close enough and large enough to enclose the RA and Dec of any fake source extractions inserted, since the association routines reject sources outside of designated extraction regions.

tkp.testutil.db_subs.example_extractedsource_tuple(ra=123.123, dec=10.5, ra_fit_err=0.001388888888888889, dec_fit_err=0.0016666666666666668, peak=0.015, peak_err=0.0005, flux=0.015, flux_err=0.0005, sigma=15.0, beam_maj=100.0, beam_min=100.0, beam_angle=45.0, ew_sys_err=20.0, ns_sys_err=20.0, error_radius=10.0, fit_type=0, chisq=5.0, reduced_chisq=1.5)

Generates an example ‘fake extraction’ for unit testing.

Note that while RA and Dec are arbitrary, they should (usually) be close to the RA and Dec of any fake images used, since the association routines reject sources outside of designated extraction regions.

tkp.testutil.db_subs.generate_timespaced_dbimages_data(n_images, timedelta_between_images=datetime.timedelta(1), **kwargs)

Generate a list of image data dictionaries.

The image-data dicts are identical except for having the taustart_ts advanced by a fixed timedelta for each entry.

These can be used to create known entries in the image table, for unit-testing.

A subset of the image-data defaults may be overridden by passing the relevant dictionary values as keyword args.

tkp.testutil.db_subs.get_newsources_for_dataset(dsid)

Returns dicts representing all newsources for this dataset.

Parameters:dsid – Dataset id Returns:

(list of dicts) Each dict represents one newsource.

The dict keys are all the columns in the newsources table, plus the ‘taustart_ts’ from the image table, which represents the trigger time.

Return type:list

tkp.testutil.db_subs.get_sources_filtered_by_final_variability(dataset_id, eta_min, v_min)

Search the database to find high-variability lightcurves.

Uses the variability associated with the last datapoint in a lightcurve as the key criteria.

Parameters:

• dataset_id (int) – Dataset to search
• eta_min (float) – Minimum value of eta-index to return.
• v_min (float) – Minimum value of V-index to return.

Returns:

(list of dicts) Each dict represents a runningcatalog_flux entry

matching the filter criteria.

Return type:

list

tkp.testutil.db_subs.insert_image_and_simulated_sources(dataset, image_params, mock_sources, new_source_sigma_margin, deruiter_radius=3.7)

Simulates the standard database image-and-source insertion logic using mock sources.

Parameters:

• dataset – The dataset object
• image_params (dict) – Contains the image properties.
• mock_sources (list of MockSource) – The mock sources to simulate.
• new_source_sigma_margin (float) – Parameter passed to source-association routines.
• deruiter_radius (float) – Parameter passed to source-association routines.

Returns:

3-tuple (image, list of blind extractions, list of forced fits).

tkp.testutil.db_subs.lightcurve_metrics(src_list)

Calculates various metrics for a lightcurve made up of source extractions

These are normally calculated internally in the database - this function serves as a sanity check, and is used for unit-testing purposes.

Returns a list of dictionaries, the nth dict representing the value of the metrics after processing the first n extractions in the lightcurve. The dict keys mirror the column names in the database, to make cross-checking of results trivial.

Final note: this function is very inefficient, recalculating over the first n extractions for each step. We could make it iterative, updating the weighted averages as we do in the database. However, this way provides a stronger cross-check that our iterative SQL approaches are correct - less chance of making the same mistakes in two languages!

tkp.testutil.decorators

tkp.testutil.decorators.high_ram_requirements()

Used to disable tests that break Travis due to out-of-memory issues.

tkp.testutil.decorators.requires_test_db_managed()

This decorator is used to disable tests that do potentially low level database management operations like destroy and create. You can enable these tests by setting the TKP_TESTDBMANAGEMENT environment variable.

tkp.testutil.mock

Mock / synthetic data objects for use in testing.

tkp.testutil.mock.make_wcs(crval=None, cdelt=None, crpix=None)

Make a WCS object for insertion into a synthetic image.

Parameters:

• crval (tuple) – Tuple of (RA, Dec) in decimal degrees at the reference position.
• crpix (tuple) – Tuple of (x,y) co-ordinates describing the reference pixel location corresponding to the crval sky-position.
• cdelt (tuple) – Tuple of (cdelt0, cdelt1) in decimal degrees. This is the pixel width in degrees of arc, but not necessarily aligned to RA, Dec unless crota is (0,0). If that is the case, then typically cdelt0 is negative since the x-axis is in direction of West (decreasing RA).