tkp.testutil – helper functions for writing tests


A collection of back end db query subroutines used for unittesting

tkp.testutil.db_queries.convert_to_cartesian(conn, ra, decl)[source]

Returns tuple (x,y,z)

tkp.testutil.db_queries.get_assoc_entries(db, runcat_id)[source]

Return the full history of variability indices for a runcat entry, ordered by time.


class tkp.testutil.db_subs.ExtractedSourceTuple(ra, dec, ra_fit_err, dec_fit_err, peak, peak_err, flux, flux_err, sigma, beam_maj, beam_min, beam_angle, ew_sys_err, ns_sys_err, error_radius, fit_type, chisq, reduced_chisq)

Return self as a plain tuple. Used by copy and pickle.


Exclude the OrderedDict from pickling


Return a nicely formatted representation string


Alias for field number 11


Alias for field number 9


Alias for field number 10


Alias for field number 16


Alias for field number 1


Alias for field number 3


Alias for field number 14


Alias for field number 12


Alias for field number 15


Alias for field number 6


Alias for field number 7


Alias for field number 13


Alias for field number 4


Alias for field number 5


Alias for field number 0


Alias for field number 2


Alias for field number 17


Alias for field number 8

class tkp.testutil.db_subs.MockSource(template_extractedsource, lightcurve)[source]

Defines a MockSource for generating mock source lists.

(These can be used to test the database routines.)

The lightcurve-dict entries define the times of non-zero flux (we do not support time-ranges here, discretely defined datapoints are sufficiently complex for the current unit-test suite). In this case, any undefined datetimes requested will produce a zero-flux measurement. A defaultdict may be supplied to simulate a steady-flux source.

  • template_extractedsource (ExtractedSourceTuple) – This defines everything except the flux and significance of the extraction (i.e. position, fit error, beam properties, etc.).
  • lightcurve (dict) –

    A dict mapping datetime -> flux value [Jy]. Any undefined datetimes will produce a zero-flux measurement. A defaultdict with constant-valued default may be supplied to represent a steady source, e.g.

    >>>MockSource(base_source, defaultdict(lambda:steady_flux_val))
simulate_extraction(db_image, extraction_type, rms_attribute='rms_min')[source]

Simulate extraction process, returns extracted source or none.

Uses the database image properties (extraction region, rms values) to determine if this source would be extracted in the given image, and return an extraction or None accordingly.

  • db_image (int) – Database Image object.
  • extraction_type – Valid values are ‘blind’, ‘ff_nd’. If ‘blind’ then we only return an extracted source if the flux is above rms_value * detection_threshold.
  • rms_attribute (str) – Valid values are ‘rms_min’, ‘rms_max’. Determines which rms value we use when deciding if this source will be seen in a blind extraction.

ExtractedSourceTuple or None.

value_at_dtime(dtime, image_rms)[source]

Returns an extractedsource for a given datetime.

If lightcurve is defined but does not contain the requested datetime, then peak, flux, sigma are all set to zero.

tkp.testutil.db_subs.deRuiter_radius(src1, src2)[source]

Calculates the De Ruiter radius for two sources


Use with caution!

NB. Not the same as a freshly initialised database.
All the sequence counters are offset.

Defines the canonical default image-data for unit-testing the database.

By defining this in one place we make it simple to make changes. A subset of the default values may be overridden by passing the keys as keyword-args.

Note that while RA, Dec and extraction radius are arbitrary, they should (usually) be close enough and large enough to enclose the RA and Dec of any fake source extractions inserted, since the association routines reject sources outside of designated extraction regions.

tkp.testutil.db_subs.example_extractedsource_tuple(ra=123.123, dec=10.5, ra_fit_err=0.001388888888888889, dec_fit_err=0.0016666666666666668, peak=0.015, peak_err=0.0005, flux=0.015, flux_err=0.0005, sigma=15.0, beam_maj=100.0, beam_min=100.0, beam_angle=45.0, ew_sys_err=20.0, ns_sys_err=20.0, error_radius=10.0, fit_type=0, chisq=5.0, reduced_chisq=1.5)[source]

Generates an example ‘fake extraction’ for unit testing.

Note that while RA and Dec are arbitrary, they should (usually) be close to the RA and Dec of any fake images used, since the association routines reject sources outside of designated extraction regions.

tkp.testutil.db_subs.generate_timespaced_dbimages_data(n_images, timedelta_between_images=datetime.timedelta(1), **kwargs)[source]

Generate a list of image data dictionaries.

The image-data dicts are identical except for having the taustart_ts advanced by a fixed timedelta for each entry.

These can be used to create known entries in the image table, for unit-testing.

A subset of the image-data defaults may be overridden by passing the relevant dictionary values as keyword args.


Returns dicts representing all newsources for this dataset.

Parameters:dsid – Dataset id
Returns:(list of dicts) Each dict represents one newsource. The dict keys are all the columns in the newsources table, plus the ‘taustart_ts’ from the image table, which represents the trigger time.
Return type:list
tkp.testutil.db_subs.get_sources_filtered_by_final_variability(dataset_id, eta_min, v_min)[source]

Search the database to find high-variability lightcurves.

Uses the variability associated with the last datapoint in a lightcurve as the key criteria.

  • dataset_id (int) – Dataset to search
  • eta_min (float) – Minimum value of eta-index to return.
  • v_min (float) – Minimum value of V-index to return.

(list of dicts) Each dict represents a runningcatalog_flux entry matching the filter criteria.

Return type:


tkp.testutil.db_subs.insert_image_and_simulated_sources(dataset, image_params, mock_sources, new_source_sigma_margin, deruiter_radius=3.7)[source]

Simulates the standard database image-and-source insertion logic using mock sources.

  • dataset – The dataset object
  • image_params (dict) – Contains the image properties.
  • mock_sources (list of MockSource) – The mock sources to simulate.
  • new_source_sigma_margin (float) – Parameter passed to source-association routines.
  • deruiter_radius (float) – Parameter passed to source-association routines.

3-tuple (image, list of blind extractions, list of forced fits).


Calculates various metrics for a lightcurve made up of source extractions

These are normally calculated internally in the database - this function serves as a sanity check, and is used for unit-testing purposes.

Returns a list of dictionaries, the nth dict representing the value of the metrics after processing the first n extractions in the lightcurve. The dict keys mirror the column names in the database, to make cross-checking of results trivial.

Final note: this function is very inefficient, recalculating over the first n extractions for each step. We could make it iterative, updating the weighted averages as we do in the database. However, this way provides a stronger cross-check that our iterative SQL approaches are correct - less chance of making the same mistakes in two languages!



Used to disable tests that break Travis due to out-of-memory issues.


This decorator is used to disable tests that do potentially low level database management operations like destroy and create. You can enable these tests by setting the TKP_TESTDBMANAGEMENT environment variable.


Mock data objects for use in testing.