tkp.db.general – general database functions

A collection of back end subroutines (mostly SQL queries).

In this module we collect together various routines that don’t fit into a more specific collection.


Insert dataset with description as given by argument.

DB function insertDataset() sets the necessary default values.

tkp.db.general.insert_extracted_sources(image_id, results, extract_type, ff_runcat_ids=None, ff_monitor_ids=None)[source]

Insert all detections from sourcefinder into the extractedsource table.

Besides the source properties from sourcefinder, we calculate additional attributes that are increase performance in other tasks.

The strict sequence from results (the sourcefinder detections) is given below. Note the units between sourcefinder and database. (0) ra [deg], (1) dec [deg], (2) ra_fit_err [deg], (3) decl_fit_err [deg], (4) peak_flux [Jy], (5) peak_flux_err [Jy], (6) int_flux [Jy], (7) int_flux_err [Jy], (8) significance detection level, (9) beam major width (arcsec), (10) - minor width (arcsec), (11) - parallactic angle [deg], (12) ew_sys_err [arcsec], (13) ns_sys_err [arcsec], (14) error_radius [arcsec] (15) gaussian fit (bool) (16), (17) chisq, reduced_chisq (float)

ra_fit_err and decl_fit_err are the 1-sigma errors from the gaussian fit, in degrees. Note that for a source located towards the poles the ra_fit_err increases with absolute declination. error_radius is a pessimistic on-sky error estimate in arcsec. ew_sys_err and ns_sys_err represent the telescope dependent systematic errors and are in arcsec. An on-sky error (declination independent, and used in de ruiter calculations) is then: uncertainty_ew^2 = ew_sys_err^2 + error_radius^2 uncertainty_ns^2 = ns_sys_err^2 + error_radius^2 The units of uncertainty_ew and uncertainty_ns are in degrees. The error on RA is given by ra_err. For a source with an RA of ra and an error of ra_err, its RA lies in the range [ra-ra_err, ra+ra_err]. ra_err^2 = ra_fit_err^2 + [alpha_inflate(ew_sys_err,decl)]^2 decl_err^2 = decl_fit_err^2 + ns_sys_err^2. The units of ra_err and decl_err are in degrees. Here alpha_inflate() is the RA inflation function, it converts an angular on-sky distance to a ra distance at given declination.

Input argument “extract” tells whether the source detections originate from: ‘blind’: blind source extraction ‘ff_nd’: from forced fits at null detection locations ‘ff_ms’: from forced fits at monitoringlist positions

Input argument ff_runcat is not empty in the case of forced fits from null detections. It contains the runningcatalog ids from which the source positions were derived for the forced fits. In that case the runcat ids will be inserted into the extractedsource table as well, to simplify further null-detection processing. For blind extractions this list is empty (None).

For all extracted sources additional parameters are calculated, and appended to the sourcefinder data. Appended and converted are:

  • the image id to which the extracted sources belong to
  • the zone in which an extracted source falls is calculated, based on its declination. We adopt a zoneheight of 1 degree, so the floor of the declination represents the zone.
  • the positional errors are converted from degrees to arcsecs
  • the Cartesian coordinates of the source position
  • ra * cos(radians(decl)), this is very often being used in source-distance calculations
tkp.db.general.insert_image(dataset, freq_eff, freq_bw, taustart_ts, tau_time, beam_smaj_pix, beam_smin_pix, beam_pa_rad, deltax, deltay, url, centre_ra, centre_decl, xtr_radius, rms_qc, rms_min, rms_max, detection_thresh, analysis_thresh)[source]

Insert an image for a given dataset.

  • dataset (int) – ID of parent dataset.
  • freq_eff – See Image table definitions.
  • freq_bw – See Image table definitions.
  • taustart_ts – See Image table definitions.
  • taus_time – See Image table definitions.
  • beam_smaj_pix (float) – Restoring beam semimajor axis length in pixels. (Converted to degrees before storing to database).
  • beam_smin_pix (float) – Restoring beam semiminor axis length in pixels. (Converted to degrees before storing to database).
  • beam_pa_rad (float) – Restoring beam parallactic angle in radians. (Converted to degrees before storing to database).
  • deltax (float) – Degrees per pixel increment in x-direction.
  • deltay (float) – Degrees per pixel increment in y-direction.
  • centre_ra (float) – Image central RA co-ord, in degrees.
  • centre_decl (float) – Image central Declination co-ord, in degrees.
  • xtr_radius (float) – Radius in degrees from field centre that will be used for source extraction.
tkp.db.general.insert_monitor_positions(dataset_id, positions)[source]

Add entries to the monitor table.

  • dataset_id (int) – Positions will only be monitored when processing this dataset.
  • positions (list of tuples) – List of (RA, decl) tuples in decimal degrees.

Obtain a light curve for a specific extractedsource

Parameters:xtrsrcid (int) – the source identifier that corresponds to a point on the light curve. Note that the point does not have to be the start (first) point of the light curve.
Returns:a list of tuples, each containing: - observation start time as a datetime.datetime object - integration time (float) - integrated flux (float) - integrated flux error (float) - database ID of this particular source - frequency band ID - stokes
Return type:list

Update dataset start-of-processing timestamp.