tkp.steps – Define logic for each pipeline stage


tkp.steps.forced_fitting.perform_forced_fits(fit_posns, fit_ids, accessor, extraction_params)[source]

Perform forced source measurements on an image based on a list of positions.

  • fit_posns (tuple) – List of (RA, Dec) tuples: Positions to be fit.
  • fit_ids – List of identifiers for each requested fit position.
  • image_path (str) – path to image for measurements.
  • extraction_params (dict) – source extraction parameters, as a dictionary.

A matched pair of lists (serialized_fits, ids), corresponding to successfully fitted positions. NB returned lists may be shorter than input lists if some fits are unsuccessful.

Return type:



Various subroutines used in the main pipeline flow.

We keep them separately to make the pipeline logic easier to read at a glance.

class tkp.steps.misc.ImageMetadataForSort(url, timestamp, frequency)

Create new instance of ImageMetadataForSort(url, timestamp, frequency)


Alias for field number 2


Alias for field number 1


Alias for field number 0

tkp.steps.misc.check_job_configs_match(job_config_1, job_config_2)[source]

Check if job configs match, except dataset_id which we expect to change.


groups a list of TRAP images per time step.

Per time step the images are order per frequency. We could add other ordering logic (per stoke) later on.

Parameters:metadatas (tuple) – list of ImageMetadataForSort
List of tuples. The list is sorted by timestamp.
Each tuple has the timestamp as a first element, and a list of ImageMetadataForSort sorted by frequency as the second element.
Return type:tuple

Locates the job_params.cfg in ‘job_directory’ and loads via ConfigParser.

tkp.steps.misc.setup_logging(log_dir, debug, use_colorlog, basename='trap')[source]

Sets up logging to stdout, + info/debug level logfiles in log_dir.

  • log_dir (string) – log directory
  • debug (bool) – do we want debug level logging on stdout?
  • basename (string) – basename of the log file


This step is used for the storing of images and metadata to the database

tkp.steps.persistence.create_dataset(dataset_id, description)[source]

Creates a dataset if it doesn’t exists Note: Should only be used in a master recipe :returns: the database ID of this dataset

tkp.steps.persistence.extract_metadatas(accessors, rms_est_sigma, rms_est_fraction)[source]

Extracts metadata and rms_qc values from the list of images.

  • accessors – list of image accessors
  • rms_est_sigma – used for RMS calculation, see tkp.quality.statistics
  • rms_est_fraction – used for RMS calculation, see tkp.quality.statistics

a list of metadata’s. The metadata will be False if extraction failed.

paths (tuple): list of paths to a astronomical image which can be opened with
Returns:of HDUlist objects
Return type:tuple
tkp.steps.persistence.store_images_in_db(images_metadata, extraction_radius_pix, dataset_id, bandwidth_max)[source]

Add images to database. Note that all images in one dataset should be inserted in one go, since the order is very important here. If you don’t add them all in once, you should make sure they are added in the correct order e.g. sorted by observation time.

Note: Should only be used in a master recipe

  • images_metadata – list of dicts containing image metadata
  • extraction_radius_pix – (float) Used to calculate the ‘skyregion’
  • dataset_id – dataset id to be used. don’t use value from parset file since this can be -1 (TraP way of setting auto increment)

the database ID of this dataset


Pretty print the classified transients


All generic quality checking routines.

tkp.steps.quality.reject_check(accessor, job_config)[source]

Check if an image passes the quality checks.

If not, a rejection reason is returned.

  • accessor – a TKP accessor representing the image
  • job_config – parset file location with quality check parameters

(rejection ID, description) if rejected, else None

tkp.steps.quality.reject_image(image_id, reason, comment)[source]

Adds a rejection for an image to the database


class tkp.steps.source_extraction.ExtractionResults(sources, rms_min, rms_max)

Create new instance of ExtractionResults(sources, rms_min, rms_max)


Alias for field number 2


Alias for field number 1


Alias for field number 0

tkp.steps.source_extraction.extract_sources(accessor, extraction_params)[source]

Extract sources from an image.


a tuple of image DB object and accessor

extraction_params: dictionary containing at least the detection and
analysis threshold and the association radius, the last one a multiplication factor of the de Ruiter radius.
Returns:list of ExtractionResults named tuples containing source measurements, min RMS value and max RMS value