tkp.quality – Data-quality control

Data-quality checking related code.

The quality checks are described in the LOFAR Transients Key Science Project Quality Control Document V1.1 and on the wiki.

tkp.quality.restoringbeam

tkp.quality.restoringbeam.beam_invalid(semibmaj, semibmin, theta, oversampled_x=30, elliptical_x=2.0)[source]

Are the beam shape properties ok?

Parameters:semibmaj/semibmin – size of the beam in pixels
Returns:True/False
tkp.quality.restoringbeam.highly_elliptical(semibmaj, semibmin, x=2.0)[source]

If the beam is highly elliptical it can cause source association problems within TraP. Again further testing is required to determine exactly where the cut needs to be.

Parameters:Semibmaj/semibmin – describe the beam size in pixels
Returns:True if the beam is highly elliptical, False otherwise
tkp.quality.restoringbeam.infinite(smaj, smin, bpa)[source]

If the beam is not correctly fitted by AWimager, one or more parameters will be recorded as infinite.

Parameters:
  • smaj – Semi-major axis (arbitrary units)
  • smin – Semi-minor axis
  • bpa – Postion angle
tkp.quality.restoringbeam.not_full_fieldofview(nx, ny, cellsize, fov)[source]

This has been raised as an interesting test, as if the full field of view (FOV) has not been imaged we may want to image the full dataset. The imaged FOV information can be estimated using the number of pixels and the size of the pixels.

Parameters:
  • nx – number of pixels in x direction
  • ny – number of pixels in y direction
Returns:

True if the full FOV is imaged, False otherwise

tkp.quality.restoringbeam.oversampled(semibmaj, semibmin, x=30)[source]

It has been identified that having too many pixels across the restoring beam can lead to bad images, however further testing is required to determine the exact number.

Parameters:Semibmaj/semibmin – describe the beam size in pixels
Returns:True if beam is oversampled, False otherwise
tkp.quality.restoringbeam.undersampled(semibmaj, semibmin)[source]

We want more than 2 pixels across the beam major and minor axes.

Parameters:Semibmaj/semibmin – describe the beam size in pixels
Returns:True if beam is undersampled, False otherwise

tkp.quality.brightsource

tkp.quality.brightsource.check_for_valid_ephemeris(measures)[source]

Checks whether the ephemeris data in use by measures is valid. measures should already have a valid reference frame.

tkp.quality.brightsource.is_bright_source_near(accessor, distance=20)[source]

Checks if there is any of the bright radio sources defined in targets near the center of the image.

Parameters:
  • accessor – a TKP accessor
  • distance – maximum allowed distance of a bright source (in degrees)
Returns:

False if not bright source is near, description of source if a bright source is near

tkp.quality.rms

tkp.quality.rms.clip(data, sigma=3)[source]

Remove all values above a threshold from the array. Uses iterative clipping at sigma value until nothing more is getting clipped. :param data: a numpy array

tkp.quality.rms.reject_historical_rms(image_id, session, history=100, est_sigma=4, rms_max=100.0, rms_min=0.0)[source]

Check if the RMS value of an image lies within a range defined by a gaussian fit on the histogram calculated from the last x RMS values in this subband. Upper and lower bound are then controlled by est_sigma multiplied with the sigma of the gaussian.

Parameters:
  • image_id (int) – database ID of the image we want to check
  • session (sqlalchemy.orm.session.Session) – the database session
  • history (int) – the number of timestamps we want to use for histogram
  • est_sigma (float) – sigma multiplication factor
  • rms_max (float) – global maximum rms for image quality check
  • rms_min (float) – global minimum rms for image quality check
Returns:

None if not rejected, (rejectreason, comment) if rejected

Return type:

bool

tkp.quality.rms.rms(data)[source]

Returns the RMS of the data about the median. :param data: a numpy array

tkp.quality.rms.rms_invalid(rms, noise, low_bound=1, high_bound=50)[source]

Is the RMS value of an image outside the plausible range?

Parameters:
  • rms – RMS value of an image, can be computed with tkp.quality.statistics.rms
  • noise – Theoretical noise level of instrument, can be calculated with tkp.lofar.noise.noise_level
  • low_bound – multiplied with noise to define lower threshold
  • high_bound – multiplied with noise to define upper threshold
Returns:

True/False

tkp.quality.rms.rms_with_clipped_subregion(data, rms_est_sigma=3, rms_est_fraction=4)[source]

RMS for quality-control.

Root mean square value calculated from central region of an image. We sigma-clip the input-data in an attempt to exclude source-pixels and keep only background-pixels.

Parameters:
  • data – A numpy array
  • rms_est_sigma – sigma value used for clipping
  • rms_est_fraction – determines size of subsection, result will be 1/fth of the image size where f=rms_est_fraction

returns the rms value of a iterative sigma clipped subsection of an image

tkp.quality.rms.subregion(data, f=4)[source]

Returns the inner region of a image, according to f.

Resulting area is 4/(f*f) of the original. :param data: a numpy array