# Variability and new-source detection stage¶

(See also the relevant configuration parameters.)

## Variability index calculation¶

After all images for a given timestep have been processed and the resulting source measurements have been assigned to runningcatalog entries (effectively lightcurves), variability indices are calculated for the most recent timestep, and stored as part of the association recorded in the assocxtrsource table.

Note that a single runningcatalog source may contain entries from multiple independent frequency bands. The variability indices are calculated independently for each frequency band, hence the \(\nu\) suffix in the calculations below denotes an index over the different bands.

For a comprehensive discussion of the transient and variability detection algorithms currently being employed, see Scheers (2011) chapter 3. Here, we provide a brief outline.

We define two metrics for identifying variability in a lightcurve. The flux coefficient of variation, which we denote \(V_\nu\), is defined as

where \(\overline{I_{\nu}}\) is the mean flux of all measurements in the lightcurve at frequency \(\nu\), \(s_{\nu}\) is the standard deviation of those flux measurements and \(N\) is the number of measurements.

The second metric is \(\eta_{\nu}\), which is defined based on reduced \(\chi^2\) statistics as

Where \(\overline{I_{\nu}}^*\) is the average of the flux measurements
weighed by their uncertainties. \(\eta_{\nu}\) is the \(\chi^{2}\)
probability distribution. The probability that the source is “flat” (i.e. has
no significant variability) is then the integral of the distribution from the
measured value of \(\eta_{\nu}\) to \(\infty\); the probability that
it *isn’t* flat is thus 1 minus this quantity.

See also the appendices on the database schema for details of how these are iteratively updated.

## ‘New source’ detection¶

We also attempt to identify any newly extracted sources which we suspect
are intrinsically variable in nature (i.e. they are getting brighter, as
opposed to our observations getting deeper or even simply looking at a
previously unobserved patch of sky). The algorithm for evaluating new
sources is encoded by
`tkp.db.associations._determine_newsource_previous_limits()`

.
Sources we deem to be intrinsically ‘new’ are then recorded in the
newsource table.