Testing & Continuous Integration¶
The tests
directory contains a (reasonably) comprehensive TraP test suite.
Tests are written following the Python unittest conventions. Although they
can be run using just the standard Python library tools, you might find that
nose provides a more convenient
interface.
Requirements¶
Whenever any changes are made to the code, all pre-extant tests must continue to pass. If the change involves changing the result of one or more tests, any such changes must be clearly explained and check during Code Review.
New features and bug fixes must be accompanied by appropriate tests. These should both demonstrate the correct operation of the code with good input data as well as demonstrating how it responds to bad inputs. Again, the code reviewer should check that the tests provided are clear and comprehensive.
Test data¶
Many of the tests require data files which are distributed separately from the
TraP. Having cloned the TraP repository (into the directory tkp
in this
example), fetch the appropriate test data as follows:
$ cd tkp
$ git submodule init
$ git submodule update
In future, running git submodule update
again will fetch the latest
version of the test data if required.
By default, the test suite will look for data in the appropriate subdirectory
of your checked out Trap source. However, if the data has been installed
elsewhere, or if you are running the test suite against an installed copy of
the TraP, you can specify the data location by exporting the TKP_TESTPATH
environment variable.
If the data is not available, the relevant tests will be skipped. The partial test suite should still complete successfully; note that your copy of the TraP will not be fully tested.
Database¶
Many of the tests require interaction with the pipeline database. A convient way to configure the database is by using environment variables. For example:
$ export TKP_DBENGINE=xxx
$ export TKP_DBNAME=xxx
$ export TKP_DBHOST=xxx
$ export TKP_DBPORT=xxx
$ export TKP_DBUSER=xxx
$ export TKP_DBPASSWORD=xxx
$ trap-manage.py initdb
If you do not have or need a database, you can disable all the tests which
require it by exporting the variable TKP_DISABLEDB
. The partial test suite
should still complete successfully, but your copy of the TraP will not be
fully tested.
Running the tests¶
Within the tests
directory, use the runtests.py
script to start the
test suite using nose
:
$ cd tests
$ python runtests.py -v
Command line arguments (such as -v
, above) are passed onwards to nose
;
you can use them, for example, to select a particular subset of the suite to
run.
Often it is convenient to run the TraP against a work-in-progress version of
the TraP while continuing to use other libraries and tools installed on the
system. This just requires setting the PYTHONPATH
environment variable to
the root of the development tree:
$ cd tkp
$ export PYTHONPATH=$(pwd):${PYTHONPATH}
$ cd tests
$ python runtests.py -v
Alternatively, you may choose to work with a virtualenv and install the TraP in ‘development mode’:
$ pip install --editable .
Continuous integration¶
There is a Jenkins instance at https://jenkins.transientskp.org/ which builds
and tests all code committed to the master
branch as well as any incoming
pull requests. This constitutes the reference system on which tests are
required to pass: a failure here should result in a change being rejected,
regardless of its successful operation on some other system.