On Tue, 2006-10-31 at 15:38 +1100, Andrew Perry wrote:
(responding to essentially the entire thread)
I would strongly prefer a separate directory tree of files to
prevent
clutter. I've seen many projects take this approach where the
tests
are dislocated from the main tree and very much prefer the
isolation.
This is what I've done with the current function/system
tests. If the
separation ever causes the tests to get out of sync with the
main
tree, then these will instantly cause a failure of the test
suite and
hence synchronisation between different directory trees won't
be an
issue.
I've been following the Biopython project for a few years, and in my
mind this project is a reasonable example of how unit tests in python
can be used.
The Biopython tests are organised like:
Tests/test_Fred.py
with all tests in one directory, separate from the modules that are
being tested, and using the naming scheme that Gary suggests.
The problem here is the namespace, in that several modules within relax
have the same name (eg. the module model_free.py is found in prompt/ and
specific_fns/). For this reason any dedicated unit-test directory would
have to mirror the sub-directory structure of the main code-base. The
simplicity and lack of clutter which seems to be the principle
attraction of the isolated test directory is thus lost.
They also hold any data required for input/comparison for the tests in
subdirectories under the Tests directory, named module-wise like:
Tests/Fred/fred_input_data_001.dat
Tests/Fred/fred_input_data_002.dat
This seems a better option than a single dedicated data directory within
the unit-test directory - the single data directory would also have to
mirror the code sub-directory structure.
Chris
Biopython doesn't seem to have much in the way of the higher level
system/regression-style tests, due to the nature of that project being
a mostly a collection of small modules, as far as I can tell they
don't require testing of higher level 'protocols' which such as relax
uses.
As far as running the test suite (ie parsing directories for
test_Fred.py files and running them) .. here's a link to the Biopython
code that does just that (clean-room warning: link is to BSD-ish
licensed code :) ):
http://cvs.biopython.org/cgi-bin/viewcvs/viewcvs.cgi/biopython/Tests/run_tests.py?rev=1.7&cvsroot=biopython&content-type=ext/vnd.viewcvs-markup
It even uses the unittestgui with the all-important 'green line', that
goes red if any test fails :)
So, if I understand correctly, the plan is to have the test_suite with
something like:
test_suite/unit_tests (for the real module-wise unit tests, using
Pythons unittest module)
and
test_suite/system_tests (which is closer to what is in the current
test_suite, and could be at the same high level as the users scripts
in sample_scripts. Eventually the existing tests can be modified to
use the unittest framework for verifying 'correctness' of operation).
Cheers,
Andrew
_______________________________________________
relax (http://nmr-relax.com)
This is the relax-devel mailing list
relax-devel@xxxxxxx
To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-devel
Re: unit tests vs... the current test_suite