Re: unit tests vs... the current test_suite -- October 31, 2006 - 09:31

On 10/31/06, Andrew Perry <ajperry@xxxxxxxxxxxxxx> wrote:
> (responding to essentially the entire thread)
>
> I've been following the Biopython project for a few years, and in my mind
> this project is a reasonable example of how unit tests in python can be
> used.
>
> The Biopython tests are organised like:
>
> Tests/test_Fred.py
>
> with all tests in one directory, separate from the modules that are being
> tested, and using the naming scheme that Gary suggests.
>
> They also hold any data required for input/comparison for the tests in
> subdirectories under the Tests directory, named module-wise like:
>
> Tests/Fred/fred_input_data_001.dat
> Tests/Fred/fred_input_data_002.dat
>
> Biopython doesn't seem to have much in the way of the higher level
> system/regression-style tests, due to the nature of that project being a
> mostly a collection of small modules, as far as I can tell they don't
> require testing of higher level 'protocols' which such as relax uses.
>
> As far as running the test suite (ie parsing directories for test_Fred.py
> files and running them) .. here's a link to the Biopython code that does
> just that (clean-room warning: link is to BSD-ish licensed code :) ):
>
> http://cvs.biopython.org/cgi-bin/viewcvs/viewcvs.cgi/biopython/Tests/run_tests.py?rev=1.7&cvsroot=biopython&content-type=ext/vnd.viewcvs-markup
>
> It even uses the unittestgui with the all-important 'green line', that goes
> red if any test fails :)
>
> So, if I understand correctly, the plan is to have the test_suite with
> something like:
>
> test_suite/unit_tests    (for the real module-wise unit tests, using Pythons
> unittest module)
>
> and
>
> test_suite/system_tests   (which is closer to what is in the current
> test_suite, and could be at the same high level as  the users scripts in
> sample_scripts. Eventually the existing tests can be modified to use the
> unittest framework for verifying 'correctness' of operation).

That's a great summary of the thread.  As the 1.2 line is now in
maintenance mode, these modifications should go into the 1.3 line.  I
have therefore created the branch of the 1.3 line located at
'branches/test_suite' within the repository
(http://svn.gna.org/viewcvs/relax/) for the changes.

The unittest GUI would probably be quite useful.  The green bar
feature is pretty cool.  However it would be best to wait until the
function/system tests use enough of the unittest framework so that the
entire test suite can run through this TK GUI.  You could then run
'relax --test-suite' and have the GUI pop up.

Cheers,

Edward