Dear all,
Note that in the 1.3 line I have made some significant changes to how
the system / functional tests are implemented. There are three major
changes which will significantly simplify the writting of these tests.
1. The first change is that I have written a Python class called a
test runner (RelaxTestRunner). This inherits from the normal Python
unittest TextTestRunner class. The important difference though is
that STDOUT is being caught! This means that the hundreds of lines of
useless relax print outs when running the unit tests are now gone.
Significantly this framework, when applied to the system tests, will
allow the normal print out messages to be suppressed. When an error
or failure of the test occurs, the print out for that test is then
prepended to the print out of the failure at the end of the tests. If
no error occurs, nothing is printed.
2. The system / functional tests have been migrated to the unittest
framework. This will make implementation of the tests so much simpler
than the old setup. You write a class which inherits from TestCase,
write the setUp() and tearDown() methods if you wish, and write the
tests as methods beginning with 'def test_'. See
http://docs.python.org/lib/testcase-objects.html for implementation
details. The only complication is that if you add a new class with
its own tests, this class will need to be registered in the file
'test_suite/system_tests/__init__.py. The reason for this is because
Gary's powerful unit_test_runner.py code is not yet being used by the
system tests. I had a few problems making this fly.
3. Scripts! A system / functional test is supposed to mimic how
relax is operated. It should be a mini version (i.e. less or
truncated data) of a normal script you would run. So now you can just
write a script in 'test_suite/system_tests/scripts', place the
necessary data in 'test_suite/system_tests/data', and then write a
test which runs the script and then checks that the appropriate result
has been achieved. I will soon be migrating most of the tests to use
scripts, as the current tests are essentially scripts written in an
ugly way. An example, in the 1.3 line, to see how this is done is the
test_create_m4() method of the test_suite/system_tests/model_free.py
file.
Regards,
Edward
A new framework for the system/functional tests (and improved unit tests).