On Mon, 2006-10-30 at 11:56 +0000, Gary S. Thompson wrote:
Hi Andrew
Unit tests have been on my radar as well as I have used them under
java. As ususal having done things the wrong way round I am now
implementing unit tests for the ieee float functions. Here is my suggestion:
0. use unittest, it is a tried and tested methodology
1. if you have a file fred.py it should be accompanied by a unit test
files test_fred.py with the correct main function
2. if you have test data to go with fred.py it should be in a directory
test_data_fred
3. modify the relax test suite to scan the source directories for files
with the correct names and apply the tests...
as a general note you ought to write the unit tests first ;-) before
coding if you want to stick to the 'true philosophy' of this methodology
so the idea is
1. suggest feature
2. define relevant unit tests
3. code unit tests
4. code feature
5. check / correct / add more unit tests as needed
regards
gary
It seems to me there are a couple of different issues here.
The first is would it be useful to have unit-tests for relax. From my
point of view the answer is a clear yes, and I agree with Gary's
suggestions regarding their organisation.
The second issue raised in Andrew's post is whether the Python unit-test
mechanisms are worth using to drive the current test-suite. Again, I
think the answer is yes: the test-suite is getting big and complex
enough to justify using this mechanism. It should make coding new tests
easier, and eliminate the need for much of the repeditive code in the
current suite.
As to why I think these are separate issues: the current test-suite
(indeed the entire relax code base) contains not one single unit-test.
Unit-tests, as their name implies, are intended to test the smallest
possible unit of code in isolation (a single function or method, at most
a single class). The type of test coded in the test-suite is much higher
level than this, and is better termed a regression test - it is intended
to test that all 'units' in the program are working together correctly.
A complete set of regression tests should catch all bugs in relax, but
might not reveal much about the nature of the bugs. On the other hand,
unit tests will miss bugs of regression, but should identify more detail
of the nature and location of the bugs they do catch. Both types of
testing are important in a working package like relax.
I think its important that the distinction remains clear between the
relax test-suite, which should be a reasonably exhaustive set of
regression tests, and the relax unit-tests which will remain incomplete
for the forseeable future. Gary's suggestion above is appropriate for
the organisation of unit-tests, but the test-suite I think properly
belongs as a seperate set of tests.
All of that said, there is no reason why we can't use the Unittest
module as the testing mechanism for the test-suite.
Chris
Re: unit tests vs... the current test_suite