Re: Multiple test cases and the splitting up of large modules. -- March 08, 2007 - 10:00

On 3/8/07, Gary S. Thompson <garyt@xxxxxxxxxxxxxxx> wrote:
> Edward d'Auvergne wrote:
>
> >On Tue, 2007-03-06 at 09:49 +0000, Chris MacRaild wrote:
> >
> >
> >>On Tue, 2007-03-06 at 08:44 퍍㐙ꫲ S. Thompson wrote:
> >>
> >>
> >>>Edward d'Auvergne wrote:
> >>>
> >>>
> >>>
> >>snip
> >>
> >>
> >>>>Also do you think that we should have a single unit test
> >>>>file per relax module?  I can see value in not enforcing this rule.
> >>>>For example the 'specific_fns.model_free' module contains a very large
> >>>>number of class methods.  This could be better served by a directory
> >>>>such as 'test_suite/unit_tests/specific_fns/model_free/' and allowing
> >>>>that directory to have a separate unit test model per class methods
> >>>>(each containing numerous tests of that one method).  As each relax
> >>>>function or method will require numerous tests for proper functioning,
> >>>>proper throwing of RelaxErrors, etc, the unit test modules could
> >>>>become quite large and unwieldy, especially if the self.setUp and
> >>>>self.tearDown() methods need to be different for different
> >>>>functions/methods.
> >>>>
> >>>>
> >>>>
> >>>It would be ideal to have one file per because then we can check for
> >>>basic coverage and overlooked unit tests. However there is nothing to
> >>>stop you adding imports to auxiliary classes to carry out the individual
> >>>tests....
> >>>
> >>>
> >>>
> >>Although I take the point about multiple tests, setUps and tearDowns per
> >>method, I'd generally suggest that if the unit test file is getting too
> >>unwieldy, the related module probably is too. So perhaps we should
> >>enforce this rule, and split modules when neccessary, not split unit
> >>test files. (still more work, still more breakages, but that might be
> >>the cost of a manageable code base)
> >>
> >>
> >
> >
> >We could have multiple test case classes derived from TestCase within
> >the single module file (corresponding to the single relax source
> >file).  That way we can, if needed, have one class per function. I
> >think this is really important as the data structures, etc that need to
> >be created for that one function will be different to what is needed
> >for another function (if common self.setUp() and self.tearDown()
> >methods are needed for different functions, we can simply use
> >inheritance or the same class).  For example the setUp() for the
> >'calc_jw()' function of the module 'maths_fns.jw_mf.py' may need to be
> >different to that of the subsequent function 'calc_S2_jw()' (then again
> >maybe not).  In fact each of the 128 functions in that single module
> >may need different setUp() and tearDown() methods (that code was an
> >absolute nightmare to debug back in 2003, but this setup is very
> >important for numerical efficiency).  A better example might be the
> >'self.execute()' and 'self.create()' methods of the module
> >'generic_fns.dasha' which execute Dasha and create Dasha input files
> >respectively.  The tearDown() method for the input file creation tests
> >will require the deletion of the temporary input files whereas
> >tearDown() for Dasha execution will be different.
> >
> >
> Hi Ed
>     Setup and teardown whould only be for common stae between a set of
> unit tests... os function specific setup and teardown whould go in the
> inviviual functions. If a number of   functions share specific setup and
> teardown functionality that could go in a separate utiltiy function.
>
> On another note one of the things that the the unit test framework
> allows us to do is to return a test suite when we load a unit test from
> a file (testsuite and test case are polymorphic)... Obviously this
> allows multiple different test_cases or suites to be loaded from one
> file.. However. overall i agree with chris that if the the files are too
> complex they should be split

Yep, I've already set up a framework for splitting the most complex of
relax's modules, 'specific_fns/model_free.py'.  This can be used for
other modules.  Even if we do break up the complex modules, there will
still be a need for different setUp and tearDown methods for different
categories of relax functions (of the same module).  These special
methods are important as the Python unittest code will execute
tearDown no matter how badly the code fails during the test and setUp
is rerun for each of the unittests of the TestCase.  These setups and
cleanups shouldn't be in the individual functions.  Unittest will take
all the classes subclassed from TestCase in the given module and run
them.  So if we have a different TestCase class for each time we need
different setups, then we can have one test suite module corresponding
to one relax module - mimicking the directory and file structure -
while still retaining the flexibility of doing completely different
things in the unit tests of that module.  Another example of where
setUp and tearDown need to be different between two functions of a
relax module is in 'generic_fns/structure.py'.  For the method
add_atom(), we will need to create the 'self.atomic_data' structure
prior to the unit tests.  This structure is however not used by the
method uniform_vect_dist_spherical_angles(), and hence the setUp for
its tests will be different.  For the method write_pdb_file(), the
resultant file will need to be deleted by the tearDown() TestCase
method.

Cheers,

Edward