Posted by Edward d'Auvergne on December 02, 2006 - 14:28:
> I would like a clear distinction between the different classes of
> tests so that relax can generate a report in the end. I.e. all the
> unit tests are clustered into a single TestSuite (which can be
> composed of multiple TestSuite objects for grouping) and all the
> system/functional tests clustered into a single separate TestSuite
> object. Although the system/functional tests could possibly use the
> unittest framework, the unit tests and system tests don't have to be
> within the same framework.
>
No indeed and this is one of the thoughts we had. The TestSuite tree of
instances should be something like this
system_tests
|
--------------------------------------------
| | | |
model_selection jw_mapping ... unit_tests
|
-----------------------------
| | |
test_x test_y ...
(if the tree doesn't display you need to use a monospaced font)
I don't know if I would classify the unit tests as system tests though
(for example see http://en.wikipedia.org/wiki/Unit_testing
and
http://en.wikipedia.org/wiki/System_testing). Instead I was thinking
of the following structure for the tests:
test suite:
unit tests:
test_relax.py
test 1
test 2
test_float.py
test 1
test 2
system / functional tests:
model-free:
reading:
test 1
test 2
optimisation:
test 1
test 2
reduced spectral density mapping:
test 1
test 2
The unit tests can run first, printing a message saying that the tests
passed. Then the system/functional tests can be run, printing a
similar message. For example if both pass the following could be
printed (with the unittest print out in between):
Unit tests ................. [ passed ]
Functional tests ........... [ passed ]
Test suite ................. [ passed ]
I.e. the first ideally tests if all parts of relax work on a function
by function level and the second ideally runs all types of analyses on
the system level. There's no need to print out a message for each of
the system tests. relax could however print out '[ OK ]' or '[ passed
]' messages for each category of unit and system tests if the
--test-suite command line flag is used together with --debug (or some
other flag).
> I envisage the system tests to work quite differently in that the
> instance 'self.relax' is passed into each test class (this is
> essential). Most of the automatic stuff in the unit tests will be
> unnecessary. For example functions such as
> 'get_module_relative_path()', etc. will be unnecessary.
Indeed this is true and my plan is to create a super class which just
did test discovery without having to find relative paths etc the root
path would just be provided
A base class placed in the base directory 'test_suite'?
yes
(We talked about subclassing TestCase as well so it has an idea of it's
directory and the root test_suite directory to give tests places to find
data from)
That's probably the best way to do it. Maybe the modified TestCase
could be called RelaxTestCase?
sounds iseal
> Actually on
> the subject of these functions, the ones not defined as class methods,
> they could all be shifted into the classes and accessed by typing
> 'self.get_module_relative_path()'. Apart from the print statements at
> the end of 'unit_test_runner.py', these functions are only accessed by
> the other class methods. These may as well be converted to an object
> oriented approach.
>
Now that is an interesting question: Should they go in the class? Most
probably. However, should they be virtual methods that can be overidden?
Almost definitley not in my opinion (overridden methods should always be
designed to be so) So I suggest they go in as static methods of the
class [this was my plan anyway I just have to get down to trying it]
also most of these methods should be private [they are not part of the
public interface])
That sounds like a really good setup. By static method, do you mean
using the @staticmethod function decorator? If so, is there a
benefit? And by virtual method do you mean a standard class method?
the benefit is that people cant start overriding a static method and claiming that its an interface that we should be maintaining. Also its good style when operating in a function where there is no need to use self such as utility methods.
e.g here is a quote from
http://www.rexx.com/~dkuhlman/python_101/python_101.htmlsnip----------------------
- Most of the time, almost always, implement plain
instance methods. Implement an instance method whenever the
method needs access to the values that are specific to the
instance or needs to call other methods that have access to
instance specific values. If the method needs self, then
you probably need an instance method.
- Implement a class method (1) when the method
does not need access to instance variables and (2) when you
do not want to require the caller of the method to create an
instance and (3) when the method needs access to class variables.
A class method may be called on either an instance or the class.
A class method gets the class as a first argument, whether it is
called on the class or the instance. If the method needs access
to the class but does not need self, then think class method.
- Implement a static method if you merely want to put
the code of the method within the scope of the class, perhaps for
purposes of organizing your code, but the method needs access to
neither class nor instance variables (though you can access class
variables through the class itself). A static method may be called
on either an instance or the class. A static method gets neither
the class nor the instance as an argument.
unsnip------------------
Making the methods private is ok by me. I haven't used private names,
but in the situation where someone is adding tests to the test suite -
this would be quite useful. For sanity, I would use the following
ordering of the class methods throughout all of relax. First would
come the special methods 'self.__xxx__()' ordered alphabetically with,
of course, 'self.__init__()' as the very first method. Then in
alphabetical order would be the private methods 'self.__xxx()'.
Finally, and again alphabetically ordered, would be the standard class
methods.
can I suggest a slightly different order (if this isn't already set in stone of course, otherwise I will obey ;-)
constants
variables
special methods
standard instance methods (watch out class methods can mean a different thing
private variables
private methods
all alphabetically listed
This makes for a logical ordering for the general reader (i.e. anm extrenal user of the class) as the things any one other that the developer of the class wants to read come first
This leads to another question what is the minimum python version that
we are aiming for in the 1.3 branch?
(i.e. what facilities are available to us)
Which version do people think would be best? I would say 2.4. Would
you like to start a new thread to discuss this Gary?
can do
It's best that
everyone sees this discussion as it would probably be lost in this
thread. I'll respond to the rest in a second post as this is getting
too big.
Edward
Re(2): r2860 - /branches/test_suite/test_suite/unit_tests/unit_test_runner.py