>> > I'll look into this once I
>> > attempt to migrate the system/function tests to use parts or all of
>> > the unittest framework. At the moment though, we could simply have
>> > relax run the unit tests followed by the system/function tests (or the
>> > other way around).
>>
>>
>> The intention was to have a functional test that ran the unit_tester ;-)
>
>
> It could be implemented that way. It may be better to keep a
> separation between the different types of test though, especially
> considering how big the unit tests may become. Then relax can present
> a report of each category, and only pass if all categories pass. It
> will allow more categories of tests to be added separately - for
> example installation tests, compilation tests, etc.
>
>
My thoughts are that unit tests should be a category in the test suite.
Certainly running relax with a command line flag such as -test or
-test_suite should run all the tests available as a default (because
this is what the user generally wants). If we need the ability to test
sub sets of tests then we need some extra options (philosophically we
should be aiming for users testers and in fact everyone to run all the
tests by default, we catch more bugs that way...)
As in my previous post, I would say that both the unit tests and the
system/functional tests would be classified as two categories of tests
at the highest level. For the running of different categories of
tests separately, which would be useful for when you are coding, what
would be the best way to do this? What type of granularity should
this be implemented to? For example, could the unit tests be run by
themselves using say the --unit-test flag? Or should the running of
individual categories of tests be lower level (maybe even individual
tests)?
>> Two other questions
>>
>> 1. do you use the '_'function_name convention for private functions
>
>
> Currently relax doesn't have private functions - I don't see a need
> for this,
Actually I see quite a strong impetus for private methods/classes in
code that is used/developed by more than one person. In general a class
should define a public interface (bits shat shouldn't go away) and
private methods (implimentation details)...
Although I don't have a problem with having private names in relax, I
don't believe that these are very important within the Python
language. This can be seen by grepping the Python source code. For
example if I go into my Python installation directory
(/usr/lib/python2.4), I get the following method/function counts:
[edward@p2800 python2.4]$ grep -rIP "def __[a-z]*__\(" * | wc -l
4495
[edward@p2800 python2.4]$ grep -rIP "def __[a-z]*\(" * | wc -l
80
[edward@p2800 python2.4]$ grep -rIP "def [a-z]*\(" * | wc -l
7674
If just the site-packages are grepped:
[edward@p2800 site-packages]$ grep -rIP "def __[a-z]*__\(" * | wc -l
2244
[edward@p2800 site-packages]$ grep -rIP "def __[a-z]*\(" * | wc -l
45
[edward@p2800 site-packages]$ grep -rIP "def [a-z]*\(" * | wc -l
3119
These numbers are very telling. Also, I don't think we need to
enforce the fact that certain methods of the public interface
shouldn't be overwritten. If a user does this, then that is their
problem. For example after importing the 'math' module, I can without
problem do the following:
math.sin = 1
> except of course for the eventual conversion of 'float.py'
> into a separate Python module. I have used the 'self._xxxx()' style
> only in the automatic diffusion tensor object creation code and this
> only hides these functions by a search statement within the
> 'self.__repr__()' methods of the relax data structure
> 'self.relax.data'. I know Python fudges private functions using the
> 'self.__xxxx()' notation, but I have a feeling site-packages do this
> differently, not through private objects but by other methods. I
> could be wrong though.
>
Actually I am pretty sure you are right. Though if you ask guido he
would tell you that the __ thing isn't a fudge it is just a marker
(almost all public/private systems can be compromised by a determined
programmer so why bother with anything other than information?)
Further points. My intention wa to rename all the system tests so they
were of the form test_<function>.py so that we can use the unit test
automatic discovery and then rewrite main.py to use a TestRunner and
TestCase setup (headings etc would then come from instance variables) I
can try and create something that does what is already in parallel using
the new infrastructure (maybe we should leave the test suite branch up
for the moment?)
The first thing that needs to be done is making 'relax --test-suite'
work. I would then suggest the following structure of the system
tests. In 'test_suite/system_tests' create the directories
'model_free', etc. Then 'model_free.py' could be split (by svn cp)
into multiple 'test_<category>.py' files. For example
'test_suite/system_tests/model_free/test_optimisation.py'. This could
be done for all the system tests.
As for headings, these could be incorporated into one large printout
when a failure occurs. I.e. for debugging merge the heading, the
relax print outs, and the traceback messages into a single string
which is written to stderr. If you would like to do the work on the
unit tests and the system/functional tests, this can occur directly in
the 1.3 line. As long a relax runs, it shouldn't matter what state
the test suite is in in the unstable 1.3 line.
Edward
Re: r2860 - /branches/test_suite/test_suite/unit_tests/unit_test_runner.py