Hi,
Yes, Windows has caused many precision compromises in the test-suite.
It was especially bad with the optimisation tests as it decreased the
number of minimisation iterations and hence caused all results to
change! Fortunately the final results were reasonable (in the cases I
tested), just difficult for testing purposes. One way to handle this
would have been to manually code the expected result from each
platform, but I would prefer to have one constant test for all
systems.
A suggestion. Would it be a good idea to ask some people (maybe those
registered on the devel mailing list) to become testers that would verify,
using different platforms / systems, that the test suite works fine. I
suggest this idea because I feel that we could try to make the test-suite as
tight as possible if we can test it on different
platforms / systems with volunteers... This could also help in finding
potential bugs...
Having people doing testing would be very useful. As the 1.3 line
develops and more unit and system tests are added to relax, this
testing will prove even more useful. However at the moment I don't
think there are enough people who would volunteer to consistently test
relax prior to each release. This is why I haven't used the 'alpha',
'beta', 'rc' (release candidate) type system in relax. Also the 1.2
line is quite stable now, especially the model-free code. My papers
describing relax will hopefully be published soon (they are
progressing slowly but surely) and maybe that will increase the number
of people using relax (and hence indirectly the number of testers and
the number of different CPUs and operating systems).
For the 1.2 line, more system tests would be useful to better cover
the full abilities of the program (the current test-suite is very
limited and small). These system tests are designed to be an
automatic yet quick version of a user running one of their scripts and
seeing if there are issues. But not much is tested.
If ever this could be a good idea, here are the platforms / systems I have
access to :
This is a good idea!
a) x86 laptop (Intel Pentium M CPU) running Gentoo Linux
b) x86 desktop (Intel Pentium 4 CPU) running Gentoo Linux
c) x86 desktop (2 Intel Xeon CPUs) running Gentoo Linux
These three chips may cause slightly different precisions in
calculations. Hopefully if the test-suite passes on one, it will pass
on the rest.
d) dual core x86 desktop (Intel Pentium 4) running 64 bit Red Hat
Enterprise Linux
Is this a 64 bit chip?
e) x86-64 desktop (AMD Athlon 64 CPU) running Gentoo Linux
Separate testing on 32 and 64 bit GNU/Linux systems would be good. My
production machine is an x86-64 GNU/Linux laptop and part of the
generation of a relax release is to run the test-suite to see if
anything fails. So I have that one covered.
f) x86-64 cluster (AMD Opteron CPUs) running Gentoo Linux
g) x86-64 cluster (Intel Xeon Dual Core CPUs) running Gentoo Linux
These two will be very beneficial for testing Gary's "multi_processor"
branch once the 1.3 line progresses a bit and this branch is merged
into the main line. This code in combination with a cluster will be
very powerful, and testing on different clusters or multi-core systems
will be good. In the future, more test-suite code will be present for
the testing of these new relax abilities.
Cheers,
Edward
Re: Precision in the test-suite