On 10/20/06, Gary S. Thompson <garyt@xxxxxxxxxxxxxxx> wrote:
This is quite an interesting result ;-) If you are doing float maths and
don't call any higher level functions the results from the two test
cases _ought_ to be the same if both platforms impliment ieee-754.
However, there are some caveats once you call high level functions all
bets are off as there as there can be impementation/algorithm
dependancies (certainly the c standard on first reading says nothing
about what rounding sin etc use!..?). Now there are further complcations
as some compilers use fused multiply-add instructions and other
optimisation which are not coveredby ieee-754 and are not standardised
by ieee-754 in terms of their rounding behaviour....( except in C99
which has a pragma to control this sort of thing: FP_CONTRACT)
The Numeric function calls which use BLAS and LAPACK I'm sure would be
notorious here.
So here my thought. What we have here are regression tests so we either
1. define a set of results for each test on each particular platform
(you have a mode there someone can run the tests on a version we believe
works, and then say e-mails them to us for inclusion) We then store them
and then use those results only for that platform
2. define a s set of result for each test which encompases worst case
performance (as long as it is reasonable), run the tests on a variety of
platforms and if it fails on some platforms decide on a case by case
basis if the result is reasonable and downgrade you regression tests
till it works everywhere.
I would go for 2. its a lot easier to work with and much more likley to
be used by the user for testing their implementation
I agree! There are too many variables to sanely handle point 1. The
model-free parameter tests should be tight but the optimisation stats
tests should be set to the hypothetical worst case. The question is,
how would you initially define 'worst case' when building these tests?
Edward
Re: Optimisation tests in the test suite.