> The function tolerance between iterations is set to 1e-25 (I'll get to
> the importance of this in my next post).
Is this the only termination criterion? If so, why are we seeing
apparently normal optimisations terminating with chi2 >> 1e-25 ?
I guess another possibly related question is why is this only happening
for the BFGS and Newton optimisation tests - is there something special
about these algorithms that makes chi2 poorly determined in these test
cases?
No, relax uses by default the function tolerance (default of 1e-25)
and the maximum number of iterations (default of 1e7). As the tests
in the test suite use noise-free relaxation data back-calculated from
known model-free parameter values, the chi-squared value in these
cases at the minimum is zero (to infinite precision!). However as
collected relaxation data is noisy, the chi-squared value should not
be zero (an exception is when the number of parameters of the model is
greater than or equal to the number of relaxation data points).
The failure of the BFGS and Newton tests is likely to be because of
the high level functions (Numeric/LAPACK matrix operations in this
case) mentioned by Gary in his post at
https://mail.gna.org/public/relax-devel/2006-10/msg00117.html
(Message-id: <4538955F.3090300@xxxxxxxxxxxxxxx>).
> The test is to be within
> 'value +/- value*error' where the error is 1e-8. This equation
> removes the problem of the different scaling between the model-free
> parameters (the 1e12 difference between S2 and te, etc.).
This is fine for finite values like S2 and te. The issue here is that
the expected value for chi2 in these tests is 0 (assuming a perfect
optimisation). It seems to me that the best way to ensure that
optimisation is behaving correctly in these cases is to test for the
following:
1) The optimised values are correct to within some tolerance (1e-8
relative error seems about right here)
2) Termination is normal. ie. the optimiser has thrown no errors or
warnings, and has not reached its maximum number of iterations.
Good point! This currently isn't tested. Errors are caught but
optimisation warnings (in self.relax.data.res[run][i].warning) aren't.
3) Chi2 is small ( <= 1e-20 seems about right based on the few values
reported so far, but something less restrictive might be required)
That test value should work ok for this noise-free data. Would you
like to have a go changing this value?
On reflection, it is probably worth having at least some tests where we
expect a finite chi2. Testing for that value then should be much easier
to deal with.
I could add some noisy synthetic data for the tests. For testing the
ensemble of algorithms and the myriad of combinations, I don't know if
this is worth the extra effort. It could be useful for testing the
default algorithms though.
Edward
Re: Optimisation tests in the test suite.