On 10/26/06, Chris MacRaild <c.a.macraild@xxxxxxxxxxx> wrote:
I'm still failing on the grid search, both BFGS and the Newton
+backtracking tests (though the reasons for failure are new).
For the grid search I have:
Parameter values: array([ 1., 0., 0.])
Function value: 3.9844117908982244
Iterations: 1331
Function calls: 1331
Gradient calls: 0
Hessian calls: 0
Warning: None
The chi-squared value of '3.9844117908982244' should be within +/-
9.9999999999999995e-21 of 3.9844117908982288.
I'll change this one to the error tolerance of 1e-8 as you suggest below.
For both BFGS tests:
Parameter values: array([ 9.70000000e-01, 2.04800000e+03,
1.49000000e-01])
Function value: 4.1720494200991728e-21
Iterations: 157
Function calls: 699
Gradient calls: 164
Hessian calls: 0
Warning: None
The te value of '2047.9999998929304' should be within +/- 1e-08 of
2048.0.
I'm surprised by this difference! I might change the te scaling to
1e9 (nanoseconds) and leave the error tolerances as they are. That
will fix this issue.
And for the Newton+backtracking test:
Parameter values: array([ 9.70000000e-01, 2.04800000e+03,
1.49000000e-01])
Function value: 7.5138787163096146e-28
Iterations: 18
Function calls: 96
Gradient calls: 23
Hessian calls: 18
Warning: None
The function count of '96.0' should be greater than 55.0.
Now you are at a higher precision, the previous results were lower
precision? Bizarrely the iteration count is identical? I'll use
these values instead within the test!
My suggestions:
It was quite appropriate to test finite values with a relative tolerance
of 1e-8. This is the test that should be applied for the grid search
chi2 and all finite parameter values.
And the optimisation algorithms which have been terminated prematurely.
It is clearly not appropriate to test finite chi2 values to an absolute
tolerance of 1e-20. Rather, use the relative 1e-8 tolerance for finite
chi2 and a simple chi2 < 1e-20 test for values expected to be (close to)
zero.
I've changed all the optimisation tests to do this. I'll commit the
changes tomorrow.
Obviously the Newton failure there is a simple typo
Thanks, it's fixed.
Edward
Re: Optimisation tests in the test suite.