On 10/20/06, Alexandar Hansen <viochemist@xxxxxxxxx> wrote:
Maybe I'm missing something, but to me 1e-21 is the same as 1e-27 in
computer language. Let's assume a small amount of error in measurements of
1%. With that amount of random error, I couldn't possibly expect much less
than 1e-3 or 1e-4 as an overall chi2. Is there significance to having this
high of precision when our measurements could never reach it?
Very much so. The function tolerance between iterations is set to
1e-25. In model-free analysis this is the chi-squared function
difference between two iterations of the optimisation algorithm (not
to be confused with the iterations of the Method of Multipliers
constraint algorithm, the iterations of the step length selection
algorithm for the line search optimisation algorithms, the iterations
over the runs of the 'full_analysis.py' script, or the iterations over
the global models of the 'full_analysis.py' script (there are many
other iterations as well, but I should probably stop now)). The
reason for this is because the sum of many small steps can be quite
large.
As I showed on my poster at ICMRBS (and will hopefully soon be
published), the model-free space is quite convoluted. In the two
timescale models, it is common to have a long convoluted shallow
tunnel through the model-free space close the the local/global
minimum. If a function tolerance of 1e-5 is used, then the
optimisation algorithms are unable to slide down the tunnel to reach
the minimum. The result is that ts is overestimated by an order of
magnitude and the S2f and S2s parameters have their values swapped!!!
Although the change in chi-squared is small for each step, the overall
change from the top to the bottom of the tunnel is large. This is why
high precision optimisation after a very rough grid search is
essential for correct model-free results.
This relates to the fact that the changes in the optimised function
value are not correlated to changes in the parameter values. Rather
the change in parameter values is related to the complex curvature of
the space which itself is directly correlated with the experimental
errors. In order to reach the local minimum, if a part of the space
which needs to be traversed is shallow and curved (a shallow valley, a
narrow hole through the space, a broad yet long and twisted plane or
valley, etc.) then you will need a small enough function tolerance to
continue past these geometric features. In the model-free space,
1e-25 is optimal.
Edward
P.S. As for 1e-21 being the same as 1e-27, this is only the case if
you are talking about differences between values many orders of
magnitude greater that this. For absolute floating point values, the
value of the exponent part of the floating point doesn't influence the
precision of the rest of the number.
Re: Optimisation tests in the test suite.