Hi Edward.
I also think it is some math some where.
I have a feeling, that it is the creating of Monte Carlo data with 2 sigma?
and then some log/exp calculation of R2eff.
If the errors are 2 x times over estimated, the chi2 values are 4 as
small, and the
space should be the same?
best
Troels
2014-08-29 17:06 GMT+02:00 Edward d'Auvergne <edward@xxxxxxxxxxxxx>:
I've just added the 2D Grace plots for this to the repository (r25444,
http://article.gmane.org/gmane.science.nmr.relax.scm/23194). They are
also attached to the task for easier access
(https://gna.org/task/index.php?7822#comment107). From these plots I
see that the I0 error appears to be reasonable, but that the R2eff
errors are overestimated by 1.9555.
The plots are very, very different. It is clear that
chi2_jacobian=True just gives rubbish. It is also clear that there is
a perfect correlation in R2eff when chi2_jacobian=False. The plot
shows absolutely no scattering, therefore this indicates a crystal
clear mathematical error somewhere. I wonder where that could be. It
may not be a factor of 2, as the correlation is 1.9555. So it might
be a bug that is more complicated. In any case, overestimating the
errors by ~2 and performing a dispersion analysis is not possible.
This will significantly change the curvature of the optimisation space
and will also have a huge effect on statistical comparisons between
models.
Regards,
Edward
On 29 August 2014 16:56, Troels Emtekær Linnet <tlinnet@xxxxxxxxxxxxx>
wrote:
The default is now chi2_jacobian=False.
You will hopefully see, that the errors are double.
Best
Troels
2014-08-29 16:43 GMT+02:00 Edward d'Auvergne <edward@xxxxxxxxxxxxx>:
Terrible ;) For R2eff, the correlation is 2.748 and the points are
spread out all over the place. For I0, the correlation is 3.5 and the
points are also spread out everywhere. Maybe I should try with the
change from:
relax_disp.r2eff_err_estimate(chi2_jacobian=True)
to:
relax_disp.r2eff_err_estimate(chi2_jacobian=False)
How should this be used?
Cheers,
Edward
On 29 August 2014 16:33, Troels Emtekær Linnet <tlinnet@xxxxxxxxxxxxx>
wrote:
Do you mean terrible or double?
Best
Troels
2014-08-29 16:15 GMT+02:00 Edward d'Auvergne <edward@xxxxxxxxxxxxx>:
Hi Troels,
I really cannot follow and judge how the techniques compare. I must
be getting old. So to remedy this, I have created the
test_suite/shared_data/dispersion/Kjaergaard_et_al_2013/exp_error_analysis/
directory (r25437,
http://article.gmane.org/gmane.science.nmr.relax.scm/23187). This
contains 3 scripts for comparing R2eff and I0 parameters for the 2
parameter exponential curve-fitting:
1) A simple script to perform Monte Carlo simulation error analysis.
This is run with 10,000 simulations to act as the gold standard.
2) A simple script to perform covariance matrix error analysis.
3) A simple script to generate 2D Grace plots to visualise the
differences. Now I can see how good the covariance matrix technique
is :)
Could you please check and see if I have used the
relax_disp.r2eff_err_estimate user function correctly? The Grace
plots show that the error estimates are currently terrible.
Cheers,
Edward
_______________________________________________
relax (http://www.nmr-relax.com)
This is the relax-devel mailing list
relax-devel@xxxxxxx
To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-devel
Re: Comparison of Monte Carlo simulations vs. covariance matrix.