Hi Troels,
You should be very careful with your interpretation here. The
curvature of the chi-squared space does not correlate with the
parameter errors! Well, it most cases it doesn't. You will see this
if you map the space for different Monte Carlo simulations. Some
extreme edge cases might help in understanding the problem. Lets say
you have a kex value of 100 with a real error of 1000. In this case,
you could still have a small, perfectly quadratic minimum. But this
minimum will jump all over the place with the simulations. Another
extreme example might be kex of 100 with a real error of 0.00000001.
In this case, the chi-squared space could look similar to the
screenshot you attached to the task ( http://gna.org/task/?7882).
However Monte Carlo simulations may hardly perturb the chi-squared
space. I have observed scenarios similar to these hypothetical cases
with the Lipari and Szabo model-free protein dynamics analysis.
There is one case where the chi-squared space and error space match,
and that is at the limit of the minimum when the chi-squared space
becomes quadratic. This happens when you zoom right into the minimum.
The correlation matrix approach makes this assumption. Monte Carlo
simulations do not. In fact, Monte Carlo simulations are the gold
standard. There is no technique which is better than Monte Carlo
simulations, if you use enough simulations. You can only match it by
deriving exact symbolic error equations.
Therefore you really should investigate how your optimisation space is
perturbed by Monte Carlo simulations to understand the correlation -
or non-correlation - of the chi-squared curvature and the parameter
errors. Try mapping the minimum for the simulations and see if the
distribution of minima matches the chi-squared curvature
(http://gna.org/task/download.php?file_id=23527).
Regards,
Edward
On 16 January 2015 at 17:14, Troels E. Linnet
<NO-REPLY.INVALID-ADDRESS@xxxxxxx> wrote:
URL:
<http://gna.org/task/?7882>
Summary: Implement Monte-Carlo simulation, where errors are
generated with width of standard deviation or residuals
Project: relax
Submitted by: tlinnet
Submitted on: Fri 16 Jan 2015 04:14:30 PM UTC
Should Start On: Fri 16 Jan 2015 12:00:00 AM UTC
Should be Finished on: Fri 16 Jan 2015 12:00:00 AM UTC
Category: relax's source code
Priority: 5 - Normal
Status: In Progress
Percent Complete: 0%
Assigned to: tlinnet
Open/Closed: Open
Discussion Lock: Any
Effort: 0.00
_______________________________________________________
Details:
This is implemented due to strange results.
A relaxation dispersion on data with 61 spins, and a monte carlo simulation
with 500 steps, showed un-expected low errors.
-------
results.read(file=fname_results, dir=dir_results)
# Number of MC
mc_nr = 500
monte_carlo.setup(number=mc_nr)
monte_carlo.create_data()
monte_carlo.initial_values()
minimise.execute(min_algor='simplex', func_tol=1e-25, max_iter=int(1e7),
constraints=True)
monte_carlo.error_analysis()
--------
The kex was 2111 and with error 16.6.
When performing a dx.map, some weird results was found:
i_sort dw_sort pA_sort kex_sort chi2_sort
471 4.50000 0.99375 2125.00000 4664.31083
470 4.50000 0.99375 1750.00000 4665.23872
So, even a small change with chi2, should reflect a larger
deviation with kex.
It seems, that change of R2eff values according to their errors, is not
"enough".
According to the regression book of Graphpad
http://www.graphpad.com/faq/file/Prism4RegressionBook.pdf
Page 33, and 104.
Standard deviation of residuals is:
Sxy = sqrt(SS/(N-p))
where SS is sum of squares. N - p, is the number of degrees of freedom.
In relax, SS is spin.chi2, and is weighted.
The random scatter to each R2eff point should be drawn from a gaussian
distribution with a mean of Zero and SD equal to Sxy.
Additional, find the 2.5 and 97.5 percentile for each parameter.
The range between these values is the confidence interval.
_______________________________________________________
File Attachments:
-------------------------------------------------------
Date: Fri 16 Jan 2015 04:14:30 PM UTC Name: Screenshot-1.png Size: 161kB
By: tlinnet
<http://gna.org/task/download.php?file_id=23527>
_______________________________________________________
Reply to this item at:
<http://gna.org/task/?7882>
_______________________________________________
Message sent via/by Gna!
http://gna.org/
_______________________________________________
relax (http://www.nmr-relax.com)
This is the relax-devel mailing list
relax-devel@xxxxxxx
To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-devel
Re: [task #7882] Implement Monte-Carlo simulation, where errors are generated with width of standard deviation or residuals