While we are on the topic of model selection/optimization, I am
having some trouble with relax 1.2's full_analysis.py script. I have
optimized the local_tm run, and I ran the initial run on the oblate
tensor (init). When I attempt to optimize the oblate run (round_1),
I see the following error a:
relax> results.write(run='final', file='results', dir='oblate/round_1/
aic', force=1, format='columnar', compress_type=1)Opening the file
'oblate/round_1/aic/results.bz2' for writing.RelaxError: The sequence
data for the run 'final' does not exist.
Any ideas?
Thanks,
Doug
On Apr 30, 2007, at 10:11 AM, Clare-Louise Evans wrote:
> Dear Edward,
>
> Sorry for the thread hijack, but I noticed in the PS section of the
> quote Hongyan has included below your comment regarding convergence.
>
> I am having problems with convergence at the moment when running the
> full_analysis.py script. I have run the optimisation of the diffusion
> models on the same PC. For the oblate, prolate and ellipsoid models I
> have failed to reach convergence after 30+ rounds. The sphere model
> converged within 5 rounds. However, when I look at the output the
> only
> difference between the chi-squared and other parameter values is in
> the
> 10th and later decimal places. Surely, with having such small
> differences between the values they can be considered to have
> converged. However, in your comment you state they have to be
> identical. If this really is the case then I'm not sure how to
> proceed
> given that I'm failing to reach convergence on these 3 models?
>
> Apologies if this ties in to the question I asked yesterday regarding
> full_analysis.py.
>
> Kind regards
> Clare
>
> Hongyan Li wrote:
>> Dear Edward,
>> Thanks for your early suggestion regarding different model
>> selections. I have
>> been busy on other stuff and only recently got time to focus on
>> this subject
>> again.
>> I have run all the diffusion models e.g. isotropic, prolate,
>> oblate and
>> ellipsoid and model selections were made within each model (in aic
>> directory).
>> I am not sure how to write a script to select different models using
>> 'model_selection()'. Could you please do me a favor?
>> Your help is highly appreciated!
>> Best wishes,
>> Hongyan
>>
>> Quoting Edward d'Auvergne <edward.dauvergne@xxxxxxxxx>:
>>
>>
>>> Hi,
>>>
>>> To compare the results what you need to employ is a technique
>>> from the
>>> statistical field of model selection. The spherical diffusion
>>> (isotropic) + all model-free models of all selected residues is one
>>> single mathematical model. The prolate and oblate spheroids
>>> (prolate
>>> and oblate axially symmetric anisotropic diffusion tensors) + all
>>> model-free models, and the ellipsoid (fully anisotropic or three
>>> different eigenvalues) + all model-free models, are three additional
>>> mathematical models. Therefore to compare these four different
>>> models
>>> you need to select the model which best represents your relaxation
>>> data. These models are, however, not nested and therefore cannot be
>>> compared using ANOVA F-tests! Firstly the three types of diffusion
>>> tensor are not nested (there is a reference from Dominique Marion's
>>> group in which they say ANOVA statistics cannot be used but I can't
>>> find it at the moment (although it shouldn't be too hard to track
>>> down, it's related to Tensor)). Secondly the model-free models
>>> selected will be different between the four models. Hence chi-
>>> squared
>>> and F-tests cannot be used.
>>>
>>> A useful reference (I'm not at all biased ;) for this problem is my
>>> paper d'Auvergne, E. J. and Gooley, P. R. (2003), (see
>>> http://www.nmr-relax.com/refs.html for the full reference). On page
>>> 37 at the end of that paper I discuss how AIC model selection is
>>> perfect for selecting between these non-nested models. The AIC
>>> criterion is still
>>>
>>> AIC = chi2 + 2k,
>>>
>>> however chi2 is the minimised chi-squared value for the complete
>>> model
>>> and k is the sum of the number of diffusion parameters and number of
>>> model-free parameters for all spin systems. BIC model selection is
>>> likely to work quite well as well. If you have four runs, one for
>>> each of the diffusion models, then the relax user function
>>> 'model_selection()' is designed to select between these models. I
>>> hope this helps and hasn't been too biased.
>>>
>>> Cheers,
>>>
>>> Edward
>>>
>>>
>>> P.S. Prior to model selection between the diffusion models, the
>>> diffusion models must have fully converged. Multiple iterations of
>>> optimisation of the model-free models, AIC model selection, and
>>> optimisation of all parameters together (diffusion tensor + model-
>>> free
>>> parameter of all residues) must be executed. Convergence is when
>>> two
>>> iterations possess identical chi-squared values, identical model-
>>> free
>>> models, and identical parameter values.
>>>
>>>
>>> On 3/2/07, Hongyan Li <hylichem@xxxxxxxxxxxx> wrote:
>>>
>>>> Dear relax users,
>>>> I have managed to use Relax to run my dynamics data by both
>>>> isotropic and
>>>> axial-oblate models. My qestion is how to compare the results, by
>>>>
>>> chi-square??
>>>
>>>> what is the criteria to make judgment that which residues is beter
>>>>
>>> simulated
>>>
>>>> with which model?
>>>>
>>>> Thanks for your kind help!
>>>>
>>>> Best wishes,
>>>>
>>>> Hongyan
>>>>
>>>> Dr. Hongyan Li
>>>> Department of Chemistry
>>>> The University of Hong Kong
>>>> Pokfulam Road
>>>> Hong Kong
>>>>
>>>>
>>>> _______________________________________________
>>>> relax (http://nmr-relax.com)
>>>>
>>>> This is the relax-users mailing list
>>>> relax-users@xxxxxxx
>>>>
>>>> To unsubscribe from this list, get a password
>>>> reminder, or change your subscription options,
>>>> visit the list information page at
>>>> https://mail.gna.org/listinfo/relax-users
>>>>
>>>>
>>
>>
>> Dr. Hongyan Li
>> Department of Chemistry
>> The University of Hong Kong
>> Pokfulam Road
>> Hong Kong
>>
>>
>> _______________________________________________
>> relax (http://nmr-relax.com)
>>
>> This is the relax-users mailing list
>> relax-users@xxxxxxx
>>
>> To unsubscribe from this list, get a password
>> reminder, or change your subscription options,
>> visit the list information page at
>> https://mail.gna.org/listinfo/relax-users
>>
>
> This message has been checked for viruses but the contents of an
> attachment
> may still contain software viruses, which could damage your
> computer system:
> you are advised to perform your own checks. Email communications
> with the
> University of Nottingham may be monitored as permitted by UK
> legislation.
>
>
> _______________________________________________
> relax (http://nmr-relax.com)
>
> This is the relax-users mailing list
> relax-users@xxxxxxx
>
> To unsubscribe from this list, get a password
> reminder, or change your subscription options,
> visit the list information page at
> https://mail.gna.org/listinfo/relax-users
_______________________________________________
relax (http://nmr-relax.com)
This is the relax-users mailing list
relax-users@xxxxxxx
To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-users
Re: model selection