Re: Weird performance of grid search and order of spectrum.error_analysis -- April 22, 2014 - 10:14

Hi Troels,

This third thread at
http://thread.gmane.org/gmane.science.nmr.relax.devel/5302 will cover
the error analysis.


> Another thing that worries me, is the order how you perform the error 
> analysis:
> If I set:
> ********************************************
> relax> spectrum.replicated(spectrum_ids=['Z_A1', 'Z_A15'])
> relax> spectrum.replicated(spectrum_ids=['Z_B1', 'Z_B18'])
> relax> spectrum.error_analysis(subset=['Z_A0', 'Z_A1', 'Z_A2', 'Z_A3',
> 'Z_A4', 'Z_A5', 'Z_A6', 'Z_A7', 'Z_A8', 'Z_A9', 'Z_A10', 'Z_A11',
> 'Z_A12', 'Z_A13', 'Z_A14', 'Z_A15', 'Z_A16'])
> Intensity measure:  Peak heights.
> Replicated spectra:  Yes.
> All spectra replicated:  No.
>
> Replicated spectra:  ['Z_A1', 'Z_A15']
> Standard deviation:  1900.52054868
>
> Variance averaging over all spectra.
> Standard deviation for all spins:  1900.5205486798784
>
> relax> spectrum.error_analysis(subset=['Z_B0', 'Z_B1', 'Z_B2', 'Z_B3',
> 'Z_B4', 'Z_B5', 'Z_B6', 'Z_B7', 'Z_B8', 'Z_B9', 'Z_B10', 'Z_B11',
> 'Z_B12', 'Z_B13', 'Z_B14', 'Z_B15', 'Z_B16', 'Z_B17', 'Z_B18'])
> Intensity measure:  Peak heights.
> Replicated spectra:  Yes.
> All spectra replicated:  No.
>
> Replicated spectra:  ['Z_B1', 'Z_B18']
> Standard deviation:  5629.56857527
>
> Variance averaging over all spectra.
> Standard deviation for all spins:  2562.7669744222535
> *************************
> relax> spectrum.replicated(spectrum_ids=['Z_A1', 'Z_A15'])
> relax> spectrum.replicated(spectrum_ids=['Z_B1', 'Z_B18'])
>
> relax> spectrum.error_analysis(subset=['Z_B0', 'Z_B1', 'Z_B2', 'Z_B3',
> 'Z_B4', 'Z_B5', 'Z_B6', 'Z_B7', 'Z_B8', 'Z_B9', 'Z_B10', 'Z_B11',
> 'Z_B12', 'Z_B13', 'Z_B14', 'Z_B15', 'Z_B16', 'Z_B17', 'Z_B18'])
> Intensity measure:  Peak heights.
> Replicated spectra:  Yes.
> All spectra replicated:  No.
>
> Replicated spectra:  ['Z_B1', 'Z_B18']
> Standard deviation:  5629.56857527
>
> Variance averaging over all spectra.
> Standard deviation for all spins:  5629.5685752716654
>
> relax> spectrum.error_analysis(subset=['Z_A0', 'Z_A1', 'Z_A2', 'Z_A3',
> 'Z_A4', 'Z_A5', 'Z_A6', 'Z_A7', 'Z_A8', 'Z_A9', 'Z_A10', 'Z_A11',
> 'Z_A12', 'Z_A13', 'Z_A14', 'Z_A15', 'Z_A16'])
> Intensity measure:  Peak heights.
> Replicated spectra:  Yes.
> All spectra replicated:  No.
>
> Replicated spectra:  ['Z_A1', 'Z_A15']
> Standard deviation:  1900.52054868
>
> Variance averaging over all spectra.
> Standard deviation for all spins:  5386.8126508475143
> *************************
> The difference is:
> Standard deviation for all spins:  2562.7669744222535
> or
> Standard deviation for all spins:  5386.8126508475143

I knew we should have had better system test coverage of this!  The
problem was probably introduced when I added the subset argument to
the spectrum.error_analysis user function
(http://thread.gmane.org/gmane.science.nmr.relax.scm/17239) for the
relaxation dispersion analysis.

It should be very easy for you to create a system test for this!  Note
that the result of the "variance averaging over all spectra" should be
the same as the single replicate standard deviation (for each subset).
 Then have a go and see if you can find where the bug in relax is ;)
I would also suggest creating a bug report for the problem.  Once you
have this sorted out, I might release relax 3.2.0 and tell everyone
that they should upgrade if they are performing dispersion analyses.

Cheers,

Edward