Re: Weird performance of grid search and order of spectrum.error_analysisRe: Weird performance of grid search and order of spectrum.error_analysisHi Troels, Note that solving this bug might change the Relax_disp.test_hansen_trunc_data system test. I can see the same problem occurring there when I run: $ ./relax -d --gui-tests Relax_disp.test_hansen_trunc_data and find where the spectrum.error_analysis user function was called. Therefore maybe a check could be added there as well, as the variance averaging is then over 2 sets of 3 replicates: """ relax> spectrum.error_analysis(subset=['500_reference.in', '500_66.667.in', '500_133.33.in', '500_133.33.in.bis', '500_200.in', '500_266.67.in', '500_333.33.in', '500_400.in', '500_466.67.in', '500_533.33.in', '500_533.33.in.bis', '500_600.in', '500_666.67.in', '500_733.33.in', '500_800.in', '500_866.67.in', '500_933.33.in', '500_933.33.in.bis', '500_1000.in']) Intensity measure: Peak heights. Replicated spectra: Yes. All spectra replicated: No. Replicated spectra: ['500_133.33.in', '500_133.33.in.bis'] Standard deviation: 132.165166899 Replicated spectra: ['500_533.33.in', '500_533.33.in.bis'] Standard deviation: 139.49874291 Replicated spectra: ['500_933.33.in', '500_933.33.in.bis'] Standard deviation: 103.633969846 Variance averaging over all spectra. Standard deviation for all spins: 126.05227793927868 relax> spectrum.error_analysis(subset=['800_reference.in', '800_66.667.in', '800_133.33.in', '800_133.33.in.bis', '800_200.in', '800_266.67.in', '800_333.33.in', '800_400.in', '800_466.67.in', '800_533.33.in', '800_533.33.in.bis', '800_600.in', '800_666.67.in', '800_733.33.in', '800_800.in', '800_866.67.in', '800_933.33.in', '800_933.33.in.bis', '800_1000.in']) Intensity measure: Peak heights. Replicated spectra: Yes. All spectra replicated: No. Replicated spectra: ['800_133.33.in', '800_133.33.in.bis'] Standard deviation: 234.398086258 Replicated spectra: ['800_533.33.in', '800_533.33.in.bis'] Standard deviation: 300.176560374 Replicated spectra: ['800_933.33.in', '800_933.33.in.bis'] Standard deviation: 258.534014697 Variance averaging over all spectra. Standard deviation for all spins: 170.37263779019497 """ Here I've deleted the debugging printouts. For the first set, the variance averaging is:
sqrt((132.165166899**2 + 139.49874291**2 + 103.633969846**2) / 3)
126.0522779391583 For the second set:
sqrt((234.398086258**2 + 300.176560374**2 + 258.534014697**2) / 3)
265.7619655473215 I don't think you'll have many problems debugging this one! The hard part will be updating the dispersion GUI tutorial in the manual as those numbers will slightly change. I'll wait for the bug to be closed first though before working on that. Cheers, Edward On 22 April 2014 10:13, Edward d'Auvergne <edward@xxxxxxxxxxxxx> wrote:
Hi Troels, This third thread at http://thread.gmane.org/gmane.science.nmr.relax.devel/5302 will cover the error analysis.Another thing that worries me, is the order how you perform the error analysis: If I set: ******************************************** relax> spectrum.replicated(spectrum_ids=['Z_A1', 'Z_A15']) relax> spectrum.replicated(spectrum_ids=['Z_B1', 'Z_B18']) relax> spectrum.error_analysis(subset=['Z_A0', 'Z_A1', 'Z_A2', 'Z_A3', 'Z_A4', 'Z_A5', 'Z_A6', 'Z_A7', 'Z_A8', 'Z_A9', 'Z_A10', 'Z_A11', 'Z_A12', 'Z_A13', 'Z_A14', 'Z_A15', 'Z_A16']) Intensity measure: Peak heights. Replicated spectra: Yes. All spectra replicated: No. Replicated spectra: ['Z_A1', 'Z_A15'] Standard deviation: 1900.52054868 Variance averaging over all spectra. Standard deviation for all spins: 1900.5205486798784 relax> spectrum.error_analysis(subset=['Z_B0', 'Z_B1', 'Z_B2', 'Z_B3', 'Z_B4', 'Z_B5', 'Z_B6', 'Z_B7', 'Z_B8', 'Z_B9', 'Z_B10', 'Z_B11', 'Z_B12', 'Z_B13', 'Z_B14', 'Z_B15', 'Z_B16', 'Z_B17', 'Z_B18']) Intensity measure: Peak heights. Replicated spectra: Yes. All spectra replicated: No. Replicated spectra: ['Z_B1', 'Z_B18'] Standard deviation: 5629.56857527 Variance averaging over all spectra. Standard deviation for all spins: 2562.7669744222535 ************************* relax> spectrum.replicated(spectrum_ids=['Z_A1', 'Z_A15']) relax> spectrum.replicated(spectrum_ids=['Z_B1', 'Z_B18']) relax> spectrum.error_analysis(subset=['Z_B0', 'Z_B1', 'Z_B2', 'Z_B3', 'Z_B4', 'Z_B5', 'Z_B6', 'Z_B7', 'Z_B8', 'Z_B9', 'Z_B10', 'Z_B11', 'Z_B12', 'Z_B13', 'Z_B14', 'Z_B15', 'Z_B16', 'Z_B17', 'Z_B18']) Intensity measure: Peak heights. Replicated spectra: Yes. All spectra replicated: No. Replicated spectra: ['Z_B1', 'Z_B18'] Standard deviation: 5629.56857527 Variance averaging over all spectra. Standard deviation for all spins: 5629.5685752716654 relax> spectrum.error_analysis(subset=['Z_A0', 'Z_A1', 'Z_A2', 'Z_A3', 'Z_A4', 'Z_A5', 'Z_A6', 'Z_A7', 'Z_A8', 'Z_A9', 'Z_A10', 'Z_A11', 'Z_A12', 'Z_A13', 'Z_A14', 'Z_A15', 'Z_A16']) Intensity measure: Peak heights. Replicated spectra: Yes. All spectra replicated: No. Replicated spectra: ['Z_A1', 'Z_A15'] Standard deviation: 1900.52054868 Variance averaging over all spectra. Standard deviation for all spins: 5386.8126508475143 ************************* The difference is: Standard deviation for all spins: 2562.7669744222535 or Standard deviation for all spins: 5386.8126508475143I knew we should have had better system test coverage of this! The problem was probably introduced when I added the subset argument to the spectrum.error_analysis user function (http://thread.gmane.org/gmane.science.nmr.relax.scm/17239) for the relaxation dispersion analysis. It should be very easy for you to create a system test for this! Note that the result of the "variance averaging over all spectra" should be the same as the single replicate standard deviation (for each subset). Then have a go and see if you can find where the bug in relax is ;) I would also suggest creating a bug report for the problem. Once you have this sorted out, I might release relax 3.2.0 and tell everyone that they should upgrade if they are performing dispersion analyses. Cheers, Edward