Hi Ed, That's fine, I now see the reason... Windows again..! Also, don't worry with the patch I proposed, it took me less than 2 minutes to produce it... A suggestion. Would it be a good idea to ask some people (maybe those registered on the devel mailing list) to become testers that would verify, using different platforms / systems, that the test suite works fine. I suggest this idea because I feel that we could try to make the test-suite as tight as possible if we can test it on different platforms / systems with volunteers... This could also help in finding potential bugs... If ever this could be a good idea, here are the platforms / systems I have access to : a) x86 laptop (Intel Pentium M CPU) running Gentoo Linux b) x86 desktop (Intel Pentium 4 CPU) running Gentoo Linux c) x86 desktop (2 Intel Xeon CPUs) running Gentoo Linux d) dual core x86 desktop (Intel Pentium 4) running 64 bit Red Hat Enterprise Linux e) x86-64 desktop (AMD Athlon 64 CPU) running Gentoo Linux f) x86-64 cluster (AMD Opteron CPUs) running Gentoo Linux g) x86-64 cluster (Intel Xeon Dual Core CPUs) running Gentoo Linux Cheers Séb :) Edward d'Auvergne wrote: Hi, As you noted, the precision differences in the test-suite are hard coded into relax as tolerances. I've tried to chase down the point of entry of these tolerances for the reduced spectral density mapping tests but cannot find anything in the commit messages in the repository nor in the mailing list archives. So I'll have to explain their origin as best I can using my slowly fading memory :) If anyone else can remember these discussions and this precise example, please say. So, I was the one who introduced these specific tolerances. This was preceded by many discussions with Alex, Chris, and Gary in the mailing lists about optimisation precisions on different architectures, operating systems, etc. See the thread starting at https://mail.gna.org/public/relax-devel/2006-10/msg00114.html, specifically the post at https://mail.gna.org/public/relax-devel/2006-10/msg00139.html for a summary. Now when I coded these J(w) mapping tests I had, at the time, access to 32 bit i686 GNU/Linux and 32 bit Windows. The tolerances were much smaller in my pre-committed code but when I tested it on Windows, the test-suite failed!!! This could be because of the Windows versions of Python or Numeric (the likely cause), or some other Windows specific issue. So I set these tolerances with the idea that for a final result, this type of precision will be reasonable. For calculations and results prior to the final result truncation will cause round-off error, but in this case a difference of 1e-6 in a manuscript for values on the order of 1 will not matter. So in the end, this compromise was just to make Windows happy! I no longer have access to 32 bit Windows, but will sometime soon setup a relax development environment (http://www.nmr-relax.com/windows_devel.html) for 64 bit Windows (I had a feeling that I would one day have a use for the Windows install that came with my laptop :). I have a feeling, though, that your patches will cause the test-suite to fail under Windows. Sorry, I should have mentioned this prior to you making these patches. Regards, Edward On 10/22/07, Sebastien Morin <sebastien.morin.1@xxxxxxxxx> wrote:Hi again... Here are two patches if the changes I proposed for the precision in the jw_mapping and consistency tests test-suites were to be accepted as relevant... 1. patch_precision_in_test-suite_1.2-r3370_jw_mapping For jw_mapping in the main branch. 2. patch_precision_in_test-suite_1.2-r3370_consistency_tests (consistency tests branch) For consistency tests in the consistency tests branch. If not relevant, just discard the files.. Cheers Séb :) Sebastien Morin wrote: Hi again After a look in the test_suite/consistency.py and test_suite/jw_mapping.py files, I realized that the 6th decimal precision behaviour was directly encoded. At line ~80 in test_suite/consistency.py : if abs(self.relax.data.res[self.run][index].j0 - j0[index])j0[index]/1e6:print 'Error in residue', self.relax.data.res[self.run][index].num, 'j0 calculated value' return if abs(self.relax.data.res[self.run][index].f_eta - f_eta[index]) > f_eta[index]/1e6: print 'Error in residue', self.relax.data.res[self.run][index].num, 'f_eta calculated value' return if abs(self.relax.data.res[self.run][index].f_r2 - f_r2[index]) > f_r2[index]/1e6: print 'Error in residue', self.relax.data.res[self.run][index].num, 'f_r2 calculated value' return The code specifies that the difference between the calculated value and the reference one should not be ore than a ppm of the reference value... I understand that this is to prevent differences in machine precision from affecting the test, but the 1/1000000 factor seems big and I feel machine rpecision is a lot more... I tried (on my 32 bit x86 Linux machine) to change the code and see its behaviour and it seems that we could simplify this part of code to render it more logic and stringent : if abs(self.relax.data.res[self.run][index].j0 != j0[index]): print 'Error in residue', self.relax.data.res[self.run][index].num, 'j0 calculated value' return if abs(self.relax.data.res[self.run][index].f_eta != f_eta[index]): print 'Error in residue', self.relax.data.res[self.run][index].num, 'f_eta calculated value' return if abs(self.relax.data.res[self.run][index].f_r2 != f_r2[index]): print 'Error in residue', self.relax.data.res[self.run][index].num, 'f_r2 calculated value' return With that change, the reference values could be erroneous to the 16th decimal without affecting the test. Thus, the 15th decimal is now critical, which is more like the other values I pointed in my first post (which must be related to machine precision and portability between architecture, OS, scientific libraries, etc. Sorry if, again, I talk this point (We talked about this in July : https://mail.gna.org/public/relax-devel/2007-07/msg00050.html), but this seems important to me as other reference values are tested in a much less relaxed way... Cheers Séb Sebastien Morin wrote: Hi all As I was verifying that the consistency tests code is ready to be incorporated into the main stream, I found out some strange behaviour in the test suite (1.2, r3370, main stream and consistency tests branch). This behaviour has something to do with, maybe, the precision of numbers we use in relax or maybe also machine precision (two topics I'm really not familiar with)... 1. test_suite/consistency_tests.py (consistency tests branch) If someone changes the following lines (line ~70) : # Correct consistency functions values: j0 = [4.0958793960056238e-09, 3.7976266046729745e-09] f_eta = [0.35164988964635652, 0.32556427866911447] f_r2 = [2.0611470814962761e-09, 1.9117396355237641e-09] for # Correct consistency functions values: j0 = [1.00001*4.0958793960056238e-09, 1.00001*3.7976266046729745e-09] f_eta = [1.00001*0.35164988964635652, 1.00001*0.32556427866911447] f_r2 = [1.00001*2.0611470814962761e-09, 1.00001*1.9117396355237641e-09] The test-suite will not pass. However, if someone changes the lines so they become : # Correct consistency functions values: j0 = [1.000001*4.0958793960056238e-09, 1.000001*3.7976266046729745e-09] f_eta = [1.000001*0.35164988964635652, 1.000001*0.32556427866911447] f_r2 = [1.000001*2.0611470814962761e-09, 1.000001*1.9117396355237641e-09] The test-suite will pass. So what's the real change in the values..? Let's take the first (left) value for f_eta. FAIL : 0.35164988964635652 * 1.00001 = 0.3516534061452529835652 PASS : 0.35164988964635652 * 1.000001 = 0.35165024129624616635652 The real change here happens at the 6th decimal. I'm not really familiar with machine precision and how it's dealt with, but this 6th decimal shouldn't be limiting... 2. test_suite/jw_mapping.py (main stream) The exact same behaviour applies for the values of j0, jwx and jwh in that file. This is quite normal has the code for consistency testing was inspired by the jw mapping code... 3. test_suite/model_free.py (main stream) If someone changes the following lines (line ~910) : # Load the relaxation data. self.relax.interpreter._Relax_data.read(self.run, 'R1', '600', 600.0 * 1e6, 'r1.600.out', dir=path) for : # Load the relaxation data. self.relax.interpreter._Relax_data.read(self.run, 'R1', '600', 1.00000000000001*600.0 * 1e6, 'r1.600.out', dir=path) The test-suite won't pass. However, with : # Load the relaxation data. self.relax.interpreter._Relax_data.read(self.run, 'R1', '600', 1.000000000000001*600.0 * 1e6, 'r1.600.out', dir=path) The test-suite will pass. So what's the real change in the values..? FAIL : 1.00000000000001 * 600.0 = 600.000000000006 = 6.00000000000006 * 10^2 PASS : 1.000000000000001 * 600.0 = 600.0000000000006 = 6.000000000000006 * 10^2 Here, the change is seen at the 14th decimal... Should this be limiting in our calculations..? For the diffusion tensor (line ~920), the 15th decimal seems critical as the following won't pass : # Setup other values. self.relax.interpreter._Diffusion_tensor.init(self.run, 1.000000000000001*10e-9, fixed=1) self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10, 'bond_length') self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa') and the following will : # Setup other values. self.relax.interpreter._Diffusion_tensor.init(self.run, 1.0000000000000001*10e-9, fixed=1) self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10, 'bond_length') self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa') For the r value (line ~920), the 13th decimal seems critical as the following won't pass : # Setup other values. self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9, fixed=1) self.relax.interpreter._Value.set(self.run, 1.0000000000001*1.02 * 1e-10, 'bond_length') self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa') and the following will : # Setup other values. self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9, fixed=1) self.relax.interpreter._Value.set(self.run, 1.00000000000001*1.02 * 1e-10, 'bond_length') self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa') For the CSA value (line ~920), the 12th decimal seems critical as the following won't pass : # Setup other values. self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9, fixed=1) self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10, 'bond_length') self.relax.interpreter._Value.set(self.run, 1.000000000001*-160 * 1e-6, 'csa') and the following will : # Setup other values. self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9, fixed=1) self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10, 'bond_length') self.relax.interpreter._Value.set(self.run, 1.0000000000001*-160 * 1e-6, 'csa') Ok... So what's the point with that ? Is it something we should care about ? Or is it insignificant..? Will this behaviour in precision affect the rest of the calculations ? Why is there variation in the way values are affected ? As I said, I'm really not familiar with precision stuff... So, if my question is naive, don't worry with it, I'm sorry... Cheers Séb -- ______________________________________ _______________________________________________ | | || Sebastien Morin || ||| Etudiant au PhD en biochimie ||| |||| Laboratoire de resonance magnetique nucleaire |||| ||||| Dr Stephane Gagne ||||| |||| CREFSIP (Universite Laval, Quebec, CANADA) |||| ||| 1-418-656-2131 #4530 ||| || || |_______________________________________________| ______________________________________ ________________________________ _______________________________________________ relax (http://nmr-relax.com) This is the relax-devel mailing list relax-devel@xxxxxxx To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-devel -- ______________________________________ _______________________________________________ | | || Sebastien Morin || ||| Etudiant au PhD en biochimie ||| |||| Laboratoire de resonance magnetique nucleaire |||| ||||| Dr Stephane Gagne ||||| |||| CREFSIP (Universite Laval, Quebec, CANADA) |||| ||| 1-418-656-2131 #4530 ||| || || |_______________________________________________| ______________________________________ -- ______________________________________ _______________________________________________ | | || Sebastien Morin || ||| Etudiant au PhD en biochimie ||| |||| Laboratoire de resonance magnetique nucleaire |||| ||||| Dr Stephane Gagne ||||| |||| CREFSIP (Universite Laval, Quebec, CANADA) |||| ||| 1-418-656-2131 #4530 ||| || || |_______________________________________________| ______________________________________ Index: test_suite/jw_mapping.py =================================================================== --- test_suite/jw_mapping.py (revision 3370) +++ test_suite/jw_mapping.py (working copy) @@ -79,13 +79,13 @@ print 'Residue', self.relax.data.res[self.run][index].num, 'unexpectedly not selected' return - if abs(self.relax.data.res[self.run][index].j0 - j0[index])j0[index]/1e6:+ if abs(self.relax.data.res[self.run][index].j0 != j0[index]): print 'Error in residue', self.relax.data.res[self.run][index].num, 'j0 calculated value' return - if abs(self.relax.data.res[self.run][index].jwh - jwh[index]) > jwh[index]/1e6: + if abs(self.relax.data.res[self.run][index].jwh != jwh[index]): print 'Error in residue', self.relax.data.res[self.run][index].num, 'jwh calculated value' return - if abs(self.relax.data.res[self.run][index].jwx - jwx[index]) > jwx[index]/1e6: + if abs(self.relax.data.res[self.run][index].jwx != jwx[index]): print 'Error in residue', self.relax.data.res[self.run][index].num, 'jwx calculated value' return Index: test_suite/consistency_tests.py =================================================================== --- test_suite/consistency_tests.py (revision 3370) +++ test_suite/consistency_tests.py (working copy) @@ -81,13 +81,13 @@ print 'Residue', self.relax.data.res[self.run][index].num, 'unexpectedly not selected' return - if abs(self.relax.data.res[self.run][index].j0 - j0[index])j0[index]/1e6:+ if abs(self.relax.data.res[self.run][index].j0 != j0[index]): print 'Error in residue', self.relax.data.res[self.run][index].num, 'j0 calculated value' return - if abs(self.relax.data.res[self.run][index].f_eta - f_eta[index]) > f_eta[index]/1e6: + if abs(self.relax.data.res[self.run][index].f_eta != f_eta[index]): print 'Error in residue', self.relax.data.res[self.run][index].num, 'f_eta calculated value' return - if abs(self.relax.data.res[self.run][index].f_r2 - f_r2[index]) > f_r2[index]/1e6: + if abs(self.relax.data.res[self.run][index].f_r2 != f_r2[index]): print 'Error in residue', self.relax.data.res[self.run][index].num, 'f_r2 calculated value' return _______________________________________________ relax (http://nmr-relax.com) This is the relax-devel mailing list relax-devel@xxxxxxx To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-devel -- ______________________________________ _______________________________________________ | | || Sebastien Morin || ||| Etudiant au PhD en biochimie ||| |||| Laboratoire de resonance magnetique nucleaire |||| ||||| Dr Stephane Gagne ||||| |||| CREFSIP (Universite Laval, Quebec, CANADA) |||| ||| 1-418-656-2131 #4530 ||| || || |_______________________________________________| ______________________________________ |