Precision in the test-suite -- October 22, 2007 - 18:26

Hi all

As I was verifying that the consistency tests code is ready to be incorporated into the main stream, I found out some strange behaviour in the test suite (1.2, r3370, main stream and consistency tests branch).

This behaviour has something to do with, maybe, the precision of numbers we use in relax or maybe also machine precision (two topics I'm really not familiar with)...

1. test_suite/consistency_tests.py (consistency tests branch)


If someone changes the following lines (line ~70) :

        # Correct consistency functions values:
        j0 = [4.0958793960056238e-09, 3.7976266046729745e-09]
        f_eta = [0.35164988964635652, 0.32556427866911447]
        f_r2 = [2.0611470814962761e-09, 1.9117396355237641e-09]

for

        # Correct consistency functions values:
        j0 = [1.00001*4.0958793960056238e-09, 1.00001*3.7976266046729745e-09]
        f_eta = [1.00001*0.35164988964635652, 1.00001*0.32556427866911447]
        f_r2 = [1.00001*2.0611470814962761e-09, 1.00001*1.9117396355237641e-09]

The test-suite will not pass. However, if someone changes the lines so they become :

        # Correct consistency functions values:
        j0 = [1.000001*4.0958793960056238e-09, 1.000001*3.7976266046729745e-09]
        f_eta = [1.000001*0.35164988964635652, 1.000001*0.32556427866911447]
        f_r2 = [1.000001*2.0611470814962761e-09, 1.000001*1.9117396355237641e-09]

The test-suite will pass. So what's the real change in the values..? Let's take the first (left) value for f_eta.

        FAIL : 0.35164988964635652 * 1.00001  = 0.3516534061452529835652
        PASS : 0.35164988964635652 * 1.000001 = 0.35165024129624616635652

The real change here happens at the 6th decimal. I'm not really familiar with machine precision and how it's dealt with, but this 6th decimal shouldn't be limiting...




2. test_suite/jw_mapping.py (main stream)


The exact same behaviour applies for the values of j0, jwx and jwh in that file. This is quite normal has the code for consistency testing was inspired by the jw mapping code...




3. test_suite/model_free.py (main stream)


If someone changes the following lines (line ~910) :

        # Load the relaxation data.
        self.relax.interpreter._Relax_data.read(self.run, 'R1', '600', 600.0 * 1e6, 'r1.600.out', dir=path)

for :

        # Load the relaxation data.
        self.relax.interpreter._Relax_data.read(self.run, 'R1', '600', 1.00000000000001*600.0 * 1e6, 'r1.600.out', dir=path)

The test-suite won't pass. However, with :

        # Load the relaxation data.
        self.relax.interpreter._Relax_data.read(self.run, 'R1', '600', 1.000000000000001*600.0 * 1e6, 'r1.600.out', dir=path)

The test-suite will pass. So what's the real change in the values..?

        FAIL : 1.00000000000001  * 600.0 = 600.000000000006  = 6.00000000000006  * 10^2
        PASS : 1.000000000000001 * 600.0 = 600.0000000000006 = 6.000000000000006 * 10^2

Here, the change is seen at the 14th decimal... Should this be limiting in our calculations..?


For the diffusion tensor (line ~920), the 15th decimal seems critical as the following won't pass :

        # Setup other values.
        self.relax.interpreter._Diffusion_tensor.init(self.run, 1.000000000000001*10e-9, fixed=1)
        self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10, 'bond_length')
        self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa')

and the following will :

        # Setup other values.
        self.relax.interpreter._Diffusion_tensor.init(self.run, 1.0000000000000001*10e-9, fixed=1)
        self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10, 'bond_length')
        self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa')



For the r value (line ~920), the 13th decimal seems critical as the following won't pass :

        # Setup other values.
        self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9, fixed=1)
        self.relax.interpreter._Value.set(self.run, 1.0000000000001*1.02 * 1e-10, 'bond_length')
        self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa')

and the following will :

        # Setup other values.
        self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9, fixed=1)
        self.relax.interpreter._Value.set(self.run, 1.00000000000001*1.02 * 1e-10, 'bond_length')
        self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa')


For the CSA value (line ~920), the 12th decimal seems critical as the following won't pass :

        # Setup other values.
        self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9, fixed=1)
        self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10, 'bond_length')
        self.relax.interpreter._Value.set(self.run, 1.000000000001*-160 * 1e-6, 'csa')

and the following will :

        # Setup other values.
        self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9, fixed=1)
        self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10, 'bond_length')
        self.relax.interpreter._Value.set(self.run, 1.0000000000001*-160 * 1e-6, 'csa')



Ok... So what's the point with that ? Is it something we should care about ? Or is it insignificant..? Will this behaviour in precision affect the rest of the calculations ? Why is there variation in the way values are affected ?

As I said, I'm really not familiar with precision stuff... So, if my question is naive, don't worry with it, I'm sorry...

Cheers


Séb
--

         ______________________________________    
     _______________________________________________
    |                                               |
   || Sebastien Morin                               ||
  ||| Etudiant au PhD en biochimie                  |||
 |||| Laboratoire de resonance magnetique nucleaire ||||
||||| Dr Stephane Gagne                             |||||
 |||| CREFSIP (Universite Laval, Quebec, CANADA)    ||||
  ||| 1-418-656-2131 #4530                          |||
   ||                                               ||
    |_______________________________________________|
         ______________________________________