Hi again
After a look in the test_suite/consistency.py and
test_suite/jw_mapping.py files, I realized that the 6th decimal
precision behaviour was directly encoded.
At line ~80 in test_suite/consistency.py :
if abs(self.relax.data.res[self.run][index].j0 -
j0[index]) > j0[index]/1e6:
print 'Error in residue',
self.relax.data.res[self.run][index].num, 'j0 calculated value'
return
if abs(self.relax.data.res[self.run][index].f_eta -
f_eta[index]) > f_eta[index]/1e6:
print 'Error in residue',
self.relax.data.res[self.run][index].num, 'f_eta calculated value'
return
if abs(self.relax.data.res[self.run][index].f_r2 -
f_r2[index]) > f_r2[index]/1e6:
print 'Error in residue',
self.relax.data.res[self.run][index].num, 'f_r2 calculated value'
return
The code specifies that the difference between the calculated value and
the reference one should not be ore than a ppm of the reference
value... I understand that this is to prevent differences in machine
precision from affecting the test, but the 1/1000000 factor seems big
and I feel machine rpecision is a lot more...
I tried (on my 32 bit x86 Linux machine) to change the code and see its
behaviour and it seems that we could simplify this part of code to
render it more logic and stringent :
if
abs(self.relax.data.res[self.run][index].j0 != j0[index]):
print 'Error in residue',
self.relax.data.res[self.run][index].num, 'j0 calculated value'
return
if abs(self.relax.data.res[self.run][index].f_eta !=
f_eta[index]):
print 'Error in residue',
self.relax.data.res[self.run][index].num, 'f_eta calculated value'
return
if abs(self.relax.data.res[self.run][index].f_r2 !=
f_r2[index]):
print 'Error in residue',
self.relax.data.res[self.run][index].num, 'f_r2 calculated value'
return
With that change, the reference values could be erroneous to the 16th
decimal without affecting the test. Thus, the 15th decimal is now
critical, which is more like the other values I pointed in my first
post (which must be related to machine precision and portability
between architecture, OS, scientific libraries, etc.
Sorry if, again, I talk this point (We talked about this in July :
https://mail.gna.org/public/relax-devel/2007-07/msg00050.html), but
this seems important to me as other reference values are tested in a
much less relaxed way...
Cheers
Séb
Sebastien Morin wrote:
Hi
all
As I was verifying that the consistency tests code is ready to be
incorporated into the main stream, I found out some strange behaviour
in the test suite (1.2, r3370, main stream and consistency tests
branch).
This behaviour has something to do with, maybe, the precision of
numbers we use in relax or maybe also machine precision (two topics I'm
really not familiar with)...
1. test_suite/consistency_tests.py (consistency tests branch)
If someone changes the following lines (line ~70)
:
# Correct consistency functions values:
j0 = [4.0958793960056238e-09, 3.7976266046729745e-09]
f_eta = [0.35164988964635652, 0.32556427866911447]
f_r2 = [2.0611470814962761e-09, 1.9117396355237641e-09]
for
# Correct consistency functions values:
j0 = [1.00001*4.0958793960056238e-09,
1.00001*3.7976266046729745e-09]
f_eta = [1.00001*0.35164988964635652,
1.00001*0.32556427866911447]
f_r2 = [1.00001*2.0611470814962761e-09,
1.00001*1.9117396355237641e-09]
The test-suite will not pass. However, if someone changes the lines so
they become :
# Correct consistency functions values:
j0 = [1.000001*4.0958793960056238e-09,
1.000001*3.7976266046729745e-09]
f_eta = [1.000001*0.35164988964635652,
1.000001*0.32556427866911447]
f_r2 = [1.000001*2.0611470814962761e-09,
1.000001*1.9117396355237641e-09]
The test-suite will pass. So what's the real change in the values..?
Let's take the first (left) value for f_eta.
FAIL : 0.35164988964635652 * 1.00001 = 0.3516534061452529835652
PASS : 0.35164988964635652 * 1.000001 =
0.35165024129624616635652
The real change here happens at the 6th decimal. I'm not really
familiar with machine precision and how it's dealt with, but this 6th
decimal shouldn't be limiting...
2. test_suite/jw_mapping.py (main stream)
The exact same behaviour applies for the values of j0, jwx and jwh in
that file. This is quite normal has the code for consistency testing
was inspired by the jw mapping code...
3. test_suite/model_free.py (main stream)
If someone changes the following lines (line ~910) :
# Load the relaxation data.
self.relax.interpreter._Relax_data.read(self.run, 'R1', '600',
600.0 * 1e6, 'r1.600.out', dir=path)
for :
# Load the relaxation data.
self.relax.interpreter._Relax_data.read(self.run,
'R1', '600', 1.00000000000001*600.0
* 1e6, 'r1.600.out', dir=path)
The test-suite won't pass. However, with :
# Load the relaxation data.
self.relax.interpreter._Relax_data.read(self.run,
'R1', '600', 1.000000000000001*600.0 * 1e6, 'r1.600.out', dir=path)
The test-suite will pass. So what's the real change in the values..?
FAIL : 1.00000000000001 * 600.0 = 600.000000000006
= 6.00000000000006 * 10^2
PASS : 1.000000000000001
* 600.0 = 600.0000000000006 = 6.000000000000006
* 10^2
Here, the change is seen at the 14th decimal... Should this be limiting
in our calculations..?
For the diffusion tensor (line ~920), the 15th
decimal seems critical as the following won't pass :
# Setup other values.
self.relax.interpreter._Diffusion_tensor.init(self.run,
1.000000000000001*10e-9, fixed=1)
self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10,
'bond_length')
self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa')
and the following will :
# Setup other values.
self.relax.interpreter._Diffusion_tensor.init(self.run,
1.0000000000000001*10e-9, fixed=1)
self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10,
'bond_length')
self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa')
For the r value (line ~920), the 13th decimal
seems critical as the following won't pass :
# Setup other values.
self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9,
fixed=1)
self.relax.interpreter._Value.set(self.run,
1.0000000000001*1.02 * 1e-10, 'bond_length')
self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa')
and the following will :
# Setup other values.
self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9,
fixed=1)
self.relax.interpreter._Value.set(self.run,
1.00000000000001*1.02 * 1e-10, 'bond_length')
self.relax.interpreter._Value.set(self.run, -160 * 1e-6, 'csa')
For the CSA value (line ~920), the 12th decimal seems critical as the
following won't pass :
# Setup other values.
self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9,
fixed=1)
self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10,
'bond_length')
self.relax.interpreter._Value.set(self.run, 1.000000000001*-160
* 1e-6, 'csa')
and the following will :
# Setup other values.
self.relax.interpreter._Diffusion_tensor.init(self.run, 10e-9,
fixed=1)
self.relax.interpreter._Value.set(self.run, 1.02 * 1e-10,
'bond_length')
self.relax.interpreter._Value.set(self.run,
1.0000000000001*-160 * 1e-6, 'csa')
Ok... So what's the point with that ? Is it something we should care
about ? Or is it insignificant..? Will this behaviour in precision
affect the rest of the calculations ? Why is there variation in the way
values are affected ?
As I said, I'm really not familiar with precision stuff... So, if my
question is naive, don't worry with it, I'm sorry...
Cheers
Séb
--
______________________________________
_______________________________________________
| |
|| Sebastien Morin ||
||| Etudiant au PhD en biochimie |||
|||| Laboratoire de resonance magnetique nucleaire ||||
||||| Dr Stephane Gagne |||||
|||| CREFSIP (Universite Laval, Quebec, CANADA) ||||
||| 1-418-656-2131 #4530 |||
|| ||
|_______________________________________________|
______________________________________
_______________________________________________
relax (http://nmr-relax.com)
This is the relax-devel mailing list
relax-devel@xxxxxxx
To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-devel
--
______________________________________
_______________________________________________
| |
|| Sebastien Morin ||
||| Etudiant au PhD en biochimie |||
|||| Laboratoire de resonance magnetique nucleaire ||||
||||| Dr Stephane Gagne |||||
|||| CREFSIP (Universite Laval, Quebec, CANADA) ||||
||| 1-418-656-2131 #4530 |||
|| ||
|_______________________________________________|
______________________________________
|
Re: Precision in the test-suite