[bug #20821] Optimisation failure / round error on windows 64 bit -- June 20, 2013 - 09:18

Follow-up Comment #5, bug #20821 (project relax):

Reply to bug #20821: (https://gna.org/bugs/?20821) Optimisation failure /
round error on windows 64 bit

I experienced more precision problems.

I had a terrible need, to turn down precision, quite a lot.

So much, that I think something is wrong with the MS 
Visual Studio Express """ Professional """ 64 bit compilation.

-            self.assertAlmostEqual(spin.s2, s2, msg=mesg)
+            self.assertAlmostEqual(spin.s2, s2, places=6, msg=mesg)

-            self.assertAlmostEqual(spin.te / 1e-12, te, 5, msg=mesg)
+            self.assertAlmostEqual(spin.te / 1e-12, te, places=1, msg=mesg)

-            self.assertAlmostEqual(spin.rex * (2.0 * pi *
cdp.spectrometer_frq[cdp.ri_ids[0]])**2, rex * (2.0 * pi *
cdp.spectrometer_frq[cdp.ri_ids[0]])**2, msg=mesg)
+            self.assertAlmostEqual(spin.rex * (2.0 * pi *
cdp.spectrometer_frq[cdp.ri_ids[0]])**2, rex * (2.0 * pi *
cdp.spectrometer_frq[cdp.ri_ids[0]])**2, places=4, msg=mesg)

        # Optimisation values.
        s2 = 0.9699999999999995
        te = 2048.000000000022283
        rex = 0.14900000000000566 / (2.0 * pi *
cdp.spectrometer_frq[cdp.ri_ids[0]])**2

My value
# s2: 0.9700002183674102 : Diff -2.18367410709e-07
# te: 2048.015293187 Diff: -0.0152931869779
# rex: 0.14899473115727899 Diff: 5.26884272667e-06

# 64-bit Windows 7 (https://gna.org/bugs/?20821).
# System: Windows
# Release: 7
# Version: 6.1.7601
# Architecture: 64bit WindowsPE
# Machine: AMD64
# Processor: Intel64 Family 6 Model 37 Stepping 2, GenuineIntel
# Python version: 2.7.5
# numpy version: 1.7.1
# Compilation: Visual Studio Express 2012, Professional, for 64 bit
compilation
# s2: 0.9700002183674102
# te: 2048.015293187
# rex: 0.14899473115727899
# chi2: 2.3195994119090742e-10
# iter: 116
# f_count: 411
# g_count: 411
# h_count: 0
# warning: None

(file #18123)
    _______________________________________________________

Additional Item Attachment:

File name: patch3                         Size:2 KB


    _______________________________________________________

Reply to this item at:

  <http://gna.org/bugs/?20821>

_______________________________________________
  Message sent via/by Gna!
  http://gna.org/