Re: System testsRe: System testsHi Ed, I just tested if the problems were still present after the many changes the 1.3 line has been made to. The problem is still present so I will fill a bug report... Cheers, Séb Edward d'Auvergne wrote:
Hi, I'm not sure what is causing these problems, but the issue is not important. Nevertheless, could you create a bug report for this so I can make the system tests work on all systems? Cheers, Edward On Mon, May 25, 2009 at 8:06 PM, Sébastien Morin<sebastien.morin.1@xxxxxxxxx> wrote:Hi Ed, [Olivier Fisette is in CC since he helped me tests several things on different machines...] I just applied r9061. The two failures are still present on the three failing machines, and still not on the one machine that already was fine... Hence, r9061 doesn't seem to change anything... ... Running the tests three times in a row (on one of the failing machines), I get the following variations: ======================================= Constrained BFGS opt, backtracking line search {S2=0.970, te=2048, Rex=0.149} ======================================= s2: 0.9700000000014879 te: 2048.0000002802667 rex: 0.14899999997179250 chi2: 2.8554367542689649e-20 iter: 206 f_count: 906 g_count: 214 h_count: 0 warning: None s2: 0.9700000000018254 te: 2048.0000003419927 rex: 0.14899999996525742 chi2: 4.2518045538462617e-20 iter: 161 f_count: 691 g_count: 168 h_count: 0 warning: None s2: 0.9699999999994500 te: 2047.9999998975570 rex: 0.14900000001042418 chi2: 3.8190859969802086e-21 iter: 189 f_count: 1050 g_count: 198 h_count: 0 warning: None ======================================= Constrained BFGS opt, More and Thuente line search {S2=0.970, te=2048, Rex=0.149} ======================================= s2: 0.9700000000000611 te: 2048.0000000116133 rex: 0.14899999999884833 chi2: 4.858663172235886e-23 iter: 122 f_count: 436 g_count: 436 h_count: 0 warning: None s2: 0.9700000000000610 te: 2048.0000000116065 rex: 0.14899999999884858 chi2: 4.8676674275557319e-23 iter: 120 f_count: 382 g_count: 382 h_count: 0 warning: None s2: 0.9700000000000601 te: 2048.0000000114451 rex: 0.14899999999886407 chi2: 4.7283344579074642e-23 iter: 120 f_count: 322 g_count: 322 h_count: 0 warning: None Are these variations normal, or bigger than usual ??? ... Program versions are as follow on these Gentoo Linux system where the failures are present: System: Linux Release: 2.6.28-gentoo-r5 Version: #1 SMP Sat Apr 25 13:31:51 EDT 2009 Win32 version: Distribution: Architecture: 32bit ELF Machine: i686 Processor: Intel(R) Pentium(R) M processor 1.80GHz Python version: 2.5.4 numpy version: 1.2.1 System: Linux Release: 2.6.28-gentoo-r5 Version: #1 SMP Mon Apr 27 14:04:48 EDT 2009 Win32 version: Distribution: Architecture: 32bit ELF Machine: i686 Processor: Intel(R) Pentium(R) 4 CPU 3.20GHz Python version: 2.5.4 numpy version: 1.2.1 System: Linux Release: 2.6.28-gentoo-r5 Version: #1 SMP Mon Apr 27 14:02:44 EDT 2009 Win32 version: Distribution: Architecture: 32bit ELF Machine: i686 Processor: Intel(R) Pentium(R) 4 CPU 3.20GHz Python version: 2.5.4 numpy version: 1.2.1 ... I also checked on a system for which the tests passed... The only notable difference was the kernel being 2.6.24 on this system, whereas it was 2.6.28 on the others: System: Linux Release: 2.6.24-gentoo-r8 Version: #1 SMP Wed Jun 18 09:14:35 EDT 2008 Win32 version: Distribution: Architecture: 32bit ELF Machine: i686 Processor: Intel(R) Xeon(TM) CPU 2.80GHz Python version: 2.5.4 numpy version: 1.2.1 Moreover, if using other 64 bit machines with some recent kernels (2.6.27 or 2.6.28), the tests pass... ... Could theses failures be limited to 32-bit systems running a recent Linux kernel (> 2.6.24)..? Regards, Séb :) Edward d'Auvergne wrote:Hi, These failures are not too bad. The first is just because of the precision differences of the combination of the operating system, CPU architecture, Python version, and numpy version. Would it be possible to try this on python 2.4? If the exact cause can be found, we can catch it on line 388 of test_suite/system_tests/model_free.py (in the 1.3 line) and add an exception. If the hardware and software appears to be the same, try modifying the code so that this test fails on all systems. Then you'll have the printout of CPU and software details to help track down a difference. The second test is a worry though. I'm not sure what's happening, but the precision is horrible on this setup. The unittest.TestCase.assertAlmostEqual() method is used to check this to the default of 7 decimal places. I'll shift the test from picoseconds to nanoseconds to fix the problem in this case, but still something is not ideal with this combination of hardware+software. Regards, Edward On Fri, May 15, 2009 at 6:34 PM, Sébastien Morin <sebastien.morin.1@xxxxxxxxx> wrote:Hi, Running the system tests, depending on the actual Linux system I use, the errors I get are different... Prsently, the system tests on relax-1.3 should give rise to 3 errors. On one system in the lab here, we observe this behaviour. Fine. However, on others systems, we observe two additional failures: ============================================ ============================================ FAIL: Constrained BFGS opt, backtracking line search {S2=0.970, te=2048, Rex=0.149} Parameter values: array([ 9.70000000e-01, 2.04800000e+03, 1.49000000e-01]) Function value: 4.762657780645096e-23 Iterations: 120 Function calls: 386 Gradient calls: 386 Hessian calls: 0 Warning: None Traceback (most recent call last): File "/home/semor/pse-4/collaborations/relax/relax-1.3/test_suite/system_tests/model_free.py", line 394, in test_opt_constr_bfgs_mt_S2_0_970_te_2048_Rex_0_149 self.value_test(spin, select, s2, te, rex, chi2, iter, f_count, g_count, h_count, warning) File "/home/semor/pse-4/collaborations/relax/relax-1.3/test_suite/system_tests/model_free.py", line 1109, in value_test self.assertEqual(spin.f_count, f_count, msg=mesg) AssertionError: Optimisation failure. System: Linux Release: 2.6.28-gentoo-r5 Version: #1 SMP Sat Apr 25 13:31:51 EDT 2009 Win32 version: Distribution: Architecture: 32bit ELF Machine: i686 Processor: Intel(R) Pentium(R) M processor 1.80GHz Python version: 2.5.4 numpy version: 1.2.1 s2: 0.9700000000000604 te: 2048.0000000114946 rex: 0.14899999999885985 chi2: 4.762657780645096e-23 iter: 120 f_count: 386 g_count: 386 h_count: 0 warning: None ============================================ ============================================ ============================================ ============================================ FAIL: Constrained BFGS opt, More and Thuente line search {S2=0.970, te=2048, Rex=0.149} Parameter values: array([ 9.70000000e-01, 2.04800000e+03, 1.49000000e-01]) Function value: 1.9223825944220359e-20 Iterations: 157 Function calls: 722 Gradient calls: 164 Hessian calls: 0 Warning: None Traceback (most recent call last): File "/home/semor/pse-4/collaborations/relax/relax-1.3/test_suite/system_tests/model_free.py", line 337, in test_opt_constr_bfgs_back_S2_0_970_te_2048_Rex_0_149 self.value_test(spin, select, s2, te, rex, chi2, iter, f_count, g_count, h_count, warning) File "/home/semor/pse-4/collaborations/relax/relax-1.3/test_suite/system_tests/model_free.py", line 1105, in value_test self.assertAlmostEqual(spin.te / 1e-12, te, msg=mesg) AssertionError: Optimisation failure. System: Linux Release: 2.6.28-gentoo-r5 Version: #1 SMP Sat Apr 25 13:31:51 EDT 2009 Win32 version: Distribution: Architecture: 32bit ELF Machine: i686 Processor: Intel(R) Pentium(R) M processor 1.80GHz Python version: 2.5.4 numpy version: 1.2.1 s2: 0.9700000000012307 te: 2048.0000002299716 rex: 0.14899999997647859 chi2: 1.9223825944220359e-20 iter: 157 f_count: 722 g_count: 164 h_count: 0 warning: None ============================================ ============================================ All dependencies are up-to-date and at the same versions (python, scientific python, numpy, numeric, minfx). Moreover, the system tests used to pass equally on all these machines, so it might not be a precision issue, although this is not impossible since the different machines have slightly different architecture... Someone has any clue what is causing this problem..? Could this be a bug in relax test suite ? Cheers, Séb -- Sébastien Morin PhD Student S. Gagné NMR Laboratory Université Laval & PROTEO Québec, Canada _______________________________________________ relax (http://nmr-relax.com) This is the relax-devel mailing list relax-devel@xxxxxxx To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-devel-- Sébastien Morin PhD Student S. Gagné NMR Laboratory Université Laval & PROTEO Québec, Canada
-- Sébastien Morin PhD Student S. Gagné NMR Laboratory Université Laval & PROTEO Québec, Canada