Hi, This appears to be another bug associated with this type of grid search. Hopefully I'll have this one solved soon, but it's proving more difficult. Cheers, Edward On 14 April 2011 13:28, Tiago Pais <tpais@xxxxxxxxxxx> wrote:
Hi, But the odd thing is that, removing the first spin from the analysis the problem repeats itself but now starting on the previous third spin. So, the first spin to be analyzed seems to work well but something happens after that that interferes with the subsequent grid searches. If I use Newton minimization after that I loose the constraint for the S2f, right? Regards Tiago P -----Original Message----- From: edward.dauvergne@xxxxxxxxx [mailto:edward.dauvergne@xxxxxxxxx] On Behalf Of Edward d'Auvergne Sent: quinta-feira, 14 de Abril de 2011 9:05 To: Tiago Pais Cc: relax-devel@xxxxxxx Subject: Re: [bug #18019] Gridsearch appears not to be working when upper/lower bounds are set Hi, I noticed that as well. This is a consequence of the constraint that S2f = 0.111. If you optimise using Newton minimisation after that, the results are quite different. Higher models cannot be used unless you collect data at multiple field strengths. I'm not sure what you can do here. Regards, Edward On 13 April 2011 09:54, Tiago Pais <tpais@xxxxxxxxxxx> wrote:Hi Edward, The grid_search seems to be working well for the S2 parameters but the ts fails, returning zero to all spins after the first one. It is strange because after grid_search finishes the first spin you no longer see the values of the ts changing. In the end S2 and S2s seem fine but ts returns zeros for all other spins that are not the first of the sequence. Regards, Tiago -----Original Message----- From: edward.dauvergne@xxxxxxxxx [mailto:edward.dauvergne@xxxxxxxxx] On Behalf Of Edward d'Auvergne Sent: terça-feira, 12 de Abril de 2011 16:27 To: Tiago Pais Cc: relax-devel@xxxxxxx Subject: Re: [bug #18019] Gridsearch appears not to be working when upper/lower bounds are set Hi, A grid search can find close to the minimum. If it is fine enough, it should find it reasonably well. The only problem is if the space is highly convoluted, and then a course grid search can be way off. This happens from model m5 a lot, unfortunately. Have a look at my "Optimisation of NMR dynamic models I" paper for a description of this problem (http://www.nmr-relax.com/refs.html). Another problem in this case the assumption of a fast motion at S2f = 0.111 and tf = 0, and then a single motion on top of that, but that is not something I have tackled yet. Do you have a reference for doing this? Regards, Edward On 12 April 2011 17:15, Tiago Pais <tpais@xxxxxxxxxxx> wrote:Yes, I was just playing with those values also and I have just seen that evenasmall increase is already giving different results. As for the zoominggridsearch I don't really know how to implement it and it would probably take more time than I can spare at the moment, unfortunately. One more question: will grid_search find the real minimum? Thanks for all the effort. Cheers Tiago P -----Original Message----- From: edward.dauvergne@xxxxxxxxx [mailto:edward.dauvergne@xxxxxxxxx] On Behalf Of Edward d'Auvergne Sent: terça-feira, 12 de Abril de 2011 16:02 To: Tiago Pais Cc: relax-devel@xxxxxxx Subject: Re: [bug #18019] Gridsearch appears not to be working when upper/lower bounds are set Hi, You might just need to increase the number of grid points to maybe [1, 2001, 2001]. If you don't implement a zooming grid search, you may need to increase the points even more! And the limits for ts should probably be increased to the nanosecond timescale - trust me, you will see motions at that scale! I'm running the following command now and it's looking quite good: relax> grid_search(lower=[0.111, 0, 0.0], upper=[0.111, 1.0, 2e-9], inc=[1, 2001, 2001], constraints=True, verbosity=1) It does take a while without the zooming though. As for those skipped points, they are valid. The S2 value is less than S2f hence, if used, would mean that S2s is >1. If the constraints are violated, the grid points are skipped to speed up the computation time. Regards, Edward On 12 April 2011 16:33, Tiago Pais <tpais@xxxxxxxxxxx> wrote:It seems to work fine for the first spin but it seems to go crazy on the following spins (although it seems strange that the S2 value in thearrayisalways the same): Fitting to spin '#snPHSinDiffFrame2CHnoHs_mol1:14&:L@119&@CD1' ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Grid search ~~~~~~~~~~~ Searching through 256 grid nodes. k: 0 xk: [ 0.111, 0, 0] fk:16513.3641708k: 1 xk: [ 0.111, 0.066667, 0] fk:10390.3333598k: 2 xk: [ 0.111, 0, 13.333] fk:7092.56902759k: 3 xk: [ 0.111, 0.066667, 13.333] fk: 4780.221211 k: 5 xk: [ 0.111, 0.066667, 26.667] fk:2820.12641286k: 7 xk: [ 0.111, 0.066667, 40] fk:2039.04303462k: 9 xk: [ 0.111, 0.066667, 53.333] fk:1700.9699681k: 11 xk: [ 0.111, 0.066667, 66.667] fk:1546.15216207k: 13 xk: [ 0.111, 0.066667, 80] fk:1476.32033148k: 15 xk: [ 0.111, 0.066667, 93.333] fk:1454.68522627Fitting to spin '#snPHSinDiffFrame2CHnoHs_mol1:37&:L@506&@CD1' ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Grid search ~~~~~~~~~~~ Searching through 256 grid nodes. k: 0 xk: [ 0.111, 0, 0] fk:5632.02607401k: 1 xk: [ 0.111, 0.066667, 0] fk:4534.98672847And if I increase the verbosity you can see that it just goes through alotof nodes that violate the constraint and the grid point is skipped: k: 2 xk: [ 0.111, 0.066667, 0] Constraint violated, skipping grid point. ci: array([ 1.11000000e-01, 8.89000000e-01, 1.00000000e+00, 0.00000000e+00, -8.89000000e-01, 1.33333333e+14, -1.33333333e+14]) k: 2 xk: [ 0.111, 0.066667, 0] Constraint violated, skipping grid point. ci: array([ 1.11000000e-01, 8.89000000e-01, 0.00000000e+00, 1.00000000e+00, 1.11000000e-01, 1.46666667e+14, -1.46666667e+14]) Cheers TP -----Original Message----- From: edward.dauvergne@xxxxxxxxx [mailto:edward.dauvergne@xxxxxxxxx] On Behalf Of Edward d'Auvergne Sent: terça-feira, 12 de Abril de 2011 15:06 To: Tiago Pais Cc: relax-devel@xxxxxxx Subject: Re: [bug #18019] Gridsearch appears not to be working when upper/lower bounds are set Hi, The relax version is the most recent, and the --info print out indicates that the minfx used is correct (assuming the new version of relax is at /usr/local/relax-1.3). This is quite peculiar :S If you use the grid search command: relax> grid_search(lower=[0.111, 0, 0.0], upper=[0.111, 1.0, 200e-12], inc=[1, 16, 16], constraints=True, verbosity=1) What do you see? Try copying and pasting this line into the script. Cheers, Edward On 12 April 2011 15:39, Tiago Pais <tpais@xxxxxxxxxxx> wrote:Sorry Edward, still not working... The checked out revision of the repository relax version is number12819The errors appear to be exactly the same. What kind of values are you getting for S2 and tauS? Here, I am getting 0 for all tauS and 0.06667forall S2. $relax --info plots the following: relax repository checkout Molecular dynamics by NMR data analysis Copyright (C) 2001-2006 Edward d'Auvergne Copyright (C) 2006-2011 the relax developmentteamThis is free software which you are welcome to modify and redistributeunderthe conditions of the GNU General Public License (GPL). This program, including all modules,islicensed under the GPL and comes with absolutely no warranty. For details type 'GPL' withintherelax prompt. Assistance in using the relax prompt and scripting interface can beaccessedby typing 'help' within the prompt. Hardware information: Machine: i686 Processor: System information: System: Linux Release: 2.6.31-23-generic Version: #74-Ubuntu SMP Mon Feb 28 21:32:57 UTC 2011 GNU/Linux version: Ubuntu 9.10 karmic Distribution: Ubuntu 9.10 karmic Full platform string: Linux-2.6.31-23-generic-i686-with-Ubuntu-9.10-karmic Software information: Architecture: 32bit ELF Python version: 2.6.4 Python branch: tags/r264 Python build: r264:75706, Dec 7 2009 18:45:15 Python compiler: GCC 4.4.1 Python implementation: CPython Python revision: 75706 Numpy version: 1.3.0 Libc version: glibc 2.4 Python packages (most are optional): Package Installed Version Path minfx True Unknown /usr/local/relax-1.3/minfx bmrblib False numpy True 1.3.0 /usr/lib/python2.6/dist-packages/numpy ScientificPython True 2.8 /usr/lib/python2.6/dist-packages/Scientific wxPython False mpi4py False epydoc False optparse True 1.5.3 /usr/lib/python2.6/optparse.pyc readline True /usr/lib/python2.6/lib-dynload/readline.so profile True /usr/lib/python2.6/profile.pyc bz2 True /usr/lib/python2.6/lib-dynload/bz2.so gzip True /usr/lib/python2.6/gzip.pyc os.devnull True /usr/lib/python2.6/os.pyc Compiled relax C modules: Relaxation curve fitting: True Sorry for all the trouble. Cheers Tiago P -----Original Message----- From: edward.dauvergne@xxxxxxxxx [mailto:edward.dauvergne@xxxxxxxxx] On Behalf Of Edward d'Auvergne Sent: terça-feira, 12 de Abril de 2011 13:02 To: Tiago Pais Cc: relax-devel@xxxxxxx Subject: Re: [bug #18019] Gridsearch appears not to be working when upper/lower bounds are set Hi, Where is the new one installed? Maybe relax is not using it. Are the errors 100% the same? It's working for me perfectly with the truncated data. I removed residue 37, and the grid search gives: ----- relax> grid_search(lower=[0.111, 0, 0.0], upper=[0.111, 1.0, 2.0000000000000001e-10], inc=[1, 16, 16], constraints=True, verbosity=1) Over-fit spin deselection. RelaxWarning: The spin '#snPHSinDiffFrame2CHnoHs_mol1:37&:L@506&@CD1' has been deselected because of missing relaxation data. Only the model-free parameters for single spins will be used. Fitting to spin '#snPHSinDiffFrame2CHnoHs_mol1:14&:L@119&@CD1' ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Grid search ~~~~~~~~~~~ Searching through 256 grid nodes. k: 0 xk: [ 0.111, 0, 0] fk:16513.3641708k: 1 xk: [ 0.111, 0.066667, 0] fk:10390.3333598k: 2 xk: [ 0.111, 0, 13.333] fk:7092.56902759k: 3 xk: [ 0.111, 0.066667, 13.333] fk:4780.221211k: 5 xk: [ 0.111, 0.066667, 26.667] fk:2820.12641286k: 7 xk: [ 0.111, 0.066667, 40] fk:2039.04303462k: 9 xk: [ 0.111, 0.066667, 53.333] fk:1700.9699681k: 11 xk: [ 0.111, 0.066667, 66.667] fk:1546.15216207k: 13 xk: [ 0.111, 0.066667, 80] fk:1476.32033148k: 15 xk: [ 0.111, 0.066667, 93.333] fk:1454.68522627----- Maybe you need to newest version of relax as well. Try: $ svn co http://svn.gna.org/svn/relax/1.3 relax-1.3 $ cd relax-1.3 $ svn co http://svn.gna.org/svn/minfx/trunk/minfx $ cd .. $ scons and then see if things run? Cheers, Edward On 12 April 2011 13:49, Tiago Pais <tpais@xxxxxxxxxxx> wrote:Hi, I have deleted the previous minfx and installed the new one but It'sstillnot working. I also tried to increase the interval of S2f and put some increments but the same error occurs. Did you manage to do it with the truncated data? TiagoP -----Original Message----- From: edward.dauvergne@xxxxxxxxx [mailto:edward.dauvergne@xxxxxxxxx]OnBehalf Of Edward d'Auvergne Sent: terça-feira, 12 de Abril de 2011 10:27 To: Tiago Pais Cc: relax-devel@xxxxxxx Subject: Re: [bug #18019] Gridsearch appears not to be working when upper/lower bounds are set Hi, I found the problem to be in minfx. Strangely, this used to work before as I had performed grid searches with an increment of 1 in one of the dimensions. The problem is that when there is an increment of one, you need to pick a value between the lower and upper bound values. Minfx is now taking the average value. To use this, you can delete the minfx directory in the 1.3.8 installation, and get the new version of minfx by typing: $ svn co http://svn.gna.org/svn/minfx/trunk/minfx If you don't have access to svn, you might have to wait until I release a new version of relax. However due to holidays, I will unfortunately not be able to do this before mid May. Regards, Edward On 12 April 2011 10:38, Tiago Pais <tpais@xxxxxxxxxxx> wrote:Hi Edward, Yes I am sorry about the private label. I set it initially to private because I was going to include the structure of the protein attached,butthen I realized that I didn't need to send the whole thing.EventuallyIforgot to set the label back to public. So, yes, you can use the infoasyouwish to try and solve the issue - including the test suite. Hope to hear from you soon. Cheers TP -----Original Message----- From: Edward d'Auvergne [mailto:edward.dauvergne@xxxxxxxxx] Sent: terça-feira, 12 de Abril de 2011 7:51 To: anonymous Cc: Tiago Pais; relax-devel@xxxxxxx Subject: Re: [bug #18019] Gridsearch appears not to be working when upper/lower bounds are set Hi, Is there any reason this bug report has been made private? The data is so truncated, there's nothing much anyone can do with it. The privacy tag is usually reserved for security issues whereby it needs to be kept private until a fix is found and released. Then the bug can be made public. I'm looking into this now, but I'm not exactly sure what is happening. Is it ok if I include this data, or part of it, into the relax test suite? This will significantly help in the debugging and will make sure this bug never returns! Cheers, Edward On 11 April 2011 13:42, anonymous <NO-REPLY.INVALID-ADDRESS@xxxxxxx>wrote:URL: <http://gna.org/bugs/?18019> Summary: Gridsearch appears not to be working when upper/lower bounds are set Project: relax Submitted by: None Submitted on: Mon 11 Apr 2011 11:42:49 AM UTC Category: relax's source code Severity: 3 - Normal Priority: 5 - Normal Status: None Privacy: Private Assigned to: None Originator Name: Tiago Pais Originator Email: tpais@xxxxxxxxxxx Open/Closed: Open Discussion Lock: Any Release: 1.3.8 Operating System: GNU/Linux _______________________________________________________ Details: My system is Ubuntu-9.04: When I perform a gridsearch, exclusively for m5, with the lower andupperbounds set to: lower=[0.111, 0, -0.0], upper=[0.111, 1.00, 200] Ikeepseeingthe following message during the calculations: "Min f: 3474.82 k: 3860 xk: [ 0.111, 0.066667, -0] fk: nan Increment: array([ 5, 16, 16]) Params: array([ 0.111, 1. , Inf]) Min params: array([ 0.111 , 0.06666667, -0. ]) f: nan Min f: 3474.82 k: 3861 xk: [ 0.111, 0.066667, -0] fk: nan Increment: array([ 6, 16, 16]) Params: array([ 0.111, 1. , Inf]) Min params: array([ 0.111 , 0.06666667, -0. ]) f: nan " And at the end the tauS is zero for all spins and the S2s is alwayscloseto1. Attached follows the script that I am using and a truncated sampleofmydataset (I hope it is not too truncated!) as well as the results that Iamgetting. If you need more info let me know. Thanks Regards Tiago P _______________________________________________________ File Attachments: ------------------------------------------------------- Date: Mon 11 Apr 2011 11:42:49 AM UTC Name: ErrorReportRelax.tar.gzSize:8kB By: None <http://gna.org/bugs/download.php?file_id=12833> _______________________________________________________ Reply to this item at: <http://gna.org/bugs/?18019> _______________________________________________ Message sent via/by Gna! http://gna.org/ _______________________________________________ relax (http://nmr-relax.com) This is the relax-devel mailing list relax-devel@xxxxxxx To unsubscribe from this list, get a password reminder, or change your subscription options, visit the list information page at https://mail.gna.org/listinfo/relax-devel__________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6034 (20110411) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6034 (20110411) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com__________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6034 (20110411) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6035 (20110412) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com__________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6035 (20110412) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6035 (20110412) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com__________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6035 (20110412) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6035 (20110412) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com__________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6036 (20110412) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6036 (20110412) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com__________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6036 (20110412) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __________ Information from ESET NOD32 Antivirus, version of virussignaturedatabase 6037 (20110412) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com__________ Information from ESET NOD32 Antivirus, version of virus signature database 6039 (20110413) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com __________ Information from ESET NOD32 Antivirus, version of virus signature database 6040 (20110414) __________ The message was checked by ESET NOD32 Antivirus. http://www.eset.com