Re: looping the optimization in full_analysis.pyRe: looping the optimization in full_analysis.pyDoug On Oct 24, 2007, at 9:24 AM, Sébastien Morin wrote:
Hi againYou're right. This could be quite useful especially for multi- processorsystems...Also, as you say, there could be other levels of queuing (that people morefamiliar with this topic than me could propose)... Ciao Séb Selon Douglas Kojetin <douglas.kojetin@xxxxxxxxx>, 24.10.2007:Hi, I think it would be worth adding the code, as a queueing system within the full_analysis script could spawn multiple tensor optimizations at the same time, depending on the number of processors available on the computer (or cluster). For example, it would first run local_tm. Then, on a 2 CPU system, spawn the sphere and prolate runs. When one of those finish, start the next run in the queue (oblate). Then when another run finishes, run the next (ellipsoid). That is one level of queueing, but I'm sure this could also be used to queue and run individual model optimizations within a tensor optimization run. For example, m1, m2 ... through m8 for the sphere tensor optimization. Doug On Oct 24, 2007, at 8:48 AM, Sebastien Morin wrote:Hi again This is exactly what I was thinking. However, there's a simple way to queue things like that out of relax (at least in Unix systems)... But, then, you need multiple scripts. Let's say you want to run every tensor in full_analysis.py one after the other... You could do something like : ( relax -t log_local_tm ./full_analysis_local_tm.py ; relax -t log_sphere ./full_analysis_sphere.py ; relax -t log_prolate ./ full_analysis_prolate.py ; relax -t log_oblate ./ full_analysis_oblate.py ; relax -t log_ellipsoid ./ full_analysis_ellipsoid.py ; relax -t log_final ./ full_analysis_final.py ) & This would produce different logs for the different diffusion tensor optimization and also be in the background... A run would start when the previous run would finish or crash. This ain't perfect but is it worth spending time to modify the code when a command line like this one can do the same thing ? Let me know what you think. Cheers Séb Douglas Kojetin wrote:Perhaps it might be useful to build some sort of automatic queueing routine into relax, or the full_analysis script directly? It could first run the local_tm run, then queue up and run the other tensor runs. Doug On Oct 24, 2007, at 8:25 AM, Sebastien Morin wrote:Hi Doug This extreme looping should be added as an option in the full_analysis.py script. I'm not sure I would routinely use it as the computing time must be very long, but it could be useful some times... When the threading over multiple CPUs will be implemented, this option would also certainly be useful... I would suggest you submit a patch to relax-devel@xxxxxxx to add this feature as an option the user can activate or not depending on its needs... Cheers ! Séb :) Douglas Kojetin wrote:I guess I could also mentione that I've 'hacked' the full_analysis.py script to automatically loop through local_tm, sphere, prolate, oblate and ellipsoid runs as well. This, however, produces one extremely large output file, so the modifications mentioned below would be useful. Doug On Oct 22, 2007, at 4:22 PM, Sebastien Morin wrote:Hi Doug ! Great ! I didn't know about that feature (and some others that "relax - help" outputs)... Séb :) Douglas Kojetin wrote:Hi Séb, I've been using the -t flag creating one large log, so I think this:For analysis where some kind of looping is done (like here), the logs could be separated into different steps. For the full_analysis.py script, logging could be done independantly for each round and added to the other directories created... aic/ m0/ m1/ m2/ m3/ m4/ m5/ m6/ m7/ m8/ m9/ opt/ logand this:Maybe, also, two versions of the log could created, one detailed and one only logging main topics so one can easily follow the processes. This could be interesting for people not familiar with the details of relax and of the approaches used for calculation/optimization...would be extremely useful. Doug On Oct 22, 2007, at 4:15 PM, Sebastien Morin wrote:Hi all ! First of all, I think that the new feature for automatic looping is quite interesting..! However, I think that something could be done to make the analysis easier. When one starts a relax run, text is sent to the standard output. I usually redirect this text to a log file so I can analyze if any trouble arises... I think that logging should be automatic. This would not mean that standard output would be suppress, but rather that standard output would be, also, saved in a text file along with the results. For analysis where some kind of looping is done (like here), the logs could be separated into different steps. For the full_analysis.py script, logging could be done independantly for each round and added to the other directories created... aic/ m0/ m1/ m2/ m3/ m4/ m5/ m6/ m7/ m8/ m9/ opt/ log You may say that we can manually do that, but when using the automatic looping options, one would get a huge log... Maybe, also, two versions of the log could created, one detailed and one only logging main topics so one can easily follow the processes. This could be interesting for people not familiar with the details of relax and of the approaches used for calculation/optimization... Let me know what you think of this and also if I'm being clear or not... Cheers Séb :)-- ______________________________________ _______________________________________________ | | || Sebastien Morin || ||| Etudiant au PhD en biochimie ||| |||| Laboratoire de resonance magnetique nucleaire |||| ||||| Dr Stephane Gagne ||||| |||| CREFSIP (Universite Laval, Quebec, CANADA) |||| ||| 1-418-656-2131 #4530 ||| || || |_______________________________________________| ______________________________________-- ______________________________________ _______________________________________________ | | || Sebastien Morin || ||| Etudiant au PhD en biochimie ||| |||| Laboratoire de resonance magnetique nucleaire |||| ||||| Dr Stephane Gagne ||||| |||| CREFSIP (Universite Laval, Quebec, CANADA) |||| ||| 1-418-656-2131 #4530 ||| || || |_______________________________________________| ______________________________________-- ______________________________________ _______________________________________________ | | || Sebastien Morin || ||| Etudiant au PhD en biochimie ||| |||| Laboratoire de resonance magnetique nucleaire |||| ||||| Dr Stephane Gagne ||||| |||| CREFSIP (Universite Laval, Quebec, CANADA) |||| ||| 1-418-656-2131 #4530 ||| || || |_______________________________________________| ______________________________________------------------------ Sébastien Morin Étudiant M.Sc. Biochimie Laboratoire S. Gagné 3252 Pav. Marchand (Université Laval) Tél : (418) 656-2131 #4530 Fax : (418) 656-7176 e-mail : sebastien.morin.1@xxxxxxxxx