Re: r3261 - in /branches/multi_processor: generic_fns/ multi/ prompt/ specific_fns/ -- April 12, 2007

Edward d'Auvergne wrote:

Gary,

I've had a look at the changes and I'm a little alarmed by the comment
'# these may need to be in c they cause an pprox 10% slowdown'.  What
is the 'multi/PrependStringIO.py' module actually used for?  There
isn't much in this module explaining its function.  If the slowdown is
during the minimisation, do we need to prepend 'S> ' and 'E> '?

This merely currently a reminder to me to do some checking! I am noteven sure of the data as most of the overhead comes from the outputcomes from writing to stdout and some of the numbers I have in my headmay come from runs thatouput to files. The commands are quite simplethey prepend output from the masters and slaves with some text thatindicates which stream and which processor the output came from e.g. ona multiprocessor you get


M S> script
M S>
M S>
M S>
M S>                                      relax repository checkout
M S>

M S> Protein dynamics by NMR relaxation dataanalysis

M S>
M S>                              Copyright (C) 2001-2006 Edward d'Auvergne
M S>

M S> This is free software which you are welcome to modify andredistribute under the conditions of theM S> GNU General Public License (GPL). This program, including allmodules, is licensed under the GPLM S> and comes with absolutely no warranty. For details type 'GPL'.Assistance in using this program

M S> can be accessed by typing 'help'.
M S>
M S> script = 'test_small.py'

M S>----------------------------------------------------------------------------------------------------

M S> # Script for model-free analysis.
M S>
M S> dataPaths = ['test_data/noe750.dat',
M S>              'test_data/T1_750_hgts.bs',
M S>              'test_data/T2_750_hgts.bs']
M S>
M S> dataTypes = [('NOE', '750', 750.8e6),
M S>              ('R1', '750', 750.8e6),
M S>              ('R2', '750', 750.8e6)]
M S>
M S> # Set the run names (also the names of preset model-free models).
M S> #runs = ['tm1', 'tm2', 'tm3', 'tm4', 'tm5', 'tm6', 'tm7', 'tm8', 'tm9']
M S> runs = ['m1', 'm2', 'm3', 'm4', 'm5', 'm6', 'm7', 'm8', 'm9']
M S>
M S> # Nuclei type
M S> nuclei('N')
M S>
M S> test=True
M S> # Loop over the runs.
M S> for name in runs:
M S>     # Create the run.
M S>     run.create(name, 'mf')
M S>
M S>     # Load the sequence.
M S>     #sequence.read(name, 'noe.500.out')
M S>
M S>     # Load a PDB file.
M S>     structure.read_pdb(name, 'test_data/test.pdb')
M S>     #pdb.read(name, 'test_data/test.pdb')
M S>     structure.vectors(name, proton='HN')
M S>
M S>     # Load the relaxation data.
M S>     for dataSet in xrange(len(dataPaths)):

M S> relax_data.read(name, dataTypes[dataSet][0],dataTypes[dataSet][1], dataTypes[dataSet][2], dataPaths[dataSet])

M S>
M S>     # Setup other values.
M S>     diffusion_tensor.init(name, 1e-8, fixed=1)

M S> #diffusion_tensor.init(name, (1e-8, 1.0, 60, 290),param_types=0, spheroid_type='oblate', fixed=0)

M S>     value.set(name, 1.02 * 1e-10, 'bond_length')
M S>     value.set(name, -160 * 1e-6, 'csa')
M S>     #value.set(name, 0.970, 's2')
M S>     #value.set(name, 1.0, 's2f')
M S>     #value.set(name, 2048e-12, 'te')
M S>     #value.set(name, 2048e-12, 'tf')
M S>     #value.set(name, 2048e-12, 'ts')
M S>     #value.set(name, 0.149/(2*pi*600e6)**2, 'rex')
M S>
M S>     # Select the model-free model.
M S>     model_free.select_model(run=name, model=name)
M S>     #fix(name, 'all_res')
M S>
M S>     # Minimise.
M S>     print '***- grid searching ' + name
M S>     grid_search(name, inc=11)
M S>     if not test:
M S>         print '***- minimise' + name
M S>         minimise('newton', run=name)
M S>
M S> if test:
M S>     for name in runs:
M S>         print '***- minimise' + name
M S>         minimise('newton', run=name)
M S>
M S> for name in runs:
M S>     # Write the results.
M S>     results.write(run=name, file='results', force=1)
M S>
M S> # Save the program state.
M S> state.save('save', force=1)

M S>----------------------------------------------------------------------------------------------------

M S>
M S> relax> nuclei(heteronuc='N')
M S>
M S> relax> run.create(run='m1', run_type='mf')
M S>

M S> relax> structure.read_pdb(run='m1', file='test_data/test.pdb',dir=None, model=None, load_seq=1)

M S> Loading all structures from the PDB file.
M S> Structure('test_data/test.pdb'):
M S>   Peptide chain BP of length 28
M S>
M S>
M S> Loading the sequence from the PDB file.
M S>
M S>

M S> relax> structure.vectors(run='m1', heteronuc='N', proton='HN',res_num=None, res_name=None)

M S>
M S> Calculating the unit XH vectors from the structure.
M S>

M S> relax> relax_data.read(run='m1', ri_label='NOE', frq_label='750',frq=750800000.0, file='test_data/noe750.dat', dir=None, num_col=0,name_col=1, data_

col=2, error_col=3, sep=None)
M S> Opening the file 'test_data/noe750.dat' for reading.
M S>

M S> relax> relax_data.read(run='m1', ri_label='R1', frq_label='750',frq=750800000.0, file='test_data/T1_750_hgts.bs', dir=None, num_col=0,name_col=1, da

ta_col=2, error_col=3, sep=None)
M S> Opening the file 'test_data/T1_750_hgts.bs' for reading.
M S>

M S> relax> relax_data.read(run='m1', ri_label='R2', frq_label='750',frq=750800000.0, file='test_data/T2_750_hgts.bs', dir=None, num_col=0,name_col=1, da

ta_col=2, error_col=3, sep=None)
M S> Opening the file 'test_data/T2_750_hgts.bs' for reading.
M S>

M S> relax> diffusion_tensor.init(run='m1', params=1e-08,time_scale=1.0, d_scale=1.0, angle_units='deg', param_types=0,spheroid_type=None, fixed=1)

M S>

M S> relax> value.set(run='m1', value=1.0200000000000001e-10,param='bond_length', res_num=None, res_name=None)

M S>

M S> relax> value.set(run='m1', value=-0.00015999999999999999,param='csa', res_num=None, res_name=None)

M S>
M S> relax> model_free.select_model(run='m1', model='m1', res_num=None)
M S> ***- grid searching m1
M S>

M S> relax> grid_search(run='m1', lower=None, upper=None, inc=11,constraints=1, print_flag=1)

M S> Only the model-free parameters for single residues will be used.
M S> 1 S>
1 S>
1 S> Fitting to residue: 29 LYS
1 S> ~~~~~~~~~~~~~~~~~~~~~~~~~~

1 S> Unconstrained grid search size: 11 (constraints may decrease thissize).

1 S>
1 S>
1 S> Grid search
1 S> ~~~~~~~~~~~
1 S>
1 S> Searching the grid.

1 S> k: 0 xk: array([0.]) fk:8.6505190311418673e+2001 S> k: 1 xk: array([0.1]) fk:2911.93159252202351 S> k: 2 xk: array([0.2]) fk:2169.38430919890611 S> k: 3 xk: array([0.3]) fk:1609.73862258595051 S> k: 4 xk: array([0.4]) fk:1232.99453268315641 S> k: 5 xk: array([0.5]) fk:1039.15203949052531 S> k: 6 xk: array([0.6]) fk:1028.2111430080565

1 S>
1 S> Parameter values: array([ 0.6])
1 S> Function value:   1028.2111430080565
1 S> Iterations:       11
1 S> Function calls:   11
1 S> Gradient calls:   0
1 S> Hessian calls:    0
1 S> Warning:          None
1 S>
1 S> 2 S>
2 S>
2 S> Fitting to residue: 28 GLY
2 S> ~~~~~~~~~~~~~~~~~~~~~~~~~~

2 S> Unconstrained grid search size: 11 (constraints may decrease thissize).

2 S>
2 S>
2 S> Grid search
2 S> ~~~~~~~~~~~
2 S>
2 S> Searching the grid.

2 S> k: 0 xk: array([0.]) fk:2.2887432739557032e+2002 S> k: 1 xk: array([0.1]) fk:5117.06884672076832 S> k: 2 xk: array([0.2]) fk:4166.23028174156122 S> k: 3 xk: array([0.3]) fk:3333.41183918349862 S> k: 4 xk: array([0.4]) fk:2618.6135190465772 S> k: 5 xk: array([0.5]) fk:2021.83532133080122 S> k: 6 xk: array([0.6]) fk:1543.07724603616862 S> k: 7 xk: array([0.7]) fk:1182.33929316267972 S> k: 8 xk: array([0.8]) fk:939.621462710333732 S> k: 9 xk: array([0.9]) fk:814.923754679132342 S> k: 10 xk: array([1.]) fk:808.2461690690742

2 S>
2 S> Parameter values: array([ 1.])
2 S> Function value:   808.2461690690742
2 S> Iterations:       11
2 S> Function calls:   11
2 S> Gradient calls:   0
2 S> Hessian calls:    0
2 S> Warning:          None
2 S>
2 S> 3 S>
3 S>
3 S> Fitting to residue: 27 ALA
3 S> ~~~~~~~~~~~~~~~~~~~~~~~~~~

3 S> Unconstrained grid search size: 11 (constraints may decrease thissize).

3 S>
3 S>
3 S> Grid search
3 S> ~~~~~~~~~~~
3 S>
3 S> Searching the grid.

3 S> k: 0 xk: array([0.]) fk:3.9211841976276823e+2003 S> k: 1 xk: array([0.1]) fk:3619.63763594916963 S> k: 2 xk: array([0.2]) fk:2774.61359091210583 S> k: 3 xk: array([0.3]) fk:2061.22411530736243 S> k: 4 xk: array([0.4]) fk:1479.46920913493663 S> k: 5 xk: array([0.5]) fk:1029.34887239483123 S> k: 6 xk: array([0.6]) fk:710.863105087045143 S> k: 7 xk: array([0.7]) fk:524.01190721157823 S> k: 8 xk: array([0.8]) fk:468.79527876843048

3 S>
3 S> Parameter values: array([ 0.8])
3 S> Function value:   468.79527876843048
3 S> Iterations:       11
3 S> Function calls:   11
3 S> Gradient calls:   0
3 S> Hessian calls:    0
3 S> Warning:          None
3 S>
3 S> 4 S>
4 S>
4 S> Fitting to residue: 26 LYS
4 S> ~~~~~~~~~~~~~~~~~~~~~~~~~~

4 S> Unconstrained grid search size: 11 (constraints may decrease thissize).

4 S>
4 S>
4 S> Grid search
4 S> ~~~~~~~~~~~
4 S>
4 S> Searching the grid.

4 S> k: 0 xk: array([0.]) fk:2.6874496103198057e+2004 S> k: 1 xk: array([0.1]) fk:2589.26208962865444 S> k: 2 xk: array([0.2]) fk:2035.07246735557154 S> k: 3 xk: array([0.3]) fk:1564.72211406039954 S> k: 4 xk: array([0.4]) fk:1178.21102974313684 S> k: 5 xk: array([0.5]) fk:875.539214403784974 S> k: 6 xk: array([0.6]) fk:656.706668042343154 S> k: 7 xk: array([0.7]) fk:521.71339065881174 S> k: 8 xk: array([0.8]) fk:470.55938225319028

4 S>
4 S> Parameter values: array([ 0.8])
4 S> Function value:   470.55938225319028
4 S> Iterations:       11
4 S> Function calls:   11
4 S> Gradient calls:   0
4 S> Hessian calls:    0
4 S> Warning:          None
4 S>
4 S> 5 S>
5 S>
5 S> Fitting to residue: 25 ASP
5 S> ~~~~~~~~~~~~~~~~~~~~~~~~~~

5 S> Unconstrained grid search size: 11 (constraints may decrease thissize).

5 S>

....

obviously I will add some flags to squash this if the user wants to at alater date, but it is currently very useful to me to see that is going on

In 'specific_fns/model_free.py' most of the changes where objects are
no longer placed into 'self' will be eliminated in the 1.3 line.  Most
class methods will probably be distributed into different modules
within the directory 'specific_fns/model_free/' and converted to
simple functions.  This will complicate the porting of the branch into
a patch for the 1.3 line.

I guessed this would be the case. I expected that we will have to make anew branch from the 1.3.x line when it stabalises, merge what can bemegered from multi and manually cross port the rest..

On last thing is that I think that the changes to
'prompt/minimisation.py' belong elsewhere.  The call to
'self.relax.processor.run_queue()' should be in the generic code
(specifically 'generic_fns/minimisation.py') to allow any type of UI
to have access to the MPI code.  The code in the 'prompt/' directory
is solely used to set up the behaviour of the user functions
accessible through the prompt and script UIs, is used to test the
validity of the arguments and then execute the generic or specific
code.  Then if a graphical user interface (GUI) or some new UI is
added to relax, they will all automatically be MPI compatible.

That seems fine. However, there is a problem here the run_queue methodneeds to be called after all the the minimisations that are notinterrelated have been queued. How will this work with th command pipes?

e.g. when we have commands that take runs as opposed to run willgeneric.minimise.minimise get called with a runs parameter instead of run?


regards
gary

Cheers,

Edward


On 4/8/07, garyt@xxxxxxxxxxxxxxx <garyt@xxxxxxxxxxxxxxx> wrote:

Author: varioustoxins
Date: Sat Apr  7 22:11:21 2007
New Revision: 3261

URL: http://svn.gna.org/viewcvs/relax?rev=3261&view=rev
Log:
Major update - monte carlo simulations supported (trivially)
also implimented: chunky queues, exception throwing on slaves, and rank
annotated output. note uniprocessor support currently broken.

Added:
   branches/multi_processor/multi/PrependStringIO.py
Modified:
   branches/multi_processor/generic_fns/minimise.py
   branches/multi_processor/multi/commands.py
   branches/multi_processor/multi/mpi4py_processor.py
   branches/multi_processor/multi/processor.py
   branches/multi_processor/prompt/minimisation.py
   branches/multi_processor/specific_fns/model_free.py

[This mail would be too long, it was shortened to contain the URLs only.]

Modified: branches/multi_processor/generic_fns/minimise.py
URL: 
http://svn.gna.org/viewcvs/relax/branches/multi_processor/generic_fns/minimise.py?rev=3261&r1=3260&r2=3261&view=diff

Added: branches/multi_processor/multi/PrependStringIO.py
URL: 
http://svn.gna.org/viewcvs/relax/branches/multi_processor/multi/PrependStringIO.py?rev=3261&view=auto

Modified: branches/multi_processor/multi/commands.py
URL: 
http://svn.gna.org/viewcvs/relax/branches/multi_processor/multi/commands.py?rev=3261&r1=3260&r2=3261&view=diff

Modified: branches/multi_processor/multi/mpi4py_processor.py
URL: 
http://svn.gna.org/viewcvs/relax/branches/multi_processor/multi/mpi4py_processor.py?rev=3261&r1=3260&r2=3261&view=diff

Modified: branches/multi_processor/multi/processor.py
URL: 
http://svn.gna.org/viewcvs/relax/branches/multi_processor/multi/processor.py?rev=3261&r1=3260&r2=3261&view=diff

Modified: branches/multi_processor/prompt/minimisation.py
URL: 
http://svn.gna.org/viewcvs/relax/branches/multi_processor/prompt/minimisation.py?rev=3261&r1=3260&r2=3261&view=diff

Modified: branches/multi_processor/specific_fns/model_free.py
URL: 
http://svn.gna.org/viewcvs/relax/branches/multi_processor/specific_fns/model_free.py?rev=3261&r1=3260&r2=3261&view=diff


_______________________________________________
relax (http://nmr-relax.com)

This is the relax-commits mailing list
relax-commits@xxxxxxx

To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-commits


_______________________________________________
relax (http://nmr-relax.com)

This is the relax-devel mailing list
relax-devel@xxxxxxx

To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-devel

.



--
-------------------------------------------------------------------
Dr Gary Thompson
Astbury Centre for Structural Molecular Biology,
University of Leeds, Astbury Building,
Leeds, LS2 9JT, West-Yorkshire, UK             Tel. +44-113-3433024
email: garyt@xxxxxxxxxxxxxxx                   Fax  +44-113-2331407
-------------------------------------------------------------------

Re: r3261 - in /branches/multi_processor: generic_fns/ multi/ prompt/ specific_fns/

Header

Content

Related Messages