how to parallelise model_free minimise -- March 23, 2007 - 15:55

Dear Ed and all
   I have started looking at how to parallelise the calls to  
model_free.minimise as discussed in our previous message but am having 
some problems with the function....

The first is that it is huge and seems to have a large amount of special 
casing and checking built in.

The second one is to work out how many different modes it can be called 
in. From waht I can see I need to look for param_set and only 
paralellise if self.param_set is one of mf or local_mf (diff and alle 
being either to trivial or too hard to optimise respectivley ;-)

Then there comes the parameters passed

first there is Mf  and generic_minimse which seem to take a huge number 
orf parameters:

          self.mf = Mf(init_params=self.param_vector, 
param_set=self.param_set, diff_type=diff_type,
                        diff_params=diff_params, 
scaling_matrix=self.scaling_matrix, num_res=num_res,
                        equations=equations, param_types=param_types, 
param_values=param_values,
                        relax_data=relax_data, errors=relax_error, 
bond_length=r, csa=csa, num_frq=num_frq,
                        frq=frq, num_ri=num_ri, 
remap_table=remap_table, noe_r1_table=noe_r1_table,
                        ri_labels=ri_labels, gx=self.relax.data.gx, 
gh=self.relax.data.gh,
                        g_ratio=self.relax.data.g_ratio, 
h_bar=self.relax.data.h_bar,
                        mu0=self.relax.data.mu0, num_params=num_params, 
vectors=xh_unit_vectors)

and generic

               results = generic_minimise(func=self.mf.func, 
dfunc=self.mf.dfunc, d2func=self.mf.d2func, args=(),
                                          x0=self.param_vector, 
min_algor=min_algor, min_options=min_options,
                                          func_tol=func_tol, 
grad_tol=grad_tol, maxiter=max_iterations,
                                          full_output=1, 
print_flag=print_flag, A=A, b=b)

The questions here are

1. that are all these parameters either I misreading thing or else not 
undertanding because i couldn't find definitions for every thing
2. what is going to change between runs or even over runs of the relax 
program.

clearly some things don't change at all and it could even be asked why 
for example h_bar is a parameter to Mf  (ther maybe something deep I am 
missing here?)
some other things may only change at  specific points in the program For 
example the vectors of the molecules should only change when the vecors 
function of structure or pdb is called
other things are per residue but which of them???
and other things are differing by type of model and minimisation....

as a note typical input parameters for a local tm calculation are
-#-initialise Mf residue - 3 LEU
-#--------------
-#-
-#-init_params [ 1000.]
-#-param_se local_tm
-#-diff_type sphere
-#-diff_params None
-#-scaling_matrix [ [  1.00000000e-12]]
-#-num_res 1
-#-equations ['mf_orig']
-#-param_types [['local_tm']]
-#-param_values None
-#-relax_data [array([  0.8293,  12.85  ,   0.9528,  12.57  ,   0.0983])]
-#-errors [array([ 0.023 ,  0.13  ,  0.0253,  0.171 ,  0.0278])]
-#-bond_length [1.0200000000000001e-10]
-#-csa= [-0.00017199999999999998]
-#-num_frq [2]
-#-frq [[750800000.0, 599.71900000000005]]
-#-num_ri [5]
-#-remap_table [[0, 0, 1, 1, 1]]
-#-noe_r1_table [[None, None, None, None, 2]]
-#-ri_labels [['R1', 'R2', 'R1', 'R2', 'NOE']]
-#-gx -27126000.0
-#-gh 267522212.0
-#-g_ratio -9.862206444
-#-h_bar 1.05457159642e-34
-#-mu0 1.25663706144e-06
-#-num_params [1]
-#-vectors [None]
-#-
-#-generic minimisation residue - 3 LEU
-#-----------------------------
-#-
-#-constraints 1
-#-func <bound method Mf.func_local_tm of <maths_fns.mf.Mf instance at 
0x4079fc0c>> -#-dfunc <bound method Mf.dfunc_local_tm of 
<maths_fns.mf.Mf instance a
t 0x4079fc0c>>
-#-d2func <bound method Mf.d2func_local_tm of <maths_fns.mf.Mf instance 
at 0x4079fc0c>>
-#-args ()
-#-print x0 [ 1000.]
-#-print min_algor Method of Multipliers
-#-min_options ('newton',)
-#-func_tol 1e-25
-#-grad_tol None
-#-maxiter 10000000
-#-full_output 1
-#-print_flag 1
-#-constrained
-#-A [[ 1.]
-#-b [      0. -200000.]



As an aside when the redesign of the spin_loops and minimise /model 
loops cuts in it would be a good idea 9from the paralle point of view) 
to have the spin loop  running faster than the minimse/model loop


so you could split by residue for prallelising but send of all the 
required model minimisations for each model at the same time which would 
give implicicit load balancing and coarser gains on homogeouds parallel 
computers:

i.e for six residues and 3 nodes

node 1 calculates
res 1 [m1 m2 m3...]
res 2 [[m1 m2 m3..]

node 2 calculates
res 3 [m1 m2 m3...]
res 4 [[m1 m2 m3..]

node 3 calculates
res 5 [m1 m2 m3...]
res 6 [[m1 m2 m3..]

this obviously places some limitations of the design of the minimisation 
function as it might needs to have a set and tear down region that cope with 
this batched data....


regard

gary
--
-------------------------------------------------------------------
Dr Gary Thompson
Astbury Centre for Structural Molecular Biology,
University of Leeds, Astbury Building,
Leeds, LS2 9JT, West-Yorkshire, UK             Tel. +44-113-3433024
email: garyt@xxxxxxxxxxxxxxx                   Fax  +44-113-2331407
-------------------------------------------------------------------