On 3/21/07, Gary S. Thompson <garyt@xxxxxxxxxxxxxxx> wrote:
Edward d'Auvergne wrote:
>On Mon, 2007-03-19 at 22:28 +0000, Gary S. Thompson wrote:
>
>
>> I also had a look at save state in state.py and this
>>seems quite heavy I presume that it dumps the complete program state to
>>a pickle and then rejuvenates it at the other side?
>>
>>
>
>This pickles solely the relax data storage object (which is now a
>singleton) as this is the program state. All permanent data and
>settings in relax must be stored in this object. This object in a
>pickled state is not what you'd need for MPI though - the object is too
>big for inter-node communication.
>
not necessarily it depends on how coarse grained your calculations are
and how long they run for. What I am doing with the multi frameworks is
effectivley just a wrapper round something similar to the therad code
plus at the moment an implimentation and that uses complete state dumps
(obviously if I can use more focused dumps things become faster)
This structure is BIG! Especially if Monte Carlo simulations are run.
I'd avoid sending it at all costs! Unless sending many, many
megabytes between the nodes is acceptable.
>Maybe there should be a model-free specific method called
>'minimise_mpi()' which is copied from the current 'minimise()' method.
>I think this is the area of relax which should be targeted.
>
yes but not mpi just a multiprocessor specific version
Ah. I get what you are trying to do!
>>One thing to note here is that I will at some stage try and rewrite
>>commands to keep the slave states in sync as we run so we don't have to
>>save the whole state. But that is for a later day, or never if you
>>consider that to not be the way to go...
>>
>>
>
>If you are working at the model-free 'minimise()' method level I think
>you will get the best efficiency out of a cluster! The granulatity
>would be perfect - not too fine that inter-node communication is the
>limiting factor and not too coarse that the used nodes of the cluster
>will be underutilised.
>
>At this low level the program state and the relax data storage object do
>not even come into play. Hence the slave program state will be
>untouched and remain at the initial state for as long as it exists.
>This will probably be the simplest solution to implement as well. This
>is what I eventually plan to do for the grid computing but you are
>welcome to beat me to it.
>
>ok i wll go for it
>
>
>>more questions
>>
>>where should I be attacking the division problem?
>>
>>
>
>Unless you see a point of using threads in the MPI code, don't attack
>the division problem. That one is my problem!
>
>
>
>
sorry I don't follow I need to divide upr the jobs otherwise I have
nothing to work with??
Sorry, I though you meant separating the grid computing code from the
threading code.
>> my main thought was to
>>effectivley add restrictions to a some commands. So consider grid search
>>I would add an extra parameter at the generic and functional levels
>>which would give a range of steps within the current parameters to
>>calculate.... e.g here are the ranges which give a grid of 10x10x10 ie
>>1000 steps. slave 1. you calculate 1-250 slave 2. 251-500 and so on.....
>>is this the correct way to go?
>>
>>
>
>Subdividing the grid search will be an interesting problem! Should it
>be at the 'generic_fns' level, the 'specific_fns' level, or implemented
>directly into the minimisation package? I think that the 'specific_fns'
>level, again within the 'minimise()' model-free method (copied, modified
>for MPI, and renamed to 'minimise_mpi()') would be the best place to
>target.
>
>An algorithm to subdivide the grid would be useful. Then an algorithm
>to collect the results and determine which subspace of the grid has the
>point with the lowest chi2 value should be used. I.e. this will be an
>MPI-oriented grid search over a number of standard grid searches.
>
>However your best MPI gains are likely to be achieved by sending each
>grid search to a different node. This higher level would be shared with
>the standard model-free optimisation code and hence you don't need to
>worry about writing separate MPI code for the grid search and for the
>minimisation. Slight improvements may be achieved by breaking up the
>grid search, but I would personally tackle this later on.
>
>
>
again I need to think about this. However if this uses divisons by model
again it will perform poorly as the different models will take different
amounts of time to calculate so many processors will sit idle...
again if I am not undertsanding properly please accept my apologies
relax is very heavily layered and alot of names are repeated multiple
times it can be quite had to follow whatis going on in the code base ;-)
Idle time will be inevitable. Especially when it comes to the 'all'
model-free minimisation instance (the optimisation of all model-free
model parameters for all residues together with all diffusion
parameters). That cannot be avoided.
Bye,
Edward
Re: relax, MPI, and Grid computing.