Re: Monte-Carlo simulation for a clustering of 78 residues in a dispersion analysis is extremely slow -- October 08, 2013 - 11:54

Hi,

The clustering analysis can take quite a long time, especially with
Monte Carlo simulations.  If you cluster 78 spins with data at 2 field
strengths and use the CR72 model, the problem changes from 78 small 5D
problems (the dimensions are:  R02_field1 + R02_field2 + pA + dw +
kex) to one large 313D problem (the dimensions are:  78 * (R02_field1
+ R02_field2 + pA + dw) + kex).  For the field of numerical
optimisation, that is a huge difference.  Optimising a 313 dimensional
problem is difficult and long.  Then add Monte Carlo simulations on
top of that and 58 hours actually seems quite reasonable.

But now that the Monte Carlo simulations are parallelised, if you have
a cluster with 50 nodes the calculation with 50 Monte Carlo
simulations should drop from 58 hours to between 1 to 2 hours.  Give
it a go on your cluster and see how much things have speed up when
using OpenMPI!

Regards,

Edward




On 8 October 2013 11:39, Troels Emtekær Linnet <tlinnet@xxxxxxxxx> wrote:
> Hi Edward.
>
> Last time I did a clustering analysis for 78 spins, with no parallisation,
> with 50 monte carlo simulations, it took 58 Hours.
>
> I compare that to a run with no clustering, which took 8 Hours.
>
> Is the clustering calculation so much more expensive?
>
> Best
> Troels
>
>
>
> Troels Emtekær Linnet
>
>
> 2013/10/7 Edward d'Auvergne <edward@xxxxxxxxxxxxx>
> > Hi Troels,
> >
> > This is exactly as I would expect as the parallisation is currently at
> > the level of the spin cluster.  Therefore if you have only one
> > cluster, using MPI will not result in any speed ups.  This was
> > mentioned in the commit message:
> >
> > http://thread.gmane.org/gmane.science.nmr.relax.scm/18737
> >
> > I am currently considering how to implement an additional
> > parallisiation at the Monte Carlo simulation level.  My idea is to
> > have the multi-processor box singleton used in the minimise user
> > function backend, specifically in the pipe_control.minimise.minimise()
> > function.  Instead of running processor.run_queue() at the end of the
> > minimise() method of the specific_analyses.relax_disp.api.Relax_disp
> > class, it is run at the end of the pipe_control.minimise.minimise()
> > function.  This might involve less that 10 lines of code changed to
> > fully implement Monte Carlo simulation parallisation.  But the problem
> > is that it needs testing to see how the non-parallised analyses handle
> > this.
> >
> > Regards,
> >
> > Edward
> >
> > On 15 September 2013 19:14, Troels Emtekær Linnet <tlinnet@xxxxxxxxx>
> > wrote:
> > > Hi Edward.
> > >
> > > I try to perform some speed-analysis.
> > > As mentioned here:
> > >
> > > http://wiki.nmr-relax.com/OpenMPI#Results
> > >
> > > I am very happy to see, that the analysis time go down from 8 hours, to
> > > 2
> > > hours, when you change to a computer with 24 CPU's.
> > >
> > > Now I am trying to make a cluster analysis.
> > > But making the monte-carlo analysis for a clustering of 78 residues,
> > > takes
> > > extremely long time.
> > > This is for the test, where I only use 1 CPU.
> > >
> > > After 1 Hour, it has not even passed simulation 1.
> > > For the faster computer, is has passed simulation 3.
> > > But the time for 50 MC simulation, now looks like to take extremely long
> > > time.
> > >
> > > Do you know, if this is expected?
> > >
> > >
> > >
> > > Troels Emtekær Linnet
> > >
> > > _______________________________________________
> > > relax (http://www.nmr-relax.com)
> > >
> > > This is the relax-devel mailing list
> > > relax-devel@xxxxxxx
> > >
> > > To unsubscribe from this list, get a password
> > > reminder, or change your subscription options,
> > > visit the list information page at
> > > https://mail.gna.org/listinfo/relax-devel