Re: r3236 - /branches/multi_processor/relax -- March 19, 2007 - 16:56

Edward d'Auvergne wrote:

> Gary,
>
> It might be important to note that the code that you commented out was
> actually grid computing code rather than threading code.
> Unfortunately I called my grid computing code 'threading'!  Half of it
> could probably kept as is although relabelled to 'grid'.

Can you give an outline of how the grid code works I found it fairly 
convoluted when i tried to look at it....

>   For example
> the 'threading' user function class which sets up grid computing.  In
> the future I'll probably want to use my algorithm for handling very
> slow machines on a grid (which avoids relax having to wait for a slow
> machine to terminate) and the setting up of slave relax processes
> (using the 'relax --thread' invocation).  Although the grid computing
> code is currently broken, this is only because there is a problem with
> the handling of SSH tunnel breakages.  I also have in mind some
> optimisations for minimising data flow through the tunnel 


can you outline this?

> and Andrew
> Perry has had ideas about using heartbeats from the grid machine relax
> processes to probe for dead tunnels and processes.


again this would be interesting but I would have to look at it quite 
carefully as mpi has some threading limitations I believe...

> The model-free threads, which are true threads, could possibly also be
> kept and utilised by the MPI implementation.  They could be optimised
> though and any grid computing code separated from it.
>
> Cheers,
>
> Edward
Hi Ed
my thought on this was to turn  the treading code into an implimentation 
of the multi code so my plan was to rip it out and then put it back ;-) 
The architecture devised should be able to cope with ssh tunnels just as 
well as a thread or a mpi invocation I think... If not I would be happy 
to adapt to any ideas you have.

The basic idea is as you can see is to send pickled commands containing 
data to the slave which then returns a pickled result object,  and I 
think that this quite possible down an ssh tunnel. If not text commands 
with embedded text would also work.



regards
gary

> On 3/19/07, garyt@xxxxxxxxxxxxxxx <garyt@xxxxxxxxxxxxxxx> wrote:
>
> > Author: varioustoxins
> > Date: Mon Mar 19 11:58:24 2007
> > New Revision: 3236
> >
> > URL: http://svn.gna.org/viewcvs/relax?rev=3236&view=rev
> > Log:
> > marked more threading code for reletion
> >
> >
> > Modified:
> >     branches/multi_processor/relax
> >
> > Modified: branches/multi_processor/relax
> > URL: 
> > http://svn.gna.org/viewcvs/relax/branches/multi_processor/relax?rev=3236&r1=3235&r2=3236&view=diff 
> >
> > ============================================================================== 
> >
> > --- branches/multi_processor/relax (original)
> > +++ branches/multi_processor/relax Mon Mar 19 11:58:24 2007
> > @@ -171,23 +171,25 @@
> >              # Run the interpreter.
> >              self.interpreter = Interpreter(self)
> >              self.interpreter.run()
> > -
> > +            print 'exit'
> > +
> > +        # FIXME no more threading
> >          # Threading mode.
> > -        elif mode == 'thread':
> > -            # Print the PID prior to IO redirection, and then flush 
> > the buffer to send it to the parent.
> > -            print self.pid
> > -            sys.stdout.flush()
> > -
> > -            # Logging (silent IO redirection).
> > -            if self.log_file:
> > -                self.IO.log(log_file, print_flag=0)
> > -
> > -            # Tee (silent IO redirection).
> > -            elif self.tee_file:
> > -                self.IO.tee(tee_file, print_flag=0)
> > -
> > -            # Execute the script.
> > -            self.threading.execute()
> > +#        elif mode == 'thread':
> > +#            # Print the PID prior to IO redirection, and then flush 
> > the buffer to send it to the parent.
> > +#            print self.pid
> > +#            sys.stdout.flush()
> > +#
> > +#            # Logging (silent IO redirection).
> > +#            if self.log_file:
> > +#                self.IO.log(log_file, print_flag=0)
> > +#
> > +#            # Tee (silent IO redirection).
> > +#            elif self.tee_file:
> > +#                self.IO.tee(tee_file, print_flag=0)
> > +#
> > +#            # Execute the script.
> > +#            self.threading.execute()
> >
> >          # Execute the relax test suite
> >          elif mode == 'test suite':
> > @@ -553,8 +555,8 @@
> >      module_path = '.'.join(('multi',processor_name))
> >
> >
> > -    modules = import_module(module_path,verbose=True)
> > -    print modules
> > +    modules = import_module(module_path)
> > +    #print modules
> >      if hasattr(modules[-1],class_name):
> >          clazz =  getattr(modules[-1], class_name)
> >      else:
> >
> >
> > _______________________________________________
> > relax (http://nmr-relax.com)
> >
> > This is the relax-commits mailing list
> > relax-commits@xxxxxxx
> >
> > To unsubscribe from this list, get a password
> > reminder, or change your subscription options,
> > visit the list information page at
> > https://mail.gna.org/listinfo/relax-commits
>
> .


--
-------------------------------------------------------------------
Dr Gary Thompson
Astbury Centre for Structural Molecular Biology,
University of Leeds, Astbury Building,
Leeds, LS2 9JT, West-Yorkshire, UK             Tel. +44-113-3433024
email: garyt@xxxxxxxxxxxxxxx                   Fax  +44-113-2331407
-------------------------------------------------------------------