Re: r3208 - /branches/multi_processor/relax -- March 16, 2007 - 16:11

garyt@xxxxxxxxxxxxxxx wrote:

Hi
  This is  an initial try at creating a multiprocessor intialization 
setup for relax

It uses the idea of a specific class that determines how to load relax 
nd setup communication depending on which communication methodology you 
use. The loading of the class is done dynamically. Currently there is 
only a single launcher multi.uni_processor and this will load and launch 
a single instance of relax on a single processor.  The reason for this 
prelaunch methodology is that the semantics for different communication 
stratergies are differerent.

for example

1. for mpi once your programme instance has started and mpi is 
initialised you just need to run a single interpreter which will 
communicate with all other interpreteres via mpi calls
2. for a threaded system when we start we have to create a set of 
threads and all the interpreters to go in them and set their 
communicatiosn channels up...

there are still a few bits of refactoring and deciding which modes work 
with which processing setup to be defined. I think we need two modes:  
the current relax run type mode and a multiprocessor type mode, but 
anyway it works ;-)


regards
gary

> Author: varioustoxins
> Date: Fri Mar 16 12:47:30 2007
> New Revision: 3208
>
> URL: http://svn.gna.org/viewcvs/relax?rev=3208&view=rev
> Log:
> initial work to add multiprocessor startup and refactoring of main Relax
> class
>
> Modified:
>    branches/multi_processor/relax
>
> Modified: branches/multi_processor/relax
> URL: 
> http://svn.gna.org/viewcvs/relax/branches/multi_processor/relax?rev=3208&r1=3207&r2=3208&view=diff 
>
> ============================================================================== 
>
> --- branches/multi_processor/relax (original)
> +++ branches/multi_processor/relax Fri Mar 16 12:47:30 2007
> @@ -103,7 +103,7 @@
>
>         # Debugging flag (default is off).
>         __builtin__.Debug = 0
> -        +
>         # Pedantic flag (default is off).
>         __builtin__.Pedantic = 0
>
> @@ -137,25 +137,27 @@
>         # Setup the object containing the thread setup functions.
>         self.threading = Threading(self)
>
> +        # FIXME removed
>         # Process the command line arguments and determine the relax 
> mode.
> -        mode, log_file, tee_file = self.arguments()
> -        +        #mode, log_file, tee_file = self.arguments()
> +
>         # Initialise the warnings sytem.
>         RelaxWarnings()
>         __builtin__.warn = warn
> -                +
> +    def run(self, mode):
>         # Show the version number and exit.
>         if mode == 'version':
>             print 'relax ' + self.version
>             sys.exit()
>
>         # Logging.
> -        if log_file and mode != 'thread':
> -            self.IO.log(log_file)
> +        if self.log_file and mode != 'thread':
> +            self.IO.log(self.log_file)
>
>         # Tee.
> -        elif tee_file and mode != 'thread':
> -            self.IO.tee(tee_file)
> +        elif self.tee_file and mode != 'thread':
> +            self.IO.tee(self.tee_file)
>
>         # Run the interpreter for the prompt or script modes.
>         if mode == 'prompt' or mode == 'script':
> @@ -173,11 +175,11 @@
>             sys.stdout.flush()
>
>             # Logging (silent IO redirection).
> -            if log_file:
> +            if self.log_file:
>                 self.IO.log(log_file, print_flag=0)
>
>             # Tee (silent IO redirection).
> -            elif tee_file:
> +            elif self.tee_file:
>                 self.IO.tee(tee_file, print_flag=0)
>
>             # Execute the script.
> @@ -213,7 +215,7 @@
>             self.licence()
>
>
> -    def arguments(self):
> +    def arguments(self,args):
>         """Function for processing the command line arguments."""
>
>         # Parser object.
> @@ -228,16 +230,20 @@
>         parser.add_option('--test', action='store_true', dest='test', 
> default=0, help='run relax in test mode')
>         parser.add_option('--test-suite', action='store_true', 
> dest='test_suite', default=0, help='execute the relax test suite')
>         parser.add_option('--unit-test', action='store_true', 
> dest='unit_test', default=0, help='execute the relax unit tests (part 
> of the test suite)')
> -        parser.add_option('--thread', action='store_true', 
> dest='thread', default=0, help='run relax in threading mode (this mode 
> should not be invoked by a user)')
> +# FIXME remove threading
> +# threading dprecated
> +#        parser.add_option('--thread', action='store_true', 
> dest='thread', default=0, help='run relax in threading mode (this mode 
> should not be invoked by a user)')
>         parser.add_option('-v', '--version', action='store_true', 
> dest='version', default=0, help='show the version number and exit')
> +        parser.add_option('-m', '--multi', action='store', 
> type='string', dest='multiprocessor', default='uni', help='set multi 
> processor method')
> +        parser.add_option('-n', '--processors', action='store', 
> type='int', dest='n_processors', default=1, help='set number of 
> processors (may be ignored)')
>
>         # Parse the options.
> -        (options, args) = parser.parse_args()
> +        (options, args) = parser.parse_args(args)
>
>         # Debugging flag.
>         if options.debug:
>             __builtin__.Debug = 1
> -        +
>         # Pedantic flag.
>         if options.pedantic:
>             __builtin__.Pedantic = 1
> @@ -249,9 +255,9 @@
>                 parser.error("the logging and tee options cannot be 
> set simultaneously")
>
>             # The log file.
> -            log_file = options.log
> +            self.log_file = options.log
>         else:
> -            log_file = None
> +            self.log_file = None
>
>         # Tee.
>         if options.tee:
> @@ -260,9 +266,9 @@
>                 parser.error("the tee and logging options cannot be 
> set simultaneously")
>
>             # The tee file.
> -            tee_file = options.tee
> +            self.tee_file = options.tee
>         else:
> -            tee_file = None
> +            self.tee_file = None
>
>         # Number of positional arguements should only be 0 or 1.  1 
> should be the script file.
>         if len(args) > 1:
> @@ -276,6 +282,9 @@
>             # Test if the script file exists.
>             if not access(self.script_file, F_OK):
>                 parser.error("the script file " + `self.script_file` + 
> " does not exist")
> +
> +
> +
>
>
>         # Determine the relax mode and test for mutually exclusive modes.
> @@ -294,8 +303,9 @@
>             # Exclusive modes.
>             if options.test:
>                 parser.error("executing the relax test suite and 
> running relax in test mode are mutually exclusive")
> -            elif options.thread:
> -                parser.error("executing the relax test suite and 
> running relax in thread mode are mutually exclusive")
> +# FIXME: delete
> +#            elif options.thread:
> +#                parser.error("executing the relax test suite and 
> running relax in thread mode are mutually exclusive")
>             elif options.licence:
>                 parser.error("executing the relax test suite and 
> running relax in licence mode are mutually exclusive")
>
> @@ -315,28 +325,30 @@
>             # Exclusive modes.
>             if options.test_suite or options.unit_test:
>                 parser.error("the relax test mode and executing the 
> test suite are mutually exclusive")
> -            elif options.thread:
> -                parser.error("the relax modes test and thread are 
> mutually exclusive")
> +# FIXME delete
> +#            elif options.thread:
> +#                parser.error("the relax modes test and thread are 
> mutually exclusive")
>             elif options.licence:
>                 parser.error("the relax modes test and licence are 
> mutually exclusive")
>
>             # Set the mode.
>             mode = 'test'
>
> +        # FIXME threading deprecated
>         # Threaded mode.
> -        elif options.thread:
> -            # Make sure a script is supplied.
> -            if not self.script_file:
> -                parser.error("in threaded mode, a script is required")
> -
> -            # Exclusive modes.
> -            if options.test:
> -                parser.error("the relax modes thread and test are 
> mutually exclusive")
> -            elif options.licence:
> -                parser.error("the relax modes thread and licence are 
> mutually exclusive")
> -
> -            # Set the mode.
> -            mode = 'thread'
> +#        elif options.thread:
> +#            # Make sure a script is supplied.
> +#            if not self.script_file:
> +#                parser.error("in threaded mode, a script is required")
> +#
> +#            # Exclusive modes.
> +#            if options.test:
> +#                parser.error("the relax modes thread and test are 
> mutually exclusive")
> +#            elif options.licence:
> +#                parser.error("the relax modes thread and licence are 
> mutually exclusive")
> +#
> +#            # Set the mode.
> +#            mode = 'thread'
>
>         # Licence mode.
>         elif options.licence:
> @@ -349,22 +361,44 @@
>                 parser.error("the relax licence mode and executing the 
> test suite are mutually exclusive")
>             elif options.test:
>                 parser.error("the relax modes licence and test are 
> mutually exclusive")
> -            elif options.thread:
> -                parser.error("the relax modes licence and thread are 
> mutually exclusive")
> +# FIXME delete
> +#            elif options.thread:
> +#                parser.error("the relax modes licence and thread are 
> mutually exclusive")
>
>             # Set the mode.
>             mode = 'licence'
>
> +        # multi processor
> +        elif options.multiprocessor != 'uni':
> +
> +            # Exclusive modes.
> +            if options.test_suite or options.unit_test:
> +                parser.error("the relax multi processor mode and 
> executing the test suite are mutually exclusive")
> +            elif options.licence:
> +                parser.error("the relax multi processor multi 
> processor and licence are mutually exclusive")
> +
> +            self.multiprocessor_type = options.multiprocessor
> +            self.n_processors = options.n_processors
> +
> +            mode='multi'
> +
>         # Script mode.
>         elif self.script_file:
> +            #FIXME check for uniprocessor and n_processors
> +            self.multiprocessor_type = options.multiprocessor
> +            self.n_processors = options.n_processors
>             mode = 'script'
> +
>
>         # Prompt mode (default).
>         else:
> +            #FIXME check for uniprocessor and n_processors
> +            self.multiprocessor_type = options.multiprocessor
> +            self.n_processors = options.n_processors
>             mode = 'prompt'
>
>         # Return.
> -        return mode, log_file, tee_file
> +        return mode
>
>
>     def licence(self):
> @@ -459,16 +493,77 @@
>         # Exit.
>         sys.exit()
>
> +# FIXME: code shared with unit testing framework not changed from 
> unit tests version
> +# replace unit test version with this
> +def import_module(module_path, verbose=False):
> +    ''' import the python module named by module_path
> +
> +        @type module_path: a string containing a dot separated module 
> path
> +        @param module_path: a module path in python dot separated format
> +                            note: this currently doesn't support 
> relative module
> +                            paths as defined by pep328 and python 2.5
> +
> +        @type verbose: Boolean
> +        @param verbose: whether to report sucesses and failures for 
> debugging
> +
> +        @rtype:     list of class module instances or None
> +        @return:    the module path as a list of module instances or 
> None
> +                    if the module path cannot be found in the python 
> path
> +
> +        '''
> +
> +    module = None
> +    result = None
> +
> +    try:
> +        module = __import__(module_path,globals(),  locals(), [])
> +        print 'loaded module %s' % module_path
> +    except Exception, e:
> +        if verbose:
> +            print 'failed to load module_path %s' % module_path
> +            print 'exception:',e
> +
> +
> +    if module != None:
> +        result = [module]
> +        components = module_path.split('.')
> +        for component in components[1:]:
> +            module = getattr(module, component)
> +            result.append(module)
> +    return result
> +
> +#FIXME: mode not required should be an instance variable of relax?
> +def load_multiprocessor(mode,relax_instance):
> +
> +    processor_name = relax_instance.multiprocessor_type + '_processor'
> +    class_name= processor_name[0].upper() + processor_name[1:]
> +    module_path = '.'.join(('multi',processor_name))
> +
> +
> +    modules = import_module(module_path)
> +    clazz =  getattr(modules[-1], class_name)
> +    object = clazz(mode,relax_instance)
> +    return object
>
> if __name__ == "__main__":
> +    #import multi.uni_processor
> +    for elem in  sys.modules:
> +        if elem == 'multi.uni_processor':
> +            print 'found',elem
>     # Change this flag to 1 for code profiling.
>     profile_flag = 0
>
>     if not profile_flag:
> -        Relax()
> +        relax_instance = Relax()
> +        mode=relax_instance.arguments(sys.argv[1:])
> +        multi_processor = load_multiprocessor(mode,relax_instance)
> +        multi_processor.run()
> +
>     else:
>         def print_stats(stats, status=0):
>             pstats.Stats(stats).sort_stats('time', 'name').print_stats()
> -
> +        #FIXME: profiling won't work with multi processors.
>         profile.Profile.print_stats = print_stats
>         profile.run('Relax()')
> +
> +
>
>
> _______________________________________________
> relax (http://nmr-relax.com)
>
> This is the relax-commits mailing list
> relax-commits@xxxxxxx
>
> To unsubscribe from this list, get a password
> reminder, or change your subscription options,
> visit the list information page at
> https://mail.gna.org/listinfo/relax-commits
>
> .
>
>  


--
-------------------------------------------------------------------
Dr Gary Thompson
Astbury Centre for Structural Molecular Biology,
University of Leeds, Astbury Building,
Leeds, LS2 9JT, West-Yorkshire, UK             Tel. +44-113-3433024
email: garyt@xxxxxxxxxxxxxxx                   Fax  +44-113-2331407
-------------------------------------------------------------------