r27870 - in /branches/frame_order_cleanup: ./ multi/processor.py -- September 25, 2015 - 14:12

Author: bugman
Date: Fri Sep 25 14:12:18 2015
New Revision: 27870

URL: http://svn.gna.org/viewcvs/relax?rev=27870&view=rev
Log:
Merged revisions 27844-27845 via svnmerge from 
svn+ssh://bugman@xxxxxxxxxxx/svn/relax/trunk

........
  r27844 | tlinnet | 2015-05-27 03:09:57 +0200 (Wed, 27 May 2015) | 6 lines
  
  Suggestion Fix 1, in multi.processor.run_queue().
  
  This fix changes, that the amount of simulations is not chunked up before 
sending each chunk to a CPU.
  Rather, all jobs are to be submitted after each other, and finished for 
themselves.
  
  Bug #23618: (https://gna.org/bugs/index.php?23618): queuing system for 
multi processors is not well designed.
........
  r27845 | tlinnet | 2015-05-27 03:09:59 +0200 (Wed, 27 May 2015) | 3 lines
  
  Suggestion for fix 2, where jobs are continously replenished when other 
jobs are finished.
  
  Bug #23618: (https://gna.org/bugs/index.php?23618): queuing system for 
multi processors is not well designed.
........

Modified:
    branches/frame_order_cleanup/   (props changed)
    branches/frame_order_cleanup/multi/processor.py

Propchange: branches/frame_order_cleanup/
------------------------------------------------------------------------------
--- svnmerge-integrated (original)
+++ svnmerge-integrated Fri Sep 25 14:12:18 2015
@@ -1 +1 @@
-/trunk:1-27797,27800-27843
+/trunk:1-27797,27800-27845

Modified: branches/frame_order_cleanup/multi/processor.py
URL: 
http://svn.gna.org/viewcvs/relax/branches/frame_order_cleanup/multi/processor.py?rev=27870&r1=27869&r2=27870&view=diff
==============================================================================
--- branches/frame_order_cleanup/multi/processor.py     (original)
+++ branches/frame_order_cleanup/multi/processor.py     Fri Sep 25 14:12:18 
2015
@@ -585,6 +585,8 @@
 
         running_set = set()
         idle_set = set([i for i in range(1, self.processor_size()+1)])
+        all_jobs = list(reversed(xrange(1, len(queue)+1)))
+        completed_jobs = []
 
         if self.threaded_result_processing:
             result_queue = Threaded_result_queue(self)
@@ -606,8 +608,9 @@
             while len(running_set) != 0:
                 # Debugging printout.
                 if verbosity.level():
-                    print('\nIdle set:    %s' % idle_set)
-                    print('Running set: %s' % running_set)
+                    print('\n')
+                    print('Running nr of jobs: %i' % len(running_set))
+                    print('Completed jobs: %s' % len(completed_jobs))
 
                 # Get the result.
                 result = self.master_receive_result()
@@ -616,6 +619,13 @@
                 if result.completed:
                     idle_set.add(result.rank)
                     running_set.remove(result.rank)
+                    completed_jobs.append(all_jobs.pop())
+                    if len(queue) != 0:
+                        # Add new to que
+                        command = queue.pop()
+                        dest = result.rank
+                        self.master_queue_command(command=command, dest=dest)
+                        running_set.add(dest)
 
                 # Add to the result queue for instant or threaded processing.
                 result_queue.put(result)
@@ -633,8 +643,8 @@
         """
 
         #FIXME: need a finally here to cleanup exceptions states
-        lqueue = self.chunk_queue(self.command_queue)
-        self.run_command_queue(lqueue)
+        #lqueue = self.chunk_queue(self.command_queue)
+        self.run_command_queue(self.command_queue)
 
         del self.command_queue[:]
         self.memo_map.clear()