On Nov 26, 2007 3:32 AM, Chris MacRaild <macraild@xxxxxxxxxxx> wrote:
On Nov 23, 2007 7:14 PM, Gary Thompson <garyt@xxxxxxxxxxxxxxx> wrote:
Hi Ed
from looking at the error here I think the problem is that python
internally thinks that Data() is still the classes constructor and is
suprised to find that it is a different object...
I think the answer is on the following page
http://docs.python.org/lib/pickle-inst.html you need to define
__getstate__ and __setstate__ but...
Its Worse than that he's Dead Jim
---------------------------------
There is possibly more that you have to do here, we need to check what
happens if two state pickles are read in: do we get two different
singletons ! What happens if we read in a pickle after already creating
a Data object...
I'm having problems with my computer at home, so cant test any of this
properly, but have a couple of ideas
One solution is to change names just before we pickle, then change
back after unpickling.
saveData = Data
Data = saveData.__class__
should restore sanity, and allow:
pickle.dump(saveData, file)
It is a pretty crude hack, however, and as Gary says, raises lots of
questions as to what will happen if a state is loaded on top of the
existing, etc...
I've tried the suggestion but the unit test gives:
======================================================================
ERROR: The pickling and saving of the relax data storage singleton.
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/nmr5/edau/relax/relax-1.3/test_suite/unit_tests/state_testing_base.py",
line 104, in test_save
self.state.save_state(state=self.tmp_file)
File "/nmr5/edau/relax/relax-1.3/generic_fns/state.py", line 77, in
save_state
save_store = relax_data_store
UnboundLocalError: local variable 'relax_data_store' referenced before
assignment
======================================================================
Removing the line 'Data = saveData.__class__' gives:
======================================================================
ERROR: The pickling and saving of the relax data storage singleton.
----------------------------------------------------------------------
Traceback (most recent call last):
File
"/nmr5/edau/relax/relax-1.3/test_suite/unit_tests/state_testing_base.py",
line 104, in test_save
self.state.save_state(state=self.tmp_file)
File "/nmr5/edau/relax/relax-1.3/generic_fns/state.py", line 80, in
save_state
dump(save_store, file, 1)
PicklingError: Can't pickle <class 'data.Data'>: it's not the same
object as data.Data
======================================================================
I'm not sure if this approach will work unless we play with maybe
__weakref__ and similar things.
Another (better?) option would be to do saving and restoring state at
a lower level. So instead of simply pickling the whole Data object, we
have save and restore methods of the Data class that do the pickling
in a more controled way. This seems to me more true to the intent of
the singleton pattern, avoiding the complications Gary refers to. Also
the control over what gets saved and how might be useful.
A quick sketch of the sort of thing I'm thinking:
class Data(dict):
...
def _save(file):
P = Pickler(file)
dont_save = [] # a list of attributes that don't need saving,
eg. methods
for name, attr in self.__dict__.items():
if not name in dont_save:
P.dump((name,attr))
def _restore(file):
P = Unpickler(file)
while True:
try:
name, attr = U.load()
except EOFError:
break
setAttr(self, name, attr)
Then the user commands save_state and restore_state are just
front-ends to Data._save and Data._restore. Pickle needn't wory itself
with our unusual namespace, because only attributes of Data are
pickled, not Data itself. Save and restore functions are mehtods of
the Singleton object, so there is no risk of breaking the Singleton
pattern. Finally, we have the basis of a mechanism there to control
what gets saved/restored and how.
This is a very good idea! Maybe an alternative, that is very similar
in spirit, is to create functions that pickle already knows about.
For example the __reduce__() method
(http://docs.python.org/lib/node321.html). This one function should
work for both pickling and unpickling. I don't know if this method is
useful for unpickling a singleton though. Currently the
generic_fns.state.load_state() function just resets the singleton
(clearing out all data) and then dumps the objects of the pickled
object back into the existing singleton (the key; PipeContainer pairs
are also placed back into the singleton). Anyway, I like the idea of
the methods within the singleton.
Regards,
Edward
Re: Pickling problems with the relax data storage singleton.