Re: Pickling problems with the relax data storage singleton. -- November 26, 2007

On Nov 26, 2007 2:32 AM, Chris MacRaild <macraild@xxxxxxxxxxx> wrote:

On Nov 23, 2007 7:14 PM, Gary Thompson <garyt@xxxxxxxxxxxxxxx> wrote:

> >>Hi Ed
> >>
> >>from looking at the error here I think the problem is that python
> >>internally thinks that Data() is still the classes constructor and is
> >>suprised to find that it is a different object...
> >>
> >>I think the answer is on the following page
> >>http://docs.python.org/lib/pickle-inst.html you need to define
> >>__getstate__ and __setstate__ but...
> >>
> >>Its Worse than that he's Dead Jim
> >>---------------------------------
> >>
> >>There is possibly more that you have to do here, we need to check what
> >>happens if two state pickles are read in: do we get two different
> >>singletons ! What happens if we read in a pickle after already creating
> >>a Data object...
> >>

I'm having problems with my computer at home, so cant test any of this
properly, but have a couple of ideas

One solution is to change names just before we pickle, then change
back after unpickling.

saveData = Data
Data = "">
should restore sanity, and allow:

pickle.dump(saveData, file)

It is a pretty crude hack, however, and as Gary says, raises lots of
questions as to what will happen if a state is loaded on top of the
existing, etc...

yes this was one of my thoughts as well, crude but effective ;-)

Another (better?) option would be to do saving and restoring state at
a lower level. So instead of simply pickling the whole Data object, we
have save and restore methods of the Data class that do the pickling
in a more controled way. This seems to me more true to the intent of
the singleton pattern, avoiding the complications Gary refers to. Also
the control over what gets saved and how might be useful.

A quick sketch of the sort of thing I'm thinking:

class Data(dict):
...
def _save(file):
P = Pickler(file)
dont_save = [] # a list of attributes that don't need saving,
eg. methods
for name, attr in self.__dict__.items():
if not name in dont_save:
P.dump((name,attr))

def _restore(file):
P = Unpickler(file)
while True:
try:
name, attr = U.load()
except EOFError:
break
setAttr(self, name, attr)

Then the user commands save_state and restore_state are just
front-ends to Data._save and Data._restore. Pickle needn't wory itself
with our unusual namespace, because only attributes of Data are
pickled, not Data itself. Save and restore functions are mehtods of
the Singleton object, so there is no risk of breaking the Singleton
pattern. Finally, we have the basis of a mechanism there to control
what gets saved/restored and how.

Cheers,

Chris

This seems good but have some other possibly interesting questions

1. should data be a singleton? what happens if i need to load two data hierachies for example for comparison between two runs (If i am barking up the wrong tree about the current design here please disregard this)
2. we have to be careful we don't pickle any python state (what is python state can change between versions) How do we identify what is python state, create an empty object?
3. Chris's will add an extra requirement for maintenance in the future as all new fields have to be registered for saving
4. what are the implications for cross version compatability, this may not be a requirement, but if it is then we will have impliment code to shim old formats into newer ones
5. If you put an XP hat on you would go with the simple but crude method for the moment and put in place a more complicated methodology if and whn we need it (could we design an interface that could deal with both and then impliment the more complicated stratergy if and when we need it?

regards
gary

> >>we definitley need to write some tests for these cases
> >>
> >>
> >>regards
> >>gary
> >>
> >>--
> >>-------------------------------------------------------------------
> >>Dr Gary Thompson
> >>Astbury Centre for Structural Molecular Biology,
> >>University of Leeds, Astbury Building,
> >>Leeds, LS2 9JT, West-Yorkshire, UK Tel. +44-113-3433024
> >>email: garyt@xxxxxxxxxxxxxxx Fax +44-113-2331407
> >>-------------------------------------------------------------------
> >>
> >>
> >>
> >>
> >>
> >
> >.
> >
> >
> >
>
>
> --
> -------------------------------------------------------------------
> Dr Gary Thompson
> Astbury Centre for Structural Molecular Biology,
> University of Leeds, Astbury Building,
> Leeds, LS2 9JT, West-Yorkshire, UK Tel. +44-113-3433024
> email: garyt@xxxxxxxxxxxxxxx Fax +44-113-2331407
> -------------------------------------------------------------------
>
>
>
>
> _______________________________________________
> relax (http://nmr-relax.com)
>
> This is the relax-devel mailing list
> relax-devel@xxxxxxx
>
> To unsubscribe from this list, get a password
> reminder, or change your subscription options,
> visit the list information page at
> https://mail.gna.org/listinfo/relax-devel
>

_______________________________________________
relax (http://nmr-relax.com)

This is the relax-devel mailing list
relax-devel@xxxxxxx

To unsubscribe from this list, get a password
reminder, or change your subscription options,
visit the list information page at
https://mail.gna.org/listinfo/relax-devel

Re: Pickling problems with the relax data storage singleton.

Header

Content

Related Messages