Re: Persist a class (not an instance)
David Wahler wrote: Kent Johnson wrote: Is there a way to persist a class definition (not a class instance, the actual class) so it can be restored later? A naive approach using pickle doesn't work: [snip] The idea is to persist classes that are created and modified at runtime. I couldn't resist the challenge, so I decided to take a crack at it. My code is below. (I hope it's OK to post it even though it's a bit on the long side.) So far, it seems to work OK; the biggest caveat to be aware of is that functions' global context is not preserved. My approach was to use pickle's __reduce__ protocol to store functions and classes. Of course, you can't modify the built-in function and classobj types, so I subclassed Pickler to override them. The advantage of this is that you don't need an extension to the pickling data format, and you can use the standard unpickler. (The custom module still needs to have been imported, as it adds the classobj type to __builtins__.) Unfortunately, I'm not sure how to go about making it work for new-style classes. It would seem to involve messing with dictproxy and descriptor objects, and that's getting me into more unfamiliar territory. I'm sure there's a better way to do this; this seemed like the simplest thing that could possibly work. This is actually pretty sweet. It seems to me that you'll be fine with new-style classes if you just save dict(ob.__dict__) instead of trying to save __dict__ directly, as that'll get rid of the dictproxy part. There's no generic way to save descriptors, as far as I know, but you can always register reducers for specific types, like property, and user-defined descriptor classes are likely to be picklable anyway. As for functions' global context, you could look to see if there's a __name__ present, in which case you can save a reference to that module's __dict__. Otherwise, simply pickle the func_globals as-is. Some folks might just want to do that anyway, if the code isn't actually being loaded from a module. Of course, the classic caveat regarding pickles and security applies to all this. That is, pickles and security don't mix. If you want one, you can't really get the other. ;-) -- http://mail.python.org/mailman/listinfo/python-list
Re: Persist a class (not an instance)
Kent Johnson enlightened us with: Is there a way to persist a class definition (not a class instance, the actual class) so it can be restored later? From the docs: Similarly, classes are pickled by named reference, so the same restrictions in the unpickling environment apply. Note that none of the class's code or data is pickled [...] Sybren -- The problem with the world is stupidity. Not saying there should be a capital punishment for stupidity, but why don't we just take the safety labels off of everything and let the problem solve itself? Frank Zappa -- http://mail.python.org/mailman/listinfo/python-list
Re: Persist a class (not an instance)
Sybren Stuvel wrote: Kent Johnson enlightened us with: Is there a way to persist a class definition (not a class instance, the actual class) so it can be restored later? From the docs: Similarly, classes are pickled by named reference, so the same restrictions in the unpickling environment apply. Note that none of the class's code or data is pickled [...] OK that confirms that pickle won't work. Is there another approach that will? Kent -- http://mail.python.org/mailman/listinfo/python-list
Re: Persist a class (not an instance)
Kent Johnson enlightened us with: OK that confirms that pickle won't work. Is there another approach that will? Well, since the classes are created at runtime as well, you could compile them using the appropriate API and call exec() on them. Sybren -- The problem with the world is stupidity. Not saying there should be a capital punishment for stupidity, but why don't we just take the safety labels off of everything and let the problem solve itself? Frank Zappa -- http://mail.python.org/mailman/listinfo/python-list
Re: Persist a class (not an instance)
Kent Johnson [EMAIL PROTECTED] wrote: Is there a way to persist a class definition (not a class instance, the actual class) so it can be restored later? A naive approach using pickle doesn't work: You can use copy_reg to customize pickling behavior. In this case, you'd need a custom metaclass to use as the type for your picklable classes. Moreover, if the class has attributes that you also want to pickle, such as methods or properties, you'll have to arrange for custom pickling of *them*, too. So, yes, there are ways, but not simple ones. Alex -- http://mail.python.org/mailman/listinfo/python-list
Re: Persist a class (not an instance)
Kent Johnson wrote:
Is there a way to persist a class definition (not a class instance,
the actual class) so it can be restored later? A naive approach
using pickle doesn't work:
[snip]
The idea is to persist classes that are created and modified at runtime.
I couldn't resist the challenge, so I decided to take a crack at it. My
code is below. (I hope it's OK to post it even though it's a bit on the
long side.) So far, it seems to work OK; the biggest caveat to be aware
of is that functions' global context is not preserved.
My approach was to use pickle's __reduce__ protocol to store functions
and classes. Of course, you can't modify the built-in function and
classobj types, so I subclassed Pickler to override them. The advantage
of this is that you don't need an extension to the pickling data
format, and you can use the standard unpickler. (The custom module
still needs to have been imported, as it adds the classobj type to
__builtins__.)
Unfortunately, I'm not sure how to go about making it work for
new-style classes. It would seem to involve messing with dictproxy and
descriptor objects, and that's getting me into more unfamiliar
territory.
I'm sure there's a better way to do this; this seemed like the
simplest thing that could possibly work.
-- David
#
# code_pickle.py
import sys, copy_reg, pickle, new, marshal, types, StringIO
# Needed to unserialize old-style classes
sys.modules['__builtin__'].classobj = new.classobj
# from http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/439096
def get_cell_value(cell):
return type(lambda: 0)(
(lambda x: lambda: x)(0).func_code, {}, None, None, (cell,)
)()
def func_constructor(name, code, defaults, closure):
return new.function(marshal.loads(code), globals(), name,
defaults, closure)
class CodePickler(pickle.Pickler):
def __init__(self, *args, **kwargs):
pickle.Pickler.__init__(self, *args, **kwargs)
self.dispatch = self.dispatch.copy()
self.dispatch[types.ClassType] = CodePickler.do_class
self.dispatch[types.FunctionType] = CodePickler.do_function
def save(self, ob, *args, **kwargs):
print ob
pickle.Pickler.save(self, ob, *args, **kwargs)
def do_class(self, ob):
if ob in (types.__dict__.values()):
self.save_global(ob)
else:
args = (ob.__name__, ob.__bases__, ob.__dict__)
self.save_reduce(type(ob), args)
def do_function(self, ob):
if ob == func_constructor:
self.save_global(ob)
else:
if ob.func_closure:
closure = tuple(map(get_cell_value, ob.func_closure))
else:
closure = None
args = (ob.func_name, marshal.dumps(ob.func_code),
ob.func_defaults, closure)
self.save_reduce(func_constructor, args)
def dumps(ob):
s = StringIO.StringIO()
CodePickler(s).dump(ob)
return s.getvalue()
# Example:
#
# import code_pickle
# class Foo:
# def show(self):
# print Foo!
#
# s = code_pickle.dumps(Foo)
# --
# import code_pickle, pickle
# Foo = pickle.loads(s)
# Foo().show
#
--
http://mail.python.org/mailman/listinfo/python-list
