New submission from Craig Citro <craigci...@gmail.com>: Currently, it's impossible to use the usual pickle mechanisms to pickle a dynamically created class, even if the user requests a different pickling mechanism via copy_reg. The attached patch makes this customization possible by simply transposing two blocks of code.
Longer explanation: Classes are pickled by name, which is of course problematic when trying to pickle a dynamically created class. The natural solution would be to create a __reduce__ method on the metaclass. However, as mentioned in issue 494904, this won't work -- the issue is that for any class C with a metaclass M, C.__reduce__ is the unbound method for instances of class C, as opposed to the bound method M.__reduce__ for C (i.e. viewing C as an instance of M). Guido's patch on that ticket does the sensible thing, which is force the class to be pickled by name -- this is fine, until you want to pickle a class that's created at runtime. The copy_reg module exists to handle custom pickling of objects, which is exactly what's needed here. However, the code from #494904 that checks for instances of a metaclass does this *just before* it looks at the copy_reg dispatch table. The patches just reorder these tests in pickle.py and cPickle.c, so that one can register a custom pickler for instances of a metaclass. Comments: First, let me preemptively say that I do indeed have a use case for this fix. We ran into this bug working on Sage (http://www.sagemath.org), where we create lots of dynamic classes at runtime (to model mathematical relationships between different kinds of objects). There's a healthy mix of dynamic and non-dynamic classes floating around, and we want the user to be able to pickle them without having to know anything about how the class was created. We could create our own pickling function, but we'd much rather just use the default Python mechanisms, especially since it's such a small fix. Second, this patch is fairly "safe," in the sense that it's pretty unlikely it can break any existing code. The only way it could break is if someone created a dynamic class, registered a pickler for it with copy_reg, and was depending on calls to pickle to fail. Third, there's a few extra lines of code in the chunk the patch moves around coming from ticket #502085. These are there to deal with versions of Boost circa 2002. Is it worth removing these? The attached patch is against trunk (r77421). At least as I write this, the same patch should apply against the py3k branch (up to renaming copy_reg to copyreg, anyway), but I'm happy to do the legwork of rebasing it as needed. Also, I'd happily review something else in exchange for a review of this. We'll be keeping this patch around in Sage until it gets merged, so the sooner the better. Authors: Nicolas Thiery (nthi...@users.sf.net) first hunted this down and wrote a patch for cPickle.c. I did the (fairly trivial) work of mirroring the fix in pickle.py, and added the tests in pickletester. ---------- components: Library (Lib) files: dynamic_class_copyreg.patch keywords: patch messages: 97702 nosy: craigcitro severity: normal status: open title: Pickling of classes with a metaclass and copy_reg type: behavior versions: Python 2.5, Python 2.6, Python 2.7, Python 3.1, Python 3.2 Added file: http://bugs.python.org/file15853/dynamic_class_copyreg.patch _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue7689> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: http://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
