On 2008-05-17 16:59, Alexandre Vassalotti wrote:
On Sat, May 17, 2008 at 5:05 AM, M.-A. Lemburg <[EMAIL PROTECTED]> wrote:
I'd like to bring a potential problem to attention that is caused
by the recent module renaming approach:
Object serialization protocols like e.g. pickle usually store the
complete module path to the object class together with the object.
Thanks for bringing this up. I was aware of the problem myself, but I
hadn't yet worked out a good solution to it.
It can also happen in storage setups where Python
objects are stored using e.g. pickle, ZODB being a prominent
example. As soon as a Python 2.6 application starts writing to
such storages, Python 2.5 and lower versions will no longer be
able to read back all the data.
The opposite problem exists for Python 3.0, too. Pickle streams
written by Python 2.x applications will not be readable by Python 3.0.
And, one solution to this is to use Python 2.6 to regenerate pickle
stream.
Another solution would be to write a 2to3 pickle converter using the
pickletools module. It is surely not the most elegant or robust
solution, but I could work.
I'm not really worried much about going from 2.x to 3.x.
Breakage is allowed for that transition.
However, the case is different for going from 2.5 to 2.6. Breakage
should be avoided if at all possible.
Now, I think there's a way to solve this puzzle:
Instead of renaming the modules (e.g. Queue -> queue), we leave
the code in the existing modules and packages and instead add
the new module names and package structure with pointers and
redirects to the existing 2.5 modules.
This would certainly work for simple modules, but what about packages?
For packages, you can't use the ``sys.modules[__name__] = Queue`` to
preserve module identity. Therefore, pickle will use the new package
name when writing its streams. So, we are back to the same problem
again.
A possible solution could be writing a compatibility layer for the
Pickler class, which would map new module names to their old at
runtime. Again, this is neither an elegant, nor robust, solution, but
it should work in most cases.
While it's possible to fix pickle (at least the Python version),
this would not help with other serialization formats that rely
on the .__module__ attribute mapping to an existing module.
It's better to address the problem at the module level.
Perhaps I have a misunderstanding of the reasoning behind
doing the renaming in the 2.x branch, but it appears that
the only reason is to get used to the new names. That's a
rather low priority argument in comparison to the breakage
the renaming will cause in the 2.x branch.
I think it's much better to have 2to3.py do the renaming
and only add warnings to the renamed modules in 2.x
(without actually applying any renaming).
It would also be possible to seed sys.modules with module
proxy objects (see e.g. mx.Misc.LazyModule from egenix-mx-base)
which only turn into real module object if the module is
referenced.
This would allow adding a "from __future__ import new_module_names"
which then results in loading proxies for all renamed modules
(without actually loading the modules until they are used under
their new names).
--
Marc-Andre Lemburg
eGenix.com
Professional Python Services directly from the Source (#1, May 18 2008)
>>> Python/Zope Consulting and Support ... http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ... http://python.egenix.com/
________________________________________________________________________
:::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,MacOSX for free ! ::::
eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
Registered at Amtsgericht Duesseldorf: HRB 46611
_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
