On Mon, Jun 14, 2010 at 3:43 AM, Jens Vagelpohl <j...@dataflake.org> wrote: > Hi all, > > In a system using Zope 2.11 and ZODB 3.7.0b3 I'm having a problem with > old transaction records containing pickle data that fails on > cPickle.Unpickler.noload as employed by ZODB.serialize.referencesf, I > get an AttributeError from inside the cPickle module:
It would be interesting to see the traceback. > def referencesf(p, oids=None): > """Return a list of object ids found in a pickle > > A list may be passed in, in which case, information is > appended to it. > > Weak references are not included. > """ > > refs = [] > u = cPickle.Unpickler(cStringIO.StringIO(p)) > u.persistent_load = refs > u.noload() > u.noload() > > # Now we have a list of referencs. Need to convert to list of > # oids: > <snip> > > The noload failure means I cannot pack the database right now. Since > only historical transactions contain the problematic attribute and I > don't care about those anymore Note that: - in ZODB 3.9, there's an option top pack without doing GC, and - ZODB 3.9 ZEO servers can be used with older clients. > I have introduced a simple change locally > that ignores the failure: > > def referencesf(p, oids=None): > """Return a list of object ids found in a pickle > > A list may be passed in, in which case, information is > appended to it. > > Weak references are not included. > """ > > refs = [] > u = cPickle.Unpickler(cStringIO.StringIO(p)) > u.persistent_load = refs > try: > u.noload() > u.noload() > except AttributeError: > pass > > # Now we have a list of referencs. Need to convert to list of > # oids: > <snip> That's a bad idea. > With this change I can pack the database and a few simple check > (fsrefs.py etc) indicate the database is consistent after the pack. Cool. You got lucky. :) > My question: Is there any risk associated with ignoring the failure > during packing? Yes. You might ignore references to objects causing them to be removed while still used. > The latest object versions for the persistent objects in > question do not have the problematic attribute anymore, so the latest > records are "safe" and will not cause the AttributeError, anyway. Right, it worked out for your DB. Jim -- Jim Fulton _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
