Re: [ZODB-Dev] [zopefoundation/ZODB] 49919d: test for POSKeyError during transaction commit
Le 05/02/14 15:44, Jim Fulton a écrit : When setting a key value pair on a new (not already committed) instance of a standard BTree, readCurrent method is not called on the connection. This is with your change, right? No, it is without my change. My understanding is that it is due to the fact that _p_jar and _p_oid are only set during transaction commit. They can be set earlier by calling the connection add method. This is used often for frameworks that use object ids at the application level. I do not understand what you mean. However, with a new BTree instance that also inherits from Acquisition.Implicit, readCurrent method is called on ZODB connection when setting key value pair. The only explanation I found is that this instance _p_jar attribute has a value (acquired in a way or another ?). You could also simulate this by adding an object to a connection using a connection's add method. Wanna update the test to use this technique instead? I'll make a try. But I think I remember having already tried (I have been working on this a few times since september). So if I do not succeed in a reasonable amount of time, I'll come back here. In this case, when readCurrent is called on an object created during a savepoint and this savepoint is rolled back, the oid is leftover in the Connection._readCurrent mapping. This leads to the POSKeyError when committing later as checkCurrentSerialInTransaction cannot check the object since it went away at rollback. This brings us to the fix I propose: calls to readCurrent should not track objects with oid equal to z64. ... This was a very long explanation which I hope will help to confirm the fix or to come up with a better one. PS: keep in mind that english is not my mothertongue. :) You do very well. I think your fix is correct. As you point out, It doesn't make sense to guard against conflicts on new objects. I think a cleaner test could be written using the connection add method. Jim -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] [zopefoundation/ZODB] 49919d: test for POSKeyError during transaction commit
Le 03/02/14 20:53, Tres Seaver a écrit : I wish you hadn't pushed that -- some of these changes are definitely inappropriate on the 3.10 branch (adding an Acquisition dependency is definitely wrong). Tres. Acquisition is added as a test dependency. Any hint how to replicate the bug without acquisition is welcome. Which other change is inappropriate ? -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] POSKeyError when transaction.commit after savepoint.rollback and connection.readCurrent
Hi all, It needed time and obstinacy to find how to write a test that triggers the same POSKeyError during commit as triggered by some Plone 3 to 4 migrations. For more details about the error, see http://rpatterson.net/blog/poskeyerror-during-commit The test is now pushed to the 3.10 branch on github : https://github.com/zopefoundation/ZODB/commit/49919d31cfa73410cede0af969d80527a9fd852c The fix is pushed as well. https://github.com/zopefoundation/ZODB/commit/9ce470f70acc9655aaf915e5a74678b43f6312da Both commits are referenced in an issue https://github.com/zopefoundation/ZODB/issues/16 I have a request and a question : - Can we get a new 3.10.X release (iow 3.10.6) with the fix ? I am happy to do it myself if needed. - As the test above depends on Acquisition (which is not ported to Python 3), what should be the strategy to include my work in master ? Pull the fix only ? Pull the fix and do some magic with the test so that it runs only in Python 2 ? Other suggestion ? Regards, -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] POSKeyError when transaction.commit after savepoint.rollback and connection.readCurrent
Le 03/02/14 15:12, Marius Gedminas a écrit : Could you tell us why that test depends on Acquisition? Is it possible to replicate the bug using pure Python code? Marius Gedminas I have not been able to replicate the bug with pure Python code. Reason why it took me so much time. IIRC, the issue is that on newly created objects that inherit from Acquisition.Implicit, the _p_jar attribute has a value (I guess it is acquired). Without acquisition, because a newly created BTree._p_jar does not have any value before transaction is committed, any call to readCurrent is then discarded. I understand this gives the feeling that the fix should be in Acquisition rather than in ZODB. I have tried to make a fix for the Acquisition code but was unable to understand enough. Nevertheless, my understanding of the readCurrent API makes me think that it is actually useless to keep reference to newly created objects (iow in same transaction). And that the fix I propose makes sense anyway. -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] POSKeyError when transaction.commit after savepoint.rollback and connection.readCurrent
Le 17/09/13 10:44, Godefroid Chapelle a écrit : Hi, Plone 3 to 4 migration occasionally triggers a POSKeyError during transaction commit. See http://rpatterson.net/blog/poskeyerror-during-commit I encountered the issue yesterday. I committed a test and a fix in branch 3.10. https://github.com/zopefoundation/ZODB/commit/6457bcfd07b3b77f240cbc6d60f222f0d40e8d5d Before the fix, the test triggered a traceback very similar to the one seen when migrating Plone. As I am not an expert of ZODB internals, I'd like someone to review my fix even if the test suite does not break. I confirm that I am not a ZODB internals expert :-( The fix does avoid the issue triggered by my test. But does NOT fix the Plone migration issue. IOW, the test is not reproducing the issue triggered by the Plone migration. This is what I have found : At the end of the Connection.commit() method, oids found in Connection._readCurrent mapping are checked for mismatch between the committed tid and the one stored in _readCurrent mapping. If there is a mismatch, the object must be invalidated in cache. In the Plone migration issue, when getting to the step where the committed tid is fetched, a POSKeyError is triggered by getTid() method. I found out that the tid stored in _readCurrrent for this falty oid is equal to z64 (eight nul chars). This is what I deduce : the oid, tid pair (where tid is null) corresponds to an object that was added to the transaction but not completely cleaned up by the transaction rollback (I am looking at rollback because my migration does call savepoint.rollback() five times and there are five oids with null tid leftover in Connection._readCurrent when the POSKeyError is triggered. But this might just be a coincidence.) Any hint where to add the call to pop those falty oids out of _readCurrent during transaction rollback, or to enlighten me about things I misunderstand is welcome. Regards -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] POSKeyError when transaction.commit after savepoint.rollback and connection.readCurrent
Hi, Plone 3 to 4 migration occasionally triggers a POSKeyError during transaction commit. See http://rpatterson.net/blog/poskeyerror-during-commit I encountered the issue yesterday. I committed a test and a fix in branch 3.10. https://github.com/zopefoundation/ZODB/commit/6457bcfd07b3b77f240cbc6d60f222f0d40e8d5d Before the fix, the test triggered a traceback very similar to the one seen when migrating Plone. As I am not an expert of ZODB internals, I'd like someone to review my fix even if the test suite does not break. Thanks -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Progress report: porting persistent, BTrees to Python3
Le 15/12/12 01:52, Tres Seaver a écrit : I fixed the remainig issues in persistent and released 4.0.5 today: its tests properly exercise the C extensions Under Python 3.2 / 3.3. I want to express my thanks to you, Tres, for taking care of that work ! This port of ZODB to Python 3 is really a crucial step for the ZTK ecosystem. After the work already done on zope.interface and zope.component. Further, I'd like to also thank Jim for his work on porting buildout. When this will be finished, porting the rest of the ZTK should be much easier, which hopefully implies that more of us will be able to participate. Regards, -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] TypeError: ('object.__new__(SyncedLanguages) is not safe, use Persistence.Persistent.__new__()', ...
Hi,
I have been fighting all day to understand a bug with the unregistration
of a utility from a local site.
After unregistration, some instance is left in the _subscribers list of
the utilities AdapterRegistry in my local site.
This avoids me to remove the code as it does leave a broken object.
I have come to the conclusion that the code that first registered the
utility is the culprit : the utility itself is not persistent.
It seems that the utility instance is unserialized at least two times
during unregistration, which leads (if I understand well) to two
different objects in memory.
This breaks the unsubscription algorithm that is based on object identity.
I naively tried to introduce the Persistent base class into the utility
calss. When I try to access the utility instance again, I get the error
mentioned in the subject of this mail :
TypeError: ('object.__new__(SyncedLanguages) is not safe, use
Persistence.Persistent.__new__()', function _reconstructor at
0x1004297d0, (class
'Products.LinguaPlone.vocabulary.SyncedLanguages', type 'object', None))
If I understand well, the a posteriori introduction of Persistent cannot
happen alone.
Can anyone tell me what I should do add to the class I try to make it
actually persistent ?
PS: It is late here, I hope I have not told too many silly things
--
Godefroid Chapelle (aka __gotcha) http://bubblenet.be
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] TypeError: ('object.__new__(SyncedLanguages) is not safe, use Persistence.Persistent.__new__()', ...
Le 31/12/10 02:02, Godefroid Chapelle a écrit :
Hi,
I have been fighting all day to understand a bug with the unregistration
of a utility from a local site.
After unregistration, some instance is left in the _subscribers list of
the utilities AdapterRegistry in my local site.
This avoids me to remove the code as it does leave a broken object.
I have come to the conclusion that the code that first registered the
utility is the culprit : the utility itself is not persistent.
It seems that the utility instance is unserialized at least two times
during unregistration, which leads (if I understand well) to two
different objects in memory.
This breaks the unsubscription algorithm that is based on object identity.
I naively tried to introduce the Persistent base class into the utility
calss. When I try to access the utility instance again, I get the error
mentioned in the subject of this mail :
TypeError: ('object.__new__(SyncedLanguages) is not safe, use
Persistence.Persistent.__new__()',function _reconstructor at
0x1004297d0, (class
'Products.LinguaPlone.vocabulary.SyncedLanguages',type 'object', None))
If I understand well, the a posteriori introduction of Persistent cannot
happen alone.
Can anyone tell me what I should do add to the class I try to make it
actually persistent ?
Let's try to make an understandable question :
Can anyone tell me what I should add to the class to make it actually
persistent, while making the old instances unserializable ?
PS: It is late here, I hope I have not told too many silly things
--
Godefroid Chapelle (aka __gotcha) http://bubblenet.be
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] TypeError: ('object.__new__(SyncedLanguages) is not safe, use Persistence.Persistent.__new__()', ...
Le 31/12/10 04:08, Marius Gedminas a écrit :
On Fri, Dec 31, 2010 at 02:02:36AM +0100, Godefroid Chapelle wrote:
I have been fighting all day to understand a bug with the unregistration
of a utility from a local site.
After unregistration, some instance is left in the _subscribers list of
the utilities AdapterRegistry in my local site.
Sounds like https://bugs.launchpad.net/zope.component/+bug/551049
This avoids me to remove the code as it does leave a broken object.
I have come to the conclusion that the code that first registered the
utility is the culprit : the utility itself is not persistent.
It seems that the utility instance is unserialized at least two times
during unregistration, which leads (if I understand well) to two
different objects in memory.
This breaks the unsubscription algorithm that is based on object identity.
Actually, half of it is based on identity, and half on equality; when
these disagree, you end up with inconsistent state.
Yup
Your example is a good argument for making both halves use equality
rather than identity.
I naively tried to introduce the Persistent base class into the utility
calss. When I try to access the utility instance again, I get the error
mentioned in the subject of this mail :
TypeError: ('object.__new__(SyncedLanguages) is not safe, use
Persistence.Persistent.__new__()',function _reconstructor at
0x1004297d0, (class
'Products.LinguaPlone.vocabulary.SyncedLanguages',type 'object', None))
If I understand well, the a posteriori introduction of Persistent cannot
happen alone.
Can anyone tell me what I should do add to the class I try to make it
actually persistent ?
I think a more fruitful approach would be to fix the zope.component bug
and then unregister the utility in the normal way.
Marius, did you already try to fix zope.component ? Is it because it is
hard that you accepted the Won't fix ? Or because of lack of time ?
Alternatively, you could hack up a __new__ method to always return the
same instance, so you can unregister it cleanly before removing the
code. This will not work if you have more than one instance in the DB.
Could that __new__ method insert Persistent as a base class if not present ?
Fixing up _subscribers directly is also a possible workaround, if you're
feeling brave.
Marius Gedminas
Thanks for your help.
--
Godefroid Chapelle (aka __gotcha) http://bubblenet.be
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/
ZODB-Dev mailing list - ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] undo (and storage interface) brokenness
On 24/12/09 03:09, Martin Aspeli wrote: Hanno Schlichting wrote: Unfortunately, I think the people most likely to have used it in a pinch are less likely to be reading this list. If the current API is broken, we should fix it. If that means fixing some of the UI in Zope to use a new API, that's OK too. Few people will have customised this or relied on the old API. If we had to say you only get step-by-step undo from the most recent transaction, then that'd probably be OK too. In this case, we should have a UI that lets you specify a number (undo N transactions) or a limit (undo up to and including transaction X). I think people understand that undoing a single transaction that's not the most recent one is error-prone anyway. Martin Some of my customers are low-budget customers. They rather have me restore some content with a few minutes of work even at the cost (for them) of undoing later transactions. IOW, I'll support Martin's statement : step-by-step undo should be kept if possible. -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Repozo tests -- not
Jim Fulton wrote: snip No one seems to be willing to step up and take responsibility. I guess everyone assumes I will. Jim I'll take the responsibility of transforming the manual test to an automated... -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Repozo tests -- not
Godefroid Chapelle wrote: Jim Fulton wrote: snip No one seems to be willing to step up and take responsibility. I guess everyone assumes I will. Jim I'll take the responsibility of transforming the manual test to an automated... Done in svn.zope.org/repos/main/ZODB/branches/test_repozo I added code changes from revisions 105888, 105915 and 105919. Tests pass for me on MacOSX with 2.4, 2.5 and 2.6 -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Repozo tests -- not
Tres Seaver wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Godefroid Chapelle wrote: Godefroid Chapelle wrote: Jim Fulton wrote: snip No one seems to be willing to step up and take responsibility. I guess everyone assumes I will. Jim I'll take the responsibility of transforming the manual test to an automated... Done in svn.zope.org/repos/main/ZODB/branches/test_repozo I added code changes from revisions 105888, 105915 and 105919. Tests pass for me on MacOSX with 2.4, 2.5 and 2.6 Thanks for throwing yourself on that grenade, Godefroid. I have checked in changes to your branch which: - - Use a tempdir for backup / restore. Sure. - - Clean out module-scope imports. Can you tell me why this is better ? - - Avoid shellling out to run repozo, but rather use its main(), passing argv. I thought of this but decided to not go that way to test that repozo can actually run on a living database. Even if the code that was there still needs to be modified to achieve that goal. - - Suppress noisy output, unless NOISY_REPOZO_TEST_OUTPUT is in the environment. Cool ! Remaining cleanup would be to remove the Monte Carlo use of random to generate permutations: the test takes a *long* time to run, because it tries to run 100 times and use random.random to get permutations of the three choices run: these should be six separate tests, likely. I had thought of this as well. This will be my next task. Tres. - -- -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Repozo tests -- not
Tres Seaver wrote: - - Avoid shellling out to run repozo, but rather use its main(), passing argv. I thought of this but decided to not go that way to test that repozo can actually run on a living database. Even if the code that was there still needs to be modified to achieve that goal. Computing the path to repozo to run as a script was a bit wonky: I just changed the 'main()' function to take optional arguments, which also gave me more flexibility in how thigs run (e.g., turning off noisy output). I suspect I do not express myself correctly. I try to say that repozo should be tested against a database being mutated (the current test only exercise backups when the db is closed). As far as I can understand, THE reason for repozo is to backup without stopping the ZODB. -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Moving temporary storage from each zeo client to zeo server
Hi, I found the following https://weblion.psu.edu/trac/weblion/wiki/TemporaryStorageInZeo where Weblion documents how to move temporary storage from each zeo client to zeo server in Zope2. I understand the pro (no need for sticky sessions). What would be the cons ? Thanks -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] problem with _p_mtime
Alan Runyan wrote: We run consistency checks on the ZODB - the same as we do on SVN -- nightly. http://svn.zope.org/ZODB/trunk/src/ZODB/scripts - fstest.py - checkbtrees.py - fsrefs.py When reading documentation of fsrefs.py, I see it is not recommended to run it again a live FileStorage. Do you stop your servers nightly to do it ? We have never had corruption in years. But I just wanted to ask what people are doing to find any problems with consistency; so the problem can be solved ASAP. Miles, have you run any of these scripts across your production ZODB? cheers alan -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Re: Ruby/Smalltalk OODB
Wichert Akkerman wrote: Martijn Faassen wrote: What I find interesting is that Python has had such a thing for about a decade (the ZODB), and a mostly vaporware announcement in the Ruby world makes such a splash. That's because most things Zope are not very good with PR. Smart people are usually very demanding with themselves, which leads to the bad pattern of not advertizing our good stuff because it is not perfect. Time to change our patterns, and advertize stuff before they are perfect ;-) -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: ZODB Benchmarks
Godefroid Chapelle wrote: Jim Fulton wrote: On Dec 6, 2007, at 2:40 PM, Godefroid Chapelle wrote: Jim Fulton wrote: On Nov 6, 2007, at 2:40 PM, Sidnei da Silva wrote: Despite this change there are still a huge amount of unexplained calls to the 'persistent_id' method of the ObjectWriter in serialize.py. Why 'unexplained'? 'persistent_id' is called from the Pickler instance being used in ObjectWriter._dump(). It is called for each and every single object reachable from the main object, due to the way Pickler works (I believe). Maybe persistent_id can be analysed and optimized for the most common cases? Yup. Note that there is a undocumented feature in cPickle that I added years ago to deal with this issue but never got around to pursuing. Maybe someone else would be able to spend the time to try it out and report back. If you set inst_persistent_id, rather than persistent_id, on a pickler, then the hook will only be called for instances. This should eliminate that vast majority of the calls. Note that this feature was added back when testing was minimal or non-existent, so it is untested, however, the implementation is simple enough. :) Do you mean that the ZODB has enough tests now that making the change and running the tests might already be a good proof ? No, I mean that pickle and cPickle lack tests for this feature. Or should we be more prudent ? It would be nice to try this out with ZODB to see if it makes much difference. If it does, then that would provide extra motivation for me to add the missing test. Roché Compaan said he would try it out, but I just realized that he might have been waiting for me. Laurent (cced) tried it today and it seems it does make a difference. Our benchmark is running this night with bigger amount of content. We will be back with results tomorrow. We can measure some benefit. For tests on a ZODB prefilled with 100k instances of an archetypes class, update of an instance : 12% improval insert of an instance : 15% improval If it would, What do you mean by 'If it would' ? If we can measure a benefit. Jim -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: ZODB Benchmarks
Jim Fulton wrote: On Dec 7, 2007, at 10:55 AM, Godefroid Chapelle wrote: Jim Fulton wrote: It sounds like I should write some pickle and cPickle tests and we should update the ZODB trunk to take advantage of this. (/me fears gettimg mired in Python 3.) Jim Would you do that on Python 2.4, 2.5 or ... ? I would do it on ?. ;) I'd help with pleasure... I guess I'd need to do it for 2.4, 2.5, trunk and hope that pickle hasn't been ported to Python 3 yet. :) This is just a fairly simple test, so shouldn't be a big deal. ... but I fear it would take you longer to explain what to do than to do it. Jim -- Jim Fulton Zope Corporation -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
[ZODB-Dev] Re: ZODB Benchmarks
Jim Fulton wrote: On Nov 6, 2007, at 2:40 PM, Sidnei da Silva wrote: Despite this change there are still a huge amount of unexplained calls to the 'persistent_id' method of the ObjectWriter in serialize.py. Why 'unexplained'? 'persistent_id' is called from the Pickler instance being used in ObjectWriter._dump(). It is called for each and every single object reachable from the main object, due to the way Pickler works (I believe). Maybe persistent_id can be analysed and optimized for the most common cases? Yup. Note that there is a undocumented feature in cPickle that I added years ago to deal with this issue but never got around to pursuing. Maybe someone else would be able to spend the time to try it out and report back. If you set inst_persistent_id, rather than persistent_id, on a pickler, then the hook will only be called for instances. This should eliminate that vast majority of the calls. Note that this feature was added back when testing was minimal or non-existent, so it is untested, however, the implementation is simple enough. :) Do you mean that the ZODB has enough tests now that making the change and running the tests might already be a good proof ? Or should we be more prudent ? If it would, What do you mean by 'If it would' ? then of course we should contribute documentation and a test to the Python source tree. Jim -- Jim Fulton Zope Corporation -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
