Re: [ZODB-Dev] ZODB 3.9/3.8 incompatibility (was Re: Data.fs size grows non-stop)

2009-12-17 Thread Kapil Thangavelu 
On Thu, Dec 10, 2009 at 8:12 PM, Marius Gedminas mar...@gedmin.as wrote:

 On Thu, Dec 10, 2009 at 02:07:06PM -0500, Jim Fulton wrote:
  On Thu, Dec 10, 2009 at 1:50 PM, Marius Gedminas mar...@gedmin.as
 wrote:
   On Thu, Dec 10, 2009 at 12:41:11PM -0500, Jim Fulton wrote:
   On Thu, Dec 10, 2009 at 11:54 AM, Marius Gedminas mar...@gedmin.as
 wrote:
   In 3.9 for FileStorage, if you give a starting tid that is toward the
   end of the file,
   it will scan backward, saving a lot of time.
  
   Ah, but how can I get a valid tid that is toward the end of the file?
   I assume that if I give a tid that doesn't exist, I'll get an error --
   which is the behavior oldstate() exhibits -- rather than the next
   existing transaction after that timestamp, or I could fudge by getting
   the current timestamp and subtracting 24 hours.
 
  No, the iterator starts at the first transactions who's tid is = the
  start tid.
 
  So, guess a time and use repr(ZODB.TimeStamp.TimeStamp(...))

 Cool!  Thank you,

 Marius Gedminas
 --


another way is to use the fstools.prev_txn starting at the end of the file
 to grab the newest transactions, the script i send to the list last week
uses it to poke a large zodb to just get the contents of the last 20
transactions without iterating through the whole thing or guessing a
timestamp.

https://mail.zope.org/pipermail/zodb-dev/attachments/20091207/531fc1e4/attachment.obj

cheers,

kapil
___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
https://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] Plone Get Paid and zodb corruption

2009-07-06 Thread Kapil Thangavelu 
the referenced code, was moving an object from a session (ram zodb ) to a
persistent zodb, and making it explicitly not a cross database reference via
pickling. shouldn't pickling the object remove any database reference, all
the _p_ variables should be gone, and thus the pickle laundered object is
unattached to a db. ie.

 import transaction
 from ZODB.FileStorage import FileStorage
 from ZODB.DB import DB
 from persistent import Persistent
 class bar( Persistent ): pass
...
 a = bar()
 db = DB(FileStorage('tmp'))
 root = db.open().root()
 root['foo'] = a
 transaction.commit()
 root['foo']
__main__.bar object at 0x8a530
 root['foo']._p_jar
Connection at 0062d090
 root['foo']._p_oid
'\x00\x00\x00\x00\x00\x00\x00\x01'
 from cPickle import dumps
 from cPickle import loads
 x = loads(dumps(root['foo']))
 x
__main__.bar object at 0x7352b0
 x._p_jar
 x._p_oid


although the error message is clear. its not clear if the referenced code
section is responsible.

-kapil

On Mon, Jun 29, 2009 at 11:57 AM, Jim Fulton j...@zope.com wrote:

 Your application is creating cross-database references. The version of
 ZODB you're using on the ZEO server doesn't support packing databases
 with cross References. Upgrade your ZEO server to ZODB 3.8.1 or ZODB
 3.9.

 Jim


 On Jun 29, 2009, at 11:54 AM, rafael wrote:

  Hi,
 
  I don't know if you are aware of Plone Getpaid issue 209
 
  http://code.google.com/p/getpaid/issues/detail?id=209
 
  The title of this issue is not 100% correct. As I get this
  corruption without restarting the server.  There are some tracebacks
  over there.
 
  I think the following lines of the payment processor might have to
  do with it
 
   # make cart safe for persistence by using pickling
  order.shopping_cart = loads(dumps(cart))
  order.user_id = getSecurityManager().getUser().getId()
  order_manager.store(order)
 
  # have to wait for the order to be created and the cart
  added for this to work
  order.finance_workflow.fireTransition('authorize')
 
  # save html for button - we'll destroy the cart later on
  html = button.cart_post_button(order)
 
  # and destroy the cart
  cart_util.destroy(self.context)
 
  return html
 
 
 
  Recently, I tried moving from zeo to filestorage to see if this was
  related. I get the same error, but now instead of breaking my site
  (givving errors for different actions), it seems it is just
  preventing me from packing... The tracebak I get when packing is:
 
  Traceback (innermost last):
 
• Module ZPublisher.Publish, line 119, in publish
• Module ZPublisher.mapply, line 88, in mapply
• Module ZPublisher.Publish, line 42, in call_object
• Module string, line 3, in _facade
• Module AccessControl.requestmethod, line 64, in _curried
• Module App.ApplicationManager, line 431, in manage_pack
• Module ZODB.DB, line 624, in pack
• Module ZODB.FileStorage.FileStorage, line 1352, in pack
• Module ZODB.FileStorage.fspack, line 482, in pack
• Module ZODB.FileStorage.fspack, line 228, in findReachable
• Module ZODB.FileStorage.fspack, line 304, in
  findReachableAtPacktime
• Module ZODB.FileStorage.fspack, line 377, in findrefs
• Module ZODB.serialize, line 645, in referencesf
  KeyError: 'n'
 
  Thanks,
 
  Rafael
  ___
  For more information about ZODB, see the ZODB Wiki:
  http://www.zope.org/Wikis/ZODB/
 
  ZODB-Dev mailing list  -  ZODB-Dev@zope.org
  http://mail.zope.org/mailman/listinfo/zodb-dev

 --
 Jim Fulton
 Zope Corporation


 ___
 For more information about ZODB, see the ZODB Wiki:
 http://www.zope.org/Wikis/ZODB/

 ZODB-Dev mailing list  -  ZODB-Dev@zope.org
 http://mail.zope.org/mailman/listinfo/zodb-dev

___
For more information about ZODB, see the ZODB Wiki:
http://www.zope.org/Wikis/ZODB/

ZODB-Dev mailing list  -  ZODB-Dev@zope.org
http://mail.zope.org/mailman/listinfo/zodb-dev