[ZODB-Dev] problem with _p_mtime
Hi, I've moved a FileStorage from one (old) machine to another (new) machine, but when I mount it on the new machine I get a lot of time errrors: Traceback (innermost last): Module ZPublisher.Publish, line 115, in publish Module ZPublisher.mapply, line 88, in mapply Module ZPublisher.Publish, line 41, in call_object Module Shared.DC.Scripts.Bindings, line 311, in __call__ Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec Module App.special_dtml, line 176, in _exec Module DocumentTemplate.DT_Let, line 76, in render Module DocumentTemplate.DT_In, line 703, in renderwob Module DocumentTemplate.DT_With, line 76, in render Module DocumentTemplate.DT_Var, line 214, in render Module App.PersistentExtra, line 43, in bobobase_modification_time Module DateTime.DateTime, line 509, in __init__ Module DateTime.DateTime, line 760, in _parse_args Module DateTime.DateTime, line 437, in safelocaltime TimeError: The time 98040302366.810165 is beyond the range of this Python implementation. I googled around on this problem and didn't find much, but I did find that as a temporary fix, I've put a try/except in App.PersistentExtra, so if it generates an exception, it returns the current date and time. This keeps things working. Out of interst, can anyone tell me why there is a difference? On both machines, python was compiled from source with the same options, so I'm not sure what this is down to. More importantly, what's the most straightforward way to set all the dates to usable values? From my POV, it's fine to set them all to the current date, but I don't know how to go about doing this exhaustively. Thanks for your help, Miles ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] strange commit error
Hi, Thanks for your help >> >>(from ZODB.serialize.py - line 350) >> >>if obj._p_jar._implicitlyAdding(oid): >>raise InvalidObjectReference( >>"A new object is reachable from multiple >> databases. " >>"Won't try to guess which one was correct!" >>) >> >> Can anyone shed any light on what this error means?! > > It means a new object is referenced from objects in multiple databases > and it doesn't know what database to assign it to. After some digging into the objects, I think I understand this. It's come up as follows: - I load object A from the old database - Object A has an attribute B, which is an object also in the old database - I create a fresh copy of A, but leave the attribute B remained a reference to B in the old database - When I went to commit A, I got an error because A.B points to the old database still Is this right? > >> Given a persistent >> object, what is the correct way to get a completely fresh copy that is >> not tied to any database at all, so I can transfer these old objects >> to >> the current database. > > I would use the database export/import API to export it from one > database and import it to another. (There are issues with doing this > in Zope 3, but I assume you're using Zope 2. Sorry if I'm being thick - I thought the code below did indeed use the export/import API: f=tempfile.TemporaryFile() old_ob._p_jar.exportFile(old_ob._p_oid,f) f.seek(0) container = app.new_folder ob=container._p_jar.importFile(f) Is this what you meant? If so, I'll do some more digging. Thanks, Miles ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] problem with _p_mtime
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miles wrote: > Hi, > > I've moved a FileStorage from one (old) machine to another (new) > machine, but when I mount it on the new machine I get a lot of time errrors: > > Traceback (innermost last): >Module ZPublisher.Publish, line 115, in publish >Module ZPublisher.mapply, line 88, in mapply >Module ZPublisher.Publish, line 41, in call_object >Module Shared.DC.Scripts.Bindings, line 311, in __call__ >Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec >Module App.special_dtml, line 176, in _exec >Module DocumentTemplate.DT_Let, line 76, in render >Module DocumentTemplate.DT_In, line 703, in renderwob >Module DocumentTemplate.DT_With, line 76, in render >Module DocumentTemplate.DT_Var, line 214, in render >Module App.PersistentExtra, line 43, in bobobase_modification_time >Module DateTime.DateTime, line 509, in __init__ >Module DateTime.DateTime, line 760, in _parse_args >Module DateTime.DateTime, line 437, in safelocaltime > TimeError: The time 98040302366.810165 is beyond the range of this > Python implementation. > > I googled around on this problem and didn't find much, but I did find > that as a temporary fix, I've put a try/except in App.PersistentExtra, > so if it generates an exception, it returns the current date and time. > This keeps things working. > > Out of interst, can anyone tell me why there is a difference? On both > machines, python was compiled from source with the same options, so I'm > not sure what this is down to. > > More importantly, what's the most straightforward way to set all the > dates to usable values? From my POV, it's fine to set them all to the > current date, but I don't know how to go about doing this exhaustively. > > Thanks for your help, For each machine: what version of Python and Zope? what OS? Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJN/BX+gerLs4ltQ4RAkcrAJ41ZnKq8GMFiSWLf9bk5/3DH855rgCgs7sX zVNGq01iU9Bo4pI+MQ/ATTI= =Mkwi -END PGP SIGNATURE- ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] Are there any IBlobStorage implementations outside of the ZODB implementation packages?
On Dec 4, 2008, at 1:27 AM, Christian Theune wrote: > On Wed, 2008-12-03 at 14:19 -0500, Jim Fulton wrote: >> I need to add a method to IBlobStorage. I don't think there are any >> IBlobStorage implementations outside of the ZODB project (other than >> zc.beforestorage). If this is the case, I don't need to add backward >> compatibility support. Please let me know if you know of an another >> IBlobStorage implementation. > > gocept.zeoraid is one. If you change this for 3.9, I'd be happy to > update our code without you doing BBB. Cool. Hopefully, this will appear on the trunk today. Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] problem with _p_mtime
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Miles Waller wrote: > Tres Seaver wrote: >> -BEGIN PGP SIGNED MESSAGE- >> Hash: SHA1 >> >> Miles wrote: >> >>> Hi, >>> >>> I've moved a FileStorage from one (old) machine to another (new) >>> machine, but when I mount it on the new machine I get a lot of time errrors: >>> >>> Traceback (innermost last): >>>Module ZPublisher.Publish, line 115, in publish >>>Module ZPublisher.mapply, line 88, in mapply >>>Module ZPublisher.Publish, line 41, in call_object >>>Module Shared.DC.Scripts.Bindings, line 311, in __call__ >>>Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec >>>Module App.special_dtml, line 176, in _exec >>>Module DocumentTemplate.DT_Let, line 76, in render >>>Module DocumentTemplate.DT_In, line 703, in renderwob >>>Module DocumentTemplate.DT_With, line 76, in render >>>Module DocumentTemplate.DT_Var, line 214, in render >>>Module App.PersistentExtra, line 43, in bobobase_modification_time >>>Module DateTime.DateTime, line 509, in __init__ >>>Module DateTime.DateTime, line 760, in _parse_args >>>Module DateTime.DateTime, line 437, in safelocaltime >>> TimeError: The time 98040302366.810165 is beyond the range of this >>> Python implementation. >>> >>> I googled around on this problem and didn't find much, but I did find >>> that as a temporary fix, I've put a try/except in App.PersistentExtra, >>> so if it generates an exception, it returns the current date and time. >>> This keeps things working. >>> >>> Out of interst, can anyone tell me why there is a difference? On both >>> machines, python was compiled from source with the same options, so I'm >>> not sure what this is down to. >>> >>> More importantly, what's the most straightforward way to set all the >>> dates to usable values? From my POV, it's fine to set them all to the >>> current date, but I don't know how to go about doing this exhaustively. >>> >>> Thanks for your help, >>> >> For each machine: what version of Python and Zope? what OS? >> > Old machine: Python 2.3.5, Zope 2.8.6, OS is Red Hat 8.0 (!) > > New machine: Python 2.4.3, Zope 2.9.8, OS is Ubuntu 7.10 > > I had discounted python/zope version differences as other databases had > upgraded fine. (CC'ing the newsgroup) Zope is defending against a change in Python's time module, which in turn is due to a change in the platform's time_t: $ bin/python2.3 Python 2.3.7 (#1, Aug 20 2008, 12:39:04) [GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> int(98040302366.81016) 98040302366L >>> from time import gmtime, localtime >>> gmtime(int(98040302366.81016)) (1901, 12, 13, 20, 45, 52, 4, 347, 0) >>> localtime(int(98040302366.81016)) (1901, 12, 13, 15, 45, 52, 4, 347, 0) $ bin/python2.4 Python 2.4.4 (#2, Nov 14 2007, 13:58:47) [GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2 Type "help", "copyright", "credits" or "license" for more information. >>> int(98040302366.81016) 98040302366L >>> from time import gmtime, localtime >>> gmtime(int(98040302366.81016)) Traceback (most recent call last): File "", line 1, in ? ValueError: timestamp out of range for platform time_t >>> localtime(int(98040302366.81016)) Traceback (most recent call last): File "", line 1, in ? ValueError: timestamp out of range for platform time_t As you can see, the Python 2.3 version has a wraparound problem. I have no idea how you got such an odd value stored for your '_p_mtime', unless you have been using Guido's time machine to run your appserver in 2039. ;) Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFJOAcs+gerLs4ltQ4RAooLAKCbzF9wBkwhtnhISmIGqIt22y7+rACfThhx IyjRAFhHuPE+LAVSj4Ct3Mk= =iq+a -END PGP SIGNATURE- ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] strange commit error
On Dec 4, 2008, at 8:46 AM, Miles wrote: > Hi, > > Thanks for your help > > > >>> >>> (from ZODB.serialize.py - line 350) >>> >>> if obj._p_jar._implicitlyAdding(oid): >>> raise InvalidObjectReference( >>> "A new object is reachable from multiple >>> databases. " >>> "Won't try to guess which one was correct!" >>> ) >>> >>> Can anyone shed any light on what this error means?! >> >> It means a new object is referenced from objects in multiple >> databases >> and it doesn't know what database to assign it to. > > After some digging into the objects, I think I understand this. It's > come up as follows: > > - I load object A from the old database > - Object A has an attribute B, which is an object also in the old > database > - I create a fresh copy of A, but leave the attribute B remained a > reference to B in the old database > - When I went to commit A, I got an error because A.B points to the > old database still > > Is this right? No. The error is about a *new* object that is refered to from both databases. > > >> >>> Given a persistent >>> object, what is the correct way to get a completely fresh copy >>> that is >>> not tied to any database at all, so I can transfer these old objects >>> to >>> the current database. >> >> I would use the database export/import API to export it from one >> database and import it to another. (There are issues with doing this >> in Zope 3, but I assume you're using Zope 2. > > Sorry if I'm being thick - I thought the code below did indeed use the > export/import API: > > f=tempfile.TemporaryFile() > old_ob._p_jar.exportFile(old_ob._p_oid,f) > f.seek(0) > container = app.new_folder > ob=container._p_jar.importFile(f) > > Is this what you meant? If so, I'll do some more digging. Oops, I didn't notice that you were already doing this. This is odd. I'm at a loss. - Does your application involve multiple databases, aside from the databases you are copying data between? - How complex is the object in question? Jim -- Jim Fulton Zope Corporation ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] problem with _p_mtime
Miles wrote at 2008-12-4 13:39 +: >I've moved a FileStorage from one (old) machine to another (new) >machine, but when I mount it on the new machine I get a lot of time errrors: > >Traceback (innermost last): > Module ZPublisher.Publish, line 115, in publish > Module ZPublisher.mapply, line 88, in mapply > Module ZPublisher.Publish, line 41, in call_object > Module Shared.DC.Scripts.Bindings, line 311, in __call__ > Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec > Module App.special_dtml, line 176, in _exec > Module DocumentTemplate.DT_Let, line 76, in render > Module DocumentTemplate.DT_In, line 703, in renderwob > Module DocumentTemplate.DT_With, line 76, in render > Module DocumentTemplate.DT_Var, line 214, in render > Module App.PersistentExtra, line 43, in bobobase_modification_time > Module DateTime.DateTime, line 509, in __init__ > Module DateTime.DateTime, line 760, in _parse_args > Module DateTime.DateTime, line 437, in safelocaltime >TimeError: The time 98040302366.810165 is beyond the range of this >Python implementation. I expect that your storage is damanged and contains some garbage at a place where a serial should be. This could probably be fixed -- but other parts of your storage might be damanged, too. -- Dieter ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] problem with _p_mtime
We run consistency checks on the ZODB - the same as we do on SVN -- nightly. http://svn.zope.org/ZODB/trunk/src/ZODB/scripts - fstest.py - checkbtrees.py - fsrefs.py We have never had corruption in years. But I just wanted to ask what people are doing to find any problems with consistency; so the problem can be solved ASAP. Miles, have you run any of these scripts across your production ZODB? cheers alan On Thu, Dec 4, 2008 at 12:45 PM, Dieter Maurer <[EMAIL PROTECTED]> wrote: > Miles wrote at 2008-12-4 13:39 +: >>I've moved a FileStorage from one (old) machine to another (new) >>machine, but when I mount it on the new machine I get a lot of time errrors: >> >>Traceback (innermost last): >> Module ZPublisher.Publish, line 115, in publish >> Module ZPublisher.mapply, line 88, in mapply >> Module ZPublisher.Publish, line 41, in call_object >> Module Shared.DC.Scripts.Bindings, line 311, in __call__ >> Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec >> Module App.special_dtml, line 176, in _exec >> Module DocumentTemplate.DT_Let, line 76, in render >> Module DocumentTemplate.DT_In, line 703, in renderwob >> Module DocumentTemplate.DT_With, line 76, in render >> Module DocumentTemplate.DT_Var, line 214, in render >> Module App.PersistentExtra, line 43, in bobobase_modification_time >> Module DateTime.DateTime, line 509, in __init__ >> Module DateTime.DateTime, line 760, in _parse_args >> Module DateTime.DateTime, line 437, in safelocaltime >>TimeError: The time 98040302366.810165 is beyond the range of this >>Python implementation. > > I expect that your storage is damanged and contains some garbage > at a place where a serial should be. > > This could probably be fixed -- but other parts of your storage > might be damanged, too. > > > > -- > Dieter > ___ > For more information about ZODB, see the ZODB Wiki: > http://www.zope.org/Wikis/ZODB/ > > ZODB-Dev mailing list - ZODB-Dev@zope.org > http://mail.zope.org/mailman/listinfo/zodb-dev > -- Alan Runyan Enfold Systems, Inc. http://www.enfoldsystems.com/ phone: +1.713.942.2377x111 fax: +1.832.201.8856 ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
Re: [ZODB-Dev] problem with _p_mtime
Alan Runyan wrote: > We run consistency checks on the ZODB - the same as we do on SVN -- nightly. > > http://svn.zope.org/ZODB/trunk/src/ZODB/scripts > - fstest.py > - checkbtrees.py > - fsrefs.py When reading documentation of fsrefs.py, I see it is not recommended to run it again a live FileStorage. Do you stop your servers nightly to do it ? > > We have never had corruption in years. > But I just wanted to ask what people are doing to find any problems > with consistency; so the problem can be solved ASAP. > > Miles, have you run any of these scripts across your production ZODB? > > cheers > alan -- Godefroid Chapelle (aka __gotcha) http://bubblenet.be ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
