Re: [ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Benji York
On Mon, Sep 14, 2009 at 3:52 PM, Shane Hathaway wrote: > Benji York wrote: >> >> On Mon, Sep 14, 2009 at 2:44 PM, Shane Hathaway >> wrote: >>> >>> People have been looking at these performance charts for quite a while: >> >> Are the benchmarks publicly available?  I looked through the blog posts

Re: [ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Shane Hathaway
Benji York wrote: > On Mon, Sep 14, 2009 at 2:44 PM, Shane Hathaway wrote: >> People have been looking at these performance charts for quite a while: > > Are the benchmarks publicly available? I looked through the blog posts > and svn.zope.org but couldn't find them. http://svn.zope.org/relstor

Re: [ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Benji York
On Mon, Sep 14, 2009 at 2:44 PM, Shane Hathaway wrote: > People have been looking at these performance charts for quite a while: Are the benchmarks publicly available? I looked through the blog posts and svn.zope.org but couldn't find them. -- Benji York Senior Software Engineer Zope Corporatio

Re: [ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Shane Hathaway
Chris Withers wrote: > Shane Hathaway wrote: >> Chris Withers wrote: >>> I thought RelStorage was slow at packing? >> Not anymore. I optimized packing last December/January and it has been >> plenty fast ever since, AFAIK. > > Interesting. Got any metrics on its speed compared to FileStorage, >

Re: [ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Chris Withers
Shane Hathaway wrote: > Chris Withers wrote: >> I thought RelStorage was slow at packing? > > Not anymore. I optimized packing last December/January and it has been > plenty fast ever since, AFAIK. Interesting. Got any metrics on its speed compared to FileStorage, including packing? cheers,

Re: [ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Shane Hathaway
Chris Withers wrote: > I thought RelStorage was slow at packing? Not anymore. I optimized packing last December/January and it has been plenty fast ever since, AFAIK. Shane ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/W

Re: [ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Chris Withers
Shane Hathaway wrote: >> There is a .old file there, but it was last modified over a week ago. > > The analysis phase of packing can take a long time. A week is too much, > though. :-) Yeah, this is off the back of an automated pack/backup/unpack/verify to keep two storages in sync on two data

Re: [ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Shane Hathaway
Chris Withers wrote: > I've got a ZODB that's ballooned and I'm now having problems packing > with zeopack. > > I tried with zeopack, which appeared to just hang (still in top, but no > cpu usage for an hour or two), so I ctrl-c'ed it. > > Now when I try, I just get the text "Already packing" b

Re: [ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Jim Fulton
On Mon, Sep 14, 2009 at 10:43 AM, David Glick wrote: > On Sep 14, 2009, at 7:04 AM, Jim Fulton wrote: >> >> On Mon, Sep 14, 2009 at 7:10 AM, Chris Withers >> wrote: >>> >>> Jim Fulton wrote: ZODB doesn't provide any direct support because it has no notion of tree. >>> >>> Does it h

Re: [ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread David Glick
On Sep 14, 2009, at 7:04 AM, Jim Fulton wrote: > On Mon, Sep 14, 2009 at 7:10 AM, Chris Withers > wrote: >> Jim Fulton wrote: >>> >>> ZODB doesn't provide any direct support because it has no notion >>> of tree. >> >> Does it have any notion of object size on disk for a given object? > > Not d

[ZODB-Dev] Problems packing Zope 2.9.8 storage

2009-09-14 Thread Chris Withers
Hi All, I've got a ZODB that's ballooned and I'm now having problems packing with zeopack. I tried with zeopack, which appeared to just hang (still in top, but no cpu usage for an hour or two), so I ctrl-c'ed it. Now when I try, I just get the text "Already packing" back. There is a .old file

Re: [ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Jim Fulton
On Mon, Sep 14, 2009 at 7:10 AM, Chris Withers wrote: > Jim Fulton wrote: >> >> ZODB doesn't provide any direct support because it has no notion of tree. > > Does it have any notion of object size on disk for a given object? Not directly, but you can do: len(ob._p_jar.db().storage.load(ob._p_o

Re: [ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Jens Vagelpohl
On Sep 14, 2009, at 13:27 , Chris Withers wrote: Jens Vagelpohl wrote: Sadly, when you're trying to figure out which folder is using all the space on a disk that's running out of space, that's not practical... How about simply writing a script that crawls the ZODB and gives you paths and s

Re: [ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Andreas Jung
On 14.09.09 13:27, Chris Withers wrote: > Jens Vagelpohl wrote: > >> >>> Sadly, when you're trying to figure out which folder is using all the >>> space on a disk that's running out of space, that's not practical... >>> >> How about simply writing a script that crawls the ZODB and gi

Re: [ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Chris Withers
Jens Vagelpohl wrote: > >> Sadly, when you're trying to figure out which folder is using all the >> space on a disk that's running out of space, that's not practical... > > How about simply writing a script that crawls the ZODB and gives you > paths and sizes? Indeed, and what's the magic incan

Re: [ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Jens Vagelpohl
On Sep 14, 2009, at 13:10 , Chris Withers wrote: Jim Fulton wrote: ZODB doesn't provide any direct support because it has no notion of tree. Does it have any notion of object size on disk for a given object? In Zope 2, you could export the folder and look at the size of the export. Sad

Re: [ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Chris Withers
Jim Fulton wrote: > ZODB doesn't provide any direct support because it has no notion of tree. Does it have any notion of object size on disk for a given object? > In Zope 2, you could export the folder and look at the size of the export. Sadly, when you're trying to figure out which folder is us

Re: [ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Jim Fulton
On Mon, Sep 14, 2009 at 5:55 AM, Chris Withers wrote: > Hi All, > > As the subject line says with the following definitions: > > - size = size on disk, in Mb > > - subtee = Zope folderish type things, in this case... ZODB doesn't provide any direct support because it has no notion of tree. In Zo

Re: [ZODB-Dev] CRC check failed when running repozo

2009-09-14 Thread Chris Withers
Adam GROSZER wrote: > Hello, > > +1 on that. Sometimes copy operations (over the network) screw up some > bits. Happens rarely but then it hits hard. > > Monday, September 14, 2009, 10:13:39 AM, you wrote: > > CT> Hmm. Haven't seen that. My guess would be some low-level data corruption. > > CT>

[ZODB-Dev] How do I find out the size of a subtree of the object graph?

2009-09-14 Thread Chris Withers
Hi All, As the subject line says with the following definitions: - size = size on disk, in Mb - subtee = Zope folderish type things, in this case... cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk __

Re: [ZODB-Dev] CRC check failed when running repozo

2009-09-14 Thread Adam GROSZER
Hello, +1 on that. Sometimes copy operations (over the network) screw up some bits. Happens rarely but then it hits hard. Monday, September 14, 2009, 10:13:39 AM, you wrote: CT> Hmm. Haven't seen that. My guess would be some low-level data corruption. CT> Christian -- Best regards, Adam GR

Re: [ZODB-Dev] CRC check failed when running repozo

2009-09-14 Thread Pedro Ferreira
I've sent a mail to this list on 05/26/09 reporting a similar issue. Regards, Pedro Christian Theune wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 09/14/2009 10:06 AM, Chris Withers wrote: > >> Hi All, >> >> Anyone ever see this? >> >> Traceback (most recent call last): >>

Re: [ZODB-Dev] CRC check failed when running repozo

2009-09-14 Thread Christian Theune
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/14/2009 10:06 AM, Chris Withers wrote: > Hi All, > > Anyone ever see this? > > Traceback (most recent call last): >File "/opt/Zope-2.9//bin/repozo.py", line 517, in ? > main() >File "/opt/Zope-2.9//bin/repozo.py", line 510, in main

[ZODB-Dev] CRC check failed when running repozo

2009-09-14 Thread Chris Withers
Hi All, Anyone ever see this? Traceback (most recent call last): File "/opt/Zope-2.9//bin/repozo.py", line 517, in ? main() File "/opt/Zope-2.9//bin/repozo.py", line 510, in main do_backup(options) File "/opt/Zope-2.9//bin/repozo.py", line 450, in do_backup reposz, reposum