Re: [ZODB-Dev] PGStorage

2008-01-24 Thread Jim Fulton
On Jan 24, 2008, at 11:59 AM, Shane Hathaway wrote: Instead, I've been comparing ZEO+FileStorage with RelStorage +PostgreSQL. Makes sense, although I'm not sure how much. :) In that light, my most recent graphs suggest that RelStorage wins especially as you scale up. YMMV of course! Ben

Re: [ZODB-Dev] PGStorage

2008-01-24 Thread Dieter Maurer
Zvezdan Petkovic wrote at 2008-1-23 17:15 -0500: >On Jan 23, 2008, at 4:05 PM, Flavio Coelho wrote: >> sorry, I never meant to email you personally I have been wrong: Flavio has not forgotten the list, I had not looked carefully enough. Sorry! -- Dieter

Re: [ZODB-Dev] PGStorage

2008-01-24 Thread Shane Hathaway
Andreas Jung wrote: Excellent. I re-tried my benchmarks of course. Copy&Paste of a complete new Plone site is now nearly as fast as with Filestorage (just by looking on my watch). The tests for commiting 100 objects are also blazing fast: roughly 1.2 - 1.5 seconds for committing. That's roughly

Re: [ZODB-Dev] PGStorage

2008-01-24 Thread Andreas Jung
--On 24. Januar 2008 11:21:28 -0400 David Pratt <[EMAIL PROTECTED]> wrote: Hi Andreas. No need to rely on rumors. The current source is available here: http://pgstorage.cvs.sourceforge.net/pgstorage/ Excellent. I re-tried my benchmarks of course. Copy&Paste of a complete new Plone site

Re: [ZODB-Dev] PGStorage

2008-01-24 Thread David Pratt
Hi Andreas. No need to rely on rumors. The current source is available here: http://pgstorage.cvs.sourceforge.net/pgstorage/ Regards, David Andreas Jung wrote: --On 24. Januar 2008 15:42:01 +0100 Andreas Jung <[EMAIL PROTECTED]> wrote: --On 23. Januar 2008 18:17:18 +0100 Andreas Jung <

Re: [ZODB-Dev] PGStorage

2008-01-24 Thread Andreas Jung
--On 24. Januar 2008 15:42:01 +0100 Andreas Jung <[EMAIL PROTECTED]> wrote: --On 23. Januar 2008 18:17:18 +0100 Andreas Jung <[EMAIL PROTECTED]> wrote: --On 22. Januar 2008 21:17:45 -0500 Stephan Richter <[EMAIL PROTECTED]> wrote: On Tuesday 22 January 2008, Dieter Maurer wrote: "Orac

Re: [ZODB-Dev] PGStorage

2008-01-24 Thread Andreas Jung
--On 23. Januar 2008 18:17:18 +0100 Andreas Jung <[EMAIL PROTECTED]> wrote: --On 22. Januar 2008 21:17:45 -0500 Stephan Richter <[EMAIL PROTECTED]> wrote: On Tuesday 22 January 2008, Dieter Maurer wrote: "OracleStorage" was abandoned because it was almost an order or magnitude slower than

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Dieter Maurer
Alan Runyan wrote at 2008-1-23 13:32 -0600: > ... >each record in a catalog may have an object for each index/metadata attribute >you are capturing. and possibly a few others. Each catalog entries contain >ts of object per object being indexed.That is my understanding. It does not completely

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Dieter Maurer
Alan Runyan wrote at 2008-1-23 11:24 -0600: > ... >My understanding is >the catalog is what makes storages a misery. I do not think that this is true. The catalog only often contains lots of objects -- and maybe some of them with not so good persistency design. >CMF/portal_catalog mounted as Fil

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Zvezdan Petkovic
On Jan 23, 2008, at 4:05 PM, Flavio Coelho wrote: BTW: stay on the list. I do not like personal emails. sorry, I never meant to email you personally, this due to the default configuration of this list, that sets the to the poster instead of to the list...So I may have been too quick to the

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Flavio Coelho
On Jan 23, 2008 6:45 PM, Dieter Maurer <[EMAIL PROTECTED]> wrote: > Flavio Coelho wrote at 2008-1-22 17:43 -0200: > > ... > >Actually what I am trying to run away from is the "packing monster" ;-) > > Jim has optimized pack consideraly (--> "zc.FileStorage"). > > I, too, have worked on pack optimi

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Dieter Maurer
Flavio Coelho wrote at 2008-1-22 17:43 -0200: > ... >Actually what I am trying to run away from is the "packing monster" ;-) Jim has optimized pack consideraly (--> "zc.FileStorage"). I, too, have worked on pack optimization the last few days (we cannot yet use Jims work because we are using ZODB

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Andreas Jung
--On 23. Januar 2008 13:32:29 -0600 Alan Runyan <[EMAIL PROTECTED]> wrote: Likely but I don#t know how to setup my instance in order to make the copied instance making use of mounted catalog. What about creating a Plone site with a FileStorage. Then mount a subfolder into PGStorage say /Plon

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Alan Runyan
> Likely but I don#t know how to setup my instance in order to make the > copied instance making use of mounted catalog. What about creating a Plone site with a FileStorage. Then mount a subfolder into PGStorage say /Plone/some_folder then you could see create af older /Plone/new_folder and dump

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Andreas Jung
--On 23. Januar 2008 11:24:28 -0600 Alan Runyan <[EMAIL PROTECTED]> wrote: Andreas, Could you try to mount the catalog separately? My understanding is the catalog is what makes storages a misery. Likely but I don#t know how to setup my instance in order to make the copied instance making

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread David Pratt
Graphical test comparison (open office format) http://pgstorage.cvs.sourceforge.net/pgstorage/PGStorage/tests/comparison.ods Regards, David Alan Runyan wrote: Andreas, Could you try to mount the catalog separately? My understanding is the catalog is what makes storages a misery. CMF/portal_c

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Alan Runyan
Andreas, Could you try to mount the catalog separately? My understanding is the catalog is what makes storages a misery. CMF/portal_catalog mounted as Filestorage and CMF could be mounted as PGStorage. I presume you would see much more reasonable performance? cheers -- Alan Runyan Enfold Sys

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Andreas Jung
--On 22. Januar 2008 21:17:45 -0500 Stephan Richter <[EMAIL PROTECTED]> wrote: On Tuesday 22 January 2008, Dieter Maurer wrote: "OracleStorage" was abandoned because it was almost an order or magnitude slower than "FileStorage". Actually, Lovely Systems uses PGStorage because it is faster

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread David Pratt
Cool. Jim Fulton wrote: Berkeley DB storage didnt' work out the first time around. My experience optimizing packing for FileStorage reminded me how much I want an alternative to FileStorage for large active databases. I intend to revisit Berkeley DB storage one of these days. __

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Jim Fulton
On Jan 23, 2008, at 9:59 AM, Benji York wrote: Flavio Coelho wrote: Actually what I am trying to run away from is the "packing monster" ;-) Jim has done a great deal of work on packing (that will go into 3.9 I presume) and is available now in zc.FileStorage. that should make your pack

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Jim Fulton
On Jan 23, 2008, at 9:44 AM, Alan Runyan wrote: Jim, What would you consider a "large active database"? I don't have specific metrics. Generally databases who's daily growth is measured in Gigabytes and who's size is significantly reduced by packing is what I have in mind. Jim -- Ji

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Benji York
Flavio Coelho wrote: Actually what I am trying to run away from is the "packing monster" ;-) Jim has done a great deal of work on packing (that will go into 3.9 I presume) that should make your pack 3 to 6 times faster (depending on if you do garbage collection at pack time or not). I want

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Alan Runyan
Jim, What would you consider a "large active database"? curiously, alan ___ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Jim Fulton
On Jan 22, 2008, at 8:44 PM, Marius Gedminas wrote: On Tue, Jan 22, 2008 at 05:43:42PM -0200, Flavio Coelho wrote: Actually what I am trying to run away from is the "packing monster" ;-) I want to be able to use an OO database without the inconvenience of having it growing out of control

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Flavio Coelho
Thats a good thing to know, I will do some testing of my own... Thanks Flávio On Jan 23, 2008 2:36 AM, David Pratt <[EMAIL PROTECTED]> wrote: > Yes, Shane had done some benchmarking about a year or so ago. PGStorage > was actually faster with small writes but slower for larger ones. As far > as

Re: [ZODB-Dev] PGStorage

2008-01-23 Thread Flavio Coelho
Then perhaps I should rephrase my question: Is there a way, by means of configuration, to reduce the size of history to reduce the cost of packing? thanks, Flávio On Jan 22, 2008 11:44 PM, Marius Gedminas <[EMAIL PROTECTED]> wrote: > On Tue, Jan 22, 2008 at 05:43:42PM -0200, Flavio Coelho wrot

Re: [ZODB-Dev] PGStorage

2008-01-22 Thread Stephan Richter
On Tuesday 22 January 2008, David Pratt wrote: > BTW Stephan, where is Lovely using > it - a site example? I had read some time ago that they were exploring > it but not that it was being used. I think they are using it for video.vol.at, but you should contact them making sure they actually use

Re: [ZODB-Dev] PGStorage

2008-01-22 Thread David Pratt
Yes, Shane had done some benchmarking about a year or so ago. PGStorage was actually faster with small writes but slower for larger ones. As far as packing, as a zodb implementation, packing is still required to reduce the size of data in Postgres. BTW Stephan, where is Lovely using it - a site

Re: [ZODB-Dev] PGStorage

2008-01-22 Thread Stephan Richter
On Tuesday 22 January 2008, Dieter Maurer wrote: > "OracleStorage" was abandoned because it was almost an order > or magnitude slower than "FileStorage". Actually, Lovely Systems uses PGStorage because it is faster for them. Regards, Stephan -- Stephan Richter Web Software Design, Development an

Re: [ZODB-Dev] PGStorage

2008-01-22 Thread Marius Gedminas
On Tue, Jan 22, 2008 at 05:43:42PM -0200, Flavio Coelho wrote: > Actually what I am trying to run away from is the "packing monster" ;-) > > I want to be able to use an OO database without the inconvenience of having > it growing out of control and then having to spend hours packing the > database

Re: [ZODB-Dev] PGStorage

2008-01-22 Thread Flavio Coelho
Thanks Dieter, Actually what I am trying to run away from is the "packing monster" ;-) I want to be able to use an OO database without the inconvenience of having it growing out of control and then having to spend hours packing the database every once in a while. (I do a lot of writes in my DBs).

Re: [ZODB-Dev] PGStorage

2008-01-22 Thread Dieter Maurer
Flavio Coelho wrote at 2008-1-22 10:57 -0200: > ... >Can anyone tell me if PGstorage is stable enough for production use? I expect that it will behave similar to "OracleStorage". "OracleStorage" was abandoned because it was almost an order or magnitude slower than "FileStorage". Carefully think

[ZODB-Dev] PGStorage

2008-01-22 Thread Flavio Coelho
Hi, Can anyone tell me if PGstorage is stable enough for production use? thanks, -- Flávio Codeço Coelho "My grandfather once told me that there were two kinds of people: those who do the work and those who take the credit. He tol