Re: [Zope] Fwd: Building a fast, scalable yet small Zope application

2009-04-27 Thread Garito
2009/4/27 Tres Seaver > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Garito wrote: > > jajajajajajajajajajajajaja > > > > Amazing! > > > > I don't know how the other list doesn't collapse using reply-to field > > jajajajajajajajajajajajaja > > I said "religious" because I meant that people

Re: [Zope] Building a fast, scalable yet small Zope application

2009-04-27 Thread Morten W. Petersen
Hedley Roos skrev: > I've followed this thread with interest since I have a Zope site with > tens of millions of entries in BTrees. It scales well, but it requires > many tricks to make it work. > > Roche Compaan wrote these great pieces on ZODB, Data.fs size and > scalability at > http://www.upfr

Re: [Zope] relstorage on zope 2.10.5

2009-04-27 Thread Shane Hathaway
Jürgen Herrmann wrote: > the version of zopdb shipped with zope 2.10.5 is 3.7.1, so why > does the command above try to pull an alpha release of zodb? > can in "pin" the version of zodb to 3.7.1? Zope includes ZODB. You need a buildout that does not try to replace the version of ZODB that comes

Re: [Zope] Building a fast, scalable yet small Zope application

2009-04-27 Thread Peter Bengtsson
For huge inserts like that, have you looked at the more modern alternatives such as Tokyo Cabinet or MongoDB? I heard about an experiment to transfer 20 million text blobs into a Tokyo Cabinet. The first 10 million inserts were superfast but after that it started to take up to a second to insert ea

Re: [Zope] Building a fast, scalable yet small Zope application

2009-04-27 Thread Lennart Regebro
On Mon, Apr 27, 2009 at 17:57, Morten W. Petersen wrote: > OK.  Well, I'm concerned about how much a database would grow.  I'm thinking > if > I use one BTree for all the entries, would the database grow just a little > or a lot when > you start getting into the millions of entries when inserting

Re: [Zope] Building a fast, scalable yet small Zope application

2009-04-27 Thread Hedley Roos
I've followed this thread with interest since I have a Zope site with tens of millions of entries in BTrees. It scales well, but it requires many tricks to make it work. Roche Compaan wrote these great pieces on ZODB, Data.fs size and scalability at http://www.upfrontsystems.co.za/Members/roche/w

Re: [Zope] Building a fast, scalable yet small Zope application

2009-04-27 Thread Morten W. Petersen
Peter Bengtsson skrev: > From experience I find that BTrees are very fast to write to and pick > out items from. Even in the millions. (Never gone into the tens of > millions or further) > Also, when it comes to browsing stuff I find SQL faster and easier to > work with. An added advantage of a RDB

Re: [Zope] Building a fast, scalable yet small Zope application

2009-04-27 Thread Morten W. Petersen
Lennart Regebro skrev: > On Sat, Apr 25, 2009 at 13:24, Morten W. Petersen > wrote: > >> So far, I've been contemplating disabling undo (if that's possible), >> > > I doubt that it would make a difference. The Undo functionality comes > out of the database being logging, and changing that

Re: [Zope] Building a fast, scalable yet small Zope application

2009-04-27 Thread Morten W. Petersen
> I suggest you experiment a bit. Create 100 million objects, and do > some of the actions you are planning to do on them. > Right. I'm thinking of taking the time to try a simple SQL based implementation, as well as one in ZODB. I need to learn more about high-speed Zope programming as we

Re: [Zope] Fwd: Building a fast, scalable yet small Zope application

2009-04-27 Thread Tres Seaver
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Garito wrote: > jajajajajajajajajajajajaja > > Amazing! > > I don't know how the other list doesn't collapse using reply-to field > jajajajajajajajajajajajaja I said "religious" because I meant that people have strong opinions about which style they

Re: [Zope] Request time grows with memory size

2009-04-27 Thread Hedley Roos
On Mon, Apr 27, 2009 at 12:40 PM, Peter Bengtsson wrote: > What have you done to investigate memory leaks? > What external connectors are you using, like MySQL or LDAP? > It is probably not a memory leak. The graph is what I'd expect in a garbage collection scenario (ie. Python). Hedley

Re: [Zope] Building a fast, scalable yet small Zope application

2009-04-27 Thread Peter Bengtsson
>From experience I find that BTrees are very fast to write to and pick out items from. Even in the millions. (Never gone into the tens of millions or further) Also, when it comes to browsing stuff I find SQL faster and easier to work with. An added advantage of a RDBMS is that you get the indexing

Re: [Zope] Request time grows with memory size

2009-04-27 Thread Peter Bengtsson
What have you done to investigate memory leaks? What external connectors are you using, like MySQL or LDAP? 2009/4/27 Gerhard Schmidt : > HI, > > I've encounters a performance Problem with Zope. Some requests take very > long time to process while others are served very fast. All request go for >

[Zope] relstorage on zope 2.10.5

2009-04-27 Thread Jürgen Herrmann
hi there! today i thought i'ds give relstorage a try, esp. for it's small startup times compared to filestorage. i followed the instructions at http://pypi.python.org/pypi/RelStorage#installation an when calling "python2.4 setup.py install --install-lib=${INSTANCE_HOME}/lib/python" (instance hom