Re: [Zope-dev] Re: any limits on object number?
On Wed, 2004-07-14 at 18:55, Casey Duncan wrote: On Wed, 14 Jul 2004 13:46:57 +0530 Nagarjuna G. [EMAIL PROTECTED] wrote: We are builiding a large portal using Zope. We need to create a large number of objects. The data component of the objects is small, but each object carries lots of metadata. My question is: Is there any limit on the number of objects in a given folder? I am not planning to use any external RDBMS. Are there any known performace isssues when the numbeer of objects increase, particularly when we store them in the same folder? Normal zope folders should probably not be used to hold more than a few dozen items. They store a list of their children in a single ZODB record, and as their numbers increase so do the size of the transactions that change the folder. Also normal Zope folders do not handle concurrent updates and will thus not perform well when multiple users are adding items to the folder. Shane Hathaway's BTreeFolder2 product solves these problems. It is the thing to use when you want to store large numbers of objects in a single folder. It also handles concurrency much better. hth, thanks to all the responses. I will test with BTreeFolder2, and subfoldering with first character of id as another subfolder2, so that working with ZMI will also be possible without further scripting or customized views. i will report the results back to the list. Nagarjuna ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: RewriteMaps inefficient
Ganesha Shanmuganathan wrote: (..)Is there anyother way to do load balncing with Zope without Rewrite ? Not sure, but would this help? http://www.backhand.org/mod_backhand/ regards jw -- Jan-Wijbrand Kolman [EMAIL PROTECTED] ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Granularity of Persistent Objects
Greetings. I'm developing a Zope (2.7) product, and I've got a particular persistent object that's going to be a real bottleneck, in that there will be many threads trying to change the state of the object more or less simultaneously. I'd like to spread out the problem by taking member fields of the object (lists, dictionaries, etc.) and making them persistent sub-objects in their own right. That way, changes made to one of them can be saved to the ZODB without causing thread conflicts with changes being made to others, removing at least some of the thred conflicts. However, I don't know how to do this. Is it as simple as using a PersistentMapping where I'd normally use a standard Python dictionary? Or shall I wrap each one in a custom class that subclasses Persistent? Does my parent object have to handle these in any special way so that the ZODB knows to give them their own DB records? I've tried looking at how ObjectManager does things, and found it less than enlightening. Thanks, ..Ian Beatty University of Massachusetts Amherst ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] Granularity of Persistent Objects
On Thu, Jul 15, 2004 at 02:54:54PM -0400, Ian Beatty wrote: Greetings. I'm developing a Zope (2.7) product, and I've got a particular persistent object that's going to be a real bottleneck, in that there will be many threads trying to change the state of the object more or less simultaneously. I'd like to spread out the problem by taking member fields of the object (lists, dictionaries, etc.) and making them persistent sub-objects in their own right. That way, changes made to one of them can be saved to the ZODB without causing thread conflicts with changes being made to others, removing at least some of the thred conflicts. sounds reasonable... However, I don't know how to do this. Is it as simple as using a PersistentMapping where I'd normally use a standard Python dictionary? Instead of PersistentMapping, see the various flavors of BTree, e.g. OOBTree, IOBTree. These do their own conflict resolution and in your scenario that could be a big help. -- Paul Winkler http://www.slinkp.com ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: RewriteMaps inefficient (was: [Zope-dev] Apache Mod_rewrite question)
Ganesha Shanmuganathan wrote at 2004-7-14 17:26 -0500: ... Now, we use RewriteMaps only when the map files are very small (not more than some dozen entries). That is interesting to know. Right now, we use multiple map files but each is small. Is there anyother way to do load balncing with Zope without Rewrite ? We use a load balancer (a piece of hardware) for this. Alternatively, pond (or similar) is a software load balancer implemented in Python. ... maps with the same name ... The problem is the cache treats both the file as the same since they have the same name! Hence if you visit foo.com first, the data stored in foomap.txt is returned from the cache next time you visit foo.com or bar.com. This causes apache to redirect into the wrong zope site. We did not observe this behaviour... btw, Did you use the same map name (like, in different sections maybe) for the different map files. Yes. The same map file name mapping to different files in different virtual hosts. -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [ZODB-Dev] Getting rid of ZODB versions (was Re: [ZWeb] Are Versions evil?)
Jim Fulton wrote at 2004-7-15 10:00 -0400: ... Given this, I'd like to formally announce the intention to deprecate versions in both Zope and ZODB. We will formally deprecate versions when we have a way to maintain long-running transactions in Zope. I have a good idea how they would work and I'd be happy to advise volunteers on how to proceed. Any volunteers? Any objections? Versions are nothing else than long running transactions (which the additional but not essential feature to be able to join such a transaction). Locking might be necessary for long running transactions as the longer the transaction the higher becomes the risk of conflicts and the more costly a redo of the transaction. When we get long running transactions I will be happy to let Versions alone. -- Dieter ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
[Zope-dev] Re: [ZODB-Dev] Getting rid of ZODB versions (was Re: [ZWeb] Are Versions evil?)
On Thu, Jul 15, 2004 at 05:18:37PM -0400, Jim Fulton wrote: I don't think anyone find versions useful in cases where locks matter. With versions, if you get resource contention, someone loses. Long-running subtransactions as I'm proposing would rely on mult-version concurrency control, rather than locks. In either case, if you have resource contention, you are probably out of luck. Locks are important, however, in cases such as generating unique IDs based on integer attribtues in a database. This is something we never managed to sort out in IndexedCatalog, and I wonder if anyone else ever has. Take care, -- Christian Robottom Reis | http://async.com.br/~kiko/ | [+55 16] 3361 2331 ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
