[Zope-dev] Re: Catalog performance

[Seb Bacon]

Nguyen Quan Son wrote:
Hi,
I have a problem with performance and memory consumption when trying to do some 
statistics, using following code:
...
docs = container.portal_catalog(meta_type='Document', ...)
for doc in docs:
    obj = doc.getObject()
    value = obj.attr
    ...
With about 10.000 documents this Python script takes 10 minutes and more than 500MB of 
memory, after that I had to restart Zope. I
am running Zope 2.6.1 + Plone 1.0 on Windows 2000, Xeon P4 with 1GB RAM.
What's wrong with this code? Any suggestion is appreciated.
With getObject(), you're loading entire objects into memory in order to 
grab a single attribute.  This is very wasteful.  Try putting the 
attribute into the metadata for the catalog and grabbing it from there. 
 Then you can do:

 for doc in docs:
     value = doc.attr
seb



_______________________________________________
Zope-Dev maillist  -  [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
**  No cross posts or HTML encoding!  **
(Related lists - 
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )