Re: [Zope-dev] catalog performance: query plan

2008-11-10 Thread Roché Compaan
ed about indexes beyond this point - they're all returning results in less than a millisecond. > Can you provide information on the corpus / configuration / test plan > you used to generate these results? It's basically a Plone site with 300,000 remember based users and roughly 150,0

Re: [Zope-dev] catalog performance: query plan

2008-11-10 Thread Roché Compaan
e_or_created. I have previously done benchmarks with query plan and it didn't make any noticeable difference. What might be true or is becoming more likely is that indexes are used where they don't fit the use case rather than that the indexes themselves need optimisation. -- Roché Compa

Re: [Zope-dev] catalog performance: query plan

2008-11-09 Thread Roché Compaan
GroupId |ZCTextIndex |0.105|94962.39 getUserId|ZCTextIndex |0.105|95165.88 getFullname |ZCTextIndex |0.104|95313.06 getRoles |FieldIndex |0.104|95385.31 getUserName |ZCTextIndex |0.103|

Re: [Zope-dev] ZCatalog caching with memcached

2008-10-27 Thread Roché Compaan
On Mon, 2008-10-27 at 13:41 +0100, Jens Vagelpohl wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > > On Oct 27, 2008, at 13:32 , Roché Compaan wrote: > > > On Mon, 2008-10-27 at 13:23 +0100, Jens Vagelpohl wrote: > >> When it comes to integrating a

Re: [Zope-dev] ZCatalog caching with memcached

2008-10-27 Thread Roché Compaan
h have a cache of 20 objects. Our queries are CPU bound, not memory or IO bound. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-d

Re: [Zope-dev] ZCatalog caching with memcached

2008-10-27 Thread Roché Compaan
On Mon, 2008-10-27 at 13:23 +0100, Jens Vagelpohl wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > > On Oct 27, 2008, at 13:08 , Roché Compaan wrote: > > > On Sun, 2008-10-26 at 14:07 -0400, Tres Seaver wrote: > >> - Plone uses too many indexes, and

Re: [Zope-dev] ZCatalog caching with memcached

2008-10-27 Thread Roché Compaan
On Sun, 2008-10-26 at 14:07 -0400, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Roché Compaan wrote: > > On Sat, 2008-10-25 at 09:20 +0200, Hedley Roos wrote: > >>> Have you measures the time needs for some "standard" ZCatalog q

Re: [Zope-dev] ZCatalog caching with memcached

2008-10-26 Thread Roché Compaan
On Sun, 2008-10-26 at 18:50 +0100, Andreas Jung wrote: > On 26.10.2008 18:43 Uhr, Roché Compaan wrote: > > On Sat, 2008-10-25 at 09:20 +0200, Hedley Roos wrote: > > > > > I suspect specific indexes are just performing suboptimally and needs to > > be improved. Extend

Re: [Zope-dev] ZCatalog caching with memcached

2008-10-26 Thread Roché Compaan
Hi Fabio The funkload tests were project specific. I plan to write up my findings and to do benchmarks on a standard Plone instance and blog about it. This will unfortunately have to wait since I'm on holiday this week :-) -- Roché Compaan Upfront Systems

Re: [Zope-dev] ZCatalog caching with memcached

2008-10-26 Thread Roché Compaan
cache the IISet result sets. I suspect specific indexes are just performing suboptimally and needs to be improved. ExtendPathIndex in Plone seems to be one of them. The effect on performance is really awesome, now we just need to fine tune the implementation. -- Roc

Re: [Zope-dev] Re: Zope 3 catalog

2007-11-02 Thread Roché Compaan
On Fri, 2007-11-02 at 11:44 -0700, Ross Patterson wrote: > Roché Compaan <[EMAIL PROTECTED]> writes: > > > Where can one find Zope 3 catalog? Or is it simply zope.index? > > I think zope.app.catalog is what you're looking for. Thanks. I was looking at the Zope 3

[Zope-dev] Zope 3 catalog

2007-11-02 Thread Roché Compaan
Where can one find Zope 3 catalog? Or is it simply zope.index? -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross

Re: [Zope-dev] Re: More resilient indexes

2007-09-19 Thread Roché Compaan
On Tue, 2007-09-18 at 20:01 +0200, Dieter Maurer wrote: > Roché Compaan wrote at 2007-9-18 08:55 +0200: > > ... > >Sorry if I was unclear but what I'm really asking is if it is possible > >to improve the conflict handling of the current indexes that we have in &g

Re: [Zope-dev] Re: More resilient indexes

2007-09-17 Thread Roché Compaan
On Mon, 2007-09-17 at 19:57 -0400, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Roché Compaan wrote: > > Hi there > > > > Are the current indexing solutions that we have in Zope 2 and 3 the best > > that we can have? Is th

[Zope-dev] More resilient indexes

2007-09-17 Thread Roché Compaan
ally would like to make the ZODB work for me here. Are there solutions here? Can one employ some of the QueueCatalog conflict resolution strategies to make indexes more resilient? Or should one use some locking strategy instead? -- Roché Compaan Upfront Systems

[Zope-dev] Bug in Traversable.py

2007-03-23 Thread Roché Compaan
raise Unauthorized(name) If you agree that his is a bug, I'll log it. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zo

Re: [Zope-dev] Re: Caching ZCatalog results

2007-02-25 Thread Roché Compaan
r. Maybe this is more than enough, I don't know, but with apps like Plone that relies heavily on the catalog, optimisation of catalog operations can surely help improving scalability. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za

Re: [Zope-dev] Re: Caching ZCatalog results

2007-02-25 Thread Roché Compaan
On Sat, 2007-02-24 at 09:48 +0100, Dieter Maurer wrote: > Roché Compaan wrote at 2007-2-23 22:00 +0200: > > ... > >Thanks for that pointer. It's good that way, it should make invalidation > >easier. It could be as simple as invalidating any cached result that > &

Re: [Zope-dev] Re: Caching ZCatalog results

2007-02-23 Thread Roché Compaan
On Fri, 2007-02-23 at 20:43 +0100, Dieter Maurer wrote: > Roché Compaan wrote at 2007-2-23 18:44 +0200: > > ... > >Cool idea. I haven't done any coding involving OFS.Cache though. Looking > >at it briefly it looks like one can modify the catalog to subclass > &g

Re: [Zope-dev] Re: Caching ZCatalog results

2007-02-23 Thread Roché Compaan
On Fri, 2007-02-23 at 12:09 -0500, Tres Seaver wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > Tres Seaver wrote: > > Roché Compaan wrote: > >>> On Fri, 2007-02-23 at 06:55 -0500, Tres Seaver wrote: > >>>> Roché Compaan wrote: > >&

Re: [Zope-dev] Re: Caching ZCatalog results

2007-02-23 Thread Roché Compaan
On Fri, 2007-02-23 at 06:55 -0500, Tres Seaver wrote: > Roché Compaan wrote: > > I'm curious, has anybody played around with the idea of caching ZCatalog > > results and if I submitted a patch to do this would it be excepted? > > > > I quickly coded some basic

[Zope-dev] Caching ZCatalog results

2007-02-23 Thread Roché Compaan
ith a Plone site that is a heavy user of the catalog) -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ Zope-Dev maillist - Zope-Dev@zope.org http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML enc

Re: [Zope-dev] ZCatalog getObject broken

2005-03-03 Thread Roché Compaan
On Thu, 2005-03-03 at 14:56 +, Chris Withers wrote: > Roché Compaan wrote: > > +obj = self.aq_parent.unrestrictedTraverse(self.getPath(), None) > > +if obj and securityManager.validate(obj, obj, None, None): > > +return obj > > +el

[Zope-dev] ZCatalog getObject broken

2005-02-25 Thread Roché Compaan
found. But, it won't work because restrictedTraverse already fails when traversing the 'documents' folder. I would propose that getObject does an unrestrictedTraverse of the path and then checks if the user has permission to access that the object. -- Roché Compaan U

Re: [Zope-dev] Making a ZSQL.DA fully multi-threaded?

2004-07-10 Thread Roché Compaan
without bugging the list too > much? I don't think there are any comprehensive documentation about writing DA's but '_v_' attributes are *volatile* attributes and they are not persisted, see http://zope.org/Wikis/ZODB/VolatileAttributes. -- Roché Compaan Upfront Systems

Re: [Zope-dev] KeywordIndex errors: This should not happen.

2002-09-30 Thread Roché Compaan
On Mon, 30 Sep 2002 09:50:56 -0400 Casey Duncan <[EMAIL PROTECTED]> wrote: > On Monday 30 September 2002 08:36 am, Roché Compaan wrote: > > Don't know if this will help but these errors specifically occur with > > 'lines' properties with Keyword Indexes on the

Re: [Zope-dev] KeywordIndex errors: This should not happen.

2002-09-30 Thread Roché Compaan
feel right. I would have thought that only unique values should be stored in the index data. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za On Mon, 30 Sep 2002 14:17:17 +0200 Jean Jordaan <[EMAIL PROTECTED]> wrote: > Hi all > > Our app has b

Re: [Zope] Re: [Zope-dev] Interesting namespace observation

2002-05-27 Thread Roché Compaan
On Mon, 2002-05-27 at 18:31, Leonardo Rochael Almeida wrote: > On Mon, 2002-05-27 at 11:45, Roché Compaan wrote: > > [...] > > > > > > This is not a valid syntax, you should have:: > > > > Note the double quotes around the getProxies call. In orde

[Zope-dev] Interesting namespace observation

2002-05-27 Thread Roché Compaan
hat makes it impossible for the subsequent tag to find the "getProxies" method in the root. Any ideas why? -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za ___ Zope-Dev maillist - [EMAIL PROTECTED] http

RE: [Zope-dev] Zope process killed

2002-05-18 Thread Roché Compaan
> ;-) It very nice to hear that I can simply upgrade and my problem will go away :) And it did! Just thought I'd mention this since I didn't find anything in the collector about it. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za

[Zope-dev] Zope process killed

2002-05-17 Thread Roché Compaan
7;complex': , 'EOFError': , 'min': , 'reorder': , 'OSError': , 'same_type': , 'RuntimeError': , 'LookupError': , 'apply': , 'EnvironmentError': , 'unicode': , 'ValueError':

[Zope-dev] ZPatterns and Transactions

2001-05-31 Thread Roché Compaan
I have a ZPatterns application that uses the ZODB as storage. When I delete an object only the name of the script that deleted the object is shown in the Undo log. Is there any way to determine the ID of the object that was deleted? Is there a way that one can get a handle on the object that wa