On Sun, 2008-08-24 at 08:08 +0200, [EMAIL PROTECTED] wrote: > Tres Seaver wrote at 2008-8-22 16:45 -0400: > > ... > >I recall a pre-Zope (for me, 10 years ago) rule of thumb that text > >indexing imposed an order of magnitude of overhead on the actual corpus, > >with improvements possible only via batching or post-processing / > >compresstion (incremental indexing is worst-case). > > And this is especially true for indexes supporting a term frequency > based ranking and which uses "IISet" at places where "IITreeSet" were > more appropriate. > > With "TextIndexNG3", one can get rid of the overhead of > term frequency based ranking (in case one does not need it) > > Using "AdvancedQuery" (and parsing the text subqueries oneself), > one can use a "Managable SimpleTextIndex" which > tries very hard to be as efficient as possible for large data sets > (and does not support term frequency based ranking).
Thanks for the feedback. I'll re-run the tests without any text indexes, as well as run it with other implementations such as TextIndexNG3 and SimpleTextIndex and compare the results. -- Roché Compaan Upfront Systems http://www.upfrontsystems.co.za _______________________________________________ For more information about ZODB, see the ZODB Wiki: http://www.zope.org/Wikis/ZODB/ ZODB-Dev mailing list - ZODB-Dev@zope.org http://mail.zope.org/mailman/listinfo/zodb-dev
