Has the physical path of the object changed? If not, the newer (2.3.0 + )
catalog stuff should be smart enough to figure out whether anything inside
the object has changed during catalog_object. If nothing has changed, none
of the indexes or metadata columns should be updated. We're scrambling
right now to write tests for this kind of thing. :-(
Note that the algoritm is simple - for each index, compare the what exists
in the index to what is to be put in. If they're the same, do nothing. If
they're different, reindex. I wasn't able to understand completely from
your description whether the object method your're attempting to index via a
TextIndex actually returns different data or not when you recatalog it.
Does it?
There's a huge possibility that the merge stuff isn't working completely
because until now there have been no deterministic tests.
----- Original Message -----
From: "R. David Murray " <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Saturday, March 03, 2001 2:29 PM
Subject: Re: [Zope-dev] ZCatalog hackery
> I've got an analogous but different problem with ZCatalog udpates. I'd
> like to ask for ideas about the best way to handle this.
>
> I've got a ZPatterns based ap that is pulling data from a postgresql
> database. But associated with the object created out of the database
> are signficant chunks of html (Author bios, book descriptions). I've
> stored these html chunks in the database as well, though I don't think
> that matters to the problem or possible solution strategies.
>
> The issue is that parts of the database get updated periodically from an
> external source. That is, the author and book tables get replaced
> wholesale. But the Book and Author objects are cataloged using
> ZCatalog, doing a full text index on a combination of fields from
> the external database and the html chunks. As far as I can see
> at the moment, this means that every object has to get uncataloged
> and recataloged, meaning every time the database update happens
> the zodb inflates considerably, even though very little data has
> actually changed.
>
> So far I can think of two simple solutions to this problem: (1) pack
> often (2) put the Catalog into a mounted storage backed by a
> non-undoable storage. I'd rather not do either one of these
> (the first for obvious reasons, the second simply because I don't
> want to take the time to learn how to set up a non-undoable storage).
> Am I missing some other obvious options? It seems like there *ought*
> to be a way to avoid the overhead of updating the catalog for objects
> that haven't really changed.
>
> --RDM
>
>
> _______________________________________________
> Zope-Dev maillist - [EMAIL PROTECTED]
> http://lists.zope.org/mailman/listinfo/zope-dev
> ** No cross posts or HTML encoding! **
> (Related lists -
> http://lists.zope.org/mailman/listinfo/zope-announce
> http://lists.zope.org/mailman/listinfo/zope )
>
_______________________________________________
Zope-Dev maillist - [EMAIL PROTECTED]
http://lists.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://lists.zope.org/mailman/listinfo/zope-announce
http://lists.zope.org/mailman/listinfo/zope )
