Mike Klaas wrote on 12/05/2006 11:38 AM: > On 12/5/06, negrinv <[EMAIL PROTECTED]> wrote: > >> Chris Hostetter wrote: > >> > If the code was not already in the core, and someone asked about >> adding it >> > I would argue against doing so on the grounds that some helpfull >> utility >> > methods (possibly in a contrib) would be just as usefull, and would >> have >> > no performance cost for people who don't care about compression. >> > >> Perhaps, if you look at compression on its own, but once you see >> compression >> in the context of all the other field options it makes sense to have it >> added to Lucene, it's about having everything in one place for ease of >> implementation that offsets the performance issue, in my opinion. > > Note that built-in compression is deprecated, for similar reasons as > are being given for the encrypted fields.
Built-in compression is also memory-hungry and slow due to the copying it does. External compression is much faster, especially if you extend Field binary values to support a binary length parameter (which I submitted a patch for a long time ago). Here is another argument against adding Field encryption to the lucene core. Changes in index format make life complex for any implementations that deal with index files directly. There are a number of Lucene sister projects that do this, plus a number of applications. I have a fast bulk updater that directly manipulates index files and am busy upgrading it right now to the 2.1 index format with lockless commits (which is not fully documented in the new index file formats, by the way, e.g. the <segment>N.sM separate norm files are missing). It's a pain. In general, I think changes to Lucene index format should only be driven by compelling benefits. Moving encryption from external to internal to get a minor application simplification is not sufficiently compelling to me. Chuck --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
