Le Samedi 23 Décembre 2006 00:32, Michael Busch a écrit :
> Nicolas Lalevée wrote:
> > I have just looked at it. It looks great :)
>
> Thanks! :-)
>
> > But I still doesn't understand why a new entry in the fieldinfo is
> > needed.
>
> The entry is not really *needed*, but I use it for
> backwards-compatibility and as an optimization for fields that don't
> have any tokens with payloads. For fields with payloads the
> PositionDelta is shifted one bit, so for certain values this means that
> the VInt needs an extra byte. I have an index with about 500k web
> documents and measured, that about 8% of all PositionDelta values would
> need one extra byte in case PositionDelta is shifted. For my index that
> means roughly 4% growth of the total index size. With using a fieldbit,
> payloads can be disabled for a field and therefore the shifting of
> PositionDelta can be avoided. Furthermore, if the payload-fieldbit is
> not enabled, then the index format does not change at all.
>
> > There is the same for TermVector. And code like that fail for no obvious
> > reason :
> >
> > Document doc = new Document();
> > doc.add(new Field("f1", "v1", Store.YES, Index.TOKENIZED,
> > TermVector.WITH_POSITIONS_OFFSETS));
> > doc.add(new Field("f1", "v2", Store.YES, Index.TOKENIZED,
> > TermVector.NO));
> >
> > RAMDirectory ram = new RAMDirectory();
> > IndexWriter writer = new IndexWriter(ram, new StandardAnalyzer(), true);
> > writer.addDocument(doc);
> > writer.close();
> >
> > Knowing a little bit about how lucene works, I have an idea why this
> > fail, but can we avoid this ?
> >
> > Nicolas
>
> In the payload case there is no problem like this one. There is no new
> Field option that can be used to set the fieldbit explicitly. The bit is
> set automatically for a field as soon as the first Token of that field
> that carries a payload is encountered.ok, thanks for the explaination. I looked closer to how indexing works, and in fact the issue I was talking about was I think a bug. Filling a jira issue. -- Nicolas LALEVÉE Solutions & Technologies ANYWARE TECHNOLOGIES Tel : +33 (0)5 61 00 52 90 Fax : +33 (0)5 61 00 51 46 http://www.anyware-tech.com --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
