It is still open to you how you handle it. On my projects I normally only
store string fields. If I compress them, they are binary. So I use a similar
approach like you to compress only large fields and store the others as
string fields like before.
When I retrieve the contents of the documents I use
Document.getField("name") and then use the Fieldable.isBinary() to detect if
it was stored binary (and therefore compressed) or if it is string.
-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: u...@thetaphi.de
> -----Original Message-----
> From: Ivan Vasilev [mailto:ivasi...@sirma.bg]
> Sent: Tuesday, December 29, 2009 11:50 AM
> To: java-user@lucene.apache.org
> Subject: Re: Compressing field content with Lucene 3.0
>
> 10x Uwe for your answer,
>
> It is good news that data compressed with Field.Store.COMPRESS with 2.4
> will be retrieved properly from 3.0.
>
> From your answer I understand that in 3.0 there is no API way to
> compress some of the values of some field and not to compress other
> values for the same field. We should choose either to compress all of
> the values of that field (and keep them as bin values) or not to
> compress any of values for that field?
>
> Cheers,
> Ivan
>
>
> Uwe Schindler wrote:
> > If it is a 2.4 index, you can read it without any problems. It is only
> no
> > longer possible to add fields with Field.Store.COMPRESS. Nothing more
> > changed.
> >
> > If you want to add field with some compression, you have to compress
> > yourself e.g. to a byte[]. You can then add this byte[] as a binary
> stored
> > field. How you deal with this is in your responsibility.
> >
> > -----
> > Uwe Schindler
> > H.-H.-Meier-Allee 63, D-28213 Bremen
> > http://www.thetaphi.de
> > eMail: u...@thetaphi.de
> >
> >
> >> -----Original Message-----
> >> From: Ivan Vasilev [mailto:ivasi...@sirma.bg]
> >> Sent: Monday, December 28, 2009 7:13 PM
> >> To: LUCENE MAIL LIST
> >> Subject: Compressing field content with Lucene 3.0
> >>
> >> Hi Guys,
> >>
> >> Could you give me advice how to deal with Lucene 3.0 with 2.4 indexes
> >> that contain compressed data.
> >>
> >> Our case is following - we have code like this:
> >>
> >> Field.Store fieldStored = storedFieldsSet.contains(fieldName) ?
> >> (fieldValue.length() >= COMPRESS_THRESHOLD ? Field.Store.COMPRESS :
> >> Field.Store.YES) : Field.Store.NO;
> >> Field.Index fieldIndexed = indexedFieldsSet.contains(fieldName) ?
> >> Field.Index.ANALYZED : Field.Index.NO;
> >> doc.add(new Field(fieldName, fieldValue, fieldStored, fieldIndexed));
> >>
> >> So for one and the same field some values are compressed in the indexes
> >> other values are not.
> >> Reading Lucene documentation I understand that all the values for some
> >> field should be either compressed or not compressed OR we should
> somehow
> >> keep which values for which fields are compressed. If this is so it
> >> would be very difficult to migrate our system to Lucene 3.0 without
> need
> >> of re indexing.
> >> If I am not right could you tell me please:
> >> 1. How to read these indexes (created with the code above with Lucene
> >> 2.4) with Lucene 3.0? I mean when fetching field values how to know
> when
> >> to use CompressionTools.decompressString(..) and when to skip it?
> >> 2. Is there possibility in Lucene 3.0 again to compress values of some
> >> docs for certain field and for other docs the values for the same field
> >> not to be compressed?
> >>
> >> Cheers,
> >> Ivan
> >>
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> >> For additional commands, e-mail: java-user-h...@lucene.apache.org
> >>
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> > For additional commands, e-mail: java-user-h...@lucene.apache.org
> >
> >
> > __________ NOD32 3990 (20090406) Information __________
> >
> > This message was checked by NOD32 antivirus system.
> > http://www.eset.com
> >
> >
> >
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
> For additional commands, e-mail: java-user-h...@lucene.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscr...@lucene.apache.org
For additional commands, e-mail: java-user-h...@lucene.apache.org
