Hi Hossman, Thanks for your reply.when i index the search fields in my lucene document,it occupy 20% of the original size.how can i reduce the reduce the index size.
hossman_lucene wrote: > > > : I need to store all the attributes of the document i index as part of > the > : index. And I need to get the size of the files as close to 20% of the > : original size as possible. If anyone can help with this I can pay a > nominal > : fee. Please contact me if anyone can help. > > Let's be clear about something here: Lucene is an "indexing" technology, > not a compression technology. Lucene indexes *can* be smaller then the > source corpus they index for a variety of reasons (most notably because > the inverted index structure allows a more susinect represetation of the > same words appearing in multiple docuemnts, but things like stop word > removal play a big part as well) but this reduced size only applies to the > *indexed* fields. If you "store" data in Lucene there is nothing > intrinisic in the Lucene that makes it smaller. If you have a chunk of > text, and you tell Lucene to store that text, and index that text, it > doesn't matter how space efficient the "index" is, Lucene still still > needs to store all of that text. > > Here's a few simple steps to achieve what you want, and i won't charge you > anything for it... > > 1) take your existing code for building an index, and change it so that > all of the options to "store" field values are commented out, and only the > "indexed" fields exist. > 2) index your corpus of size X, note the size of the index Y. compute > the value Z such that: Z = (0.20 * X) - Y > 3) go find a compression library A that can compress your corpus of size > X down to size Z > 4) go edit your orriginal code to use algorithm A to compress all of > your stored fields before adding them to Lucene. > > Viola! > > > > -Hoss > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [EMAIL PROTECTED] > For additional commands, e-mail: [EMAIL PROTECTED] > > > -- View this message in context: http://www.nabble.com/Need-Lucene-Compression-help----can-pay-nominal-fee-tf3881801.html#a11188737 Sent from the Lucene - Java Users mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]