Re: Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread Xiaocheng Luan
You can store the fields in the index itself if you want, without indexing them (just flag it as stored/unindexed). I believe storing fields should not incur the "norms" size problem, please correct me if I'm wrong. Thanks, Xiaocheng maureen tanuwidjaja <[EMAIL PROTECTED]> wrote: Ya...I think i

Re: Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread maureen tanuwidjaja
Ya...I think i will store it in the database so that later it could be used in scoring/ranking for retrieval...:) Another thing i would like to see is whether the precision or recall will be much affaected by this... Regards, Maureen Xiaocheng Luan <[EMAIL PROTECTED]> wrote:One side

Re: Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread Xiaocheng Luan
One side-effect of turning off the norms may be that the scoring/ranking will be different? Do you need to search by each of these many fields? If not, you probably don't have to index these fields (but store them for retrieval?). Just a thought. Xiaocheng Michael McCandless <[EMAIL PROTECTED]>

Re: Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread maureen tanuwidjaja
Hi Mike, How to disable/turn off the norm?is it while indexing? Thanks, Maureen - Need Mail bonding? Go to the Yahoo! Mail Q&A for great tips from Yahoo! Answers users.

Re: Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread Michael McCandless
"maureen tanuwidjaja" <[EMAIL PROTECTED]> wrote: > "The only simple workaround I can think of is to set maxMergeDocs to > keep all segments "small". But then you may have too many segments > with time. Either that or find a way to reduce the number of unique > fields that you actually need to

Re: Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread Michael McCandless
"Michael McCandless" <[EMAIL PROTECTED]> wrote: > The only simple workaround I can think of is to set maxMergeDocs to > keep all segments "small". But then you may have too many segments > with time. Either that or find a way to reduce the number of unique > fields that you actually need to sto

Re: Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread maureen tanuwidjaja
Oops sorry,mistyping.. I have the searching result in 30 SECONDS to 3 minutes, which is actually quite unacceptable for the "search engine" I build...Is there any recommendation on how faster searching could be done? maureen tanuwidjaja <[EMAIL PROTECTED]> wrote: Hi mike "The on

Re: Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread maureen tanuwidjaja
Hi mike "The only simple workaround I can think of is to set maxMergeDocs to keep all segments "small". But then you may have too many segments with time. Either that or find a way to reduce the number of unique fields that you actually need to store." It is not possible for me to reduce

Re: Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread Michael McCandless
"maureen tanuwidjaja" <[EMAIL PROTECTED]> wrote: > "One thing that stands out in your listing is: your norms file > (_1ke1.nrm) is enormous compared to all other files. Are you indexing > many tiny docs where each docs has highly variable fields or > something?" > > Ya I also confuse

Re: Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread maureen tanuwidjaja
Hi Mike.. "One thing that stands out in your listing is: your norms file (_1ke1.nrm) is enormous compared to all other files. Are you indexing many tiny docs where each docs has highly variable fields or something?" Ya I also confuse why this nrm file is trmendous in size. I am ind

Re: Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread Michael McCandless
"maureen tanuwidjaja" <[EMAIL PROTECTED]> wrote: > How much actually the disk space needed to optimize the index?The > explanation given in documentation seems to be very different with the > practical situation > > I have an index file of size 18.6 G and I am going to optimize it.I

Urgent : How much actually the disk space needed to optimize the index?

2007-03-13 Thread maureen tanuwidjaja
Dear All How much actually the disk space needed to optimize the index?The explanation given in documentation seems to be very different with the practical situation I have an index file of size 18.6 G and I am going to optimize it.I keep this index in mobile Hard Disk with capacit