we now meet the same situation and want to implement like this:
we add new documents to a RAMDirectory and search two indice-- the
index in disk and the RAM index.
regularly(e.g. every hour we flush the RAMDirecotry into disk and make
a new segment)
to prevent error. before add to RAMDirecotry,we write the document
into log file.
and after flushing, we delete corresponding lines in the log file
if the program corrput. we will redo the log and add them into RAMDirectory.
Any one has done similar work?
2010/12/1 Li Li <fancye...@gmail.com>:
> you may implement your own MergePolicy to keep on large index and
> merge all other small ones
> or simply set merge factor to 2 and the largest index not be merged by
> set maxMergeDocs less than the docs in the largest one.
> So there is one large index and a small one. when adding a little
> docs, they will be merged into the small one. and you can, e.g. weekly
> optimize the index and merge all indice into one index.
>
> 2010/11/30 stockii <st...@shopgate.com>:
>>
>> Hello.
>>
>> index is about 28 Million documents large. When i starts an delta-import is
>> look at modified. but delta import takes to long. over an hour need solr for
>> delta.
>>
>> thats my query. all sessions from the last hour should updated and all
>> changed. i think its normal that solr need long time for the querys. how can
>> i optimize this ?
>>
>> deltaQuery="SELECT id FROM sessions
>> WHERE created BETWEEN DATE_ADD( NOW(), INTERVAL - 10 HOUR ) AND NOW()
>> OR modified BETWEEN '${dataimporter.last_index_time}' AND DATE_ADD( NOW(),
>> INTERVAL - 1 HOUR ) "
>> --
>> View this message in context:
>> http://lucene.472066.n3.nabble.com/Best-practice-for-Delta-every-2-Minutes-tp1992714p1992714.html
>> Sent from the Solr - User mailing list archive at Nabble.com.
>>
>
