We are facing the same problem here. For now, our solution is having the crawler notify the searcher whenever there is an updated segment, and the searcher gets a new bean when this happens. Now my question is, does the searcher load everything into memory, including segments, on startup? Because it seems that if I delete segments or replace them while the searcher is running, it doesn't affect the search result. But if it does load everything into memory, isn't that a LOT of memory we are talking about?
On 1/27/06, Stefan Groschupf <[EMAIL PROTECTED]> wrote: > > This is difficult today. > We had somehow hack the nutch bean to get this done. If I would have > more time I would clean up things and publish, but you know time. :-O > Anyway we had a seperated thread that is scanning the folder and > compare the result to the last scan, if something changes we created > a new search and detailer, runs one query against them to warm up and > than replace the searcher and detailer in the nutch bean. > I notice that this is a common need and I will put this in my todo > list on a little higher position. :) Anyway for now there is no > solution. > Stefan > > Am 27.01.2006 um 05:03 schrieb Chun Wei Ho: > > > Hi, > > > > We are running a nutch crawler on one machine and a web search > > application on searching an index using NutchBean on another. > > Periodically we would like to copy the updated crawl index from the > > crawl machine to replace the one used by the search application, > > without resulting in any broken queries or search application > > downtime. > > > > Could anyone give us some pointers on how they are doing this or > > setting it up? Thanks :) > > > >
