Re: Detecting an empty index during start-up
Thanks Chris. I dug into the SolrCore code and after reading some of the code I ended up going with core.getNewestSearcher(true) and this fixed the problem. David On Thu, Mar 24, 2011 at 7:20 PM, Chris Hostetter hossman_luc...@fucit.orgwrote: : I am not familiar with Solr internals, so the approach I wanted to take was : to basically check the numDocs property of the index during start-up and set : a READABLE state in the ZooKeeper node if it's greater than 0. I also : planned to create a commit hook for replication and updating which : controlled the READABLE property based on numDocs also. : : This just leaves the problem of finding out the number of documents during : start-up. I planned to have something like: Most of the ZK stuff you mentioned is over my head, but i get the general gist of what you want: * a hook on startup that checks numDocs * if not empty, trigger some logic My suggestion would be to implement this as a firstSearcher SolrEventListener. when that runs, you'll have easy access to a SOlrIndexSearcher (and you won't even have to refcount it) and you can fire whatever logic you want based on what you find when looking at it. -Hoss
Re: Detecting an empty index during start-up
On 3/25/11 11:25 AM, David McLaughlin wrote: Thanks Chris. I dug into the SolrCore code and after reading some of the code I ended up going with core.getNewestSearcher(true) and this fixed the problem. FYI, openNew=true is not implemented and can result in an UnsupportedOperationException. For now it's better to pass openNew=false and be prepared to get a null. -- Best regards, Andrzej Bialecki ___. ___ ___ ___ _ _ __ [__ || __|__/|__||\/| Information Retrieval, Semantic Web ___|||__|| \| || | Embedded Unix, System Integration http://www.sigram.com Contact: info at sigram dot com
Detecting an empty index during start-up
Hi,
In our Solr deployment we have a cluster of replicated Solr cores, with the
small change that we have dynamic master look-up using ZooKeeper. The
problem I am trying to solve is to make sure that when a new Solr core joins
the cluster it isn't made available to any search services until it has been
filled with data.
I am not familiar with Solr internals, so the approach I wanted to take was
to basically check the numDocs property of the index during start-up and set
a READABLE state in the ZooKeeper node if it's greater than 0. I also
planned to create a commit hook for replication and updating which
controlled the READABLE property based on numDocs also.
This just leaves the problem of finding out the number of documents during
start-up. I planned to have something like:
int numDocs = 0;
RefCountedSolrIndexSearcher searcher = core.getSearcher();
try {
numDocs = searcher.get().getIndexReader().numDocs();
} finally {
searcher.decref();
}
but getSearcher's documentation specifically says don't use it from the
inform method. I missed this at first and of course I got a deadlock
(although only when I had more than one core on the same Solr instance).
Is there a simpler way to do what I want? Or will I just need to have a
thread which waits until the Searcher is available before setting the
state?
Thanks,
David
Re: Detecting an empty index during start-up
: I am not familiar with Solr internals, so the approach I wanted to take was : to basically check the numDocs property of the index during start-up and set : a READABLE state in the ZooKeeper node if it's greater than 0. I also : planned to create a commit hook for replication and updating which : controlled the READABLE property based on numDocs also. : : This just leaves the problem of finding out the number of documents during : start-up. I planned to have something like: Most of the ZK stuff you mentioned is over my head, but i get the general gist of what you want: * a hook on startup that checks numDocs * if not empty, trigger some logic My suggestion would be to implement this as a firstSearcher SolrEventListener. when that runs, you'll have easy access to a SOlrIndexSearcher (and you won't even have to refcount it) and you can fire whatever logic you want based on what you find when looking at it. -Hoss
