Re: Detecting an empty index during start-up

[David McLaughlin]

Thanks Chris. I dug into the SolrCore code and after reading some of the
code I ended up going with core.getNewestSearcher(true) and this fixed the
problem.


David

On Thu, Mar 24, 2011 at 7:20 PM, Chris Hostetter
hossman_luc...@fucit.orgwrote:

 : I am not familiar with Solr internals, so the approach I wanted to take
 was
 : to basically check the numDocs property of the index during start-up and
 set
 : a READABLE state in the ZooKeeper node if it's greater than 0. I also
 : planned to create a commit hook for replication and updating which
 : controlled the READABLE property based on numDocs also.
 :
 : This just leaves the problem of finding out the number of documents
 during
 : start-up. I planned to have something like:

 Most of the ZK stuff you mentioned is over my head, but i get the general
 gist of what you want:

  * a hook on startup that checks numDocs
  * if not empty, trigger some logic

 My suggestion would be to implement this as a firstSearcher
 SolrEventListener.  when that runs, you'll have easy access to a
 SOlrIndexSearcher (and you won't even have to refcount it) and you can
 fire whatever logic you want based on what you find when looking at it.


 -Hoss

Detecting an empty index during start-up

[David McLaughlin]

Hi,

In our Solr deployment we have a cluster of replicated Solr cores, with the
small change that we have dynamic master look-up using ZooKeeper. The
problem I am trying to solve is to make sure that when a new Solr core joins
the cluster it isn't made available to any search services until it has been
filled with data.

I am not familiar with Solr internals, so the approach I wanted to take was
to basically check the numDocs property of the index during start-up and set
a READABLE state in the ZooKeeper node if it's greater than 0. I also
planned to create a commit hook for replication and updating which
controlled the READABLE property based on numDocs also.

This just leaves the problem of finding out the number of documents during
start-up. I planned to have something like:

int numDocs = 0;
RefCountedSolrIndexSearcher searcher = core.getSearcher();
try {
   numDocs = searcher.get().getIndexReader().numDocs();
} finally {
searcher.decref();
}

but getSearcher's documentation specifically says don't use it from the
inform method. I missed this at first and of course I got a deadlock
(although only when I had more than one core on the same Solr instance).

Is there a simpler way to do what I want? Or will I just need to have a
thread which waits until the Searcher is available before setting the
state?

Thanks,
David

Adding metadata to a Solr schema

[David McLaughlin]

Hi,

I need to add some meta data to a schema file in Solr - such a version and
current transaction id. I need to be able to query Solr to get this
information. What would be the best way to do this?

Thanks,
David

Re: Adding metadata to a Solr schema

[David McLaughlin]

Thanks Otis, yes it is the former and it definitely solves my problem for my
static metadata.

I realise now though that for dynamic values like transaction id, I probably
need a different method for storing this metadata though. Is there a
standard way of adding metadata to a Solr core and being able to set/get
this data at runtime?

Thanks,
David

On Wed, Jan 19, 2011 at 8:56 PM, Otis Gospodnetic 
otis_gospodne...@yahoo.com wrote:

 David,

 I'm not sure if you are asking about adding this to the schema.xml file or
 to
 the Solr schema and therefore the Solr index?
 If the former, you could put it in comments, then get the schema via HTTP
 (see
 Admin UI for the URL), and grep for your line from there.
 If the latter, this sounds like 2 fields.  Not sure if every document would
 have
 them, of you just need 1 doc with this data...

 Otis
 
 Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch
 Lucene ecosystem search :: http://search-lucene.com/



 - Original Message 
  From: David McLaughlin mesave...@gmail.com
  To: solr-user@lucene.apache.org
  Sent: Wed, January 19, 2011 2:13:30 PM
  Subject: Adding metadata to a Solr schema
 
  Hi,
 
  I need to add some meta data to a schema file in Solr - such a  version
 and
  current transaction id. I need to be able to query Solr to get  this
  information. What would be the best way to do  this?
 
  Thanks,
  David