Let me start with 4) Have you tried simply using multiple threads to send your docs to a single Solr instance/core? You should get about the same performance as what you are trying with your approach below, but without the headache of managing multiple cores and index merging (not yet possible to do programatically).
Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch ----- Original Message ---- > From: vivek sar <vivex...@gmail.com> > To: solr-user@lucene.apache.org > Sent: Tuesday, March 31, 2009 1:59:01 PM > Subject: Merging Solr Indexes > > Hi, > > As part of speeding up the index process I'm thinking of spawning > multiple threads which will write to different temporary SolrCores. > Once the index process is done I want to merge all the indexes in > temporary cores to a master core. For ex., if I want one SolrCore per > day then every index cycle I'll spawn 4 threads which will index into > some temporary index and once they are done I want to merge all these > into the day core. My questions, > > 1) I want to use the same schema and solrconfig.xml for all cores > without duplicating them - how do I do that? > 2) How do I merge the temporary Solr cores into one master core > programmatically? I've read the wiki on "MergingSolrIndexes", but I > want to do it programmatically (like in Lucene - > writer.addIndexes(..)) once the temporary indices are done. > 3) Can I remove the temporary indices once the merge process is done? > 4) Is this the right strategy to speed up indexing? > > Thanks, > -vivek
