Hi, I am facing multiple issues with solr and I am not sure what happens in each case. I am quite naive in Solr and there are some scenarios I'd like to discuss with you.
We have a huge volume of documents to be indexed. Somewhere about 5 million. We have a full indexer script which essentially picks up all the documents from database and updates into Solr and an incremental script which adds new documents to Solr.. Relevant areas of my config file goes like <unlockOnStartup>false</unlockOnStartup> <deletionPolicy class="solr.SolrDeletionPolicy"> <!-- Keep only optimized commit points --> <str name="keepOptimizedOnly">false</str> <!-- The maximum number of commit points to be kept --> <str name="maxCommitsToKeep">1</str> </deletionPolicy> <updateHandler class="solr.DirectUpdateHandler2"> <autoCommit> <maxDocs>100000</maxDocs> </autoCommit> </updateHandler> <requestHandler name="/replication" class="solr.ReplicationHandler"> <lst name="master"> <str name="enable">${enable.master:false}</str> <str name="replicateAfter">startup</str> <str name="replicateAfter">commit</str> </lst> <lst name="slave"> <str name="enable">${enable.slave:false}</str> <str name="masterUrl">http://hostname:port/solr/core0/replication</str> </lst> </requestHandler> Sometimes, while the full indexer script breaks while adding documents to Solr. The script adds the documents and then commits the operation. So, when the script breaks, we have a huge lot of data which has been updated but not committed. Next, the incremental index script executes, and figures out all the new entries, adds them to Solr. It works successfully and commits the operation. - Will the commit by incremental indexer script also commit the previously uncommitted changes made by full indexer script before it broke? Sometimes, while during execution, Solr's avg response time 9avg resp time for last 10 requests, read from log file) goes as high as 9000ms (which I am still unclear why, any ideas how to start hunting for the problem?), so the watchdog process restarts Solr (because it causes a pile of requests queue at application server, which causes app server to crash). On my local environment, I performed the same experiment by adding docs to Solr, killing the process and restarting it. I found that the uncommitted changes were applied and searchable. However, the updates were uncommitted. Could you explain me as to how is this happening, or is there a configuration that can be adjusted for this? Also, what would the index state be if after the restarting Solr, a commit is applied or a commit is not applied? I'd be happy to provide any other information that might be needed. *Pranav Prakash* "temet nosce" Twitter <http://twitter.com/pranavprakash> | Blog <http://blog.myblive.com> | Google <http://www.google.com/profiles/pranny> On Tue, Jun 28, 2011 at 20:55, Shalin Shekhar Mangar <shalinman...@gmail.com > wrote: > On Tue, Jun 28, 2011 at 4:18 PM, Pranav Prakash <pra...@gmail.com> wrote: > > > > > I am not sure what is the index number value? It looks like an epoch > time, > > but in my case, this points to one month back. However, i can see > documents > > which were added last week, to be in the index. > > > > The index version shown on the dashboard is the time at which the most > recent index segment was created. I'm not sure why it has a value older > than > a month if a commit has happened after that time. > > > > > Even after I did a commit, the index number did not change? Isn't it > > supposed to change on every commit? If not, is there a way to look into > the > > last index time? > > > > Yeah, it changes after every commit which added/deleted a document. > > > > Also, this page > > http://wiki.apache.org/solr/SolrReplication#Replication_Dashboard shows > a > > Replication Dashboard. How is this dashboard invoked? Is there any URL > > which > > needs to be called? > > > > > If you have configured replication correctly, the admin dashboard should > show a "Replication" link right next to the "Schema Browser" link. The path > should be /admin/replication/index.jsp > > -- > Regards, > Shalin Shekhar Mangar. >