On 11/6/2013 1:18 PM, kaustubh147 wrote:
I am attaching a small log file with debug option enabled.. log shows
following process
1. first start of solr cluster
2. create collection "collection1"
3. shutdown cluster
4. start cluster again
error is in only the 4th step...and it is coming after solr is trying to
close data directory for some reason... class name is
org.apache.solr.core.CachingDirectoryFactory..
Thanks,
Kaustubh
solr_logs_for_post.txt
<http://lucene.472066.n3.nabble.com/file/n4099641/solr_logs_for_post.txt>
Here's a relevant excerpt from that log:
Caused by: org.apache.lucene.store.LockObtainFailedException: Lock
obtain timed out:
NativeFSLock@/mnt/emc/app_name/data-prod-refresh/SolrCloud/SolrHome3/solr/collection1/data/index/write.lock
at org.apache.lucene.store.Lock.obtain(Lock.java:84)
at org.apache.lucene.index.IndexWriter.<init>(IndexWriter.java:673)
at
org.apache.solr.update.SolrIndexWriter.<init>(SolrIndexWriter.java:77)
at
org.apache.solr.update.SolrIndexWriter.create(SolrIndexWriter.java:64)
at
org.apache.solr.update.DefaultSolrCoreState.createMainIndexWriter(DefaultSolrCoreState.java:267)
at
org.apache.solr.update.DefaultSolrCoreState.getIndexWriter(DefaultSolrCoreState.java:110)
at org.apache.solr.core.SolrCore.openNewSearcher(SolrCore.java:1440)
... 15 more
In a nutshell, it's trying to create the "write.lock" file in your index
directory and obtain a lock on it. Either the file already exists from
the previous Solr start, or the lock is failing.
You may need to change your lock type. Currently it's "native" but you
might need "simple" or "none" instead, for NFS to work properly.
You might also need to upgrade to NFSv4, which has much better locking
capability than earlier versions. Unfortunately it also tends to have
mounting problems, leading people to downgrade the version to 3 on their
servers.
There might be network problems, or bugs in your NFS server or NFS
client code, etc. The path suggests that the storage is an EMC device.
Check with them for possible bugs with their NFS implementation - you
might need newer software/firmware for the device. NIC firmware on your
server is another possible problem point.
Running with your index on NFS is problematic. I have never personally
tried it, but I see messages here and on IRC about problems with it. I
believe there are some people who do it successfully, perhaps someone
who's faced these problems and beat them could offer insight.
Thanks,
Shawn
