Hi,
I’ve got an odd problem where I’m unable to create a collection as it’s
claiming a lock is held at /solr/solr-data/data/tech-data-001/index/write.lock,
but that location (/solr/solr-data/data/tech-data-001) doesn’t exist on any of
the nodes on the cluster.
Does anyone know what might be causing this?
Thanks
Karl
18:52:17.366
[OverseerThreadFactory-80-thread-5-processing-n:solr-8.search-solr.svc.cluster.local:80_solr]
ERROR org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler -
Error from shard solr-0.search-solr.svc.cluster.local:80_solr:
{responseHeader={status=0,QTime=0},STATUS=failed,Response=Error CREATEing
SolrCore 'tech-data-001_shard1_replica_n6': Unable to create core
[tech-data-001_shard1_replica_n6] Caused by: Lock held by this virtual machine:
/solr/solr-data/data/tech-data-001/index/write.lock}
18:52:18.381
[OverseerThreadFactory-80-thread-5-processing-n:solr-8.search-solr.svc.cluster.local:80_solr]
ERROR org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler -
Cleaning up collection [tech-data-001].
18:52:18.520 [Finalizer] ERROR org.apache.solr.update.SolrIndexWriter -
SolrIndexWriter was not closed prior to finalize(), indicates a bug -- POSSIBLE
RESOURCE LEAK!!!
18:52:18.520 [Finalizer] ERROR org.apache.solr.update.SolrIndexWriter - Error
closing IndexWriter
java.lang.NullPointerException: null
at
org.apache.lucene.index.IndexWriter.doFlush(IndexWriter.java:3927)
~[lucene-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:47:53]
at
org.apache.lucene.index.IndexWriter.flush(IndexWriter.java:3887)
~[lucene-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:47:53]
at
org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1235)
~[lucene-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:47:53]
at
org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1278)
~[lucene-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:47:53]
at
org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:298)
[solr-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:02]
at
org.apache.solr.update.SolrIndexWriter.finalize(SolrIndexWriter.java:357)
[solr-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:02]
at java.lang.System$2.invokeFinalize(System.java:2125) [?:?]
at java.lang.ref.Finalizer.runFinalizer(Finalizer.java:87) [?:?]
at
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:171) [?:?]
Suppressed: java.lang.NullPointerException
at
org.apache.lucene.index.IndexWriter.rollbackInternalNoCommit(IndexWriter.java:2435)
~[lucene-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:47:53]
at
org.apache.lucene.index.IndexWriter.rollbackInternal(IndexWriter.java:2410)
~[lucene-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:47:53]
at
org.apache.lucene.index.IndexWriter.shutdown(IndexWriter.java:1241)
~[lucene-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:47:53]
at
org.apache.lucene.index.IndexWriter.close(IndexWriter.java:1278)
~[lucene-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:47:53]
at
org.apache.solr.update.SolrIndexWriter.close(SolrIndexWriter.java:298)
[solr-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:02]
at
org.apache.solr.update.SolrIndexWriter.finalize(SolrIndexWriter.java:357)
[solr-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:02]
at
java.lang.System$2.invokeFinalize(System.java:2125) [?:?]
at
java.lang.ref.Finalizer.runFinalizer(Finalizer.java:87) [?:?]
at
java.lang.ref.Finalizer$FinalizerThread.run(Finalizer.java:171) [?:?]
18:52:18.561
[OverseerThreadFactory-80-thread-5-processing-n:solr-8.search-solr.svc.cluster.local:80_solr]
ERROR org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler -
Collection: tech-data-001 operation: create
failed:org.apache.solr.common.SolrException: Underlying core creation failed
while creating collection: tech-data-001
at
org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:371)
at
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:266)
at
org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:524)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:218)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
18:56:03.371
[OverseerThreadFactory-80-thread-5-processing-n:solr-8.search-solr.svc.cluster.local:80_solr]
ERROR org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler -
Error from shard: http://solr-0.search-solr.svc.cluster.local:80/solr
org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException: Error
from server at http://solr-0.search-solr.svc.cluster.local:80/solr: Error
CREATEing SolrCore 'tech-data-001_shard1_replica_n1': Unable to create core
[tech-data-001_shard1_replica_n1] Caused by: Lock held by this virtual machine:
/solr/solr-data/data/tech-data-001/index/write.lock
at
org.apache.solr.client.solrj.impl.HttpSolrClient.executeMethod(HttpSolrClient.java:681)
~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:04]
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:266)
~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:04]
at
org.apache.solr.client.solrj.impl.HttpSolrClient.request(HttpSolrClient.java:248)
~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:04]
at
org.apache.solr.client.solrj.SolrClient.request(SolrClient.java:1290)
~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:04]
at
org.apache.solr.handler.component.HttpShardHandlerFactory$1.request(HttpShardHandlerFactory.java:169)
~[solr-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:02]
at
org.apache.solr.handler.component.ShardRequestor.call(ShardRequestor.java:130)
~[solr-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:02]
at
org.apache.solr.handler.component.ShardRequestor.call(ShardRequestor.java:41)
~[solr-core-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:02]
at java.util.concurrent.FutureTask.run(FutureTask.java:264)
~[?:?]
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
at java.util.concurrent.FutureTask.run(FutureTask.java:264)
~[?:?]
at
com.codahale.metrics.InstrumentedExecutorService$InstrumentedRunnable.run(InstrumentedExecutorService.java:180)
~[metrics-core-4.1.5.jar:4.1.5]
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:218)
~[solr-solrj-8.8.1.jar:8.8.1 64f3b496bfee762a9d2dbff40700f457f4464dfe - tjp -
2021-02-16 15:52:04]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
[?:?]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
[?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
18:56:03.372
[OverseerThreadFactory-80-thread-5-processing-n:solr-8.search-solr.svc.cluster.local:80_solr]
ERROR org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler -
Cleaning up collection [tech-data-001].
18:56:03.499
[OverseerThreadFactory-80-thread-5-processing-n:solr-8.search-solr.svc.cluster.local:80_solr]
ERROR org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler -
Collection: tech-data-001 operation: create
failed:org.apache.solr.common.SolrException: Underlying core creation failed
while creating collection: tech-data-001
at
org.apache.solr.cloud.api.collections.CreateCollectionCmd.call(CreateCollectionCmd.java:371)
at
org.apache.solr.cloud.api.collections.OverseerCollectionMessageHandler.processMessage(OverseerCollectionMessageHandler.java:266)
at
org.apache.solr.cloud.OverseerTaskProcessor$Runner.run(OverseerTaskProcessor.java:524)
at
org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:218)
at
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
Unless expressly stated otherwise in this email, this e-mail is sent on behalf
of Auto Trader Limited Registered Office: 1 Tony Wilson Place, Manchester,
Lancashire, M15 4FN (Registered in England No. 03909628). Auto Trader Limited
is part of the Auto Trader Group Plc group. This email and any files
transmitted with it are confidential and may be legally privileged, and
intended solely for the use of the individual or entity to whom they are
addressed. If you have received this email in error please notify the sender.
This email message has been swept for the presence of computer viruses.
