Hi,
I am facing a weird problem where solr shuts down everyday at exactly
the same time for no apparent reason. I don't know how to begin to
debug this. I am sharing some system info and log entries below. I
would appreciate any tips and insights into understanding this issue.
Best Regards,
===
OS: Windows Server 2022
Solr: 9.7.0 (675a41516e3f3bacfc975590773e7abdca444ff4)
Lucene: 9.11.1 (0c087dfdd10e0f6f3f6faecc6af4415e671a9e69)
JVM: Eclipse Adoptium OpenJDK 64-Bit Server VM 17.0.13 17.0.13+11
Service Manager: nssm
[solr.log]
2025-01-25 11:34:13.667 INFO (main) [c: s: r: x: t:] o.e.j.s.Server
jetty-10.0.22; built: 2024-06-27T16:03:51.502Z; git:
5c8471e852d377fd726ad9b1692c35ffc5febb09; jvm 17.0.13+11
2025-01-25 11:34:18.746 WARN (main) [c: s: r: x: t:]
o.e.j.u.DeprecationWarning Using @Deprecated Class
org.eclipse.jetty.servlet.listener.ELContextCleaner
2025-01-25 11:34:18.775 INFO (main) [c: s: r: x: t:]
o.a.s.s.CoreContainerProvider Using logger factory
org.apache.logging.slf4j.Log4jLoggerFactory
(More Entries Follow)
[solr.log.10] (There is only one entry, indicating failed start attempt by nssm)
2025-01-25 11:34:11.534 INFO (main) [c: s: r: x: t:] o.e.j.s.Server
jetty-10.0.22; built: 2024-06-27T16:03:51.502Z; git:
5c8471e852d377fd726ad9b1692c35ffc5febb09; jvm 17.0.13+11
[solr.log.9]
2025-01-24 11:34:14.861 INFO (main) [c: s: r: x: t:] o.e.j.s.Server
jetty-10.0.22; built: 2024-06-27T16:03:51.502Z; git:
5c8471e852d377fd726ad9b1692c35ffc5febb09; jvm 17.0.13+11
2025-01-24 11:34:15.317 WARN (main) [c: s: r: x: t:]
o.e.j.u.DeprecationWarning Using @Deprecated Class
org.eclipse.jetty.servlet.listener.ELContextCleaner
2025-01-24 11:34:15.345 INFO (main) [c: s: r: x: t:]
o.a.s.s.CoreContainerProvider Using logger factory
org.apache.logging.slf4j.Log4jLoggerFactory
[more entries]
2025-01-25 11:23:24.717 INFO
(searcherExecutor-17-thread-34-processing-xxxxx null-1445) [c: s: r:
x:xxxxx t:null-1445] o.a.s.c.SolrCore Registered new searcher autowarm
time: 0 ms
2025-01-25 11:23:24.721 INFO (qtp948250363-142-null-1445) [c: s: r:
x:xxxxx t:null-1445] o.a.s.u.p.LogUpdateProcessorFactory webapp=/solr
path=/update params={_ttl_=%2B40DAYS&commit=true}{add=[164862/328/69
(1822219781590220800)], commit=} 0 171
2025-01-25 11:32:44.874 INFO (qtp948250363-153-null-1446) [c: s: r:
x:xxxxx t:null-1446] o.a.s.c.S.Request webapp=/solr path=/get
params={id=127369/10104/17} status=0 QTime=0
[solr.log.8]
2025-01-24 11:34:10.131 INFO (main) [c: s: r: x: t:] o.e.j.s.Server
jetty-10.0.22; built: 2024-06-27T16:03:51.502Z; git:
5c8471e852d377fd726ad9b1692c35ffc5febb09; jvm 17.0.13+11
2025-01-24 11:34:10.571 WARN (main) [c: s: r: x: t:]
o.e.j.u.DeprecationWarning Using @Deprecated Class
org.eclipse.jetty.servlet.listener.ELContextCleaner
2025-01-24 11:34:10.599 INFO (main) [c: s: r: x: t:]
o.a.s.s.CoreContainerProvider Using logger factory
org.apache.logging.slf4j.Log4jLoggerFactory
[more entries]
2025-01-24 11:34:13.123 INFO (main) [c: s: r: x: t:]
o.e.j.s.h.ContextHandler Started
o.e.j.w.WebAppContext@797501a{solr-jetty-context.xml,/solr,file:///C:/solr-9.7.0/server/solr-webapp/webapp/,AVAILABLE}{C:\solr-9.7.0\server/solr-webapp/webapp}
2025-01-24 11:34:13.129 INFO (main) [c: s: r: x: t:]
o.e.j.s.RequestLogWriter Opened
C:\solr-9.7.0\server\logs\2025_01_24.request.log
2025-01-24 11:34:13.139 INFO (main) [c: s: r: x: t:]
o.e.j.s.AbstractConnector Started ServerConnector@20b5f2ac{HTTP/1.1,
(http/1.1, h2c)}{0.0.0.0:8983}
2025-01-24 11:34:13.140 INFO (main) [c: s: r: x: t:] o.e.j.s.Server
Started Server@5241cf67{STARTING}[10.0.22,sto=0] @4135ms
[solr.log.7]
2025-01-24 10:53:21.627 INFO (main) [c: s: r: x: t:] o.e.j.s.Server
jetty-10.0.22; built: 2024-06-27T16:03:51.502Z; git:
5c8471e852d377fd726ad9b1692c35ffc5febb09; jvm 17.0.13+11
2025-01-24 10:53:26.840 WARN (main) [c: s: r: x: t:]
o.e.j.u.DeprecationWarning Using @Deprecated Class
org.eclipse.jetty.servlet.listener.ELContextCleaner
2025-01-24 10:53:26.868 INFO (main) [c: s: r: x: t:]
o.a.s.s.CoreContainerProvider Using logger factory
org.apache.logging.slf4j.Log4jLoggerFactory
[more entries]
2025-01-24 11:34:06.422 INFO (JettyShutdownThread) [c: s: r: x: t:]
o.e.j.s.Server Stopped Server@5241cf67{STOPPING}[10.0.22,sto=0]
2025-01-24 11:34:06.424 INFO (JettyShutdownThread) [c: s: r: x: t:]
o.e.j.s.AbstractConnector Stopped ServerConnector@20b5f2ac{HTTP/1.1,
(http/1.1, h2c)}{0.0.0.0:8983}
2025-01-24 11:34:06.432 INFO (JettyShutdownThread) [c: s: r: x: t:]
o.a.s.c.CoreContainer Shutting down CoreContainer instance=1390430007
2025-01-24 11:34:06.433 INFO (coreCloseExecutor-28-thread-1) [c: s:
r: x:xxxxx t:] o.a.s.c.SolrCore CLOSING SolrCore
org.apache.solr.core.SolrCore@65296d7f xxxxx
2025-01-24 11:34:06.434 INFO (coreCloseExecutor-28-thread-2) [c: s:
r: x:xxxxx t:] o.a.s.c.SolrCore CLOSING SolrCore
org.apache.solr.core.SolrCore@760dbf95 xxxxx
2025-01-24 11:34:06.434 INFO (coreCloseExecutor-28-thread-1) [c: s:
r: x:xxxxx t:] o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.core.xxxxx tag=SolrCore@65296d7f
[solr.log.6] (This doesn't fit the pattern, might be a manuel restart
to change settings)
2025-01-23 14:46:14.068 INFO (main) [c: s: r: x: t:] o.e.j.s.Server
jetty-10.0.22; built: 2024-06-27T16:03:51.502Z; git:
5c8471e852d377fd726ad9b1692c35ffc5febb09; jvm 17.0.13+11
2025-01-23 14:46:19.287 WARN (main) [c: s: r: x: t:]
o.e.j.u.DeprecationWarning Using @Deprecated Class
org.eclipse.jetty.servlet.listener.ELContextCleaner
2025-01-23 14:46:19.315 INFO (main) [c: s: r: x: t:]
o.a.s.s.CoreContainerProvider Using logger factory
org.apache.logging.slf4j.Log4jLoggerFactory
[more entries]
2025-01-24 10:53:15.574 INFO (JettyShutdownThread) [c: s: r: x: t:]
o.e.j.s.Server Stopped Server@5241cf67{STOPPING}[10.0.22,sto=0]
2025-01-24 10:53:15.576 INFO (JettyShutdownThread) [c: s: r: x: t:]
o.e.j.s.AbstractConnector Stopped ServerConnector@20b5f2ac{HTTP/1.1,
(http/1.1, h2c)}{0.0.0.0:8983}
2025-01-24 10:53:15.579 INFO (JettyShutdownThread) [c: s: r: x: t:]
o.a.s.c.CoreContainer Shutting down CoreContainer instance=1390430007
2025-01-24 10:53:15.581 INFO (coreCloseExecutor-40-thread-1) [c: s:
r: x:xxxxx t:] o.a.s.c.SolrCore CLOSING SolrCore
org.apache.solr.core.SolrCore@395fb567 xxxxx
2025-01-24 10:53:15.581 INFO (coreCloseExecutor-40-thread-2) [c: s:
r: x:xxxxx t:] o.a.s.c.SolrCore CLOSING SolrCore
org.apache.solr.core.SolrCore@697eeec4 xxxxx
2025-01-24 10:53:15.581 INFO (coreCloseExecutor-40-thread-1) [c: s:
r: x:xxxxx t:] o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.core.xxxxx tag=SolrCore@395fb567
2025-01-24 10:53:15.582 INFO (coreCloseExecutor-40-thread-2) [c: s:
r: x:xxxxx t:] o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.core.xxxxx tag=SolrCore@697eeec4
2025-01-24 10:53:15.583 INFO (coreCloseExecutor-40-thread-2) [c: s:
r: x:xxxxx t:] o.a.s.u.p.DocExpirationUpdateProcessorFactory
Triggering Graceful close of DocExpiration Executor
2025-01-24 10:53:15.585 INFO (coreCloseExecutor-40-thread-1) [c: s:
r: x:xxxxx t:] o.a.s.u.DirectUpdateHandler2 Committing on
IndexWriter.close() ... SKIPPED (unnecessary).
2025-01-24 10:53:15.585 INFO (coreCloseExecutor-40-thread-2) [c: s:
r: x:xxxxx t:] o.a.s.u.DirectUpdateHandler2 Committing on
IndexWriter.close() ... SKIPPED (unnecessary).
2025-01-24 10:53:15.588 INFO (JettyShutdownThread) [c: s: r: x: t:]
o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.node tag=null
2025-01-24 10:53:15.588 INFO (JettyShutdownThread) [c: s: r: x: t:]
o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.jvm tag=null
2025-01-24 10:53:15.589 INFO (JettyShutdownThread) [c: s: r: x: t:]
o.a.s.m.SolrMetricManager Closing metric reporters for
registry=solr.jetty tag=null
2025-01-24 10:53:15.608 INFO (JettyShutdownThread) [c: s: r: x: t:]
o.e.j.s.h.ContextHandler Stopped
o.e.j.w.WebAppContext@797501a{solr-jetty-context.xml,/solr,null,STOPPED}{C:\solr-9.7.0\server/solr-webapp/webapp}
[solr.log.5]
2025-01-23 11:34:14.687 INFO (main) [c: s: r: x: t:] o.e.j.s.Server
jetty-10.0.22; built: 2024-06-27T16:03:51.502Z; git:
5c8471e852d377fd726ad9b1692c35ffc5febb09; jvm 17.0.13+11
2025-01-23 11:34:19.838 WARN (main) [c: s: r: x: t:]
o.e.j.u.DeprecationWarning Using @Deprecated Class
org.eclipse.jetty.servlet.listener.ELContextCleaner
2025-01-23 11:34:19.867 INFO (main) [c: s: r: x: t:]
o.a.s.s.CoreContainerProvider Using logger factory
org.apache.logging.slf4j.Log4jLoggerFactory
[more entries]
2025-01-23 14:44:17.913 INFO (qtp948250363-76-null-286) [c: s: r: x:
t:null-286] o.a.s.s.HttpSolrCall [admin] webapp=null path=/admin/cores
params={indexInfo=false&wt=json&_=1737643390088} status=0 QTime=0
2025-01-23 14:44:17.921 INFO (qtp948250363-26-null-287) [c: s: r: x:
t:null-287] o.a.s.s.HttpSolrCall [admin] webapp=null
path=/admin/info/system params={wt=json&_=1737643390088} status=0
QTime=8
2025-01-23 14:44:17.930 INFO (qtp948250363-65-null-288) [c: s: r: x:
t:null-288] o.a.s.s.HttpSolrCall [admin] webapp=null
path=/admin/info/system params={wt=json&_=1737643390088} status=0
QTime=7
