[
https://issues.apache.org/jira/browse/NIFI-3839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16001246#comment-16001246
]
Joseph Percivall commented on NIFI-3839:
----------------------------------------
Yup the old instance would have inherently been stopped since it was restarting
the docker container with the new image.
All configuration options and paths would have stayed the same but I don't
think the jetty working dir was something that was persisted to disk (so would
have been wiped when the docker container restarted).
Data was flowing and the only noticeable issue was not being able to access
provenance so doubt it was related to disk space, file permissions, etc.
> Problem accessing provenance initially when upgrading from 1.1.2 to 1.2.0
> -------------------------------------------------------------------------
>
> Key: NIFI-3839
> URL: https://issues.apache.org/jira/browse/NIFI-3839
> Project: Apache NiFi
> Issue Type: Bug
> Affects Versions: 1.2.0
> Environment: centos 7, docker instance, OpenJDK 1.8
> Reporter: Joseph Percivall
> Priority: Minor
>
> Upgraded a long running dev docker 1.1.2 instance to 1.2.0 RC1. Did not
> change any configuration. After leaving it running for a couple days, when I
> attempted to access provenance either by right-clicking the component or by
> accessing it through the hamburger menu I got a 500 error due to a NPE. I see
> the stacktrace in the window that should have the provenance events
> (stacktraces from nifi-app log below). Upgraded the version to RC2
> (restarting the container) and afterwards it worked.
> There are no tickets resolved in RC2 that appear to relate to this issue.
> One thing of note relating to provenance, there would have been events that
> expired between being started with 1.2.0-RC1 and the provenance being queried.
> The docker container was running the latest version of Centos7 with OpenJdk
> java 1.8.0 and the provenance volume was mounted (so it persisted through
> restarts). Also, the instance is secure with certs and I had the correct
> policies to access the provenance.
> I have not been able to reproduce and restarting the instance appeared to
> resolve the issue. I am logging an issue relating to the RC because I believe
> there may be an issue related to upgrading 1.1.2 -> 1.2.0 and provenance
> events expiring while keeping an old prov repo but is solved by restarting.
> For the stacktraces, I pulled a couple lines before/after the actual
> stacktrace in order to provide context and maybe some pertinent context.
> Stacktrace 1:
> {code}
> 2017-05-08 14:05:36,943 INFO [Provenance Maintenance Thread-2]
> o.a.n.p.lucene.UpdateMinimumEventId Updated Minimum Event ID for Provenance
> Event Repository - Minimum Event ID now 589858079
> 2017-05-08 14:05:36,943 INFO [Provenance Maintenance Thread-2]
> o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration
> Action org.apache.nifi.provenance.lucene.UpdateMinimumEventId@6340141b on
> Provenance Event file ./provenance_repository/589858075.prov.gz in 1 millis
> 2017-05-08 14:05:36,944 INFO [Provenance Maintenance Thread-2]
> o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Event file
> ./provenance_repository/589858075.prov.gz
> 2017-05-08 14:05:36,944 INFO [Provenance Maintenance Thread-2]
> o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Table-of-Contents
> file ./provenance_repository/toc/589858075.toc
> 2017-05-08 14:05:36,944 INFO [Provenance Maintenance Thread-2]
> o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration
> Action org.apache.nifi.provenance.expiration.FileRemovalAction@16a8507e on
> Provenance Event file ./provenance_repository/589858075.prov.gz in 402049
> nanos
> 2017-05-08 14:05:38,879 WARN [NiFi Web Server-18211]
> org.eclipse.jetty.server.HttpChannel /nifi/provenance
> javax.servlet.ServletException: org.eclipse.jetty.servlet.ServletHolder$1:
> java.lang.NullPointerException
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:138)
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:561)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:564)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
> at
> org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:258)
> at
> org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:147)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
> at
> org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:122)
> at
> org.eclipse.jetty.util.thread.strategy.ExecutingExecutionStrategy.invoke(ExecutingExecutionStrategy.java:58)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:201)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:133)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.eclipse.jetty.servlet.ServletHolder$1:
> java.lang.NullPointerException
> at
> org.eclipse.jetty.servlet.ServletHolder.makeUnavailable(ServletHolder.java:596)
> at
> org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:655)
> at
> org.eclipse.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:498)
> at
> org.eclipse.jetty.servlet.ServletHolder.ensureInstance(ServletHolder.java:785)
> at
> org.eclipse.jetty.servlet.ServletHolder.prepare(ServletHolder.java:770)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:538)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1593)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1239)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:481)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1141)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:118)
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:561)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:564)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
> at
> org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:258)
> at
> org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:147)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
> ... 3 common frames omitted
> Caused by: java.lang.NullPointerException: null
> at
> org.apache.jasper.servlet.JspServlet.handleMissingResource(JspServlet.java:397)
> at
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:387)
> at org.apache.jasper.servlet.JspServlet.init(JspServlet.java:138)
> at
> org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:637)
> ... 32 common frames omitted
> 2017-05-08 14:05:39,462 INFO [Provenance Maintenance Thread-1]
> o.a.n.p.lucene.UpdateMinimumEventId Updated Minimum Event ID for Provenance
> Event Repository - Minimum Event ID now 589858259
> 2017-05-08 14:05:39,462 INFO [Provenance Maintenance Thread-1]
> o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration
> Action org.apache.nifi.provenance.lucene.UpdateMinimumEventId@6340141b on
> Provenance Event file ./provenance_repository/589858080.prov.gz in 10 millis
> {code}
> Stacktrace 2:
> {code}
> 2017-05-08 14:07:21,100 INFO [Provenance Maintenance Thread-2]
> o.a.n.p.lucene.UpdateMinimumEventId Updated Minimum Event ID for Provenance
> Event Repository - Minimum Event ID now 589873378
> 2017-05-08 14:07:21,100 INFO [Provenance Maintenance Thread-2]
> o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration
> Action org.apache.nifi.provenance.lucene.UpdateMinimumEventId@6340141b on
> Provenance Event file ./provenance_repository/589873199.prov.gz in 11 millis
> 2017-05-08 14:07:21,101 INFO [Provenance Maintenance Thread-2]
> o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Event file
> ./provenance_repository/589873199.prov.gz
> 2017-05-08 14:07:21,101 INFO [Provenance Maintenance Thread-2]
> o.a.n.p.lucene.DeleteIndexAction Removed expired Provenance Table-of-Contents
> file ./provenance_repository/toc/589873199.toc
> 2017-05-08 14:07:21,101 INFO [Provenance Maintenance Thread-2]
> o.a.n.p.PersistentProvenanceRepository Successfully performed Expiration
> Action org.apache.nifi.provenance.expiration.FileRemovalAction@16a8507e on
> Provenance Event file ./provenance_repository/589873199.prov.gz in 562266
> nanos
> 2017-05-08 14:07:21,379 WARN [NiFi Web Server-18263]
> org.eclipse.jetty.server.HttpChannel /nifi/provenance
> javax.servlet.ServletException: org.eclipse.jetty.servlet.ServletHolder$1:
> java.lang.NullPointerException
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:138)
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:561)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:564)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
> at
> org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:258)
> at
> org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:147)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
> at
> org.eclipse.jetty.util.thread.Invocable.invokePreferred(Invocable.java:122)
> at
> org.eclipse.jetty.util.thread.strategy.ExecutingExecutionStrategy.invoke(ExecutingExecutionStrategy.java:58)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:201)
> at
> org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:133)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:672)
> at
> org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:590)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: org.eclipse.jetty.servlet.ServletHolder$1:
> java.lang.NullPointerException
> at
> org.eclipse.jetty.servlet.ServletHolder.makeUnavailable(ServletHolder.java:596)
> at
> org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:655)
> at
> org.eclipse.jetty.servlet.ServletHolder.getServlet(ServletHolder.java:498)
> at
> org.eclipse.jetty.servlet.ServletHolder.ensureInstance(ServletHolder.java:785)
> at
> org.eclipse.jetty.servlet.ServletHolder.prepare(ServletHolder.java:770)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:538)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
> at
> org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
> at
> org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1593)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1239)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
> at
> org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:481)
> at
> org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1562)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1141)
> at
> org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
> at
> org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:118)
> at
> org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:561)
> at
> org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
> at org.eclipse.jetty.server.Server.handle(Server.java:564)
> at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
> at
> org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
> at
> org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
> at
> org.eclipse.jetty.io.ssl.SslConnection.onFillable(SslConnection.java:258)
> at
> org.eclipse.jetty.io.ssl.SslConnection$3.succeeded(SslConnection.java:147)
> at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:110)
> at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
> ... 3 common frames omitted
> Caused by: java.lang.NullPointerException: null
> at
> org.apache.jasper.servlet.JspServlet.handleMissingResource(JspServlet.java:397)
> at
> org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:387)
> at org.apache.jasper.servlet.JspServlet.init(JspServlet.java:138)
> at
> org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:637)
> ... 32 common frames omitted
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
