Joe, We will be setting up partitions for these things - it makes sense. However, it there a way to configure max space usage? I would like nifi to monitor it's upper bound so that stuff doesn't get corrupted when the disk fills up.
Thanks, Mike. On Tue, Apr 4, 2017 at 3:20 PM, Joe Witt <joe.w...@gmail.com> wrote: > Mike > > It looks like the flow file repo has partial written data at the end and > is problematic on restart. Strongly recommend configuring per best > practices and making sure the flow file repo in particular is in its own > logical partition. Also, we can make that more resilient on startup and > there is a jira. > > On Apr 4, 2017 5:36 PM, "Mikhail Sosonkin" <mikh...@synack.com> wrote: > >> Hello Nifi: >> >> Has anyone face this issue? This happened after I tried restarting nifi >> after it filled up the disk. >> >> Thanks, >> Mike. >> >> 2017-04-04 21:29:46,490 WARN [main] org.apache.nifi.web.server.JettyServer >> Failed to start web server... shutting down. >> java.lang.NullPointerException: null >> at org.apache.nifi.controller.repository.SchemaRepositoryRecord >> Serde.deserializeRecord(SchemaRepositoryRecordSerde.java:115) >> ~[nifi-framework-core-1.1.1.jar:1.1.1] >> at org.apache.nifi.controller.repository.SchemaRepositoryRecord >> Serde.deserializeEdit(SchemaRepositoryRecordSerde.java:105) >> ~[nifi-framework-core-1.1.1.jar:1.1.1] >> at org.apache.nifi.controller.repository.SchemaRepositoryRecord >> Serde.deserializeEdit(SchemaRepositoryRecordSerde.java:46) >> ~[nifi-framework-core-1.1.1.jar:1.1.1] >> at org.wali.MinimalLockingWriteAheadLog$Partition.recoverNextTr >> ansaction(MinimalLockingWriteAheadLog.java:1072) >> ~[nifi-write-ahead-log-1.1.1.jar:1.1.1] >> at org.wali.MinimalLockingWriteAheadLog.recoverFromEdits(Minima >> lLockingWriteAheadLog.java:459) ~[nifi-write-ahead-log-1.1.1.jar:1.1.1] >> at >> org.wali.MinimalLockingWriteAheadLog.recoverRecords(MinimalLockingWriteAheadLog.java:301) >> ~[nifi-write-ahead-log-1.1.1.jar:1.1.1] >> at org.apache.nifi.controller.repository.WriteAheadFlowFileRepo >> sitory.loadFlowFiles(WriteAheadFlowFileRepository.java:346) >> ~[nifi-framework-core-1.1.1.jar:1.1.1] >> at >> org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:700) >> ~[nifi-framework-core-1.1.1.jar:1.1.1] >> at >> org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:701) >> ~[nifi-framework-core-1.1.1.jar:1.1.1] >> at >> org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:508) >> ~[nifi-framework-core-1.1.1.jar:1.1.1] >> at org.apache.nifi.web.contextlistener.ApplicationStartupContex >> tListener.contextInitialized(ApplicationStartupContextListener.java:69) >> ~[na:na] >> at org.eclipse.jetty.server.handler.ContextHandler.callContextI >> nitialized(ContextHandler.java:837) ~[jetty-server-9.3.9.v20160517 >> .jar:9.3.9.v20160517] >> at org.eclipse.jetty.servlet.ServletContextHandler.callContextI >> nitialized(ServletContextHandler.java:533) ~[jetty-servlet-9.3.9.v2016051 >> 7.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:810) >> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] >> at org.eclipse.jetty.servlet.ServletContextHandler.startContext >> (ServletContextHandler.java:345) ~[jetty-servlet-9.3.9.v2016051 >> 7.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1404) >> ~[jetty-webapp-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1366) >> ~[jetty-webapp-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:772) >> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:262) >> ~[jetty-servlet-9.3.9.v20160517.jar:9.3.9.v20160517] >> at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:520) >> ~[jetty-webapp-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) >> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) >> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114) >> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) >> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) >> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) >> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106) >> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) >> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:231) >> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) >> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132) >> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] >> at org.eclipse.jetty.server.Server.start(Server.java:411) >> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:106) >> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61) >> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] >> at org.eclipse.jetty.server.Server.doStart(Server.java:378) >> ~[jetty-server-9.3.9.v20160517.jar:9.3.9.v20160517] >> at >> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) >> ~[jetty-util-9.3.9.v20160517.jar:9.3.9.v20160517] >> at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:675) >> ~[nifi-jetty-1.1.1.jar:1.1.1] >> at org.apache.nifi.NiFi.<init>(NiFi.java:156) >> [nifi-runtime-1.1.1.jar:1.1.1] >> at org.apache.nifi.NiFi.main(NiFi.java:262) >> [nifi-runtime-1.1.1.jar:1.1.1] >> 2017-04-04 21:29:46,491 INFO [Thread-1] org.apache.nifi.NiFi Initiating >> shutdown of Jetty web server... >> >> >> >> This email may contain material that is confidential for the sole use of >> the intended recipient(s). Any review, reliance or distribution or >> disclosure by others without express permission is strictly prohibited. If >> you are not the intended recipient, please contact the sender and delete >> all copies of this message. >> > -- This email may contain material that is confidential for the sole use of the intended recipient(s). Any review, reliance or distribution or disclosure by others without express permission is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies of this message.
