[jira] [Commented] (NIFI-4208) Node failed to join cluster due to NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127448#comment-16127448 ] ASF GitHub Bot commented on NIFI-4208: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2086 > Node failed to join cluster due to NullPointerException > --- > > Key: NIFI-4208 > URL: https://issues.apache.org/jira/browse/NIFI-4208 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Critical > Fix For: 1.4.0 > > > A clustered node ran out of disk space. Upon restart, I came across the > following error: > 2017-07-20 09:03:00,988 ERROR [main] o.a.nifi.controller.StandardFlowService > Failed to load flow from cluster due to: > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:945) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:515) > at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:800) > at org.apache.nifi.NiFi.(NiFi.java:160) > at org.apache.nifi.NiFi.main(NiFi.java:267) > Caused by: java.lang.NullPointerException: null > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:43) > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:26) > at > org.wali.MinimalLockingWriteAheadLog$Partition.recoverNextTransaction(MinimalLockingWriteAheadLog.java:1132) > at > org.wali.MinimalLockingWriteAheadLog.recoverFromEdits(MinimalLockingWriteAheadLog.java:459) > at > org.wali.MinimalLockingWriteAheadLog.recoverRecords(MinimalLockingWriteAheadLog.java:301) > at > org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.loadFlowFiles(WriteAheadFlowFileRepository.java:381) > at > org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:713) > at > org.apache.nifi.controller.StandardFlowService.initializeController(StandardFlowService.java:955) > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:927) > ... 4 common frames omitted -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4208) Node failed to join cluster due to NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127447#comment-16127447 ] ASF subversion and git services commented on NIFI-4208: --- Commit abca9d1464fe0d6a5deefb28260465a310a97bd4 in nifi's branch refs/heads/master from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=abca9d1 ] NIFI-4208: Fixed bug in SchemaRepositoryRecordSerde that would return null from deserializeEdit if there was no data; the interface documents that null cannot be returned from this method, as it is called only when data is expected to exist. As a result, if there is no data, we should throw EOFException instead, and the write-ahead log will handle that appropriately. This closes #2086. Signed-off-by: Andy LoPresto> Node failed to join cluster due to NullPointerException > --- > > Key: NIFI-4208 > URL: https://issues.apache.org/jira/browse/NIFI-4208 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Critical > Fix For: 1.4.0 > > > A clustered node ran out of disk space. Upon restart, I came across the > following error: > 2017-07-20 09:03:00,988 ERROR [main] o.a.nifi.controller.StandardFlowService > Failed to load flow from cluster due to: > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:945) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:515) > at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:800) > at org.apache.nifi.NiFi.(NiFi.java:160) > at org.apache.nifi.NiFi.main(NiFi.java:267) > Caused by: java.lang.NullPointerException: null > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:43) > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:26) > at > org.wali.MinimalLockingWriteAheadLog$Partition.recoverNextTransaction(MinimalLockingWriteAheadLog.java:1132) > at > org.wali.MinimalLockingWriteAheadLog.recoverFromEdits(MinimalLockingWriteAheadLog.java:459) > at > org.wali.MinimalLockingWriteAheadLog.recoverRecords(MinimalLockingWriteAheadLog.java:301) > at > org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.loadFlowFiles(WriteAheadFlowFileRepository.java:381) > at > org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:713) > at > org.apache.nifi.controller.StandardFlowService.initializeController(StandardFlowService.java:955) > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:927) > ... 4 common frames omitted -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4208) Node failed to join cluster due to NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127446#comment-16127446 ] ASF GitHub Bot commented on NIFI-4208: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2086 I verified the new unit test passed successfully. I then started the app normally, ran a heavy flow, and debugged at the `checkpoint()` method to ensure I captured a valid `snapshot` file. I modified this by truncating the file after a transaction ID, yielding an incomplete record. I stopped the application, moved the modified `snapshot` into place, and restarted the app. On loading, NiFi stopped with the following expected error in the `nifi-app.log`: ``` org.apache.nifi.web.NiFiCoreException: Unable to start Flow Controller. at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:88) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:876) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532) at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:839) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1480) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1442) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:799) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:540) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:290) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131) at org.eclipse.jetty.server.Server.start(Server.java:452) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) at org.eclipse.jetty.server.Server.doStart(Server.java:419) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:707) at org.apache.nifi.NiFi.(NiFi.java:160) at org.apache.nifi.NiFi.main(NiFi.java:268) Caused by: java.io.EOFException: Failed to read field 'Repository Record Update' at org.apache.nifi.repository.schema.SchemaRecordReader.readField(SchemaRecordReader.java:115) at org.apache.nifi.repository.schema.SchemaRecordReader.readRecord(SchemaRecordReader.java:72) at org.apache.nifi.controller.repository.SchemaRepositoryRecordSerde.deserializeRecord(SchemaRepositoryRecordSerde.java:124) at org.apache.nifi.controller.repository.SchemaRepositoryRecordSerde.deserializeRecord(SchemaRepositoryRecordSerde.java:46) at org.wali.MinimalLockingWriteAheadLog.recoverFromSnapshot(MinimalLockingWriteAheadLog.java:380) at org.wali.MinimalLockingWriteAheadLog.recoverRecords(MinimalLockingWriteAheadLog.java:300) at org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.loadFlowFiles(WriteAheadFlowFileRepository.java:381) at org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:723) at org.apache.nifi.controller.StandardFlowService.initializeController(StandardFlowService.java:956) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:535) at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) ... 28 common frames omitted Suppressed:
[jira] [Commented] (NIFI-4208) Node failed to join cluster due to NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127326#comment-16127326 ] ASF GitHub Bot commented on NIFI-4208: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/2086 Reviewing... > Node failed to join cluster due to NullPointerException > --- > > Key: NIFI-4208 > URL: https://issues.apache.org/jira/browse/NIFI-4208 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Critical > Fix For: 1.4.0 > > > A clustered node ran out of disk space. Upon restart, I came across the > following error: > 2017-07-20 09:03:00,988 ERROR [main] o.a.nifi.controller.StandardFlowService > Failed to load flow from cluster due to: > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:945) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:515) > at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:800) > at org.apache.nifi.NiFi.(NiFi.java:160) > at org.apache.nifi.NiFi.main(NiFi.java:267) > Caused by: java.lang.NullPointerException: null > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:43) > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:26) > at > org.wali.MinimalLockingWriteAheadLog$Partition.recoverNextTransaction(MinimalLockingWriteAheadLog.java:1132) > at > org.wali.MinimalLockingWriteAheadLog.recoverFromEdits(MinimalLockingWriteAheadLog.java:459) > at > org.wali.MinimalLockingWriteAheadLog.recoverRecords(MinimalLockingWriteAheadLog.java:301) > at > org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.loadFlowFiles(WriteAheadFlowFileRepository.java:381) > at > org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:713) > at > org.apache.nifi.controller.StandardFlowService.initializeController(StandardFlowService.java:955) > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:927) > ... 4 common frames omitted -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4208) Node failed to join cluster due to NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127321#comment-16127321 ] Mark Payne commented on NIFI-4208: -- [~joewitt] - sorry, just figured out what was going on here. Looks like we were returning a null when the interface documents that null is not allowed and instead we should throw EOFException (because we've already found some data for the record). Posted a PR to address. The nice thing is that the bug affects reading the data, not writing it. As a result, if a user runs into this issue, the patch should allow the NiFi instance to startup properly, once applied. > Node failed to join cluster due to NullPointerException > --- > > Key: NIFI-4208 > URL: https://issues.apache.org/jira/browse/NIFI-4208 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Critical > Fix For: 1.4.0 > > > A clustered node ran out of disk space. Upon restart, I came across the > following error: > 2017-07-20 09:03:00,988 ERROR [main] o.a.nifi.controller.StandardFlowService > Failed to load flow from cluster due to: > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:945) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:515) > at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:800) > at org.apache.nifi.NiFi.(NiFi.java:160) > at org.apache.nifi.NiFi.main(NiFi.java:267) > Caused by: java.lang.NullPointerException: null > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:43) > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:26) > at > org.wali.MinimalLockingWriteAheadLog$Partition.recoverNextTransaction(MinimalLockingWriteAheadLog.java:1132) > at > org.wali.MinimalLockingWriteAheadLog.recoverFromEdits(MinimalLockingWriteAheadLog.java:459) > at > org.wali.MinimalLockingWriteAheadLog.recoverRecords(MinimalLockingWriteAheadLog.java:301) > at > org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.loadFlowFiles(WriteAheadFlowFileRepository.java:381) > at > org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:713) > at > org.apache.nifi.controller.StandardFlowService.initializeController(StandardFlowService.java:955) > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:927) > ... 4 common frames omitted -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4208) Node failed to join cluster due to NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127314#comment-16127314 ] ASF GitHub Bot commented on NIFI-4208: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2086 NIFI-4208: Fixed bug in SchemaRepositoryRecordSerde that would return… … null from deserializeEdit if there was no data; the interface documents that null cannot be returned from this method, as it is called only when data is expected to exist. As a result, if there is no data, we should throw EOFException instead, and the write-ahead log will handle that appropriately Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4208 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2086.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2086 commit a8cf805381489b35a56062a0cdf52eea6fa4fb76 Author: Mark PayneDate: 2017-08-15T14:39:46Z NIFI-4208: Fixed bug in SchemaRepositoryRecordSerde that would return null from deserializeEdit if there was no data; the interface documents that null cannot be returned from this method, as it is called only when data is expected to exist. As a result, if there is no data, we should throw EOFException instead, and the write-ahead log will handle that appropriately > Node failed to join cluster due to NullPointerException > --- > > Key: NIFI-4208 > URL: https://issues.apache.org/jira/browse/NIFI-4208 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Critical > Fix For: 1.4.0 > > > A clustered node ran out of disk space. Upon restart, I came across the > following error: > 2017-07-20 09:03:00,988 ERROR [main] o.a.nifi.controller.StandardFlowService > Failed to load flow from cluster due to: > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:945) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:515) > at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:800) > at org.apache.nifi.NiFi.(NiFi.java:160) > at org.apache.nifi.NiFi.main(NiFi.java:267) > Caused by: java.lang.NullPointerException: null > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:43) > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:26) > at >
[jira] [Commented] (NIFI-4208) Node failed to join cluster due to NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-4208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16122601#comment-16122601 ] Joseph Witt commented on NIFI-4208: --- [~markap14] Any more thoughts/plans on this? I see it is tagged to 1.4.0 and said to be critical but no update since then. > Node failed to join cluster due to NullPointerException > --- > > Key: NIFI-4208 > URL: https://issues.apache.org/jira/browse/NIFI-4208 > Project: Apache NiFi > Issue Type: Bug >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Critical > Fix For: 1.4.0 > > > A clustered node ran out of disk space. Upon restart, I came across the > following error: > 2017-07-20 09:03:00,988 ERROR [main] o.a.nifi.controller.StandardFlowService > Failed to load flow from cluster due to: > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > org.apache.nifi.cluster.ConnectionException: Failed to connect node to > cluster due to: java.lang.NullPointerException > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:945) > at > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:515) > at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:800) > at org.apache.nifi.NiFi.(NiFi.java:160) > at org.apache.nifi.NiFi.main(NiFi.java:267) > Caused by: java.lang.NullPointerException: null > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:43) > at > org.apache.nifi.controller.repository.RepositoryRecordSerde.getRecordIdentifier(RepositoryRecordSerde.java:26) > at > org.wali.MinimalLockingWriteAheadLog$Partition.recoverNextTransaction(MinimalLockingWriteAheadLog.java:1132) > at > org.wali.MinimalLockingWriteAheadLog.recoverFromEdits(MinimalLockingWriteAheadLog.java:459) > at > org.wali.MinimalLockingWriteAheadLog.recoverRecords(MinimalLockingWriteAheadLog.java:301) > at > org.apache.nifi.controller.repository.WriteAheadFlowFileRepository.loadFlowFiles(WriteAheadFlowFileRepository.java:381) > at > org.apache.nifi.controller.FlowController.initializeFlow(FlowController.java:713) > at > org.apache.nifi.controller.StandardFlowService.initializeController(StandardFlowService.java:955) > at > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:927) > ... 4 common frames omitted -- This message was sent by Atlassian JIRA (v6.4.14#64029)