Hi, As a plan for tomorrow – I have generated new keystores, truststores, client certts etc. for all nodes in my cluster using the
From: Bryan Bende <bbe...@gmail.com> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org> Date: Wednesday, 19 October 2016 at 15:33 To: "users@nifi.apache.org" <users@nifi.apache.org> Subject: Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups Trying to think of things to check here... Does every node have nifi.remote.input.secure=true in nifi.properties and the URL in the RPG is an https URL? On Wed, Oct 19, 2016 at 10:25 AM, Conrad Crampton <conrad.cramp...@secdata.com<mailto:conrad.cramp...@secdata.com>> wrote: One other thing… The RPGs have an unlocked padlock on them saying S2S is not secure. Conrad From: Bryan Bende <bbe...@gmail.com<mailto:bbe...@gmail.com>> Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Wednesday, 19 October 2016 at 15:20 To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups Ok that does seem like a TLS/SSL issue... Is this a single cluster doing site-to-site to itself? On Wed, Oct 19, 2016 at 10:06 AM, Joe Witt <joe.w...@gmail.com<mailto:joe.w...@gmail.com>> wrote: thanks conrad - did get it. Bryan is being more helpful that I so I went silent :-) On Wed, Oct 19, 2016 at 10:02 AM, Conrad Crampton <conrad.cramp...@secdata.com<mailto:conrad.cramp...@secdata.com>> wrote: > Hi Joe, > Yep, > Tried removing the RPG that referenced the NCM and adding new one with > one of the datanodes as url. > That sort of worked, but kept getting errors about the NCM not being > available for the ports and therefore couldn’t actually enable the port I > needed to for that RPG. > Thanks > Conrad > > (sending again as don’t know if the stupid header ‘spoofed’ is stopping > getting though – apologies if already sent) > > On 19/10/2016, 14:12, "Joe Witt" > <joe.w...@gmail.com<mailto:joe.w...@gmail.com>> wrote: > > Conrad, > > For s2s now you can just point at any of the nodes in the cluster. > Have you tried changing the URL or removing and adding new RPG > entries? > > Thanks > Joe > > On Wed, Oct 19, 2016 at 8:38 AM, Conrad Crampton > <conrad.cramp...@secdata.com<mailto:conrad.cramp...@secdata.com>> > wrote: > > Hi, > > > > I have finally taken the plunge to upgrade my cluster from 0.6.1 to > 1.0.0. > > > > 6 nodes with a NCM. > > > > With the removal of NCM in 1.0.0 I believe I now have an issue > where none of > > my Remote Process Groups work as they previously did because they > were > > configured to connect to the NCM (as the RPG url) which now doesn’t > exist. > > > > I have tried converting my NCM to a node but whilst I can get it > running > > (sort of) when I try and connect to the cluster I get something > like this in > > my logs… > > > > > > > > 2016-10-19 13:14:44,109 ERROR [main] > o.a.nifi.controller.StandardFlowService > > Failed to load flow from cluster due to: > > org.apache.nifi.controller.UninheritableFlowException: Failed to > connect > > node to cluster because local flow is different than cluster flow. > > > > org.apache.nifi.controller.UninheritableFlowException: Failed to > connect > > node to cluster because local flow is different than cluster flow. > > > > at > > > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:879) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:493) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:746) > > [nifi-jetty-1.0.0.jar:1.0.0] > > > > at org.apache.nifi.NiFi.<init>(NiFi.java:152) > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > at org.apache.nifi.NiFi.main(NiFi.java:243) > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > Caused by: org.apache.nifi.controller.UninheritableFlowException: > Proposed > > Authorizer is not inheritable by the flow controller because of > Authorizer > > differences: Proposed Authorizations do not match current > Authorizations > > > > at > > > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:252) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1435) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:83) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:671) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:857) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > ... 4 common frames omitted > > > > 2016-10-19 13:14:44,414 ERROR [main] > o.a.n.c.c.node.NodeClusterCoordinator > > Event Reported for ncm-cm1.mis-cds.local:9090 -- Node disconnected > from > > cluster due to > org.apache.nifi.controller.UninheritableFlowException: Failed > > to connect node to cluster because local flow is different than > cluster > > flow. > > > > 2016-10-19 13:14:44,420 ERROR [Shutdown Cluster Coordinator] > > org.apache.nifi.NiFi An Unknown Error Occurred in Thread > Thread[Shutdown > > Cluster Coordinator,5,main]: java.lang.NullPointerException > > > > 2016-10-19 13:14:44,423 ERROR [Shutdown Cluster Coordinator] > > org.apache.nifi.NiFi > > > > java.lang.NullPointerException: null > > > > at > > > java.util.concurrent.ConcurrentHashMap.putVal(ConcurrentHashMap.java:1011) > > ~[na:1.8.0_51] > > > > at > > > java.util.concurrent.ConcurrentHashMap.put(ConcurrentHashMap.java:1006) > > ~[na:1.8.0_51] > > > > at > > > org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.updateNodeStatus(NodeClusterCoordinator.java:570) > > ~[nifi-framework-cluster-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.cluster.coordination.node.NodeClusterCoordinator.shutdown(NodeClusterCoordinator.java:119) > > ~[nifi-framework-cluster-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.controller.StandardFlowService$1.run(StandardFlowService.java:330) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_51] > > > > 2016-10-19 13:14:44,448 WARN [main] > o.a.n.c.l.e.CuratorLeaderElectionManager > > Failed to close Leader Selector for Cluster Coordinator > > > > java.lang.IllegalStateException: Already closed or has not been > started > > > > at > > > com.google.common.base.Preconditions.checkState(Preconditions.java:173) > > ~[guava-18.0.jar:na] > > > > at > > > org.apache.curator.framework.recipes.leader.LeaderSelector.close(LeaderSelector.java:270) > > ~[curator-recipes-2.11.0.jar:na] > > > > at > > > org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.stop(CuratorLeaderElectionManager.java:159) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.controller.FlowController.shutdown(FlowController.java:1303) > > [nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.controller.StandardFlowService.stop(StandardFlowService.java:339) > > [nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:753) > > [nifi-jetty-1.0.0.jar:1.0.0] > > > > at org.apache.nifi.NiFi.<init>(NiFi.java:152) > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > at org.apache.nifi.NiFi.main(NiFi.java:243) > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > 2016-10-19 13:14:45,062 WARN [Cluster Socket Listener] > > org.apache.nifi.io.socket.SocketListener Failed to communicate with > Unknown > > Host due to java.net.SocketException: Socket closed > > > > java.net.SocketException: Socket closed > > > > at java.net.PlainSocketImpl.socketAccept(Native Method) > > ~[na:1.8.0_51] > > > > at > > > java.net<http://java.net>.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:404) > > ~[na:1.8.0_51] > > > > at java.net.ServerSocket.implAccept(ServerSocket.java:545) > > ~[na:1.8.0_51] > > > > at > > > sun.security.ssl.SSLServerSocketImpl.accept(SSLServerSocketImpl.java:348) > > ~[na:1.8.0_51] > > > > at > > > org.apache.nifi.io.socket.SocketListener$2.run(SocketListener.java:112) > > ~[nifi-socket-utils-1.0.0.jar:1.0.0] > > > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51] > > > > 2016-10-19 13:14:45,064 WARN [main] > org.apache.nifi.web.server.JettyServer > > Failed to start web server... shutting down. > > > > java.lang.Exception: Unable to load flow due to: > java.io.IOException: > > org.apache.nifi.controller.UninheritableFlowException: Failed to > connect > > node to cluster because local flow is different than cluster flow. > > > > at > > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:755) > > ~[nifi-jetty-1.0.0.jar:1.0.0] > > > > at org.apache.nifi.NiFi.<init>(NiFi.java:152) > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > at org.apache.nifi.NiFi.main(NiFi.java:243) > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > Caused by: java.io.IOException: > > org.apache.nifi.controller.UninheritableFlowException: Failed to > connect > > node to cluster because local flow is different than cluster flow. > > > > at > > > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:497) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > org.apache.nifi.web.server.JettyServer.start(JettyServer.java:746) > > ~[nifi-jetty-1.0.0.jar:1.0.0] > > > > ... 2 common frames omitted > > > > Caused by: org.apache.nifi.controller.UninheritableFlowException: > Failed to > > connect node to cluster because local flow is different than > cluster flow. > > > > at > > > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:879) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:493) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > ... 3 common frames omitted > > > > Caused by: org.apache.nifi.controller.UninheritableFlowException: > Proposed > > Authorizer is not inheritable by the flow controller because of > Authorizer > > differences: Proposed Authorizations do not match current > Authorizations > > > > at > > > org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:252) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1435) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:83) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:671) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > at > > > org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:857) > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > ... 4 common frames omitted > > > > [root@ncm-cm1 logs]# > > > > > > > > I don’t know if the ‘Proposed Authorizer is not inheritable…’ > exception is > > part of the problem too. > > > > The docs weren’t very clear on whether (when upgrading and using > the legacy > > support of the authorized-user.xml path required the nodes to be > also added > > to the authorizers.xml. > > > > I did add them in the end as various attempts to get the cluster up > and > > running without them failed (as each server didn’t seem to have > rights to do > > anything. > > > > > > > > I have a lot of RPG in my work flows as I am ingesting many syslog > data > > sources and this was the recommended pattern to distribute the data > > (listensyslog…run on primary, output to port (RPG), pick up in rest > of data > > flow), > > > > > > > > Any suggestions on where to start trying to get this working? > > > > I’ve tried creating a new RPG on one on the datanodes and > connecting the > > syslog to that which sort of worked but then I have a bunch of > other errors > > when trying to enable the ports to do with not being able to > connect to > > (what was) the NCM. > > > > > > > > Thanks > > > > Conrad > > > > > > > > SecureData, combating cyber threats > > > > ________________________________ > > > > The information contained in this message or any of its attachments > may be > > privileged and confidential and intended for the exclusive use of > the > > intended recipient. If you are not the intended recipient any > disclosure, > > reproduction, distribution or other dissemination or use of this > > communications is strictly prohibited. The views expressed in this > email are > > those of the individual and not necessarily of SecureData Europe > Ltd. Any > > prices quoted are only valid if followed up by a formal written > quote. > > > > SecureData Europe Limited. Registered in England & Wales 04365896. > > Registered Address: SecureData House, Hermitage Court, Hermitage > Lane, > > Maidstone, Kent, ME16 9NT > > > ***This email originated outside SecureData*** > > Click > https://www.mailcontrol.com/sr/tAj77!!uP0XGX2PQPOmvUu5zZAYN1Mos55ZMH65vS49VoLnJlQAkvDtaSciXa9lO25LWvxYjTGeVGm43FW9a3A==<https://www.mailcontrol.com/sr/MZbqvYs5QwJvpeaetUwhCQ==> > to report this email as spam. > > > >
