Yes http site-to-site was added recently so setting that to disabled should be fine and not related.
If you are using all the same keystores and truststores from before, then I can't think of why the nodes wouldn't be able to communicate securely. Unless anyone else has some other ideas, you may need to turn on SSL debug (-Djavax.net.debug=all) to see why the handshake is failing. Is there anything interesting/related in nifi-user.log? On Wed, Oct 19, 2016 at 10:38 AM, Conrad Crampton < conrad.cramp...@secdata.com> wrote: > Hi, > > Yes, every nifi.properties is set thus – with host and port different for > each. > > > > # Site to Site properties > > nifi.remote.input.socket.host=ncm.xxxxxxx > > nifi.remote.input.socket.port=9870 > > nifi.remote.input.secure=true > > nifi.remote.input.http.enabled=false > > nifi.remote.input.http.transaction.ttl=30 sec > > > > You’ll obviously notice that I have http disabled. I set this as this was > a new setting which I didn’t have before (it was only RAW in previous > versions wasn’t it?) > > > > Does this make a difference? > > > > Thanks > > Conrad > > > > *From: *Bryan Bende <bbe...@gmail.com> > *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org> > *Date: *Wednesday, 19 October 2016 at 15:33 > > *To: *"users@nifi.apache.org" <users@nifi.apache.org> > *Subject: *Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups > > > > Trying to think of things to check here... > > > > Does every node have nifi.remote.input.secure=true in nifi.properties and > the URL in the RPG is an https URL? > > > > On Wed, Oct 19, 2016 at 10:25 AM, Conrad Crampton < > conrad.cramp...@secdata.com> wrote: > > One other thing… > > The RPGs have an unlocked padlock on them saying S2S is not secure. > > Conrad > > > > *From: *Bryan Bende <bbe...@gmail.com> > *Reply-To: *"users@nifi.apache.org" <users@nifi.apache.org> > *Date: *Wednesday, 19 October 2016 at 15:20 > *To: *"users@nifi.apache.org" <users@nifi.apache.org> > *Subject: *Re: Upgrade 0.6.1 to 1.0.0 problems with Remote Process Groups > > > > Ok that does seem like a TLS/SSL issue... > > > > Is this a single cluster doing site-to-site to itself? > > > > On Wed, Oct 19, 2016 at 10:06 AM, Joe Witt <joe.w...@gmail.com> wrote: > > thanks conrad - did get it. Bryan is being more helpful that I so I > went silent :-) > > On Wed, Oct 19, 2016 at 10:02 AM, Conrad Crampton > > <conrad.cramp...@secdata.com> wrote: > > Hi Joe, > > Yep, > > Tried removing the RPG that referenced the NCM and adding new one > with one of the datanodes as url. > > That sort of worked, but kept getting errors about the NCM not being > available for the ports and therefore couldn’t actually enable the port I > needed to for that RPG. > > Thanks > > Conrad > > > > (sending again as don’t know if the stupid header ‘spoofed’ is stopping > getting though – apologies if already sent) > > > > On 19/10/2016, 14:12, "Joe Witt" <joe.w...@gmail.com> wrote: > > > > Conrad, > > > > For s2s now you can just point at any of the nodes in the > cluster. > > Have you tried changing the URL or removing and adding new RPG > > entries? > > > > Thanks > > Joe > > > > On Wed, Oct 19, 2016 at 8:38 AM, Conrad Crampton > > <conrad.cramp...@secdata.com> wrote: > > > Hi, > > > > > > I have finally taken the plunge to upgrade my cluster from > 0.6.1 to 1.0.0. > > > > > > 6 nodes with a NCM. > > > > > > With the removal of NCM in 1.0.0 I believe I now have an issue > where none of > > > my Remote Process Groups work as they previously did because > they were > > > configured to connect to the NCM (as the RPG url) which now > doesn’t exist. > > > > > > I have tried converting my NCM to a node but whilst I can get > it running > > > (sort of) when I try and connect to the cluster I get > something like this in > > > my logs… > > > > > > > > > > > > 2016-10-19 13:14:44,109 ERROR [main] o.a.nifi.controller. > StandardFlowService > > > Failed to load flow from cluster due to: > > > org.apache.nifi.controller.UninheritableFlowException: Failed > to connect > > > node to cluster because local flow is different than cluster > flow. > > > > > > org.apache.nifi.controller.UninheritableFlowException: Failed > to connect > > > node to cluster because local flow is different than cluster > flow. > > > > > > at > > > org.apache.nifi.controller.StandardFlowService. > loadFromConnectionResponse(StandardFlowService.java:879) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.controller.StandardFlowService.load( > StandardFlowService.java:493) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.web.server.JettyServer.start(JettyServer. > java:746) > > > [nifi-jetty-1.0.0.jar:1.0.0] > > > > > > at org.apache.nifi.NiFi.<init>(NiFi.java:152) > > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > > > at org.apache.nifi.NiFi.main(NiFi.java:243) > > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > > > Caused by: org.apache.nifi.controller.UninheritableFlowException: > Proposed > > > Authorizer is not inheritable by the flow controller because > of Authorizer > > > differences: Proposed Authorizations do not match current > Authorizations > > > > > > at > > > org.apache.nifi.controller.StandardFlowSynchronizer.sync( > StandardFlowSynchronizer.java:252) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.controller.FlowController.synchronize( > FlowController.java:1435) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.persistence.StandardXMLFlowConfigurationDA > O.load(StandardXMLFlowConfigurationDAO.java:83) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.controller.StandardFlowService.loadFromBytes( > StandardFlowService.java:671) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.controller.StandardFlowService. > loadFromConnectionResponse(StandardFlowService.java:857) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > ... 4 common frames omitted > > > > > > 2016-10-19 13:14:44,414 ERROR [main] o.a.n.c.c.node. > NodeClusterCoordinator > > > Event Reported for ncm-cm1.mis-cds.local:9090 -- Node > disconnected from > > > cluster due to > > org.apache.nifi.controller.UninheritableFlowException: > Failed > > > to connect node to cluster because local flow is different > than cluster > > > flow. > > > > > > 2016-10-19 13:14:44,420 ERROR [Shutdown Cluster Coordinator] > > > org.apache.nifi.NiFi An Unknown Error Occurred in Thread > Thread[Shutdown > > > Cluster Coordinator,5,main]: java.lang.NullPointerException > > > > > > 2016-10-19 13:14:44,423 ERROR [Shutdown Cluster Coordinator] > > > org.apache.nifi.NiFi > > > > > > java.lang.NullPointerException: null > > > > > > at > > > java.util.concurrent.ConcurrentHashMap.putVal( > ConcurrentHashMap.java:1011) > > > ~[na:1.8.0_51] > > > > > > at > > > java.util.concurrent.ConcurrentHashMap.put( > ConcurrentHashMap.java:1006) > > > ~[na:1.8.0_51] > > > > > > at > > > org.apache.nifi.cluster.coordination.node. > NodeClusterCoordinator.updateNodeStatus(NodeClusterCoordinator.java:570) > > > ~[nifi-framework-cluster-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.cluster.coordination.node. > NodeClusterCoordinator.shutdown(NodeClusterCoordinator.java:119) > > > ~[nifi-framework-cluster-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.controller.StandardFlowService$1.run( > StandardFlowService.java:330) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at java.lang.Thread.run(Thread.java:745) > ~[na:1.8.0_51] > > > > > > 2016-10-19 13:14:44,448 WARN [main] o.a.n.c.l.e. > CuratorLeaderElectionManager > > > Failed to close Leader Selector for Cluster Coordinator > > > > > > java.lang.IllegalStateException: Already closed or has not > been started > > > > > > at > > > com.google.common.base.Preconditions.checkState( > Preconditions.java:173) > > > ~[guava-18.0.jar:na] > > > > > > at > > > org.apache.curator.framework.recipes.leader.LeaderSelector. > close(LeaderSelector.java:270) > > > ~[curator-recipes-2.11.0.jar:na] > > > > > > at > > > org.apache.nifi.controller.leader.election. > CuratorLeaderElectionManager.stop(CuratorLeaderElectionManager.java:159) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.controller.FlowController.shutdown( > FlowController.java:1303) > > > [nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.controller.StandardFlowService.stop( > StandardFlowService.java:339) > > > [nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.web.server.JettyServer.start(JettyServer. > java:753) > > > [nifi-jetty-1.0.0.jar:1.0.0] > > > > > > at org.apache.nifi.NiFi.<init>(NiFi.java:152) > > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > > > at org.apache.nifi.NiFi.main(NiFi.java:243) > > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > > > 2016-10-19 13:14:45,062 WARN [Cluster Socket Listener] > > > org.apache.nifi.io.socket.SocketListener Failed to > communicate with Unknown > > > Host due to java.net.SocketException: Socket closed > > > > > > java.net.SocketException: Socket closed > > > > > > at java.net.PlainSocketImpl.socketAccept(Native > Method) > > > ~[na:1.8.0_51] > > > > > > at > > > java.net.AbstractPlainSocketImpl.accept( > AbstractPlainSocketImpl.java:404) > > > ~[na:1.8.0_51] > > > > > > at java.net.ServerSocket.implAccept(ServerSocket.java: > 545) > > > ~[na:1.8.0_51] > > > > > > at > > > sun.security.ssl.SSLServerSocketImpl.accept( > SSLServerSocketImpl.java:348) > > > ~[na:1.8.0_51] > > > > > > at > > > org.apache.nifi.io.socket.SocketListener$2.run( > SocketListener.java:112) > > > ~[nifi-socket-utils-1.0.0.jar:1.0.0] > > > > > > at java.lang.Thread.run(Thread.java:745) [na:1.8.0_51] > > > > > > 2016-10-19 13:14:45,064 WARN [main] org.apache.nifi.web.server. > JettyServer > > > Failed to start web server... shutting down. > > > > > > java.lang.Exception: Unable to load flow due to: > java.io.IOException: > > > org.apache.nifi.controller.UninheritableFlowException: Failed > to connect > > > node to cluster because local flow is different than cluster > flow. > > > > > > at > > > org.apache.nifi.web.server.JettyServer.start(JettyServer. > java:755) > > > ~[nifi-jetty-1.0.0.jar:1.0.0] > > > > > > at org.apache.nifi.NiFi.<init>(NiFi.java:152) > > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > > > at org.apache.nifi.NiFi.main(NiFi.java:243) > > > [nifi-runtime-1.0.0.jar:1.0.0] > > > > > > Caused by: java.io.IOException: > > > org.apache.nifi.controller.UninheritableFlowException: Failed > to connect > > > node to cluster because local flow is different than cluster > flow. > > > > > > at > > > org.apache.nifi.controller.StandardFlowService.load( > StandardFlowService.java:497) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.web.server.JettyServer.start(JettyServer. > java:746) > > > ~[nifi-jetty-1.0.0.jar:1.0.0] > > > > > > ... 2 common frames omitted > > > > > > Caused by: org.apache.nifi.controller.UninheritableFlowException: > Failed to > > > connect node to cluster because local flow is different than > cluster flow. > > > > > > at > > > org.apache.nifi.controller.StandardFlowService. > loadFromConnectionResponse(StandardFlowService.java:879) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.controller.StandardFlowService.load( > StandardFlowService.java:493) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > ... 3 common frames omitted > > > > > > Caused by: org.apache.nifi.controller.UninheritableFlowException: > Proposed > > > Authorizer is not inheritable by the flow controller because > of Authorizer > > > differences: Proposed Authorizations do not match current > Authorizations > > > > > > at > > > org.apache.nifi.controller.StandardFlowSynchronizer.sync( > StandardFlowSynchronizer.java:252) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.controller.FlowController.synchronize( > FlowController.java:1435) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.persistence.StandardXMLFlowConfigurationDA > O.load(StandardXMLFlowConfigurationDAO.java:83) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.controller.StandardFlowService.loadFromBytes( > StandardFlowService.java:671) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > at > > > org.apache.nifi.controller.StandardFlowService. > loadFromConnectionResponse(StandardFlowService.java:857) > > > ~[nifi-framework-core-1.0.0.jar:1.0.0] > > > > > > ... 4 common frames omitted > > > > > > [root@ncm-cm1 logs]# > > > > > > > > > > > > I don’t know if the ‘Proposed Authorizer is not inheritable…’ > exception is > > > part of the problem too. > > > > > > The docs weren’t very clear on whether (when upgrading and > using the legacy > > > support of the authorized-user.xml path required the nodes to > be also added > > > to the authorizers.xml. > > > > > > I did add them in the end as various attempts to get the > cluster up and > > > running without them failed (as each server didn’t seem to > have rights to do > > > anything. > > > > > > > > > > > > I have a lot of RPG in my work flows as I am ingesting many > syslog data > > > sources and this was the recommended pattern to distribute the > data > > > (listensyslog…run on primary, output to port (RPG), pick up in > rest of data > > > flow), > > > > > > > > > > > > Any suggestions on where to start trying to get this working? > > > > > > I’ve tried creating a new RPG on one on the datanodes and > connecting the > > > syslog to that which sort of worked but then I have a bunch of > other errors > > > when trying to enable the ports to do with not being able to > connect to > > > (what was) the NCM. > > > > > > > > > > > > Thanks > > > > > > Conrad > > > > > > > > > > > > SecureData, combating cyber threats > > > > > > ________________________________ > > > > > > The information contained in this message or any of its > attachments may be > > > privileged and confidential and intended for the exclusive use > of the > > > intended recipient. If you are not the intended recipient any > disclosure, > > > reproduction, distribution or other dissemination or use of > this > > > communications is strictly prohibited. The views expressed in > this email are > > > those of the individual and not necessarily of SecureData > Europe Ltd. Any > > > prices quoted are only valid if followed up by a formal > written quote. > > > > > > SecureData Europe Limited. Registered in England & Wales > 04365896. > > > Registered Address: SecureData House, Hermitage Court, > Hermitage Lane, > > > Maidstone, Kent, ME16 9NT > > > > > > ***This email originated outside SecureData*** > > > > Click https://www.mailcontrol.com/sr/tAj77!! > uP0XGX2PQPOmvUu5zZAYN1Mos55ZMH65vS49VoLnJlQAkvDtaSciXa9lO25L > WvxYjTGeVGm43FW9a3A== > <https://www.mailcontrol.com/sr/MZbqvYs5QwJvpeaetUwhCQ==> to report this > email as spam. > > > > > > > > > > > > >
