Matt - Thanks. It worked. So I have two independent work flows. ( I am trying to be detailed so that this archive mail may help some one achieve site to site setup)
Flow 1 Listent http — > Nifi flow (SitetoSite) (Here is asked me to Flow 2 Inputport —> Putfile. I tried to set secure site-to-site, I see following errors. Here both nodes are set with https keys. 2015-10-13 16:38:41,475 ERROR [Site-to-Site Worker Thread-254] o.a.n.r.io.socket.ssl.SSLSocketChannel org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel@2b7eba71 Failed to connect due to {} javax.net.ssl.SSLHandshakeException: null cert chain at sun.security.ssl.Handshaker.checkThrown(Handshaker.java:1431) ~[na:1.8.0_45] at sun.security.ssl.SSLEngineImpl.checkTaskThrown(SSLEngineImpl.java:535) ~[na:1.8.0_45] at sun.security.ssl.SSLEngineImpl.writeAppRecord(SSLEngineImpl.java:1214) ~[na:1.8.0_45] at sun.security.ssl.SSLEngineImpl.wrap(SSLEngineImpl.java:1186) ~[na:1.8.0_45] at javax.net.ssl.SSLEngine.wrap(SSLEngine.java:469) ~[na:1.8.0_45] at org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.performHandshake(SSLSocketChannel.java:187) ~[nifi-utils-0.3.0.jar:0.3.0] at org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.connect(SSLSocketChannel.java:140) ~[nifi-utils-0.3.0.jar:0.3.0] at org.apache.nifi.remote.SocketRemoteSiteListener$1$1.run(SocketRemoteSiteListener.java:155) [nifi-site-to-site-0.3.0.jar:0.3.0] at java.lang.Thread.run(Thread.java:745) [na:1.8.0_45] Caused by: javax.net.ssl.SSLHandshakeException: null cert chain at sun.security.ssl.Alerts.getSSLException(Alerts.java:192) ~[na:1.8.0_45] at sun.security.ssl.SSLEngineImpl.fatal(SSLEngineImpl.java:1666) ~[na:1.8.0_45] at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:304) ~[na:1.8.0_45] at sun.security.ssl.Handshaker.fatalSE(Handshaker.java:292) ~[na:1.8.0_45] at sun.security.ssl.ServerHandshaker.clientCertificate(ServerHandshaker.java:1804) ~[na:1.8.0_45] at sun.security.ssl.ServerHandshaker.processMessage(ServerHandshaker.java:222) ~[na:1.8.0_45] at sun.security.ssl.Handshaker.processLoop(Handshaker.java:979) ~[na:1.8.0_45] at sun.security.ssl.Handshaker$1.run(Handshaker.java:919) ~[na:1.8.0_45] at sun.security.ssl.Handshaker$1.run(Handshaker.java:916) ~[na:1.8.0_45] at java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_45] at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker.java:1369) ~[na:1.8.0_45] at org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.performTasks(SSLSocketChannel.java:250) ~[nifi-utils-0.3.0.jar:0.3.0] at org.apache.nifi.remote.io.socket.ssl.SSLSocketChannel.performHandshake(SSLSocketChannel.java:236) ~[nifi-utils-0.3.0.jar:0.3.0] ... 3 common frames omitted 2015-10-13 16:38:41,475 ERROR [Site-to-Site Worker Thread-254] o.a.nifi.remote.SocketRemoteSiteListener RemoteSiteListener Unable to accept connection from Socket[unconnected] due to javax.net.ssl.SSLException: Inbound closed before receiving peer's close_notify: possible truncation attack? From: Matt Clarke <matt.clarke....@gmail.com<mailto:matt.clarke....@gmail.com>> Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Tuesday, October 13, 2015 at 2:51 PM To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: site to site setup - Remore instance not configured So you're "nifi flow" RPG should not have any connection lines coming out of it. The input port should not have any connections coming in to it. You should have what looks like two separate independent flows on your graph. Flow 1 would consist of the listenHTTP connecting to the RPG directly. When draw the connection to the RPG, the connection window will ask you which input port you want to connect with. Flow 2 would consist of the input port you picked in flow 1 connecting directly to the putFile. The RPG will handle the load balancing for you automatically. Thanks, Matt Sent from my Verizon Wireless 4G LTE DROID Chakrader Dewaragatla <chakrader.dewaraga...@lifelock.com<mailto:chakrader.dewaraga...@lifelock.com>> wrote: Sweet, we need load balancer data on other end. how do I make it work ? Here is the nifi canvas screenshot. http://tinyurl.com/nq9fbqr From: Matthew Clarke <matt.clarke....@gmail.com<mailto:matt.clarke....@gmail.com>> Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Tuesday, October 13, 2015 at 2:17 PM To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: site to site setup - Remore instance not configured Yes, the Remote Process Group (RPG, also known as the site-to-site) will talk to the NCM which will provide connected nodes status information. It will then load balancer data to the nodes in that cluster. On Oct 13, 2015 5:06 PM, "Chakrader Dewaragatla" <chakrader.dewaraga...@lifelock.com<mailto:chakrader.dewaraga...@lifelock.com>> wrote: Thanks Matthew it worked, no errors this time. As noted below, we would like to consume http data on primary node and send it back to cluster for processing. I have site to site setup with http listener(on primary node) —> Input port —> SitetoSite to NCM cluster instance —> putfile —> S3upload . Does my setup achieve the purpose? Data movement from http listener to input port is not working and no errors reported. Site-to-site admin document has limited information to understand. Thanks, -Chakri From: Matthew Clarke <matt.clarke....@gmail.com<mailto:matt.clarke....@gmail.com>> Reply-To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Monday, October 12, 2015 at 4:49 PM To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: site to site setup - Remore instance not configured For Site to Site properties the input socket host is optional. When it is configured it should be set the hostname or IP of the the system the NiFi instance is running on. you must however configure an input socket port on every instance. This includes all nodes and the NCM. If you cluster is configured to run securely (HTTPS), you should also have input secure set to true. Make sure you have the ports you use open in the firewalls between systems. On Oct 12, 2015 7:41 PM, "Chakrader Dewaragatla" <chakrader.dewaraga...@lifelock.com<mailto:chakrader.dewaraga...@lifelock.com>> wrote: Hi – I have a use case to collect http post requests on a nifi-clustered setup. My cluster has three nodes. Ncm —> Slave 1 (Primary) —> Slave 2. I would like to setup a http listener on primary node and establish site to site connection to the same cluster for further processing data. So http payload receive as follows http post —> Slave 1 (ListenHttp )— > (Site-to-site) — > NCM (put file) and (S3 upload) (I assume this data process by two slaves nodes) I have following error at site-to-site setup : Remote instance Is not configured for site-to-site communications at this time. I followed the admin doc to set the properties, as follows (on slaves). # Site to Site properties nifi.remote.input.socket.host=10.83.14.59 (NCM ip) nifi.remote.input.socket.port= nifi.remote.input.secure=false Any thoughts? Thanks, -Chakri ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________ ________________________________ The information contained in this transmission may contain privileged and confidential information. It is intended only for the use of the person(s) named above. If you are not the intended recipient, you are hereby notified that any review, dissemination, distribution or duplication of this communication is strictly prohibited. If you are not the intended recipient, please contact the sender by reply email and destroy all copies of the original message. ________________________________