Re: problem creating simple cluster

[Brian Jeltema]

I had assigned  values to nifi.web.http.host, but that didn't work (the first 
time).
After a bunch of fiddling around, I tried that again, and now the cluster is 
working.
I don't know what the original problem was, but I'm good now.

Thanks.

On Dec 7, 2016, at 9:06 PM, Matthew Clarke  wrote:

> In the NiFi.properties file is a property just above your http.port=8080 for 
> the http.host= . If that field is left blank. UI requested that are 
> replicated between node's may end up using localhost. Make sure this property 
> is set to the Nodes hostname or IP addresses on every node. A restart will be 
> needed before any changes to the NiFi.properties file to take affect.
> 
> Thanks, 
> Matt
> 
> On Dec 7, 2016 8:21 PM, "Koji Kawamura"  wrote:
> Sorry, I overlooked the nifi.properties settings you shared.
> Would you share what you can see on the NiFi "Cluster window", from
> right top Hamburger menu, especially the 'Node Address' column?
> 
> Thanks,
> Koji
> 
> On Thu, Dec 8, 2016 at 10:10 AM, Koji Kawamura  wrote:
> > Hi Brian,
> >
> > Are those three node running on a same host using different port? Or
> > running on different hosts?
> > nifi.properties has nifi.cluster.node.address configuration, which is
> > used by a NiFi node to tell how other NiFi nodes should access the
> > node.
> >
> > If the property is blank, NiFi uses 'localhost' as node hostname.
> > I think that's why the node tried to replicate the request to 
> > 'localhost:8080'.
> >
> > If so, the property should be set with a hostname that is accessible
> > from other nodes.
> >
> > Or, if there's any firewall among nodes,
> > nifi.web.http.port
> > nifi.cluster.node.protocol.port
> > should be opened.
> >
> > I sometimes forget this on AWS with security group setting then get
> > timeout error.
> >
> > Thanks,
> > Koji
> >
> > On Thu, Dec 8, 2016 at 2:59 AM, Brian Jeltema  wrote:
> >> The cluster is not running securely, so I don’t believe that file is
> >> relavent. In the stack trace,
> >> the reference to /nifi-api/flow/current-user is misleading - I think any
> >> nifi-api call has problems.
> >>
> >> On Dec 7, 2016, at 12:37 PM, James Wing  wrote:
> >>
> >> Brian,
> >>
> >> Did you add entries for the node DNs in the conf/authorizers.xml file?
> >> Something like:
> >>
> >> 
> >> CN=node1.nifi, ...
> >> CN=node2.nifi, ...
> >> ...
> >>
> >> Thanks,
> >>
> >> James
> >>
> >> On Wed, Dec 7, 2016 at 8:28 AM, Brian Jeltema  wrote:
> >>>
> >>> I’m trying to create my first cluster using NiFi 1.1.0. It’s a simple
> >>> 3-node unsecure configuration with each node running embedded
> >>> zookeeper. The instances all come up and the zookeeper quarum is
> >>> reached.
> >>>
> >>> If I bring up the UI for the node that is elected as the
> >>> cluster coordinator, it works as expected, and shows that 3 nodes
> >>> are participating in the cluster.
> >>>
> >>> However, if I attempt to display the UI on the non-coordinator nodes,
> >>> after a delay of about 10 seconds an error page is returned. The
> >>> logs contains a stream of exceptions similar to the following:
> >>>
> >>> 2016-12-07 11:08:17,914 WARN [Replicate Request Thread-2]
> >>> o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET
> >>> /nifi-api/flow/current-user to localhost:8080 due to {}
> >>> com.sun.jersey.api.client.ClientHandlerException:
> >>> java.net.SocketTimeoutException: Read timed out
> >>> at
> >>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at com.sun.jersey.api.client.Client.handle(Client.java:652)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at
> >>> com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at
> >>> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:578)
> >>> ~[nifi-framework-cluster-1.1.0.jar:1.1.0]
> >>> at
> >>> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:770)
> >>> ~[nifi-framework-cluster-1.1.0.jar:1.1.0]
> >>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> >>> [na:1.8.0_101]
> >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_101]
> >>> at
> >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> >>> 

Re: problem creating simple cluster

[Matthew Clarke]

In the NiFi.properties file is a property just above your http.port=8080
for the http.host= . If that field is left blank. UI requested that are
replicated between node's may end up using localhost. Make sure this
property is set to the Nodes hostname or IP addresses on every node. A
restart will be needed before any changes to the NiFi.properties file to
take affect.

Thanks,
Matt

On Dec 7, 2016 8:21 PM, "Koji Kawamura"  wrote:

> Sorry, I overlooked the nifi.properties settings you shared.
> Would you share what you can see on the NiFi "Cluster window", from
> right top Hamburger menu, especially the 'Node Address' column?
>
> Thanks,
> Koji
>
> On Thu, Dec 8, 2016 at 10:10 AM, Koji Kawamura 
> wrote:
> > Hi Brian,
> >
> > Are those three node running on a same host using different port? Or
> > running on different hosts?
> > nifi.properties has nifi.cluster.node.address configuration, which is
> > used by a NiFi node to tell how other NiFi nodes should access the
> > node.
> >
> > If the property is blank, NiFi uses 'localhost' as node hostname.
> > I think that's why the node tried to replicate the request to
> 'localhost:8080'.
> >
> > If so, the property should be set with a hostname that is accessible
> > from other nodes.
> >
> > Or, if there's any firewall among nodes,
> > nifi.web.http.port
> > nifi.cluster.node.protocol.port
> > should be opened.
> >
> > I sometimes forget this on AWS with security group setting then get
> > timeout error.
> >
> > Thanks,
> > Koji
> >
> > On Thu, Dec 8, 2016 at 2:59 AM, Brian Jeltema 
> wrote:
> >> The cluster is not running securely, so I don’t believe that file is
> >> relavent. In the stack trace,
> >> the reference to /nifi-api/flow/current-user is misleading - I think any
> >> nifi-api call has problems.
> >>
> >> On Dec 7, 2016, at 12:37 PM, James Wing  wrote:
> >>
> >> Brian,
> >>
> >> Did you add entries for the node DNs in the conf/authorizers.xml file?
> >> Something like:
> >>
> >> 
> >> CN=node1.nifi, ...
> >> CN=node2.nifi, ...
> >> ...
> >>
> >> Thanks,
> >>
> >> James
> >>
> >> On Wed, Dec 7, 2016 at 8:28 AM, Brian Jeltema 
> wrote:
> >>>
> >>> I’m trying to create my first cluster using NiFi 1.1.0. It’s a simple
> >>> 3-node unsecure configuration with each node running embedded
> >>> zookeeper. The instances all come up and the zookeeper quarum is
> >>> reached.
> >>>
> >>> If I bring up the UI for the node that is elected as the
> >>> cluster coordinator, it works as expected, and shows that 3 nodes
> >>> are participating in the cluster.
> >>>
> >>> However, if I attempt to display the UI on the non-coordinator nodes,
> >>> after a delay of about 10 seconds an error page is returned. The
> >>> logs contains a stream of exceptions similar to the following:
> >>>
> >>> 2016-12-07 11:08:17,914 WARN [Replicate Request Thread-2]
> >>> o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request
> GET
> >>> /nifi-api/flow/current-user to localhost:8080 due to {}
> >>> com.sun.jersey.api.client.ClientHandlerException:
> >>> java.net.SocketTimeoutException: Read timed out
> >>> at
> >>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(
> URLConnectionClientHandler.java:155)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at com.sun.jersey.api.client.Client.handle(Client.java:652)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at
> >>> com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(
> GZIPContentEncodingFilter.java:123)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at com.sun.jersey.api.client.WebResource.access$200(
> WebResource.java:74)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at com.sun.jersey.api.client.WebResource$Builder.get(
> WebResource.java:509)
> >>> ~[jersey-client-1.19.jar:1.19]
> >>> at
> >>> org.apache.nifi.cluster.coordination.http.replication.
> ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.
> java:578)
> >>> ~[nifi-framework-cluster-1.1.0.jar:1.1.0]
> >>> at
> >>> org.apache.nifi.cluster.coordination.http.replication.
> ThreadPoolRequestReplicator$NodeHttpRequest.run(
> ThreadPoolRequestReplicator.java:770)
> >>> ~[nifi-framework-cluster-1.1.0.jar:1.1.0]
> >>> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
> >>> [na:1.8.0_101]
> >>> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> [na:1.8.0_101]
> >>> at
> >>> java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> >>> [na:1.8.0_101]
> >>> at
> >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> >>> [na:1.8.0_101]
> >>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> >>> Caused by: java.net.SocketTimeoutException: Read timed out
> >>> at java.net.SocketInputStream.socketRead0(Native 

Re: problem creating simple cluster

[Koji Kawamura]

Sorry, I overlooked the nifi.properties settings you shared.
Would you share what you can see on the NiFi "Cluster window", from
right top Hamburger menu, especially the 'Node Address' column?

Thanks,
Koji

On Thu, Dec 8, 2016 at 10:10 AM, Koji Kawamura  wrote:
> Hi Brian,
>
> Are those three node running on a same host using different port? Or
> running on different hosts?
> nifi.properties has nifi.cluster.node.address configuration, which is
> used by a NiFi node to tell how other NiFi nodes should access the
> node.
>
> If the property is blank, NiFi uses 'localhost' as node hostname.
> I think that's why the node tried to replicate the request to 
> 'localhost:8080'.
>
> If so, the property should be set with a hostname that is accessible
> from other nodes.
>
> Or, if there's any firewall among nodes,
> nifi.web.http.port
> nifi.cluster.node.protocol.port
> should be opened.
>
> I sometimes forget this on AWS with security group setting then get
> timeout error.
>
> Thanks,
> Koji
>
> On Thu, Dec 8, 2016 at 2:59 AM, Brian Jeltema  wrote:
>> The cluster is not running securely, so I don’t believe that file is
>> relavent. In the stack trace,
>> the reference to /nifi-api/flow/current-user is misleading - I think any
>> nifi-api call has problems.
>>
>> On Dec 7, 2016, at 12:37 PM, James Wing  wrote:
>>
>> Brian,
>>
>> Did you add entries for the node DNs in the conf/authorizers.xml file?
>> Something like:
>>
>> 
>> CN=node1.nifi, ...
>> CN=node2.nifi, ...
>> ...
>>
>> Thanks,
>>
>> James
>>
>> On Wed, Dec 7, 2016 at 8:28 AM, Brian Jeltema  wrote:
>>>
>>> I’m trying to create my first cluster using NiFi 1.1.0. It’s a simple
>>> 3-node unsecure configuration with each node running embedded
>>> zookeeper. The instances all come up and the zookeeper quarum is
>>> reached.
>>>
>>> If I bring up the UI for the node that is elected as the
>>> cluster coordinator, it works as expected, and shows that 3 nodes
>>> are participating in the cluster.
>>>
>>> However, if I attempt to display the UI on the non-coordinator nodes,
>>> after a delay of about 10 seconds an error page is returned. The
>>> logs contains a stream of exceptions similar to the following:
>>>
>>> 2016-12-07 11:08:17,914 WARN [Replicate Request Thread-2]
>>> o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET
>>> /nifi-api/flow/current-user to localhost:8080 due to {}
>>> com.sun.jersey.api.client.ClientHandlerException:
>>> java.net.SocketTimeoutException: Read timed out
>>> at
>>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
>>> ~[jersey-client-1.19.jar:1.19]
>>> at com.sun.jersey.api.client.Client.handle(Client.java:652)
>>> ~[jersey-client-1.19.jar:1.19]
>>> at
>>> com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
>>> ~[jersey-client-1.19.jar:1.19]
>>> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
>>> ~[jersey-client-1.19.jar:1.19]
>>> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>>> ~[jersey-client-1.19.jar:1.19]
>>> at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509)
>>> ~[jersey-client-1.19.jar:1.19]
>>> at
>>> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:578)
>>> ~[nifi-framework-cluster-1.1.0.jar:1.1.0]
>>> at
>>> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:770)
>>> ~[nifi-framework-cluster-1.1.0.jar:1.1.0]
>>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>>> [na:1.8.0_101]
>>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_101]
>>> at
>>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>>> [na:1.8.0_101]
>>> at
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>>> [na:1.8.0_101]
>>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
>>> Caused by: java.net.SocketTimeoutException: Read timed out
>>> at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.8.0_101]
>>> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
>>> ~[na:1.8.0_101]
>>> at java.net.SocketInputStream.read(SocketInputStream.java:170)
>>> ~[na:1.8.0_101]
>>> at java.net.SocketInputStream.read(SocketInputStream.java:141)
>>> ~[na:1.8.0_101]
>>> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
>>> ~[na:1.8.0_101]
>>> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
>>> ~[na:1.8.0_101]
>>> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
>>> ~[na:1.8.0_101]
>>> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
>>> ~[na:1.8.0_101]
>>> at 

Re: problem creating simple cluster

[Koji Kawamura]

Hi Brian,

Are those three node running on a same host using different port? Or
running on different hosts?
nifi.properties has nifi.cluster.node.address configuration, which is
used by a NiFi node to tell how other NiFi nodes should access the
node.

If the property is blank, NiFi uses 'localhost' as node hostname.
I think that's why the node tried to replicate the request to 'localhost:8080'.

If so, the property should be set with a hostname that is accessible
from other nodes.

Or, if there's any firewall among nodes,
nifi.web.http.port
nifi.cluster.node.protocol.port
should be opened.

I sometimes forget this on AWS with security group setting then get
timeout error.

Thanks,
Koji

On Thu, Dec 8, 2016 at 2:59 AM, Brian Jeltema  wrote:
> The cluster is not running securely, so I don’t believe that file is
> relavent. In the stack trace,
> the reference to /nifi-api/flow/current-user is misleading - I think any
> nifi-api call has problems.
>
> On Dec 7, 2016, at 12:37 PM, James Wing  wrote:
>
> Brian,
>
> Did you add entries for the node DNs in the conf/authorizers.xml file?
> Something like:
>
> 
> CN=node1.nifi, ...
> CN=node2.nifi, ...
> ...
>
> Thanks,
>
> James
>
> On Wed, Dec 7, 2016 at 8:28 AM, Brian Jeltema  wrote:
>>
>> I’m trying to create my first cluster using NiFi 1.1.0. It’s a simple
>> 3-node unsecure configuration with each node running embedded
>> zookeeper. The instances all come up and the zookeeper quarum is
>> reached.
>>
>> If I bring up the UI for the node that is elected as the
>> cluster coordinator, it works as expected, and shows that 3 nodes
>> are participating in the cluster.
>>
>> However, if I attempt to display the UI on the non-coordinator nodes,
>> after a delay of about 10 seconds an error page is returned. The
>> logs contains a stream of exceptions similar to the following:
>>
>> 2016-12-07 11:08:17,914 WARN [Replicate Request Thread-2]
>> o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET
>> /nifi-api/flow/current-user to localhost:8080 due to {}
>> com.sun.jersey.api.client.ClientHandlerException:
>> java.net.SocketTimeoutException: Read timed out
>> at
>> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
>> ~[jersey-client-1.19.jar:1.19]
>> at com.sun.jersey.api.client.Client.handle(Client.java:652)
>> ~[jersey-client-1.19.jar:1.19]
>> at
>> com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
>> ~[jersey-client-1.19.jar:1.19]
>> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
>> ~[jersey-client-1.19.jar:1.19]
>> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
>> ~[jersey-client-1.19.jar:1.19]
>> at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509)
>> ~[jersey-client-1.19.jar:1.19]
>> at
>> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:578)
>> ~[nifi-framework-cluster-1.1.0.jar:1.1.0]
>> at
>> org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:770)
>> ~[nifi-framework-cluster-1.1.0.jar:1.1.0]
>> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>> [na:1.8.0_101]
>> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_101]
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>> [na:1.8.0_101]
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>> [na:1.8.0_101]
>> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
>> Caused by: java.net.SocketTimeoutException: Read timed out
>> at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.8.0_101]
>> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
>> ~[na:1.8.0_101]
>> at java.net.SocketInputStream.read(SocketInputStream.java:170)
>> ~[na:1.8.0_101]
>> at java.net.SocketInputStream.read(SocketInputStream.java:141)
>> ~[na:1.8.0_101]
>> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
>> ~[na:1.8.0_101]
>> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
>> ~[na:1.8.0_101]
>> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
>> ~[na:1.8.0_101]
>> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
>> ~[na:1.8.0_101]
>> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
>> ~[na:1.8.0_101]
>> at
>> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536)
>> ~[na:1.8.0_101]
>> at
>> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
>> ~[na:1.8.0_101]
>> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
>> ~[na:1.8.0_101]
>> at
>> 

Re: problem creating simple cluster

[James Wing]

Brian,

Did you add entries for the node DNs in the conf/authorizers.xml file?
Something like:


CN=node1.nifi, ...
CN=node2.nifi, ...
...

Thanks,

James

On Wed, Dec 7, 2016 at 8:28 AM, Brian Jeltema  wrote:

> I’m trying to create my first cluster using NiFi 1.1.0. It’s a simple
> 3-node unsecure configuration with each node running embedded
> zookeeper. The instances all come up and the zookeeper quarum is
> reached.
>
> If I bring up the UI for the node that is elected as the
> cluster coordinator, it works as expected, and shows that 3 nodes
> are participating in the cluster.
>
> However, if I attempt to display the UI on the non-coordinator nodes,
> after a delay of about 10 seconds an error page is returned. The
> logs contains a stream of exceptions similar to the following:
>
> 2016-12-07 11:08:17,914 WARN [Replicate Request Thread-2] 
> o.a.n.c.c.h.r.ThreadPoolRequestReplicator
> Failed to replicate request GET /nifi-api/flow/current-user to
> localhost:8080 due to {}
> com.sun.jersey.api.client.ClientHandlerException: 
> java.net.SocketTimeoutException:
> Read timed out
> at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(
> URLConnectionClientHandler.java:155) ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.Client.handle(Client.java:652)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(
> GZIPContentEncodingFilter.java:123) ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509)
> ~[jersey-client-1.19.jar:1.19]
> at org.apache.nifi.cluster.coordination.http.replication.
> ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:578)
> ~[nifi-framework-cluster-1.1.0.jar:1.1.0]
> at org.apache.nifi.cluster.coordination.http.replication.
> ThreadPoolRequestReplicator$NodeHttpRequest.run(
> ThreadPoolRequestReplicator.java:770) ~[nifi-framework-cluster-1.1.
> 0.jar:1.1.0]
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_101]
> at java.util.concurrent.FutureTask.run(FutureTask.java:266) [na:1.8.0_101]
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> [na:1.8.0_101]
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> [na:1.8.0_101]
> at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
> Caused by: java.net.SocketTimeoutException: Read timed out
> at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.8.0_101]
> at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
> ~[na:1.8.0_101]
> at java.net.SocketInputStream.read(SocketInputStream.java:170)
> ~[na:1.8.0_101]
> at java.net.SocketInputStream.read(SocketInputStream.java:141)
> ~[na:1.8.0_101]
> at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
> ~[na:1.8.0_101]
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
> ~[na:1.8.0_101]
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> ~[na:1.8.0_101]
> at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704)
> ~[na:1.8.0_101]
> at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
> ~[na:1.8.0_101]
> at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536)
> ~[na:1.8.0_101]
> at 
> sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
> ~[na:1.8.0_101]
> at java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480)
> ~[na:1.8.0_101]
> at com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(
> URLConnectionClientHandler.java:253) ~[jersey-client-1.19.jar:1.19]
> at com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(
> URLConnectionClientHandler.java:153) ~[jersey-client-1.19.jar:1.19]
> ... 12 common frames omitted
>
> The nifi.properties file contains the following, with
> nifi.cluster.node.address set appropriately on each node
>
>  # cluster node properties (only configure for cluster nodes) #
>  nifi.cluster.is.node=true
>  nifi.cluster.node.address=172.31.1.247
>  nifi.cluster.node.protocol.port=8081
>  nifi.cluster.node.protocol.threads=10
>  nifi.cluster.node.event.history.size=25
>  nifi.cluster.node.connection.timeout=5 sec
>  nifi.cluster.node.read.timeout=5 sec
>  nifi.cluster.firewall.file=
>  nifi.cluster.flow.election.max.wait.time=5 mins
>  nifi.cluster.flow.election.max.candidates=3
>
>  # zookeeper properties, used for cluster management #
>  nifi.zookeeper.connect.string=172.31.13.177:2181,172.31.1.247:2181,
> 172.31.1.69:2181
>  nifi.zookeeper.connect.timeout=3 secs
>  nifi.zookeeper.session.timeout=3 

problem creating simple cluster

[Brian Jeltema]

I’m trying to create my first cluster using NiFi 1.1.0. It’s a simple
3-node unsecure configuration with each node running embedded
zookeeper. The instances all come up and the zookeeper quarum is
reached.

If I bring up the UI for the node that is elected as the 
cluster coordinator, it works as expected, and shows that 3 nodes
are participating in the cluster.

However, if I attempt to display the UI on the non-coordinator nodes,
after a delay of about 10 seconds an error page is returned. The
logs contains a stream of exceptions similar to the following:

2016-12-07 11:08:17,914 WARN [Replicate Request Thread-2] 
o.a.n.c.c.h.r.ThreadPoolRequestReplicator Failed to replicate request GET 
/nifi-api/flow/current-user to localhost:8080 due to {}
com.sun.jersey.api.client.ClientHandlerException: 
java.net.SocketTimeoutException: Read timed out
at 
com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:155)
 ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.Client.handle(Client.java:652) 
~[jersey-client-1.19.jar:1.19]
at 
com.sun.jersey.api.client.filter.GZIPContentEncodingFilter.handle(GZIPContentEncodingFilter.java:123)
 ~[jersey-client-1.19.jar:1.19]
at com.sun.jersey.api.client.WebResource.handle(WebResource.java:682) 
~[jersey-client-1.19.jar:1.19]
at 
com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74) 
~[jersey-client-1.19.jar:1.19]
at 
com.sun.jersey.api.client.WebResource$Builder.get(WebResource.java:509) 
~[jersey-client-1.19.jar:1.19]
at 
org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator.replicateRequest(ThreadPoolRequestReplicator.java:578)
 ~[nifi-framework-cluster-1.1.0.jar:1.1.0]
at 
org.apache.nifi.cluster.coordination.http.replication.ThreadPoolRequestReplicator$NodeHttpRequest.run(ThreadPoolRequestReplicator.java:770)
 ~[nifi-framework-cluster-1.1.0.jar:1.1.0]
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) 
[na:1.8.0_101]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) 
[na:1.8.0_101]
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) 
[na:1.8.0_101]
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) 
[na:1.8.0_101]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_101]
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[na:1.8.0_101]
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) 
~[na:1.8.0_101]
at java.net.SocketInputStream.read(SocketInputStream.java:170) 
~[na:1.8.0_101]
at java.net.SocketInputStream.read(SocketInputStream.java:141) 
~[na:1.8.0_101]
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246) 
~[na:1.8.0_101]
at java.io.BufferedInputStream.read1(BufferedInputStream.java:286) 
~[na:1.8.0_101]
at java.io.BufferedInputStream.read(BufferedInputStream.java:345) 
~[na:1.8.0_101]
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:704) 
~[na:1.8.0_101]
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647) 
~[na:1.8.0_101]
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1536)
 ~[na:1.8.0_101]
at 
sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1441)
 ~[na:1.8.0_101]
at 
java.net.HttpURLConnection.getResponseCode(HttpURLConnection.java:480) 
~[na:1.8.0_101]
at 
com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:253)
 ~[jersey-client-1.19.jar:1.19]
at 
com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:153)
 ~[jersey-client-1.19.jar:1.19]
... 12 common frames omitted

The nifi.properties file contains the following, with nifi.cluster.node.address 
set appropriately on each node

 # cluster node properties (only configure for cluster nodes) #
 nifi.cluster.is.node=true
 nifi.cluster.node.address=172.31.1.247
 nifi.cluster.node.protocol.port=8081
 nifi.cluster.node.protocol.threads=10
 nifi.cluster.node.event.history.size=25
 nifi.cluster.node.connection.timeout=5 sec
 nifi.cluster.node.read.timeout=5 sec
 nifi.cluster.firewall.file=
 nifi.cluster.flow.election.max.wait.time=5 mins
 nifi.cluster.flow.election.max.candidates=3

 # zookeeper properties, used for cluster management #
 
nifi.zookeeper.connect.string=172.31.13.177:2181,172.31.1.247:2181,172.31.1.69:2181
 nifi.zookeeper.connect.timeout=3 secs
 nifi.zookeeper.session.timeout=3 secs
 nifi.zookeeper.root.node=/nifi

I’ve run out of ideas; I presume I’ve overlooked some simple configuration 
value, but I can’t find it.
Any help out there?

Thanks