Paresh,
Please look in your sending NiFi logs for the following log line:
INFO [Timer-Driven Process Thread-6]
o.a.n.r.c.socket.EndpointConnectionPool New Weighted Distribution of Nodes:
Node[<*FQDN-node1>*:0] will receive 33.333333333333336% of data
Node[<*FQDN-node2>*:0] will receive 33.333333333333336% of data
Node[<*FQDN-node3>*:0] will receive 33.333333333333336% of data
Do the hostnames in those lines look correct?
Did you configure all three S2S line on every Node in your receiving
cluster?
# Site to Site properties
nifi.remote.input.socket.host=<*FQDN-node1> <-- not
recommended that this is left blank. This should be set to an IP or FQDN
that is resolvable and reachable by any sending NiFi.
Leaving this blank may cause host to resolve to localhost. This will
result in sending system trying to send to itself. *
nifi.remote.input.socket.port=<*some port> <-- use
netstat to verifiy that this port shows up as a LISTEN port*
nifi.remote.input.secure=true <--
verify that this is only set to true if the NiFi has been configured to run
securely (HTTPS)
Thanks,
Matt
On Thu, Feb 4, 2016 at 2:09 PM, Paresh Shah <paresh.s...@lifelock.com>
wrote:
> The problem is not quite what we thought.
>
> What we see is that we generate a bunch of flowFiles on the sender
> processor and then commit. But all these get delivered to the same remote
> node. What we are expecting is they should be split depending on the no of
> nodes and the loads on each node.
>
>
> Paresh
>
> On 2/4/16, 10:53 AM, "Paresh Shah" <paresh.s...@lifelock.com> wrote:
>
> >We have the pipeline as follows.
> >
> >Sender Side
> > P1..->Pn -> RPG ( connected to ForwardInputPort )
> >
> >Receiver Side
> >ForwardInputPort -> ProcessGroup
> >
> > Inside the processGroup
> > InputPort-> P1 Š-> Pn
> >
> >We see that we sent 10 flow files to the remote cluster. There we see the
> >same set of flow files being sent to different nodes of the cluster. We
> >are confirming this by looking at the DataProvnance and the fileNames in
> >there. They are the same ones for two nodes of the cluster.
> >
> >This behaviour seems to incorrect.
> >
> >Paresh
> >
> >________________________________
> >The information contained in this transmission may contain privileged and
> >confidential information. It is intended only for the use of the
> >person(s) named above. If you are not the intended recipient, you are
> >hereby notified that any review, dissemination, distribution or
> >duplication of this communication is strictly prohibited. If you are not
> >the intended recipient, please contact the sender by reply email and
> >destroy all copies of the original message.
> >________________________________
>
> ________________________________
> The information contained in this transmission may contain privileged and
> confidential information. It is intended only for the use of the person(s)
> named above. If you are not the intended recipient, you are hereby notified
> that any review, dissemination, distribution or duplication of this
> communication is strictly prohibited. If you are not the intended
> recipient, please contact the sender by reply email and destroy all copies
> of the original message.
> ________________________________
>
