Hi Faisai,
Adding ControlRate processor before sending FlowFiles via RPG, you can
throttle the rate of sending data, that should help reducing the
probability for receiving side to get full.
If the current overall throughput is acceptable for your use-case, and
you don't see any data loss, then y
Hi Joe/Koji,
I cant seem to figure out a way to reduce the back pressure or to find the
root cause of the errors
1.Unable to communicate with remote instance Peer [] due to
java.io.EOFException; closing connection
2.indicates that port 37e64bd0-5326-3c3f-80f4-42a828dea1d5's destination is
ful
you can allow for larger backlogs by increasing the backpressure thresholds
OR you can add additional nodes OR you can expire data.
The whole point of the backpressure and pressure release features are to
let you be in control of how many resources are dedicated to buffering
data. However, in the
Hi Koji,
I moved onto another cluster of Nifi nodes , did the same configuration for
S2S there and boom.. the same error message all over the logs.(nothing on
the bulletin board)
Could it be because of the back pressure as i also get the error
-(indicates that port 8c77c1b0-0164-1000--05
Hi Faisai,
I think both error messages indicating the same thing, that is network
communication is closed in the middle of a Site-to-Site transaction.
That can be happen due to many reasons, such as freaky network, or
manually stop the port or RPG while some transaction is being
processed. I don't
Hi Koji,
In the subsequent tests the above error did not come but now we are getting
errors on the RPG :
RemoteGroupPort[name=1_pk_ip,targets=http://xx.prod.xx.local:9090/nifi/]
failed to communicate with remote NiFi instance due to
java.io.IOException: Failed to confirm transaction with
Peer
Hi Koji ,
Thank you for your reply. I updated the logback.xml and ran the test again.
I can see an additional error in the app.log which is as below.
o.a.nifi.remote.SocketRemoteSiteListener
java.io.EOFException: null
at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:340)
Hello,
1. The error message sounds like the client disconnects in the middle
of Site-to-Site communication. Enabling debug log would show more
information, by adding at conf/logback.xml.
2. I'd suggest checking if your 4 nodes receive data evenly (well
distributed). Connection status history, 'Q
Hi, I've got two questions
1.We are using Remote Process Group with Raw transport protocol to
distribute the data across four node cluster. I see the nifi app log has a
lot of instance of the below error
1. o.a.nifi.remote.SocketRemoteSiteListener Unable to communicate
with remote instance Pe