Pierre Villard created NIFI-16084:
-------------------------------------
Summary: Load-balanced connection desynchronizes its socket after
an in-flight transaction is unregistered
Key: NIFI-16084
URL: https://issues.apache.org/jira/browse/NIFI-16084
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Reporter: Pierre Villard
Assignee: Pierre Villard
On a cluster, when a load-balanced connection's partition is unregistered while
a transfer transaction is in flight, which happens for all partitions on a
cluster topology change or restart
(SocketLoadBalancedFlowFileQueue.setNodeIdentifiers → partition stop() →
NioAsyncLoadBalanceClient.unregister()), the in-flight LoadBalanceSession is
cancelled. LoadBalanceSession.cancel() only flips the session state; it does
not close the PeerChannel and does not send ABORT_TRANSACTION to the receiving
node.
The client retains and reuses that same open socket for the next transaction,
and every transaction begins by writing protocol-version byte 1 (0x01). The
receiving node, still mid-stream on the abandoned transaction, reads that 0x01
where it expects a protocol indicator, and StandardLoadBalanceProtocol aborts
with:
{code:java}
Failed to receive FlowFiles for Load Balancing due to java.io.IOException:
Expected to receive Transaction Completion Indicator from Peer <host> but
instead received a value of 1{code}
The sender never receives CONFIRM_COMPLETE_TRANSACTION, so it re-queues the
batch. With the "Partition by attribute" strategy (isRebalanceOnFailure=false)
the batch is pinned to the same node and retried indefinitely, leaving
FlowFiles stuck in the connection.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
