[
https://issues.apache.org/jira/browse/NIFI-15892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Turcsanyi resolved NIFI-15892.
------------------------------------
Resolution: Won't Do
The issue will be addressed using a different approach in NIFI-15935.
> ConsumeAMQP should nack batched deliveries when ProcessSession commit fails
> ---------------------------------------------------------------------------
>
> Key: NIFI-15892
> URL: https://issues.apache.org/jira/browse/NIFI-15892
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Affects Versions: 2.9.0
> Reporter: Claudio Mattioni
> Priority: Major
> Labels: AMQP
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> ConsumeAMQP with Auto-Acknowledge Messages set to false and Batch Size
> greater than 1 can leave RabbitMQ deliveries unacknowledged until the broker
> closes the channel due to consumer_timeout.
> Current behavior observed with NiFi/MiNiFi 2.9.0:
> - ConsumeAMQP is configured with:
> - Auto-Acknowledge Messages = false
> - Batch Size > 1
> - Prefetch Count > 1
> - RabbitMQ eventually logs:
> Consumer ... has timed out waiting for a consumer acknowledgement of a
> delivery.
> Timeout used: 1800000 ms.
> operation none caused a channel exception precondition_failed:
> delivery acknowledgement on channel 1 timed out.
> - MiNiFi then logs:
> AMQP failure, dropping the client
> AMQPException: AMQP client has lost connection
> Consumer is closed, discarding message
> This resembles the older NIFI-5896 behavior, where Batch Size > 1 with
> Auto-Acknowledge disabled could result in messages remaining unacknowledged.
> NIFI-5896 was closed with a workaround of Batch Size = 1, but the current
> ConsumeAMQP implementation still appears to acknowledge batched messages only
> after ProcessSession commit, using a cumulative ack for the last delivery tag.
> The current implementation handles the successful commit path, but does not
> explicitly nack/requeue outstanding batched deliveries when the
> ProcessSession commit fails or cannot complete successfully. In such cases,
> messages can remain unacknowledged until the RabbitMQ channel is closed by
> consumer_timeout.
> Proposed improvement:
> When Auto-Acknowledge Messages is false and a batch has been consumed:
> 1. On successful ProcessSession commit:
> - basicAck(lastDeliveryTag, true)
> 2. On ProcessSession commit failure:
> - basicNack(lastDeliveryTag, true, true)
> This would preserve the documented at-least-once behavior while allowing
> Batch Size > 1 to be used safely.
> P.S. This is marked as Major because it can make ConsumeAMQP unreliable with
> Auto-Acknowledge=false and Batch Size > 1, causing RabbitMQ channels to be
> closed due to consumer acknowledgement timeout. A workaround exists by using
> Batch Size = 1, but that significantly limits throughput.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
