[
https://issues.apache.org/jira/browse/CAMEL-25700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-25700 started by shashank.
----------------------------------------
> camel-aws2-sqs - with extendMessageVisibility=true and maxMessagesPerPoll >
> 1, the messages that wait in the batch are not extended and are delivered a
> second time
> -------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25700
> URL: https://issues.apache.org/jira/browse/CAMEL-25700
> Project: Camel
> Issue Type: Bug
> Components: camel-aws2-sqs
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> With {{extendMessageVisibility=true}} the SQS consumer keeps the messages it
> is processing invisible: a background task ({{TimeoutExtender}}) calls
> {{ChangeMessageVisibilityBatch}} for every message it knows about before its
> visibility timeout runs out.
> A poll receives up to {{maxMessagesPerPoll}} messages at once, and their
> visibility timeout starts at that receive. {{Sqs2Consumer.processBatch}} then
> routes them one after the other, and registers each message with the extender
> only when its own turn comes ({{timeoutExtender.add(exchange)}} inside the
> loop, line 151 on main d46d51c24550). While the first messages are processed
> (synchronous route), the ones that wait behind them are not extended at all.
> When the processing of the earlier messages takes longer than
> {{visibilityTimeout}}, SQS makes the waiting messages visible again and
> delivers them to another consumer (or to the next poll of this one): they are
> processed twice, and the delete of the first copy uses a receipt handle that
> is no longer current.
> This is the scenario of CAMEL-10587 (2016, camel-aws):
> {{maxMessagesPerPoll=10}}, {{extendMessageVisibility=true}},
> {{visibilityTimeout=30}}, a slow route, "after 30 seconds only 1 message is
> in flight". That ticket was closed as Abandoned; its only commit
> (c6a15565acdf, 2017) started the extender's executor before the first poll;
> the per-message registration stayed in the loop, and camel-aws2-sqs (and the
> batched extender of CAMEL-19656) kept it.
> Trigger: {{extendMessageVisibility=true}} (the option users set because their
> processing is slow) with {{maxMessagesPerPoll}} greater than 1 and a
> synchronous route where the messages before the last one together take longer
> than {{visibilityTimeout}}. Example: 10 messages per poll, 10 seconds each,
> {{visibilityTimeout=30}}: at least messages 4 to 10 are delivered again (2
> and 3 are extended only at about the moment they expire). With the default
> {{maxMessagesPerPoll}} (one message per poll) nothing changes.
> h3. Reproduction
> New {{SqsConsumerExtendMessageVisibilityBatchTest}} with the module's
> {{AmazonSQSClientMock}} (no AWS): two messages on the queue,
> {{maxMessagesPerPoll=2&visibilityTimeout=2&extendMessageVisibility=true}},
> the route holds the first message until the extender has sent a
> {{ChangeMessageVisibilityBatch}} entry for the receipt handle of the second
> (at most 10 seconds). On main it never does:
> {noformat}
> [the visibility of the second message of the batch is extended while the
> first message is processed (otherwise SQS makes it visible again and delivers
> it a second time)]
> Expecting value to be true but was false
> {noformat}
> h3. Proposed fix
> {{processBatch}} registers every message of the batch with the extender
> before it routes the first one, and removes the messages it does not route
> (the consumer is stopping or suspending, or the processor threw) so that they
> become visible again as they do today. The extender's own rule (extend an
> entry when its deadline is near) is unchanged, so a message that is processed
> quickly is still not extended.
> A second new test suspends the route while the first message is processed:
> the second message is not routed and must not be extended afterwards (it
> passes on main, where the message is never registered; it guards the removal).
> camel-aws2-sqs unit tests with the fix: 77 run, 0 failures, 0 skipped. Not
> run: the integration tests (LocalStack), a real queue.
> How it was found: reading the ack and visibility paths of the SQS consumer in
> a sweep of acknowledge/commit/delete timing; the test confirmed it.
> Affected: main (tested). camel-4.23.x, camel-4.22.x and camel-4.18.x register
> the message with the extender inside the processing loop the same way (line
> 151; read, not run).
> Duplicate check (2026-10-16): JIRA text "extendMessageVisibility"
> (CAMEL-10587, CAMEL-18780, CAMEL-17156, CAMEL-5944, CAMEL-16956: none open
> about the batch), "TimeoutExtender" / "visibility extender" (CAMEL-19562,
> CAMEL-19656, CAMEL-21550, CAMEL-21659, CAMEL-21769: start delay, batching,
> deadlines, an error that extended forever; none about the waiting messages of
> a batch); GitHub pull requests "sqs extend", "Sqs2Consumer visibility"
> (#9108, #10828, #17262, #16592, #16945, #11102; all merged, none changes
> where a message is registered).
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
