[
https://issues.apache.org/jira/browse/CAMEL-21659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Rasmussen updated CAMEL-21659:
------------------------------------
Description:
After upgrading to 4.8.3, we started seeing the following warnings in our logs,
quite often:
{noformat}
2025-01-27T06:54:34.226Z
WARN 11 --- [X] [TimeoutExtender]
o.a.c.component.aws2.sqs.Sqs2Consumer ::
[BatchResultErrorEntry(Id=FBDB3014CD64A96-0000000000093816,
SenderFault=true, Code=ReceiptHandleIsInvalid, Message=Message does
not exist or is not available for visibility timeout change)]{noformat}
The error occurs when an attempt to extend message visibility for a message
that has already been deleted in SQS.
In CAMEL-19656, the implementation was switched to using the batched call, but
failed to implement the proper ignore in case of above error – as the old
implementation had.
I also noticed that the current implementation extends ALL messages in the
queue of non-processed – not just those that are about to expire. This happens
because the TimeoutExtender, currently will make a copy of all the values in
the Map, and extend them, in batches.
In case of 100 messages, this will require 10 requests, even though the last
messages in the queue might not need to be extended yet.
If the route is used together with the threads EIP, then the default queue size
is 1000, which makes above algorithm really inefficient.
I suggest introducing a concept of a deadline to each entry in the
TimeoutExtender. The deadline is set to now() plus 80% of the
visibilityTimeout. When sending requests, only messages that will expire before
the next execution of the TimeoutExtender will be extended. The TimeoutExtender
task itself should run more frequently, but will pick up less messages each
time. I suggest it running every visibilityTimeout/2. In a typical
configuration, this would result in executing TimeoutExtender every 10 seconds.
was:
After upgrading to 4.8.3, we I started seeing the following warnings in our
logs, quite often:
{noformat}
2025-01-27T06:54:34.226Z
WARN 11 --- [X] [TimeoutExtender]
o.a.c.component.aws2.sqs.Sqs2Consumer ::
[BatchResultErrorEntry(Id=FBDB3014CD64A96-0000000000093816,
SenderFault=true, Code=ReceiptHandleIsInvalid, Message=Message does
not exist or is not available for visibility timeout change)]{noformat}
The error occurs when an attempt to extend message visibility for a message
that has already been deleted in SQS.
In CAMEL-19656, the implementation was switched to using the batched call, but
failed to implement the proper ignore in case of above error – as the old
implementation had.
I also noticed that the current implementation extends ALL messages in the
queue of non-processed – not just those that are about to expire. This happens
because the TimeoutExtender, currently will make a copy of all the values in
the Map, and extend them, in batches.
In case of 100 messages, this will require 10 requests, even though the last
messages in the queue might not need to be extended yet.
If the route is used together with the threads EIP, then the default queue size
is 1000, which makes above algorithm really inefficient.
I suggest introducing a concept of a deadline to each entry in the
TimeoutExtender. The deadline is set to now() plus 80% of the
visibilityTimeout. When sending requests, only messages that will expire before
the next execution of the TimeoutExtender will be extended. The TimeoutExtender
task itself should run more frequently, but will pick up less messages each
time. I suggest it running every visibilityTimeout/2. In a typical
configuration, this would result in executing TimeoutExtender every 10 seconds.
> Improve aws sqs TimeoutExtender by introducing deadline concept to messages
> ---------------------------------------------------------------------------
>
> Key: CAMEL-21659
> URL: https://issues.apache.org/jira/browse/CAMEL-21659
> Project: Camel
> Issue Type: Improvement
> Components: camel-aws2-sqs
> Affects Versions: 4.8.3
> Reporter: Simon Rasmussen
> Assignee: Simon Rasmussen
> Priority: Major
> Fix For: 4.8.4, 4.10.0
>
>
> After upgrading to 4.8.3, we started seeing the following warnings in our
> logs, quite often:
> {noformat}
> 2025-01-27T06:54:34.226Z
> WARN 11 --- [X] [TimeoutExtender]
> o.a.c.component.aws2.sqs.Sqs2Consumer ::
> [BatchResultErrorEntry(Id=FBDB3014CD64A96-0000000000093816,
> SenderFault=true, Code=ReceiptHandleIsInvalid, Message=Message does
> not exist or is not available for visibility timeout change)]{noformat}
> The error occurs when an attempt to extend message visibility for a message
> that has already been deleted in SQS.
> In CAMEL-19656, the implementation was switched to using the batched call,
> but failed to implement the proper ignore in case of above error – as the old
> implementation had.
> I also noticed that the current implementation extends ALL messages in the
> queue of non-processed – not just those that are about to expire. This
> happens because the TimeoutExtender, currently will make a copy of all the
> values in the Map, and extend them, in batches.
> In case of 100 messages, this will require 10 requests, even though the last
> messages in the queue might not need to be extended yet.
> If the route is used together with the threads EIP, then the default queue
> size is 1000, which makes above algorithm really inefficient.
> I suggest introducing a concept of a deadline to each entry in the
> TimeoutExtender. The deadline is set to now() plus 80% of the
> visibilityTimeout. When sending requests, only messages that will expire
> before the next execution of the TimeoutExtender will be extended. The
> TimeoutExtender task itself should run more frequently, but will pick up less
> messages each time. I suggest it running every visibilityTimeout/2. In a
> typical configuration, this would result in executing TimeoutExtender every
> 10 seconds.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
