[
https://issues.apache.org/jira/browse/CAMEL-23602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-23602 started by Federico Mariani.
------------------------------------------------
> threads EIP does not respect maxQueueSize when using virtual threads
> --------------------------------------------------------------------
>
> Key: CAMEL-23602
> URL: https://issues.apache.org/jira/browse/CAMEL-23602
> Project: Camel
> Issue Type: Bug
> Components: came-core
> Affects Versions: 4.20.0
> Reporter: Simon Rasmussen
> Assignee: Federico Mariani
> Priority: Major
>
> We are looking into switching from platform threads to virtual threads. We
> have multiple routes, consuming from SQS queues. To achieve high throughput
> today, we use threads EIP like this:
>
> {code:java}
> from(someSqsQueue?&waitTimeSeconds=10&maxMessagesPerPoll=10&extendMessageVisibility=true).threads().maxQueueSize(50).bean(service);{code}
> as you can see, we use the threads EIP to do the actual processing. This way,
> we have a single consumer thread, using long poll, to fetch up to 10 messages
> at a time, and offload them to a pool of threads to get processed. The max
> queue size is set relatively low, because if it gets full, then
> extendMessageVisibility kicks in, and we don’t want to do this across a too
> large set of messages. It also has the benefit that the polling thread will
> pick up processing, effectively backing off from pulling in more load.
> Now, if we switch to using virtual threads, we observe that the max queue
> size is ignored, and we keep pulling in messages unlimited – isn’t this a bug
> in threads EIP when combined with virtual threads?
> I can use a custom executor service with threads to overcome this, but is
> that really the best way?
> Our motivation for switching to virtual threads is due to having too many
> platform threads when the number of consumers using this pattern increases
> within an application.
> see also:
> https://camel.zulipchat.com/#narrow/channel/257298-camel/topic/Unbounded.20threads.20EIP.20when.20using.20virtual.20threads.3F/with/596960264
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
