[
https://issues.apache.org/jira/browse/CAMEL-23267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18072521#comment-18072521
]
Thomas Gantenbein commented on CAMEL-23267:
-------------------------------------------
I agree that the remove operation should stay O(1), so I'm absolutely fine with
_not_ removing items from that Deque. I also agree that a real alternative
based on some existing library is worth considering.
Just to explain the motivation for the LinkedHashMap thing: I didn't see it as
an alternative to the SimpleLRUCache, but as an alternative for the "backend"
of the MemoryIdempotentRepository, which is used by the file component and
others as an inProgressRepository. Looking from outside, I wondered "Why does
that inProgressRepository even _need_ a 'cache'? There's nothing to be
'cached', but to be stored and removed if failed or done." The fact that the
MemoryIdempotentRepository used the SimpleLRUCache looked more like "hey, it
does the job that needs to be done and has no risk of becoming a memory leak,
so let's use that, even if we don't really need cache and LRU semantics for the
inProgressRepository". Thread-safety is guaranteed, I thought, by the locks
used by the MemoryIdempotentRepository, but I might well be wrong.
Closing the same as you did: That's only my point of view. And I can absolutely
understand if you and others disagree.
> File consumer does not release items in SimpleLRUCache
> ------------------------------------------------------
>
> Key: CAMEL-23267
> URL: https://issues.apache.org/jira/browse/CAMEL-23267
> Project: Camel
> Issue Type: Bug
> Components: camel-core
> Reporter: Thomas Gantenbein
> Assignee: Nicolas Filotto
> Priority: Minor
> Fix For: 4.14.6, 4.18.2, 4.19.0
>
>
> See also [#camel > file consumer does not release items in
> SimpleLRUCache|https://camel.zulipchat.com/#narrow/channel/257298-camel/topic/file.20consumer.20does.20not.20release.20items.20in.20SimpleLRUCache/with/581658577]
> *Observation*
> By default, the file component is using an MemoryIdempotentRepository as its
> inProgressRepository. This, in turn, is using by default an instance of
> SimpleLRU cache. The SimpleLRU cache maintains a Map with the values
> themselves ("delegate") as well as a list of recent changes ("lastChanges").
> The GenericFileOnCompletion _does_ remove the absolute path of the processed
> file from the "delegate" in the SimpleLRUCache, but it does not remove it
> from the lastChanges queue. So when the queue is full (2 times the cache
> capacity = 100'000), whenever a file is added to the inProgressRepository,
> that Map with lastChanges gets iterated over and copied. And it never gets
> below a size of 100'000 again.
> This leads to high CPU load as soon as many files are processed by an
> endpoint after 100'000 files have been processed.
> *Proposed solution*
> Replace the SimpleLRUCache with a LinkedHashMap as the backing store of the
> MemoryIdempotentRepository used as the default inProgressRepository of the
> file component.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
