Thomas Gantenbein created CAMEL-23267:
-----------------------------------------
Summary: Lightweight inProgressRepository for file component
Key: CAMEL-23267
URL: https://issues.apache.org/jira/browse/CAMEL-23267
Project: Camel
Issue Type: Improvement
Components: camel-file
Reporter: Thomas Gantenbein
See also [#camel > file consumer does not release items in
SimpleLRUCache|https://camel.zulipchat.com/#narrow/channel/257298-camel/topic/file.20consumer.20does.20not.20release.20items.20in.20SimpleLRUCache/with/581658577]
*Observation*
By default, the file component is using an MemoryIdempotentRepository as its
inProgressRepository. This, in turn, is using by default an instance of
SimpleLRU cache. The SimpleLRU cache maintains a Map with the values themselves
("delegate") as well as a list of recent changes ("lastChanges").
The GenericFileOnCompletion _does_ remove the absolute path of the processed
file from the "delegate" in the SimpleLRUCache, but it does not remove it from
the lastChanges queue. So when the queue is full (2 times the cache capacity =
100'000), whenever a file is added to the inProgressRepository, that Map with
lastChanges gets iterated over and copied. And it never gets below a size of
100'000 again.
This leads to high CPU load as soon as many files are processed by an endpoint
after 100'000 files have been processed.
*Proposed solution*
Replace the SimpleLRUCache with a LinkedHashMap as the backing store of the
MemoryIdempotentRepository used as the default inProgressRepository of the file
component.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
