[
https://issues.apache.org/jira/browse/CAMEL-25213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Claus Ibsen resolved CAMEL-25213.
---------------------------------
Resolution: Fixed
> camel-support - FileStateRepository can lose the stored offsets: stop
> rewrites the file without the lock, the rewrite truncates the file first, and
> an incomplete line prevents a restart
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-25213
> URL: https://issues.apache.org/jira/browse/CAMEL-25213
> Project: Camel
> Issue Type: Bug
> Components: camel-support
> Reporter: shashank
> Assignee: shashank
> Priority: Minor
> Fix For: 4.23.0
>
>
> {{FileStateRepository}} is the file based {{StateRepository}} documented for
> the camel-kafka {{offsetRepository}} option (and usable as the MongoDB change
> stream token repository). It keeps the state in a {{HashMap}} and appends a
> {{key=value}} line to the file on every {{setState}}; when the file reaches
> {{maxFileStoreSize}}, and on every {{doStop}}, it rewrites the whole file
> from the map ({{trunkStore}}). Three defects can lose the stored state:
> # *{{doStop}} does not hold the lock.* {{setState}}/{{getState}} hold
> {{cacheAndStoreLock}}, {{doStop}} calls {{trunkStore()}} and
> {{cache.clear()}} without it. A {{setState}} while the repository stops
> (another route sharing the repository, a Kafka consumer thread still
> committing after {{shutdownTimeout}}, or the MongoDB change stream thread,
> which {{MongoDbChangeStreamsConsumer.doStop}} does not wait for) modifies the
> map while {{trunkStore}} iterates it: {{doStop}} fails with
> {{ConcurrentModificationException}} and the file keeps only the lines written
> until then.
> # *The rewrite truncates the file first.* {{trunkStore}} opens {{new
> FileOutputStream(fileStore)}} (truncate) and then writes the entries, so any
> failure or crash while rewriting (on every stop, and every time the file
> reaches the size limit) leaves the file empty or partial.
> # *An incomplete line prevents a restart.* {{appendToStore}} writes key,
> {{=}}, value and newline as four separate writes; if the process dies in
> between, the last line has no {{=}}, and {{loadStore}} fails with
> {{StringIndexOutOfBoundsException}} ({{line.substring(0, -1)}}), so the
> repository, and the consumer that starts it, cannot start.
> For a Kafka consumer, a lost offset means the partition restarts from
> {{auto.offset.reset}} ({{latest}} by default, so the records in between are
> skipped; {{earliest}} replays the partition), because the component disables
> Kafka's own commits when an offset repository is set.
> The {{doStop}} path has not held the lock since the class was added
> (CAMEL-20199 only turned the {{synchronized}} blocks into the
> {{ReentrantLock}}).
> h3. Reproduction
> Deterministic tests against the class (three runs each):
> * a map as 1st level cache whose iteration pauses while {{stop()}} rewrites
> the file; another thread calls {{setState}} meanwhile: {{stop()}} fails with
> {{ConcurrentModificationException}}, and after a restart three of the five
> stored keys are gone (the file holds only the first two lines);
> * a map whose iteration fails half way while {{stop()}} rewrites the file:
> the file loses the entries after the failure;
> * a store file {{"key1=value1\nkey2=value2\nkey3"}}: {{start()}} fails with
> {{StringIndexOutOfBoundsException: Range [0, -1) out of bounds for length 4}}.
> h3. Proposed fix
> * {{doStop}} holds {{cacheAndStoreLock}}, as {{reset()}} does.
> * {{trunkStore}} writes into {{<file>.tmp}}, syncs it and moves it over the
> file store ({{ATOMIC_MOVE}}, with a plain replace where the file system does
> not support it); on failure the temporary file is deleted and the store is
> unchanged.
> * {{appendToStore}} writes the line with a single write.
> * {{loadStore}} skips a line without {{=}} with a WARN.
> No API or file format change. Tests: three new tests in
> {{FileStateRepositoryTest}}, which fail without the fix.
> Affected: all versions (the same code at camel-3.x and 4.x, now in
> camel-support).
> Duplicate check (2026-09-30): JIRA text "FileStateRepository",
> "StateRepository", "offsetRepository": only CAMEL-20212 (move to
> camel-support), CAMEL-12732 and CAMEL-13710 (Kafka manual commit and
> examples), CAMEL-23994 (Kafka offset handling, not the repository). GitHub
> pull requests "FileStateRepository": none for this.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
