shashank created CAMEL-26088:
--------------------------------
Summary: camel-wal - a state update after a roll-over writes at a
stale file position (corrupt log) or marks a newer record, and a shorter record
leaves the end of the old content in the file
Key: CAMEL-26088
URL: https://issues.apache.org/jira/browse/CAMEL-26088
Project: Camel
Issue Type: Bug
Reporter: shashank
Assignee: shashank
{{LogWriter}} keeps the last {{maxRecordCount}} entries in {{TransactionLog}},
a circular buffer with a layer per loop (main 6a2f66ddec3e). After a roll-over
each {{append}} writes the new entry and then the entries of the previous layer
again after it. {{WriteAheadResumeStrategy}} marks each entry
{{PROCESSED}}/{{FAILED}} from the callback of the wrapped strategy, so updates
of the entries appended just before a roll-over arrive after it.
# {{updateState(CachedEntryInfo, ...)}} writes at {{entryInfo.getPosition()}},
the position of the first append, which holds another record once the roll-over
wrote the entries again with records of different sizes: {{LogReader}} then
fails with {{BufferTooSmallException ... 1667457792 bytes needed}} or reads a
wrong record.
# {{TransactionLog.canUpdate}} accepts an update from any older layer when its
index is above the current index: an update two or more layers late marks the
newer record in that slot. It also rejects the entry at the current index (not
replaced yet), so that update is dropped and the record stays {{NEW}} in the
log although it was processed.
# When the new entry is shorter than the one it replaces, the end of the old
content stays after the last entry and {{LogReader}} fails
({{BufferTooSmallException ... 814950 bytes needed}}), without any update.
So the log holds wrong states or cannot be read. (On main a restart does not
read it yet: {{WriteAheadResumeStrategy.start()}} truncates the file before
{{loadCache()}} reads it, a separate defect, see below.)
*Fix*: {{canUpdate}} accepts the same layer, or the previous layer at or after
the current index, never older layers. {{TransactionLog.EntryContainer}} keeps
the current file position of the entry, set by every write, and {{updateState}}
writes there. {{append}} truncates the file after the rewritten entries.
{{append}}, {{updateState}} and {{reset}} take the writer's lock (the one flush
takes), as the update reads the position the append moves and they can run on
different threads.
*Tests*: new {{LogWriterUpdateAfterRollOverPositionTest}}, 4 tests, all fail on
main (the two {{BufferTooSmallException}}s, {{expected: <{dd=NEW, bb=PROCESSED,
cc=NEW}> but was: <{dd=NEW, bb=NEW, cc=NEW}>}}, {{expected: <{g=NEW, e=NEW,
f=NEW}> but was: <{g=NEW, e=NEW, f=PROCESSED}>}}); two {{TransactionLogTest}}
assertions that encoded the off-by-one are corrected. Unit tests of camel-wal
with the fix: 27 run, 0 failures.
*Related*: {{WriteAheadResumeStrategy.start()}} creates the {{LogWriter}},
which opens the file with {{TRUNCATE_EXISTING}}, before {{loadCache()}} reads
it, so the records of the previous run are never replayed (checked with a test:
135 bytes before {{start()}}, 12 after, nothing replayed). Separate JIRA.
*Not changed*: {{LogReader}} gives a {{PersistedLogEntry}} the channel position
after its buffer was filled, not the entry's position, so
{{updateState(PersistedLogEntry, ...)}} during the replay looks wrong too
(read, not run); follow-up.
*How it was found*: spec mining of {{TransactionLogTest.canUpdate}} over all
layer/index pairs plus roll-overs with records of different sizes.
*Affected*: main, camel-4.23.x, camel-4.22.x, camel-4.18.x and camel-4.14.x
(same code). Read, not run.
_Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
