[
https://issues.apache.org/jira/browse/CAMEL-26088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on CAMEL-26088 started by shashank.
----------------------------------------
> camel-wal - a state update after a roll-over writes at a stale file position
> (corrupt log) or marks a newer record, and a shorter record leaves the end of
> the old content in the file
> --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: CAMEL-26088
> URL: https://issues.apache.org/jira/browse/CAMEL-26088
> Project: Camel
> Issue Type: Bug
> Reporter: shashank
> Assignee: shashank
> Priority: Major
>
> {{LogWriter}} keeps the last {{maxRecordCount}} entries in
> {{TransactionLog}}, a circular buffer with a layer per loop (main
> 6a2f66ddec3e). After a roll-over each {{append}} writes the new entry and
> then the entries of the previous layer again after it.
> {{WriteAheadResumeStrategy}} marks each entry {{PROCESSED}}/{{FAILED}} from
> the callback of the wrapped strategy, so updates of the entries appended just
> before a roll-over arrive after it.
> # {{updateState(CachedEntryInfo, ...)}} writes at
> {{entryInfo.getPosition()}}, the position of the first append, which holds
> another record once the roll-over wrote the entries again with records of
> different sizes: {{LogReader}} then fails with {{BufferTooSmallException ...
> 1667457792 bytes needed}} or reads a wrong record.
> # {{TransactionLog.canUpdate}} accepts an update from any older layer when
> its index is above the current index: an update two or more layers late marks
> the newer record in that slot. It also rejects the entry at the current index
> (not replaced yet), so that update is dropped and the record stays {{NEW}} in
> the log although it was processed.
> # When the new entry is shorter than the one it replaces, the end of the old
> content stays after the last entry and {{LogReader}} fails
> ({{BufferTooSmallException ... 814950 bytes needed}}), without any update.
> So the log holds wrong states or cannot be read. (On main a restart does not
> read it yet: {{WriteAheadResumeStrategy.start()}} truncates the file before
> {{loadCache()}} reads it, a separate defect, see below.)
> *Fix*: {{canUpdate}} accepts the same layer, or the previous layer at or
> after the current index, never older layers.
> {{TransactionLog.EntryContainer}} keeps the current file position of the
> entry, set by every write, and {{updateState}} writes there. {{append}}
> truncates the file after the rewritten entries. {{append}}, {{updateState}}
> and {{reset}} take the writer's lock (the one flush takes), as the update
> reads the position the append moves and they can run on different threads.
> *Tests*: new {{LogWriterUpdateAfterRollOverPositionTest}}, 4 tests, all fail
> on main (the two {{BufferTooSmallException}}s, {{expected: <{dd=NEW,
> bb=PROCESSED, cc=NEW}> but was: <{dd=NEW, bb=NEW, cc=NEW}>}}, {{expected:
> <{g=NEW, e=NEW, f=NEW}> but was: <{g=NEW, e=NEW, f=PROCESSED}>}}); two
> {{TransactionLogTest}} assertions that encoded the off-by-one are corrected.
> Unit tests of camel-wal with the fix: 27 run, 0 failures.
> *Related*: {{WriteAheadResumeStrategy.start()}} creates the {{LogWriter}},
> which opens the file with {{TRUNCATE_EXISTING}}, before {{loadCache()}} reads
> it, so the records of the previous run are never replayed (checked with a
> test: 135 bytes before {{start()}}, 12 after, nothing replayed). Separate
> JIRA.
> *Not changed*: {{LogReader}} gives a {{PersistedLogEntry}} the channel
> position after its buffer was filled, not the entry's position, so
> {{updateState(PersistedLogEntry, ...)}} during the replay looks wrong too
> (read, not run); follow-up.
> *How it was found*: spec mining of {{TransactionLogTest.canUpdate}} over all
> layer/index pairs plus roll-overs with records of different sizes.
> *Affected*: main, camel-4.23.x, camel-4.22.x, camel-4.18.x and camel-4.14.x
> (same code). Read, not run.
> _Filed with Claude Code on behalf of allthingssecurity._
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
