Aleksander Nek created HDFS-18022:
-------------------------------------
Summary: JournalNode serves discarded edits from RPC cache after
rolling-upgrade rollback
Key: HDFS-18022
URL: https://issues.apache.org/jira/browse/HDFS-18022
Project: Hadoop HDFS
Issue Type: Bug
Components: journal-node
Affects Versions: 3.6.0
Reporter: Aleksander Nek
h2. Problem
Journal.discardSegments() removes on-disk segments and rewinds committedTxnId,
but keeps the old RPC edits cache and highestWrittenTxId/FJM readable
transaction state. With dfs.ha.tail-edits.in-progress=true, a restarted
NameNode can replay intentionally discarded operations from warm JournalNodes
after rolling-upgrade rollback.
h2. Minimal reproduction
# Start a journal epoch and finalize segment [1,3].
# Write transactions [4,6] to a second segment (either finalized or still in
progress). Verify getJournaledEdits(4,10) returns those edits.
# Successfully discardSegments(4).
# Request getJournaledEdits(4,10) again.
Expected: no transactions at or after 4, and highest available transaction 3.
Actual: highestWrittenTxId remains 6 and the cache still contains the discarded
edits. In an HA integration reproducer, namespace entries created after the
rollback image reappear after rollback.
h2. Proposed fix and validation
Invalidate the RPC cache after successful discard, rewind highest/FJM readable
TXID, and abort a discarded open segment. Snapshot the cache and readable
boundary together for RPC readers. A rejected mid-segment discard must leave
the existing cache unchanged.
Before-fix unit tests fail for both finalized and in-progress segments
(expected highest 3, actual 6). The focused patch adds coverage for both,
reused TXIDs in a new epoch, and rejected discard. All 19 TestJournal cases
pass on trunk c815ec8e1e98f20e3f0ae2637665d4adb9196660 (JDK 17, macOS ARM64). A
separate warm-JN HA integration test also confirms discarded paths stay absent
without restarting JournalNodes.
This is distinct from HDFS-17710 (edits cached before a failed durable write)
and HDFS-15485 (JNStorage properties after directory rollback). Here the edits
were successfully written and are subsequently discarded deliberately. No
wire/disk format change or new configuration is required.
Contains content generated by OpenAI Codex.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]