[
https://issues.apache.org/jira/browse/HDFS-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13921602#comment-13921602
]
Jing Zhao commented on HDFS-6038:
---------------------------------
After offline discussion with [~szetszwo], the 002 patch did the following
change:
1. Persist a length field for each editlog op to indicate the total length of
the op
2. In JournalNode, instead of calling EditLogFileInputStream#validateEditLog to
get the last txid of an in-progress editlog segment, we add a new method
scanEditLog which does not decode each editlog op. Instead, the new method
reads the length and txid of each op, and use the length to quickly jump to the
next op.
The 002 patch is just a preliminary patch to demonstrate the idea. Still need
to fix unit tests and run system tests.
> JournalNode hardcodes NameNodeLayoutVersion in the edit log file
> ----------------------------------------------------------------
>
> Key: HDFS-6038
> URL: https://issues.apache.org/jira/browse/HDFS-6038
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, ha, hdfs-client, namenode
> Reporter: Haohui Mai
> Assignee: Jing Zhao
> Attachments: HDFS-6038.000.patch, HDFS-6038.001.patch,
> HDFS-6038.002.patch
>
>
> In HA setup, the JNs receive edit logs (blob) from the NN and write into edit
> log files. In order to write well-formed edit log files, the JNs prepend a
> header for each edit log file.
> The problem is that the JN hard-codes the version (i.e.,
> {{NameNodeLayoutVersion}} in the edit log, therefore it generates incorrect
> edit logs when the newer release bumps the {{NameNodeLayoutVersion}} during
> rolling upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
