[
https://issues.apache.org/jira/browse/HDFS-6038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923725#comment-13923725
]
Jing Zhao commented on HDFS-6038:
---------------------------------
bq. DataOutputBuffer.writeInt can be simplified as below.
Here we cannot call DataOutputBuffer#writeInt. The issue is that
DataOutputStream#writeInt will increase the total number of written bytes by 4
(DataOutputStream#written), and the total number of written bytes will later be
retrieved by EditsDoubleBuffer#countReadyBytes, and used by
QuorumOutputStream#flushAndSync to determine the size of the data flushed to
JNs. Since our writeInt(int, int) method is actually modifying some previous
data, the total number of bytes written should be unchanged. Directly calling
DataOutputBuffer#writeInt there will append extra 4 bytes (0x00000000) for each
editlog transaction recorded in JNs.
> JournalNode hardcodes NameNodeLayoutVersion in the edit log file
> ----------------------------------------------------------------
>
> Key: HDFS-6038
> URL: https://issues.apache.org/jira/browse/HDFS-6038
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, ha, hdfs-client, namenode
> Reporter: Haohui Mai
> Assignee: Jing Zhao
> Attachments: HDFS-6038.000.patch, HDFS-6038.001.patch,
> HDFS-6038.002.patch, HDFS-6038.003.patch, HDFS-6038.004.patch,
> HDFS-6038.005.patch
>
>
> In HA setup, the JNs receive edit logs (blob) from the NN and write into edit
> log files. In order to write well-formed edit log files, the JNs prepend a
> header for each edit log file.
> The problem is that the JN hard-codes the version (i.e.,
> {{NameNodeLayoutVersion}} in the edit log, therefore it generates incorrect
> edit logs when the newer release bumps the {{NameNodeLayoutVersion}} during
> rolling upgrade.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
