[
https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13880077#comment-13880077
]
Tsz Wo (Nicholas), SZE commented on HDFS-5754:
----------------------------------------------
- The patch adds the new system properties (HDFS_SERVICE_LAYOUT etc.) for
initializing LayoutVersion.map. It seems that it won't work for unit tests
since a test may run both NN and DN but we only have one map. I think we need
two maps. Do you agree?
- If HdfsConstants.LAYOUT_VERSION is initialized with NameNode.Feature.values()
as below, the code in DN using HdfsConstants.LAYOUT_VERSION is incorrect. I
guess it needs two LAYOUT_VERSIONs.
{code}
//HdfsConstants
public static final int LAYOUT_VERSION = LayoutVersion
- .getCurrentLayoutVersion();
+ .getCurrentLayoutVersion(NameNode.Feature.values());
{code}
- LayoutFeatureComparator.compare(..) could be simplified as below
{code}
public int compare(LayoutFeature arg0, LayoutFeature arg1) {
return arg0.getLayoutVersion() - arg1.getLayoutVersion();
}
{code}
- In the patch, the Feature enum in LayoutVersion, NameNode and DataNode are
very similar. We could add a new class FeatureInfo for reducing repeated code
as shown in FeatureInfo.patch.
> Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
> ------------------------------------------------------------------------
>
> Key: HDFS-5754
> URL: https://issues.apache.org/jira/browse/HDFS-5754
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: datanode, namenode
> Reporter: Tsz Wo (Nicholas), SZE
> Assignee: Brandon Li
> Attachments: HDFS-5754.001.patch, HDFS-5754.002.patch,
> HDFS-5754.003.patch, HDFS-5754.004.patch, HDFS-5754.006.patch,
> HDFS-5754.007.patch, HDFS-5754.008.patch
>
>
> Currently, LayoutVersion defines the on-disk data format and supported
> features of the entire cluster including NN and DNs. LayoutVersion is
> persisted in both NN and DNs. When a NN/DN starts up, it checks its
> supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a
> different LayoutVersion than NN cannot register with the NN.
> We propose to split LayoutVersion into two independent values that are local
> to the nodes:
> - NamenodeLayoutVersion - defines the on-disk data format in NN, including
> the format of FSImage, editlog and the directory structure.
> - DatanodeLayoutVersion - defines the on-disk data format in DN, including
> the format of block data file, metadata file, block pool layout, and the
> directory structure.
> The LayoutVersion check will be removed in DN registration. If
> NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling
> upgrade, then only rollback is supported and downgrade is not.
--
This message was sent by Atlassian JIRA
(v6.1.5#6160)
