[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Attachment: HDFS-5754.013.patch Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: FeatureInfo.patch, HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch, HDFS-5754.004.patch, HDFS-5754.006.patch, HDFS-5754.007.patch, HDFS-5754.008.patch, HDFS-5754.009.patch, HDFS-5754.010.patch, HDFS-5754.012.patch, HDFS-5754.013.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-5754: - Hadoop Flags: Reviewed +1 patch looks good. Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Fix For: HDFS-5535 (Rolling upgrades) Attachments: FeatureInfo.patch, HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch, HDFS-5754.004.patch, HDFS-5754.006.patch, HDFS-5754.007.patch, HDFS-5754.008.patch, HDFS-5754.009.patch, HDFS-5754.010.patch, HDFS-5754.012.patch, HDFS-5754.013.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Attachment: HDFS-5754.012.patch Uploaded a new patch to address the comments. Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: FeatureInfo.patch, HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch, HDFS-5754.004.patch, HDFS-5754.006.patch, HDFS-5754.007.patch, HDFS-5754.008.patch, HDFS-5754.009.patch, HDFS-5754.010.patch, HDFS-5754.012.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Attachment: HDFS-5754.009.patch Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: FeatureInfo.patch, HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch, HDFS-5754.004.patch, HDFS-5754.006.patch, HDFS-5754.007.patch, HDFS-5754.008.patch, HDFS-5754.009.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Attachment: HDFS-5754.010.patch Rebased the patch with the HDFS-5535 branch along with a couple unit test fixes. Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: FeatureInfo.patch, HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch, HDFS-5754.004.patch, HDFS-5754.006.patch, HDFS-5754.007.patch, HDFS-5754.008.patch, HDFS-5754.009.patch, HDFS-5754.010.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tsz Wo (Nicholas), SZE updated HDFS-5754: - Attachment: FeatureInfo.patch Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: FeatureInfo.patch, HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch, HDFS-5754.004.patch, HDFS-5754.006.patch, HDFS-5754.007.patch, HDFS-5754.008.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Attachment: HDFS-5754.006.patch Currently there is one LayoutVersion instance per NameNode process. After we split it into NameNode LayoutVersion and DataNode LayoutVersion, we will have one instance in NameNode, and one instance in DataNode. This could be a problem for unit tests since MiniDFSCluster has both NameNode and DataNode as threads. Therefore, I've added a static LayoutVersion variable in both NameNode and DataNode to make sure that MiniDFSCluster won't have problem running unit tests later on. I've uploaded a new patch. The features before rolling upgrade are duplicated in both NameNode.Feature and DataNode.Feature just to simplify the code change. I can keep the common features in LayoutVersion class if it's preferred. Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch, HDFS-5754.004.patch, HDFS-5754.006.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Attachment: HDFS-5754.007.patch Uploaded a new patch, which keeps the features previous than ROLLING_UPGRADE_MARKER in LayoutVersion calss. Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch, HDFS-5754.004.patch, HDFS-5754.006.patch, HDFS-5754.007.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Attachment: HDFS-5754.008.patch Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch, HDFS-5754.004.patch, HDFS-5754.006.patch, HDFS-5754.007.patch, HDFS-5754.008.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Attachment: HDFS-5754.004.patch Thank you, Nicholas! I've uploaded a new patch to address your comments. The new patch is based on HDFS-5535 branch. I can make a trunk patch if you think trunk is a better choice. Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch, HDFS-5754.004.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Status: Open (was: Patch Available) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch, HDFS-5754.004.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Attachment: HDFS-5754.003.patch Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: HDFS-5754.001.patch, HDFS-5754.002.patch, HDFS-5754.003.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Attachment: HDFS-5754.001.patch Uploaded the initial patch. It added a datanode layout version and removed namenode version check during datanode registration. Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: HDFS-5754.001.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Status: Patch Available (was: Open) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: HDFS-5754.001.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Updated] (HDFS-5754) Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion
[ https://issues.apache.org/jira/browse/HDFS-5754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Li updated HDFS-5754: - Attachment: HDFS-5754.002.patch Patch rebased. Split LayoutVerion into NamenodeLayoutVersion and DatanodeLayoutVersion Key: HDFS-5754 URL: https://issues.apache.org/jira/browse/HDFS-5754 Project: Hadoop HDFS Issue Type: Sub-task Components: datanode, namenode Reporter: Tsz Wo (Nicholas), SZE Assignee: Brandon Li Attachments: HDFS-5754.001.patch, HDFS-5754.002.patch Currently, LayoutVersion defines the on-disk data format and supported features of the entire cluster including NN and DNs. LayoutVersion is persisted in both NN and DNs. When a NN/DN starts up, it checks its supported LayoutVersion against the on-disk LayoutVersion. Also, a DN with a different LayoutVersion than NN cannot register with the NN. We propose to split LayoutVersion into two independent values that are local to the nodes: - NamenodeLayoutVersion - defines the on-disk data format in NN, including the format of FSImage, editlog and the directory structure. - DatanodeLayoutVersion - defines the on-disk data format in DN, including the format of block data file, metadata file, block pool layout, and the directory structure. The LayoutVersion check will be removed in DN registration. If NamenodeLayoutVersion or DatanodeLayoutVersion is changed in a rolling upgrade, then only rollback is supported and downgrade is not. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
