wangzhihui created HDFS-17463:
---------------------------------
Summary: Title ： Support the switch StringTable Split ID feature
Key: HDFS-17463
URL: https://issues.apache.org/jira/browse/HDFS-17463
Project: Hadoop HDFS
Issue Type: Improvement
Components: namenode
Affects Versions: 3.3.4, 3.3.3, 3.3.5, 3.2.0
Reporter: wangzhihui
Attachments: Image_struct.png, error.png
desc:
*
Hadoop 3.2 introduced optimization features for HDFS StringTable
(b60ca37914b22550e3630fa02742d40697decb3), It resulted in lower versions of
Hadoop upgraded to 3.2 and later versions not supporting downgrade operations.
!error.png!
* This issue has also been discussed in HDFS-14831, and it is recommended to
revert the feature, but it cannot fundamentally solve the problem。
*
Therefore, we have added an optimization to support downgrading
Solution:
* First, we will add the "dfs. image. save. splitId. stringTable" conf switch
"StringTable optimization feature" is enabled
* When the conf value is false, an Image file compatible with lower versions
of HDFS is generated to support downgrading.
*
The difference in HDFS Image file format between Hadoop 3.1.1 and Hadoop 3.2 is
shown in the following figure.
* With the sub-sections feature introduced in HDFS-14617, Protobuf can support
compatible reading.
*
The data structure causing incompatible differences is mainly StringTable.
!Image_struct.png|width=396,height=163!
* In "dfs.image.save.splitId.stringTable = false " the Id generation order of
StringTable starts from 0 to Integer.Max. When true, the Id value range follows
the latest rules.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
