[
https://issues.apache.org/jira/browse/HDFS-16147?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
liuyongpan updated HDFS-16147:
------------------------------
Release Note:
In HDFS-14617, it allows the inode and inode directory sections of the fsimage
to be loaded in parallel. But it can't turn on parallelism and compression at
the same time. I fixed this defect.
1、When compression and parallel are turned on at the same time, oiv can still
work normally. Although each section is split into multiple sections and
compressed, it can still be parsed correctly.
2、If you create a parallel compressed image with this patch, and then try to
load it in a NN without this patch and parallel loading disabled, NN is still
able to load it.
Both of these may sound crazy, but it works. This may happen because the
decompressed code can splits each section correctly.
was:
In HDFS-14617, it allows the inode and inode directory sections of the fsimage
to be loaded in parallel. But it can't turn on parallelism and compression at
the same time. I fixed this defect.
1、When compression and parallel are turned on at the same time, oiv can still
work normally. Although each section is split into multiple sections and
compressed, it can still be parsed correctly.
2、If you create a parallel compressed image with this patch, and then try to
load it in a NN without this patch and parallel loading disabled, NN is still
able to load it.
Both of these may sound crazy, but it works. This may happen because the
decompressed code can splits each section correctly
> load fsimage with parallelization and compression
> -------------------------------------------------
>
> Key: HDFS-16147
> URL: https://issues.apache.org/jira/browse/HDFS-16147
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: namanode
> Affects Versions: 3.3.0
> Reporter: liuyongpan
> Priority: Minor
> Attachments: HDFS-16147.001.patch, HDFS-16147.002.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
