[
https://issues.apache.org/jira/browse/HDFS-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leitao Guo updated HDFS-7692:
-----------------------------
Description:
{code:title=BlockPoolSliceStorage#loadBpStorageDirectories(...)|borderStyle=solid}
for (File dataDir : dataDirs) {
if (containsStorageDir(dataDir)) {
throw new IOException(
"BlockPoolSliceStorage.recoverTransitionRead: " +
"attempt to load an used block storage: " + dataDir);
}
StorageDirectory sd =
loadStorageDirectory(datanode, nsInfo, dataDir, startOpt);
succeedDirs.add(sd);
}
{code}
In the above code the storage directories will be analyzed one by one, which is
really time consuming when upgrading HDFS with datanodes have dozens of large
volumes. MultiThread dataDirs analyzing should be supported here to speedup
upgrade.
was:
{code:title=BlockPoolSliceStorage#loadBpStorageDirectories(...)|borderStyle=solid}
for (File dataDir : dataDirs) {
if (containsStorageDir(dataDir)) {
throw new IOException(
"BlockPoolSliceStorage.recoverTransitionRead: " +
"attempt to load an used block storage: " + dataDir);
}
StorageDirectory sd =
loadStorageDirectory(datanode, nsInfo, dataDir, startOpt);
succeedDirs.add(sd);
}
{code}
In the above code the storage directories will be analyzed one by one, which is
really time consuming when upgrading HDFS with datanodes have dozens of large
volumes.
> BlockPoolSliceStorage#loadBpStorageDirectories(...) should support
> MultiThread to speedup the upgrade of block pool at multi storage directories.
> -------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-7692
> URL: https://issues.apache.org/jira/browse/HDFS-7692
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.5.2
> Reporter: Leitao Guo
>
> {code:title=BlockPoolSliceStorage#loadBpStorageDirectories(...)|borderStyle=solid}
> for (File dataDir : dataDirs) {
> if (containsStorageDir(dataDir)) {
> throw new IOException(
> "BlockPoolSliceStorage.recoverTransitionRead: " +
> "attempt to load an used block storage: " + dataDir);
> }
> StorageDirectory sd =
> loadStorageDirectory(datanode, nsInfo, dataDir, startOpt);
> succeedDirs.add(sd);
> }
> {code}
> In the above code the storage directories will be analyzed one by one, which
> is really time consuming when upgrading HDFS with datanodes have dozens of
> large volumes. MultiThread dataDirs analyzing should be supported here to
> speedup upgrade.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
