[
https://issues.apache.org/jira/browse/HDFS-7692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leitao Guo updated HDFS-7692:
-----------------------------
Description:
{code:title=DataStorage#addStorageLocations(...)|borderStyle=solid}
for (StorageLocation dataDir : dataDirs) {
File root = dataDir.getFile();
... ...
bpStorage.recoverTransitionRead(datanode, nsInfo, bpDataDirs, startOpt);
addBlockPoolStorage(bpid, bpStorage);
... ...
successVolumes.add(dataDir);
}
{code}
In the above code the storage directories will be analyzed one by one, which is
really time consuming when upgrading HDFS with datanodes have dozens of large
volumes. MultiThread dataDirs analyzing should be supported here to speedup
upgrade.
was:
{code:title=DataStorage#addStorageLocations...)|borderStyle=solid}
for (StorageLocation dataDir : dataDirs) {
File root = dataDir.getFile();
... ...
bpStorage.recoverTransitionRead(datanode, nsInfo, bpDataDirs, startOpt);
addBlockPoolStorage(bpid, bpStorage);
... ...
successVolumes.add(dataDir);
}
{code}
In the above code the storage directories will be analyzed one by one, which is
really time consuming when upgrading HDFS with datanodes have dozens of large
volumes. MultiThread dataDirs analyzing should be supported here to speedup
upgrade.
> DataStorage#addStorageLocations(...) should support MultiThread to speedup
> the upgrade of block pool at multi storage directories.
> ----------------------------------------------------------------------------------------------------------------------------------
>
> Key: HDFS-7692
> URL: https://issues.apache.org/jira/browse/HDFS-7692
> Project: Hadoop HDFS
> Issue Type: Improvement
> Components: datanode
> Affects Versions: 2.5.2
> Reporter: Leitao Guo
>
> {code:title=DataStorage#addStorageLocations(...)|borderStyle=solid}
> for (StorageLocation dataDir : dataDirs) {
> File root = dataDir.getFile();
> ... ...
> bpStorage.recoverTransitionRead(datanode, nsInfo, bpDataDirs,
> startOpt);
> addBlockPoolStorage(bpid, bpStorage);
> ... ...
> successVolumes.add(dataDir);
> }
> {code}
> In the above code the storage directories will be analyzed one by one, which
> is really time consuming when upgrading HDFS with datanodes have dozens of
> large volumes. MultiThread dataDirs analyzing should be supported here to
> speedup upgrade.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
