[
https://issues.apache.org/jira/browse/HDFS-7967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15802511#comment-15802511
]
Kihwal Lee commented on HDFS-7967:
----------------------------------
The 2.8 patch works fine with jdk8. The unit tests ran fine too.
{noformat}
Results :
Tests run: 4100, Failures: 0, Errors: 0, Skipped: 17
{noformat}
But it doesn't compile with jdk7.
{noformat}
[ERROR]
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java:[47,25]
error: package java.util.function does not exist
[ERROR]
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeStorageInfo.java:[23,25]
error: package java.util.function does not exist
[ERROR]
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java:[3999,10]
error: cannot find symbol
[ERROR] symbol: class Predicate
location: class BlockManager
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeStorageInfo.java:[325,40]
error: cannot find symbol
[ERROR] symbol: class Predicate
location: class DatanodeStorageInfo
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeStorageInfo.java:[92,18]
error: cannot find symbol
[ERROR] symbol: class Predicate
location: class DatanodeStorageInfo.BlockIterator
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeStorageInfo.java:[96,18]
error: cannot find symbol
[ERROR]
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java:[4000,10]
error: cannot find symbol
[ERROR] symbol: class Predicate
location: class BlockManager
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java:[4015,10]
error: BlockManager.BalancerBlockIterator is not abstract and does not
override abstract method remove() in Iterator
[ERROR]
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java:[4023,11]
error: cannot find symbol
[ERROR] symbol: method sort(Comparator<DatanodeStorageInfo.BlockIterator>)
location: variable iters of type ArrayList<DatanodeStorageInfo.BlockIterator>
/testptch/hadoop/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java:[4042,13]
error: cannot find symbol
{noformat}
> Reduce the performance impact of the balancer
> ---------------------------------------------
>
> Key: HDFS-7967
> URL: https://issues.apache.org/jira/browse/HDFS-7967
> Project: Hadoop HDFS
> Issue Type: Sub-task
> Components: namenode
> Affects Versions: 2.0.0-alpha
> Reporter: Daryn Sharp
> Assignee: Daryn Sharp
> Priority: Critical
> Attachments: HDFS-7967-branch-2.8.patch, HDFS-7967-branch-2.patch
>
>
> The balancer needs to query for blocks to move from overly full DNs. The
> block lookup is extremely inefficient. An iterator of the node's blocks is
> created from the iterators of its storages' blocks. A random number is
> chosen corresponding to how many blocks will be skipped via the iterator.
> Each skip requires costly scanning of triplets.
> The current design also only considers node imbalances while ignoring
> imbalances within the nodes's storages. A more efficient and intelligent
> design may eliminate the costly skipping of blocks via round-robin selection
> of blocks from the storages based on remaining capacity.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-h...@hadoop.apache.org
