[
https://issues.apache.org/jira/browse/HADOOP-1912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12539275
]
Hadoop QA commented on HADOOP-1912:
-----------------------------------
+1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12368396/replace6.patch
against trunk revision r590273.
@author +1. The patch does not contain any @author tags.
javadoc +1. The javadoc tool did not generate any warning messages.
javac +1. The applied patch does not generate any new compiler warnings.
findbugs +1. The patch does not introduce any new Findbugs warnings.
core tests +1. The patch passed core unit tests.
contrib tests +1. The patch passed contrib unit tests.
Test results:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1046/testReport/
Findbugs warnings:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1046/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
Checkstyle results:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1046/artifact/trunk/build/test/checkstyle-errors.html
Console output:
http://lucene.zones.apache.org:8080/hudson/job/Hadoop-Patch/1046/console
This message is automatically generated.
> Datanode should support block replacement
> -----------------------------------------
>
> Key: HADOOP-1912
> URL: https://issues.apache.org/jira/browse/HADOOP-1912
> Project: Hadoop
> Issue Type: New Feature
> Components: dfs
> Affects Versions: 0.14.1
> Reporter: Hairong Kuang
> Assignee: Hairong Kuang
> Fix For: 0.16.0
>
> Attachments: replace.patch, replace1.patch, replace2.patch,
> replace3.patch, replace4.patch, replace5.patch, replace6.patch
>
>
> This jira Data Node's support for rebalancing (HADOOP-1652). When a balancer
> decides to move a block B from Source S to Destination D. It also chooses a
> proxy source PS, which contains a replica of B, to speed up block copy. The
> block placement is carried in the following steps:
> 1. A block copy command is sent to datanode PS in the format of
> "OP_BLOCK_COPY <block_id_of_B> <source S> <destination D>". It requests PS to
> copy B to datanode D.
> 2. PS then transfers block B to datanode D with a block replacement command
> to D in the format of "OP_BLOCK_REPLACEMENT <block_id_of_B> <source S>
> <data_of_B>".
> 3. Datanode D writes the block B to its disk and then sends a name node a
> blockReceived RPC informing the namenode that a block B is received and
> please delete a replica of B from source S if there is any excessive replica.
> 4. The namenode then adds datanode D to block B's map and removes an exesive
> replicas of B in favor of datanode S.
> In addition, each data node has a limited bandwidth for rebalancing. The
> default value for the bandwidth is 5MB/s. Throttling is done at both source &
> destination sides. Each data node limits maximum number of concurrent data
> transfers (including both sending and receiving) for the rebalancing purpose
> to be 5. In the worst case, each data transfer has a limited bandwidth of
> 1MB/s. Each sender & receiver has a Throttler. The primary method of the
> class is "throttle( int numOfBytes )". The parameter numOfBytes indicates the
> total number of bytes that the caller has sent or received since the last
> throttle is called. The method calculates the caller's I/O rate. If the rate
> is faster than the bandwidth limit, it sleeps to slow down the data transfer.
> After it wakes up, it adjusts its bandwidth limit if the number of concurrent
> data transfers is changed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
