tomscut commented on pull request #4101: URL: https://github.com/apache/hadoop/pull/4101#issuecomment-1085245717
> @tomscut Thanks for your review. Yes, background reconstruction is necessary. With missing block(s) of block group, client has to read `dataUnits` blocks in block group to decode the target data. This causes `dataUnits` times of network traffic and corresponding cpu costs. In our product cluster, we encountered backgroud reconstruction exhausts the network traffic of DN node. This leads to the failure of client read/write. The speed of read/write and reconstruction is in conflict, we can not have both of them actually. And in practice, we set throttler of EC reconstruction to half of available network traffic, for example, 500 MB/s to 1GB/s NIC. In this PR, we set the default value to 0, which indicates there is no limit to reconstruction at all, to be sure there is no side effect to upgrade. Thanks @cndaimin for your explanation. Hi @tasanuma @Hexiaoqiao, could you please give me some advice? Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-issues-h...@hadoop.apache.org
