[jira] [Commented] (MAPREDUCE-1831) BlockPlacement policy for RAID
[ https://issues.apache.org/jira/browse/MAPREDUCE-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13017000#comment-13017000 ] Hudson commented on MAPREDUCE-1831: --- Integrated in Hadoop-Mapreduce-trunk #643 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk/643/]) > BlockPlacement policy for RAID > -- > > Key: MAPREDUCE-1831 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1831 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.23.0 > > Attachments: MAPREDUCE-1831-v2.txt, MAPREDUCE-1831.20100610.txt, > MAPREDUCE-1831.txt, MAPREDUCE-1831.v1.1.txt, test.result > > > Raid introduce the new dependency between blocks within a file. > The blocks help decode each other. Therefore we should avoid put them on the > same machine. > The proposed BlockPlacementPolicy does the following > 1. When writing parity blocks, it avoid the parity blocks and source blocks > sit together. > 2. When reducing replication number, it deletes the blocks that sits with > other dependent blocks. > 3. It does not change the way we write normal files. It only has different > behavior when processing raid files. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] Commented: (MAPREDUCE-1831) BlockPlacement policy for RAID
[ https://issues.apache.org/jira/browse/MAPREDUCE-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970260#action_12970260 ] Scott Chen commented on MAPREDUCE-1831: --- The failed test is pointed out by MAPREDUCE-2121. And it is irrelevant to this change. > BlockPlacement policy for RAID > -- > > Key: MAPREDUCE-1831 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1831 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.23.0 > > Attachments: MAPREDUCE-1831-v2.txt, MAPREDUCE-1831.20100610.txt, > MAPREDUCE-1831.txt, MAPREDUCE-1831.v1.1.txt, test.result > > > Raid introduce the new dependency between blocks within a file. > The blocks help decode each other. Therefore we should avoid put them on the > same machine. > The proposed BlockPlacementPolicy does the following > 1. When writing parity blocks, it avoid the parity blocks and source blocks > sit together. > 2. When reducing replication number, it deletes the blocks that sits with > other dependent blocks. > 3. It does not change the way we write normal files. It only has different > behavior when processing raid files. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1831) BlockPlacement policy for RAID
[ https://issues.apache.org/jira/browse/MAPREDUCE-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12970257#action_12970257 ] Scott Chen commented on MAPREDUCE-1831: --- {code} [exec] [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system test framework. The patch passed system test framework compile. [exec] [exec] [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == {code} > BlockPlacement policy for RAID > -- > > Key: MAPREDUCE-1831 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1831 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.23.0 > > Attachments: MAPREDUCE-1831-v2.txt, MAPREDUCE-1831.20100610.txt, > MAPREDUCE-1831.txt, MAPREDUCE-1831.v1.1.txt > > > Raid introduce the new dependency between blocks within a file. > The blocks help decode each other. Therefore we should avoid put them on the > same machine. > The proposed BlockPlacementPolicy does the following > 1. When writing parity blocks, it avoid the parity blocks and source blocks > sit together. > 2. When reducing replication number, it deletes the blocks that sits with > other dependent blocks. > 3. It does not change the way we write normal files. It only has different > behavior when processing raid files. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1831) BlockPlacement policy for RAID
[ https://issues.apache.org/jira/browse/MAPREDUCE-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969887#action_12969887 ] Ramkumar Vadali commented on MAPREDUCE-1831: +1 looks good. Please run unit-tests and test-patch. > BlockPlacement policy for RAID > -- > > Key: MAPREDUCE-1831 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1831 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.23.0 > > Attachments: MAPREDUCE-1831-v2.txt, MAPREDUCE-1831.20100610.txt, > MAPREDUCE-1831.txt, MAPREDUCE-1831.v1.1.txt > > > Raid introduce the new dependency between blocks within a file. > The blocks help decode each other. Therefore we should avoid put them on the > same machine. > The proposed BlockPlacementPolicy does the following > 1. When writing parity blocks, it avoid the parity blocks and source blocks > sit together. > 2. When reducing replication number, it deletes the blocks that sits with > other dependent blocks. > 3. It does not change the way we write normal files. It only has different > behavior when processing raid files. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1831) BlockPlacement policy for RAID
[ https://issues.apache.org/jira/browse/MAPREDUCE-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969871#action_12969871 ] Scott Chen commented on MAPREDUCE-1831: --- Here's the review board. https://reviews.apache.org/r/159/ > BlockPlacement policy for RAID > -- > > Key: MAPREDUCE-1831 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1831 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.23.0 > > Attachments: MAPREDUCE-1831-v2.txt, MAPREDUCE-1831.20100610.txt, > MAPREDUCE-1831.txt, MAPREDUCE-1831.v1.1.txt > > > Raid introduce the new dependency between blocks within a file. > The blocks help decode each other. Therefore we should avoid put them on the > same machine. > The proposed BlockPlacementPolicy does the following > 1. When writing parity blocks, it avoid the parity blocks and source blocks > sit together. > 2. When reducing replication number, it deletes the blocks that sits with > other dependent blocks. > 3. It does not change the way we write normal files. It only has different > behavior when processing raid files. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1831) BlockPlacement policy for RAID
[ https://issues.apache.org/jira/browse/MAPREDUCE-1831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12969589#action_12969589 ] Scott Chen commented on MAPREDUCE-1831: --- I have remove the dependency of MAPREDUCE-1861. This patch does chooseTarget for parity file. This does not need MAPREDUCE-1861. I will also rebase the patch in MAPREDUCE-1861. > BlockPlacement policy for RAID > -- > > Key: MAPREDUCE-1831 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-1831 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: contrib/raid >Affects Versions: 0.23.0 >Reporter: Scott Chen >Assignee: Scott Chen > Fix For: 0.23.0 > > Attachments: MAPREDUCE-1831-v2.txt, MAPREDUCE-1831.20100610.txt, > MAPREDUCE-1831.txt, MAPREDUCE-1831.v1.1.txt > > > Raid introduce the new dependency between blocks within a file. > The blocks help decode each other. Therefore we should avoid put them on the > same machine. > The proposed BlockPlacementPolicy does the following > 1. When writing parity blocks, it avoid the parity blocks and source blocks > sit together. > 2. When reducing replication number, it deletes the blocks that sits with > other dependent blocks. > 3. It does not change the way we write normal files. It only has different > behavior when processing raid files. -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.