[jira] Commented: (MAPREDUCE-1816) HAR files used for RAID parity need to have configurable partfile size
[ https://issues.apache.org/jira/browse/MAPREDUCE-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12926065#action_12926065 ] Hudson commented on MAPREDUCE-1816: --- Integrated in Hadoop-Mapreduce-trunk-Commit #523 (See [https://hudson.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/523/]) HAR files used for RAID parity need to have configurable partfile size -- Key: MAPREDUCE-1816 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1816 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Affects Versions: 0.20.1 Reporter: Ramkumar Vadali Assignee: Ramkumar Vadali Priority: Minor Fix For: 0.22.0 Attachments: MAPREDUCE-1816.patch, MAPREDUCE-1816.patch.1 RAID parity files are merged into HAR archives periodically. This is required to reduce the number of files that the NameNode has to track. The number of files present in a HAR archive depends on the size of HAR part files - higher the size, lower the number of files. The size of HAR part files is configurable through the setting har.partfile.size, but that is a global setting. This task introduces a new setting specific to raid.har.partfile.size, that is used in-turn to set har.partfile.size -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1816) HAR files used for RAID parity need to have configurable partfile size
[ https://issues.apache.org/jira/browse/MAPREDUCE-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12916263#action_12916263 ] Ramkumar Vadali commented on MAPREDUCE-1816: Test results: ant test: All succeeded except TestUlimit, which is an unrelated failure. ant test-patch: [exec] [exec] -1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] -1 tests included. The patch doesn't appear to include any new or modified tests. [exec] Please justify why no new tests are needed for this patch. [exec] Also please list what manual steps were performed to verify this patch. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] [exec] +1 release audit. The applied patch does not increase the total number of release audit warnings. [exec] [exec] +1 system tests framework. The patch passed system tests framework compile. [exec] [exec] [exec] [exec] [exec] == [exec] == [exec] Finished build. [exec] == [exec] == I think it is OK to not have a unit-test for this patch since this only changes configuration parameters. I have tested this manually and verified that it works HAR files used for RAID parity need to have configurable partfile size -- Key: MAPREDUCE-1816 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1816 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Affects Versions: 0.20.1 Reporter: Ramkumar Vadali Assignee: Ramkumar Vadali Priority: Minor Fix For: 0.22.0 Attachments: MAPREDUCE-1816.patch, MAPREDUCE-1816.patch.1 RAID parity files are merged into HAR archives periodically. This is required to reduce the number of files that the NameNode has to track. The number of files present in a HAR archive depends on the size of HAR part files - higher the size, lower the number of files. The size of HAR part files is configurable through the setting har.partfile.size, but that is a global setting. This task introduces a new setting specific to raid.har.partfile.size, that is used in-turn to set har.partfile.size -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1816) HAR files used for RAID parity need to have configurable partfile size
[ https://issues.apache.org/jira/browse/MAPREDUCE-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12916342#action_12916342 ] Rodrigo Schmidt commented on MAPREDUCE-1816: Do I need to +1 again? :) +1 HAR files used for RAID parity need to have configurable partfile size -- Key: MAPREDUCE-1816 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1816 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Affects Versions: 0.20.1 Reporter: Ramkumar Vadali Assignee: Ramkumar Vadali Priority: Minor Fix For: 0.22.0 Attachments: MAPREDUCE-1816.patch, MAPREDUCE-1816.patch.1 RAID parity files are merged into HAR archives periodically. This is required to reduce the number of files that the NameNode has to track. The number of files present in a HAR archive depends on the size of HAR part files - higher the size, lower the number of files. The size of HAR part files is configurable through the setting har.partfile.size, but that is a global setting. This task introduces a new setting specific to raid.har.partfile.size, that is used in-turn to set har.partfile.size -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1816) HAR files used for RAID parity need to have configurable partfile size
[ https://issues.apache.org/jira/browse/MAPREDUCE-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12885400#action_12885400 ] Rodrigo Schmidt commented on MAPREDUCE-1816: +1 Patch looks good. HAR files used for RAID parity need to have configurable partfile size -- Key: MAPREDUCE-1816 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1816 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Affects Versions: 0.20.1 Reporter: Ramkumar Vadali Priority: Minor Fix For: 0.22.0 Attachments: MAPREDUCE-1816.patch, MAPREDUCE-1816.patch.1 RAID parity files are merged into HAR archives periodically. This is required to reduce the number of files that the NameNode has to track. The number of files present in a HAR archive depends on the size of HAR part files - higher the size, lower the number of files. The size of HAR part files is configurable through the setting har.partfile.size, but that is a global setting. This task introduces a new setting specific to raid.har.partfile.size, that is used in-turn to set har.partfile.size -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1816) HAR files used for RAID parity need to have configurable partfile size
[ https://issues.apache.org/jira/browse/MAPREDUCE-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12884267#action_12884267 ] Hadoop QA commented on MAPREDUCE-1816: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12448457/MAPREDUCE-1816.patch.1 against trunk revision 959509. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/278/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/278/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/278/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/278/console This message is automatically generated. HAR files used for RAID parity need to have configurable partfile size -- Key: MAPREDUCE-1816 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1816 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Affects Versions: 0.20.1 Reporter: Ramkumar Vadali Priority: Minor Fix For: 0.22.0 Attachments: MAPREDUCE-1816.patch, MAPREDUCE-1816.patch.1 RAID parity files are merged into HAR archives periodically. This is required to reduce the number of files that the NameNode has to track. The number of files present in a HAR archive depends on the size of HAR part files - higher the size, lower the number of files. The size of HAR part files is configurable through the setting har.partfile.size, but that is a global setting. This task introduces a new setting specific to raid.har.partfile.size, that is used in-turn to set har.partfile.size -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1816) HAR files used for RAID parity need to have configurable partfile size
[ https://issues.apache.org/jira/browse/MAPREDUCE-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12871531#action_12871531 ] Rodrigo Schmidt commented on MAPREDUCE-1816: Hi Ram, Your patch is changing spaces and tabs which makes it hard to check. Please change it. You need some unit tests as well. Thanks for working on this! HAR files used for RAID parity need to have configurable partfile size -- Key: MAPREDUCE-1816 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1816 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Affects Versions: 0.20.1 Reporter: Ramkumar Vadali Priority: Minor Attachments: MAPREDUCE-1816.patch RAID parity files are merged into HAR archives periodically. This is required to reduce the number of files that the NameNode has to track. The number of files present in a HAR archive depends on the size of HAR part files - higher the size, lower the number of files. The size of HAR part files is configurable through the setting har.partfile.size, but that is a global setting. This task introduces a new setting specific to raid.har.partfile.size, that is used in-turn to set har.partfile.size -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Commented: (MAPREDUCE-1816) HAR files used for RAID parity need to have configurable partfile size
[ https://issues.apache.org/jira/browse/MAPREDUCE-1816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=12871954#action_12871954 ] Hadoop QA commented on MAPREDUCE-1816: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12445530/MAPREDUCE-1816.patch against trunk revision 947758. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/205/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/205/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/205/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Mapreduce-Patch-h4.grid.sp2.yahoo.net/205/console This message is automatically generated. HAR files used for RAID parity need to have configurable partfile size -- Key: MAPREDUCE-1816 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1816 Project: Hadoop Map/Reduce Issue Type: Improvement Components: contrib/raid Affects Versions: 0.20.1 Reporter: Ramkumar Vadali Priority: Minor Attachments: MAPREDUCE-1816.patch RAID parity files are merged into HAR archives periodically. This is required to reduce the number of files that the NameNode has to track. The number of files present in a HAR archive depends on the size of HAR part files - higher the size, lower the number of files. The size of HAR part files is configurable through the setting har.partfile.size, but that is a global setting. This task introduces a new setting specific to raid.har.partfile.size, that is used in-turn to set har.partfile.size -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
