[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13466618#comment-13466618 ] Matt Foley commented on MAPREDUCE-4511: --- In branch-1.1, committed 2-line change to this patch, r1392179, to avoid use of interface HasFileDescriptor, which doesn't exist until 1.2.0. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Fix For: 1.1.0, 2.0.2-alpha Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_branch-1_rev7.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435804#comment-13435804 ] Todd Lipcon commented on MAPREDUCE-4511: I don't know if Ahmed ran benchmarks on the current iteration of the patch, but I saw 10-20% improvement on terasort and some other reduce-heavy customer workloads with this patch on branch-1 back when I originally proposed it. So I don't see why the new implementation wouldn't have similar gains. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Fix For: 1.2.0, 2.2.0-alpha Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_branch-1_rev7.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435901#comment-13435901 ] Ahmed Radwan commented on MAPREDUCE-4511: - For branch-12 last iterations, I have seen 7-10% improvement on terasort and other custom example jobs. The actual % improvement may differ based on the jobs and how heavy their reduce-side is. I have also used the default readahead length in my tests, tuning this parameter for different setups can also have an impact. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Fix For: 1.2.0, 2.2.0-alpha Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_branch-1_rev7.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435944#comment-13435944 ] Hudson commented on MAPREDUCE-4511: --- Integrated in Hadoop-Hdfs-trunk #1136 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1136/]) MAPREDUCE-4511. Add IFile readahead (ahmed via tucu) (Revision 1373669) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1373669 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFile.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFileInputStream.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestIFileStreams.java Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Fix For: 1.2.0, 2.2.0-alpha Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_branch-1_rev7.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435975#comment-13435975 ] Hudson commented on MAPREDUCE-4511: --- Integrated in Hadoop-Mapreduce-trunk #1168 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1168/]) MAPREDUCE-4511. Add IFile readahead (ahmed via tucu) (Revision 1373669) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1373669 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFile.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFileInputStream.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestIFileStreams.java Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Fix For: 1.2.0, 2.2.0-alpha Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_branch-1_rev7.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13434860#comment-13434860 ] Hadoop QA commented on MAPREDUCE-4511: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12540738/MAPREDUCE-4511_trunk_rev5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2732//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2732//console This message is automatically generated. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[
https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435502#comment-13435502
]
Alejandro Abdelnur commented on MAPREDUCE-4511:
---
Looks good, some minor improvements:
*IFileInputStream init*:
{code}
if (conf != null){
readahead = conf.getBoolean(mapreduce.ifile.readahead, true);
readaheadLength = conf.getInt(mapreduce.ifile.readahead.bytes,
4 * 1024 * 1024);
} else {
readahead = true;
readaheadLength = 4 * 1024 * 1024;
}
{code}
How about?
{code}
conf = (conf !=null) ? conf : new Configuration();
readahead = conf.getBoolean(mapreduce.ifile.readahead, true);
readaheadLength = conf.getInt(mapreduce.ifile.readahead.bytes,
{code}
Instead using literals, we should have constants keys and default in
MRConfig.java and have the defaults in mapred-default.xml.
*IFileInputStream.getFileDescriptorIfAvail*: Use a local var for FileDescriptor
initialized to null, set it in the IF/ELIF and have a single return at the end
of the method.
*IFileInputSTream.doReadahead()*: the first condition should be 'raPool !=
null', if there is not readahead pool avail, no point in checking if the job is
configured to do it and if there is a FD
*testcase* just tests there are no regressions, right? I think it is OK, would
not be that easy to test it anyway.
Add IFile readahead
---
Key: MAPREDUCE-4511
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv1, mrv2, performance
Reporter: Ahmed Radwan
Assignee: Ahmed Radwan
Attachments: MAPREDUCE-4511_branch1.patch,
MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch,
MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch,
MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch,
MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch,
MAPREDUCE-4511_trunk_rev5.patch
This ticket is to add IFile readahead as part of HADOOP-7714.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435566#comment-13435566 ] Ahmed Radwan commented on MAPREDUCE-4511: - Thanks Tucu for the review! I agree about testing. I have updated the patches per your comments. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[
https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435568#comment-13435568
]
Alejandro Abdelnur commented on MAPREDUCE-4511:
---
+1 for patch for trunk, pending jenkins.
I don't think patch for branch-1 is correct, you should not add those constants
to the {{JobConf}} as they won't be in trunk/branch-2 thus breaking backwards
compatibility. They should be somewhere else, just not in the {{JobConf}}.
Add IFile readahead
---
Key: MAPREDUCE-4511
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv1, mrv2, performance
Reporter: Ahmed Radwan
Assignee: Ahmed Radwan
Attachments: MAPREDUCE-4511_branch1.patch,
MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch,
MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch,
MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_trunk.patch,
MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch,
MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch,
MAPREDUCE-4511_trunk_rev6.patch
This ticket is to add IFile readahead as part of HADOOP-7714.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435591#comment-13435591 ] Hadoop QA commented on MAPREDUCE-4511: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12541143/MAPREDUCE-4511_trunk_rev6.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: org.apache.hadoop.mapreduce.lib.input.TestCombineFileInputFormat +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2741//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2741//console This message is automatically generated. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435594#comment-13435594 ] Alejandro Abdelnur commented on MAPREDUCE-4511: --- test failure seems unrelated, it has been going one for a while already. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435595#comment-13435595 ] Ahmed Radwan commented on MAPREDUCE-4511: - Thanks Tucu, I see you point about keeping the JobConf properties compatible with MR2. I have updated the branch-1 patch to have these constants in the class itself. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_branch-1_rev7.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435604#comment-13435604 ] Alejandro Abdelnur commented on MAPREDUCE-4511: --- +1 on rev7 for branch-1. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Fix For: 1.2.0, 2.2.0-alpha Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_branch-1_rev7.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435605#comment-13435605 ] Hudson commented on MAPREDUCE-4511: --- Integrated in Hadoop-Common-trunk-Commit #2583 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2583/]) MAPREDUCE-4511. Add IFile readahead (ahmed via tucu) (Revision 1373669) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1373669 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFile.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFileInputStream.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestIFileStreams.java Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Fix For: 1.2.0, 2.2.0-alpha Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_branch-1_rev7.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435608#comment-13435608 ] Hudson commented on MAPREDUCE-4511: --- Integrated in Hadoop-Hdfs-trunk-Commit #2648 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2648/]) MAPREDUCE-4511. Add IFile readahead (ahmed via tucu) (Revision 1373669) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1373669 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFile.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFileInputStream.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestIFileStreams.java Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Fix For: 1.2.0, 2.2.0-alpha Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_branch-1_rev7.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435647#comment-13435647 ] Hudson commented on MAPREDUCE-4511: --- Integrated in Hadoop-Mapreduce-trunk-Commit #2612 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2612/]) MAPREDUCE-4511. Add IFile readahead (ahmed via tucu) (Revision 1373669) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1373669 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFile.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/IFileInputStream.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRConfig.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestIFileStreams.java Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Fix For: 1.2.0, 2.2.0-alpha Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_branch-1_rev7.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13435758#comment-13435758 ] Arun C Murthy commented on MAPREDUCE-4511: -- Did we do any performance benchmark and see any improvements? Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv1, mrv2, performance Reporter: Ahmed Radwan Assignee: Ahmed Radwan Fix For: 1.2.0, 2.2.0-alpha Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_branch-1_rev6.patch, MAPREDUCE-4511_branch-1_rev7.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch, MAPREDUCE-4511_trunk_rev6.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[
https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=1340#comment-1340
]
Alejandro Abdelnur commented on MAPREDUCE-4511:
---
We need HADOOP-7754 in order for this patch to apply and be test-patch-ed, in
the mean time, a better name for the method {{determineFd}} would be
{{getFileDescriptorIfAvail}}.
Add IFile readahead
---
Key: MAPREDUCE-4511
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv1, mrv2
Reporter: Ahmed Radwan
Assignee: Ahmed Radwan
Attachments: MAPREDUCE-4511_branch1.patch,
MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch,
MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_trunk.patch,
MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch,
MAPREDUCE-4511_trunk_rev4.patch
This ticket is to add IFile readahead as part of HADOOP-7714.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433465#comment-13433465 ] Ahmed Radwan commented on MAPREDUCE-4511: - Here are the updated patches with the new method name. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13433470#comment-13433470 ] Hadoop QA commented on MAPREDUCE-4511: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12540738/MAPREDUCE-4511_trunk_rev5.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. -1 javac. The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2725//console This message is automatically generated. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch-1_rev5.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch, MAPREDUCE-4511_trunk_rev5.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13432727#comment-13432727 ] Hadoop QA commented on MAPREDUCE-4511: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12540461/MAPREDUCE-4511_trunk_rev4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. -1 javac. The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2723//console This message is automatically generated. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch-1_rev4.patch, MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch, MAPREDUCE-4511_trunk_rev4.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13431580#comment-13431580 ] Ahmed Radwan commented on MAPREDUCE-4511: - Here are the new patches based on HADOOP-7754 updated patches. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13431584#comment-13431584 ] Hadoop QA commented on MAPREDUCE-4511: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539961/MAPREDUCE-4511_trunk_rev3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. -1 javac. The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2718//console This message is automatically generated. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch-1_rev3.patch, MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch, MAPREDUCE-4511_trunk_rev3.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13430732#comment-13430732 ] Ahmed Radwan commented on MAPREDUCE-4511: - Here are updated versions for both trunk and branch-1 adding the ability to disable and configure the readahead and its length, respectively. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_trunk.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13430744#comment-13430744 ] Hadoop QA commented on MAPREDUCE-4511: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539743/MAPREDUCE-4511_branch-1_rev2.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2717//console This message is automatically generated. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch-1_rev2.patch, MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_trunk.patch, MAPREDUCE-4511_trunk_rev2.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13429034#comment-13429034 ] Ahmed Radwan commented on MAPREDUCE-4511: - Here is the trunk patch. Similarly this patch requires the trunk patch from HADOOP-7754. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_trunk.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13429347#comment-13429347 ] Hadoop QA commented on MAPREDUCE-4511: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539253/MAPREDUCE-4511_trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 javac. The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2711//console This message is automatically generated. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan Attachments: MAPREDUCE-4511_branch1.patch, MAPREDUCE-4511_trunk.patch This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4511) Add IFile readahead
[ https://issues.apache.org/jira/browse/MAPREDUCE-4511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13427968#comment-13427968 ] Ahmed Radwan commented on MAPREDUCE-4511: - Here is the updated branch-1 patch based on Todd's HADOOP-7714 patches. Note that this patch requires HADOOP-7754 patch. Add IFile readahead --- Key: MAPREDUCE-4511 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4511 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Reporter: Ahmed Radwan Assignee: Ahmed Radwan This ticket is to add IFile readahead as part of HADOOP-7714. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
