[jira] [Commented] (MAPREDUCE-2492) [MAPREDUCE] The new MapReduce API should make available task's progress to the task
[ https://issues.apache.org/jira/browse/MAPREDUCE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13038637#comment-13038637 ] Hudson commented on MAPREDUCE-2492: --- Integrated in Hadoop-Mapreduce-trunk #689 (See [https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk/689/]) MAPREDUCE-2492. The new MapReduce API should make available task's progress to the task. (amarrk) amarrk : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVNview=revrev=1126591 Files : * /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/UtilsForTests.java * /hadoop/mapreduce/trunk/CHANGES.txt * /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestReporter.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/TaskAttemptContextImpl.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/Task.java * /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapreduce/TestTaskContext.java * /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapreduce/MapReduceTestUtil.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/lib/map/WrappedMapper.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/Reporter.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/TaskAttemptContext.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/lib/map/MultithreadedMapper.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/lib/reduce/WrappedReducer.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/lib/chain/ChainReduceContextImpl.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/lib/output/MultipleOutputs.java * /hadoop/mapreduce/trunk/src/contrib/mrunit/src/java/org/apache/hadoop/mrunit/mock/MockReporter.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/lib/chain/ChainMapContextImpl.java * /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestMapProgress.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/lib/join/Parser.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/MapTask.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/StatusReporter.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/ReduceTask.java * /hadoop/mapreduce/trunk/src/contrib/mrunit/src/java/org/apache/hadoop/mrunit/mapreduce/mock/MockReporter.java * /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapreduce/task/TaskAttemptContextImpl.java [MAPREDUCE] The new MapReduce API should make available task's progress to the task --- Key: MAPREDUCE-2492 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2492 Project: Hadoop Map/Reduce Issue Type: Improvement Components: task Affects Versions: 0.23.0 Reporter: Amar Kamat Assignee: Amar Kamat Fix For: 0.23.0 Attachments: MAPREDUCE-2492-v1.3.patch, MAPREDUCE-2492-v1.4.patch, MAPREDUCE-2492-v1.5.patch, MAPREDUCE-2492-v1.6.patch, MAPREDUCE-2492-v1.7.patch There is no way to get the task's current progress in the new MapReduce API. It would be nice to make it available so that the task (map/reduce) can use it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2492) [MAPREDUCE] The new MapReduce API should make available task's progress to the task
[ https://issues.apache.org/jira/browse/MAPREDUCE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13038004#comment-13038004 ] Hadoop QA commented on MAPREDUCE-2492: -- +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480106/MAPREDUCE-2492-v1.7.patch against trunk revision 1125599. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/293//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/293//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/293//console This message is automatically generated. [MAPREDUCE] The new MapReduce API should make available task's progress to the task --- Key: MAPREDUCE-2492 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2492 Project: Hadoop Map/Reduce Issue Type: Improvement Components: task Affects Versions: 0.23.0 Reporter: Amar Kamat Assignee: Amar Kamat Attachments: MAPREDUCE-2492-v1.3.patch, MAPREDUCE-2492-v1.4.patch, MAPREDUCE-2492-v1.5.patch, MAPREDUCE-2492-v1.6.patch, MAPREDUCE-2492-v1.7.patch There is no way to get the task's current progress in the new MapReduce API. It would be nice to make it available so that the task (map/reduce) can use it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2492) [MAPREDUCE] The new MapReduce API should make available task's progress to the task
[ https://issues.apache.org/jira/browse/MAPREDUCE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13038084#comment-13038084 ] Tom White commented on MAPREDUCE-2492: -- This is an incompatible change, since it adds a method to the public @Stable mapred.Reporter interface. Is it possible to rework this to only change the new API, as the title suggests? This would then be a compatible change since the classes that have been changed in the new API are private or @Evolving. Also, it looks like this wasn't reviewed before being committed. [MAPREDUCE] The new MapReduce API should make available task's progress to the task --- Key: MAPREDUCE-2492 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2492 Project: Hadoop Map/Reduce Issue Type: Improvement Components: task Affects Versions: 0.23.0 Reporter: Amar Kamat Assignee: Amar Kamat Fix For: 0.23.0 Attachments: MAPREDUCE-2492-v1.3.patch, MAPREDUCE-2492-v1.4.patch, MAPREDUCE-2492-v1.5.patch, MAPREDUCE-2492-v1.6.patch, MAPREDUCE-2492-v1.7.patch There is no way to get the task's current progress in the new MapReduce API. It would be nice to make it available so that the task (map/reduce) can use it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2492) [MAPREDUCE] The new MapReduce API should make available task's progress to the task
[ https://issues.apache.org/jira/browse/MAPREDUCE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13038134#comment-13038134 ] Tom White commented on MAPREDUCE-2492: -- I think it makes sense to have Reporter provide the task's progress to the map/reduce task attempts. I would prefer marking this change as incompatible. It's certainly an improvement, but does it warrant an incompatible change to an interface marked as stable? There may be an argument that it does, but I would have expected to see some discussion about this before it was committed. Why not only change the new API and tell users that they need to use that one if they want to use this feature? [MAPREDUCE] The new MapReduce API should make available task's progress to the task --- Key: MAPREDUCE-2492 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2492 Project: Hadoop Map/Reduce Issue Type: Improvement Components: task Affects Versions: 0.23.0 Reporter: Amar Kamat Assignee: Amar Kamat Fix For: 0.23.0 Attachments: MAPREDUCE-2492-v1.3.patch, MAPREDUCE-2492-v1.4.patch, MAPREDUCE-2492-v1.5.patch, MAPREDUCE-2492-v1.6.patch, MAPREDUCE-2492-v1.7.patch There is no way to get the task's current progress in the new MapReduce API. It would be nice to make it available so that the task (map/reduce) can use it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2492) [MAPREDUCE] The new MapReduce API should make available task's progress to the task
[
https://issues.apache.org/jira/browse/MAPREDUCE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13038214#comment-13038214
]
Chris Douglas commented on MAPREDUCE-2492:
--
This is partially my fault. I only had time for a quick pass over the patch,
and had missed the mapred API change among the updates to {{mapreduce.lib}}
classes.
I agree with Tom. It's a useful feature, but changing only the new API is
probably the better course. Any breakage is unlikely- it's adding, not removing
a method from a framework type almost never implemented by users- but I'd lean
away from any modifications to the old APIs unless they affect correctness.
[MAPREDUCE] The new MapReduce API should make available task's progress to
the task
---
Key: MAPREDUCE-2492
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2492
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: task
Affects Versions: 0.23.0
Reporter: Amar Kamat
Assignee: Amar Kamat
Fix For: 0.23.0
Attachments: MAPREDUCE-2492-v1.3.patch, MAPREDUCE-2492-v1.4.patch,
MAPREDUCE-2492-v1.5.patch, MAPREDUCE-2492-v1.6.patch,
MAPREDUCE-2492-v1.7.patch
There is no way to get the task's current progress in the new MapReduce API.
It would be nice to make it available so that the task (map/reduce) can use
it.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2492) [MAPREDUCE] The new MapReduce API should make available task's progress to the task
[ https://issues.apache.org/jira/browse/MAPREDUCE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13037539#comment-13037539 ] Hadoop QA commented on MAPREDUCE-2492: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12480023/MAPREDUCE-2492-v1.5.patch against trunk revision 1125599. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.mapred.TestReporter +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/290//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/290//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/290//console This message is automatically generated. [MAPREDUCE] The new MapReduce API should make available task's progress to the task --- Key: MAPREDUCE-2492 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2492 Project: Hadoop Map/Reduce Issue Type: Improvement Components: task Affects Versions: 0.23.0 Reporter: Amar Kamat Assignee: Amar Kamat Attachments: MAPREDUCE-2492-v1.3.patch, MAPREDUCE-2492-v1.4.patch, MAPREDUCE-2492-v1.5.patch There is no way to get the task's current progress in the new MapReduce API. It would be nice to make it available so that the task (map/reduce) can use it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2492) [MAPREDUCE] The new MapReduce API should make available task's progress to the task
[ https://issues.apache.org/jira/browse/MAPREDUCE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13037298#comment-13037298 ] Hadoop QA commented on MAPREDUCE-2492: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12479987/MAPREDUCE-2492-v1.4.patch against trunk revision 1125599. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these core unit tests: org.apache.hadoop.mapred.TestReporter -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/288//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/288//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/288//console This message is automatically generated. [MAPREDUCE] The new MapReduce API should make available task's progress to the task --- Key: MAPREDUCE-2492 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2492 Project: Hadoop Map/Reduce Issue Type: Improvement Components: task Affects Versions: 0.23.0 Reporter: Amar Kamat Assignee: Amar Kamat Attachments: MAPREDUCE-2492-v1.3.patch, MAPREDUCE-2492-v1.4.patch There is no way to get the task's current progress in the new MapReduce API. It would be nice to make it available so that the task (map/reduce) can use it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2492) [MAPREDUCE] The new MapReduce API should make available task's progress to the task
[ https://issues.apache.org/jira/browse/MAPREDUCE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13037311#comment-13037311 ] Amar Kamat commented on MAPREDUCE-2492: --- I downloaded the latest patch and ran the failed test. The testcase which failed on Hudson passed on my local box. Attached here (http://pastebin.com/qaKZ57TE) is the run log. See line no 13, 37, 53 for progress values in map/reduce task cleanup. [MAPREDUCE] The new MapReduce API should make available task's progress to the task --- Key: MAPREDUCE-2492 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2492 Project: Hadoop Map/Reduce Issue Type: Improvement Components: task Affects Versions: 0.23.0 Reporter: Amar Kamat Assignee: Amar Kamat Attachments: MAPREDUCE-2492-v1.3.patch, MAPREDUCE-2492-v1.4.patch There is no way to get the task's current progress in the new MapReduce API. It would be nice to make it available so that the task (map/reduce) can use it. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2492) [MAPREDUCE] The new MapReduce API should make available task's progress to the task
[
https://issues.apache.org/jira/browse/MAPREDUCE-2492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13037231#comment-13037231
]
Chris Douglas commented on MAPREDUCE-2492:
--
Adding progress to {{TaskAttemptContext}} makes sense. Minor points on the
tests:
* Please use JUnit4 annotations instead of extending {{TestCase}} for new tests
* Using {{@BeforeClass}} instead of an instance initializer block would be
easier to read and would fail appropriately
* Deleting test.build.data (defaulting to /tmp) recursively when the test
runs could have bad side-effects. Please use a subdir of {{rootTestDir}} instead
* Cleaning up after the test would also be a good idea ({{@AfterClass}}). Right
now, many of the tests will not delete output when the test fails.
[MAPREDUCE] The new MapReduce API should make available task's progress to
the task
---
Key: MAPREDUCE-2492
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2492
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: task
Affects Versions: 0.23.0
Reporter: Amar Kamat
Assignee: Amar Kamat
Attachments: MAPREDUCE-2492-v1.3.patch
There is no way to get the task's current progress in the new MapReduce API.
It would be nice to make it available so that the task (map/reduce) can use
it.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
