[jira] [Created] (MAPREDUCE-5328) ClientToken should not be set in the environment

2013-06-16 Thread Omkar Vinit Joshi (JIRA) 
Omkar Vinit Joshi created MAPREDUCE-5328:


 Summary: ClientToken should not be set in the environment
 Key: MAPREDUCE-5328
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5328
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Omkar Vinit Joshi
Assignee: Omkar Vinit Joshi


This is to track Map Reduce related changes. related to YARN-610

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5326) Add version to shuffle header

2013-06-16 Thread Vinod Kumar Vavilapalli (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684918#comment-13684918
 ] 

Vinod Kumar Vavilapalli commented on MAPREDUCE-5326:


Also, I think this is internal and so is safe to do beyond 2.1. It isn't 
directly exposed to client, only MR and may be TEZ. So..

> Add version to shuffle header
> -
>
> Key: MAPREDUCE-5326
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5326
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Affects Versions: 2.0.4-alpha
>Reporter: Arun C Murthy
>Assignee: Zhijie Shen
>Priority: Blocker
> Fix For: 2.1.0-beta
>
>
> We need to add a version to the shuffle header to allow for 
> forward-compatibility etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5326) Add version to shuffle header

2013-06-16 Thread Arun C Murthy (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684904#comment-13684904
 ] 

Arun C Murthy commented on MAPREDUCE-5326:
--

Good news is that we can add this in a follow-on release (2.1.1-beta)  without 
breaking existing apps etc.

I think that is the right soln. ATM, it's too close for comfort to muck around 
with this late in the game.

> Add version to shuffle header
> -
>
> Key: MAPREDUCE-5326
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5326
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Affects Versions: 2.0.4-alpha
>Reporter: Arun C Murthy
>Assignee: Zhijie Shen
>Priority: Blocker
> Fix For: 2.1.0-beta
>
>
> We need to add a version to the shuffle header to allow for 
> forward-compatibility etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5326) Add version to shuffle header

2013-06-16 Thread Arun C Murthy (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684901#comment-13684901
 ] 

Arun C Murthy commented on MAPREDUCE-5326:
--

More - we also need to add /v1/ to the shuffle request to be safe...

> Add version to shuffle header
> -
>
> Key: MAPREDUCE-5326
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5326
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Affects Versions: 2.0.4-alpha
>Reporter: Arun C Murthy
>Assignee: Zhijie Shen
>Priority: Blocker
> Fix For: 2.1.0-beta
>
>
> We need to add a version to the shuffle header to allow for 
> forward-compatibility etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5326) Add version to shuffle header

2013-06-16 Thread Arun C Murthy (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684890#comment-13684890
 ] 

Arun C Murthy commented on MAPREDUCE-5326:
--

I think, for now, we should just add name (mapreduce) & version (1.0.0) in the 
http response header and assert them in the fetcher.

Thoughts?



> Add version to shuffle header
> -
>
> Key: MAPREDUCE-5326
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5326
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Affects Versions: 2.0.4-alpha
>Reporter: Arun C Murthy
>Assignee: Zhijie Shen
>Priority: Blocker
> Fix For: 2.1.0-beta
>
>
> We need to add a version to the shuffle header to allow for 
> forward-compatibility etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAPREDUCE-5326) Add version to shuffle header

2013-06-16 Thread Arun C Murthy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun C Murthy updated MAPREDUCE-5326:
-

Assignee: Zhijie Shen  (was: Arun C Murthy)

> Add version to shuffle header
> -
>
> Key: MAPREDUCE-5326
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5326
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Affects Versions: 2.0.4-alpha
>Reporter: Arun C Murthy
>Assignee: Zhijie Shen
>Priority: Blocker
> Fix For: 2.1.0-beta
>
>
> We need to add a version to the shuffle header to allow for 
> forward-compatibility etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5130) Add missing job config options to mapred-default.xml

2013-06-16 Thread Arun C Murthy (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684865#comment-13684865
 ] 

Arun C Murthy commented on MAPREDUCE-5130:
--

Also, while we are at this, can you please add mapreduce.(map|reduce).env - 
again, commented out as mapreduce.(map|reduce).child.java.opts? Tx.

> Add missing job config options to mapred-default.xml
> 
>
> Key: MAPREDUCE-5130
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5130
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.0.4-alpha
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: MAPREDUCE-5130-1.patch, MAPREDUCE-5130-1.patch, 
> MAPREDUCE-5130-2.patch, MAPREDUCE-5130-3.patch, MAPREDUCE-5130-4.patch, 
> MAPREDUCE-5130.patch, MAPREDUCE-5130.patch
>
>
> I came across that mapreduce.map.child.java.opts and 
> mapreduce.reduce.child.java.opts were missing in mapred-default.xml.  I'll do 
> a fuller sweep to see what else is missing before posting a patch.
> List so far:
> mapreduce.map/reduce.child.java.opts
> mapreduce.map/reduce.memory.mb
> mapreduce.job.jvm.numtasks
> mapreduce.input.lineinputformat.linespermap
> mapreduce.task.combine.progress.records

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAPREDUCE-5130) Add missing job config options to mapred-default.xml

2013-06-16 Thread Arun C Murthy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5130?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun C Murthy updated MAPREDUCE-5130:
-

Status: Open  (was: Patch Available)

[~sandyryza] Unfortunately, as [~qwertymaniac] mentioned in the context of 
mapreduce.{map,reduce}.child.java.opts, we can't add 
mapreduce.{map,reduce}.memory.mb since it will mask 
mapred.job.{map,reduce}.memory.mb.

> Add missing job config options to mapred-default.xml
> 
>
> Key: MAPREDUCE-5130
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5130
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.0.4-alpha
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: MAPREDUCE-5130-1.patch, MAPREDUCE-5130-1.patch, 
> MAPREDUCE-5130-2.patch, MAPREDUCE-5130-3.patch, MAPREDUCE-5130-4.patch, 
> MAPREDUCE-5130.patch, MAPREDUCE-5130.patch
>
>
> I came across that mapreduce.map.child.java.opts and 
> mapreduce.reduce.child.java.opts were missing in mapred-default.xml.  I'll do 
> a fuller sweep to see what else is missing before posting a patch.
> List so far:
> mapreduce.map/reduce.child.java.opts
> mapreduce.map/reduce.memory.mb
> mapreduce.job.jvm.numtasks
> mapreduce.input.lineinputformat.linespermap
> mapreduce.task.combine.progress.records

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Comment Edited] (MAPREDUCE-5130) Add missing job config options to mapred-default.xml

2013-06-16 Thread Arun C Murthy (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684862#comment-13684862
 ] 

Arun C Murthy edited comment on MAPREDUCE-5130 at 6/17/13 12:28 AM:


[~sandyryza] Unfortunately, as [~qwertymaniac] mentioned in the context of 
mapreduce.(map|reduce).child.java.opts, we can't add 
mapreduce.(map|reduce).memory.mb since it will mask 
mapred.job.(map|reduce).memory.mb.

  was (Author: acmurthy):
[~sandyryza] Unfortunately, as [~qwertymaniac] mentioned in the context of 
mapreduce.{map,reduce}.child.java.opts, we can't add 
mapreduce.{map,reduce}.memory.mb since it will mask 
mapred.job.{map,reduce}.memory.mb.
  
> Add missing job config options to mapred-default.xml
> 
>
> Key: MAPREDUCE-5130
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5130
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: documentation
>Affects Versions: 2.0.4-alpha
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: MAPREDUCE-5130-1.patch, MAPREDUCE-5130-1.patch, 
> MAPREDUCE-5130-2.patch, MAPREDUCE-5130-3.patch, MAPREDUCE-5130-4.patch, 
> MAPREDUCE-5130.patch, MAPREDUCE-5130.patch
>
>
> I came across that mapreduce.map.child.java.opts and 
> mapreduce.reduce.child.java.opts were missing in mapred-default.xml.  I'll do 
> a fuller sweep to see what else is missing before posting a patch.
> List so far:
> mapreduce.map/reduce.child.java.opts
> mapreduce.map/reduce.memory.mb
> mapreduce.job.jvm.numtasks
> mapreduce.input.lineinputformat.linespermap
> mapreduce.task.combine.progress.records

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5171) Expose blacklisted nodes from the MR AM REST API

2013-06-16 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684823#comment-13684823
 ] 

Hudson commented on MAPREDUCE-5171:
---

Integrated in Hadoop-trunk-Commit #3942 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/3942/])
MAPREDUCE-5171. Expose blacklisted nodes from the MR AM REST API. (sandyr 
via tucu) (Revision 1493600)

 Result = SUCCESS
tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493600
Files : 
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/AppContext.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/MRAppMaster.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/rm/RMContainerRequestor.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/AMWebServices.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/webapp/dao/BlacklistedNodesInfo.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MockAppContext.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAMWebServices.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistory.java


> Expose blacklisted nodes from the MR AM REST API 
> -
>
> Key: MAPREDUCE-5171
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5171
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: applicationmaster
>Affects Versions: 2.0.3-alpha
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Fix For: 2.1.0-beta
>
> Attachments: MAPREDUCE-5171-1.patch, MAPREDUCE-5171.patch
>
>
> It would be useful to expose some the list of nodes that an MR AM has 
> blacklisted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAPREDUCE-5171) Expose blacklisted nodes from the MR AM REST API

2013-06-16 Thread Alejandro Abdelnur (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alejandro Abdelnur updated MAPREDUCE-5171:
--

   Resolution: Fixed
Fix Version/s: 2.1.0-beta
 Hadoop Flags: Reviewed
   Status: Resolved  (was: Patch Available)

Thanks Sandy. Committed to trunk, branch-2 and branch-2.1-beta.

> Expose blacklisted nodes from the MR AM REST API 
> -
>
> Key: MAPREDUCE-5171
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5171
> Project: Hadoop Map/Reduce
>  Issue Type: Improvement
>  Components: applicationmaster
>Affects Versions: 2.0.3-alpha
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Fix For: 2.1.0-beta
>
> Attachments: MAPREDUCE-5171-1.patch, MAPREDUCE-5171.patch
>
>
> It would be useful to expose some the list of nodes that an MR AM has 
> blacklisted.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (MAPREDUCE-5327) TestMRJobs and TestUberAM fail at verifying counters

2013-06-16 Thread Alejandro Abdelnur (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alejandro Abdelnur resolved MAPREDUCE-5327.
---

Resolution: Invalid

MAPREDUCE-5310 stopped setting the MIN in the local clusterinfo. That is why 
the tests were failing, because the slot-millis calculation was querying the 
MIN from the local clusterinfo.

YARN-787 removed the MIN from cluster info and wired the MIN from configuration 
for the calculation of slot-millis. 

So, YARN-787 fixed what MAPREDUCE-5310 broke.

I've just run those tests on the head of trunk and 2.1-beta and they are 
passing.

Closing this one as invalid

> TestMRJobs and TestUberAM fail at verifying counters
> 
>
> Key: MAPREDUCE-5327
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5327
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Zhijie Shen
>Assignee: Alejandro Abdelnur
>Priority: Critical
>
> See the test report in YARN-829 and YARN-830:
> * https://builds.apache.org/job/PreCommit-YARN-Build/1269//testReport/
> * https://builds.apache.org/job/PreCommit-YARN-Build/1270//testReport/
> The failure seems to be related to:
> {code}
> Assert
> .assertTrue(counters.findCounter(JobCounter.SLOTS_MILLIS_MAPS) != null
> && counters.findCounter(JobCounter.SLOTS_MILLIS_MAPS).getValue() 
> != 0);
> {code}
> in TestMRJobs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5327) TestMRJobs and TestUberAM fail at verifying counters

2013-06-16 Thread Alejandro Abdelnur (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684776#comment-13684776
 ] 

Alejandro Abdelnur commented on MAPREDUCE-5327:
---

On it. It seems it broke with MAPREDUCE-5310, test-patch did not detect it as 
it did not run those tests.

> TestMRJobs and TestUberAM fail at verifying counters
> 
>
> Key: MAPREDUCE-5327
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5327
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Zhijie Shen
>Assignee: Alejandro Abdelnur
>Priority: Critical
>
> See the test report in YARN-829 and YARN-830:
> * https://builds.apache.org/job/PreCommit-YARN-Build/1269//testReport/
> * https://builds.apache.org/job/PreCommit-YARN-Build/1270//testReport/
> The failure seems to be related to:
> {code}
> Assert
> .assertTrue(counters.findCounter(JobCounter.SLOTS_MILLIS_MAPS) != null
> && counters.findCounter(JobCounter.SLOTS_MILLIS_MAPS).getValue() 
> != 0);
> {code}
> in TestMRJobs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5184) Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2

2013-06-16 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684749#comment-13684749
 ] 

Hudson commented on MAPREDUCE-5184:
---

Integrated in Hadoop-trunk-Commit #3938 (See 
[https://builds.apache.org/job/Hadoop-trunk-Commit/3938/])
MAPREDUCE-5184. Document compatibility for MapReduce applications in 
hadoop-2 vis-a-vis hadoop-1. Contributed by Zhijie Shen. (Revision 1493570)

 Result = SUCCESS
acmurthy : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493570
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/site/apt/MapReduce_Compatibility_Hadoop1_Hadoop2.apt.vm
* /hadoop/common/trunk/hadoop-project/src/site/site.xml
* 
/hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-site/src/site/apt/index.apt.vm


> Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2
> 
>
> Key: MAPREDUCE-5184
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5184
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Arun C Murthy
>Assignee: Zhijie Shen
> Fix For: 2.1.0-beta
>
> Attachments: MAPREDUCE-5184.1.patch, MAPREDUCE-5184.2.patch, 
> MAPREDUCE-5184.3.patch, MAPREDUCE-5184.4.patch, MAPREDUCE-5184.4.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (MAPREDUCE-5184) Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2

2013-06-16 Thread Arun C Murthy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun C Murthy resolved MAPREDUCE-5184.
--

Resolution: Fixed

I just committed this. Thanks [~zhijin]!

> Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2
> 
>
> Key: MAPREDUCE-5184
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5184
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Arun C Murthy
>Assignee: Zhijie Shen
> Fix For: 2.1.0-beta
>
> Attachments: MAPREDUCE-5184.1.patch, MAPREDUCE-5184.2.patch, 
> MAPREDUCE-5184.3.patch, MAPREDUCE-5184.4.patch, MAPREDUCE-5184.4.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAPREDUCE-5184) Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2

2013-06-16 Thread Arun C Murthy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun C Murthy updated MAPREDUCE-5184:
-

Attachment: MAPREDUCE-5184.4.patch

Updated patch to include this link in ToC in the site.

> Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2
> 
>
> Key: MAPREDUCE-5184
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5184
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Arun C Murthy
>Assignee: Zhijie Shen
> Fix For: 2.1.0-beta
>
> Attachments: MAPREDUCE-5184.1.patch, MAPREDUCE-5184.2.patch, 
> MAPREDUCE-5184.3.patch, MAPREDUCE-5184.4.patch, MAPREDUCE-5184.4.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5327) TestMRJobs and TestUberAM fail at verifying counters

2013-06-16 Thread Arun C Murthy (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684743#comment-13684743
 ] 

Arun C Murthy commented on MAPREDUCE-5327:
--

[~tucu00] Can you please take a look at this? Seems related to YARN-787. Tx!

> TestMRJobs and TestUberAM fail at verifying counters
> 
>
> Key: MAPREDUCE-5327
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5327
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Zhijie Shen
>Assignee: Alejandro Abdelnur
>Priority: Critical
>
> See the test report in YARN-829 and YARN-830:
> * https://builds.apache.org/job/PreCommit-YARN-Build/1269//testReport/
> * https://builds.apache.org/job/PreCommit-YARN-Build/1270//testReport/
> The failure seems to be related to:
> {code}
> Assert
> .assertTrue(counters.findCounter(JobCounter.SLOTS_MILLIS_MAPS) != null
> && counters.findCounter(JobCounter.SLOTS_MILLIS_MAPS).getValue() 
> != 0);
> {code}
> in TestMRJobs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAPREDUCE-5327) TestMRJobs and TestUberAM fail at verifying counters

2013-06-16 Thread Arun C Murthy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun C Murthy updated MAPREDUCE-5327:
-

Assignee: Alejandro Abdelnur

> TestMRJobs and TestUberAM fail at verifying counters
> 
>
> Key: MAPREDUCE-5327
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5327
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Reporter: Zhijie Shen
>Assignee: Alejandro Abdelnur
>Priority: Critical
>
> See the test report in YARN-829 and YARN-830:
> * https://builds.apache.org/job/PreCommit-YARN-Build/1269//testReport/
> * https://builds.apache.org/job/PreCommit-YARN-Build/1270//testReport/
> The failure seems to be related to:
> {code}
> Assert
> .assertTrue(counters.findCounter(JobCounter.SLOTS_MILLIS_MAPS) != null
> && counters.findCounter(JobCounter.SLOTS_MILLIS_MAPS).getValue() 
> != 0);
> {code}
> in TestMRJobs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAPREDUCE-5184) Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2

2013-06-16 Thread Arun C Murthy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun C Murthy updated MAPREDUCE-5184:
-

Attachment: MAPREDUCE-5184.4.patch

Updated patch.

I'll commit this momentarily so that I can unblock 2.1.0-beta. Please file 
follow-on jiras if necessary.

> Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2
> 
>
> Key: MAPREDUCE-5184
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5184
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Arun C Murthy
>Assignee: Zhijie Shen
> Fix For: 2.1.0-beta
>
> Attachments: MAPREDUCE-5184.1.patch, MAPREDUCE-5184.2.patch, 
> MAPREDUCE-5184.3.patch, MAPREDUCE-5184.4.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAPREDUCE-5184) Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2

2013-06-16 Thread Arun C Murthy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun C Murthy updated MAPREDUCE-5184:
-

Status: Open  (was: Patch Available)

I have some minor cosmetic changes, otherwise it looks good.

[~kkambatl] I also clarified, a bit, on what we mean by source-compatibility.

> Document MR Binary Compatibility vis-a-vis hadoop-1 and hadoop-2
> 
>
> Key: MAPREDUCE-5184
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5184
> Project: Hadoop Map/Reduce
>  Issue Type: Sub-task
>  Components: documentation
>Reporter: Arun C Murthy
>Assignee: Zhijie Shen
> Fix For: 2.1.0-beta
>
> Attachments: MAPREDUCE-5184.1.patch, MAPREDUCE-5184.2.patch, 
> MAPREDUCE-5184.3.patch
>
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (MAPREDUCE-5327) TestMRJobs and TestUberAM fail at verifying counters

2013-06-16 Thread Zhijie Shen (JIRA) 
Zhijie Shen created MAPREDUCE-5327:
--

 Summary: TestMRJobs and TestUberAM fail at verifying counters
 Key: MAPREDUCE-5327
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5327
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Zhijie Shen
Priority: Critical


See the test report in YARN-829 and YARN-830:
* https://builds.apache.org/job/PreCommit-YARN-Build/1269//testReport/
* https://builds.apache.org/job/PreCommit-YARN-Build/1270//testReport/

The failure seems to be related to:

{code}
Assert
.assertTrue(counters.findCounter(JobCounter.SLOTS_MILLIS_MAPS) != null
&& counters.findCounter(JobCounter.SLOTS_MILLIS_MAPS).getValue() != 
0);
{code}

in TestMRJobs.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAPREDUCE-5326) Add version to shuffle header

2013-06-16 Thread Arun C Murthy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun C Murthy updated MAPREDUCE-5326:
-

Affects Version/s: 2.0.4-alpha

> Add version to shuffle header
> -
>
> Key: MAPREDUCE-5326
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5326
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Affects Versions: 2.0.4-alpha
>Reporter: Arun C Murthy
>Assignee: Arun C Murthy
>Priority: Blocker
>
> We need to add a version to the shuffle header to allow for 
> forward-compatibility etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Updated] (MAPREDUCE-5326) Add version to shuffle header

2013-06-16 Thread Arun C Murthy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun C Murthy updated MAPREDUCE-5326:
-

Target Version/s: 2.1.0-beta
   Fix Version/s: 2.1.0-beta

> Add version to shuffle header
> -
>
> Key: MAPREDUCE-5326
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5326
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>Affects Versions: 2.0.4-alpha
>Reporter: Arun C Murthy
>Assignee: Arun C Murthy
>Priority: Blocker
> Fix For: 2.1.0-beta
>
>
> We need to add a version to the shuffle header to allow for 
> forward-compatibility etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Created] (MAPREDUCE-5326) Add version to shuffle header

2013-06-16 Thread Arun C Murthy (JIRA) 
Arun C Murthy created MAPREDUCE-5326:


 Summary: Add version to shuffle header
 Key: MAPREDUCE-5326
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5326
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Arun C Murthy
Assignee: Arun C Murthy
Priority: Blocker


We need to add a version to the shuffle header to allow for 
forward-compatibility etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5192) Separate TCE resolution from fetch

2013-06-16 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684667#comment-13684667
 ] 

Hudson commented on MAPREDUCE-5192:
---

Integrated in Hadoop-Mapreduce-trunk #1459 (See 
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1459/])
MAPREDUCE-5192. Allow for alternate resolutions of TaskCompletionEvents. 
Contributed by Chris Douglas. (Revision 1493445)

 Result = FAILURE
acmurthy : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493445
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/dev-support/findbugs-exclude.xml
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/EventFetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Shuffle.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleScheduler.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestEventFetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestFetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java


> Separate TCE resolution from fetch
> --
>
> Key: MAPREDUCE-5192
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5192
> Project: Hadoop Map/Reduce
>  Issue Type: Task
>  Components: task
>Reporter: Chris Douglas
>Assignee: Chris Douglas
>Priority: Minor
> Fix For: 2.1.0-beta
>
> Attachments: M5192-1.patch, M5192-2.patch, M5192-3.patch, M5192.patch
>
>
> The {{EventFetcher}} thread grounds task completion events as URIs before 
> passing them to the {{ShuffleScheduler}}. If the former deferred this to the 
> scheduler, one could interpret the TCE metadata differently

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5194) Heed interrupts during Fetcher shutdown

2013-06-16 Thread Arun C Murthy (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684655#comment-13684655
 ] 

Arun C Murthy commented on MAPREDUCE-5194:
--

[~chris.douglas] some questions:

# Should we make openConnection synchronized and add an analogous 
(synchronized) closeConnection method? 
# In copyMapsFromHost should we check for stopped before we call openConnection?
# Also, should we call disconnect in the case stopped is true after the call to 
Fetcher.connect in copyMapsFromHost?


> Heed interrupts during Fetcher shutdown
> ---
>
> Key: MAPREDUCE-5194
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5194
> Project: Hadoop Map/Reduce
>  Issue Type: Task
>  Components: task
>Reporter: Chris Douglas
>Assignee: Chris Douglas
>Priority: Minor
> Fix For: 2.1.0-beta
>
> Attachments: M5194-1.patch, M5194.patch
>
>
> In the current implementation, {{Fetcher}} instances usually exit gracefully 
> when the shuffle succeeds. When it fails, threads are interrupted, but may 
> continue running harmlessly until the JVM shuts down.
> However, to generate consistent checkpoints, these threads should exit 
> cleanly to quiesce the state of the shuffle.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5192) Separate TCE resolution from fetch

2013-06-16 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684646#comment-13684646
 ] 

Hudson commented on MAPREDUCE-5192:
---

Integrated in Hadoop-Hdfs-trunk #1432 (See 
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1432/])
MAPREDUCE-5192. Allow for alternate resolutions of TaskCompletionEvents. 
Contributed by Chris Douglas. (Revision 1493445)

 Result = FAILURE
acmurthy : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493445
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/dev-support/findbugs-exclude.xml
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/EventFetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Shuffle.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleScheduler.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestEventFetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestFetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java


> Separate TCE resolution from fetch
> --
>
> Key: MAPREDUCE-5192
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5192
> Project: Hadoop Map/Reduce
>  Issue Type: Task
>  Components: task
>Reporter: Chris Douglas
>Assignee: Chris Douglas
>Priority: Minor
> Fix For: 2.1.0-beta
>
> Attachments: M5192-1.patch, M5192-2.patch, M5192-3.patch, M5192.patch
>
>
> The {{EventFetcher}} thread grounds task completion events as URIs before 
> passing them to the {{ShuffleScheduler}}. If the former deferred this to the 
> scheduler, one could interpret the TCE metadata differently

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Resolved] (MAPREDUCE-5211) Reducer intermediate files can collide during merge

2013-06-16 Thread Arun C Murthy (JIRA) 

 [ 
https://issues.apache.org/jira/browse/MAPREDUCE-5211?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Arun C Murthy resolved MAPREDUCE-5211.
--

  Resolution: Fixed
Target Version/s: 0.23.8, 2.1.0-beta  (was: 2.1.0-beta, 0.23.8)

Not a problem in branch-2 per [~jlowe] comment.

> Reducer intermediate files can collide during merge
> ---
>
> Key: MAPREDUCE-5211
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5211
> Project: Hadoop Map/Reduce
>  Issue Type: Bug
>  Components: mrv2
>Affects Versions: 0.23.7
>Reporter: Jason Lowe
>Assignee: Jason Lowe
>Priority: Blocker
>  Labels: 2.0.4.2
> Fix For: 0.23.8
>
> Attachments: MAPREDUCE-5211.branch-0.23.patch
>
>
> The OnDiskMerger.merge method constructs an output path that is not unique to 
> a reduce attempt, and as a result can result in a file collision with other 
> reducers from the same app that are running on the same node.  In addition 
> the name of the output file is based on MapOutput.toString which may not be 
> unique in light of multi-pass merges on disk since the mapId will be null and 
> the basename ends up as "MapOutput(null, DISK)"

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

[jira] [Commented] (MAPREDUCE-5192) Separate TCE resolution from fetch

2013-06-16 Thread Hudson (JIRA) 

[ 
https://issues.apache.org/jira/browse/MAPREDUCE-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13684617#comment-13684617
 ] 

Hudson commented on MAPREDUCE-5192:
---

Integrated in Hadoop-Yarn-trunk #242 (See 
[https://builds.apache.org/job/Hadoop-Yarn-trunk/242/])
MAPREDUCE-5192. Allow for alternate resolutions of TaskCompletionEvents. 
Contributed by Chris Douglas. (Revision 1493445)

 Result = SUCCESS
acmurthy : 
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1493445
Files : 
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/dev-support/findbugs-exclude.xml
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/EventFetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Fetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/Shuffle.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleScheduler.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/ShuffleSchedulerImpl.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestEventFetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestFetcher.java
* 
/hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/task/reduce/TestShuffleScheduler.java


> Separate TCE resolution from fetch
> --
>
> Key: MAPREDUCE-5192
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5192
> Project: Hadoop Map/Reduce
>  Issue Type: Task
>  Components: task
>Reporter: Chris Douglas
>Assignee: Chris Douglas
>Priority: Minor
> Fix For: 2.1.0-beta
>
> Attachments: M5192-1.patch, M5192-2.patch, M5192-3.patch, M5192.patch
>
>
> The {{EventFetcher}} thread grounds task completion events as URIs before 
> passing them to the {{ShuffleScheduler}}. If the former deferred this to the 
> scheduler, one could interpret the TCE metadata differently

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira