[jira] [Commented] (MAPREDUCE-5280) Mapreduce API: ClusterMetrics incompatibility issues with MR1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671973#comment-13671973 ] Hudson commented on MAPREDUCE-5280: --- Integrated in Hadoop-trunk-Commit #3826 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3826/]) MAPREDUCE-5280. Bring back removed constructor and a method in mapreduce ClusterMetrics for binary compatibility with 1.x APIs. Contributed by Mayank Bansal. (Revision 1488458) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488458 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/ClusterStatus.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/JobClient.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/ClusterMetrics.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestNetworkedJob.java > Mapreduce API: ClusterMetrics incompatibility issues with MR1 > - > > Key: MAPREDUCE-5280 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5280 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Fix For: 2.1.0-beta > > Attachments: MAPREDUCE-5280-trunk-1.patch > > > 1. Constructor has one fewer parameters: numGraylistedTrackers > 2. getGrayListedTaskTrackerCount() is removed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5280) Mapreduce API: ClusterMetrics incompatibility issues with MR1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-5280: --- Resolution: Fixed Fix Version/s: 2.1.0-beta Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed this to trunk and branch-2. Thanks Mayank! > Mapreduce API: ClusterMetrics incompatibility issues with MR1 > - > > Key: MAPREDUCE-5280 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5280 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Fix For: 2.1.0-beta > > Attachments: MAPREDUCE-5280-trunk-1.patch > > > 1. Constructor has one fewer parameters: numGraylistedTrackers > 2. getGrayListedTaskTrackerCount() is removed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5280) Mapreduce API: ClusterMetrics incompatibility issues with MR1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671956#comment-13671956 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-5280: +1, looks good. Checking this in. > Mapreduce API: ClusterMetrics incompatibility issues with MR1 > - > > Key: MAPREDUCE-5280 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5280 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Attachments: MAPREDUCE-5280-trunk-1.patch > > > 1. Constructor has one fewer parameters: numGraylistedTrackers > 2. getGrayListedTaskTrackerCount() is removed -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5273) Protected variables are removed from CombineFileRecordReader in both mapred and mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671936#comment-13671936 ] Hudson commented on MAPREDUCE-5273: --- Integrated in Hadoop-trunk-Commit #3825 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3825/]) MAPREDUCE-5273. Bring back a couple of protected variables in mapred and mapreduce CombineFileRecordReader for binary compatibility with 1.x APIs. Contributed by Mayank Bansal. (Revision 1488442) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488442 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/CombineFileRecordReader.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/CombineFileRecordReader.java > Protected variables are removed from CombineFileRecordReader in both mapred > and mapreduce > - > > Key: MAPREDUCE-5273 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5273 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Fix For: 2.1.0-beta > > Attachments: MAPREDUCE-5273-trunk-1.patch > > > Two protected variables are removed from CombineFileRecordReader in both > mapred and mapreduce: > * FileSystem fs > * Class> rrClass -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5273) Protected variables are removed from CombineFileRecordReader in both mapred and mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-5273: --- Resolution: Fixed Fix Version/s: 2.1.0-beta Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed this to trunk and branch-2. Thanks Mayank! > Protected variables are removed from CombineFileRecordReader in both mapred > and mapreduce > - > > Key: MAPREDUCE-5273 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5273 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Fix For: 2.1.0-beta > > Attachments: MAPREDUCE-5273-trunk-1.patch > > > Two protected variables are removed from CombineFileRecordReader in both > mapred and mapreduce: > * FileSystem fs > * Class> rrClass -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5273) Protected variables are removed from CombineFileRecordReader in both mapred and mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-5273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671925#comment-13671925 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-5273: It's so trivial it's funny. These variables aren't even used otherwise. +1. Checking this in. > Protected variables are removed from CombineFileRecordReader in both mapred > and mapreduce > - > > Key: MAPREDUCE-5273 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5273 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Attachments: MAPREDUCE-5273-trunk-1.patch > > > Two protected variables are removed from CombineFileRecordReader in both > mapred and mapreduce: > * FileSystem fs > * Class> rrClass -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-5285) Update MR App to use immutable ApplicationAttemptID, ContainerID, NodeID after YARN-735
[ https://issues.apache.org/jira/browse/MAPREDUCE-5285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Siddharth Seth resolved MAPREDUCE-5285. --- Resolution: Fixed Fix Version/s: 2.1.0-beta Committed as part of YARN-735. > Update MR App to use immutable ApplicationAttemptID, ContainerID, NodeID > after YARN-735 > --- > > Key: MAPREDUCE-5285 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5285 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Jian He > Fix For: 2.1.0-beta > > -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5231) Constructor of DBInputFormat.DBRecordReader in mapred is changed
[ https://issues.apache.org/jira/browse/MAPREDUCE-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671915#comment-13671915 ] Hudson commented on MAPREDUCE-5231: --- Integrated in Hadoop-trunk-Commit #3823 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3823/]) MAPREDUCE-5231. Bring back a constructor in mapred's DBInputFormat.DBRecordReader for binary compatibility with 1.x mapred APIs. Contributed by Zhijie Shen. (Revision 1488436) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488436 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/lib/db/DBInputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/db/DBInputFormat.java > Constructor of DBInputFormat.DBRecordReader in mapred is changed > > > Key: MAPREDUCE-5231 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5231 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.1.0-beta > > Attachments: MAPREDUCE-5231.1.patch > > > The constructor of DBInputFormat.DBRecordReader in mapred is changed from MR1 > to RM2. Though MAPREDUCE-716 tried to deal with the API difference. However, > if DBInputFormat.DBRecordReader is extended, the incompatibility around the > constructor is still there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5231) Constructor of DBInputFormat.DBRecordReader in mapred is changed
[ https://issues.apache.org/jira/browse/MAPREDUCE-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-5231: --- Resolution: Fixed Fix Version/s: 2.1.0-beta Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed this to trunk and branch-2. Thanks Zhijie! > Constructor of DBInputFormat.DBRecordReader in mapred is changed > > > Key: MAPREDUCE-5231 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5231 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Fix For: 2.1.0-beta > > Attachments: MAPREDUCE-5231.1.patch > > > The constructor of DBInputFormat.DBRecordReader in mapred is changed from MR1 > to RM2. Though MAPREDUCE-716 tried to deal with the API difference. However, > if DBInputFormat.DBRecordReader is extended, the incompatibility around the > constructor is still there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5231) Constructor of DBInputFormat.DBRecordReader in mapred is changed
[ https://issues.apache.org/jira/browse/MAPREDUCE-5231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671901#comment-13671901 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-5231: This DB* code is a bit of a mind-bender w.r.t mapred and mapreduce impls. Looks good, +1. Checking this in. > Constructor of DBInputFormat.DBRecordReader in mapred is changed > > > Key: MAPREDUCE-5231 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5231 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Zhijie Shen > Attachments: MAPREDUCE-5231.1.patch > > > The constructor of DBInputFormat.DBRecordReader in mapred is changed from MR1 > to RM2. Though MAPREDUCE-716 tried to deal with the API difference. However, > if DBInputFormat.DBRecordReader is extended, the incompatibility around the > constructor is still there. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5150) Backport 2009 terasort (MAPREDUCE-639) to branch-1
[
https://issues.apache.org/jira/browse/MAPREDUCE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671892#comment-13671892
]
Hadoop QA commented on MAPREDUCE-5150:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12578622/MAPREDUCE-5150-branch-1.patch
against trunk revision .
{color:red}-1 patch{color}. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3714//console
This message is automatically generated.
> Backport 2009 terasort (MAPREDUCE-639) to branch-1
> --
>
> Key: MAPREDUCE-5150
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5150
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: examples
>Affects Versions: 1.2.0
>Reporter: Gera Shegalov
>Priority: Minor
> Attachments: MAPREDUCE-5150-branch-1.patch
>
>
> Users evaluate performance of Hadoop clusters using different benchmarks such
> as TeraSort. However, terasort version in branch-1 is outdated. It works on
> teragen dataset that cannot exceed 4 billion unique keys and it does not have
> the fast non-sampling partitioner SimplePartitioner either.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5283) Over 10 different tests have near identical implementations of AppContext
[ https://issues.apache.org/jira/browse/MAPREDUCE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671890#comment-13671890 ] Jason Lowe commented on MAPREDUCE-5283: --- +1, lgtm. Waiting for the dust to settle this weekend on 2.0.5 / 2.1.0 and to give others a chance to comment. > Over 10 different tests have near identical implementations of AppContext > - > > Key: MAPREDUCE-5283 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5283 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: applicationmaster, test >Affects Versions: 2.0.4-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: MAPREDUCE-5283-1.patch, MAPREDUCE-5283.patch > > > I'm trying to add a method to AppContext for MAPREDUCE-5171, and I have to go > into nearly every test file for MR web services to make sure their > TestAppContext implements it. I propose having a common implementation of > AppContext that all these tests can use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5150) Backport 2009 terasort (MAPREDUCE-639) to branch-1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated MAPREDUCE-5150: - Affects Version/s: (was: 1.0.4) > Backport 2009 terasort (MAPREDUCE-639) to branch-1 > -- > > Key: MAPREDUCE-5150 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5150 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: examples >Affects Versions: 1.2.0 >Reporter: Gera Shegalov >Priority: Minor > Attachments: MAPREDUCE-5150-branch-1.patch > > > Users evaluate performance of Hadoop clusters using different benchmarks such > as TeraSort. However, terasort version in branch-1 is outdated. It works on > teragen dataset that cannot exceed 4 billion unique keys and it does not have > the fast non-sampling partitioner SimplePartitioner either. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5150) Backport 2009 terasort (MAPREDUCE-639) to branch-1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated MAPREDUCE-5150: - Status: Patch Available (was: Open) > Backport 2009 terasort (MAPREDUCE-639) to branch-1 > -- > > Key: MAPREDUCE-5150 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5150 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: examples >Affects Versions: 1.2.0 >Reporter: Gera Shegalov >Priority: Minor > Attachments: MAPREDUCE-5150-branch-1.patch > > > Users evaluate performance of Hadoop clusters using different benchmarks such > as TeraSort. However, terasort version in branch-1 is outdated. It works on > teragen dataset that cannot exceed 4 billion unique keys and it does not have > the fast non-sampling partitioner SimplePartitioner either. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5150) Backport 2009 terasort (MAPREDUCE-639) to branch-1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gera Shegalov updated MAPREDUCE-5150: - Affects Version/s: 1.2.0 > Backport 2009 terasort (MAPREDUCE-639) to branch-1 > -- > > Key: MAPREDUCE-5150 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5150 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: examples >Affects Versions: 1.0.4, 1.2.0 >Reporter: Gera Shegalov >Priority: Minor > Attachments: MAPREDUCE-5150-branch-1.patch > > > Users evaluate performance of Hadoop clusters using different benchmarks such > as TeraSort. However, terasort version in branch-1 is outdated. It works on > teragen dataset that cannot exceed 4 billion unique keys and it does not have > the fast non-sampling partitioner SimplePartitioner either. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5268) Improve history server startup performance
[ https://issues.apache.org/jira/browse/MAPREDUCE-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671884#comment-13671884 ] Jason Lowe commented on MAPREDUCE-5268: --- Will commit this after the 2.0.5 / 2.1.0 dust has cleared which should be done this weekend. > Improve history server startup performance > -- > > Key: MAPREDUCE-5268 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5268 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver >Affects Versions: 0.23.7, 2.0.4-alpha >Reporter: Jason Lowe >Assignee: Karthik Kambatla > Attachments: mr-5268.patch, mr-5268.patch, mr-5268.patch, > mr-5268.patch, mr-5268.patch, mr-5268-prelim.patch > > > The history server can easily take many minutes to startup when there are a > significant number of jobs to scan in the done directory. However the > scanning of files is not the bottleneck, rather it's the heavy use of > ConcurrentSkipListMap.size in HistoryFileManager. > ConcurrentSkipListMap.size is a very expensive operation, especially on maps > with many entries, as it has to scan every entry to compute the size. We > should avoid calling this method or at least minimize its use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5283) Over 10 different tests have near identical implementations of AppContext
[
https://issues.apache.org/jira/browse/MAPREDUCE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671851#comment-13671851
]
Hadoop QA commented on MAPREDUCE-5283:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585671/MAPREDUCE-5283-1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 15 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3713//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3713//console
This message is automatically generated.
> Over 10 different tests have near identical implementations of AppContext
> -
>
> Key: MAPREDUCE-5283
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5283
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: applicationmaster, test
>Affects Versions: 2.0.4-alpha
>Reporter: Sandy Ryza
>Assignee: Sandy Ryza
> Attachments: MAPREDUCE-5283-1.patch, MAPREDUCE-5283.patch
>
>
> I'm trying to add a method to AppContext for MAPREDUCE-5171, and I have to go
> into nearly every test file for MR web services to make sure their
> TestAppContext implements it. I propose having a common implementation of
> AppContext that all these tests can use.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5283) Over 10 different tests have near identical implementations of AppContext
[ https://issues.apache.org/jira/browse/MAPREDUCE-5283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sandy Ryza updated MAPREDUCE-5283: -- Attachment: MAPREDUCE-5283-1.patch > Over 10 different tests have near identical implementations of AppContext > - > > Key: MAPREDUCE-5283 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5283 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: applicationmaster, test >Affects Versions: 2.0.4-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Attachments: MAPREDUCE-5283-1.patch, MAPREDUCE-5283.patch > > > I'm trying to add a method to AppContext for MAPREDUCE-5171, and I have to go > into nearly every test file for MR web services to make sure their > TestAppContext implements it. I propose having a common implementation of > AppContext that all these tests can use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5192) Separate TCE resolution from fetch
[
https://issues.apache.org/jira/browse/MAPREDUCE-5192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated MAPREDUCE-5192:
-
Assignee: Chris Douglas
> Separate TCE resolution from fetch
> --
>
> Key: MAPREDUCE-5192
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5192
> Project: Hadoop Map/Reduce
> Issue Type: Task
> Components: task
>Reporter: Chris Douglas
>Assignee: Chris Douglas
>Priority: Minor
> Fix For: 2.1.0-beta
>
> Attachments: M5192-1.patch, M5192-2.patch, M5192.patch
>
>
> The {{EventFetcher}} thread grounds task completion events as URIs before
> passing them to the {{ShuffleScheduler}}. If the former deferred this to the
> scheduler, one could interpret the TCE metadata differently
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-4404) Adding Test case for TestMRJobClient to verify the user name
[ https://issues.apache.org/jira/browse/MAPREDUCE-4404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Bansal resolved MAPREDUCE-4404. -- Resolution: Fixed > Adding Test case for TestMRJobClient to verify the user name > > > Key: MAPREDUCE-4404 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4404 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: client >Affects Versions: 0.22.0 >Reporter: Mayank Bansal >Assignee: Mayank Bansal >Priority: Minor > Fix For: 0.22.1 > > Attachments: MAPREDUCE-4404-22.patch > > > Adding Test case for TestMRJobClient to verify the user name -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5268) Improve history server startup performance
[ https://issues.apache.org/jira/browse/MAPREDUCE-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671798#comment-13671798 ] Jason Lowe commented on MAPREDUCE-5268: --- +1 for the latest patch. Thanks so much for the rapid turnarounds on the patches and your patience, Karthik! > Improve history server startup performance > -- > > Key: MAPREDUCE-5268 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5268 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver >Affects Versions: 0.23.7, 2.0.4-alpha >Reporter: Jason Lowe >Assignee: Karthik Kambatla > Attachments: mr-5268.patch, mr-5268.patch, mr-5268.patch, > mr-5268.patch, mr-5268.patch, mr-5268-prelim.patch > > > The history server can easily take many minutes to startup when there are a > significant number of jobs to scan in the done directory. However the > scanning of files is not the bottleneck, rather it's the heavy use of > ConcurrentSkipListMap.size in HistoryFileManager. > ConcurrentSkipListMap.size is a very expensive operation, especially on maps > with many entries, as it has to scan every entry to compute the size. We > should avoid calling this method or at least minimize its use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5275) Mapreduce API: TokenCache incompatibility issues with MR1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671782#comment-13671782 ] Hudson commented on MAPREDUCE-5275: --- Integrated in Hadoop-trunk-Commit #3822 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/3822/]) MAPREDUCE-5275. Bring back a couple of APIs in mapreduce.security.TokenCache for binary compatibility with 1.x mapreduce APIs. Contributed by Mayank Bansal. (Revision 1488369) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488369 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/security/TokenCache.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/security/TestTokenCache.java > Mapreduce API: TokenCache incompatibility issues with MR1 > - > > Key: MAPREDUCE-5275 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5275 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Fix For: 2.1.0-beta > > Attachments: MAPREDUCE-5275-trunk-1.patch, > MAPREDUCE-5275-trunk-2.patch, MAPREDUCE-5275-trunk-3.patch, > MAPREDUCE-5275-trunk-4.patch > > > There're following incompatibility issues: > * Token getDelegationToken(Credentials, String) is > removed > * Credentials loadTokens(String, Configuration) changes to Credentials > loadTokens(String, JobConf) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5275) Mapreduce API: TokenCache incompatibility issues with MR1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-5275: --- Resolution: Fixed Fix Version/s: 2.1.0-beta Hadoop Flags: Reviewed Status: Resolved (was: Patch Available) Committed this to trunk and branch-2. Thanks Mayank! Tx for the early review Zhijie! > Mapreduce API: TokenCache incompatibility issues with MR1 > - > > Key: MAPREDUCE-5275 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5275 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Fix For: 2.1.0-beta > > Attachments: MAPREDUCE-5275-trunk-1.patch, > MAPREDUCE-5275-trunk-2.patch, MAPREDUCE-5275-trunk-3.patch, > MAPREDUCE-5275-trunk-4.patch > > > There're following incompatibility issues: > * Token getDelegationToken(Credentials, String) is > removed > * Credentials loadTokens(String, Configuration) changes to Credentials > loadTokens(String, JobConf) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5275) Mapreduce API: TokenCache incompatibility issues with MR1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671774#comment-13671774 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-5275: +1, looks good now. Checking this in.. > Mapreduce API: TokenCache incompatibility issues with MR1 > - > > Key: MAPREDUCE-5275 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5275 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Attachments: MAPREDUCE-5275-trunk-1.patch, > MAPREDUCE-5275-trunk-2.patch, MAPREDUCE-5275-trunk-3.patch, > MAPREDUCE-5275-trunk-4.patch > > > There're following incompatibility issues: > * Token getDelegationToken(Credentials, String) is > removed > * Credentials loadTokens(String, Configuration) changes to Credentials > loadTokens(String, JobConf) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5199) AppTokens file can/should be removed
[
https://issues.apache.org/jira/browse/MAPREDUCE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671756#comment-13671756
]
Hadoop QA commented on MAPREDUCE-5199:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585649/MAPREDUCE-5199.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 core tests{color}. The patch failed these unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient:
org.apache.hadoop.mapreduce.v2.app.TestRecovery
org.apache.hadoop.mapreduce.v2.app.TestMRApp
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3712//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3712//console
This message is automatically generated.
> AppTokens file can/should be removed
>
>
> Key: MAPREDUCE-5199
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5199
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: security
>Affects Versions: 3.0.0, 2.1.0-beta
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: MAPREDUCE-5199.patch, MAPREDUCE-5199.patch,
> MAPREDUCE-5199.patch
>
>
> All the required tokens are propagated to AMs and containers via
> startContainer(), no need for explicitly creating the app-token file that we
> have today..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5281) Mapreduce API: Counter changes from non-abstract class to interface
[ https://issues.apache.org/jira/browse/MAPREDUCE-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671722#comment-13671722 ] Zhijie Shen commented on MAPREDUCE-5281: As Counter becomes interface, there's no way to add the implemented methods. However, the four aforementioned methods must exist in the interface's implementations. boolean equals(Object) and int hashCode() comes from Object, while void readFields(DataInput) and void write(DataOutput) will be implemented as they are part of the interface (Counter extends Writable). > Mapreduce API: Counter changes from non-abstract class to interface > --- > > Key: MAPREDUCE-5281 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5281 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Zhijie Shen > > Therefore, significant changes in Counter: > 1. Two Constructors are removed; > 2. Following methods are removed: > * boolean equals(Object) > * int hashCode() > * void readFields(DataInput) > * void write(DataOutput) > Fix of this issue may break 0.23. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5199) AppTokens file can/should be removed
[ https://issues.apache.org/jira/browse/MAPREDUCE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated MAPREDUCE-5199: --- Attachment: MAPREDUCE-5199.patch Uploaded wrong patch. > AppTokens file can/should be removed > > > Key: MAPREDUCE-5199 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5199 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: security >Affects Versions: 3.0.0, 2.1.0-beta >Reporter: Vinod Kumar Vavilapalli >Assignee: Daryn Sharp >Priority: Blocker > Attachments: MAPREDUCE-5199.patch, MAPREDUCE-5199.patch, > MAPREDUCE-5199.patch > > > All the required tokens are propagated to AMs and containers via > startContainer(), no need for explicitly creating the app-token file that we > have today.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5268) Improve history server startup performance
[
https://issues.apache.org/jira/browse/MAPREDUCE-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671705#comment-13671705
]
Hadoop QA commented on MAPREDUCE-5268:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585646/mr-5268.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3711//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3711//console
This message is automatically generated.
> Improve history server startup performance
> --
>
> Key: MAPREDUCE-5268
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5268
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver
>Affects Versions: 0.23.7, 2.0.4-alpha
>Reporter: Jason Lowe
>Assignee: Karthik Kambatla
> Attachments: mr-5268.patch, mr-5268.patch, mr-5268.patch,
> mr-5268.patch, mr-5268.patch, mr-5268-prelim.patch
>
>
> The history server can easily take many minutes to startup when there are a
> significant number of jobs to scan in the done directory. However the
> scanning of files is not the bottleneck, rather it's the heavy use of
> ConcurrentSkipListMap.size in HistoryFileManager.
> ConcurrentSkipListMap.size is a very expensive operation, especially on maps
> with many entries, as it has to scan every entry to compute the size. We
> should avoid calling this method or at least minimize its use.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5268) Improve history server startup performance
[ https://issues.apache.org/jira/browse/MAPREDUCE-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-5268: Attachment: mr-5268.patch Thanks for the patient reviews, Jason. My bad forgot to rename the corresponding test - fixed that now. Also, the tests now use MRBuilderUtils instead of FakeJob. Reviewed the patch carefully, hopefully this takes care of everything. > Improve history server startup performance > -- > > Key: MAPREDUCE-5268 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5268 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver >Affects Versions: 0.23.7, 2.0.4-alpha >Reporter: Jason Lowe >Assignee: Karthik Kambatla > Attachments: mr-5268.patch, mr-5268.patch, mr-5268.patch, > mr-5268.patch, mr-5268.patch, mr-5268-prelim.patch > > > The history server can easily take many minutes to startup when there are a > significant number of jobs to scan in the done directory. However the > scanning of files is not the bottleneck, rather it's the heavy use of > ConcurrentSkipListMap.size in HistoryFileManager. > ConcurrentSkipListMap.size is a very expensive operation, especially on maps > with many entries, as it has to scan every entry to compute the size. We > should avoid calling this method or at least minimize its use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4019) -list-attempt-ids is not working
[
https://issues.apache.org/jira/browse/MAPREDUCE-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671614#comment-13671614
]
Jason Lowe commented on MAPREDUCE-4019:
---
Thanks for picking up the torch on this, Ashwin. Couple of comments:
* Whitespace was removed before/after a number of methods which would be nice
to have back to match the style of the rest of the code
* getCluster should be named something like createCluster to indicate it is
creating a new object each time it's called
* getTaskTypes() can simply be StringUtils.join(taskTypes, " ")
> -list-attempt-ids is not working
> -
>
> Key: MAPREDUCE-4019
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4019
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: client
>Affects Versions: 2.0.0-alpha, 3.0.0
>Reporter: B Anil Kumar
>Priority: Minor
> Attachments: MAPREDUCE-4019-4.txt, MAPREDUCE-4019.patch,
> MAPREDUCE-4019.patch, MAPREDUCE-4019.txt
>
>
> while executing {noformat}bin/mapred job -list-attempt-ids job_id map
> running{noformat}, we are getting IllegalArgumentexception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5199) AppTokens file can/should be removed
[
https://issues.apache.org/jira/browse/MAPREDUCE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671602#comment-13671602
]
Hadoop QA commented on MAPREDUCE-5199:
--
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585632/MAPREDUCE-5199.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 5 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3710//console
This message is automatically generated.
> AppTokens file can/should be removed
>
>
> Key: MAPREDUCE-5199
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5199
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: security
>Affects Versions: 3.0.0, 2.0.5-beta
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Daryn Sharp
>Priority: Blocker
> Attachments: MAPREDUCE-5199.patch, MAPREDUCE-5199.patch
>
>
> All the required tokens are propagated to AMs and containers via
> startContainer(), no need for explicitly creating the app-token file that we
> have today..
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5199) AppTokens file can/should be removed
[ https://issues.apache.org/jira/browse/MAPREDUCE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated MAPREDUCE-5199: --- Attachment: MAPREDUCE-5199.patch bq. JobImpl.fsTokens should be renamed too. Done bq. Jobtoken itself could be set in jobCredentials and passed to TaskImpl etc. Perhaps another ticket. Yes, will defer since it's out of scope for removing appTokens file. bq. initAndStartAppMaster Adding the credentials to conf is necessary for the sake of MAPREDUCE-5240. Please remove the comment " // shouldn't be necessary" Removed. bq. And it is better to add to conf before init() happens. We never know if the original conf is cached as part of init(). Done. Although the first thing init() does is call the method to setup the credentials, I changed it. bq. appMasterUgi is useless. Perhaps remove it in a separate ticket. I too thought it was useless, but it's not. That's how the AM constructs a UGI as the submitting user in insecure mode. Otherwise the UGI is the NM user. The issue is the launch context sets $USER which in turn is used by the AM to construct the "unnecessary" UGI. However if $HADOOP_USER_NAME=$USER is also defined in the launch context, the UGI will create the login UGI as the submitter. bq. TestMRAppMaster: Wherever you say appToken shouldn't be present, you should explicitly validate it. Seems like that comment is invalid in some cases. It does validate by checking that there's 1 token, and that it's the expected non-app token. bq. MRJobConf.APPLICATION_TOKENS_FILE should just be removed along with the java comments. This was supposed to be private anyways. Done. I considered this too, but was worried if other projects might be using it. bq. downloadTokensAndSetupUGI -> setupJobCredentialsAndUGI like Sid proposed. Done. > AppTokens file can/should be removed > > > Key: MAPREDUCE-5199 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5199 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: security >Affects Versions: 3.0.0, 2.0.5-beta >Reporter: Vinod Kumar Vavilapalli >Assignee: Daryn Sharp >Priority: Blocker > Attachments: MAPREDUCE-5199.patch, MAPREDUCE-5199.patch > > > All the required tokens are propagated to AMs and containers via > startContainer(), no need for explicitly creating the app-token file that we > have today.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5199) AppTokens file can/should be removed
[ https://issues.apache.org/jira/browse/MAPREDUCE-5199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daryn Sharp updated MAPREDUCE-5199: --- Status: Patch Available (was: Open) > AppTokens file can/should be removed > > > Key: MAPREDUCE-5199 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5199 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: security >Affects Versions: 3.0.0, 2.0.5-beta >Reporter: Vinod Kumar Vavilapalli >Assignee: Daryn Sharp >Priority: Blocker > Attachments: MAPREDUCE-5199.patch, MAPREDUCE-5199.patch > > > All the required tokens are propagated to AMs and containers via > startContainer(), no need for explicitly creating the app-token file that we > have today.. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5268) Improve history server startup performance
[ https://issues.apache.org/jira/browse/MAPREDUCE-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671582#comment-13671582 ] Jason Lowe commented on MAPREDUCE-5268: --- Wow, that was fast! Unfortunately ConcurrentSkipListMapWithSize is still showing up in filenames. Also taking a closer look at the unit tests, FakeJobId should be removed and replaced with MRBuilderUtils.newJobId. There's a form of that method where we can simply pass numbers for the cluster timestamp, app ID, and job ID directly. > Improve history server startup performance > -- > > Key: MAPREDUCE-5268 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5268 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver >Affects Versions: 0.23.7, 2.0.4-alpha >Reporter: Jason Lowe >Assignee: Karthik Kambatla > Attachments: mr-5268.patch, mr-5268.patch, mr-5268.patch, > mr-5268.patch, mr-5268-prelim.patch > > > The history server can easily take many minutes to startup when there are a > significant number of jobs to scan in the done directory. However the > scanning of files is not the bottleneck, rather it's the heavy use of > ConcurrentSkipListMap.size in HistoryFileManager. > ConcurrentSkipListMap.size is a very expensive operation, especially on maps > with many entries, as it has to scan every entry to compute the size. We > should avoid calling this method or at least minimize its use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5268) Improve history server startup performance
[
https://issues.apache.org/jira/browse/MAPREDUCE-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671571#comment-13671571
]
Hadoop QA commented on MAPREDUCE-5268:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585626/mr-5268.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3709//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3709//console
This message is automatically generated.
> Improve history server startup performance
> --
>
> Key: MAPREDUCE-5268
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5268
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: jobhistoryserver
>Affects Versions: 0.23.7, 2.0.4-alpha
>Reporter: Jason Lowe
>Assignee: Karthik Kambatla
> Attachments: mr-5268.patch, mr-5268.patch, mr-5268.patch,
> mr-5268.patch, mr-5268-prelim.patch
>
>
> The history server can easily take many minutes to startup when there are a
> significant number of jobs to scan in the done directory. However the
> scanning of files is not the bottleneck, rather it's the heavy use of
> ConcurrentSkipListMap.size in HistoryFileManager.
> ConcurrentSkipListMap.size is a very expensive operation, especially on maps
> with many entries, as it has to scan every entry to compute the size. We
> should avoid calling this method or at least minimize its use.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5268) Improve history server startup performance
[ https://issues.apache.org/jira/browse/MAPREDUCE-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Kambatla updated MAPREDUCE-5268: Attachment: mr-5268.patch Updated the helper class name to JobIdHistoryFileInfoMap > Improve history server startup performance > -- > > Key: MAPREDUCE-5268 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5268 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver >Affects Versions: 0.23.7, 2.0.4-alpha >Reporter: Jason Lowe >Assignee: Karthik Kambatla > Attachments: mr-5268.patch, mr-5268.patch, mr-5268.patch, > mr-5268.patch, mr-5268-prelim.patch > > > The history server can easily take many minutes to startup when there are a > significant number of jobs to scan in the done directory. However the > scanning of files is not the bottleneck, rather it's the heavy use of > ConcurrentSkipListMap.size in HistoryFileManager. > ConcurrentSkipListMap.size is a very expensive operation, especially on maps > with many entries, as it has to scan every entry to compute the size. We > should avoid calling this method or at least minimize its use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5268) Improve history server startup performance
[ https://issues.apache.org/jira/browse/MAPREDUCE-5268?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671535#comment-13671535 ] Jason Lowe commented on MAPREDUCE-5268: --- Thanks for updating so quickly, Karthik. Sorry, I should have been more clear on my nit with the templates. I was surprised to see such a generic type, both in name and template use, for a specific, private use-case. If the name is something so generic as ConcurrentSkipListMapWithSize then I think it is appropriate to use templates with that name since the name implies it is generic. If we think the templates are overkill then I think it's better to name it more specifically for its purpose, e.g.: JobHistoryInfoMap. I do like the AtomicInteger change. And as far as the size inconsistency concern, that's already an issue because of the very nature of concurrency whether we synchronize the methods or not. As soon as we get/compute the size, it could be wrong as someone else comes in and changes the map immediately afterwards. In this case the size is only being used to loosely cap the size of the job cache, and it's not critical if we're a few jobs over or a few jobs under that limit. So I think we're really close. I'd like to see either the ConcurrentSkipListMapWithSize name changed to be more specific to its use case or keep the name and have the templates restored. I don't care too much either way. > Improve history server startup performance > -- > > Key: MAPREDUCE-5268 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5268 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver >Affects Versions: 0.23.7, 2.0.4-alpha >Reporter: Jason Lowe >Assignee: Karthik Kambatla > Attachments: mr-5268.patch, mr-5268.patch, mr-5268.patch, > mr-5268-prelim.patch > > > The history server can easily take many minutes to startup when there are a > significant number of jobs to scan in the done directory. However the > scanning of files is not the bottleneck, rather it's the heavy use of > ConcurrentSkipListMap.size in HistoryFileManager. > ConcurrentSkipListMap.size is a very expensive operation, especially on maps > with many entries, as it has to scan every entry to compute the size. We > should avoid calling this method or at least minimize its use. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5286) startContainer call should use the ContainerToken instead of Container [YARN-684]
[ https://issues.apache.org/jira/browse/MAPREDUCE-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671500#comment-13671500 ] Hudson commented on MAPREDUCE-5286: --- Integrated in Hadoop-Mapreduce-trunk #1442 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1442/]) MAPREDUCE-5286. Change MapReduce to use ContainerTokenIdentifier instead of the entire Container in the startContainer call - YARN-684. Contributed by Vinod Kumar Vavilapalli. (Revision 1488087) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488087 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java > startContainer call should use the ContainerToken instead of Container > [YARN-684] > - > > Key: MAPREDUCE-5286 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5286 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Vinod Kumar Vavilapalli > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5286.txt > > > MapReduce counterpart of YARN-684. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5282) Update MR App to use immutable ApplicationID after YARN-716
[ https://issues.apache.org/jira/browse/MAPREDUCE-5282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671497#comment-13671497 ] Hudson commented on MAPREDUCE-5282: --- Integrated in Hadoop-Mapreduce-trunk #1442 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1442/]) YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth. (Revision 1487994) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1487994 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestStagingCleanup.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAppController.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestShufflePlugin.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistory.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationIdPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestYarnClient.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Apps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/MockApps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerReboot.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerShutdown.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdater.java * /hadoop/common
[jira] [Commented] (MAPREDUCE-5261) TestRMContainerAllocator is exiting and failing the build
[ https://issues.apache.org/jira/browse/MAPREDUCE-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671494#comment-13671494 ] Hudson commented on MAPREDUCE-5261: --- Integrated in Hadoop-Mapreduce-trunk #1442 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1442/]) MAPREDUCE-5261. Reverting the patch as it is no longer needed. (Revision 1488032) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488032 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java > TestRMContainerAllocator is exiting and failing the build > - > > Key: MAPREDUCE-5261 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5261 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.0.5-beta >Reporter: Jason Lowe >Assignee: Omkar Vinit Joshi > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5261.20130521.1.patch, > MAPREDUCE-5261.20130521.2.patch, MAPREDUCE-5261.20130521.patch, > MAPREDUCE-5261.20130529.patch, MAPREDUCE-5261.20130529.patch, > MAPREDUCE-5261.patch > > > Recent builds are failing because TestRMContainerAllocator is exiting rather > than succeeding or failing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5228) Enum Counter is removed from FileInputFormat and FileOutputFormat of both mapred and mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671489#comment-13671489 ] Hudson commented on MAPREDUCE-5228: --- Integrated in Hadoop-Mapreduce-trunk #1442 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1442/]) MAPREDUCE-5228. Bring back FileInputFormat.Counter and FileOuputFormat.Counter for binary compatibility with 1.x mapred APIs. Contributed by Mayank Bansal. (Revision 1488060) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488060 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Counters.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileOutputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestJobCounters.java > Enum Counter is removed from FileInputFormat and FileOutputFormat of both > mapred and mapreduce > -- > > Key: MAPREDUCE-5228 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5228 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5228-trunk-1.patch, > MAPREDUCE-5228-trunk-2.patch, MAPREDUCE-5228-trunk-3.patch, > MAPREDUCE-5228-trunk-4.patch, MAPREDUCE-5228-trunk-5.patch, > MAPREDUCE-5228-trunk-6.patch > > > The enum was used by findCounter(Enum key) to find a specific counter object. > Now it seems to be replaced by FileInputFormatCounter and > FileOutputFormatCounter. Now the enum seems to be only used internally, but > not sure whether it will be used externally when users extend FileFormat. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5176) Preemptable annotations (to support preemption in MR)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671490#comment-13671490 ] Hudson commented on MAPREDUCE-5176: --- Integrated in Hadoop-Mapreduce-trunk #1442 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1442/]) MAPREDUCE-5176. Add annotation for tagging tasks as responsive to preemption. Contributed by Carlo Curino, cdouglas (Revision 1488103) Result = SUCCESS cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488103 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Reducer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/PartialFileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/PartialOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/annotation * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/annotation/Checkpointable.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestPreemptableFileOutputCommitter.java > Preemptable annotations (to support preemption in MR) > - > > Key: MAPREDUCE-5176 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5176 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Reporter: Carlo Curino >Assignee: Carlo Curino > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5176.1.patch, MAPREDUCE-5176.2.patch, > MAPREDUCE-5176.3.patch, MAPREDUCE-5176.patch > > > Proposing a patch that introduces a new annotation @Preemptable that > represents to the framework property of user-supplied classes (e.g., Reducer, > OutputCommiter). The intended semantics is that a tagged class is safe to be > preempted between invocations. > (this is in spirit similar to the Output Contracts of [Nephele/PACT | > https://stratosphere.eu/sites/default/files/papers/ComparingMapReduceAndPACTs_11.pdf]) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5286) startContainer call should use the ContainerToken instead of Container [YARN-684]
[ https://issues.apache.org/jira/browse/MAPREDUCE-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671437#comment-13671437 ] Hudson commented on MAPREDUCE-5286: --- Integrated in Hadoop-Hdfs-trunk #1416 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1416/]) MAPREDUCE-5286. Change MapReduce to use ContainerTokenIdentifier instead of the entire Container in the startContainer call - YARN-684. Contributed by Vinod Kumar Vavilapalli. (Revision 1488087) Result = FAILURE sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488087 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java > startContainer call should use the ContainerToken instead of Container > [YARN-684] > - > > Key: MAPREDUCE-5286 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5286 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Vinod Kumar Vavilapalli > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5286.txt > > > MapReduce counterpart of YARN-684. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5282) Update MR App to use immutable ApplicationID after YARN-716
[ https://issues.apache.org/jira/browse/MAPREDUCE-5282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671434#comment-13671434 ] Hudson commented on MAPREDUCE-5282: --- Integrated in Hadoop-Hdfs-trunk #1416 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1416/]) YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth. (Revision 1487994) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1487994 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestStagingCleanup.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAppController.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestShufflePlugin.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistory.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationIdPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestYarnClient.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Apps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/MockApps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerReboot.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerShutdown.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdater.java * /hadoop/common/trunk/had
[jira] [Commented] (MAPREDUCE-5261) TestRMContainerAllocator is exiting and failing the build
[ https://issues.apache.org/jira/browse/MAPREDUCE-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671431#comment-13671431 ] Hudson commented on MAPREDUCE-5261: --- Integrated in Hadoop-Hdfs-trunk #1416 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1416/]) MAPREDUCE-5261. Reverting the patch as it is no longer needed. (Revision 1488032) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488032 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java > TestRMContainerAllocator is exiting and failing the build > - > > Key: MAPREDUCE-5261 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5261 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.0.5-beta >Reporter: Jason Lowe >Assignee: Omkar Vinit Joshi > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5261.20130521.1.patch, > MAPREDUCE-5261.20130521.2.patch, MAPREDUCE-5261.20130521.patch, > MAPREDUCE-5261.20130529.patch, MAPREDUCE-5261.20130529.patch, > MAPREDUCE-5261.patch > > > Recent builds are failing because TestRMContainerAllocator is exiting rather > than succeeding or failing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5228) Enum Counter is removed from FileInputFormat and FileOutputFormat of both mapred and mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671425#comment-13671425 ] Hudson commented on MAPREDUCE-5228: --- Integrated in Hadoop-Hdfs-trunk #1416 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1416/]) MAPREDUCE-5228. Bring back FileInputFormat.Counter and FileOuputFormat.Counter for binary compatibility with 1.x mapred APIs. Contributed by Mayank Bansal. (Revision 1488060) Result = FAILURE vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488060 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Counters.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileOutputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestJobCounters.java > Enum Counter is removed from FileInputFormat and FileOutputFormat of both > mapred and mapreduce > -- > > Key: MAPREDUCE-5228 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5228 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5228-trunk-1.patch, > MAPREDUCE-5228-trunk-2.patch, MAPREDUCE-5228-trunk-3.patch, > MAPREDUCE-5228-trunk-4.patch, MAPREDUCE-5228-trunk-5.patch, > MAPREDUCE-5228-trunk-6.patch > > > The enum was used by findCounter(Enum key) to find a specific counter object. > Now it seems to be replaced by FileInputFormatCounter and > FileOutputFormatCounter. Now the enum seems to be only used internally, but > not sure whether it will be used externally when users extend FileFormat. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5176) Preemptable annotations (to support preemption in MR)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671426#comment-13671426 ] Hudson commented on MAPREDUCE-5176: --- Integrated in Hadoop-Hdfs-trunk #1416 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1416/]) MAPREDUCE-5176. Add annotation for tagging tasks as responsive to preemption. Contributed by Carlo Curino, cdouglas (Revision 1488103) Result = FAILURE cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488103 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Reducer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/PartialFileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/PartialOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/annotation * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/annotation/Checkpointable.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestPreemptableFileOutputCommitter.java > Preemptable annotations (to support preemption in MR) > - > > Key: MAPREDUCE-5176 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5176 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Reporter: Carlo Curino >Assignee: Carlo Curino > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5176.1.patch, MAPREDUCE-5176.2.patch, > MAPREDUCE-5176.3.patch, MAPREDUCE-5176.patch > > > Proposing a patch that introduces a new annotation @Preemptable that > represents to the framework property of user-supplied classes (e.g., Reducer, > OutputCommiter). The intended semantics is that a tagged class is safe to be > preempted between invocations. > (this is in spirit similar to the Output Contracts of [Nephele/PACT | > https://stratosphere.eu/sites/default/files/papers/ComparingMapReduceAndPACTs_11.pdf]) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-5287) Create a generic InputFormat wrapping any other InputFormat, to control the number of map tasks
nicu marasoiu created MAPREDUCE-5287: Summary: Create a generic InputFormat wrapping any other InputFormat, to control the number of map tasks Key: MAPREDUCE-5287 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5287 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv1, performance Reporter: nicu marasoiu I wrote a generic InputFormat that wraps any other InputFormat, and creates CompositeInputSplits to reduce the number of map tasks in a controllable manner while preserving data locality. A correspondent CompositeRecordReader is written to iterate through underlying RecordReaders as created by the underlying InputFormat for each underlying raw split. An application to this is to group TableSplits when the raw splits are coming from multiple regions and are filtered with key ranges. We use this to shard/distribute a time based incremental access to an hbase table. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5286) startContainer call should use the ContainerToken instead of Container [YARN-684]
[ https://issues.apache.org/jira/browse/MAPREDUCE-5286?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671329#comment-13671329 ] Hudson commented on MAPREDUCE-5286: --- Integrated in Hadoop-Yarn-trunk #226 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/226/]) MAPREDUCE-5286. Change MapReduce to use ContainerTokenIdentifier instead of the entire Container in the startContainer call - YARN-684. Contributed by Vinod Kumar Vavilapalli. (Revision 1488087) Result = SUCCESS sseth : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488087 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/launcher/ContainerLauncherImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/launcher/TestContainerLauncher.java > startContainer call should use the ContainerToken instead of Container > [YARN-684] > - > > Key: MAPREDUCE-5286 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5286 > Project: Hadoop Map/Reduce > Issue Type: Task >Reporter: Siddharth Seth >Assignee: Vinod Kumar Vavilapalli > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5286.txt > > > MapReduce counterpart of YARN-684. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5282) Update MR App to use immutable ApplicationID after YARN-716
[ https://issues.apache.org/jira/browse/MAPREDUCE-5282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671326#comment-13671326 ] Hudson commented on MAPREDUCE-5282: --- Integrated in Hadoop-Yarn-trunk #226 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/226/]) YARN-716. Making ApplicationID immutable. Contributed by Siddharth Seth. MAPREDUCE-5282. Updating MR App to use immutable ApplicationID after YARN-716. Contributed by Siddharth Seth. (Revision 1487994) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1487994 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/MRApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRuntimeEstimators.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestStagingCleanup.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TestTaskImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/webapp/TestAppController.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/main/java/org/apache/hadoop/mapreduce/TypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/TestTypeConverter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-common/src/test/java/org/apache/hadoop/mapreduce/v2/util/TestMRApps.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/TestShufflePlugin.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/src/main/java/org/apache/hadoop/mapreduce/v2/hs/JobHistory.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestResourceMgrDelegate.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestYARNRunner.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-shuffle/src/main/java/org/apache/hadoop/mapred/ShuffleHandler.java * /hadoop/common/trunk/hadoop-yarn-project/CHANGES.txt * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/ApplicationId.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ApplicationIdPBImpl.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-client/src/test/java/org/apache/hadoop/yarn/client/TestYarnClient.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Apps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/BuilderUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/ConverterUtils.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/MockApps.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestContainerLaunchRPC.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/test/java/org/apache/hadoop/yarn/TestRPC.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestEventFlow.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerReboot.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeManagerShutdown.java * /hadoop/common/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNodeStatusUpdater.java * /hadoop/common/trunk/hadoo
[jira] [Commented] (MAPREDUCE-5261) TestRMContainerAllocator is exiting and failing the build
[ https://issues.apache.org/jira/browse/MAPREDUCE-5261?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671323#comment-13671323 ] Hudson commented on MAPREDUCE-5261: --- Integrated in Hadoop-Yarn-trunk #226 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/226/]) MAPREDUCE-5261. Reverting the patch as it is no longer needed. (Revision 1488032) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488032 Files : * /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/SecurityUtil.java * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/test/java/org/apache/hadoop/mapreduce/v2/app/TestRMContainerAllocator.java > TestRMContainerAllocator is exiting and failing the build > - > > Key: MAPREDUCE-5261 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5261 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.0.5-beta >Reporter: Jason Lowe >Assignee: Omkar Vinit Joshi > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5261.20130521.1.patch, > MAPREDUCE-5261.20130521.2.patch, MAPREDUCE-5261.20130521.patch, > MAPREDUCE-5261.20130529.patch, MAPREDUCE-5261.20130529.patch, > MAPREDUCE-5261.patch > > > Recent builds are failing because TestRMContainerAllocator is exiting rather > than succeeding or failing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5176) Preemptable annotations (to support preemption in MR)
[ https://issues.apache.org/jira/browse/MAPREDUCE-5176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671319#comment-13671319 ] Hudson commented on MAPREDUCE-5176: --- Integrated in Hadoop-Yarn-trunk #226 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/226/]) MAPREDUCE-5176. Add annotation for tagging tasks as responsive to preemption. Contributed by Carlo Curino, cdouglas (Revision 1488103) Result = SUCCESS cdouglas : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488103 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/Reducer.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/PartialFileOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/PartialOutputCommitter.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/annotation * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/annotation/Checkpointable.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/test/java/org/apache/hadoop/mapreduce/lib/output/TestPreemptableFileOutputCommitter.java > Preemptable annotations (to support preemption in MR) > - > > Key: MAPREDUCE-5176 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5176 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: mrv2 >Reporter: Carlo Curino >Assignee: Carlo Curino > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5176.1.patch, MAPREDUCE-5176.2.patch, > MAPREDUCE-5176.3.patch, MAPREDUCE-5176.patch > > > Proposing a patch that introduces a new annotation @Preemptable that > represents to the framework property of user-supplied classes (e.g., Reducer, > OutputCommiter). The intended semantics is that a tagged class is safe to be > preempted between invocations. > (this is in spirit similar to the Output Contracts of [Nephele/PACT | > https://stratosphere.eu/sites/default/files/papers/ComparingMapReduceAndPACTs_11.pdf]) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5228) Enum Counter is removed from FileInputFormat and FileOutputFormat of both mapred and mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-5228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671318#comment-13671318 ] Hudson commented on MAPREDUCE-5228: --- Integrated in Hadoop-Yarn-trunk #226 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/226/]) MAPREDUCE-5228. Bring back FileInputFormat.Counter and FileOuputFormat.Counter for binary compatibility with 1.x mapred APIs. Contributed by Mayank Bansal. (Revision 1488060) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1488060 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/Counters.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileInputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/FileOutputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileInputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/input/FileOutputFormat_Counter.properties * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/lib/output/FileOutputFormat.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapred/TestJobCounters.java > Enum Counter is removed from FileInputFormat and FileOutputFormat of both > mapred and mapreduce > -- > > Key: MAPREDUCE-5228 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5228 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5228-trunk-1.patch, > MAPREDUCE-5228-trunk-2.patch, MAPREDUCE-5228-trunk-3.patch, > MAPREDUCE-5228-trunk-4.patch, MAPREDUCE-5228-trunk-5.patch, > MAPREDUCE-5228-trunk-6.patch > > > The enum was used by findCounter(Enum key) to find a specific counter object. > Now it seems to be replaced by FileInputFormatCounter and > FileOutputFormatCounter. Now the enum seems to be only used internally, but > not sure whether it will be used externally when users extend FileFormat. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5036) Default shuffle handler port should not be 8080
[ https://issues.apache.org/jira/browse/MAPREDUCE-5036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671311#comment-13671311 ] Tom White commented on MAPREDUCE-5036: -- +1 I'll commit this soon unless there are any objections to the new port number. > Default shuffle handler port should not be 8080 > --- > > Key: MAPREDUCE-5036 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5036 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Affects Versions: 2.0.3-alpha >Reporter: Sandy Ryza >Assignee: Sandy Ryza > Fix For: 2.0.5-beta > > Attachments: MAPREDUCE-5036-13562.patch, MAPREDUCE-5036.patch > > > The shuffle handler port (mapreduce.shuffle.port) defaults to 8080. This is > a pretty common port for web services, and is likely to cause unnecessary > port conflicts. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-5275) Mapreduce API: TokenCache incompatibility issues with MR1
[
https://issues.apache.org/jira/browse/MAPREDUCE-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13671246#comment-13671246
]
Hadoop QA commented on MAPREDUCE-5275:
--
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12585561/MAPREDUCE-5275-trunk-4.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. The javadoc tool did not generate any
warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3708//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/3708//console
This message is automatically generated.
> Mapreduce API: TokenCache incompatibility issues with MR1
> -
>
> Key: MAPREDUCE-5275
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-5275
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
>Reporter: Zhijie Shen
>Assignee: Mayank Bansal
> Attachments: MAPREDUCE-5275-trunk-1.patch,
> MAPREDUCE-5275-trunk-2.patch, MAPREDUCE-5275-trunk-3.patch,
> MAPREDUCE-5275-trunk-4.patch
>
>
> There're following incompatibility issues:
> * Token getDelegationToken(Credentials, String) is
> removed
> * Credentials loadTokens(String, Configuration) changes to Credentials
> loadTokens(String, JobConf)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5275) Mapreduce API: TokenCache incompatibility issues with MR1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Bansal updated MAPREDUCE-5275: - Status: Patch Available (was: Open) > Mapreduce API: TokenCache incompatibility issues with MR1 > - > > Key: MAPREDUCE-5275 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5275 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Attachments: MAPREDUCE-5275-trunk-1.patch, > MAPREDUCE-5275-trunk-2.patch, MAPREDUCE-5275-trunk-3.patch, > MAPREDUCE-5275-trunk-4.patch > > > There're following incompatibility issues: > * Token getDelegationToken(Credentials, String) is > removed > * Credentials loadTokens(String, Configuration) changes to Credentials > loadTokens(String, JobConf) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5275) Mapreduce API: TokenCache incompatibility issues with MR1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Bansal updated MAPREDUCE-5275: - Attachment: MAPREDUCE-5275-trunk-4.patch Fixing the findbug warning. Thanks, Mayank > Mapreduce API: TokenCache incompatibility issues with MR1 > - > > Key: MAPREDUCE-5275 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5275 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Attachments: MAPREDUCE-5275-trunk-1.patch, > MAPREDUCE-5275-trunk-2.patch, MAPREDUCE-5275-trunk-3.patch, > MAPREDUCE-5275-trunk-4.patch > > > There're following incompatibility issues: > * Token getDelegationToken(Credentials, String) is > removed > * Credentials loadTokens(String, Configuration) changes to Credentials > loadTokens(String, JobConf) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-5275) Mapreduce API: TokenCache incompatibility issues with MR1
[ https://issues.apache.org/jira/browse/MAPREDUCE-5275?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mayank Bansal updated MAPREDUCE-5275: - Status: Open (was: Patch Available) > Mapreduce API: TokenCache incompatibility issues with MR1 > - > > Key: MAPREDUCE-5275 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-5275 > Project: Hadoop Map/Reduce > Issue Type: Sub-task >Reporter: Zhijie Shen >Assignee: Mayank Bansal > Attachments: MAPREDUCE-5275-trunk-1.patch, > MAPREDUCE-5275-trunk-2.patch, MAPREDUCE-5275-trunk-3.patch, > MAPREDUCE-5275-trunk-4.patch > > > There're following incompatibility issues: > * Token getDelegationToken(Credentials, String) is > removed > * Credentials loadTokens(String, Configuration) changes to Credentials > loadTokens(String, JobConf) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
