[jira] [Created] (MAPREDUCE-2935) MRV1 test compilation broken after HDFS-2197
MRV1 test compilation broken after HDFS-2197
Key: MAPREDUCE-2935
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2935
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: test
Reporter: Vinod Kumar Vavilapalli
Runing the following:
At the trunk level: {{mvn clean install package -Dtar -Pdist
-Dmaven.test.skip.exec=true}}
In hadoop-mapreduce-project: {{ant jar-test -Dresolvers=internal}}
yields the errors:
{code}
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java:62:
cannot find symbol
[javac] symbol : method
getRpcServer(org.apache.hadoop.hdfs.server.namenode.NameNode)
[javac] location: class
org.apache.hadoop.hdfs.server.namenode.NameNodeAdapter
[javac] Set> protocolsWithAcls =
NameNodeAdapter.getRpcServer(dfs.getNameNode())
[javac]^
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java:79:
cannot find symbol
[javac] symbol : method
getRpcServer(org.apache.hadoop.hdfs.server.namenode.NameNode)
[javac] location: class
org.apache.hadoop.hdfs.server.namenode.NameNodeAdapter
[javac] protocolsWithAcls =
NameNodeAdapter.getRpcServer(dfs.getNameNode())
[javac] 2 errors
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2935) MRV1 test compilation broken after HDFS-2197
[
https://issues.apache.org/jira/browse/MAPREDUCE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097813#comment-13097813
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2935:
Not sure of the fix, looked at the patch at HDFS-2197 and it doesn't seem like
the RpcServer is exposed to the outside world anymore. Added Todd who worked
on HDFS-2197 on the watch list and commenting on that ticket too.
> MRV1 test compilation broken after HDFS-2197
>
>
> Key: MAPREDUCE-2935
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2935
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Reporter: Vinod Kumar Vavilapalli
>
> Runing the following:
> At the trunk level: {{mvn clean install package -Dtar -Pdist
> -Dmaven.test.skip.exec=true}}
> In hadoop-mapreduce-project: {{ant jar-test -Dresolvers=internal}}
> yields the errors:
> {code}
> [javac]
> /home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java:62:
> cannot find symbol
> [javac] symbol : method
> getRpcServer(org.apache.hadoop.hdfs.server.namenode.NameNode)
> [javac] location: class
> org.apache.hadoop.hdfs.server.namenode.NameNodeAdapter
> [javac] Set> protocolsWithAcls =
> NameNodeAdapter.getRpcServer(dfs.getNameNode())
> [javac]^
> [javac]
> /home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java:79:
> cannot find symbol
> [javac] symbol : method
> getRpcServer(org.apache.hadoop.hdfs.server.namenode.NameNode)
> [javac] location: class
> org.apache.hadoop.hdfs.server.namenode.NameNodeAdapter
> [javac] protocolsWithAcls =
> NameNodeAdapter.getRpcServer(dfs.getNameNode())
> [javac] 2 errors
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
Contrib Raid compilation broken after HDFS-1620 --- Key: MAPREDUCE-2936 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 Project: Hadoop Map/Reduce Issue Type: Bug Reporter: Vinod Kumar Vavilapalli After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization and runing the following: At the trunk level: mvn clean install package -Dtar -Pdist -Dmaven.test.skip.exec=true In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[
https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097820#comment-13097820
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2936:
The errors:
{code}
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:34:
cannot find symbol
[javac] symbol : class FSConstants
[javac] location: package org.apache.hadoop.hdfs.protocol
[javac] import org.apache.hadoop.hdfs.protocol.FSConstants;
[javac] ^
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java:49:
cannot find symbol
[javac] symbol : class FSConstants
[javac] location: package org.apache.hadoop.hdfs.protocol
[javac] import org.apache.hadoop.hdfs.protocol.FSConstants;
[javac] ^
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java:50:
package org.apache.hadoop.hdfs.protocol.FSConstants does not exist
[javac] import
org.apache.hadoop.hdfs.protocol.FSConstants.DatanodeReportType;
[javac] ^
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java:53:
cannot find symbol
[javac] symbol : class HdfsConstants
[javac] location: package org.apache.hadoop.hdfs.server.common
[javac] import org.apache.hadoop.hdfs.server.common.HdfsConstants;
[javac]^
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:392:
cannot find symbol
[javac] symbol : variable FSConstants
[javac] location: class
org.apache.hadoop.hdfs.server.datanode.RaidBlockSender
[javac] maxChunksPerPacket =
(Math.max(FSConstants.IO_FILE_BUFFER_SIZE,
[javac]^
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:392:
operator + cannot be applied to ,int
[javac] maxChunksPerPacket =
(Math.max(FSConstants.IO_FILE_BUFFER_SIZE,
[javac] ^
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:394:
operator - cannot be applied to ,int
[javac] + bytesPerChecksum -
1)/bytesPerChecksum;
[javac] ^
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:400:
cannot find symbol
[javac] symbol : variable FSConstants
[javac] location: class
org.apache.hadoop.hdfs.server.datanode.RaidBlockSender
[javac] (FSConstants.IO_FILE_BUFFER_SIZE + bytesPerChecksum -
1)/bytesPerChecksum);
[javac] ^
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:400:
operator + cannot be applied to FSConstants.IO_FILE_BUFFER_SIZE,int
[javac] (FSConstants.IO_FILE_BUFFER_SIZE + bytesPerChecksum -
1)/bytesPerChecksum);
[javac] ^
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java:400:
operator - cannot be applied to ,int
[javac] (FSConstants.IO_FILE_BUFFER_SIZE + bytesPerChecksum -
1)/bytesPerChecksum);
[javac] ^
[javac]
/home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java:615:
cannot find symbol
[javac] symbol : variable DatanodeReportType
[javac] location: class org.apache.hadoop.raid.BlockFixer.BlockFixerHelper
[javac] dfs.getClient().datanodeReport(DatanodeReportType.LIVE);
[javac]^
[javac]
/home/vinodkv/Workspace/eclips
[jira] [Updated] (MAPREDUCE-2933) Change allocate call to return ContainerStatus for completed containers rather than Container
[ https://issues.apache.org/jira/browse/MAPREDUCE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2933: - Status: Patch Available (was: Open) > Change allocate call to return ContainerStatus for completed containers > rather than Container > -- > > Key: MAPREDUCE-2933 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2933 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, nodemanager, resourcemanager >Reporter: Arun C Murthy >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2933.patch > > > Change allocate call to return ContainerStatus for completed containers > rather than Container, we should do this all the way from the NodeManager too. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2933) Change allocate call to return ContainerStatus for completed containers rather than Container
[ https://issues.apache.org/jira/browse/MAPREDUCE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2933: - Attachment: MAPREDUCE-2933.patch Using ContainerStatus everywhere from allocate call to NodeManager-ResourceManager heartbeat. > Change allocate call to return ContainerStatus for completed containers > rather than Container > -- > > Key: MAPREDUCE-2933 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2933 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, nodemanager, resourcemanager >Reporter: Arun C Murthy >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2933.patch > > > Change allocate call to return ContainerStatus for completed containers > rather than Container, we should do this all the way from the NodeManager too. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli reassigned MAPREDUCE-2936: -- Assignee: Vinod Kumar Vavilapalli > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2936: --- Attachment: MAPREDUCE-2936-20110906.txt Patch that should fix the issue. You can use it to proceed with builds. Please don't commit this yet, I will confirm if HDFS-1620 can fix it to be a backwards-compatible change. > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097880#comment-13097880 ] Harsh J commented on MAPREDUCE-2936: Apologies. I overlooked contrib/raid for these constants. I believe since those classes aren't publicly consumable ones, it should be OK to just have the renaming done as provided in your patch? +1 to that patch, btw, lgtm. Also, does the HDFS-1620 affect TestServiceLevelAuthorization? I'm slightly confused if its just RAID that got affected or things beyond that too in MR land. > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097896#comment-13097896 ] Amar Kamat commented on MAPREDUCE-2936: --- This patch solves the compilation problem for me. +1. > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2936: --- Status: Patch Available (was: Open) > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097899#comment-13097899 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-2936: bq. Also, does the HDFS-1620 affect TestServiceLevelAuthorization? I'm slightly confused if its just RAID that got affected or things beyond that too in MR land. That is a different one (MAPREDUCE-2935), we are dealing with multiple issues here. Given HDFS-1620 clearly says the constants are private, I think this patch should be fine. (Ah, I hate these tools not being rebuilt whenever HDFS changes). > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097911#comment-13097911 ] Hadoop QA commented on MAPREDUCE-2936: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493137/MAPREDUCE-2936-20110906.txt against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/614//console This message is automatically generated. > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2882) TestLineRecordReader depends on ant jars
[ https://issues.apache.org/jira/browse/MAPREDUCE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097938#comment-13097938 ] Hudson commented on MAPREDUCE-2882: --- Integrated in Hadoop-Hdfs-trunk #785 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/785/]) MAPREDUCE-2882. TestLineRecordReader depends on ant jars. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165449 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestLineRecordReader.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/UtilsForTests.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/lib/input/TestLineRecordReader.java > TestLineRecordReader depends on ant jars > > > Key: MAPREDUCE-2882 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2882 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.23.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Fix For: 0.23.0 > > Attachments: mr-2882.txt > > > This test is currently importing an ant utility class to read a file - this > dependency doesn't work in mavenized land. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097942#comment-13097942 ] Hudson commented on MAPREDUCE-2697: --- Integrated in Hadoop-Hdfs-trunk #785 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/785/]) MAPREDUCE-2697. Enhance CapacityScheduler to cap concurrently running applications per-queue & per-user. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165403 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ContainerIdPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerContext.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/Queue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/resources/capacity-scheduler.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java > Enhance CS to cap concurrently running jobs > --- > > Key: MAPREDUCE-2697 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Reporter: Arun C Murthy >Assignee: Arun C Murthy > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, > MAPREDUCE-2697.patch, MAPREDUCE-2697.patch > > > Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2908) Fix findbugs warnings in Map Reduce.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli reassigned MAPREDUCE-2908: -- Assignee: Vinod Kumar Vavilapalli (was: Mahadev konar) There are 13 errors now. Bugs are escaping because of Jenkins' broken links, we are close to fixing that. I am going to fix the warnings for the time being. > Fix findbugs warnings in Map Reduce. > > > Key: MAPREDUCE-2908 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2908 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Mahadev konar >Assignee: Vinod Kumar Vavilapalli >Priority: Critical > Fix For: 0.23.0 > > > In the current trunk/0.23 codebase there are 5 findbugs warnings which cause > the precommit CI builds to -1 the patches. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097980#comment-13097980 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-2936: bq. -1 tests included. The patch doesn't appear to include any new or modified tests. The patch fixes compilation. Current tests should continue to verify the code. Unfortunately they aren't run as part of the automated builds. Will file a ticket. bq.-1 findbugs. The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings. None of the warnings are new and due to patch. I'll fix them via MAPREDUCE-2908. > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097988#comment-13097988 ] Hudson commented on MAPREDUCE-2936: --- Integrated in Hadoop-Hdfs-trunk-Commit #912 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/912/]) MAPREDUCE-2936. Contrib Raid compilation broken after HDFS-1620. Contributed by Vinod Kumar Vavilapalli. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165657 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13097991#comment-13097991 ] Hudson commented on MAPREDUCE-2936: --- Integrated in Hadoop-Mapreduce-trunk-Commit #846 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/846/]) MAPREDUCE-2936. Contrib Raid compilation broken after HDFS-1620. Contributed by Vinod Kumar Vavilapalli. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165657 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vinod Kumar Vavilapalli updated MAPREDUCE-2936: --- Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) I just committed this trunk and branch 0.23. > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2690) Construct the web page for default scheduler
[ https://issues.apache.org/jira/browse/MAPREDUCE-2690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098024#comment-13098024 ] Eric Payne commented on MAPREDUCE-2690: --- With MAPREDUCE-2690.2011-09-05.patch, I fixed a warning that had resulted from my original patch. However, the other 13 findbugs warnings that are mentioned by the above 'Hadoop QA' comment were not introduced by this patch and existed prior to this patch. > Construct the web page for default scheduler > > > Key: MAPREDUCE-2690 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2690 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Eric Payne > Fix For: 0.23.0 > > Attachments: DefaultSchedulerPage.jpg, > MAPREDUCE-2690.2011-09-02.patch, MAPREDUCE-2690.2011-09-05.patch > > > Currently, the web page for default scheduler reads as "Under construction". > This is a long known issue, but could not find a tracking ticket. Hence > opening one. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2935) MRV1 test compilation broken after HDFS-2197
[
https://issues.apache.org/jira/browse/MAPREDUCE-2935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Todd Lipcon reassigned MAPREDUCE-2935:
--
Assignee: Todd Lipcon
> MRV1 test compilation broken after HDFS-2197
>
>
> Key: MAPREDUCE-2935
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2935
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: test
>Reporter: Vinod Kumar Vavilapalli
>Assignee: Todd Lipcon
>
> Runing the following:
> At the trunk level: {{mvn clean install package -Dtar -Pdist
> -Dmaven.test.skip.exec=true}}
> In hadoop-mapreduce-project: {{ant jar-test -Dresolvers=internal}}
> yields the errors:
> {code}
> [javac]
> /home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java:62:
> cannot find symbol
> [javac] symbol : method
> getRpcServer(org.apache.hadoop.hdfs.server.namenode.NameNode)
> [javac] location: class
> org.apache.hadoop.hdfs.server.namenode.NameNodeAdapter
> [javac] Set> protocolsWithAcls =
> NameNodeAdapter.getRpcServer(dfs.getNameNode())
> [javac]^
> [javac]
> /home/vinodkv/Workspace/eclipse-workspace/apache-git/hadoop-common/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/security/authorize/TestServiceLevelAuthorization.java:79:
> cannot find symbol
> [javac] symbol : method
> getRpcServer(org.apache.hadoop.hdfs.server.namenode.NameNode)
> [javac] location: class
> org.apache.hadoop.hdfs.server.namenode.NameNodeAdapter
> [javac] protocolsWithAcls =
> NameNodeAdapter.getRpcServer(dfs.getNameNode())
> [javac] 2 errors
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michael Noll updated MAPREDUCE-2398:
Attachment: MAPREDUCE-2398_v2-0.20.203.0.patch
New version of patch, integrating Harsh J's suggestions. This patch is based
on Hadoop release 0.20.203.0.
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 0.22.0
>Reporter: Michael Noll
>Priority: Minor
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2926) 500 Error in ResourceManager UI
[
https://issues.apache.org/jira/browse/MAPREDUCE-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098068#comment-13098068
]
Robert Joseph Evans commented on MAPREDUCE-2926:
I just tested with trunk
git-svn-id: https://svn.apache.org/repos/asf/hadoop/common/trunk@1165657
13f79535-47bb-0310-9956-ffa450edef68
And The RM UI comes up just fine for me, no problems. Mahadev, I am not sure
what is causing your problems, but I thought you should be aware of it. I will
try on 0.23 too.
> 500 Error in ResourceManager UI
> ---
>
> Key: MAPREDUCE-2926
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2926
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0, 0.24.0
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2926-v1.patch, MR-2926-v2.patch
>
>
> When accessing the resource manager UI the following is returned
> {noformat}
> Problem accessing /. Reason:
> org.codehaus.jackson.type.JavaType.(Ljava/lang/Class;)V
> Caused by:
> java.lang.NoSuchMethodError:
> org.codehaus.jackson.type.JavaType.(Ljava/lang/Class;)V
> at org.codehaus.jackson.map.type.TypeBase.(TypeBase.java:15)
> at org.codehaus.jackson.map.type.SimpleType.(SimpleType.java:45)
> at org.codehaus.jackson.map.type.SimpleType.(SimpleType.java:40)
> at
> org.codehaus.jackson.map.type.TypeBindings.(TypeBindings.java:20)
> at
> org.codehaus.jackson.map.type.TypeFactory._fromType(TypeFactory.java:530)
> at org.codehaus.jackson.map.type.TypeFactory.type(TypeFactory.java:63)
> at org.codehaus.jackson.map.ObjectMapper.(ObjectMapper.java:179)
> at org.apache.hadoop.yarn.webapp.Controller.(Controller.java:43)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> com.google.inject.DefaultConstructionProxyFactory$2.newInstance(DefaultConstructionProxyFactory.java:81)
> at
> com.google.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
> at
> com.google.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:111)
> at com.google.inject.InjectorImpl$4$1.call(InjectorImpl.java:758)
> at com.google.inject.InjectorImpl.callInContext(InjectorImpl.java:804)
> at com.google.inject.InjectorImpl$4.get(InjectorImpl.java:754)
> at com.google.inject.InjectorImpl.getInstance(InjectorImpl.java:793)
> at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:136)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:216)
> at
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:141)
> at
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:93)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:63)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:122)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:110)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:892)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAv
[jira] [Commented] (MAPREDUCE-2398) MRBench: setting the baseDir parameter has no effect
[
https://issues.apache.org/jira/browse/MAPREDUCE-2398?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098071#comment-13098071
]
Hadoop QA commented on MAPREDUCE-2398:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12493159/MAPREDUCE-2398_v2-0.20.203.0.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
+1 tests included. The patch appears to include 3 new or modified tests.
-1 patch. The patch command could not apply the patch.
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/615//console
This message is automatically generated.
> MRBench: setting the baseDir parameter has no effect
>
>
> Key: MAPREDUCE-2398
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2398
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: benchmarks
>Affects Versions: 0.20.2, 0.22.0
>Reporter: Michael Noll
>Priority: Minor
> Fix For: 0.22.0
>
> Attachments: MAPREDUCE-2398-trunk.patch, MAPREDUCE-2398_0.20.2.patch,
> MAPREDUCE-2398_v2-0.20.203.0.patch
>
>
> The optional {{-baseDir}} parameter lets user specify the base DFS path for
> output/input of MRBench.
> However, the two private variables {{INPUT_DIR}} and {{OUTPUT_DIR}}
> (MRBench.java) are not updated in the case that the default value of
> {{-baseDir}} is actually overwritten by the user. Hence any input and output
> is always written to the default locations ({{/benchmarks/MRBench/...}}),
> even though the user-supplied location for {{-baseDir}} is created (and
> eventually deleted again) on HDFS.
> The bug affects at least Hadoop 0.20.2 and the current trunk (r1082703) as of
> March 21, 2011.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2926) 500 Error in ResourceManager UI
[
https://issues.apache.org/jira/browse/MAPREDUCE-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098084#comment-13098084
]
Robert Joseph Evans commented on MAPREDUCE-2926:
I just tested with branch-0.23 too
git-svn-id:
https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.23@1165692
13f79535-47bb-0310-9956-ffa450edef68
And the RM UI comes up just fine again.
> 500 Error in ResourceManager UI
> ---
>
> Key: MAPREDUCE-2926
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2926
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0, 0.24.0
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2926-v1.patch, MR-2926-v2.patch
>
>
> When accessing the resource manager UI the following is returned
> {noformat}
> Problem accessing /. Reason:
> org.codehaus.jackson.type.JavaType.(Ljava/lang/Class;)V
> Caused by:
> java.lang.NoSuchMethodError:
> org.codehaus.jackson.type.JavaType.(Ljava/lang/Class;)V
> at org.codehaus.jackson.map.type.TypeBase.(TypeBase.java:15)
> at org.codehaus.jackson.map.type.SimpleType.(SimpleType.java:45)
> at org.codehaus.jackson.map.type.SimpleType.(SimpleType.java:40)
> at
> org.codehaus.jackson.map.type.TypeBindings.(TypeBindings.java:20)
> at
> org.codehaus.jackson.map.type.TypeFactory._fromType(TypeFactory.java:530)
> at org.codehaus.jackson.map.type.TypeFactory.type(TypeFactory.java:63)
> at org.codehaus.jackson.map.ObjectMapper.(ObjectMapper.java:179)
> at org.apache.hadoop.yarn.webapp.Controller.(Controller.java:43)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> com.google.inject.DefaultConstructionProxyFactory$2.newInstance(DefaultConstructionProxyFactory.java:81)
> at
> com.google.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
> at
> com.google.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:111)
> at com.google.inject.InjectorImpl$4$1.call(InjectorImpl.java:758)
> at com.google.inject.InjectorImpl.callInContext(InjectorImpl.java:804)
> at com.google.inject.InjectorImpl$4.get(InjectorImpl.java:754)
> at com.google.inject.InjectorImpl.getInstance(InjectorImpl.java:793)
> at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:136)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:216)
> at
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:141)
> at
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:93)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:63)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:122)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:110)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:892)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
>
[jira] [Commented] (MAPREDUCE-2933) Change allocate call to return ContainerStatus for completed containers rather than Container
[
https://issues.apache.org/jira/browse/MAPREDUCE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098093#comment-13098093
]
Vinod Kumar Vavilapalli commented on MAPREDUCE-2933:
Looked at the patch, mostly looks good. Few comments, majority of them minor:
- RMAppAttemptContainerFinishedEvent can have ContainerStatus instead of
container.
- CapacityScheduler.java and FifoScheduler.java:
-- In {{removeNode()}} method, the status for the CompletedContainer can be
"Container on lost node" or "Lost node" or some such instead of "Lost
container". Thoughts?
-- {{createAbnormalContainerStatus()}} method can be shared between CS and
FifoScheduler. Either via {{BuilderUtils}} or a new {{SchedulerUtils}}.
- {{SchedulerApp.containerCompleted()}}: The containerStatus field that you so
laboriously passed around isn't finally used :) Actually you will still need
the conditional that was there before, except you send out the same event but
with different ContainerStatus objects. We need to make sure that diagnostics
and the state for Containers that really ran on the NM retain the remote
information.
- {{LeafQueue.assignReservedContainer()}}: For the completedContainer, we can
set set the diagnostics to be say "Reserved container no longer needed" or
something similar.
- {{ContainerImpl}}: Removal of "@SuppressWarnings("unchecked") // dispatcher
not typed". Though I don't prefer one way to the other, so far most of the
people whose MRV2 patches I happened to review seemed to have liked this, so
let's leave it like that, lest somebody add it again :)
- {{TestNodeStatusUpdater}}: Two assert statements on memory values (+152
pre-patch and +172 post-patch) seem to have been accidentally removed.
Similarly one more statement at +127 pre-patch(+149 post-patch).
> Change allocate call to return ContainerStatus for completed containers
> rather than Container
> --
>
> Key: MAPREDUCE-2933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2933
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, resourcemanager
>Reporter: Arun C Murthy
>Assignee: Arun C Murthy
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2933.patch
>
>
> Change allocate call to return ContainerStatus for completed containers
> rather than Container, we should do this all the way from the NodeManager too.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2572) Throttle the deletion of data from the distributed cache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans updated MAPREDUCE-2572: --- Status: Open (was: Patch Available) Now that YARN is in trunk the patch really does not apply any more. I will take a look at the cache deletion policy and see if I should just close this JIRA or not. > Throttle the deletion of data from the distributed cache > > > Key: MAPREDUCE-2572 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2572 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: distributed-cache >Affects Versions: 0.23.0 >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > Fix For: 0.23.0 > > Attachments: MR-2572-trunk-v1.patch, THROTTLING-security-v1.patch > > > When deleting entries from the distributed cache we do so in a background > thread. Once the size limit of the distributed cache is reached all unused > entries are deleted. MAPREDUCE-2494 changes this so that entries are deleted > in LRU order until the usage falls below a given threshold. In either of > these cases we are periodically flooding a disk with delete requests which > can slow down all IO operations to a drive. It would be better to be able to > throttle this deletion so that it is spread out over a longer period of time. > This jira is to add in this throttling. > On investigating it seems much simpler to backport MPAREDUCE-2494 to 20S > before implementing this change rather then try to implement it without LRU > deletion, because LRU goes a long way towards reducing the load on the disk > anyways. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2745) [MR-279] NM UI should get a read-only view instead of the actual NMContext
[ https://issues.apache.org/jira/browse/MAPREDUCE-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anupam Seth updated MAPREDUCE-2745: --- Attachment: MAPREDUCE-2745-branch-0_23_v2.patch > [MR-279] NM UI should get a read-only view instead of the actual NMContext > --- > > Key: MAPREDUCE-2745 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2745 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Anupam Seth > Labels: newbie > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2745-branch-0_23.patch, > MAPREDUCE-2745-branch-0_23_v2.patch > > > NMContext is modifiable, the UI should only get read-only access. Just like > the AM web-ui. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2745) [MR-279] NM UI should get a read-only view instead of the actual NMContext
[ https://issues.apache.org/jira/browse/MAPREDUCE-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098114#comment-13098114 ] Anupam Seth commented on MAPREDUCE-2745: Thanks Vinod for the help and the feedback! I have uploaded new patch - kindly review and let me know if it doesn't look good. Test patch returns the following, but again all the findbugs warnings are in packages other than what I have touched and unrelated to any of my changes. -1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 26 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. > [MR-279] NM UI should get a read-only view instead of the actual NMContext > --- > > Key: MAPREDUCE-2745 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2745 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Anupam Seth > Labels: newbie > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2745-branch-0_23.patch, > MAPREDUCE-2745-branch-0_23_v2.patch > > > NMContext is modifiable, the UI should only get read-only access. Just like > the AM web-ui. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098128#comment-13098128 ] Harsh J commented on MAPREDUCE-2936: Thanks Vinod! > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2933) Change allocate call to return ContainerStatus for completed containers rather than Container
[
https://issues.apache.org/jira/browse/MAPREDUCE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Arun C Murthy updated MAPREDUCE-2933:
-
Status: Open (was: Patch Available)
Thanks for the review Vinod, I'll fix.
I removed the asserts in {{TestNodeStatusUpdater}} since the container resource
isn't available anymore.
> Change allocate call to return ContainerStatus for completed containers
> rather than Container
> --
>
> Key: MAPREDUCE-2933
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2933
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: applicationmaster, mrv2, nodemanager, resourcemanager
>Reporter: Arun C Murthy
>Assignee: Arun C Murthy
>Priority: Blocker
> Fix For: 0.23.0
>
> Attachments: MAPREDUCE-2933.patch
>
>
> Change allocate call to return ContainerStatus for completed containers
> rather than Container, we should do this all the way from the NodeManager too.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2926) 500 Error in ResourceManager UI
[
https://issues.apache.org/jira/browse/MAPREDUCE-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098158#comment-13098158
]
Mahadev konar commented on MAPREDUCE-2926:
--
Bobby,
Ill verify it today and close the jira if it works out for me.
> 500 Error in ResourceManager UI
> ---
>
> Key: MAPREDUCE-2926
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2926
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0, 0.24.0
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2926-v1.patch, MR-2926-v2.patch
>
>
> When accessing the resource manager UI the following is returned
> {noformat}
> Problem accessing /. Reason:
> org.codehaus.jackson.type.JavaType.(Ljava/lang/Class;)V
> Caused by:
> java.lang.NoSuchMethodError:
> org.codehaus.jackson.type.JavaType.(Ljava/lang/Class;)V
> at org.codehaus.jackson.map.type.TypeBase.(TypeBase.java:15)
> at org.codehaus.jackson.map.type.SimpleType.(SimpleType.java:45)
> at org.codehaus.jackson.map.type.SimpleType.(SimpleType.java:40)
> at
> org.codehaus.jackson.map.type.TypeBindings.(TypeBindings.java:20)
> at
> org.codehaus.jackson.map.type.TypeFactory._fromType(TypeFactory.java:530)
> at org.codehaus.jackson.map.type.TypeFactory.type(TypeFactory.java:63)
> at org.codehaus.jackson.map.ObjectMapper.(ObjectMapper.java:179)
> at org.apache.hadoop.yarn.webapp.Controller.(Controller.java:43)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> com.google.inject.DefaultConstructionProxyFactory$2.newInstance(DefaultConstructionProxyFactory.java:81)
> at
> com.google.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
> at
> com.google.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:111)
> at com.google.inject.InjectorImpl$4$1.call(InjectorImpl.java:758)
> at com.google.inject.InjectorImpl.callInContext(InjectorImpl.java:804)
> at com.google.inject.InjectorImpl$4.get(InjectorImpl.java:754)
> at com.google.inject.InjectorImpl.getInstance(InjectorImpl.java:793)
> at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:136)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:216)
> at
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:141)
> at
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:93)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:63)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:122)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:110)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:892)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.
[jira] [Updated] (MAPREDUCE-2933) Change allocate call to return ContainerStatus for completed containers rather than Container
[ https://issues.apache.org/jira/browse/MAPREDUCE-2933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2933: - Attachment: MAPREDUCE-2933.patch Updated patch for review while I continue to test. > Change allocate call to return ContainerStatus for completed containers > rather than Container > -- > > Key: MAPREDUCE-2933 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2933 > Project: Hadoop Map/Reduce > Issue Type: Sub-task > Components: applicationmaster, mrv2, nodemanager, resourcemanager >Reporter: Arun C Murthy >Assignee: Arun C Murthy >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2933.patch, MAPREDUCE-2933.patch > > > Change allocate call to return ContainerStatus for completed containers > rather than Container, we should do this all the way from the NodeManager too. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2749) [MR-279] NM registers with RM even before it starts various servers
[ https://issues.apache.org/jira/browse/MAPREDUCE-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098166#comment-13098166 ] Thomas Graves commented on MAPREDUCE-2749: -- Hey Vinod, thanks for the feedback. what is we move the ContainerTokenSecretManager to the top level NodeManager composite and also pass it into the NodeStatusUpdater so it can update it as soon as it register with RM? That way by the time the NodeStatusUpdater returns from its start() routine and can start sending heartbeats to rm everything is initialized and we won't have any race as to when the RM schedules containers for the rm. > [MR-279] NM registers with RM even before it starts various servers > --- > > Key: MAPREDUCE-2749 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2749 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2749-v2.patch, MAPREDUCE-2749.patch > > > In case NM eventually fails to start the ContainerManager server because of > say a port clash, RM will have to wait for expiry to detect the NM crash. > It is desirable to make NM register with RM only after it can start all of > its components successfully. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098178#comment-13098178 ] Aaron T. Myers commented on MAPREDUCE-2936: --- Thanks a lot for taking care of this, Vinod. I'm sorry I missed this when committing it. In the future, this should be noticed earlier once HADOOP-7612 goes in. > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2882) TestLineRecordReader depends on ant jars
[ https://issues.apache.org/jira/browse/MAPREDUCE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098202#comment-13098202 ] Hudson commented on MAPREDUCE-2882: --- Integrated in Hadoop-Mapreduce-trunk #808 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/808/]) MAPREDUCE-2882. TestLineRecordReader depends on ant jars. Contributed by Todd Lipcon. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165449 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestLineRecordReader.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/UtilsForTests.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/lib/input/TestLineRecordReader.java > TestLineRecordReader depends on ant jars > > > Key: MAPREDUCE-2882 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2882 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.23.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Fix For: 0.23.0 > > Attachments: mr-2882.txt > > > This test is currently importing an ant utility class to read a file - this > dependency doesn't work in mavenized land. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098204#comment-13098204 ] Hudson commented on MAPREDUCE-2936: --- Integrated in Hadoop-Mapreduce-trunk #808 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/808/]) MAPREDUCE-2936. Contrib Raid compilation broken after HDFS-1620. Contributed by Vinod Kumar Vavilapalli. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165657 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/datanode/RaidBlockSender.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/raid/BlockFixer.java > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2697) Enhance CS to cap concurrently running jobs
[ https://issues.apache.org/jira/browse/MAPREDUCE-2697?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098208#comment-13098208 ] Hudson commented on MAPREDUCE-2697: --- Integrated in Hadoop-Mapreduce-trunk #808 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/808/]) MAPREDUCE-2697. Enhance CapacityScheduler to cap concurrently running applications per-queue & per-user. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165403 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/src/main/java/org/apache/hadoop/yarn/api/records/impl/pb/ContainerIdPBImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerConfiguration.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacitySchedulerContext.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/LeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/ParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/Queue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/resources/capacity-scheduler.xml * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestApplicationLimits.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestLeafQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestParentQueue.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/TestUtils.java > Enhance CS to cap concurrently running jobs > --- > > Key: MAPREDUCE-2697 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2697 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Reporter: Arun C Murthy >Assignee: Arun C Murthy > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2697.patch, MAPREDUCE-2697.patch, > MAPREDUCE-2697.patch, MAPREDUCE-2697.patch > > > Enhance CS to cap concurrently running jobs ala 0.20.203 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2745) [MR-279] NM UI should get a read-only view instead of the actual NMContext
[ https://issues.apache.org/jira/browse/MAPREDUCE-2745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098219#comment-13098219 ] Hadoop QA commented on MAPREDUCE-2745: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493169/MAPREDUCE-2745-branch-0_23_v2.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 13 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/616//console This message is automatically generated. > [MR-279] NM UI should get a read-only view instead of the actual NMContext > --- > > Key: MAPREDUCE-2745 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2745 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Anupam Seth > Labels: newbie > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2745-branch-0_23.patch, > MAPREDUCE-2745-branch-0_23_v2.patch > > > NMContext is modifiable, the UI should only get read-only access. Just like > the AM web-ui. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2801) Include the native libs in java.library.path
[ https://issues.apache.org/jira/browse/MAPREDUCE-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Joseph Evans reassigned MAPREDUCE-2801: -- Assignee: Robert Joseph Evans > Include the native libs in java.library.path > - > > Key: MAPREDUCE-2801 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2801 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Robert Joseph Evans > Fix For: 0.23.0 > > > For the child tasks in mrv2, java.library.path is set to just $PWD and the > native libs are not included. Whereas in 0.20.x, java.library.path for child > tasks was set to :$PWD -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-2936: - Fix Version/s: 0.23.0 > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098266#comment-13098266 ] Robert Joseph Evans commented on MAPREDUCE-2936: Can we get this merged to branch-0.23 too? > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098267#comment-13098267 ] Arun C Murthy commented on MAPREDUCE-2936: -- It's already merged into 23 > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098285#comment-13098285 ] Harsh J commented on MAPREDUCE-2936: I failed to notice this earlier, but HDFS-1620 wasn't put into 0.23. Why was this necessary to be put into 0.23 in that case? Looking over atm's commits in HDFS-1620, I don't see a 0.23 commit but I may be wrong since I do not know how those svn callbacks work (and if they're supposed to log back here for branch commits). In any case, if HDFS-1620 was put into 0.23, we need to update its fix versions field too, just as it was done here :) > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2929) Move task-controller from bin to libexec
[ https://issues.apache.org/jira/browse/MAPREDUCE-2929?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098297#comment-13098297 ] Milind Bhandarkar commented on MAPREDUCE-2929: -- linux task-controller is scheduled to be removed from 0.23. (See MAPREDUCE-2767). > Move task-controller from bin to libexec > > > Key: MAPREDUCE-2929 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2929 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: task-controller >Affects Versions: 0.20.204.0, 0.23.0 > Environment: Java, Redhat 5.6 >Reporter: Eric Yang > > Linux task-controller is hard coded to $HADOOP_HOME/bin. Ideally, it should > be moved to $HADOOP_PREFIX/libexec for "ant binary" layout, or the updated > file structure layout for trunk. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2874) ApplicationId printed in 2 different formats and has 2 different toString routines that are used
[
https://issues.apache.org/jira/browse/MAPREDUCE-2874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Payne reassigned MAPREDUCE-2874:
-
Assignee: Eric Payne
> ApplicationId printed in 2 different formats and has 2 different toString
> routines that are used
>
>
> Key: MAPREDUCE-2874
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2874
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Thomas Graves
>Assignee: Eric Payne
> Fix For: 0.23.0
>
>
> Looks like the ApplicationId is now printed in 2 different formats.
> ApplicationIdPBImpl.java has a toString routine that prints it in the format:
> return "application_" + this.getClusterTimestamp() + "_" + this.getId();
> While the webapps use
> ./hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/Apps.java
> toString that prints it like:
> return _join("app", id.getClusterTimestamp(), id.getId());
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098338#comment-13098338 ] Robert Joseph Evans commented on MAPREDUCE-2936: Arun, my bad. I was getting compilation errors and thought it was because this had not gone into 0.23, but it looks like it was put into 0.23 but it should not have gone in. HdfsConstants does not have IO_FILE_BUFFER_SIZE in it, that is in FSConstants in 0.23. Thanks Harsh for seeing that. > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2901) Build should fail sanely if protoc isn't on PATH
[ https://issues.apache.org/jira/browse/MAPREDUCE-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098346#comment-13098346 ] Joep Rottinghuis commented on MAPREDUCE-2901: - Yeah, that is the same error I see with libprotoc 2.3.0: [INFO] --- exec-maven-plugin:1.2:exec (generate-sources) @ hadoop-yarn-api --- main: [INFO] Executed tasks [INFO] [INFO] --- exec-maven-plugin:1.2:exec (generate-sources) @ hadoop-yarn-api --- yarn_protos.proto:22:8: Option "java_generate_equals_and_hash" unknown. [INFO] [INFO] Reactor Summary: [INFO] [INFO] hadoop-yarn-api ... FAILURE [1.224s] [INFO] hadoop-yarn-common SKIPPED [INFO] hadoop-yarn-server-common . SKIPPED [INFO] hadoop-yarn-server-nodemanager SKIPPED [INFO] hadoop-yarn-server-resourcemanager SKIPPED [INFO] hadoop-yarn-server-tests .. SKIPPED [INFO] hadoop-yarn-server SKIPPED [INFO] hadoop-yarn ... SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 1.518s [INFO] Finished at: Tue Sep 06 12:33:18 PDT 2011 [INFO] Final Memory: 7M/227M [INFO] [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec (generate-sources) on project hadoop-yarn-api: Command execution failed. Process exited with an error: 1(Exit value: 1) -> [Help 1] > Build should fail sanely if protoc isn't on PATH > > > Key: MAPREDUCE-2901 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2901 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli > > It now fails "[ERROR] Failed to execute goal > org.codehaus.mojo:exec-maven-plugin:1.2:exec (generate-sources) on project > hadoop-yarn-api: Command execution failed. Process exited with an error: > 1(Exit value: 1) -> [Help 1]". > Which doesn't help much. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2901) Build should fail sanely if protoc isn't on PATH
[ https://issues.apache.org/jira/browse/MAPREDUCE-2901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098360#comment-13098360 ] Joep Rottinghuis commented on MAPREDUCE-2901: - After configure make install of protoc the yarn build came through clean. In my case I uninstalled libprotoc through the synaptic UI. The default ./configure step did not work and resulted in the following error when running protoc --version: protoc: error while loading shared libraries: libprotobuf.so.7: cannot open shared object file: No such file or directory I did a make unintall, make distclean, ./configure --prefix /usr, make, make install and then things worked fine. > Build should fail sanely if protoc isn't on PATH > > > Key: MAPREDUCE-2901 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2901 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli > > It now fails "[ERROR] Failed to execute goal > org.codehaus.mojo:exec-maven-plugin:1.2:exec (generate-sources) on project > hadoop-yarn-api: Command execution failed. Process exited with an error: > 1(Exit value: 1) -> [Help 1]". > Which doesn't help much. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-2789) [MR:279] Update the scheduling info on CLI
[ https://issues.apache.org/jira/browse/MAPREDUCE-2789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Payne reassigned MAPREDUCE-2789: - Assignee: Eric Payne > [MR:279] Update the scheduling info on CLI > -- > > Key: MAPREDUCE-2789 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2789 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Eric Payne > Fix For: 0.23.0 > > > The scheduling information such as number of containers running, memory usage > and reservations per job is not available on bin/mapred job -list CLI. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2767) Remove Linux task-controller from 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098372#comment-13098372 ] Hudson commented on MAPREDUCE-2767: --- Integrated in Hadoop-Common-trunk-Commit #837 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/837/]) MAPREDUCE-2767. Remove Linux task-controller. Contributed by Milind Bhandarkar. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165864 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/build.xml * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/.autom4te.cfg * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/Makefile.am * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/configuration.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/configuration.h * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/configure.ac * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/main.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/task-controller.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/task-controller.h * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/tests/test-task-controller.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/ClusterWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestDebugScriptWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestJobExecutionAsDifferentUser.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestKillSubProcessesWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestLocalizationWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestTrackerDistributedCacheManagerWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/pipes/TestPipesAsDifferentUser.java > Remove Linux task-controller from 0.22 branch > - > > Key: MAPREDUCE-2767 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2767 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.22.0, 0.23.0, 0.24.0 >Reporter: Milind Bhandarkar >Assignee: Milind Bhandarkar >Priority: Blocker > Fix For: 0.22.0, 0.23.0, 0.24.0 > > Attachments: MR2767-22.patch, MR2767-23.patch, MR2767-trunk.patch, > TEST-org.apache.hadoop.mapred.TestMiniMRChildTask.txt, testlog.txt > > > There's a potential security hole in the task-controller as it stands. Based > on the discussion on general@, removing task-controller from the 0.22 branch > will pave way for 0.22.0 release. (This was done for the 0.21.0 release as > well: see MAPREDUCE-2014.) We can roll a 0.22.1 release with the > task-controller when it is fixed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2767) Remove Linux task-controller from 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098375#comment-13098375 ] Hudson commented on MAPREDUCE-2767: --- Integrated in Hadoop-Hdfs-trunk-Commit #914 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/914/]) MAPREDUCE-2767. Remove Linux task-controller. Contributed by Milind Bhandarkar. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165864 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/build.xml * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/.autom4te.cfg * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/Makefile.am * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/configuration.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/configuration.h * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/configure.ac * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/main.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/task-controller.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/task-controller.h * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/tests/test-task-controller.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/ClusterWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestDebugScriptWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestJobExecutionAsDifferentUser.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestKillSubProcessesWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestLocalizationWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestTrackerDistributedCacheManagerWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/pipes/TestPipesAsDifferentUser.java > Remove Linux task-controller from 0.22 branch > - > > Key: MAPREDUCE-2767 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2767 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.22.0, 0.23.0, 0.24.0 >Reporter: Milind Bhandarkar >Assignee: Milind Bhandarkar >Priority: Blocker > Fix For: 0.22.0, 0.23.0, 0.24.0 > > Attachments: MR2767-22.patch, MR2767-23.patch, MR2767-trunk.patch, > TEST-org.apache.hadoop.mapred.TestMiniMRChildTask.txt, testlog.txt > > > There's a potential security hole in the task-controller as it stands. Based > on the discussion on general@, removing task-controller from the 0.22 branch > will pave way for 0.22.0 release. (This was done for the 0.21.0 release as > well: see MAPREDUCE-2014.) We can roll a 0.22.1 release with the > task-controller when it is fixed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2767) Remove Linux task-controller from 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098379#comment-13098379 ] Hudson commented on MAPREDUCE-2767: --- Integrated in Hadoop-Mapreduce-trunk-Commit #848 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/848/]) MAPREDUCE-2767. Remove Linux task-controller. Contributed by Milind Bhandarkar. shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165864 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/build.xml * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/.autom4te.cfg * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/Makefile.am * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/configuration.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/configuration.h * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/configure.ac * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/main.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/task-controller.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/task-controller.h * /hadoop/common/trunk/hadoop-mapreduce-project/src/c++/task-controller/tests/test-task-controller.c * /hadoop/common/trunk/hadoop-mapreduce-project/src/java/org/apache/hadoop/mapred/LinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/ClusterWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestDebugScriptWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestJobExecutionAsDifferentUser.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestKillSubProcessesWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestLocalizationWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestTrackerDistributedCacheManagerWithLinuxTaskController.java * /hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/pipes/TestPipesAsDifferentUser.java > Remove Linux task-controller from 0.22 branch > - > > Key: MAPREDUCE-2767 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2767 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.22.0, 0.23.0, 0.24.0 >Reporter: Milind Bhandarkar >Assignee: Milind Bhandarkar >Priority: Blocker > Fix For: 0.22.0, 0.23.0, 0.24.0 > > Attachments: MR2767-22.patch, MR2767-23.patch, MR2767-trunk.patch, > TEST-org.apache.hadoop.mapred.TestMiniMRChildTask.txt, testlog.txt > > > There's a potential security hole in the task-controller as it stands. Based > on the discussion on general@, removing task-controller from the 0.22 branch > will pave way for 0.22.0 release. (This was done for the 0.21.0 release as > well: see MAPREDUCE-2014.) We can roll a 0.22.1 release with the > task-controller when it is fixed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2801) Include the native libs in java.library.path
[ https://issues.apache.org/jira/browse/MAPREDUCE-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098388#comment-13098388 ] Robert Joseph Evans commented on MAPREDUCE-2801: In the 0.20.20X code the tasktracker via the TaskRunner.java added in its own java.library.path to the options passed down to the child. Under YARN this will not work. It is the application master that determines the command line that is used to launch the container not the nodemanager. So if the application master is running on a system with a different java.library.path on it then the system the container is running on then it will not work (Think Hadoop is installed in a different location on different machines). Also the application master does not have the java.library.path set because the client did not set it when launching. I think the best way to solve this is to add in code to the nodemanager that would do a replace a @hadoop-java.library.path@ with the nodemanager's java.library.path similar to how the application master will replace @taskid@ with the task id for the application. Any thoughts on this? > Include the native libs in java.library.path > - > > Key: MAPREDUCE-2801 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2801 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Robert Joseph Evans > Fix For: 0.23.0 > > > For the child tasks in mrv2, java.library.path is set to just $PWD and the > native libs are not included. Whereas in 0.20.x, java.library.path for child > tasks was set to :$PWD -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2687) Non superusers unable to launch apps in both secure and non-secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2687: - Status: Patch Available (was: Open) Looks like there is an issue with my secure env caused by random timing issues. This patch seems to work. > Non superusers unable to launch apps in both secure and non-secure cluster > -- > > Key: MAPREDUCE-2687 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Mahadev konar >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch > > > Apps of non superuser fail to succeed in both secure and non-secure > environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) > is able to launch apps successfully. However, when a normal user submits a > job, the job fails. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2687) Non superusers unable to launch apps in both secure and non-secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2687: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thanks Mahadev. > Non superusers unable to launch apps in both secure and non-secure cluster > -- > > Key: MAPREDUCE-2687 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Mahadev konar >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch > > > Apps of non superuser fail to succeed in both secure and non-secure > environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) > is able to launch apps successfully. However, when a normal user submits a > job, the job fails. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2767) Remove Linux task-controller from 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098397#comment-13098397 ] Arun C Murthy commented on MAPREDUCE-2767: -- Konstantin - can you pls merge the trunk commit to branch-0.23 too? Thanks. > Remove Linux task-controller from 0.22 branch > - > > Key: MAPREDUCE-2767 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2767 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.22.0, 0.23.0, 0.24.0 >Reporter: Milind Bhandarkar >Assignee: Milind Bhandarkar >Priority: Blocker > Fix For: 0.22.0, 0.23.0, 0.24.0 > > Attachments: MR2767-22.patch, MR2767-23.patch, MR2767-trunk.patch, > TEST-org.apache.hadoop.mapred.TestMiniMRChildTask.txt, testlog.txt > > > There's a potential security hole in the task-controller as it stands. Based > on the discussion on general@, removing task-controller from the 0.22 branch > will pave way for 0.22.0 release. (This was done for the 0.21.0 release as > well: see MAPREDUCE-2014.) We can roll a 0.22.1 release with the > task-controller when it is fixed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2687) Non superusers unable to launch apps in both secure and non-secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098399#comment-13098399 ] Hudson commented on MAPREDUCE-2687: --- Integrated in Hadoop-Common-trunk-Commit #838 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/838/]) MAPREDUCE-2687. Fix NodeManager to use the right version of LocalDirAllocator.getLocalPathToWrite. Contributed by mahadev & acmurthy. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165890 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java > Non superusers unable to launch apps in both secure and non-secure cluster > -- > > Key: MAPREDUCE-2687 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Mahadev konar >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch > > > Apps of non superuser fail to succeed in both secure and non-secure > environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) > is able to launch apps successfully. However, when a normal user submits a > job, the job fails. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2687) Non superusers unable to launch apps in both secure and non-secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098401#comment-13098401 ] Hudson commented on MAPREDUCE-2687: --- Integrated in Hadoop-Hdfs-trunk-Commit #915 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/915/]) MAPREDUCE-2687. Fix NodeManager to use the right version of LocalDirAllocator.getLocalPathToWrite. Contributed by mahadev & acmurthy. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165890 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java > Non superusers unable to launch apps in both secure and non-secure cluster > -- > > Key: MAPREDUCE-2687 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Mahadev konar >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch > > > Apps of non superuser fail to succeed in both secure and non-secure > environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) > is able to launch apps successfully. However, when a normal user submits a > job, the job fails. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2687) Non superusers unable to launch apps in both secure and non-secure cluster
[ https://issues.apache.org/jira/browse/MAPREDUCE-2687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098402#comment-13098402 ] Hudson commented on MAPREDUCE-2687: --- Integrated in Hadoop-Mapreduce-trunk-Commit #849 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/849/]) MAPREDUCE-2687. Fix NodeManager to use the right version of LocalDirAllocator.getLocalPathToWrite. Contributed by mahadev & acmurthy. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165890 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/launcher/ContainerLaunch.java > Non superusers unable to launch apps in both secure and non-secure cluster > -- > > Key: MAPREDUCE-2687 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2687 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Mahadev konar >Priority: Blocker > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2687.patch, MAREDUCE-2687.patch > > > Apps of non superuser fail to succeed in both secure and non-secure > environment. Only the superuser(i.e. one who started/owns the mrv2 cluster) > is able to launch apps successfully. However, when a normal user submits a > job, the job fails. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2936) Contrib Raid compilation broken after HDFS-1620
[ https://issues.apache.org/jira/browse/MAPREDUCE-2936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mahadev konar updated MAPREDUCE-2936: - Fix Version/s: (was: 0.23.0) 0.24.0 I just reverted this from 0.23 branch. Thanks Harsh and Bobby for pointing it out! > Contrib Raid compilation broken after HDFS-1620 > --- > > Key: MAPREDUCE-2936 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2936 > Project: Hadoop Map/Reduce > Issue Type: Bug >Reporter: Vinod Kumar Vavilapalli >Assignee: Vinod Kumar Vavilapalli > Fix For: 0.24.0 > > Attachments: MAPREDUCE-2936-20110906.txt > > > After working around MAPREDUCE-2935 by removing TestServiceLevelAuthorization > and runing the following: > At the trunk level: mvn clean install package -Dtar -Pdist > -Dmaven.test.skip.exec=true > In hadoop-mapreduce-project: ant compile-contrib -Dresolvers=internal > yields 14 errors. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2767) Remove Linux task-controller from 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098410#comment-13098410 ] Hudson commented on MAPREDUCE-2767: --- Integrated in Hadoop-Common-trunk-Commit #839 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/839/]) MAPREDUCE-2767. CHANGES.txt shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165894 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt > Remove Linux task-controller from 0.22 branch > - > > Key: MAPREDUCE-2767 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2767 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.22.0, 0.23.0, 0.24.0 >Reporter: Milind Bhandarkar >Assignee: Milind Bhandarkar >Priority: Blocker > Fix For: 0.22.0, 0.23.0, 0.24.0 > > Attachments: MR2767-22.patch, MR2767-23.patch, MR2767-trunk.patch, > TEST-org.apache.hadoop.mapred.TestMiniMRChildTask.txt, testlog.txt > > > There's a potential security hole in the task-controller as it stands. Based > on the discussion on general@, removing task-controller from the 0.22 branch > will pave way for 0.22.0 release. (This was done for the 0.21.0 release as > well: see MAPREDUCE-2014.) We can roll a 0.22.1 release with the > task-controller when it is fixed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2767) Remove Linux task-controller from 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098411#comment-13098411 ] Hudson commented on MAPREDUCE-2767: --- Integrated in Hadoop-Hdfs-trunk-Commit #916 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/916/]) MAPREDUCE-2767. CHANGES.txt shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165894 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt > Remove Linux task-controller from 0.22 branch > - > > Key: MAPREDUCE-2767 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2767 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.22.0, 0.23.0, 0.24.0 >Reporter: Milind Bhandarkar >Assignee: Milind Bhandarkar >Priority: Blocker > Fix For: 0.22.0, 0.23.0, 0.24.0 > > Attachments: MR2767-22.patch, MR2767-23.patch, MR2767-trunk.patch, > TEST-org.apache.hadoop.mapred.TestMiniMRChildTask.txt, testlog.txt > > > There's a potential security hole in the task-controller as it stands. Based > on the discussion on general@, removing task-controller from the 0.22 branch > will pave way for 0.22.0 release. (This was done for the 0.21.0 release as > well: see MAPREDUCE-2014.) We can roll a 0.22.1 release with the > task-controller when it is fixed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2767) Remove Linux task-controller from 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098412#comment-13098412 ] Hudson commented on MAPREDUCE-2767: --- Integrated in Hadoop-Mapreduce-trunk-Commit #850 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/850/]) MAPREDUCE-2767. CHANGES.txt shv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165894 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt > Remove Linux task-controller from 0.22 branch > - > > Key: MAPREDUCE-2767 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2767 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.22.0, 0.23.0, 0.24.0 >Reporter: Milind Bhandarkar >Assignee: Milind Bhandarkar >Priority: Blocker > Fix For: 0.22.0, 0.23.0, 0.24.0 > > Attachments: MR2767-22.patch, MR2767-23.patch, MR2767-trunk.patch, > TEST-org.apache.hadoop.mapred.TestMiniMRChildTask.txt, testlog.txt > > > There's a potential security hole in the task-controller as it stands. Based > on the discussion on general@, removing task-controller from the 0.22 branch > will pave way for 0.22.0 release. (This was done for the 0.21.0 release as > well: see MAPREDUCE-2014.) We can roll a 0.22.1 release with the > task-controller when it is fixed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2767) Remove Linux task-controller from 0.22 branch
[ https://issues.apache.org/jira/browse/MAPREDUCE-2767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Shvachko updated MAPREDUCE-2767: --- Resolution: Fixed Hadoop Flags: [Reviewed] Status: Resolved (was: Patch Available) I just committed this. Thank you Milind. > Remove Linux task-controller from 0.22 branch > - > > Key: MAPREDUCE-2767 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2767 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: security >Affects Versions: 0.22.0, 0.23.0, 0.24.0 >Reporter: Milind Bhandarkar >Assignee: Milind Bhandarkar >Priority: Blocker > Fix For: 0.22.0, 0.23.0, 0.24.0 > > Attachments: MR2767-22.patch, MR2767-23.patch, MR2767-trunk.patch, > TEST-org.apache.hadoop.mapred.TestMiniMRChildTask.txt, testlog.txt > > > There's a potential security hole in the task-controller as it stands. Based > on the discussion on general@, removing task-controller from the 0.22 branch > will pave way for 0.22.0 release. (This was done for the 0.21.0 release as > well: see MAPREDUCE-2014.) We can roll a 0.22.1 release with the > task-controller when it is fixed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2917) Corner case in container reservations
[ https://issues.apache.org/jira/browse/MAPREDUCE-2917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2917: - Resolution: Fixed Status: Resolved (was: Patch Available) Forgot to resolve the issue. > Corner case in container reservations > - > > Key: MAPREDUCE-2917 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2917 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, resourcemanager >Affects Versions: 0.23.0 >Reporter: Arun C Murthy >Assignee: Arun C Murthy > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2917.patch, MAPREDUCE-2917.patch, > MAPREDUCE-2917.patch > > > Saw a corner case in container reservations where the node on which the AM is > running was reserved, and hence never fulfilled leaving the application > hanging. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2922) create-c++-task-controller-configure called during ant build create-c++-configure and fails
[ https://issues.apache.org/jira/browse/MAPREDUCE-2922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098449#comment-13098449 ] Arun C Murthy commented on MAPREDUCE-2922: -- Thomas, can we drop this since MAPREDUCE-2767 went it? If so, please go ahead and close this as 'duplicate'. Thanks. > create-c++-task-controller-configure called during ant build > create-c++-configure and fails > > > Key: MAPREDUCE-2922 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2922 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves >Priority: Minor > Fix For: 0.23.0 > > Attachments: configure.patch > > > if you call ant create-c++-configure it tries to run > create-c++-task-controller-configure, which fails with error about >>> > [exec] Makefile.am: C objects in subdir but `AM_PROG_CC_C_O'. We don't need > to build the task-controller anymore in mrv2 so simply remove it from > build.xml > cmdline used to build: > ant -Dresolvers=internal -Dcompile.c++=true -Dcompile.native=true > create-c++-configure binary -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2926) 500 Error in ResourceManager UI
[
https://issues.apache.org/jira/browse/MAPREDUCE-2926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mahadev konar resolved MAPREDUCE-2926.
--
Resolution: Duplicate
I just verified. I dont see the issue anymore. Marking it as Duplicate of
HADOOP-7606.
> 500 Error in ResourceManager UI
> ---
>
> Key: MAPREDUCE-2926
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2926
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0, 0.24.0
>Reporter: Robert Joseph Evans
>Assignee: Robert Joseph Evans
> Fix For: 0.23.0, 0.24.0
>
> Attachments: MR-2926-v1.patch, MR-2926-v2.patch
>
>
> When accessing the resource manager UI the following is returned
> {noformat}
> Problem accessing /. Reason:
> org.codehaus.jackson.type.JavaType.(Ljava/lang/Class;)V
> Caused by:
> java.lang.NoSuchMethodError:
> org.codehaus.jackson.type.JavaType.(Ljava/lang/Class;)V
> at org.codehaus.jackson.map.type.TypeBase.(TypeBase.java:15)
> at org.codehaus.jackson.map.type.SimpleType.(SimpleType.java:45)
> at org.codehaus.jackson.map.type.SimpleType.(SimpleType.java:40)
> at
> org.codehaus.jackson.map.type.TypeBindings.(TypeBindings.java:20)
> at
> org.codehaus.jackson.map.type.TypeFactory._fromType(TypeFactory.java:530)
> at org.codehaus.jackson.map.type.TypeFactory.type(TypeFactory.java:63)
> at org.codehaus.jackson.map.ObjectMapper.(ObjectMapper.java:179)
> at org.apache.hadoop.yarn.webapp.Controller.(Controller.java:43)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> at
> com.google.inject.DefaultConstructionProxyFactory$2.newInstance(DefaultConstructionProxyFactory.java:81)
> at
> com.google.inject.ConstructorInjector.construct(ConstructorInjector.java:85)
> at
> com.google.inject.ConstructorBindingImpl$Factory.get(ConstructorBindingImpl.java:111)
> at com.google.inject.InjectorImpl$4$1.call(InjectorImpl.java:758)
> at com.google.inject.InjectorImpl.callInContext(InjectorImpl.java:804)
> at com.google.inject.InjectorImpl$4.get(InjectorImpl.java:754)
> at com.google.inject.InjectorImpl.getInstance(InjectorImpl.java:793)
> at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:136)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:216)
> at
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:141)
> at
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:93)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:63)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:122)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:110)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:892)
> at
> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
> at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
> at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
> at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
> at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> at org.mortbay.jetty.Server.handle(Server.java:326)
> at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
> at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
> at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
> at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
> at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
> at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
> at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedT
[jira] [Updated] (MAPREDUCE-2864) Renaming of configuration property names in yarn
[ https://issues.apache.org/jira/browse/MAPREDUCE-2864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2864: - Status: Open (was: Patch Available) Robert, sorry for coming late into this - the patch has gone stale. :( I'm concerned we are proliferating too many things here: YarnConfiguration and MRJobConfig are sufficient I think - trying to differentiate YarnConfig into Admin/User/Internal etc. is an overkill IMO. I'm ok with YarnConfiguration, but maybe we can do with YarnServerConfig and YarnClientConfig? Similarly, we can just do with MRJobConfig for now - MR is going to be a Yarn client, thus we can just stick all configs there? > Renaming of configuration property names in yarn > > > Key: MAPREDUCE-2864 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2864 > Project: Hadoop Map/Reduce > Issue Type: Improvement > Components: jobhistoryserver, mrv2, nodemanager, resourcemanager >Affects Versions: 0.23.0, 0.24.0 >Reporter: Robert Joseph Evans >Assignee: Robert Joseph Evans > Fix For: 0.23.0, 0.24.0 > > Attachments: MR-2864-v1.patch, MR-2864-v2.patch, MR-2864-v3.patch, > MR-2864-v4.patch, update.pl, yarnConfigs.csv > > > Now that YARN has been put in to trunk we should do something similar to > MAPREDUCE-849. We should go back and look at all of the configurations that > have been added in and rename them as needed to be consistent and subdivided > by component. > # We should use all lowercase in the config names. e.g., we should use > appsmanager instead of appsManager etc. > # history server config names should be prefixed with mapreduce instead of > yarn. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2800) clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files
[ https://issues.apache.org/jira/browse/MAPREDUCE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2800: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thanks Sid! > clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files > - > > Key: MAPREDUCE-2800 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2800 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Siddharth Seth > Fix For: 0.23.0, 0.24.0 > > Attachments: MR2800.patch, MR2800_1.patch, MR2800_2.patch > > > clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 for all the > map tasks for the last 4 progress interval in the jobhistory files. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2774) [MR-279] Add a startup msg while starting RM/NM
[ https://issues.apache.org/jira/browse/MAPREDUCE-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2774: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thanks Venu! > [MR-279] Add a startup msg while starting RM/NM > --- > > Key: MAPREDUCE-2774 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2774 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Venu Gopala Rao >Priority: Minor > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2774.patch > > > Add a startup msg while starting NM/RM indicating the version, build details > etc. This will help in easier parsing of logs and debugging. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2774) [MR-279] Add a startup msg while starting RM/NM
[ https://issues.apache.org/jira/browse/MAPREDUCE-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098480#comment-13098480 ] Hudson commented on MAPREDUCE-2774: --- Integrated in Hadoop-Common-trunk-Commit #840 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/840/]) MAPREDUCE-2774. Add startup message to ResourceManager & NodeManager on startup. Contributed by Venu Gopala Rao. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165933 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java > [MR-279] Add a startup msg while starting RM/NM > --- > > Key: MAPREDUCE-2774 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2774 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Venu Gopala Rao >Priority: Minor > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2774.patch > > > Add a startup msg while starting NM/RM indicating the version, build details > etc. This will help in easier parsing of logs and debugging. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2800) clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files
[ https://issues.apache.org/jira/browse/MAPREDUCE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098479#comment-13098479 ] Hudson commented on MAPREDUCE-2800: --- Integrated in Hadoop-Common-trunk-Commit #840 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/840/]) MAPREDUCE-2800. Set final progress for tasks to ensure all task information is correctly logged to JobHistory. Contributed by Siddharth Seth. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165930 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java > clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files > - > > Key: MAPREDUCE-2800 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2800 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Siddharth Seth > Fix For: 0.23.0, 0.24.0 > > Attachments: MR2800.patch, MR2800_1.patch, MR2800_2.patch > > > clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 for all the > map tasks for the last 4 progress interval in the jobhistory files. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2774) [MR-279] Add a startup msg while starting RM/NM
[ https://issues.apache.org/jira/browse/MAPREDUCE-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098483#comment-13098483 ] Hudson commented on MAPREDUCE-2774: --- Integrated in Hadoop-Hdfs-trunk-Commit #917 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/917/]) MAPREDUCE-2774. Add startup message to ResourceManager & NodeManager on startup. Contributed by Venu Gopala Rao. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165933 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java > [MR-279] Add a startup msg while starting RM/NM > --- > > Key: MAPREDUCE-2774 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2774 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Venu Gopala Rao >Priority: Minor > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2774.patch > > > Add a startup msg while starting NM/RM indicating the version, build details > etc. This will help in easier parsing of logs and debugging. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2800) clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files
[ https://issues.apache.org/jira/browse/MAPREDUCE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098482#comment-13098482 ] Hudson commented on MAPREDUCE-2800: --- Integrated in Hadoop-Hdfs-trunk-Commit #917 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/917/]) MAPREDUCE-2800. Set final progress for tasks to ensure all task information is correctly logged to JobHistory. Contributed by Siddharth Seth. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165930 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java > clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files > - > > Key: MAPREDUCE-2800 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2800 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Siddharth Seth > Fix For: 0.23.0, 0.24.0 > > Attachments: MR2800.patch, MR2800_1.patch, MR2800_2.patch > > > clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 for all the > map tasks for the last 4 progress interval in the jobhistory files. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2800) clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files
[ https://issues.apache.org/jira/browse/MAPREDUCE-2800?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098484#comment-13098484 ] Hudson commented on MAPREDUCE-2800: --- Integrated in Hadoop-Mapreduce-trunk-Commit #851 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/851/]) MAPREDUCE-2800. Set final progress for tasks to ensure all task information is correctly logged to JobHistory. Contributed by Siddharth Seth. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165930 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-app/src/main/java/org/apache/hadoop/mapreduce/v2/app/job/impl/TaskAttemptImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapred/MapTask.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/src/test/java/org/apache/hadoop/mapreduce/v2/TestMRJobs.java > clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files > - > > Key: MAPREDUCE-2800 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2800 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Siddharth Seth > Fix For: 0.23.0, 0.24.0 > > Attachments: MR2800.patch, MR2800_1.patch, MR2800_2.patch > > > clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 for all the > map tasks for the last 4 progress interval in the jobhistory files. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2774) [MR-279] Add a startup msg while starting RM/NM
[ https://issues.apache.org/jira/browse/MAPREDUCE-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098485#comment-13098485 ] Hudson commented on MAPREDUCE-2774: --- Integrated in Hadoop-Mapreduce-trunk-Commit #851 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/851/]) MAPREDUCE-2774. Add startup message to ResourceManager & NodeManager on startup. Contributed by Venu Gopala Rao. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165933 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ResourceManager.java > [MR-279] Add a startup msg while starting RM/NM > --- > > Key: MAPREDUCE-2774 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2774 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Ramya Sunil >Assignee: Venu Gopala Rao >Priority: Minor > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2774.patch > > > Add a startup msg while starting NM/RM indicating the version, build details > etc. This will help in easier parsing of logs and debugging. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2836) Provide option to fail jobs when submitted to non-existent pools.
[
https://issues.apache.org/jira/browse/MAPREDUCE-2836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098489#comment-13098489
]
Todd Lipcon commented on MAPREDUCE-2836:
Hi Ahmed,
I took another detailed look at this and some potential issues.
- In jobAdded(), it constructs a new JobInfo and puts it in the {{infos}} map
before it calls {{poolMgr.addJob}}. So, in the case that an exception is
thrown, the job will end up orphaned in the {{infos}} map.
- this implementation has the bug that, if the allocations file is reloaded
with one of the pools removed, users will still be allowed to submit jobs to
it. That doesn't seem quite right.
- in JobTracker.addJob, it looks like it just catches and warns if the jobAdded
notification fails. It doesn't actually reject the job submission. I think your
test doesn't catch this since it's using a fake JT and not the real
implementation.
- Maybe you can add a functional test that uses MiniMRCluster to see the above
bug?
One minor thing I noticed: we don't usually use camelCase in the configs.
Perhaps {{mapred.fairscheduler.allow.undeclared.pools}} would be more
consistent.
I think I agree with Matei's earlier comment that it would be better to add a
hook in the scheduler API like "checkJobSubmission(JobInProgress job)" that
could be called after aclsManager.checkAccess and checkQueueValidity in
submitJob. Then, to fix the other bug with pool reloading, add a new
{{{Set validPools}} which is loaded in {{reloadAllocs}}. At submission
time you can simply check if the pool exists in that list. Then getPool() etc
wouldn't need to be changed at all. Does that sound reasonable?
> Provide option to fail jobs when submitted to non-existent pools.
> -
>
> Key: MAPREDUCE-2836
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2836
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: contrib/fair-share
>Reporter: Jeff Bean
>Assignee: Ahmed Radwan
>Priority: Minor
> Attachments: MAPREDUCE-2836.patch, MAPREDUCE-2836_rev2.patch,
> MAPREDUCE-2836_rev3.patch, MAPREDUCE-2836_rev4.patch
>
>
> In some environments, it might be desirable to explicitly specify the fair
> scheduler pools and to explicitly fail jobs that are not submitted to any of
> the pools.
> Current behavior of the fair scheduler is to submit jobs to a default pool if
> a pool name isn't specified or to create a pool with the new name if the pool
> name doesn't already exist. There should be a configuration option for the
> fair scheduler that causes it to noisily fail the job if it's submitted to a
> pool that isn't pre-specified or if the specified pool doesn't exist.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2882) TestLineRecordReader depends on ant jars
[ https://issues.apache.org/jira/browse/MAPREDUCE-2882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon resolved MAPREDUCE-2882. Resolution: Fixed Hadoop Flags: [Reviewed] > TestLineRecordReader depends on ant jars > > > Key: MAPREDUCE-2882 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2882 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.23.0 >Reporter: Todd Lipcon >Assignee: Todd Lipcon >Priority: Minor > Fix For: 0.23.0 > > Attachments: mr-2882.txt > > > This test is currently importing an ant utility class to read a file - this > dependency doesn't work in mavenized land. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2934) MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process
[
https://issues.apache.org/jira/browse/MAPREDUCE-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098493#comment-13098493
]
Todd Lipcon commented on MAPREDUCE-2934:
+1 once HADOOP-7607 is committed
> MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process
> --
>
> Key: MAPREDUCE-2934
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2934
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
>Affects Versions: 0.24.0
>Reporter: Aaron T. Myers
>Assignee: Aaron T. Myers
> Fix For: 0.24.0
>
> Attachments: mapreduce-2934.0.patch
>
>
> Once HADOOP-7607 goes in, {{ProtoOverHadoopRpcEngine.stopProxy}} will need to
> be removed or at least have its {{@Override}} annotation removed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2934) MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process
[
https://issues.apache.org/jira/browse/MAPREDUCE-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aaron T. Myers updated MAPREDUCE-2934:
--
Status: Patch Available (was: Open)
> MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process
> --
>
> Key: MAPREDUCE-2934
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2934
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
>Affects Versions: 0.24.0
>Reporter: Aaron T. Myers
>Assignee: Aaron T. Myers
> Fix For: 0.24.0
>
> Attachments: mapreduce-2934.0.patch
>
>
> Once HADOOP-7607 goes in, {{ProtoOverHadoopRpcEngine.stopProxy}} will need to
> be removed or at least have its {{@Override}} annotation removed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2934) MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process
[
https://issues.apache.org/jira/browse/MAPREDUCE-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098510#comment-13098510
]
Hadoop QA commented on MAPREDUCE-2934:
--
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12493099/mapreduce-2934.0.patch
against trunk revision .
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified
tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
+1 javadoc. The javadoc tool did not generate any warning messages.
-1 javac. The patch appears to cause tar ant target to fail.
-1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.
+1 release audit. The applied patch does not increase the total number of
release audit warnings.
-1 core tests. The patch failed these unit tests:
+1 contrib tests. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/618//testReport/
Console output:
https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/618//console
This message is automatically generated.
> MR portion of HADOOP-7607 - Simplify the RPC proxy cleanup process
> --
>
> Key: MAPREDUCE-2934
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2934
> Project: Hadoop Map/Reduce
> Issue Type: Improvement
> Components: mrv2
>Affects Versions: 0.24.0
>Reporter: Aaron T. Myers
>Assignee: Aaron T. Myers
> Fix For: 0.24.0
>
> Attachments: mapreduce-2934.0.patch
>
>
> Once HADOOP-7607 goes in, {{ProtoOverHadoopRpcEngine.stopProxy}} will need to
> be removed or at least have its {{@Override}} annotation removed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2655) MR279: Audit logs for YARN
[ https://issues.apache.org/jira/browse/MAPREDUCE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2655: - Attachment: MAPREDUCE-2655-v4.patch Minor update to merge with some recent changes. Patch looks good. +1 > MR279: Audit logs for YARN > --- > > Key: MAPREDUCE-2655 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2655 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2655-v2.patch, MAPREDUCE-2655-v3.patch, > MAPREDUCE-2655-v4.patch, MAPREDUCE-2655.patch, nm-audit.log, rm-audit.log > > > We need audit logs for YARN components: > ResourceManager: > - All the refresh* protocol access points - refreshQueues, refreshNodes, > refreshProxyUsers, > refreshUserToGroupMappings. > - All app-submissions, app-kills to RM. > - Illegal and successful(?) AM registrations. > - Illegal container allocations/deallocations from AMs > - Successful container allocations/deallocations from AMs too? > NodeManager: > - Illegal container launches from AMs > - Successful container launches from AMs too? > Not sure if we need audit logs from MR AMs. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2655) MR279: Audit logs for YARN
[ https://issues.apache.org/jira/browse/MAPREDUCE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2655: - Resolution: Fixed Status: Resolved (was: Patch Available) I just committed this. Thanks Thomas! > MR279: Audit logs for YARN > --- > > Key: MAPREDUCE-2655 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2655 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2655-v2.patch, MAPREDUCE-2655-v3.patch, > MAPREDUCE-2655-v4.patch, MAPREDUCE-2655.patch, nm-audit.log, rm-audit.log > > > We need audit logs for YARN components: > ResourceManager: > - All the refresh* protocol access points - refreshQueues, refreshNodes, > refreshProxyUsers, > refreshUserToGroupMappings. > - All app-submissions, app-kills to RM. > - Illegal and successful(?) AM registrations. > - Illegal container allocations/deallocations from AMs > - Successful container allocations/deallocations from AMs too? > NodeManager: > - Illegal container launches from AMs > - Successful container launches from AMs too? > Not sure if we need audit logs from MR AMs. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2655) MR279: Audit logs for YARN
[ https://issues.apache.org/jira/browse/MAPREDUCE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098524#comment-13098524 ] Hudson commented on MAPREDUCE-2655: --- Integrated in Hadoop-Common-trunk-Commit #841 (See [https://builds.apache.org/job/Hadoop-Common-trunk-Commit/841/]) Adding new files for MAPREDUCE-2655. MAPREDUCE-2655. Add audit logs to ResourceManager and NodeManager. Contributed by Thomas Graves. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165950 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NMAuditLogger.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNMAuditLogger.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAuditLogger.java acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165949 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/MockRMApp.java > MR279: Audit logs for YARN > --- > > Key: MAPREDUCE-2655 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2655 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2655-v2.patch, MAPREDUCE-2655-v3.patch, > MAPREDUCE-2655-v4.patch, MAPREDUCE-2655.patch, nm-audit.log, rm-audit.log > > > We need audit logs for YARN components: > ResourceManager: > - All the refresh* protocol access points - refreshQueues, refreshNodes, > refreshProxyUsers, > refreshUserToGroupMappings. > - All app-submissions, app-kills to RM. > - Illegal and successful(?) AM registrations. > - Illegal container allocations/deallocations from AMs > - Successful container allocations/deallocations from AMs too? > NodeManager: > - Illegal container launches from AMs > - Successful container launches from AMs too? > Not sure if we need audit logs from MR AMs. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2655) MR279: Audit logs for YARN
[ https://issues.apache.org/jira/browse/MAPREDUCE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098528#comment-13098528 ] Hudson commented on MAPREDUCE-2655: --- Integrated in Hadoop-Mapreduce-trunk-Commit #852 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/852/]) Adding new files for MAPREDUCE-2655. MAPREDUCE-2655. Add audit logs to ResourceManager and NodeManager. Contributed by Thomas Graves. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165950 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NMAuditLogger.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNMAuditLogger.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAuditLogger.java acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165949 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/MockRMApp.java > MR279: Audit logs for YARN > --- > > Key: MAPREDUCE-2655 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2655 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2655-v2.patch, MAPREDUCE-2655-v3.patch, > MAPREDUCE-2655-v4.patch, MAPREDUCE-2655.patch, nm-audit.log, rm-audit.log > > > We need audit logs for YARN components: > ResourceManager: > - All the refresh* protocol access points - refreshQueues, refreshNodes, > refreshProxyUsers, > refreshUserToGroupMappings. > - All app-submissions, app-kills to RM. > - Illegal and successful(?) AM registrations. > - Illegal container allocations/deallocations from AMs > - Successful container allocations/deallocations from AMs too? > NodeManager: > - Illegal container launches from AMs > - Successful container launches from AMs too? > Not sure if we need audit logs from MR AMs. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2655) MR279: Audit logs for YARN
[ https://issues.apache.org/jira/browse/MAPREDUCE-2655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098529#comment-13098529 ] Hudson commented on MAPREDUCE-2655: --- Integrated in Hadoop-Hdfs-trunk-Commit #918 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/918/]) Adding new files for MAPREDUCE-2655. MAPREDUCE-2655. Add audit logs to ResourceManager and NodeManager. Contributed by Thomas Graves. acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165950 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NMAuditLogger.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/TestNMAuditLogger.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAuditLogger.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestRMAuditLogger.java acmurthy : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1165949 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/ContainerManagerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/ContainerImpl.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/AdminService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ApplicationMasterService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/ClientRMService.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/RMAppManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/SchedulerApp.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/capacity/CapacityScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fifo/FifoScheduler.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/rmapp/MockRMApp.java > MR279: Audit logs for YARN > --- > > Key: MAPREDUCE-2655 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2655 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Thomas Graves >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2655-v2.patch, MAPREDUCE-2655-v3.patch, > MAPREDUCE-2655-v4.patch, MAPREDUCE-2655.patch, nm-audit.log, rm-audit.log > > > We need audit logs for YARN components: > ResourceManager: > - All the refresh* protocol access points - refreshQueues, refreshNodes, > refreshProxyUsers, > refreshUserToGroupMappings. > - All app-submissions, app-kills to RM. > - Illegal and successful(?) AM registrations. > - Illegal container allocations/deallocations from AMs > - Successful container allocations/deallocations from AMs too? > NodeManager: > - Illegal container launches from AMs > - Successful container launches from AMs too? > Not sure if we need audit logs from MR AMs. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2170) Send out last-minute load averages in TaskTrackerStatus
[ https://issues.apache.org/jira/browse/MAPREDUCE-2170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098572#comment-13098572 ] Harsh J commented on MAPREDUCE-2170: If there's still interest for this, I can update it up for the hadoop-yarn framework (ContainersMonitor). > Send out last-minute load averages in TaskTrackerStatus > --- > > Key: MAPREDUCE-2170 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2170 > Project: Hadoop Map/Reduce > Issue Type: New Feature > Components: jobtracker >Affects Versions: 0.22.0 >Reporter: Harsh J >Assignee: Harsh J >Priority: Critical > Fix For: 0.23.0 > > Attachments: mapreduce.loadaverage.r3.diff, > mapreduce.loadaverage.r4.diff, mapreduce.loadaverage.r5.diff, > mapreduce.loadaverage.r6.diff > > Original Estimate: 20m > Remaining Estimate: 20m > > Load averages could be useful in scheduling. This patch looks to extend the > existing Linux resource plugin (via /proc/loadavg file) to allow transmitting > load averages of the last one minute via the TaskTrackerStatus. > Patch is up for review, with test cases added, at: > https://reviews.apache.org/r/20/ -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2932) Missing instrumentation plugin class shouldn't crash the TT startup per design
[
https://issues.apache.org/jira/browse/MAPREDUCE-2932?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098576#comment-13098576
]
Harsh J commented on MAPREDUCE-2932:
I s'pose NodeManagerMetrics is the newer equivalent of the older, pluggable
instrumentation of TT. Since NodeManagerMetrics is not an interface, and nor
does the framework allow things to just be plugged in (to merely override the
metrics class) for this, I do not think this problem exists MR2 (0.23/trunk).
> Missing instrumentation plugin class shouldn't crash the TT startup per design
> --
>
> Key: MAPREDUCE-2932
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2932
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
>Affects Versions: 0.20.2
>Reporter: Harsh J
>Assignee: Harsh J
>Priority: Trivial
> Attachments: 0.20-MAPREDUCE-2932.r1.diff
>
>
> Per the implementation of the TaskTracker instrumentation plugin
> implementation (from 2008), a ClassNotFoundException during loading up of an
> configured TaskTracker instrumentation class shouldn't have hampered TT start
> up at all.
> But, there is one class-fetching call outside try/catch, which makes TT fall
> down with a RuntimeException if there's a class not found. Would be good to
> include this line into the try/catch itself.
> Strace would appear as:
> {code}
> 2011-08-25 11:45:38,470 ERROR org.apache.hadoop.mapred.TaskTracker: Can not
> start task tracker because java.lang.RuntimeException:
> java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapred.CustomInstPlugin
> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java)
> at
> org.apache.hadoop.mapred.TaskTracker.getInstrumentationClass(TaskTracker.java)
>
> at org.apache.hadoop.mapred.TaskTracker.initialize(TaskTracker.java)
> {code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2937) Errors in Application failures are not shown in the client trace.
Errors in Application failures are not shown in the client trace. - Key: MAPREDUCE-2937 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2937 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Mahadev konar Priority: Critical Fix For: 0.23.0, 0.24.0 The client side does not show enough information on why the job failed. Here is step to reproduce it: 1) set the scheduler to be capacity scheduler with queues a, b 2) submit a job to a queue that is not a,b The job just fails without saying why it failed. We should have enough trace log at the client side to let the user know why it failed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2937) Errors in Application failures are not shown in the client trace.
[
https://issues.apache.org/jira/browse/MAPREDUCE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098578#comment-13098578
]
Mahadev konar commented on MAPREDUCE-2937:
--
Here is the stack trace I get:
{code}
11/09/07 01:04:42 INFO mapreduce.JobSubmitter: Cleaning up the staging area
.staging/job_1315351928719_0007
RemoteTrace: at Local Trace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl: failed to
run job
at
org.apache.hadoop.yarn.factories.impl.pb.YarnRemoteExceptionFactoryPBImpl.createYarnRemoteException(YarnRemoteExceptionFactoryPBImpl.java:39)
at
org.apache.hadoop.yarn.ipc.RPCUtil.getRemoteException(RPCUtil.java:47)
at org.apache.hadoop.mapred.YARNRunner.submitJob(YARNRunner.java:250)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:377)
at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1072)
at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1069)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1135)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1069)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1089)
at org.apache.hadoop.examples.RandomWriter.run(RandomWriter.java:283)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.examples.RandomWriter.main(RandomWriter.java:294)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:189)
{code}
> Errors in Application failures are not shown in the client trace.
> -
>
> Key: MAPREDUCE-2937
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-2937
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.0
>Reporter: Mahadev konar
>Priority: Critical
> Fix For: 0.23.0, 0.24.0
>
>
> The client side does not show enough information on why the job failed. Here
> is step to reproduce it:
> 1) set the scheduler to be capacity scheduler with queues a, b
> 2) submit a job to a queue that is not a,b
> The job just fails without saying why it failed. We should have enough trace
> log at the client side to let the user know why it failed.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2749) [MR-279] NM registers with RM even before it starts various servers
[ https://issues.apache.org/jira/browse/MAPREDUCE-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098577#comment-13098577 ] Vinod Kumar Vavilapalli commented on MAPREDUCE-2749: Perfect, that should work. +1 for your proposal. Thanks! > [MR-279] NM registers with RM even before it starts various servers > --- > > Key: MAPREDUCE-2749 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2749 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Vinod Kumar Vavilapalli >Assignee: Thomas Graves > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2749-v2.patch, MAPREDUCE-2749.patch > > > In case NM eventually fails to start the ContainerManager server because of > say a port clash, RM will have to wait for expiry to detect the NM crash. > It is desirable to make NM register with RM only after it can start all of > its components successfully. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2938) Missing log stmt for app submission fail CS
Missing log stmt for app submission fail CS --- Key: MAPREDUCE-2938 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2938 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, scheduler Affects Versions: 0.23.0 Reporter: Arun C Murthy Assignee: Arun C Murthy Priority: Trivial Fix For: 0.23.0 Missing log stmt for app submission fail CS -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2938) Missing log stmt for app submission fail CS
[ https://issues.apache.org/jira/browse/MAPREDUCE-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2938: - Attachment: MAPREDUCE-2938.patch Trivial patch > Missing log stmt for app submission fail CS > --- > > Key: MAPREDUCE-2938 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2938 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, scheduler >Affects Versions: 0.23.0 >Reporter: Arun C Murthy >Assignee: Arun C Murthy >Priority: Trivial > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2938.patch > > > Missing log stmt for app submission fail CS -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2938) Missing log stmt for app submission fail CS
[ https://issues.apache.org/jira/browse/MAPREDUCE-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2938: - Status: Patch Available (was: Open) > Missing log stmt for app submission fail CS > --- > > Key: MAPREDUCE-2938 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2938 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, scheduler >Affects Versions: 0.23.0 >Reporter: Arun C Murthy >Assignee: Arun C Murthy >Priority: Trivial > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2938.patch > > > Missing log stmt for app submission fail CS -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2939) Ant setup on hadoop7 jenkins host
Ant setup on hadoop7 jenkins host - Key: MAPREDUCE-2939 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2939 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 0.22.0 Environment: Jenkins: https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-22-branch/65/console Reporter: Joep Rottinghuis Fix For: 0.22.0 >From the build error it looks like a) ant is not set up on the machine b) $ANT_HOME point to the wrong spot -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2939) Ant setup on hadoop7 jenkins host
[ https://issues.apache.org/jira/browse/MAPREDUCE-2939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13098612#comment-13098612 ] Joep Rottinghuis commented on MAPREDUCE-2939: - Could somebody with access to the Jenkins host take a look and adjust $ANT_HOME or the ant installation there on host hadoop7? Here is a snippet from the build: /homes/hudson/tools/ant/latest/bin/ant -Dversion=2011-08-29_22-31-26 -Declipse.home=/homes/hudson/tools/eclipse/latest -Dfindbugs.home=/homes/hudson/tools/findbugs/latest -Dforrest.home=/homes/hudson/tools/forrest/latest -Dcompile.c++=true -Dcompile.native=true clean create-c++-configure tar findbugs nightly/hudsonBuildHadoopNightly.sh: 1: /homes/hudson/tools/ant/latest/bin/ant: not found > Ant setup on hadoop7 jenkins host > - > > Key: MAPREDUCE-2939 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2939 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: build >Affects Versions: 0.22.0 > Environment: Jenkins: > https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-22-branch/65/console >Reporter: Joep Rottinghuis > Fix For: 0.22.0 > > > From the build error it looks like > a) ant is not set up on the machine > b) $ANT_HOME point to the wrong spot -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2938) Missing log stmt for app submission fail CS
[ https://issues.apache.org/jira/browse/MAPREDUCE-2938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2938: - Resolution: Fixed Status: Resolved (was: Patch Available) I committed the trivial patch. > Missing log stmt for app submission fail CS > --- > > Key: MAPREDUCE-2938 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2938 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, scheduler >Affects Versions: 0.23.0 >Reporter: Arun C Murthy >Assignee: Arun C Murthy >Priority: Trivial > Fix For: 0.23.0 > > Attachments: MAPREDUCE-2938.patch > > > Missing log stmt for app submission fail CS -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2937) Errors in Application failures are not shown in the client trace.
[ https://issues.apache.org/jira/browse/MAPREDUCE-2937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated MAPREDUCE-2937: - Attachment: MAPREDUCE-2937.patch Trivial patch > Errors in Application failures are not shown in the client trace. > - > > Key: MAPREDUCE-2937 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2937 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.0 >Reporter: Mahadev konar >Priority: Critical > Fix For: 0.23.0, 0.24.0 > > Attachments: MAPREDUCE-2937.patch > > > The client side does not show enough information on why the job failed. Here > is step to reproduce it: > 1) set the scheduler to be capacity scheduler with queues a, b > 2) submit a job to a queue that is not a,b > The job just fails without saying why it failed. We should have enough trace > log at the client side to let the user know why it failed. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
