[jira] [Commented] (MAPREDUCE-4318) TestRecoveryManagershould not use raw and deprecated configuration parameters.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290847#comment-13290847 ] Benoy Antony commented on MAPREDUCE-4318: - The other option will be to define the new way fo defining the queue configuration. This is followed in some other tests. I'll try that and attach a new patch > TestRecoveryManagershould not use raw and deprecated configuration parameters. > -- > > Key: MAPREDUCE-4318 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4318 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.22.1 >Reporter: Konstantin Shvachko >Assignee: Benoy Antony > Attachments: MAPREDUCE-4318.patch > > > TestRecoveryManager should not use deprecated config keys, and should use > constants for the keys where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4318) TestRecoveryManagershould not use raw and deprecated configuration parameters.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Benoy Antony updated MAPREDUCE-4318: Attachment: MAPREDUCE-4318.patch Fixed the hardcoded keys. Still need to use deprecated config key for mapred.queue.names > TestRecoveryManagershould not use raw and deprecated configuration parameters. > -- > > Key: MAPREDUCE-4318 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4318 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.22.1 >Reporter: Konstantin Shvachko >Assignee: Benoy Antony > Attachments: MAPREDUCE-4318.patch > > > TestRecoveryManager should not use deprecated config keys, and should use > constants for the keys where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Work started] (MAPREDUCE-4318) TestRecoveryManagershould not use raw and deprecated configuration parameters.
[ https://issues.apache.org/jira/browse/MAPREDUCE-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on MAPREDUCE-4318 started by Benoy Antony. > TestRecoveryManagershould not use raw and deprecated configuration parameters. > -- > > Key: MAPREDUCE-4318 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4318 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: test >Affects Versions: 0.22.1 >Reporter: Konstantin Shvachko >Assignee: Benoy Antony > > TestRecoveryManager should not use deprecated config keys, and should use > constants for the keys where possible. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4260) Investigate use of JobObject to spawn tasks on Windows
[ https://issues.apache.org/jira/browse/MAPREDUCE-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bikas Saha updated MAPREDUCE-4260: -- Attachment: MAPREDUCE-4260.branch-1-win.patch Attaching a patch with 1) bug fixes 2) native code placed in winutils 3) changing TestKillSubProcesses test the new code. There is a small hack in MiniDFSCluster to make the test run in the debugger by working around Path issues on Windows. Several jira's track the Path issues. > Investigate use of JobObject to spawn tasks on Windows > -- > > Key: MAPREDUCE-4260 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4260 > Project: Hadoop Map/Reduce > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Bikas Saha >Assignee: Bikas Saha > Attachments: MAPREDUCE-4260.branch-1-win.patch, MAPREDUCE-4260.patch, > test.cpp > > > Currently, the Windows version spawns the task as a normal cmd shell from > which other downstream exe's are spawned. However, this is not bullet proof > because if an intermediate process exits before its child exits, then the > parent child process tree relationship cannot be constructed. Windows has a > concept of JobObject that is similar to the setsid behavior used in Linux. > The initial spawned task could be launched within its JobObject. Thereafter, > process termination, memory management etc could be operated on the JobObject. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-4306) Problem running Distributed Shell applications as a user other than the one started the daemons
[ https://issues.apache.org/jira/browse/MAPREDUCE-4306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ahmed Radwan reassigned MAPREDUCE-4306: --- Assignee: Ahmed Radwan > Problem running Distributed Shell applications as a user other than the one > started the daemons > --- > > Key: MAPREDUCE-4306 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4306 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.0.0-alpha >Reporter: Ahmed Radwan >Assignee: Ahmed Radwan > > Using the tarball, if you start the yarn daemons using one user and then > switch to a different user. You can successfully run MR jobs, but DS jobs > fail to run. Only able to run DS jobs using the user who started the daemons. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4322) Fix command-line length abort issues on Windows
[
https://issues.apache.org/jira/browse/MAPREDUCE-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Mitic updated MAPREDUCE-4322:
--
Attachment: MAPREDUCE-4322-branch-1-win.patch
Attaching the patch.
The fix is to separate the classpath into an environment variable instead of
passing it via "java -classpath".
In some test cases we've seen command line length go slightly above 8192
characters what is the Windows command line limit. ~4k goes into the classpath,
and the rest goes on other command line arguments. By separating out the
classpath we now have plenty of room for other args.
The patch also introduces checks on the command length before it is executed,
and surfaces a nice error message if the length exceeds the limit. Otherwise,
we would only see that the child task exited with non 0 code, and we would not
have any context on the reason for a failure.
> Fix command-line length abort issues on Windows
> ---
>
> Key: MAPREDUCE-4322
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4322
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Environment: Windows, downstream applications with long aggregate
> classpaths
>Reporter: John Gordon
>Assignee: Ivan Mitic
> Attachments: MAPREDUCE-4322-branch-1-win.patch
>
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> When a task is started on the tasktracker, it creates a small batch file to
> invoke java and runs that batch. Within the batch file, the invocation of
> Java currently has -classpath ${CLASSPATH} inline to the command. That line
> often exceeds 8000 characters. This is ok for most linux distributions
> because the line limit env variable is often set much higher than this.
> However, for Windows this cause cmd to abort execution. This surfaces in
> Hadoop as an unknown failure mode for the task.
> I think the easiest and most natural way to fix this is to push the
> -classpath option into a config file to take the longest variable part of the
> line and put it somewhere that scales better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2454) Allow external sorter plugin for MR
[ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290582#comment-13290582 ] Hadoop QA commented on MAPREDUCE-2454: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531175/mapreduce-2454.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient: org.apache.hadoop.mapred.TestReduceFetchFromPartialMem +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2442//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2442//console This message is automatically generated. > Allow external sorter plugin for MR > --- > > Key: MAPREDUCE-2454 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Mariappan Asokan >Priority: Minor > Labels: features, performance, plugin, sort > Attachments: HadoopSortPlugin.pdf, KeyValueIterator.java, > MR-2454-trunkPatchPreview.gz, MapOutputSorter.java, > MapOutputSorterAbstract.java, ReduceInputSorter.java, mapreduce-2454.patch, > mr-2454-on-mr-279-build82.patch.gz > > > Define interfaces and some abstract classes in the Hadoop framework to > facilitate external sorter plugins both on the Map and Reduce sides. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Assigned] (MAPREDUCE-4322) Fix command-line length abort issues on Windows
[
https://issues.apache.org/jira/browse/MAPREDUCE-4322?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Mitic reassigned MAPREDUCE-4322:
-
Assignee: Ivan Mitic
> Fix command-line length abort issues on Windows
> ---
>
> Key: MAPREDUCE-4322
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4322
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: tasktracker
> Environment: Windows, downstream applications with long aggregate
> classpaths
>Reporter: John Gordon
>Assignee: Ivan Mitic
> Original Estimate: 12h
> Remaining Estimate: 12h
>
> When a task is started on the tasktracker, it creates a small batch file to
> invoke java and runs that batch. Within the batch file, the invocation of
> Java currently has -classpath ${CLASSPATH} inline to the command. That line
> often exceeds 8000 characters. This is ok for most linux distributions
> because the line limit env variable is often set much higher than this.
> However, for Windows this cause cmd to abort execution. This surfaces in
> Hadoop as an unknown failure mode for the task.
> I think the easiest and most natural way to fix this is to push the
> -classpath option into a config file to take the longest variable part of the
> line and put it somewhere that scales better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2454) Allow external sorter plugin for MR
[ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mariappan Asokan updated MAPREDUCE-2454: Attachment: (was: mapreduce-2454.patch) > Allow external sorter plugin for MR > --- > > Key: MAPREDUCE-2454 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Mariappan Asokan >Priority: Minor > Labels: features, performance, plugin, sort > Attachments: HadoopSortPlugin.pdf, KeyValueIterator.java, > MR-2454-trunkPatchPreview.gz, MapOutputSorter.java, > MapOutputSorterAbstract.java, ReduceInputSorter.java, mapreduce-2454.patch, > mr-2454-on-mr-279-build82.patch.gz > > > Define interfaces and some abstract classes in the Hadoop framework to > facilitate external sorter plugins both on the Map and Reduce sides. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2454) Allow external sorter plugin for MR
[ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mariappan Asokan updated MAPREDUCE-2454: Attachment: mapreduce-2454.patch > Allow external sorter plugin for MR > --- > > Key: MAPREDUCE-2454 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Mariappan Asokan >Priority: Minor > Labels: features, performance, plugin, sort > Attachments: HadoopSortPlugin.pdf, KeyValueIterator.java, > MR-2454-trunkPatchPreview.gz, MapOutputSorter.java, > MapOutputSorterAbstract.java, ReduceInputSorter.java, mapreduce-2454.patch, > mapreduce-2454.patch, mr-2454-on-mr-279-build82.patch.gz > > > Define interfaces and some abstract classes in the Hadoop framework to > facilitate external sorter plugins both on the Map and Reduce sides. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-2454) Allow external sorter plugin for MR
[ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290514#comment-13290514 ] Mariappan Asokan commented on MAPREDUCE-2454: - Fixed findbugs and javadoc warnings. > Allow external sorter plugin for MR > --- > > Key: MAPREDUCE-2454 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Mariappan Asokan >Priority: Minor > Labels: features, performance, plugin, sort > Attachments: HadoopSortPlugin.pdf, KeyValueIterator.java, > MR-2454-trunkPatchPreview.gz, MapOutputSorter.java, > MapOutputSorterAbstract.java, ReduceInputSorter.java, mapreduce-2454.patch, > mr-2454-on-mr-279-build82.patch.gz > > > Define interfaces and some abstract classes in the Hadoop framework to > facilitate external sorter plugins both on the Map and Reduce sides. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2454) Allow external sorter plugin for MR
[ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mariappan Asokan updated MAPREDUCE-2454: Status: Patch Available (was: Open) > Allow external sorter plugin for MR > --- > > Key: MAPREDUCE-2454 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Mariappan Asokan >Priority: Minor > Labels: features, performance, plugin, sort > Attachments: HadoopSortPlugin.pdf, KeyValueIterator.java, > MR-2454-trunkPatchPreview.gz, MapOutputSorter.java, > MapOutputSorterAbstract.java, ReduceInputSorter.java, mapreduce-2454.patch, > mr-2454-on-mr-279-build82.patch.gz > > > Define interfaces and some abstract classes in the Hadoop framework to > facilitate external sorter plugins both on the Map and Reduce sides. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2454) Allow external sorter plugin for MR
[ https://issues.apache.org/jira/browse/MAPREDUCE-2454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mariappan Asokan updated MAPREDUCE-2454: Status: Open (was: Patch Available) > Allow external sorter plugin for MR > --- > > Key: MAPREDUCE-2454 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-2454 > Project: Hadoop Map/Reduce > Issue Type: New Feature >Affects Versions: 2.0.0-alpha, 3.0.0 >Reporter: Mariappan Asokan >Priority: Minor > Labels: features, performance, plugin, sort > Attachments: HadoopSortPlugin.pdf, KeyValueIterator.java, > MR-2454-trunkPatchPreview.gz, MapOutputSorter.java, > MapOutputSorterAbstract.java, ReduceInputSorter.java, mapreduce-2454.patch, > mr-2454-on-mr-279-build82.patch.gz > > > Define interfaces and some abstract classes in the Hadoop framework to > facilitate external sorter plugins both on the Map and Reduce sides. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4320) gridmix mainClass wrong in pom.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-4320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290464#comment-13290464 ] Hadoop QA commented on MAPREDUCE-4320: -- -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12531151/MAPREDUCE-4320.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. -1 findbugs. The patch appears to introduce 11 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-tools/hadoop-gridmix. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2441//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2441//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-gridmix.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2441//console This message is automatically generated. > gridmix mainClass wrong in pom.xml > -- > > Key: MAPREDUCE-4320 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4320 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/gridmix >Affects Versions: 0.23.3 >Reporter: Thomas Graves >Assignee: Thomas Graves > Attachments: MAPREDUCE-4320.patch > > > when trying to run gridmix its actually trying to run > org.apache.hadoop.tools.HadoopArchives. > the pom.xml needs to be fixed to have correct mainClass: > org.apache.hadoop.mapred.gridmix.Gridmix -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4322) Fix command-line length abort issues on Windows
John Gordon created MAPREDUCE-4322:
--
Summary: Fix command-line length abort issues on Windows
Key: MAPREDUCE-4322
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4322
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: tasktracker
Environment: Windows, downstream applications with long aggregate
classpaths
Reporter: John Gordon
When a task is started on the tasktracker, it creates a small batch file to
invoke java and runs that batch. Within the batch file, the invocation of Java
currently has -classpath ${CLASSPATH} inline to the command. That line often
exceeds 8000 characters. This is ok for most linux distributions because the
line limit env variable is often set much higher than this. However, for
Windows this cause cmd to abort execution. This surfaces in Hadoop as an
unknown failure mode for the task.
I think the easiest and most natural way to fix this is to push the -classpath
option into a config file to take the longest variable part of the line and put
it somewhere that scales better.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4321) DefaultTaskController fails to launch tasks on Windows
[
https://issues.apache.org/jira/browse/MAPREDUCE-4321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Mitic updated MAPREDUCE-4321:
--
Attachment: MAPREDUCE-4321-branch-1-win.patch
Attaching the patch.
> DefaultTaskController fails to launch tasks on Windows
> --
>
> Key: MAPREDUCE-4321
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4321
> Project: Hadoop Map/Reduce
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Ivan Mitic
>Assignee: Ivan Mitic
> Attachments: MAPREDUCE-4321-branch-1-win.patch
>
>
> DefaultTaskController#launchTask tries to run the child JVM task with the
> following command line:
> {code}cmd.exe /c /c:/some/path.../taskjvm.cmd{code}
> And this fails because the given path is prefixed with a forward slash. This
> also causes a number of tests to fail:
> org.apache.hadoop.conf.TestNoDefaultsJobConf
> org.apache.hadoop.fs.TestCopyFiles
> org.apache.hadoop.mapred.TestBadRecords
> org.apache.hadoop.mapred.TestClusterMRNotification
> org.apache.hadoop.mapred.TestCompressedEmptyMapOutputs
> org.apache.hadoop.mapred.TestControlledMapReduceJob
> org.apache.hadoop.mapred.TestCustomOutputCommitter
> org.apache.hadoop.mapred.TestEmptyJob
> org.apache.hadoop.mapred.TestFileOutputFormat
> org.apache.hadoop.mapred.TestIsolationRunner
> org.apache.hadoop.mapred.TestJavaSerialization
> org.apache.hadoop.mapred.TestJobCleanup
> org.apache.hadoop.mapred.TestJobCounters
> org.apache.hadoop.mapred.TestJobHistoryServer
> org.apache.hadoop.mapred.TestJobInProgressListener
> org.apache.hadoop.mapred.TestJobKillAndFail
> org.apache.hadoop.mapred.TestJobName
> ...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4320) gridmix mainClass wrong in pom.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-4320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated MAPREDUCE-4320: - Status: Patch Available (was: Open) > gridmix mainClass wrong in pom.xml > -- > > Key: MAPREDUCE-4320 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4320 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/gridmix >Affects Versions: 0.23.3 >Reporter: Thomas Graves >Assignee: Thomas Graves > Attachments: MAPREDUCE-4320.patch > > > when trying to run gridmix its actually trying to run > org.apache.hadoop.tools.HadoopArchives. > the pom.xml needs to be fixed to have correct mainClass: > org.apache.hadoop.mapred.gridmix.Gridmix -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4320) gridmix mainClass wrong in pom.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-4320?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290415#comment-13290415 ] Thomas Graves commented on MAPREDUCE-4320: -- this is what you get right now: hadoop jar ./hadoop-0.23.3-SNAPSHOT/share/hadoop/tools/lib/hadoop-gridmix-0.23.3-SNAPSHOT.jar 12/06/06 21:09:12 WARN mapred.JobConf: The variable mapred.child.ulimit is no longer used. 12/06/06 21:09:12 WARN conf.Configuration: mapred.used.genericoptionsparser is deprecated. Instead, use mapreduce.client.genericoptionsparser.used archive -archiveName NAME -p * Invalid usage. with fix it outputs gridmix usage: hadoop jar ./hadoop-0.23.3-SNAPSHOT/share/hadoop/tools/lib/hadoop-gridmix-0.23.3-SNAPSHOT.jar Generic options supported are -conf specify an application configuration file -D use value for given property -fs specify a namenode -jt specify a job tracker -files specify comma separated files to be copied to the map reduce cluster -libjars specify comma separated jar files to include in the classpath. -archives specify comma separated archives to be unarchived on the compute machines. The general command line syntax is bin/hadoop command [genericOptions] [commandOptions] Usage: gridmix [-generate ] [-users URI] [-Dname=value ...] e.g. gridmix -generate 100m foo - Options: -generate : Generate input data of size MiB under /input/ and generate distributed cache data under /distributedCache/. -users : URI that contains the users list. Configuration parameters: General parameters: gridmix.output.directory : Output directory gridmix.client.submit.threads: Submitting threads gridmix.client.pending.queue.depth : Queued job desc gridmix.user.resolve.class : User resolution class gridmix.job.type : Job types (LOADJOB|SLEEPJOB) Parameters related to job submission: gridmix.job-submission.default-queue : Default queue gridmix.job-submission.use-queue-in-trace: Enable/disable using queues in trace gridmix.job-submission.policy: Job submission policy (REPLAY|STRESS|SERIAL) Parameters specific for LOADJOB: gridmix.key.fraction : Key fraction of rec Parameters specific for SLEEPJOB: gridmix.sleep.maptask-only : Whether to ignore reduce tasks gridmix.sleep.fake-locations : Number of fake locations for map tasks gridmix.sleep.max-map-time : Maximum map task runtime in mili-sec gridmix.sleep.max-reduce-time: Maximum reduce task runtime in mili-sec (merge+reduce) Parameters specific for STRESS submission throttling policy: gridmix.throttle.jobs-to-tracker-ratio : jobs vs task-tracker ratio gridmix.throttle.maps.task-to-slot-ratio : maps vs map-slot ratio gridmix.throttle.reduces.task-to-slot-ratio : reduces vs reduce-slot ratio gridmix.throttle.maps.max-slot-share-per-job : map-slot share per job gridmix.throttle.reducess.max-slot-share-per-job : reduce-slot share per job > gridmix mainClass wrong in pom.xml > -- > > Key: MAPREDUCE-4320 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4320 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/gridmix >Affects Versions: 0.23.3 >Reporter: Thomas Graves >Assignee: Thomas Graves > Attachments: MAPREDUCE-4320.patch > > > when trying to run gridmix its actually trying to run > org.apache.hadoop.tools.HadoopArchives. > the pom.xml needs to be fixed to have correct mainClass: > org.apache.hadoop.mapred.gridmix.Gridmix -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4320) gridmix mainClass wrong in pom.xml
[ https://issues.apache.org/jira/browse/MAPREDUCE-4320?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated MAPREDUCE-4320: - Attachment: MAPREDUCE-4320.patch > gridmix mainClass wrong in pom.xml > -- > > Key: MAPREDUCE-4320 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4320 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/gridmix >Affects Versions: 0.23.3 >Reporter: Thomas Graves >Assignee: Thomas Graves > Attachments: MAPREDUCE-4320.patch > > > when trying to run gridmix its actually trying to run > org.apache.hadoop.tools.HadoopArchives. > the pom.xml needs to be fixed to have correct mainClass: > org.apache.hadoop.mapred.gridmix.Gridmix -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4321) DefaultTaskController fails to launch tasks on Windows
Ivan Mitic created MAPREDUCE-4321:
-
Summary: DefaultTaskController fails to launch tasks on Windows
Key: MAPREDUCE-4321
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4321
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 1.0.0
Reporter: Ivan Mitic
Assignee: Ivan Mitic
DefaultTaskController#launchTask tries to run the child JVM task with the
following command line:
{code}cmd.exe /c /c:/some/path.../taskjvm.cmd{code}
And this fails because the given path is prefixed with a forward slash. This
also causes a number of tests to fail:
org.apache.hadoop.conf.TestNoDefaultsJobConf
org.apache.hadoop.fs.TestCopyFiles
org.apache.hadoop.mapred.TestBadRecords
org.apache.hadoop.mapred.TestClusterMRNotification
org.apache.hadoop.mapred.TestCompressedEmptyMapOutputs
org.apache.hadoop.mapred.TestControlledMapReduceJob
org.apache.hadoop.mapred.TestCustomOutputCommitter
org.apache.hadoop.mapred.TestEmptyJob
org.apache.hadoop.mapred.TestFileOutputFormat
org.apache.hadoop.mapred.TestIsolationRunner
org.apache.hadoop.mapred.TestJavaSerialization
org.apache.hadoop.mapred.TestJobCleanup
org.apache.hadoop.mapred.TestJobCounters
org.apache.hadoop.mapred.TestJobHistoryServer
org.apache.hadoop.mapred.TestJobInProgressListener
org.apache.hadoop.mapred.TestJobKillAndFail
org.apache.hadoop.mapred.TestJobName
...
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4320) gridmix mainClass wrong in pom.xml
Thomas Graves created MAPREDUCE-4320: Summary: gridmix mainClass wrong in pom.xml Key: MAPREDUCE-4320 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4320 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/gridmix Affects Versions: 0.23.3 Reporter: Thomas Graves Assignee: Thomas Graves when trying to run gridmix its actually trying to run org.apache.hadoop.tools.HadoopArchives. the pom.xml needs to be fixed to have correct mainClass: org.apache.hadoop.mapred.gridmix.Gridmix -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4311) Capacity scheduler.xml does not accept decimal values for capacity and maximum-capacity settings
[ https://issues.apache.org/jira/browse/MAPREDUCE-4311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290362#comment-13290362 ] Karthik Kambatla commented on MAPREDUCE-4311: - The fix seems fairly straightforward - modify the capacities from int to float. The classes that call getCapacity() eventually convert it to a float value, anyway. I ll upload a patch with the changes later today. > Capacity scheduler.xml does not accept decimal values for capacity and > maximum-capacity settings > > > Key: MAPREDUCE-4311 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4311 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: contrib/capacity-sched, mrv2 >Affects Versions: 0.23.3 >Reporter: Thomas Graves >Assignee: Karthik Kambatla > > if capacity scheduler capacity or max capacity set with decimal it errors: > - Error starting ResourceManager > java.lang.NumberFormatException: For input string: "10.5" > at > java.lang.NumberFormatException.forInputString(NumberFormatException.java:48) > at java.lang.Integer.parseInt(Integer.java:458) > at java.lang.Integer.parseInt(Integer.java:499) > at org.apache.hadoop.conf.Configuration.getInt(Configuration.java:713) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacitySchedulerConfiguration.getCapacity(CapacitySchedulerConfiguration.java:147) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.LeafQueue.(LeafQueue.java:147) > at > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.parseQueue(CapacityScheduler.java:297) > at > 0.20 used to take decimal and this could be an issue on large clusters that > would have queues with small allocations. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3927) Shuffle hang when set map.failures.percent
[ https://issues.apache.org/jira/browse/MAPREDUCE-3927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290207#comment-13290207 ] Jason Lowe commented on MAPREDUCE-3927: --- +1 (non-binding), looks good to me. Thanks Kamesh! > Shuffle hang when set map.failures.percent > -- > > Key: MAPREDUCE-3927 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3927 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 0.23.3, 2.0.0-alpha, 2.0.1-alpha, trunk >Reporter: MengWang >Assignee: Bhallamudi Venkata Siva Kamesh >Priority: Critical > Labels: patch > Attachments: MAPREDUCE-3927-1.patch, MAPREDUCE-3927-2.patch, > MAPREDUCE-3927-3.patch, MAPREDUCE-3927.patch, MAPREDUCE-3927.patch > > > When set mapred.max.map.failures.percent and there does have some failed > maps, then shuffle will hang -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4292) Job is hanging forever when some maps are failing always
[ https://issues.apache.org/jira/browse/MAPREDUCE-4292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290202#comment-13290202 ] Jason Lowe commented on MAPREDUCE-4292: --- Nishan or Devaraj, have we been able to confirm that this issue only occurs when mapred.max.map.failures.percent (or mapreduce.map.failures.maxpercent) is set to a non-zero value? I'm curious if you have been able to reproduce job hangs without that property being set. > Job is hanging forever when some maps are failing always > > > Key: MAPREDUCE-4292 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4292 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.0.0-alpha >Reporter: Nishan Shetty >Priority: Critical > Attachments: syslog > > > Set property "mapred.reduce.tasks" to some value greater than zero > I have a job in which some maps are failing always. > Observations: > 1.Map phase is completing with 100%(with succeeded and failed maps). > 2.Reduce phase is not progressing further after 32%. > 3.After map phase is completed job is hanging forever. > Expected that job should be failed after waiting for some time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4319) MRClientProtocolPBClientImpl creates proxy for AM ignoring custom SocketFactory
[
https://issues.apache.org/jira/browse/MAPREDUCE-4319?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Subroto Sanyal updated MAPREDUCE-4319:
--
Attachment: screenshot-1.jpg
The screenshot shows the Call Hierarchy
> MRClientProtocolPBClientImpl creates proxy for AM ignoring custom
> SocketFactory
> ---
>
> Key: MAPREDUCE-4319
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4319
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: mrv2
>Affects Versions: 0.23.1
>Reporter: Subroto Sanyal
> Attachments: screenshot-1.jpg
>
>
> Use a Customized socket factory and set in the configuration.
> The constructor of MRClientProtocolPBClientImpl creates proxy with
> DefaultSocketFactory
> {code}
> public static ProtocolProxy getProtocolProxy(Class protocol,
> long clientVersion,
> InetSocketAddress addr, Configuration conf)
> throws IOException {
> return getProtocolProxy(protocol, clientVersion, addr, conf, NetUtils
> .getDefaultSocketFactory(conf));
> }
> {code}
> It should use NetUtils.getSocketFactory(Configuration conf,Class clazz)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4319) MRClientProtocolPBClientImpl creates proxy for AM ignoring custom SocketFactory
Subroto Sanyal created MAPREDUCE-4319:
-
Summary: MRClientProtocolPBClientImpl creates proxy for AM
ignoring custom SocketFactory
Key: MAPREDUCE-4319
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4319
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Subroto Sanyal
Use a Customized socket factory and set in the configuration.
The constructor of MRClientProtocolPBClientImpl creates proxy with
DefaultSocketFactory
{code}
public static ProtocolProxy getProtocolProxy(Class protocol,
long clientVersion,
InetSocketAddress addr, Configuration conf)
throws IOException {
return getProtocolProxy(protocol, clientVersion, addr, conf, NetUtils
.getDefaultSocketFactory(conf));
}
{code}
It should use NetUtils.getSocketFactory(Configuration conf,Class clazz)
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4313) TestTokenCache doesn't compile due TokenCache.getDelegationToken compilation error
[
https://issues.apache.org/jira/browse/MAPREDUCE-4313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290167#comment-13290167
]
Hudson commented on MAPREDUCE-4313:
---
Integrated in Hadoop-Mapreduce-trunk #1102 (See
[https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1102/])
MAPREDUCE-4313. TestTokenCache doesn't compile due
TokenCache.getDelegationToken compilation error (bobby) (Revision 1346406)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346406
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/security/TestTokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/system
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/system/test
> TestTokenCache doesn't compile due TokenCache.getDelegationToken compilation
> error
> --
>
> Key: MAPREDUCE-4313
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4313
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
>Reporter: Eli Collins
>Assignee: Robert Joseph Evans
>Priority: Blocker
> Fix For: 2.0.1-alpha, 3.0.0
>
>
> Saw this on the trunk Jenkins job:
> {noformat}
> compile-mapred-test:
> [mkdir] Created dir:
> /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/build/test/mapred/classes
> [mkdir] Created dir:
> /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/build/test/mapred/testjar
> [mkdir] Created dir:
> /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/build/test/mapred/testshell
> [javac] Compiling 95 source files to
> /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/build/test/mapred/classes
> [javac]
> /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/security/TestTokenCache.java:292:
> incompatible types
> [javac] found : org.apache.hadoop.security.token.Token
> [javac] required:
> org.apache.hadoop.security.token.Token
> [javac] Token nnt =
> TokenCache.getDelegationToken(
> [javac]
> ^
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] 1 error
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3873) Nodemanager is not getting decommisioned if the absolute ip is given in exclude file.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290166#comment-13290166 ] Hudson commented on MAPREDUCE-3873: --- Integrated in Hadoop-Mapreduce-trunk #1102 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1102/]) MAPREDUCE-3873. Fixed NodeManagers' decommissioning at RM to accept IP addresses also. Contributed by xieguiming and vinodkv. (Revision 1346671) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346671 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java > Nodemanager is not getting decommisioned if the absolute ip is given in > exclude file. > - > > Key: MAPREDUCE-3873 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3873 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, nodemanager >Affects Versions: 0.23.0 >Reporter: Nishan Shetty >Assignee: xieguiming >Priority: Minor > Fix For: 2.0.1-alpha > > Attachments: MAPREDUCE-3873-20120502.1.txt, > MAPREDUCE-3873-20120502.txt, MAPREDUCE-3873-20120502.txt, > MAPREDUCE-3873-20120502.txt, MAPREDUCE-3873.patch > > > Configure absolute ip in "yarn.resourcemanager.nodes.exclude-path" and try to > decommission the node. > It is not getting decommisioned.But if the hostname is given, decommissioning > is happening. > I have also given the ip-host mapping of each machine in /etc/hosts.(i,e in > every machine the other machines ip-host mapping is specified). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-3873) Nodemanager is not getting decommisioned if the absolute ip is given in exclude file.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290085#comment-13290085 ] Hudson commented on MAPREDUCE-3873: --- Integrated in Hadoop-Hdfs-trunk #1069 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/1069/]) MAPREDUCE-3873. Fixed NodeManagers' decommissioning at RM to accept IP addresses also. Contributed by xieguiming and vinodkv. (Revision 1346671) Result = SUCCESS vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346671 Files : * /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/NodesListManager.java * /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/java/org/apache/hadoop/yarn/server/resourcemanager/TestResourceTrackerService.java > Nodemanager is not getting decommisioned if the absolute ip is given in > exclude file. > - > > Key: MAPREDUCE-3873 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-3873 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2, nodemanager >Affects Versions: 0.23.0 >Reporter: Nishan Shetty >Assignee: xieguiming >Priority: Minor > Fix For: 2.0.1-alpha > > Attachments: MAPREDUCE-3873-20120502.1.txt, > MAPREDUCE-3873-20120502.txt, MAPREDUCE-3873-20120502.txt, > MAPREDUCE-3873-20120502.txt, MAPREDUCE-3873.patch > > > Configure absolute ip in "yarn.resourcemanager.nodes.exclude-path" and try to > decommission the node. > It is not getting decommisioned.But if the hostname is given, decommissioning > is happening. > I have also given the ip-host mapping of each machine in /etc/hosts.(i,e in > every machine the other machines ip-host mapping is specified). -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (MAPREDUCE-4313) TestTokenCache doesn't compile due TokenCache.getDelegationToken compilation error
[
https://issues.apache.org/jira/browse/MAPREDUCE-4313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13290086#comment-13290086
]
Hudson commented on MAPREDUCE-4313:
---
Integrated in Hadoop-Hdfs-trunk #1069 (See
[https://builds.apache.org/job/Hadoop-Hdfs-trunk/1069/])
MAPREDUCE-4313. TestTokenCache doesn't compile due
TokenCache.getDelegationToken compilation error (bobby) (Revision 1346406)
Result = SUCCESS
bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1346406
Files :
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
*
/hadoop/common/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/security/TestTokenCache.java
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/system
* /hadoop/common/trunk/hadoop-mapreduce-project/src/test/system/test
> TestTokenCache doesn't compile due TokenCache.getDelegationToken compilation
> error
> --
>
> Key: MAPREDUCE-4313
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-4313
> Project: Hadoop Map/Reduce
> Issue Type: Bug
> Components: build, test
>Reporter: Eli Collins
>Assignee: Robert Joseph Evans
>Priority: Blocker
> Fix For: 2.0.1-alpha, 3.0.0
>
>
> Saw this on the trunk Jenkins job:
> {noformat}
> compile-mapred-test:
> [mkdir] Created dir:
> /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/build/test/mapred/classes
> [mkdir] Created dir:
> /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/build/test/mapred/testjar
> [mkdir] Created dir:
> /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/build/test/mapred/testshell
> [javac] Compiling 95 source files to
> /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/build/test/mapred/classes
> [javac]
> /home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapreduce/security/TestTokenCache.java:292:
> incompatible types
> [javac] found : org.apache.hadoop.security.token.Token
> [javac] required:
> org.apache.hadoop.security.token.Token
> [javac] Token nnt =
> TokenCache.getDelegationToken(
> [javac]
> ^
> [javac] Note: Some input files use or override a deprecated API.
> [javac] Note: Recompile with -Xlint:deprecation for details.
> [javac] 1 error
> {noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4290) JobStatus.getState() API is giving ambiguous values
[ https://issues.apache.org/jira/browse/MAPREDUCE-4290?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-4290: - Description: For failed job getState() API is giving status as SUCCEEDED if we use JobClient.getAllJobs() for retrieving all jobs info from RM. (was: For failed job getRunState() API is giving status as SUCCEEDED and for successful job its giving status as RUNNING) Summary: JobStatus.getState() API is giving ambiguous values (was: JobStatus.getRunState() API is giving ambiguous values) > JobStatus.getState() API is giving ambiguous values > --- > > Key: MAPREDUCE-4290 > URL: https://issues.apache.org/jira/browse/MAPREDUCE-4290 > Project: Hadoop Map/Reduce > Issue Type: Bug > Components: mrv2 >Affects Versions: 2.0.0-alpha >Reporter: Nishan Shetty >Assignee: Devaraj K > > For failed job getState() API is giving status as SUCCEEDED if we use > JobClient.getAllJobs() for retrieving all jobs info from RM. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
