[jira] [Created] (MAPREDUCE-4169) Container Logs appear in unsorted order
Container Logs appear in unsorted order --- Key: MAPREDUCE-4169 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4169 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.3, 2.0.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Minor container logs (stdout, stderr, syslog) in the nodemanager ui and jobhistory ui appear in unsorted order where the order displayed is based on what file was created first. This jira will have the results be displayed in a consistent order. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4165) Committing is misspelled as commiting in task logs
Committing is misspelled as commiting in task logs -- Key: MAPREDUCE-4165 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4165 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.0.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Trivial -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-4092) commitJob Exception does not fail job (0.20 regression)
commitJob Exception does not fail job (0.20 regression) --- Key: MAPREDUCE-4092 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4092 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.2 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Blocker Attachments: MAPREDUCE-4092.patch If commitJob throws an exception JobImpl will swallow the exception with a warning and succeed the Job. This is a break from 0.20 and 1.0 where commitJob exception will fail the job org.apache.hadoop.mapreduce.v2.app.job.impl.JobImpl: Could not do commit for Job -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3918) proc_historyserver no longer in command line arguments for HistoryServer
proc_historyserver no longer in command line arguments for HistoryServer Key: MAPREDUCE-3918 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3918 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Attachments: MAPREDUCE-3918.patch This java arg is missing from the command line and needs to be replaced -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3904) Job history produced with mapreduce.cluster.acls.enabled false can not be viewed with mapreduce.cluster.acls.enabled true
Job history produced with mapreduce.cluster.acls.enabled false can not be viewed with mapreduce.cluster.acls.enabled true - Key: MAPREDUCE-3904 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3904 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Job history page displays 'null'. It looks like job history files only populate job acls when mapreduce.cluster.acls.enabled is true. Upon reading job history files, getAcls can return null, throwing an exception on the HsJobBlock page. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3878) byauth bypass paths doesn't work on jobhistory job page
byauth bypass paths doesn't work on jobhistory job page --- Key: MAPREDUCE-3878 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3878 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Critical If jobhistory/job.* is included in the byauth.bypass.paths. URL will always show Null user. This differs from 0.20 where this setting disables security on this page. Bypassing the filter essentially passes a null user to AppController where an exception is thrown. If a null user is detected, we should assume security is disabled on this page. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3857) Grep example ignores mapred.job.queue.name
Grep example ignores mapred.job.queue.name -- Key: MAPREDUCE-3857 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3857 Project: Hadoop Map/Reduce Issue Type: Bug Components: examples Affects Versions: 1.0.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Attachments: MAPREDUCE-3857-branch-1.patch Grep example creates to jobs as part of its implementation. The first job correctly uses the configuration settings. The second job ignores configuration settings. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3716) java.io.File.createTempFile fails in map/reduce tasks
java.io.File.createTempFile fails in map/reduce tasks - Key: MAPREDUCE-3716 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3716 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Blocker container-launch.sh specifies java option java.io.tmpdir when executing the java process for the child container but fails to create the tmpdir. All uses of createTempFile and other commands relying on java.io.tmpdir will fail when called from child container jvms. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3690) mapred.ClientServiceDelegate.java:getProxy prints the proxy without http:// prefix
mapred.ClientServiceDelegate.java:getProxy prints the proxy without http:// prefix -- Key: MAPREDUCE-3690 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3690 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Jonathan Eagles Example job client stdout capture 2012-01-18 10:30:00,980 INFO mapred.ClientServiceDelegate (ClientServiceDelegate.java:getProxy(178)) - The url to track the job: machine.example.com:8088/proxy/application_1326903944546_0002/ -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3623) Authorization of NM <=> RM with simple authentication mistakenly attempts kerberos when yarn.nodemanager.principal is defined
Authorization of NM <=> RM with simple authentication mistakenly attempts
kerberos when yarn.nodemanager.principal is defined
-
Key: MAPREDUCE-3623
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3623
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2
Affects Versions: 0.23.1, 0.24.0
Reporter: Jonathan Eagles
MAPREDUCE-3617 addresses the default values of yarn.nodemanager.principal and
yarn.resourcemanager.principal
I have enabled authorization with simple authentication. NM <=> RM still
attempts kerberos authentication. If simple authentication is enabled
yarn.nodemanager.principal and yarn.resourcemanager.principal values should be
ignored and simple authentication should be used.
{code:xml|title=core-site.xml snippet}
hadoop.security.authentication
simple
hadoop.security.authorization
true
{code}
{code:xml|title=yarn-site.xml snippet}
The Kerberos principal for the resource manager.
yarn.resourcemanager.principal
rm/sightbusy-lx@LOCALHOST
The kerberos principal for the node manager.
yarn.nodemanager.principal
nm/sightbusy-lx@LOCALHOST
{code}
{noformat:title=nodemanager.out snippet}
2012-01-03 16:40:00,793 INFO nodemanager.NodeStatusUpdaterImpl
(NodeStatusUpdaterImpl.java:registerWithRM(176)) - Connected to ResourceManager
at machine.example.com:8025
2012-01-03 16:40:00,845 ERROR service.CompositeService
(CompositeService.java:start(72)) - Error starting services
org.apache.hadoop.yarn.server.nodemanager.NodeManager
org.apache.avro.AvroRuntimeException:
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:149)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.start(NodeManager.java:167)
at
org.apache.hadoop.yarn.server.nodemanager.NodeManager.main(NodeManager.java:242)
Caused by: java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.registerNodeManager(ResourceTrackerPBClientImpl.java:66)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.registerWithRM(NodeStatusUpdaterImpl.java:182)
at
org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl.start(NodeStatusUpdaterImpl.java:145)
... 3 more
Caused by: com.google.protobuf.ServiceException:
org.apache.hadoop.security.authorize.AuthorizationException: User user
(auth:SIMPLE) is not authorized for protocol interface
org.apache.hadoop.yarn.proto.ResourceTracker$ResourceTrackerService$BlockingInterface,
expected client Kerberos principal is nm/sightbusy-lx@LOCALHOST
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:139)
at $Proxy24.registerNodeManager(Unknown Source)
at
org.apache.hadoop.yarn.server.api.impl.pb.client.ResourceTrackerPBClientImpl.registerNodeManager(ResourceTrackerPBClientImpl.java:59)
... 5 more
Caused by: org.apache.hadoop.security.authorize.AuthorizationException: User
user (auth:SIMPLE) is not authorized for protocol interface
org.apache.hadoop.yarn.proto.ResourceTracker$ResourceTrackerService$BlockingInterface,
expected client Kerberos principal is nm/sightbusy-lx@LOCALHOST
at org.apache.hadoop.ipc.Client.call(Client.java:1085)
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:136)
... 7 more
2012-01-03 16:40:00,846 WARN event.AsyncDispatcher
(AsyncDispatcher.java:run(78)) - AsyncDispatcher thread interrupted
java.lang.InterruptedException
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:1961)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:1996)
at
java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:399)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:76)
at java.lang.Thread.run(Thread.java:662)
2012-01-03 16:40:00,846 INFO service.AbstractService
(AbstractService.java:stop(75)) - Service:Dispatcher is stopped.
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3617) Remove yarn default values for resource manager and nodemanager principal
Remove yarn default values for resource manager and nodemanager principal - Key: MAPREDUCE-3617 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3617 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1, 0.24.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Default values should be empty since no use can be made of them without correct values defined. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3577) Classpath error when installing via separate tarballs
Classpath error when installing via separate tarballs - Key: MAPREDUCE-3577 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3577 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1, 0.24.0 Reporter: Jonathan Eagles Priority: Critical Classpath error when using the three tarballs to do a hadoop installation. In particular, HADOOP_PREFIX is guessed incorrectly in yarn-config.sh and passed to hadoop-config.sh. Work around is to manually set HADOOP_PREFIX. Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.conf.Configuration at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3522) Capacity Scheduler ACLs not inherited by default
Capacity Scheduler ACLs not inherited by default Key: MAPREDUCE-3522 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3522 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1, 0.24.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Hierarchical Queues do not inherit parent ACLs correctly by default. Instead, if no value is specified for submit or administer acls, then all access is granted. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3518) mapred queue -info -showJobs throws NPE
mapred queue -info -showJobs throws NPE --- Key: MAPREDUCE-3518 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3518 Project: Hadoop Map/Reduce Issue Type: Bug Components: client, mrv2 Affects Versions: 0.23.1, 0.24.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles mapred queue -info default -showJobs Exception in thread "main" java.lang.NullPointerException at org.apache.hadoop.mapreduce.tools.CLI.displayJobList(CLI.java:572) at org.apache.hadoop.mapred.JobQueueClient.displayQueueInfo(JobQueueClient.java:190) at org.apache.hadoop.mapred.JobQueueClient.run(JobQueueClient.java:103) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83) at org.apache.hadoop.mapred.JobQueueClient.main(JobQueueClient.java:234) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3510) Capacity Scheduler inherited ACLs not displayed by mapred queue -showacls
Capacity Scheduler inherited ACLs not displayed by mapred queue -showacls - Key: MAPREDUCE-3510 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3510 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/capacity-sched, mrv2 Affects Versions: 0.23.1, 0.24.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles mapred queue -showacls does not show inherited acls -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3496) Yarn initializes ACL operations from capacity scheduler config in a non-deterministic order
Yarn initializes ACL operations from capacity scheduler config in a non-deterministic order --- Key: MAPREDUCE-3496 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3496 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1, 0.24.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles 'mapred queue -showacls' does not output put acls in a predictable manner. This is a regression from previous versions. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3448) TestCombineOutputCollector javac unchecked warning on mocked generics
TestCombineOutputCollector javac unchecked warning on mocked generics - Key: MAPREDUCE-3448 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3448 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.24.0, 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Minor [javac] found : org.apache.hadoop.mapred.IFile.Writer [javac] required: org.apache.hadoop.mapred.IFile.Writer [javac] Writer mockWriter = mock(Writer.class); [javac] ^ [javac] /home/jeagles/hadoop/trunk/hadoop-mapreduce-project/src/test/mapred/org/apache/hadoop/mapred/TestCombineOutputCollector.java:125: warning: [unchecked] unchecked conversion [javac] found : org.apache.hadoop.mapred.IFile.Writer [javac] required: org.apache.hadoop.mapred.IFile.Writer [javac] Writer mockWriter = mock(Writer.class); [javac] ^ [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] 2 warnings -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3437) Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT
Branch 23 fails to build with Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT -- Key: MAPREDUCE-3437 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3437 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Blocker [INFO] Scanning for projects... [ERROR] The build could not read 1 project -> [Help 1] [ERROR] [ERROR] The project org.apache.hadoop:hadoop-mapreduce-examples:0.24.0-SNAPSHOT (/home/jeagles/hadoop/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/pom.xml) has 1 error [ERROR] Non-resolvable parent POM: Failure to find org.apache.hadoop:hadoop-project:pom:0.24.0-SNAPSHOT in http://stormwalk.champ.corp.yahoo.com:8081/nexus/content/groups/public was cached in the local repository, resolution will not be reattempted until the update interval of nexus has elapsed or updates are forced and 'parent.relativePath' points at wrong local POM @ line 17, column 11 -> [Help 2] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException [ERROR] [Help 2] http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3413) RM web ui applications not sorted in any order by default
RM web ui applications not sorted in any order by default - Key: MAPREDUCE-3413 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3413 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.24.0, 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3411) Performance Upgrade for jQuery
Performance Upgrade for jQuery -- Key: MAPREDUCE-3411 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3411 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.24.0, 0.23.1 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Priority: Minor jQuery 1.6.4 is almost twice as fast as current version 1.4.4 on modern browsers on some operations. There are also many modern browser compatibility fixes http://jsperf.com/jquery-15-unique-traversal/15 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3324) HttpServer tools are not accessible through the UI Tools
HttpServer tools are not accessible through the UI Tools Key: MAPREDUCE-3324 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3324 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver, mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3195) MapReduce PreCommit Trunk build is silently failing when running tests
MapReduce PreCommit Trunk build is silently failing when running tests -- Key: MAPREDUCE-3195 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3195 Project: Hadoop Map/Reduce Issue Type: Bug Components: build Affects Versions: 0.23.0, 0.24.0 Reporter: Jonathan Eagles Priority: Blocker MapReduce PreCommit build is silently failing only running a very small portion of tests. The build then errors out, yet +1 it given to the patch. Last known Success build - 307 tests run and passed https://builds.apache.org/view/G-L/view/Hadoop/job/PreCommit-MAPREDUCE-Build/990/testReport/ First known Error build - 69 tests run and passed https://builds.apache.org/view/G-L/view/Hadoop/job/PreCommit-MAPREDUCE-Build/994/testReport/ Snippet from failed build log - Errors out and then +1 the patch https://builds.apache.org/view/G-L/view/Hadoop/job/PreCommit-MAPREDUCE-Build/994/console [INFO] [INFO] Reactor Summary: [INFO] [INFO] hadoop-yarn-api ... SUCCESS [19.512s] [INFO] hadoop-yarn-common FAILURE [13.835s] [INFO] hadoop-yarn-server-common . SKIPPED [INFO] hadoop-yarn-server-nodemanager SKIPPED [INFO] hadoop-yarn-server-resourcemanager SKIPPED [INFO] hadoop-yarn-server-tests .. SKIPPED [INFO] hadoop-yarn-server SKIPPED [INFO] hadoop-yarn-applications-distributedshell . SKIPPED [INFO] hadoop-yarn-applications .. SKIPPED [INFO] hadoop-yarn-site .. SKIPPED [INFO] hadoop-yarn ... SKIPPED [INFO] hadoop-mapreduce-client-core .. SKIPPED [INFO] hadoop-mapreduce-client-common SKIPPED [INFO] hadoop-mapreduce-client-shuffle ... SKIPPED [INFO] hadoop-mapreduce-client-app ... SKIPPED [INFO] hadoop-mapreduce-client-hs SKIPPED [INFO] hadoop-mapreduce-client-jobclient . SKIPPED [INFO] hadoop-mapreduce-client ... SKIPPED [INFO] hadoop-mapreduce .. SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 33.784s [INFO] Finished at: Wed Oct 12 12:03:22 UTC 2011 [INFO] Final Memory: 40M/630M [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:2.2-beta-5:single (tar) on project hadoop-yarn-common: Failed to create assembly: Error adding file 'org.apache.hadoop:hadoop-yarn-api:jar:0.24.0-SNAPSHOT' to archive: /home/jenkins/jenkins-slave/workspace/PreCommit-MAPREDUCE-Build/trunk/hadoop-mapreduce-project/hadoop-yarn/hadoop-yarn-api/target/classes isn't a file. -> [Help 1] [ERROR] [ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch. [ERROR] Re-run Maven using the -X switch to enable full debug logging. [ERROR] [ERROR] For more information about the errors and possible solutions, please read the following articles: [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException [ERROR] [ERROR] After correcting the problems, you can resume the build with the command [ERROR] mvn -rf :hadoop-yarn-common == == Running contrib tests. == == /bin/kill -9 27543 kill: No such process NOP == == Checking the integrity of system test framework code. == == /bin/kill -9 27548 kill: No such process NOP +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12498577/MR3166.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The p
[jira] [Created] (MAPREDUCE-3182) loadgen ignore -m command line when using writing random data
loadgen ignore -m command line when using writing random data - Key: MAPREDUCE-3182 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3182 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, test Affects Versions: 0.23.0, 0.24.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles If no input directories are specified, loadgen goes into a special mode where random data is generated and written. In that mode, setting the number of mappers (-m command line option) is overridden by a calculation. Instead, it should take into consideration the user specified number of mappers and fall back to the calculation. In addition, update the documentation as well to match the new behavior in the code. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3179) Incorrect exit code for hadoop-mapreduce-test tests when exception thrown
Incorrect exit code for hadoop-mapreduce-test tests when exception thrown - Key: MAPREDUCE-3179 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3179 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, test Affects Versions: 0.23.0, 0.24.0 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Exit code for test jar is 0 despite exception thrown hadoop jar hadoop-mapreduce-test-0.23.0-SNAPSHOT.jar loadgen -Dmapreduce.job.acl-view -m 18 -r 0 -outKey org.apache.hadoop.io.Text -outValue org.apache.hadoop.io.Text -indir nonexistentdir Loadgen output snippet org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://machine.name.example.com:9000/user/exampleuser/nonexistentdir at org.apache.hadoop.mapred.FileInputFormat.listStatus(FileInputFormat.java:234) at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:254) at org.apache.hadoop.mapreduce.JobSubmitter.writeOldSplits(JobSubmitter.java:470) at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:462) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:358) at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1159) at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1135) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1156) at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:539) at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:778) at org.apache.hadoop.mapred.GenericMRLoadGenerator.run(GenericMRLoadGenerator.java:200) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69) at org.apache.hadoop.mapred.GenericMRLoadGenerator.main(GenericMRLoadGenerator.java:214) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72) at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144) at org.apache.hadoop.test.MapredTestDriver.run(MapredTestDriver.java:111) at org.apache.hadoop.test.MapredTestDriver.main(MapredTestDriver.java:118) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:189) -bash-3.2$ echo $? 0 This differs from example jar which correctly returns the correct exit code hadoop jar hadoop-mapreduce-examples-0.23.0-SNAPSHOT.jar wordcount nonexistentdir /outputdir wordcount output snippet org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://machine.name.example.com:9000/user/exampleuser/nonexistentdir at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:243) at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:269) at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:443) at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:460) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:358) at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1159) at org.apache.hadoop.mapreduce.Job$2.run(Job.java:1156) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1135) at org.apache.hadoop.mapreduce.Job.submit(Job.java:1156) at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1176) at org.apache.hadoop.examples.WordCount.main(WordCount.java:84) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util
[jira] [Created] (MAPREDUCE-3132) Running a set of methods in a Single Test Class
Running a set of methods in a Single Test Class --- Key: MAPREDUCE-3132 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3132 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Reporter: Jonathan Eagles Assignee: Jonathan Eagles Instead of running every test method in a class, limit to specific testing methods as describe in the link below. http://maven.apache.org/plugins/maven-surefire-plugin/examples/single-test.html Upgrade to the latest version of maven-surefire-plugin that has this feature. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3117) Uber jobs are hanging
Uber jobs are hanging - Key: MAPREDUCE-3117 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3117 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Jonathan Eagles Ran a simple wordcount with uber jobs enabled and job hangs hadoop jar hadoop-mapreduce-examples-0.23.0-SNAPSHOT.jar wordcount -Dmapreduce.job.ubertask.enable=true container syslog is filled with repeating 2011-09-28 16:55:04,157 INFO org.apache.hadoop.mapred.TaskAttemptListenerImpl: MapCompletionEvents request from attempt_1317246458306_0002_r_00_0. fromEventID 0 maxEvents 1 2011-09-28 16:55:04,157 DEBUG org.apache.hadoop.mapreduce.task.reduce.EventFetcher: Got 0 map completion events from 0 2011-09-28 16:55:04,157 DEBUG org.apache.hadoop.mapreduce.task.reduce.EventFetcher: GetMapEventsThread about to sleep for 1000 -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
