[jira] [Created] (MAPREDUCE-5766) Ping messages from attempts should be moved to DEBUG
Ramya Sunil created MAPREDUCE-5766: -- Summary: Ping messages from attempts should be moved to DEBUG Key: MAPREDUCE-5766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5766 Project: Hadoop Map/Reduce Issue Type: Bug Components: applicationmaster Reporter: Ramya Sunil Priority: Minor Fix For: 0.24.0 Messages such as org.apache.hadoop.mapred.TaskAttemptListenerImpl: Ping from attempt_1391416522080_0015_m_00_0 in AM logs should be moved to DEBUG. -- This message was sent by Atlassian JIRA (v6.1.5#6160)
Re: [VOTE] Release Apache Hadoop 2.2.0
+1 Deployed on secure and nonsecure clusters. Ran some hdfs, MR and yarn tests. Seems good to me. On Mon, Oct 7, 2013 at 12:21 PM, Alejandro Abdelnur t...@cloudera.comwrote: +1 * downloaded source tarball * verified MD5 * verified signature * verified CHANGES.txt files, release # and date * run 'mvn apache-rat:check' successfully * built distribution * setup speudo cluster * started HDFS/YARN * run some HTTFS tests * run a couple of MR examples * run a few tests using Llama AM On Mon, Oct 7, 2013 at 12:07 PM, Tassapol Athiapinya tathiapi...@hortonworks.com wrote: +1 for the release. I have deployed a multinode cluster and extensively tested MR speculative execution, YARN CLI and YARN distributed shell. There were couple of issues I encountered while testing MAPREDUCE-5533, YARN-1168, YARN-1167, YARN-1157, YARN-1131, YARN-1118, YARN-1117 and all of them have been fixed. Thanks, Tassapol On Oct 7, 2013, at 12:00 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc0) for hadoop-2.2.0 that I would like to get released - this release fixes a small number of bugs and some protocol/api issues which should ensure they are now stable and will not change in hadoop-2.x. The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.2.0-rc0 The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.2.0-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun P.S.: Thanks to Colin, Andrew, Daryn, Chris and others for helping nail down the symlinks-related issues. I'll release note the fact that we have disabled it in 2.2. Also, thanks to Vinod for some heavy-lifting on the YARN side in the last couple of weeks. -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/ -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You. -- Alejandro -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
Re: [VOTE] Release Apache Hadoop 2.1.0-beta
-1. Some of the cli and distcp system tests which use hftp:// and webhdfs:// are failing on secure cluster (HDFS-4841 and HDFS-4952/HDFS-4896). This is a regression and we need to make sure they work before we call a release. On Wed, Jun 26, 2013 at 1:17 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, I've created a release candidate (rc0) for hadoop-2.1.0-beta that I would like to get released. This release represents a *huge* amount of work done by the community (639 fixes) which includes several major advances including: # HDFS Snapshots # Windows support # YARN API stabilization # MapReduce Binary Compatibility with hadoop-1.x # Substantial amount of integration testing with rest of projects in the ecosystem The RC is available at: http://people.apache.org/~acmurthy/hadoop-2.1.0-beta-rc0/ The RC tag in svn is here: http://svn.apache.org/repos/asf/hadoop/common/tags/release-2.1.0-beta-rc0 The maven artifacts are available via repository.apache.org. Please try the release and vote; the vote will run for the usual 7 days. thanks, Arun -- Arun C. Murthy Hortonworks Inc. http://hortonworks.com/
Re: Heads up: branch-2.1-beta
We have started testing branch-2.1-beta and for most parts the code looks very stable. We have deployed both secure and non-secure multinode clusters. We had some minor hiccups with some of our e2e tests breaking due to additional setsid info being logged by the bin scripts and errors while building with snappy turned on. Both are now fixed. Some of the e2e tests had to be tweaked due to configuration changes in branch-2.1-beta such as yarn.nodemanager.delete.debug-delay-sec has to be set to 0 to invoke the cleanup of container tmp files, fs.trash.interval is now a server side config etc. We are now seeing failures while configuring short circuit read and job summary logs(most likely configuration issues) and tests which use hftp:// and webhdfs:// are failing on secure clusters. Will file jiras after initial investigation. Thanks, Ramya On Tue, Jun 4, 2013 at 8:32 AM, Arun C Murthy a...@hortonworks.com wrote: Folks, The vast majority of of the planned features and API work is complete, thanks to everyone who contributed! I've created a branch-2.1-beta branch from which I anticipate I can make the first of our beta releases very shortly. For now the remaining work is to wrap up loose ends i.e. last minute api work (e.g. YARN-759 showed up last night for consideration), bug-fixes etc.; then run this through a battery of unit/system/integration tests and do a final review before we ship. There is more work remaining on documentation (e.g. HADOOP-9517) and I plan to personally focus on it this week - obviously help reviewing docs is very welcome. Committers, from now, please please exercise your judgement on where you commit. Typically, features should go into branch-2 with 2.3.0 as the version on jira (fix-version 2.3.0 is ready). The expectation is that 2.2.0 will be limited to content in branch-2.1-beta and we stick to stabilizing it henceforth (I've deliberately not created 2.2.0 fix-version on jira yet). thanks, Arun
[jira] [Resolved] (MAPREDUCE-858) NPE in heartbeat if mapred.job.tracker.history.completed.location is not writable
[
https://issues.apache.org/jira/browse/MAPREDUCE-858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramya Sunil resolved MAPREDUCE-858.
---
Resolution: Fixed
This is no longer an issue. JT gracefully shuts down if
mapred.job.tracker.history.completed.location is not writable.
NPE in heartbeat if mapred.job.tracker.history.completed.location is not
writable
Key: MAPREDUCE-858
URL: https://issues.apache.org/jira/browse/MAPREDUCE-858
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: jobtracker
Affects Versions: 0.20.1, 0.21.0
Reporter: Ramya Sunil
Priority: Minor
If mapred.job.tracker.history.completed.location has been configured to
write to a location which is not writable by JT, NullPointerException is
thrown in TT heartbeat. Below is the Exception obtained:
{noformat}
2009-08-13 07:56:02,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler
handler on port, call
heartbeat(org.apache.hadoop.mapred.TaskTrackerStatus@1e7a6ae, false, false,
true, 1775) from ip:port: error: java.io.IOException:
java.lang.NullPointerException
java.io.IOException: java.lang.NullPointerException
at
org.apache.hadoop.mapred.JobHistory$JobHistoryFilesManager.moveToDone(JobHistory.java:215)
at
org.apache.hadoop.mapred.JobHistory$JobInfo.markCompleted(JobHistory.java:1071)
at
org.apache.hadoop.mapred.JobTracker.finalizeJob(JobTracker.java:2413)
at
org.apache.hadoop.mapred.JobInProgress.garbageCollect(JobInProgress.java:2729)
at
org.apache.hadoop.mapred.JobInProgress.jobComplete(JobInProgress.java:2327)
at
org.apache.hadoop.mapred.JobInProgress.completedTask(JobInProgress.java:2259)
at
org.apache.hadoop.mapred.JobInProgress.updateTaskStatus(JobInProgress.java:957)
at
org.apache.hadoop.mapred.JobTracker.updateTaskStatuses(JobTracker.java:3946)
at
org.apache.hadoop.mapred.JobTracker.processHeartbeat(JobTracker.java:3123)
at org.apache.hadoop.mapred.JobTracker.heartbeat(JobTracker.java:2861)
at sun.reflect.GeneratedMethodAccessor7.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:508)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:959)
at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:955)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.ipc.Server$Handler.run(Server.java:953)
{noformat}
Instead of an NPE, it would be helpful if an useful error message is logged.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
Re: [Vote] Merge branch-trunk-win to trunk
+1 for the merge. As someone who has been testing the code for many months now, both on singlenode and multinode clusters, I am very confident about the stability and the quality of the code. I have run several regression tests to verify distributed cache, streaming, compression, capacity scheduler, job history and many more features in HDFS and MR. - Ramya On Thu, Feb 28, 2013 at 3:08 PM, sanjay Radia san...@hortonworks.comwrote: +1 Java has done the bulk of the work in making Hadoop multi-platform. Windows specific code is a tiny percentage of the code. Jeninks support for windows is going help us keep the platform portable going forward. I expect that the vast majority of new commits have no problems. I propose that we start by fixing problems that Jenkins raises but not block new commits for too long if the author does not have a windows box or if a volunteer does not step up. sanjay
[jira] [Created] (MAPREDUCE-3841) Broken Server metrics and Local logs link under the tools menu
Broken Server metrics and Local logs link under the tools menu -- Key: MAPREDUCE-3841 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3841 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Local logs link redirects to the cluster page and Server metrics opens an empty page on the RM/JHS homepage. So does the links from nodemanager UI. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3821) NPE while running Shuffle benchmark
NPE while running Shuffle benchmark
---
Key: MAPREDUCE-3821
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3821
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Ramya Sunil
Priority: Critical
hadoop jar hadoop-mapreduce-test.jar loadgen -outKey org.apache.hadoop.io.Text
-outValue org.apache.hadoop.io.Text
The tasks fail with the following exception:
{noformat}
Error: java.lang.NullPointerException
at org.apache.hadoop.fs.Path.init(Path.java:69)
at org.apache.hadoop.fs.Path.init(Path.java:58)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingJobAttemptsPath(FileOutputCommitter.java:118)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:167)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getJobAttemptPath(FileOutputCommitter.java:149)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getPendingTaskAttemptsPath(FileOutputCommitter.java:185)
at
org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:209)
at
org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:100)
at
org.apache.hadoop.mapred.FileOutputCommitter.getTaskAttemptPath(FileOutputCommitter.java:94)
at
org.apache.hadoop.mapred.FileOutputCommitter.needsTaskCommit(FileOutputCommitter.java:176)
at
org.apache.hadoop.mapred.OutputCommitter.needsTaskCommit(OutputCommitter.java:248)
at org.apache.hadoop.mapred.Task.isCommitRequired(Task.java:955)
at org.apache.hadoop.mapred.Task.done(Task.java:912)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:331)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3780) RM assigns containers to killed applications
RM assigns containers to killed applications Key: MAPREDUCE-3780 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3780 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Fix For: 0.23.1 RM attempts to assign containers to killed applications. The applications were killed when they were inactive and waiting for AM allocation. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3781) Fix history for apps which were terminated before the AM launch
Fix history for apps which were terminated before the AM launch --- Key: MAPREDUCE-3781 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3781 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Fix For: 0.23.1 Currently the history for applications which were terminated/killed/failed before the AM was launched redirects to a page that does not exist. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3783) Fix queue -list to display the correct capacity of queues
Fix queue -list to display the correct capacity of queues --- Key: MAPREDUCE-3783 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3783 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Priority: Minor Occasionally, the capacity of the queue as displayed by queue -list has incorrect values. For e.g: yarn.scheduler.capacity.root.queues=a,b yarn.scheduler.capacity.root.b.queues=b1,b2,b3 yarn.scheduler.capacity.root.b.b1.capacity=30 yarn.scheduler.capacity.root.b.b2.capacity=30 yarn.scheduler.capacity.root.b.b3.capacity=40 $ mapred queue -list Queue Name : b Queue State : running Scheduling Info : Capacity: 40.0, MaximumCapacity: 1.0, CurrentCapacity: 0.0 == Queue Name : b1 Queue State : running Scheduling Info : Capacity: *30.02*, MaximumCapacity: 1.0, CurrentCapacity: 0.0 == Queue Name : b2 Queue State : running Scheduling Info : Capacity: *30.02*, MaximumCapacity: 1.0, CurrentCapacity: 0.0 ... -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3784) maxActiveApplications(|PerUser) per queue is too low for small clusters
maxActiveApplications(|PerUser) per queue is too low for small clusters --- Key: MAPREDUCE-3784 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3784 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil We ran into this issue while testing on small clusters. On a 7node cluster with 8G per node, for a queue with absolute capacity 30%, user limit 100%, maxActiveApplications and maxActiveApplicationsPerUser is calculated to be 1. This means that even though the queue has 17GB(0.3*8*7), only 1 user can run 1 app at a given time queuing up rest of the apps/users. This hurts performance on small clusters. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3786) No checks for misconfigured userlimit
No checks for misconfigured userlimit - Key: MAPREDUCE-3786 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3786 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Priority: Minor Currently, there are no checks being made for misconfigured userLimit (such as negative values/values 100) This can potentially be a problem if the RM comes up with incorrect userLimit values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3777) used mem and util have negative values after a queue addition
used mem and util have negative values after a queue addition - Key: MAPREDUCE-3777 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3777 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Assignee: Arun C Murthy Fix For: 0.23.1 After a queue addition to capacity scheduler and submission of an application, root queue utilization and used memory have negative values. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3759) ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes
ClassCastException thrown in -list-active-trackers when there are a few unhealthy nodes --- Key: MAPREDUCE-3759 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3759 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Fix For: 0.23.1 When there are a few blacklisted nodes in the cluster, bin/mapred job -list-active-trackers throws java.lang.ClassCastException: org.apache.hadoop.yarn.server.resourcemanager.resource.Resources$1 cannot be cast to org.apache.hadoop.yarn.api.records.impl.pb.ResourcePBImpl -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3760) Blacklisted NMs should not appear in Active nodes list
Blacklisted NMs should not appear in Active nodes list -- Key: MAPREDUCE-3760 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3760 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Priority: Minor Fix For: 0.23.1 Blacklisted NMs appear in both Active Nodes and Unhealthy nodes on the RM UI. This should be fixed. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3761) AM info in job -list does not reflect the actual AM hostname
AM info in job -list does not reflect the actual AM hostname Key: MAPREDUCE-3761 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3761 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Fix For: 0.23.1 The AM info field on bin/mapred job -list currently has a value resourcemanager hostname:8088/proxy/appID. This info is irrelevant unless it shows the real information of where the AM was launched. This needs to be fixed to show the AM host details. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3763) Failed refreshQueues due to misconfiguration prevents further refreshing of queues
Failed refreshQueues due to misconfiguration prevents further refreshing of queues -- Key: MAPREDUCE-3763 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3763 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Fix For: 0.23.1 Stumbled upon this problem while refreshing queues with incorrect configuration. The exact scenario was: 1. Added a new queue newQueue without defining its capacity. 2. bin/mapred queue -refreshQueues fails correctly with Illegal capacity of -1 for queue root.newQueue 3. However, after defining the capacity of newQueue followed by a second bin/mapred queue -refreshQueues throws org.apache.hadoop.metrics2.MetricsException: Metrics source QueueMetrics,q0=root,q1=newQueue already exists! Also see Hadoop:name=QueueMetrics,q0=root,q1=newQueue,service=ResourceManager metrics being available even though the queue was not added. The expected behavior would be to refresh the queues correctly and allow addition of newQueue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3746) Nodemanagers are not automatically shut down after decommissioning
Nodemanagers are not automatically shut down after decommissioning -- Key: MAPREDUCE-3746 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3746 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Fix For: 0.23.1 Nodemanagers are not automatically shutdown after decommissioning. MAPREDUCE-2775 does not seem to fix the issue. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3747) Memory Total is not refreshed until an app is launched
Memory Total is not refreshed until an app is launched -- Key: MAPREDUCE-3747 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3747 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Fix For: 0.23.1 Memory Total on the RM UI is not refreshed until an application is launched. This is a problem when the cluster is started for the first time or when there are any lost/decommissioned NMs. When the cluster is started for the first time, Active Nodes is 0 but the Memory Total=0. Also when there are any lost/decommissioned nodes, Memory Total has wrong value. This is a useful tool for cluster admins and has to be updated correctly without having the need to submit an app each time. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3748) Move CS related nodeUpdate log messages to DEBUG
Move CS related nodeUpdate log messages to DEBUG Key: MAPREDUCE-3748 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3748 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.1 Reporter: Ramya Sunil Priority: Minor Fix For: 0.23.1 Currently, the RM has nodeUpdate logs per NM per second such as the following: 2012-01-27 21:51:32,429 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: nodeUpdate: nodemanager1:port1 clusterResources: memory: 57344 2012-01-27 21:51:32,510 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: nodeUpdate: nodemanager2:port2 clusterResources: memory: 57344 2012-01-27 21:51:33,094 INFO org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler: nodeUpdate: nodemanager1:port1 clusterResources: memory: 57344 Debugging is difficult with huge amount of logs such as this. These logs need to be moved to DEBUG. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3646) Remove redundant URL info from mapred job output
Remove redundant URL info from mapred job output
--
Key: MAPREDUCE-3646
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3646
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 0.23.1
Reporter: Ramya Sunil
Priority: Minor
Fix For: 0.23.1
The URL information to track the job is printed for all the mapred jobmrv2
commands. This information is redundant and has to be removed.
E.g:
{noformat}
-bash-3.2$ mapred job -list
Total jobs:3
JobId State StartTime UserNameQueue PriorityMaps
Reduces UsedContainers RsvdContainers UsedMem RsvdMem NeededMem AM info
12/01/09 22:20:15 INFO mapred.ClientServiceDelegate: The url to track the job:
RM host:8088/proxy/application ID 1/
job ID 1 RUNNING 1326147596446 ramya default NORMAL 10 10 21
0 22528M 0M 22528M RM host:8088/proxy/application ID 1/
12/01/09 22:20:15 INFO mapred.ClientServiceDelegate: The url to track the job:
RM host:8088/proxy/application ID 2/
job ID 2 RUNNING 1326147603726 ramya default NORMAL 10 10 11
0 12288M 0M 12288M RM host:8088/proxy/application ID 2/
12/01/09 22:20:16 INFO mapred.ClientServiceDelegate: The url to track the job:
RM host:8088/proxy/application ID 3/
job ID 3 RUNNING 1326147520126 ramya default NORMAL 10 10 21
0 22528M 0M 22528M RM host:8088/proxy/application ID 3/
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3647) Pipes job fails with Illegal text protocol
Pipes job fails with Illegal text protocol Key: MAPREDUCE-3647 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3647 Project: Hadoop Map/Reduce Issue Type: Bug Components: pipes Affects Versions: 0.23.1 Reporter: Ramya Sunil Fix For: 0.23.1 Pipes job fail with Hadoop Pipes Exception: Illegal text protocol command -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3368) compile-mapred-test fails
compile-mapred-test fails - Key: MAPREDUCE-3368 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3368 Project: Hadoop Map/Reduce Issue Type: Bug Components: build, mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Priority: Critical Fix For: 0.23.0 compile-mapred-test target is failing once again. Details: https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-Mapreduce-0.23-Build/83/consoleFull -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3305) Fix -list-blacklisted-trackers to print the blacklisted NMs
Fix -list-blacklisted-trackers to print the blacklisted NMs --- Key: MAPREDUCE-3305 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3305 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 bin/mapred job -list-blacklisted-trackers currently prints getBlacklistedTrackers - Not implemented yet This is a long pending issue. Could not find a tracking ticket, hence opening one. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3307) Improve logging on the console during job execution
Improve logging on the console during job execution --- Key: MAPREDUCE-3307 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3307 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 There is a lot of redundant information being printed on the console and a not so intuitive flow of events. We should improve the logging on console during job execution. More details in the next comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3308) MR builds failing due to download failure
MR builds failing due to download failure - Key: MAPREDUCE-3308 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3308 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Assignee: Giridharan Kesavan Priority: Critical Fix For: 0.23.0 MR builds are failing due to unresolved dependencies. [ivy:resolve] :: problems summary :: [ivy:resolve] WARNINGS [ivy:resolve] [FAILED ] org.apache.commons#commons-daemon;1.0.3!commons-daemon.jar: (0ms) [ivy:resolve] fs: tried [ivy:resolve] /home/jenkins/.m2/repository/org/apache/commons/commons-daemon/1.0.3/commons-daemon-1.0.3.jar [ivy:resolve] apache-snapshot: tried [ivy:resolve] https://repository.apache.org/content/repositories/snapshots/org/apache/commons/commons-daemon/1.0.3/commons-daemon-1.0.3.jar [ivy:resolve] maven2: tried [ivy:resolve] http://repo1.maven.org/maven2/org/apache/commons/commons-daemon/1.0.3/commons-daemon-1.0.3.jar [ivy:resolve] :: [ivy:resolve] :: FAILED DOWNLOADS:: [ivy:resolve] :: ^ see resolution messages for details ^ :: [ivy:resolve] :: [ivy:resolve] :: org.apache.commons#commons-daemon;1.0.3!commons-daemon.jar [ivy:resolve] :: [ivy:resolve] [ivy:resolve] :: USE VERBOSE OR DEBUG MESSAGE LEVEL FOR MORE DETAILS -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3309) Report the AM of an application in the UI
Report the AM of an application in the UI - Key: MAPREDUCE-3309 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3309 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 Make provision to report the AM hostname of an application in the RM/JHS UI. It is difficult to trace back the AM on which an app ran when there are 100+ jobs in history. Digging through the logs is an option but since there is no consistency maintained in naming of apps in UI/logs/local dirs (MAPREDUCE-2793), debugging is all the more harder. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3282) bin/mapred job -list throws exception
bin/mapred job -list throws exception - Key: MAPREDUCE-3282 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3282 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 bin/mapred job -list throws exception when mapreduce.framework.name is set to yarn -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3283) yarn classpath CLI does not display the complete classpath
yarn classpath CLI does not display the complete classpath
--
Key: MAPREDUCE-3283
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3283
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
bin/yarn classpath does not display the complete classpath. Below is how the
classpath looks like:
{noformat}
$HADOOP_CONF_DIR:$HADOOP_CONF_DIR::$TOOLS_JAR:$HADOOP_COMMON_HOME/*:$HADOOP_COMMON_HOME/lib/*:$HADOOP_HDFS_HOME/*:$HADOOP_HDFS_HOME/lib/*:
$HADOOP_MAPRED_HOME/bin/../modules/*:$HADOOP_MAPRED_HOME/bin/../lib/*
{noformat}
* has to be substituted with the actual jars. Also, $HADOOP_CONF_DIR appears
twice in the classpath
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3288) Mapreduce 23 builds failing
Mapreduce 23 builds failing --- Key: MAPREDUCE-3288 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3288 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Priority: Blocker Fix For: 0.23.0 Hadoop mapreduce 0.23 builds are failing. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3290) list-active-trackers throws NPE
list-active-trackers throws NPE --- Key: MAPREDUCE-3290 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3290 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 bin/mapred -list-active-trackers throws NPE in mrV2. Trace in the next comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3291) App fail to launch due to delegation token not found in cache
App fail to launch due to delegation token not found in cache - Key: MAPREDUCE-3291 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3291 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 In secure mode, saw an app failure due to org.apache.hadoop.security.token.SecretManager$InvalidToken: token (HDFS_DELEGATION_TOKEN token id for user) can't be found in cache Exception in the next comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3293) Reason for application failure is not correctly reported
Reason for application failure is not correctly reported Key: MAPREDUCE-3293 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3293 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 When apps fail, the reason for failure is not correctly reflected in the UI. For one such app failure, the UI reports Application appID failed 1 times due to . Failing the application. which is not very helpful. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3263) compile-mapred-test target fails
compile-mapred-test target fails Key: MAPREDUCE-3263 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3263 Project: Hadoop Map/Reduce Issue Type: Bug Components: build, mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Compile mapred test target is broken due to which the builds are not archiving the test jars. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3269) Jobsummary logs not being moved to a separate file
Jobsummary logs not being moved to a separate file
--
Key: MAPREDUCE-3269
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3269
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Priority: Blocker
The jobsummary logs are not being moved to a separate file. Below is the
configuration in log4j.properties:
{noformat}
mapred.jobsummary.logger=INFO,console
log4j.logger.org.apache.hadoop.mapreduce.jobhistory.JobSummary=${mapred.jobsummary.logger}
log4j.additivity.org.apache.hadoop.mapreduce.jobhistory.JobSummary=false
log4j.appender.JSA=org.apache.log4j.DailyRollingFileAppender
log4j.appender.JSA.File=${hadoop.log.dir}/mapred-jobsummary.log
log4j.appender.JSA.layout=org.apache.log4j.PatternLayout
log4j.appender.JSA.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
log4j.appender.JSA.DatePattern=.-MM-dd
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3271) Lost nodes list and count not updated
Lost nodes list and count not updated - Key: MAPREDUCE-3271 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3271 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 When nodemanagers are lost, the Lost Nodes list and the count is not incremented. Either we, 1. Fix the lost nodes list when a nodemanager is lost - The problem with tracking lost nodes is, if the nodemanager joins back, there would be duplicate entries in active and lost nodes with different port numbers. 2. Do not track lost nodemanagers -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3272) Lost NMs fail to rejoin
Lost NMs fail to rejoin
---
Key: MAPREDUCE-3272
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3272
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
Lost nodemanagers fail to join back.
When the NM is lost, RM log reads
{noformat}
INFO org.apache.hadoop.yarn.util.AbstractLivelinessMonitor: Expired:host:port
Timed out after 600 secs
INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl:
Processing host:port of type EXPIRE
INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl: Removed
Node host:port
INFO org.apache.hadoop.yarn.server.resourcemanager.rmnode.RMNodeImpl:
host:port Node Transitioned from RUNNING to LOST
{noformat}
When the NM joins back, RM log reads
{noformat}
INFO org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService: Node
not found rebooting host:port
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3254) Streaming jobs failing with PipeMapRunner ClassNotFoundException
Streaming jobs failing with PipeMapRunner ClassNotFoundException Key: MAPREDUCE-3254 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3254 Project: Hadoop Map/Reduce Issue Type: Bug Components: contrib/streaming, mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil ClassNotFoundException: org.apache.hadoop.streaming.PipeMapRunner encountered while running streaming jobs. Stack trace in the next comment. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3243) Invalid tracking URL for streaming jobs
Invalid tracking URL for streaming jobs
---
Key: MAPREDUCE-3243
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3243
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/streaming, mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
The tracking URL for streaming jobs currently display http://N/A;
{noformat}
INFO streaming.StreamJob: To kill this job, run:
INFO streaming.StreamJob: hadoop job -kill jobID
INFO streaming.StreamJob: Tracking URL: http://N/A
INFO mapreduce.Job: Running job: jobID
INFO mapreduce.Job: map 0% reduce 0%
INFO mapred.ClientServiceDelegate: Tracking Url of JOB is host:port
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3051) HADOOP_CONF_DIR exported twice in the classpath
HADOOP_CONF_DIR exported twice in the classpath --- Key: MAPREDUCE-3051 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3051 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Priority: Minor Fix For: 0.23.0 HADOOP_CONF_DIR is exported twice in the classpath during RM, NM and container startup time. Not an issue so far but seems redundant. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2763) IllegalArgumentException while using the dist cache
[ https://issues.apache.org/jira/browse/MAPREDUCE-2763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramya Sunil resolved MAPREDUCE-2763. Resolution: Fixed I see this issue being fixed in the latest code base. Hence resolving this. IllegalArgumentException while using the dist cache --- Key: MAPREDUCE-2763 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2763 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 IllegalArgumentException is seen while using distributed cache to cache some files and custom jars in classpath. A simple way to reproduce this error is by using a streaming job: hadoop jar hadoop-streaming.jar -libjars file://path to custom jar -input path to input file -output out -mapper cat -reducer NONE -cacheFile hdfs://path to some file#linkname This is a regression introduced and the same command works fine on 0.20.x -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3043) Missing containers info on the nodes page
Missing containers info on the nodes page - Key: MAPREDUCE-3043 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3043 Project: Hadoop Map/Reduce Issue Type: Bug Components: resourcemanager Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 The containers info on the nodes page on the RM seems to be missing. This was useful in understanding the usage on each of the nodemanagers. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3044) Pipes jobs stuck without making progress
Pipes jobs stuck without making progress Key: MAPREDUCE-3044 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3044 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 A simple example pipes job gets stuck without making any progress. The AM is launched but the maps do not make any progress. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-3045) Elapsed time filter on jobhistory server displays incorrect table entries
Elapsed time filter on jobhistory server displays incorrect table entries - Key: MAPREDUCE-3045 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3045 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobhistoryserver, mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Priority: Minor Fix For: 0.23.0 The elapsed time filter on the jobhistory server filters incorrect information. For e.g. on a cluster where the elapsed time of all the tasks is either 7 or 8sec, the filter displays non null table entries for 1sec or 3sec -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2843) [MR-279] Node entries on the RM UI are not sortable
[MR-279] Node entries on the RM UI are not sortable --- Key: MAPREDUCE-2843 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2843 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 The nodemanager entries on the RM UI is not sortable unlike the other web pages. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2844) [MR-279] Incorrect node ID info
[MR-279] Incorrect node ID info Key: MAPREDUCE-2844 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2844 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 The node ID info for the nodemanager entires on the RM UI incorrectly displays the value of $yarn.server.nodemanager.address instead of the ID. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2821) [MR-279] Missing fields in job summary logs
[MR-279] Missing fields in job summary logs Key: MAPREDUCE-2821 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2821 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Priority: Minor Fix For: 0.23.0 The following fields are missing in the job summary logs in mrv2: - numSlotsPerMap - numSlotsPerReduce - clusterCapacity (Earlier known as clusterMapCapacity and clusterReduceCapacity in 0.20.x) The first two fields are important to know if the job was a High RAM job or not and the last field is important to know the total available resource in the cluster during job execution. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2834) [MR-279] Enable dense update for file sink metrics
[MR-279] Enable dense update for file sink metrics -- Key: MAPREDUCE-2834 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2834 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 Currently, if File sink is enabled for MRAppMaster or Resourcemanager, it does not populate the file with all the available attributes. It would be useful for debugging and admin purpose to have all the metrics populated in the file. For eg: MRAppMaster metrics currently logs value only for JobsRunning even though the total available job level metrics are JobsCompleted, JobsFailed, JobsKilled, JobsPreparing etc -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-2799) [MR-279] NPE is throwing on job -status Invalid Job ID/Job Id doesn't exist
[
https://issues.apache.org/jira/browse/MAPREDUCE-2799?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ramya Sunil resolved MAPREDUCE-2799.
Resolution: Duplicate
Devaraj, there is already a known bug MAPREDUCE-2686 for the same. Hence
resolving this as duplicate.
[MR-279] NPE is throwing on job -status Invalid Job ID/Job Id doesn't exist
-
Key: MAPREDUCE-2799
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2799
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Devaraj K
Assignee: Devaraj K
{code:xml}
Exception in thread main java.lang.NullPointerException
at
org.apache.hadoop.mapred.ClientServiceDelegate.refreshProxy(ClientServiceDelegate.java:113)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:101)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getRefreshedProxy(ClientServiceDelegate.java:94)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:384)
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:515)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:154)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:223)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1074)
{code}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2800) clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files
clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 in jhist files - Key: MAPREDUCE-2800 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2800 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 clockSplits, cpuUsages, vMemKbytes, physMemKbytes is set to -1 for all the map tasks for the last 4 progress interval in the jobhistory files. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2801) Include the native libs in java.library.path
Include the native libs in java.library.path - Key: MAPREDUCE-2801 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2801 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 For the child tasks in mrv2, java.library.path is set to just $PWD and the native libs are not included. Whereas in 0.20.x, java.library.path for child tasks was set to path to native libs:$PWD -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2802) [MR-279] Jobhistory filenames should have jobID to help in better parsing
[MR-279] Jobhistory filenames should have jobID to help in better parsing -- Key: MAPREDUCE-2802 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2802 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Priority: Minor Fix For: 0.23.0 For jobID such as job_1312933838300_0007, jobhistory file names are named as job%5F1312933838300%5F0007_submit_time_ramya_jobname_finish_time_1_1_SUCCEEDED.jhist It would be easier for parsing if the jobIDs were a part of the filenames. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2789) [MR:279] Update the scheduling info on CLI
[MR:279] Update the scheduling info on CLI -- Key: MAPREDUCE-2789 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2789 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 The scheduling information such as number of containers running, memory usage and reservations per job is not available on bin/mapred job -list CLI. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2790) [MR-279] Add additional field for storing the AM/job history info on CLI
[MR-279] Add additional field for storing the AM/job history info on CLI
Key: MAPREDUCE-2790
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2790
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
bin/mapred job [-list [all]] displays the AM or job history location in the
SchedulingInfo field. An additional column has to be added to display the
AM/job history information. Currently, the output reads:
{noformat}
JobId State StartTime UserNameQueue Priority
SchedulingInfo
jobID FAILED 0 ramya default NORMAL AM
information/job history location
{noformat}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2791) [MR-279] Missing/incorrect info on job -status CLI
[MR-279] Missing/incorrect info on job -status CLI --- Key: MAPREDUCE-2791 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2791 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 There are a couple of details missing/incorrect on the job -status command line output for completed jobs: 1. Incorrect job file 2. map() completion is always 0 3. reduce() completion is always set to 0 4. history URL is empty 5. Missing launched map tasks 6. Missing launched reduce tasks -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2792) [MR-279] Replace IP addresses with hostnames
[MR-279] Replace IP addresses with hostnames Key: MAPREDUCE-2792 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2792 Project: Hadoop Map/Reduce Issue Type: Improvement Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 Currently, all the logs, UI, CLI have IP addresses of the NM/RM, which are difficult to manage. It will be useful to have hostnames like in 0.20.x for easier debugging and maintenance purpose. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2793) [MR-279] Maintain consistency in naming appIDs, jobIDs and attemptIDs
[MR-279] Maintain consistency in naming appIDs, jobIDs and attemptIDs -- Key: MAPREDUCE-2793 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2793 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 appIDs, jobIDs and attempt/container ids are not consistently named in the logs, console and UI. For consistency purpose, they all have to follow a common naming convention. Currently, For appID = On the RM UI: app_1308259676864_5 On the JHS UI: No appID Console/logs: No appID mapred-local dirs are named as: application_1308259676864_0005 For jobID = On the RM UI: job_1308259676864_5_5 JHS UI: job_1308259676864_5_5 Console/logs: job_1308259676864_0005 mapred-local dirs are named as: No jobID For attemptID On the RM UI: attempt_1308259676864_5_5_m_24_0 JHS attempt_1308259676864_5_5_m_24_0 Console/logs: attempt_1308259676864_0005_m_24_0 mapred-local dirs are named as: container_1308259676864_0005_24 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2794) [MR-279] Incorrect metrics value for AvailableGB per queue per user
[MR-279] Incorrect metrics value for AvailableGB per queue per user --- Key: MAPREDUCE-2794 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2794 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 AvailableGB per queue is not the same as AvailableGB per queue per user when the user limit is set to 100%. i.e. if the total available GB of the cluster is 60, and queue default has 92% capacity with 100% as the user limit, AvailableGB per queue default = 55 (i.e. 0.92*60) whereas AvailableGB per queue for user ramya is 56 (however it should be 55 = 0.92*60*1) Also, unlike the AvailableGB/queue, AvailableGB/queue/user is not decremented when user ramya is running apps on the default queue. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2795) [MR-279] AppsKilled is never incremented
[MR-279] AppsKilled is never incremented Key: MAPREDUCE-2795 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2795 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 AppsKilled metrics is never incremented even though there are killed jobs in the system. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2796) [MR-279] Start time for all the apps is set to 0
[MR-279] Start time for all the apps is set to 0 Key: MAPREDUCE-2796 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2796 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 The start time for all the apps in the output of job -list is set to 0 -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2774) Add a startup msg while starting RM/NM
Add a startup msg while starting RM/NM -- Key: MAPREDUCE-2774 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2774 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Priority: Minor Fix For: 0.23.0 Add a startup msg while starting NM/RM indicating the version, build details etc. This will help in easier parsing of logs and debugging. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2775) [MR-279] Decommissioned node does not shutdown
[MR-279] Decommissioned node does not shutdown -- Key: MAPREDUCE-2775 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2775 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 A Nodemanager which is decommissioned by an admin via refreshnodes does not automatically shutdown. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2766) [MR-279] Set correct permissions for files in dist cache
[MR-279] Set correct permissions for files in dist cache Key: MAPREDUCE-2766 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2766 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 Currently, the files in both public and private dist cache are having 777 permission. Also, the group ownership of files on private cache have to be set to $TT_SPECIAL_GROUP -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2768) [MR-279] NMs not being blacklisted as determined by health scripts
[MR-279] NMs not being blacklisted as determined by health scripts -- Key: MAPREDUCE-2768 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2768 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 The NMs are not being blacklisted via the node health script. Below is the configuration used: yarn.server.nodemanager.healthchecker.script.path=path to node health script which blacklists a NM yarn.server.nodemanager.healthchecker.interval=10 yarn.server.nodemanager.healthchecker.script.timeout=12 The node continues to be healthy forever. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2762) [MR-279] - Cleanup staging dir after job completion
[MR-279] - Cleanup staging dir after job completion
---
Key: MAPREDUCE-2762
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2762
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Fix For: 0.23.0
The files created under the staging dir have to be deleted after job
completion. Currently, all job.* files remain forever in the
${yarn.apps.stagingDir}
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2763) IllegalArgumentException while using the dist cache
IllegalArgumentException while using the dist cache --- Key: MAPREDUCE-2763 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2763 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 IllegalArgumentException is seen while using distributed cache to cache some files and custom jars in classpath. A simple way to reproduce this error is by using a streaming job: hadoop jar hadoop-streaming.jar -libjars file://path to custom jar -input path to input file -output out -mapper cat -reducer NONE -cacheFile hdfs://path to some file#linkname This is a regression introduced and the same command works fine on 0.20.x -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2754) MR-279: Error messages are being moved to stdout instead of stderr
MR-279: Error messages are being moved to stdout instead of stderr -- Key: MAPREDUCE-2754 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2754 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Priority: Minor Fix For: 0.23.0 Error messages flagging the reason for app failures are currently being moved to stdout of container logs instead of stderr. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2755) MR-279: AM writes logs to stderr
MR-279: AM writes logs to stderr Key: MAPREDUCE-2755 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2755 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 Currently the AM logs are written to $YARN_LOG_DIR/appID/containerID/stderr. In order to maintain consistency with other container logs, it probably should be moved to syslog. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2757) [MR-279] Redundant file: directory created in appcache
[MR-279] Redundant file: directory created in appcache
-
Key: MAPREDUCE-2757
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2757
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Ramya Sunil
Priority: Minor
Fix For: 0.23.0
A redundant directory called file: is being created under
${yarn.server.nodemanager.local-dir}/usercache/${username}/appcache/appID which
further has a directory structure
${yarn.server.nodemanager.local-dir}/usercache/${username}/(appcache|filecache)/appID/filecache
which is empty.
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2758) MR-279: Log info about the location of dist cache
MR-279: Log info about the location of dist cache - Key: MAPREDUCE-2758 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2758 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Priority: Minor Fix For: 0.23.0 Currently, there is no log info available about the actual location of the file/archive in dist cache being used by the task except for the ln command in task.sh. We need to log this information to help in debugging esp in those cases where there are more than one archive with the same name. In 0.20.x, in task logs, one could find log info such as the following: INFO org.apache.hadoop.mapred.TaskRunner: Creating symlink: distcache location/archive - mapred.local.dir/archive -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2737) Update the progress of jobs on client side
Update the progress of jobs on client side -- Key: MAPREDUCE-2737 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2737 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 The progress of the jobs are not being correctly updated on the client side. The map progress halts at 66% and both map/reduce progress % does not display 100 when the job completes. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2738) Missing cluster level stats on the RM UI
Missing cluster level stats on the RM UI Key: MAPREDUCE-2738 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2738 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 Cluster usage information such as the following are currently not available in the RM UI. - Total number of apps submitted so far - Total number of containers running/total memory usage - Total capacity of the cluster (in terms of memory) - Reserved memory - Total number of NMs - sorting based on Node IDs is an option but when there are lost NMs or restarted NMs, the node ids does not correspond to the actual value - Blacklisted NMs - sorting based on health-status and counting manually is not very straight forward - Excluded NMs - Handle to the jobhistory server -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (MAPREDUCE-1986) History files fail to move to DONE folder when hadoop.job.history.location is configured to a HDFS path
[ https://issues.apache.org/jira/browse/MAPREDUCE-1986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ramya Sunil resolved MAPREDUCE-1986. Resolution: Duplicate This issue is exactly the same as MAPREDUCE-2463. Since there is already a patch available in MAPREDUCE-2463, resolving this as duplicate. Thanks Bhallamudi. History files fail to move to DONE folder when hadoop.job.history.location is configured to a HDFS path --- Key: MAPREDUCE-1986 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1986 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.2 Reporter: Ramya Sunil When hadoop.job.history.location is configured to a HDFS path and when the job retires, the history files are never moved to mapred.job.tracker.history.completed.location and they remain forever in hadoop.job.history.location . The jobhistory is never viewable thereafter. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2689) InvalidStateTransisiton when AM is not assigned to a job
InvalidStateTransisiton when AM is not assigned to a job Key: MAPREDUCE-2689 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2689 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 In cases where an AM is not being assigned to a job, RELEASED at COMPLETED invalid event is observed. This is easily reproducible in cases such as MAPREDUCE-2687. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2690) Construct the web page for default scheduler
Construct the web page for default scheduler Key: MAPREDUCE-2690 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2690 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 Currently, the web page for default scheduler reads as Under construction. This is a long known issue, but could not find a tracking ticket. Hence opening one. -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (MAPREDUCE-2677) MR-279: 404 error while accessing pages from history server
MR-279: 404 error while accessing pages from history server --- Key: MAPREDUCE-2677 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2677 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 0.23.0 Reporter: Ramya Sunil Fix For: 0.23.0 Accessing the following pages from the history server, causes 404 HTTP error 1. Cluster- About 2. Cluster - Applications 3. Cluster - Scheduler 4. Application - About -- This message is automatically generated by JIRA. For more information on JIRA, see: http://www.atlassian.com/software/jira
