[jira] [Updated] (MAPREDUCE-3193) FileInputFormat doesn't read files recursively in the input path dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3193: - Affects Version/s: 0.23.2 FileInputFormat doesn't read files recursively in the input path dir Key: MAPREDUCE-3193 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3193 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Affects Versions: 0.23.2, 1.0.2, 2.0.0, 3.0.0 Reporter: Ramgopal N Assignee: Devaraj K Attachments: MAPREDUCE-3193-1.patch, MAPREDUCE-3193-2.patch, MAPREDUCE-3193-2.patch, MAPREDUCE-3193.patch, MAPREDUCE-3193.security.patch java.io.FileNotFoundException is thrown,if input file is more than one folder level deep and the job is getting failed. Example:Input file is /r1/r2/input.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2720) MR-279: Write a simple Java application
[ https://issues.apache.org/jira/browse/MAPREDUCE-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-2720: - Attachment: MAPREDUCE-2720.patch MR-279: Write a simple Java application --- Key: MAPREDUCE-2720 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2720 Project: Hadoop Map/Reduce Issue Type: New Feature Components: mrv2 Reporter: Sharad Agarwal Assignee: Devaraj K Attachments: MAPREDUCE-2720.patch Currently for isolation purposes, many simple java applications run in cluster with 1 map only job. (eg. Oozie). This is not really required with nextgen hadoop (mrv2) and *non-MR* apps are first class and easy to write. A simple hadoop java app can be written which runs in the cluster in the user space. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2720) MR-279: Write a simple Java application
[ https://issues.apache.org/jira/browse/MAPREDUCE-2720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-2720: - Affects Version/s: 3.0.0 2.0.0 Status: Patch Available (was: Open) I have attached first level patch for review. Please give your comments/suggestions on the patch. MR-279: Write a simple Java application --- Key: MAPREDUCE-2720 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2720 Project: Hadoop Map/Reduce Issue Type: New Feature Components: mrv2 Affects Versions: 2.0.0, 3.0.0 Reporter: Sharad Agarwal Assignee: Devaraj K Attachments: MAPREDUCE-2720.patch Currently for isolation purposes, many simple java applications run in cluster with 1 map only job. (eg. Oozie). This is not really required with nextgen hadoop (mrv2) and *non-MR* apps are first class and easy to write. A simple hadoop java app can be written which runs in the cluster in the user space. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4159) Job is running in Uber mode after setting mapreduce.job.ubertask.maxreduces to zero
[ https://issues.apache.org/jira/browse/MAPREDUCE-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-4159: - Assignee: Devaraj K Status: Patch Available (was: Open) I have attached patch to address this issue. Job is running in Uber mode after setting mapreduce.job.ubertask.maxreduces to zero - Key: MAPREDUCE-4159 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4159 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.0.0 Reporter: Nishan Shetty Assignee: Devaraj K Attachments: MAPREDUCE-4159.patch 1.Configure mapreduce.job.ubertask.enable to true 2.Configure mapreduce.job.ubertask.maxreduces to 0(zero) 3.Run job such that it has one reducer(more than mapreduce.job.ubertask.maxreduces value) Observe that job is running in Uber mode instead of normal mode(non uber mode) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4159) Job is running in Uber mode after setting mapreduce.job.ubertask.maxreduces to zero
[ https://issues.apache.org/jira/browse/MAPREDUCE-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-4159: - Attachment: MAPREDUCE-4159.patch Job is running in Uber mode after setting mapreduce.job.ubertask.maxreduces to zero - Key: MAPREDUCE-4159 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4159 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.0.0 Reporter: Nishan Shetty Attachments: MAPREDUCE-4159.patch 1.Configure mapreduce.job.ubertask.enable to true 2.Configure mapreduce.job.ubertask.maxreduces to 0(zero) 3.Run job such that it has one reducer(more than mapreduce.job.ubertask.maxreduces value) Observe that job is running in Uber mode instead of normal mode(non uber mode) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4159) Job is running in Uber mode after setting mapreduce.job.ubertask.maxreduces to zero
[ https://issues.apache.org/jira/browse/MAPREDUCE-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-4159: - Attachment: MAPREDUCE-4159-1.patch Job is running in Uber mode after setting mapreduce.job.ubertask.maxreduces to zero - Key: MAPREDUCE-4159 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4159 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.0.0 Reporter: Nishan Shetty Assignee: Devaraj K Attachments: MAPREDUCE-4159-1.patch, MAPREDUCE-4159.patch 1.Configure mapreduce.job.ubertask.enable to true 2.Configure mapreduce.job.ubertask.maxreduces to 0(zero) 3.Run job such that it has one reducer(more than mapreduce.job.ubertask.maxreduces value) Observe that job is running in Uber mode instead of normal mode(non uber mode) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4159) Job is running in Uber mode after setting mapreduce.job.ubertask.maxreduces to zero
[ https://issues.apache.org/jira/browse/MAPREDUCE-4159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-4159: - Affects Version/s: 3.0.0 Status: Patch Available (was: Open) Thanks a lot Robert for looking into the patch. I have updated the patch with the optimization. Job is running in Uber mode after setting mapreduce.job.ubertask.maxreduces to zero - Key: MAPREDUCE-4159 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4159 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2 Affects Versions: 2.0.0, 3.0.0 Reporter: Nishan Shetty Assignee: Devaraj K Attachments: MAPREDUCE-4159-1.patch, MAPREDUCE-4159.patch 1.Configure mapreduce.job.ubertask.enable to true 2.Configure mapreduce.job.ubertask.maxreduces to 0(zero) 3.Run job such that it has one reducer(more than mapreduce.job.ubertask.maxreduces value) Observe that job is running in Uber mode instead of normal mode(non uber mode) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4074) Client continuously retries to RM When RM goes down before launching Application Master
[
https://issues.apache.org/jira/browse/MAPREDUCE-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4074:
-
Status: Open (was: Patch Available)
Cancelling patch to address Thomas review comments.
Client continuously retries to RM When RM goes down before launching
Application Master
---
Key: MAPREDUCE-4074
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4074
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Devaraj K
Attachments: MAPREDUCE-4074-1.patch, MAPREDUCE-4074-2.patch,
MAPREDUCE-4074.patch
Client continuously tries to RM and logs the below messages when the RM goes
down before launching App Master.
I feel exception should be thrown or break the loop after finite no of
retries.
{code:xml}
28/03/12 07:15:03 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 0 time(s).
28/03/12 07:15:04 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 1 time(s).
28/03/12 07:15:05 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 2 time(s).
28/03/12 07:15:06 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 3 time(s).
28/03/12 07:15:07 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 4 time(s).
28/03/12 07:15:08 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 5 time(s).
28/03/12 07:15:09 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 6 time(s).
28/03/12 07:15:10 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 7 time(s).
28/03/12 07:15:11 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 8 time(s).
28/03/12 07:15:12 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 9 time(s).
28/03/12 07:15:13 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 0 time(s).
28/03/12 07:15:14 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 1 time(s).
28/03/12 07:15:15 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 2 time(s).
28/03/12 07:15:16 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 3 time(s).
28/03/12 07:15:17 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 4 time(s).
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4074) Client continuously retries to RM When RM goes down before launching Application Master
[
https://issues.apache.org/jira/browse/MAPREDUCE-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4074:
-
Status: Open (was: Patch Available)
Client continuously retries to RM When RM goes down before launching
Application Master
---
Key: MAPREDUCE-4074
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4074
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Devaraj K
Attachments: MAPREDUCE-4074.patch
Client continuously tries to RM and logs the below messages when the RM goes
down before launching App Master.
I feel exception should be thrown or break the loop after finite no of
retries.
{code:xml}
28/03/12 07:15:03 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 0 time(s).
28/03/12 07:15:04 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 1 time(s).
28/03/12 07:15:05 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 2 time(s).
28/03/12 07:15:06 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 3 time(s).
28/03/12 07:15:07 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 4 time(s).
28/03/12 07:15:08 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 5 time(s).
28/03/12 07:15:09 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 6 time(s).
28/03/12 07:15:10 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 7 time(s).
28/03/12 07:15:11 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 8 time(s).
28/03/12 07:15:12 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 9 time(s).
28/03/12 07:15:13 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 0 time(s).
28/03/12 07:15:14 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 1 time(s).
28/03/12 07:15:15 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 2 time(s).
28/03/12 07:15:16 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 3 time(s).
28/03/12 07:15:17 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 4 time(s).
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4008) ResourceManager throws MetricsException on start up saying QueueMetrics MBean already exists
[
https://issues.apache.org/jira/browse/MAPREDUCE-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4008:
-
Component/s: (was: resourcemanager)
ResourceManager throws MetricsException on start up saying QueueMetrics MBean
already exists
Key: MAPREDUCE-4008
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4008
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, scheduler
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4008.patch
{code:xml}
2012-03-14 15:22:23,089 WARN org.apache.hadoop.metrics2.util.MBeans: Error
creating MBean object name:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default
org.apache.hadoop.metrics2.MetricsException:
org.apache.hadoop.metrics2.MetricsException:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:117)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newMBeanName(DefaultMetricsSystem.java:102)
at org.apache.hadoop.metrics2.util.MBeans.getMBeanName(MBeans.java:91)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:55)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:218)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:93)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:243)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:288)
at $Proxy6.postStart(Unknown Source)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:183)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:155)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:54)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.start(ResourceManager.java:454)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:588)
Caused by: org.apache.hadoop.metrics2.MetricsException:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:113)
... 19 more
2012-03-14 15:22:23,090 WARN org.apache.hadoop.metrics2.util.MBeans: Failed
to register MBean null
javax.management.RuntimeOperationsException: Exception occurred trying to
register the MBean
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:969)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:57)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:218)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:93)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:243)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:288)
at $Proxy6.postStart(Unknown Source)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:183)
at
[jira] [Updated] (MAPREDUCE-4048) NullPointerException exception while accessing the Application Master UI
[
https://issues.apache.org/jira/browse/MAPREDUCE-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4048:
-
Component/s: mrv2
NullPointerException exception while accessing the Application Master UI
Key: MAPREDUCE-4048
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4048
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4048.patch
{code:xml}
2012-03-21 10:21:31,838 ERROR [2145015588@qtp-957250718-801]
org.apache.hadoop.yarn.webapp.Dispatcher: error handling URI:
/mapreduce/attempts/job_1332261815858_2_8/m/KILLED
java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:150)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
...
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.lang.NullPointerException
at com.google.common.base.Joiner.toString(Joiner.java:317)
at com.google.common.base.Joiner.appendTo(Joiner.java:97)
at com.google.common.base.Joiner.appendTo(Joiner.java:127)
at com.google.common.base.Joiner.join(Joiner.java:158)
at com.google.common.base.Joiner.join(Joiner.java:166)
at
org.apache.hadoop.yarn.util.StringHelper.join(StringHelper.java:102)
at
org.apache.hadoop.mapreduce.v2.app.webapp.AppController.badRequest(AppController.java:319)
at
org.apache.hadoop.mapreduce.v2.app.webapp.AppController.attempts(AppController.java:286)
... 36 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3494) The RM should handle the graceful shutdown of the NM.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3494: - Component/s: (was: resourcemanager) The RM should handle the graceful shutdown of the NM. - Key: MAPREDUCE-3494 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3494 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 2.0.0, 3.0.0 Reporter: Ravi Teja Ch N V Assignee: Devaraj K Attachments: MAPREDUCE-3494.1.patch, MAPREDUCE-3494.2.patch, MAPREDUCE-3494.patch Instead of waiting for the NM expiry, RM should remove and handle the NM, which is shutdown gracefully. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4135:
-
Attachment: MAPREDUCE-4135.patch
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4135:
-
Affects Version/s: 2.0.0
Status: Patch Available (was: Open)
As Alejandro mentioned in MAPREDUCE-4137, Calling FileSystem.closeAll() from
MRAppMaster is not required because the FileSystem has its own shutdown hook
ClientFinalizer which does closeAll().
I have removed it and attached the patch. I have not added any tests because it
is just code removal and verified it manually.
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4135:
-
Status: Open (was: Patch Available)
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4135.patch, MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4135:
-
Attachment: MAPREDUCE-4135.patch
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4135.patch, MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4135) MRAppMaster throws IllegalStateException while shutting down
[
https://issues.apache.org/jira/browse/MAPREDUCE-4135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4135:
-
Status: Patch Available (was: Open)
Re-attaching the same patch to run Hadoop QA.
MRAppMaster throws IllegalStateException while shutting down
Key: MAPREDUCE-4135
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4135
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4135.patch, MAPREDUCE-4135.patch
Always MRAppMaster throws IllegalStateException in the stderr while shutting
down. It doesn't look good having this exception in the stderr file of
MRAppMaster container.
{code:xml}
WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use
org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files.
Exception in thread Thread-1 java.lang.IllegalStateException: Shutdown in
progress
at
java.lang.ApplicationShutdownHooks.remove(ApplicationShutdownHooks.java:55)
at java.lang.Runtime.removeShutdownHook(Runtime.java:220)
at org.apache.hadoop.fs.FileSystem$Cache.remove(FileSystem.java:2148)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2180)
at org.apache.hadoop.fs.FileSystem$Cache.closeAll(FileSystem.java:2157)
at org.apache.hadoop.fs.FileSystem.closeAll(FileSystem.java:361)
at
org.apache.hadoop.mapreduce.v2.app.MRAppMaster$MRAppMasterShutdownHook.run(MRAppMaster.java:1014)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4019) -list-attempt-ids is not working
[
https://issues.apache.org/jira/browse/MAPREDUCE-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4019:
-
Status: Open (was: Patch Available)
-list-attempt-ids is not working
-
Key: MAPREDUCE-4019
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4019
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 0.23.1, 0.23.2
Reporter: B Anil Kumar
Priority: Blocker
Attachments: MAPREDUCE-4019.patch, MAPREDUCE-4019.patch
while executing {noformat}bin/mapred job -list-attempt-ids job_id map
running{noformat}, we are getting IllegalArgumentexception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4019) -list-attempt-ids is not working
[
https://issues.apache.org/jira/browse/MAPREDUCE-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4019:
-
Affects Version/s: (was: 0.23.2)
(was: 0.23.1)
3.0.0
2.0.0
Status: Patch Available (was: Open)
I have updated the patch with test case.
-list-attempt-ids is not working
-
Key: MAPREDUCE-4019
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4019
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.0.0, 3.0.0
Reporter: B Anil Kumar
Priority: Blocker
Attachments: MAPREDUCE-4019.patch, MAPREDUCE-4019.patch
while executing {noformat}bin/mapred job -list-attempt-ids job_id map
running{noformat}, we are getting IllegalArgumentexception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4019) -list-attempt-ids is not working
[
https://issues.apache.org/jira/browse/MAPREDUCE-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4019:
-
Attachment: MAPREDUCE-4019.patch
-list-attempt-ids is not working
-
Key: MAPREDUCE-4019
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4019
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 2.0.0, 3.0.0
Reporter: B Anil Kumar
Priority: Blocker
Attachments: MAPREDUCE-4019.patch, MAPREDUCE-4019.patch
while executing {noformat}bin/mapred job -list-attempt-ids job_id map
running{noformat}, we are getting IllegalArgumentexception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4048) NullPointerException exception while accessing the Application Master UI
[
https://issues.apache.org/jira/browse/MAPREDUCE-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4048:
-
Affects Version/s: (was: 0.23.1)
3.0.0
2.0.0
Status: Patch Available (was: Open)
I have attached patch to fix this.
NullPointerException exception while accessing the Application Master UI
Key: MAPREDUCE-4048
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4048
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4048.patch
{code:xml}
2012-03-21 10:21:31,838 ERROR [2145015588@qtp-957250718-801]
org.apache.hadoop.yarn.webapp.Dispatcher: error handling URI:
/mapreduce/attempts/job_1332261815858_2_8/m/KILLED
java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:150)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
...
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.lang.NullPointerException
at com.google.common.base.Joiner.toString(Joiner.java:317)
at com.google.common.base.Joiner.appendTo(Joiner.java:97)
at com.google.common.base.Joiner.appendTo(Joiner.java:127)
at com.google.common.base.Joiner.join(Joiner.java:158)
at com.google.common.base.Joiner.join(Joiner.java:166)
at
org.apache.hadoop.yarn.util.StringHelper.join(StringHelper.java:102)
at
org.apache.hadoop.mapreduce.v2.app.webapp.AppController.badRequest(AppController.java:319)
at
org.apache.hadoop.mapreduce.v2.app.webapp.AppController.attempts(AppController.java:286)
... 36 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4048) NullPointerException exception while accessing the Application Master UI
[
https://issues.apache.org/jira/browse/MAPREDUCE-4048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4048:
-
Attachment: MAPREDUCE-4048.patch
NullPointerException exception while accessing the Application Master UI
Key: MAPREDUCE-4048
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4048
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4048.patch
{code:xml}
2012-03-21 10:21:31,838 ERROR [2145015588@qtp-957250718-801]
org.apache.hadoop.yarn.webapp.Dispatcher: error handling URI:
/mapreduce/attempts/job_1332261815858_2_8/m/KILLED
java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor50.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:150)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
...
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
Caused by: java.lang.NullPointerException
at com.google.common.base.Joiner.toString(Joiner.java:317)
at com.google.common.base.Joiner.appendTo(Joiner.java:97)
at com.google.common.base.Joiner.appendTo(Joiner.java:127)
at com.google.common.base.Joiner.join(Joiner.java:158)
at com.google.common.base.Joiner.join(Joiner.java:166)
at
org.apache.hadoop.yarn.util.StringHelper.join(StringHelper.java:102)
at
org.apache.hadoop.mapreduce.v2.app.webapp.AppController.badRequest(AppController.java:319)
at
org.apache.hadoop.mapreduce.v2.app.webapp.AppController.attempts(AppController.java:286)
... 36 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3494) The RM should handle the graceful shutdown of the NM.
[ https://issues.apache.org/jira/browse/MAPREDUCE-3494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3494: - Affects Version/s: (was: 0.23.0) 3.0.0 2.0.0 Fix Version/s: (was: 0.24.0) The RM should handle the graceful shutdown of the NM. - Key: MAPREDUCE-3494 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3494 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager, resourcemanager Affects Versions: 2.0.0, 3.0.0 Reporter: Ravi Teja Ch N V Assignee: Devaraj K Attachments: MAPREDUCE-3494.1.patch, MAPREDUCE-3494.2.patch, MAPREDUCE-3494.patch Instead of waiting for the NM expiry, RM should remove and handle the NM, which is shutdown gracefully. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4008) ResourceManager throws MetricsException on start up saying QueueMetrics MBean already exists
[
https://issues.apache.org/jira/browse/MAPREDUCE-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4008:
-
Affects Version/s: (was: 0.24.0)
3.0.0
2.0.0
ResourceManager throws MetricsException on start up saying QueueMetrics MBean
already exists
Key: MAPREDUCE-4008
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4008
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, resourcemanager, scheduler
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4008.patch
{code:xml}
2012-03-14 15:22:23,089 WARN org.apache.hadoop.metrics2.util.MBeans: Error
creating MBean object name:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default
org.apache.hadoop.metrics2.MetricsException:
org.apache.hadoop.metrics2.MetricsException:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:117)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newMBeanName(DefaultMetricsSystem.java:102)
at org.apache.hadoop.metrics2.util.MBeans.getMBeanName(MBeans.java:91)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:55)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:218)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:93)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:243)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:288)
at $Proxy6.postStart(Unknown Source)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:183)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:155)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:54)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.start(ResourceManager.java:454)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:588)
Caused by: org.apache.hadoop.metrics2.MetricsException:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:113)
... 19 more
2012-03-14 15:22:23,090 WARN org.apache.hadoop.metrics2.util.MBeans: Failed
to register MBean null
javax.management.RuntimeOperationsException: Exception occurred trying to
register the MBean
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:969)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:57)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:218)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:93)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:243)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:288)
at $Proxy6.postStart(Unknown Source)
at
[jira] [Updated] (MAPREDUCE-4076) Stream job fails with ZipException when use yarn jar command
[
https://issues.apache.org/jira/browse/MAPREDUCE-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4076:
-
Affects Version/s: (was: 0.23.1)
Stream job fails with ZipException when use yarn jar command
Key: MAPREDUCE-4076
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4076
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Blocker
Attachments: MAPREDUCE-4076.patch
Stream job fails with ZipException when use yarn jar command and executes
successfully with hadoop jar command.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./yarn jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output /test/output/1 -mapper cat -reducer wc
packageJobJar: []
[/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin/$%7Bhadoop.home.dir%7D/hadoop-$%7Buser.name%7D/hadoop-unjar4241129353499211360/]
/tmp/streamjob7683981905208294893.jar tmpDir=null
Exception in thread main java.io.IOException: java.util.zip.ZipException:
ZIP file must have at least one entry
at org.apache.hadoop.streaming.JarBuilder.merge(JarBuilder.java:82)
at
org.apache.hadoop.streaming.StreamJob.packageJobJar(StreamJob.java:707)
at
org.apache.hadoop.streaming.StreamJob.setJobConf(StreamJob.java:948)
at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:127)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3193) FileInputFormat doesn't read files recursively in the input path dir
[ https://issues.apache.org/jira/browse/MAPREDUCE-3193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3193: - Component/s: mrv1 Affects Version/s: (was: 0.24.0) 3.0.0 2.0.0 1.0.2 Fix Version/s: (was: 1.1.0) (was: 0.24.0) FileInputFormat doesn't read files recursively in the input path dir Key: MAPREDUCE-3193 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3193 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv1, mrv2 Affects Versions: 1.0.2, 2.0.0, 3.0.0 Reporter: Ramgopal N Assignee: Devaraj K Attachments: MAPREDUCE-3193-1.patch, MAPREDUCE-3193-2.patch, MAPREDUCE-3193-2.patch, MAPREDUCE-3193.patch, MAPREDUCE-3193.security.patch java.io.FileNotFoundException is thrown,if input file is more than one folder level deep and the job is getting failed. Example:Input file is /r1/r2/input.txt -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4019) -list-attempt-ids is not working
[
https://issues.apache.org/jira/browse/MAPREDUCE-4019?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4019:
-
Priority: Blocker (was: Minor)
-list-attempt-ids is not working
-
Key: MAPREDUCE-4019
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4019
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client
Affects Versions: 0.23.1, 0.23.2
Reporter: B Anil Kumar
Priority: Blocker
Attachments: MAPREDUCE-4019.patch
while executing {noformat}bin/mapred job -list-attempt-ids job_id map
running{noformat}, we are getting IllegalArgumentexception.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4117) mapred job -status throws NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4117:
-
Attachment: MAPREDUCE-4117.patch
mapred job -status throws NullPointerException
--
Key: MAPREDUCE-4117
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4117
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client, mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Critical
Attachments: MAPREDUCE-4117.patch
{code:xml}
dev@ubuntudev-linux:~/hadoop/hadoop-trunk/bin$ ./mapred job -status
job_1333408894669_0001
Exception in thread main java.lang.NullPointerException
at org.apache.hadoop.mapreduce.Job.getTaskFailureEventString(Job.java:512)
at org.apache.hadoop.mapreduce.Job.toString(Job.java:463)
at java.lang.String.valueOf(String.java:2838)
at java.io.PrintStream.println(PrintStream.java:788)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:255)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1244)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4117) mapred job -status throws NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4117:
-
Status: Patch Available (was: Open)
I have attached patch to fix this issue.
mapred job -status throws NullPointerException
--
Key: MAPREDUCE-4117
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4117
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client, mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Critical
Attachments: MAPREDUCE-4117.patch
{code:xml}
dev@ubuntudev-linux:~/hadoop/hadoop-trunk/bin$ ./mapred job -status
job_1333408894669_0001
Exception in thread main java.lang.NullPointerException
at org.apache.hadoop.mapreduce.Job.getTaskFailureEventString(Job.java:512)
at org.apache.hadoop.mapreduce.Job.toString(Job.java:463)
at java.lang.String.valueOf(String.java:2838)
at java.io.PrintStream.println(PrintStream.java:788)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:255)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1244)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4117) mapred job -status throws NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4117:
-
Attachment: MAPREDUCE-4117-1.patch
mapred job -status throws NullPointerException
--
Key: MAPREDUCE-4117
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4117
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client, mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Critical
Attachments: MAPREDUCE-4117-1.patch, MAPREDUCE-4117.patch
{code:xml}
dev@ubuntudev-linux:~/hadoop/hadoop-trunk/bin$ ./mapred job -status
job_1333408894669_0001
Exception in thread main java.lang.NullPointerException
at org.apache.hadoop.mapreduce.Job.getTaskFailureEventString(Job.java:512)
at org.apache.hadoop.mapreduce.Job.toString(Job.java:463)
at java.lang.String.valueOf(String.java:2838)
at java.io.PrintStream.println(PrintStream.java:788)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:255)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1244)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4117) mapred job -status throws NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4117:
-
Status: Patch Available (was: Open)
I am sorry Jon. It's my bad, I have mentioned the name wrongly in the above
comment.
Thanks for looking and identifying all the cases. I have attached the patch as
per your suggestion.
mapred job -status throws NullPointerException
--
Key: MAPREDUCE-4117
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4117
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client, mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Critical
Attachments: MAPREDUCE-4117-1.patch, MAPREDUCE-4117.patch
{code:xml}
dev@ubuntudev-linux:~/hadoop/hadoop-trunk/bin$ ./mapred job -status
job_1333408894669_0001
Exception in thread main java.lang.NullPointerException
at org.apache.hadoop.mapreduce.Job.getTaskFailureEventString(Job.java:512)
at org.apache.hadoop.mapreduce.Job.toString(Job.java:463)
at java.lang.String.valueOf(String.java:2838)
at java.io.PrintStream.println(PrintStream.java:788)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:255)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1244)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4117) mapred job -status throws NullPointerException
[
https://issues.apache.org/jira/browse/MAPREDUCE-4117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4117:
-
Status: Open (was: Patch Available)
mapred job -status throws NullPointerException
--
Key: MAPREDUCE-4117
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4117
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: client, mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Critical
Attachments: MAPREDUCE-4117-1.patch, MAPREDUCE-4117.patch
{code:xml}
dev@ubuntudev-linux:~/hadoop/hadoop-trunk/bin$ ./mapred job -status
job_1333408894669_0001
Exception in thread main java.lang.NullPointerException
at org.apache.hadoop.mapreduce.Job.getTaskFailureEventString(Job.java:512)
at org.apache.hadoop.mapreduce.Job.toString(Job.java:463)
at java.lang.String.valueOf(String.java:2838)
at java.io.PrintStream.println(PrintStream.java:788)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:255)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1244)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4057) Compilation error in RAID
[
https://issues.apache.org/jira/browse/MAPREDUCE-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4057:
-
Attachment: MAPREDUCE-4057.patch
Compilation error in RAID
--
Key: MAPREDUCE-4057
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4057
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/raid
Reporter: Tsz Wo (Nicholas), SZE
Attachments: MAPREDUCE-4057.patch
{noformat}
[javac] Compiling 33 source files to
/Users/szetszwo/hadoop/t2/hadoop-mapreduce-project/build/contrib/raid/classes
[javac]
/Users/szetszwo/hadoop/t2/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java:42:
unreported exception org.apache.hadoop.ipc.StandbyException; must be caught
or declared to be thrown
[javac] return namesystem.getFileInfo(src, resolveLink);
[javac] ^
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4057) Compilation error in RAID
[
https://issues.apache.org/jira/browse/MAPREDUCE-4057?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4057:
-
Status: Patch Available (was: Open)
I have attached patch to fix these compilation errors.
Compilation error in RAID
--
Key: MAPREDUCE-4057
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4057
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: contrib/raid
Reporter: Tsz Wo (Nicholas), SZE
Attachments: MAPREDUCE-4057.patch
{noformat}
[javac] Compiling 33 source files to
/Users/szetszwo/hadoop/t2/hadoop-mapreduce-project/build/contrib/raid/classes
[javac]
/Users/szetszwo/hadoop/t2/hadoop-mapreduce-project/src/contrib/raid/src/java/org/apache/hadoop/hdfs/server/namenode/NameNodeRaidUtil.java:42:
unreported exception org.apache.hadoop.ipc.StandbyException; must be caught
or declared to be thrown
[javac] return namesystem.getFileInfo(src, resolveLink);
[javac] ^
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4106) Fix skipping tests in mapreduce
[ https://issues.apache.org/jira/browse/MAPREDUCE-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-4106: - Summary: Fix skipping tests in mapreduce (was: Correct skipping tests in mapreduce) Fix skipping tests in mapreduce --- Key: MAPREDUCE-4106 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4106 Project: Hadoop Map/Reduce Issue Type: Test Components: mrv2 Affects Versions: 2.0.0, 3.0.0 Reporter: Devaraj K There are 22 tests skipping in hadoop-mapreduce-client-jobclient module, all these can be corrected as part of this umbrella jira. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4108) Fix tests in org.apache.hadoop.util.TestRunJar
[
https://issues.apache.org/jira/browse/MAPREDUCE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4108:
-
Summary: Fix tests in org.apache.hadoop.util.TestRunJar (was: Correct
org.apache.hadoop.util.TestRunJar)
Fix tests in org.apache.hadoop.util.TestRunJar
--
Key: MAPREDUCE-4108
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4108
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4108.patch
Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.util.TestRunJar
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.031 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4107) Fix tests in org.apache.hadoop.ipc.TestSocketFactory
[
https://issues.apache.org/jira/browse/MAPREDUCE-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4107:
-
Summary: Fix tests in org.apache.hadoop.ipc.TestSocketFactory (was:
Correct org.apache.hadoop.ipc.TestSocketFactory)
Fix tests in org.apache.hadoop.ipc.TestSocketFactory
Key: MAPREDUCE-4107
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4107
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4107.patch
Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.ipc.TestSocketFactory
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.031 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4110) Fix tests in org.apache.hadoop.mapred.TestMiniMRClasspath org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
[
https://issues.apache.org/jira/browse/MAPREDUCE-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4110:
-
Attachment: MAPREDUCE-4110.patch
Fix tests in org.apache.hadoop.mapred.TestMiniMRClasspath
org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
---
Key: MAPREDUCE-4110
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4110
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2, test
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4110.patch
Sub Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.mapred.TestMiniMRClasspath
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.032 sec
{code}
{code:xml}
Running org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.031 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4110) Fix tests in org.apache.hadoop.mapred.TestMiniMRClasspath org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
[
https://issues.apache.org/jira/browse/MAPREDUCE-4110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4110:
-
Status: Patch Available (was: Open)
Attached patch to fix this.
Fix tests in org.apache.hadoop.mapred.TestMiniMRClasspath
org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
---
Key: MAPREDUCE-4110
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4110
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2, test
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4110.patch
Sub Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.mapred.TestMiniMRClasspath
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.032 sec
{code}
{code:xml}
Running org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.031 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4111) Fix tests in org.apache.hadoop.mapred.TestJobName
[
https://issues.apache.org/jira/browse/MAPREDUCE-4111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4111:
-
Attachment: MAPREDUCE-4111.patch
Fix tests in org.apache.hadoop.mapred.TestJobName
-
Key: MAPREDUCE-4111
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4111
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2, test
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4111.patch
Sub Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.mapred.TestJobName
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.031 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4111) Fix tests in org.apache.hadoop.mapred.TestJobName
[
https://issues.apache.org/jira/browse/MAPREDUCE-4111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4111:
-
Status: Patch Available (was: Open)
Attached patch to fix this.
Fix tests in org.apache.hadoop.mapred.TestJobName
-
Key: MAPREDUCE-4111
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4111
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2, test
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4111.patch
Sub Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.mapred.TestJobName
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.031 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4112) Fix tests org.apache.hadoop.mapred.TestClusterMapReduceTestCase
[
https://issues.apache.org/jira/browse/MAPREDUCE-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4112:
-
Attachment: MAPREDUCE-4112.patch
Fix tests org.apache.hadoop.mapred.TestClusterMapReduceTestCase
---
Key: MAPREDUCE-4112
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4112
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2, test
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4112.patch
Sub Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.mapred.TestClusterMapReduceTestCase
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.032 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4113) Fix tests org.apache.hadoop.mapred.TestClusterMRNotification
[
https://issues.apache.org/jira/browse/MAPREDUCE-4113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4113:
-
Status: Patch Available (was: Open)
Attached patch to fix this.
Fix tests org.apache.hadoop.mapred.TestClusterMRNotification
Key: MAPREDUCE-4113
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4113
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2, test
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4113.patch
Sub Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.mapred.TestClusterMRNotification
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.031 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4113) Fix tests org.apache.hadoop.mapred.TestClusterMRNotification
[
https://issues.apache.org/jira/browse/MAPREDUCE-4113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4113:
-
Attachment: MAPREDUCE-4113.patch
Fix tests org.apache.hadoop.mapred.TestClusterMRNotification
Key: MAPREDUCE-4113
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4113
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2, test
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4113.patch
Sub Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.mapred.TestClusterMRNotification
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.031 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4112) Fix tests org.apache.hadoop.mapred.TestClusterMapReduceTestCase
[
https://issues.apache.org/jira/browse/MAPREDUCE-4112?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4112:
-
Status: Patch Available (was: Open)
Attached patch to fix this.
Fix tests org.apache.hadoop.mapred.TestClusterMapReduceTestCase
---
Key: MAPREDUCE-4112
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4112
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2, test
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4112.patch
Sub Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.mapred.TestClusterMapReduceTestCase
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.032 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4107) Correct org.apache.hadoop.ipc.TestSocketFactory
[
https://issues.apache.org/jira/browse/MAPREDUCE-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4107:
-
Attachment: MAPREDUCE-4107.patch
Correct org.apache.hadoop.ipc.TestSocketFactory
---
Key: MAPREDUCE-4107
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4107
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4107.patch
Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.ipc.TestSocketFactory
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.031 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4108) Correct org.apache.hadoop.util.TestRunJar
[
https://issues.apache.org/jira/browse/MAPREDUCE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4108:
-
Attachment: MAPREDUCE-4108.patch
Correct org.apache.hadoop.util.TestRunJar
-
Key: MAPREDUCE-4108
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4108
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4108.patch
Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.util.TestRunJar
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.031 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4108) Correct org.apache.hadoop.util.TestRunJar
[
https://issues.apache.org/jira/browse/MAPREDUCE-4108?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4108:
-
Status: Patch Available (was: Open)
Attached patch to fix this.
Correct org.apache.hadoop.util.TestRunJar
-
Key: MAPREDUCE-4108
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4108
Project: Hadoop Map/Reduce
Issue Type: Sub-task
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4108.patch
Project : *hadoop-mapreduce-client-jobclient*
{code:xml}
Running org.apache.hadoop.util.TestRunJar
Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.031 sec
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4094) Mapreduce-trunk test cases are failing
[
https://issues.apache.org/jira/browse/MAPREDUCE-4094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4094:
-
Description:
https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1037/
{code:xml}
Failed tests:
testDefaultCleanupAndAbort(org.apache.hadoop.mapred.TestJobCleanup): Done file
/home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/test-dir/test-job-cleanup/output-0/_SUCCESS
missing for job job_1333287145240_0001
testCustomAbort(org.apache.hadoop.mapred.TestJobCleanup): Done file
/home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/test-dir/test-job-cleanup/output-1/_SUCCESS
missing for job job_1333287145240_0002
testCustomCleanup(org.apache.hadoop.mapred.TestJobCleanup): Done file
/home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/test-dir/test-job-cleanup/output-2/_custom_cleanup
missing for job job_1333287145240_0003
testHeapUsageCounter(org.apache.hadoop.mapred.TestJobCounters): Job
job_1333287188908_0001 failed!
testTaskTempDir(org.apache.hadoop.mapred.TestMiniMRChildTask)
testTaskEnv(org.apache.hadoop.mapred.TestMiniMRChildTask): The environment
checker job failed.
testTaskOldEnv(org.apache.hadoop.mapred.TestMiniMRChildTask): The environment
checker job failed.
testJob(org.apache.hadoop.mapred.TestMiniMRClientCluster)
testLazyOutput(org.apache.hadoop.mapreduce.TestMapReduceLazyOutput)
testSpeculativeExecution(org.apache.hadoop.mapreduce.v2.TestSpeculativeExecution)
testSleepJob(org.apache.hadoop.mapreduce.v2.TestMRJobs)
testRandomWriter(org.apache.hadoop.mapreduce.v2.TestMRJobs)
testDistributedCache(org.apache.hadoop.mapreduce.v2.TestMRJobs)
testValidProxyUser(org.apache.hadoop.mapreduce.v2.TestMiniMRProxyUser)
Tests in error:
testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem):
Job failed!
testWithDFS(org.apache.hadoop.mapred.TestJobSysDirWithDFS): Job failed!
testReduceFromPartialMem(org.apache.hadoop.mapred.TestReduceFetchFromPartialMem):
Job failed!
testLazyOutput(org.apache.hadoop.mapred.TestLazyOutput): Job failed!
testFailingMapper(org.apache.hadoop.mapreduce.v2.TestMRJobs): 0
org.apache.hadoop.mapreduce.v2.TestMROldApiJobs: Failed to Start
org.apache.hadoop.mapreduce.v2.TestMROldApiJobs
org.apache.hadoop.mapreduce.v2.TestUberAM: Failed to Start
org.apache.hadoop.mapreduce.v2.TestMRJobs
testDefaultCleanupAndAbort(org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter):
Failed to Start org.apache.hadoop.mapred.MiniMRCluster
testCustomAbort(org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter):
Failed to Start org.apache.hadoop.mapred.MiniMRCluster
testCustomCleanup(org.apache.hadoop.mapreduce.lib.output.TestJobOutputCommitter):
Failed to Start org.apache.hadoop.mapred.MiniMRCluster
testChild(org.apache.hadoop.mapreduce.TestChild): Failed to Start
org.apache.hadoop.mapred.MiniMRCluster
Tests run: 404, Failures: 14, Errors: 11, Skipped: 22
{code}
was:
{code:xml}
Failed tests:
testDefaultCleanupAndAbort(org.apache.hadoop.mapred.TestJobCleanup): Done file
/home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/test-dir/test-job-cleanup/output-0/_SUCCESS
missing for job job_1333287145240_0001
testCustomAbort(org.apache.hadoop.mapred.TestJobCleanup): Done file
/home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/test-dir/test-job-cleanup/output-1/_SUCCESS
missing for job job_1333287145240_0002
testCustomCleanup(org.apache.hadoop.mapred.TestJobCleanup): Done file
/home/jenkins/jenkins-slave/workspace/Hadoop-Mapreduce-trunk/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-jobclient/target/test-dir/test-job-cleanup/output-2/_custom_cleanup
missing for job job_1333287145240_0003
testHeapUsageCounter(org.apache.hadoop.mapred.TestJobCounters): Job
job_1333287188908_0001 failed!
testTaskTempDir(org.apache.hadoop.mapred.TestMiniMRChildTask)
testTaskEnv(org.apache.hadoop.mapred.TestMiniMRChildTask): The environment
checker job failed.
testTaskOldEnv(org.apache.hadoop.mapred.TestMiniMRChildTask): The environment
checker job failed.
testJob(org.apache.hadoop.mapred.TestMiniMRClientCluster)
testLazyOutput(org.apache.hadoop.mapreduce.TestMapReduceLazyOutput)
testSpeculativeExecution(org.apache.hadoop.mapreduce.v2.TestSpeculativeExecution)
testSleepJob(org.apache.hadoop.mapreduce.v2.TestMRJobs)
[jira] [Updated] (MAPREDUCE-4093) Improve RM WebApp start up when proxy address is not set
[
https://issues.apache.org/jira/browse/MAPREDUCE-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4093:
-
Attachment: MAPREDUCE-4093.patch
Improve RM WebApp start up when proxy address is not set
Key: MAPREDUCE-4093
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4093
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Attachments: MAPREDUCE-4093.patch
{code:title=ResourceManager.java|borderStyle=solid}
protected void startWepApp() {
BuilderApplicationMasterService builder =
WebApps.$for(cluster, ApplicationMasterService.class, masterService,
ws).at(
this.conf.get(YarnConfiguration.RM_WEBAPP_ADDRESS,
YarnConfiguration.DEFAULT_RM_WEBAPP_ADDRESS));
if(YarnConfiguration.getRMWebAppHostAndPort(conf).
equals(YarnConfiguration.getProxyHostAndPort(conf))) {
AppReportFetcher fetcher = new AppReportFetcher(conf,
getClientRMService());
builder.withServlet(ProxyUriUtils.PROXY_SERVLET_NAME,
ProxyUriUtils.PROXY_PATH_SPEC, WebAppProxyServlet.class);
builder.withAttribute(WebAppProxy.FETCHER_ATTRIBUTE, fetcher);
String proxy = YarnConfiguration.getProxyHostAndPort(conf);
String[] proxyParts = proxy.split(:);
builder.withAttribute(WebAppProxy.PROXY_HOST_ATTRIBUTE, proxyParts[0]);
}
webApp = builder.start(new RMWebApp(this));
}
{code}
In the above code, YarnConfiguration.getProxyHostAndPort(conf) is invoking
twice. getProxyHostAndPort() internally invokes getRMWebAppHostAndPort()
which resolves RM web app address when proxy address is not set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4093) Improve RM WebApp start up when proxy address is not set
[
https://issues.apache.org/jira/browse/MAPREDUCE-4093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4093:
-
Status: Patch Available (was: Open)
I have updated the patch for this.
Improve RM WebApp start up when proxy address is not set
Key: MAPREDUCE-4093
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4093
Project: Hadoop Map/Reduce
Issue Type: Improvement
Components: mrv2
Affects Versions: 2.0.0, 3.0.0
Reporter: Devaraj K
Attachments: MAPREDUCE-4093.patch
{code:title=ResourceManager.java|borderStyle=solid}
protected void startWepApp() {
BuilderApplicationMasterService builder =
WebApps.$for(cluster, ApplicationMasterService.class, masterService,
ws).at(
this.conf.get(YarnConfiguration.RM_WEBAPP_ADDRESS,
YarnConfiguration.DEFAULT_RM_WEBAPP_ADDRESS));
if(YarnConfiguration.getRMWebAppHostAndPort(conf).
equals(YarnConfiguration.getProxyHostAndPort(conf))) {
AppReportFetcher fetcher = new AppReportFetcher(conf,
getClientRMService());
builder.withServlet(ProxyUriUtils.PROXY_SERVLET_NAME,
ProxyUriUtils.PROXY_PATH_SPEC, WebAppProxyServlet.class);
builder.withAttribute(WebAppProxy.FETCHER_ATTRIBUTE, fetcher);
String proxy = YarnConfiguration.getProxyHostAndPort(conf);
String[] proxyParts = proxy.split(:);
builder.withAttribute(WebAppProxy.PROXY_HOST_ATTRIBUTE, proxyParts[0]);
}
webApp = builder.start(new RMWebApp(this));
}
{code}
In the above code, YarnConfiguration.getProxyHostAndPort(conf) is invoking
twice. getProxyHostAndPort() internally invokes getRMWebAppHostAndPort()
which resolves RM web app address when proxy address is not set.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4076) Stream job fails with ZipException when use yarn jar command
[
https://issues.apache.org/jira/browse/MAPREDUCE-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4076:
-
Priority: Blocker (was: Critical)
Affects Version/s: 3.0.0
2.0.0
Stream job fails with ZipException when use yarn jar command
Key: MAPREDUCE-4076
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4076
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1, 2.0.0, 3.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Blocker
Attachments: MAPREDUCE-4076.patch
Stream job fails with ZipException when use yarn jar command and executes
successfully with hadoop jar command.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./yarn jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output /test/output/1 -mapper cat -reducer wc
packageJobJar: []
[/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin/$%7Bhadoop.home.dir%7D/hadoop-$%7Buser.name%7D/hadoop-unjar4241129353499211360/]
/tmp/streamjob7683981905208294893.jar tmpDir=null
Exception in thread main java.io.IOException: java.util.zip.ZipException:
ZIP file must have at least one entry
at org.apache.hadoop.streaming.JarBuilder.merge(JarBuilder.java:82)
at
org.apache.hadoop.streaming.StreamJob.packageJobJar(StreamJob.java:707)
at
org.apache.hadoop.streaming.StreamJob.setJobConf(StreamJob.java:948)
at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:127)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3879) yarn script has vestiges of jsvc that need to be cleaned up
[
https://issues.apache.org/jira/browse/MAPREDUCE-3879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3879:
-
Resolution: Duplicate
Status: Resolved (was: Patch Available)
Dup of MAPREDUCE-3578.
yarn script has vestiges of jsvc that need to be cleaned up
---
Key: MAPREDUCE-3879
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3879
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, nodemanager
Affects Versions: 0.23.0
Reporter: Roman Shaposhnik
Assignee: Roman Shaposhnik
Attachments: MAPREDUCE-3879.patch.txt
If nodemanager is started under root (I know, I know). The following is
displayed:
{noformat}
/usr/lib/jvm/java-6-sun/bin/java -Dproc_nodemanager -Xmx1000m -jvm server
-Dhadoop.log.dir=/var/log/yarn -Dyarn.log.dir=/var/log/yarn
-Dhadoop.log.file=yarn-yarn-nodemanager-ahmed-laptop.log
-Dyarn.log.file=yarn-yarn-nodemanager-ahmed-laptop.log
-Dyarn.home.dir=/usr/lib/hadoop -Dhadoop.root.logger=INFO,DRFA
-Dyarn.root.logger=INFO,DRFA -Djava.library.path=/usr/lib/hadoop/lib/native
-classpath
/etc/hadoop/conf:/etc/hadoop/conf:/etc/hadoop/conf:/usr/lib/hadoop/share/hadoop/common/*::/usr/lib/hadoop/*:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/:/usr/lib/hadoop/share/hadoop/hdfs/*:/usr/lib/hadoop/share/hadoop/mapreduce/*:/share/hadoop/mapreduce/*:/usr/lib/hadoop/share/hadoop/mapreduce/*:/usr/lib/hadoop/share/hadoop/mapreduce/lib/*:/etc/hadoop/conf/nm-config/log4j.properties
org.apache.hadoop.yarn.server.nodemanager.NodeManager
Unrecognized option: -jvm
Could not create the Java virtual machine.
{noformat}
The culprit is this bit of code that looks suspiciously like what used to be
in bin/hdfs in support of jsvc launch:
{noformat}
elif [ $COMMAND = nodemanager ] ; then
CLASSPATH=${CLASSPATH}:$YARN_CONF_DIR/nm-config/log4j.properties
CLASS='org.apache.hadoop.yarn.server.nodemanager.NodeManager'
if [[ $EUID -eq 0 ]]; then
YARN_OPTS=$YARN_OPTS -jvm server $YARN_NODEMANAGER_OPTS
else
YARN_OPTS=$YARN_OPTS -server $YARN_NODEMANAGER_OPTS
fi
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3916) various issues with running yarn proxyserver
[
https://issues.apache.org/jira/browse/MAPREDUCE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3916:
-
Attachment: MAPREDUCE-3916.patch
various issues with running yarn proxyserver
Key: MAPREDUCE-3916
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3916
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, resourcemanager, webapps
Affects Versions: 0.23.1
Reporter: Roman Shaposhnik
Assignee: Devaraj K
Attachments: MAPREDUCE-3916.patch
Seem like yarn proxyserver is not operational when running out of the 0.23.1
RC2 tarball.
# Setting yarn.web-proxy.address to match yarn.resourcemanager.address
doesn't disable the proxyserver (althought not setting yarn.web-proxy.address
at all correctly disable it and produces a message:
org.apache.hadoop.yarn.YarnException: yarn.web-proxy.address is not set so
the proxy will not run). This contradicts the documentation provided for
yarn.web-proxy.address in yarn-default.xml
# Setting yarn.web-proxy.address and running the service results in the
following:
{noformat}
$ ./sbin/yarn-daemon.sh start proxyserver
starting proxyserver, logging to
/tmp/hadoop-0.23.1/logs/yarn-rvs-proxyserver-ahmed-laptop.out
/usr/java/64/jdk1.6.0_22/bin/java -Dproc_proxyserver -Xmx1000m
-Dhadoop.log.dir=/tmp/hadoop-0.23.1/logs
-Dyarn.log.dir=/tmp/hadoop-0.23.1/logs
-Dhadoop.log.file=yarn-rvs-proxyserver-ahmed-laptop.log
-Dyarn.log.file=yarn-rvs-proxyserver-ahmed-laptop.log -Dyarn.home.dir=
-Dyarn.id.str=rvs -Dhadoop.root.logger=INFO,DRFA -Dyarn.root.logger=INFO,DRFA
-Djava.library.path=/tmp/hadoop-0.23.1/lib/native
-Dyarn.policy.file=hadoop-policy.xml -Dhadoop.log.dir=/tmp/hadoop-0.23.1/logs
-Dyarn.log.dir=/tmp/hadoop-0.23.1/logs
-Dhadoop.log.file=yarn-rvs-proxyserver-ahmed-laptop.log
-Dyarn.log.file=yarn-rvs-proxyserver-ahmed-laptop.log
-Dyarn.home.dir=/tmp/hadoop-0.23.1 -Dhadoop.root.logger=INFO,DRFA
-Dyarn.root.logger=INFO,DRFA
-Djava.library.path=/tmp/hadoop-0.23.1/lib/native -classpath
/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/share/hadoop/common/lib/*:/tmp/hadoop-0.23.1/share/hadoop/common/*:/tmp/hadoop-0.23.1/share/hadoop/hdfs:/tmp/hadoop-0.23.1/share/hadoop/hdfs/lib/*:/tmp/hadoop-0.23.1/share/hadoop/hdfs/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/lib/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/lib/*
org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer
{noformat}
with the following message found in the logs:
{noformat}
2012-02-24 09:26:31,099 FATAL
org.apache.hadoop.yarn.server.webproxy.WebAppProxy: Could not start proxy web
server
java.io.FileNotFoundException: webapps/proxy not found in CLASSPATH
at
org.apache.hadoop.http.HttpServer.getWebAppsPath(HttpServer.java:532)
at org.apache.hadoop.http.HttpServer.init(HttpServer.java:224)
at org.apache.hadoop.http.HttpServer.init(HttpServer.java:164)
at
org.apache.hadoop.yarn.server.webproxy.WebAppProxy.start(WebAppProxy.java:85)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
at
org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer.main(WebAppProxyServer.java:76)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3916) various issues with running yarn proxyserver
[
https://issues.apache.org/jira/browse/MAPREDUCE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3916:
-
Labels: mrv2 (was: )
Target Version/s: 2.0.0, 3.0.0
Affects Version/s: 3.0.0
2.0.0
Status: Patch Available (was: Open)
{quote}Setting yarn.web-proxy.address to match yarn.resourcemanager.address
doesn't disable the proxyserver (althought not setting yarn.web-proxy.address
at all correctly disable it and produces a message:
org.apache.hadoop.yarn.YarnException: yarn.web-proxy.address is not set so the
proxy will not run). This contradicts the documentation provided for
yarn.web-proxy.address in yarn-default.xml{quote}
If we don't set 'yarn.web-proxy.address', then the proxy server will not start
as separate process and start up fails with the message
'org.apache.hadoop.yarn.YarnException: yarn.web-proxy.address is not set so the
proxy will not run' and proxy will run as part of the RM.
If we give the value for 'yarn.web-proxy.address' which matches to the
'yarn.resourcemanager.address' then proxy server start up fails with Address
already in use. And also it is not starting as part of the RM. It is not as
per the documentation. I have modified the description for this property
accordingly.
{quote}
Setting yarn.web-proxy.address and running the service results in the following:
{quote}
I have added the 'hadoop-yarn-common\src\main\resources\webapps\proxy' dir with
.keep file in it to resolve this problem as done for other components.
various issues with running yarn proxyserver
Key: MAPREDUCE-3916
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3916
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, resourcemanager, webapps
Affects Versions: 0.23.1, 2.0.0, 3.0.0
Reporter: Roman Shaposhnik
Assignee: Devaraj K
Labels: mrv2
Attachments: MAPREDUCE-3916.patch
Seem like yarn proxyserver is not operational when running out of the 0.23.1
RC2 tarball.
# Setting yarn.web-proxy.address to match yarn.resourcemanager.address
doesn't disable the proxyserver (althought not setting yarn.web-proxy.address
at all correctly disable it and produces a message:
org.apache.hadoop.yarn.YarnException: yarn.web-proxy.address is not set so
the proxy will not run). This contradicts the documentation provided for
yarn.web-proxy.address in yarn-default.xml
# Setting yarn.web-proxy.address and running the service results in the
following:
{noformat}
$ ./sbin/yarn-daemon.sh start proxyserver
starting proxyserver, logging to
/tmp/hadoop-0.23.1/logs/yarn-rvs-proxyserver-ahmed-laptop.out
/usr/java/64/jdk1.6.0_22/bin/java -Dproc_proxyserver -Xmx1000m
-Dhadoop.log.dir=/tmp/hadoop-0.23.1/logs
-Dyarn.log.dir=/tmp/hadoop-0.23.1/logs
-Dhadoop.log.file=yarn-rvs-proxyserver-ahmed-laptop.log
-Dyarn.log.file=yarn-rvs-proxyserver-ahmed-laptop.log -Dyarn.home.dir=
-Dyarn.id.str=rvs -Dhadoop.root.logger=INFO,DRFA -Dyarn.root.logger=INFO,DRFA
-Djava.library.path=/tmp/hadoop-0.23.1/lib/native
-Dyarn.policy.file=hadoop-policy.xml -Dhadoop.log.dir=/tmp/hadoop-0.23.1/logs
-Dyarn.log.dir=/tmp/hadoop-0.23.1/logs
-Dhadoop.log.file=yarn-rvs-proxyserver-ahmed-laptop.log
-Dyarn.log.file=yarn-rvs-proxyserver-ahmed-laptop.log
-Dyarn.home.dir=/tmp/hadoop-0.23.1 -Dhadoop.root.logger=INFO,DRFA
-Dyarn.root.logger=INFO,DRFA
-Djava.library.path=/tmp/hadoop-0.23.1/lib/native -classpath
/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/share/hadoop/common/lib/*:/tmp/hadoop-0.23.1/share/hadoop/common/*:/tmp/hadoop-0.23.1/share/hadoop/hdfs:/tmp/hadoop-0.23.1/share/hadoop/hdfs/lib/*:/tmp/hadoop-0.23.1/share/hadoop/hdfs/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/lib/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/lib/*
org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer
{noformat}
with the following message found in the logs:
{noformat}
2012-02-24 09:26:31,099 FATAL
org.apache.hadoop.yarn.server.webproxy.WebAppProxy: Could not start proxy web
server
java.io.FileNotFoundException: webapps/proxy not found in CLASSPATH
at
org.apache.hadoop.http.HttpServer.getWebAppsPath(HttpServer.java:532)
at org.apache.hadoop.http.HttpServer.init(HttpServer.java:224)
at org.apache.hadoop.http.HttpServer.init(HttpServer.java:164)
at
org.apache.hadoop.yarn.server.webproxy.WebAppProxy.start(WebAppProxy.java:85)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
at
[jira] [Updated] (MAPREDUCE-3916) various issues with running yarn proxyserver
[
https://issues.apache.org/jira/browse/MAPREDUCE-3916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3916:
-
Priority: Critical (was: Major)
Target Version/s: 2.0.0, 3.0.0 (was: 3.0.0, 2.0.0)
various issues with running yarn proxyserver
Key: MAPREDUCE-3916
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3916
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, resourcemanager, webapps
Affects Versions: 0.23.1, 2.0.0, 3.0.0
Reporter: Roman Shaposhnik
Assignee: Devaraj K
Priority: Critical
Labels: mrv2
Attachments: MAPREDUCE-3916.patch
Seem like yarn proxyserver is not operational when running out of the 0.23.1
RC2 tarball.
# Setting yarn.web-proxy.address to match yarn.resourcemanager.address
doesn't disable the proxyserver (althought not setting yarn.web-proxy.address
at all correctly disable it and produces a message:
org.apache.hadoop.yarn.YarnException: yarn.web-proxy.address is not set so
the proxy will not run). This contradicts the documentation provided for
yarn.web-proxy.address in yarn-default.xml
# Setting yarn.web-proxy.address and running the service results in the
following:
{noformat}
$ ./sbin/yarn-daemon.sh start proxyserver
starting proxyserver, logging to
/tmp/hadoop-0.23.1/logs/yarn-rvs-proxyserver-ahmed-laptop.out
/usr/java/64/jdk1.6.0_22/bin/java -Dproc_proxyserver -Xmx1000m
-Dhadoop.log.dir=/tmp/hadoop-0.23.1/logs
-Dyarn.log.dir=/tmp/hadoop-0.23.1/logs
-Dhadoop.log.file=yarn-rvs-proxyserver-ahmed-laptop.log
-Dyarn.log.file=yarn-rvs-proxyserver-ahmed-laptop.log -Dyarn.home.dir=
-Dyarn.id.str=rvs -Dhadoop.root.logger=INFO,DRFA -Dyarn.root.logger=INFO,DRFA
-Djava.library.path=/tmp/hadoop-0.23.1/lib/native
-Dyarn.policy.file=hadoop-policy.xml -Dhadoop.log.dir=/tmp/hadoop-0.23.1/logs
-Dyarn.log.dir=/tmp/hadoop-0.23.1/logs
-Dhadoop.log.file=yarn-rvs-proxyserver-ahmed-laptop.log
-Dyarn.log.file=yarn-rvs-proxyserver-ahmed-laptop.log
-Dyarn.home.dir=/tmp/hadoop-0.23.1 -Dhadoop.root.logger=INFO,DRFA
-Dyarn.root.logger=INFO,DRFA
-Djava.library.path=/tmp/hadoop-0.23.1/lib/native -classpath
/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/etc/hadoop:/tmp/hadoop-0.23.1/share/hadoop/common/lib/*:/tmp/hadoop-0.23.1/share/hadoop/common/*:/tmp/hadoop-0.23.1/share/hadoop/hdfs:/tmp/hadoop-0.23.1/share/hadoop/hdfs/lib/*:/tmp/hadoop-0.23.1/share/hadoop/hdfs/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/lib/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/*:/tmp/hadoop-0.23.1/share/hadoop/mapreduce/lib/*
org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer
{noformat}
with the following message found in the logs:
{noformat}
2012-02-24 09:26:31,099 FATAL
org.apache.hadoop.yarn.server.webproxy.WebAppProxy: Could not start proxy web
server
java.io.FileNotFoundException: webapps/proxy not found in CLASSPATH
at
org.apache.hadoop.http.HttpServer.getWebAppsPath(HttpServer.java:532)
at org.apache.hadoop.http.HttpServer.init(HttpServer.java:224)
at org.apache.hadoop.http.HttpServer.init(HttpServer.java:164)
at
org.apache.hadoop.yarn.server.webproxy.WebAppProxy.start(WebAppProxy.java:85)
at
org.apache.hadoop.yarn.service.CompositeService.start(CompositeService.java:68)
at
org.apache.hadoop.yarn.server.webproxy.WebAppProxyServer.main(WebAppProxyServer.java:76)
{noformat}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4076) Stream job fails with ZipException when use yarn jar command
[
https://issues.apache.org/jira/browse/MAPREDUCE-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4076:
-
Attachment: MAPREDUCE-4076.patch
Stream job fails with ZipException when use yarn jar command
Key: MAPREDUCE-4076
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4076
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Critical
Attachments: MAPREDUCE-4076.patch
Stream job fails with ZipException when use yarn jar command and executes
successfully with hadoop jar command.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./yarn jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output /test/output/1 -mapper cat -reducer wc
packageJobJar: []
[/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin/$%7Bhadoop.home.dir%7D/hadoop-$%7Buser.name%7D/hadoop-unjar4241129353499211360/]
/tmp/streamjob7683981905208294893.jar tmpDir=null
Exception in thread main java.io.IOException: java.util.zip.ZipException:
ZIP file must have at least one entry
at org.apache.hadoop.streaming.JarBuilder.merge(JarBuilder.java:82)
at
org.apache.hadoop.streaming.StreamJob.packageJobJar(StreamJob.java:707)
at
org.apache.hadoop.streaming.StreamJob.setJobConf(StreamJob.java:948)
at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:127)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4076) Stream job fails with ZipException when use yarn jar command
[
https://issues.apache.org/jira/browse/MAPREDUCE-4076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4076:
-
Status: Patch Available (was: Open)
When we use 'yarn jar' command, RunJar.java tries to create temp directory if
doesn't exist using configuration property hadoop.tmp.dir. When it gets from
the conf object, it will get the value as
${hadoop.home.dir}/hadoop-${user.name}. Here these vars are not replaced with
system properties because of unavailability of 'hadoop.home.dir' system
property. It will create the temp dir with the same name(i.e
${hadoop.home.dir}/hadoop-${user.name}) in the current dir.
StreamJob unjars and keeps classes in the directory
current-dir/${hadoop.home.dir}/hadoop-${user.name}, and then it tries to find
org/apache/hadoop/streaming/StreamJob.class in the classpath and it gets the
path as
curent-dir/$%7Bhadoop.home.dir%7D/hadoop-$%7Buser.name%7D/hadoop-unjar8421477351848586067/
due to special chars in the directory name. And finally fails to merge from
this path to the job jar file.
If we do the same with 'hadoop jar', it will get the prop as
$HADOOP_HOME/hadoop-username which is replaced with 'hadoop.home.dir' and
'user.name' properties , it will create the temp dir properly uses the same for
other things to do and works fine.
I have attached the patch to address the above problem by adding the
hadoop.home.dir system property in yarn file.
Stream job fails with ZipException when use yarn jar command
Key: MAPREDUCE-4076
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4076
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Critical
Attachments: MAPREDUCE-4076.patch
Stream job fails with ZipException when use yarn jar command and executes
successfully with hadoop jar command.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./yarn jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output /test/output/1 -mapper cat -reducer wc
packageJobJar: []
[/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin/$%7Bhadoop.home.dir%7D/hadoop-$%7Buser.name%7D/hadoop-unjar4241129353499211360/]
/tmp/streamjob7683981905208294893.jar tmpDir=null
Exception in thread main java.io.IOException: java.util.zip.ZipException:
ZIP file must have at least one entry
at org.apache.hadoop.streaming.JarBuilder.merge(JarBuilder.java:82)
at
org.apache.hadoop.streaming.StreamJob.packageJobJar(StreamJob.java:707)
at
org.apache.hadoop.streaming.StreamJob.setJobConf(StreamJob.java:948)
at org.apache.hadoop.streaming.StreamJob.run(StreamJob.java:127)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at
org.apache.hadoop.streaming.HadoopStreaming.main(HadoopStreaming.java:50)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.apache.hadoop.util.RunJar.main(RunJar.java:200)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4074) Client continuously retries to RM When RM goes down before launching Application Master
[
https://issues.apache.org/jira/browse/MAPREDUCE-4074?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4074:
-
Summary: Client continuously retries to RM When RM goes down before
launching Application Master (was: Client continuosly reties to RM When RM
goes down before launching Application Master)
Client continuously retries to RM When RM goes down before launching
Application Master
---
Key: MAPREDUCE-4074
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4074
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.23.1
Reporter: Devaraj K
Client continuously tries to RM and logs the below messages when the RM goes
down before launching App Master.
I feel exception should be thrown or break the loop after finite no of
retries.
{code:xml}
28/03/12 07:15:03 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 0 time(s).
28/03/12 07:15:04 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 1 time(s).
28/03/12 07:15:05 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 2 time(s).
28/03/12 07:15:06 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 3 time(s).
28/03/12 07:15:07 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 4 time(s).
28/03/12 07:15:08 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 5 time(s).
28/03/12 07:15:09 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 6 time(s).
28/03/12 07:15:10 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 7 time(s).
28/03/12 07:15:11 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 8 time(s).
28/03/12 07:15:12 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 9 time(s).
28/03/12 07:15:13 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 0 time(s).
28/03/12 07:15:14 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 1 time(s).
28/03/12 07:15:15 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 2 time(s).
28/03/12 07:15:16 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 3 time(s).
28/03/12 07:15:17 INFO ipc.Client: Retrying connect to server:
linux-f330.site/10.18.40.182:8032. Already tried 4 time(s).
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2237) Lost heartbeat response containing MapTask throws NPE when it is resent
[ https://issues.apache.org/jira/browse/MAPREDUCE-2237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-2237: - Resolution: Not A Problem Status: Resolved (was: Patch Available) It is no more problem in active versions. Lost heartbeat response containing MapTask throws NPE when it is resent --- Key: MAPREDUCE-2237 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2237 Project: Hadoop Map/Reduce Issue Type: Bug Components: jobtracker Affects Versions: 0.20.2, 0.21.0, 0.22.0 Reporter: Todd Lipcon Assignee: Devaraj K Fix For: 0.24.0 Attachments: MAPREDUCE-2237-0.20.patch, MAPREDUCE-2237-0.23.1.patch, MAPREDUCE-2237-0.23.2.patch, MAPREDUCE-2237-0.23.patch When the JT sends a heartbeat response, it records it in trackerToHeartbeatResponseMap. But after MapTask writes its input split, it sets that split to null (assumedly to save memory?). So, if the heartbeat response is lost, and the JT needs to resend it, it will throw NPE since the split information has been lost. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4077) Issues while using Hadoop Streaming job
[
https://issues.apache.org/jira/browse/MAPREDUCE-4077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4077:
-
Summary: Issues while using Hadoop Streaming job (was: Issue while using
Hadoop Streaming job)
Issues while using Hadoop Streaming job
---
Key: MAPREDUCE-4077
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4077
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
When we use -file option it says deprecated and use -files.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop
jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output /test/output/3 -mapper cat -reducer wc -file hadoop
02/02/19 10:55:51 WARN streaming.StreamJob: -file option is deprecated,
please use generic option -files instead.
{code}
But when we use -files option, it says unrecognized option.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop
jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output
/test/output/3 -mapper cat -reducer wc -files hadoop
02/02/19 10:56:42 ERROR streaming.StreamJob: Unrecognized option: -files
Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
{code}
When we use -archives option, it says unrecognized option.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop
jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output
/test/output/3 -mapper cat -reducer wc -archives testarchive.rar
02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
{code}
But in the options it will display the usage of the -archives.
{code:xml}
linux-f330:/home/devaraj/hadoop/trunk/hadoop-0.24.0-SNAPSHOT/bin # ./hadoop
jar
../share/hadoop/tools/lib/hadoop-streaming-0.24.0-SNAPSHOT.jar -input /hadoop
-output
/test/output/3 -mapper cat -reducer wc -archives testarchive.rar
02/02/19 11:05:43 ERROR streaming.StreamJob: Unrecognized option: -archives
Usage: $HADOOP_PREFIX/bin/hadoop jar hadoop-streaming.jar [options]
..
..
-libjars comma separated list of jarsspecify comma separated jar files
to include in the classpath.
-archives comma separated list of archivesspecify comma separated
archives to be unarchived on the compute machines.
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2548) Log improvements in DBOutputFormat.java and CounterGroup.java
[
https://issues.apache.org/jira/browse/MAPREDUCE-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-2548:
-
Attachment: MAPREDUCE-2548-1.patch
Log improvements in DBOutputFormat.java and CounterGroup.java
-
Key: MAPREDUCE-2548
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2548
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Devaraj K
Assignee: Devaraj K
Fix For: 0.24.0
Attachments: MAPREDUCE-2548-1.patch, MAPREDUCE-2548.patch
1. Instead of the printing the stack trace on the console, It can be logged.
{code:title=DBOutputFormat.java|borderStyle=solid}
public void write(K key, V value) throws IOException {
try {
key.write(statement);
statement.addBatch();
} catch (SQLException e) {
e.printStackTrace();
}
}
{code}
2. Missing resource information can be logged.
{code:title=CounterGroup .java|borderStyle=solid}
protected CounterGroup(String name) {
this.name = name;
try {
bundle = getResourceBundle(name);
}
catch (MissingResourceException neverMind) {
}
displayName = localize(CounterGroupName, name);
}
private String localize(String key, String defaultValue) {
String result = defaultValue;
if (bundle != null) {
try {
result = bundle.getString(key);
}
catch (MissingResourceException mre) {
}
}
return result;
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2548) Log improvements in DBOutputFormat.java and CounterGroup.java
[
https://issues.apache.org/jira/browse/MAPREDUCE-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-2548:
-
Status: Open (was: Patch Available)
Log improvements in DBOutputFormat.java and CounterGroup.java
-
Key: MAPREDUCE-2548
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2548
Project: Hadoop Map/Reduce
Issue Type: Improvement
Reporter: Devaraj K
Assignee: Devaraj K
Fix For: 0.24.0
Attachments: MAPREDUCE-2548-1.patch, MAPREDUCE-2548.patch
1. Instead of the printing the stack trace on the console, It can be logged.
{code:title=DBOutputFormat.java|borderStyle=solid}
public void write(K key, V value) throws IOException {
try {
key.write(statement);
statement.addBatch();
} catch (SQLException e) {
e.printStackTrace();
}
}
{code}
2. Missing resource information can be logged.
{code:title=CounterGroup .java|borderStyle=solid}
protected CounterGroup(String name) {
this.name = name;
try {
bundle = getResourceBundle(name);
}
catch (MissingResourceException neverMind) {
}
displayName = localize(CounterGroupName, name);
}
private String localize(String key, String defaultValue) {
String result = defaultValue;
if (bundle != null) {
try {
result = bundle.getString(key);
}
catch (MissingResourceException mre) {
}
}
return result;
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-2548) Log improvements in DBOutputFormat.java and CounterGroup.java
[
https://issues.apache.org/jira/browse/MAPREDUCE-2548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-2548:
-
Fix Version/s: (was: 0.24.0)
Affects Version/s: 2.0.0
Status: Patch Available (was: Open)
Thank you harsh for taking look into the patch. Sorry for the late response.
There is no issue with CounterGroup.java in the latest. I have updated the
message in DBOutputFormat.java.
Can you please have look into the updated patch?
Log improvements in DBOutputFormat.java and CounterGroup.java
-
Key: MAPREDUCE-2548
URL: https://issues.apache.org/jira/browse/MAPREDUCE-2548
Project: Hadoop Map/Reduce
Issue Type: Improvement
Affects Versions: 2.0.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-2548-1.patch, MAPREDUCE-2548.patch
1. Instead of the printing the stack trace on the console, It can be logged.
{code:title=DBOutputFormat.java|borderStyle=solid}
public void write(K key, V value) throws IOException {
try {
key.write(statement);
statement.addBatch();
} catch (SQLException e) {
e.printStackTrace();
}
}
{code}
2. Missing resource information can be logged.
{code:title=CounterGroup .java|borderStyle=solid}
protected CounterGroup(String name) {
this.name = name;
try {
bundle = getResourceBundle(name);
}
catch (MissingResourceException neverMind) {
}
displayName = localize(CounterGroupName, name);
}
private String localize(String key, String defaultValue) {
String result = defaultValue;
if (bundle != null) {
try {
result = bundle.getString(key);
}
catch (MissingResourceException mre) {
}
}
return result;
}
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4064) Job History Link in RM UI is redirecting to the URL which contains Job Id twice
[
https://issues.apache.org/jira/browse/MAPREDUCE-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4064:
-
Attachment: MAPREDUCE-4064-1.patch
Job History Link in RM UI is redirecting to the URL which contains Job Id
twice
---
Key: MAPREDUCE-4064
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4064
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4064-1.patch, MAPREDUCE-4064.patch
{code:xml}
http://xx.x.x.x:19888/jobhistory/job/job_1332435449546_0001/jobhistory/job/job_1332435449546_0001
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4064) Job History Link in RM UI is redirecting to the URL which contains Job Id twice
[
https://issues.apache.org/jira/browse/MAPREDUCE-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4064:
-
Status: Patch Available (was: Open)
Thank you Robert for looking into the patch.
Before posting I tested the patch including all the links in the AM/History UI
but I couldn't find out the cosmetic problems(caused by previous patch) in the
AM UI due to the issue MAPREDUCE-3173.
Now I have updated the patch with the change applies to only redirected history
URL. Can you please review this patch?
Job History Link in RM UI is redirecting to the URL which contains Job Id
twice
---
Key: MAPREDUCE-4064
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4064
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4064-1.patch, MAPREDUCE-4064.patch
{code:xml}
http://xx.x.x.x:19888/jobhistory/job/job_1332435449546_0001/jobhistory/job/job_1332435449546_0001
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4064) Job History Link in RM UI is redirecting to the URL which contains Job Id twice
[
https://issues.apache.org/jira/browse/MAPREDUCE-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4064:
-
Attachment: MAPREDUCE-4064.patch
Job History Link in RM UI is redirecting to the URL which contains Job Id
twice
---
Key: MAPREDUCE-4064
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4064
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4064.patch
{code:xml}
http://xx.x.x.x:19888/jobhistory/job/job_1332435449546_0001/jobhistory/job/job_1332435449546_0001
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4064) Job History Link in RM UI is redirecting to the URL which contains Job Id twice
[
https://issues.apache.org/jira/browse/MAPREDUCE-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4064:
-
Status: Patch Available (was: Open)
I have attached patch to fix this. Please review this.
Job History Link in RM UI is redirecting to the URL which contains Job Id
twice
---
Key: MAPREDUCE-4064
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4064
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4064.patch
{code:xml}
http://xx.x.x.x:19888/jobhistory/job/job_1332435449546_0001/jobhistory/job/job_1332435449546_0001
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3947) yarn.app.mapreduce.am.resource.mb not documented
[ https://issues.apache.org/jira/browse/MAPREDUCE-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3947: - Attachment: MAPREDUCE-3947.patch yarn.app.mapreduce.am.resource.mb not documented Key: MAPREDUCE-3947 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3947 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.24.0, 0.23.3 Reporter: Todd Lipcon Priority: Minor Labels: newbie Attachments: MAPREDUCE-3947.patch This configuration is useful but doesn't appear to be documented anywhere. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3947) yarn.app.mapreduce.am.resource.mb not documented
[ https://issues.apache.org/jira/browse/MAPREDUCE-3947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3947: - Labels: mrv2 (was: newbie) Status: Patch Available (was: Open) I have added this property to mapred-default.xml file. yarn.app.mapreduce.am.resource.mb not documented Key: MAPREDUCE-3947 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3947 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.24.0, 0.23.3 Reporter: Todd Lipcon Priority: Minor Labels: mrv2 Attachments: MAPREDUCE-3947.patch This configuration is useful but doesn't appear to be documented anywhere. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4064) Job History Link in RM UI is redirecting to the URL which contains Job Id twice
[
https://issues.apache.org/jira/browse/MAPREDUCE-4064?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4064:
-
Component/s: mrv2
Job History Link in RM UI is redirecting to the URL which contains Job Id
twice
---
Key: MAPREDUCE-4064
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4064
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
{code:xml}
http://xx.x.x.x:19888/jobhistory/job/job_1332435449546_0001/jobhistory/job/job_1332435449546_0001
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4054) Fix issues in start-all.sh and stop-all.sh
[ https://issues.apache.org/jira/browse/MAPREDUCE-4054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-4054: - Component/s: (was: mrv2) Description: 1. Warning message Execute start-all.sh and stop-all.sh scripts it displays message like This script is Deprecated. Instead use start-dfs.sh/stop-dfs.sh and start-mapred.sh/stop-mapred.sh. Instead of start-mapred.sh/stop-mapred.sh it should be start-yarn.sh/stop-yarn.sh. 2. start/stop hdfs 3. start/stop yarn was:Execute start-all.sh and stop-all.sh scripts it displays message like This script is Deprecated. Instead use start-dfs.sh/stop-dfs.sh and start-mapred.sh/stop-mapred.sh. Instead of start-mapred.sh/stop-mapred.sh it should be start-yarn.sh/stop-yarn.sh. Priority: Major (was: Trivial) Assignee: Devaraj K Summary: Fix issues in start-all.sh and stop-all.sh (was: Executing start-all.sh and stop-all.sh scripts not displaying proper message) Fix issues in start-all.sh and stop-all.sh -- Key: MAPREDUCE-4054 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4054 Project: Hadoop Map/Reduce Issue Type: Bug Affects Versions: 0.24.0 Reporter: Nishan Shetty Assignee: Devaraj K 1. Warning message Execute start-all.sh and stop-all.sh scripts it displays message like This script is Deprecated. Instead use start-dfs.sh/stop-dfs.sh and start-mapred.sh/stop-mapred.sh. Instead of start-mapred.sh/stop-mapred.sh it should be start-yarn.sh/stop-yarn.sh. 2. start/stop hdfs 3. start/stop yarn -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4047) NullPointerException during the map task merge phase
[
https://issues.apache.org/jira/browse/MAPREDUCE-4047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4047:
-
Affects Version/s: 0.23.1
Assignee: Devaraj K
NullPointerException during the map task merge phase
Key: MAPREDUCE-4047
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4047
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
While executing the tera sort with 1TB data, got the below exception
{code:xml}
2012-03-20 21:02:18,526 FATAL [IPC Server handler 14 on 31776]
org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task:
attempt_1332261815858_0002_m_003124_0 - exited :
java.lang.NullPointerException
at org.apache.hadoop.mapred.IFile$Writer.init(IFile.java:135)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.mergeParts(MapTask.java:1845)
at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.flush(MapTask.java:1438)
at
org.apache.hadoop.mapred.MapTask$NewOutputCollector.close(MapTask.java:663)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:730)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:332)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:147)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:142)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-4008) ResourceManager throws MetricsException on start up saying QueueMetrics MBean already exists
[
https://issues.apache.org/jira/browse/MAPREDUCE-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4008:
-
Attachment: MAPREDUCE-4008.patch
ResourceManager throws MetricsException on start up saying QueueMetrics MBean
already exists
Key: MAPREDUCE-4008
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4008
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, resourcemanager
Affects Versions: 0.24.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4008.patch
{code:xml}
2012-03-14 15:22:23,089 WARN org.apache.hadoop.metrics2.util.MBeans: Error
creating MBean object name:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default
org.apache.hadoop.metrics2.MetricsException:
org.apache.hadoop.metrics2.MetricsException:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:117)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newMBeanName(DefaultMetricsSystem.java:102)
at org.apache.hadoop.metrics2.util.MBeans.getMBeanName(MBeans.java:91)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:55)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:218)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:93)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:243)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:288)
at $Proxy6.postStart(Unknown Source)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:183)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:155)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:54)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.start(ResourceManager.java:454)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:588)
Caused by: org.apache.hadoop.metrics2.MetricsException:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:113)
... 19 more
2012-03-14 15:22:23,090 WARN org.apache.hadoop.metrics2.util.MBeans: Failed
to register MBean null
javax.management.RuntimeOperationsException: Exception occurred trying to
register the MBean
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:969)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:57)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:218)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:93)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:243)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:288)
at $Proxy6.postStart(Unknown Source)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:183)
at
[jira] [Updated] (MAPREDUCE-4008) ResourceManager throws MetricsException on start up saying QueueMetrics MBean already exists
[
https://issues.apache.org/jira/browse/MAPREDUCE-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4008:
-
Status: Patch Available (was: Open)
@Jason, You are right, It happens only for the fifo scheduler . In the
FifoScheduler.setConf() method it was registering default QueueMetrics. This
setConf method is getting called twice(as part of scheduler instance creation
and reinitialize()) and causing the exception logging.
I have moved the default QueueMetrics registration to reinitialize() to call
only once..
ResourceManager throws MetricsException on start up saying QueueMetrics MBean
already exists
Key: MAPREDUCE-4008
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4008
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, resourcemanager
Affects Versions: 0.24.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4008.patch
{code:xml}
2012-03-14 15:22:23,089 WARN org.apache.hadoop.metrics2.util.MBeans: Error
creating MBean object name:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default
org.apache.hadoop.metrics2.MetricsException:
org.apache.hadoop.metrics2.MetricsException:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:117)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newMBeanName(DefaultMetricsSystem.java:102)
at org.apache.hadoop.metrics2.util.MBeans.getMBeanName(MBeans.java:91)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:55)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:218)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:93)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:243)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:288)
at $Proxy6.postStart(Unknown Source)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:183)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:155)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:54)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.start(ResourceManager.java:454)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:588)
Caused by: org.apache.hadoop.metrics2.MetricsException:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:113)
... 19 more
2012-03-14 15:22:23,090 WARN org.apache.hadoop.metrics2.util.MBeans: Failed
to register MBean null
javax.management.RuntimeOperationsException: Exception occurred trying to
register the MBean
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:969)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:57)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:218)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:93)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:243)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
[jira] [Updated] (MAPREDUCE-4008) ResourceManager throws MetricsException on start up saying QueueMetrics MBean already exists
[
https://issues.apache.org/jira/browse/MAPREDUCE-4008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-4008:
-
Component/s: scheduler
ResourceManager throws MetricsException on start up saying QueueMetrics MBean
already exists
Key: MAPREDUCE-4008
URL: https://issues.apache.org/jira/browse/MAPREDUCE-4008
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, resourcemanager, scheduler
Affects Versions: 0.24.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-4008.patch
{code:xml}
2012-03-14 15:22:23,089 WARN org.apache.hadoop.metrics2.util.MBeans: Error
creating MBean object name:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default
org.apache.hadoop.metrics2.MetricsException:
org.apache.hadoop.metrics2.MetricsException:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:117)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newMBeanName(DefaultMetricsSystem.java:102)
at org.apache.hadoop.metrics2.util.MBeans.getMBeanName(MBeans.java:91)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:55)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:218)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:93)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:243)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:288)
at $Proxy6.postStart(Unknown Source)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:183)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:155)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:54)
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.start(ResourceManager.java:454)
at
org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.main(ResourceManager.java:588)
Caused by: org.apache.hadoop.metrics2.MetricsException:
Hadoop:service=ResourceManager,name=QueueMetrics,q0=default already exists!
at
org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newObjectName(DefaultMetricsSystem.java:113)
... 19 more
2012-03-14 15:22:23,090 WARN org.apache.hadoop.metrics2.util.MBeans: Failed
to register MBean null
javax.management.RuntimeOperationsException: Exception occurred trying to
register the MBean
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:969)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
at
com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
at
com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:57)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.startMBeans(MetricsSourceAdapter.java:218)
at
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter.start(MetricsSourceAdapter.java:93)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.registerSource(MetricsSystemImpl.java:243)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$1.postStart(MetricsSystemImpl.java:227)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl$3.invoke(MetricsSystemImpl.java:288)
at $Proxy6.postStart(Unknown Source)
at
org.apache.hadoop.metrics2.impl.MetricsSystemImpl.start(MetricsSystemImpl.java:183)
at
[jira] [Updated] (MAPREDUCE-3869) Distributed shell application fails with NoClassDefFoundError
[
https://issues.apache.org/jira/browse/MAPREDUCE-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3869:
-
Status: Patch Available (was: Open)
Distributed shell application fails with NoClassDefFoundError
-
Key: MAPREDUCE-3869
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3869
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Blocker
Fix For: 0.23.2
Attachments: MAPREDUCE-3869-1.patch, MAPREDUCE-3869-2.patch,
MAPREDUCE-3869.patch
Distributed shell application always fails to start the application master
with the following error.
\\
{code:xml}
12/02/16 05:35:25 FATAL distributedshell.ApplicationMaster: Error running
ApplicationMaster
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/ipc/YarnRPC
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.init(ApplicationMaster.java:252)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.main(ApplicationMaster.java:195)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.ipc.YarnRPC
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
... 2 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3869) Distributed shell application fails with NoClassDefFoundError
[
https://issues.apache.org/jira/browse/MAPREDUCE-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3869:
-
Status: Open (was: Patch Available)
Distributed shell application fails with NoClassDefFoundError
-
Key: MAPREDUCE-3869
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3869
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Blocker
Fix For: 0.23.2
Attachments: MAPREDUCE-3869-1.patch, MAPREDUCE-3869-2.patch,
MAPREDUCE-3869.patch
Distributed shell application always fails to start the application master
with the following error.
\\
{code:xml}
12/02/16 05:35:25 FATAL distributedshell.ApplicationMaster: Error running
ApplicationMaster
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/ipc/YarnRPC
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.init(ApplicationMaster.java:252)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.main(ApplicationMaster.java:195)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.ipc.YarnRPC
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
... 2 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3869) Distributed shell application fails with NoClassDefFoundError
[
https://issues.apache.org/jira/browse/MAPREDUCE-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3869:
-
Attachment: MAPREDUCE-3869-2.patch
Distributed shell application fails with NoClassDefFoundError
-
Key: MAPREDUCE-3869
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3869
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Blocker
Fix For: 0.23.2
Attachments: MAPREDUCE-3869-1.patch, MAPREDUCE-3869-2.patch,
MAPREDUCE-3869.patch
Distributed shell application always fails to start the application master
with the following error.
\\
{code:xml}
12/02/16 05:35:25 FATAL distributedshell.ApplicationMaster: Error running
ApplicationMaster
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/ipc/YarnRPC
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.init(ApplicationMaster.java:252)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.main(ApplicationMaster.java:195)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.ipc.YarnRPC
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
... 2 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3869) Distributed shell application fails with NoClassDefFoundError
[
https://issues.apache.org/jira/browse/MAPREDUCE-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3869:
-
Attachment: MAPREDUCE-3869-1.patch
Distributed shell application fails with NoClassDefFoundError
-
Key: MAPREDUCE-3869
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3869
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Blocker
Fix For: 0.23.2
Attachments: MAPREDUCE-3869-1.patch, MAPREDUCE-3869.patch
Distributed shell application always fails to start the application master
with the following error.
\\
{code:xml}
12/02/16 05:35:25 FATAL distributedshell.ApplicationMaster: Error running
ApplicationMaster
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/ipc/YarnRPC
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.init(ApplicationMaster.java:252)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.main(ApplicationMaster.java:195)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.ipc.YarnRPC
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
... 2 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3869) Distributed shell application fails with NoClassDefFoundError
[
https://issues.apache.org/jira/browse/MAPREDUCE-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3869:
-
Status: Patch Available (was: Open)
Thanks Hitesh and Vinod for looking into the patch.
I'm sorry for the late response.
I have updated the patch with the suggested change.
Distributed shell application fails with NoClassDefFoundError
-
Key: MAPREDUCE-3869
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3869
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.1
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Blocker
Fix For: 0.23.2
Attachments: MAPREDUCE-3869-1.patch, MAPREDUCE-3869.patch
Distributed shell application always fails to start the application master
with the following error.
\\
{code:xml}
12/02/16 05:35:25 FATAL distributedshell.ApplicationMaster: Error running
ApplicationMaster
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/ipc/YarnRPC
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.init(ApplicationMaster.java:252)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.main(ApplicationMaster.java:195)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.ipc.YarnRPC
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
... 2 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3348) mapred job -status fails to give info even if the job is present in History
[
https://issues.apache.org/jira/browse/MAPREDUCE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3348:
-
Attachment: MAPREDUCE-3348-1.patch
mapred job -status fails to give info even if the job is present in History
---
Key: MAPREDUCE-3348
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3348
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Devaraj K
Assignee: Devaraj K
Fix For: 0.23.2
Attachments: MAPREDUCE-3348-1.patch, MAPREDUCE-3348.patch
It is trying to get the app report from the RM for the job, RM throws
exception when it doesn't find and then it is giving the same exception
without trying from History Server.
{code}
11/11/03 08:47:27 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy
for protocol interface org.apache.hadoop.mapred
uce.v2.api.MRClientProtocol
11/11/03 08:47:28 WARN mapred.ClientServiceDelegate: Exception thrown by
remote end.
RemoteTrace:
at LocalTrace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl:
Trying to get information for an absent applicat
ion
application_1320278804241_0002
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:142)
at $Proxy6.getApplicationReport(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getApplicationReport(ClientRMProtocolPBClie
ntImpl.java:111)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getApplicationReport(ResourceMgrDelegate.java:321)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:137)
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:273)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:353)
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:429)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:240)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1106)
Exception in thread main RemoteTrace:
at Local Trace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl:
Trying to get information for an absent applicat
ion
application_1320278804241_0002
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:142)
at $Proxy6.getApplicationReport(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getApplicationReport(ClientRMProtocolPBClie
ntImpl.java:111)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getApplicationReport(ResourceMgrDelegate.java:321)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:137)
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:273)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:353)
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:429)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:240)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1106)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3348) mapred job -status fails to give info even if the job is present in History
[
https://issues.apache.org/jira/browse/MAPREDUCE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3348:
-
Status: Patch Available (was: Open)
Thanks Vinod for taking a look into the patch. I have addressed the comments in
the updated patch.
I'm sorry for the late response.
mapred job -status fails to give info even if the job is present in History
---
Key: MAPREDUCE-3348
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3348
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.23.0
Reporter: Devaraj K
Assignee: Devaraj K
Fix For: 0.23.2
Attachments: MAPREDUCE-3348-1.patch, MAPREDUCE-3348.patch
It is trying to get the app report from the RM for the job, RM throws
exception when it doesn't find and then it is giving the same exception
without trying from History Server.
{code}
11/11/03 08:47:27 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy
for protocol interface org.apache.hadoop.mapred
uce.v2.api.MRClientProtocol
11/11/03 08:47:28 WARN mapred.ClientServiceDelegate: Exception thrown by
remote end.
RemoteTrace:
at LocalTrace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl:
Trying to get information for an absent applicat
ion
application_1320278804241_0002
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:142)
at $Proxy6.getApplicationReport(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getApplicationReport(ClientRMProtocolPBClie
ntImpl.java:111)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getApplicationReport(ResourceMgrDelegate.java:321)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:137)
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:273)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:353)
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:429)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:240)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1106)
Exception in thread main RemoteTrace:
at Local Trace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl:
Trying to get information for an absent applicat
ion
application_1320278804241_0002
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:142)
at $Proxy6.getApplicationReport(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getApplicationReport(ClientRMProtocolPBClie
ntImpl.java:111)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getApplicationReport(ResourceMgrDelegate.java:321)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:137)
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:273)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:353)
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:429)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:240)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1106)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3869) Distributed shell application fails with NoClassDefFoundError
[
https://issues.apache.org/jira/browse/MAPREDUCE-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3869:
-
Attachment: MAPREDUCE-3869.patch
Distributed shell application fails with NoClassDefFoundError
-
Key: MAPREDUCE-3869
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3869
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.24.0, 0.23.2
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Blocker
Attachments: MAPREDUCE-3869.patch
Distributed shell application always fails to start the application master
with the following error.
\\
{code:xml}
12/02/16 05:35:25 FATAL distributedshell.ApplicationMaster: Error running
ApplicationMaster
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/ipc/YarnRPC
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.init(ApplicationMaster.java:252)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.main(ApplicationMaster.java:195)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.ipc.YarnRPC
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
... 2 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3869) Distributed shell application fails with NoClassDefFoundError
[
https://issues.apache.org/jira/browse/MAPREDUCE-3869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3869:
-
Status: Patch Available (was: Open)
Here client is setting the classpath environment with $YARN_HOME/modules/* and
$YARN_HOME/lib/* but those are not present and causing the
NoClassDefFoundError.
I have provided trivial and direct patch which fixes this issue.
Distributed shell application fails with NoClassDefFoundError
-
Key: MAPREDUCE-3869
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3869
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.24.0, 0.23.2
Reporter: Devaraj K
Assignee: Devaraj K
Priority: Blocker
Attachments: MAPREDUCE-3869.patch
Distributed shell application always fails to start the application master
with the following error.
\\
{code:xml}
12/02/16 05:35:25 FATAL distributedshell.ApplicationMaster: Error running
ApplicationMaster
java.lang.NoClassDefFoundError: org/apache/hadoop/yarn/ipc/YarnRPC
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.init(ApplicationMaster.java:252)
at
org.apache.hadoop.yarn.applications.distributedshell.ApplicationMaster.main(ApplicationMaster.java:195)
Caused by: java.lang.ClassNotFoundException:
org.apache.hadoop.yarn.ipc.YarnRPC
at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
... 2 more
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3348) mapred job -status fails to give info even if the job is present in History
[
https://issues.apache.org/jira/browse/MAPREDUCE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3348:
-
Attachment: MAPREDUCE-3348.patch
mapred job -status fails to give info even if the job is present in History
---
Key: MAPREDUCE-3348
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3348
Project: Hadoop Map/Reduce
Issue Type: Bug
Affects Versions: 0.24.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-3348.patch
It is trying to get the app report from the RM for the job, RM throws
exception when it doesn't find and then it is giving the same exception
without trying from History Server.
{code}
11/11/03 08:47:27 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy
for protocol interface org.apache.hadoop.mapred
uce.v2.api.MRClientProtocol
11/11/03 08:47:28 WARN mapred.ClientServiceDelegate: Exception thrown by
remote end.
RemoteTrace:
at LocalTrace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl:
Trying to get information for an absent applicat
ion
application_1320278804241_0002
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:142)
at $Proxy6.getApplicationReport(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getApplicationReport(ClientRMProtocolPBClie
ntImpl.java:111)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getApplicationReport(ResourceMgrDelegate.java:321)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:137)
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:273)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:353)
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:429)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:240)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1106)
Exception in thread main RemoteTrace:
at Local Trace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl:
Trying to get information for an absent applicat
ion
application_1320278804241_0002
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:142)
at $Proxy6.getApplicationReport(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getApplicationReport(ClientRMProtocolPBClie
ntImpl.java:111)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getApplicationReport(ResourceMgrDelegate.java:321)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:137)
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:273)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:353)
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:429)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:240)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1106)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3348) mapred job -status fails to give info even if the job is present in History
[
https://issues.apache.org/jira/browse/MAPREDUCE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3348:
-
Status: Patch Available (was: Open)
mapred job -status fails to give info even if the job is present in History
---
Key: MAPREDUCE-3348
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3348
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.24.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-3348.patch
It is trying to get the app report from the RM for the job, RM throws
exception when it doesn't find and then it is giving the same exception
without trying from History Server.
{code}
11/11/03 08:47:27 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy
for protocol interface org.apache.hadoop.mapred
uce.v2.api.MRClientProtocol
11/11/03 08:47:28 WARN mapred.ClientServiceDelegate: Exception thrown by
remote end.
RemoteTrace:
at LocalTrace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl:
Trying to get information for an absent applicat
ion
application_1320278804241_0002
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:142)
at $Proxy6.getApplicationReport(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getApplicationReport(ClientRMProtocolPBClie
ntImpl.java:111)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getApplicationReport(ResourceMgrDelegate.java:321)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:137)
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:273)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:353)
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:429)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:240)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1106)
Exception in thread main RemoteTrace:
at Local Trace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl:
Trying to get information for an absent applicat
ion
application_1320278804241_0002
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:142)
at $Proxy6.getApplicationReport(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getApplicationReport(ClientRMProtocolPBClie
ntImpl.java:111)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getApplicationReport(ResourceMgrDelegate.java:321)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:137)
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:273)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:353)
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:429)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:240)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1106)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3348) mapred job -status fails to give info even if the job is present in History
[
https://issues.apache.org/jira/browse/MAPREDUCE-3348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3348:
-
Component/s: mrv2
mapred job -status fails to give info even if the job is present in History
---
Key: MAPREDUCE-3348
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3348
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2
Affects Versions: 0.24.0
Reporter: Devaraj K
Assignee: Devaraj K
Attachments: MAPREDUCE-3348.patch
It is trying to get the app report from the RM for the job, RM throws
exception when it doesn't find and then it is giving the same exception
without trying from History Server.
{code}
11/11/03 08:47:27 INFO ipc.HadoopYarnRPC: Creating a HadoopYarnProtoRpc proxy
for protocol interface org.apache.hadoop.mapred
uce.v2.api.MRClientProtocol
11/11/03 08:47:28 WARN mapred.ClientServiceDelegate: Exception thrown by
remote end.
RemoteTrace:
at LocalTrace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl:
Trying to get information for an absent applicat
ion
application_1320278804241_0002
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:142)
at $Proxy6.getApplicationReport(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getApplicationReport(ClientRMProtocolPBClie
ntImpl.java:111)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getApplicationReport(ResourceMgrDelegate.java:321)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:137)
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:273)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:353)
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:429)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:240)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1106)
Exception in thread main RemoteTrace:
at Local Trace:
org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl:
Trying to get information for an absent applicat
ion
application_1320278804241_0002
at
org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Invoker.invoke(ProtoOverHadoopRpcEngine.java:142)
at $Proxy6.getApplicationReport(Unknown Source)
at
org.apache.hadoop.yarn.api.impl.pb.client.ClientRMProtocolPBClientImpl.getApplicationReport(ClientRMProtocolPBClie
ntImpl.java:111)
at
org.apache.hadoop.mapred.ResourceMgrDelegate.getApplicationReport(ResourceMgrDelegate.java:321)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getProxy(ClientServiceDelegate.java:137)
at
org.apache.hadoop.mapred.ClientServiceDelegate.invoke(ClientServiceDelegate.java:273)
at
org.apache.hadoop.mapred.ClientServiceDelegate.getJobStatus(ClientServiceDelegate.java:353)
at
org.apache.hadoop.mapred.YARNRunner.getJobStatus(YARNRunner.java:429)
at org.apache.hadoop.mapreduce.Cluster.getJob(Cluster.java:186)
at org.apache.hadoop.mapreduce.tools.CLI.run(CLI.java:240)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:69)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:83)
at org.apache.hadoop.mapred.JobClient.main(JobClient.java:1106)
{code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Attachment: MAPREDUCE-3034-4.patch NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Priority: Critical Attachments: MAPREDUCE-3034-1.patch, MAPREDUCE-3034-2.patch, MAPREDUCE-3034-3.patch, MAPREDUCE-3034-4.patch, MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Patch Available (was: Open) Thanks a lot Arun for looking into the patch. bq. 1.set/get on isRebooted needs to be synchronized I updated the patch with this change. bq. 2.On reboot, we should be kill existing containers, if any? I tested with the patch with/without having running containers in the NM. If any containers are running, it will stop all those containers as part of NM service stop. Thank you so much Eric for verifying and describing all the cases. NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Priority: Critical Attachments: MAPREDUCE-3034-1.patch, MAPREDUCE-3034-2.patch, MAPREDUCE-3034-3.patch, MAPREDUCE-3034-4.patch, MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Attachment: MAPREDUCE-3034-3.patch NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Priority: Critical Attachments: MAPREDUCE-3034-1.patch, MAPREDUCE-3034-2.patch, MAPREDUCE-3034-3.patch, MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Open (was: Patch Available) NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Priority: Critical Attachments: MAPREDUCE-3034-1.patch, MAPREDUCE-3034-2.patch, MAPREDUCE-3034-3.patch, MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Patch Available (was: Open) Thanks a lot Robert for taking look at the patch. I have updated the patch with the above comments fixed. NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Priority: Critical Attachments: MAPREDUCE-3034-1.patch, MAPREDUCE-3034-2.patch, MAPREDUCE-3034-3.patch, MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Attachment: MAPREDUCE-3034-2.patch NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Priority: Critical Attachments: MAPREDUCE-3034-1.patch, MAPREDUCE-3034-2.patch, MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Patch Available (was: Open) Thanks Eric for revewing and testing the patch. I have updated the patch with the suggested change. NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Priority: Critical Attachments: MAPREDUCE-3034-1.patch, MAPREDUCE-3034-2.patch, MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[
https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Devaraj K updated MAPREDUCE-3034:
-
Attachment: MAPREDUCE-3034-1.patch
{quote}
* Major:
** {{NodeManager.stateChanged()}}:
*** In the case of {{STATE.STOPPED}} *both* {{stop()}} and {{reboot()}} need to
be called.
*** I don't see anywhere in the call hierarchy where {{NodeManager.stop()}} is
ever called in the reboot case.
*** Following the logic through, it looks like this is the call sequence from
your patch:
{{NodeStatusUpdaterImpl.startStatusUpdater().run()}}
* calls: {{NodeStatusUpdaterImpl.reboot()}}
* calls: {{AbstractService.stop()}}
* calls: {{AbstractService.changeState()}}
* calls: {{NodeManager.stateChanged()}}
* Which *does not* call {{NodeManager.stop()}} if
{{NodeStatusUpdaterImpl.isRebooted}} is set.
{quote}
stop() was calling from reboot method. Anyway now I have moved it to common for
SHUTDOWN and REBOOT cases.
{quote}
** {{NodeManager}}:
*** In order for the new private static variable {{nodeManager}} to be fully
effective, you should take out the local {{NodeManager}} declaration within
{{main()}}. Otherwise, the new {{getNodeManager()}} method won't always return
the NodeManager instance.
{quote}
It was added only for reboot testability purpose. Now I have refactored the
code.
{quote}
** Please look into the new findbugs and core test failures.
{quote}
These test failures are not related to this patch.
{quote}
* Minor:
** {{NodeManager.main()}} / {{NodeManager.reboot()}}:
*** Rather than copy the same code from {{main()}} into the new {{reboot()}}
method, I would create a common method (call it {{commonNMMain}}, for example).
You will probably need a parameter flag to tell you whether or not this is
called from {{main()}} or from {{reboot()}}.
{quote}
I have refactored the code accordingly.
{quote}
** {{NodeManager.reboot()}}:
*** You should handle the case where {{nodeManagerShutdownHook}} is {{null}}.
Even though it should not be {{null}}, if it is {{null}}, you should create a
new instance of it.
{quote}
There will not be a case where it will be null. This condition is added only
for test purpose.
{quote}
** {{NodeManager.reboot()}}:
*** Rather than calling {{getConfig()}} when calling {{nodeManager.init()}}, I
would argue that it would be better to create a new instance of conf so that
any updates to the NM configs will be picked up. I guess it could be argued the
other way, too. That is, you could make the case that the grid owner doesn't
want the NM configs to change just because the RM was restarted. However, I
think it should be a new NM config because the RM will have a new config.
{quote}
I don't feel it is a good idea to reload the configuration when the NM gets
rebooted. If the user want to update any property then they should restart the
NM manually.
{quote}
** {{NodeStatusUpdaterImpl}}:
*** {{YarnRPC rpc;}} is declared but never used.
{quote}
It was my mistake, I was doing some thing and forgot to remove. I have updated
now.
{quote}
** {{TestNodeStatusUpdater}}:
*** I don't see where it is testing the RM-restart / NM-reboot case.
{quote}
This test case verifies that when NM gets reboot signal, current instance will
be stopped and new instance will be started successfully.
NM should act on a REBOOT command from RM
-
Key: MAPREDUCE-3034
URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034
Project: Hadoop Map/Reduce
Issue Type: Bug
Components: mrv2, nodemanager
Affects Versions: 0.23.0, 0.24.0
Reporter: Vinod Kumar Vavilapalli
Assignee: Devaraj K
Priority: Critical
Attachments: MAPREDUCE-3034-1.patch, MAPREDUCE-3034.patch, MR-3034.txt
RM sends a reboot command to NM in some cases, like when it gets lost and
rejoins back. In such a case, NM should act on the command and
reboot/reinitalize itself.
This is akin to TT reinitialize on order from JT. We will need to shutdown
all the services properly and reinitialize - this should automatically take
care of killing of containers, cleaning up local temporary files etc.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Patch Available (was: Open) I updated the patch by incorporating the above comments. NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Priority: Critical Attachments: MAPREDUCE-3034-1.patch, MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Open (was: Patch Available) Thanks Eric for reviewing the patch. Cancelling the patch to address Eric review comments. NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Priority: Critical Attachments: MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Attachment: MAPREDUCE-3034.patch I have updated the first level patch for review. Please review and give your comments. NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Attachments: MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (MAPREDUCE-3034) NM should act on a REBOOT command from RM
[ https://issues.apache.org/jira/browse/MAPREDUCE-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj K updated MAPREDUCE-3034: - Target Version/s: 0.23.0, 0.24.0 (was: 0.24.0, 0.23.0) Status: Patch Available (was: Reopened) NM should act on a REBOOT command from RM - Key: MAPREDUCE-3034 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3034 Project: Hadoop Map/Reduce Issue Type: Bug Components: mrv2, nodemanager Affects Versions: 0.23.0, 0.24.0 Reporter: Vinod Kumar Vavilapalli Assignee: Devaraj K Attachments: MAPREDUCE-3034.patch, MR-3034.txt RM sends a reboot command to NM in some cases, like when it gets lost and rejoins back. In such a case, NM should act on the command and reboot/reinitalize itself. This is akin to TT reinitialize on order from JT. We will need to shutdown all the services properly and reinitialize - this should automatically take care of killing of containers, cleaning up local temporary files etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
