[jira] [Updated] (YARN-3349) treat all exceptions as failure in testFSRMStateStoreClientRetry
[
https://issues.apache.org/jira/browse/YARN-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhihai xu updated YARN-3349:
Attachment: YARN-3349.000.patch
treat all exceptions as failure in testFSRMStateStoreClientRetry
Key: YARN-3349
URL: https://issues.apache.org/jira/browse/YARN-3349
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Minor
Attachments: YARN-3349.000.patch
treat all exceptions as failure in testFSRMStateStoreClientRetry.
Currently the exception could only be replicated to 0 nodes instead of
minReplication (=1) is not treated as failure in
testFSRMStateStoreClientRetry.
{code}
// TODO 0 datanode exception will not be retried by dfs client,
fix
// that separately.
if (!e.getMessage().contains(could only be replicated +
to 0 nodes instead of minReplication (=1))) {
assertionFailedInThread.set(true);
}
{code}
With YARN-2820(Retry in FileSystemRMStateStore), we needn't treat this
exception specially. We can remove the check and treat all exceptions as
failure in testFSRMStateStoreClientRetry.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3171) Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui
[ https://issues.apache.org/jira/browse/YARN-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-3171: Summary: Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui (was: Sort by Application id, AppA doesn't work in ATS web ui) Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui Key: YARN-3171 URL: https://issues.apache.org/jira/browse/YARN-3171 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Affects Versions: 2.6.0 Reporter: Jeff Zhang Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-3171.20150213-1.patch, YARN-3171.20150215-1.patch, ats_webui.png The order doesn't change when I click the column header -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3171) Sort by Application id, AppA doesn't work in ATS web ui
[ https://issues.apache.org/jira/browse/YARN-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-3171: Summary: Sort by Application id, AppA doesn't work in ATS web ui (was: Sort by application id doesn't work in ATS web ui) Sort by Application id, AppA doesn't work in ATS web ui --- Key: YARN-3171 URL: https://issues.apache.org/jira/browse/YARN-3171 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Affects Versions: 2.6.0 Reporter: Jeff Zhang Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-3171.20150213-1.patch, YARN-3171.20150215-1.patch, ats_webui.png The order doesn't change when I click the column header -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3171) Sort by application id doesn't work in ATS web ui
[ https://issues.apache.org/jira/browse/YARN-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-3171: Issue Type: Sub-task (was: Bug) Parent: YARN-3299 Sort by application id doesn't work in ATS web ui - Key: YARN-3171 URL: https://issues.apache.org/jira/browse/YARN-3171 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Affects Versions: 2.6.0 Reporter: Jeff Zhang Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-3171.20150213-1.patch, YARN-3171.20150215-1.patch, ats_webui.png The order doesn't change when I click the column header -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-2854) The document about timeline service and generic service needs to be updated
[
https://issues.apache.org/jira/browse/YARN-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naganarasimha G R updated YARN-2854:
Attachment: YARN-2854.20150315-1_trunk_addendum.patch
Hi [~zjshen], Seems like for trunk patch has been applied already, so have
attached addendum patch (for trunk) for the few small changes (if its difficult
to revert).
Difference are :
# Grammatical corrections in Current status
# Grammatical corrections for
yarn.timeline-service.generic-application-history.enabled property description
# Indentation corerction for one property
# Command to start TLS as daemon modified to
{{$HADOOP_YARN_HOME/sbin/yarn-daemon.sh start timelineserver}}
The document about timeline service and generic service needs to be updated
---
Key: YARN-2854
URL: https://issues.apache.org/jira/browse/YARN-2854
Project: Hadoop YARN
Issue Type: Improvement
Components: timelineserver
Reporter: Zhijie Shen
Assignee: Naganarasimha G R
Priority: Critical
Attachments: TimelineServer.html, YARN-2854.20141120-1.patch,
YARN-2854.20150128.1.patch, YARN-2854.20150304.1.patch,
YARN-2854.20150311-1.patch, YARN-2854.20150313-1.patch,
YARN-2854.20150314-1.patch, YARN-2854.20150314-1_branch2.patch,
YARN-2854.20150315-1_trunk_addendum.patch, timeline_structure.jpg
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (YARN-3349) treat all exceptions as failure in testFSRMStateStoreClientRetry
zhihai xu created YARN-3349:
---
Summary: treat all exceptions as failure in
testFSRMStateStoreClientRetry
Key: YARN-3349
URL: https://issues.apache.org/jira/browse/YARN-3349
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Minor
treat all exceptions as failure in testFSRMStateStoreClientRetry.
Currently the exception could only be replicated to 0 nodes instead of
minReplication (=1) is not treated as failure in testFSRMStateStoreClientRetry.
{code}
// TODO 0 datanode exception will not be retried by dfs client, fix
// that separately.
if (!e.getMessage().contains(could only be replicated +
to 0 nodes instead of minReplication (=1))) {
assertionFailedInThread.set(true);
}
{code}
With YARN-2820(Retry in FileSystemRMStateStore), we needn't treat this
exception specially. We can remove the check and treat all exceptions as
failure in testFSRMStateStoreClientRetry.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3171) Sort by application id doesn't work in ATS web ui
[ https://issues.apache.org/jira/browse/YARN-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naganarasimha G R updated YARN-3171: Attachment: YARN-3171.20150215-1.patch Rebasing the patch based on changes made from YARN-1809. No test cases added as its UI issue but have tested it manually and working fine Sort by application id doesn't work in ATS web ui - Key: YARN-3171 URL: https://issues.apache.org/jira/browse/YARN-3171 Project: Hadoop YARN Issue Type: Bug Components: timelineserver Affects Versions: 2.6.0 Reporter: Jeff Zhang Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-3171.20150213-1.patch, YARN-3171.20150215-1.patch, ats_webui.png The order doesn't change when I click the column header -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3171) Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui
[
https://issues.apache.org/jira/browse/YARN-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362284#comment-14362284
]
Hadoop QA commented on YARN-3171:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704646/YARN-3171.20150215-1.patch
against trunk revision bc9cb3e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6964//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6964//console
This message is automatically generated.
Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web
ui
Key: YARN-3171
URL: https://issues.apache.org/jira/browse/YARN-3171
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Affects Versions: 2.6.0
Reporter: Jeff Zhang
Assignee: Naganarasimha G R
Priority: Minor
Attachments: YARN-3171.20150213-1.patch, YARN-3171.20150215-1.patch,
ats_webui.png
The order doesn't change when I click the column header
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3349) treat all exceptions as failure in testFSRMStateStoreClientRetry
[
https://issues.apache.org/jira/browse/YARN-3349?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362323#comment-14362323
]
Hadoop QA commented on YARN-3349:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704654/YARN-3349.000.patch
against trunk revision bc9cb3e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 5 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6966//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6966//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6966//console
This message is automatically generated.
treat all exceptions as failure in testFSRMStateStoreClientRetry
Key: YARN-3349
URL: https://issues.apache.org/jira/browse/YARN-3349
Project: Hadoop YARN
Issue Type: Improvement
Components: test
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Minor
Attachments: YARN-3349.000.patch
treat all exceptions as failure in testFSRMStateStoreClientRetry.
Currently the exception could only be replicated to 0 nodes instead of
minReplication (=1) is not treated as failure in
testFSRMStateStoreClientRetry.
{code}
// TODO 0 datanode exception will not be retried by dfs client,
fix
// that separately.
if (!e.getMessage().contains(could only be replicated +
to 0 nodes instead of minReplication (=1))) {
assertionFailedInThread.set(true);
}
{code}
With YARN-2820(Retry in FileSystemRMStateStore), we needn't treat this
exception specially. We can remove the check and treat all exceptions as
failure in testFSRMStateStoreClientRetry.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-2854) The document about timeline service and generic service needs to be updated
[
https://issues.apache.org/jira/browse/YARN-2854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362296#comment-14362296
]
Hadoop QA commented on YARN-2854:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704653/YARN-2854.20150315-1_trunk_addendum.patch
against trunk revision bc9cb3e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in .
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6965//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6965//console
This message is automatically generated.
The document about timeline service and generic service needs to be updated
---
Key: YARN-2854
URL: https://issues.apache.org/jira/browse/YARN-2854
Project: Hadoop YARN
Issue Type: Improvement
Components: timelineserver
Reporter: Zhijie Shen
Assignee: Naganarasimha G R
Priority: Critical
Attachments: TimelineServer.html, YARN-2854.20141120-1.patch,
YARN-2854.20150128.1.patch, YARN-2854.20150304.1.patch,
YARN-2854.20150311-1.patch, YARN-2854.20150313-1.patch,
YARN-2854.20150314-1.patch, YARN-2854.20150314-1_branch2.patch,
YARN-2854.20150315-1_trunk_addendum.patch, timeline_structure.jpg
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-1621) Add CLI to list rows of task attempt ID, container ID, host of container, state of container
[
https://issues.apache.org/jira/browse/YARN-1621?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362450#comment-14362450
]
Bartosz Ługowski commented on YARN-1621:
Any comments?
Add CLI to list rows of task attempt ID, container ID, host of container,
state of container
--
Key: YARN-1621
URL: https://issues.apache.org/jira/browse/YARN-1621
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.2.0
Reporter: Tassapol Athiapinya
Assignee: Bartosz Ługowski
Attachments: YARN-1621.1.patch, YARN-1621.2.patch, YARN-1621.3.patch,
YARN-1621.4.patch, YARN-1621.5.patch
As more applications are moved to YARN, we need generic CLI to list rows of
task attempt ID, container ID, host of container, state of container. Today
if YARN application running in a container does hang, there is no way to find
out more info because a user does not know where each attempt is running in.
For each running application, it is useful to differentiate between
running/succeeded/failed/killed containers.
{code:title=proposed yarn cli}
$ yarn application -list-containers -applicationId appId [-containerState
state of container]
where containerState is optional filter to list container in given state only.
container state can be running/succeeded/killed/failed/all.
A user can specify more than one container state at once e.g. KILLED,FAILED.
task attempt ID container ID host of container state of container
{code}
CLI should work with running application/completed application. If a
container runs many task attempts, all attempts should be shown. That will
likely be the case of Tez container-reuse application.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3110) Few issues in ApplicationHistory web ui
[
https://issues.apache.org/jira/browse/YARN-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naganarasimha G R updated YARN-3110:
Summary: Few issues in ApplicationHistory web ui (was: Faulty link and
state in ApplicationHistory when aplication is in unassigned state)
Few issues in ApplicationHistory web ui
---
Key: YARN-3110
URL: https://issues.apache.org/jira/browse/YARN-3110
Project: Hadoop YARN
Issue Type: Sub-task
Components: applications, timelineserver
Affects Versions: 2.6.0
Reporter: Bibin A Chundatt
Assignee: Naganarasimha G R
Priority: Minor
Attachments: YARN-3110.20150209-1.patch, YARN-3110.20150315-1.patch
Application state and History link wrong when Application is in unassigned
state
1.Configure capacity schedular with queue size as 1 also max Absolute Max
Capacity: 10.0%
(Current application state is Accepted and Unassigned from resource manager
side)
2.Submit application to queue and check the state and link in Application
history
State= null and History link shown as N/A in applicationhistory page
Kill the same application . In timeline server logs the below is show when
selecting application link.
{quote}
2015-01-29 15:39:50,956 ERROR org.apache.hadoop.yarn.webapp.View: Failed to
read the AM container of the application attempt
appattempt_1422467063659_0007_01.
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerOnTimelineStore.getContainer(ApplicationHistoryManagerOnTimelineStore.java:162)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerOnTimelineStore.getAMContainer(ApplicationHistoryManagerOnTimelineStore.java:184)
at
org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:160)
at
org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.yarn.server.webapp.AppBlock.render(AppBlock.java:156)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:67)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:77)
at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
at
org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at
org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:845)
at
org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:56)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController.app(AHSController.java:38)
at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:153)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
at
[jira] [Updated] (YARN-3110) Faulty link and state in ApplicationHistory when aplication is in unassigned state
[
https://issues.apache.org/jira/browse/YARN-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naganarasimha G R updated YARN-3110:
Issue Type: Sub-task (was: Bug)
Parent: YARN-3299
Faulty link and state in ApplicationHistory when aplication is in unassigned
state
--
Key: YARN-3110
URL: https://issues.apache.org/jira/browse/YARN-3110
Project: Hadoop YARN
Issue Type: Sub-task
Components: applications, timelineserver
Affects Versions: 2.6.0
Reporter: Bibin A Chundatt
Assignee: Naganarasimha G R
Priority: Minor
Attachments: YARN-3110.20150209-1.patch, YARN-3110.20150315-1.patch
Application state and History link wrong when Application is in unassigned
state
1.Configure capacity schedular with queue size as 1 also max Absolute Max
Capacity: 10.0%
(Current application state is Accepted and Unassigned from resource manager
side)
2.Submit application to queue and check the state and link in Application
history
State= null and History link shown as N/A in applicationhistory page
Kill the same application . In timeline server logs the below is show when
selecting application link.
{quote}
2015-01-29 15:39:50,956 ERROR org.apache.hadoop.yarn.webapp.View: Failed to
read the AM container of the application attempt
appattempt_1422467063659_0007_01.
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerOnTimelineStore.getContainer(ApplicationHistoryManagerOnTimelineStore.java:162)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerOnTimelineStore.getAMContainer(ApplicationHistoryManagerOnTimelineStore.java:184)
at
org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:160)
at
org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.yarn.server.webapp.AppBlock.render(AppBlock.java:156)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:67)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:77)
at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
at
org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at
org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:845)
at
org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:56)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController.app(AHSController.java:38)
at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:153)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:834)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:795)
at
com.google.inject.servlet.FilterDefinition.doFilter(FilterDefinition.java:163)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:58)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
at
org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
at
[jira] [Commented] (YARN-3034) [Aggregator wireup] Implement RM starting its ATS writer
[
https://issues.apache.org/jira/browse/YARN-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362519#comment-14362519
]
Hadoop QA commented on YARN-3034:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704680/YARN-3034.20150316-1.patch
against trunk revision bc9cb3e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 1 new
or modified test files.
{color:red}-1 javac{color:red}. The patch appears to cause the build to
fail.
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6969//console
This message is automatically generated.
[Aggregator wireup] Implement RM starting its ATS writer
Key: YARN-3034
URL: https://issues.apache.org/jira/browse/YARN-3034
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Sangjin Lee
Assignee: Naganarasimha G R
Attachments: YARN-3034-20150312-1.patch, YARN-3034.20150205-1.patch,
YARN-3034.20150316-1.patch
Per design in YARN-2928, implement resource managers starting their own ATS
writers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3110) Faulty link and state in ApplicationHistory when aplication is in unassigned state
[
https://issues.apache.org/jira/browse/YARN-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naganarasimha G R updated YARN-3110:
Attachment: YARN-3110.20150315-1.patch
Hi [~zjshen] [~xgong],
Main issue for which this jira was raised by reporter has been closed by
YARN-1809,
but some issue which i had identified and fixed earlier, seems still to be
present even after 1809 :
# Progress bar is always shown as 100%. we can either not show it in AHS or can
be handled such that when application is in accepted state it will be 0% and
once finished it will be 100% (have chosen the later approach)
# In Application listing page in ATS, {{State}} is always shown as N/A till it
gets completed. Initially we can show as {{Assigned}} and when app is completed
final state can be shown.
# In app attempt page if the am container is not created and before that if app
is killed {{null}} is displayed for AM container instead N/A can be shown.
Faulty link and state in ApplicationHistory when aplication is in unassigned
state
--
Key: YARN-3110
URL: https://issues.apache.org/jira/browse/YARN-3110
Project: Hadoop YARN
Issue Type: Bug
Components: applications, timelineserver
Affects Versions: 2.6.0
Reporter: Bibin A Chundatt
Assignee: Naganarasimha G R
Priority: Minor
Attachments: YARN-3110.20150209-1.patch, YARN-3110.20150315-1.patch
Application state and History link wrong when Application is in unassigned
state
1.Configure capacity schedular with queue size as 1 also max Absolute Max
Capacity: 10.0%
(Current application state is Accepted and Unassigned from resource manager
side)
2.Submit application to queue and check the state and link in Application
history
State= null and History link shown as N/A in applicationhistory page
Kill the same application . In timeline server logs the below is show when
selecting application link.
{quote}
2015-01-29 15:39:50,956 ERROR org.apache.hadoop.yarn.webapp.View: Failed to
read the AM container of the application attempt
appattempt_1422467063659_0007_01.
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerOnTimelineStore.getContainer(ApplicationHistoryManagerOnTimelineStore.java:162)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerOnTimelineStore.getAMContainer(ApplicationHistoryManagerOnTimelineStore.java:184)
at
org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:160)
at
org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.yarn.server.webapp.AppBlock.render(AppBlock.java:156)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:67)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:77)
at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
at
org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at
org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:845)
at
org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:56)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController.app(AHSController.java:38)
at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.yarn.webapp.Dispatcher.service(Dispatcher.java:153)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.sun.jersey.spi.container.servlet.ServletContainer.doFilter(ServletContainer.java:900)
at
[jira] [Updated] (YARN-3341) Fix findbugs warning:BC_UNCONFIRMED_CAST at FSSchedulerNode.reserveResource
[
https://issues.apache.org/jira/browse/YARN-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhihai xu updated YARN-3341:
Attachment: YARN-3341.000.patch
Fix findbugs warning:BC_UNCONFIRMED_CAST at FSSchedulerNode.reserveResource
---
Key: YARN-3341
URL: https://issues.apache.org/jira/browse/YARN-3341
Project: Hadoop YARN
Issue Type: Improvement
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Minor
Labels: findbugs
Attachments: YARN-3341.000.patch
Fix findbugs warning:BC_UNCONFIRMED_CAST at FSSchedulerNode.reserveResource
The warning message is
{code}
Unchecked/unconfirmed cast from
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt
to org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt
in
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerNode.reserveResource(SchedulerApplicationAttempt,
Priority, RMContainer)
{code}
The code which cause the warning is
{code}
this.reservedAppSchedulable = (FSAppAttempt) application;
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3110) Few issues in ApplicationHistory web ui
[
https://issues.apache.org/jira/browse/YARN-3110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362473#comment-14362473
]
Hadoop QA commented on YARN-3110:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704668/YARN-3110.20150315-1.patch
against trunk revision bc9cb3e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-applicationhistoryservice
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6967//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6967//console
This message is automatically generated.
Few issues in ApplicationHistory web ui
---
Key: YARN-3110
URL: https://issues.apache.org/jira/browse/YARN-3110
Project: Hadoop YARN
Issue Type: Sub-task
Components: applications, timelineserver
Affects Versions: 2.6.0
Reporter: Bibin A Chundatt
Assignee: Naganarasimha G R
Priority: Minor
Attachments: YARN-3110.20150209-1.patch, YARN-3110.20150315-1.patch
Application state and History link wrong when Application is in unassigned
state
1.Configure capacity schedular with queue size as 1 also max Absolute Max
Capacity: 10.0%
(Current application state is Accepted and Unassigned from resource manager
side)
2.Submit application to queue and check the state and link in Application
history
State= null and History link shown as N/A in applicationhistory page
Kill the same application . In timeline server logs the below is show when
selecting application link.
{quote}
2015-01-29 15:39:50,956 ERROR org.apache.hadoop.yarn.webapp.View: Failed to
read the AM container of the application attempt
appattempt_1422467063659_0007_01.
java.lang.NullPointerException
at
org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerOnTimelineStore.getContainer(ApplicationHistoryManagerOnTimelineStore.java:162)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.ApplicationHistoryManagerOnTimelineStore.getAMContainer(ApplicationHistoryManagerOnTimelineStore.java:184)
at
org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:160)
at
org.apache.hadoop.yarn.server.webapp.AppBlock$3.run(AppBlock.java:157)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at
org.apache.hadoop.yarn.server.webapp.AppBlock.render(AppBlock.java:156)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:67)
at
org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:77)
at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
at
org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
at
org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:845)
at
org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:56)
at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212)
at
org.apache.hadoop.yarn.server.applicationhistoryservice.webapp.AHSController.app(AHSController.java:38)
at sun.reflect.GeneratedMethodAccessor63.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
[jira] [Commented] (YARN-3341) Fix findbugs warning:BC_UNCONFIRMED_CAST at FSSchedulerNode.reserveResource
[
https://issues.apache.org/jira/browse/YARN-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362509#comment-14362509
]
Hadoop QA commented on YARN-3341:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704670/YARN-3341.000.patch
against trunk revision bc9cb3e.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:red}-1 findbugs{color}. The patch appears to introduce 4 new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6968//testReport/
Findbugs warnings:
https://builds.apache.org/job/PreCommit-YARN-Build/6968//artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6968//console
This message is automatically generated.
Fix findbugs warning:BC_UNCONFIRMED_CAST at FSSchedulerNode.reserveResource
---
Key: YARN-3341
URL: https://issues.apache.org/jira/browse/YARN-3341
Project: Hadoop YARN
Issue Type: Improvement
Reporter: zhihai xu
Assignee: zhihai xu
Priority: Minor
Labels: findbugs
Attachments: YARN-3341.000.patch
Fix findbugs warning:BC_UNCONFIRMED_CAST at FSSchedulerNode.reserveResource
The warning message is
{code}
Unchecked/unconfirmed cast from
org.apache.hadoop.yarn.server.resourcemanager.scheduler.SchedulerApplicationAttempt
to org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSAppAttempt
in
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSSchedulerNode.reserveResource(SchedulerApplicationAttempt,
Priority, RMContainer)
{code}
The code which cause the warning is
{code}
this.reservedAppSchedulable = (FSAppAttempt) application;
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (YARN-3034) [Aggregator wireup] Implement RM starting its ATS writer
[
https://issues.apache.org/jira/browse/YARN-3034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Naganarasimha G R updated YARN-3034:
Attachment: YARN-3034.20150316-1.patch
Attaching patch with following changes :
# fixed [~sjlee0]'s review comments on resourcemanager/pom.xml
# Fixed [~vinodkv]'s Comments :
#* {{yarn.system-metrics-publisher.enabled}} added
#* {{yarn.resourcemanager.system-metrics-publisher.enabled}} deprecated and
req modifications
#* {{yarn.timeline-service.version}} added with default as V1
[Aggregator wireup] Implement RM starting its ATS writer
Key: YARN-3034
URL: https://issues.apache.org/jira/browse/YARN-3034
Project: Hadoop YARN
Issue Type: Sub-task
Components: timelineserver
Reporter: Sangjin Lee
Assignee: Naganarasimha G R
Attachments: YARN-3034-20150312-1.patch, YARN-3034.20150205-1.patch,
YARN-3034.20150316-1.patch
Per design in YARN-2928, implement resource managers starting their own ATS
writers.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3190) NM can't aggregate logs: token can't be found in cache
[
https://issues.apache.org/jira/browse/YARN-3190?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362584#comment-14362584
]
zhihai xu commented on YARN-3190:
-
This looks like an issue fixed by YARN-2704, YARN-2790 and YARN-2964. Could you
check whether YARN-2704, YARN-2790 and YARN-2964 can fix your issue?
NM can't aggregate logs: token can't be found in cache
---
Key: YARN-3190
URL: https://issues.apache.org/jira/browse/YARN-3190
Project: Hadoop YARN
Issue Type: Bug
Components: nodemanager
Affects Versions: 2.5.0
Environment: CDH 5.3.1
HA HDFS
Kerberos
Reporter: Andrejs Dubovskis
Priority: Minor
In rare cases node manager can not aggregate logs: generating exception:
{code}
2015-02-12 13:04:03,703 INFO
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl:
Starting aggregate log-file for app application_1423661043235_2150 at
/tmp/logs/catalyst/logs/application_1423661043235_2150/catdn001.intrum.net_8041.tmp
2015-02-12 13:04:03,707 INFO
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Deleting
absolute path :
/data5/yarn/nm/usercache/catalyst/appcache/application_1423661043235_2150/container_1423661043235_2150_01_000442
2015-02-12 13:04:03,707 INFO
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Deleting
absolute path :
/data6/yarn/nm/usercache/catalyst/appcache/application_1423661043235_2150/container_1423661043235_2150_01_000442
2015-02-12 13:04:03,707 INFO
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Deleting
absolute path :
/data7/yarn/nm/usercache/catalyst/appcache/application_1423661043235_2150/container_1423661043235_2150_01_000442
2015-02-12 13:04:03,709 INFO
org.apache.hadoop.yarn.server.nodemanager.LinuxContainerExecutor: Deleting
absolute path :
/data1/yarn/nm/usercache/catalyst/appcache/application_1423661043235_2150
2015-02-12 13:04:03,709 WARN org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:catalyst (auth:SIMPLE)
cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 2334644 for catalyst) can't be found in
cache
2015-02-12 13:04:03,709 WARN org.apache.hadoop.ipc.Client: Exception
encountered while connecting to the server :
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 2334644 for catalyst) can't be found in
cache
2015-02-12 13:04:03,709 WARN org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:catalyst (auth:SIMPLE)
cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 2334644 for catalyst) can't be found in
cache
2015-02-12 13:04:03,712 WARN org.apache.hadoop.security.UserGroupInformation:
PriviledgedActionException as:catalyst (auth:SIMPLE)
cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 2334644 for catalyst) can't be found in
cache
2015-02-12 13:04:03,712 ERROR
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl:
Cannot create writer for app application_1423661043235_2150. Disabling
log-aggregation for this app.
org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
token (HDFS_DELEGATION_TOKEN token 2334644 for catalyst) can't be found in
cache
at org.apache.hadoop.ipc.Client.call(Client.java:1411)
at org.apache.hadoop.ipc.Client.call(Client.java:1364)
at
org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
at com.sun.proxy.$Proxy19.getServerDefaults(Unknown Source)
at
org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getServerDefaults(ClientNamenodeProtocolTranslatorPB.java:259)
at sun.reflect.GeneratedMethodAccessor114.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:187)
at
org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
at com.sun.proxy.$Proxy20.getServerDefaults(Unknown Source)
at
org.apache.hadoop.hdfs.DFSClient.getServerDefaults(DFSClient.java:966)
at org.apache.hadoop.fs.Hdfs.getServerDefaults(Hdfs.java:159)
at
[jira] [Commented] (YARN-3171) Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui
[ https://issues.apache.org/jira/browse/YARN-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362690#comment-14362690 ] Naganarasimha G R commented on YARN-3171: - Thanks for reviewing and committing [~xgong]. Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui Key: YARN-3171 URL: https://issues.apache.org/jira/browse/YARN-3171 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Affects Versions: 2.6.0 Reporter: Jeff Zhang Assignee: Naganarasimha G R Priority: Minor Fix For: 2.7.0 Attachments: YARN-3171.20150213-1.patch, YARN-3171.20150215-1.patch, ats_webui.png The order doesn't change when I click the column header -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3171) Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui
[ https://issues.apache.org/jira/browse/YARN-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362691#comment-14362691 ] Hudson commented on YARN-3171: -- FAILURE: Integrated in Hadoop-trunk-Commit #7329 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/7329/]) YARN-3171. Sort by Application id, AppAttempt and ContainerID doesn't (xgong: rev 3ff1ba2a7b00fdf06270d00b2193bde4b56b06b3) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-common/src/main/java/org/apache/hadoop/yarn/server/webapp/WebPageUtils.java * hadoop-yarn-project/CHANGES.txt Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui Key: YARN-3171 URL: https://issues.apache.org/jira/browse/YARN-3171 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Affects Versions: 2.6.0 Reporter: Jeff Zhang Assignee: Naganarasimha G R Priority: Minor Fix For: 2.7.0 Attachments: YARN-3171.20150213-1.patch, YARN-3171.20150215-1.patch, ats_webui.png The order doesn't change when I click the column header -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3350) YARN RackResolver spams logs with messages at info level
[ https://issues.apache.org/jira/browse/YARN-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-3350: Attachment: YARN-3350.patch YARN RackResolver spams logs with messages at info level Key: YARN-3350 URL: https://issues.apache.org/jira/browse/YARN-3350 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.6.0 Reporter: Wilfred Spiegelenburg Assignee: Wilfred Spiegelenburg Attachments: YARN-3350.patch, yarn-RackResolver-log.txt When you run an application the container logs shows a lot of messages for the RackResolver: 2015-03-10 00:58:30,483 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved node175.example.com to /rack15 A real world example for a large job was generating 20+ messages in 2 milliseconds during a sustained period of time flooding the logs causing the node to run out of disk space. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3350) YARN RackResolver spams logs with messages at info level
[ https://issues.apache.org/jira/browse/YARN-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wilfred Spiegelenburg updated YARN-3350: Attachment: yarn-RackResolver-log.txt Extract from a log which shows the messages logged by the RackResolver YARN RackResolver spams logs with messages at info level Key: YARN-3350 URL: https://issues.apache.org/jira/browse/YARN-3350 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.6.0 Reporter: Wilfred Spiegelenburg Assignee: Wilfred Spiegelenburg Attachments: yarn-RackResolver-log.txt When you run an application the container logs shows a lot of messages for the RackResolver: 2015-03-10 00:58:30,483 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved node175.example.com to /rack15 A real world example for a large job was generating 20+ messages in 2 milliseconds during a sustained period of time flooding the logs causing the node to run out of disk space. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3343) TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate sometime fails in trunk
[ https://issues.apache.org/jira/browse/YARN-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362737#comment-14362737 ] Rohith commented on YARN-3343: -- Attached the patch with 60seconds timeout. TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate sometime fails in trunk --- Key: YARN-3343 URL: https://issues.apache.org/jira/browse/YARN-3343 Project: Hadoop YARN Issue Type: Test Reporter: Xuan Gong Assignee: Rohith Priority: Minor Attachments: 0001-YARN-3343.patch Error Message test timed out after 3 milliseconds Stacktrace java.lang.Exception: test timed out after 3 milliseconds at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901) at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293) at java.net.InetAddress.getAllByName0(InetAddress.java:1246) at java.net.InetAddress.getAllByName(InetAddress.java:1162) at java.net.InetAddress.getAllByName(InetAddress.java:1098) at java.net.InetAddress.getByName(InetAddress.java:1048) at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:563) at org.apache.hadoop.yarn.server.resourcemanager.NodesListManager.isValidNode(NodesListManager.java:147) at org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.nodeHeartbeat(ResourceTrackerService.java:367) at org.apache.hadoop.yarn.server.resourcemanager.MockNM.nodeHeartbeat(MockNM.java:178) at org.apache.hadoop.yarn.server.resourcemanager.MockNM.nodeHeartbeat(MockNM.java:136) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.waitForState(MockRM.java:206) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate(TestCapacitySchedulerNodeLabelUpdate.java:157) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-3343) TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate sometime fails in trunk
[ https://issues.apache.org/jira/browse/YARN-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith updated YARN-3343: - Attachment: 0001-YARN-3343.patch TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate sometime fails in trunk --- Key: YARN-3343 URL: https://issues.apache.org/jira/browse/YARN-3343 Project: Hadoop YARN Issue Type: Test Reporter: Xuan Gong Assignee: Rohith Priority: Minor Attachments: 0001-YARN-3343.patch Error Message test timed out after 3 milliseconds Stacktrace java.lang.Exception: test timed out after 3 milliseconds at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901) at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293) at java.net.InetAddress.getAllByName0(InetAddress.java:1246) at java.net.InetAddress.getAllByName(InetAddress.java:1162) at java.net.InetAddress.getAllByName(InetAddress.java:1098) at java.net.InetAddress.getByName(InetAddress.java:1048) at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:563) at org.apache.hadoop.yarn.server.resourcemanager.NodesListManager.isValidNode(NodesListManager.java:147) at org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.nodeHeartbeat(ResourceTrackerService.java:367) at org.apache.hadoop.yarn.server.resourcemanager.MockNM.nodeHeartbeat(MockNM.java:178) at org.apache.hadoop.yarn.server.resourcemanager.MockNM.nodeHeartbeat(MockNM.java:136) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.waitForState(MockRM.java:206) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate(TestCapacitySchedulerNodeLabelUpdate.java:157) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (YARN-1453) [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments
[ https://issues.apache.org/jira/browse/YARN-1453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Akira AJISAKA reassigned YARN-1453: --- Assignee: Akira AJISAKA (was: Andrew Purtell) [JDK8] Fix Javadoc errors caused by incorrect or illegal tags in doc comments - Key: YARN-1453 URL: https://issues.apache.org/jira/browse/YARN-1453 Project: Hadoop YARN Issue Type: Bug Affects Versions: 2.7.0 Reporter: Andrew Purtell Assignee: Akira AJISAKA Priority: Minor Attachments: 1453-branch-2.patch, 1453-branch-2.patch, 1453-trunk.patch, 1453-trunk.patch, YARN-1453-02.patch Javadoc is more strict by default in JDK8 and will error out on malformed or illegal tags found in doc comments. Although tagged as JDK8 all of the required changes are generic Javadoc cleanups. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3171) Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui
[ https://issues.apache.org/jira/browse/YARN-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362688#comment-14362688 ] Xuan Gong commented on YARN-3171: - Committed to trunk/branch-2/branch-2.7. Thanks, Naganarasimha! Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui Key: YARN-3171 URL: https://issues.apache.org/jira/browse/YARN-3171 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Affects Versions: 2.6.0 Reporter: Jeff Zhang Assignee: Naganarasimha G R Priority: Minor Fix For: 2.7.0 Attachments: YARN-3171.20150213-1.patch, YARN-3171.20150215-1.patch, ats_webui.png The order doesn't change when I click the column header -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (YARN-3350) YARN RackResolver spams logs with messages at info level
Wilfred Spiegelenburg created YARN-3350: --- Summary: YARN RackResolver spams logs with messages at info level Key: YARN-3350 URL: https://issues.apache.org/jira/browse/YARN-3350 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.6.0 Reporter: Wilfred Spiegelenburg Assignee: Wilfred Spiegelenburg When you run an application the container logs shows a lot of messages for the RackResolver: 2015-03-10 00:58:30,483 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved node175.example.com to /rack15 A real world example for a large job was generating 20+ messages in 2 milliseconds during a sustained period of time flooding the logs causing the node to run out of disk space. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (YARN-3343) TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate sometime fails in trunk
[ https://issues.apache.org/jira/browse/YARN-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rohith reassigned YARN-3343: Assignee: Rohith TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate sometime fails in trunk --- Key: YARN-3343 URL: https://issues.apache.org/jira/browse/YARN-3343 Project: Hadoop YARN Issue Type: Test Reporter: Xuan Gong Assignee: Rohith Priority: Minor Error Message test timed out after 3 milliseconds Stacktrace java.lang.Exception: test timed out after 3 milliseconds at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901) at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293) at java.net.InetAddress.getAllByName0(InetAddress.java:1246) at java.net.InetAddress.getAllByName(InetAddress.java:1162) at java.net.InetAddress.getAllByName(InetAddress.java:1098) at java.net.InetAddress.getByName(InetAddress.java:1048) at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:563) at org.apache.hadoop.yarn.server.resourcemanager.NodesListManager.isValidNode(NodesListManager.java:147) at org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.nodeHeartbeat(ResourceTrackerService.java:367) at org.apache.hadoop.yarn.server.resourcemanager.MockNM.nodeHeartbeat(MockNM.java:178) at org.apache.hadoop.yarn.server.resourcemanager.MockNM.nodeHeartbeat(MockNM.java:136) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.waitForState(MockRM.java:206) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate(TestCapacitySchedulerNodeLabelUpdate.java:157) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3171) Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui
[ https://issues.apache.org/jira/browse/YARN-3171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362686#comment-14362686 ] Xuan Gong commented on YARN-3171: - +1 LGTM Sort by Application id, AppAttempt ContainerID doesn't work in ATS / RM web ui Key: YARN-3171 URL: https://issues.apache.org/jira/browse/YARN-3171 Project: Hadoop YARN Issue Type: Sub-task Components: timelineserver Affects Versions: 2.6.0 Reporter: Jeff Zhang Assignee: Naganarasimha G R Priority: Minor Attachments: YARN-3171.20150213-1.patch, YARN-3171.20150215-1.patch, ats_webui.png The order doesn't change when I click the column header -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-3350) YARN RackResolver spams logs with messages at info level
[
https://issues.apache.org/jira/browse/YARN-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362718#comment-14362718
]
Hadoop QA commented on YARN-3350:
-
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12704712/YARN-3350.patch
against trunk revision 3ff1ba2.
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 2.0.3) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/6970//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/6970//console
This message is automatically generated.
YARN RackResolver spams logs with messages at info level
Key: YARN-3350
URL: https://issues.apache.org/jira/browse/YARN-3350
Project: Hadoop YARN
Issue Type: Improvement
Affects Versions: 2.6.0
Reporter: Wilfred Spiegelenburg
Assignee: Wilfred Spiegelenburg
Attachments: YARN-3350.patch, yarn-RackResolver-log.txt
When you run an application the container logs shows a lot of messages for
the RackResolver:
2015-03-10 00:58:30,483 INFO [RMCommunicator Allocator]
org.apache.hadoop.yarn.util.RackResolver: Resolved node175.example.com to
/rack15
A real world example for a large job was generating 20+ messages in 2
milliseconds during a sustained period of time flooding the logs causing the
node to run out of disk space.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3343) TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate sometime fails in trunk
[ https://issues.apache.org/jira/browse/YARN-3343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362733#comment-14362733 ] Rohith commented on YARN-3343: -- I verified in the trunk, able to reproduce it. It is just matter of test timeout. 30s is not enough to complete the test. TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate sometime fails in trunk --- Key: YARN-3343 URL: https://issues.apache.org/jira/browse/YARN-3343 Project: Hadoop YARN Issue Type: Test Reporter: Xuan Gong Assignee: Rohith Priority: Minor Error Message test timed out after 3 milliseconds Stacktrace java.lang.Exception: test timed out after 3 milliseconds at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:901) at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1293) at java.net.InetAddress.getAllByName0(InetAddress.java:1246) at java.net.InetAddress.getAllByName(InetAddress.java:1162) at java.net.InetAddress.getAllByName(InetAddress.java:1098) at java.net.InetAddress.getByName(InetAddress.java:1048) at org.apache.hadoop.net.NetUtils.normalizeHostName(NetUtils.java:563) at org.apache.hadoop.yarn.server.resourcemanager.NodesListManager.isValidNode(NodesListManager.java:147) at org.apache.hadoop.yarn.server.resourcemanager.ResourceTrackerService.nodeHeartbeat(ResourceTrackerService.java:367) at org.apache.hadoop.yarn.server.resourcemanager.MockNM.nodeHeartbeat(MockNM.java:178) at org.apache.hadoop.yarn.server.resourcemanager.MockNM.nodeHeartbeat(MockNM.java:136) at org.apache.hadoop.yarn.server.resourcemanager.MockRM.waitForState(MockRM.java:206) at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerNodeLabelUpdate.testNodeUpdate(TestCapacitySchedulerNodeLabelUpdate.java:157) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (YARN-53) Add protocol to YARN to support GetGroups
[
https://issues.apache.org/jira/browse/YARN-53?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Allen Wittenauer updated YARN-53:
-
Fix Version/s: (was: 2.0.0-alpha)
2.0.3-alpha
Add protocol to YARN to support GetGroups
-
Key: YARN-53
URL: https://issues.apache.org/jira/browse/YARN-53
Project: Hadoop YARN
Issue Type: Bug
Components: resourcemanager
Reporter: Alejandro Abdelnur
Assignee: Bo Wang
Labels: patch
Fix For: 2.0.3-alpha
Attachments: MAPREDUCE-4268.patch, YARN-53-v2.patch,
YARN-53-v3.patch, YARN-53-v4.patch, YARN-53-v5.patch, YARN-53.patch
This is a regression from Hadoop1, as hadoop mrgroups fails with:
{code}
Exception in thread main java.lang.UnsupportedOperationException
at
org.apache.hadoop.mapred.tools.GetGroups.getProtocolAddress(GetGroups.java:50)
at
org.apache.hadoop.tools.GetGroupsBase.getUgmProtocol(GetGroupsBase.java:98)
at org.apache.hadoop.tools.GetGroupsBase.run(GetGroupsBase.java:71)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
at org.apache.hadoop.mapred.tools.GetGroups.main(GetGroups.java:54)
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-3350) YARN RackResolver spams logs with messages at info level
[ https://issues.apache.org/jira/browse/YARN-3350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14362771#comment-14362771 ] Wilfred Spiegelenburg commented on YARN-3350: - No tests for this change: it is just a simple log level change without any further code changes. BTW: this same issue was encountered by Spark as an application using the RackResolver and they changed the log level from their side to prevent log flooding via SPARK-5393 YARN RackResolver spams logs with messages at info level Key: YARN-3350 URL: https://issues.apache.org/jira/browse/YARN-3350 Project: Hadoop YARN Issue Type: Improvement Affects Versions: 2.6.0 Reporter: Wilfred Spiegelenburg Assignee: Wilfred Spiegelenburg Attachments: YARN-3350.patch, yarn-RackResolver-log.txt When you run an application the container logs shows a lot of messages for the RackResolver: 2015-03-10 00:58:30,483 INFO [RMCommunicator Allocator] org.apache.hadoop.yarn.util.RackResolver: Resolved node175.example.com to /rack15 A real world example for a large job was generating 20+ messages in 2 milliseconds during a sustained period of time flooding the logs causing the node to run out of disk space. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
