[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13891084#comment-13891084 ] Arun C Murthy commented on YARN-1206: - I haven't seen much progress here, [~jianhe] I'm inclined to move this to 2.4? Thoughts? Thanks. > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Priority: Blocker > > With log aggregation disabled, when container is running, its logs link works > properly, but after the application is finished, the link shows 'Container > does not exist.' -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13891672#comment-13891672 ] Jian He commented on YARN-1206: --- I remember this was working on branch-2, but failed on trunk, it should be fine > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Priority: Blocker > > With log aggregation disabled, when container is running, its logs link works > properly, but after the application is finished, the link shows 'Container > does not exist.' -- This message was sent by Atlassian JIRA (v6.1.5#6160)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[
https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13918064#comment-13918064
]
Rohith commented on YARN-1206:
--
I am able to reproduce this issue in today's trunk with log aggregation
disable. I verified hadoop-2.1 , this issue does not ocure.
I just going through fix for YARN-649, found there is null check for container
in ContainerLogsUtils.getContainerLogDirs() method.
{noformat}
if (container == null) {
throw new YarnException("Container does not exist.");
}
{noformat}
In hadoop-2.1, above piece of code not there. I am not pretty sure why this is
added.!!
Basically if container is COMPLETED than it will be removed from NMContext (
NodeStatusUpdaterImpl.updateAndGetContainerStatuses() ). NM does not have any
information regarding this container.
Is it really required to have this check ?
> Container logs link is broken on RM web UI after application finished
> -
>
> Key: YARN-1206
> URL: https://issues.apache.org/jira/browse/YARN-1206
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jian He
>Priority: Blocker
>
> With log aggregation disabled, when container is running, its logs link works
> properly, but after the application is finished, the link shows 'Container
> does not exist.'
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[
https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13919317#comment-13919317
]
Hadoop QA commented on YARN-1206:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12632490/YARN-1206.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3240//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3240//console
This message is automatically generated.
> Container logs link is broken on RM web UI after application finished
> -
>
> Key: YARN-1206
> URL: https://issues.apache.org/jira/browse/YARN-1206
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jian He
>Assignee: Rohith
>Priority: Blocker
> Attachments: YARN-1206.patch
>
>
> With log aggregation disabled, when container is running, its logs link works
> properly, but after the application is finished, the link shows 'Container
> does not exist.'
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13935658#comment-13935658 ] Jian He commented on YARN-1206: --- Thanks for the patch ! LGTM, can you add a comment say why we should not check container == null ? > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Assignee: Rohith >Priority: Blocker > Attachments: YARN-1206.patch > > > With log aggregation disabled, when container is running, its logs link works > properly, but after the application is finished, the link shows 'Container > does not exist.' -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937439#comment-13937439 ] Rohith commented on YARN-1206: -- Hi Jian, Thank you for looking into patch. There are 2 scenarios. 1. When Container is Running, NM maintains all running container from NMContext. Accessing container logs link is validated by getting container from NMContext(container == null ) which works fine. 2. When Container is Completed, NM removes completed container from NMContext(NodeStatusUpdaterImpl.removeCompletedContainersFromContext()). After application is finished, accessing container logs link tying to validate containerId from NMContext which is already removed in heartbeat. When log aggregation is disabled, request for container log view forwarded to NM. But NM does not have completed container list. So, for already finished containers, we need not to check for null. > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Assignee: Rohith >Priority: Blocker > Attachments: YARN-1206.patch > > > With log aggregation disabled, when container is running, its logs link works > properly, but after the application is finished, the link shows 'Container > does not exist.' -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[
https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13937762#comment-13937762
]
Hadoop QA commented on YARN-1206:
-
{color:green}+1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12635068/YARN-1206.1.patch
against trunk revision .
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:green}+1 tests included{color}. The patch appears to include 2 new
or modified test files.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:green}+1 javadoc{color}. There were no new javadoc warning messages.
{color:green}+1 eclipse:eclipse{color}. The patch built with
eclipse:eclipse.
{color:green}+1 findbugs{color}. The patch does not introduce any new
Findbugs (version 1.3.9) warnings.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:green}+1 core tests{color}. The patch passed unit tests in
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.
{color:green}+1 contrib tests{color}. The patch passed contrib unit tests.
Test results:
https://builds.apache.org/job/PreCommit-YARN-Build/3373//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/3373//console
This message is automatically generated.
> Container logs link is broken on RM web UI after application finished
> -
>
> Key: YARN-1206
> URL: https://issues.apache.org/jira/browse/YARN-1206
> Project: Hadoop YARN
> Issue Type: Bug
>Reporter: Jian He
>Assignee: Rohith
>Priority: Blocker
> Attachments: YARN-1206.1.patch, YARN-1206.patch
>
>
> With log aggregation disabled, when container is running, its logs link works
> properly, but after the application is finished, the link shows 'Container
> does not exist.'
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13938168#comment-13938168 ] Vinod Kumar Vavilapalli commented on YARN-1206: --- [~jianhe], can you look this in the context of YARN-1685 and see if these two patches go together? > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Assignee: Rohith >Priority: Blocker > Attachments: YARN-1206.1.patch, YARN-1206.patch > > > With log aggregation disabled, when container is running, its logs link works > properly, but after the application is finished, the link shows 'Container > does not exist.' -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13938174#comment-13938174 ] Vinod Kumar Vavilapalli commented on YARN-1206: --- Okay, never mind, they are completely orthogonal. > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Assignee: Rohith >Priority: Blocker > Attachments: YARN-1206.1.patch, YARN-1206.patch > > > With log aggregation disabled, when container is running, its logs link works > properly, but after the application is finished, the link shows 'Container > does not exist.' -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13938332#comment-13938332 ] Jian He commented on YARN-1206: --- LGTM, tested on single node cluster and also in the case of RM restart, patch works properly. Thanks Rohith ! > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Assignee: Rohith >Priority: Blocker > Attachments: YARN-1206.1.patch, YARN-1206.patch > > > With log aggregation disabled, when container is running, its logs link works > properly, but after the application is finished, the link shows 'Container > does not exist.' -- This message was sent by Atlassian JIRA (v6.2#6252)
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769192#comment-13769192 ] Zhijie Shen commented on YARN-1206: --- I observed "Container does not exist." for the containers whose Map/Reduce task is completed, but the whole MR application is still running > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Priority: Blocker > Labels: 2.1.1-beta > > When container is running, its logs link works properly, but after the > application is finished, the link shows 'Container does not exist.' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769199#comment-13769199 ] Lohit Vijayarenu commented on YARN-1206: We have seen case where upon enabling log aggregation, container link are broken because logs are aggregated to HDFS. If link is not updated to point ot history server, those will be broken links. Broken here is to say that nodemangaer will not be able to display logs (since they are aggregated to hdfs) One way to reproduce this is, run application to completion, then click application link, something like http://hadoop-rm-host:port/cluster/app/application_1379365648572_0001 Then click on 'logs' link next to ApplicationMaster attempt. This will point to page on NM displaying the below message "Failed while trying to construct the redirect url to the log server. Log Server url may not be configured Unknown container. Container either has not started or has already completed or doesn't belong to this node at all." > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Priority: Blocker > Labels: 2.1.1-beta > > When container is running, its logs link works properly, but after the > application is finished, the link shows 'Container does not exist.' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769201#comment-13769201 ] Jian He commented on YARN-1206: --- Hi [~lohit], thanks for clarifying, here the situation is that log aggregation is disabled. > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Priority: Blocker > Labels: 2.1.1-beta > > With log aggregation disabled, when container is running, its logs link works > properly, but after the application is finished, the link shows 'Container > does not exist.' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13770430#comment-13770430 ] Sandy Ryza commented on YARN-1206: -- Did this work differently prior to YARN-649? My impression was that this was the existing behavior. > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Priority: Blocker > Labels: 2.1.1-beta > > With log aggregation disabled, when container is running, its logs link works > properly, but after the application is finished, the link shows 'Container > does not exist.' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (YARN-1206) Container logs link is broken on RM web UI after application finished
[ https://issues.apache.org/jira/browse/YARN-1206?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13771425#comment-13771425 ] Jian He commented on YARN-1206: --- Yeah, it worked. > Container logs link is broken on RM web UI after application finished > - > > Key: YARN-1206 > URL: https://issues.apache.org/jira/browse/YARN-1206 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Jian He >Priority: Blocker > > With log aggregation disabled, when container is running, its logs link works > properly, but after the application is finished, the link shows 'Container > does not exist.' -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
