[jira] [Commented] (YARN-4670) add logging when a node is AM-blacklisted
[
https://issues.apache.org/jira/browse/YARN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15133507#comment-15133507
]
Sangjin Lee commented on YARN-4670:
---
{quote}
volume of logging depends on the number of nodes which are free and and how
many are blacklisted for the application and how long other nodes are occupied.
but IMO it would be rare scenario volume explodes and we can have logging for
it in AppSchedulingInfo.isBlackListed , Thoughts ?
{quote}
Yes that sounds like a plan.
> add logging when a node is AM-blacklisted
> -
>
> Key: YARN-4670
> URL: https://issues.apache.org/jira/browse/YARN-4670
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.8.0
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
>Priority: Trivial
>
> Today there is not much logging happening when a node is blacklisted for an
> AM (see YARN-2005). We can add a little more logging to see this activity
> easily from the RM logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4670) add logging when a node is AM-blacklisted
[
https://issues.apache.org/jira/browse/YARN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131643#comment-15131643
]
Naganarasimha G R commented on YARN-4670:
-
hi [~sjlee0],
To an extent now we wil be able to find it out after YARN-4307 and YARN-3946,
and also in the trunk code i am able to see a debug log for the same in
{{SchedulerAppUtils.isBlackListed}}, anything more is planned for this ?
> add logging when a node is AM-blacklisted
> -
>
> Key: YARN-4670
> URL: https://issues.apache.org/jira/browse/YARN-4670
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: resourcemanager
>Affects Versions: 2.8.0
>Reporter: Sangjin Lee
>Assignee: Sangjin Lee
>Priority: Trivial
>
> Today there is not much logging happening when a node is blacklisted for an
> AM (see YARN-2005). We can add a little more logging to see this activity
> easily from the RM logs.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (YARN-4670) add logging when a node is AM-blacklisted
[ https://issues.apache.org/jira/browse/YARN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131820#comment-15131820 ] Naganarasimha G R commented on YARN-4670: - YARN-3946 informs if the app's AM is stuck and if it misses to schedule on a node due to its blacklisting for launching AM's for this app. bq. Regarding logging in SchedulerAppUtils.isBlackListed(), does that get used for the AM blacklisting too? It's not obvious to me. *SchedulerAppUtils.isBlackListed* -> *SchedulerApplicationAttempt.isBlackListed* -> *AppSchedulingInfo.isBlackListed* and finally in the last call they are checking for the AM black list so in a way logging is there but as you said its not too obvious. ??volume of logging?? depends on the number of nodes which are free and and how many are blacklisted for the application and how long other nodes are occupied. but IMO it would be rare scenario volume explodes and we can have logging for it in *AppSchedulingInfo.isBlackListed* , Thoughts ? > add logging when a node is AM-blacklisted > - > > Key: YARN-4670 > URL: https://issues.apache.org/jira/browse/YARN-4670 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.8.0 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Trivial > > Today there is not much logging happening when a node is blacklisted for an > AM (see YARN-2005). We can add a little more logging to see this activity > easily from the RM logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4670) add logging when a node is AM-blacklisted
[ https://issues.apache.org/jira/browse/YARN-4670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15131726#comment-15131726 ] Sangjin Lee commented on YARN-4670: --- Thanks for the info. I missed YARN-4307. Regarding logging in SchedulerAppUtils.isBlackListed(), does that get used for the *AM* blacklisting too? It's not obvious to me. I was looking more at RMAppAttemptImpl.sendAMContainerToNM(). Also, it would be good if this can be logged at the INFO level, as I don't think the volume of this logging is going to be too much and logging this during the normal operation would be useful? > add logging when a node is AM-blacklisted > - > > Key: YARN-4670 > URL: https://issues.apache.org/jira/browse/YARN-4670 > Project: Hadoop YARN > Issue Type: Improvement > Components: resourcemanager >Affects Versions: 2.8.0 >Reporter: Sangjin Lee >Assignee: Sangjin Lee >Priority: Trivial > > Today there is not much logging happening when a node is blacklisted for an > AM (see YARN-2005). We can add a little more logging to see this activity > easily from the RM logs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
