[jira] [Updated] (YARN-8459) Improve logs of Capacity Scheduler to better debug invalid states
[ https://issues.apache.org/jira/browse/YARN-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8459: - Attachment: YARN-8459.004.patch > Improve logs of Capacity Scheduler to better debug invalid states > - > > Key: YARN-8459 > URL: https://issues.apache.org/jira/browse/YARN-8459 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.1.0 >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Major > Attachments: YARN-8459.001.patch, YARN-8459.002.patch, > YARN-8459.003.patch, YARN-8459.004.patch > > > Improve logs in CS to better debug invalid states -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8459) Improve logs of Capacity Scheduler to better debug invalid states
[ https://issues.apache.org/jira/browse/YARN-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8459: - Priority: Major (was: Critical) > Improve logs of Capacity Scheduler to better debug invalid states > - > > Key: YARN-8459 > URL: https://issues.apache.org/jira/browse/YARN-8459 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.1.0 >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Major > Attachments: YARN-8459.001.patch, YARN-8459.002.patch, > YARN-8459.003.patch > > > Improve logs in CS to better debug invalid states -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8459) Improve logs of Capacity Scheduler to better debug invalid states
[ https://issues.apache.org/jira/browse/YARN-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8459: - Description: Improve logs in CS to better debug invalid states (was: Improve logs in CS to better ) > Improve logs of Capacity Scheduler to better debug invalid states > - > > Key: YARN-8459 > URL: https://issues.apache.org/jira/browse/YARN-8459 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.1.0 >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Critical > Attachments: YARN-8459.001.patch, YARN-8459.002.patch, > YARN-8459.003.patch > > > Improve logs in CS to better debug invalid states -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8459) Improve logs of Capacity Scheduler to better debug invalid states
[ https://issues.apache.org/jira/browse/YARN-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wangda Tan updated YARN-8459: - Attachment: YARN-8459.003.patch > Improve logs of Capacity Scheduler to better debug invalid states > - > > Key: YARN-8459 > URL: https://issues.apache.org/jira/browse/YARN-8459 > Project: Hadoop YARN > Issue Type: Bug > Components: capacity scheduler >Affects Versions: 3.1.0 >Reporter: Wangda Tan >Assignee: Wangda Tan >Priority: Critical > Attachments: YARN-8459.001.patch, YARN-8459.002.patch, > YARN-8459.003.patch > > > Improve logs in CS to better debug invalid states -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8459) Improve logs of Capacity Scheduler to better debug invalid states
[
https://issues.apache.org/jira/browse/YARN-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-8459:
-
Description: Improve logs in CS to better (was: Thanks [~gopalv] for
reporting this issue.
In async mode, capacity scheduler can allocate/reserve containers on node/app
when node/app is being removed ({{doneApplicationAttempt}}/{{removeNode}}).
This will cause some issues, for example.
a. Container for app_1 reserved on node_x.
b. At the same time, app_1 is being removed.
c. Reserve on node operation finished after app_1 removed
({{doneApplicationAttempt}}).
For all the future runs, the node_x is completely blocked by the invalid
reservation. It keep reporting "Trying to schedule for a finished app, please
double check" for the node_x.
We need a fix to make sure this won't happen.)
> Improve logs of Capacity Scheduler to better debug invalid states
> -
>
> Key: YARN-8459
> URL: https://issues.apache.org/jira/browse/YARN-8459
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 3.1.0
>Reporter: Wangda Tan
>Assignee: Wangda Tan
>Priority: Critical
> Attachments: YARN-8459.001.patch, YARN-8459.002.patch,
> YARN-8459.003.patch
>
>
> Improve logs in CS to better
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8459) Improve logs of Capacity Scheduler to better debug invalid states
[
https://issues.apache.org/jira/browse/YARN-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-8459:
-
Summary: Improve logs of Capacity Scheduler to better debug invalid states
(was: Capacity Scheduler should properly handle container allocation on
app/node when app/node being removed by scheduler)
> Improve logs of Capacity Scheduler to better debug invalid states
> -
>
> Key: YARN-8459
> URL: https://issues.apache.org/jira/browse/YARN-8459
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 3.1.0
>Reporter: Wangda Tan
>Assignee: Wangda Tan
>Priority: Blocker
> Attachments: YARN-8459.001.patch, YARN-8459.002.patch
>
>
> Thanks [~gopalv] for reporting this issue.
> In async mode, capacity scheduler can allocate/reserve containers on node/app
> when node/app is being removed ({{doneApplicationAttempt}}/{{removeNode}}).
> This will cause some issues, for example.
> a. Container for app_1 reserved on node_x.
> b. At the same time, app_1 is being removed.
> c. Reserve on node operation finished after app_1 removed
> ({{doneApplicationAttempt}}).
> For all the future runs, the node_x is completely blocked by the invalid
> reservation. It keep reporting "Trying to schedule for a finished app, please
> double check" for the node_x.
> We need a fix to make sure this won't happen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
[jira] [Updated] (YARN-8459) Improve logs of Capacity Scheduler to better debug invalid states
[
https://issues.apache.org/jira/browse/YARN-8459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wangda Tan updated YARN-8459:
-
Priority: Critical (was: Blocker)
> Improve logs of Capacity Scheduler to better debug invalid states
> -
>
> Key: YARN-8459
> URL: https://issues.apache.org/jira/browse/YARN-8459
> Project: Hadoop YARN
> Issue Type: Bug
> Components: capacity scheduler
>Affects Versions: 3.1.0
>Reporter: Wangda Tan
>Assignee: Wangda Tan
>Priority: Critical
> Attachments: YARN-8459.001.patch, YARN-8459.002.patch
>
>
> Thanks [~gopalv] for reporting this issue.
> In async mode, capacity scheduler can allocate/reserve containers on node/app
> when node/app is being removed ({{doneApplicationAttempt}}/{{removeNode}}).
> This will cause some issues, for example.
> a. Container for app_1 reserved on node_x.
> b. At the same time, app_1 is being removed.
> c. Reserve on node operation finished after app_1 removed
> ({{doneApplicationAttempt}}).
> For all the future runs, the node_x is completely blocked by the invalid
> reservation. It keep reporting "Trying to schedule for a finished app, please
> double check" for the node_x.
> We need a fix to make sure this won't happen.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
