[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16661616#comment-16661616 ] Chen Yufei commented on YARN-8513: -- Our YARN cluster is now working fine. Thanks [~cheers

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16654575#comment-16654575 ] Weiwei Yang commented on YARN-8513: --- Thanks [~leftnoteasy], [~hustnn], I created YARN-88

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652886#comment-16652886 ] niu commented on YARN-8513: --- [~cheersyang] [~leftnoteasy] Thanks. > CapacityScheduler infinite

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16652645#comment-16652645 ] Wangda Tan commented on YARN-8513: -- Sounds like a plan, default value set to 100 may make

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651349#comment-16651349 ] Weiwei Yang commented on YARN-8513: --- Hi [~leftnoteasy]/[~cyfdecyf]/[~hustnn]/[~Tao Yang]

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16651024#comment-16651024 ] Chen Yufei commented on YARN-8513: -- Make some corrections to my previous comment.   The

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649797#comment-16649797 ] Chen Yufei commented on YARN-8513: -- [~Tao Yang] I encounter this problem again today. I s

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16623040#comment-16623040 ] Tao Yang commented on YARN-8513: Hi, [~cyfdecyf] Dose your cluster have empty resource typ

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611742#comment-16611742 ] Chen Yufei commented on YARN-8513: -- [~cheersyang] Thanks for your help. I haven't run int

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16611677#comment-16611677 ] Weiwei Yang commented on YARN-8513: --- Hi [~cyfdecyf] Did the config changes I suggested

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16610141#comment-16610141 ] Chen Yufei commented on YARN-8513: -- [~leftnoteasy] Thanks for taking time to investigate

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16608263#comment-16608263 ] niu commented on YARN-8513: --- [~leftnoteasy] . No problem. Take your time. Let me try first. > C

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16608213#comment-16608213 ] Wangda Tan commented on YARN-8513: -- [~hustnn], I agree that it is still a problem, but r

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607901#comment-16607901 ] niu commented on YARN-8513: --- Thanks [~leftnoteasy] for your effort to look at this problem. In

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607495#comment-16607495 ] Wangda Tan commented on YARN-8513: -- Spent good amount of time to check the issue. I foun

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602522#comment-16602522 ] niu commented on YARN-8513: --- Debug dump: {code:java} 2018-09-03 11:44:11,175 DEBUG org.apache.h

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602519#comment-16602519 ] niu commented on YARN-8513: --- Attached the debug log. > CapacityScheduler infinite loop when que

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602507#comment-16602507 ] niu commented on YARN-8513: --- I also tested. This happens when node label is not used. Does it ca

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602326#comment-16602326 ] Wangda Tan commented on YARN-8513: -- And btw, I found a comment in LeafQueue: {code:java}

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16602323#comment-16602323 ] Wangda Tan commented on YARN-8513: -- Interesting, it must be caused by CS allocation doesn

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16601223#comment-16601223 ] niu commented on YARN-8513: --- Thank. I will try it tomorrow. > CapacityScheduler infinite loop

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16601171#comment-16601171 ] Janus Chow commented on YARN-8513: -- [~leftnoteasy] Can reproduce as follow: 1.Specify 2

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16601138#comment-16601138 ] Janus Chow commented on YARN-8513: -- Set two queues: - queue1:     - capacity: 50     - ma

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16593141#comment-16593141 ] niu commented on YARN-8513: --- OK. I will try. > CapacityScheduler infinite loop when queue is ne

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16591878#comment-16591878 ] Wangda Tan commented on YARN-8513: -- [~hustnn], what is the cause of "Failed to accept all

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16591337#comment-16591337 ] niu commented on YARN-8513: --- I also check the logic for printing out of the error "Failed to acc

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16588238#comment-16588238 ] Chen Yufei commented on YARN-8513: -- [~leftnoteasy] My original config did not have the tw

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586854#comment-16586854 ] Wangda Tan commented on YARN-8513: -- Interesting, [~cheersyang], I can only think about

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16586811#comment-16586811 ] Weiwei Yang commented on YARN-8513: --- Discussed with [~cyfdecyf] in the slack channel, it

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585359#comment-16585359 ] Weiwei Yang commented on YARN-8513: --- Thanks for the info [~cyfdecyf]. To work more effic

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16585036#comment-16585036 ] Chen Yufei commented on YARN-8513: -- [~cheersyang] Thanks for looking into this issue. Th

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16584825#comment-16584825 ] Weiwei Yang commented on YARN-8513: --- Hi [~cyfdecyf] >From the RM log you uploaded, in 1

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16584787#comment-16584787 ] Chen Yufei commented on YARN-8513: -- New jstack/top and RM logs are uploaded and prefixed

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583270#comment-16583270 ] Chen Yufei commented on YARN-8513: -- [~leftnoteasy] Thanks for your help. I'll enable capa

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16583026#comment-16583026 ] Wangda Tan commented on YARN-8513: -- [~cyfdecyf], Could u upload logs/jstacks for 3.1.0

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16582584#comment-16582584 ] Chen Yufei commented on YARN-8513: -- [~leftnoteasy] We encounter the same problem twice to

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16580862#comment-16580862 ] Chen Yufei commented on YARN-8513: -- We got infinite loops two times recently with 2.9.1,

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16552536#comment-16552536 ] niu commented on YARN-8513: --- We also met this problem in 2.9.1. > CapacityScheduler infinite lo

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16551551#comment-16551551 ] Yuanbo Liu commented on YARN-8513: -- Sorry for the late response. Quite busy this week. I

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16544742#comment-16544742 ] Chen Yufei commented on YARN-8513: -- [~yuanbo] I've uploaded jstack and top log when the p

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16544234#comment-16544234 ] Chen Yufei commented on YARN-8513: -- [~leftnoteasy] Thanks for your suggestions. I'll depl

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16544231#comment-16544231 ] Chen Yufei commented on YARN-8513: -- [~yuanbo] I'll capture those info when I encounter th

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543689#comment-16543689 ] Wangda Tan commented on YARN-8513: -- [~cyfdecyf], I couldn't find the error message on the

[jira] [Commented] (YARN-8513) CapacityScheduler infinite loop when queue is near fully utilized

[ https://issues.apache.org/jira/browse/YARN-8513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16542719#comment-16542719 ] Yuanbo Liu commented on YARN-8513: -- [~cyfdecyf] Can you reproduce this issue and capture