[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739642#comment-14739642 ] Hudson commented on YARN-4106: -- FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #354 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/354/]) YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev 77666105b4557d5706e5844a4ca286917d966c5f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Fix For: 2.8.0 > > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739625#comment-14739625 ] Hudson commented on YARN-4106: -- FAILURE: Integrated in Hadoop-Hdfs-trunk #2293 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk/2293/]) YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev 77666105b4557d5706e5844a4ca286917d966c5f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Fix For: 2.8.0 > > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739454#comment-14739454 ] Hudson commented on YARN-4106: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk #2316 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2316/]) YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev 77666105b4557d5706e5844a4ca286917d966c5f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java * hadoop-yarn-project/CHANGES.txt > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Fix For: 2.8.0 > > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739383#comment-14739383 ] Hudson commented on YARN-4106: -- FAILURE: Integrated in Hadoop-Yarn-trunk #1106 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk/1106/]) YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev 77666105b4557d5706e5844a4ca286917d966c5f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java * hadoop-yarn-project/CHANGES.txt > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Fix For: 2.8.0 > > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739309#comment-14739309 ] Hudson commented on YARN-4106: -- FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #374 (See [https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/374/]) YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev 77666105b4557d5706e5844a4ca286917d966c5f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Fix For: 2.8.0 > > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739290#comment-14739290 ] Hudson commented on YARN-4106: -- FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #368 (See [https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/368/]) YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev 77666105b4557d5706e5844a4ca286917d966c5f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Fix For: 2.8.0 > > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739172#comment-14739172 ] Hudson commented on YARN-4106: -- FAILURE: Integrated in Hadoop-trunk-Commit #8430 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/8430/]) YARN-4106. NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM. (Bibin A Chundatt via wangda) (wangda: rev 77666105b4557d5706e5844a4ca286917d966c5f) * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/ConfigurationNodeLabelsProvider.java * hadoop-yarn-project/CHANGES.txt * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/AbstractNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/java/org/apache/hadoop/yarn/server/nodemanager/nodelabels/TestConfigurationNodeLabelsProvider.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/NodeStatusUpdaterImpl.java > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Fix For: 2.8.0 > > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739113#comment-14739113 ] Wangda Tan commented on YARN-4106: -- bq. ... may be Labels Manager can support additional method which adds the missing labels first and then updates the mapping Doing this could be hard to manage: for example, how to deal with node label removal, you can do that when reference count of a label becomes zero, but resource request could be rejected if we remove a existing label. I would not prefer to add more flexibility to node partition, since it will likely break something. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Fix For: 2.8.0 > > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739097#comment-14739097 ] Naganarasimha G R commented on YARN-4106: - bq. Without a centralized node partition collection, capacity planning will be not straightforward. yes idea is to still have this collection but only difference being if the labels sent from NM is not present @ RM, then may be Labels Manager can support additional method which adds the missing labels first and then updates the mapping. thoughts? Yes it makes more sense when we support node constraint, but if we want to support more flexibility then we can think of supporting this too. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Fix For: 2.8.0 > > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739078#comment-14739078 ] Wangda Tan commented on YARN-4106: -- [~Naganarasimha], bq. Wangda Tan As far as this patch its fine but was wondering to increase usability do we need to support YARN-2728, Support for disabling the Centralized NodeLabel validation in Distributed Node Label Configuration setup ? Since we only support node partition, and node partition relates to capacity planning, etc. Without a centralized node partition collection, capacity planning will be not straightforward. I think YARN-2728 makes more sense when we support node constraint. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Fix For: 2.8.0 > > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739064#comment-14739064 ] Naganarasimha G R commented on YARN-4106: - +1, Applied the latest patch, ran the test cases and applied YARN-2729 on top of this patch and script was also running successfully. Latest patch LGTM. [~leftnoteasy] As far as this patch its fine but was wondering to increase usability do we need to support YARN-2728, ??Support for disabling the Centralized NodeLabel validation in Distributed Node Label Configuration setup?? ? > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14739052#comment-14739052 ] Wangda Tan commented on YARN-4106: -- +1 to latest patch. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14738175#comment-14738175 ] Hadoop QA commented on YARN-4106: - \\ \\ | (/) *{color:green}+1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 41s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 8m 2s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 9m 56s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 36s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 27s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 33s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 11s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | yarn tests | 7m 34s | Tests passed in hadoop-yarn-server-nodemanager. | | | | 46m 26s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12755060/0008-YARN-4106.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / f153710 | | hadoop-yarn-server-nodemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/9074/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/9074/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/9074/console | This message was automatically generated. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch, 0008-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14738065#comment-14738065 ] Hadoop QA commented on YARN-4106: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 40s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 56s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 5s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 37s | The applied patch generated 1 new checkstyle issues (total was 38, now 39). | | {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 31s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 34s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 15s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | yarn tests | 7m 42s | Tests passed in hadoop-yarn-server-nodemanager. | | | | 46m 47s | | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12755044/0007-YARN-4106.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / f153710 | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/9073/artifact/patchprocess/diffcheckstylehadoop-yarn-server-nodemanager.txt | | hadoop-yarn-server-nodemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/9073/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/9073/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/9073/console | This message was automatically generated. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch, 0007-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14737546#comment-14737546 ] Wangda Tan commented on YARN-4106: -- [~bibinchundatt], Jenkins cannot show findbugs report properly. You should try to run "mvn clean compile findbugs:findbugs" under nodemanager project, and check target/findbugs.xml. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14736158#comment-14736158 ] Bibin A Chundatt commented on YARN-4106: Findbug report not showing anything looks like build report problem > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14736112#comment-14736112 ] Hadoop QA commented on YARN-4106: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 37s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 55s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 4s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 36s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 1s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 30s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 35s | The patch built with eclipse:eclipse. | | {color:red}-1{color} | findbugs | 1m 15s | The patch appears to introduce 1 new Findbugs (version 3.0.0) warnings. | | {color:green}+1{color} | yarn tests | 7m 36s | Tests passed in hadoop-yarn-server-nodemanager. | | | | 46m 34s | | \\ \\ || Reason || Tests || | FindBugs | module:hadoop-yarn-server-nodemanager | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12754800/0006-YARN-4106.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / a153b96 | | Findbugs warnings | https://builds.apache.org/job/PreCommit-YARN-Build/9051/artifact/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html | | hadoop-yarn-server-nodemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/9051/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/9051/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf907.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/9051/console | This message was automatically generated. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14736025#comment-14736025 ] Bibin A Chundatt commented on YARN-4106: Hi [~leftnoteasy] Thnks for comments. Updates patch uploaded > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch, > 0006-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14735207#comment-14735207 ] Wangda Tan commented on YARN-4106: -- Thanks update [~bibinchundatt], and comments from [~Naganarasimha]. Few minor comments: 1) Make failLabelResendInterval to final before we can configure it. 2) testConfigTimer sleep time is too much, I don't know if Clock can be used in Timer, I think you can set NM_NODE_LABELS_PROVIDER_FETCH_INTERVAL_MS to lower value, like 1000, and sleep 1500 ms. 3) With changes in your patch, testNodeLabelsFromConfig doesn't need sleep any more? > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > --- > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side > *This jira also handles the below issue too* > Timer Task not getting triggered in Nodemanager for Label update in > nodemanager for distributed scheduling > Task is supposed to trigger every > {{yarn.nodemanager.node-labels.provider.fetch-interval-ms}} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733442#comment-14733442 ] Bibin A Chundatt commented on YARN-4106: Ran locally the same testcases its working fine. Looks like not related to the patch uploaded. Please do review. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733416#comment-14733416 ] Hadoop QA commented on YARN-4106: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 9s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 46s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 12s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 22s | The applied patch does not increase the total number of release audit warnings. | | {color:green}+1{color} | checkstyle | 0m 37s | There were no new checkstyle issues. | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 28s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 13s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | yarn tests | 7m 41s | Tests failed in hadoop-yarn-server-nodemanager. | | | | 46m 4s | | \\ \\ || Reason || Tests || | Timed out tests | org.apache.hadoop.yarn.server.nodemanager.TestNodeStatusUpdater | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12754457/0005-YARN-4106.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / 1dbd8e3 | | hadoop-yarn-server-nodemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/9021/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/9021/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/9021/console | This message was automatically generated. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch, 0005-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14733251#comment-14733251 ] Bibin A Chundatt commented on YARN-4106: [~leftnoteasy] Thnks for looking into the issue and [~Naganarasimha] for clearing first point. Will try to update testcase for trigger check too. Testcases is working perfectly fine locally for me in eclipse. i will recheck and update. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14731697#comment-14731697 ] Naganarasimha G R commented on YARN-4106: - Thanks for the comments [~leftnoteasy], bq. Does this mean we cannot get labels when initializing ConfigurationNodeLabelsProvider? Actually {{timerTask.run();}} in {{serviceStart}} is responsible to fetch the labels for the very first time. Reason for this modification is, if some class extends this in future and forgets to set the labels during the start then it will be missed hence trying to take care of this in AbstractProvider class itself and secondly we cannot choose {{intervalTime to 0}} because there is configuration that timer task need not run at all, so timer itself will not be created in this case, considering all this, he has modified as per the patch. bq. Add a test for new behavior of this JIRA Well for second issue : i felt changes were related to the design level and scenarios are the same. only thing we have missed is to check whether the timer is triggered after the configured interval and the new labels are set, i think we can have one test case for this. Apart from this were you referring to the test case for first issue also ? > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14731620#comment-14731620 ] Wangda Tan commented on YARN-4106: -- [~bibinchundatt], Thanks for update, Some comments: - Not sure if I completely understand the logic. {{updateNodeLabelsFromConfig}} of {{ConfigurationNodeLabelsProvider}} is invoked only when TimerTask scheduled. And TimerTask of AbstractNodeLabelsProvider delayed started for {{intervalTime}}. Does this mean we cannot get labels when initializing ConfigurationNodeLabelsProvider? If so, I think we need to call updateNodeLabelsFromConfig in serviceInit of ConfigurationNodeLabelsProvider. (Another choice is make the delay time from intervalTime to 0, but it runs in another thread, so the behavior is undetermined. To make it simple, I suggest to add it in serviceInit). - Add a test for new behavior of this JIRA? > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch, 0004-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730432#comment-14730432 ] Hadoop QA commented on YARN-4106: - \\ \\ | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote || Subsystem || Runtime || Comment || | {color:blue}0{color} | pre-patch | 16m 41s | Pre-patch trunk compilation is healthy. | | {color:green}+1{color} | @author | 0m 0s | The patch does not contain any @author tags. | | {color:green}+1{color} | tests included | 0m 0s | The patch appears to include 1 new or modified test files. | | {color:green}+1{color} | javac | 7m 59s | There were no new javac warning messages. | | {color:green}+1{color} | javadoc | 10m 5s | There were no new javadoc warning messages. | | {color:green}+1{color} | release audit | 0m 23s | The applied patch does not increase the total number of release audit warnings. | | {color:red}-1{color} | checkstyle | 0m 37s | The applied patch generated 3 new checkstyle issues (total was 38, now 41). | | {color:green}+1{color} | whitespace | 0m 0s | The patch has no lines that end in whitespace. | | {color:green}+1{color} | install | 1m 31s | mvn install still works. | | {color:green}+1{color} | eclipse:eclipse | 0m 32s | The patch built with eclipse:eclipse. | | {color:green}+1{color} | findbugs | 1m 14s | The patch does not introduce any new Findbugs (version 3.0.0) warnings. | | {color:red}-1{color} | yarn tests | 7m 3s | Tests failed in hadoop-yarn-server-nodemanager. | | | | 46m 9s | | \\ \\ || Reason || Tests || | Failed unit tests | hadoop.yarn.server.nodemanager.nodelabels.TestConfigurationNodeLabelsProvider | | | hadoop.yarn.server.nodemanager.TestNodeManagerReboot | | | hadoop.yarn.server.nodemanager.TestNodeManagerResync | | | hadoop.yarn.server.nodemanager.TestNodeStatusUpdaterForLabels | \\ \\ || Subsystem || Report/Notes || | Patch URL | http://issues.apache.org/jira/secure/attachment/12754152/0003-YARN-4106.patch | | Optional Tests | javadoc javac unit findbugs checkstyle | | git revision | trunk / c83d13c | | checkstyle | https://builds.apache.org/job/PreCommit-YARN-Build/9007/artifact/patchprocess/diffcheckstylehadoop-yarn-server-nodemanager.txt | | hadoop-yarn-server-nodemanager test log | https://builds.apache.org/job/PreCommit-YARN-Build/9007/artifact/patchprocess/testrun_hadoop-yarn-server-nodemanager.txt | | Test Results | https://builds.apache.org/job/PreCommit-YARN-Build/9007/testReport/ | | Java | 1.7.0_55 | | uname | Linux asf905.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux | | Console output | https://builds.apache.org/job/PreCommit-YARN-Build/9007/console | This message was automatically generated. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch, > 0003-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730380#comment-14730380 ] Naganarasimha G R commented on YARN-4106: - [~bibinchundatt], Good catch, Basically this was added initially for testing and later removed but got missed over here so basically its sufficient that {{intervalTime}} itself is used as {{delay}} in {{AbstractNodeLabelsProvider.serviceStart}} and also we can remove {{startTime}} & in {{ConfigurationNodeLabelsProvider.serviceInit}} we can remove {{startTime}} and instead of calling {{updateNodeLabelsFromConfig}} in {{serviceInit}} we can override and invoke {{timertask.run}} in {{serviceStart()}}. Please correct the test cases accordingly. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14730338#comment-14730338 ] Bibin A Chundatt commented on YARN-4106: Hi [~leftnoteasy] Also checking distributed labels further found that Timer task is not getting triggers as expected. In {{ConfigurationNodeLabelsProvider#serviceInit}} {{startTime }} is getting set wrongly {code} if (intervalTime != DISABLE_NODE_LABELS_PROVIDER_FETCH_TIMER) { startTime = new Date().getTime() + intervalTime; } {code} Need to remove the same. [~Naganarasimha] have a look at the same will update patch further after your comments.. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > Attachments: 0001-YARN-4106.patch, 0002-YARN-4106.patch > > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727678#comment-14727678 ] Naganarasimha G R commented on YARN-4106: - Thanks for the thoughts [~leftnoteasy], Initially we also thought of the same approach but only thing is: At design level NMDistributedNodeLabelsHandler just asks for labels from provider and AbstractNodeLabelsProvider has the configuration of fetch-interval which periodically fetches the labels using the timer, so design wise would it be better to use the fetch-interval configuration in NMDistributedNodeLabelsHandler for only in the failure case ? or is it good to introduce some new configuration only for failure case here, thoughts ? > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14727590#comment-14727590 ] Wangda Tan commented on YARN-4106: -- [~Naganarasimha], I think do the sync on periodic intervals seems reasonable to me. Maybe we can do this: when a label is not accepted by RM, it will retry every fetch-interval. If a label is accepted by RM, it won't resend unless labels updated in the node. > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14726816#comment-14726816 ] Naganarasimha G R commented on YARN-4106: - Thanks for raising this issue [~bibinchundatt], This issue is happening because we are trying to avoid sending labels everytime to RM as if sent everytime RM needs to validate it and refresh the Label mapping for each Node's HB and its a unnecessary overload @ RM side. So only in this failure case, wherein the Labels are not accepted by RM then may be we can send the labels as part of each HB (or on periodic intervals) send the labels from NM to RM . Thoughts ? cc/ [~leftnoteasy] > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (YARN-4106) NodeLabels for NM in distributed mode is not updated even after clusterNodelabel addition in RM
[ https://issues.apache.org/jira/browse/YARN-4106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14726802#comment-14726802 ] Bibin A Chundatt commented on YARN-4106: Observation = Logs shown below are when Cluster node labels are not available in RM {code} 2015-09-02 10:21:02,766 ERROR org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Node Labels {1} reported from NM with ID host-:64318 was rejected from RM with exception message as : Not all labels being replaced contained by known label collections, please check, new labels=[1] {code} # On restart of NM its updating fine # Looks like once failure happens NM not resending nodelabel status in heartbeat to RM > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > > > Key: YARN-4106 > URL: https://issues.apache.org/jira/browse/YARN-4106 > Project: Hadoop YARN > Issue Type: Bug >Reporter: Bibin A Chundatt >Assignee: Bibin A Chundatt >Priority: Critical > > NodeLabels for NM in distributed mode is not updated even after > clusterNodelabel addition in RM > Steps to reproduce > === > # Configure nodelabel in distributed mode > yarn.node-labels.configuration-type=distributed > provider = config > yarn.nodemanager.node-labels.provider.fetch-interval-ms=12ms > # Start RM the NM > # Once NM is registration is done add nodelabels in RM > Nodelabels not getting updated in RM side -- This message was sent by Atlassian JIRA (v6.3.4#6332)