[ https://issues.apache.org/jira/browse/YARN-10475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17223668#comment-17223668 ]
Jim Brennan commented on YARN-10475: ------------------------------------ Thanks for the suggestion [~bibinchundatt]! I think a plugin for calculating the heartbeat interval is definitely possible. The configs as specified I think could remain for enabling scaling and setting up the parameters - there is nothing specific about cpu utilization in those properties. Would you be ok with a follow-up Jira to move the calculation into a plugin? Do you have any suggestions for alternate calculations? > Scale RM-NM heartbeat interval based on node utilization > -------------------------------------------------------- > > Key: YARN-10475 > URL: https://issues.apache.org/jira/browse/YARN-10475 > Project: Hadoop YARN > Issue Type: Improvement > Components: yarn > Affects Versions: 2.10.1, 3.4.1 > Reporter: Jim Brennan > Assignee: Jim Brennan > Priority: Minor > Attachments: YARN-10475.001.patch, YARN-10475.002.patch, > YARN-10475.003.patch > > > Add the ability to scale the RM-NM heartbeat interval based on node cpu > utilization compared to overall cluster cpu utilization. If a node is > over-utilized compared to the rest of the cluster, it's heartbeat interval > slows down. If it is under-utilized compared to the rest of the cluster, > it's heartbeat interval speeds up. > This is a feature we have been running with internally in production for > several years. It was developed by [~nroberts], based on the observation > that larger faster nodes on our cluster were under-utilized compared to > smaller slower nodes. > This feature is dependent on [YARN-10450], which added cluster-wide > utilization metrics. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org