[
https://issues.apache.org/jira/browse/YARN-10475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17221774#comment-17221774
]
Jim Brennan commented on YARN-10475:
------------------------------------
This adds the following {{yarn.resourcemanager.nodemanagers}} configuration
properties:
{{heartbeat-interval-scaling-enable}}
* enables heartbeat interval scaling, defaults to false
{{heartbeat-interval-min-ms}}
* If heart-beat interval scaling is enabled, this is the minimum heart-beat
interval in milliseconds.
{{heartbeat-interval-max-ms}}
* If heart-beat interval scaling is enabled, this is the maximum heart-beat
interval in milliseconds.
{{heartbeat-interval-speedup-factor}}
* This controls the degree of adjustment when speeding up heartbeat intervals.
At 1.0, 20% lesser than average CPU utilization will result in a 20% decrease
in heartbeat interval.
{{heartbeat-interval-slowdown-factor}}
* This controls the degree of adjustment when slowing down heartbeat intervals.
At 1.0, 20% greater than average CPU utilization will result in a 20% increase
in heartbeat interval.
> Scale RM-NM heartbeat interval based on node utilization
> --------------------------------------------------------
>
> Key: YARN-10475
> URL: https://issues.apache.org/jira/browse/YARN-10475
> Project: Hadoop YARN
> Issue Type: Improvement
> Components: yarn
> Affects Versions: 2.10.1, 3.4.1
> Reporter: Jim Brennan
> Assignee: Jim Brennan
> Priority: Minor
>
> Add the ability to scale the RM-NM heartbeat interval based on node cpu
> utilization compared to overall cluster cpu utilization. If a node is
> over-utilized compared to the rest of the cluster, it's heartbeat interval
> slows down. If it is under-utilized compared to the rest of the cluster,
> it's heartbeat interval speeds up.
> This is a feature we have been running with internally in production for
> several years. It was developed by [~nroberts], based on the observation
> that larger faster nodes on our cluster were under-utilized compared to
> smaller slower nodes.
> This feature is dependent on [YARN-10450], which added cluster-wide
> utilization metrics.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
