[
https://issues.apache.org/jira/browse/YARN-4821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15267826#comment-15267826
]
Naganarasimha G R commented on YARN-4821:
-----------------------------------------
Thanks [~sjlee0],
It was majorly WIP patch usingthe existing classes just to check if the
approach is fine, i agree with most of your comments
bq. My preference would be to use a true multiple. If we're going to emit every
n-th time, we should let users define n as the config.
It was oversight it was intended to be as you described, i will correct it in
next patch
bq. It would add to the garbage collection pressure.
Agree thought of using existing class but seems tob not appropriate here will
rework on it and coming up new class we can have arrays for each resource type
and have some variable to indicate number of monitor data already stored and
have a compute avg logic which avoids creation of unnecessary intermediate
objects.
bq. What if we are still accumulating (without publishing) when the container
is finished?
I had plans to publish when the container has finished with what ever existing
data available, this was one edge case which i was aware of and thought of
handling at the end :)
bq. let's instantiate this map in serviceInit() only if NM timeline publisher
is enabled
agree
bq. I am puzzled by this. Why are we re-defining this memory metric to be in
MB? Is this necessary as part of this patch?
when using REsource utilization i realized they were converting into MB and
then accumulating at container level and sending across to RM. I felt this
solution was correct and we too should capture MB only as in most of the cases
we containers will be configured in MB (and default value also is KB) so felt
there was no use in capturing bytes level data as it would not be of much use
as mostly we will be handling in MB's and GB's. Also it would unnecessary
storage of data. Thoughts?
> Have a separate NM timeline publishing-interval
> -----------------------------------------------
>
> Key: YARN-4821
> URL: https://issues.apache.org/jira/browse/YARN-4821
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-2928
> Reporter: Sangjin Lee
> Assignee: Naganarasimha G R
> Labels: yarn-2928-1st-milestone
> Attachments: YARN-4821-YARN-2928.v1.001.patch
>
>
> Currently the interval with which NM publishes container CPU and memory
> metrics is tied to {{yarn.nodemanager.resource-monitor.interval-ms}} whose
> default is 3 seconds. This is too aggressive.
> There should be a separate configuration that controls how often
> {{NMTimelinePublisher}} publishes container metrics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
