[
https://issues.apache.org/jira/browse/YARN-6435?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15954779#comment-15954779
]
Varun Saxena commented on YARN-6435:
------------------------------------
Thanks [~rohithsharma] for raising the issue.
A similar limit exists for Application Table too. So probably change should be
made there too.
However, having said that if we increase max versions to max value of integer,
it will directly impact the storage footprint for the table and space required
to store corresponding data.
Especially when it comes to entity table, the table can be quite large.
If a metric is printed every 10 second, this means we can at max retrieve a
time series for 3 hours approximately which sounds quite less.
Maybe we can provide a configuration to control how many versions should we
store for metric column family with a higher suitable default?
[~vrushalic], your thoughts on the same?
> [ATSv2] Can't retrieve more than 1000 versions of metrics in time series
> ------------------------------------------------------------------------
>
> Key: YARN-6435
> URL: https://issues.apache.org/jira/browse/YARN-6435
> Project: Hadoop YARN
> Issue Type: Bug
> Components: timelineserver
> Reporter: Rohith Sharma K S
> Assignee: Rohith Sharma K S
> Priority: Critical
> Attachments: YARN-6435.0001.patch
>
>
> It is observed that, even though *metricslimit* is set to 1500, maximum
> number of metrics values retrieved is 1000.
> This is due to, while creating EntityTable, metrics column family max version
> is specified as 1000 which is hardcoded in
> {{EntityTable#DEFAULT_METRICS_MAX_VERSIONS}}. So, HBase will return max
> version with following {{MIN(cf max version , user provided max version)}}.
> This behavior is contradicting the documentation which claims that
> {code}
> metricslimit - If specified, defines the number of metrics to return.
> Considered only if fields contains METRICS/ALL or metricstoretrieve is
> specified. Ignored otherwise. The maximum possible value for metricslimit can
> be maximum value of Integer. If it is not specified or has a value less than
> 1, and metrics have to be retrieved, then metricslimit will be considered as
> 1 i.e. latest single value of metric(s) will be returned.
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
