[
https://issues.apache.org/jira/browse/YARN-4455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16091980#comment-16091980
]
Varun Saxena commented on YARN-4455:
------------------------------------
[~vrushalic],
bq. We should not be using the supplemented timestamp for other tables, since
it will change the meaning of the cell timestamp for entity and application
tables. The HBase TTL will not work if we modify the cell timestamp. We would
like clean up of old metrics to happen after a certain expiration period.
I see. I am not sure how it works in HBase but it is likely that cell timestamp
+ TTL would be checked during compaction to delete data which means metrics
would go away on first compaction. If this is how behavior is, you are correct
that we should store original timestamps instead of supplemented timestamps in
Application and entity table. As I said above, I do not find it necessary to
store supplemented timestamp for these 2 tables. Amount of metric data in Flow
run table would anyways be controlled.
bq. do you know where this is a common code path for flowrun table and the
application/entity tables. Is it during reading?
I was referring to code during read/write in ColumnHelper. I was assuming this
was intentional. However, your point about TTL is most likely correct.
bq. Did you see any issues while trying to write regular timestamps?
No. We can return regular timestamps in all cases. I guess we would need to
return a truncated timestamp from FlowRunCoprocessor as well, even though we
always return single value.
I think we can raise another JIRA to handle this.
> Support fetching metrics by time range
> --------------------------------------
>
> Key: YARN-4455
> URL: https://issues.apache.org/jira/browse/YARN-4455
> Project: Hadoop YARN
> Issue Type: Sub-task
> Components: timelineserver
> Affects Versions: YARN-5355
> Reporter: Varun Saxena
> Assignee: Varun Saxena
> Labels: YARN-5355, yarn-5355-merge-blocker
> Attachments: YARN-4455-YARN-5355.01.patch,
> YARN-4455-YARN-5355.02.patch, YARN-4455-YARN-5355.03.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-issues-h...@hadoop.apache.org
