[
https://issues.apache.org/jira/browse/HIVE-8574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14226517#comment-14226517
]
Marcelo Vanzin commented on HIVE-8574:
--------------------------------------
Actually, after a quick look at the code again, this might not be a problem.
Metrics are kept per-job handle. Job handles are managed by the code submitting
jobs - leave them for garbage collection and metrics go away.
So unless we're worried about a single job creating so many tasks that it will
run the driver out of memory with all the metrics data, this shouldn't really
be an issue.
> Enhance metrics gathering in Spark Client [Spark Branch]
> --------------------------------------------------------
>
> Key: HIVE-8574
> URL: https://issues.apache.org/jira/browse/HIVE-8574
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Marcelo Vanzin
> Assignee: Marcelo Vanzin
>
> The current implementation of metrics gathering in the Spark client is a
> little hacky. First, it's awkward to use (and the implementation is also
> pretty ugly). Second, it will just collect metrics indefinitely, so in the
> long term it turns into a huge memory leak.
> We need a simplified interface and some mechanism for disposing of old
> metrics.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
