[
https://issues.apache.org/jira/browse/HIVE-18690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sahil Takiar updated HIVE-18690:
--------------------------------
Resolution: Fixed
Fix Version/s: 4.0.0
Status: Resolved (was: Patch Available)
Pushed to master. Thanks Aihua for the review!
Decided not to wait for Hive QA because the most recent patch just fixed
checkstyle issue and the previous run came back green.
> Integrate with Spark OutputMetrics
> ----------------------------------
>
> Key: HIVE-18690
> URL: https://issues.apache.org/jira/browse/HIVE-18690
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Sahil Takiar
> Assignee: Sahil Takiar
> Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-18690.1.patch, HIVE-18690.2.patch,
> HIVE-18690.3.patch, HIVE-18690.4.patch, HIVE-18690.5.patch, HIVE-18690.6.patch
>
>
> Spark has an {{OutputMetrics}} it uses to expose records / bytes written. We
> currently don't integrate with it and the Spark UI shows a blank value for
> output records / bytes. We have our own customer accumulators instead (like
> {{HIVE_RECORDS_OUT}}).
> Spark exposes the {{OutputMetrics}} object inside individual tasks via the
> {{TaskContext.get()}} method. We can use this method to access the
> {{OutputMetrics}} object and update it.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
