[
https://issues.apache.org/jira/browse/HIVE-2446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13106625#comment-13106625
]
[email protected] commented on HIVE-2446:
-----------------------------------------------------
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/1909/#review1935
-----------------------------------------------------------
Ship it!
- Ning
On 2011-09-16 17:35:58, Robert Surówka wrote:
bq.
bq. -----------------------------------------------------------
bq. This is an automatically generated e-mail. To reply, visit:
bq. https://reviews.apache.org/r/1909/
bq. -----------------------------------------------------------
bq.
bq. (Updated 2011-09-16 17:35:58)
bq.
bq.
bq. Review request for hive and Ning Zhang.
bq.
bq.
bq. Summary
bq. -------
bq.
bq. The purpose of this change is to allow publication or storage of counters
while the job is running.
bq.
bq. Introduced two new variables to hive-default.xml and HiveConf.java:
"hive.client.stats.publishers" and "hive.client.stats.counters". First one
specifies classes names, whose instances will be executed by
HadoopJobExecHelper.java (similarly as hooks are) in its method
progress(ExecDriverTaskHandle): MapRedStats. Second one specifies list of
counters that any client stat publishers should publish or stored. Details
regarding format of this list is up to a specific deployment (it is up to
client stats publishers to parse it), yet it is required to use display names
of counter groups and counters.
bq.
bq. Added interface ClientStatsPublishers in org.apache.hadoop.hive.ql.stats
package, that must be implemented by all stats publishers.
bq.
bq. Added code to progress(ExecDriverTaskHandle): MapRedStats from
HadoopJobExecHelper.java that puts counters' values to a Java map and then
executes registered client stats publishers giving them that map and running
job id. Added two new methods to HadoopJobExecHelper:
extractAllCounterValues(Counters) and getClientStatsPublishers() that are used
by code from previous sentence.
bq.
bq. Made cosmetic changes in one other class
bq.
bq.
bq. This addresses bug HIVE-2446.
bq. https://issues.apache.org/jira/browse/HIVE-2446
bq.
bq.
bq. Diffs
bq. -----
bq.
bq. trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 1171283
bq. trunk/conf/hive-default.xml 1171283
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/HadoopJobExecHelper.java
1171283
bq. trunk/ql/src/java/org/apache/hadoop/hive/ql/history/HiveHistory.java
1171283
bq.
trunk/ql/src/java/org/apache/hadoop/hive/ql/stats/ClientStatsPublisher.java
PRE-CREATION
bq.
bq. Diff: https://reviews.apache.org/r/1909/diff
bq.
bq.
bq. Testing
bq. -------
bq.
bq. Run some random tests, and still running the unit tests.
bq.
bq.
bq. Thanks,
bq.
bq. Robert
bq.
bq.
> Introduction of client statistics publishers possibility
> --------------------------------------------------------
>
> Key: HIVE-2446
> URL: https://issues.apache.org/jira/browse/HIVE-2446
> Project: Hive
> Issue Type: Improvement
> Components: Clients, Statistics
> Reporter: Robert Surówka
> Assignee: Robert Surówka
> Priority: Minor
> Attachments: HIVE-2446.1.patch, HIVE-2446.1.patch, HIVE-2446.2.patch
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> The purpose of this change is to allow publication or storage of counters
> while the job is running.
> Introduced two new variables to hive-default.xml and HiveConf.java:
> "hive.client.stats.publishers" and "hive.client.stats.counters". First one
> specifies classes names, whose instances will be executed by
> HadoopJobExecHelper.java (similarly as hooks are) in its method
> progress(ExecDriverTaskHandle): MapRedStats. Second one specifies list of
> counters that any client stat publishers should publish or stored. Details
> regarding format of this list is up to a specific deployment (it is up to
> client stats publishers to parse it), yet it is required to use display names
> of counter groups and counters.
> Added interface ClientStatsPublishers in org.apache.hadoop.hive.ql.stats
> package, that must be implemented by all stats publishers.
> Added code to progress(ExecDriverTaskHandle): MapRedStats from
> HadoopJobExecHelper.java that puts counters' values to a Java map and then
> executes registered client stats publishers giving them that map and running
> job id. Added two new methods to HadoopJobExecHelper:
> extractAllCounterValues(Counters) and getClientStatsPublishers() that are
> used by code from previous sentence.
> Made cosmetic changes in two other classes
>
--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira
