[
https://issues.apache.org/jira/browse/HIVE-9128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14249201#comment-14249201
]
Jimmy Xiang commented on HIVE-9128:
-----------------------------------
Played with this setting on spark. With query 28, the total time (ms, sum up
all the durations for the corresponding method):
|| || with the option off || with the option on ||
| writing the plan | 1124 | 156 |
| reading the plan | 299 | 51 |
It looks like this option does give us some benefit. However, the gain is not
very significant, since the query itself takes about 80+ seconds.
> Evaluate hive.rpc.query.plan performance [Spark Branch]
> -------------------------------------------------------
>
> Key: HIVE-9128
> URL: https://issues.apache.org/jira/browse/HIVE-9128
> Project: Hive
> Issue Type: Sub-task
> Components: Spark
> Reporter: Brock Noland
> Assignee: Jimmy Xiang
>
> Tez uses
> [hive.rpc.query.plan|https://github.com/apache/hive/blob/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java#L1874]
> which is used in {{Utilities.java}}. Basically instead of writing the query
> plan to HDFS, the query plan is placed in the JobConf object and then
> de-serialized form there.
> We should do some evaluation to see which is more performant for us. We might
> need to place some timings in {{Utilities}} to understand this if the PerfLog
> doesn't have enough information today.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
