Xuefu Zhang created HIVE-7526:
---------------------------------
Summary: Research to use groupby transformation to replace Hive
existing partitionByKey and SparkCollector combination
Key: HIVE-7526
URL: https://issues.apache.org/jira/browse/HIVE-7526
Project: Hive
Issue Type: Task
Components: Spark
Reporter: Xuefu Zhang
Currently SparkClient shuffles data by calling paritionByKey(). This
transformation outputs <key, value> tuples. However, Hive's ExecMapper expects
<key, iterator<value>> tuples, and Spark's groupByKey() seems outputing this
directly. Thus, using groupByKey, we may be able to avoid its own key
clustering mechanism (in HiveReduceFunction). This research is to have a try.
--
This message was sent by Atlassian JIRA
(v6.2#6252)
