[
https://issues.apache.org/jira/browse/SPARK-4849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14990050#comment-14990050
]
Yin Huai commented on SPARK-4849:
---------------------------------
oh, actually SPARK-5354. SPARK-11410 adds APIs to specify how to shuffle data.
> Pass partitioning information (distribute by) to In-memory caching
> ------------------------------------------------------------------
>
> Key: SPARK-4849
> URL: https://issues.apache.org/jira/browse/SPARK-4849
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 1.2.0
> Reporter: Nitin Goyal
> Assignee: Nong Li
> Priority: Critical
> Fix For: 1.6.0
>
>
> HQL "distribute by <column_name>" partitions data based on specified column
> values. We can pass this information to in-memory caching for further
> performance improvements. e..g. in Joins, an extra partition step can be
> saved based on this information.
> Refer -
> http://apache-spark-user-list.1001560.n3.nabble.com/SchemaRDD-partition-on-specific-column-values-td20350.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
