[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17059890#comment-17059890 ]
Takeshi Yamamuro commented on SPARK-17495: ------------------------------------------ Ah, I see. I've looked over the merged PRs and, in the branch-2.4 (the master, too), it seems the hive hash function is not used yet in partitioning/bucketing. So, we need more work to use the hash there. This Jira just targets the implementation of the hive-simulated hash, so I think we can close this now. > Hive hash implementation > ------------------------ > > Key: SPARK-17495 > URL: https://issues.apache.org/jira/browse/SPARK-17495 > Project: Spark > Issue Type: Sub-task > Components: SQL > Affects Versions: 2.2.0 > Reporter: Tejas Patil > Assignee: Tejas Patil > Priority: Minor > Fix For: 2.2.0 > > > Spark internally uses Murmur3Hash for partitioning. This is different from > the one used by Hive. For queries which use bucketing this leads to different > results if one tries the same query on both engines. For us, we want users to > have backward compatibility to that one can switch parts of applications > across the engines without observing regressions. -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org