[jira] [Commented] (SPARK-32464) Support skew handling on join that has one side with no query stage
[ https://issues.apache.org/jira/browse/SPARK-32464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166072#comment-17166072 ] Apache Spark commented on SPARK-32464: -- User 'wangshisan' has created a pull request for this issue: https://github.com/apache/spark/pull/29266 > Support skew handling on join that has one side with no query stage > --- > > Key: SPARK-32464 > URL: https://issues.apache.org/jira/browse/SPARK-32464 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.0.0 >Reporter: Wang, Gang >Priority: Major > > In our production environment, there are many bucket tables, which are used > to join with other tables. And there are some skewed joins now and then. > While, in current implementation, the skew join handling can only applied > when both sides of a SMJ are QueryStages. So skew join handling is not able > to deal with such cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-32464) Support skew handling on join that has one side with no query stage
[ https://issues.apache.org/jira/browse/SPARK-32464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17166071#comment-17166071 ] Wang, Gang commented on SPARK-32464: A PR [https://github.com/apache/spark/pull/29266] > Support skew handling on join that has one side with no query stage > --- > > Key: SPARK-32464 > URL: https://issues.apache.org/jira/browse/SPARK-32464 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.0.0 >Reporter: Wang, Gang >Priority: Major > > In our production environment, there are many bucket tables, which are used > to join with other tables. And there are some skewed joins now and then. > While, in current implementation, the skew join handling can only applied > when both sides of a SMJ are QueryStages. So skew join handling is not able > to deal with such cases. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
