[jira] [Commented] (SPARK-13213) BroadcastNestedLoopJoin is very slow
[ https://issues.apache.org/jira/browse/SPARK-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174441#comment-15174441 ] Apache Spark commented on SPARK-13213: -- User 'davies' has created a pull request for this issue: https://github.com/apache/spark/pull/11328 > BroadcastNestedLoopJoin is very slow > > > Key: SPARK-13213 > URL: https://issues.apache.org/jira/browse/SPARK-13213 > Project: Spark > Issue Type: Improvement > Components: SQL >Reporter: Davies Liu >Assignee: Davies Liu > Fix For: 2.0.0 > > > Since we have improve the performance of CartisianProduct, which should be > faster and robuster than BroacastNestedLoopJoin, we should do > CartisianProduct instead of BroacastNestedLoopJoin, especially when the > broadcasted table is not that small. > Today, we hit a query that take very long time but still not finished, once > decrease the threshold for broadcast (disable BroacastNestedLoopJoin), it > just finished in seconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13213) BroadcastNestedLoopJoin is very slow
[ https://issues.apache.org/jira/browse/SPARK-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155438#comment-15155438 ] Davies Liu commented on SPARK-13213: It depends, I'm open to any reasonable solution. > BroadcastNestedLoopJoin is very slow > > > Key: SPARK-13213 > URL: https://issues.apache.org/jira/browse/SPARK-13213 > Project: Spark > Issue Type: Improvement > Components: SQL >Reporter: Davies Liu > > Since we have improve the performance of CartisianProduct, which should be > faster and robuster than BroacastNestedLoopJoin, we should do > CartisianProduct instead of BroacastNestedLoopJoin, especially when the > broadcasted table is not that small. > Today, we hit a query that take very long time but still not finished, once > decrease the threshold for broadcast (disable BroacastNestedLoopJoin), it > just finished in seconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13213) BroadcastNestedLoopJoin is very slow
[ https://issues.apache.org/jira/browse/SPARK-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15155418#comment-15155418 ] Reynold Xin commented on SPARK-13213: - [~davies] what is this ticket about? Is it about making BroadcastNestedLoopJoin faster, or using CartesianProduct as much as possible? > BroadcastNestedLoopJoin is very slow > > > Key: SPARK-13213 > URL: https://issues.apache.org/jira/browse/SPARK-13213 > Project: Spark > Issue Type: Improvement > Components: SQL >Reporter: Davies Liu > > Since we have improve the performance of CartisianProduct, which should be > faster and robuster than BroacastNestedLoopJoin, we should do > CartisianProduct instead of BroacastNestedLoopJoin, especially when the > broadcasted table is not that small. > Today, we hit a query that take very long time but still not finished, once > decrease the threshold for broadcast (disable BroacastNestedLoopJoin), it > just finished in seconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13213) BroadcastNestedLoopJoin is very slow
[ https://issues.apache.org/jira/browse/SPARK-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15152051#comment-15152051 ] Apache Spark commented on SPARK-13213: -- User 'viirya' has created a pull request for this issue: https://github.com/apache/spark/pull/11251 > BroadcastNestedLoopJoin is very slow > > > Key: SPARK-13213 > URL: https://issues.apache.org/jira/browse/SPARK-13213 > Project: Spark > Issue Type: Improvement > Components: SQL >Reporter: Davies Liu > > Since we have improve the performance of CartisianProduct, which should be > faster and robuster than BroacastNestedLoopJoin, we should do > CartisianProduct instead of BroacastNestedLoopJoin, especially when the > broadcasted table is not that small. > Today, we hit a query that take very long time but still not finished, once > decrease the threshold for broadcast (disable BroacastNestedLoopJoin), it > just finished in seconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-13213) BroadcastNestedLoopJoin is very slow
[ https://issues.apache.org/jira/browse/SPARK-13213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15137551#comment-15137551 ] Davies Liu commented on SPARK-13213: [~sowen] Thanks very much for update these, I try to remember to add that recently, but may still missed sometimes. Can we mark that as required (or remember the last action as default value)? > BroadcastNestedLoopJoin is very slow > > > Key: SPARK-13213 > URL: https://issues.apache.org/jira/browse/SPARK-13213 > Project: Spark > Issue Type: Improvement > Components: SQL >Reporter: Davies Liu > > Since we have improve the performance of CartisianProduct, which should be > faster and robuster than BroacastNestedLoopJoin, we should do > CartisianProduct instead of BroacastNestedLoopJoin, especially when the > broadcasted table is not that small. > Today, we hit a query that take very long time but still not finished, once > decrease the threshold for broadcast (disable BroacastNestedLoopJoin), it > just finished in seconds. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
