[jira] [Commented] (SPARK-17076) Cardinality estimation of join operator
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15734504#comment-15734504 ] Apache Spark commented on SPARK-17076: -- User 'wzhfy' has created a pull request for this issue: https://github.com/apache/spark/pull/16228 > Cardinality estimation of join operator > --- > > Key: SPARK-17076 > URL: https://issues.apache.org/jira/browse/SPARK-17076 > Project: Spark > Issue Type: Sub-task > Components: Optimizer >Affects Versions: 2.0.0 >Reporter: Ron Hu > > support cardinality estimates for equi-join, Cartesian product join, and > outer join, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17076) Cardinality estimation of join operator
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15665664#comment-15665664 ] Ron Hu commented on SPARK-17076: Hi, I am out of office 11 /15 through 11/18 with very limited Internet access. My reply to your email will be delayed. Thanks. Best, Ron Hu > Cardinality estimation of join operator > --- > > Key: SPARK-17076 > URL: https://issues.apache.org/jira/browse/SPARK-17076 > Project: Spark > Issue Type: Sub-task > Components: Optimizer >Affects Versions: 2.0.0 >Reporter: Ron Hu > > support cardinality estimates for equi-join, Cartesian product join, and > outer join, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-17076) Cardinality estimation of join operator
[ https://issues.apache.org/jira/browse/SPARK-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15665644#comment-15665644 ] Ruslan Dautkhanov commented on SPARK-17076: --- HIVE-13076 added FK constraint to Hive / HMS. Joining two tables without declared relation can produce and number of rows. On one extreme case, joining two tables produces zero rows (if there is no overlaps in those join set of ids). On another extream case, joining two tables will produce N x M rows where N and M are number of rows in each case. Luckliy, most often joining happens using columns that can have declared constraints on them. For example, customers.customer_id = purchases.customer_id AND purchases.product_id = products.product_id If you'd build on HIVE-13076, Spark CBO will know exactly that customers.customer_id = purchases.customer_id AND purchases.product_id = products.product_id will always produce 1 row for each join (join cardinality = # of rows on the biggest table). > Cardinality estimation of join operator > --- > > Key: SPARK-17076 > URL: https://issues.apache.org/jira/browse/SPARK-17076 > Project: Spark > Issue Type: Sub-task > Components: Optimizer >Affects Versions: 2.0.0 >Reporter: Ron Hu > > support cardinality estimates for equi-join, Cartesian product join, and > outer join, etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
