Github user ioana-delaney commented on the issue: https://github.com/apache/spark/pull/15363 The following updates were made: 1. Incorporate table and column statistics into the star join detection algorithm. Fact table is chosen based on table cardinality, and dimensions are chosen based on the RI constraints. To infer column uniqueness, the algorithm uses table and column statistics. It compares the number of distinct values with the total number of rows in the table. If their relative difference is within certain limits, the column is assumed to be unique. The updated design document is uploaded to https://issues.apache.org/jira/secure/attachment/12843316/StarJoinReordering1214.doc. 2. Move star join test cases under Hive test suite, which currently supports statistics. 3. Rerun TPCDS 1TB with the new table and column statistics. The results are shown in the design doc. @wzhfy and @davies Would you please review the changes?
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org