voonhous opened a new issue, #20634: URL: https://github.com/apache/hudi/issues/20634
Part of #20625. Line references are against master `db90a0da`. ### Problem Separate from the MOR read path; it affects COW baselines. On a table whose metadata table has no column stats, the connector appears to expose no table statistics, so the planner keeps the syntactic join order. TPC-DS writes `FROM date_dim dt, store_sales, item`, which puts the fact table on the build side: the date dynamic filter then flows from the fact table onto `date_dim` and never prunes the fact scan. This is a hypothesis until confirmed. ### Steps - Confirm with `SHOW STATS` on both tables and `EXPLAIN` of a TPC-DS q03-style join. Locally, a dpp variant of `HudiMorReadBenchmark` written in TPC-DS FROM order on the stats-less generated tables should reproduce it. - Decide the fix: (a) the connector derives basic statistics (row count and size from the file listing, partition counts) when the metadata table has no column stats; or (b) document that statistics need the column stats index. ### Exit criteria - Root cause confirmed with `SHOW STATS` + `EXPLAIN` output attached here. - Either (a) merged with a test showing a table with no column stats still gets a row-count estimate and q03-style joins put the dimension on the build side, or (b) the decision recorded and documented. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
