mohitgurav20 commented on issue #25622: URL: https://github.com/apache/datafusion/issues/25622#issuecomment-5821199160
Hi @gabotechs, thanks for putting together this detailed issue and repro! This looks like a great cardinality estimation improvement for the TPC-DS epic. I took a brief look at BoundedWindowAggExec (and WindowAggExec), and it seems we're currently just appending ColumnStatistics::new_unknown() for the generated window expression columns. I'd love to take this on! My plan would be to update the statistics() implementation for the window execution nodes to properly derive bounds for ranking functions. For unpartitioned ROW_NUMBER(), we can easily establish an upper bound based on the input row count estimate. For partitioned windows or tie-aware functions like RANK(), we can compute tighter bounds using the input's partition count estimates and row counts. Could you assign this to me if no one else is currently working on it? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
