adriangb opened a new issue, #17599: URL: https://github.com/apache/datafusion/issues/17599
This patterns shows up pretty often: ```sql select expensive(col) from t where expensive(col) ``` A pathological case is variant / json: ```sql select variant_get(col, 'key') from t from variant_get(col, 'key') ``` There's two issues here: 1. Until we solve projection pushdown (https://github.com/apache/datafusion/issues/14993) if `key` is not shredded we materialize the entire `col` and then extract `key` in a `ProjectionExec`. 2. Even once that is resolved, or in the case that `key` is not shredded evaluating `variant_get(col, 'key')` itself is expensive we still re-compute `variant_get(col, 'key')` twice: once for the filter and once for the projection. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
