ianthetechie opened a new pull request, #27259: URL: https://github.com/apache/datafusion/pull/27259
## Which issue does this PR close? - Closes #27253 ## Rationale for this change The parquet pushdown was too eager in removing map columns, causing query issues. This makes the prune more conservative. See the issue for details. ## What changes are included in this PR? This PR reduces the scope of the prune, allowing for map columns to be filtered in the WHERE clause. ## What is the testing strategy for this PR? Additional SQL logic test cases are included with a reproducible test case that fails before the changes and passes after. Existing tests were insufficient, as this regression was introduced in the DataFusion 54 series (https://github.com/apache/datafusion/pull/20854). ## Are there any user-facing changes? No user-facing changes. I'm not sure how bug fix backports work, but it'd be nice to have this for at least DF 55 if that's not too much trouble. Otherwise I can maintain my own fork until contrib packages like geodatafusion and iceberg catch up. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
