alamb commented on issue #8609: URL: https://github.com/apache/datafusion/issues/8609#issuecomment-3775118875
> On the short-term, being able to configure the limit a bit above 20 could solve the majority of our problems - I'm not exactly sure how the magic constant 20 was determined, I take it this is because of tradeoffs with the implementation that translates Inlist into deeply nested binary exprs ? Yes I think so I certainly think a config setting is reasonable > A bit more ambitious/long-term: if I understand correctly the various conversations/codepaths, given that in our case we're interested in page index pruning based on min-max: I think @2010YOUY01 has been working on a more sophisticated pruning predicate system - https://github.com/apache/datafusion/pull/19609 - https://github.com/apache/datafusion/issues/19487 > I'm seeing that some previous attempts (https://github.com/apache/datafusion/pull/8669) didn't make it to master, maybe with simplified requirements we could iterate more cheaply I can't remember all the details, but this seems like a reasonable approach to me to pursue -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
