adriangb commented on PR #18192: URL: https://github.com/apache/datafusion/pull/18192#issuecomment-3447787020
It depends on how you do the math (I did `(5.7-4)/4`) but that's a detail. IMO "control plane" vs "data plane" can get a bit blurry, eg in the case of `InList` it's possible (and I think common) to have relatively large amounts of data in the "control plane". Besides: all of this is to further enable an optimization (dynamic filters) that can [make queries 25x faster](https://datafusion.apache.org/blog/2025/09/10/dynamic-filters/). We are in fact discussing using `InList` to push down join hash tables so large `InList` expressions are an excellent example. So what I'm saying is "let's make the data plane 25x faster by at the same time making the control plane use 30% less memory, that requires some breaking API changes, we should figure out what those are". DataFusion makes plenty of breaking API changes, I don't even think this is that egregious of one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
