theirix commented on PR #16325: URL: https://github.com/apache/datafusion/pull/16325#issuecomment-3185015782
Thank you so much, @aditanase! Your `LIMIT` idea looks useful and should bring performance improvements. There are numerous advanced use cases and possible data source-level optimisations for table sampling. To fully support them, the executor should be provided with all the information about sampling. This PR focuses on a simple filter propagation. As @milenkovicm suggested, introducing a special operator is more flexible and preserves sampling semantics, which could help implement such features. We can consider implementing a `LIMIT` either for the approach in this PR, or for a possible physical plan operator. I could work on an operator implementation if this approach doesn't look flexible enough for maintainers. @2010YOUY01, @alamb , what do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
