GitHub user backkem added a comment to the discussion: Search Pushdown (e.g. Vector Search) Into Table Providers
FWIW there is an existing implementation of the "full logical plan pushdown" in [datafusion-contrib/datafusion-federation](https://github.com/datafusion-contrib/datafusion-federation). Approach: we extend the TableProvider to keep track of which (remote) engine represents it. A federation optimiser finds the largest sub-plans provided by one engine. This sub-plan is passed to an engine-specific optimiser that can self-determine how much of this sub-plan it can federate. GitHub link: https://github.com/apache/datafusion/discussions/16358#discussioncomment-13425770 ---- This is an automatically sent email for github@datafusion.apache.org. To unsubscribe, please send an email to: github-unsubscr...@datafusion.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
