Thanks, that makes sense. I see I was thinking about this in reverse, the idea is to evaluate each expr and accept it fully or partially or not at all.
Once I have this working I will be happy to help add to the guide for issue 7676 if that is useful. Cory -- Cory Isaacson http://www.coryisaacson.com On Sep 27, 2023 at 10:51 AM -0600, Andy Grove <andygrov...@gmail.com>, wrote: > Hi Cory, > > You need to override supports_filters_pushdown in your TableProvider > implementation. DataFusion will call this for each filter expression to see > which ones can be pushed down. > > We should add this information to the documentation at > https://arrow.apache.org/datafusion/library-user-guide/custom-table-providers.html > .. I filed an issue for this: > > https://github.com/apache/arrow-datafusion/issues/7676 > > Hope that helps. > > Andy. > > > On Wed, Sep 27, 2023 at 8:50 AM Cory Isaacson <cory.m.isaac...@gmail.com> > > wrote: > > > I am building a custom data source and need to know how to enable push > > > down filters. Here is an example query: > > > > > > SELECT a > > > FROM t1 > > > WHERE b = 123 > > > > > > DataFusion filters the data itself, but I want it to push down the filter > > > (it does push down projections correctly). > > > > > > I added the filters arg to `create_execution_plan` and save the filters > > > in the enclosing data source struct. I checked and the filters are always > > > empty. > > > > > > However, I do see the filter in the LogicalPlan. > > > > > > Please let me know how to do this, or point me to an example. > > > > > > Thanks in advance for any assistance. > > > > > > Cory > > > > > > > > > > > > -- > > > Cory Isaacson > > > http://www.coryisaacson.com
