alamb commented on issue #22883:
URL: https://github.com/apache/datafusion/issues/22883#issuecomment-5868936738
Another thing I was thinking about was "could this mechanism work for
dynamically choosing parquet pushdown filters".
Today we have to choose between
```text
(filter)
(repartition)
(scan)
```
and full pushdown
```
(scan, filter)
```
However, with optional filters we could potentially model it like two
filters that switched at runtime between optional and required. Something like:
Plan starts like
```
(filter = OPTIONAL)
(repartition)
(scan, filter = REQUIRED)
```
And if we need to stop evaluating during the scan we could switch the
pattern to
```
(filter = REQUIRED)
(repartition)
(scan, filter = OPTIONAL)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
