Gengliang Wang created SPARK-53798:
--------------------------------------
Summary: Enable operator pushdown in DataSource V2 streaming
Key: SPARK-53798
URL: https://issues.apache.org/jira/browse/SPARK-53798
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.1.0
Reporter: Gengliang Wang
This umbrella tracks enabling operator pushdowns in DataSource V2 streaming,
bringing it closer to batch parity. Supported pushdowns (filters, projections,
aggregates, etc) will be applied directly in DSv2 readers during stream
analysis to reduce scanned data and compute cost.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
