[
https://issues.apache.org/jira/browse/SPARK-33612?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Okolnychyi updated SPARK-33612:
-------------------------------------
Description: Right now, we have a special place in the optimizer where we
run rules for constructing v2 scans. As time shows, we need more rewrite rules
for v2 tables and not all of them related to reads. One option is to rename the
current batch into something more generic but it would require changing quite
some places. Moreover, the batch contains some non-V2 logic as well. That's why
it seems better to introduce a new batch and use it for all data source v2
rewrites and beyond. (was: Right now, we have a special place in the optimizer
where we run rules for constructing v2 scans. As time shows, we need more
rewrite rules for v2 tables and not all of them related to reads. One option is
to rename the current batch into something more generic but it would require
changing quite some places. Moreover, the batch contains some non-V2 logic as
well. That's why it seems better to introduce a new batch and use it for all
data source v2 rewrites.)
> Add v2SourceRewriteRules batch to Optimizer
> -------------------------------------------
>
> Key: SPARK-33612
> URL: https://issues.apache.org/jira/browse/SPARK-33612
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 3.1.0
> Reporter: Anton Okolnychyi
> Priority: Major
>
> Right now, we have a special place in the optimizer where we run rules for
> constructing v2 scans. As time shows, we need more rewrite rules for v2
> tables and not all of them related to reads. One option is to rename the
> current batch into something more generic but it would require changing quite
> some places. Moreover, the batch contains some non-V2 logic as well. That's
> why it seems better to introduce a new batch and use it for all data source
> v2 rewrites and beyond.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
