[
https://issues.apache.org/jira/browse/SPARK-58392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-58392:
-----------------------------------
Labels: pull-request-available (was: )
> Pass all options while loading relations
> ----------------------------------------
>
> Key: SPARK-58392
> URL: https://issues.apache.org/jira/browse/SPARK-58392
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.2.0
> Reporter: Yan Yan
> Priority: Minor
> Labels: pull-request-available
>
> Follow-up to SPARK-58389 ("Pass all options while loading tables"), which
> forwarded
> all user-specified options to TableCatalog.loadTable(Identifier, TableContext,
> CaseInsensitiveStringMap) on the table read path.
> That change does not cover the RelationCatalog single-RPC read path. A
> RelationCatalog
> (a connector that exposes both tables and views in one identifier namespace)
> is resolved
> through RelationCatalog.loadRelation(Identifier) -- a single call that
> answers "table or
> view?" in one round trip. For such a catalog this is the primary plain-read
> path, and it
> currently does not receive the user options, so a connector cannot see them
> when
> producing the Table/View metadata for a plain read.
> This is the direct analogue of SPARK-58389 for the single-RPC path. Add
> RelationCatalog.loadRelation(Identifier, CaseInsensitiveStringMap) with a
> default
> implementation that ignores the options and delegates to
> loadRelation(Identifier) --
> so existing connectors are unaffected -- and forward the user options to it
> from the
> resolver's plain-read branch (which fires only when there is no time travel
> and no write
> privileges; both apply to tables only and route through the loadTable path
> instead).
> This is API completeness for the combined table+view catalog: it lets such a
> catalog
> customize the returned relation from the read options at metadata-load time,
> symmetric
> to what SPARK-58389 enables for table-only catalogs. The feature has not been
> released.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
