[
https://issues.apache.org/jira/browse/SPARK-59163?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-59163:
-----------------------------------
Labels: pull-request-available (was: )
> Reuse projected table state options across table loading and cache lookup
> -------------------------------------------------------------------------
>
> Key: SPARK-59163
> URL: https://issues.apache.org/jira/browse/SPARK-59163
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 4.3.0
> Reporter: Yan Yan
> Priority: Minor
> Labels: pull-request-available
>
> h3. Background
> SPARK-58392 and PR #57585 introduced table-state option projection for
> relation loading. During relation resolution, Spark already materializes the
> projected state-option map for the per-query table cache key.
> h3. Problem
> Some subsequent paths derive the same projection again before catalog loading
> or shared relation cache lookup. This repeats normalization and
> materialization and makes correctness depend on independently computed maps
> remaining equivalent.
> h3. Proposed change
> Reuse the already-projected state-option map for the table cache key, catalog
> load, shared relation cache lookup, and execution refresh. Also apply a minor
> internal refactoring so method names make it explicit when callers pass a
> complete option map versus already-projected state options.
> There is no user-facing behavior change.
> Related issue: SPARK-58392
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
