Vladan Vasić created SPARK-61027:
------------------------------------
Summary: Extract shared DDL and DML analysis helpers
Key: SPARK-61027
URL: https://issues.apache.org/jira/browse/SPARK-61027
Project: Spark
Issue Type: Improvement
Components: SQL
Affects Versions: 4.4.0
Reporter: Vladan Vasić
Extract command-local DDL and DML analysis from fixed-point analyzer rules so
Single Pass command
resolvers can reuse the existing operations without running fixed-point rule
traversals.
Scope:
* Field-name and position resolution for ALTER TABLE and CREATE INDEX, ALTER
validation, and
partition specification resolution and validation.
* V1/V2 INSERT preparation, write-output alignment, row-level assignment
conversion, and check
and generated-column invariant construction.
* V1 ADD/ALTER COLUMN, comment changes, CTAS/RTAS/CREATE TABLE LIKE, and
namespace creation and
location changes.
Keep existing rule traversal, pruning, case guards and ordering, fixed-point
stages, validation
timing, exception behavior, tags and metadata unchanged. Generated-expression
analysis remains an
explicit callback with the existing lazy session fallback. Expose a per-column
INSERT operation
with caller-owned alias creation and an independently callable field-path
operation.
This is a refactor only. It does not change SQL semantics, enable new Single
Pass command coverage,
add feature flags, or change golden files. Some whole-command helpers retain
legacy orchestration
and synthesized expressions that a future Single Pass implementation must
resolve inline.
Validate using existing Catalyst write analysis and SQL command suites, plus
focused alias-factory
contract coverage. This continues the helper extractions in Apache Spark PRs
#59646, #59746,
#59861, #59959 and #60099 without overlapping the SHOW helpers in SPARK-61004.
Preserve ResolveInsertionBase's existing inheritance/API and the original
configuration, catalog
and error-formatting receivers, including deferred reads and subclass overrides.
Implementation and review: https://github.com/apache/spark/pull/60171
Validation: 986 local tests, four Scala style checks, two independent
strict-equivalence audits,
and eight compiled base-versus-refactor probes passed for the prepared revision.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
