cloud-fan opened a new pull request, #58204: URL: https://github.com/apache/spark/pull/58204
### What changes were proposed in this pull request? This PR makes DML target tables regular analyzer children while keeping them out of optimizer traversal after analysis. `InsertIntoStatement` now exposes both its target table and input query as children. `V2WriteCommand` now extends `AnalysisOnlyCommand`: the analyzer sees both the table and query, while the analyzed command keeps only the query in `children` and moves the table to `innerChildren`. This lets normal analyzer rules resolve all expressions in an `IDENTIFIER` clause without adding command-specific cases to each rule. The change also preserves CTE placement on the input query, unwraps analyzer-generated aliases around V2 write targets, updates affected command extractors, and adds coverage for INSERT, INSERT OVERWRITE, REPLACE WHERE, and REPLACE USING. ### Why are the changes needed? https://github.com/apache/spark/pull/55949 moved the unresolved target identifier into `OverwriteByExpression.table`. Since that table parameter was not a child, analyzer rules could not traverse it. As a result, expressions inside the target `IDENTIFIER` clause, including nested functions, could remain unresolved. Handling only `ResolveIdentifierClause` or individual DML commands is insufficient because other rules must also resolve functions, parameters, variables, and future expression types in the target. Making the table an analysis child gives it the standard analyzer lifecycle. The table is removed from `children` after analysis because optimizer rules such as `V2ScanRelationPushDown` must not turn the V2 write target relation into a scan relation. It remains available in `innerChildren` for plan inspection and physical write planning. ### Does this PR introduce _any_ user-facing change? Yes. DML statements can resolve expressions inside a target `IDENTIFIER` clause. For example, the following INSERT REPLACE WHERE statement now resolves and executes correctly: ```sql INSERT INTO IDENTIFIER( lower(regexp_replace(:table_name, 'PLACEHOLDER', 'TBL'))) REPLACE WHERE id = 1 VALUES (1, 'updated') ``` ### How was this patch tested? Added analyzer lifecycle and end-to-end regression coverage, then ran: ``` build/sbt 'catalyst/testOnly org.apache.spark.sql.catalyst.plans.logical.V2CommandTreePatternSuite' build/sbt 'sql/testOnly org.apache.spark.sql.ParametersSuite' build/sbt 'sql/testOnly org.apache.spark.sql.execution.command.PlanResolutionSuite' build/sbt 'sql/testOnly org.apache.spark.sql.connector.DataSourceV2OptionSuite' build/sbt 'sql/testOnly org.apache.spark.sql.connector.DataSourceV2SQLSuiteV1Filter' ``` All 697 active tests passed; one existing test was ignored. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: OpenAI Codex (GPT-5) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
