srielau opened a new pull request, #59844: URL: https://github.com/apache/spark/pull/59844
### What changes were proposed in this pull request? Add an optional `mode` argument to `parse_sql(sqlStmt[, mode])`: - `PARSE` (default): existing unresolved JSON (source location, parse success, Table 39 identifier/code, table/function refs, select-list names, parameter markers). - `ANALYZE`: parse plus catalog analysis. Successful analysis adds `analyze_success`, SQL types on `select_list`, and fully qualified resolved relation/routine names (`spark_catalog.default.t`, `system.builtin.count`). Temp views and temp functions stay unqualified. Parse failures remain JSON. Analysis failures keep parse metadata and nest a STANDARD error (`analyze_success` false) without aborting the batch. Invalid `mode` raises `INVALID_PARAMETER_VALUE.PARSE_SQL_MODE`. `ANALYZE` without an active session raises `PARSE_SQL_ANALYZE_REQUIRES_SESSION`. The function remains gated by `spark.sql.function.parseSql.enabled` (default false), since 4.4.0. ### Why are the changes needed? Callers that already use `parse_sql` for statement inventory also need typed select-list columns and resolved catalog names. Keeping that behind an explicit `ANALYZE` mode preserves the unresolved, executor-safe `PARSE` contract. JIRA: https://issues.apache.org/jira/browse/SPARK-60593 ### Does this PR introduce _any_ user-facing change? Yes, for the still-experimental `parse_sql` function (flag off by default): - Signature is now `parse_sql(sqlStmt[, mode])`. One-argument calls keep `PARSE` behavior. - `ANALYZE` adds `analyze_success`, typed `select_list`, and qualified names as described above. - One-argument pretty SQL / schema strings include the default `PARSE` child, e.g. `parse_sql(SELECT 1;SELECT 2, PARSE)`. Example: ```sql SELECT parse_sql('SELECT 1 AS a', 'ANALYZE'); -- select_list includes {"name":["a"],"type":"INT"} -- analyze_success is true ``` ### How was this patch tested? - `ParseSqlSuite` and `ParseSqlAnalyzeSuite` - `ExpressionsSchemaSuite` - Regenerated SQLQueryTest goldens for `parse-sql` and new `parse-sql-analyze` - `./dev/lint-scala` ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Cursor Grok 4.6 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
