wayne-x-wang opened a new pull request, #60116: URL: https://github.com/apache/spark/pull/60116
### What changes were proposed in this pull request? Preserve the source table's default collation in `CREATE TABLE LIKE` by copying `sourceTableDesc.collation` into the new `CatalogTable` in `CreateTableLikeCommand`. Add regression coverage in `InMemoryCatalogedDDLSuite` for `UTF8_LCASE`, `UTF8_BINARY`, `UNICODE_CI`, and an unspecified table default collation. ### Why are the changes needed? [SPARK-60130](https://issues.apache.org/jira/browse/SPARK-60130) `CREATE TABLE LIKE` copies the source schema but drops the table-level default collation. Existing string columns retain their collations, while a string column subsequently added to the cloned table falls back to `UTF8_BINARY` instead of inheriting the source table's default. This can change query results. ### Does this PR introduce _any_ user-facing change? Yes. Newly cloned tables preserve the source table's default collation, so string columns added without an explicit collation inherit that default. For example, with the V1 Parquet data source: ```sql CREATE TABLE src (s STRING) USING PARQUET DEFAULT COLLATION UTF8_LCASE; CREATE TABLE dst LIKE src; ALTER TABLE dst ADD COLUMNS (t STRING); INSERT INTO dst VALUES ('a', 'a'), ('A', 'A'); SELECT COUNT(*) FROM (SELECT t FROM dst GROUP BY t) groups; ``` The result changes from `2` to `1`, consistent with the source table's `UTF8_LCASE` default. Explicit column collations remain unchanged. Tables without an explicit default continue to use `UTF8_BINARY`. ### How was this patch tested? The four new regression cases check copied column types, preservation of an explicit `UTF8_BINARY` column collation, collation inheritance after `ADD COLUMNS`, grouping of `'a'` and `'A'`, and equality of the source and target table collation metadata. Before the fix, three cases failed; after the fix, all four passed. Local validation with Java 21 passed: ```bash ./build/sbt 'sql/testOnly *InMemoryCatalogedDDLSuite -- -z "SPARK-60130"' ./build/sbt 'sql/testOnly *InMemoryCatalogedDDLSuite' ./build/sbt 'sql/testOnly *DefaultCollationStringTestSuiteV1' ./build/sbt -Phive 'hive/testOnly *HiveDDLSuite -- -z "CREATE TABLE LIKE"' ./build/sbt 'sql/scalastyle' 'sql/Test/scalastyle' ``` - New regression cases: 4 passed. - `InMemoryCatalogedDDLSuite`: 125 passed. - `DefaultCollationStringTestSuiteV1`: 191 passed. - Existing Hive `CREATE TABLE LIKE` cases: 7 passed. - SQL compile and test Scala style checks: no errors or warnings. The fork's [Build workflow, attempt 2](https://github.com/wayne-x-wang/spark/actions/runs/41813256558/attempts/2) also passed for commit `94efa8052eb3fae5854cbbac5488eb8e91aee228`. ### Was this patch authored or co-authored using generative AI tooling? OpenAI Codex, GPT-6 Astra -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
