Haricharanpanjwani opened a new pull request, #14346: URL: https://github.com/apache/gravitino/pull/14346
### What changes were proposed in this pull request? - Add optional character set and collation metadata to `StringType`, `VarCharType`, and `FixedCharType` in the Java and Python type systems. - Preserve attributed character types through JSON serialization, REST DTOs, and Java/Python clients while retaining the existing string representation when metadata is absent. - Read effective character set and collation metadata from MySQL and include it in generated CREATE and ALTER DDL. - Explicitly handle attributed character types in Flink, Spark, and Trino conversions. - Document the new API and MySQL behavior. ### Why are the changes needed? Character columns with different character sets or collations currently map to identical Gravitino types. Recreating a table from this metadata can therefore change character representation, comparison, and sorting behavior. Structured metadata is needed instead of relying on `ExternalType`, while preserving compatibility with existing character type factories and JSON representations. Fixes #13585 ### Does this PR introduce _any_ user-facing change? Yes. Character types can now expose optional `charset` and `collation` attributes. Attributed character types use an object JSON representation containing these fields. Existing factories and string JSON representations remain unchanged when both attributes are absent. MySQL tables loaded through Gravitino now carry their effective character set and collation, and table creation or alteration emits the corresponding `CHARACTER SET` and `COLLATE` clauses. ### How was this patch tested? - Added unit tests for Java and Python character type metadata and serialization. - Added compatibility tests for Flink, Spark 3.5, Spark 4.0, and Trino conversions. - Added Docker-backed MySQL round-trip coverage that loads attributed columns, recreates the table through Gravitino, and verifies type equality. - Passed focused Java module tests, 68 focused Python tests, Python lint/format checks, and the 22-test `TestMysqlTableOperations` Docker suite. - Passed the complete 47-job Fork CI workflow: https://github.com/Haricharanpanjwani/gravitino/actions/runs/40780026594 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
