funcpp opened a new pull request, #2170: URL: https://github.com/apache/datafusion-sqlparser-rs/pull/2170
## Summary This PR adds several Databricks Delta Lake SQL syntax features: ### 1. OPTIMIZE statement support Adds support for the Databricks `OPTIMIZE` statement syntax: ```sql OPTIMIZE table_name [WHERE predicate] [ZORDER BY (col1, col2, ...)] ``` Reference: https://docs.databricks.com/en/sql/language-manual/delta-optimize.html Key difference from ClickHouse: Databricks omits the `TABLE` keyword after `OPTIMIZE`. ### 2. PARTITIONED BY with optional column types Databricks allows partition columns to reference existing table columns without specifying types: ```sql CREATE TABLE t (col1 STRING, col2 INT) PARTITIONED BY (col1) CREATE TABLE t (name STRING) PARTITIONED BY (year INT, month INT) ``` Reference: https://docs.databricks.com/en/sql/language-manual/sql-ref-partition.html ### 3. STRUCT type with colon syntax Databricks uses Hive-style colon separator for struct field definitions: ```sql STRUCT<field_name: field_type, ...> ARRAY<STRUCT<finish_flag: STRING, survive_flag: STRING, score: INT>> ``` Reference: https://docs.databricks.com/en/sql/language-manual/data-types/struct-type.html The colon is optional per the spec, so both `field: type` and `field type` syntaxes are now accepted. ## Changes - Extended `OptimizeTable` AST to support Databricks-specific fields (`predicate`, `zorder`, `has_table_keyword`) - Added `parse_column_def_for_partition()` to handle optional column types in PARTITIONED BY - Added `DatabricksDialect` to STRUCT type parsing - Modified `parse_struct_field_def()` to accept optional colon separator ## Test plan - [x] Added tests for `OPTIMIZE` statement variations - [x] Added tests for `PARTITIONED BY` with/without column types - [x] Added tests for `STRUCT` type with colon syntax - [x] Verified existing ClickHouse and BigQuery tests still pass - [x] All tests pass (`cargo test`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
