Hi everyone, I'd like to start a discussion on FLIP-604: Complete VARIANT Primitive Coverage with UUID and Timestamps. [1]
FLIP-521 [2] shipped VARIANT in Flink 2.1, but it only implements Parquet Variant primitive codes 0 through 16. The open Variant spec goes through 20: TIME, nanosecond timestamps, and UUID. So a VARIANT written by Spark, Iceberg, or any Parquet writer that holds one of these values cannot be read by Flink today, which breaks the lakehouse round-trip VARIANT was meant to support. This FLIP completes the primitive set. TIME and the nanosecond timestamps reuse types Flink already has. UUID needs a type to land on, so the FLIP also adds a native 16-byte UUID type as a first-class SQL type, with casts, ordering, a UUID literal, and UUID_V4 and UUID_V7 generator functions. One point worth noting: no Calcite upgrade is required. Calcite has had a UUID type since 1.39, the same release FLIP-521 adopted for VARIANT, and Flink already runs Calcite 1.41. The 16-byte layout also matches every engine with a native UUID type, including PostgreSQL, Trino, DuckDB, ClickHouse, Snowflake, and Iceberg. Looking forward to your feedback. Best, Ramin Gharib [1] https://cwiki.apache.org/confluence/spaces/FLINK/pages/446070993/FLIP-604+Complete+VARIANT+Primitive+Coverage+with+UUID+and+Timestamps [2] https://cwiki.apache.org/confluence/spaces/FLINK/pages/349637099/FLIP-521+Integrating+Variant+Type+into+Flink+Enabling+Efficient+Semi-Structured+Data+Processing
