manner opened a new pull request, #29050: URL: https://github.com/apache/flink/pull/29050
## What is the purpose of the change The variant binary encoding spec (see [Variant Encoding](https://parquet.apache.org/docs/file-format/types/variantencoding/)) defines primitive type codes 17-20 for `TIME`, `TIMESTAMP_LTZ_NS`, `TIMESTAMP_NS`, and `UUID` that are currently missing in Flink. This PR adds support for codes 17-19 (`TIME`, `TIMESTAMP_LTZ_NS`, `TIMESTAMP_NS`) to `BinaryVariant`, so that a `LocalTime` and a nanosecond-precision `Instant`/`LocalDateTime` can be represented in a `Variant` without lossy truncation to microseconds. `UUID` (code 20) is intentionally out of scope here and will be added in a separate ticket. ## Brief change log - Added `BinaryVariantUtil` constants for primitive codes 17-19, extended `getType()`/`valueSize()`/`getLong()` to handle them, and added a `TIME_FORMATTER` for JSON rendering - Added `Variant.Type.TIME`/`TIMESTAMP_NS`/`TIMESTAMP_LTZ_NS` and the corresponding `getTime()`/`getDateTimeNanos()`/`getInstantNanos()` accessors to the `Variant` interface - Added `VariantBuilder.of(LocalTime)`; made `of(Instant)`/`of(LocalDateTime)` precision-aware so a value with no sub-microsecond component keeps using the existing compact micros encoding, and only switches to the new nanosecond encoding when the value actually needs it - Implemented `appendTime`/`appendTimestampNanos`/`appendTimestampLtzNanos` in `BinaryVariantInternalBuilder`, and the matching read/`get()`/`toJson()` support in `BinaryVariant` ## Verifying this change This change added tests and can be verified as follows: - Round-trip and generic `get()` dispatch tests for the new types (`BinaryVariantTest#testScalarVariant`) - Dedicated precision-dispatch tests verifying `Instant`/`LocalDateTime` pick the existing `TIMESTAMP_LTZ`/`TIMESTAMP` encoding for microsecond-aligned values and the new `TIMESTAMP_LTZ_NS`/`TIMESTAMP_NS` encoding otherwise, including that the mismatched accessor throws `VariantTypeException` (`BinaryVariantTest#testNanosecondPrecisionVariant`) - A test verifying that `LocalTime` silently truncates below microsecond precision, since `TIME` has no nanosecond-precision counterpart in the variant spec (`BinaryVariantTest#testTimeSubMicrosecondTruncation`) - JSON rendering assertions for `TIME`/`TIMESTAMP_NS`/`TIMESTAMP_LTZ_NS` (`BinaryVariantTest#testToJsonScalar`) ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): **no** - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: **yes** (`Variant` and `VariantBuilder` are `@PublicEvolving`; this adds new enum constants and new interface methods) - The serializers: **no** - The runtime per-record code paths (performance sensitive): **no** - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: **no** - The S3 file system connector: **no** ## Documentation - Does this pull request introduce a new feature? **yes** - If yes, how is the feature documented? **not documented** (documentation is tracked separately in [FLINK-40494](https://issues.apache.org/jira/browse/FLINK-40494)) --- ##### Was generative AI tooling used to co-author this PR? - [X] Yes (please specify the tool below) Generated-by: Claude Sonnet 5 (Claude Code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
