xiangfu0 opened a new pull request, #20028: URL: https://github.com/apache/pinot/pull/20028
## What Adds the Variant value codec and the `VARIANT` result column type to `pinot-common`. ## Changes - `VariantUtils`: path compilation and evaluation, typed extraction (`ResultType`), JSON rendering and parsing, type inspection, and the reusable cursor used on hot paths. It uses `parquet-variant` (new managed dependency; `parquet-jackson` is excluded). - Scalar functions in `VariantFunctions`: `variantGet`, `tryVariantGet`, `variantExists`, `isVariantNull`, `variantTypeOf`, `variantToJson`, `parseJsonToVariant`/`parseJson`, `tryParseJsonToVariant`/`tryParseJson`. They take and return `byte[]`, so ingestion transforms can use them. - `InbuiltFunctionEvaluator` plans variant calls with literal path and type operands once per transformer, instead of re-parsing the path on every row. - `DataSchema.ColumnDataType.VARIANT` (backed by `BYTES`, rendered as JSON), plus `DataBlockEquals` and `LiteralContext` handling. `FunctionRegistry.canonicalize` now uses `Locale.ROOT`. - The data block and data table test helpers learn VARIANT, because they enumerate every `ColumnDataType`. ## Compatibility `ColumnDataType` goes on the wire by name. An older peer fails only when a VARIANT-typed result is actually in flight, which needs a VARIANT column. No existing column type changes. ## Depends on 1/8. ## About this PR This is **part 2 of 8** splitting #19101 (end-to-end Parquet VARIANT support) into layered PRs. #19101 stays open as the umbrella with the full feature description; [`docs/design/VARIANT.md`](https://github.com/apache/pinot/blob/xiangfu0/variant-split/01-spi-foundation/docs/design/VARIANT.md) (added in 1/8) is the design reference. The base branch is the previous PR in the stack, so the diff below is exactly this layer. Each layer compiles and passes its own tests on top of the one below it, and the top of the stack has the same tree as #19101. > VARIANT is not usable end to end until the whole stack has landed. Raw-value opacity guards arrive in 5/8 (single-stage) and 6/8 (multi-stage), so please don't create VARIANT columns on a build that has only part of the stack. The stack should land within one release. ### Full stack (merge bottom → top) _filled in once all PRs are open_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
