xiangfu0 opened a new pull request, #20031: URL: https://github.com/apache/pinot/pull/20031
## What Adds query-time Variant functions to the single-stage engine and keeps raw VARIANT values out of every operation that would need equality, hashing, ordering or pattern matching. ## Changes - Transform functions `variantGet`, `tryVariantGet`, `variantExists`, `isVariantNull`, `variantTypeOf`, `variantToJson`, `parseJsonToVariant`/`tryParseJsonToVariant`, registered in `TransformFunctionType` with Calcite type inference. These functions require `enableNullHandling`. - Compile-time folding skips JSON-to-Variant calls, because a folded result would degrade to a plain BYTES literal (`CompileTimeFunctionsInvoker`). - Guards reject raw VARIANT in filter predicates (validated on the server before segment pruning), IN/NOT IN, comparisons, CAST, COALESCE, `ANY_VALUE`, ORDER BY, GROUP BY, DISTINCT and general aggregations. `jsonExtractScalar` doesn't accept VARIANT as a results type. `COUNT(raw_variant)` stays supported. - Selecting a raw VARIANT requires null handling, because the empty-bytes sentinel is how SQL null is told apart from a Variant null. The broker turns a raw VARIANT result without null handling into a query-validation error. ## Depends on 4/8 (stacking order only; the code itself needs 1/8–3/8). ## About this PR This is **part 5 of 8** splitting #19101 (end-to-end Parquet VARIANT support) into layered PRs. #19101 stays open as the umbrella with the full feature description; [`docs/design/VARIANT.md`](https://github.com/apache/pinot/blob/xiangfu0/variant-split/01-spi-foundation/docs/design/VARIANT.md) (added in 1/8) is the design reference. The base branch is the previous PR in the stack, so the diff below is exactly this layer. Each layer compiles and passes its own tests on top of the one below it, and the top of the stack has the same tree as #19101. > VARIANT is not usable end to end until the whole stack has landed. Raw-value opacity guards arrive in 5/8 (single-stage) and 6/8 (multi-stage), so please don't create VARIANT columns on a build that has only part of the stack. The stack should land within one release. ### Full stack (merge bottom → top) _filled in once all PRs are open_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
