nssalian opened a new pull request, #1206:
URL: https://github.com/apache/arrow-go/pull/1206
### Rationale for this change
Reading one field from a variant column reassembles the whole value per row
(`VariantArray.Value`) then navigates to the field in Go. When the variant is
shredded, that field is often already a typed column, so the full reassembly is
wasted work.
### What changes are included in this PR?
- Adds `VariantGet(input, GetOptions{Path, AsType, Safe, Mem})`: extracts a
path by following the shredded `typed_value` columns as far as possible and
only reassembling the residual `value` per-row for the rest. Mirrors the
arrow-rs
[variant_get](https://github.com/apache/arrow-rs/blob/main/parquet-variant-compute/src/variant_get.rs)
- `AsType == nil` returns a `VariantArray` pointing at the path; set it to
get a typed array. Also extracts the variant-leaf -> typed-builder cast matrix
from `shreddedPrimitiveBuilder.tryTyped` into a shared
`appendVariantToTypedBuilder` so the writer and `VariantGet` share it
(`tryTyped` behavior unchanged).
### Follow-ups (out of scope here):
- Array-index columnar push-down - index steps currently use the per-row
fallback. The columnar `take` kernel lives in `arrow/compute`, which
`arrow/extensions` cannot import (cycle); needs a local gather to push indices
into the shredded columns.
- Lenient cross-type casts - only exact-match + numeric widening are
supported; string->bool, numeric->bool, decimal->float, etc. are not ported.
- Nested `AsType` output - struct/list target types return
`arrow.ErrNotImplemented`; would need a recursive per-field builder.
### Are these changes tested?
- `variant_get_test.go` covers the extraction paths (typed and variant
output, nested path, array index, missing field, fallback, null rows,
dictionary-encoded metadata, nested-type rejection).
- `variant_get_internal_test.go` adds a white-box check that the
perfect-shredding fast path fires and a `CheckedAllocator` leak check.
### Are there any user-facing changes?
Yes - new exported API: `VariantGet`, `GetOptions`, `VariantPath`,
`VariantPathElement`, `VariantPathField`, `VariantPathIndex`. Additive only.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
