alamb commented on PR #7403: URL: https://github.com/apache/arrow-rs/pull/7403#issuecomment-2797228880
Thank you for this PR @scovich > It would be better to leverage a general variant library for variant bit-wrangling instead of doing it all manually here. - 100% agree. I think we'll do this as part of https://github.com/apache/arrow-rs/issues/6736. I am currently working on breaking up the work into some smaller pieces so we can get the code flowing > TBD Where/how to expose this functionality through a public API In my mind this functionality feels like a "computation kernel" (aka similarly to the functions in https://docs.rs/arrow/latest/arrow/compute/index.html) The signature seems like it would roughly be something like: ```rust /// Covert text stored as JSON in an input `StringArray`, `LargeStringArray` or `StringViewArray` into /// a single "Variant" array (`StructArray` with an extension type) fn json_to_variant(input: &ArrayRef) -> ArrayRef { ... } ``` Since the arrow-json crate is currently for converting `JSON to arrow` it is not 100% clear to me that this functionality belongs in the `arrow-json` crate at all, espcially as variant is not part of the "core" arrow spec it seems. > Still TBD how to assemble a bunch of variant metadata+value pairs inside an arrow array data that can eventually become a usable arrow array I think we will sort this out as part of implementing varint in https://github.com/apache/arrow-rs/issues/6736. TLDR is via a `StructArray` annotated with an extension type I think. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org