debugmiller opened a new pull request, #8998:
URL: https://github.com/apache/arrow-rs/pull/8998
# Which issue does this PR close?
- Closes #8987.
# What changes are included in this PR?
With this change you can provide a Variant annotated Field to the arrow-json
reader and it will deserialize JSON into that column as a Variant.
For example
```rust
let variant_array = VariantArrayBuilder::new(0).build();
let struct_field = Schema::new(vec![
Field::new("id", DataType::Int32, false),
// call VariantArray::field to get the correct Field
variant_array.field("var"),
]);
let builder = ReaderBuilder::new(Arc::new(struct_field.clone()));
let result = builder
.with_struct_mode(StructMode::ObjectOnly)
.build(Cursor::new(b"{\"id\": 1, \"var\": [\"mixed data\", 1]}"));
```
This is my first PR to this project, so I wanted to get this up for
feedback. Some things to point out:
- I added this under an arrow-json feature `variant_experimental` which
matches the feature in other crates. Does this align with expectations for how
this feature would be exposed?
- `arrow-json` must include `parquet-variant-compute` so to avoid a circular
dependency I had to modify `parquet-variant-compute` to include `arrow` sub
crates (e.g. `arrow-schema`) instead of including through `arrow`. That makes
up a decent chunk of the changes in this PR and could be pulled out as a
standalone change but I was not sure if it was intentional that
`parquet-variant-compute` included from `arrow` directly.
- It does not attempt to deserialize to Decimal (which aligns with the
existing code in `parquet-variant-compute`), numbers are deserialized to either
i64 or f64.
# Are these changes tested?
Unit test was added. Open to feedback on whether tests are adequate as-is.
# Are there any user-facing changes?
Yes, users can now provide
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
