matanper opened a new pull request, #1293:
URL: https://github.com/apache/arrow-go/pull/1293
### Rationale for this change
Writing shredded VARIANT values to Parquet fails with `lists with non-zero
length null components are not supported` when a nested list has a null,
missing, or scalar parent between populated rows. For example:
```json
{"obj":{"items":[1]}}
{"obj":null}
{"obj":{"items":[2]}}
```
`shreddedObjBuilder.AppendMissing` marks the parent struct null, but
recursively calling `shreddedArrayBuilder.AppendMissing` adds a valid list and
a placeholder child element. The Parquet writer rejects the resulting skipped
child range.
Fixes #1292.
### What changes are included in this PR?
Make `shreddedArrayBuilder.AppendMissing` append a null list without adding
child elements. The production change is two added lines (including a comment)
and four removed lines.
Add a self-contained Parquet write/read regression for null, missing, and
scalar parent values between populated lists. It verifies all decoded JSON
values, row count, and checked-allocator cleanup. No external fixtures are
required.
### Are these changes tested?
All three regression cases reproduce the original write error before the fix
and pass afterward.
- `go test ./parquet/pqarrow -run TestShreddedVariantListsUnderMissingParent
-count=1`
- `go test ./arrow/extensions ./parquet/variant`
### Are there any user-facing changes?
These valid shredded VARIANT values can now be written to Parquet while
preserving null, missing, and scalar parent values. No public API changes.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
