Subodh-17 opened a new pull request, #12135:
URL: https://github.com/apache/arrow-rs/pull/12135
### Which issue does this PR close?
Closes #11948.
### Rationale for this change
In `cast_to_variant`, converting Arrow `StructArray` or `MapArray` to
Parquet Variant in strict mode (`options.safe == false`) silently overwrote
duplicate object keys instead of returning an error. Duplicate keys with null
values could also bypass underlying builder checks because null fields are
skipped during builder insertion.
In strict mode, duplicate keys within the same object should be rejected
with an `ArrowError::InvalidArgumentError`. Safe mode retains its existing
permissive overwrite behavior.
### What changes are included in this PR?
1. **Struct conversion**
- Checks `struct_array.column_names()` for duplicate names in strict mode.
- Returns an error when a non-null Struct row contains duplicate field
names.
- Null Struct rows continue to produce null variants.
2. **Map conversion**
- Uses a per-row `HashSet` to detect duplicate keys in strict mode.
- Detects duplicates even when the later value is null.
- Keeps keys in separate rows and nested sibling objects independent.
3. **Regression tests**
- Adds 10 targeted tests covering Struct and Map duplicates, null-valued
duplicates, differing value types, nested objects, row independence, and
safe-mode behavior.
### Are these changes tested?
Previously reported verification:
- `cargo test -p parquet-variant-compute`: 381 unit tests and 15 doctests
passed.
- `cargo test -p parquet-variant --lib`: 178 unit tests passed.
- Strict targeted tests: 9 passed.
- Safe-mode targeted test: 1 passed.
- Clippy with `-D warnings` passed.
- `cargo fmt --check` passed.
- `git diff --check` passed.
These are results from the implementation and audit stages, not newly
executed tests in this PR-creation stage.
### Are there any user-facing changes?
Strict `cast_to_variant` conversions (`CastOptions { safe: false, .. }`)
containing duplicate Struct or Map keys now return
`ArrowError::InvalidArgumentError` instead of silently overwriting earlier
values.
Safe conversions (`safe: true`) retain the existing permissive behavior.
---
*AI Tooling Disclosure: AI tools were used during initial problem
reproduction, test scaffolding, and implementation drafting. The contributor
reviewed and refined the implementation, error-handling paths, and test
assertions, and is responsible for understanding, debugging, and maintaining
the submitted changes.*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
