Abhisheklearn12 opened a new pull request, #11351:
URL: https://github.com/apache/arrow-rs/pull/11351
# Which issue does this PR close?
- Closes #10359.
# Rationale for this change
When an object repeats a field name, `shred_variant` appends both values to
that field's builder. That field's column ends up one row longer than the
others, and arrow-array panics. If the duplicate field is not shredded, there
is no panic: only the last value is kept and the earlier ones are silently
dropped.
`Variant::try_new` rejects such objects, but `shred_variant` reads rows with
shallow validation only, so they reach the shredder.
# What changes are included in this PR?
A repeated field name now returns `InvalidArgumentError` ("Duplicate field
name: ..."), as the issue requests:
- Shredded fields: checked with the existing `seen` scratch buffer. The
issue's proposed `HashSet` fix predates #10882, which replaced the per-row set
with this buffer.
- Unshredded fields: the object builder now validates unique fields, and
`insert_bytes` becomes `try_insert_bytes`, so the error is returned instead of
panicking.
Valid input is unaffected: both checks fire only when a field name repeats
within one object. Unshredded repeats are detected by field id, so an object
whose dictionary is (invalidly) flagged sorted yet repeats a name still passes
through unchanged, as before. Catching it would need a name-based check per
row, like the per-row HashSet that #10882 removed for performance.
# Are these changes tested?
Yes. `test_object_with_duplicate_field_names_errors` covers a duplicate
referenced by the same field id and by two ids naming the same string, each
with the field shredded and unshredded. The test fails if either half of the
fix is removed.
# Are there any significant user-facing changes?
No API changes. An object with a repeated field name is invalid input; it
now returns an error instead of panicking or returning `Ok` with only the last
value kept.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
