Alb3e3 commented on PR #51027: URL: https://github.com/apache/arrow/pull/51027#issuecomment-5479989670
Fixed in de746b3cb. I removed `ARROW:integration:allow_unregistered_extension` and registered the existing `dense_union_extension_type()` / `sparse_union_extension_type()` test types in the C++ integration executable. The generated storage schemas and serialized metadata now match those registered types. Removing the bypass also exposed a related integration JSON reader issue: validity parsing was selected from the logical `EXTENSION` type, so it incorrectly required `VALIDITY` for union storage. It now selects validity layout from the physical storage type, matching the IPC writer's behavior. I verified the coverage is non-vacuous: - The C++ JSON reader reports both fields as `extension<sparse-union-extension>` / `extension<dense-union-extension>`. - JSON -> IPC -> validate -> JSON -> validate passes for the zero-length and populated batches. - Temporarily reverting only the writer fix from 3e6f730 makes validation fail on the populated sparse union: buffer #1 is 0 bytes, expected at least 7 bytes. - `arrow-json-integration-test` passes and both C++ files are clang-format 18.1.8 clean. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
