alliasgher opened a new pull request, #876:
URL: https://github.com/apache/iceberg-go/pull/876
## Summary
Partition columns of Iceberg `FixedType` were being written into manifest
partition schemas as Avro `bytes` with a stale TODO in `schema_conversions.go`,
because hamba/avro cannot dispatch `[N]byte` arrays inside a union
(hamba/avro#571). That silently produced spec-non-compliant manifests — any
compliant reader expects `fixed[N]` for those columns and sees a type mismatch
on round-trip.
This PR works around the upstream limitation and emits the correct schema:
* `partitionTypeToAvroSchema` now builds a per-length named fixed schema
(`fixed_N`) wrapped in the usual nullable union.
* Track FixedType partition fields in a new `partFieldIDToFixedPartSize` map
threaded from `getFieldIDMap` through both `ManifestWriter` and
`ManifestReader`. The existing decimal-scale tracking (stored in the
misleadingly named `fieldIDToFixedSize`) is unchanged.
* At encode time, `convertFixedValue` pads/truncates the partition `[]byte`
to the declared size and hands hamba/avro a `{"fixed_N": [N]byte}` union
branch, which it does accept.
* At decode time, `unwrapFixedValue` reverses the wrap back into `[]byte` so
downstream Iceberg code keeps its existing slice-based shape.
Fixes #845
## Test plan
- [x] `go test ./...`
- [x] `go vet ./...`
- [x] New regression test `TestFixedPartitionColumnAvroSchema` asserts both
the on-wire schema (fixed branch with the right size) and that a non-nil fixed
partition value survives encode → decode as the original `[]byte`.
Signed-off-by: Ali <[email protected]>
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
