Jeffail opened a new pull request, #936: URL: https://github.com/apache/iceberg-go/pull/936
## Problem `NewUpdateSchema` seeds its fresh-id counter from `txn.meta.CurrentSchema().HighestFieldID()`. The Iceberg spec reserves `last-column-id` on table metadata as the monotonic counter for this purpose; Java's [`SchemaUpdate`](https://github.com/apache/iceberg/blob/main/core/src/main/java/org/apache/iceberg/SchemaUpdate.java) seeds from `metadata.lastColumnId()`. When the current schema's highest id is lower than the lifetime max — after a previous evolution added columns that were later dropped, or after a schema swap — `AddColumn` allocates ids already referenced by fields in historical schemas. Catalogs that index the full schema history (e.g. AWS Glue's `schemasToGlueColumns`) then reject the commit with a duplicate field id error. ## Fix Seed `lastColumnID` from `txn.meta.LastColumnID()` in `NewUpdateSchema`. Adds a public `LastColumnID()` getter on `*MetadataBuilder` — the `Metadata` interface already exposes this value, but `txn.meta` is the builder. `MetadataBuilder.AddSchema` already keeps `last-column-id` monotonic via `max(b.lastColumnId, schema.HighestFieldID())`, so no other allocator sites need changes. ## Testing Adds `TestAddColumnMonotonicFieldIDs`: constructs metadata whose `last-column-id` exceeds the current schema's `HighestFieldID()` (by adding a higher-id schema while leaving the original as current) and asserts that `AddColumn` allocates above `last-column-id`. Confirmed failing on `main` before the fix and passing after. `go test ./...` passes. Fixes #935 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
