Hi all, Kicking off this thread to discuss the initial REST spec changes needed for Iceberg V4: 1. root-manifest in Snapshot: We need the returned Snapshot to have a root-manifest . (oneOf of root-manifest or manifest-list). For V4 tables, manifest-list will not be set. 2. Add table-location: LoadTableResult, CommitTableResponse, and UnregisterTableResult all need to return table-location. When defined, this will be the absolute path of the table root. (Allows the paths inside the metadata to be relative making table migration easier). 3. Make metadata-location in CommitTableResponse optional: This is for catalog managed tables.
Some of these changes are not backwards compatible: `manifest-list` in Snapshot is a required field right now. Some clients might rely on this field being set in the response. If we allow this to be empty when `root-manifest` is set, these clients could break. The same problem applies to `metadata-location`. Furthermore, many endpoints (unregister, table, etc) transitively depend on Snapshot. If we create a new SnapshotV2, many endpoints will need a version bump to V2. I have a PR that shows what this would roughly look like: (approach 1) https://github.com/apache/iceberg/pull/17543 . The diff is pretty big and we end up duplicating many classes and endpoints. Another way of solving this would be to make a required response field optional (manifest-list and metadata-location) without bumping the model/api versions. This might be a more practical approach assuming that most clients don't break this way. This PR shows how minimal these changes are: (approach 2) https://github.com/apache/iceberg/pull/17642 I lean toward the in-place approach, assuming few clients actually require these fields to be present. Does that seem reasonable, or are there compatibility concerns I'm missing? Regards, Dhruv Arya
