jerryshao opened a new issue, #13305:
URL: https://github.com/apache/gravitino/issues/13305
### Describe the subtask
Renaming a tagged column drops its tags, and the tag still lists the old
column name.
Steps (lakehouse-iceberg catalog):
1. Create a table with columns `c1` and `c2`, and tag `c1`.
2. Rename `c1` to `c1_new` via `alterTable` (`renameColumn`).
3. `GET .../objects/column/...c1_new/tags` returns an empty list.
4. `GET .../tags/{tag}/objects` still lists `...c1`.
Table renames work correctly; only column renames are affected.
Cause:
- For external catalogs, `TableOperationDispatcher.updateColumnsIfNecessary`
matches catalog columns to stored `ColumnEntity`s by name only. `alterTable`
never passes it the `TableChange.RenameColumn` changes. A rename is therefore
stored as a delete of `c1` plus a new column `c1_new` with a fresh id
(`idGenerator.nextId()`), and every relation keyed by the old id is lost.
- Related issue, which must be fixed together:
`selectColumnIdByTableIdAndName` filters by `column_name` before picking the
latest version. Once a rename keeps the column id, the old name still resolves
to the renamed column, even after it is dropped. A tag on `t.c1` would then
land on `c1_new`. Managed/Lance tables, which already keep the id on rename,
are affected today.
Proposed fix:
- In `alterTable`, build an old→new name map from top-level `RenameColumn`
changes (`fieldName.length == 1`) and use it in `updateColumnsIfNecessary` to
keep the existing column id.
- Chained renames (`a→b`, `b→c`) should resolve to `a→c`.
- A rename plus `addColumn` of the old name in the same request must not
give the new column the old id.
- Change `selectColumnIdByTableIdAndName` to resolve against each column's
latest row. A name matches only if it is the current name of a live column.
- Tests:
- Tags and policies follow a renamed column.
- The old name no longer resolves.
- Chained renames work.
- A rename plus re-adding the old name gives the new column no tags.
- An Iceberg integration test.
Out of scope: columns renamed outside Gravitino, which are still matched by
name on load.
Target: `main`, backport to `branch-1.3`.
### Parent issue
#13303
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
