jerryshao opened a new issue, #13304:
URL: https://github.com/apache/gravitino/issues/13304
### Describe the subtask
Dropping a tagged column leaves the column in the tag's object list.
Steps (lakehouse-iceberg catalog):
1. Create table `t` with columns `c1` and `c2`, and tag `c1`.
2. Drop `c1` via `alterTable` (`deleteColumn`).
3. `GET /api/metalakes/{m}/objects/column/{catalog}.{schema}.t.c1/tags`
correctly returns `NoSuchMetadataObjectException`.
4. `GET /api/metalakes/{m}/tags/{tag}/objects` still lists
`{catalog}.{schema}.t.c1`.
Cause:
- Dropping a column inserts a version row with `column_op_type = DELETE` and
`deleted_at = 0` (`TableColumnMetaService.updateColumnPOsFromTableDiff`). The
column's tag, policy and owner relations are not touched.
- `MetadataObjectService.getColumnObjectsFullName` (via
`listColumnPOsByColumnIds`) returns a name for that `DELETE` row.
`TagMetaService.listAssociatedMetadataObjectsForTag` therefore keeps listing
the column. Policy object lists and role securable objects go through the same
function.
- `OrphanedMetadataObjectRelationService` treats a column as live whenever
any of its rows has `deleted_at = 0`, so these relations are never collected.
`branch-1.3` has no orphan GC at all.
Proposed fix:
- In `getColumnObjectsFullName`, return `null` for a column whose latest row
is `DELETE`.
- When `updateColumnPOsFromTableDiff` writes `DELETE` rows, soft-delete the
tag, policy and owner relations of those column ids in the same transaction.
- On `main`, update the orphan GC liveness check for columns.
- Tests:
- Drop a tagged column, then check that the tag's object list no longer
contains it. Also check the relation rows directly.
- Re-add a column with the same name, then check that it has no tags and
the tag doesn't list it.
- On `branch-1.3`, check that the cached tag→objects list is refreshed.
Target: `main`, backport to `branch-1.3`.
### Parent issue
#13303
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
