roryqi commented on PR #12257: URL: https://github.com/apache/gravitino/pull/12257#issuecomment-5178971104
Since this feature is scoped to Iceberg REST Catalog, I wonder whether we need a generic `entity_deletion` model and a new `table_meta.deletion_id` column. I would prefer keeping the Entity Store metadata and its relations unchanged during the recovery window, and extending the existing `iceberg_cleanup_job` instead: - add `entity_id`, `retention_expires_at`, and a `RETAINED` state; - use the cleanup-job ID as the immutable `deletionId`; - while `RETAINED`, use the row as the durable tombstone to hide the table and reserve its identifier, as the current cleanup job already does for name reuse; - on undrop, consume the retained job and expose the unchanged Entity Store metadata again; - on expiry or explicit purge, transition `RETAINED -> PENDING`, then reuse the existing worker, heartbeat, CAS, and retry flow. This avoids reversing the current Entity Store `deleted_at` cascade for columns, owners, tags, policies, and other relations. It also avoids introducing a generic deletion abstraction and changing `table_meta` for an IRC-only feature. Could the design explain which concrete requirement cannot be satisfied by this approach? In particular, if the native Gravitino table API and non-Iceberg entities are outside the product scope, it is not clear what requires the separate `entity_deletion` record plus the relationship pointer in `table_meta`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
