Hi Dmitri, JB, Yufei,

Thanks, everyone.

Dmitri, agreed. For Phase 1, we'll make sure the data is
fully cleaned up before a soft-deleted identifier can be
reused.

In PR #5544 <https://github.com/apache/polaris/pull/5544>, When a
soft-deleted table is permanently deleted, its
identifier will remain reserved until the entity cleanup
and all child tasks it spawns have completed. Only then
will the name be released, so another entity can't reuse
it while the old data files are still there.

One thing from the PR review is worth calling out:
currently, the identifier is released before the async
cleanup finishes. Just scheduling cleanup isn't enough.
I'll make the release wait for cleanup to finish and add
a test that pauses cleanup, verifies recreation is
blocked while cleanup is in progress, and confirms it
succeeds only after cleanup completes.

This also means cleanup can't be skipped when releasing
a name for reuse. Legal-hold catalogs won't be affected,
since they never release the identifier.

Yufei, for the compliance/legal-hold use case, the main
requirement is a recoverable DROP with a retention
period. The table's catalog state and metadata are
retained during that period, the table is hidden from the
Iceberg REST API, and its identifier remains reserved.

This supports legal hold, where the table and its data
must remain preserved during the retention window, as
well as Nessie-to-Polaris migration, where a dropped but
retained table may need to be recovered.

The key point is that another table can't reuse the name
until the retained data has actually been purged. The
cleanup gating above ensures that.

Let me know if this matches what you had in mind, and
I'll update the PR accordingly.

Thanks,
Prithvi

Reply via email to