yuqi1129 opened a new pull request, #12454: URL: https://github.com/apache/gravitino/pull/12454
### What changes were proposed in this pull request? Add database-backed optimistic concurrency control and transaction boundaries for metalake writes. - Advance the metalake OCC version on every alter, and guard alter and delete with a compare-and-set on the observed version. - Classify a failed CAS as either a stale-version conflict or an already-missing entity. - Keep the root CAS, the non-empty check or the cascade cleanup, and the change-log writes in one database transaction. - Lock the catalog rows before a metalake cascade snapshot, then CAS-delete descendant catalogs and schemas with their observed identifier-and-version pairs, so a concurrent child write is reported instead of silently dropped. - Add the shared `OptimisticLockException` factories used by the follow-up catalog and schema PRs. This is the first of three PRs that replace #12350, which reviewers found too large. The stack is metalake -> catalog -> schema; each PR is independently green. Two cross-entity tests in `TestMetalakeMetaService` (concurrent schema alter during a metalake cascade, and metalake cascade racing a schema create) land with the schema PR, because the behaviour they assert only exists once schema writes take the catalog row lock and bump the schema version. ### Why are the changes needed? Managed metalake operations previously consisted of multiple independent reads and writes. Concurrent alter and delete requests could overwrite newer metadata, and a cascade delete could run partial cleanup while another writer was still modifying descendants. Fix: #12451 ### Does this PR introduce _any_ user-facing change? Concurrent metalake version conflicts are reported as HTTP 409. If the observed entity was deleted or renamed away, alter reports not found and drop preserves its idempotent false result. ### How was this patch tested? - `./gradlew :core:test :core:javadoc :catalogs:catalog-fileset:test :catalogs:catalog-kafka:test -PskipITs` (H2) - New tests in `TestMetalakeMetaService`, `TestMetalakeManager`, `TestExceptionUtils`, `TestPOConverters`. - MySQL and PostgreSQL coverage for the new `<foreach>` CAS delete and the PostgreSQL lock syntax is left to CI (`-PskipDockerTests=false`). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
