lasdf1234 commented on code in PR #12050: URL: https://github.com/apache/gravitino/pull/12050#discussion_r3612499147
########## design-docs/iceberg-idempotency-key.md: ########## @@ -0,0 +1,316 @@ +<!-- + Licensed to the Apache Software Foundation (ASF) under one + or more contributor license agreements. See the NOTICE file + distributed with this work for additional information + regarding copyright ownership. The ASF licenses this file + to you under the Apache License, Version 2.0 (the + "License"); you may not use this file except in compliance + with the License. You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, + software distributed under the License is distributed on an + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + KIND, either express or implied. See the License for the + specific language governing permissions and limitations + under the License. +--> + +# Design: Idempotency-Key Support for the Gravitino Iceberg REST Server + +| Field | Value | +| ------- | ------------------------------------------------------- | +| Status | Draft | +| Authors | @laserninja | +| Created | 2026-07-16 | +| Module | `iceberg/iceberg-rest-server`, `iceberg/iceberg-common` | + +--- + +## 1. Background + +The Iceberg REST spec defines an optional `Idempotency-Key` header (UUIDv7) on +all mutation endpoints (create/update/delete for tables, namespaces, and views). +When present, the server must: + +1. Execute the mutation on first receipt and store the response. +2. Return the stored response on subsequent retries with the same key. +3. Advertise the key reuse window via `idempotency-key-lifetime` in + `GET /v1/config`. + +Without this, network failures and client retries can cause duplicate +table/namespace/view creation or deletion. + +### 1.1 Multi-Node Problem + +Gravitino's Iceberg REST server can run as multiple replicas behind a load +balancer. A purely in-memory cache (e.g., Caffeine) is node-local: + +- **Retry hits different node**: The second node has no record of the first + execution and re-runs the mutation (duplicate side-effect). +- **Node crash**: All cached responses are lost; retries execute again. + +A production-grade solution requires shared, durable storage for idempotency +records. + +### 1.2 Prior Art + +Apache Polaris introduced database-backed idempotency support in +[apache/polaris#3205](https://github.com/apache/polaris/pull/3205) with a +Postgres-backed `IdempotencyStore` SPI. Their design includes heartbeat-based +liveness, executor ownership, crash reconciliation, and minimal response +reconstruction. Gravitino can adopt a simplified version of this pattern that +fits its existing relational backend and shorter mutation lifecycle. + +--- + +## 2. Goals + +1. Prevent duplicate side-effects when clients retry mutations with the same key. +2. Work correctly in multi-node deployments (shared storage). +3. Survive node failures (durable state). +4. Minimal latency overhead on the happy path (single extra DB round-trip). +5. Configurable storage backend (pluggable SPI). +6. Spec-compliant: validate UUIDv7 format, advertise lifetime in config. Review Comment: In my opinion, the design document should include an explanation of the RFC 9562 protocol that UUIDv7 adheres to. It is also necessary to explain why your design follows the RFC 9562 protocol. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
