Hi Dmitri, Thank you for reviewing. As you said, we previously explored and dropped TransactionContext APIs with opaque IDs because it created a very chatty protocol and also led to complex transaction state management on Server side, link to old thread below.
Would you add a section to the existing document on the approach you are thinking - Opaque IDs without the chatty protocol and complex transaction state management on Catalog Server? Then we can compare all of them and discuss the best path forward. Thank you! Older thread - https://lists.apache.org/thread/q7vgnfwdxng5q6mq45m0psghzy7553r7 -Jagdeep On Thu, Jun 19, 2025 at 10:42 AM Dmitri Bourlatchkov <di...@apache.org> wrote: > Thanks for driving this proposal, Maninder! > > From my POV the need for Catalogs to provide a monotonic sequence number > has deep implications on the catalog implementations. I added a related > comment to the doc as well. > > The document does a good job at discussing the client operation. I'd > appreciate it if the server-side impact were considered in more depth too, > since the proposal implies changes on both sides. > > I know that an opaque "commit ID" was considered before, however, if I'm > not mistaken previous discussions revolved around the idea of > a TransactionContext as an entity exposed via new APIs for sharing state > between clients/engines and the catalog. I'd like to revisit the idea of > opaque transaction IDs (managed by the catalog) but without the use > of TransactionContext. I made a brief comment about that in the doc, and > I'm willing to expand on this. I believe it can be implemented > without having a durable context object to represent a transaction between > the client and the catalog. > > The main idea for "opaque commit IDs" is to allow more flexibility for > Catalog implementations, while keeping the same client-side guarantees > (snapshot isolation, causally consistent multi-table changes, etc.). > > Thanks, > Dmitri. > > On Mon, Jun 16, 2025 at 9:32 PM Maninderjit Singh < > parmar.maninder...@gmail.com> wrote: > >> Hi Iceberg dev community, >> >> We have been iterating on the Multi Table Transactions proposal and have >> merged the proposals for using catalog authored timestamps and sequence >> numbers together as well incorporated feedback from the community: >> Proposal: Multi-table multi-statement transactions for Apache Iceberg >> REST Catalog >> <https://drive.google.com/open?id=1KVgUJc1WgftHfLz118vMbEE7HV8_pUDk4s-GJFDyAOE> >> >> We have captured the tradeoffs involved with each approach as well as the >> reasoning for making those choices. We would love to hear your opinions on >> the consolidated proposal and which approach is more suitable for your >> requirements and why. >> >> Thank you in advance! >> >
