> From the Iceberg client's > perspective, it builds the table metadata locally and expects to > commit the new tables and updates in a single server roundtrip.
Ayush, why do we need a stage creation REST call if the engines commit both in a single server roundtrip? Can you share more details about the use case? One related question: if the engine skips the stage-create request, how does it obtain the credentials needed to write the table data and metadata? That said, I think it is reasonable not to reject an updateForStagedCreate in a multi-table transaction. My understanding is that the updateForStagedCreate REST call validates constraints and persists the table independently. Whether or not the stage-create REST call occurred, the catalog should be able to process a complete and valid updateForStagedCreate. In that case, omitting any stage-create call may also be acceptable during a commitTransaction. It doesn't affect the persistence at all. Yufei On Thu, Jul 9, 2026 at 7:34 PM Dmitri Bourlatchkov <[email protected]> wrote: > Hi Ayush, > > Sorry about the delay. I'll try to review soon :) > > Cheers, > Dmitri. > > On Wed, Jul 1, 2026 at 1:28 PM Ayush Saxena <[email protected]> wrote: > > > Hi Folks, > > I recently opened a PR to introduce support for staged table creations > > within the commitTransaction REST API endpoint [1]. Because this > > touches core transactional boundaries, I wanted to bring it to the dev > > list for discussion and feedback. > > > > Currently, Polaris throws a BadRequestException if a > > CommitTransactionRequest contains an updateForStagedCreate (identified > > by the AssertTableDoesNotExist requirement). This prevents compute > > engines from executing multi-table transactions that involve creating > > new tables alongside updating existing ones. From the Iceberg client's > > perspective, it builds the table metadata locally and expects to > > commit the new tables and updates in a single server roundtrip. > > > > This PR aims to remove this restriction so Polaris can process mixed > > batches of staged creates and regular updates atomically. > > > > I'd love to get the community's thoughts on supporting this flow and > > hear any high-level feedback you might have. The PR is here [1] > > > > -Ayush > > > > [1] https://github.com/apache/polaris/pull/4939 > > >
