Thanks Dmitri for the update. Given that the community has agreed that multiple Persistence implementations for DynamoDB are acceptable, I suggest moving forward with https://github.com/apache/polaris/pull/5622, pending appropriate review.
Yufei On Thu, Oct 1, 2026 at 11:25 AM Dmitri Bourlatchkov <[email protected]> wrote: > Hi All, > > Recap from today's community sync (please add/correct me if I missed > anything): > > * The community agrees that having multiple Persistence implementations for > DynamoDB is acceptable. > > * Documentation should be crear about naming (config) and > feature differences between these implementations to allow users to make > intelligent decisions. > > * As usual new Persistence implementations are to be labelled as "beta" or > "experimental" initially. > > * Since alternative Persistence modules are optional for end users (opt-in > config), it is reasonable to develop the contribution in a series of small > / fast PRs merged to `main` sequentially. > > * Performance tests / benchmarks with due rigor are encouraged. These will > hopefully provide a solid foundation to compare different approaches. > > Cheers, > Dmitri. > > On Fri, Sep 25, 2026 at 5:59 PM Yuewei Zhou <[email protected]> > wrote: > > > Hi all, > > > > Following the discussion on the GitHub issue, and as suggested there, I'd > > like to bring this proposal to the list for wider visibility and > feedback. > > > > Summary: a new, opt-in persistence.type=dynamodb backend that reuses the > > existing AtomicOperationMetaStoreManager (per-entity compare-and-set) > over > > a new DynamoDbBasePersistence adapter - the same shape relational-jdbc > > uses, so the manager, catalog API, and entity model are unchanged. The > > motivation is a serverless, AWS-native metastore, so AWS deployments > don't > > have to run a stateful Postgres or MongoDB purely to hold catalog > metadata. > > > > On the issue we aligned that this fits Polaris's usual pattern for > optional > > backends - a self-contained, pluggable module, selected by config, with > no > > default change, no migration, and no new dependency or enforcement of any > > kind on users or downstream projects that don't enable it - which is a > > reasonable basis to start building on and kick off a series of PRs. I'd > of > > course still welcome input on the design as it takes shape. > > > > One trade-off I'd especially like thoughts on: this per-entity-CAS > approach > > versus layering DynamoDB under the existing NoSQL framework (single > > named-pointer / HEAD commit) - roughly, independent-write concurrency on > > one side vs. atomic multi-entity commits and cross-entity validation on > the > > other. > > > > GitHub issue: https://github.com/apache/polaris/issues/5615 > > Design doc: > > > > > https://docs.google.com/document/d/1vc7FCvznlfdBoHe0DJQo1vRzcd0AXHFX4cGrFhuiF9Y/edit?usp=sharing > > > > Thanks, > > ZephyrYWZhou > > >
