Hi All, Recap from today's community sync (please add/correct me if I missed anything):
* The community agrees that having multiple Persistence implementations for DynamoDB is acceptable. * Documentation should be crear about naming (config) and feature differences between these implementations to allow users to make intelligent decisions. * As usual new Persistence implementations are to be labelled as "beta" or "experimental" initially. * Since alternative Persistence modules are optional for end users (opt-in config), it is reasonable to develop the contribution in a series of small / fast PRs merged to `main` sequentially. * Performance tests / benchmarks with due rigor are encouraged. These will hopefully provide a solid foundation to compare different approaches. Cheers, Dmitri. On Fri, Sep 25, 2026 at 5:59 PM Yuewei Zhou <[email protected]> wrote: > Hi all, > > Following the discussion on the GitHub issue, and as suggested there, I'd > like to bring this proposal to the list for wider visibility and feedback. > > Summary: a new, opt-in persistence.type=dynamodb backend that reuses the > existing AtomicOperationMetaStoreManager (per-entity compare-and-set) over > a new DynamoDbBasePersistence adapter - the same shape relational-jdbc > uses, so the manager, catalog API, and entity model are unchanged. The > motivation is a serverless, AWS-native metastore, so AWS deployments don't > have to run a stateful Postgres or MongoDB purely to hold catalog metadata. > > On the issue we aligned that this fits Polaris's usual pattern for optional > backends - a self-contained, pluggable module, selected by config, with no > default change, no migration, and no new dependency or enforcement of any > kind on users or downstream projects that don't enable it - which is a > reasonable basis to start building on and kick off a series of PRs. I'd of > course still welcome input on the design as it takes shape. > > One trade-off I'd especially like thoughts on: this per-entity-CAS approach > versus layering DynamoDB under the existing NoSQL framework (single > named-pointer / HEAD commit) - roughly, independent-write concurrency on > one side vs. atomic multi-entity commits and cross-entity validation on the > other. > > GitHub issue: https://github.com/apache/polaris/issues/5615 > Design doc: > > https://docs.google.com/document/d/1vc7FCvznlfdBoHe0DJQo1vRzcd0AXHFX4cGrFhuiF9Y/edit?usp=sharing > > Thanks, > ZephyrYWZhou >
