Thanks Alex for putting together the proposal and for elaborating on the potential path toward pluggable persistence. I agree that runtime datasource activation helps address some of the Quarkus limitations and provides a practical mechanism for supporting multiple datasources.
That said, I think this may oversimplify the use cases around splitting tables across different datasources. The main motivation for multiple datasources is workload isolation. For example, events, metrics, and potentially other high traffic tables may need to be isolated from the critical metastore tables. But once we allow each store type to be mapped either to the same datasource or to different datasources, the persistence model becomes harder to reason about. For example, if two tables are configured to use the same datasource, should the persistence layer rely on database joins? If they use different datasources, the same logic would need to fall back to in memory joins or avoid joins entirely. This means query and transaction behavior may depend on the deployment configuration, which feels confusing and fragile. I think we probably need a more explicit design here. We should decide whether store types are truly isolated persistence boundaries, or whether they are just logical groupings that may still be colocated. That decision affects schema evolution, migrations, validation, query patterns, and consistency semantics [1]. So while the proposed configuration mechanism may be useful, I don't think it fully resolves the architectural questions around multi datasource persistence. I would prefer to discuss the persistence model more holistically before we introduce flexible mapping across store types. A good starting point is the use case, e.g., event persistence. 1. https://github.com/apache/polaris/issues/3890 Yufei On Thu, Jun 18, 2026 at 12:03 PM Romain Manni-Bucau <[email protected]> wrote: > Hi Dmitri, > > Assuming you put metadata and metrics/events (maybe metrics but these ones > where not under my radar) in different databases to leverage different > tuning and aggregation level but still have exactly once guarantee. > I assume at some point there are or will be some writes in the same request > (stat updates, meta refresh/audit etc). > > While a single request/background task is guaranteed to be single > datasource backed there is no real topic if it is what you have in mind. > > Romain Manni-Bucau > @rmannibucau <https://x.com/rmannibucau> | .NET Blog > <https://dotnetbirdie.github.io/> | Blog <https://rmannibucau.github.io/> > | Old > Blog <http://rmannibucau.wordpress.com> | Github > <https://github.com/rmannibucau> | LinkedIn > <https://www.linkedin.com/in/rmannibucau> | Book > < > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > Javaccino founder (Java/.NET service - contact via linkedin) > > > Le jeu. 18 juin 2026 à 19:58, Dmitri Bourlatchkov <[email protected]> a > écrit : > > > Hi Romain, > > > > In what cases / API calls do you envision XA to be used in Polaris? > > > > I'm asking just for my education, as I do not see how / where it could be > > relevant ATM. > > > > Thanks, > > Dmitri. > > > > On Thu, Jun 18, 2026 at 1:11 PM Romain Manni-Bucau < > [email protected]> > > wrote: > > > > > Ok, good it was already thought. > > > > > > Think worse case an user can set the 3 same or enable XA using agroal > > > capabilities ([1]) > > > > > > Thanks > > > > > > [1] > > > > > > > > > https://quarkus.io/guides/datasource#quarkus-agroal_quarkus-datasource-jdbc-transactions > > > > > > Romain Manni-Bucau > > > @rmannibucau <https://x.com/rmannibucau> | .NET Blog > > > <https://dotnetbirdie.github.io/> | Blog < > https://rmannibucau.github.io/ > > > > > > | Old > > > Blog <http://rmannibucau.wordpress.com> | Github > > > <https://github.com/rmannibucau> | LinkedIn > > > <https://www.linkedin.com/in/rmannibucau> | Book > > > < > > > > > > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > > > > > Javaccino founder (Java/.NET service - contact via linkedin) > > > > > > > > > Le jeu. 18 juin 2026 à 18:17, Alexandre Dutra <[email protected]> a > > écrit > > > : > > > > > > > Hi Romain, > > > > > > > > You raise good points. > > > > > > > > Datasource multitenancy in Quarkus would have helped a lot during > this > > > > old discussion: [1] where we eventually decided to go with one > > > > datasource for N realms, out of better choices by then. But that ship > > > > has sailed. > > > > > > > > And good question about XA: the proposal in [3960] does not include > > > > any transactional coordination between datasources, afaict, and the > > > > outbox pattern doesn't fit anymore in the design we picked for events > > > > (and probably for metrics). This pattern was considered initially > > > > iirc, as it would have allowed exactly-once semantics for events; but > > > > in the end, we went with separate tables updated independently (~= > > > > at-most-once semantics). > > > > > > > > Thanks, > > > > Alex > > > > > > > > [1]: > https://lists.apache.org/thread/kvt3gd29hyxlhnwxr9b674s555jlvjjs > > > > [3960]: https://github.com/apache/polaris/pull/3960 > > > > > > > > On Thu, Jun 18, 2026 at 8:38 AM Romain Manni-Bucau > > > > <[email protected]> wrote: > > > > > > > > > > Hi Alexandre, > > > > > > > > > > Can be worth shiming in the quarkus thread about it: > > > > > https://github.com/quarkusio/quarkus/issues/11861 before merging a > > new > > > > way > > > > > to do it (even if it delays from a few weeks the feature) I think. > > > > > > > > > > Technically one thing the 1 database -> N databases (not speaking > > about > > > > > tenants where there is no challenge but meta+metrics+events) brings > > the > > > > > transactional challenge, do you go XA with this shift or do you > plan > > > some > > > > > recovery (kind of outbox pattern with some challenges)? > > > > > > > > > > Romain Manni-Bucau > > > > > @rmannibucau <https://x.com/rmannibucau> | .NET Blog > > > > > <https://dotnetbirdie.github.io/> | Blog < > > > https://rmannibucau.github.io/> > > > > | Old > > > > > Blog <http://rmannibucau.wordpress.com> | Github > > > > > <https://github.com/rmannibucau> | LinkedIn > > > > > <https://www.linkedin.com/in/rmannibucau> | Book > > > > > < > > > > > > > > > > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > > > > > > > > Javaccino founder (Java/.NET service - contact via linkedin) > > > > > > > > > > > > > > > Le mer. 17 juin 2026 à 21:37, Alexandre Dutra <[email protected]> > a > > > > écrit : > > > > > > > > > > > Hi all, > > > > > > > > > > > > I'd like to elaborate on the "pluggable persistence" concept > > defined > > > > > > my previous message, and how it might evolve in the future: > > > > > > > > > > > > The proposal from PR [3960] originally suggested three distinct > > > "store > > > > > > types": metastore, metrics, and events. Although that PR was > closed > > > > > > because of inactivity, the concept remains relevant. If we decide > > to > > > > > > support multiple datasources for each store type, I envision the > > > > > > following approach: > > > > > > > > > > > > Polaris would provide datasource definitions for every > combination > > of > > > > > > store type and JDBC driver (such as "postgresql-metastore" or > > > > > > "h2-metrics"). The current "enabler property" would be replaced > by > > a > > > > > > map of properties keyed by the store type. For instance: > > > > > > > > > > > > > > > > > > > > > > > > > > > > polaris.persistence.relational.jdbc.metastore.datasource=postgresql-metastore > > > > > > > > > > > > polaris.persistence.relational.jdbc.metrics.datasource=postgresql-metrics > > > > > > > > > polaris.persistence.relational.jdbc.events.datasource=postgresql-events > > > > > > > > > > > > In this configuration, three separate PostgreSQL datasources > would > > be > > > > > > enabled, each managing its own connection pool and settings. > > > > > > > > > > > > Alternatively, if a user wants to use a single datasource for all > > > > > > data, they could simply map every store type to the same name > > (e.g., > > > > > > "postgresql-metastore"). This would result in only one active > > > > > > datasource and a single connection pool for the metastore, > metrics, > > > > > > and events. > > > > > > > > > > > > It is still open for debate whether the default setup should > > > > > > prioritize a single shared datasource or separate ones for each > > store > > > > > > type. > > > > > > > > > > > > While this is primarily speculative at this point, since the > > feature > > > > > > hasn't been finalized, it demonstrates that we have a viable path > > > > > > forward should we choose to implement this architecture. > > > > > > > > > > > > Thanks, > > > > > > Alex > > > > > > > > > > > > [3960]: https://github.com/apache/polaris/pull/3960 > > > > > > > > > > > > On Wed, Jun 17, 2026 at 9:26 PM Alexandre Dutra < > [email protected] > > > > > > > wrote: > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > I have recently submitted a PR [4812] that introduces support > for > > > > > > > multiple datasources with possibility of runtime activation. > > > > > > > > > > > > > > I would like to open a discussion regarding the intent behind > > this > > > > > > > proposal and the advantages it offers. > > > > > > > > > > > > > > While Quarkus is a powerful platform, it presents certain > > > challenges, > > > > > > > particularly with JDBC datasources. Currently, the datasource > > type > > > > > > > must be defined at build time using the "db-kind" property. > This > > > > makes > > > > > > > managing multiple datasources difficult, especially for > Polaris, > > > > where > > > > > > > users should ideally be able to select the appropriate > datasource > > > > (and > > > > > > > driver) based on their specific requirements. > > > > > > > > > > > > > > The core of this proposal relies on a key fact: while "db-kind" > > is > > > > > > > fixed at build time, the "active" property can be toggled at > > > runtime. > > > > > > > Combined with named datasources, this property becomes the > > missing > > > > > > > lever to achieve runtime datasource activation. > > > > > > > > > > > > > > Under this change, Polaris must include predefined named > > > datasources > > > > > > > for every supported driver. Users simply choose which ones to > > > > activate > > > > > > > at runtime using an enabler property: > > > > > > > "polaris.persistence.relational.jdbc.datasource", which > > identifies > > > > the > > > > > > > named datasource for activation. (Worth noting: an inactive > > > > datasource > > > > > > > does not consume any resources at runtime.) > > > > > > > > > > > > > > As a result, the server will now include the H2 driver by > > default, > > > as > > > > > > > it is a prerequisite for selecting an H2 datasource at runtime. > > We > > > > > > > will need to follow a similar approach for any other JDBC > drivers > > > we > > > > > > > intend to support in the future. > > > > > > > > > > > > > > This architectural shift is necessary for several reasons: > > > > > > > > > > > > > > 1. Flexible defaults: It allows us to use JDBC + H2 as the > > default > > > > for > > > > > > > server images (see these related discussions for details: [1] > > [2]) > > > > > > > while enabling users to switch to PostgreSQL or other drivers > > *via > > > > > > > configuration alone*, thus eliminating the need to rebuild > > Polaris > > > > > > > entirely. > > > > > > > > > > > > > > 2. Pluggable persistence: It enables the persistence layer to > > > connect > > > > > > > to multiple datasources simultaneously. For instance, one could > > > use a > > > > > > > specific datasource for main persistence and another for > metrics > > – > > > a > > > > > > > concept previously discussed in [3960] that lacked an easy > > > > > > > implementation path in my opinion. > > > > > > > > > > > > > > I look forward to hearing your thoughts on this proposal. > > > > > > > > > > > > > > Thanks, > > > > > > > Alex > > > > > > > > > > > > > > [4812]: https://github.com/apache/polaris/pull/4812 > > > > > > > [3960]: https://github.com/apache/polaris/pull/3960 > > > > > > > > > > > > > > [1]: > > > > https://lists.apache.org/thread/yw8l026g2smdk7gdg7k61tdcvdwcncqw > > > > > > > [2]: > > > > https://lists.apache.org/thread/nzoljc1ohnsq4f5o28dh4opqkqw3p09h > > > > > > > > > > > > > > > >
