Hi all, I'd like to elaborate on the "pluggable persistence" concept defined my previous message, and how it might evolve in the future:
The proposal from PR [3960] originally suggested three distinct "store types": metastore, metrics, and events. Although that PR was closed because of inactivity, the concept remains relevant. If we decide to support multiple datasources for each store type, I envision the following approach: Polaris would provide datasource definitions for every combination of store type and JDBC driver (such as "postgresql-metastore" or "h2-metrics"). The current "enabler property" would be replaced by a map of properties keyed by the store type. For instance: polaris.persistence.relational.jdbc.metastore.datasource=postgresql-metastore polaris.persistence.relational.jdbc.metrics.datasource=postgresql-metrics polaris.persistence.relational.jdbc.events.datasource=postgresql-events In this configuration, three separate PostgreSQL datasources would be enabled, each managing its own connection pool and settings. Alternatively, if a user wants to use a single datasource for all data, they could simply map every store type to the same name (e.g., "postgresql-metastore"). This would result in only one active datasource and a single connection pool for the metastore, metrics, and events. It is still open for debate whether the default setup should prioritize a single shared datasource or separate ones for each store type. While this is primarily speculative at this point, since the feature hasn't been finalized, it demonstrates that we have a viable path forward should we choose to implement this architecture. Thanks, Alex [3960]: https://github.com/apache/polaris/pull/3960 On Wed, Jun 17, 2026 at 9:26 PM Alexandre Dutra <[email protected]> wrote: > > Hi all, > > I have recently submitted a PR [4812] that introduces support for > multiple datasources with possibility of runtime activation. > > I would like to open a discussion regarding the intent behind this > proposal and the advantages it offers. > > While Quarkus is a powerful platform, it presents certain challenges, > particularly with JDBC datasources. Currently, the datasource type > must be defined at build time using the "db-kind" property. This makes > managing multiple datasources difficult, especially for Polaris, where > users should ideally be able to select the appropriate datasource (and > driver) based on their specific requirements. > > The core of this proposal relies on a key fact: while "db-kind" is > fixed at build time, the "active" property can be toggled at runtime. > Combined with named datasources, this property becomes the missing > lever to achieve runtime datasource activation. > > Under this change, Polaris must include predefined named datasources > for every supported driver. Users simply choose which ones to activate > at runtime using an enabler property: > "polaris.persistence.relational.jdbc.datasource", which identifies the > named datasource for activation. (Worth noting: an inactive datasource > does not consume any resources at runtime.) > > As a result, the server will now include the H2 driver by default, as > it is a prerequisite for selecting an H2 datasource at runtime. We > will need to follow a similar approach for any other JDBC drivers we > intend to support in the future. > > This architectural shift is necessary for several reasons: > > 1. Flexible defaults: It allows us to use JDBC + H2 as the default for > server images (see these related discussions for details: [1] [2]) > while enabling users to switch to PostgreSQL or other drivers *via > configuration alone*, thus eliminating the need to rebuild Polaris > entirely. > > 2. Pluggable persistence: It enables the persistence layer to connect > to multiple datasources simultaneously. For instance, one could use a > specific datasource for main persistence and another for metrics – a > concept previously discussed in [3960] that lacked an easy > implementation path in my opinion. > > I look forward to hearing your thoughts on this proposal. > > Thanks, > Alex > > [4812]: https://github.com/apache/polaris/pull/4812 > [3960]: https://github.com/apache/polaris/pull/3960 > > [1]: https://lists.apache.org/thread/yw8l026g2smdk7gdg7k61tdcvdwcncqw > [2]: https://lists.apache.org/thread/nzoljc1ohnsq4f5o28dh4opqkqw3p09h
