Hi All, The way I see it is that Alex's PR [4812] is an incremental improvement at the technical level in our Quarkus server.
It does not impose or require any redesign right now. Yet, it enables more options for future consideration. I think it is worth merging "as is" for now. Then we can resume the discussions about realm and database association. [4812] https://github.com/apache/polaris/pull/4812 Cheers, Dmitri. On Thu, Jun 25, 2026 at 12:01 PM Alexandre Dutra <[email protected]> wrote: > Hi Yufei, > > > > if Polaris managed the connection pools itself using HikariCP [...] > would that also solve both the "no rebuild" requirement? > > Yes, that would work. > > However, we need to consider some significant tradeoffs: > > - Manual lifecycle management becomes necessary (handling shutdown > hooks, pool eviction during catalog drops, etc.). > > - Native image support is lost. While Polaris official images don't > use them yet, this would block that path for future development. Also, > integrators may be building Polaris in native mode today. > > - Loss of @DataSource CDI injection. > > - Loss of integrated health checks and metrics. > > - Loss of Dev Services (though this is less critical as we don't > currently use them). > > > Am I missing something? If not, this seems worth prototyping. > > To me, sacrificing the above benefits just to allow runtime selection > of a single (!) datasource feels overkill. My PR [4812] achieves the > same objective without these drawbacks. What makes the HikariCP > approach preferable in your view? > > If the goal is to revisit the datasource-per-realm discussion, we > should prioritize a design document over a POC. Such a change would > fundamentally reshape Polaris internals. We still haven't settled on > the core architecture: should it be database-per-realm, per-catalog, > per-store-type (as seen in [3960]), or a combination of these? > > Thanks, > Alex > > On Thu, Jun 25, 2026 at 2:30 AM Yufei Gu <[email protected]> wrote: > > > > Thanks for the clear explanation, Alex. I think the use cases for > avoiding > > rebuilds make sense. > > > > One question: if Polaris managed the connection pools itself using > HikariCP > > (or Agroal/DBCP directly), instead of relying on Quarkus' datasource > > extension, would that also solve both the "no rebuild" requirement and > the > > dynamic datasource problem? > > > > My understanding is that, in JVM mode, users could simply provide the > > appropriate JDBC driver JAR on the classpath or Polaris can ship them if > > possible and configure the driver class and JDBC URL at runtime, without > > rebuilding Polaris. If that's correct, this seems to address the primary > > goal while also avoiding Quarkus' build time datasource limitation. > > > > For example, switching databases would simply become: > > polaris.persistence.jdbc.driver=com.mysql.cj.jdbc.Driver > > polaris.persistence.jdbc.url=jdbc:mysql://... > > > > HikariCP can load the driver at runtime, as long as the MySQL driver JAR > is > > available on the classpath. > > > > Am I missing something? If not, this seems worth prototyping. I'd be > happy > > to help build a small POC to validate the approach. > > > > Yufei > > > > > > On Wed, Jun 24, 2026 at 6:50 AM Alexandre Dutra <[email protected]> > wrote: > > > > > Hi Yufei, > > > > > > The primary goal is straightforward: we want to eliminate the need for > > > users to rebuild Polaris simply to switch their database provider > > > (e.g., moving from PostgreSQL to MySQL). > > > > > > Let's take the MySQL PR [4281] as an example: as it is designed today, > > > users are forced to rebuild Polaris using the includeMysqlDriver build > > > property. This constraint prevents them from using official Polaris > > > images and forces the maintenance of custom images. > > > > > > With the proposed changes in [4812], users would instead simply use > > > the official image and apply a runtime configuration: > > > polaris.persistence.relational.jdbc.datasource=mysql. This effectively > > > removes the burden of custom builds and image management. > > > > > > To clarify, [4812] does not change ANYTHING in the current datasource > > > design. It's still one single datasource for all data. > > > > > > I briefly mentioned multi-datasource to demonstrate that my changes > > > wouldn't block [3960] if we decide to pursue it later. As a reminder, > > > [3960] was introducing the ability to use a separate datasource for > > > metrics and events. > > > > > > If you'd like to revisit the datasource-per-store-type design from > > > [3960], or revive the datasource-per-realm design [1], I'm happy to > > > discuss those further. But [4812] is really only about runtime > > > selection of fixed datasources. > > > > > > Thanks, > > > Alex > > > > > > [3960]: https://github.com/apache/polaris/pull/3960 > > > [4281]: https://github.com/apache/polaris/pull/4281 > > > [4812]: https://github.com/apache/polaris/pull/4812 > > > > > > [1]: https://lists.apache.org/thread/kvt3gd29hyxlhnwxr9b674s555jlvjjs > > > > > > On Wed, Jun 24, 2026 at 3:24 AM Yufei Gu <[email protected]> wrote: > > > > > > > > Hi Alex, > > > > > > > > I'm still trying to understand the intended use cases behind PR 4812. > > > > > > > > The examples discussed so far focus on runtime selection between H2, > > > > PostgreSQL, MySQL, etc. In those cases, it seems the requirement is > to > > > > choose one datasource implementation at runtime, while the actual > > > > datasource definitions remain known at build time. > > > > > > > > The use case I'm more familiar with is multi realm support, here is > an > > > > example[1]. In that model, each realm could potentially have its own > > > > database configuration, which implies dynamic datasource creation and > > > > lifecycle management at runtime. The number of datasources is not > known > > > > ahead of time and can change as realms are added or removed. If > Polaris > > > > needs to support that model in the future, The Quarkus datasource > > > extension > > > > may not be the right abstraction, since it is fundamentally centered > > > around > > > > datasources defined at build time. PR 4812 works around this > limitation > > > by > > > > predeclaring datasource definitions and activating them selectively, > but > > > > that approach does not seem to generalize well to truly dynamic > > > datasource > > > > provisioning. > > > > > > > > For dynamic realm backed datasources, the cleaner design may be a > Polaris > > > > managed pool registry using DBCP, HikariCP, or Agroal directly, > rather > > > than > > > > relying on Quarkus datasource configuration. That would allow > datasource > > > > pools to be created, updated, and removed dynamically as realms are > > > managed. > > > > > > > > Could you elaborate a bit more on the concrete use cases you have in > mind > > > > for PR 4812? I'm trying to understand whether the primary goal is > runtime > > > > selection among a fixed set of datasources, or whether it is > intended as > > > a > > > > stepping stone toward more dynamic datasource management. > > > > > > > > 1. https://github.com/apache/polaris/discussions/4761 > > > > > > > > Thanks, > > > > Yufei > > > > > > > > > > > > > > > > > > > > On Fri, Jun 19, 2026 at 12:47 PM Romain Manni-Bucau < > > > [email protected]> > > > > wrote: > > > > > > > > > Guess it depends if you do use native-image or not, didn't check in > > > latest > > > > > version except the github sources of some datasource but the trick > was > > > > > working well in java mode (guess the recommended mode for polaris) > so > > > I'm > > > > > not sure why it wouldn't work - and in native mode the entrypoint > is > > > not > > > > > even a java command so not a real concern, isnt it? > > > > > > > > > > > > > > > From what I do see in h2, db2 etc modules there is nothing crazy > > > requiring > > > > > build time wiring (for java mode once again) so I think it will > work. > > > > > Worse case you can easily do a quarkus-jdbc-generic since at the > end > > > it is > > > > > mainly hints for native-image and a few auto loading (driver for > ex) > > > but > > > > > using any pool programmatically - as polaris does handle the > datasource > > > > > programmatically for ex - will does it. > > > > > > > > > > So overall think it works but happy to look if it doesn't work. > > > > > > > > > > Romain Manni-Bucau > > > > > @rmannibucau <https://x.com/rmannibucau> | .NET Blog > > > > > <https://dotnetbirdie.github.io/> | Blog < > > > https://rmannibucau.github.io/> > > > > > | Old > > > > > Blog <http://rmannibucau.wordpress.com> | Github > > > > > <https://github.com/rmannibucau> | LinkedIn > > > > > <https://www.linkedin.com/in/rmannibucau> | Book > > > > > < > > > > > > > > > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > > > > > > > > > Javaccino founder (Java/.NET service - contact via linkedin) > > > > > > > > > > > > > > > Le ven. 19 juin 2026 à 18:51, Alexandre Dutra <[email protected]> > a > > > écrit > > > > > : > > > > > > > > > > > Hi Romain, > > > > > > > > > > > > I don't think your trick would work with Quarkus, would it? As > the > > > > > > driver has to be present at build time. > > > > > > > > > > > > Also, I think we want to achieve an "it just works" kind of user > > > > > > experience. That's the main reason for including h2 as we want a > > > > > > simple "docker run apache/polaris" to "just work". > > > > > > > > > > > > Thanks, > > > > > > Alex > > > > > > > > > > > > On Fri, Jun 19, 2026 at 6:46 PM Romain Manni-Bucau > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > Why providing h2 in the image? I know a common trick I do use > is > > > to set > > > > > > the > > > > > > > classpath in my entry point to something like > > > > > > > "myjar1.jar:myjar2.jar:.....:/opt/app/extensions/custom/*" - > with > > > jib I > > > > > > > do > <extraClasspath>/opt/app/extensions/custom/*</extraClasspath>. > > > > > > > This enables to mount the JDBC driver in this custom folder and > > > get it > > > > > > > working OOTB, with helm it is just a matter of mounting either > an > > > OCI > > > > > > image > > > > > > > with just the driver in this folder or using an init container > to > > > copy > > > > > it > > > > > > > using an empty dir/ephemeral volume as pivot folder. > > > > > > > The good thing about that is: no default driver required but > open > > > to > > > > > any > > > > > > > so no related CVE (h2 got several by the past which would be a > > > negative > > > > > > > aspect for prod when using pg for ex). > > > > > > > > > > > > > > Romain Manni-Bucau > > > > > > > @rmannibucau <https://x.com/rmannibucau> | .NET Blog > > > > > > > <https://dotnetbirdie.github.io/> | Blog < > > > > > https://rmannibucau.github.io/> > > > > > > | Old > > > > > > > Blog <http://rmannibucau.wordpress.com> | Github > > > > > > > <https://github.com/rmannibucau> | LinkedIn > > > > > > > <https://www.linkedin.com/in/rmannibucau> | Book > > > > > > > < > > > > > > > > > > > > > > > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > > > > > > > > > > > > Javaccino founder (Java/.NET service - contact via linkedin) > > > > > > > > > > > > > > > > > > > > > Le ven. 19 juin 2026 à 17:18, Dmitri Bourlatchkov < > > > [email protected]> a > > > > > > > écrit : > > > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > > > Focusing on the technical capabilities of PR [4812], I think > it > > > > > > provides a > > > > > > > > valuable enhancement and we should merge it. > > > > > > > > > > > > > > > > I do not see any adverse effects of this PR on existing > > > > > functionality, > > > > > > but > > > > > > > > it certainly opens new use cases like H2 inside pre-built > docker > > > > > > images and > > > > > > > > MySQL (potentially). > > > > > > > > > > > > > > > > [4812] https://github.com/apache/polaris/pull/4812 > > > > > > > > > > > > > > > > Cheers, > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > On Fri, Jun 19, 2026 at 4:59 AM Alexandre Dutra < > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > Although the conversation has shifted toward > multi-datasource > > > > > > support, > > > > > > > > > the primary focus in this thread remains [4812]: the > ability to > > > > > > > > > activate a datasource at runtime, even in single-source > > > scenarios. > > > > > > > > > > > > > > > > > > I want to emphasize that this functionality is essential > > > > > > independently > > > > > > > > > of the multi-datasource debate. For example, it is > required to > > > > > > provide > > > > > > > > > out-of-the-box support for JDBC + H2 within Polaris > images, and > > > > > also > > > > > > > > > to provide support for more databases (e.g. MySQL). > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > Alex > > > > > > > > > > > > > > > > > > [4812]: https://github.com/apache/polaris/pull/4812 > > > > > > > > > > > > > > > > > > On Thu, Jun 18, 2026 at 9:28 PM Dmitri Bourlatchkov < > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Hi Romain, > > > > > > > > > > > > > > > > > > > > Thanks for the info! > > > > > > > > > > > > > > > > > > > > Metrics requests generally read from the MetaStore but > write > > > only > > > > > > in > > > > > > > > the > > > > > > > > > > Metrics datasource. > > > > > > > > > > > > > > > > > > > > The latest thinking about Events is to isolate them from > the > > > > > > initial > > > > > > > > > > request via the Quarkus Event Bus (async delivery), IIRC. > > > > > > > > > > > > > > > > > > > > The new Idempotency feature [3] might be the first case > > > where we > > > > > > > > (could) > > > > > > > > > do > > > > > > > > > > writes into multiple datasources. However, the way it is > > > designed > > > > > > does > > > > > > > > > not > > > > > > > > > > assume or require coordination of changes / commits > between > > > the > > > > > > > > MetaStore > > > > > > > > > > and the Idempotency data store, AFAIK. > > > > > > > > > > > > > > > > > > > > In general, this is a very interesting topic. It has > > > implications > > > > > > for > > > > > > > > our > > > > > > > > > > (MetaStore) Persistence interface contracts too. IIRC, > > > previous > > > > > [1] > > > > > > > > > > discussions generally settled on the idea that each > > > Persistence > > > > > API > > > > > > > > call > > > > > > > > > > forms one atomic (and durable) change. Yet, I do not > think we > > > > > have > > > > > > > > > > completely ironed out all wrinkles [2] there. The tricky > > > aspect > > > > > is > > > > > > that > > > > > > > > > > Persistence may be backed by a non-transactional > database. > > > > > > Currently we > > > > > > > > > > have JDBC and MongoDB, which could be said to be > > > transactional, > > > > > > still > > > > > > > > the > > > > > > > > > > NoSQL Persistence in Polaris is built without assuming Tx > > > > > > capabilities, > > > > > > > > > so > > > > > > > > > > it will also work on non-transactional databases. > > > > > > > > > > > > > > > > > > > > This is just for context :) > > > > > > > > > > > > > > > > > > > > [1] > > > > > > https://lists.apache.org/thread/fh6t3mgzdb5ft0dt5q7qxm1pj1x5po75 > > > > > > > > > > > > > > > > > > > > [2] > > > > > > https://lists.apache.org/thread/rf5orxs815zs4h64p4rwp03q3pbgxb5r > > > > > > > > > > > > > > > > > > > > [3] > > > > > > https://lists.apache.org/thread/ksf5b5y3jt8dmft5kgblbwms8dhs4fn9 > > > > > > > > > > > > > > > > > > > > Cheers, > > > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > On Thu, Jun 18, 2026 at 3:03 PM Romain Manni-Bucau < > > > > > > > > > [email protected]> > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi Dmitri, > > > > > > > > > > > > > > > > > > > > > > Assuming you put metadata and metrics/events (maybe > > > metrics but > > > > > > these > > > > > > > > > ones > > > > > > > > > > > where not under my radar) in different databases to > > > leverage > > > > > > > > different > > > > > > > > > > > tuning and aggregation level but still have exactly > once > > > > > > guarantee. > > > > > > > > > > > I assume at some point there are or will be some > writes in > > > the > > > > > > same > > > > > > > > > request > > > > > > > > > > > (stat updates, meta refresh/audit etc). > > > > > > > > > > > > > > > > > > > > > > While a single request/background task is guaranteed > to be > > > > > single > > > > > > > > > > > datasource backed there is no real topic if it is what > you > > > have > > > > > > in > > > > > > > > > mind. > > > > > > > > > > > > > > > > > > > > > > Romain Manni-Bucau > > > > > > > > > > > @rmannibucau <https://x.com/rmannibucau> | .NET Blog > > > > > > > > > > > <https://dotnetbirdie.github.io/> | Blog < > > > > > > > > > https://rmannibucau.github.io/> > > > > > > > > > > > | Old > > > > > > > > > > > Blog <http://rmannibucau.wordpress.com> | Github > > > > > > > > > > > <https://github.com/rmannibucau> | LinkedIn > > > > > > > > > > > <https://www.linkedin.com/in/rmannibucau> | Book > > > > > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > > > > > > > > > > > > > > > > > > > > > Javaccino founder (Java/.NET service - contact via > > > linkedin) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Le jeu. 18 juin 2026 à 19:58, Dmitri Bourlatchkov < > > > > > > [email protected]> > > > > > > > > a > > > > > > > > > > > écrit : > > > > > > > > > > > > > > > > > > > > > > > Hi Romain, > > > > > > > > > > > > > > > > > > > > > > > > In what cases / API calls do you envision XA to be > used > > > in > > > > > > Polaris? > > > > > > > > > > > > > > > > > > > > > > > > I'm asking just for my education, as I do not see > how / > > > where > > > > > > it > > > > > > > > > could be > > > > > > > > > > > > relevant ATM. > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Dmitri. > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jun 18, 2026 at 1:11 PM Romain Manni-Bucau < > > > > > > > > > > > [email protected]> > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > Ok, good it was already thought. > > > > > > > > > > > > > > > > > > > > > > > > > > Think worse case an user can set the 3 same or > enable > > > XA > > > > > > using > > > > > > > > > agroal > > > > > > > > > > > > > capabilities ([1]) > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks > > > > > > > > > > > > > > > > > > > > > > > > > > [1] > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://quarkus.io/guides/datasource#quarkus-agroal_quarkus-datasource-jdbc-transactions > > > > > > > > > > > > > > > > > > > > > > > > > > Romain Manni-Bucau > > > > > > > > > > > > > @rmannibucau <https://x.com/rmannibucau> | .NET > Blog > > > > > > > > > > > > > <https://dotnetbirdie.github.io/> | Blog < > > > > > > > > > > > https://rmannibucau.github.io/ > > > > > > > > > > > > > > > > > > > > > > > > > > | Old > > > > > > > > > > > > > Blog <http://rmannibucau.wordpress.com> | Github > > > > > > > > > > > > > <https://github.com/rmannibucau> | LinkedIn > > > > > > > > > > > > > <https://www.linkedin.com/in/rmannibucau> | Book > > > > > > > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > > > > > > > > > > > > > > > > > > > > > > > > > Javaccino founder (Java/.NET service - contact via > > > > > linkedin) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Le jeu. 18 juin 2026 à 18:17, Alexandre Dutra < > > > > > > [email protected] > > > > > > > > > > > > > > > > > > a > > > > > > > > > > > > écrit > > > > > > > > > > > > > : > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Romain, > > > > > > > > > > > > > > > > > > > > > > > > > > > > You raise good points. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Datasource multitenancy in Quarkus would have > helped > > > a > > > > > lot > > > > > > > > during > > > > > > > > > > > this > > > > > > > > > > > > > > old discussion: [1] where we eventually decided > to go > > > > > with > > > > > > one > > > > > > > > > > > > > > datasource for N realms, out of better choices by > > > then. > > > > > But > > > > > > > > that > > > > > > > > > ship > > > > > > > > > > > > > > has sailed. > > > > > > > > > > > > > > > > > > > > > > > > > > > > And good question about XA: the proposal in > [3960] > > > does > > > > > > not > > > > > > > > > include > > > > > > > > > > > > > > any transactional coordination between > datasources, > > > > > > afaict, and > > > > > > > > > the > > > > > > > > > > > > > > outbox pattern doesn't fit anymore in the design > we > > > > > picked > > > > > > for > > > > > > > > > events > > > > > > > > > > > > > > (and probably for metrics). This pattern was > > > considered > > > > > > > > initially > > > > > > > > > > > > > > iirc, as it would have allowed exactly-once > > > semantics for > > > > > > > > > events; but > > > > > > > > > > > > > > in the end, we went with separate tables updated > > > > > > independently > > > > > > > > > (~= > > > > > > > > > > > > > > at-most-once semantics). > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > Alex > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1]: > > > > > > > > > > > > > > > > https://lists.apache.org/thread/kvt3gd29hyxlhnwxr9b674s555jlvjjs > > > > > > > > > > > > > > [3960]: > https://github.com/apache/polaris/pull/3960 > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jun 18, 2026 at 8:38 AM Romain > Manni-Bucau > > > > > > > > > > > > > > <[email protected]> wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi Alexandre, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Can be worth shiming in the quarkus thread > about > > > it: > > > > > > > > > > > > > > > > https://github.com/quarkusio/quarkus/issues/11861 > > > > > before > > > > > > > > > merging a > > > > > > > > > > > > new > > > > > > > > > > > > > > way > > > > > > > > > > > > > > > to do it (even if it delays from a few weeks > the > > > > > > feature) I > > > > > > > > > think. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Technically one thing the 1 database -> N > databases > > > > > (not > > > > > > > > > speaking > > > > > > > > > > > > about > > > > > > > > > > > > > > > tenants where there is no challenge but > > > > > > meta+metrics+events) > > > > > > > > > brings > > > > > > > > > > > > the > > > > > > > > > > > > > > > transactional challenge, do you go XA with this > > > shift > > > > > or > > > > > > do > > > > > > > > you > > > > > > > > > > > plan > > > > > > > > > > > > > some > > > > > > > > > > > > > > > recovery (kind of outbox pattern with some > > > challenges)? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Romain Manni-Bucau > > > > > > > > > > > > > > > @rmannibucau <https://x.com/rmannibucau> | > .NET > > > Blog > > > > > > > > > > > > > > > <https://dotnetbirdie.github.io/> | Blog < > > > > > > > > > > > > > https://rmannibucau.github.io/> > > > > > > > > > > > > > > | Old > > > > > > > > > > > > > > > Blog <http://rmannibucau.wordpress.com> | > Github > > > > > > > > > > > > > > > <https://github.com/rmannibucau> | LinkedIn > > > > > > > > > > > > > > > <https://www.linkedin.com/in/rmannibucau> | > Book > > > > > > > > > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > https://www.packtpub.com/en-us/product/java-ee-8-high-performance-9781788473064 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Javaccino founder (Java/.NET service - contact > via > > > > > > linkedin) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Le mer. 17 juin 2026 à 21:37, Alexandre Dutra < > > > > > > > > > [email protected]> > > > > > > > > > > > a > > > > > > > > > > > > > > écrit : > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I'd like to elaborate on the "pluggable > > > persistence" > > > > > > > > concept > > > > > > > > > > > > defined > > > > > > > > > > > > > > > > my previous message, and how it might evolve > in > > > the > > > > > > future: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The proposal from PR [3960] originally > suggested > > > > > three > > > > > > > > > distinct > > > > > > > > > > > > > "store > > > > > > > > > > > > > > > > types": metastore, metrics, and events. > Although > > > that > > > > > > PR > > > > > > > > was > > > > > > > > > > > closed > > > > > > > > > > > > > > > > because of inactivity, the concept remains > > > relevant. > > > > > > If we > > > > > > > > > decide > > > > > > > > > > > > to > > > > > > > > > > > > > > > > support multiple datasources for each store > > > type, I > > > > > > > > envision > > > > > > > > > the > > > > > > > > > > > > > > > > following approach: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Polaris would provide datasource definitions > for > > > > > every > > > > > > > > > > > combination > > > > > > > > > > > > of > > > > > > > > > > > > > > > > store type and JDBC driver (such as > > > > > > "postgresql-metastore" > > > > > > > > or > > > > > > > > > > > > > > > > "h2-metrics"). The current "enabler property" > > > would > > > > > be > > > > > > > > > replaced > > > > > > > > > > > by > > > > > > > > > > > > a > > > > > > > > > > > > > > > > map of properties keyed by the store type. > For > > > > > > instance: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > polaris.persistence.relational.jdbc.metastore.datasource=postgresql-metastore > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > polaris.persistence.relational.jdbc.metrics.datasource=postgresql-metrics > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > polaris.persistence.relational.jdbc.events.datasource=postgresql-events > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > In this configuration, three separate > PostgreSQL > > > > > > > > datasources > > > > > > > > > > > would > > > > > > > > > > > > be > > > > > > > > > > > > > > > > enabled, each managing its own connection > pool > > > and > > > > > > > > settings. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Alternatively, if a user wants to use a > single > > > > > > datasource > > > > > > > > > for all > > > > > > > > > > > > > > > > data, they could simply map every store type > to > > > the > > > > > > same > > > > > > > > name > > > > > > > > > > > > (e.g., > > > > > > > > > > > > > > > > "postgresql-metastore"). This would result in > > > only > > > > > one > > > > > > > > active > > > > > > > > > > > > > > > > datasource and a single connection pool for > the > > > > > > metastore, > > > > > > > > > > > metrics, > > > > > > > > > > > > > > > > and events. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > It is still open for debate whether the > default > > > setup > > > > > > > > should > > > > > > > > > > > > > > > > prioritize a single shared datasource or > separate > > > > > ones > > > > > > for > > > > > > > > > each > > > > > > > > > > > > store > > > > > > > > > > > > > > > > type. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > While this is primarily speculative at this > > > point, > > > > > > since > > > > > > > > the > > > > > > > > > > > > feature > > > > > > > > > > > > > > > > hasn't been finalized, it demonstrates that > we > > > have a > > > > > > > > viable > > > > > > > > > path > > > > > > > > > > > > > > > > forward should we choose to implement this > > > > > > architecture. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Alex > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [3960]: > > > https://github.com/apache/polaris/pull/3960 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Jun 17, 2026 at 9:26 PM Alexandre > Dutra < > > > > > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Hi all, > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I have recently submitted a PR [4812] that > > > > > introduces > > > > > > > > > support > > > > > > > > > > > for > > > > > > > > > > > > > > > > > multiple datasources with possibility of > > > runtime > > > > > > > > > activation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I would like to open a discussion > regarding the > > > > > > intent > > > > > > > > > behind > > > > > > > > > > > > this > > > > > > > > > > > > > > > > > proposal and the advantages it offers. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > While Quarkus is a powerful platform, it > > > presents > > > > > > certain > > > > > > > > > > > > > challenges, > > > > > > > > > > > > > > > > > particularly with JDBC datasources. > Currently, > > > the > > > > > > > > > datasource > > > > > > > > > > > > type > > > > > > > > > > > > > > > > > must be defined at build time using the > > > "db-kind" > > > > > > > > property. > > > > > > > > > > > This > > > > > > > > > > > > > > makes > > > > > > > > > > > > > > > > > managing multiple datasources difficult, > > > especially > > > > > > for > > > > > > > > > > > Polaris, > > > > > > > > > > > > > > where > > > > > > > > > > > > > > > > > users should ideally be able to select the > > > > > > appropriate > > > > > > > > > > > datasource > > > > > > > > > > > > > > (and > > > > > > > > > > > > > > > > > driver) based on their specific > requirements. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > The core of this proposal relies on a key > fact: > > > > > while > > > > > > > > > "db-kind" > > > > > > > > > > > > is > > > > > > > > > > > > > > > > > fixed at build time, the "active" property > can > > > be > > > > > > toggled > > > > > > > > > at > > > > > > > > > > > > > runtime. > > > > > > > > > > > > > > > > > Combined with named datasources, this > property > > > > > > becomes > > > > > > > > the > > > > > > > > > > > > missing > > > > > > > > > > > > > > > > > lever to achieve runtime datasource > activation. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Under this change, Polaris must include > > > predefined > > > > > > named > > > > > > > > > > > > > datasources > > > > > > > > > > > > > > > > > for every supported driver. Users simply > choose > > > > > which > > > > > > > > ones > > > > > > > > > to > > > > > > > > > > > > > > activate > > > > > > > > > > > > > > > > > at runtime using an enabler property: > > > > > > > > > > > > > > > > > > > > "polaris.persistence.relational.jdbc.datasource", > > > > > > which > > > > > > > > > > > > identifies > > > > > > > > > > > > > > the > > > > > > > > > > > > > > > > > named datasource for activation. (Worth > > > noting: an > > > > > > > > inactive > > > > > > > > > > > > > > datasource > > > > > > > > > > > > > > > > > does not consume any resources at runtime.) > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > As a result, the server will now include > the H2 > > > > > > driver by > > > > > > > > > > > > default, > > > > > > > > > > > > > as > > > > > > > > > > > > > > > > > it is a prerequisite for selecting an H2 > > > datasource > > > > > > at > > > > > > > > > runtime. > > > > > > > > > > > > We > > > > > > > > > > > > > > > > > will need to follow a similar approach for > any > > > > > other > > > > > > JDBC > > > > > > > > > > > drivers > > > > > > > > > > > > > we > > > > > > > > > > > > > > > > > intend to support in the future. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > This architectural shift is necessary for > > > several > > > > > > > > reasons: > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 1. Flexible defaults: It allows us to use > JDBC > > > + H2 > > > > > > as > > > > > > > > the > > > > > > > > > > > > default > > > > > > > > > > > > > > for > > > > > > > > > > > > > > > > > server images (see these related > discussions > > > for > > > > > > details: > > > > > > > > > [1] > > > > > > > > > > > > [2]) > > > > > > > > > > > > > > > > > while enabling users to switch to > PostgreSQL or > > > > > other > > > > > > > > > drivers > > > > > > > > > > > > *via > > > > > > > > > > > > > > > > > configuration alone*, thus eliminating the > > > need to > > > > > > > > rebuild > > > > > > > > > > > > Polaris > > > > > > > > > > > > > > > > > entirely. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > 2. Pluggable persistence: It enables the > > > > > persistence > > > > > > > > layer > > > > > > > > > to > > > > > > > > > > > > > connect > > > > > > > > > > > > > > > > > to multiple datasources simultaneously. For > > > > > > instance, one > > > > > > > > > could > > > > > > > > > > > > > use a > > > > > > > > > > > > > > > > > specific datasource for main persistence > and > > > > > another > > > > > > for > > > > > > > > > > > metrics > > > > > > > > > > > > – > > > > > > > > > > > > > a > > > > > > > > > > > > > > > > > concept previously discussed in [3960] that > > > lacked > > > > > an > > > > > > > > easy > > > > > > > > > > > > > > > > > implementation path in my opinion. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > I look forward to hearing your thoughts on > this > > > > > > proposal. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > > Alex > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [4812]: > > > > > https://github.com/apache/polaris/pull/4812 > > > > > > > > > > > > > > > > > [3960]: > > > > > https://github.com/apache/polaris/pull/3960 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > [1]: > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread/yw8l026g2smdk7gdg7k61tdcvdwcncqw > > > > > > > > > > > > > > > > > [2]: > > > > > > > > > > > > > > > > > > > > > > > https://lists.apache.org/thread/nzoljc1ohnsq4f5o28dh4opqkqw3p09h > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
