Re: Proposed Public OpenAPI Changes

2025-09-08 Thread Alexandre Dutra
Hi Adnan, Thanks for bringing this topic to our attention. In general, for operations creating new resources, the event instrumentation logic could rely on the incoming request. E.g. for createCatalog, if the response does not contain the catalog resource, it is generally OK to grab it from the i

Re: [VOTE] Release Apache Polaris 1.1.0-incubating (rc0)

2025-09-03 Thread Alexandre Dutra
> > > > Task :polaris-runtime-service:intTest > > > > > > > > > > RestCatalogKeycloakFileIT > initializationError FAILED > > > > > org.junit.jupiter.api.extension.ParameterResolutionException at > > > > > ArrayList.java:1596 &g

Re: [VOTE] Release Apache Polaris 1.1.0-incubating (rc0)

2025-09-03 Thread Alexandre Dutra
sums > > > * Polaris starts both in-memory and with JDBC persistence > > > * Sanity-check read and write benchmarks complete successfully > > > > > > -- > > > > > > Pierre > > > > > > > > > On Mon, Sep 1, 2025 at 4:28 PM Alexandre

Re: [VOTE] Release Apache Polaris 1.1.0-incubating (rc0)

2025-09-01 Thread Alexandre Dutra
+1 (nb) Verified: - Signatures & checksums - Source release: no binary file found - Source release: ./gradlew rat test :checkForCopiedCode passes - Binary distribution: both server and admin tool start normally - Helm chart: installation from tarball and from repo both work (with local Docker ima

Re: Incubator report for Sep 25

2025-09-01 Thread Alexandre Dutra
Hi JB, Thanks for preparing the report, it looks great! In the proposals under discussion, I note that we've been also discussing: FGAC and S3 remote signing. I'm not sure it's necessary to list all topics under discussion though. Other than that, I fixed a minor typo. Thanks, Alex On Mon, Sep

[DISCUSS] S3 Remote Signing

2025-08-25 Thread Alexandre Dutra
Hi all, I'm starting a new thread on S3 remote signing to avoid hijacking the existing one [1]. To summarize our current progress: we have a design document [2], a Github issue [3] and an initial PR [4]. This initial PR establishes the foundation for the feature. In that PR, remote signing is ma

Re: Adding user principal tag in metrics

2025-08-25 Thread Alexandre Dutra
Hi Kostas, Thank you for the PR and for opening this discussion thread! I'm not against the proposed new metric tag, but I want to ensure everyone understands the potential consequences. Firstly, there's a risk of exposing sensitive information, specifically the principal name, within metric tag

Re: Heads-up: role-ARN option for S3 becoming optional

2025-08-21 Thread Alexandre Dutra
Hi, We just had an issue created by a user that was attempting to do use case #2 in Dennis' categorization ("Using DefaultCredentialsProvider directly without subscoping to access non-AWS s3-compat storage"): https://github.com/apache/polaris/issues/2398 This uncovered some interesting findings

Re: [DISCUSS] Merge Authenticator and ActiveRolesProvider

2025-08-21 Thread Alexandre Dutra
HttpAuthenticationMechanism. > > I'm going to check out this branch and validate, but I think it's a good > change. It helps with some changes I had planned to make, but hadn't had a > chance to work on yet. > > Mike > > On Tue, Aug 19, 2025 at 6:49 AM Alexandre Dut

Re: [DISCUSS] Labels for GitHub issues + PRs

2025-08-20 Thread Alexandre Dutra
Hi Robert, I'm +1 on your proposal, although I could see a valid reason to keep the "question" label as well, even if it's unused. Thanks, Alex On Wed, Aug 20, 2025 at 10:56 AM Robert Stupp wrote: > > Hi all, > > We currently have a lot of GitHub labels for issues and pull requests. > Many of t

Re: [RELEASE] 1.1.0-incubating release preparation

2025-08-20 Thread Alexandre Dutra
Hi all, Would it be possible to include the below PR in 1.1.0: https://github.com/apache/polaris/pull/2404 This PR deprecates `ActiveRolesProvider`, thus sending a clear message about its future removal while providing integrators with ample time to adjust their implementations accordingly. Rel

Re: Re: [DISCUSS] S3 Credential vending without STS

2025-08-19 Thread Alexandre Dutra
uss options for > > > Polaris to send specific credentials to clients (a.k.a. vended > > credentials) > > > when STS is not available. > > > > > > For the sake of clarity, let's re-title the thread for replies related to > > > the remo

Re: [DISCUSS] Merge Authenticator and ActiveRolesProvider

2025-08-19 Thread Alexandre Dutra
t. > > Mixing one > > Authenticator implementation with another Role Resolver looks really > > awkward. > > > > If some Authenticator implementations need to break this process down into > > two > > stages, it is ok, but Polaris core still receives one (immuta

Re: Re: [DISCUSS] S3 Credential vending without STS

2025-08-18 Thread Alexandre Dutra
is a big feature deserving a design doc and community discussion. Can > we have a design doc first? > > Yufei > > > On Thu, Aug 14, 2025 at 8:53 AM Alexandre Dutra wrote: > > > Hi all, > > > > I've drafted an initial version of remote signing enablement in &g

Re: Re: [DISCUSS] S3 Credential vending without STS

2025-08-14 Thread Alexandre Dutra
t; > > > > > Remote signing sounds a good idea! Looking forward to a proposal/design > > doc. > > > > > > Yufei > > > > > > > > > On Fri, Aug 1, 2025 at 8:44 AM Pat Patterson > > > wrote: > > > > >

Re: [DISCUSS] Merge polaris-service-common into polaris-runtime-service

2025-08-05 Thread Alexandre Dutra
Hello again! The PR has been merged, make sure to rebase if you have any ongoing work. Thanks, Alex On Tue, Aug 5, 2025 at 2:39 PM Robert Stupp wrote: > > +1 from mv POV to merge > > On Tue, Aug 5, 2025 at 1:58 PM Alexandre Dutra wrote: > > > > Hello all, > > &g

Re: [DISCUSS] Merge polaris-service-common into polaris-runtime-service

2025-08-05 Thread Alexandre Dutra
Hello all, We would like to merge [2233] sooner than later. Is it OK to merge today? Thanks, Alex [2233] https://github.com/apache/polaris/pull/2233 On Fri, Aug 1, 2025 at 2:18 PM Alexandre Dutra wrote: > > Hi all, > > Here is the PR: > > https://github.com/apache/polaris/pu

Re: Schema setup in admin tool and server

2025-08-05 Thread Alexandre Dutra
gt; > > > 4) options [...] > > > > > > I believe the primary schema setup case (with many options, etc.) should > > be > > > through the admin tool. > > > > > > If the server bootstraps automatically, it should use only defaults for

[DISCUSS] Merge Authenticator and ActiveRolesProvider

2025-08-04 Thread Alexandre Dutra
Hi all, ActiveRolesProvider was introduced back in January, in order to enrich SecurityContext with valid roles for a given principal. But that was before the introduction of Quarkus, and the introduction of external authentication with Quarkus Security and OIDC. TLDR: ActiveRolesProvider became

Re: [DISCUSS] Adding endpointInternal to AwsStorageConfigInfo

2025-08-02 Thread Alexandre Dutra
gt; > > Polaris in a way that Polaris connections go through a certain > > > internal > > > > > > network, while connections from query engines running outside of > > that > > > > > > deployment environment go through a different network. This is ve

Re: [DISCUSS] Merge polaris-service-common into polaris-runtime-service

2025-08-01 Thread Alexandre Dutra
Hi all, Here is the PR: https://github.com/apache/polaris/pull/2233 Thanks, Alex On Fri, Aug 1, 2025 at 12:51 PM Robert Stupp wrote: > > WFM > > On Fri, Aug 1, 2025 at 12:45 PM Alexandre Dutra wrote: > > > > Hi Robert, > > > > I would very much like to

Re: [DISCUSS] Merge polaris-service-common into polaris-runtime-service

2025-08-01 Thread Alexandre Dutra
distinct modules, which is what I've been advocating > for a "long" time ;) > > Regarding naming/where, I think we should retain "runtime" as a > container for the server and admin-tool runnables. > > WDYT? > > On Fri, Aug 1, 2025 at 12:12 PM Alexa

Schema setup in admin tool and server

2025-08-01 Thread Alexandre Dutra
Hi all, A nice recent contribution [1], still under review, proposes to create a separate admin tool command for setting up the database schema. Currently, schema setup is done as part of realm bootstrapping [2], which can happen at server bootstrap, when running the tool's bootstrap command, or

Re: [DISCUSS] Merge polaris-service-common into polaris-runtime-service

2025-08-01 Thread Alexandre Dutra
l the packages in the resulting module having a ".quarkus." token - Rename all the classes in the resulting module having a "Quarkus" prefix Thanks, Alex On Thu, Jul 31, 2025 at 8:11 PM Alexandre Dutra wrote: > > Hi Dmitri, > > Thanks for your reply. My proposal

Re: [DISCUSS] Merge polaris-service-common into polaris-runtime-service

2025-07-31 Thread Alexandre Dutra
a > Quarkus build env. is a nightmare :) ). I hope this will not interfere with > your proposal. I'm mentioning it here only because it affects > the polaris-runtime-service module (IIRC). > > Cheers, > Dmitri. > > On Thu, Jul 31, 2025 at 1:36 PM Alexandre Dutra wrot

Re: [DISCUSS] Merge polaris-service-common into polaris-runtime-service

2025-07-31 Thread Alexandre Dutra
rd > > wrote: > > > > > The project already *has* adopted Quarkus for good, so I think making the > > > module structure reflect that is fine. In addition to the configuration > > > issue, which is significant, I think cutting down on the number

Re: [DISCUSS] Merge polaris-service-common into polaris-runtime-service

2025-07-31 Thread Alexandre Dutra
ity for all tests. > Sure, not all tests have to be `@QuarkusTest`s, but I could imagine > that there will be tests that do not need Quarkus are annotated as > such. > > On Thu, Jul 31, 2025 at 12:19 PM Alexandre Dutra wrote: > > > > Hi all, > > > > The polaris

Re: [DISCUSS] Adding endpointInternal to AwsStorageConfigInfo

2025-07-31 Thread Alexandre Dutra
Hi Dmitri, I think your suggestion makes sense. We added something similar in Nessie long ago, and it is definitely useful. I left some comments in the PR. Thanks, Alex On Thu, Jul 31, 2025 at 4:12 AM Dmitri Bourlatchkov wrote: > > Hi All, > > I propose to add an `endpointInternal` optional pa

[DISCUSS] Merge polaris-service-common into polaris-runtime-service

2025-07-31 Thread Alexandre Dutra
Hi all, The polaris-service-common module is a reminiscence of the times where we were still supporting two runtimes. I think it has now become obsolete, and could be merged into polaris-runtime-service. One annoyance that polaris-service-common brings is with configuration classes that have to

Re: [PROPOSAL] Release Apache Polaris 1.0.1-incubating

2025-07-23 Thread Alexandre Dutra
+1 on the proposed plan, thanks JB! Alex On Tue, Jul 22, 2025 at 6:43 PM yun zou wrote: > > +1 on 1.0.1 with just helm fix. > > Best Regards, > Yun > > On Tue, Jul 22, 2025 at 9:05 AM Prashant Singh > wrote: > > > +1 to 1.0.1 with just helm fix ! > > > > On Tue, Jul 22, 2025 at 7:07 AM Dmitri B

Re: [Discussion] Decoupling Helm chart releases (+ nightly build proposal)

2025-07-19 Thread Alexandre Dutra
t; > > > >> choices), so having it in a separate repository is probably going > > > to > > > > > make > > > > > >> maintenance easier (to recap: I originally supported more > > frequent / > > > > > >> independent

Re: [DISCUSS] Gradle and Maven runner plugins

2025-07-17 Thread Alexandre Dutra
Hi all, If the runner plugins can save us from the dependency nightmare between Spark and Quarkus, imo that's a huge selling point. An example: the QuarkusSparkIT test lives in its own module (!), precisely because of dependency issues. Would the runner plugin allow us to move that test back to t

Re: [Discussion] Decoupling Helm chart releases (+ nightly build proposal)

2025-07-14 Thread Alexandre Dutra
Hi all, For reference and completeness, this has also been previously discussed in a much older thread: https://lists.apache.org/thread/428xb6dfrmm7xgr91p2dxoy8ptcyovs2 So far the consensus was, as Yufei pointed out, to release the Helm chart along with the Polaris server release (+docker images,