+1 (binding) Verified sigs/sums/license/build/test (Java 17)
-Dan On Thu, Feb 27, 2025 at 11:30 AM Aihua Xu <aihu...@gmail.com> wrote: > Thanks Alex. > > On Thu, Feb 27, 2025 at 10:31 AM Alex Dutra <alex.du...@dremio.com.invalid> > wrote: > >> Hi Aihua, >> >> I was indeed suspecting that you had a custom RESTClient :-) Thanks for >> digging further and clarifying! Let me know if you need my help in adapting >> your code. >> >> And with that: +1 (non-binding) from me as well. >> >> Thanks, >> >> Alex >> >> On Thu, Feb 27, 2025 at 6:45 PM Aihua Xu <aihu...@gmail.com> wrote: >> >>> Hi Alex, >>> >>> I checked our code further. We have an internal implementation >>> for RESTClient which needs an update after your change to place the token >>> in DefaultAuthSession and not pass through initHeaders anymore. The >>> existing code assumes the token coming from initHeaders and we need to make >>> the change to get the token from the headers of DefaultAuthSession. >>> >>> With that, +1 (non-binding) >>> >>> Thanks, >>> Aihua >>> >>> >>> >>> On Thu, Feb 27, 2025 at 9:26 AM Alex Dutra <alex.du...@dremio.com.invalid> >>> wrote: >>> >>>> Hi Aihua Xu, >>>> >>>> I reviewed your PR but without further details I do not agree with your >>>> change, and I am unable to reproduce the issue. Besides, we have unit >>>> tests that cover this extensively. >>>> >>>> Could you please provide a simple reproducer or a test case? >>>> >>>> Thanks! >>>> >>>> Alex >>>> >>>> On Thu, Feb 27, 2025 at 5:06 PM Aihua Xu <aihu...@gmail.com> wrote: >>>> >>>>> I'm running within our integration tests. I'm able to trace and find >>>>> the breaking change: https://github.com/apache/iceberg/pull/11992. >>>>> >>>>> Seems we need to make the following changes to pass the token: >>>>> https://github.com/apache/iceberg/pull/12415/files. Alex, can you >>>>> help take a look at this? >>>>> >>>>> if (hasCredential) { >>>>> authResponse = >>>>> OAuth2Util.fetchToken( >>>>> initClient, initHeaders, credential, scope, oauth2ServerUri, >>>>> optionalOAuthParams); >>>>> Map<String, String> authHeaders = >>>>> RESTUtil.merge(initHeaders, >>>>> OAuth2Util.authHeaders(authResponse.token())); >>>>> config = fetchConfig(initClient, authHeaders, props); >>>>> } else { >>>>> authResponse = null; >>>>> Map<String, String> authHeaders = >>>>> RESTUtil.merge(initHeaders, OAuth2Util.authHeaders(initToken)); >>>>> config = fetchConfig(initClient, authHeaders, props); >>>>> } >>>>> >>>>> >>>>> On Thu, Feb 27, 2025 at 12:46 AM Alex Dutra >>>>> <alex.du...@dremio.com.invalid> wrote: >>>>> >>>>>> Hi Aihua, >>>>>> >>>>>> I just tested 1.8.1 with Polaris OSS and I am not seeing anything >>>>>> different. Can you share your setup? >>>>>> >>>>>> Below is my Spark setup. >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Alex >>>>>> >>>>>> ./gradlew run >>>>>> >>>>>> token=$(curl -s http://localhost:8181/api/catalog/v1/oauth/tokens \ >>>>>> --user root:<secret> \ >>>>>> -d grant_type=client_credentials \ >>>>>> -d scope=PRINCIPAL_ROLE:ALL | sed -n >>>>>> 's/.*"access_token":"\([^"]*\)".*/\1/p') >>>>>> >>>>>> curl -s -H "Authorization: Bearer ${token}" \ >>>>>> -H 'Accept: application/json' \ >>>>>> -H 'Content-Type: application/json' \ >>>>>> http://localhost:8181/api/management/v1/catalogs \ >>>>>> -d '{ >>>>>> "catalog": { >>>>>> "name": "polaris_demo", >>>>>> "type": "INTERNAL", >>>>>> "readOnly": false, >>>>>> "properties": { >>>>>> "default-base-location": "file:///tmp/polaris/" >>>>>> }, >>>>>> "storageConfigInfo": { >>>>>> "storageType": "FILE", >>>>>> "allowedLocations": [ >>>>>> "file:///tmp" >>>>>> ] >>>>>> } >>>>>> } >>>>>> }' >>>>>> >>>>>> >>>>>> spark-sql \ >>>>>> --packages >>>>>> org.apache.iceberg:iceberg-spark-runtime-3.5_2.12:1.8.1 \ >>>>>> --conf >>>>>> spark.sql.extensions=org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions >>>>>> \ >>>>>> --conf >>>>>> spark.sql.catalog.polaris=org.apache.iceberg.spark.SparkCatalog \ >>>>>> --conf >>>>>> spark.sql.catalog.polaris.catalog-impl=org.apache.iceberg.rest.RESTCatalog >>>>>> \ >>>>>> --conf spark.sql.catalog.polaris.uri= >>>>>> http://127.0.0.1:8181/api/catalog \ >>>>>> --conf spark.sql.catalog.polaris.credential=root:<secret> \ >>>>>> --conf spark.sql.catalog.polaris.scope=PRINCIPAL_ROLE:ALL \ >>>>>> --conf spark.sql.catalog.polaris.warehouse=polaris_demo >>>>>> >>>>>> spark-sql (default)> create namespace polaris.foo; >>>>>> 25/02/27 09:43:24 WARN RESTSessionCatalog: Iceberg REST client is >>>>>> missing the OAuth2 server URI configuration and defaults to >>>>>> http://127.0.0.1:8181/api/catalog/v1/oauth/tokens. This automatic >>>>>> fallback will be removed in a future Iceberg release.It is recommended to >>>>>> configure the OAuth2 endpoint using the 'oauth2-server-uri' property to >>>>>> be >>>>>> prepared. This warning will disappear if the OAuth2 endpoint is >>>>>> explicitly >>>>>> configured. See https://github.com/apache/iceberg/issues/10537 >>>>>> Time taken: 0.688 seconds >>>>>> spark-sql (default)> >>>>>> >>>>>> >>>>>> On Thu, Feb 27, 2025 at 12:22 AM Aihua Xu <aihu...@gmail.com> wrote: >>>>>> >>>>>>> I tested 1.8.1 RC with Snowflake build. I'm seeing the following >>>>>>> (I'm not seeing that in 1.7.x). >>>>>>> >>>>>>> "exception": "java.io.IOException: *Authorization header is >>>>>>> missing*\n\tat >>>>>>> org.apache.polaris.service.dropwizard.auth.PolarisPrincipalAuthenticator.filter(PolarisPrincipalAuthenticator.java:43)\n\tat >>>>>>> org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:108)\n\tat >>>>>>> org.glassfish.jersey.server.ContainerFilteringStage.apply(ContainerFilteringStage.java:44)\n\tat >>>>>>> org.glassfish.jersey.process.internal.Stages.process(Stages.java:173)\n\tat >>>>>>> org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:248). >>>>>>> >>>>>>> Does anyone know which change could introduce this? >>>>>>> >>>>>>> On Wed, Feb 26, 2025 at 5:16 AM Péter Váry < >>>>>>> peter.vary.apa...@gmail.com> wrote: >>>>>>> >>>>>>>> +1 >>>>>>>> >>>>>>>> checked the signatures, checksums >>>>>>>> build and run some tests >>>>>>>> >>>>>>>> Amogh Jahagirdar <2am...@gmail.com> ezt írta (időpont: 2025. febr. >>>>>>>> 26., Sze, 6:11): >>>>>>>> >>>>>>>>> +1 (binding) >>>>>>>>> >>>>>>>>> Verified signatures, checksum, RAT checks. >>>>>>>>> Ran build and test with JDK17 >>>>>>>>> >>>>>>>>> Thanks, >>>>>>>>> Amogh Jahagirdar >>>>>>>>> >>>>>>>>> On Wed, Feb 26, 2025 at 2:30 AM Honah J. <hon...@apache.org> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> +1 (binding) >>>>>>>>>> >>>>>>>>>> - Checked signatures and checksum >>>>>>>>>> - Checked license >>>>>>>>>> - Full Build and Test >>>>>>>>>> >>>>>>>>>> Best regards, >>>>>>>>>> Honah >>>>>>>>>> >>>>>>>>>> On Tue, Feb 25, 2025 at 10:52 AM Russell Spitzer < >>>>>>>>>> russell.spit...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> +1 >>>>>>>>>>> Checked Sigs and Checksum >>>>>>>>>>> Ran Rat >>>>>>>>>>> Ran full build/test >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Tue, Feb 25, 2025 at 11:30 AM Driesprong, Fokko >>>>>>>>>>> <fo...@driesprong.frl> wrote: >>>>>>>>>>> >>>>>>>>>>>> +1 (binding) >>>>>>>>>>>> >>>>>>>>>>>> - Checked signatures and checksum >>>>>>>>>>>> - Checked licenses >>>>>>>>>>>> - Spotchecked NOTICE/LICENSE >>>>>>>>>>>> >>>>>>>>>>>> Kind regards, >>>>>>>>>>>> Fokko >>>>>>>>>>>> >>>>>>>>>>>> Op di 25 feb 2025 om 16:56 schreef Kevin Liu < >>>>>>>>>>>> kevinjq...@apache.org>: >>>>>>>>>>>> >>>>>>>>>>>>> +1 (non-binding) >>>>>>>>>>>>> >>>>>>>>>>>>> I followed "How to Verify a Release" >>>>>>>>>>>>> <https://iceberg.apache.org/how-to-release/#how-to-verify-a-release> >>>>>>>>>>>>> . >>>>>>>>>>>>> Checked out artifact from SVN, >>>>>>>>>>>>> ``` >>>>>>>>>>>>> svn checkout >>>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.8.1-rc1/ >>>>>>>>>>>>> . >>>>>>>>>>>>> ``` >>>>>>>>>>>>> >>>>>>>>>>>>> Verified >>>>>>>>>>>>> * Signature Good >>>>>>>>>>>>> * Checksum Ok >>>>>>>>>>>>> * RAT check passed. 1 unrelated error message >>>>>>>>>>>>> ``` >>>>>>>>>>>>> ERROR: Ignored 0 lines in your exclusion files as comments or >>>>>>>>>>>>> empty lines. >>>>>>>>>>>>> ``` >>>>>>>>>>>>> * Build + test passed, running on Java 17.0.6 (openjdk 17.0.6 >>>>>>>>>>>>> 2023-01-17 LTS) on M1 >>>>>>>>>>>>> * Ran a few examples on Spark >>>>>>>>>>>>> * Ran pyiceberg integration tests, >>>>>>>>>>>>> https://github.com/kevinjqliu/iceberg-python/pull/11 >>>>>>>>>>>>> >>>>>>>>>>>>> I ran the tests both with and without the docker daemon. >>>>>>>>>>>>> Without docker, a few tests failed in `iceberg-aws`, >>>>>>>>>>>>> `iceberg-azure`, and >>>>>>>>>>>>> `iceberg-kafka-connect`. There's already an issue to track this at >>>>>>>>>>>>> https://github.com/apache/iceberg/issues/12236. >>>>>>>>>>>>> I'm also continuing to see the flakey test for >>>>>>>>>>>>> `iceberg-core`'s `testConcurrentFastAppends` test. I believe this >>>>>>>>>>>>> is a >>>>>>>>>>>>> local issue with my machine. >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for running the release, Eduard! >>>>>>>>>>>>> >>>>>>>>>>>>> Best, >>>>>>>>>>>>> Kevin Liu >>>>>>>>>>>>> >>>>>>>>>>>>> On Tue, Feb 25, 2025 at 4:23 AM Jean-Baptiste Onofré < >>>>>>>>>>>>> j...@nanthrax.net> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> +1 (non binding) >>>>>>>>>>>>>> >>>>>>>>>>>>>> - Hash and checksum are good >>>>>>>>>>>>>> - LICENSE and NOTICE are OK in different distributed >>>>>>>>>>>>>> artifacts (source >>>>>>>>>>>>>> distribution, aws bundle, etc) >>>>>>>>>>>>>> - ASF header present in all expected files >>>>>>>>>>>>>> - No binary file found in the source distribution >>>>>>>>>>>>>> - Did quick smoke tests >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks, >>>>>>>>>>>>>> Regards >>>>>>>>>>>>>> JB >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Feb 24, 2025 at 1:46 PM Eduard Tudenhoefner >>>>>>>>>>>>>> <etudenhoef...@gmail.com> wrote: >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > Hi Everyone, >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > I propose that we release the following RC as the official >>>>>>>>>>>>>> Apache Iceberg 1.8.1 release. >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > The commit ID is 9ce0fcf0af7becf25ad9fc996c3bad2afdcfd33d >>>>>>>>>>>>>> > * This corresponds to the tag: apache-iceberg-1.8.1-rc1 >>>>>>>>>>>>>> > * >>>>>>>>>>>>>> https://github.com/apache/iceberg/commits/apache-iceberg-1.8.1-rc1 >>>>>>>>>>>>>> > * >>>>>>>>>>>>>> https://github.com/apache/iceberg/tree/9ce0fcf0af7becf25ad9fc996c3bad2afdcfd33d >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > The release tarball, signature, and checksums are here: >>>>>>>>>>>>>> > * >>>>>>>>>>>>>> https://dist.apache.org/repos/dist/dev/iceberg/apache-iceberg-1.8.1-rc1 >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > You can find the KEYS file here: >>>>>>>>>>>>>> > * https://downloads.apache.org/iceberg/KEYS >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > Convenience binary artifacts are staged on Nexus. The Maven >>>>>>>>>>>>>> repository URL is: >>>>>>>>>>>>>> > * >>>>>>>>>>>>>> https://repository.apache.org/content/repositories/orgapacheiceberg-1184/ >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > Please download, verify, and test. >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > Please vote in the next 72 hours. >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > [ ] +1 Release this as Apache Iceberg 1.8.1 >>>>>>>>>>>>>> > [ ] +0 >>>>>>>>>>>>>> > [ ] -1 Do not release this because... >>>>>>>>>>>>>> > >>>>>>>>>>>>>> > Only PMC members have binding votes, but other community >>>>>>>>>>>>>> members are encouraged to cast >>>>>>>>>>>>>> > non-binding votes. This vote will pass if there are 3 >>>>>>>>>>>>>> binding +1 votes and more binding >>>>>>>>>>>>>> > +1 votes than -1 votes. >>>>>>>>>>>>>> > >>>>>>>>>>>>>> >>>>>>>>>>>>>
