Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
damienburke commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1518387104 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: > When your token without id, should I use first JWK to verify your token? > Or we need to add the default id to the pulsar config, when your id is null, the pulsar will use this default id to find your JWK. The "default id" approach sounds cleaner and probably also easier to document. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve] Removed unnecessary native libraries from Docker images [pulsar]
lhotari commented on code in PR #22230: URL: https://github.com/apache/pulsar/pull/22230#discussion_r1518489699 ## docker/pulsar/build-scripts/remove-unnecessary-native-binaries.sh: ## @@ -0,0 +1,38 @@ +#!/usr/bin/env sh +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. +# + +set -e + +# Retain only native libraries for the architecture of this +# image +ARCH=$(uname -m) + +# Remove extra binaries for Netty TCNative +ls /pulsar/lib/io.netty-netty-tcnative-boringssl-static*Final-*.jar | grep -v linux-$ARCH | xargs rm + +# Prune extra libs from RocksDB JAR +mkdir /tmp/rocksdb +cd /tmp/rocksdb +ROCKSDB_JAR=$(ls /pulsar/lib/org.rocksdb-rocksdbjni-*.jar) +unzip $ROCKSDB_JAR > /dev/null + +ls librocksdbjni-* | grep -v librocksdbjni-linux-$ARCH-musl.so | xargs rm Review Comment: Since we have full glibc in the Alpine container, does that impact the library version which should be used (musl or glibc)? I don't know the details, but what I wonder about is which malloc implementation does the JVM use and which one is used by the rocksdb library and does it matter if they are different? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [Improve] Add PR-template [pulsar-client-go]
RobertIndie commented on code in PR #1196: URL: https://github.com/apache/pulsar-client-go/pull/1196#discussion_r1518484100 ## .github/PULL_REQUEST_TEMPLATE.md: ## @@ -69,3 +69,20 @@ This change added tests and can be verified as follows: - If yes, how is the feature documented? (not applicable / docs / GoDocs / not documented) - If a feature is not applicable for documentation, explain why? - If a feature is not documented yet in this PR, please create a followup issue for adding the documentation + +### Matching PR in forked repository + +PR in forked repository: +
Re: [PR] Add change log for 0.12.1 [pulsar-client-go]
RobertIndie merged PR #1189: URL: https://github.com/apache/pulsar-client-go/pull/1189 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-client-go) branch master updated: Add change log for 0.12.1 (#1189)
This is an automated email from the ASF dual-hosted git repository. zike pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-client-go.git The following commit(s) were added to refs/heads/master by this push: new 3935fd7b Add change log for 0.12.1 (#1189) 3935fd7b is described below commit 3935fd7b1efe0afe7643fdcc5687559a4d00a40b Author: Zike Yang AuthorDate: Sat Mar 9 13:33:30 2024 +0800 Add change log for 0.12.1 (#1189) ### Motivation Add 0.12.1 change log --- CHANGELOG.md | 9 + 1 file changed, 9 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 7a4c81cb..f25de209 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,15 @@ All notable changes to this project will be documented in this file. +[0.12.1] 2024-02-29 + +- [fix] Fix Infinite Loop in Reader's `HasNext` Function by @RobertIndie in [#1182](https://github.com/apache/pulsar-client-go/pull/1182) +- [fix] Fix available permits in MessageReceived by @panszobe in [#1181](https://github.com/apache/pulsar-client-go/pull/1181) +- [feat] Support partitioned topic reader by @RobertIndie in [#1178](https://github.com/apache/pulsar-client-go/pull/1178) +- [fix] Fix BytesSchema by @petermnhull in [#1173](https://github.com/apache/pulsar-client-go/pull/1173) +- [fix] Respect context cancellation in Flush by @jayshrivastava in [#1165](https://github.com/apache/pulsar-client-go/pull/1165) +- [fix] Fix SIGSEGV with zstd compression enabled by @RobertIndie in [#1164](https://github.com/apache/pulsar-client-go/pull/1164) + [0.12.0] 2024-01-10 ## What's Changed
Re: [PR] Add change log for 0.12.1 [pulsar-client-go]
RobertIndie commented on PR #1189: URL: https://github.com/apache/pulsar-client-go/pull/1189#issuecomment-1986738598 The vote has passed: https://lists.apache.org/thread/lbbj3mjcfm1cf42z88pbgszcjk5z877x -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
nodece commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1518468167 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: Ok, I will open a new PR to do that. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-site) branch main updated: Fix doc issue (#833)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new f3856a4d7b64 Fix doc issue (#833) f3856a4d7b64 is described below commit f3856a4d7b64c9b1501173ffa4ceadc467d3e295 Author: Jiwei Guo AuthorDate: Sat Mar 9 10:11:30 2024 +0800 Fix doc issue (#833) --- release-notes/versioned/pulsar-3.2.0.md | 2 +- release-notes/versioned/pulsar-3.2.1.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/release-notes/versioned/pulsar-3.2.0.md b/release-notes/versioned/pulsar-3.2.0.md index d0400445e900..abc8aadf7c02 100644 --- a/release-notes/versioned/pulsar-3.2.0.md +++ b/release-notes/versioned/pulsar-3.2.0.md @@ -227,5 +227,5 @@ sidebar_label: Apache Pulsar 3.2.0 - Upgrade Apache ZooKeeper to 3.9.1 [#20933](https://github.com/apache/pulsar/pull/20933) -> A special thanks to the following contributors (alphabetic Github IDs) who contributed to Pulsar 3.2.0: +> A special thanks to the following contributors who contributed to Pulsar 3.2.0: > lhotari, jiangpengcheng, merlimat, poorbarcode, nodece, thetumbled, hr, > BewareMyPower, RobertIndie, onobc, Technoboy-, wenbingshen, aloyszhang, > heesung-sn, massakam, liangyepianzhou, coderzc, asafm, crossoverJie, > TakaHiR07, zymap, yaalsn, rdhabalia, dragosvictor, pandalee99, AnonHxy, > hangc0276, pengxiangrui127, mattisonchao, Denovo1998, 1Jack2, shibd, > horizonzy, liudezhi2098, KevinLiLu, hanmz, erobot, Demogorgon314, tisonkun, > diegosalvi, liangyuanpeng, kecona, alpreu, tuteng, frank [...] diff --git a/release-notes/versioned/pulsar-3.2.1.md b/release-notes/versioned/pulsar-3.2.1.md index e24bb84968a6..8ede5fce771d 100644 --- a/release-notes/versioned/pulsar-3.2.1.md +++ b/release-notes/versioned/pulsar-3.2.1.md @@ -64,5 +64,5 @@ sidebar_label: Apache Pulsar 3.2.1 - Upgrade BookKeeper dependency to 4.16.4 [#21983](https://github.com/apache/pulsar/pull/21983) -> A special thanks to the following contributors (alphabetic Github IDs) who contributed to Pulsar 3.2.1: +> A special thanks to the following contributors who contributed to Pulsar 3.2.1: > mattisonchao, lhotari, poorbarcode, Technoboy-, jiangpengcheng, > dragosvictor, freeznet, liangyepianzhou, heesung-sn, zymap, massakam, > thetumbled, BewareMyPower, chenhongSZ, Shawyeok, hangc0276, 315157973, > coderzc, dao-jun.
Re: [PR] Fix doc issue [pulsar-site]
Technoboy- merged PR #833: URL: https://github.com/apache/pulsar-site/pull/833 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Fix doc issue [pulsar-site]
Technoboy- opened a new pull request, #833: URL: https://github.com/apache/pulsar-site/pull/833 ### Motivation Fix the doc, not list by alphabetic -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Release 3.2.1 [pulsar-site]
Technoboy- commented on code in PR #828: URL: https://github.com/apache/pulsar-site/pull/828#discussion_r1518453748 ## release-notes/versioned/pulsar-3.2.1.md: ## @@ -0,0 +1,68 @@ +--- +id: pulsar-3.2.1 +title: Apache Pulsar 3.2.1 +sidebar_label: Apache Pulsar 3.2.1 +--- + + 2024-03-08 + +### Broker +- Consistently add fine-grain authorization to REST API [#22202](https://github.com/apache/pulsar/pull/22202) +- Add fine-grain authorization to retention admin API [#22163](https://github.com/apache/pulsar/pull/22163) +- Print non log when delete partitioned topic failed [#22153](https://github.com/apache/pulsar/pull/22153) +- Fix broker not starting when both transactions and the Extensible Load Manager are enabled [#22139](https://github.com/apache/pulsar/pull/22139) +- Return getOwnerAsync without waiting on source broker upon Assigning and Releasing and handle role change during role init [#22112](https://github.com/apache/pulsar/pull/22112) +- Cache the internal writer when sent to system topic. [#22099](https://github.com/apache/pulsar/pull/22099) +- Enabling batch causes negative unackedMessages due to ack and delivery concurrency [#22090](https://github.com/apache/pulsar/pull/22090) +- Support running docker container with gid != 0 [#22081](https://github.com/apache/pulsar/pull/22081) +- Add an error log to troubleshoot the failure of starting broker registry [#22065](https://github.com/apache/pulsar/pull/22065) +- Set ServiceUnitStateChannel topic compaction threshold explicitly, improve getOwnerAsync, and fix other bugs [#22064](https://github.com/apache/pulsar/pull/22064) +- Fix hash collision when using a consumer name that ends with a number [#22053](https://github.com/apache/pulsar/pull/22053) +- Sanitize values before logging in apply-config-from-env.py script [#22044](https://github.com/apache/pulsar/pull/22044) +- Fix can not subscribe partitioned topic with a suffix-matched regexp [#22025](https://github.com/apache/pulsar/pull/22025) +- Do not retain the data in the system topic [#22022](https://github.com/apache/pulsar/pull/22022) +- Subscription stuck due to called Admin API analyzeSubscriptionBacklog [#22019](https://github.com/apache/pulsar/pull/22019) +- Do not try to open ML when the topic meta does not exist and do not expect to create a new one [#22004](https://github.com/apache/pulsar/pull/22004) +- Do not print an Error log when responding to HTTP-404 when calling Admin API and the topic does not exist [#21995](https://github.com/apache/pulsar/pull/21995) +- Do not close the socket if lookup failed due to LockBusyException [#21993](https://github.com/apache/pulsar/pull/21993) +- Support setting autoSkipNonRecoverableData dynamically in PersistentMessageExpiryMonitor [#21991](https://github.com/apache/pulsar/pull/21991) +- Add timeout for health check read [#21990](https://github.com/apache/pulsar/pull/21990) +- Fix schema deletion error when deleting a partitioned topic with many partitions and schema [#21977](https://github.com/apache/pulsar/pull/21977) +- Defer the ownership checks if the owner is inactive (ExtensibleLoadManager) [#21811](https://github.com/apache/pulsar/pull/21811) +- Fix compaction/replication data loss when expire messages [#21865](https://github.com/apache/pulsar/pull/21865) +- Skip loading the NAR packages if not configured [#21867](https://github.com/apache/pulsar/pull/21867) +- Improve NamespaceUnloadStrategy error message [#21880](https://github.com/apache/pulsar/pull/21880) +- Don't rollover empty ledgers based on inactivity [#21893](https://github.com/apache/pulsar/pull/21893) +- Expire messages according to ledger close time to avoid client clock skew [#21940](https://github.com/apache/pulsar/pull/21940) +- Fix reader stuck when read from compacted topic with read compact mode disable [#21969](https://github.com/apache/pulsar/pull/21969) + + +### Pulsar IO and Pulsar Functions +- [FN] Add configuration for connector & functions package url sources [#22184](https://github.com/apache/pulsar/pull/22184) +- [FN] Add missing "exception" argument to some log.error [#22140](https://github.com/apache/pulsar/pull/22140) +- [FN] Prevent putstate uses empty values [#22127](https://github.com/apache/pulsar/pull/22127) +- [FN] Optimize Function Worker startup by lazy loading and direct zip/bytecode access [#22122](https://github.com/apache/pulsar/pull/22122) +- [FN] Prevent create state table from API calls for non-exists instances [#22107](https://github.com/apache/pulsar/pull/22107) +- [FN] Use unified PackageManagement service to download packages [#21955](https://github.com/apache/pulsar/pull/21955) + +### CLI +- [Admin] Expose the offload threshold in seconds to the admin [#22101](https://github.com/apache/pulsar/pull/22101) +- [CLI] Fix the bug when set-retention specified size with -T [#22150](https://github.com/apache/pulsar/pull/22150) +- [CLI] Error out when --destinationBroker is set when
(pulsar-site) 01/01: Fix doc issue
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a commit to branch fix-doc-3 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git commit 4cde8fa860715c02e9b045c22209a37cedcc8caf Author: Jiwe Guo AuthorDate: Sat Mar 9 09:47:05 2024 +0800 Fix doc issue --- release-notes/versioned/pulsar-3.2.0.md | 2 +- release-notes/versioned/pulsar-3.2.1.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/release-notes/versioned/pulsar-3.2.0.md b/release-notes/versioned/pulsar-3.2.0.md index d0400445e900..abc8aadf7c02 100644 --- a/release-notes/versioned/pulsar-3.2.0.md +++ b/release-notes/versioned/pulsar-3.2.0.md @@ -227,5 +227,5 @@ sidebar_label: Apache Pulsar 3.2.0 - Upgrade Apache ZooKeeper to 3.9.1 [#20933](https://github.com/apache/pulsar/pull/20933) -> A special thanks to the following contributors (alphabetic Github IDs) who contributed to Pulsar 3.2.0: +> A special thanks to the following contributors who contributed to Pulsar 3.2.0: > lhotari, jiangpengcheng, merlimat, poorbarcode, nodece, thetumbled, hr, > BewareMyPower, RobertIndie, onobc, Technoboy-, wenbingshen, aloyszhang, > heesung-sn, massakam, liangyepianzhou, coderzc, asafm, crossoverJie, > TakaHiR07, zymap, yaalsn, rdhabalia, dragosvictor, pandalee99, AnonHxy, > hangc0276, pengxiangrui127, mattisonchao, Denovo1998, 1Jack2, shibd, > horizonzy, liudezhi2098, KevinLiLu, hanmz, erobot, Demogorgon314, tisonkun, > diegosalvi, liangyuanpeng, kecona, alpreu, tuteng, frank [...] diff --git a/release-notes/versioned/pulsar-3.2.1.md b/release-notes/versioned/pulsar-3.2.1.md index e24bb84968a6..8ede5fce771d 100644 --- a/release-notes/versioned/pulsar-3.2.1.md +++ b/release-notes/versioned/pulsar-3.2.1.md @@ -64,5 +64,5 @@ sidebar_label: Apache Pulsar 3.2.1 - Upgrade BookKeeper dependency to 4.16.4 [#21983](https://github.com/apache/pulsar/pull/21983) -> A special thanks to the following contributors (alphabetic Github IDs) who contributed to Pulsar 3.2.1: +> A special thanks to the following contributors who contributed to Pulsar 3.2.1: > mattisonchao, lhotari, poorbarcode, Technoboy-, jiangpengcheng, > dragosvictor, freeznet, liangyepianzhou, heesung-sn, zymap, massakam, > thetumbled, BewareMyPower, chenhongSZ, Shawyeok, hangc0276, 315157973, > coderzc, dao-jun.
(pulsar-site) branch fix-doc-3 created (now 4cde8fa86071)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a change to branch fix-doc-3 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git at 4cde8fa86071 Fix doc issue This branch includes the following new commits: new 4cde8fa86071 Fix doc issue The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
[PR] Removed unnecessary native libraries from Docker images [pulsar]
merlimat opened a new pull request, #22230: URL: https://github.com/apache/pulsar/pull/22230 ### Motivation The RocksDB jar that we are including in our Docker image is ~55 MB in size because it includes native library of RocksDB compiled for many different platforms. In reality, our Docker image is going to be used exactly on one single platorm, either `linux/x86-64` or `linux/arm64`. Therefore we can remove all the other RocksDB compiled libraries from the image. The resulting RocksDB jar is shrunk to 4.6 MB. Similarly, we can remove Netty TC natives jar that are not relevant for the container. ### Modifications Added a script to remove the unnecessary libraries. ### Verifying this change - [ ] Make sure that the change passes the CI checks. *(Please pick either of the following options)* This change is a trivial rework / code cleanup without any test coverage. *(or)* This change is already covered by existing tests, such as *(please describe tests)*. *(or)* This change added tests and can be verified as follows: *(example:)* - *Added integration tests for end-to-end deployment with large payloads (10MB)* - *Extended integration test for recovery after broker failure* ### Does this pull request potentially affect one of the following parts: *If the box was checked, please highlight the changes* - [ ] Dependencies (add or upgrade a dependency) - [ ] The public API - [ ] The schema - [ ] The default values of configurations - [ ] The threading model - [ ] The binary protocol - [ ] The REST endpoints - [ ] The admin CLI options - [ ] The metrics - [ ] Anything that affects deployment ### Documentation - [ ] `doc` - [ ] `doc-required` - [X] `doc-not-needed` - [ ] `doc-complete` ### Matching PR in forked repository PR in forked repository: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) branch master updated (95a53f3a033 -> 4effaa74b77)
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git from 95a53f3a033 [fix][client] fix Reader.hasMessageAvailable might return true after seeking to latest (#22201) add 4effaa74b77 [improve][pip] PIP-324: Alpine image (#22054) No new revisions were added by this update. Summary of changes: .github/workflows/pulsar-ci.yaml | 13 +-- build/build_java_test_image.sh | 1 - build/docker/Dockerfile| 100 build/docker/publish.sh| 57 - build/pulsar_ci_tool.sh| 48 +--- docker/glibc-package/Dockerfile| 79 + {build/docker => docker/glibc-package}/README.md | 31 ++--- docker/glibc-package/scripts/APKBUILD | 53 + .../glibc-package/scripts/glibc-bin.trigger| 3 +- .../glibc-package/scripts/ld.so.conf | 5 +- docker/pulsar/Dockerfile | 128 +++-- docker/pulsar/pom.xml | 3 - docker/pulsar/scripts/install-pulsar-client.sh | 30 - pulsar-io/kinesis/pom.xml | 2 +- tests/docker-images/java-test-image/Dockerfile | 55 + tests/docker-images/java-test-image/pom.xml| 10 +- .../docker-images/latest-version-image/Dockerfile | 29 ++--- tests/docker-images/pom.xml| 14 +++ .../tests/integration/cli/HealthCheckTest.java | 6 +- .../integration/offload/TestFileSystemOffload.java | 2 +- 20 files changed, 262 insertions(+), 407 deletions(-) delete mode 100644 build/docker/Dockerfile delete mode 100755 build/docker/publish.sh create mode 100644 docker/glibc-package/Dockerfile rename {build/docker => docker/glibc-package}/README.md (54%) create mode 100644 docker/glibc-package/scripts/APKBUILD copy pulsar-io/file/src/test/resources/org/apache/pulsar/io/file/nonGzipFile.txt => docker/glibc-package/scripts/glibc-bin.trigger (95%) mode change 100644 => 100755 copy pulsar-io/file/src/test/resources/org/apache/pulsar/io/file/nonGzipFile.txt => docker/glibc-package/scripts/ld.so.conf (93%) delete mode 100755 docker/pulsar/scripts/install-pulsar-client.sh
Re: [PR] [improve][pip] PIP-324: Alpine image [pulsar]
merlimat merged PR #22054: URL: https://github.com/apache/pulsar/pull/22054 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
svn commit: r67818 - /dev/pulsar/pulsar-3.1.3-candidate-2/ /release/pulsar/pulsar-3.1.3/
Author: penghui Date: Fri Mar 8 23:57:09 2024 New Revision: 67818 Log: Release Apache Pulsar 3.1.3 Added: release/pulsar/pulsar-3.1.3/ - copied from r67817, dev/pulsar/pulsar-3.1.3-candidate-2/ Removed: dev/pulsar/pulsar-3.1.3-candidate-2/
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
damienburke commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1518387104 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: > When your token without id, should I use first JWK to verify your token? > Or we need to add the default id to the pulsar config, when your id is null, the pulsar will use this default id to find your JWK. The "default id" approach sounds cleaner and probably also easier to document. Also @nodece - ill be offline until Sunday GMT nite, but will get to any open q's / comments then. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] Optimize seeking by timestamp [pulsar]
dao-jun commented on PR #22152: URL: https://github.com/apache/pulsar/pull/22152#issuecomment-1986407981 @lhotari I've updated the implementation, PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Release 3.1.3 [pulsar-site]
lhotari merged PR #832: URL: https://github.com/apache/pulsar-site/pull/832 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Release 3.1.3 [pulsar-site]
gaoran10 opened a new pull request, #832: URL: https://github.com/apache/pulsar-site/pull/832 ### Motivation Doc release for Pulsar 3.1.3. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve] WIP: PIP-342: OTel client metrics support [pulsar]
merlimat commented on code in PR #22179: URL: https://github.com/apache/pulsar/pull/22179#discussion_r1518210597 ## pulsar-client/src/main/java/org/apache/pulsar/client/impl/BinaryProtoLookupService.java: ## @@ -84,6 +92,21 @@ public BinaryProtoLookupService(PulsarClientImpl client, this.serviceNameResolver = new PulsarServiceNameResolver(); this.listenerName = listenerName; updateServiceUrl(serviceUrl); + +Attributes attrs = Attributes.of(AttributeKey.stringKey("transport-type"), "binary"); + +histoGetBroker = client.instrumentProvider().newLatencyHistogram("pulsar.client.lookup", +"Lookup operations", +attrs.toBuilder().put("type", "topic").build()); Review Comment: These are all kinds of "lookups" done by client. User doesn't necessarily need to differentiate them, though we have that possibility through the different attributes. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve] WIP: PIP-342: OTel client metrics support [pulsar]
merlimat commented on code in PR #22179: URL: https://github.com/apache/pulsar/pull/22179#discussion_r1518209219 ## pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java: ## @@ -258,11 +270,17 @@ public ClientCnx(ClientConfigurationData conf, EventLoopGroup eventLoopGroup, in this.idleState = new ClientCnxIdleState(this); this.clientVersion = "Pulsar-Java-v" + PulsarVersion.getVersion() + (conf.getDescription() == null ? "" : ("-" + conf.getDescription())); +this.connectionsOpenedCounter = instrumentProvider.newCounter("pulsar.client.connections.opened", Unit.Connections, Review Comment: Fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve] WIP: PIP-342: OTel client metrics support [pulsar]
merlimat commented on code in PR #22179: URL: https://github.com/apache/pulsar/pull/22179#discussion_r1518195267 ## pulsar-broker/src/test/java/org/apache/pulsar/client/metrics/ClientMetricsTest.java: ## @@ -0,0 +1,333 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.pulsar.client.metrics; + +import static org.testng.Assert.assertEquals; +import static org.testng.Assert.assertNotNull; +import static org.testng.Assert.assertTrue; +import static org.testng.Assert.fail; +import io.opentelemetry.api.OpenTelemetry; +import io.opentelemetry.api.common.Attributes; +import io.opentelemetry.sdk.OpenTelemetrySdk; +import io.opentelemetry.sdk.metrics.SdkMeterProvider; +import io.opentelemetry.sdk.metrics.data.MetricData; +import io.opentelemetry.sdk.metrics.data.MetricDataType; +import io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader; +import java.util.Map; +import java.util.TreeMap; +import java.util.concurrent.TimeUnit; +import lombok.Cleanup; +import org.apache.pulsar.client.api.Consumer; +import org.apache.pulsar.client.api.Message; +import org.apache.pulsar.client.api.Producer; +import org.apache.pulsar.client.api.ProducerConsumerBase; +import org.apache.pulsar.client.api.PulsarClient; +import org.apache.pulsar.client.api.Schema; +import org.apache.pulsar.client.api.SubscriptionType; +import org.testng.annotations.AfterMethod; +import org.testng.annotations.BeforeMethod; +import org.testng.annotations.Test; + +@Test(groups = "broker-api") +public class ClientMetricsTest extends ProducerConsumerBase { + +InMemoryMetricReader reader; +OpenTelemetry otel; + +@BeforeMethod +@Override +protected void setup() throws Exception { +super.internalSetup(); +super.producerBaseSetup(); + +this.reader = InMemoryMetricReader.create(); +SdkMeterProvider sdkMeterProvider = SdkMeterProvider.builder() +.registerMetricReader(reader) +.build(); +this.otel = OpenTelemetrySdk.builder().setMeterProvider(sdkMeterProvider).build(); +} + +@AfterMethod(alwaysRun = true) +@Override +protected void cleanup() throws Exception { +super.internalCleanup(); +} + +private Map collectMetrics() { +Map metrics = new TreeMap<>(); +for (MetricData md : reader.collectAllMetrics()) { +metrics.put(md.getName(), md); +} +return metrics; +} + +private void assertCounterValue(Map metrics, String name, long expectedValue, +Attributes expectedAttributes) { +assertEquals(getCounterValue(metrics, name, expectedAttributes), expectedValue); +} + +private long getCounterValue(Map metrics, String name, +Attributes expectedAttributes) { +MetricData md = metrics.get(name); +assertNotNull(md, "metric not found: " + name); +assertEquals(md.getType(), MetricDataType.LONG_SUM); + +for (var ex : md.getLongSumData().getPoints()) { +if (ex.getAttributes().equals(expectedAttributes)) { +return ex.getValue(); +} +} + +fail("metric attributes not found: " + expectedAttributes); +return -1; +} + +private void assertHistoCountValue(Map metrics, String name, long expectedCount, + Attributes expectedAttributes) { +assertEquals(getHistoCountValue(metrics, name, expectedAttributes), expectedCount); +} + +private long getHistoCountValue(Map metrics, String name, +Attributes expectedAttributes) { +MetricData md = metrics.get(name); +assertNotNull(md, "metric not found: " + name); +assertEquals(md.getType(), MetricDataType.HISTOGRAM); + +for (var ex : md.getHistogramData().getPoints()) { +if (ex.getAttributes().equals(expectedAttributes)) { +return ex.getCount(); +} +} + +fail("metric attributes not found: " + expectedAttributes); +return -1; +} + +@Test +public void testProducerMetrics() throws Exception { +
(pulsar-site) branch main updated: Update release-process.md
This is an automated email from the ASF dual-hosted git repository. heesung pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 23ede3255a38 Update release-process.md 23ede3255a38 is described below commit 23ede3255a38b6638259d5e4953e14a05efd7fa6 Author: Heesung Sohn <103456639+heesung...@users.noreply.github.com> AuthorDate: Fri Mar 8 11:08:39 2024 -0800 Update release-process.md run `cd $PULSAR_PATH` to clarify the command run location. --- contribute/release-process.md | 15 --- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/contribute/release-process.md b/contribute/release-process.md index ba4080c8b497..9cd87353873e 100644 --- a/contribute/release-process.md +++ b/contribute/release-process.md @@ -58,6 +58,7 @@ It is recommended to create a fresh clone of the repository to avoid any local f ```shell git clone g...@github.com:apache/pulsar.git cd pulsar +PULSAR_PATH=$(pwd) git checkout -b branch-2.X origin/master ``` @@ -133,6 +134,7 @@ In this process, the maven version of the project will always be the final one. ```shell # Bump to the release version +cd $PULSAR_PATH ./src/set-project-version.sh 2.X.0 # Some version may not update the right parent version of `protobuf-shaded/pom.xml`, please double check it. @@ -162,6 +164,7 @@ For patch releases, the tag is like `2.3.1`. Run the following command to build the artifacts: ```shell +cd $PULSAR_PATH mvn clean install -DskipTests ``` @@ -184,12 +187,9 @@ The _apache-pulsar-shell_ artifacts are distributed beginning with release 2.11. First, check that the `LICENSE` and `NOTICE` files cover all included jars for the bin package. You can use script to cross-validate `LICENSE` file with included jars: ```shell +cd $PULSAR_PATH src/check-binary-license.sh distribution/server/target/apache-pulsar-2.X.0-bin.tar.gz -``` - -Then, run Apache RAT to verify the license headers in the src package: - -```shell +# Then, run Apache RAT to verify the license headers in the src package: tar -xvzf target/apache-pulsar-2.X.0-src.tar.gz cd apache-pulsar-2.X.0-src mvn apache-rat:check @@ -285,7 +285,7 @@ However, it is simpler to do the release on a Linux arm64 / x86_64 VM directly. Run the following commands on a Linux machine (or with Mac where DOCKER_HOST points to a Linux amd64/Intel machine): ```shell -cd $PULSAR_HOME/docker +cd $PULSAR_PATH/docker ./build.sh DOCKER_USER= DOCKER_PASSWORD= DOCKER_ORG= ./publish.sh ``` @@ -294,7 +294,8 @@ DOCKER_USER= DOCKER_PASSWORD= DOCKER_ORG= DOCKER_ORG= docker login $DOCKER_ORG -u $DOCKER_USER -p
(pulsar-site) branch main updated: Update release-process.md
This is an automated email from the ASF dual-hosted git repository. heesung pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new da5afff3793d Update release-process.md da5afff3793d is described below commit da5afff3793d19ec619cef8b5b2f8516b4f00d93 Author: Heesung Sohn <103456639+heesung...@users.noreply.github.com> AuthorDate: Fri Mar 8 10:36:56 2024 -0800 Update release-process.md make sure `cd $PULSAR_PATH` when signing --- contribute/release-process.md | 1 + 1 file changed, 1 insertion(+) diff --git a/contribute/release-process.md b/contribute/release-process.md index 31518294f6d3..ba4080c8b497 100644 --- a/contribute/release-process.md +++ b/contribute/release-process.md @@ -212,6 +212,7 @@ default-key ... where `` should be replaced with the private key fingerprint for the `@apache.org` key. The key fingerprint can be found in `gpg -K` output. ```shell +cd $PULSAR_PATH # '-candidate-1' needs to be incremented in case of multiple iteration in getting #to the final release) RCVERSION=2.11.4-candidate-1
Re: [D] Golang Client: Async producer does not send messages [pulsar]
GitHub user merlimat added a comment to the discussion: Golang Client: Async producer does not send messages To add on the previous responses. `SendAsync()` will take a callback function that will be invoked when the send operation is completed, so you can track it. ```go // SendAsync a message in asynchronous mode // This call is blocked when the `maxPendingMessages` becomes full (default: 1000) // The callback will report back the message being published and // the eventual error in publishing SendAsync(context.Context, *ProducerMessage, func(MessageID, *ProducerMessage, error)) ``` Additionally, if you want to just send a bunch of messages and then wait for completion, you can use `Flush()`: ```go for i := 0; i < 100; i++ { producer.SendAsync(context.Background(), { Payload: []byte(fmt.Sprintf("hello-%d", i)), }, nil) } // Wait for all messages to be published err := producer.Flush() ``` GitHub link: https://github.com/apache/pulsar/discussions/4#discussioncomment-8723397 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@pulsar.apache.org
svn commit: r67808 [2/2] - in /dev/pulsar/pulsar-3.0.3-candidate-1: ./ connectors/
Added: dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-jdbc-sqlite-3.0.3.nar == Binary file - no diff available. Propchange: dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-jdbc-sqlite-3.0.3.nar -- svn:mime-type = application/octet-stream Added: dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-jdbc-sqlite-3.0.3.nar.asc == --- dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-jdbc-sqlite-3.0.3.nar.asc (added) +++ dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-jdbc-sqlite-3.0.3.nar.asc Fri Mar 8 18:25:14 2024 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEZKirnh4n0JGoBG6h8U9B0mVWZKAFAmXrU3QACgkQ8U9B0mVW +ZKCg8g/+MlRpqOYAWG3llYDpmdYLZfUdf+BGSpwOgZPm0wKzd9YxGNu/np5d+SGO +Q5VvqWVlWpn/y5xJ3OFtjP3+7UqZg3cnk3VdAFxEuzjnRdXRm+4z8dkNYz/Bfet/ +VPZxVOa+YcpFhgBcE7ESlpusQTdcd4OHBfhB3lEgLv4JartNhdU6IZsFjjUsWryj +m1OFqdOG/ddb4wlpN7Xc/yLYyXID7yuQK0jVOHcukCowKzI1Y6q2fmwO+90NUAIc +vSdN9+1bjnkphH4AsETAAd8NikhJp0Qc400kg4cOAP6c47nN7bXOFCD/4jfgc1Hc +kSse4vM4J3GuAVYFZ1lPydgEiEc/Y+Y08ILCnbSJKc/z3VfRnH+BFXABSS6R6lOd ++ZhFjfPbHsDf3fFIbmismdfh5XFNekzRxxTTVky4pFCD64PPeYv5ruasoFX+Tmc7 +GdxC/Q/eo8gN9sH0t50gtnYYg6gju/YOkkH4lhloXtmROWJPAwCvX3MGRq03FZsC +LeZXkhnche4qjwKmm0ohWIntGuUcFye11db+wRLP3idi6cy+3sOr0RoWytK5ae0W +ktXVaDzczU2uhQyd05HcfApKY1ctwW6N80ySDDzDje73Pey31hzyP/sUWeNqWgep +hB+KWci/RJ+IVgWRHUCA7D9B3ILDNPhVkJeSXqDGuZ7unJWz1TY= +=jEFM +-END PGP SIGNATURE- Added: dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-jdbc-sqlite-3.0.3.nar.sha512 == --- dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-jdbc-sqlite-3.0.3.nar.sha512 (added) +++ dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-jdbc-sqlite-3.0.3.nar.sha512 Fri Mar 8 18:25:14 2024 @@ -0,0 +1 @@ +8152bc56c1ac485898f2f831d5f7e9c731515ec89564ed622ca637f439f845a02e85bb9caaa38c35b43bf216c349daaf69fa4b64e51098db1ca352b04377f0e1 ./connectors/pulsar-io-jdbc-sqlite-3.0.3.nar Added: dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-kafka-3.0.3.nar == Binary file - no diff available. Propchange: dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-kafka-3.0.3.nar -- svn:mime-type = application/octet-stream Added: dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-kafka-3.0.3.nar.asc == --- dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-kafka-3.0.3.nar.asc (added) +++ dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-kafka-3.0.3.nar.asc Fri Mar 8 18:25:14 2024 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEZKirnh4n0JGoBG6h8U9B0mVWZKAFAmXrU2wACgkQ8U9B0mVW +ZKCj2w/9Gcx5Fy+hGARgTmTN+spRSwCo/40OLeRw+/vC7IpDZJ6jY4qc7CSnLsqL +ZXijJUStX2WCRkR94/3KRhHjwUcsDu7uqxs/LF5lDtYgY6EVYlGVepSJLFMQwe3b +DsfmQe3tWz9ZC44wxaK4xM3XPUnGAQcmSQ2vqSkyI8l9tMSMdwk9Cml7pVuJAn/E +GwEgdD7zTxKCZZFkaXhkfZu9u3aP9UIkFrZJVxZuqrmAcYZDkJO8f3WhdovW8Oz5 +rtHTmChsSPimzaaVmZKGHzXwui+KUjKKW4ybGqOqIp1hzmoHWAtqWz1afpGUuJJT +oTB6IvLri3XDjUP52uvNorDV5nEHyd3Riv+rTOIIN9iG13lOnkd0JjIbrE10/Ghm +HuWpu1EKNW4mxDUBmxBUERAbeu3SGtN0ehpJnNSdnyPeyezrTLnZD3vG/3DomZbu +OFKvVd8uOzVMQI47mD+PEpKldVDgtZQPS3HZk04xWV1A/qRqGFgb6zA8L4/80d1u +mq2GckQmUFnvLoAwaslVz6mQEmlwTVpmsBmpmu4DVGPoZYY0AuO7ukGgs2ZfrSn2 +D+i9XhhA97CYh6vGTDNvWAQW6qcrMExUIQqSiBclBjQlYLUtDDAJbboxyDnv1YS7 +L9rejiFFX4CzM7Mc4mX7+1Z+oeQF+VWXG3WWczCHloZVeXoUKnQ= +=8E/7 +-END PGP SIGNATURE- Added: dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-kafka-3.0.3.nar.sha512 == --- dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-kafka-3.0.3.nar.sha512 (added) +++ dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-kafka-3.0.3.nar.sha512 Fri Mar 8 18:25:14 2024 @@ -0,0 +1 @@ +8cef329b2bf92c0c0d2cc61104755bbfb7118a6dbd74c19e63e41ef4cde0b4bbd000a77b9a9b541fba32b25985da2bd839c85a73c791c796f30f73d046796a93 ./connectors/pulsar-io-kafka-3.0.3.nar Added: dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-kafka-connect-adaptor-3.0.3.nar == Binary file - no diff available. Propchange: dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-kafka-connect-adaptor-3.0.3.nar -- svn:mime-type = application/octet-stream Added: dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-kafka-connect-adaptor-3.0.3.nar.asc
svn commit: r67808 [1/2] - in /dev/pulsar/pulsar-3.0.3-candidate-1: ./ connectors/
Author: heesung Date: Fri Mar 8 18:25:14 2024 New Revision: 67808 Log: Staging artifacts and signature for Pulsar release $RCVERSION Added: dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-3.0.3-bin.tar.gz (with props) dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-3.0.3-bin.tar.gz.asc dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-3.0.3-bin.tar.gz.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-3.0.3-src.tar.gz (with props) dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-3.0.3-src.tar.gz.asc dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-3.0.3-src.tar.gz.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-offloaders-3.0.3-bin.tar.gz (with props) dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-offloaders-3.0.3-bin.tar.gz.asc dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-offloaders-3.0.3-bin.tar.gz.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-shell-3.0.3-bin.tar.gz (with props) dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-shell-3.0.3-bin.tar.gz.asc dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-shell-3.0.3-bin.tar.gz.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-shell-3.0.3-bin.zip (with props) dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-shell-3.0.3-bin.zip.asc dev/pulsar/pulsar-3.0.3-candidate-1/apache-pulsar-shell-3.0.3-bin.zip.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/ dev/pulsar/pulsar-3.0.3-candidate-1/connectors/LICENSE dev/pulsar/pulsar-3.0.3-candidate-1/connectors/README dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-aerospike-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-aerospike-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-aerospike-3.0.3.nar.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-alluxio-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-alluxio-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-alluxio-3.0.3.nar.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-batch-data-generator-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-batch-data-generator-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-batch-data-generator-3.0.3.nar.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-canal-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-canal-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-canal-3.0.3.nar.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-cassandra-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-cassandra-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-cassandra-3.0.3.nar.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-data-generator-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-data-generator-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-data-generator-3.0.3.nar.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-mongodb-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-mongodb-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-mongodb-3.0.3.nar.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-mssql-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-mssql-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-mssql-3.0.3.nar.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-mysql-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-mysql-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-mysql-3.0.3.nar.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-oracle-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-oracle-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-oracle-3.0.3.nar.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-postgres-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-postgres-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-debezium-postgres-3.0.3.nar.sha512 dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-dynamodb-3.0.3.nar (with props) dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-dynamodb-3.0.3.nar.asc dev/pulsar/pulsar-3.0.3-candidate-1/connectors/pulsar-io-dynamodb-3.0.3.nar.sha512
Re: [PR] [improve][cli] Remove cli unnecessary code [pulsar]
codecov-commenter commented on PR #22161: URL: https://github.com/apache/pulsar/pull/22161#issuecomment-1986168015 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/22161?src=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report All modified and coverable lines are covered by tests :white_check_mark: > Project coverage is 73.59%. Comparing base [(`bbc6224`)](https://app.codecov.io/gh/apache/pulsar/commit/bbc62245c5ddba1de4b1e7cee4ab49334bc36277?el=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) to head [(`2af1699`)](https://app.codecov.io/gh/apache/pulsar/pull/22161?src=pr=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). > Report is 27 commits behind head on master. Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/apache/pulsar/pull/22161/graphs/tree.svg?width=650=150=pr=acYqCpsK9J_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache)](https://app.codecov.io/gh/apache/pulsar/pull/22161?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) ```diff @@ Coverage Diff @@ ## master #22161 +/- ## + Coverage 73.57% 73.59% +0.01% - Complexity3262432633 +9 Files 1877 1877 Lines139502 139548 +46 Branches 1529915314 +15 + Hits 102638 102696 +58 + Misses2890828893 -15 - Partials 7956 7959 +3 ``` | [Flag](https://app.codecov.io/gh/apache/pulsar/pull/22161/flags?src=pr=flags_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Coverage Δ | | |---|---|---| | [inttests](https://app.codecov.io/gh/apache/pulsar/pull/22161/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `26.71% <ø> (+2.12%)` | :arrow_up: | | [systests](https://app.codecov.io/gh/apache/pulsar/pull/22161/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `24.39% <ø> (+0.06%)` | :arrow_up: | | [unittests](https://app.codecov.io/gh/apache/pulsar/pull/22161/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `72.84% <ø> (+<0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more. [see 82 files with indirect coverage changes](https://app.codecov.io/gh/apache/pulsar/pull/22161/indirect-changes?src=pr=tree-more_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
svn commit: r67806 - /dev/pulsar/pulsar-3.0.3-candidate-1/
Author: heesung Date: Fri Mar 8 18:05:15 2024 New Revision: 67806 Log: Add directory for pulsar 3.0.3-candidate-1 release Added: dev/pulsar/pulsar-3.0.3-candidate-1/
svn commit: r67805 - /dev/pulsar/pulsar-3.0.3-candidate-2/
Author: heesung Date: Fri Mar 8 18:03:45 2024 New Revision: 67805 Log: (empty) Removed: dev/pulsar/pulsar-3.0.3-candidate-2/
svn commit: r67804 - /dev/pulsar/pulsar-3.0.3-candidate-2/
Author: heesung Date: Fri Mar 8 18:01:16 2024 New Revision: 67804 Log: Add directory for pulsar 3.0.3-candidate-2 release Added: dev/pulsar/pulsar-3.0.3-candidate-2/
Re: [I] [Doc] pulsar-io-netty-3.2.1.nar appears twice on the download page [pulsar]
jashnGoyl commented on issue #9: URL: https://github.com/apache/pulsar/issues/9#issuecomment-1986121595 Hey! can I take up this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) annotated tag v3.1.3 updated (dcaf508f8e3 -> 1e5fba90444)
This is an automated email from the ASF dual-hosted git repository. rgao pushed a change to annotated tag v3.1.3 in repository https://gitbox.apache.org/repos/asf/pulsar.git *** WARNING: tag v3.1.3 was modified! *** from dcaf508f8e3 (commit) to 1e5fba90444 (tag) tagging dcaf508f8e381107125e98722f4ddab76f9303ad (commit) replaces v3.1.3-candidate-1 by rangao on Sat Mar 9 01:31:56 2024 +0800 - Log - Release v3.1.3 -BEGIN PGP SIGNATURE- iQJEBAABCgAuFiEEtWezB1n0EhojFQBl2aZsamxtvOoFAmXrS4wQHHJnYW9AYXBh Y2hlLm9yZwAKCRDZpmxqbG286s+rD/0QN0wkgvz13qiBNMMfMZ0y0IHK20cPBCqh MJLGURPqNxhEiBg06RpW3NI5dgE9tSPenyleKPC+CJMl+togqhczfngMQcEn4RIY 2DNeu6Hh4mtlXctz3m+U+J5Tj3JPvXaCq85KqnN4oYRF2vP9wsHamkXG0/NFDkFR 26ZrLy3Qtw1A2d0osEvfgGEY6UlMGVWiQ8eqo4Gr67XIkYbO8CdGaU62uw5xQW/0 hLUsla749wGvl2KEHPxgTXn5h5P4XnaBrRQ8+BUbpv7qzLNkGaPWfIV9xifhaw7g RI981UswfHBxsM+zUyHuXJBQuBoCpSL1KxhpZxFgMnjfmLKL1/1jTzqiBt3opnia WtgjMnG/nDF/CSFa1XJvtvoZrvzSFxH/tFik+SrpZwNiUMruG2kzvjQy+exRzEar xy37bIJqdnI/Nx7RuLB/sbxCD8wAqD6DWh9FJYAxlMZw6cd9ZAyVimRXEiTeJc+2 dF9kblhKxVVKe4Y07KjYI5qE/6DGXgrMax2tBQY9FL6E/KTbyRuvK4rtLJlFcdsp ogMWfsjSZswHc+oeIE2Rc7twDo2RWV77LxzMT2RkrCF1aCsjap3CysObnu22Dtmm iRvGN0Ij4CUtdxMqoXltkSEnBPPVFVhWe5PWLHLSFQQnMpQ8C1oHPu4MWb/ctbTY TEuvqhBKOw== =Smph -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
(pulsar) annotated tag v3.1.3 deleted (was 26defff1764)
This is an automated email from the ASF dual-hosted git repository. rgao pushed a change to annotated tag v3.1.3 in repository https://gitbox.apache.org/repos/asf/pulsar.git *** WARNING: tag v3.1.3 was deleted! *** tag was 26defff1764 The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
Re: [D] Golang Client: Async producer does not send messages [pulsar]
GitHub user jak78 added a comment to the discussion: Golang Client: Async producer does not send messages Hey @paulm17 welcome to the community! Since sendAsync is asynchronous, your program might exit before the message has a chance to be sent. Before exiting the program, you might need to wait for the message to be sent. Using `WaitGroups` provided by the `sync` package might help. This is the solution implemented by the test code you shared. - disclaimer: I'm not a golang expert ☺️. Just doing my best to help. GitHub link: https://github.com/apache/pulsar/discussions/4#discussioncomment-8722601 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@pulsar.apache.org
(pulsar) annotated tag v3.1.3 updated (dcaf508f8e3 -> 26defff1764)
This is an automated email from the ASF dual-hosted git repository. rgao pushed a change to annotated tag v3.1.3 in repository https://gitbox.apache.org/repos/asf/pulsar.git *** WARNING: tag v3.1.3 was modified! *** from dcaf508f8e3 (commit) to 26defff1764 (tag) tagging dcaf508f8e381107125e98722f4ddab76f9303ad (commit) replaces v3.1.3-candidate-1 by rangao on Sat Mar 9 01:11:36 2024 +0800 - Log - Release v3.1.3 -BEGIN PGP SIGNATURE- iQJEBAABCgAuFiEEtWezB1n0EhojFQBl2aZsamxtvOoFAmXrRsgQHHJnYW9AYXBh Y2hlLm9yZwAKCRDZpmxqbG286o5GD/0XAoSaaMHuilwf/Xyeia0lsNF2bt57gR6y FFmNquQDTN9E7KI8jBk07JHfxthoUeUFlKkYblVBnVKPDDnDsV+y7FAI6GeceVs3 bHB2bhAmgnh5ZlcMpEUjzWCYMdbtoPTjVYHIQu4dUxy/+voY4+dAUrHhKxufz8E3 qwfaHQH5eYubHJ2LMt/chGzuto3G/eXsAlK4Up3ZBF3MWhUpLYG2IP8VcstOJCcB rbzVwATRXg8Ht5huO9Y9sqahQ/LuMEPhbD3OdtHrKMG7Zv3RwP4IoJoXwjZVp+sN 8tUC7hhgLrxcL6tqv4RFTPrxsvSevfiWbQmwTH3ya+RykKWoX3w+GfvY95uapoto qsJvlZ47yDMsGh3szEL/o3c/TBMzaSTjUUJkLdFVxafgOKZs2NuVWdY1rUuJ2uI0 UD/TKFlxKpb48B8xOd4yc7bgxKjXqcI5tw3McJ7OFcOgaubeMfZNpbHMtDLItKr/ 00V89jb9Az+3UiuT+vARa34dLbTI2w4ttbq2PdPvEzTivYbsJHvz8Mj6byFMS8Vq B0jrTRKJVimbfdXWOdP0DVCGW6duq3fuittuyU7fpQikGM46paCMjwf5UA5ivvQT XYrbUZ01ZYh1JnF8YEWeCbxc4mSOm8a0LrgavH9x3S/EqUvr23f+OnCthLzo/DkW ecaLTKpXnA== =AXij -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
Re: [I] [Bug] Not able to set default backlog-quota retention policy , default retention policy and default TTL [pulsar]
kunjbosamiaFalkonry commented on issue #6: URL: https://github.com/apache/pulsar/issues/6#issuecomment-1986060753 yep tried using helm chart version 3.3.0 which has a appVersion of 3.0.2 . Still giving the same result. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][cli] Remove cli unnecessary code [pulsar]
nodece commented on PR #22161: URL: https://github.com/apache/pulsar/pull/22161#issuecomment-1986042844 /pulsarbot rerun-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Doc] pulsar-io-netty-3.2.1.nar appears twice on the download page [pulsar]
sebbASF opened a new issue, #9: URL: https://github.com/apache/pulsar/issues/9 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### What issue do you find in Pulsar docs? The artifact pulsar-io-netty-3.2.1.nar appears twice on the download page with different descriptions. ### What is your suggestion? Combine the entries or remove one ### Any reference? https://pulsar.apache.org/download/ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) annotated tag v3.0.3-candidate-2 deleted (was 4aebd88cf43)
This is an automated email from the ASF dual-hosted git repository. heesung pushed a change to annotated tag v3.0.3-candidate-2 in repository https://gitbox.apache.org/repos/asf/pulsar.git *** WARNING: tag v3.0.3-candidate-2 was deleted! *** tag was 4aebd88cf43 The revisions that were on this annotated tag are still contained in other references; therefore, this change does not discard any commits from the repository.
Re: [D] Golang Client: Async producer does not send messages [pulsar]
GitHub user david-streamlio added a comment to the discussion: Golang Client: Async producer does not send messages By default, pulsar producers batch their messages together before sending them to the broker. The sending of the messages occurs only after a certain threshold in terms of message count, total message size, or time has been reached. Since you are sending just one message, my guess is that none of these thresholds are met. Can you try wrapping the sendAsync command in a for loop as well? GitHub link: https://github.com/apache/pulsar/discussions/4#discussioncomment-8722203 This is an automatically sent email for commits@pulsar.apache.org. To unsubscribe, please send an email to: commits-unsubscr...@pulsar.apache.org
Re: [PR] [improve] WIP: PIP-342: OTel client metrics support [pulsar]
merlimat commented on code in PR #22179: URL: https://github.com/apache/pulsar/pull/22179#discussion_r1517940520 ## pulsar-broker/src/test/java/org/apache/pulsar/client/metrics/ClientMetricsTest.java: ## @@ -0,0 +1,333 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ +package org.apache.pulsar.client.metrics; + +import static org.testng.Assert.assertEquals; +import static org.testng.Assert.assertNotNull; +import static org.testng.Assert.assertTrue; +import static org.testng.Assert.fail; +import io.opentelemetry.api.OpenTelemetry; +import io.opentelemetry.api.common.Attributes; +import io.opentelemetry.sdk.OpenTelemetrySdk; +import io.opentelemetry.sdk.metrics.SdkMeterProvider; +import io.opentelemetry.sdk.metrics.data.MetricData; +import io.opentelemetry.sdk.metrics.data.MetricDataType; +import io.opentelemetry.sdk.testing.exporter.InMemoryMetricReader; +import java.util.Map; +import java.util.TreeMap; +import java.util.concurrent.TimeUnit; +import lombok.Cleanup; +import org.apache.pulsar.client.api.Consumer; +import org.apache.pulsar.client.api.Message; +import org.apache.pulsar.client.api.Producer; +import org.apache.pulsar.client.api.ProducerConsumerBase; +import org.apache.pulsar.client.api.PulsarClient; +import org.apache.pulsar.client.api.Schema; +import org.apache.pulsar.client.api.SubscriptionType; +import org.testng.annotations.AfterMethod; +import org.testng.annotations.BeforeMethod; +import org.testng.annotations.Test; + +@Test(groups = "broker-api") +public class ClientMetricsTest extends ProducerConsumerBase { + +InMemoryMetricReader reader; +OpenTelemetry otel; + +@BeforeMethod +@Override +protected void setup() throws Exception { +super.internalSetup(); +super.producerBaseSetup(); + +this.reader = InMemoryMetricReader.create(); +SdkMeterProvider sdkMeterProvider = SdkMeterProvider.builder() +.registerMetricReader(reader) +.build(); +this.otel = OpenTelemetrySdk.builder().setMeterProvider(sdkMeterProvider).build(); +} + +@AfterMethod(alwaysRun = true) +@Override +protected void cleanup() throws Exception { +super.internalCleanup(); +} + +private Map collectMetrics() { +Map metrics = new TreeMap<>(); +for (MetricData md : reader.collectAllMetrics()) { +metrics.put(md.getName(), md); +} +return metrics; +} + +private void assertCounterValue(Map metrics, String name, long expectedValue, +Attributes expectedAttributes) { +assertEquals(getCounterValue(metrics, name, expectedAttributes), expectedValue); +} + +private long getCounterValue(Map metrics, String name, +Attributes expectedAttributes) { +MetricData md = metrics.get(name); +assertNotNull(md, "metric not found: " + name); +assertEquals(md.getType(), MetricDataType.LONG_SUM); + +for (var ex : md.getLongSumData().getPoints()) { +if (ex.getAttributes().equals(expectedAttributes)) { +return ex.getValue(); +} +} + +fail("metric attributes not found: " + expectedAttributes); +return -1; +} + +private void assertHistoCountValue(Map metrics, String name, long expectedCount, + Attributes expectedAttributes) { +assertEquals(getHistoCountValue(metrics, name, expectedAttributes), expectedCount); +} + +private long getHistoCountValue(Map metrics, String name, +Attributes expectedAttributes) { +MetricData md = metrics.get(name); +assertNotNull(md, "metric not found: " + name); +assertEquals(md.getType(), MetricDataType.HISTOGRAM); + +for (var ex : md.getHistogramData().getPoints()) { +if (ex.getAttributes().equals(expectedAttributes)) { +return ex.getCount(); +} +} + +fail("metric attributes not found: " + expectedAttributes); +return -1; +} + +@Test +public void testProducerMetrics() throws Exception { +
Re: [I] Redelivery Count not set leading to indefinite redelivery [pulsar-client-node]
fabianwikstrom commented on issue #368: URL: https://github.com/apache/pulsar-client-node/issues/368#issuecomment-1985970763 batchIndexAckEnabled was enabled on consumer but not on broker which led to this bug -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Redelivery Count not set leading to indefinite redelivery [pulsar-client-node]
fabianwikstrom closed issue #368: Redelivery Count not set leading to indefinite redelivery URL: https://github.com/apache/pulsar-client-node/issues/368 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] getRedeliveryCount always return 0 [pulsar-client-node]
fabianwikstrom commented on issue #296: URL: https://github.com/apache/pulsar-client-node/issues/296#issuecomment-1985969445 Hi @izumo27 . We figured it out. Turns out that we have `batchIndexAckEnabled` enabled on the consumer but not on the broker and that is what led to the redelivery loop. I'll close my other issue Thank you for the quick response! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-client-go) branch master updated: [Improve]Change base image to apachepulsar/pulsar (#1195)
This is an automated email from the ASF dual-hosted git repository. zike pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-client-go.git The following commit(s) were added to refs/heads/master by this push: new 4f213796 [Improve]Change base image to apachepulsar/pulsar (#1195) 4f213796 is described below commit 4f213796979cdf6ffb3477f595b25c4544d76811 Author: crossoverJie AuthorDate: Fri Mar 8 23:53:30 2024 +0800 [Improve]Change base image to apachepulsar/pulsar (#1195) ### Motivation There are related discussion records [here](https://github.com/apache/pulsar-client-go/pull/1037). It is recommended to switch the base image to `apachepulsar/pulsar`. ### Modifications Change base image to `apachepulsar/pulsar`. --- .github/workflows/ci.yml | 2 +- Dockerfile | 13 - Makefile | 3 +-- scripts/run-ci.sh| 3 ++- 4 files changed, 12 insertions(+), 9 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9ad0b3f2..7b770051 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -38,7 +38,7 @@ jobs: runs-on: ubuntu-latest strategy: matrix: -go-version: ['1.18', '1.19', '1.20', '1.21'] +go-version: ['1.18', '1.19', '1.20', '1.21.0'] steps: - uses: actions/checkout@v3 - name: clean docker cache diff --git a/Dockerfile b/Dockerfile index 1c137fc0..62d22bf0 100644 --- a/Dockerfile +++ b/Dockerfile @@ -18,14 +18,17 @@ # Explicit version of Pulsar and Golang images should be # set via the Makefile or CLI ARG PULSAR_IMAGE=apachepulsar/pulsar:latest -ARG GOLANG_IMAGE=golang:latest +FROM $PULSAR_IMAGE +USER root +ARG GO_VERSION=1.18 -FROM $PULSAR_IMAGE as pulsar -FROM $GOLANG_IMAGE +RUN curl -L https://dl.google.com/go/go${GO_VERSION}.linux-amd64.tar.gz -o golang.tar.gz && \ +mkdir -p /pulsar/go && tar -C /pulsar -xzf golang.tar.gz -RUN apt-get update && apt-get install -y openjdk-17-jre ca-certificates +ENV PATH /pulsar/go/bin:$PATH + +RUN apt-get update && apt-get install -y git && apt-get install -y gcc -COPY --from=pulsar /pulsar /pulsar ### Add pulsar config COPY integration-tests/certs /pulsar/certs diff --git a/Makefile b/Makefile index d0442372..527d53ac 100644 --- a/Makefile +++ b/Makefile @@ -21,7 +21,6 @@ IMAGE_NAME = pulsar-client-go-test:latest PULSAR_VERSION ?= 3.2.0 PULSAR_IMAGE = apachepulsar/pulsar:$(PULSAR_VERSION) GO_VERSION ?= 1.18 -GOLANG_IMAGE = golang:$(GO_VERSION) # Golang standard bin directory. GOPATH ?= $(shell go env GOPATH) @@ -39,7 +38,7 @@ bin/golangci-lint: GOBIN=$(shell pwd)/bin go install github.com/golangci/golangci-lint/cmd/golangci-lint@v1.51.2 container: - docker build -t ${IMAGE_NAME} --build-arg GOLANG_IMAGE="${GOLANG_IMAGE}" \ + docker build -t ${IMAGE_NAME} --build-arg GO_VERSION="${GO_VERSION}" \ --build-arg PULSAR_IMAGE="${PULSAR_IMAGE}" . test: container diff --git a/scripts/run-ci.sh b/scripts/run-ci.sh index cc4f6a1e..83246a39 100755 --- a/scripts/run-ci.sh +++ b/scripts/run-ci.sh @@ -19,7 +19,8 @@ set -e -x -export GOPATH=/ +export GOPATH=/pulsar/go +export GOCACHE=/tmp/go-cache # Install dependencies go mod download
Re: [PR] [Improve]Change base image to apachepulsar/pulsar [pulsar-client-go]
RobertIndie merged PR #1195: URL: https://github.com/apache/pulsar-client-go/pull/1195 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-client-go) branch master updated: [Improve] getMessagesById gets all messages (#1194)
This is an automated email from the ASF dual-hosted git repository. zike pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-client-go.git The following commit(s) were added to refs/heads/master by this push: new 143fa232 [Improve] getMessagesById gets all messages (#1194) 143fa232 is described below commit 143fa23228621414784cefae347286e797e4ef84 Author: crossoverJie AuthorDate: Fri Mar 8 23:52:51 2024 +0800 [Improve] getMessagesById gets all messages (#1194) ### Motivation To keep consistent with the Java client. Releted PR: https://github.com/apache/pulsar/pull/21918 ### Modifications Add `getMessagesById` interface. - Co-authored-by: Zike Yang --- pulsaradmin/pkg/admin/subscription.go | 23 +++-- pulsaradmin/pkg/admin/subscription_test.go | 130 + 2 files changed, 147 insertions(+), 6 deletions(-) diff --git a/pulsaradmin/pkg/admin/subscription.go b/pulsaradmin/pkg/admin/subscription.go index 456de46c..8ddb5845 100644 --- a/pulsaradmin/pkg/admin/subscription.go +++ b/pulsaradmin/pkg/admin/subscription.go @@ -72,8 +72,11 @@ type Subscriptions interface { // PeekMessages peeks messages from a topic subscription PeekMessages(utils.TopicName, string, int) ([]*utils.Message, error) - // GetMessageByID gets message by its ledgerID and entryID + // Deprecated: Use GetMessagesByID() instead GetMessageByID(topic utils.TopicName, ledgerID, entryID int64) (*utils.Message, error) + + // GetMessagesByID gets messages by its ledgerID and entryID + GetMessagesByID(topic utils.TopicName, ledgerID, entryID int64) ([]*utils.Message, error) } type subscriptions struct { @@ -187,6 +190,18 @@ func (s *subscriptions) peekNthMessage(topic utils.TopicName, sName string, pos } func (s *subscriptions) GetMessageByID(topic utils.TopicName, ledgerID, entryID int64) (*utils.Message, error) { + messages, err := s.GetMessagesByID(topic, ledgerID, entryID) + if err != nil { + return nil, err + } + + if len(messages) == 0 { + return nil, nil + } + return messages[0], nil +} + +func (s *subscriptions) GetMessagesByID(topic utils.TopicName, ledgerID, entryID int64) ([]*utils.Message, error) { ledgerIDStr := strconv.FormatInt(ledgerID, 10) entryIDStr := strconv.FormatInt(entryID, 10) @@ -201,11 +216,7 @@ func (s *subscriptions) GetMessageByID(topic utils.TopicName, ledgerID, entryID if err != nil { return nil, err } - - if len(messages) == 0 { - return nil, nil - } - return messages[0], nil + return messages, nil } // safeRespClose is used to close a response body diff --git a/pulsaradmin/pkg/admin/subscription_test.go b/pulsaradmin/pkg/admin/subscription_test.go new file mode 100644 index ..c4ba717d --- /dev/null +++ b/pulsaradmin/pkg/admin/subscription_test.go @@ -0,0 +1,130 @@ +// Licensed to the Apache Software Foundation (ASF) under one +// or more contributor license agreements. See the NOTICE file +// distributed with this work for additional information +// regarding copyright ownership. The ASF licenses this file +// to you under the Apache License, Version 2.0 (the +// "License"); you may not use this file except in compliance +// with the License. You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package admin + +import ( + "context" + "fmt" + "sync" + "testing" + "time" + + "github.com/apache/pulsar-client-go/pulsar" + "github.com/apache/pulsar-client-go/pulsaradmin/pkg/admin/config" + "github.com/apache/pulsar-client-go/pulsaradmin/pkg/utils" + "github.com/stretchr/testify/assert" +) + +func TestGetMessagesByID(t *testing.T) { + + randomName := newTopicName() + topic := "persistent://public/default/" + randomName + + cfg := {} + admin, err := New(cfg) + assert.NoError(t, err) + assert.NotNil(t, admin) + + ctx := context.Background() + + client, err := pulsar.NewClient(pulsar.ClientOptions{ + URL: lookupURL, + }) + + assert.Nil(t, err) + defer client.Close() + + // create producer + numberMessages := 10 + batchingMaxMessages := numberMessages / 2 + producer, err := client.CreateProducer(pulsar.ProducerOptions{ + Topic: topic, + DisableBatching: false, +
Re: [PR] [Improve] getMessagesById gets all messages [pulsar-client-go]
RobertIndie merged PR #1194: URL: https://github.com/apache/pulsar-client-go/pull/1194 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
nodece commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1517846686 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: >I think your logic is saying, if the JWKS file only contains one entry - ALWAYS use it to verify. Correct? Yes. > i feel we must support verifying JWTs with our existing, non-kid key AND future kid keys - at the same time. (which is supported by my logic). When your token without id, should I use first JWK to verify your token? When with id, I can find the JWK by your id. Or we need to add the default id to the pulsar config, when your id is null, the pulsar will use this default id to find your JWK. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
nodece commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1517846686 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: >I think your logic is saying, if the JWKS file only contains one entry - ALWAYS use it to verify. Correct? Yes. > i feel we must support verifying JWTs with our existing, non-kid key AND future kid keys - at the same time. (which is supported by my logic). When your token without id, should I use first JWK to verify your token? When with id, I can find the JWK by your id. Or we need to add the default id to the pulsar config. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
nodece commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1517846686 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: >I think your logic is saying, if the JWKS file only contains one entry - ALWAYS use it to verify. Correct? Yes. > i feel we must support verifying JWTs with our existing, non-kid key AND future kid keys - at the same time. (which is supported by my logic). When your token without id, should I use first JWK to verify your token? When with id, I can find the JWK by your id. ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: >I think your logic is saying, if the JWKS file only contains one entry - ALWAYS use it to verify. Correct? Yes. > i feel we must support verifying JWTs with our existing, non-kid key AND future kid keys - at the same time. (which is supported by my logic). When your token without id, should I use first JWK to verify your token? When with id, I can find the JWK by your id. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
nodece commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1517846686 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: When your token without id, should I use first JWK to verify your token? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
damienburke commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1517836936 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: I think your logic is saying, if the JWKS file only contains one entry - ALWAYS use it to verify. Correct? If so, that does seem reasonable. So i think its the following statement I need help with: > Once all client uses the token with an id, you can add more keys to your JWKS. Our existing JWT creation process is set in stone. We for various reasons cannot update it, and those "legacy" JWTs will never have a kid. Not sure, but feels with your logic, we would be unable to migrate to multiple key support - and i feel we must support verifying JWTs with our existing, non-kid key AND future kid keys - at the same time. (which is supported by my logic). Thanks for the collaboration @nodece -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
nodece commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1517839290 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: > In particular confused as to how kid2 would resolve to JWK1. Perhaps if u can explain why this happens? You only have one key, not two keys, so you only get the JW1 to verify the token with id kid2. This is a special case. This special case can support your requirement: > we must be able to support verifying JWTs with our existing, non-kid key AND future kid keys - at the same time. Does it support configuring two keys to meet this requirement? No. Please consider which key should be used to verify your token when your token ID is empty. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
nodece commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1517556832 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: > If the kid is null, and you only have one key in the JWKS file, we can use this key to verify your token. > the JWKS file would need to store both. Sorry, this kid should be your token id. You can only set up one key in the JWKS, which must have the kid. Regardless of whether your token contains an id, you can obtain the correct key, so like: JWKS hashmap(only have one key, not two key2): ``` kid1 -> JWK1 ``` | your token id | your JWKS | | --- | --- | | null | JWK1 | | kid1 | JWK1 | | kid2 | JWK1 | > So to support the migration use case we could introduce a new optional field into the JwkResolver This is more complex rather than my way. Once all client uses the token with an id, you can add more keys to your JWKS. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
damienburke commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1517796744 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: Thanks @nodece I dont think I am putting all the pieces together in your last comment. In your use case table, you are always resolving to JWK1. In particular confused as to how kid2 would resolve to JWK1. Perhaps if u can explain why this happens? I think what would really help, if you could state the migration use case(s) your suggested logic supports? Just to clarify our particular migration requirements - we must be able to support verifying JWTs with our existing, non-kid key AND future kid keys - at the same time. (which is supported by my logic). Not every migration might need to do this - but we have a large pulsar deployment and this "cross-over" is crucial, and the only path for us. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
damienburke commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1517836936 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: I think your logic is saying, if the JWKS file only contains one entry - ALWAYS use it to verify. Correct? If so, that does seem reasonable. So i think its the following statement I need help with: > Once all client uses the token with an id, you can add more keys to your JWKS. Our existing JWT creation process is set in stone. We for various reasons cannot update it, and those "legacy" JWTs will never have a kid. Hopefully that explains our use case / perspective a bit more. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Fix version issue [pulsar-site]
Technoboy- merged PR #831: URL: https://github.com/apache/pulsar-site/pull/831 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-site) branch main updated: Fix version issue (#831)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 71342672eb8f Fix version issue (#831) 71342672eb8f is described below commit 71342672eb8fed40f04c29428dac3bb632c2d750 Author: Jiwei Guo AuthorDate: Fri Mar 8 22:52:16 2024 +0800 Fix version issue (#831) --- data/release-pulsar.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/data/release-pulsar.js b/data/release-pulsar.js index a65514e07da9..6d9a6eed8e24 100644 --- a/data/release-pulsar.js +++ b/data/release-pulsar.js @@ -88,7 +88,7 @@ module.exports = [ "vtag": "2.11.x", "releaseNotes": "/release-notes/versioned/pulsar-2.11.3/", "doc": "/docs/2.11.x", - "version": "v2.11.x" + "version": "" }, { "author": "Technoboy-", @@ -134,7 +134,7 @@ module.exports = [ "vtag": "2.10.x", "releaseNotes": "/release-notes/versioned/pulsar-2.10.5/", "doc": "/docs/2.10.x", - "version": "v2.10.x" + "version": "" }, { "author": "liangyepianzhou",
[PR] Fix version issue [pulsar-site]
Technoboy- opened a new pull request, #831: URL: https://github.com/apache/pulsar-site/pull/831 ### Motivation ![image](https://github.com/apache/pulsar-site/assets/6297296/cfa16d0d-d3af-4670-9f99-3a432811cd82) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-site) branch fix-2-version updated: Fix version issue
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a commit to branch fix-2-version in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/fix-2-version by this push: new 3b7c54e4532a Fix version issue 3b7c54e4532a is described below commit 3b7c54e4532adfb21af429ffcab4663f85113302 Author: Jiwe Guo AuthorDate: Fri Mar 8 22:44:54 2024 +0800 Fix version issue --- data/release-pulsar.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data/release-pulsar.js b/data/release-pulsar.js index 00d5098ab57d..6d9a6eed8e24 100644 --- a/data/release-pulsar.js +++ b/data/release-pulsar.js @@ -88,7 +88,7 @@ module.exports = [ "vtag": "2.11.x", "releaseNotes": "/release-notes/versioned/pulsar-2.11.3/", "doc": "/docs/2.11.x", - "version": "v2.11.x" + "version": "" }, { "author": "Technoboy-",
(pulsar-site) branch fix-2-version created (now 748282e02790)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a change to branch fix-2-version in repository https://gitbox.apache.org/repos/asf/pulsar-site.git at 748282e02790 Fix version issue This branch includes the following new commits: new 748282e02790 Fix version issue The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(pulsar-site) 01/01: Fix version issue
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a commit to branch fix-2-version in repository https://gitbox.apache.org/repos/asf/pulsar-site.git commit 748282e027900f9c17b19765d91c34e921c9fe56 Author: Jiwe Guo AuthorDate: Fri Mar 8 22:44:18 2024 +0800 Fix version issue --- data/release-pulsar.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/data/release-pulsar.js b/data/release-pulsar.js index a65514e07da9..00d5098ab57d 100644 --- a/data/release-pulsar.js +++ b/data/release-pulsar.js @@ -134,7 +134,7 @@ module.exports = [ "vtag": "2.10.x", "releaseNotes": "/release-notes/versioned/pulsar-2.10.5/", "doc": "/docs/2.10.x", - "version": "v2.10.x" + "version": "" }, { "author": "liangyepianzhou",
(pulsar-site) branch main updated: Remove "update the site" that is misleading and add release note tips
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new e6e5534e39fd Remove "update the site" that is misleading and add release note tips e6e5534e39fd is described below commit e6e5534e39fd861e34d4e4410a706dcaa88f6195 Author: Lari Hotari AuthorDate: Fri Mar 8 16:41:52 2024 +0200 Remove "update the site" that is misleading and add release note tips --- contribute/release-note-guide.md | 22 - contribute/release-process.md| 41 2 files changed, 21 insertions(+), 42 deletions(-) diff --git a/contribute/release-note-guide.md b/contribute/release-note-guide.md index d5f1bfb31ac5..87ebc2fba57f 100644 --- a/contribute/release-note-guide.md +++ b/contribute/release-note-guide.md @@ -39,10 +39,30 @@ gh release view "v2.10.2" -R apache/pulsar --json author,tagName,publishedAt gh release view "v2.10.2" -R apache/pulsar --json body --jq .body ``` + +## Generate release notes + +There isn't a definite way yet. + +Here are 2 approaches: + +Using "git log" + +```bash +git log --reverse --oneline v2.11.3..v2.11.4 | colrm 1 12 | sed 's/\] \[/][/' | perl -p -e 's/^\s+//' | sort +``` + +Alternatively using "gh pr list" + +```bash +gh pr list -L 1000 --search "is:pr is:merged label:release/2.10.6 label:cherry-picked/branch-2.10" --json title,number,url | jq -r '.[] | "\(.title) [\(.number)](\(.url))"' +``` + ## Update the release note page 1. Copy the related release notes entries and add a [versioned release note file](https://github.com/apache/pulsar-site/tree/main/release-notes/versioned). -2. Update the [version metadata files](https://github.com/apache/pulsar-site/tree/main/data) (`release-*.js`). +2. Update the [version metadata files](https://github.com/apache/pulsar-site/tree/main/data) (`release-*.js`). For apache/pulsar releases, this means updating `release-java.js` (Java client) and `release-pulsar.js` (Pulsar). +3. For every apache/pulsar release, you should add a `` entry to the corresponding place in the `releases.json` file. To preview the result, follow the instructions for [previewing content](document-preview.md#preview-changes). diff --git a/contribute/release-process.md b/contribute/release-process.md index 9a9f74541261..31518294f6d3 100644 --- a/contribute/release-process.md +++ b/contribute/release-process.md @@ -599,47 +599,6 @@ Once the docs are generated, you can add them and submit them in a PR. The expec Read more on the manual of [pytools](https://github.com/apache/pulsar-site/tree/main/tools/pytools/README.md). -## Update the site - -Clone the apache/site repo: - -```shell -git clone g...@github.com:apache/pulsar-site.git -``` - -Create a new branch from the main branch: - -```shell -git checkout -b doc_release_ -``` - -For every release, you should add a `` entry to the corresponding place in the `releases.json` file. - -:::caution - -The following steps are for feature releases only. - -::: - -1. Generate a new version of the documentation. - -```shell -yarn install -yarn run version -``` - -After you run this command, a new folder `version-` is added in the `versioned_docs` directory, a new sidebar file `version--sidebars.json` is added in the `versioned_sidebars` directory, and the new version is added in the `versions.json` file. - -:::note - -You can move the latest version under the old version in the `versions.json` file. Make sure the Algolia index works before moving 2.X.0 as the current stable. - -::: - -2. Send out a PR request for review. After your PR is approved and merged to main, the website is published automatically after the new website is built. The website is built every 6 hours. - -3. Check the new website after the website is built. Open https://pulsar.apache.org in your browsers to verify all the changes are alive. If the website build succeeds but the website is not updated, you can try to sync the git repository. Navigate to https://selfserve.apache.org/ and click the "Synchronize Git Repositories" and then select apache/pulsar-site. - ## Announce the release Once the release artifacts are available in the Apache Mirrors and the website is updated, you need to announce the release. You should email to d...@pulsar.apache.org, us...@pulsar.apache.org, and annou...@apache.org. Here is a sample content:
(pulsar-site) branch main updated: Fix version issue (#830)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 7db8f58dc7de Fix version issue (#830) 7db8f58dc7de is described below commit 7db8f58dc7debbe801643948177936409969716f Author: Jiwei Guo AuthorDate: Fri Mar 8 22:38:17 2024 +0800 Fix version issue (#830) --- data/release-pulsar.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/data/release-pulsar.js b/data/release-pulsar.js index e797afa87372..a65514e07da9 100644 --- a/data/release-pulsar.js +++ b/data/release-pulsar.js @@ -6,7 +6,7 @@ module.exports = [ "vtag": "3.2.x", "releaseNotes": "/release-notes/versioned/pulsar-3.2.1/", "doc": "/docs/3.2.x", -"version": "" +"version": "v3.2.x" }, { "author": "Technoboy-", @@ -15,7 +15,7 @@ module.exports = [ "vtag": "3.2.x", "releaseNotes": "/release-notes/versioned/pulsar-3.2.0/", "doc": "/docs/3.2.x", -"version": "v3.2.x" +"version": "" }, { "author": "AnonHxy",
Re: [PR] Fix version issue [pulsar-site]
Technoboy- merged PR #830: URL: https://github.com/apache/pulsar-site/pull/830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] PIP-226: Add JWKS support for AuthenticationProviderToken [pulsar]
damienburke commented on code in PR #22215: URL: https://github.com/apache/pulsar/pull/22215#discussion_r1517796744 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/authentication/AuthenticationProviderToken.java: ## @@ -458,4 +485,64 @@ public String getHeader(String name) { return super.getHeader(name); } } + +@Slf4j +private static final class JwkResolver implements SigningKeyResolver { +@Setter +@Getter +private static class JwksData { +private List> keys; +} + +private final Map jwks; + +public JwkResolver(String configValue) { +try { +byte[] bytes = AuthTokenUtils.readKeyFromUrl(configValue); +ObjectMapper objectMapper = ObjectMapperFactory.create(); +JwksData data = objectMapper.reader().readValue(bytes, JwksData.class); +if (data == null || data.getKeys() == null || data.getKeys().isEmpty()) { +log.warn("No keys in " + data); +jwks = Collections.emptyMap(); +return; +} +jwks = new LinkedHashMap<>(); +data.getKeys().forEach((n) -> { +Jwk jwk = Jwk.fromValues(n); +jwks.put(jwk.getId(), jwk); +}); +if (log.isDebugEnabled()) { +log.info("jwks: {}", jwks); +} +} catch (IOException e) { +log.error("Failed to get jwks from {}", configValue, e); +throw new IllegalArgumentException(e); +} +} + +private Key get(String keyId) { Review Comment: Thanks @nodece I dont think I am putting all the pieces together in your last comment. In your use case table, you are always resolving to JWK1. In particular confused as to how kid2 would resolve to JWK1. Perhaps if u can explain why this happens? I think what would really help, if you could state the migration use case(s) your suggested logic supports? Just to clarify our particular migration requirements - we must be able to support verifying JWTs with our existing, non-kid key AND future kid keys - at the same time. (which is supported by my logic). Not every migration might need to do this - but we have a large pulsar deployment and this "cross-over" is crucial, and the only path for us. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) annotated tag v3.2.1 updated (da98b8d8d0b -> 2cf29e63597)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a change to annotated tag v3.2.1 in repository https://gitbox.apache.org/repos/asf/pulsar.git *** WARNING: tag v3.2.1 was modified! *** from da98b8d8d0b (commit) to 2cf29e63597 (tag) tagging da98b8d8d0bc0c2bdd2d14d3e2e8058fb4c5dfd2 (commit) replaces v3.2.1-candidate-1 by Jiwe Guo on Fri Mar 8 22:32:47 2024 +0800 - Log - Release v3.2.1 -BEGIN PGP SIGNATURE- iQJJBAABCAAzFiEEStnMRiSmOggeXFwiZb2iVNVk0t8FAmXrIY8VHHRlY2hub2Jv eUBhcGFjaGUub3JnAAoJEGW9olTVZNLfaLEQAIFbT/p0lgK4au57XcKrtrUuJ8q7 4FceGUWWZbutkdFUmVKeRhJByZXyT48wOxrHEvzp59Yf8torxEUO+OoH9FfjBwBw h0erF+WmoymlqvVSYN/26PAvoWe7+bSrXd/S1qtq9CeEgnCbL6ECD/hOPNT/Hbkx 7yUuLEgbXvURmiKylRxLPJj6RNU+7n5ImHOPrcz8NpOdEeGAhD/DauDNSoMuKWSa 7EbeFXo0UB1M8QaReTJeW0ppYLg5W0d/1p/WBz1mErng/cd+T/RyZD0O5kcClVpM iwao+Jx3tZSuTtIQzZcfg7w18Ue+e6mSdzDq8Wnywc9foroXvvL5Hxqim2mMtuPH bW6JuTLFjGydfxjjYXGzO3XKZejf8jMPgIVlh01BdYshYfvnDBIm7lmMFKmfLPok KuBo8VVGkigdaMNMRWGpwOdOZ9HPatyezxzXZdj8DSdx+o1TGPXy3G6OdJbEXP6/ nUopYLvwYN18KlAgLXIRgEBDYe493L6TKZ3S+prw8mWfDf07xf1PhtSnku8KVW0L bzTRgu33j3dPK+fyugN8hk5wCp6cu/yroUKkbZaiV8CYQ8mPjy53/pRu+oWTi2/E mbB2LC+K9AloM5IFhEJg9P5bVVxo/lHvXl4FojGO6qcy+J4S6UiI7tCwecNl7VfB H1ZwBLepZ9l0t4tf =2CXH -END PGP SIGNATURE- --- No new revisions were added by this update. Summary of changes:
(pulsar-site) branch main updated: Use correct header level for release date
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 5b995aa223bd Use correct header level for release date 5b995aa223bd is described below commit 5b995aa223bdb7e122b155854222651b27016c8f Author: Lari Hotari AuthorDate: Fri Mar 8 16:31:54 2024 +0200 Use correct header level for release date --- release-notes/versioned/pulsar-2.11.4.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/release-notes/versioned/pulsar-2.11.4.md b/release-notes/versioned/pulsar-2.11.4.md index d2bd610224b3..d73d9df2d8d4 100644 --- a/release-notes/versioned/pulsar-2.11.4.md +++ b/release-notes/versioned/pulsar-2.11.4.md @@ -4,7 +4,7 @@ title: Apache Pulsar 2.11.4 sidebar_label: Apache Pulsar 2.11.4 --- -## 2024-03-08 + 2024-03-08 ### Broker
[PR] Fix version issue [pulsar-site]
Technoboy- opened a new pull request, #830: URL: https://github.com/apache/pulsar-site/pull/830 ### Motivation ![image](https://github.com/apache/pulsar-site/assets/6297296/f3495c9c-a6dc-4f7b-8911-b427e22021f3) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-site) branch fix-doc-issue created (now 090fea719cec)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a change to branch fix-doc-issue in repository https://gitbox.apache.org/repos/asf/pulsar-site.git at 090fea719cec Fix version issue This branch includes the following new commits: new 090fea719cec Fix version issue The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(pulsar-site) 01/01: Fix version issue
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a commit to branch fix-doc-issue in repository https://gitbox.apache.org/repos/asf/pulsar-site.git commit 090fea719cec245fefe8fffa5d7191dd606f946c Author: Jiwe Guo AuthorDate: Fri Mar 8 22:29:50 2024 +0800 Fix version issue --- data/release-pulsar.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/data/release-pulsar.js b/data/release-pulsar.js index 8ae8c38a6d80..eaed41e9118a 100644 --- a/data/release-pulsar.js +++ b/data/release-pulsar.js @@ -6,7 +6,7 @@ module.exports = [ "vtag": "3.2.x", "releaseNotes": "/release-notes/versioned/pulsar-3.2.1/", "doc": "/docs/3.2.x", -"version": "" +"version": "v3.2.x" }, { "author": "Technoboy-", @@ -15,7 +15,7 @@ module.exports = [ "vtag": "3.2.x", "releaseNotes": "/release-notes/versioned/pulsar-3.2.0/", "doc": "/docs/3.2.x", -"version": "v3.2.x" +"version": "" }, { "author": "AnonHxy",
(pulsar-site) branch main updated: Add remaining updates for 2.10.5 and 2.10.6
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new b4ba0cc3f95d Add remaining updates for 2.10.5 and 2.10.6 b4ba0cc3f95d is described below commit b4ba0cc3f95d5e3f63882d07e1ccf332137ee2e4 Author: Lari Hotari AuthorDate: Fri Mar 8 16:28:39 2024 +0200 Add remaining updates for 2.10.5 and 2.10.6 --- data/release-java.js | 2 ++ releases.json| 2 ++ 2 files changed, 4 insertions(+) diff --git a/data/release-java.js b/data/release-java.js index 78d4d6ba3e87..f8a277e5a357 100644 --- a/data/release-java.js +++ b/data/release-java.js @@ -12,6 +12,8 @@ module.exports = [ {tagName: "v2.11.2",vtag:"2.11.x",releaseNotes:"/release-notes/versioned/client-java-2.11.2/",doc:"/docs/2.11.x/client-libraries-java",version:""}, {tagName: "v2.11.1",vtag:"2.11.x",releaseNotes:"/release-notes/versioned/client-java-2.11.1/",doc:"/docs/2.11.x/client-libraries-java",version:""}, {tagName: "v2.11.0",vtag:"2.11.x",releaseNotes:"/release-notes/versioned/client-java-2.11.0/",doc:"/docs/2.11.x/client-libraries-java",version:""}, +{tagName: "v2.10.6",vtag:"2.10.x",releaseNotes:"/release-notes/versioned/client-java-2.10.6/",doc:"/docs/2.10.x/client-libraries-java",version:"v2.10.x"}, +{tagName: "v2.10.5",vtag:"2.10.x",releaseNotes:"/release-notes/versioned/client-java-2.10.5/",doc:"/docs/2.10.x/client-libraries-java",version:"v2.10.x"}, {tagName: "v2.10.4",vtag:"2.10.x",releaseNotes:"/release-notes/versioned/client-java-2.10.4/",doc:"/docs/2.10.x/client-libraries-java",version:"v2.10.x"}, {tagName: "v2.10.3",vtag:"2.10.x",releaseNotes:"/release-notes/versioned/client-java-2.10.3/",doc:"/docs/2.10.x/client-libraries-java",version:""}, {tagName: "v2.10.2",vtag:"2.10.x",releaseNotes:"/release-notes/versioned/client-java-2.10.2/",doc:"/docs/2.10.x/client-libraries-java",version:""}, diff --git a/releases.json b/releases.json index e4c911c4ca6a..14f928bfbc26 100644 --- a/releases.json +++ b/releases.json @@ -12,6 +12,8 @@ "2.11.2", "2.11.1", "2.11.0", + "2.10.6", + "2.10.5", "2.10.4", "2.10.3", "2.10.2",
Re: [PR] Release 3.2.1 [pulsar-site]
visortelle commented on code in PR #828: URL: https://github.com/apache/pulsar-site/pull/828#discussion_r1517781010 ## release-notes/versioned/pulsar-3.2.1.md: ## @@ -0,0 +1,68 @@ +--- +id: pulsar-3.2.1 +title: Apache Pulsar 3.2.1 +sidebar_label: Apache Pulsar 3.2.1 +--- + + 2024-03-08 + +### Broker +- Consistently add fine-grain authorization to REST API [#22202](https://github.com/apache/pulsar/pull/22202) +- Add fine-grain authorization to retention admin API [#22163](https://github.com/apache/pulsar/pull/22163) +- Print non log when delete partitioned topic failed [#22153](https://github.com/apache/pulsar/pull/22153) +- Fix broker not starting when both transactions and the Extensible Load Manager are enabled [#22139](https://github.com/apache/pulsar/pull/22139) +- Return getOwnerAsync without waiting on source broker upon Assigning and Releasing and handle role change during role init [#22112](https://github.com/apache/pulsar/pull/22112) +- Cache the internal writer when sent to system topic. [#22099](https://github.com/apache/pulsar/pull/22099) +- Enabling batch causes negative unackedMessages due to ack and delivery concurrency [#22090](https://github.com/apache/pulsar/pull/22090) +- Support running docker container with gid != 0 [#22081](https://github.com/apache/pulsar/pull/22081) +- Add an error log to troubleshoot the failure of starting broker registry [#22065](https://github.com/apache/pulsar/pull/22065) +- Set ServiceUnitStateChannel topic compaction threshold explicitly, improve getOwnerAsync, and fix other bugs [#22064](https://github.com/apache/pulsar/pull/22064) +- Fix hash collision when using a consumer name that ends with a number [#22053](https://github.com/apache/pulsar/pull/22053) +- Sanitize values before logging in apply-config-from-env.py script [#22044](https://github.com/apache/pulsar/pull/22044) +- Fix can not subscribe partitioned topic with a suffix-matched regexp [#22025](https://github.com/apache/pulsar/pull/22025) +- Do not retain the data in the system topic [#22022](https://github.com/apache/pulsar/pull/22022) +- Subscription stuck due to called Admin API analyzeSubscriptionBacklog [#22019](https://github.com/apache/pulsar/pull/22019) +- Do not try to open ML when the topic meta does not exist and do not expect to create a new one [#22004](https://github.com/apache/pulsar/pull/22004) +- Do not print an Error log when responding to HTTP-404 when calling Admin API and the topic does not exist [#21995](https://github.com/apache/pulsar/pull/21995) +- Do not close the socket if lookup failed due to LockBusyException [#21993](https://github.com/apache/pulsar/pull/21993) +- Support setting autoSkipNonRecoverableData dynamically in PersistentMessageExpiryMonitor [#21991](https://github.com/apache/pulsar/pull/21991) +- Add timeout for health check read [#21990](https://github.com/apache/pulsar/pull/21990) +- Fix schema deletion error when deleting a partitioned topic with many partitions and schema [#21977](https://github.com/apache/pulsar/pull/21977) +- Defer the ownership checks if the owner is inactive (ExtensibleLoadManager) [#21811](https://github.com/apache/pulsar/pull/21811) +- Fix compaction/replication data loss when expire messages [#21865](https://github.com/apache/pulsar/pull/21865) +- Skip loading the NAR packages if not configured [#21867](https://github.com/apache/pulsar/pull/21867) +- Improve NamespaceUnloadStrategy error message [#21880](https://github.com/apache/pulsar/pull/21880) +- Don't rollover empty ledgers based on inactivity [#21893](https://github.com/apache/pulsar/pull/21893) +- Expire messages according to ledger close time to avoid client clock skew [#21940](https://github.com/apache/pulsar/pull/21940) +- Fix reader stuck when read from compacted topic with read compact mode disable [#21969](https://github.com/apache/pulsar/pull/21969) + + +### Pulsar IO and Pulsar Functions +- [FN] Add configuration for connector & functions package url sources [#22184](https://github.com/apache/pulsar/pull/22184) +- [FN] Add missing "exception" argument to some log.error [#22140](https://github.com/apache/pulsar/pull/22140) +- [FN] Prevent putstate uses empty values [#22127](https://github.com/apache/pulsar/pull/22127) +- [FN] Optimize Function Worker startup by lazy loading and direct zip/bytecode access [#22122](https://github.com/apache/pulsar/pull/22122) +- [FN] Prevent create state table from API calls for non-exists instances [#22107](https://github.com/apache/pulsar/pull/22107) +- [FN] Use unified PackageManagement service to download packages [#21955](https://github.com/apache/pulsar/pull/21955) + +### CLI +- [Admin] Expose the offload threshold in seconds to the admin [#22101](https://github.com/apache/pulsar/pull/22101) +- [CLI] Fix the bug when set-retention specified size with -T [#22150](https://github.com/apache/pulsar/pull/22150) +- [CLI] Error out when --destinationBroker is set when
Re: [PR] Release 3.2.1 [pulsar-site]
visortelle commented on code in PR #828: URL: https://github.com/apache/pulsar-site/pull/828#discussion_r1517781010 ## release-notes/versioned/pulsar-3.2.1.md: ## @@ -0,0 +1,68 @@ +--- +id: pulsar-3.2.1 +title: Apache Pulsar 3.2.1 +sidebar_label: Apache Pulsar 3.2.1 +--- + + 2024-03-08 + +### Broker +- Consistently add fine-grain authorization to REST API [#22202](https://github.com/apache/pulsar/pull/22202) +- Add fine-grain authorization to retention admin API [#22163](https://github.com/apache/pulsar/pull/22163) +- Print non log when delete partitioned topic failed [#22153](https://github.com/apache/pulsar/pull/22153) +- Fix broker not starting when both transactions and the Extensible Load Manager are enabled [#22139](https://github.com/apache/pulsar/pull/22139) +- Return getOwnerAsync without waiting on source broker upon Assigning and Releasing and handle role change during role init [#22112](https://github.com/apache/pulsar/pull/22112) +- Cache the internal writer when sent to system topic. [#22099](https://github.com/apache/pulsar/pull/22099) +- Enabling batch causes negative unackedMessages due to ack and delivery concurrency [#22090](https://github.com/apache/pulsar/pull/22090) +- Support running docker container with gid != 0 [#22081](https://github.com/apache/pulsar/pull/22081) +- Add an error log to troubleshoot the failure of starting broker registry [#22065](https://github.com/apache/pulsar/pull/22065) +- Set ServiceUnitStateChannel topic compaction threshold explicitly, improve getOwnerAsync, and fix other bugs [#22064](https://github.com/apache/pulsar/pull/22064) +- Fix hash collision when using a consumer name that ends with a number [#22053](https://github.com/apache/pulsar/pull/22053) +- Sanitize values before logging in apply-config-from-env.py script [#22044](https://github.com/apache/pulsar/pull/22044) +- Fix can not subscribe partitioned topic with a suffix-matched regexp [#22025](https://github.com/apache/pulsar/pull/22025) +- Do not retain the data in the system topic [#22022](https://github.com/apache/pulsar/pull/22022) +- Subscription stuck due to called Admin API analyzeSubscriptionBacklog [#22019](https://github.com/apache/pulsar/pull/22019) +- Do not try to open ML when the topic meta does not exist and do not expect to create a new one [#22004](https://github.com/apache/pulsar/pull/22004) +- Do not print an Error log when responding to HTTP-404 when calling Admin API and the topic does not exist [#21995](https://github.com/apache/pulsar/pull/21995) +- Do not close the socket if lookup failed due to LockBusyException [#21993](https://github.com/apache/pulsar/pull/21993) +- Support setting autoSkipNonRecoverableData dynamically in PersistentMessageExpiryMonitor [#21991](https://github.com/apache/pulsar/pull/21991) +- Add timeout for health check read [#21990](https://github.com/apache/pulsar/pull/21990) +- Fix schema deletion error when deleting a partitioned topic with many partitions and schema [#21977](https://github.com/apache/pulsar/pull/21977) +- Defer the ownership checks if the owner is inactive (ExtensibleLoadManager) [#21811](https://github.com/apache/pulsar/pull/21811) +- Fix compaction/replication data loss when expire messages [#21865](https://github.com/apache/pulsar/pull/21865) +- Skip loading the NAR packages if not configured [#21867](https://github.com/apache/pulsar/pull/21867) +- Improve NamespaceUnloadStrategy error message [#21880](https://github.com/apache/pulsar/pull/21880) +- Don't rollover empty ledgers based on inactivity [#21893](https://github.com/apache/pulsar/pull/21893) +- Expire messages according to ledger close time to avoid client clock skew [#21940](https://github.com/apache/pulsar/pull/21940) +- Fix reader stuck when read from compacted topic with read compact mode disable [#21969](https://github.com/apache/pulsar/pull/21969) + + +### Pulsar IO and Pulsar Functions +- [FN] Add configuration for connector & functions package url sources [#22184](https://github.com/apache/pulsar/pull/22184) +- [FN] Add missing "exception" argument to some log.error [#22140](https://github.com/apache/pulsar/pull/22140) +- [FN] Prevent putstate uses empty values [#22127](https://github.com/apache/pulsar/pull/22127) +- [FN] Optimize Function Worker startup by lazy loading and direct zip/bytecode access [#22122](https://github.com/apache/pulsar/pull/22122) +- [FN] Prevent create state table from API calls for non-exists instances [#22107](https://github.com/apache/pulsar/pull/22107) +- [FN] Use unified PackageManagement service to download packages [#21955](https://github.com/apache/pulsar/pull/21955) + +### CLI +- [Admin] Expose the offload threshold in seconds to the admin [#22101](https://github.com/apache/pulsar/pull/22101) +- [CLI] Fix the bug when set-retention specified size with -T [#22150](https://github.com/apache/pulsar/pull/22150) +- [CLI] Error out when --destinationBroker is set when
[PR] [WIP][improve][broker] Consistently add fine-grain authorization to REST API [pulsar]
Technoboy- opened a new pull request, #7: URL: https://github.com/apache/pulsar/pull/7 ### Motivation ### Documentation - [ ] `doc` - [ ] `doc-required` - [x] `doc-not-needed` - [ ] `doc-complete` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Release 3.2.1 [pulsar-site]
Technoboy- merged PR #828: URL: https://github.com/apache/pulsar-site/pull/828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-site) branch doc_release_3.2.1 updated (809b39d8b2e3 -> d62cccf35816)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a change to branch doc_release_3.2.1 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git from 809b39d8b2e3 update add d62cccf35816 update No new revisions were added by this update. Summary of changes: release-notes/versioned/client-java-3.2.1.md | 8 release-notes/versioned/pulsar-3.2.1.md | 68 2 files changed, 76 insertions(+) create mode 100644 release-notes/versioned/client-java-3.2.1.md create mode 100644 release-notes/versioned/pulsar-3.2.1.md
Re: [I] [Bug] Not able to set default backlog-quota retention policy , default retention policy and default TTL [pulsar]
lhotari commented on issue #6: URL: https://github.com/apache/pulsar/issues/6#issuecomment-1985730779 The latest Pulsar Helm chart version is 3.3.0 , https://github.com/apache/pulsar-helm-chart/releases . All older Helm chart versions aren't supported. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Not able to set default backlog-quota retention policy , default retention policy and default TTL [pulsar]
lhotari commented on issue #6: URL: https://github.com/apache/pulsar/issues/6#issuecomment-1985728802 Please notice that 2.10.x is out of support: https://pulsar.apache.org/contribute/release-policy/#supported-versions -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-site) branch doc_release_3.2.1 updated (fb97d31e4589 -> 809b39d8b2e3)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a change to branch doc_release_3.2.1 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git from fb97d31e4589 update add 809b39d8b2e3 update No new revisions were added by this update. Summary of changes: data/release-java.js | 3 ++- data/release-pulsar.js | 9 + 2 files changed, 11 insertions(+), 1 deletion(-)
(pulsar-site) branch doc_release_3.2.1 updated (2af113a65416 -> fb97d31e4589)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a change to branch doc_release_3.2.1 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git discard 2af113a65416 update discard b1e53cc3f536 Release 3.2.1 add be352248b0ee Update release-process.md add 7751cbfcc47e Update release-process.md add d913d3b5cdc9 Update release-process.md add 2a6e9d6514b6 Update release-process.md add 0214b3152535 Update release-process.md add c5f824793239 Update release-process.md add 888c66281559 Update release-process.md add 7ae823a44f42 Release notes 2.10.6 (#829) add 61b4890ae0a1 Go back to regctl as before since multi-arch images need to be supported in copying add d289b6ecf1bc Release 3.2.1 add fb97d31e4589 update This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (2af113a65416) \ N -- N -- N refs/heads/doc_release_3.2.1 (fb97d31e4589) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: contribute/release-process.md | 80 +++ data/release-pulsar.js | 9 ++ release-notes/versioned/client-java-2.10.6.md | 18 release-notes/versioned/pulsar-2.10.6.md | 109 + static/swagger/{2.10.5 => 2.10.6}/swagger.json | 24 + .../{2.10.5 => 2.10.6}/swaggerfunctions.json | 0 .../{2.10.5 => 2.10.6}/swaggerpackages.json| 0 static/swagger/{2.10.5 => 2.10.6}/swaggersink.json | 0 .../swagger/{2.10.5 => 2.10.6}/swaggersource.json | 0 static/swagger/{2.10.5 => 2.10.6/v2}/swagger.json | 24 + .../{2.10.5 => 2.10.6/v3}/swaggerfunctions.json| 0 .../{2.10.5 => 2.10.6/v3}/swaggerpackages.json | 0 .../swagger/{2.10.5 => 2.10.6/v3}/swaggersink.json | 0 .../{2.10.5 => 2.10.6/v3}/swaggersource.json | 0 static/swagger/restApiVersions.json| 17 15 files changed, 263 insertions(+), 18 deletions(-) create mode 100644 release-notes/versioned/client-java-2.10.6.md create mode 100644 release-notes/versioned/pulsar-2.10.6.md copy static/swagger/{2.10.5 => 2.10.6}/swagger.json (99%) copy static/swagger/{2.10.5 => 2.10.6}/swaggerfunctions.json (100%) copy static/swagger/{2.10.5 => 2.10.6}/swaggerpackages.json (100%) copy static/swagger/{2.10.5 => 2.10.6}/swaggersink.json (100%) copy static/swagger/{2.10.5 => 2.10.6}/swaggersource.json (100%) copy static/swagger/{2.10.5 => 2.10.6/v2}/swagger.json (99%) copy static/swagger/{2.10.5 => 2.10.6/v3}/swaggerfunctions.json (100%) copy static/swagger/{2.10.5 => 2.10.6/v3}/swaggerpackages.json (100%) copy static/swagger/{2.10.5 => 2.10.6/v3}/swaggersink.json (100%) copy static/swagger/{2.10.5 => 2.10.6/v3}/swaggersource.json (100%)
(pulsar-site) branch doc_release_3.2.1 updated (b1e53cc3f536 -> 2af113a65416)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a change to branch doc_release_3.2.1 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git from b1e53cc3f536 Release 3.2.1 add 2af113a65416 update No new revisions were added by this update. Summary of changes: releases.json | 1 + 1 file changed, 1 insertion(+)
(pulsar-site) branch doc_release_3.2.1 updated (d9acdf5045c3 -> b1e53cc3f536)
This is an automated email from the ASF dual-hosted git repository. technoboy pushed a change to branch doc_release_3.2.1 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git omit d9acdf5045c3 Release 3.2.1 add b1e53cc3f536 Release 3.2.1 This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (d9acdf5045c3) \ N -- N -- N refs/heads/doc_release_3.2.1 (b1e53cc3f536) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes:
[I] [Bug] Not able to set fefault backlog backlog retention policy , default retention policy and default TTL [pulsar]
kunjbosamiaFalkonry opened a new issue, #6: URL: https://github.com/apache/pulsar/issues/6 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Version pulsar helm chart version - 2.10.2 k8s version -v1.28.5+k3s1 ### Minimal reproduce step When installing pulsar using pulsar helm chart added these values to broker.configData ``` backlogQuotaDefaultRetentionPolicy: "consumer_backlog_eviction" backlogQuotaDefaultLimitBytes: "322122547200" backlogQuotaDefaultLimitSecond: "1209600" defaultRetentionTimeInMinutes: "2880" defaultRetentionSizeInMB: "-1" ttlDurationDefaultInSeconds: "259200" topicLevelPoliciesEnabled: "true" ``` exec in pulsar broker pod do `bin/pulsar-admin namespaces get-backlog-quotas public/functions` ### What did you expect to see? the output should be -> `destination_storageBacklogQuotaImpl(limit=322122547200, limitSize=322122547200, limitTime=1209600, policy=consumer_backlog_eviction)` ### What did you see instead? I get no output same goes for retention policy and ttl -> i get null as output , but it should return the default ttl and default return policy values as output respectively ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] not able to login to pulsar UI [pulsar-helm-chart]
Mortom123 commented on issue #467: URL: https://github.com/apache/pulsar-helm-chart/issues/467#issuecomment-1985616250 You are installing into the `pulsar` namespace, judging by your `prepare_helm_release` command @rahuldstk. You also need to run the command to retrieve the secret inside this namespace then, like: ``` kubectl get secret -n pulsar -l component=pulsar-manager -o=jsonpath="{.items[0].data.UI_PASSWORD}" | base64 --decode ``` Forwarding the manager via: ``` kubectl port-forward -n pulsar $(kubectl get pods -n pulsar -l component=pulsar-manager -o jsonpath='{.items[0].metadata.name}') 9527:9527 ``` works for me and I can login at `localhost:9527` using `pulsar` and `pulsarpassword`. I do not use the `prepare_helm_release.sh` command. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] topic message size exceeds limit [pulsar]
ktpktr0 opened a new issue, #5: URL: https://github.com/apache/pulsar/issues/5 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Version java: 21 pulsar: 3.2.0 ### Minimal reproduce step log ``` 2024-03-08T13:43:43,369+0800 [GarbageCollectorThread-6-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Do compaction to compact those files lower than 0.2 2024-03-08T13:43:43,369+0800 [GarbageCollectorThread-6-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Compaction: entry log usage buckets before compaction [10% 20% 30% 40% 50% 60% 70% 80% 90% 100%] = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] 2024-03-08T13:43:43,369+0800 [GarbageCollectorThread-6-1] INFO org.apache.bookkeeper.bookie.GarbageCollectorThread - Compaction: entry log usage buckets[10% 20% 30% 40% 50% 60% 70% 80% 90% 100%] = [0, 0, 0, 0, 0, 0, 0, 0, 0, 0], compacted [0, 0, 0, 0, 0, 0, 0, 0, 0, 0] 2024-03-08T13:49:12,105+0800 [bookie-io-8-1] INFO org.apache.bookkeeper.proto.AuthHandler - Authentication success on server side 2024-03-08T13:49:12,105+0800 [bookie-io-8-1] INFO org.apache.bookkeeper.proto.BookieRequestHandler - Channel connected [id: 0x3087e422, L:/192.168.0.150:3181 - R:/192.168.0.250:53602] 2024-03-08T13:49:12,106+0800 [bookie-io-8-1] ERROR org.apache.bookkeeper.proto.BookieRequestHandler - Unhandled exception occurred in I/O thread or handler on [id: 0x3087e422, L:/192.168.0.150:3181 - R:/192.168.0.250:53602] io.netty.handler.codec.TooLongFrameException: Adjusted frame length exceeds 5253120: 1195725860 - discarded at io.netty.handler.codec.LengthFieldBasedFrameDecoder.fail(LengthFieldBasedFrameDecoder.java:507) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final] at io.netty.handler.codec.LengthFieldBasedFrameDecoder.failIfNecessary(LengthFieldBasedFrameDecoder.java:493) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final] at io.netty.handler.codec.LengthFieldBasedFrameDecoder.exceededFrameLength(LengthFieldBasedFrameDecoder.java:377) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final] at io.netty.handler.codec.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:423) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final] at io.netty.handler.codec.LengthFieldBasedFrameDecoder.decode(LengthFieldBasedFrameDecoder.java:333) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final] at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:529) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final] at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:468) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final] at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:290) ~[io.netty-netty-codec-4.1.104.Final.jar:4.1.104.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:444) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final] at io.netty.handler.flush.FlushConsolidationHandler.channelRead(FlushConsolidationHandler.java:152) ~[io.netty-netty-handler-4.1.104.Final.jar:4.1.104.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:442) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final] at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:412) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final] at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:440) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final] at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:420) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final] at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) ~[io.netty-netty-transport-4.1.104.Final.jar:4.1.104.Final] at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800)
(pulsar-site) branch main updated: Go back to regctl as before since multi-arch images need to be supported in copying
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 61b4890ae0a1 Go back to regctl as before since multi-arch images need to be supported in copying 61b4890ae0a1 is described below commit 61b4890ae0a18809405a466d78038b949c8d1126 Author: Lari Hotari AuthorDate: Fri Mar 8 14:27:58 2024 +0200 Go back to regctl as before since multi-arch images need to be supported in copying --- contribute/release-process.md | 20 ++-- 1 file changed, 6 insertions(+), 14 deletions(-) diff --git a/contribute/release-process.md b/contribute/release-process.md index 24d9f39e8181..9a9f74541261 100644 --- a/contribute/release-process.md +++ b/contribute/release-process.md @@ -416,19 +416,14 @@ Select the repository and click on "Release". Artifacts will now be made availab This step is performed by a Apache Pulsar PMC member. Please request help from a PMC member for completing this step. +`regctl` from [regclient](https://github.com/regclient/regclient) is needed for copying multi-arch images. Install with `brew install regclient` or with [other installation options](https://github.com/regclient/regclient/blob/main/docs/install.md) of regclient. The benefit of `regctl` over using `docker pull/tag/push` is that it will handle copying both `amd64` and the `arm64` image. + ```bash RELEASE_MANAGER_DOCKER_USER=otheruser CANDIDATE_TAG=$VERSION_WITHOUT_RC -# pulsar image -docker pull ${RELEASE_MANAGER_DOCKER_USER}/pulsar:${CANDIDATE_TAG} -docker tag ${RELEASE_MANAGER_DOCKER_USER}/pulsar:${CANDIDATE_TAG} apachepulsar/pulsar:$VERSION_WITHOUT_RC -docker push apachepulsar/pulsar:${PULSAR_VERSION} - -# pulsar-all image -docker pull ${RELEASE_MANAGER_DOCKER_USER}/pulsar-all:${CANDIDATE_TAG} -docker tag ${RELEASE_MANAGER_DOCKER_USER}/pulsar-all:${CANDIDATE_TAG} apachepulsar/pulsar-all:$VERSION_WITHOUT_RC -docker push apachepulsar/pulsar-all:${PULSAR_VERSION} +regctl image copy ${RELEASE_MANAGER_DOCKER_USER}/pulsar:${CANDIDATE_TAG} apachepulsar/pulsar:$VERSION_WITHOUT_RC +regctl image copy ${RELEASE_MANAGER_DOCKER_USER}/pulsar-all:${CANDIDATE_TAG} apachepulsar/pulsar-all:$VERSION_WITHOUT_RC ``` Go to check the result: @@ -444,11 +439,8 @@ This step is for the latest release only. ::: ``` -docker tag apachepulsar/pulsar:$VERSION_WITHOUT_RC apachepulsar/pulsar:latest -docker push apachepulsar/pulsar:latest - -docker tag apachepulsar/pulsar-all:$VERSION_WITHOUT_RC apachepulsar/pulsar-all:latest -docker push apachepulsar/pulsar-all:latest +regctl image copy apachepulsar/pulsar:$VERSION_WITHOUT_RC apachepulsar/pulsar:latest +regctl image copy apachepulsar/pulsar-all:$VERSION_WITHOUT_RC apachepulsar/pulsar-all:latest ``` ### Update project version
Re: [PR] Release notes 2.10.6 [pulsar-site]
codelipenghui merged PR #829: URL: https://github.com/apache/pulsar-site/pull/829 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-site) branch main updated: Update release-process.md
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 888c66281559 Update release-process.md 888c66281559 is described below commit 888c6628155949a22e65d089273546b8f6eabc02 Author: Lari Hotari AuthorDate: Fri Mar 8 04:01:47 2024 -0800 Update release-process.md --- contribute/release-process.md | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/contribute/release-process.md b/contribute/release-process.md index 54a3883c6dce..24d9f39e8181 100644 --- a/contribute/release-process.md +++ b/contribute/release-process.md @@ -393,7 +393,7 @@ echo https://github.com/apache/pulsar/releases/tag/v${VERSION_WITHOUT_RC} 6. Unselect "Set as the latest release" (that should be only selected for the actual latest release of Pulsar) 7. Click "Publish release". -The [Writing release notes](release-note-guide.md) guide should be followed to generate a proper release notes. +The [Writing release notes](release-note-guide.md) guide should be followed to generate a proper release notes. That is covered in the "Update the document" section. ### Release the artifacts on SVN @@ -408,7 +408,9 @@ svn move -m "Release Apache Pulsar $VERSION_WITHOUT_RC" \ ### Release Maven modules -Promote the Maven staging repository for release. Login to `https://repository.apache.org` and select the staging repository associated with the RC candidate that was approved. The naming will be like `orgapachepulsar-XYZ`. Select the repository and click on "Release". Artifacts will now be made available on Maven central. +Promote the Maven staging repository for release. Login to https://repository.apache.org and select the staging repository associated with the RC candidate that was approved. +Double check the staging repository name from the release vote email. +Select the repository and click on "Release". Artifacts will now be made available on Maven central. ### Release Docker images @@ -439,6 +441,8 @@ Ensure that newer than 3.x images support both amd64 and arm64. Older 2.x images This step is for the latest release only. +::: + ``` docker tag apachepulsar/pulsar:$VERSION_WITHOUT_RC apachepulsar/pulsar:latest docker push apachepulsar/pulsar:latest
(pulsar-site) branch main updated (0214b3152535 -> c5f824793239)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git from 0214b3152535 Update release-process.md add c5f824793239 Update release-process.md No new revisions were added by this update. Summary of changes: contribute/release-process.md | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-)
[PR] Release notes 2.10.6 [pulsar-site]
liangyepianzhou opened a new pull request, #829: URL: https://github.com/apache/pulsar-site/pull/829 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-site) 02/03: Release notes 2.10.6
This is an automated email from the ASF dual-hosted git repository. xiangying pushed a commit to branch release/2.10.6 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git commit 4c4775c4fc6cc2807a8b3c5a7f38fb0fd04db4b1 Author: Xiangying Meng <55571188+liangyepianz...@users.noreply.github.com> AuthorDate: Fri Mar 8 19:49:34 2024 +0800 Release notes 2.10.6 --- data/release-pulsar.js| 9 +++ release-notes/versioned/client-java-2.10.6.md | 18 + release-notes/versioned/pulsar-2.10.6.md | 109 ++ static/swagger/restApiVersions.json | 17 4 files changed, 153 insertions(+) diff --git a/data/release-pulsar.js b/data/release-pulsar.js index 823cf4dce5f2..0a22fd9fdb65 100644 --- a/data/release-pulsar.js +++ b/data/release-pulsar.js @@ -100,6 +100,15 @@ module.exports = [ "doc": "/docs/2.11.x", "version": "" }, +{ + "author": "liangyepianzhou", + "tagName": "v2.10.6", + "publishedAt": "2024-03-08T01:53:34Z", + "vtag": "2.10.x", + "releaseNotes": "/release-notes/versioned/pulsar-2.10.6/", + "doc": "/docs/2.10.x", + "version": "v2.10.x" +}, { "author": "liangyepianzhou", "tagName": "v2.10.5", diff --git a/release-notes/versioned/client-java-2.10.6.md b/release-notes/versioned/client-java-2.10.6.md index e69de29bb2d1..cb369a621f76 100644 --- a/release-notes/versioned/client-java-2.10.6.md +++ b/release-notes/versioned/client-java-2.10.6.md @@ -0,0 +1,18 @@ +--- +id: client-java-2.10.6 +title: Client Java 2.10.6 +sidebar_label: Client Java 2.10.6 +--- + +## Client + +### Fixes + +- [fix][client] Fix ConsumerBuilderImpl#subscribe silent stuck when using pulsar-client:3.0.x with jackson-annotations prior to 2.12.0 [21985](https://github.com/apache/pulsar/pull/21985) +- [fix] [client] fix reader.hasMessageAvailable return false when incoming queue is not empty [21259](https://github.com/apache/pulsar/pull/21259) +- [fix][client] fix same producer/consumer use more than one connection per broker [21144](https://github.com/apache/pulsar/pull/21144) +- [fix][client] Fix repeat consume when using n-ack and batched messages [21116](https://github.com/apache/pulsar/pull/21116) +- [fix][client] Avoid ack hole for chunk message [21101](https://github.com/apache/pulsar/pull/21101) +- [fix][client] Fix consumer can't consume resent chunked messages [21070](https://github.com/apache/pulsar/pull/21070) +- [fix] [cli] the variable producerName of BatchMsgContainer is null [20819](https://github.com/apache/pulsar/pull/20819) +- [fix][client] fix negative message re-delivery twice issue [20750](https://github.com/apache/pulsar/pull/20750) diff --git a/release-notes/versioned/pulsar-2.10.6.md b/release-notes/versioned/pulsar-2.10.6.md index e69de29bb2d1..a0b615ec91ba 100644 --- a/release-notes/versioned/pulsar-2.10.6.md +++ b/release-notes/versioned/pulsar-2.10.6.md @@ -0,0 +1,109 @@ +--- +id: pulsar-2.10.6 +title: Apache Pulsar 2.10.6 +sidebar_label: Apache Pulsar 2.10.6 +--- + + 2024-3-8 +## Broker + +### Improvements + +- [improve][broker] Consistently add fine-grain authorization to REST API [22202](https://github.com/apache/pulsar/pull/22202) +- [improve] [broker] Do not print an Error log when responding to `HTTP-404` when calling `Admin API` and the topic does not exist. [21995](https://github.com/apache/pulsar/pull/21995) +- [improve][proxy] Fix comment about enableProxyStatsEndpoints [21757](https://github.com/apache/pulsar/pull/21757) +- [improve][broker][PIP-318] Support not retaining null-key message during topic compaction [21578](https://github.com/apache/pulsar/pull/21578) +- [improve] [client] Merge lookup requests for the same topic [21232](https://github.com/apache/pulsar/pull/21232) +- [improve] [broker] Let the producer request success at the first time if the previous one is inactive [21220](https://github.com/apache/pulsar/pull/21220) +- [improve] [proxy] Not close the socket if lookup failed caused by too many requests [21216](https://github.com/apache/pulsar/pull/21216) +- [improve] [broker] Not close the socket if lookup failed caused by bundle unloading or metadata ex [21211](https://github.com/apache/pulsar/pull/21211) +- [improve] [broker] Print warn log if ssl handshake error & print ledger id when switch ledger [21201](https://github.com/apache/pulsar/pull/21201) +- [improve] [broker] improve read entry error log for troubleshooting [21169](https://github.com/apache/pulsar/pull/21169) +- [improve][proxy] Support disabling metrics endpoint [21031](https://github.com/apache/pulsar/pull/21031) +- [improve][broker] Improve cache handling for partitioned topic metadata when doing lookup [21063](https://github.com/apache/pulsar/pull/21063) +- [improve][broker] Avoid print redirect exception log when get list from bundle [20846](https://github.com/apache/pulsar/pull/20846) +- [improve][sql] Fix the wrong
(pulsar-site) 01/03: Release notes 2.10.6
This is an automated email from the ASF dual-hosted git repository. xiangying pushed a commit to branch release/2.10.6 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git commit f223278707d1c1c6bbe604c638e8f4892f055756 Author: Xiangying Meng <55571188+liangyepianz...@users.noreply.github.com> AuthorDate: Fri Mar 8 19:49:25 2024 +0800 Release notes 2.10.6 --- release-notes/versioned/client-java-2.10.6.md | 0 release-notes/versioned/pulsar-2.10.6.md | 0 2 files changed, 0 insertions(+), 0 deletions(-) diff --git a/release-notes/versioned/client-java-2.10.6.md b/release-notes/versioned/client-java-2.10.6.md new file mode 100644 index ..e69de29bb2d1 diff --git a/release-notes/versioned/pulsar-2.10.6.md b/release-notes/versioned/pulsar-2.10.6.md new file mode 100644 index ..e69de29bb2d1
(pulsar-site) branch release/2.10.6 created (now 168bdc1cefea)
This is an automated email from the ASF dual-hosted git repository. xiangying pushed a change to branch release/2.10.6 in repository https://gitbox.apache.org/repos/asf/pulsar-site.git at 168bdc1cefea Release notes 2.10.6 This branch includes the following new commits: new f223278707d1 Release notes 2.10.6 new 4c4775c4fc6c Release notes 2.10.6 new 168bdc1cefea Release notes 2.10.6 The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
svn commit: r67800 - /dev/pulsar/pulsar-2.11.4-candidate-1/ /release/pulsar/pulsar-2.11.4/
Author: lhotari Date: Fri Mar 8 11:48:30 2024 New Revision: 67800 Log: Release Apache Pulsar 2.11.4 Added: release/pulsar/pulsar-2.11.4/ - copied from r67799, dev/pulsar/pulsar-2.11.4-candidate-1/ Removed: dev/pulsar/pulsar-2.11.4-candidate-1/
(pulsar-site) branch main updated: Update release-process.md
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 0214b3152535 Update release-process.md 0214b3152535 is described below commit 0214b31525356212b0c6ef39ff5c38e89f7082f5 Author: Lari Hotari AuthorDate: Fri Mar 8 03:45:33 2024 -0800 Update release-process.md fix typo --- contribute/release-process.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/contribute/release-process.md b/contribute/release-process.md index 2a25bf35398a..4108654d814a 100644 --- a/contribute/release-process.md +++ b/contribute/release-process.md @@ -382,7 +382,7 @@ Then, you can [create a GitHub release](https://docs.github.com/en/repositories/ ```shell # open this URL and create release notes by clicking "Create release from tag" -echo https://github.com/apache/pulsar/releases/tag/pulsar-${VERSION_WITHOUT_RC} +echo https://github.com/apache/pulsar/releases/tag/v${VERSION_WITHOUT_RC} ``` 1. Open the above URL in a browser and create release notes by clicking "Create release from tag".