(pulsar-helm-chart) branch master updated: Make zookeeper healthchecks compatible with Alpine's busybox nc (#504)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-helm-chart.git
The following commit(s) were added to refs/heads/master by this push:
new 9db0ccc Make zookeeper healthchecks compatible with Alpine's busybox
nc (#504)
9db0ccc is described below
commit 9db0cccaca47c8ea08cc61bba12c3c0b4c814a72
Author: Lari Hotari
AuthorDate: Sat Jun 8 08:52:06 2024 +0300
Make zookeeper healthchecks compatible with Alpine's busybox nc (#504)
* Make zookeeper healthchecks compatible with Alpine's busybox nc
* Test Pulsar 3.3.0 image
* Use 127.0.0.1 instead of localhost in zookeeper healthchecks
- Alpine nc fails if "localhost" is used.
- perhaps it defaults to use IPv6?
* Disable testing with Pulsar 3.3.0 image until 3.3.1 is released
- the image needs "apk add bind-tools" since busybox nslookup isn't
compatible with kubernetes
---
.ci/clusters/values-pulsar-latest.yaml | 20
.github/workflows/pulsar-helm-chart-ci.yaml| 7 +++
charts/pulsar/templates/zookeeper-statefulset.yaml | 10 +-
3 files changed, 32 insertions(+), 5 deletions(-)
diff --git a/.ci/clusters/values-pulsar-latest.yaml
b/.ci/clusters/values-pulsar-latest.yaml
new file mode 100644
index 000..3b06517
--- /dev/null
+++ b/.ci/clusters/values-pulsar-latest.yaml
@@ -0,0 +1,20 @@
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+defaultPulsarImageTag: 3.3.0
\ No newline at end of file
diff --git a/.github/workflows/pulsar-helm-chart-ci.yaml
b/.github/workflows/pulsar-helm-chart-ci.yaml
index 4e4f05e..3ed4760 100644
--- a/.github/workflows/pulsar-helm-chart-ci.yaml
+++ b/.github/workflows/pulsar-helm-chart-ci.yaml
@@ -209,6 +209,13 @@ jobs:
values_file: .ci/clusters/values-pulsar-manager.yaml
shortname: pulsar-manager
include:
+ # - k8sVersion:
+ # version: "1.21.14"
+ # kind_image_tag:
v1.21.14@sha256:8a4e9bb3f415d2bb81629ce33ef9c76ba514c14d707f9797a01e3216376ba093
+ # testScenario:
+ # name: "Pulsar Latest"
+ # values_file: .ci/clusters/values-pulsar-latest.yaml
+ # shortname: pulsar-latest
- k8sVersion:
version: "1.21.14"
kind_image_tag:
v1.21.14@sha256:8a4e9bb3f415d2bb81629ce33ef9c76ba514c14d707f9797a01e3216376ba093
diff --git a/charts/pulsar/templates/zookeeper-statefulset.yaml
b/charts/pulsar/templates/zookeeper-statefulset.yaml
index 5a7ea79..8f0ef6e 100755
--- a/charts/pulsar/templates/zookeeper-statefulset.yaml
+++ b/charts/pulsar/templates/zookeeper-statefulset.yaml
@@ -156,9 +156,9 @@ spec:
name: "{{ template "pulsar.fullname" . }}-{{
.Values.zookeeper.component }}"
{{- $zkConnectCommand := "" -}}
{{- if and .Values.tls.enabled .Values.tls.zookeeper.enabled }}
-{{- $zkConnectCommand = print "openssl s_client -quiet -crlf -connect
localhost:" .Values.zookeeper.ports.clientTls " -cert
/pulsar/certs/zookeeper/tls.crt -key /pulsar/certs/zookeeper/tls.key" -}}
+{{- $zkConnectCommand = print "openssl s_client -quiet -crlf -connect
127.0.0.1:" .Values.zookeeper.ports.clientTls " -cert
/pulsar/certs/zookeeper/tls.crt -key /pulsar/certs/zookeeper/tls.key" -}}
{{- else -}}
-{{- $zkConnectCommand = print "nc -q 1 localhost "
.Values.zookeeper.ports.client -}}
+{{- $zkConnectCommand = print "nc 127.0.0.1 "
.Values.zookeeper.ports.client -}}
{{- end }}
{{- if .Values.zookeeper.probe.readiness.enabled }}
{{- if and (semverCompare "<1.25-0" .Capabilities.KubeVersion.Version)
.Values.rbac.enabled .Values.rbac.psp }}
@@ -172,7 +172,7 @@ spec:
- "{{ .Values.zookeeper.probe.readiness.timeoutSeconds }}"
- bash
- -c
-- 'echo ruok | {{ $zkConnectCommand }} | grep imok'
+- '{ echo ruok; sleep 1; } | {{ $zkConnectCommand }} | grep imok'
initialDelaySeconds: {{
.Values.zookeeper.probe.readiness.initialDelaySeconds }}
periodSeconds: {{
Re: [PR] Make zookeeper healthchecks compatible with Alpine's busybox nc [pulsar-helm-chart]
lhotari merged PR #504: URL: https://github.com/apache/pulsar-helm-chart/pull/504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] one topic suddenly cannot be consumed,others is ok [pulsar]
danielnesaraj commented on issue #21082:
URL: https://github.com/apache/pulsar/issues/21082#issuecomment-2155810564
I am facing this issue randomly as well.
Symptoms: delivery rate to one of the partitions randomly and occasionally
drops to 0 and backlog builds.
Server version: 3.0.3
Java client SDK version: 3.0.4 (chose this version due to the batch ack bug
fix)
What we do:
- We are acknowledging the message on success or negative acknowledging in
other cases.
- We do not set ack timeout.
- We use shared subscription type.
I am able to reproduce this by publishing a huge number of messages into a
partitioned topic using the pulsar-perf tool and using java client consumer.
Today, we are handling this by unloading the topic.
The following are the stats:
```
{
"msgRateIn" : 162.02194280364785,
"msgThroughputIn" : 96448.48058300688,
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesInCounter" : 64639785639,
"msgInCounter" : 110172884,
"bytesOutCounter" : 64282820474,
"msgOutCounter" : 109563883,
"averageMsgSize" : 595.2803608823001,
"msgChunkPublished" : false,
"storageSize" : 523822779,
"backlogSize" : 506085747,
"publishRateLimitedTimes" : 0,
"earliestMsgPublishTimeInBacklogs" : 0,
"offloadedStorageSize" : 0,
"lastOffloadLedgerId" : 0,
"lastOffloadSuccessTimeStamp" : 0,
"lastOffloadFailureTimeStamp" : 0,
"ongoingTxnCount" : 0,
"abortedTxnCount" : 0,
"committedTxnCount" : 0,
"publishers" : [
{
"accessMode" : "Shared",
"msgRateIn" : 0.7333462313290705,
"msgThroughputIn" : 436.9243512338996,
"averageMsgSize" : 595.7954545454545,
"chunkedMessageRate" : 0.0,
"producerId" : 1,
"supportsPartialProducer" : false,
"metadata" : { },
"address" : "/10.0.102.132:34006",
"producerName" : "pulsarv3-60-52435",
"connectedSince" : "2024-06-08T02:46:07.584459326Z",
"clientVersion" : "Pulsar-Java-v3.0.4"
}, {
"accessMode" : "Shared",
"msgRateIn" : 0.9494403702612061,
"msgThroughputIn" : 595.2991121537763,
"averageMsgSize" : 627.0,
"chunkedMessageRate" : 0.0,
"producerId" : 1,
"supportsPartialProducer" : false,
"metadata" : { },
"address" : "/10.0.108.53:46610",
"producerName" : "pulsarv3-60-52441",
"connectedSince" : "2024-06-08T03:12:03.764684205Z",
"clientVersion" : "Pulsar-Java-v3.0.4"
}, {
"accessMode" : "Shared",
"msgRateIn" : 0.783947174283,
"msgThroughputIn" : 433.3242896064604,
"averageMsgSize" : 649.975,
"chunkedMessageRate" : 0.0,
"producerId" : 1,
"supportsPartialProducer" : false,
"metadata" : { },
"address" : "/10.0.100.13:42184",
"producerName" : "pulsarv3-60-52443",
"connectedSince" : "2024-06-08T03:10:14.740170844Z",
"clientVersion" : "Pulsar-Java-v3.0.4"
}, {
"accessMode" : "Shared",
"msgRateIn" : 0.6833453002698462,
"msgThroughputIn" : 403.7904046253064,
"averageMsgSize" : 590.9024390243902,
"chunkedMessageRate" : 0.0,
"producerId" : 1,
"supportsPartialProducer" : false,
"metadata" : { },
"address" : "/10.0.100.13:51930",
"producerName" : "pulsarv3-60-52459",
"connectedSince" : "2024-06-08T03:01:03.355132469Z",
"clientVersion" : "Pulsar-Java-v3.0.4"
}
// more entries removed for brevity
],
"waitingPublishers" : 0,
"subscriptions" : {
"subscription-name-for-app" : {
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 64282820474,
"msgOutCounter" : 109563883,
"msgRateRedeliver" : 0.0,
"messageAckRate" : 0.0,
"chunkedMessageRate" : 0,
"msgBacklog" : 860487,
"backlogSize" : 506085747,
"earliestMsgPublishTimeInBacklog" : 0,
"msgBacklogNoDelayed" : 860487,
"blockedSubscriptionOnUnackedMsgs" : false,
"msgDelayed" : 0,
"unackedMessages" : 0,
"type" : "Shared",
"msgRateExpired" : 0.0,
"totalMsgExpired" : 0,
"lastExpireTimestamp" : 1717816240629,
"lastConsumedFlowTimestamp" : 1717811557813,
"lastConsumedTimestamp" : 1717810973704,
"lastAckedTimestamp" : 1717811020957,
"lastMarkDeleteAdvancedTimestamp" : 1717811020957,
"consumers" : [
{
"msgRateOut" : 0.0,
"msgThroughputOut" : 0.0,
"bytesOutCounter" : 103841680,
"msgOutCounter" : 177401,
"msgRateRedeliver" : 0.0,
"messageAckRate" : 0.0,
"chunkedMessageRate" : 0.0,
"consumerName" : "0fbbf",
"availablePermits" : 99,
"unackedMessages" : 0,
(pulsar-site) branch main updated: Docs sync done from apache/pulsar (#e236f49)
This is an automated email from the ASF dual-hosted git repository. urfree pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new 1112c5474ef3 Docs sync done from apache/pulsar (#e236f49) 1112c5474ef3 is described below commit 1112c5474ef317d8e0bcd4050f6c6fec7075eb60 Author: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> AuthorDate: Sat Jun 8 01:35:52 2024 + Docs sync done from apache/pulsar (#e236f49) --- .../next/config/reference-configuration-broker.md | 26 -- .../config/reference-configuration-standalone.md | 26 -- 2 files changed, 48 insertions(+), 4 deletions(-) diff --git a/static/reference/next/config/reference-configuration-broker.md b/static/reference/next/config/reference-configuration-broker.md index 4165194386c4..88496d0c7e26 100644 --- a/static/reference/next/config/reference-configuration-broker.md +++ b/static/reference/next/config/reference-configuration-broker.md @@ -609,8 +609,30 @@ Average resource usage difference threshold to determine a broker whether to be **Category**: Load Balancer +### loadBalancerBandwidthInResourceWeight +BandwidthIn Resource Usage Weight + +**Type**: `double` + +**Default**: `1.0` + +**Dynamic**: `true` + +**Category**: Load Balancer + +### loadBalancerBandwidthOutResourceWeight +BandwidthOut Resource Usage Weight + +**Type**: `double` + +**Default**: `1.0` + +**Dynamic**: `true` + +**Category**: Load Balancer + ### loadBalancerBandwithInResourceWeight -BandwithIn Resource Usage Weight +BandwidthIn Resource Usage Weight, Deprecated: Use loadBalancerBandwidthInResourceWeight **Type**: `double` @@ -621,7 +643,7 @@ BandwithIn Resource Usage Weight **Category**: Load Balancer ### loadBalancerBandwithOutResourceWeight -BandwithOut Resource Usage Weight +BandwidthOut Resource Usage Weight, Deprecated: Use loadBalancerBandwidthOutResourceWeight **Type**: `double` diff --git a/static/reference/next/config/reference-configuration-standalone.md b/static/reference/next/config/reference-configuration-standalone.md index 4165194386c4..88496d0c7e26 100644 --- a/static/reference/next/config/reference-configuration-standalone.md +++ b/static/reference/next/config/reference-configuration-standalone.md @@ -609,8 +609,30 @@ Average resource usage difference threshold to determine a broker whether to be **Category**: Load Balancer +### loadBalancerBandwidthInResourceWeight +BandwidthIn Resource Usage Weight + +**Type**: `double` + +**Default**: `1.0` + +**Dynamic**: `true` + +**Category**: Load Balancer + +### loadBalancerBandwidthOutResourceWeight +BandwidthOut Resource Usage Weight + +**Type**: `double` + +**Default**: `1.0` + +**Dynamic**: `true` + +**Category**: Load Balancer + ### loadBalancerBandwithInResourceWeight -BandwithIn Resource Usage Weight +BandwidthIn Resource Usage Weight, Deprecated: Use loadBalancerBandwidthInResourceWeight **Type**: `double` @@ -621,7 +643,7 @@ BandwithIn Resource Usage Weight **Category**: Load Balancer ### loadBalancerBandwithOutResourceWeight -BandwithOut Resource Usage Weight +BandwidthOut Resource Usage Weight, Deprecated: Use loadBalancerBandwidthOutResourceWeight **Type**: `double`
[PR] [feat][broker] PIP-264: Add OpenTelemetry producer metrics [pulsar]
dragosvictor opened a new pull request, #22882: URL: https://github.com/apache/pulsar/pull/22882 [PIP-264](https://github.com/apache/pulsar/blob/master/pip/pip-264.md) ### Motivation Adds producer related metrics to the OpenTelemetry pipeline. ### Modifications - Adds the metrics described in https://github.com/apache/pulsar-site/pull/911/ to the collector runs. - Relocates the metric backend data-sources (`Producer#msgIn`, `Producer#chunkedMessageRate` and `Producer#msgDrop`) from the `Producer` class to `PublisherStatsImpl` and `NonPersistentPublisherStatsImpl`, for a cleaner design. - Add field `Rate.totalValueAdder` keeping track of cumulative values added to the Rate, further simplifying the design. ### Verifying this change - [x] Make sure that the change passes the CI checks. This change added tests and can be verified as follows: - Added basic messaging test in `OpenTelemetryProducerStatsTest` - Updated existing test `` to cover the message drop counter functionality ### Does this pull request potentially affect one of the following parts: - [ ] Dependencies (add or upgrade a dependency) - [ ] The public API - [ ] The schema - [ ] The default values of configurations - [ ] The threading model - [ ] The binary protocol - [ ] The REST endpoints - [ ] The admin CLI options - [x] The metrics: As described in https://github.com/apache/pulsar-site/pull/911/ - [ ] Anything that affects deployment ### Documentation - [ ] `doc` - [x] `doc-required` - [ ] `doc-not-needed` - [ ] `doc-complete` ### Matching PR in forked repository PR in forked repository: https://github.com/dragosvictor/pulsar/pull/25 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [feat][doc] PIP-264: Add OpenTelemetry producer metrics reference [pulsar-site]
dragosvictor opened a new pull request, #911: URL: https://github.com/apache/pulsar-site/pull/911 ### ✅ Contribution Checklist - [x] I read the [contribution guide](https://pulsar.apache.org/contribute/document-contribution/) - [x] I updated the [versioned docs](https://pulsar.apache.org/contribute/document-contribution/#update-versioned-docs) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [feat][broker] PIP-264: Add web executor metrics reference [pulsar-site]
dragosvictor commented on code in PR #907:
URL: https://github.com/apache/pulsar-site/pull/907#discussion_r1631745985
##
docs/reference-metrics-opentelemetry.md:
##
@@ -402,3 +402,26 @@ The number of pending topic load operations in the broker.
When it reaches thres
The maximum number of pending topic load operations in the broker. Equal to
"maxConcurrentTopicLoadRequest" defined in broker.conf.
* Type: UpDownCounter
* Unit: `{operation}`
+
+### Web Executor Service metrics
+
+ pulsar.web.executor.thread.limit
Review Comment:
Yes, these would be the same metrics across the other components.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
(pulsar) branch lh-migrate-to-gradle-develocity updated (79a482b6311 -> bff6b4aa0fa)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch lh-migrate-to-gradle-develocity in repository https://gitbox.apache.org/repos/asf/pulsar.git from 79a482b6311 Test if GRADLE_ENTERPRISE_ACCESS_KEY is supported add bff6b4aa0fa Handle DEVELOCITY_ACCESS_KEY in enabled rule No new revisions were added by this update. Summary of changes: .mvn/develocity.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Re: [PR] [improve][ci] Migrate from Gradle Enterprise to Develocity [pulsar]
lhotari commented on code in PR #22880: URL: https://github.com/apache/pulsar/pull/22880#discussion_r1631654320 ## .gitignore: ## @@ -95,5 +95,5 @@ pulsar-client/src/test/java/org/apache/pulsar/client/avro/generated test-reports/ -# Gradle Enterprise -.mvn/.gradle-enterprise/ Review Comment: Done -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][ci] Migrate from Gradle Enterprise to Develocity [pulsar]
lhotari commented on code in PR #22880: URL: https://github.com/apache/pulsar/pull/22880#discussion_r1631651151 ## .mvn/develocity.xml: ## @@ -19,25 +19,22 @@ under the License. --> -https://www.gradle.com/gradle-enterprise-maven; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; -xsi:schemaLocation="https://www.gradle.com/gradle-enterprise-maven https://www.gradle.com/schema/gradle-enterprise-maven.xsd;> - -https://www.gradle.com/gradle-enterprise-maven; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; -xsi:schemaLocation="https://www.gradle.com/gradle-enterprise-maven https://www.gradle.com/schema/gradle-enterprise-maven.xsd;> -
(pulsar) branch lh-migrate-to-gradle-develocity updated (3bb53a773ec -> 79a482b6311)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch lh-migrate-to-gradle-develocity in repository https://gitbox.apache.org/repos/asf/pulsar.git discard 3bb53a773ec Migrate from Gradle Enterprise to Develocity add e31dbdd9782 [improve][build] Require Java 17 or Java 21 for building Pulsar (#22875) add 5a8db368785 [improve][broker] Reuse topic OpenTelemetry attributes (#22876) add e236f49f4a8 [fix][misc] Add proper nslookup (included in bind-tools) to docker image (#22878) add c24772c3af7 Migrate from Gradle Enterprise to Develocity add 79a482b6311 Test if GRADLE_ENTERPRISE_ACCESS_KEY is supported This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (3bb53a773ec) \ N -- N -- N refs/heads/lh-migrate-to-gradle-develocity (79a482b6311) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .gitignore | 2 + .mvn/develocity.xml| 3 +- docker/pulsar/Dockerfile | 5 +- pom.xml| 4 +- .../broker/service/PersistentTopicAttributes.java | 73 ++ .../org/apache/pulsar/broker/service/Topic.java| 5 ++ .../pulsar/broker/service/TopicAttributes.java | 44 + .../service/nonpersistent/NonPersistentTopic.java | 16 + .../broker/service/persistent/PersistentTopic.java | 14 + .../broker/stats/OpenTelemetryTopicStats.java | 54 .../opentelemetry/OpenTelemetryAttributes.java | 17 + 11 files changed, 190 insertions(+), 47 deletions(-) create mode 100644 pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PersistentTopicAttributes.java create mode 100644 pulsar-broker/src/main/java/org/apache/pulsar/broker/service/TopicAttributes.java
Re: [PR] [improve][ci] Migrate from Gradle Enterprise to Develocity [pulsar]
lhotari commented on code in PR #22880: URL: https://github.com/apache/pulsar/pull/22880#discussion_r1631651911 ## .gitignore: ## @@ -95,5 +95,5 @@ pulsar-client/src/test/java/org/apache/pulsar/client/avro/generated test-reports/ -# Gradle Enterprise -.mvn/.gradle-enterprise/ Review Comment: valid point -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][ci] Migrate from Gradle Enterprise to Develocity [pulsar]
lhotari commented on code in PR #22880: URL: https://github.com/apache/pulsar/pull/22880#discussion_r1631651151 ## .mvn/develocity.xml: ## @@ -19,25 +19,22 @@ under the License. --> -https://www.gradle.com/gradle-enterprise-maven; xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance; -xsi:schemaLocation="https://www.gradle.com/gradle-enterprise-maven https://www.gradle.com/schema/gradle-enterprise-maven.xsd;> -
Re: [PR] [improve][ci] Migrate from Gradle Enterprise to Develocity [pulsar]
lhotari commented on code in PR #22880:
URL: https://github.com/apache/pulsar/pull/22880#discussion_r1631650602
##
.mvn/develocity.xml:
##
@@ -19,25 +19,22 @@
under the License.
-->
-https://www.gradle.com/gradle-enterprise-maven;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
-xsi:schemaLocation="https://www.gradle.com/gradle-enterprise-maven
https://www.gradle.com/schema/gradle-enterprise-maven.xsd;>
-
#{env['GRADLE_ENTERPRISE_ACCESS_KEY']?.trim() > '' and
!(env['GITHUB_HEAD_REF']?.matches('(?i).*(experiment|wip|private).*') or
env['GITHUB_REPOSITORY']?.matches('(?i).*(experiment|wip|private).*'))}
https://ge.apache.org
false
+
#{env['GRADLE_ENTERPRISE_ACCESS_KEY'].substring(env['GRADLE_ENTERPRISE_ACCESS_KEY'].indexOf('=')
+ 1)}
- true
true
true
#{isFalse(env['GITHUB_ACTIONS'])}
-ALWAYS
-true
Review Comment:
[followed the migration
guide](https://docs.gradle.com/develocity/maven-extension/legacy/#develocity_migration)
and checked in IntelliJ that the new XML was following the new XML schema.
These elements have been removed / replaced and that's why I removed them.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
(pulsar) branch branch-3.3 updated: [fix][misc] Add proper nslookup (included in bind-tools) to docker image (#22878)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a commit to branch branch-3.3 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-3.3 by this push: new e0ae847ca75 [fix][misc] Add proper nslookup (included in bind-tools) to docker image (#22878) e0ae847ca75 is described below commit e0ae847ca75b5bb6066bd62f24e4ba3cfa537b49 Author: Lari Hotari AuthorDate: Fri Jun 7 23:07:01 2024 +0300 [fix][misc] Add proper nslookup (included in bind-tools) to docker image (#22878) (cherry picked from commit e236f49f4a8f3645eba3b755c6114b63585fa02e) --- docker/pulsar/Dockerfile | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docker/pulsar/Dockerfile b/docker/pulsar/Dockerfile index f3fea0e1e9d..b75519fa91a 100644 --- a/docker/pulsar/Dockerfile +++ b/docker/pulsar/Dockerfile @@ -81,7 +81,7 @@ FROM apachepulsar/glibc-base:2.38 as glibc FROM alpine:3.19.1 ENV LANG C.UTF-8 -# Install some utilities +# Install some utilities, some are required by Pulsar scripts RUN apk add --no-cache \ bash \ python3 \ @@ -91,7 +91,8 @@ RUN apk add --no-cache \ gcompat \ ca-certificates \ procps \ -curl +curl \ +bind-tools # Upgrade all packages to get latest versions with security fixes RUN apk upgrade --no-cache
[I] Pulsar Standalone: --wipe-data does not work with RocksDB backend in 3.2.3 [pulsar]
JamesTreleaven opened a new issue, #22881: URL: https://github.com/apache/pulsar/issues/22881 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Read release policy - [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker. ### Version OS="Ubuntu 23.04", Java Version = openjdk version "17.0.9", Pulsar Version(client+broker) = 3.2.3 ### Minimal reproduce step # Start Pulsar and create topic $ apache-pulsar-3.2.3/bin/pulsar standalone ### in second window $ apache-pulsar-3.2.3/bin/pulsar-client produce persistent://public/default/exampleTopic --num-produce 2 --messages "Hello Pulsar" $ apache-pulsar-3.2.3/bin/pulsar-admin persistent list public/default Then I stop 'pulsar standalone' the first window with a ctrl-c. Then I restart it: # Restart Pulsar and attempt to --wipe-data $ apache-pulsar-3.2.3/bin/pulsar standalone --wipe-data ### in second window $ apache-pulsar-3.2.3/bin/pulsar-admin persistent list public/default ### What did you expect to see? Empty output - zero lines outputted indicating no topics exist in public/default. ### What did you see instead? I saw: "persistent://public/default/exampleTopic" ### Anything else? This issue seems to be related to issue #16741 which is claimed to be fixed. I have these two environment variables set: export JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64 export PULSAR_CLIENT_CONF=/home/james/apache-pulsar-3.2.3/conf/client.conf I installed pulsar by downloading apache-pulsar-3.2.3-bin.tar.gz and unpacking it into /home/james. When I run 'pulsar standalone' it says "Starting BK with RocksDb metadata store". ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix][misc] Add proper nslookup (included in bind-tools) to docker image [pulsar]
lhotari merged PR #22878: URL: https://github.com/apache/pulsar/pull/22878 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] nslookup in apachepulsar/pulsar:3.3.0 isn't compatible with kubernetes search domains [pulsar]
lhotari closed issue #22877: [Bug] nslookup in apachepulsar/pulsar:3.3.0 isn't compatible with kubernetes search domains URL: https://github.com/apache/pulsar/issues/22877 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) branch master updated: [fix][misc] Add proper nslookup (included in bind-tools) to docker image (#22878)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new e236f49f4a8 [fix][misc] Add proper nslookup (included in bind-tools) to docker image (#22878) e236f49f4a8 is described below commit e236f49f4a8f3645eba3b755c6114b63585fa02e Author: Lari Hotari AuthorDate: Fri Jun 7 23:07:01 2024 +0300 [fix][misc] Add proper nslookup (included in bind-tools) to docker image (#22878) --- docker/pulsar/Dockerfile | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/docker/pulsar/Dockerfile b/docker/pulsar/Dockerfile index f3fea0e1e9d..b75519fa91a 100644 --- a/docker/pulsar/Dockerfile +++ b/docker/pulsar/Dockerfile @@ -81,7 +81,7 @@ FROM apachepulsar/glibc-base:2.38 as glibc FROM alpine:3.19.1 ENV LANG C.UTF-8 -# Install some utilities +# Install some utilities, some are required by Pulsar scripts RUN apk add --no-cache \ bash \ python3 \ @@ -91,7 +91,8 @@ RUN apk add --no-cache \ gcompat \ ca-certificates \ procps \ -curl +curl \ +bind-tools # Upgrade all packages to get latest versions with security fixes RUN apk upgrade --no-cache
Re: [PR] [fix][misc] Add proper nslookup (included in bind-tools) to docker image [pulsar]
codecov-commenter commented on PR #22878: URL: https://github.com/apache/pulsar/pull/22878#issuecomment-2155469549 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/22878?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report All modified and coverable lines are covered by tests :white_check_mark: > Project coverage is 73.29%. Comparing base [(`bbc6224`)](https://app.codecov.io/gh/apache/pulsar/commit/bbc62245c5ddba1de4b1e7cee4ab49334bc36277?dropdown=coverage=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) to head [(`9147edf`)](https://app.codecov.io/gh/apache/pulsar/commit/9147edf38c71dd2a8b4dfb622ae2d247981ec775?dropdown=coverage=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). > Report is 363 commits behind head on master. Additional details and impacted files [](https://app.codecov.io/gh/apache/pulsar/pull/22878?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) ```diff @@ Coverage Diff @@ ## master #22878 +/- ## - Coverage 73.57% 73.29% -0.28% - Complexity3262432702 +78 Files 1877 1889 +12 Lines139502 141885+2383 Branches 1529915565 +266 + Hits 102638 103997+1359 - Misses2890829877 +969 - Partials 7956 8011 +55 ``` | [Flag](https://app.codecov.io/gh/apache/pulsar/pull/22878/flags?src=pr=flags_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Coverage Δ | | |---|---|---| | [inttests](https://app.codecov.io/gh/apache/pulsar/pull/22878/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `27.20% <ø> (+2.62%)` | :arrow_up: | | [systests](https://app.codecov.io/gh/apache/pulsar/pull/22878/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `24.70% <ø> (+0.37%)` | :arrow_up: | | [unittests](https://app.codecov.io/gh/apache/pulsar/pull/22878/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `72.33% <ø> (-0.51%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more. [see 392 files with indirect coverage changes](https://app.codecov.io/gh/apache/pulsar/pull/22878/indirect-changes?src=pr=tree-more_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] Consumer with MaxDegreeOfParallelism eventually hangs [pulsar-dotpulsar]
smbecker opened a new issue, #223: URL: https://github.com/apache/pulsar-dotpulsar/issues/223 ### Description I have a batch process that when it kicks off, will fill a topic with around 100k messages in a very short period of time. I have a series of consumers that start pulling from that topic. If I use the default `MaxDegreeOfParallelism=1`, then everything finishes successfully as expected. I wanted to play around with how changes to `MaxDegreeOfParallelism` impact performance. My first test was using `MaxDegreeOfParallelism=10`. Unfortunately, after a few thousand messages, it just stopped consuming. I am still trying to work through where it is hung up. If I restart the process, it will pick back up and get through another few thousand before getting stuck again. Any idea where to start debugging this issue? I am using `MessagePrefetchCount=1` and `SubscriptionType=KeyShared` for what its worth. ### Reproduction Steps In my case, having a topic with tens of thousands of messages in backlog and trying to process it with `MaxDegreeOfParallelism=10`. ### Expected behavior It should process the messages as expected ### Actual behavior It stops processing messages after a few thousand or so ### Regression? _No response_ ### Known Workarounds Change `MaxDegreeOfParallelism=1` ### Configuration _No response_ ### Other information _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][ci] Migrate from Gradle Enterprise to Develocity [pulsar]
heesung-sn commented on code in PR #22880:
URL: https://github.com/apache/pulsar/pull/22880#discussion_r1631622170
##
.gitignore:
##
@@ -95,5 +95,5 @@
pulsar-client/src/test/java/org/apache/pulsar/client/avro/generated
test-reports/
-# Gradle Enterprise
-.mvn/.gradle-enterprise/
Review Comment:
should we include both instead of replacing it?
##
.mvn/develocity.xml:
##
@@ -19,25 +19,22 @@
under the License.
-->
-https://www.gradle.com/gradle-enterprise-maven;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
-xsi:schemaLocation="https://www.gradle.com/gradle-enterprise-maven
https://www.gradle.com/schema/gradle-enterprise-maven.xsd;>
-
-https://www.gradle.com/gradle-enterprise-maven;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
-xsi:schemaLocation="https://www.gradle.com/gradle-enterprise-maven
https://www.gradle.com/schema/gradle-enterprise-maven.xsd;>
-
#{env['GRADLE_ENTERPRISE_ACCESS_KEY']?.trim() > '' and
!(env['GITHUB_HEAD_REF']?.matches('(?i).*(experiment|wip|private).*') or
env['GITHUB_REPOSITORY']?.matches('(?i).*(experiment|wip|private).*'))}
https://ge.apache.org
false
+
#{env['GRADLE_ENTERPRISE_ACCESS_KEY'].substring(env['GRADLE_ENTERPRISE_ACCESS_KEY'].indexOf('=')
+ 1)}
- true
true
true
#{isFalse(env['GITHUB_ACTIONS'])}
-ALWAYS
-true
Review Comment:
nit: I am wondering why we need to update these configs? Could you explain a
little here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
(pulsar) branch branch-3.3 updated: [improve][broker] Reuse topic OpenTelemetry attributes (#22876)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new 3a5532d7f1d [improve][broker] Reuse topic OpenTelemetry attributes
(#22876)
3a5532d7f1d is described below
commit 3a5532d7f1ddd786f742049ca9942564a67f7768
Author: Dragos Misca
AuthorDate: Fri Jun 7 12:13:04 2024 -0700
[improve][broker] Reuse topic OpenTelemetry attributes (#22876)
(cherry picked from commit 5a8db3687852c21b7d533a64e98563fa323a0897)
---
.../broker/service/PersistentTopicAttributes.java | 73 ++
.../org/apache/pulsar/broker/service/Topic.java| 5 ++
.../pulsar/broker/service/TopicAttributes.java | 44 +
.../service/nonpersistent/NonPersistentTopic.java | 16 +
.../broker/service/persistent/PersistentTopic.java | 14 +
.../broker/stats/OpenTelemetryTopicStats.java | 54
.../opentelemetry/OpenTelemetryAttributes.java | 17 +
7 files changed, 182 insertions(+), 41 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PersistentTopicAttributes.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PersistentTopicAttributes.java
new file mode 100644
index 000..048edafe884
--- /dev/null
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PersistentTopicAttributes.java
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.service;
+
+import io.opentelemetry.api.common.Attributes;
+import lombok.Getter;
+import org.apache.pulsar.common.naming.TopicName;
+import org.apache.pulsar.opentelemetry.OpenTelemetryAttributes;
+
+@Getter
+public class PersistentTopicAttributes extends TopicAttributes {
+
+private final Attributes timeBasedQuotaAttributes;
+private final Attributes sizeBasedQuotaAttributes;
+
+private final Attributes compactionSuccessAttributes;
+private final Attributes compactionFailureAttributes;
+
+private final Attributes transactionActiveAttributes;
+private final Attributes transactionCommittedAttributes;
+private final Attributes transactionAbortedAttributes;
+
+public PersistentTopicAttributes(TopicName topicName) {
+super(topicName);
+
+timeBasedQuotaAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.BacklogQuotaType.TIME.attributes)
+.build();
+sizeBasedQuotaAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.BacklogQuotaType.SIZE.attributes)
+.build();
+
+transactionActiveAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.TransactionStatus.ACTIVE.attributes)
+.build();
+transactionCommittedAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.TransactionStatus.COMMITTED.attributes)
+.build();
+transactionAbortedAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.TransactionStatus.ABORTED.attributes)
+.build();
+
+compactionSuccessAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.CompactionStatus.SUCCESS.attributes)
+.build();
+compactionFailureAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.CompactionStatus.FAILURE.attributes)
+.build();
+}
+}
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Topic.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Topic.java
index e902de8a45a..50a28c79792 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Topic.java
(pulsar) branch lh-migrate-to-gradle-develocity updated (d488fdb81a5 -> 3bb53a773ec)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch lh-migrate-to-gradle-develocity in repository https://gitbox.apache.org/repos/asf/pulsar.git discard d488fdb81a5 Migrate from Gradle Enterprise to Develocity add 3bb53a773ec Migrate from Gradle Enterprise to Develocity This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (d488fdb81a5) \ N -- N -- N refs/heads/lh-migrate-to-gradle-develocity (3bb53a773ec) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .mvn/develocity.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(pulsar) branch master updated: [improve][broker] Reuse topic OpenTelemetry attributes (#22876)
This is an automated email from the ASF dual-hosted git repository.
heesung pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 5a8db368785 [improve][broker] Reuse topic OpenTelemetry attributes
(#22876)
5a8db368785 is described below
commit 5a8db3687852c21b7d533a64e98563fa323a0897
Author: Dragos Misca
AuthorDate: Fri Jun 7 12:13:04 2024 -0700
[improve][broker] Reuse topic OpenTelemetry attributes (#22876)
---
.../broker/service/PersistentTopicAttributes.java | 73 ++
.../org/apache/pulsar/broker/service/Topic.java| 5 ++
.../pulsar/broker/service/TopicAttributes.java | 44 +
.../service/nonpersistent/NonPersistentTopic.java | 16 +
.../broker/service/persistent/PersistentTopic.java | 14 +
.../broker/stats/OpenTelemetryTopicStats.java | 54
.../opentelemetry/OpenTelemetryAttributes.java | 17 +
7 files changed, 182 insertions(+), 41 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PersistentTopicAttributes.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PersistentTopicAttributes.java
new file mode 100644
index 000..048edafe884
--- /dev/null
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/PersistentTopicAttributes.java
@@ -0,0 +1,73 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+package org.apache.pulsar.broker.service;
+
+import io.opentelemetry.api.common.Attributes;
+import lombok.Getter;
+import org.apache.pulsar.common.naming.TopicName;
+import org.apache.pulsar.opentelemetry.OpenTelemetryAttributes;
+
+@Getter
+public class PersistentTopicAttributes extends TopicAttributes {
+
+private final Attributes timeBasedQuotaAttributes;
+private final Attributes sizeBasedQuotaAttributes;
+
+private final Attributes compactionSuccessAttributes;
+private final Attributes compactionFailureAttributes;
+
+private final Attributes transactionActiveAttributes;
+private final Attributes transactionCommittedAttributes;
+private final Attributes transactionAbortedAttributes;
+
+public PersistentTopicAttributes(TopicName topicName) {
+super(topicName);
+
+timeBasedQuotaAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.BacklogQuotaType.TIME.attributes)
+.build();
+sizeBasedQuotaAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.BacklogQuotaType.SIZE.attributes)
+.build();
+
+transactionActiveAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.TransactionStatus.ACTIVE.attributes)
+.build();
+transactionCommittedAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.TransactionStatus.COMMITTED.attributes)
+.build();
+transactionAbortedAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.TransactionStatus.ABORTED.attributes)
+.build();
+
+compactionSuccessAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.CompactionStatus.SUCCESS.attributes)
+.build();
+compactionFailureAttributes = Attributes.builder()
+.putAll(commonAttributes)
+
.putAll(OpenTelemetryAttributes.CompactionStatus.FAILURE.attributes)
+.build();
+}
+}
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Topic.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Topic.java
index e902de8a45a..50a28c79792 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Topic.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Topic.java
@@ -384,4
Re: [I] Excessive memory allocation in OTel broker metrics [pulsar]
heesung-sn closed issue #22817: Excessive memory allocation in OTel broker metrics URL: https://github.com/apache/pulsar/issues/22817 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] Reuse topic OpenTelemetry attributes [pulsar]
heesung-sn merged PR #22876: URL: https://github.com/apache/pulsar/pull/22876 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) branch lh-migrate-to-gradle-develocity updated (8080c7fa659 -> d488fdb81a5)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch lh-migrate-to-gradle-develocity in repository https://gitbox.apache.org/repos/asf/pulsar.git discard 8080c7fa659 Migrate from Gradle Enterprise to Develocity add d488fdb81a5 Migrate from Gradle Enterprise to Develocity This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (8080c7fa659) \ N -- N -- N refs/heads/lh-migrate-to-gradle-develocity (d488fdb81a5) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .mvn/develocity.xml | 1 + 1 file changed, 1 insertion(+)
(pulsar) branch lh-migrate-to-gradle-develocity updated (7f1134c0101 -> 8080c7fa659)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch lh-migrate-to-gradle-develocity in repository https://gitbox.apache.org/repos/asf/pulsar.git discard 7f1134c0101 Migrate from Gradle Enterprise to Develocity add 8080c7fa659 Migrate from Gradle Enterprise to Develocity This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (7f1134c0101) \ N -- N -- N refs/heads/lh-migrate-to-gradle-develocity (8080c7fa659) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. No new revisions were added by this update. Summary of changes: .mvn/develocity.xml | 1 - 1 file changed, 1 deletion(-)
Re: [PR] [improve][broker] Reuse topic OpenTelemetry attributes [pulsar]
codecov-commenter commented on PR #22876: URL: https://github.com/apache/pulsar/pull/22876#issuecomment-2155376310 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/22876?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report Attention: Patch coverage is `94.25287%` with `5 lines` in your changes missing coverage. Please review. > Project coverage is 73.25%. Comparing base [(`bbc6224`)](https://app.codecov.io/gh/apache/pulsar/commit/bbc62245c5ddba1de4b1e7cee4ab49334bc36277?dropdown=coverage=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) to head [(`e30dae9`)](https://app.codecov.io/gh/apache/pulsar/commit/e30dae9e49a9edad3c8cebf0a54114b583133e4d?dropdown=coverage=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). > Report is 362 commits behind head on master. Additional details and impacted files [](https://app.codecov.io/gh/apache/pulsar/pull/22876?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) ```diff @@ Coverage Diff @@ ## master #22876 +/- ## - Coverage 73.57% 73.25% -0.33% - Complexity3262432994 +370 Files 1877 1891 +14 Lines139502 141935+2433 Branches 1529915565 +266 + Hits 102638 103970+1332 - Misses2890829939+1031 - Partials 7956 8026 +70 ``` | [Flag](https://app.codecov.io/gh/apache/pulsar/pull/22876/flags?src=pr=flags_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Coverage Δ | | |---|---|---| | [inttests](https://app.codecov.io/gh/apache/pulsar/pull/22876/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `27.25% <87.35%> (+2.67%)` | :arrow_up: | | [systests](https://app.codecov.io/gh/apache/pulsar/pull/22876/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `24.66% <2.29%> (+0.34%)` | :arrow_up: | | [unittests](https://app.codecov.io/gh/apache/pulsar/pull/22876/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `72.27% <94.25%> (-0.57%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more. | [Files](https://app.codecov.io/gh/apache/pulsar/pull/22876?dropdown=coverage=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Coverage Δ | | |---|---|---| | [...lsar/broker/service/PersistentTopicAttributes.java](https://app.codecov.io/gh/apache/pulsar/pull/22876?src=pr=tree=pulsar-broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpulsar%2Fbroker%2Fservice%2FPersistentTopicAttributes.java_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1BlcnNpc3RlbnRUb3BpY0F0dHJpYnV0ZXMuamF2YQ==) | `100.00% <100.00%> (ø)` | | | [...n/java/org/apache/pulsar/broker/service/Topic.java](https://app.codecov.io/gh/apache/pulsar/pull/22876?src=pr=tree=pulsar-broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpulsar%2Fbroker%2Fservice%2FTopic.java_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1RvcGljLmphdmE=) | `33.33% <ø> (-3.04%)` | :arrow_down: | | [.../apache/pulsar/broker/service/TopicAttributes.java](https://app.codecov.io/gh/apache/pulsar/pull/22876?src=pr=tree=pulsar-broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Fpulsar%2Fbroker%2Fservice%2FTopicAttributes.java_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#diff-cHVsc2FyLWJyb2tlci9zcmMvbWFpbi9qYXZhL29yZy9hcGFjaGUvcHVsc2FyL2Jyb2tlci9zZXJ2aWNlL1RvcGljQXR0cmlidXRlcy5qYXZh) | `100.00% <100.00%> (ø)` | | |
Re: [I] [Bug] maven build fails with Java 22 [pulsar]
lhotari closed issue #22874: [Bug] maven build fails with Java 22 URL: https://github.com/apache/pulsar/issues/22874 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][build] Require Java 17 or Java 21 for building Pulsar [pulsar]
lhotari merged PR #22875: URL: https://github.com/apache/pulsar/pull/22875 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) branch master updated (5af05951754 -> e31dbdd9782)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git from 5af05951754 [fix][broker] Support advertised listeners when gracefully transferring topics (ExtensibleLoadManagerImpl only) (#22862) add e31dbdd9782 [improve][build] Require Java 17 or Java 21 for building Pulsar (#22875) No new revisions were added by this update. Summary of changes: pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
(pulsar) branch lh-migrate-to-gradle-develocity updated (46861765c97 -> 7f1134c0101)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch lh-migrate-to-gradle-develocity in repository https://gitbox.apache.org/repos/asf/pulsar.git discard 46861765c97 Migrate from Gradle Enterprise to Develocity new 7f1134c0101 Migrate from Gradle Enterprise to Develocity This update added new revisions after undoing existing revisions. That is to say, some revisions that were in the old version of the branch are not in the new version. This situation occurs when a user --force pushes a change and generates a repository containing something like this: * -- * -- B -- O -- O -- O (46861765c97) \ N -- N -- N refs/heads/lh-migrate-to-gradle-develocity (7f1134c0101) You should already have received notification emails for all of the O revisions, and so the following emails describe only the N revisions from the common base, B. Any revisions marked "omit" are not gone; other references still refer to them. Any revisions marked "discard" are gone forever. The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .gitignore | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
(pulsar) 01/01: Migrate from Gradle Enterprise to Develocity
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch lh-migrate-to-gradle-develocity
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 7f1134c0101af44b367fe27dec06f78fdbc46ef4
Author: Lari Hotari
AuthorDate: Fri Jun 7 21:41:43 2024 +0300
Migrate from Gradle Enterprise to Develocity
---
.gitignore | 4 ++--
...-user-data.groovy => develocity-custom-user-data.groovy} | 0
.mvn/{gradle-enterprise.xml => develocity.xml} | 13 +
.mvn/extensions.xml | 6 +++---
4 files changed, 10 insertions(+), 13 deletions(-)
diff --git a/.gitignore b/.gitignore
index cd00c442000..3ff6d91e58e 100644
--- a/.gitignore
+++ b/.gitignore
@@ -95,5 +95,5 @@
pulsar-client/src/test/java/org/apache/pulsar/client/avro/generated
test-reports/
-# Gradle Enterprise
-.mvn/.gradle-enterprise/
+# Gradle Develocity
+.mvn/.develocity/
diff --git a/.mvn/gradle-enterprise-custom-user-data.groovy
b/.mvn/develocity-custom-user-data.groovy
similarity index 100%
rename from .mvn/gradle-enterprise-custom-user-data.groovy
rename to .mvn/develocity-custom-user-data.groovy
diff --git a/.mvn/gradle-enterprise.xml b/.mvn/develocity.xml
similarity index 77%
rename from .mvn/gradle-enterprise.xml
rename to .mvn/develocity.xml
index b9ae41d5799..004d757d410 100644
--- a/.mvn/gradle-enterprise.xml
+++ b/.mvn/develocity.xml
@@ -19,25 +19,22 @@
under the License.
-->
-https://www.gradle.com/gradle-enterprise-maven;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
-xsi:schemaLocation="https://www.gradle.com/gradle-enterprise-maven
https://www.gradle.com/schema/gradle-enterprise-maven.xsd;>
-
#{env['GRADLE_ENTERPRISE_ACCESS_KEY']?.trim() > '' and
!(env['GITHUB_HEAD_REF']?.matches('(?i).*(experiment|wip|private).*') or
env['GITHUB_REPOSITORY']?.matches('(?i).*(experiment|wip|private).*'))}
https://ge.apache.org
false
+#{env['GRADLE_ENTERPRISE_ACCESS_KEY']}
- true
true
true
#{isFalse(env['GITHUB_ACTIONS'])}
-ALWAYS
-true
#{{'0.0.0.0'}}
@@ -50,4 +47,4 @@
false
-
+
\ No newline at end of file
diff --git a/.mvn/extensions.xml b/.mvn/extensions.xml
index bc051debf61..4a2117925f1 100644
--- a/.mvn/extensions.xml
+++ b/.mvn/extensions.xml
@@ -23,12 +23,12 @@
xsi:schemaLocation="http://maven.apache.org/EXTENSIONS/1.0.0
http://maven.apache.org/xsd/core-extensions-1.0.0.xsd;>
com.gradle
-gradle-enterprise-maven-extension
-1.20.1
+develocity-maven-extension
+1.21.4
com.gradle
common-custom-user-data-maven-extension
-1.12.5
+2.0
(pulsar) branch lh-migrate-to-gradle-develocity created (now 46861765c97)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch lh-migrate-to-gradle-develocity in repository https://gitbox.apache.org/repos/asf/pulsar.git at 46861765c97 Migrate from Gradle Enterprise to Develocity This branch includes the following new commits: new 46861765c97 Migrate from Gradle Enterprise to Develocity The 1 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference.
(pulsar) 01/01: Migrate from Gradle Enterprise to Develocity
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch lh-migrate-to-gradle-develocity
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 46861765c97e6597ede55dd481ed14c9a81281c2
Author: Lari Hotari
AuthorDate: Fri Jun 7 21:41:43 2024 +0300
Migrate from Gradle Enterprise to Develocity
---
.gitignore | 1 +
...-user-data.groovy => develocity-custom-user-data.groovy} | 0
.mvn/{gradle-enterprise.xml => develocity.xml} | 13 +
.mvn/extensions.xml | 6 +++---
4 files changed, 9 insertions(+), 11 deletions(-)
diff --git a/.gitignore b/.gitignore
index cd00c442000..1248a6fb3fa 100644
--- a/.gitignore
+++ b/.gitignore
@@ -97,3 +97,4 @@ test-reports/
# Gradle Enterprise
.mvn/.gradle-enterprise/
+.mvn/.develocity/
diff --git a/.mvn/gradle-enterprise-custom-user-data.groovy
b/.mvn/develocity-custom-user-data.groovy
similarity index 100%
rename from .mvn/gradle-enterprise-custom-user-data.groovy
rename to .mvn/develocity-custom-user-data.groovy
diff --git a/.mvn/gradle-enterprise.xml b/.mvn/develocity.xml
similarity index 77%
rename from .mvn/gradle-enterprise.xml
rename to .mvn/develocity.xml
index b9ae41d5799..004d757d410 100644
--- a/.mvn/gradle-enterprise.xml
+++ b/.mvn/develocity.xml
@@ -19,25 +19,22 @@
under the License.
-->
-https://www.gradle.com/gradle-enterprise-maven;
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
-xsi:schemaLocation="https://www.gradle.com/gradle-enterprise-maven
https://www.gradle.com/schema/gradle-enterprise-maven.xsd;>
-
#{env['GRADLE_ENTERPRISE_ACCESS_KEY']?.trim() > '' and
!(env['GITHUB_HEAD_REF']?.matches('(?i).*(experiment|wip|private).*') or
env['GITHUB_REPOSITORY']?.matches('(?i).*(experiment|wip|private).*'))}
https://ge.apache.org
false
+#{env['GRADLE_ENTERPRISE_ACCESS_KEY']}
- true
true
true
#{isFalse(env['GITHUB_ACTIONS'])}
-ALWAYS
-true
#{{'0.0.0.0'}}
@@ -50,4 +47,4 @@
false
-
+
\ No newline at end of file
diff --git a/.mvn/extensions.xml b/.mvn/extensions.xml
index bc051debf61..4a2117925f1 100644
--- a/.mvn/extensions.xml
+++ b/.mvn/extensions.xml
@@ -23,12 +23,12 @@
xsi:schemaLocation="http://maven.apache.org/EXTENSIONS/1.0.0
http://maven.apache.org/xsd/core-extensions-1.0.0.xsd;>
com.gradle
-gradle-enterprise-maven-extension
-1.20.1
+develocity-maven-extension
+1.21.4
com.gradle
common-custom-user-data-maven-extension
-1.12.5
+2.0
Re: [PR] [improve][build] Require Java 17 or Java 21 for building Pulsar [pulsar]
codecov-commenter commented on PR #22875: URL: https://github.com/apache/pulsar/pull/22875#issuecomment-2155339921 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/22875?dropdown=coverage=pr=h1_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) Report All modified and coverable lines are covered by tests :white_check_mark: > Project coverage is 73.26%. Comparing base [(`bbc6224`)](https://app.codecov.io/gh/apache/pulsar/commit/bbc62245c5ddba1de4b1e7cee4ab49334bc36277?dropdown=coverage=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) to head [(`794be98`)](https://app.codecov.io/gh/apache/pulsar/commit/794be98f8ea58b4b566ea3004a826f88e5453714?dropdown=coverage=desc_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache). > Report is 361 commits behind head on master. Additional details and impacted files [](https://app.codecov.io/gh/apache/pulsar/pull/22875?src=pr=tree_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) ```diff @@ Coverage Diff @@ ## master #22875 +/- ## - Coverage 73.57% 73.26% -0.32% - Complexity3262432680 +56 Files 1877 1889 +12 Lines139502 141871+2369 Branches 1529915565 +266 + Hits 102638 103941+1303 - Misses2890829898 +990 - Partials 7956 8032 +76 ``` | [Flag](https://app.codecov.io/gh/apache/pulsar/pull/22875/flags?src=pr=flags_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | Coverage Δ | | |---|---|---| | [inttests](https://app.codecov.io/gh/apache/pulsar/pull/22875/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `27.45% <ø> (+2.87%)` | :arrow_up: | | [systests](https://app.codecov.io/gh/apache/pulsar/pull/22875/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `24.75% <ø> (+0.43%)` | :arrow_up: | | [unittests](https://app.codecov.io/gh/apache/pulsar/pull/22875/flags?src=pr=flag_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) | `72.29% <ø> (-0.56%)` | :arrow_down: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache#carryforward-flags-in-the-pull-request-comment) to find out more. [see 389 files with indirect coverage changes](https://app.codecov.io/gh/apache/pulsar/pull/22875/indirect-changes?src=pr=tree-more_medium=referral_source=github_content=comment_campaign=pr+comments_term=apache) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) 02/06: [improve][broker] Reduce number of OpenTelemetry consumer attributes (#22837)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit da5329d0adfac8d2fa500cec7f8e99f43f9510aa
Author: Dragos Misca
AuthorDate: Tue Jun 4 12:57:21 2024 -0700
[improve][broker] Reduce number of OpenTelemetry consumer attributes
(#22837)
(cherry picked from commit 8276f218f576e81c212cedf8b3691f7c1a654e0e)
---
.../org/apache/pulsar/broker/service/Consumer.java | 36 +++
.../broker/stats/OpenTelemetryConsumerStats.java | 54 ++
.../stats/OpenTelemetryConsumerStatsTest.java | 34 +-
.../opentelemetry/OpenTelemetryAttributes.java | 5 --
4 files changed, 51 insertions(+), 78 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
index fe9fbe6a400..1b19a408124 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
@@ -25,6 +25,7 @@ import com.google.common.base.MoreObjects;
import com.google.common.util.concurrent.AtomicDouble;
import io.netty.util.concurrent.Future;
import io.netty.util.concurrent.Promise;
+import io.opentelemetry.api.common.Attributes;
import java.time.Instant;
import java.util.ArrayList;
import java.util.BitSet;
@@ -35,6 +36,7 @@ import java.util.Objects;
import java.util.Optional;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.atomic.AtomicIntegerFieldUpdater;
+import java.util.concurrent.atomic.AtomicReferenceFieldUpdater;
import java.util.concurrent.atomic.LongAdder;
import java.util.stream.Collectors;
import lombok.Getter;
@@ -69,6 +71,7 @@ import org.apache.pulsar.common.util.FutureUtil;
import org.apache.pulsar.common.util.collections.BitSetRecyclable;
import org.apache.pulsar.common.util.collections.ConcurrentLongLongPairHashMap;
import
org.apache.pulsar.common.util.collections.ConcurrentLongLongPairHashMap.LongPair;
+import org.apache.pulsar.opentelemetry.OpenTelemetryAttributes;
import
org.apache.pulsar.transaction.common.exception.TransactionConflictException;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
@@ -158,6 +161,10 @@ public class Consumer {
@Getter
private final Instant connectedSince = Instant.now();
+private volatile Attributes openTelemetryAttributes;
+private static final AtomicReferenceFieldUpdater
OPEN_TELEMETRY_ATTRIBUTES_FIELD_UPDATER =
+AtomicReferenceFieldUpdater.newUpdater(Consumer.class,
Attributes.class, "openTelemetryAttributes");
+
public Consumer(Subscription subscription, SubType subType, String
topicName, long consumerId,
int priorityLevel, String consumerName,
boolean isDurable, TransportCnx cnx, String appId,
@@ -230,6 +237,8 @@ public class Consumer {
.getPulsar().getConfiguration().isAcknowledgmentAtBatchIndexLevelEnabled();
this.schemaType = schemaType;
+
+OPEN_TELEMETRY_ATTRIBUTES_FIELD_UPDATER.set(this, null);
}
@VisibleForTesting
@@ -262,6 +271,7 @@ public class Consumer {
this.isAcknowledgmentAtBatchIndexLevelEnabled = false;
this.schemaType = null;
MESSAGE_PERMITS_UPDATER.set(this, availablePermits);
+OPEN_TELEMETRY_ATTRIBUTES_FIELD_UPDATER.set(this, null);
}
public SubType subType() {
@@ -1202,4 +1212,30 @@ public class Consumer {
}
private static final Logger log = LoggerFactory.getLogger(Consumer.class);
+
+public Attributes getOpenTelemetryAttributes() {
+if (openTelemetryAttributes != null) {
+return openTelemetryAttributes;
+}
+return OPEN_TELEMETRY_ATTRIBUTES_FIELD_UPDATER.updateAndGet(this,
oldValue -> {
+if (oldValue != null) {
+return oldValue;
+}
+var topicName = TopicName.get(subscription.getTopic().getName());
+
+var builder = Attributes.builder()
+.put(OpenTelemetryAttributes.PULSAR_CONSUMER_NAME,
consumerName)
+.put(OpenTelemetryAttributes.PULSAR_CONSUMER_ID,
consumerId)
+.put(OpenTelemetryAttributes.PULSAR_SUBSCRIPTION_NAME,
subscription.getName())
+.put(OpenTelemetryAttributes.PULSAR_SUBSCRIPTION_TYPE,
subType.toString())
+.put(OpenTelemetryAttributes.PULSAR_DOMAIN,
topicName.getDomain().toString())
+.put(OpenTelemetryAttributes.PULSAR_TENANT,
topicName.getTenant())
+.put(OpenTelemetryAttributes.PULSAR_NAMESPACE,
topicName.getNamespace())
+.put(OpenTelemetryAttributes.PULSAR_TOPIC,
topicName.getPartitionedTopicName());
+if (topicName.isPartitioned()) {
+
(pulsar) 03/06: [improve][ci] Add arm64 image build (#22755)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 999f18aa3afd0557cc73d0f4ceb70af0c143ee60
Author: Zixuan Liu
AuthorDate: Thu Jun 6 16:11:00 2024 +0800
[improve][ci] Add arm64 image build (#22755)
Signed-off-by: Zixuan Liu
(cherry picked from commit fb03d159129d10fdf1c277ba7b040025bed2ca61)
---
.github/workflows/pulsar-ci.yaml | 15 ++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/.github/workflows/pulsar-ci.yaml b/.github/workflows/pulsar-ci.yaml
index c15d51f9cfc..1160a0d1ec3 100644
--- a/.github/workflows/pulsar-ci.yaml
+++ b/.github/workflows/pulsar-ci.yaml
@@ -465,6 +465,12 @@ jobs:
timeout-minutes: 60
needs: ['preconditions', 'build-and-license-check']
if: ${{ needs.preconditions.outputs.docs_only != 'true'}}
+strategy:
+ fail-fast: false
+ matrix:
+platform:
+ - linux/amd64
+ - linux/arm64
env:
GRADLE_ENTERPRISE_ACCESS_KEY: ${{ secrets.GE_ACCESS_TOKEN }}
CI_JDK_MAJOR_VERSION: ${{ needs.preconditions.outputs.jdk_major_version
}}
@@ -509,14 +515,21 @@ jobs:
cd $HOME
$GITHUB_WORKSPACE/build/pulsar_ci_tool.sh
restore_tar_from_github_actions_artifacts pulsar-maven-repository-binaries
- - name: Build java-test-image docker image
+ - name: Set up QEMU
+uses: docker/setup-qemu-action@v3
+with:
+ platforms: arm64
+
+ - name: Build java-test-image docker image - ${{ matrix.platform }}
run: |
# build docker image
DOCKER_CLI_EXPERIMENTAL=enabled mvn -B -am -pl
docker/pulsar,tests/docker-images/java-test-image install
-Pcore-modules,-main,integrationTests,docker \
+ -Ddocker.platforms=${{ matrix.platform }} \
-Dmaven.test.skip=true -DskipSourceReleaseAssembly=true \
-Dspotbugs.skip=true -Dlicense.skip=true -Dcheckstyle.skip=true
-Drat.skip=true
- name: save docker image apachepulsar/java-test-image:latest to Github
artifact cache
+if: ${{ matrix.platform == 'linux/amd64' }}
run: |
$GITHUB_WORKSPACE/build/pulsar_ci_tool.sh
docker_save_image_to_github_actions_artifacts
apachepulsar/java-test-image:latest pulsar-java-test-image
(pulsar) branch branch-3.3 updated (9677d569a69 -> e1b3c303fea)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch branch-3.3 in repository https://gitbox.apache.org/repos/asf/pulsar.git from 9677d569a69 [fix] [broker] fix topic partitions was expanded even if disabled topic level replication (#22769) new c147fcf0095 [feat][broker] PIP-264: Add OpenTelemetry consumer metrics (#22693) new da5329d0adf [improve][broker] Reduce number of OpenTelemetry consumer attributes (#22837) new 999f18aa3af [improve][ci] Add arm64 image build (#22755) new cb0a211a6f2 [fix][cli] Fix the shell script parameter passthrough syntax (#22867) new aaccb00cbe5 [fix][cli] Fix Pulsar standalone shutdown - bkCluster wasn't closed (#22868) new e1b3c303fea [fix][cli] Fix healthcheck script pulsar-zookeeper-ruok.sh (#22873) The 6 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .github/workflows/pulsar-ci.yaml | 15 ++- bin/bookkeeper | 12 +- bin/pulsar | 38 +++--- docker/pulsar/scripts/pulsar-zookeeper-ruok.sh | 2 +- .../java/org/apache/pulsar/PulsarStandalone.java | 10 ++ .../org/apache/pulsar/PulsarStandaloneStarter.java | 58 +++-- .../org/apache/pulsar/broker/PulsarService.java| 8 ++ .../org/apache/pulsar/broker/service/Consumer.java | 68 +- .../broker/stats/OpenTelemetryConsumerStats.java | 142 + .../org/apache/pulsar/PulsarStandaloneTest.java| 48 ++- .../stats/OpenTelemetryConsumerStatsTest.java | 121 ++ .../broker/testcontext/PulsarTestContext.java | 1 + .../pulsar/client/api/BrokerServiceLookupTest.java | 1 + .../configurations/pulsar_broker_test.conf | 26 ++-- .../pulsar_broker_test_standalone.conf | 26 ++-- ...pulsar_broker_test_standalone_with_rocksdb.conf | 26 ++-- .../configurations/standalone_no_client_auth.conf | 4 +- .../pulsar/metadata/bookkeeper/BKCluster.java | 43 --- .../opentelemetry/OpenTelemetryAttributes.java | 41 ++ 19 files changed, 588 insertions(+), 102 deletions(-) create mode 100644 pulsar-broker/src/main/java/org/apache/pulsar/broker/stats/OpenTelemetryConsumerStats.java create mode 100644 pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/OpenTelemetryConsumerStatsTest.java
(pulsar) 04/06: [fix][cli] Fix the shell script parameter passthrough syntax (#22867)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit cb0a211a6f206e8aa220f3919f17915558960292
Author: Lari Hotari
AuthorDate: Fri Jun 7 15:25:35 2024 +0300
[fix][cli] Fix the shell script parameter passthrough syntax (#22867)
(cherry picked from commit c81c0f684f8c55c2e39739c6e1de935dff2085d6)
---
bin/bookkeeper | 12 ++--
bin/pulsar | 38 +++---
2 files changed, 25 insertions(+), 25 deletions(-)
diff --git a/bin/bookkeeper b/bin/bookkeeper
index 0cc07dd49ab..13d092f4c99 100755
--- a/bin/bookkeeper
+++ b/bin/bookkeeper
@@ -214,20 +214,20 @@ OPTS="$OPTS $BK_METADATA_OPTIONS"
#Change to BK_HOME to support relative paths
cd "$BK_HOME"
if [ $COMMAND == "bookie" ]; then
-exec $JAVA $OPTS $JMX_ARGS org.apache.bookkeeper.server.Main --conf
$BOOKIE_CONF $@
+exec $JAVA $OPTS $JMX_ARGS org.apache.bookkeeper.server.Main --conf
$BOOKIE_CONF "$@"
elif [ $COMMAND == "autorecovery" ]; then
-exec $JAVA $OPTS $JMX_ARGS
org.apache.bookkeeper.replication.AutoRecoveryMain --conf $BOOKIE_CONF $@
+exec $JAVA $OPTS $JMX_ARGS
org.apache.bookkeeper.replication.AutoRecoveryMain --conf $BOOKIE_CONF "$@"
elif [ $COMMAND == "localbookie" ]; then
NUMBER=$1
shift
-exec $JAVA $OPTS $JMX_ARGS org.apache.bookkeeper.util.LocalBookKeeper
$NUMBER $BOOKIE_CONF $@
+exec $JAVA $OPTS $JMX_ARGS org.apache.bookkeeper.util.LocalBookKeeper
$NUMBER $BOOKIE_CONF "$@"
elif [ $COMMAND == "upgrade" ]; then
-exec $JAVA $OPTS org.apache.bookkeeper.bookie.FileSystemUpgrade --conf
$BOOKIE_CONF $@
+exec $JAVA $OPTS org.apache.bookkeeper.bookie.FileSystemUpgrade --conf
$BOOKIE_CONF "$@"
elif [ $COMMAND == "shell" ]; then
ENTRY_FORMATTER_ARG="-DentryFormatterClass=${ENTRY_FORMATTER_CLASS:-org.apache.bookkeeper.util.StringEntryFormatter}"
-exec $JAVA $OPTS $ENTRY_FORMATTER_ARG
org.apache.bookkeeper.bookie.BookieShell -conf $BOOKIE_CONF $@
+exec $JAVA $OPTS $ENTRY_FORMATTER_ARG
org.apache.bookkeeper.bookie.BookieShell -conf $BOOKIE_CONF "$@"
elif [ $COMMAND == "help" -o $COMMAND == "--help" -o $COMMAND == "-h" ]; then
bookkeeper_help;
else
-exec $JAVA $OPTS $COMMAND $@
+exec $JAVA $OPTS $COMMAND "$@"
fi
diff --git a/bin/pulsar b/bin/pulsar
index ab0029af5b0..f6061601d88 100755
--- a/bin/pulsar
+++ b/bin/pulsar
@@ -329,56 +329,56 @@ fi
cd "$PULSAR_HOME"
if [ $COMMAND == "broker" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"pulsar-broker.log"}
-exec $JAVA $LOG4J2_SHUTDOWN_HOOK_DISABLED $OPTS
-Dpulsar.log.file=$PULSAR_LOG_FILE org.apache.pulsar.PulsarBrokerStarter
--broker-conf $PULSAR_BROKER_CONF $@
+exec $JAVA $LOG4J2_SHUTDOWN_HOOK_DISABLED $OPTS
-Dpulsar.log.file=$PULSAR_LOG_FILE org.apache.pulsar.PulsarBrokerStarter
--broker-conf $PULSAR_BROKER_CONF "$@"
elif [ $COMMAND == "bookie" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"bookkeeper.log"}
-exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.bookkeeper.server.Main --conf $PULSAR_BOOKKEEPER_CONF $@
+exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.bookkeeper.server.Main --conf $PULSAR_BOOKKEEPER_CONF "$@"
elif [ $COMMAND == "zookeeper" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"zookeeper.log"}
-exec $JAVA ${ZK_OPTS} $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain $PULSAR_ZK_CONF $@
+exec $JAVA ${ZK_OPTS} $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain $PULSAR_ZK_CONF "$@"
elif [ $COMMAND == "global-zookeeper" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"global-zookeeper.log"}
# Allow global ZK to turn into read-only mode when it cannot reach the
quorum
OPTS="${OPTS} ${ZK_OPTS} -Dreadonlymode.enabled=true"
-exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain $PULSAR_GLOBAL_ZK_CONF $@
+exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain $PULSAR_GLOBAL_ZK_CONF "$@"
elif [ $COMMAND == "configuration-store" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"configuration-store.log"}
# Allow global ZK to turn into read-only mode when it cannot reach the
quorum
OPTS="${OPTS} ${ZK_OPTS} -Dreadonlymode.enabled=true"
-exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain
$PULSAR_CONFIGURATION_STORE_CONF $@
+exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain
$PULSAR_CONFIGURATION_STORE_CONF "$@"
elif [ $COMMAND == "proxy" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"pulsar-proxy.log"}
-exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.pulsar.proxy.server.ProxyServiceStarter --config $PULSAR_PROXY_CONF
$@
+exec $JAVA
(pulsar) 05/06: [fix][cli] Fix Pulsar standalone shutdown - bkCluster wasn't closed (#22868)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit aaccb00cbe54081abcbf40acc2b60cb82e7dbb1e
Author: Lari Hotari
AuthorDate: Fri Jun 7 18:36:52 2024 +0300
[fix][cli] Fix Pulsar standalone shutdown - bkCluster wasn't closed (#22868)
(cherry picked from commit c5cc25ebdc3a32d002b944e77fb59c9ccd1f14c1)
---
.../java/org/apache/pulsar/PulsarStandalone.java | 10
.../org/apache/pulsar/PulsarStandaloneStarter.java | 58 --
.../org/apache/pulsar/PulsarStandaloneTest.java| 48 --
.../configurations/pulsar_broker_test.conf | 26 +-
.../pulsar_broker_test_standalone.conf | 26 +-
...pulsar_broker_test_standalone_with_rocksdb.conf | 26 +-
.../configurations/standalone_no_client_auth.conf | 4 +-
.../pulsar/metadata/bookkeeper/BKCluster.java | 43 ++--
8 files changed, 167 insertions(+), 74 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
index b785448cdac..7f80aa29f53 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
@@ -420,18 +420,22 @@ public class PulsarStandalone implements AutoCloseable {
try {
if (fnWorkerService != null) {
fnWorkerService.stop();
+fnWorkerService = null;
}
if (broker != null) {
broker.close();
+broker = null;
}
if (bkCluster != null) {
bkCluster.close();
+bkCluster = null;
}
if (bkEnsemble != null) {
bkEnsemble.stop();
+bkEnsemble = null;
}
} catch (Exception e) {
log.error("Shutdown failed: {}", e.getMessage(), e);
@@ -496,5 +500,11 @@ public class PulsarStandalone implements AutoCloseable {
ShutdownUtil.triggerImmediateForcefulShutdown(exitCode);
}
+public String getBrokerServiceUrl() {
+return broker.getBrokerServiceUrl();
+}
+public String getWebServiceUrl() {
+return broker.getWebServiceAddress();
+}
}
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandaloneStarter.java
b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandaloneStarter.java
index 0ab731591da..29feac8cb46 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandaloneStarter.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandaloneStarter.java
@@ -19,9 +19,12 @@
package org.apache.pulsar;
import static org.apache.commons.lang3.StringUtils.isBlank;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Strings;
import java.io.FileInputStream;
import java.util.Arrays;
+import lombok.AccessLevel;
+import lombok.Setter;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.logging.log4j.LogManager;
@@ -38,6 +41,9 @@ public class PulsarStandaloneStarter extends PulsarStandalone
{
@Option(names = {"-g", "--generate-docs"}, description = "Generate docs")
private boolean generateDocs = false;
+private Thread shutdownThread;
+@Setter(AccessLevel.PACKAGE)
+private boolean testMode;
public PulsarStandaloneStarter(String[] args) throws Exception {
@@ -108,30 +114,54 @@ public class PulsarStandaloneStarter extends
PulsarStandalone {
}
}
}
+}
+@Override
+public synchronized void start() throws Exception {
registerShutdownHook();
+super.start();
}
protected void registerShutdownHook() {
-Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+if (shutdownThread != null) {
+throw new IllegalStateException("Shutdown hook already
registered");
+}
+shutdownThread = new Thread(() -> {
try {
-if (fnWorkerService != null) {
-fnWorkerService.stop();
-}
-
-if (broker != null) {
-broker.close();
-}
-
-if (bkEnsemble != null) {
-bkEnsemble.stop();
-}
+doClose(false);
} catch (Exception e) {
log.error("Shutdown failed: {}", e.getMessage(), e);
} finally {
-LogManager.shutdown();
+if (!testMode) {
+LogManager.shutdown();
+}
}
-}));
+});
+Runtime.getRuntime().addShutdownHook(shutdownThread);
+}
+
+// simulate running the shutdown
(pulsar) 06/06: [fix][cli] Fix healthcheck script pulsar-zookeeper-ruok.sh (#22873)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit e1b3c303feab07d1328fc68af13e8244fe566264
Author: Lari Hotari
AuthorDate: Fri Jun 7 20:00:00 2024 +0300
[fix][cli] Fix healthcheck script pulsar-zookeeper-ruok.sh (#22873)
(cherry picked from commit 80d1cf9e44f14b0753054e4c59419fbaf247481c)
---
docker/pulsar/scripts/pulsar-zookeeper-ruok.sh | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/docker/pulsar/scripts/pulsar-zookeeper-ruok.sh
b/docker/pulsar/scripts/pulsar-zookeeper-ruok.sh
index 7a0228c2386..045258696ff 100755
--- a/docker/pulsar/scripts/pulsar-zookeeper-ruok.sh
+++ b/docker/pulsar/scripts/pulsar-zookeeper-ruok.sh
@@ -20,7 +20,7 @@
# Check ZK server status
-status=$(echo ruok | nc -q 1 localhost 2181)
+status=$({ echo ruok; sleep 1; } | nc 127.0.0.1 2181)
if [ "$status" == "imok" ]; then
exit 0
else
(pulsar) 01/06: [feat][broker] PIP-264: Add OpenTelemetry consumer metrics (#22693)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit c147fcf00957fd897f870c47cc21f5e93d8869c3
Author: Dragos Misca
AuthorDate: Fri May 10 15:35:03 2024 -0700
[feat][broker] PIP-264: Add OpenTelemetry consumer metrics (#22693)
(cherry picked from commit e558cfe9836256065befb3ff6d6043eca10aa5ef)
---
.../org/apache/pulsar/broker/PulsarService.java| 8 +
.../org/apache/pulsar/broker/service/Consumer.java | 32 +++-
.../broker/stats/OpenTelemetryConsumerStats.java | 170 +
.../stats/OpenTelemetryConsumerStatsTest.java | 151 ++
.../broker/testcontext/PulsarTestContext.java | 1 +
.../pulsar/client/api/BrokerServiceLookupTest.java | 1 +
.../opentelemetry/OpenTelemetryAttributes.java | 46 ++
7 files changed, 408 insertions(+), 1 deletion(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
index db1e1dbe402..6482ead1f5a 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java
@@ -109,6 +109,7 @@ import
org.apache.pulsar.broker.service.TransactionBufferSnapshotServiceFactory;
import org.apache.pulsar.broker.service.schema.SchemaRegistryService;
import org.apache.pulsar.broker.service.schema.SchemaStorageFactory;
import org.apache.pulsar.broker.stats.MetricsGenerator;
+import org.apache.pulsar.broker.stats.OpenTelemetryConsumerStats;
import org.apache.pulsar.broker.stats.OpenTelemetryTopicStats;
import org.apache.pulsar.broker.stats.PulsarBrokerOpenTelemetry;
import org.apache.pulsar.broker.stats.prometheus.PrometheusMetricsServlet;
@@ -254,6 +255,7 @@ public class PulsarService implements AutoCloseable,
ShutdownService {
private MetricsGenerator metricsGenerator;
private final PulsarBrokerOpenTelemetry openTelemetry;
private OpenTelemetryTopicStats openTelemetryTopicStats;
+private OpenTelemetryConsumerStats openTelemetryConsumerStats;
private TransactionMetadataStoreService transactionMetadataStoreService;
private TransactionBufferProvider transactionBufferProvider;
@@ -632,8 +634,13 @@ public class PulsarService implements AutoCloseable,
ShutdownService {
brokerClientSharedTimer.stop();
monotonicSnapshotClock.close();
+if (openTelemetryConsumerStats != null) {
+openTelemetryConsumerStats.close();
+openTelemetryConsumerStats = null;
+}
if (openTelemetryTopicStats != null) {
openTelemetryTopicStats.close();
+openTelemetryTopicStats = null;
}
asyncCloseFutures.add(EventLoopUtil.shutdownGracefully(ioEventLoopGroup));
@@ -777,6 +784,7 @@ public class PulsarService implements AutoCloseable,
ShutdownService {
}
openTelemetryTopicStats = new OpenTelemetryTopicStats(this);
+openTelemetryConsumerStats = new OpenTelemetryConsumerStats(this);
localMetadataSynchronizer =
StringUtils.isNotBlank(config.getMetadataSyncEventTopic())
? new PulsarMetadataEventSynchronizer(this,
config.getMetadataSyncEventTopic())
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
index 89a9bab497d..fe9fbe6a400 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/Consumer.java
@@ -25,6 +25,7 @@ import com.google.common.base.MoreObjects;
import com.google.common.util.concurrent.AtomicDouble;
import io.netty.util.concurrent.Future;
import io.netty.util.concurrent.Promise;
+import java.time.Instant;
import java.util.ArrayList;
import java.util.BitSet;
import java.util.Collections;
@@ -90,7 +91,9 @@ public class Consumer {
private final Rate msgOut;
private final Rate msgRedeliver;
private final LongAdder msgOutCounter;
+private final LongAdder msgRedeliverCounter;
private final LongAdder bytesOutCounter;
+private final LongAdder messageAckCounter;
private final Rate messageAckRate;
private volatile long lastConsumedTimestamp;
@@ -152,6 +155,9 @@ public class Consumer {
@Getter
private final SchemaType schemaType;
+@Getter
+private final Instant connectedSince = Instant.now();
+
public Consumer(Subscription subscription, SubType subType, String
topicName, long consumerId,
int priorityLevel, String consumerName,
boolean isDurable, TransportCnx cnx, String appId,
@@ -182,8 +188,10 @@ public class Consumer {
this.msgOut =
(pulsar) 01/02: [fix] [broker] fix deadlock when disable topic level Geo-Replication (#22738)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 6626390cebd381e8be324a06c8d5215e2ee072c8
Author: fengyubiao
AuthorDate: Sun May 19 11:28:21 2024 +0800
[fix] [broker] fix deadlock when disable topic level Geo-Replication
(#22738)
(cherry picked from commit 6372b9c8f5448757cdb56dc70aaf0ebb52a1063e)
---
.../broker/admin/impl/PersistentTopicsBase.java| 6 +-
.../broker/service/OneWayReplicatorTest.java | 59 +++
.../broker/service/OneWayReplicatorTestBase.java | 172 +
.../service/OneWayReplicatorUsingGlobalZKTest.java | 97
4 files changed, 306 insertions(+), 28 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
index 3461eb2d140..9c2746a0293 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
@@ -3274,12 +3274,14 @@ public class PersistentTopicsBase extends AdminResource
{
}
Set replicationClusters = Sets.newHashSet(clusterIds);
return validatePoliciesReadOnlyAccessAsync()
-.thenCompose(__ -> {
+.thenAccept(__ -> {
if (replicationClusters.contains("global")) {
throw new RestException(Status.PRECONDITION_FAILED,
"Cannot specify global in the list of
replication clusters");
}
-Set clusters = clusters();
+})
+.thenCompose(__ -> clustersAsync())
+.thenCompose(clusters -> {
List> futures = new
ArrayList<>(replicationClusters.size());
for (String clusterId : replicationClusters) {
if (!clusters.contains(clusterId)) {
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/OneWayReplicatorTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/OneWayReplicatorTest.java
index fae72e8eac2..a5f1339e95f 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/OneWayReplicatorTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/OneWayReplicatorTest.java
@@ -689,4 +689,63 @@ public class OneWayReplicatorTest extends
OneWayReplicatorTestBase {
admin2.topics().delete(topicName);
});
}
+
+@Test
+public void testDeleteNonPartitionedTopic() throws Exception {
+final String topicName = BrokerTestUtil.newUniqueName("persistent://"
+ replicatedNamespace + "/tp_");
+admin1.topics().createNonPartitionedTopic(topicName);
+
+// Verify replicator works.
+verifyReplicationWorks(topicName);
+
+// Disable replication.
+setTopicLevelClusters(topicName, Arrays.asList(cluster1), admin1,
pulsar1);
+setTopicLevelClusters(topicName, Arrays.asList(cluster2), admin2,
pulsar2);
+
+// Delete topic.
+admin1.topics().delete(topicName);
+admin2.topics().delete(topicName);
+
+// Verify the topic was deleted.
+assertFalse(pulsar1.getPulsarResources().getTopicResources()
+.persistentTopicExists(TopicName.get(topicName)).join());
+assertFalse(pulsar2.getPulsarResources().getTopicResources()
+.persistentTopicExists(TopicName.get(topicName)).join());
+}
+
+@Test
+public void testDeletePartitionedTopic() throws Exception {
+final String topicName = BrokerTestUtil.newUniqueName("persistent://"
+ replicatedNamespace + "/tp_");
+admin1.topics().createPartitionedTopic(topicName, 2);
+
+// Verify replicator works.
+verifyReplicationWorks(topicName);
+
+// Disable replication.
+setTopicLevelClusters(topicName, Arrays.asList(cluster1), admin1,
pulsar1);
+setTopicLevelClusters(topicName, Arrays.asList(cluster2), admin2,
pulsar2);
+
+// Delete topic.
+admin1.topics().deletePartitionedTopic(topicName);
+if (!usingGlobalZK) {
+admin2.topics().deletePartitionedTopic(topicName);
+}
+
+// Verify the topic was deleted.
+
assertFalse(pulsar1.getPulsarResources().getNamespaceResources().getPartitionedTopicResources()
+.partitionedTopicExists(TopicName.get(topicName)));
+
assertFalse(pulsar2.getPulsarResources().getNamespaceResources().getPartitionedTopicResources()
+.partitionedTopicExists(TopicName.get(topicName)));
+if (!usingGlobalZK) {
+// So far, the topic partitions on the remote cluster are needed
to delete manually when using
(pulsar) branch branch-3.3 updated (6e3adc81755 -> 9677d569a69)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch branch-3.3 in repository https://gitbox.apache.org/repos/asf/pulsar.git from 6e3adc81755 Bump version to 3.3.1-SNAPSHOT new 6626390cebd [fix] [broker] fix deadlock when disable topic level Geo-Replication (#22738) new 9677d569a69 [fix] [broker] fix topic partitions was expanded even if disabled topic level replication (#22769) The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../broker/admin/impl/PersistentTopicsBase.java| 17 +- .../broker/service/OneWayReplicatorTest.java | 124 +- .../broker/service/OneWayReplicatorTestBase.java | 185 + .../service/OneWayReplicatorUsingGlobalZKTest.java | 107 4 files changed, 399 insertions(+), 34 deletions(-) create mode 100644 pulsar-broker/src/test/java/org/apache/pulsar/broker/service/OneWayReplicatorUsingGlobalZKTest.java
(pulsar) 02/02: [fix] [broker] fix topic partitions was expanded even if disabled topic level replication (#22769)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/pulsar.git
commit 9677d569a69d6bf5afe1822044416474d6ab6aa0
Author: fengyubiao
AuthorDate: Tue May 28 11:13:12 2024 +0800
[fix] [broker] fix topic partitions was expanded even if disabled topic
level replication (#22769)
(cherry picked from commit 55ad4b22ba2e94029c2e1c01b67b22cb237e5ecc)
---
.../broker/admin/impl/PersistentTopicsBase.java| 11 +++-
.../broker/service/OneWayReplicatorTest.java | 65 +-
.../broker/service/OneWayReplicatorTestBase.java | 13 -
.../service/OneWayReplicatorUsingGlobalZKTest.java | 10
4 files changed, 93 insertions(+), 6 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
index 9c2746a0293..eec6370cec6 100644
---
a/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
+++
b/pulsar-broker/src/main/java/org/apache/pulsar/broker/admin/impl/PersistentTopicsBase.java
@@ -35,6 +35,7 @@ import java.util.ArrayList;
import java.util.Base64;
import java.util.Collections;
import java.util.HashMap;
+import java.util.HashSet;
import java.util.List;
import java.util.Map;
import java.util.Optional;
@@ -452,7 +453,14 @@ public class PersistentTopicsBase extends AdminResource {
if (!policies.isPresent()) {
return
CompletableFuture.completedFuture(null);
}
-final Set replicationClusters =
policies.get().replication_clusters;
+// Combine namespace level policies and topic
level policies.
+Set replicationClusters =
policies.get().replication_clusters;
+TopicPolicies topicPolicies =
+
pulsarService.getTopicPoliciesService().getTopicPoliciesIfExists(topicName);
+if (topicPolicies != null) {
+replicationClusters = new
HashSet<>(topicPolicies.getReplicationClusters());
+}
+// Do check replicated clusters.
if (replicationClusters.size() == 0) {
return
CompletableFuture.completedFuture(null);
}
@@ -468,6 +476,7 @@ public class PersistentTopicsBase extends AdminResource {
// The replication clusters just has the
current cluster itself.
return
CompletableFuture.completedFuture(null);
}
+// Do sync operation to other clusters.
List> futures =
replicationClusters.stream()
.map(replicationCluster ->
admin.clusters().getClusterAsync(replicationCluster)
.thenCompose(clusterData ->
pulsarService.getBrokerService()
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/OneWayReplicatorTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/OneWayReplicatorTest.java
index a5f1339e95f..3dcd787a0cd 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/OneWayReplicatorTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/service/OneWayReplicatorTest.java
@@ -495,8 +495,17 @@ public class OneWayReplicatorTest extends
OneWayReplicatorTestBase {
admin2.topics().createPartitionedTopic(topicName, 2);
admin1.topics().setReplicationClusters(topicName,
Arrays.asList(cluster1, cluster2));
// Check the partitioned topic has been created at the remote cluster.
-PartitionedTopicMetadata topicMetadata2 =
admin2.topics().getPartitionedTopicMetadata(topicName);
-assertEquals(topicMetadata2.partitions, 2);
+Awaitility.await().untilAsserted(() -> {
+PartitionedTopicMetadata topicMetadata2 =
admin2.topics().getPartitionedTopicMetadata(topicName);
+assertEquals(topicMetadata2.partitions, 2);
+});
+
+// Expand partitions
+admin2.topics().updatePartitionedTopic(topicName, 3);
+Awaitility.await().untilAsserted(() -> {
+PartitionedTopicMetadata topicMetadata2 =
admin2.topics().getPartitionedTopicMetadata(topicName);
+assertEquals(topicMetadata2.partitions, 3);
+});
// cleanup.
admin1.topics().setReplicationClusters(topicName,
Arrays.asList(cluster1));
waitReplicatorStopped(partition0);
@@ -748,4 +757,56 @@
(pulsar) branch branch-3.3 updated: Bump version to 3.3.1-SNAPSHOT
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a commit to branch branch-3.3 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-3.3 by this push: new 6e3adc81755 Bump version to 3.3.1-SNAPSHOT 6e3adc81755 is described below commit 6e3adc81755b1f568887310b3dc1ca1a77aba119 Author: Lari Hotari AuthorDate: Fri Jun 7 21:13:24 2024 +0300 Bump version to 3.3.1-SNAPSHOT --- bouncy-castle/bc/pom.xml | 2 +- bouncy-castle/bcfips-include-test/pom.xml | 2 +- bouncy-castle/bcfips/pom.xml | 2 +- bouncy-castle/pom.xml | 2 +- buildtools/pom.xml| 4 ++-- distribution/io/pom.xml | 2 +- distribution/offloaders/pom.xml | 2 +- distribution/pom.xml | 2 +- distribution/server/pom.xml | 2 +- distribution/shell/pom.xml| 2 +- docker/pom.xml| 2 +- docker/pulsar-all/pom.xml | 2 +- docker/pulsar/pom.xml | 2 +- jclouds-shaded/pom.xml| 2 +- managed-ledger/pom.xml| 2 +- microbench/pom.xml| 2 +- pom.xml | 4 ++-- pulsar-bom/pom.xml| 4 ++-- pulsar-broker-auth-athenz/pom.xml | 2 +- pulsar-broker-auth-oidc/pom.xml | 2 +- pulsar-broker-auth-sasl/pom.xml | 2 +- pulsar-broker-common/pom.xml | 2 +- pulsar-broker/pom.xml | 2 +- pulsar-cli-utils/pom.xml | 2 +- pulsar-client-1x-base/pom.xml | 2 +- pulsar-client-1x-base/pulsar-client-1x/pom.xml| 2 +- pulsar-client-1x-base/pulsar-client-2x-shaded/pom.xml | 2 +- pulsar-client-admin-api/pom.xml | 2 +- pulsar-client-admin-shaded/pom.xml| 2 +- pulsar-client-admin/pom.xml | 2 +- pulsar-client-all/pom.xml | 2 +- pulsar-client-api/pom.xml | 2 +- pulsar-client-auth-athenz/pom.xml | 2 +- pulsar-client-auth-sasl/pom.xml | 2 +- pulsar-client-messagecrypto-bc/pom.xml| 2 +- pulsar-client-shaded/pom.xml | 2 +- pulsar-client-tools-api/pom.xml | 2 +- pulsar-client-tools-customcommand-example/pom.xml | 2 +- pulsar-client-tools-test/pom.xml | 2 +- pulsar-client-tools/pom.xml | 2 +- pulsar-client/pom.xml | 2 +- pulsar-common/pom.xml | 2 +- pulsar-config-validation/pom.xml | 2 +- pulsar-docs-tools/pom.xml | 2 +- pulsar-functions/api-java/pom.xml | 2 +- pulsar-functions/instance/pom.xml | 2 +- pulsar-functions/java-examples-builtin/pom.xml| 2 +- pulsar-functions/java-examples/pom.xml| 2 +- pulsar-functions/localrun-shaded/pom.xml | 2 +- pulsar-functions/localrun/pom.xml | 2 +- pulsar-functions/pom.xml | 2 +- pulsar-functions/proto/pom.xml| 2 +- pulsar-functions/runtime-all/pom.xml | 2 +- pulsar-functions/runtime/pom.xml | 2 +- pulsar-functions/secrets/pom.xml | 2 +- pulsar-functions/utils/pom.xml| 2 +- pulsar-functions/worker/pom.xml | 2 +- pulsar-io/aerospike/pom.xml | 2 +- pulsar-io/alluxio/pom.xml | 2 +- pulsar-io/aws/pom.xml | 2 +- pulsar-io/azure-data-explorer/pom.xml | 2 +- pulsar-io/batch-data-generator/pom.xml| 2 +- pulsar-io/batch-discovery-triggerers/pom.xml | 2 +- pulsar-io/canal/pom.xml | 2 +- pulsar-io/cassandra/pom.xml | 2 +- pulsar-io/common/pom.xml | 2 +- pulsar-io/core/pom.xml| 2 +- pulsar-io/data-generator/pom.xml | 2 +- pulsar-io/debezium/core/pom.xml | 2 +- pulsar-io/debezium/mongodb/pom.xml| 2 +- pulsar-io/debezium/mssql/pom.xml | 2 +- pulsar-io/debezium/mysql/pom.xml | 2 +- pulsar-io/debezium/oracle/pom.xml
Re: [PR] [fix][misc] Topic name from persistence name should decode local name [pulsar]
lhotari commented on PR #22879: URL: https://github.com/apache/pulsar/pull/22879#issuecomment-2155294849 @Shawyeok Just curious to know what the impact of the previous behavior was. Did it cause actual problems? Any error messages in logs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix][misc] Add proper nslookup (included in bind-tools) to docker image [pulsar]
lhotari closed pull request #22878: [fix][misc] Add proper nslookup (included in bind-tools) to docker image URL: https://github.com/apache/pulsar/pull/22878 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix][misc] Add proper nslookup (included in bind-tools) to docker image [pulsar]
lhotari commented on PR #22878: URL: https://github.com/apache/pulsar/pull/22878#issuecomment-2155285331 Pulsar cluster gets initialized properly with Apache Pulsar Helm chart when using nslookup from bind-tools package. Here's the way I tested: https://github.com/lhotari/pulsar-helm-chart/pull/10/files#diff-f34da55ca08f1a30591d8b0b3e885bcc678537b2a9a4aadea4f190806b374ddcR3 . Pushed the image to https://hub.docker.com/r/lhotari/pulsar-all/tags (`docker pull lhotari/pulsar-all:3.3.0-nslookup`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix][misc] Add proper nslookup (included in bind-tools) to docker image [pulsar]
merlimat commented on PR #22878: URL: https://github.com/apache/pulsar/pull/22878#issuecomment-2155261885 > > I assume `bind-utils` is light-weight. > > It is. The downside is that it could contain vulnerabilities which show up in scanning. However there doesn't seem to be an alternative for backwards compatibility with Pulsar helm charts that use nslookup to check when another component is available. After we do the `RUN apk upgrade --no-cache` step, it's picking up the latest version, with no vulnerabilities -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix][misc] Add proper nslookup (included in bind-tools) to docker image [pulsar]
lhotari commented on PR #22878: URL: https://github.com/apache/pulsar/pull/22878#issuecomment-2155259343 The correct package name in Alpine is `bind-tools`. Testing it soon with Apache Pulsar Helm chart. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [improve][broker] Reuse topic OpenTelemetry attributes [pulsar]
dao-jun closed pull request #22876: [improve][broker] Reuse topic OpenTelemetry attributes URL: https://github.com/apache/pulsar/pull/22876 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [fix][misc] Topic name from persistence naming encoding should decode local name [pulsar]
Shawyeok opened a new pull request, #22879: URL: https://github.com/apache/pulsar/pull/22879 ### Motivation The methods `TopicName#getPersistenceNamingEncoding` and `TopicName#fromPersistenceNamingEncoding` should be symmetric. Currently, the `localName` part of the topic name is encoded into the persistence name, but it is not decoded back properly. ### Modifications This update add decoding logic for the `localName` part of the topic name. ### Verifying this change - [ ] Make sure that the change passes the CI checks. This change added tests and can be verified as follows: - A new test case has been added to `org.apache.pulsar.common.naming.TopicNameTest#testFromPersistenceNamingEncoding` that includes topic names with special characters. ### Does this pull request potentially affect one of the following parts: *If the box was checked, please highlight the changes* - [ ] Dependencies (add or upgrade a dependency) - [ ] The public API - [ ] The schema - [ ] The default values of configurations - [ ] The threading model - [ ] The binary protocol - [ ] The REST endpoints - [ ] The admin CLI options - [ ] The metrics - [ ] Anything that affects deployment ### Documentation - [ ] `doc` - [ ] `doc-required` - [x] `doc-not-needed` - [ ] `doc-complete` ### Matching PR in forked repository PR in forked repository: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix][misc] Add proper nslookup (included in bind-utils) to docker image [pulsar]
lhotari commented on PR #22878: URL: https://github.com/apache/pulsar/pull/22878#issuecomment-2155249661 > I assume `bind-utils` is light-weight. It is. The downside is that it could contain vulnerabilities which show up in scanning. However there doesn't seem to be an alternative for backwards compatibility with Pulsar helm charts that use nslookup to check when another component is available. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) branch master updated (80d1cf9e44f -> 5af05951754)
This is an automated email from the ASF dual-hosted git repository. heesung pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git from 80d1cf9e44f [fix][cli] Fix healthcheck script pulsar-zookeeper-ruok.sh (#22873) add 5af05951754 [fix][broker] Support advertised listeners when gracefully transferring topics (ExtensibleLoadManagerImpl only) (#22862) No new revisions were added by this update. Summary of changes: .../apache/pulsar/broker/service/ServerCnx.java| 41 ++--- .../ExtensibleLoadManagerImplBaseTest.java | 15 ++-- .../extensions/ExtensibleLoadManagerImplTest.java | 65 -- ...LoadManagerImplWithAdvertisedListenersTest.java | 98 ++ ...dManagerImplWithTransactionCoordinatorTest.java | 4 +- 5 files changed, 200 insertions(+), 23 deletions(-) create mode 100644 pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/extensions/ExtensibleLoadManagerImplWithAdvertisedListenersTest.java
Re: [PR] [fix][broker] Support advertised listeners when gracefully transferring topics (ExtensibleLoadManagerImpl only) [pulsar]
heesung-sn merged PR #22862: URL: https://github.com/apache/pulsar/pull/22862 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix][cli] Fix healthcheck script pulsar-zookeeper-ruok.sh [pulsar]
lhotari commented on PR #22873: URL: https://github.com/apache/pulsar/pull/22873#issuecomment-215571 Created #22877 and #22878 to address the nslookup issue. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [fix][misc] Add proper nslookup (included in bind-utils) to docker image [pulsar]
lhotari opened a new pull request, #22878: URL: https://github.com/apache/pulsar/pull/22878 Fixes: #22877 ### Motivation The default `nslookup` in Alpine is based on busybox and it isn't compatible with Kubernetes. See #22877 for more details. ### Modifications Install `bind-utils` package to get proper `nslookup` ### Documentation - [ ] `doc` - [ ] `doc-required` - [x] `doc-not-needed` - [ ] `doc-complete` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] nslookup in apachepulsar/pulsar:3.3.0 isn't compatible with kubernetes search domains [pulsar]
lhotari opened a new issue, #22877: URL: https://github.com/apache/pulsar/issues/22877 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Read release policy - [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker. ### Version apachepulsar/pulsar:3.3.0 image ### Minimal reproduce step See https://github.com/lhotari/pulsar-helm-chart/pull/10 ### What did you expect to see? Pulsar cluster would start up without errors. ### What did you see instead? Init containers waiting for Zookeeper to come available are failing: ``` Server:10.96.0.10 Address:10.96.0.10:53 ** server can't find pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar: NXDOMAIN ** server can't find pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar: NXDOMAIN ``` ### Anything else? Debugged the problem. FQDN nslookup works, partial domain name doesn't. ``` pulsar-ci-pulsar-init-pfdv8:/pulsar$ nslookup pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar.svc.cluster.local Server: 10.96.0.10 Address:10.96.0.10:53 Name: pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar.svc.cluster.local Address: 10.244.1.17 pulsar-ci-pulsar-init-pfdv8:/pulsar$ nslookup pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar Server: 10.96.0.10 Address:10.96.0.10:53 ** server can't find pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar: NXDOMAIN ** server can't find pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar: NXDOMAIN ``` /etc/resolv.conf is ``` search pulsar.svc.cluster.local svc.cluster.local cluster.local 1evhaya2a4vu1idbbq22av5vmh.bx.internal.cloudapp.net nameserver 10.96.0.10 options ndots:5 ``` The busybox nslookup isn't compatible with search domains and ndots. It is necessary to install bind-utils package with `apk add bind-utils`. Found this information in https://github.com/docker-library/busybox/issues/48#issuecomment-2069324947 ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] bin/pulsar-zookeeper-ruok.sh fails with apachepulsar/pulsar:3.3.0 image [pulsar]
lhotari closed issue #22872: [Bug] bin/pulsar-zookeeper-ruok.sh fails with apachepulsar/pulsar:3.3.0 image URL: https://github.com/apache/pulsar/issues/22872 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) branch master updated (d6dc4d3957e -> 80d1cf9e44f)
This is an automated email from the ASF dual-hosted git repository. lhotari pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git from d6dc4d3957e [fix][misc] Disable JFR based telemetry collection since it's not used (#22869) add 80d1cf9e44f [fix][cli] Fix healthcheck script pulsar-zookeeper-ruok.sh (#22873) No new revisions were added by this update. Summary of changes: docker/pulsar/scripts/pulsar-zookeeper-ruok.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Re: [PR] [fix][cli] Fix healthcheck script pulsar-zookeeper-ruok.sh [pulsar]
lhotari merged PR #22873: URL: https://github.com/apache/pulsar/pull/22873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [improve][broker] Reuse topic OpenTelemetry attributes [pulsar]
dragosvictor opened a new pull request, #22876: URL: https://github.com/apache/pulsar/pull/22876 Fixes https://github.com/apache/pulsar/issues/22817 ### Motivation Successive collections of OpenTelemetry metrics re-allocate all attribute sets related to a topic. Since topics are a high-cardinality object in Pulsar, this puts pressure on the GC. This PR proposes caching these attribute sets. ### Modifications - Adds classes `TopicAttributes` and `PersistentTopicAttributes` to store all attribute sets relevant for a topic. - Add and lazily initialize these attribute fields in the `NonPersistentTopic` and `PersistentTopic` objects. If OpenTelemetry is not enabled, these objects never get created, saving memory. - Modify `OpenTelemetryTopicStats` to use the cached objects instead of re-allocating them every run. - Relocate some of the attributes into interface `OpenTelemetryAttributes`, for easier a cleaner design. ### Verifying this change - [x] Make sure that the change passes the CI checks. This change is already covered by existing tests, such as `org.apache.pulsar.broker.stats.OpenTelemetryTopicStatsTest`. ### Does this pull request potentially affect one of the following parts: *If the box was checked, please highlight the changes* - [ ] Dependencies (add or upgrade a dependency) - [ ] The public API - [ ] The schema - [ ] The default values of configurations - [ ] The threading model - [ ] The binary protocol - [ ] The REST endpoints - [ ] The admin CLI options - [ ] The metrics - [ ] Anything that affects deployment ### Documentation - [ ] `doc` - [ ] `doc-required` - [x] `doc-not-needed` - [ ] `doc-complete` ### Matching PR in forked repository PR in forked repository: https://github.com/dragosvictor/pulsar/pull/31 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Make zookeeper healthchecks compatible with Alpine's busybox nc [pulsar-helm-chart]
lhotari commented on PR #504: URL: https://github.com/apache/pulsar-helm-chart/pull/504#issuecomment-2155199759 Looks like busybox nslookup isn't compatible. https://github.com/docker-library/busybox/issues/48#issuecomment-2069324947 will need `apk add bind-tools` in Alpine image. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Make zookeeper healthchecks compatible with Alpine's busybox nc [pulsar-helm-chart]
lhotari commented on PR #504: URL: https://github.com/apache/pulsar-helm-chart/pull/504#issuecomment-2155173197 found https://github.com/k3s-io/k3s/issues/6561#issuecomment-1443590459 , but not a proper resolution. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Make zookeeper healthchecks compatible with Alpine's busybox nc [pulsar-helm-chart]
lhotari commented on PR #504: URL: https://github.com/apache/pulsar-helm-chart/pull/504#issuecomment-2155169975 Some DNS config issue: ``` pulsar-ci-pulsar-init-pfdv8:/pulsar$ nslookup pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar.svc.cluster.local Server: 10.96.0.10 Address:10.96.0.10:53 Name: pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar.svc.cluster.local Address: 10.244.1.17 pulsar-ci-pulsar-init-pfdv8:/pulsar$ nslookup pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar Server: 10.96.0.10 Address:10.96.0.10:53 ** server can't find pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar: NXDOMAIN ** server can't find pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar: NXDOMAIN ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] Make zookeeper healthchecks compatible with Alpine's busybox nc [pulsar-helm-chart]
lhotari commented on PR #504: URL: https://github.com/apache/pulsar-helm-chart/pull/504#issuecomment-2155168715 There's some other issue with the Alpine image: ``` Server:10.96.0.10 Address:10.96.0.10:53 ** server can't find pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar: NXDOMAIN ** server can't find pulsar-ci-zookeeper-0.pulsar-ci-zookeeper.pulsar: NXDOMAIN ``` /etc/resolv.conf is ``` search pulsar.svc.cluster.local svc.cluster.local cluster.local 1evhaya2a4vu1idbbq22av5vmh.bx.internal.cloudapp.net nameserver 10.96.0.10 options ndots:5 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix][cli] Fix healthcheck script pulsar-zookeeper-ruok.sh [pulsar]
lhotari commented on PR #22873: URL: https://github.com/apache/pulsar/pull/22873#issuecomment-2155164966 The problem in Apache Pulsar Helm chart is unrelated to the healthcheck. It works now with this syntax. There's some other DNS lookup related issue with the Alpine image. Something that could be possibly resolved with a `ndots: 5` config. I'll investigate. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) branch branch-3.3 updated: [fix][misc] Disable JFR based telemetry collection since it's not used (#22869)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch branch-3.3
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-3.3 by this push:
new 189d1f0a49d [fix][misc] Disable JFR based telemetry collection since
it's not used (#22869)
189d1f0a49d is described below
commit 189d1f0a49debdae66c54cdfeadd79c5f541c20d
Author: Lari Hotari
AuthorDate: Fri Jun 7 19:18:48 2024 +0300
[fix][misc] Disable JFR based telemetry collection since it's not used
(#22869)
---
.../java/org/apache/pulsar/opentelemetry/OpenTelemetryService.java | 3 +++
1 file changed, 3 insertions(+)
diff --git
a/pulsar-opentelemetry/src/main/java/org/apache/pulsar/opentelemetry/OpenTelemetryService.java
b/pulsar-opentelemetry/src/main/java/org/apache/pulsar/opentelemetry/OpenTelemetryService.java
index eb09e64fe73..b5610fc485b 100644
---
a/pulsar-opentelemetry/src/main/java/org/apache/pulsar/opentelemetry/OpenTelemetryService.java
+++
b/pulsar-opentelemetry/src/main/java/org/apache/pulsar/opentelemetry/OpenTelemetryService.java
@@ -102,6 +102,9 @@ public class OpenTelemetryService implements Closeable {
// For a list of exposed metrics, see
https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/
runtimeMetricsReference.set(RuntimeMetrics.builder(openTelemetrySdkReference.get())
+// disable JFR based telemetry and use only JMX telemetry
+.disableAllFeatures()
+// enable experimental JMX telemetry in addition
.enableExperimentalJmxTelemetry()
.build());
}
Re: [PR] [fix][misc] Disable JFR based telemetry collection since it's not used [pulsar]
heesung-sn commented on PR #22869: URL: https://github.com/apache/pulsar/pull/22869#issuecomment-2155152469 LGTM. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) branch master updated: [fix][misc] Disable JFR based telemetry collection since it's not used (#22869)
This is an automated email from the ASF dual-hosted git repository.
mmerli pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new d6dc4d3957e [fix][misc] Disable JFR based telemetry collection since
it's not used (#22869)
d6dc4d3957e is described below
commit d6dc4d3957e13b392c55324f3607a86d37a835a7
Author: Lari Hotari
AuthorDate: Fri Jun 7 19:18:48 2024 +0300
[fix][misc] Disable JFR based telemetry collection since it's not used
(#22869)
---
.../java/org/apache/pulsar/opentelemetry/OpenTelemetryService.java | 3 +++
1 file changed, 3 insertions(+)
diff --git
a/pulsar-opentelemetry/src/main/java/org/apache/pulsar/opentelemetry/OpenTelemetryService.java
b/pulsar-opentelemetry/src/main/java/org/apache/pulsar/opentelemetry/OpenTelemetryService.java
index eb09e64fe73..b5610fc485b 100644
---
a/pulsar-opentelemetry/src/main/java/org/apache/pulsar/opentelemetry/OpenTelemetryService.java
+++
b/pulsar-opentelemetry/src/main/java/org/apache/pulsar/opentelemetry/OpenTelemetryService.java
@@ -102,6 +102,9 @@ public class OpenTelemetryService implements Closeable {
// For a list of exposed metrics, see
https://opentelemetry.io/docs/specs/semconv/runtime/jvm-metrics/
runtimeMetricsReference.set(RuntimeMetrics.builder(openTelemetrySdkReference.get())
+// disable JFR based telemetry and use only JMX telemetry
+.disableAllFeatures()
+// enable experimental JMX telemetry in addition
.enableExperimentalJmxTelemetry()
.build());
}
Re: [PR] [fix][misc] Disable JFR based telemetry collection since it's not used [pulsar]
merlimat merged PR #22869: URL: https://github.com/apache/pulsar/pull/22869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] maven build fails with Java 22 [pulsar]
lhotari commented on issue #22874: URL: https://github.com/apache/pulsar/issues/22874#issuecomment-2155103223 One possibility is to enforce that a supported JVM versions is used for the compilation. We currently support Java 17 and Java 21 for the master branch. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] maven build fails with Java 22 [pulsar]
lhotari opened a new issue, #22874: URL: https://github.com/apache/pulsar/issues/22874 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Read release policy - [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker. ### Version master branch ### Minimal reproduce step ``` mvn install -DskipTests ``` ### What did you expect to see? The build completes without errors. ### What did you see instead? There are multiple errors about `java.lang.IllegalArgumentException: Unsupported class file major version 66` ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) branch master updated: [fix][cli] Fix Pulsar standalone shutdown - bkCluster wasn't closed (#22868)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new c5cc25ebdc3 [fix][cli] Fix Pulsar standalone shutdown - bkCluster
wasn't closed (#22868)
c5cc25ebdc3 is described below
commit c5cc25ebdc3a32d002b944e77fb59c9ccd1f14c1
Author: Lari Hotari
AuthorDate: Fri Jun 7 18:36:52 2024 +0300
[fix][cli] Fix Pulsar standalone shutdown - bkCluster wasn't closed (#22868)
---
.../java/org/apache/pulsar/PulsarStandalone.java | 10
.../org/apache/pulsar/PulsarStandaloneStarter.java | 58 --
.../org/apache/pulsar/PulsarStandaloneTest.java| 48 --
.../configurations/pulsar_broker_test.conf | 26 +-
.../pulsar_broker_test_standalone.conf | 26 +-
...pulsar_broker_test_standalone_with_rocksdb.conf | 26 +-
.../configurations/standalone_no_client_auth.conf | 4 +-
.../pulsar/metadata/bookkeeper/BKCluster.java | 43 ++--
8 files changed, 167 insertions(+), 74 deletions(-)
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
index b785448cdac..7f80aa29f53 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandalone.java
@@ -420,18 +420,22 @@ public class PulsarStandalone implements AutoCloseable {
try {
if (fnWorkerService != null) {
fnWorkerService.stop();
+fnWorkerService = null;
}
if (broker != null) {
broker.close();
+broker = null;
}
if (bkCluster != null) {
bkCluster.close();
+bkCluster = null;
}
if (bkEnsemble != null) {
bkEnsemble.stop();
+bkEnsemble = null;
}
} catch (Exception e) {
log.error("Shutdown failed: {}", e.getMessage(), e);
@@ -496,5 +500,11 @@ public class PulsarStandalone implements AutoCloseable {
ShutdownUtil.triggerImmediateForcefulShutdown(exitCode);
}
+public String getBrokerServiceUrl() {
+return broker.getBrokerServiceUrl();
+}
+public String getWebServiceUrl() {
+return broker.getWebServiceAddress();
+}
}
diff --git
a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandaloneStarter.java
b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandaloneStarter.java
index 0ab731591da..29feac8cb46 100644
--- a/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandaloneStarter.java
+++ b/pulsar-broker/src/main/java/org/apache/pulsar/PulsarStandaloneStarter.java
@@ -19,9 +19,12 @@
package org.apache.pulsar;
import static org.apache.commons.lang3.StringUtils.isBlank;
+import com.google.common.annotations.VisibleForTesting;
import com.google.common.base.Strings;
import java.io.FileInputStream;
import java.util.Arrays;
+import lombok.AccessLevel;
+import lombok.Setter;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang3.StringUtils;
import org.apache.logging.log4j.LogManager;
@@ -38,6 +41,9 @@ public class PulsarStandaloneStarter extends PulsarStandalone
{
@Option(names = {"-g", "--generate-docs"}, description = "Generate docs")
private boolean generateDocs = false;
+private Thread shutdownThread;
+@Setter(AccessLevel.PACKAGE)
+private boolean testMode;
public PulsarStandaloneStarter(String[] args) throws Exception {
@@ -108,30 +114,54 @@ public class PulsarStandaloneStarter extends
PulsarStandalone {
}
}
}
+}
+@Override
+public synchronized void start() throws Exception {
registerShutdownHook();
+super.start();
}
protected void registerShutdownHook() {
-Runtime.getRuntime().addShutdownHook(new Thread(() -> {
+if (shutdownThread != null) {
+throw new IllegalStateException("Shutdown hook already
registered");
+}
+shutdownThread = new Thread(() -> {
try {
-if (fnWorkerService != null) {
-fnWorkerService.stop();
-}
-
-if (broker != null) {
-broker.close();
-}
-
-if (bkEnsemble != null) {
-bkEnsemble.stop();
-}
+doClose(false);
} catch (Exception e) {
log.error("Shutdown failed: {}", e.getMessage(), e);
} finally {
-LogManager.shutdown();
+if (!testMode) {
+LogManager.shutdown();
+}
}
-}));
+
Re: [PR] [fix][cli] Fix Pulsar standalone shutdown - bkCluster wasn't closed [pulsar]
lhotari merged PR #22868: URL: https://github.com/apache/pulsar/pull/22868 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix][cli] Fix healthcheck script pulsar-zookeeper-ruok.sh [pulsar]
lhotari commented on PR #22873: URL: https://github.com/apache/pulsar/pull/22873#issuecomment-2155083014 I'm verifying a similar solution in https://github.com/apache/pulsar-helm-chart/pull/504. However there's some issue although I tested the solution in docker. I'll verify in k8s too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [fix][cli] Fix healthcheck script pulsar-zookeeper-ruok.sh [pulsar]
lhotari opened a new pull request, #22873: URL: https://github.com/apache/pulsar/pull/22873 Fixes #22872 ### Motivation pulsar-zookeeper-ruok.sh script doesn't work in the latest apachepulsar/pulsar:3.3.0 image. ### Modifications - make the script compatible with Alpine's default busybox nc (so that it isn't necessary to install `netcat-openbsd`) - replace "-q 1" with "sleep 1" in the input - replace "localhost" with "127.0.0.1" ### Documentation - [ ] `doc` - [ ] `doc-required` - [x] `doc-not-needed` - [ ] `doc-complete` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] Make zookeeper healthchecks compatible with Alpine's busybox nc [pulsar-helm-chart]
lhotari opened a new pull request, #504: URL: https://github.com/apache/pulsar-helm-chart/pull/504 ### Motivation The apachepulsar/pulsar:3.3.0 image contains busybox nc which doesn't support `-q 1` command line option. See #22872. ### Modifications replace `-q 1` with the use of `sleep 1` in the input. Solution from https://unix.stackexchange.com/a/274603. ### Verifying this change - [ ] Make sure that the change passes the CI checks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] bin/pulsar-zookeeper-ruok.sh fails with apachepulsar/pulsar:3.3.0 image [pulsar]
lhotari opened a new issue, #22872: URL: https://github.com/apache/pulsar/issues/22872 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Read release policy - [X] I understand that unsupported versions don't get bug fixes. I will attempt to reproduce the issue on a supported version of Pulsar client and Pulsar broker. ### Version apachepulsar/pulsar:3.3.0 image ### Minimal reproduce step ``` ❯ docker run --rm -it apachepulsar/pulsar:3.3.0 bin/pulsar-zookeeper-ruok.sh nc: unrecognized option: q BusyBox v1.36.1 (2024-05-21 13:57:33 UTC) multi-call binary. Usage: nc [OPTIONS] HOST PORT - connect nc [OPTIONS] -l -p PORT [HOST] [PORT] - listen -e PROG Run PROG after connect (must be last) -l Listen mode, for inbound connects -lk With -e, provides persistent server -p PORT Local port -s ADDR Local address -w SEC Timeout for connects and final net reads -i SEC Delay interval for lines sent -n Don't do DNS resolution -u UDP mode -b Allow broadcasts -v Verbose -o FILE Hex dump traffic -z Zero-I/O mode (scanning) ZK server is not ok ``` ### What did you expect to see? There shouldn't be an error about unrecognized options ### What did you see instead? there was an error `nc: unrecognized option: q` ### Anything else? _No response_ ### Are you willing to submit a PR? - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar-dotpulsar) branch master updated: Update release_validation_linux_macos.md
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar-dotpulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 8b33fd8 Update release_validation_linux_macos.md
8b33fd8 is described below
commit 8b33fd8913c00dacd202dd32e913cd277368a13b
Author: Lari Hotari
AuthorDate: Fri Jun 7 05:42:45 2024 -0700
Update release_validation_linux_macos.md
- fix typo
- update Pulsar docker image tag
---
docs/release_validation_linux_macos.md | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/docs/release_validation_linux_macos.md
b/docs/release_validation_linux_macos.md
index 4c0295e..7f62cf6 100644
--- a/docs/release_validation_linux_macos.md
+++ b/docs/release_validation_linux_macos.md
@@ -22,9 +22,9 @@ dotnet --info
## Validating source release
-Set exvironment variables
+Set environment variables
```shell
-export DOTPULSAR_VERSION_RC=3.1.2-rc.1
+export DOTPULSAR_VERSION_RC=3.3.0-rc.1
export DOTPULSAR_VERSION=${DOTPULSAR_VERSION_RC%-rc.*}
```
@@ -85,7 +85,7 @@ await consumer.Acknowledge(message);
Console.WriteLine("Acknowledged message");
EOF
dotnet build
-docker run --name pulsar-standalone -d --rm -it -p 8080:8080 -p 6650:6650
apachepulsar/pulsar:3.0.4 /pulsar/bin/pulsar standalone -nss -nfw
+docker run --name pulsar-standalone -d --rm -it -p 8080:8080 -p 6650:6650
apachepulsar/pulsar:3.0.5 /pulsar/bin/pulsar standalone -nss -nfw
dotnet run
docker stop pulsar-standalone
```
Re: [PR] Handle Pulsar Token bug in affected versions [pulsar-dotpulsar]
entvex closed pull request #221: Handle Pulsar Token bug in affected versions URL: https://github.com/apache/pulsar-dotpulsar/pull/221 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) branch master updated: [fix] [test] fix testGetMetrics in ExtensibleLoadManagerImplTest. (#22864)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new 6f1f7bae76d [fix] [test] fix testGetMetrics in
ExtensibleLoadManagerImplTest. (#22864)
6f1f7bae76d is described below
commit 6f1f7bae76d04d877a7e06641275a3d4a73cadba
Author: Wenzhi Feng <52550727+thetumb...@users.noreply.github.com>
AuthorDate: Fri Jun 7 20:27:27 2024 +0800
[fix] [test] fix testGetMetrics in ExtensibleLoadManagerImplTest. (#22864)
---
.../broker/loadbalance/extensions/ExtensibleLoadManagerImplTest.java| 2 ++
1 file changed, 2 insertions(+)
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/extensions/ExtensibleLoadManagerImplTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/extensions/ExtensibleLoadManagerImplTest.java
index 07855fda4d7..43c50a8ac54 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/extensions/ExtensibleLoadManagerImplTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/loadbalance/extensions/ExtensibleLoadManagerImplTest.java
@@ -1292,6 +1292,8 @@ public class ExtensibleLoadManagerImplTest extends
ExtensibleLoadManagerImplBase
@Test
public void testGetMetrics() throws Exception {
{
+ServiceConfiguration conf = getDefaultConf();
+conf.setLoadBalancerMemoryResourceWeight(1);
var brokerLoadDataReporter = mock(BrokerLoadDataReporter.class);
FieldUtils.writeDeclaredField(primaryLoadManager,
"brokerLoadDataReporter", brokerLoadDataReporter, true);
BrokerLoadData loadData = new BrokerLoadData();
Re: [I] Flaky-test: ExtensibleLoadManagerImplTest.testGetMetrics (fails consistently) [pulsar]
lhotari closed issue #22871: Flaky-test: ExtensibleLoadManagerImplTest.testGetMetrics (fails consistently) URL: https://github.com/apache/pulsar/issues/22871 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix] [test] fix testGetMetrics in ExtensibleLoadManagerImplTest. [pulsar]
lhotari merged PR #22864: URL: https://github.com/apache/pulsar/pull/22864 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix][cli] Fix the shell script parameter passthrough syntax [pulsar]
lhotari merged PR #22867: URL: https://github.com/apache/pulsar/pull/22867 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(pulsar) branch master updated: [fix][cli] Fix the shell script parameter passthrough syntax (#22867)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/master by this push:
new c81c0f684f8 [fix][cli] Fix the shell script parameter passthrough
syntax (#22867)
c81c0f684f8 is described below
commit c81c0f684f8c55c2e39739c6e1de935dff2085d6
Author: Lari Hotari
AuthorDate: Fri Jun 7 15:25:35 2024 +0300
[fix][cli] Fix the shell script parameter passthrough syntax (#22867)
---
bin/bookkeeper | 12 ++--
bin/pulsar | 38 +++---
2 files changed, 25 insertions(+), 25 deletions(-)
diff --git a/bin/bookkeeper b/bin/bookkeeper
index 0cc07dd49ab..13d092f4c99 100755
--- a/bin/bookkeeper
+++ b/bin/bookkeeper
@@ -214,20 +214,20 @@ OPTS="$OPTS $BK_METADATA_OPTIONS"
#Change to BK_HOME to support relative paths
cd "$BK_HOME"
if [ $COMMAND == "bookie" ]; then
-exec $JAVA $OPTS $JMX_ARGS org.apache.bookkeeper.server.Main --conf
$BOOKIE_CONF $@
+exec $JAVA $OPTS $JMX_ARGS org.apache.bookkeeper.server.Main --conf
$BOOKIE_CONF "$@"
elif [ $COMMAND == "autorecovery" ]; then
-exec $JAVA $OPTS $JMX_ARGS
org.apache.bookkeeper.replication.AutoRecoveryMain --conf $BOOKIE_CONF $@
+exec $JAVA $OPTS $JMX_ARGS
org.apache.bookkeeper.replication.AutoRecoveryMain --conf $BOOKIE_CONF "$@"
elif [ $COMMAND == "localbookie" ]; then
NUMBER=$1
shift
-exec $JAVA $OPTS $JMX_ARGS org.apache.bookkeeper.util.LocalBookKeeper
$NUMBER $BOOKIE_CONF $@
+exec $JAVA $OPTS $JMX_ARGS org.apache.bookkeeper.util.LocalBookKeeper
$NUMBER $BOOKIE_CONF "$@"
elif [ $COMMAND == "upgrade" ]; then
-exec $JAVA $OPTS org.apache.bookkeeper.bookie.FileSystemUpgrade --conf
$BOOKIE_CONF $@
+exec $JAVA $OPTS org.apache.bookkeeper.bookie.FileSystemUpgrade --conf
$BOOKIE_CONF "$@"
elif [ $COMMAND == "shell" ]; then
ENTRY_FORMATTER_ARG="-DentryFormatterClass=${ENTRY_FORMATTER_CLASS:-org.apache.bookkeeper.util.StringEntryFormatter}"
-exec $JAVA $OPTS $ENTRY_FORMATTER_ARG
org.apache.bookkeeper.bookie.BookieShell -conf $BOOKIE_CONF $@
+exec $JAVA $OPTS $ENTRY_FORMATTER_ARG
org.apache.bookkeeper.bookie.BookieShell -conf $BOOKIE_CONF "$@"
elif [ $COMMAND == "help" -o $COMMAND == "--help" -o $COMMAND == "-h" ]; then
bookkeeper_help;
else
-exec $JAVA $OPTS $COMMAND $@
+exec $JAVA $OPTS $COMMAND "$@"
fi
diff --git a/bin/pulsar b/bin/pulsar
index ab0029af5b0..f6061601d88 100755
--- a/bin/pulsar
+++ b/bin/pulsar
@@ -329,56 +329,56 @@ fi
cd "$PULSAR_HOME"
if [ $COMMAND == "broker" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"pulsar-broker.log"}
-exec $JAVA $LOG4J2_SHUTDOWN_HOOK_DISABLED $OPTS
-Dpulsar.log.file=$PULSAR_LOG_FILE org.apache.pulsar.PulsarBrokerStarter
--broker-conf $PULSAR_BROKER_CONF $@
+exec $JAVA $LOG4J2_SHUTDOWN_HOOK_DISABLED $OPTS
-Dpulsar.log.file=$PULSAR_LOG_FILE org.apache.pulsar.PulsarBrokerStarter
--broker-conf $PULSAR_BROKER_CONF "$@"
elif [ $COMMAND == "bookie" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"bookkeeper.log"}
-exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.bookkeeper.server.Main --conf $PULSAR_BOOKKEEPER_CONF $@
+exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.bookkeeper.server.Main --conf $PULSAR_BOOKKEEPER_CONF "$@"
elif [ $COMMAND == "zookeeper" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"zookeeper.log"}
-exec $JAVA ${ZK_OPTS} $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain $PULSAR_ZK_CONF $@
+exec $JAVA ${ZK_OPTS} $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain $PULSAR_ZK_CONF "$@"
elif [ $COMMAND == "global-zookeeper" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"global-zookeeper.log"}
# Allow global ZK to turn into read-only mode when it cannot reach the
quorum
OPTS="${OPTS} ${ZK_OPTS} -Dreadonlymode.enabled=true"
-exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain $PULSAR_GLOBAL_ZK_CONF $@
+exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain $PULSAR_GLOBAL_ZK_CONF "$@"
elif [ $COMMAND == "configuration-store" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"configuration-store.log"}
# Allow global ZK to turn into read-only mode when it cannot reach the
quorum
OPTS="${OPTS} ${ZK_OPTS} -Dreadonlymode.enabled=true"
-exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain
$PULSAR_CONFIGURATION_STORE_CONF $@
+exec $JAVA $OPTS -Dpulsar.log.file=$PULSAR_LOG_FILE
org.apache.zookeeper.server.quorum.QuorumPeerMain
$PULSAR_CONFIGURATION_STORE_CONF "$@"
elif [ $COMMAND == "proxy" ]; then
PULSAR_LOG_FILE=${PULSAR_LOG_FILE:-"pulsar-proxy.log"}
-exec $JAVA $OPTS
(pulsar) branch branch-2.10 updated: [fix][test] Fix testPerTopicStats (#22870)
This is an automated email from the ASF dual-hosted git repository.
lhotari pushed a commit to branch branch-2.10
in repository https://gitbox.apache.org/repos/asf/pulsar.git
The following commit(s) were added to refs/heads/branch-2.10 by this push:
new 647fa08bd82 [fix][test] Fix testPerTopicStats (#22870)
647fa08bd82 is described below
commit 647fa08bd8227f7ef6f1ca9b459b7cd040563bc4
Author: Zixuan Liu
AuthorDate: Fri Jun 7 20:23:28 2024 +0800
[fix][test] Fix testPerTopicStats (#22870)
Signed-off-by: Zixuan Liu
---
.../test/java/org/apache/pulsar/broker/stats/PrometheusMetricsTest.java | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/PrometheusMetricsTest.java
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/PrometheusMetricsTest.java
index 245a05acda7..722c6c02b9e 100644
---
a/pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/PrometheusMetricsTest.java
+++
b/pulsar-broker/src/test/java/org/apache/pulsar/broker/stats/PrometheusMetricsTest.java
@@ -316,7 +316,7 @@ public class PrometheusMetricsTest extends BrokerTestBase {
assertEquals(cm.size(), 1);
assertEquals(cm.get(0).tags.get("cluster"), "test");
-cm = (List) metrics.get("topic_load_failed_total");
+cm = (List) metrics.get("topic_load_failed");
assertEquals(cm.size(), 1);
assertEquals(cm.get(0).tags.get("cluster"), "test");
Re: [PR] [fix][test] Fix testPerTopicStats [pulsar]
lhotari merged PR #22870: URL: https://github.com/apache/pulsar/pull/22870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] Flaky-test: ExtensibleLoadManagerImplTest.testGetMetrics (fails consistently) [pulsar]
lhotari opened a new issue, #22871:
URL: https://github.com/apache/pulsar/issues/22871
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues)
and found nothing similar.
### Example failure
https://github.com/apache/pulsar/actions/runs/9412677464/job/25928002659?pr=22867#step:9:2239
### Exception stacktrace
```
Error: Tests run: 45, Failures: 1, Errors: 0, Skipped: 1, Time elapsed:
130.133 s <<< FAILURE! - in
org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest
Error:
org.apache.pulsar.broker.loadbalance.extensions.ExtensibleLoadManagerImplTest.testGetMetrics
Time elapsed: 0.008 s <<< FAILURE!
java.lang.AssertionError: Sets differ: expected
[dimensions=[{broker=localhost, metric=loadBalancing}],
metrics=[{brk_lb_bandwidth_in_usage=3.0, brk_lb_bandwidth_out_usage=4.0,
brk_lb_cpu_usage=1.0, brk_lb_directMemory_usage=2.0,
brk_lb_memory_usage=400.0}], dimensions=[{broker=localhost, event=Unload,
metric=sunitStateChn, result=Total}],
metrics=[{brk_sunit_state_chn_event_publish_ops_total=5}],
dimensions=[{broker=localhost, metric=sunitStateChn, result=Cancel}],
metrics=[{brk_sunit_state_chn_inactive_broker_cleanup_ops_total=7}],
dimensions=[{broker=localhost, metric=bundlesSplit, reason=Unknown,
result=Failure}], metrics=[{brk_lb_bundles_split_breakdown_total=6}],
dimensions=[{broker=localhost, event=Assigning, metric=sunitStateChn,
result=Failure}], metrics=[{brk_sunit_state_chn_subscribe_ops_total=8}],
dimensions=[{broker=localhost, metric=sunitStateChn, result=Failure,
state=Deleted}], metrics=[{brk_sunit_state_chn_owner_lookup_total=14}],
dimensions=[{broker=localhost,
metric=bundleUnloading, reason=Unknown, result=Skip}],
metrics=[{brk_lb_unload_broker_breakdown_total=9}],
dimensions=[{broker=localhost, metric=bundleUnloading, reason=NoBundles,
result=Skip}], metrics=[{brk_lb_unload_broker_breakdown_total=4}],
dimensions=[{broker=localhost, event=Init, metric=sunitStateChn,
result=Total}], metrics=[{brk_sunit_state_chn_subscribe_ops_total=1}],
dimensions=[{broker=localhost, event=Splitting, metric=sunitStateChn,
result=Failure}], metrics=[{brk_sunit_state_chn_subscribe_ops_total=12}],
dimensions=[{broker=localhost, event=Assign, metric=sunitStateChn,
result=Total}], metrics=[{brk_sunit_state_chn_event_publish_ops_total=1}],
dimensions=[{broker=localhost, metric=bundlesSplit, reason=Sessions,
result=Success}], metrics=[{brk_lb_bundles_split_breakdown_total=2}],
dimensions=[{broker=localhost, event=Split, metric=sunitStateChn,
result=Total}], metrics=[{brk_sunit_state_chn_event_publish_ops_total=3}],
dimensions=[{broker=localhost, metric=sunitState
Chn, result=Failure, state=Releasing}],
metrics=[{brk_sunit_state_chn_owner_lookup_total=10}],
dimensions=[{broker=localhost, feature=max_ema, metric=bundleUnloading,
stat=avg}], metrics=[{brk_lb_resource_usage_stats=1.5}],
dimensions=[{broker=localhost, metric=sunitStateChn, result=Failure,
state=Splitting}], metrics=[{brk_sunit_state_chn_owner_lookup_total=12}],
dimensions=[{broker=localhost, event=Assign, metric=sunitStateChn,
result=Failure}], metrics=[{brk_sunit_state_chn_event_publish_ops_total=2}],
dimensions=[{broker=localhost, event=Free, metric=sunitStateChn,
result=Total}], metrics=[{brk_sunit_state_chn_subscribe_ops_total=3}],
dimensions=[{broker=localhost, metric=bundleUnloading, reason=Overloaded,
result=Success}], metrics=[{brk_lb_unload_broker_breakdown_total=1}],
dimensions=[{broker=localhost, metric=bundleUnloading, reason=OutDatedData,
result=Skip}], metrics=[{brk_lb_unload_broker_breakdown_total=6}],
dimensions=[{broker=localhost, metric=bundleUnloading, reason=N
oBrokers, result=Skip}], metrics=[{brk_lb_unload_broker_breakdown_total=8}],
dimensions=[{broker=localhost, metric=sunitStateChn, result=Failure,
state=Assigning}], metrics=[{brk_sunit_state_chn_owner_lookup_total=8}],
dimensions=[{broker=localhost, metric=bundlesSplit, reason=MsgRate,
result=Success}], metrics=[{brk_lb_bundles_split_breakdown_total=3}],
dimensions=[{broker=localhost, event=Assigning, metric=sunitStateChn,
result=Total}], metrics=[{brk_sunit_state_chn_subscribe_ops_total=7}],
dimensions=[{broker=localhost, event=Free, metric=sunitStateChn,
result=Failure}], metrics=[{brk_sunit_state_chn_subscribe_ops_total=4}],
dimensions=[{broker=localhost, feature=max, metric=loadBalancing}],
metrics=[{brk_lb_resource_usage=0.04}], dimensions=[{broker=localhost,
metric=bundleUnloading, reason=CoolDown, result=Skip}],
metrics=[{brk_lb_unload_broker_breakdown_total=5}],
dimensions=[{broker=localhost, metric=sunitStateChn, result=Failure,
state=Init}], metrics=[{brk_sunit_state_chn_o
wner_lookup_total=2}], dimensions=[{broker=localhost, event=Deleted,
metric=sunitStateChn, result=Failure}],
metrics=[{brk_sunit_state_chn_subscribe_ops_total=14}],
dimensions=[{broker=localhost,
svn commit: r69593 - /dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/
Author: djensen Date: Fri Jun 7 11:16:57 2024 New Revision: 69593 Log: Create DotPulsar pre-release dir Added: dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/
svn commit: r69594 - in /dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1: pulsar-dotpulsar-3.3.0-src.tar.gz pulsar-dotpulsar-3.3.0-src.tar.gz.asc pulsar-dotpulsar-3.3.0-src.tar.gz.sha512
Author: djensen Date: Fri Jun 7 11:17:01 2024 New Revision: 69594 Log: Staging artifacts and signature for DotPulsar 3.3.0-rc.1 Added: dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/pulsar-dotpulsar-3.3.0-src.tar.gz (with props) dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/pulsar-dotpulsar-3.3.0-src.tar.gz.asc dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/pulsar-dotpulsar-3.3.0-src.tar.gz.sha512 Added: dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/pulsar-dotpulsar-3.3.0-src.tar.gz == Binary file - no diff available. Propchange: dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/pulsar-dotpulsar-3.3.0-src.tar.gz -- svn:mime-type = application/octet-stream Added: dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/pulsar-dotpulsar-3.3.0-src.tar.gz.asc == --- dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/pulsar-dotpulsar-3.3.0-src.tar.gz.asc (added) +++ dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/pulsar-dotpulsar-3.3.0-src.tar.gz.asc Fri Jun 7 11:17:01 2024 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEEBNpmyfPM8T5xTvymkG5uByPUwhUFAmZi7CcACgkQkG5uByPU +whUnBhAAks6yX/P91ZsCgNqXZBfCF4QZtZ4ktGrS0SAMTciWluUDX+Nzt/f/SFfe +s21sFhE4rN3fsafK8lZuP8z+d4dIRPPHNd+3olgBtOEhhm7TO3Jj2Yg0h8S/EM+/ +qyWLt63fiHjAkyDeojzNwH25qQFDhFiS5kNFhZVZO5kiAdCDAKY9p6Y/pEYqbAQL +k5ubqgL61Lp1Bp3Xma94kv2IIe2+6lyQqu3fe6k+kZnk5j4+FOhTUzKt2EO/wOhA +fJ51ptMyCQSuyJZ3hyIn7ELS9JDl9NXF/7MsMapCNx67sWLE7PLUcJ0xetDHhkQd +yhGn5tGZouQLUrcNgZn7qFYeXpXlKOvEl8TpDf1u5lqSuOEf+1543vmj2FyJ4wCY +0vQhNL47FQQGkCQ2Pxl7DmglTUrkOjP4RC+hM8cWbUEIxPpaZu0Crj2w7cU4Zyil +wVBSY67HmROgaDjAMfqV1Urk72Yf9VSp59FtZ/cGzmGcOgq6M9ggBLwEo5+Rg+tx +rz4Uy3KZ02OLSBggtYO01cdeFKBKNZ8qPfu5Fdc2eNw3WHY3FOPVEehG5ebDPPbg +EQkA+whw6sH/XPG5YwYSGtVWI1suLjitN4T8dweCUcfSyX85cjKGURYUZhbgN38p +wlbvEQNWVyKmSYBZXTlfzJEoFnq175P/LCxu0mprQR7TOGaclYA= +=NX2p +-END PGP SIGNATURE- Added: dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/pulsar-dotpulsar-3.3.0-src.tar.gz.sha512 == --- dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/pulsar-dotpulsar-3.3.0-src.tar.gz.sha512 (added) +++ dev/pulsar/pulsar-dotpulsar-3.3.0-rc.1/pulsar-dotpulsar-3.3.0-src.tar.gz.sha512 Fri Jun 7 11:17:01 2024 @@ -0,0 +1 @@ +8f629fbd6611550e4715dd1b6e1bd7fa96e302af07f4ff26be2ce46e8c627c247da696acd445cfe847c718bf56cecf0c8cf792b43faf23407a7a7162d0cc66e5 pulsar-dotpulsar-3.3.0-src.tar.gz
(pulsar-dotpulsar) tag 3.3.0-rc.1 created (now c95ee48)
This is an automated email from the ASF dual-hosted git repository. djensen pushed a change to tag 3.3.0-rc.1 in repository https://gitbox.apache.org/repos/asf/pulsar-dotpulsar.git at c95ee48 (commit) No new revisions were added by this update.
(pulsar-dotpulsar) branch master updated: Make ready for release 3.3.0-rc.1
This is an automated email from the ASF dual-hosted git repository. djensen pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-dotpulsar.git The following commit(s) were added to refs/heads/master by this push: new c95ee48 Make ready for release 3.3.0-rc.1 c95ee48 is described below commit c95ee48ade3f90cd45b532a746e2e5476e30f19c Author: entvex <1580435+ent...@users.noreply.github.com> AuthorDate: Fri Jun 7 13:10:12 2024 +0200 Make ready for release 3.3.0-rc.1 --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 18625ca..c23db24 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). -## [Unreleased] +## [3.3.0-rc.1] - 2024-06-07 ### Added
Re: [PR] [fix] [test] fix testGetMetrics in ExtensibleLoadManagerImplTest. [pulsar]
thetumbled commented on PR #22864: URL: https://github.com/apache/pulsar/pull/22864#issuecomment-2154534445 > I'm wondering why this failed test was not detected before? This is a flaky test, which can be ignored when merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [fix] [test] fix testGetMetrics in ExtensibleLoadManagerImplTest. [pulsar]
BewareMyPower commented on PR #22864: URL: https://github.com/apache/pulsar/pull/22864#issuecomment-2154530034 I'm wondering why this failed test was not detected before? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [fix][test] Fix testPerTopicStats [pulsar]
nodece opened a new pull request, #22870: URL: https://github.com/apache/pulsar/pull/22870 ### Motivation https://github.com/apache/pulsar/commit/704630b35fcf23d802ad1fc9bc27d592189a53a4 breaks the testPerTopicStats test. ### Modifications Remove the suffix `_total` from `topic_load_failed_total` metric name. ### Documentation - [ ] `doc` - [ ] `doc-required` - [x] `doc-not-needed` - [ ] `doc-complete` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
