[PR] Bump serve-static from 1.13.2 to 1.16.0 in /front-end [pulsar-manager]

2024-09-10 Thread via GitHub
dependabot[bot] opened a new pull request, #575: URL: https://github.com/apache/pulsar-manager/pull/575 Bumps [serve-static](https://github.com/expressjs/serve-static) from 1.13.2 to 1.16.0. Release notes Sourced from https://github.com/expressjs/serve-static/releases";>serve-stati

Re: [PR] [fix][broker] fix the log format error in the AdminResource [pulsar]

2024-09-10 Thread via GitHub
Technoboy- merged PR #23278: URL: https://github.com/apache/pulsar/pull/23278 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar

[I] Performance regression by 3.5.0 when batching is enabled [pulsar-client-python]

2024-09-10 Thread via GitHub
BewareMyPower opened a new issue, #226: URL: https://github.com/apache/pulsar-client-python/issues/226 Output of that with pulsar-client v.2.10.2: ``` Sent 1000 messages in 2.70 seconds Sent 2000 messages in 5.40 seconds Sent 3000 messages in 8.16 seconds Sent 4000 messages

Re: [PR] [fix][broker] fix the log format error in the AdminResource [pulsar]

2024-09-10 Thread via GitHub
codecov-commenter commented on PR #23278: URL: https://github.com/apache/pulsar/pull/23278#issuecomment-2340313710 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/23278?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments

Re: [PR] [fix] [broker] Fix system topic can not be loaded up if it contains data offloaded [pulsar]

2024-09-10 Thread via GitHub
mattisonchao commented on code in PR #23279: URL: https://github.com/apache/pulsar/pull/23279#discussion_r1751545915 ## managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java: ## @@ -2587,10 +2596,7 @@ void internalTrimLedgers(boolean isTruncate,

[I] [Bug] read entry failed cause `No such ledger exists on Bookies` with Orphan non-durable cursor [pulsar]

2024-09-10 Thread via GitHub
ethqunzhong opened a new issue, #23280: URL: https://github.com/apache/pulsar/issues/23280 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Read release policy - [X] I understand that un

Re: [PR] [fix] [broker] Fix system topic can not be loaded up if it contains data offloaded [pulsar]

2024-09-10 Thread via GitHub
poorbarcode commented on code in PR #23279: URL: https://github.com/apache/pulsar/pull/23279#discussion_r1751537917 ## managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java: ## @@ -3117,8 +3125,11 @@ public void offloadFailed(ManagedLedgerExcepti

Re: [PR] [fix] [broker] Fix system topic can not be loaded up if it contains data offloaded [pulsar]

2024-09-10 Thread via GitHub
poorbarcode commented on code in PR #23279: URL: https://github.com/apache/pulsar/pull/23279#discussion_r1751536898 ## managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ReadonlyWrapperLedgerOffloader.java: ## @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Software

Re: [PR] [fix] [broker] Fix system topic can not be loaded up if it contains data offloaded [pulsar]

2024-09-10 Thread via GitHub
mattisonchao commented on code in PR #23279: URL: https://github.com/apache/pulsar/pull/23279#discussion_r1751500211 ## managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ReadonlyWrapperLedgerOffloader.java: ## @@ -0,0 +1,69 @@ +/* + * Licensed to the Apache Softwar

Re: [PR] [feat][meta] Bump oxia java version from 0.3.2 to 0.4.5 [pulsar]

2024-09-10 Thread via GitHub
codecov-commenter commented on PR #23277: URL: https://github.com/apache/pulsar/pull/23277#issuecomment-2339992534 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/23277?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments

Re: [PR] [fix] [broker] Fix system topic can not be loaded up if it contains data offloaded [pulsar]

2024-09-10 Thread via GitHub
mattisonchao commented on code in PR #23279: URL: https://github.com/apache/pulsar/pull/23279#discussion_r1751491517 ## managed-ledger/src/main/java/org/apache/bookkeeper/mledger/impl/ManagedLedgerImpl.java: ## @@ -2572,6 +2572,17 @@ void internalTrimConsumedLedgers(CompletableF

Re: [D] Key_Shared consumer scale behavior with a partitioned topic [pulsar]

2024-09-10 Thread via GitHub
GitHub user lhotari edited a comment on the discussion: Key_Shared consumer scale behavior with a partitioned topic I'm currently working on a new design for Key_Shared AUTO_SPLIT mode which would cover the requirement of individual hash tracking. The goal is that a message will only get bloc

[PR] [fix] [broker] Fix system topic can not be loaded up if it contains data offloaded [pulsar]

2024-09-10 Thread via GitHub
poorbarcode opened a new pull request, #23279: URL: https://github.com/apache/pulsar/pull/23279 ### Motivation After https://github.com/apache/pulsar/pull/22497, the broker will not offload data or read data from offloaded for the system topic, the PR has not considered compatibility

[PR] [fix][broker] fix the log format error in the AdminResource [pulsar]

2024-09-10 Thread via GitHub
RobertIndie opened a new pull request, #23278: URL: https://github.com/apache/pulsar/pull/23278 ## Motivation The log message in `AdminResource` was missing a parameter. The `topicName` was not being included in the success log message when creating partitioned topics for rem

[PR] [feat][metadata] Bump oxia java version from 0.3.2 to 0.4.5 [pulsar]

2024-09-09 Thread via GitHub
mattisonchao opened a new pull request, #23277: URL: https://github.com/apache/pulsar/pull/23277 ### Motivation Bump oxia java version from 0.3.2 to 0.4.5 ### Modifications - Bump oxia java version from 0.3.2 to 0.4.5 ### Verifying this change - [x] Make s

Re: [PR] [fix] Bump io.grpc from 1.56.0 to 1.56.1 [pulsar]

2024-09-09 Thread via GitHub
mattisonchao merged PR #23276: URL: https://github.com/apache/pulsar/pull/23276 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@puls

Re: [PR] [fix] Bump io.grpc from 1.56.0 to 1.56.1 [pulsar]

2024-09-09 Thread via GitHub
mattisonchao commented on PR #23276: URL: https://github.com/apache/pulsar/pull/23276#issuecomment-2339694346 >Please share the context to others about why this is important for Pulsar. Pulsar is pluggable messaging system. For some plugins that is using GRPC as the RPC framework will

Re: [PR] [fix] Bump io.grpc from 1.56.0 to 1.56.1 [pulsar]

2024-09-09 Thread via GitHub
lhotari commented on PR #23276: URL: https://github.com/apache/pulsar/pull/23276#issuecomment-2339677171 > Bump io.grpc from 1.56.0 to 1.56.1 to address this super important fix. [grpc/grpc-java#10328](https://github.com/grpc/grpc-java/pull/10328) @mattisonchao Please share the contex

[PR] Bump io.grpc from 1.56.0 to 1.56.1 [pulsar]

2024-09-09 Thread via GitHub
mattisonchao opened a new pull request, #23276: URL: https://github.com/apache/pulsar/pull/23276 ### Motivation Bump io.grpc from 1.56.0 to 1.56.1 to address this super important fix. https://github.com/grpc/grpc-java/pull/10328 ### Modifications - Bump io.grpc from 1.56

Re: [PR] [fix][client] Fix broker/Client CPU reaching 100% during retriable connection failure [pulsar]

2024-09-09 Thread via GitHub
rdhabalia merged PR #23251: URL: https://github.com/apache/pulsar/pull/23251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.

Re: [I] [Bug] Broker seeing high latency, high CPU and high GC Pauses [pulsar]

2024-09-09 Thread via GitHub
rdhabalia closed issue #23250: [Bug] Broker seeing high latency, high CPU and high GC Pauses URL: https://github.com/apache/pulsar/issues/23250 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[I] Use pulsar-client-go-test docker to test TestBlueGreenMigrationTestSuite case [pulsar-client-go]

2024-09-09 Thread via GitHub
geniusjoe opened a new issue, #1282: URL: https://github.com/apache/pulsar-client-go/issues/1282 **Is your feature request related to a problem? Please describe.** When I ran `make test`, every test case worked fine except one error from `test_extensible_load_manager` : ``` ✔ Netwo

Re: [PR] RPC framework implemented by the Pulsar client [pulsar-java-contrib]

2024-09-09 Thread via GitHub
Denovo1998 commented on code in PR #6: URL: https://github.com/apache/pulsar-java-contrib/pull/6#discussion_r1750283332 ## pulsar-rpc-contrib/src/main/java/org/apache/pulsar/rpc/contrib/client/PulsarRpcClient.java: ## @@ -0,0 +1,180 @@ +/* + * Licensed under the Apache License,

Re: [PR] RPC framework implemented by the Pulsar client [pulsar-java-contrib]

2024-09-09 Thread via GitHub
Denovo1998 commented on code in PR #6: URL: https://github.com/apache/pulsar-java-contrib/pull/6#discussion_r1750297371 ## pulsar-rpc-contrib/src/main/java/org/apache/pulsar/rpc/contrib/client/ReplyListener.java: ## @@ -0,0 +1,74 @@ +/* + * Licensed under the Apache License, Ver

Re: [PR] RPC framework implemented by the Pulsar client [pulsar-java-contrib]

2024-09-09 Thread via GitHub
Denovo1998 commented on code in PR #6: URL: https://github.com/apache/pulsar-java-contrib/pull/6#discussion_r1750283847 ## pulsar-rpc-contrib/src/main/java/org/apache/pulsar/rpc/contrib/server/RequestListener.java: ## @@ -0,0 +1,88 @@ +/* + * Licensed under the Apache License, V

Re: [PR] RPC framework implemented by the Pulsar client [pulsar-java-contrib]

2024-09-09 Thread via GitHub
Denovo1998 commented on code in PR #6: URL: https://github.com/apache/pulsar-java-contrib/pull/6#discussion_r1750283332 ## pulsar-rpc-contrib/src/main/java/org/apache/pulsar/rpc/contrib/client/PulsarRpcClient.java: ## @@ -0,0 +1,180 @@ +/* + * Licensed under the Apache License,

Re: [PR] RPC framework implemented by the Pulsar client [pulsar-java-contrib]

2024-09-09 Thread via GitHub
Denovo1998 commented on code in PR #6: URL: https://github.com/apache/pulsar-java-contrib/pull/6#discussion_r1750279670 ## pulsar-rpc-contrib/src/test/java/org/apache/pulsar/rpc/contrib/SimpleRpcCallTest.java: ## @@ -0,0 +1,476 @@ +/* + * Licensed under the Apache License, Versi

Re: [PR] [fix][broker] Fix for Broker Restart Failure Due to Incomplete NAR File Extraction in /tmp Directory [pulsar]

2024-09-09 Thread via GitHub
nikhilerigila09 commented on code in PR #23274: URL: https://github.com/apache/pulsar/pull/23274#discussion_r1750129564 ## pulsar-common/src/main/java/org/apache/pulsar/common/nar/NarUnpacker.java: ## @@ -83,22 +83,31 @@ static File doUnpackNar(final File nar, final File baseWo

Re: [PR] [fix][broker] Fix for Broker Restart Failure Due to Incomplete NAR File Extraction in /tmp Directory [pulsar]

2024-09-09 Thread via GitHub
nikhilerigila09 commented on code in PR #23274: URL: https://github.com/apache/pulsar/pull/23274#discussion_r1750129564 ## pulsar-common/src/main/java/org/apache/pulsar/common/nar/NarUnpacker.java: ## @@ -83,22 +83,31 @@ static File doUnpackNar(final File nar, final File baseWo

Re: [PR] [go]: reduce dependency [pulsar-client-go]

2024-09-09 Thread via GitHub
ttys3 commented on PR #819: URL: https://github.com/apache/pulsar-client-go/pull/819#issuecomment-2337878682 > +1 to upgrading the cobra dependency However, I'm not sure if it's a good idea to split pulsar-perf-go to a separate module. In general it's recommended to have a single go module

Re: [I] [Bug] Broker Fails to Restart Due to Incomplete NAR File Extraction in `/tmp` Directory [pulsar]

2024-09-09 Thread via GitHub
lhotari commented on issue #23273: URL: https://github.com/apache/pulsar/issues/23273#issuecomment-2337766524 > * Suggested fix: Implement a mechanism that writes a ".success" file after successful NAR extraction and ensures this file is present before using the directory. An alterna

Re: [D] Key_Shared consumer scale behavior with a partitioned topic [pulsar]

2024-09-09 Thread via GitHub
GitHub user lhotari added a comment to the discussion: Key_Shared consumer scale behavior with a partitioned topic There's yet another aspect of the individual hash tracking. Currently, when new consumers join, all message delivery is blocked for the consumer until the inflight messages that

[I] [Bug] Dispatcher will be closing and reopening when Key_shared consumers have different policies (AUTO_SPLIT/STICKY, allowOutOfOrderDelivery true/false) [pulsar]

2024-09-09 Thread via GitHub
lhotari opened a new issue, #23272: URL: https://github.com/apache/pulsar/issues/23272 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Read release policy - [X] I understand that unsupp

Re: [PR] [fix][broker] fix pulsar-admin topics stats-internal caused a BK client thread a deadlock [pulsar]

2024-09-09 Thread via GitHub
codecov-commenter commented on PR #23258: URL: https://github.com/apache/pulsar/pull/23258#issuecomment-2337718462 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/23258?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments

[I] [Bug] Key_Shared subscription's "recently joined consumer" logic is unnecessary for KeySharedMode.STICKY [pulsar]

2024-09-09 Thread via GitHub
lhotari opened a new issue, #23271: URL: https://github.com/apache/pulsar/issues/23271 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Read release policy - [X] I understand that unsupp

Re: [PR] [improve][broker] Make cluster metadata init command support metadata config path [pulsar]

2024-09-09 Thread via GitHub
lhotari commented on PR #23269: URL: https://github.com/apache/pulsar/pull/23269#issuecomment-2337589985 It seems that this is related to PIP-366 #23033 and should go only in master branch? -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] [fix][broker] fix pulsar-admin topics stats-internal caused a BK client thread a deadlock [pulsar]

2024-09-09 Thread via GitHub
poorbarcode commented on code in PR #23258: URL: https://github.com/apache/pulsar/pull/23258#discussion_r1749751058 ## pulsar-broker/src/main/java/org/apache/pulsar/compaction/CompactedTopicImpl.java: ## @@ -304,8 +306,10 @@ static CompletableFuture> readEntries(LedgerHandle lh

Re: [PR] RPC framework implemented by the Pulsar client [pulsar-java-contrib]

2024-09-08 Thread via GitHub
liangyepianzhou commented on code in PR #6: URL: https://github.com/apache/pulsar-java-contrib/pull/6#discussion_r1749628176 ## pulsar-rpc-contrib/src/main/java/org/apache/pulsar/rpc/contrib/client/PulsarRpcClient.java: ## @@ -0,0 +1,180 @@ +/* + * Licensed under the Apache Lice

Re: [PR] RPC framework implemented by the Pulsar client [pulsar-java-contrib]

2024-09-08 Thread via GitHub
Denovo1998 commented on code in PR #6: URL: https://github.com/apache/pulsar-java-contrib/pull/6#discussion_r1749529755 ## pulsar-rpc-contrib/src/main/java/org/apache/pulsar/rpc/contrib/client/PulsarRpcClient.java: ## @@ -0,0 +1,180 @@ +/* + * Licensed under the Apache License,

Re: [PR] RPC framework implemented by the Pulsar client [pulsar-java-contrib]

2024-09-08 Thread via GitHub
liangyepianzhou commented on PR #6: URL: https://github.com/apache/pulsar-java-contrib/pull/6#issuecomment-2337002045 > **Feedback is not positive due to PIP-371.** [apache/pulsar#23143](https://github.com/apache/pulsar/pull/23143) [apache/pulsar#23194](https://github.com/apache/pulsar/pul

Re: [PR] [fix][broker] Execute the pending callbacks in order before ready for incoming requests [pulsar]

2024-09-08 Thread via GitHub
BewareMyPower merged PR #23266: URL: https://github.com/apache/pulsar/pull/23266 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pul

Re: [PR] [fix][broker] Execute the pending callbacks in order before ready for incoming requests [pulsar]

2024-09-08 Thread via GitHub
codecov-commenter commented on PR #23266: URL: https://github.com/apache/pulsar/pull/23266#issuecomment-2336679392 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/23266?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments

Re: [PR] [fix][broker] Execute the pending callbacks in order before ready for incoming requests [pulsar]

2024-09-07 Thread via GitHub
lhotari commented on code in PR #23266: URL: https://github.com/apache/pulsar/pull/23266#discussion_r1748053636 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java: ## @@ -1082,7 +1087,16 @@ public void start() throws PulsarServerException { }

Re: [PR] [fix][broker] Execute the pending callbacks in order before ready for incoming requests [pulsar]

2024-09-07 Thread via GitHub
lhotari commented on code in PR #23266: URL: https://github.com/apache/pulsar/pull/23266#discussion_r1748053363 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java: ## @@ -1023,7 +1024,11 @@ public void start() throws PulsarServerException { t

Re: [PR] [fix][broker] Execute the pending callbacks in order before ready for incoming requests [pulsar]

2024-09-07 Thread via GitHub
BewareMyPower commented on code in PR #23266: URL: https://github.com/apache/pulsar/pull/23266#discussion_r1748049579 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java: ## @@ -1082,7 +1087,16 @@ public void start() throws PulsarServerException { }

Re: [I] [Bug] Msg backlog & unack msg remains when using acknowledgeAsync [pulsar]

2024-09-07 Thread via GitHub
lhotari commented on issue #21958: URL: https://github.com/apache/pulsar/issues/21958#issuecomment-2335126309 I have created a proposal "PIP-377: Automatic retry for failed acknowledgements", https://github.com/apache/pulsar/pull/23267 ([rendered doc](https://github.com/lhotari/pulsar/blob/

Re: [I] [Bug] Acknowledgements lost on bookies and brokers restart resulting in messages not being delivered [pulsar]

2024-09-07 Thread via GitHub
lhotari commented on issue #22709: URL: https://github.com/apache/pulsar/issues/22709#issuecomment-2335126220 I have created a proposal "PIP-377: Automatic retry for failed acknowledgements", https://github.com/apache/pulsar/pull/23267 ([rendered doc](https://github.com/lhotari/pulsar/blob/

Re: [I] [Bug] Acknowledgements lost on bookies and brokers restart resulting in messages not being delivered [pulsar]

2024-09-07 Thread via GitHub
lhotari commented on issue #22709: URL: https://github.com/apache/pulsar/issues/22709#issuecomment-2335104724 Acknowledgements will get lost in broker restarts. That's part of current design. That will cause Key_Shared consumers to "get stuck" and the messages aren't delivered (not technica

Re: [I] [Bug] Subscription consumption stuck on consumer reconnect [pulsar]

2024-09-07 Thread via GitHub
lhotari commented on issue #21199: URL: https://github.com/apache/pulsar/issues/21199#issuecomment-2335103456 It seems that this issue might be addressed together with PIP-282 changes #21953 and other PRs #23226 (merged) and #23231 (in-progress). A remaining challenge is the lost acknowl

Re: [PR] feat: introduce ack trackers to prevent batch message loss [pulsar-client-go]

2024-09-06 Thread via GitHub
KennyChenFight commented on PR #1277: URL: https://github.com/apache/pulsar-client-go/pull/1277#issuecomment-2335051805 > > the messages added to ack_grouping_tracker are considered ready to be acknowledged. > > You can create a map to record the batch message, such as `batchMessage

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-06 Thread via GitHub
lhotari commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1747682951 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStickyKeyDispatcherMultipleConsumers.java: ## @@ -274,96 +251,58 @@ protected sync

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-06 Thread via GitHub
lhotari commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1747644825 ## conf/broker.conf: ## @@ -355,6 +355,25 @@ maxUnackedMessagesPerBroker=0 # limit/2 messages maxUnackedMessagesPerSubscriptionOnBrokerBlocked=0.16 +# For Key_Shar

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-06 Thread via GitHub
lhotari commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1747336861 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStickyKeyDispatcherMultipleConsumers.java: ## @@ -183,22 +180,6 @@ public synchron

Re: [PR] [fix][broker] Execute the pending callbacks in order before ready for incoming requests [pulsar]

2024-09-06 Thread via GitHub
lhotari commented on code in PR #23266: URL: https://github.com/apache/pulsar/pull/23266#discussion_r1747293258 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/PulsarService.java: ## @@ -1082,7 +1087,16 @@ public void start() throws PulsarServerException { }

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-06 Thread via GitHub
lhotari commented on PR #23231: URL: https://github.com/apache/pulsar/pull/23231#issuecomment-2334013781 Noticing more challenges to cover while getting into the tests. Setting this to draft mode until the concerns are addressed and tests have been added. Some of the concerns are related to

Re: [D] Key_Shared consumer scale behavior with a partitioned topic [pulsar]

2024-09-06 Thread via GitHub
GitHub user lhotari added a comment to the discussion: Key_Shared consumer scale behavior with a partitioned topic > Perhaps a future improvement could be to allow tracking an individual hash > value in the range 0-65535. @niriven This is being implemented as part of #23231. It makes improvem

Re: [PR] [improve][broker] Log warning when chunked messages appear with Key_Shared subscriptions [pulsar]

2024-09-06 Thread via GitHub
lhotari closed pull request #23265: [improve][broker] Log warning when chunked messages appear with Key_Shared subscriptions URL: https://github.com/apache/pulsar/pull/23265 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-06 Thread via GitHub
lhotari commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1746810620 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java: ## @@ -949,6 +949,48 @@ The max allowed delay for delayed delivery (in milli

[I] [Bug] Consumers getting stuck since read more entries doesn't properly handle retries with backoff [pulsar]

2024-09-06 Thread via GitHub
lhotari opened a new issue, #23264: URL: https://github.com/apache/pulsar/issues/23264 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Read release policy - [X] I understand that unsupp

Re: [PR] [improve][pip] PIP-376: Make topic policies service pluggable [pulsar]

2024-09-06 Thread via GitHub
RobertIndie commented on code in PR #23248: URL: https://github.com/apache/pulsar/pull/23248#discussion_r1746723342 ## pip/pip-376.md: ## @@ -0,0 +1,222 @@ +# PIP-376: Make Topic Policies Service Pluggable + +## Background + +### Topic Policies Service and System Topics + +[PIP-

Re: [PR] Bump cpp client version to v3.6.0 [pulsar-client-node]

2024-09-06 Thread via GitHub
shibd merged PR #392: URL: https://github.com/apache/pulsar-client-node/pull/392 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pul

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-06 Thread via GitHub
lhotari commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1746637488 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java: ## @@ -949,6 +949,48 @@ The max allowed delay for delayed delivery (in milli

[I] [Bug] Permit calculations are mixing individual message and batch message counts, similar issue in documentation [pulsar]

2024-09-06 Thread via GitHub
lhotari opened a new issue, #23263: URL: https://github.com/apache/pulsar/issues/23263 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Read release policy - [X] I understand that unsupp

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-05 Thread via GitHub
lhotari commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1746590301 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStickyKeyDispatcherMultipleConsumers.java: ## @@ -439,36 +372,162 @@ protected syn

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-05 Thread via GitHub
poorbarcode commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1746574854 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStickyKeyDispatcherMultipleConsumers.java: ## @@ -439,36 +372,162 @@ protected

Re: [PR] [fix][test] Fix flaky GetPartitionMetadataMultiBrokerTest.testCompatibilityDifferentBrokersForNonPersistentTopic [pulsar]

2024-09-05 Thread via GitHub
hanmz commented on code in PR #23259: URL: https://github.com/apache/pulsar/pull/23259#discussion_r1746514685 ## pulsar-broker/src/test/java/org/apache/pulsar/broker/admin/GetPartitionMetadataMultiBrokerTest.java: ## @@ -256,8 +256,8 @@ public void testCompatibilityDifferentBro

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-05 Thread via GitHub
lhotari commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1746499955 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStickyKeyDispatcherMultipleConsumers.java: ## @@ -439,36 +372,162 @@ protected syn

Re: [PR] Bump twisted from 23.10.0 to 24.7.0 in /tools/pytools [pulsar-site]

2024-09-05 Thread via GitHub
visortelle merged PR #960: URL: https://github.com/apache/pulsar-site/pull/960 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsa

Re: [PR] Update usecases.ts [pulsar-site]

2024-09-05 Thread via GitHub
visortelle commented on PR #955: URL: https://github.com/apache/pulsar-site/pull/955#issuecomment-2333205652 Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[PR] Bump twisted from 23.10.0 to 24.7.0 in /tools/pytools [pulsar-site]

2024-09-05 Thread via GitHub
dependabot[bot] opened a new pull request, #960: URL: https://github.com/apache/pulsar-site/pull/960 Bumps [twisted](https://github.com/twisted/twisted) from 23.10.0 to 24.7.0. Release notes Sourced from https://github.com/twisted/twisted/releases";>twisted's releases. Twist

Re: [PR] Bump webpack from 5.90.3 to 5.94.0 [pulsar-site]

2024-09-05 Thread via GitHub
visortelle merged PR #956: URL: https://github.com/apache/pulsar-site/pull/956 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsa

Re: [PR] Bump micromatch from 4.0.5 to 4.0.8 [pulsar-site]

2024-09-05 Thread via GitHub
visortelle merged PR #957: URL: https://github.com/apache/pulsar-site/pull/957 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsa

Re: [PR] Add CPP client v3.6.0 API docs [pulsar-site]

2024-09-05 Thread via GitHub
shibd merged PR #958: URL: https://github.com/apache/pulsar-site/pull/958 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apa

[PR] Bump cpp client version to v3.6.0 [pulsar-client-node]

2024-09-05 Thread via GitHub
shibd opened a new pull request, #392: URL: https://github.com/apache/pulsar-client-node/pull/392 ### Motivation Bump cpp client version to v3.6.0 ### Modifications Bump cpp client version to v3.6.0 ### Documentation - [ ] `doc-required` (Your PR nee

Re: [PR] Add release note for cpp 3.6.0 [pulsar-site]

2024-09-05 Thread via GitHub
shibd merged PR #959: URL: https://github.com/apache/pulsar-site/pull/959 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apa

[PR] Add release note for cpp 3.6.0 [pulsar-site]

2024-09-05 Thread via GitHub
shibd opened a new pull request, #959: URL: https://github.com/apache/pulsar-site/pull/959 ### ✅ Contribution Checklist - [ ] I read the [contribution guide](https://pulsar.apache.org/contribute/document-contribution/) - [ ] I updated the [versioned docs](https://pulsar.ap

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-05 Thread via GitHub
poorbarcode commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1746465968 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java: ## @@ -949,6 +949,48 @@ The max allowed delay for delayed delivery (in m

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-05 Thread via GitHub
poorbarcode commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1746465968 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java: ## @@ -949,6 +949,48 @@ The max allowed delay for delayed delivery (in m

[PR] Add CPP client v3.6.0 API docs [pulsar-site]

2024-09-05 Thread via GitHub
shibd opened a new pull request, #958: URL: https://github.com/apache/pulsar-site/pull/958 ### ✅ Contribution Checklist - [ ] I read the [contribution guide](https://pulsar.apache.org/contribute/document-contribution/) - [ ] I updated the [versioned docs](https://pulsar.ap

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-05 Thread via GitHub
poorbarcode commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1746447330 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStickyKeyDispatcherMultipleConsumers.java: ## @@ -439,36 +372,162 @@ protected

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-05 Thread via GitHub
poorbarcode commented on code in PR #23231: URL: https://github.com/apache/pulsar/pull/23231#discussion_r1746466858 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentStickyKeyDispatcherMultipleConsumers.java: ## @@ -183,22 +180,6 @@ public sync

Re: [PR] [improve][pip] PIP-374: Visibility of messages in receiverQueue for the consumers [pulsar]

2024-09-05 Thread via GitHub
coderzc commented on code in PR #23235: URL: https://github.com/apache/pulsar/pull/23235#discussion_r1746388781 ## pip/pip-374.md: ## @@ -0,0 +1,71 @@ +# PIP-374: Visibility of messages in receiverQueue for the consumers + +# Background knowledge + +When a consumer connects to t

Re: [PR] [fix][client] Fix broker/Client CPU reaching 100% during retriable connection failure [pulsar]

2024-09-05 Thread via GitHub
rdhabalia commented on PR #23251: URL: https://github.com/apache/pulsar/pull/23251#issuecomment-2332815671 done. I have sent email and will wait for 1-2 days before merging this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] [improve][broker] Add limits for Key_Shared Subscription look ahead in dispatching [pulsar]

2024-09-05 Thread via GitHub
lhotari commented on PR #23231: URL: https://github.com/apache/pulsar/pull/23231#issuecomment-2332320966 > Please do not merge this PR fast, I will finish my review tomorrow Thanks @poorbarcode. The tests are also pending. I'm planning to finish them tomorrow and hopefully your feedba

Re: [PR] [feat][broker] PIP-264: Add OpenTelemetry broker replicator metrics [pulsar]

2024-09-05 Thread via GitHub
lhotari commented on PR #22972: URL: https://github.com/apache/pulsar/pull/22972#issuecomment-2332207319 Checked again and perhaps minimizing the risk is worthwhile if it's not needed in 3.3.x. It's not a big deal to backport #23236 to branch-3.3. -- This is an automated message from the

Re: [PR] [feat][broker] PIP-264: Add OpenTelemetry broker replicator metrics [pulsar]

2024-09-05 Thread via GitHub
dragosvictor commented on PR #22972: URL: https://github.com/apache/pulsar/pull/22972#issuecomment-2332202801 It's not intended for 3.3, but it won't hurt either. No problem cherry-picking it. -- This is an automated message from the Apache Git Service. To respond to the message, please l

[I] [Bug] When ack receipts are enabled, no response is sent to the client if the topic has been unloaded or is being transferred [pulsar]

2024-09-05 Thread via GitHub
lhotari opened a new issue, #23261: URL: https://github.com/apache/pulsar/issues/23261 ### Search before asking - [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar. ### Read release policy - [X] I understand that unsupp

Re: [I] [Bug] The javadoc of ConsumerBuilder.isAckReceiptEnabled is not consistent, it references ack timeout and redelivery which isn't related to ack receipt [pulsar]

2024-09-05 Thread via GitHub
lhotari commented on issue #23249: URL: https://github.com/apache/pulsar/issues/23249#issuecomment-2332019991 Opened an ack receipt related bug report #23261 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

Re: [PR] [improve][admin] PIP-369 Change default value of `unload-scope` in `ns-isolation-policy set` [pulsar]

2024-09-05 Thread via GitHub
grssam commented on PR #23253: URL: https://github.com/apache/pulsar/pull/23253#issuecomment-2331793700 I just noticed. Why is the codecov diff comparing against a 6 months old commit rather than master's head? -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] [Release 0.5.7] post-release steps [pulsar-client-reactive]

2024-09-05 Thread via GitHub
lhotari merged PR #187: URL: https://github.com/apache/pulsar-client-reactive/pull/187 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr.

Re: [PR] [improve][admin] PIP-369 Change default value of `unload-scope` in `ns-isolation-policy set` [pulsar]

2024-09-05 Thread via GitHub
grssam commented on PR #23253: URL: https://github.com/apache/pulsar/pull/23253#issuecomment-2331646302 /pulsarbot rerun-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

Re: [PR] [improve][broker] Optimize message payload traffic for ShadowReplicator [pulsar]

2024-09-05 Thread via GitHub
lhotari commented on PR #23236: URL: https://github.com/apache/pulsar/pull/23236#issuecomment-2331325529 It looks like this would require #22972 changes for applying this cleanly to branch-3.2 . -- This is an automated message from the Apache Git Service. To respond to the message, pleas

Re: [D] How to determine if a message was consumed and the consumer details ? [pulsar]

2024-09-05 Thread via GitHub
GitHub user huxiyu added a comment to the discussion: How to determine if a message was consumed and the consumer details ? oh my god!这么重要的特性一直没有答案。我也需要... GitHub link: https://github.com/apache/pulsar/discussions/18342#discussioncomment-1020 This is an automatically sent email for

Re: [PR] [fix][broker] fix pulsar-admin topics stats-internal caused a BK client thread a deadlock [pulsar]

2024-09-05 Thread via GitHub
Technoboy- commented on code in PR #23258: URL: https://github.com/apache/pulsar/pull/23258#discussion_r1745075593 ## pulsar-broker/src/main/java/org/apache/pulsar/compaction/CompactedTopicImpl.java: ## @@ -304,8 +306,10 @@ static CompletableFuture> readEntries(LedgerHandle lh,

Re: [I] [Bug] Msg backlog & unack msg remains when using acknowledgeAsync [pulsar]

2024-09-05 Thread via GitHub
lhotari commented on issue #21958: URL: https://github.com/apache/pulsar/issues/21958#issuecomment-2330962995 > let me check tomrrow. @semistone thanks, that you be helpful. If it reproduces only within a cluster with multiple nodes and other traffic that could mean that a load balan

[PR] [fix][client] Fix concurrent lookup with properties might have different results [pulsar]

2024-09-05 Thread via GitHub
BewareMyPower opened a new pull request, #23260: URL: https://github.com/apache/pulsar/pull/23260 ### Motivation When the lookup properties are modified, the previous pending lookup might use the modified properties for lookup. ### Modifications - Combine the properties

Re: [PR] [fix][client] Fix client handle unknown exception during message-decryption and apply decryption action accordingly [pulsar]

2024-09-05 Thread via GitHub
nodece commented on code in PR #23256: URL: https://github.com/apache/pulsar/pull/23256#discussion_r1744971767 ## pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java: ## @@ -525,6 +525,10 @@ private boolean decryptDataKey(String

Re: [PR] fix: Keep compatible with old boost versions [pulsar-client-cpp]

2024-09-05 Thread via GitHub
BewareMyPower merged PR #444: URL: https://github.com/apache/pulsar-client-cpp/pull/444 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

Re: [PR] [fix][client] Fix client handle unknown exception during message-decryption and apply decryption action accordingly [pulsar]

2024-09-04 Thread via GitHub
rdhabalia commented on code in PR #23256: URL: https://github.com/apache/pulsar/pull/23256#discussion_r1744885413 ## pulsar-client-messagecrypto-bc/src/main/java/org/apache/pulsar/client/impl/crypto/MessageCryptoBc.java: ## @@ -525,6 +525,10 @@ private boolean decryptDataKey(Str

[PR] [fix][test] Fix flaky GetPartitionMetadataMultiBrokerTest.testCompatibilityDifferentBrokersForNonPersistentTopic [pulsar]

2024-09-04 Thread via GitHub
hanmz opened a new pull request, #23259: URL: https://github.com/apache/pulsar/pull/23259 ### Motivation ![image](https://github.com/user-attachments/assets/e2fe41ee-4a95-4be5-97eb-d0a12c3ec982) The root cause of this problem is that the `getBroker` method is asynchronous. When th

<    3   4   5   6   7   8   9   10   11   12   >