Re: [PR] [improve][ci] Add arm64 image build [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on code in PR #22755: URL: https://github.com/apache/pulsar/pull/22755#discussion_r1611047785 ## .github/workflows/pulsar-ci.yaml: ## @@ -465,6 +465,12 @@ jobs: timeout-minutes: 60 needs: ['preconditions', 'build-and-license-check'] if: ${{ needs

Re: [I] [Bug] parseMessageMetadata error when broker entry metadata enable with high loading [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on issue #22601: URL: https://github.com/apache/pulsar/issues/22601#issuecomment-2126291607 > I tested with that patch.. rollback all libs to version 3.2.2 and only replace pulsar-testclient.jar and pulsar-common.jar I also tried to run my test in one data center only. unf

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
BewareMyPower commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1610981655 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java: ## @@ -1122,6 +1126,32 @@ public void writeBrokerDataOnZooK

Re: [PR] [feat] PIP-188 Support blue-green migration [pulsar-client-go]

2024-05-22 Thread via GitHub
RobertIndie commented on code in PR #1210: URL: https://github.com/apache/pulsar-client-go/pull/1210#discussion_r1610912960 ## pulsar/internal/connection.go: ## @@ -928,6 +937,51 @@ func (c *connection) handleCloseProducer(closeProducer *pb.CommandCloseProducer) } }

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
heesung-sn commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1610931877 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java: ## @@ -1122,6 +1126,32 @@ public void writeBrokerDataOnZooKeep

(pulsar) branch master updated: [fix][test] Fix flaky test testShadowWrites (#22745)

2024-05-22 Thread mmerli
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new 99bff72b7d2 [fix][test] Fix flaky test testShado

Re: [I] Flaky-test: ShadowManagedLedgerImplTest.testShadowWrites [pulsar]

2024-05-22 Thread via GitHub
merlimat closed issue #22345: Flaky-test: ShadowManagedLedgerImplTest.testShadowWrites URL: https://github.com/apache/pulsar/issues/22345 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [fix][test] Fix flaky test testShadowWrites [pulsar]

2024-05-22 Thread via GitHub
merlimat merged PR #22745: URL: https://github.com/apache/pulsar/pull/22745 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.a

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
thetumbled commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1610892192 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java: ## @@ -1122,6 +1126,32 @@ public void writeBrokerDataOnZooKeep

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
thetumbled commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1610892192 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java: ## @@ -1122,6 +1126,32 @@ public void writeBrokerDataOnZooKeep

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
thetumbled commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1610892192 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java: ## @@ -1122,6 +1126,32 @@ public void writeBrokerDataOnZooKeep

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
thetumbled commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1610892192 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java: ## @@ -1122,6 +1126,32 @@ public void writeBrokerDataOnZooKeep

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
heesung-sn commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1610883878 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java: ## @@ -1122,6 +1126,32 @@ public void writeBrokerDataOnZooKeep

Re: [I] [Bug] parseMessageMetadata error when broker entry metadata enable with high loading [pulsar]

2024-05-22 Thread via GitHub
semistone commented on issue #22601: URL: https://github.com/apache/pulsar/issues/22601#issuecomment-2126072096 I tested with that patch.. rollback all libs to version 3.2.2 and only replace pulsar-testclient.jar and pulsar-common.jar I also tried to run my test in one data center on

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
thetumbled commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1610869813 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java: ## @@ -1122,6 +1126,32 @@ public void writeBrokerDataOnZooKeep

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
thetumbled commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1610869813 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java: ## @@ -1122,6 +1126,32 @@ public void writeBrokerDataOnZooKeep

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
thetumbled commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1610869813 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java: ## @@ -1122,6 +1126,32 @@ public void writeBrokerDataOnZooKeep

(pulsar-site) branch main updated: Docs sync done from apache/pulsar (#dd35981)

2024-05-22 Thread urfree
This is an automated email from the ASF dual-hosted git repository. urfree pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/pulsar-site.git The following commit(s) were added to refs/heads/main by this push: new cd2ec40d589f Docs sync done from apache/pulsar

Re: [PR] [improve][cli] PIP-353: Improve transaction message visibility for peek-message [pulsar]

2024-05-22 Thread via GitHub
codecov-commenter commented on PR #22762: URL: https://github.com/apache/pulsar/pull/22762#issuecomment-2125968926 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/22762?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
heesung-sn commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1610794576 ## pulsar-broker/src/main/java/org/apache/pulsar/broker/loadbalance/impl/ModularLoadManagerImpl.java: ## @@ -1122,6 +1126,32 @@ public void writeBrokerDataOnZooKeep

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on code in PR #22760: URL: https://github.com/apache/pulsar/pull/22760#discussion_r1610760483 ## pulsar-common/src/main/java/org/apache/pulsar/common/protocol/ByteBufPair.java: ## @@ -122,38 +123,22 @@ public void write(ChannelHandlerContext ctx, Object msg, C

(pulsar) branch branch-3.3 updated: [fix] Upgrade Alpine packages at build time to fix CVE-2023-4236 (#22763)

2024-05-22 Thread mmerli
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch branch-3.3 in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/branch-3.3 by this push: new 5c6adef9645 [fix] Upgrade Alpine package

(pulsar) branch master updated: [fix] Upgrade Alpine packages at build time to fix CVE-2023-4236 (#22763)

2024-05-22 Thread mmerli
This is an automated email from the ASF dual-hosted git repository. mmerli pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar.git The following commit(s) were added to refs/heads/master by this push: new dd359819b3a [fix] Upgrade Alpine packages at bui

Re: [PR] [fix] Upgrade Alpine packages at build time to fix CVE-2023-4236 [pulsar]

2024-05-22 Thread via GitHub
merlimat merged PR #22763: URL: https://github.com/apache/pulsar/pull/22763 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.a

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on PR #22760: URL: https://github.com/apache/pulsar/pull/22760#issuecomment-2125741440 I think I'll need to add a repro test to the Pulsar code base. While testing the recent changes, I can see that the problem occurs even when there's the read only wrapper. -- This is

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on code in PR #22760: URL: https://github.com/apache/pulsar/pull/22760#discussion_r1610625006 ## pulsar-common/src/main/java/org/apache/pulsar/common/protocol/ByteBufPair.java: ## @@ -122,38 +123,22 @@ public void write(ChannelHandlerContext ctx, Object msg, C

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on PR #22760: URL: https://github.com/apache/pulsar/pull/22760#issuecomment-2125694720 > it should return false immediately if `cumulation.isWritable(inReadableBytes)== false` or `isReadOnly == true` > Waiting the fix PR [netty/netty#14071](https://github.com/netty/nett

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on PR #22760: URL: https://github.com/apache/pulsar/pull/22760#issuecomment-2125692782 > Yes, of cause, my bad, since I didn't explain myself correctly. Thanks for the useful summary. I'll revisit the description of this PR later once this comes to a conclusion. --

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on code in PR #22760: URL: https://github.com/apache/pulsar/pull/22760#discussion_r1610605439 ## pulsar-common/src/main/java/org/apache/pulsar/common/protocol/ByteBufPair.java: ## @@ -122,38 +123,22 @@ public void write(ChannelHandlerContext ctx, Object msg, C

Re: [PR] [fix] Upgrade Busybox to fix CVE-2023-4236 [pulsar]

2024-05-22 Thread via GitHub
merlimat commented on PR #22763: URL: https://github.com/apache/pulsar/pull/22763#issuecomment-2125681549 > LGTM > > Would it be useful to upgrade all packages that are upgradeable? Does Alpine have something similar to Ubuntu's `apt-get -y dist-upgrade`? Good point. There seem

Re: [I] [Bug] Delayed message delivery receive duplicate messages when unload topic [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on issue #20099: URL: https://github.com/apache/pulsar/issues/20099#issuecomment-2125673420 Mark: The root cause is the conflicts between `cursor.readEntries` and `cursor.asyncReplayEntries`. -- This is an automated message from the Apache Git Service. To respond to

[I] Cancel MessageProcessor while allowing in-flight messages to succeed [pulsar-dotpulsar]

2024-05-22 Thread via GitHub
smbecker opened a new issue, #217: URL: https://github.com/apache/pulsar-dotpulsar/issues/217 ### Is your feature request related to a problem? Please describe I am looking to leverage [KEDA](https://keda.sh/) to support auto-scaling my Pulsar consumers based on topic volumes. One of

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on PR #22760: URL: https://github.com/apache/pulsar/pull/22760#issuecomment-2125648128 > > I'll add background to the PR description to help reviewers understand it quickly. > > @dao-jun Please share that as a comment instead. This is my PR and not yours. :) I'll up

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on PR #22760: URL: https://github.com/apache/pulsar/pull/22760#issuecomment-2125646839 Just add additional context to help understand it quickly. [CopyingEncoder](https://github.com/apache/pulsar/blob/master/pulsar-common/src/main/java/org/apache/pulsar/common/protoc

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on PR #22760: URL: https://github.com/apache/pulsar/pull/22760#issuecomment-2125597059 > I'll add background to the PR description to help reviewers understand it quickly. @dao-jun Please share that as a comment instead. This is my PR and not yours. :) -- This i

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on PR #22760: URL: https://github.com/apache/pulsar/pull/22760#issuecomment-2125577820 I'll add background to the PR description to help reviewers understand it quickly. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on code in PR #22760: URL: https://github.com/apache/pulsar/pull/22760#discussion_r1610518609 ## pulsar-common/src/main/java/org/apache/pulsar/common/protocol/ByteBufPair.java: ## @@ -122,38 +123,22 @@ public void write(ChannelHandlerContext ctx, Object msg, C

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on code in PR #22760: URL: https://github.com/apache/pulsar/pull/22760#discussion_r1610518281 ## pulsar-common/src/main/java/org/apache/pulsar/common/protocol/ByteBufPair.java: ## @@ -122,38 +123,22 @@ public void write(ChannelHandlerContext ctx, Object msg, C

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on code in PR #22760: URL: https://github.com/apache/pulsar/pull/22760#discussion_r1610502839 ## pulsar-common/src/main/java/org/apache/pulsar/common/protocol/ByteBufPair.java: ## @@ -122,38 +123,22 @@ public void write(ChannelHandlerContext ctx, Object msg, C

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on code in PR #22760: URL: https://github.com/apache/pulsar/pull/22760#discussion_r1610503475 ## pulsar-common/src/main/java/org/apache/pulsar/common/protocol/ByteBufPair.java: ## @@ -122,38 +123,22 @@ public void write(ChannelHandlerContext ctx, Object msg, C

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on code in PR #22760: URL: https://github.com/apache/pulsar/pull/22760#discussion_r1610498528 ## pulsar-common/src/main/java/org/apache/pulsar/common/protocol/ByteBufPair.java: ## @@ -122,38 +123,22 @@ public void write(ChannelHandlerContext ctx, Object msg, C

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on code in PR #22760: URL: https://github.com/apache/pulsar/pull/22760#discussion_r1610497014 ## pulsar-common/src/main/java/org/apache/pulsar/common/protocol/ByteBufPair.java: ## @@ -122,38 +123,22 @@ public void write(ChannelHandlerContext ctx, Object msg, C

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on code in PR #22760: URL: https://github.com/apache/pulsar/pull/22760#discussion_r1610491894 ## pulsar-common/src/main/java/org/apache/pulsar/common/protocol/ByteBufPair.java: ## @@ -122,38 +123,22 @@ public void write(ChannelHandlerContext ctx, Object msg, C

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on code in PR #22760: URL: https://github.com/apache/pulsar/pull/22760#discussion_r1610468237 ## pulsar-common/src/main/java/org/apache/pulsar/common/protocol/ByteBufPair.java: ## @@ -122,38 +123,22 @@ public void write(ChannelHandlerContext ctx, Object msg, C

[PR] [sec] Upgrade Busybox to fix CVE-2023-4236 [pulsar]

2024-05-22 Thread via GitHub
merlimat opened a new pull request, #22763: URL: https://github.com/apache/pulsar/pull/22763 ### Motivation Upgrade `busybox` package in Alpine at build time to get latest version with fix. ### Modifications ### Verifying this change - [ ] Make sure that t

Re: [I] [Bug] parseMessageMetadata error when broker entry metadata enable with high loading [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on issue #22601: URL: https://github.com/apache/pulsar/issues/22601#issuecomment-2125182828 > seem change to read only actually change the behavior... :( > please ignore my previous comments. `.duplicate()` or `.retainedDuplicate()` should be sufficient to replace

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on PR #22760: URL: https://github.com/apache/pulsar/pull/22760#issuecomment-2125070549 Marking this draft until feedback is received to this comment: https://github.com/netty/netty/issues/14069#issuecomment-2125067746 -- This is an automated message from the Apache Git S

Re: [I] [Bug] Delayed message delivery receive duplicate messages when unload topic [pulsar]

2024-05-22 Thread via GitHub
SennoYuki commented on issue #20099: URL: https://github.com/apache/pulsar/issues/20099#issuecomment-2124937090 ok @dao-jun , I will retry tomorrow. Hope a good result. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

Re: [PR] [Improve] PIP-313 Add GetLastMessageIDs API [pulsar-client-go]

2024-05-22 Thread via GitHub
crossoverJie commented on PR #1221: URL: https://github.com/apache/pulsar-client-go/pull/1221#issuecomment-2124901418 @RobertIndie PTAL, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[PR] [Improve] PIP-313 Add GetLastMessageIDs API [pulsar-client-go]

2024-05-22 Thread via GitHub
crossoverJie opened a new pull request, #1221: URL: https://github.com/apache/pulsar-client-go/pull/1221 ### Motivation To keep consistent with the Java client. Releted PR: https://github.com/apache/pulsar/pull/20040 ### Modifications - Add `GetLastMessageIDs`api f

Re: [PR] [improve][PIP] PIP-353: Improve transaction message visibility for peek-messages [pulsar]

2024-05-22 Thread via GitHub
shibd commented on PR #22746: URL: https://github.com/apache/pulsar/pull/22746#issuecomment-2124870534 Discuss thread: https://lists.apache.org/thread/dc7f64jtvg987ydztxpffqx88bp44lwv Vote thread: https://lists.apache.org/thread/3hz04n64ykots2qmycr3sp78oqqjlrfr -- This is an autom

Re: [PR] [improve][cli] PIP-353: Improve transaction message visibility for peek-message [pulsar]

2024-05-22 Thread via GitHub
shibd closed pull request #22762: [improve][cli] PIP-353: Improve transaction message visibility for peek-message URL: https://github.com/apache/pulsar/pull/22762 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL ab

[PR] [improve][cli] PIP-353: Improve transaction message visibility for peek-message [pulsar]

2024-05-22 Thread via GitHub
shibd opened a new pull request, #22762: URL: https://github.com/apache/pulsar/pull/22762 ### Motivation #22746 ### Modifications - Support `--show-server-marker` and `--show-txn-aborted`, `--show-txn-uncommitted` flags for peek-messages cmd. ### Verifying this chang

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on PR #22760: URL: https://github.com/apache/pulsar/pull/22760#issuecomment-2124779202 Issues reported to Netty: * https://github.com/netty/netty/issues/14069 * https://github.com/netty/netty/issues/14068 -- This is an automated message from the Apache Git Service.

Re: [PR] Copy ByteBufPair buffers when using with SSL (#2401) [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on PR #2464: URL: https://github.com/apache/pulsar/pull/2464#issuecomment-2124778203 Follow up PR to fix issues with `.copy()`: #22760 . Issues reported to Netty: * https://github.com/netty/netty/issues/14069 * https://github.com/netty/netty/issues/14068 --

Re: [PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on PR #22760: URL: https://github.com/apache/pulsar/pull/22760#issuecomment-2124712359 On Netty side, PooledByteBuf.getBytes was made thread safe by making a copy of the internalByteBuffer with https://github.com/netty/netty/pull/9120 changes. However this change is not re

Re: [I] [Bug] Delayed message delivery receive duplicate messages when unload topic [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on issue #20099: URL: https://github.com/apache/pulsar/issues/20099#issuecomment-2124631901 Few unacked message also have a chance to cause many duplicated messages, you can take a try @SennoYuki -- This is an automated message from the Apache Git Service. To respond to

Re: [PR] [fix][test] Fix flaky test testShadowWrites [pulsar]

2024-05-22 Thread via GitHub
codecov-commenter commented on PR #22745: URL: https://github.com/apache/pulsar/pull/22745#issuecomment-2124511470 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/22745?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments

Re: [I] [Bug] Delayed message delivery receive duplicate messages when unload topic [pulsar]

2024-05-22 Thread via GitHub
SennoYuki commented on issue #20099: URL: https://github.com/apache/pulsar/issues/20099#issuecomment-2124475823 @dao-jun I think it's only part reson for small batch duplicated messages. But in my scene the duplicated messages is huge. It's not possible causing the 20M and more message dupl

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
codecov-commenter commented on PR #22753: URL: https://github.com/apache/pulsar/pull/22753#issuecomment-2124447255 ## [Codecov](https://app.codecov.io/gh/apache/pulsar/pull/22753?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments

[PR] [improve][cli]New tests for operations on namespaces in pulsar-client-tools [pulsar]

2024-05-22 Thread via GitHub
sandeep-mst opened a new pull request, #22761: URL: https://github.com/apache/pulsar/pull/22761 Fixes [21743](https://github.com/apache/pulsar/issues/21743) of apache/pulsar ### Motivation Pulsar client tools doesn't have tests for listing, deleting and creating

Re: [I] [Bug] Delayed message delivery receive duplicate messages when unload topic [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on issue #20099: URL: https://github.com/apache/pulsar/issues/20099#issuecomment-2124378975 enable `AckWithResponse`, the consumer will wait until broker received the ack -- This is an automated message from the Apache Git Service. To respond to the message, please log o

Re: [I] [Bug] Delayed message delivery receive duplicate messages when unload topic [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on issue #20099: URL: https://github.com/apache/pulsar/issues/20099#issuecomment-2124375570 The problem is Pulsar optimized `consumer.ack`. When you call `consumer.ack`, the `ack` request will not send to the broker immediately, but cached in the client. When the cached

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
thetumbled commented on PR #22753: URL: https://github.com/apache/pulsar/pull/22753#issuecomment-2124351265 /pulsarbot rerun-failure-checks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [I] [Bug] Delayed message delivery receive duplicate messages when unload topic [pulsar]

2024-05-22 Thread via GitHub
dao-jun commented on issue #20099: URL: https://github.com/apache/pulsar/issues/20099#issuecomment-2124349515 ```go func test() { clientOps := pulsar.ClientOptions{ URL: "pulsar://localhost:6650", } client, err := pulsar.NewClient(clientOps)

Re: [I] [Bug] parseMessageMetadata error when broker entry metadata enable with high loading [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on issue #22601: URL: https://github.com/apache/pulsar/issues/22601#issuecomment-2124331850 I have tested #22760 with the repro case [and scripts](https://github.com/lhotari/pulsar-playground/tree/master/issues/issue22601) that are based on the modified pulsar-perf and oth

(pulsar-client-node) branch master updated: feat: Use macos-arm64 github runnner to build napi package (#386)

2024-05-22 Thread baodi
This is an automated email from the ASF dual-hosted git repository. baodi pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/pulsar-client-node.git The following commit(s) were added to refs/heads/master by this push: new 13c2859 feat: Use macos-arm64 github

Re: [PR] feat: Use macos-arm64 github runnner to build napi package [pulsar-client-node]

2024-05-22 Thread via GitHub
shibd merged PR #386: URL: https://github.com/apache/pulsar-client-node/pull/386 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pul

Re: [PR] feat: Use macos-arm64 github runnner to build napi package [pulsar-client-node]

2024-05-22 Thread via GitHub
RobertIndie commented on code in PR #386: URL: https://github.com/apache/pulsar-client-node/pull/386#discussion_r1609584235 ## .github/workflows/ci-build-release-napi.yml: ## @@ -56,31 +62,19 @@ jobs: with: python-version: ${{ matrix.python }} -

Re: [PR] [improve][authentication] Pass the authorization when user lookup transactionCoordinator topic [pulsar]

2024-05-22 Thread via GitHub
TakaHiR07 commented on PR #22744: URL: https://github.com/apache/pulsar/pull/22744#issuecomment-2124253586 @nodece @michaeljmarshall could you take a look of this problem. thx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

Re: [I] [Bug] parseMessageMetadata error when broker entry metadata enable with high loading [pulsar]

2024-05-22 Thread via GitHub
semistone commented on issue #22601: URL: https://github.com/apache/pulsar/issues/22601#issuecomment-2124240776 > @semistone I have created a PR #22760 to fix the problem. It's currently in draft state since I'm currently testing the solution to verify that it mitigates the problem.

Re: [PR] feat: Use macos-arm64 github runnner to build napi package [pulsar-client-node]

2024-05-22 Thread via GitHub
shibd commented on code in PR #386: URL: https://github.com/apache/pulsar-client-node/pull/386#discussion_r1609567491 ## .github/workflows/ci-build-release-napi.yml: ## @@ -56,31 +62,19 @@ jobs: with: python-version: ${{ matrix.python }} - -

Re: [PR] feat: Use macos-arm64 github runnner to build napi package [pulsar-client-node]

2024-05-22 Thread via GitHub
RobertIndie commented on code in PR #386: URL: https://github.com/apache/pulsar-client-node/pull/386#discussion_r1609556657 ## .github/workflows/ci-build-release-napi.yml: ## @@ -56,31 +62,19 @@ jobs: with: python-version: ${{ matrix.python }} -

Re: [I] [Bug] parseMessageMetadata error when broker entry metadata enable with high loading [pulsar]

2024-05-22 Thread via GitHub
lhotari commented on issue #22601: URL: https://github.com/apache/pulsar/issues/22601#issuecomment-2124209699 @semistone I have created a PR #22760 to fix the problem. It's currently in draft state since I'm currently testing the solution to verify that it mitigates the problem. -- This

[PR] [fix][broker] Fix multiple transfer corruption issues when TLS is enabled [pulsar]

2024-05-22 Thread via GitHub
lhotari opened a new pull request, #22760: URL: https://github.com/apache/pulsar/pull/22760 Fixes #22601 #21892 #19460 ### Motivation In Pulsar, there are multiple reported issues where the transferred output gets corrupted and fails with exceptions around invalid reader and wr

[PR] feat: Use macos-arm64 github runnner to build napi package [pulsar-client-node]

2024-05-22 Thread via GitHub
shibd opened a new pull request, #386: URL: https://github.com/apache/pulsar-client-node/pull/386 ### Motivation For now, GitHub support MacOS arm64 runnner. https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners#s

Re: [PR] [improve] [broker] apply topK machanism to ModularLoadManagerImpl. [pulsar]

2024-05-22 Thread via GitHub
Demogorgon314 commented on code in PR #22753: URL: https://github.com/apache/pulsar/pull/22753#discussion_r1609529600 ## pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java: ## @@ -2662,7 +2662,8 @@ The max allowed delay for delayed delivery (in

Re: [I] [Bug] when define Consumer in class field, and restart consumer, message received More than once [pulsar]

2024-05-22 Thread via GitHub
sdvdxl commented on issue #22759: URL: https://github.com/apache/pulsar/issues/22759#issuecomment-2124173679 this may be a mistake. other place used `consumer.unsubscribe()` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] [Bug] when define Consumer in class field, and restart consumer, message received More than once [pulsar]

2024-05-22 Thread via GitHub
sdvdxl closed issue #22759: [Bug] when define Consumer in class field, and restart consumer, message received More than once URL: https://github.com/apache/pulsar/issues/22759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] [Bug] parseMessageMetadata error when broker entry metadata enable with high loading [pulsar]

2024-05-22 Thread via GitHub
semistone commented on issue #22601: URL: https://github.com/apache/pulsar/issues/22601#issuecomment-2124039443 Thanks for helping check this issue , I don't have any progress today :( I could see there is another io thread is using that unwrap bytebuf but at least it could re