[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: d60198f621baac54bffb981d7a797a04dfa0bb5a / rongtong [ISSUE #8239] Fix the issue of potential message loss after a crash under synchronous disk flushing configuration. (#8240) Report URL: https://github.com/apache/rocketmq/actions/runs/9412254295 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "PUSH-CI" failed!
The GitHub Actions job "PUSH-CI" on rocketmq.git has failed. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: d60198f621baac54bffb981d7a797a04dfa0bb5a / rongtong [ISSUE #8239] Fix the issue of potential message loss after a crash under synchronous disk flushing configuration. (#8240) Report URL: https://github.com/apache/rocketmq/actions/runs/9412254293 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: d60198f621baac54bffb981d7a797a04dfa0bb5a / rongtong [ISSUE #8239] Fix the issue of potential message loss after a crash under synchronous disk flushing configuration. (#8240) Report URL: https://github.com/apache/rocketmq/actions/runs/9412254292 With regards, GitHub Actions via GitBox
Re: [I] [Bug] In the case of single-replica synchronous disk flushing, messages may be lost during a crash [rocketmq]
lizhimins closed issue #8239: [Bug] In the case of single-replica synchronous disk flushing, messages may be lost during a crash URL: https://github.com/apache/rocketmq/issues/8239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(rocketmq) branch develop updated: [ISSUE #8239] Fix the issue of potential message loss after a crash under synchronous disk flushing configuration. (#8240)
This is an automated email from the ASF dual-hosted git repository.
lizhimin pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/rocketmq.git
The following commit(s) were added to refs/heads/develop by this push:
new d60198f621 [ISSUE #8239] Fix the issue of potential message loss after
a crash under synchronous disk flushing configuration. (#8240)
d60198f621 is described below
commit d60198f621baac54bffb981d7a797a04dfa0bb5a
Author: rongtong
AuthorDate: Fri Jun 7 13:44:13 2024 +0800
[ISSUE #8239] Fix the issue of potential message loss after a crash under
synchronous disk flushing configuration. (#8240)
---
.../java/org/apache/rocketmq/store/CommitLog.java | 2 +-
.../apache/rocketmq/store/DefaultMessageStore.java | 12 +-
.../rocketmq/store/config/MessageStoreConfig.java | 15 ++-
.../rocketmq/store/ReputMessageServiceTest.java| 148 +
4 files changed, 171 insertions(+), 6 deletions(-)
diff --git a/store/src/main/java/org/apache/rocketmq/store/CommitLog.java
b/store/src/main/java/org/apache/rocketmq/store/CommitLog.java
index 1174eca1ba..c2150d7a32 100644
--- a/store/src/main/java/org/apache/rocketmq/store/CommitLog.java
+++ b/store/src/main/java/org/apache/rocketmq/store/CommitLog.java
@@ -651,7 +651,7 @@ public class CommitLog implements Swappable {
} else if
(this.defaultMessageStore.getMessageStoreConfig().isDuplicationEnable()) {
return this.confirmOffset;
} else {
-return getMaxOffset();
+return this.defaultMessageStore.isSyncDiskFlush() ?
getFlushedWhere() : getMaxOffset();
}
}
diff --git
a/store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java
b/store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java
index 97833351d1..a901e850ed 100644
--- a/store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java
+++ b/store/src/main/java/org/apache/rocketmq/store/DefaultMessageStore.java
@@ -2814,7 +2814,11 @@ public class DefaultMessageStore implements MessageStore
{
}
public boolean isCommitLogAvailable() {
-return this.reputFromOffset <
DefaultMessageStore.this.getConfirmOffset();
+return this.reputFromOffset < getReputEndOffset();
+}
+
+protected long getReputEndOffset() {
+return
DefaultMessageStore.this.getMessageStoreConfig().isReadUnCommitted() ?
DefaultMessageStore.this.commitLog.getMaxOffset() :
DefaultMessageStore.this.commitLog.getConfirmOffset();
}
public void doReput() {
@@ -2834,12 +2838,12 @@ public class DefaultMessageStore implements
MessageStore {
try {
this.reputFromOffset = result.getStartOffset();
-for (int readSize = 0; readSize < result.getSize() &&
reputFromOffset < DefaultMessageStore.this.getConfirmOffset() && doNext; ) {
+for (int readSize = 0; readSize < result.getSize() &&
reputFromOffset < getReputEndOffset() && doNext; ) {
DispatchRequest dispatchRequest =
DefaultMessageStore.this.commitLog.checkMessageAndReturnSize(result.getByteBuffer(),
false, false, false);
int size = dispatchRequest.getBufferSize() == -1 ?
dispatchRequest.getMsgSize() : dispatchRequest.getBufferSize();
-if (reputFromOffset + size >
DefaultMessageStore.this.getConfirmOffset()) {
+if (reputFromOffset + size > getReputEndOffset()) {
doNext = false;
break;
}
@@ -3127,7 +3131,7 @@ public class DefaultMessageStore implements MessageStore {
try {
this.reputFromOffset = result.getStartOffset();
-for (int readSize = 0; readSize < result.getSize() &&
reputFromOffset < DefaultMessageStore.this.getConfirmOffset() && doNext; ) {
+for (int readSize = 0; readSize < result.getSize() &&
reputFromOffset < getReputEndOffset() && doNext; ) {
ByteBuffer byteBuffer = result.getByteBuffer();
int totalSize =
preCheckMessageAndReturnSize(byteBuffer);
diff --git
a/store/src/main/java/org/apache/rocketmq/store/config/MessageStoreConfig.java
b/store/src/main/java/org/apache/rocketmq/store/config/MessageStoreConfig.java
index 9afc02a0c9..0060b144cf 100644
---
a/store/src/main/java/org/apache/rocketmq/store/config/MessageStoreConfig.java
+++
b/store/src/main/java/org/apache/rocketmq/store/config/MessageStoreConfig.java
@@ -413,6 +413,12 @@ public class MessageStoreConfig {
private int topicQueueLockNum = 32;
+/**
+ * If readUnCommitted is true, the dispatch of the consume queue will
exceed the confirmOffset, which may cause the client to read uncommitted
messages.
+ * For example, reput offse
Re: [PR] [ISSUE #8239] Fix the issue of potential message loss after a crash under synchronous disk flushing configuration. [rocketmq]
lizhimins merged PR #8240: URL: https://github.com/apache/rocketmq/pull/8240 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ISSUE #8127]Optimize the metric calculation logic of the time wheel [rocketmq]
3424672656 commented on PR #8128: URL: https://github.com/apache/rocketmq/pull/8128#issuecomment-2153947045 @RongtongJin PTAL~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "PUSH-CI" failed!
The GitHub Actions job "PUSH-CI" on rocketmq.git has failed. Run started by GitHub user fuyou001 (triggered by fuyou001). Head commit for run: 4be8fd43720c8635fe135404a7fd000c00bb2a15 / guyinyou <36399867+guyin...@users.noreply.github.com> [ISSUE #8265] Implement Batch Creation of Topics in RocketMQ Admin (#8267) * add UPDATE_AND_CREATE_TOPIC_LIST * support creating or updating topic config in batch - Co-authored-by: guyinyou Co-authored-by: gaoyang.cgy Report URL: https://github.com/apache/rocketmq/actions/runs/9410721673 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user fuyou001 (triggered by fuyou001). Head commit for run: 4be8fd43720c8635fe135404a7fd000c00bb2a15 / guyinyou <36399867+guyin...@users.noreply.github.com> [ISSUE #8265] Implement Batch Creation of Topics in RocketMQ Admin (#8267) * add UPDATE_AND_CREATE_TOPIC_LIST * support creating or updating topic config in batch - Co-authored-by: guyinyou Co-authored-by: gaoyang.cgy Report URL: https://github.com/apache/rocketmq/actions/runs/9410721676 With regards, GitHub Actions via GitBox
(rocketmq) branch develop updated: [ISSUE #8265] Implement Batch Creation of Topics in RocketMQ Admin (#8267)
This is an automated email from the ASF dual-hosted git repository.
fuyou pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/rocketmq.git
The following commit(s) were added to refs/heads/develop by this push:
new 4be8fd4372 [ISSUE #8265] Implement Batch Creation of Topics in
RocketMQ Admin (#8267)
4be8fd4372 is described below
commit 4be8fd43720c8635fe135404a7fd000c00bb2a15
Author: guyinyou <36399867+guyin...@users.noreply.github.com>
AuthorDate: Fri Jun 7 10:28:36 2024 +0800
[ISSUE #8265] Implement Batch Creation of Topics in RocketMQ Admin (#8267)
* add UPDATE_AND_CREATE_TOPIC_LIST
* support creating or updating topic config in batch
-
Co-authored-by: guyinyou
Co-authored-by: gaoyang.cgy
---
.../broker/processor/AdminBrokerProcessor.java | 81 ++
.../rocketmq/broker/topic/TopicConfigManager.java | 11 +-
.../rocketmq/client/impl/MQClientAPIImpl.java | 20
.../rocketmq/client/impl/MQClientAPIImplTest.java | 23
.../rocketmq/remoting/protocol/RequestCode.java| 1 +
.../protocol/body/CreateTopicListRequestBody.java | 42
.../header/CreateTopicListRequestHeader.java | 31 ++
.../rocketmq/tools/admin/DefaultMQAdminExt.java| 5 +
.../tools/admin/DefaultMQAdminExtImpl.java | 6 ++
.../apache/rocketmq/tools/admin/MQAdminExt.java| 3 +
.../rocketmq/tools/command/MQAdminStartup.java | 2 +
.../command/topic/UpdateTopicListSubCommand.java | 118 +
.../topic/UpdateTopicListSubCommandTest.java | 41 +++
13 files changed, 383 insertions(+), 1 deletion(-)
diff --git
a/broker/src/main/java/org/apache/rocketmq/broker/processor/AdminBrokerProcessor.java
b/broker/src/main/java/org/apache/rocketmq/broker/processor/AdminBrokerProcessor.java
index 40a7a461e8..44bf2a4813 100644
---
a/broker/src/main/java/org/apache/rocketmq/broker/processor/AdminBrokerProcessor.java
+++
b/broker/src/main/java/org/apache/rocketmq/broker/processor/AdminBrokerProcessor.java
@@ -116,6 +116,7 @@ import
org.apache.rocketmq.remoting.protocol.body.Connection;
import org.apache.rocketmq.remoting.protocol.body.ConsumeQueueData;
import org.apache.rocketmq.remoting.protocol.body.ConsumeStatsList;
import org.apache.rocketmq.remoting.protocol.body.ConsumerConnection;
+import org.apache.rocketmq.remoting.protocol.body.CreateTopicListRequestBody;
import org.apache.rocketmq.remoting.protocol.body.EpochEntryCache;
import org.apache.rocketmq.remoting.protocol.body.GroupList;
import org.apache.rocketmq.remoting.protocol.body.HARuntimeInfo;
@@ -243,6 +244,8 @@ public class AdminBrokerProcessor implements
NettyRequestProcessor {
switch (request.getCode()) {
case RequestCode.UPDATE_AND_CREATE_TOPIC:
return this.updateAndCreateTopic(ctx, request);
+case RequestCode.UPDATE_AND_CREATE_TOPIC_LIST:
+return this.updateAndCreateTopicList(ctx, request);
case RequestCode.DELETE_TOPIC_IN_BROKER:
return this.deleteTopic(ctx, request);
case RequestCode.GET_ALL_TOPIC_CONFIG:
@@ -536,6 +539,84 @@ public class AdminBrokerProcessor implements
NettyRequestProcessor {
return response;
}
+private synchronized RemotingCommand
updateAndCreateTopicList(ChannelHandlerContext ctx,
+RemotingCommand request) throws RemotingCommandException {
+long startTime = System.currentTimeMillis();
+
+final CreateTopicListRequestBody requestBody =
CreateTopicListRequestBody.decode(request.getBody(),
CreateTopicListRequestBody.class);
+List topicConfigList = requestBody.getTopicConfigList();
+
+StringBuilder builder = new StringBuilder();
+for (TopicConfig topicConfig : topicConfigList) {
+builder.append(topicConfig.getTopicName()).append(";");
+}
+String topicNames = builder.toString();
+LOGGER.info("AdminBrokerProcessor#updateAndCreateTopicList:
topicNames: {}, called by {}", topicNames,
RemotingHelper.parseChannelRemoteAddr(ctx.channel()));
+
+final RemotingCommand response =
RemotingCommand.createResponseCommand(null);
+
+long executionTime;
+
+try {
+// Valid topics
+for (TopicConfig topicConfig : topicConfigList) {
+String topic = topicConfig.getTopicName();
+TopicValidator.ValidateTopicResult result =
TopicValidator.validateTopic(topic);
+if (!result.isValid()) {
+response.setCode(ResponseCode.SYSTEM_ERROR);
+response.setRemark(result.getRemark());
+return response;
+}
+if
(brokerController.getBrokerConfig().isValidateSystemTopicWhenUpdateTopic()) {
+if (TopicValidator.isSystemTopic(topic)) {
+response.setCode(R
Re: [I] [Enhancement] Implement Batch Creation of Topics in RocketMQ Admin [rocketmq]
fuyou001 closed issue #8265: [Enhancement] Implement Batch Creation of Topics in RocketMQ Admin URL: https://github.com/apache/rocketmq/issues/8265 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ISSUE #8265] Implement Batch Creation of Topics in RocketMQ Admin [rocketmq]
fuyou001 merged PR #8267: URL: https://github.com/apache/rocketmq/pull/8267 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user RongtongJin (triggered by RongtongJin). Head commit for run: d9ebe887677646ec8e969c03ea0d394a299894e0 / Zhouxiang Zhan [ISSUE #8268] Fix pop orderly commitOffset when NO_MATCHED_MESSAGE (#8270) Report URL: https://github.com/apache/rocketmq/actions/runs/9410176617 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" is working again!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has succeeded. Run started by GitHub user RongtongJin (triggered by RongtongJin). Head commit for run: 4dfcd531fee0b82c4280af2b25e81118ef340a2e / RongtongJin Fix code style Report URL: https://github.com/apache/rocketmq/actions/runs/9410154456 With regards, GitHub Actions via GitBox
Re: [I] [Bug] Reopen S3 ut in tieredstorage [rocketmq]
github-actions[bot] commented on issue #6870: URL: https://github.com/apache/rocketmq/issues/6870#issuecomment-2153610521 This issue is stale because it has been open for 365 days with no activity. It will be closed in 3 days if no further activity occurs. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" is working again!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has succeeded. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: e1bf59e241da0c5ef82b5987f5530a51a08bad05 / wanghuaiyuan <3424672...@qq.com> Avoid unnecessary waiting when a response is successfully returned Report URL: https://github.com/apache/rocketmq/actions/runs/9404461405 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Coverage" is working again!
The GitHub Actions job "Coverage" on rocketmq.git has succeeded. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: e1bf59e241da0c5ef82b5987f5530a51a08bad05 / wanghuaiyuan <3424672...@qq.com> Avoid unnecessary waiting when a response is successfully returned Report URL: https://github.com/apache/rocketmq/actions/runs/9404461407 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: d9ebe887677646ec8e969c03ea0d394a299894e0 / Zhouxiang Zhan [ISSUE #8268] Fix pop orderly commitOffset when NO_MATCHED_MESSAGE (#8270) Report URL: https://github.com/apache/rocketmq/actions/runs/9404499815 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Coverage" failed!
The GitHub Actions job "Coverage" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: 1f83584db7acd22026b1132886726b15c95e06a9 / hqbfz <125714719+3424672...@users.noreply.github.com> Merge branch 'apache:develop' into optimize_request Report URL: https://github.com/apache/rocketmq/actions/runs/9404452657 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: 1f83584db7acd22026b1132886726b15c95e06a9 / hqbfz <125714719+3424672...@users.noreply.github.com> Merge branch 'apache:develop' into optimize_request Report URL: https://github.com/apache/rocketmq/actions/runs/9404452660 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: e1bf59e241da0c5ef82b5987f5530a51a08bad05 / wanghuaiyuan <3424672...@qq.com> Avoid unnecessary waiting when a response is successfully returned Report URL: https://github.com/apache/rocketmq/actions/runs/9404461414 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: 1f83584db7acd22026b1132886726b15c95e06a9 / hqbfz <125714719+3424672...@users.noreply.github.com> Merge branch 'apache:develop' into optimize_request Report URL: https://github.com/apache/rocketmq/actions/runs/9404452640 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "PR-CI" is working again!
The GitHub Actions job "PR-CI" on rocketmq.git has succeeded. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: e1bf59e241da0c5ef82b5987f5530a51a08bad05 / wanghuaiyuan <3424672...@qq.com> Avoid unnecessary waiting when a response is successfully returned Report URL: https://github.com/apache/rocketmq/actions/runs/9404461417 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: d9ebe887677646ec8e969c03ea0d394a299894e0 / Zhouxiang Zhan [ISSUE #8268] Fix pop orderly commitOffset when NO_MATCHED_MESSAGE (#8270) Report URL: https://github.com/apache/rocketmq/actions/runs/9404466962 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "PR-CI" failed!
The GitHub Actions job "PR-CI" on rocketmq.git has failed. Run started by GitHub user 3424672656 (triggered by 3424672656). Head commit for run: 1f83584db7acd22026b1132886726b15c95e06a9 / hqbfz <125714719+3424672...@users.noreply.github.com> Merge branch 'apache:develop' into optimize_request Report URL: https://github.com/apache/rocketmq/actions/runs/9404452633 With regards, GitHub Actions via GitBox
[PR] [ISSUE #8261]Avoid unnecessary waiting when a response is successfully returned [rocketmq]
3424672656 opened a new pull request, #8272: URL: https://github.com/apache/rocketmq/pull/8272 ### Which Issue(s) This PR Fixes Fixes #8261 ### Brief Description ### How Did You Test This Change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" is working again!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has succeeded. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: d1974c55353488095e5122f5ce361c150611f21a / lizhimins <707364...@qq.com> [ISSUE #8269] Support pop consumption filter in long polling service (#8271) Report URL: https://github.com/apache/rocketmq/actions/runs/9400720368 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: d9ebe887677646ec8e969c03ea0d394a299894e0 / Zhouxiang Zhan [ISSUE #8268] Fix pop orderly commitOffset when NO_MATCHED_MESSAGE (#8270) Report URL: https://github.com/apache/rocketmq/actions/runs/9400881318 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "PUSH-CI" failed!
The GitHub Actions job "PUSH-CI" on rocketmq.git has failed. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: d9ebe887677646ec8e969c03ea0d394a299894e0 / Zhouxiang Zhan [ISSUE #8268] Fix pop orderly commitOffset when NO_MATCHED_MESSAGE (#8270) Report URL: https://github.com/apache/rocketmq/actions/runs/9400881311 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Coverage" is working again!
The GitHub Actions job "Coverage" on rocketmq.git has succeeded. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: d1974c55353488095e5122f5ce361c150611f21a / lizhimins <707364...@qq.com> [ISSUE #8269] Support pop consumption filter in long polling service (#8271) Report URL: https://github.com/apache/rocketmq/actions/runs/9400720344 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "PUSH-CI" failed!
The GitHub Actions job "PUSH-CI" on rocketmq.git has failed. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: d1974c55353488095e5122f5ce361c150611f21a / lizhimins <707364...@qq.com> [ISSUE #8269] Support pop consumption filter in long polling service (#8271) Report URL: https://github.com/apache/rocketmq/actions/runs/9400720352 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: d9ebe887677646ec8e969c03ea0d394a299894e0 / Zhouxiang Zhan [ISSUE #8268] Fix pop orderly commitOffset when NO_MATCHED_MESSAGE (#8270) Report URL: https://github.com/apache/rocketmq/actions/runs/9400881295 With regards, GitHub Actions via GitBox
(rocketmq) branch develop updated: [ISSUE #8268] Fix pop orderly commitOffset when NO_MATCHED_MESSAGE (#8270)
This is an automated email from the ASF dual-hosted git repository.
lizhimin pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/rocketmq.git
The following commit(s) were added to refs/heads/develop by this push:
new d9ebe88767 [ISSUE #8268] Fix pop orderly commitOffset when
NO_MATCHED_MESSAGE (#8270)
d9ebe88767 is described below
commit d9ebe887677646ec8e969c03ea0d394a299894e0
Author: Zhouxiang Zhan
AuthorDate: Thu Jun 6 20:29:59 2024 +0800
[ISSUE #8268] Fix pop orderly commitOffset when NO_MATCHED_MESSAGE (#8270)
---
.../apache/rocketmq/broker/processor/PopMessageProcessor.java | 11 +++
1 file changed, 7 insertions(+), 4 deletions(-)
diff --git
a/broker/src/main/java/org/apache/rocketmq/broker/processor/PopMessageProcessor.java
b/broker/src/main/java/org/apache/rocketmq/broker/processor/PopMessageProcessor.java
index 3df4bec984..0304a5dab0 100644
---
a/broker/src/main/java/org/apache/rocketmq/broker/processor/PopMessageProcessor.java
+++
b/broker/src/main/java/org/apache/rocketmq/broker/processor/PopMessageProcessor.java
@@ -638,10 +638,13 @@ public class PopMessageProcessor implements
NettyRequestProcessor {
||
GetMessageStatus.MESSAGE_WAS_REMOVING.equals(result.getStatus())
||
GetMessageStatus.NO_MATCHED_LOGIC_QUEUE.equals(result.getStatus()))
&& result.getNextBeginOffset() > -1) {
-
popBufferMergeService.addCkMock(requestHeader.getConsumerGroup(), topic,
queueId, finalOffset,
-requestHeader.getInvisibleTime(), popTime, reviveQid,
result.getNextBeginOffset(),
brokerController.getBrokerConfig().getBrokerName());
-//
this.brokerController.getConsumerOffsetManager().commitOffset(channel.remoteAddress().toString(),
requestHeader.getConsumerGroup(), topic,
-//queueId, getMessageTmpResult.getNextBeginOffset());
+if (isOrder) {
+
this.brokerController.getConsumerOffsetManager().commitOffset(channel.remoteAddress().toString(),
requestHeader.getConsumerGroup(), topic,
+queueId, result.getNextBeginOffset());
+} else {
+
popBufferMergeService.addCkMock(requestHeader.getConsumerGroup(), topic,
queueId, finalOffset,
+requestHeader.getInvisibleTime(), popTime,
reviveQid, result.getNextBeginOffset(),
brokerController.getBrokerConfig().getBrokerName());
+}
}
atomicRestNum.set(result.getMaxOffset() -
result.getNextBeginOffset() + atomicRestNum.get());
Re: [PR] [ISSUE #8268] Fix pop orderly commitOffset when NO_MATCHED_MESSAGE [rocketmq]
lizhimins merged PR #8270: URL: https://github.com/apache/rocketmq/pull/8270 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Bug] Fix pop orderly commit offset when NO_MATCHED_MESSAGE [rocketmq]
lizhimins closed issue #8268: [Bug] Fix pop orderly commit offset when NO_MATCHED_MESSAGE URL: https://github.com/apache/rocketmq/issues/8268 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] Customize the port of broker fastRemotingServer [rocketmq]
liuweiGL commented on issue #2585: URL: https://github.com/apache/rocketmq/issues/2585#issuecomment-2152279358 这个端口设计真是巨臭，在 k8s 里面被恶心到了 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
(rocketmq) branch develop updated: [ISSUE #8269] Support pop consumption filter in long polling service (#8271)
This is an automated email from the ASF dual-hosted git repository.
lizhimin pushed a commit to branch develop
in repository https://gitbox.apache.org/repos/asf/rocketmq.git
The following commit(s) were added to refs/heads/develop by this push:
new d1974c5535 [ISSUE #8269] Support pop consumption filter in long
polling service (#8271)
d1974c5535 is described below
commit d1974c55353488095e5122f5ce361c150611f21a
Author: lizhimins <707364...@qq.com>
AuthorDate: Thu Jun 6 20:19:06 2024 +0800
[ISSUE #8269] Support pop consumption filter in long polling service (#8271)
---
.../longpolling/NotifyMessageArrivingListener.java | 11 --
.../broker/longpolling/PopLongPollingService.java | 44 ++
.../rocketmq/broker/longpolling/PopRequest.java| 25 +---
.../broker/processor/AckMessageProcessor.java | 13 +++
.../broker/processor/NotificationProcessor.java| 11 +-
.../broker/processor/PopMessageProcessor.java | 40 ++--
6 files changed, 107 insertions(+), 37 deletions(-)
diff --git
a/broker/src/main/java/org/apache/rocketmq/broker/longpolling/NotifyMessageArrivingListener.java
b/broker/src/main/java/org/apache/rocketmq/broker/longpolling/NotifyMessageArrivingListener.java
index e55ed2778a..1ddb9f4f8e 100644
---
a/broker/src/main/java/org/apache/rocketmq/broker/longpolling/NotifyMessageArrivingListener.java
+++
b/broker/src/main/java/org/apache/rocketmq/broker/longpolling/NotifyMessageArrivingListener.java
@@ -36,9 +36,12 @@ public class NotifyMessageArrivingListener implements
MessageArrivingListener {
@Override
public void arriving(String topic, int queueId, long logicOffset, long
tagsCode,
long msgStoreTime, byte[] filterBitMap, Map properties) {
-this.pullRequestHoldService.notifyMessageArriving(topic, queueId,
logicOffset, tagsCode,
-msgStoreTime, filterBitMap, properties);
-this.popMessageProcessor.notifyMessageArriving(topic, queueId);
-this.notificationProcessor.notifyMessageArriving(topic, queueId);
+
+this.pullRequestHoldService.notifyMessageArriving(
+topic, queueId, logicOffset, tagsCode, msgStoreTime, filterBitMap,
properties);
+this.popMessageProcessor.notifyMessageArriving(
+topic, queueId, tagsCode, msgStoreTime, filterBitMap, properties);
+this.notificationProcessor.notifyMessageArriving(
+topic, queueId, tagsCode, msgStoreTime, filterBitMap, properties);
}
}
diff --git
a/broker/src/main/java/org/apache/rocketmq/broker/longpolling/PopLongPollingService.java
b/broker/src/main/java/org/apache/rocketmq/broker/longpolling/PopLongPollingService.java
index a768fe4b9c..b5179114f3 100644
---
a/broker/src/main/java/org/apache/rocketmq/broker/longpolling/PopLongPollingService.java
+++
b/broker/src/main/java/org/apache/rocketmq/broker/longpolling/PopLongPollingService.java
@@ -35,6 +35,9 @@ import
org.apache.rocketmq.remoting.netty.NettyRemotingAbstract;
import org.apache.rocketmq.remoting.netty.NettyRequestProcessor;
import org.apache.rocketmq.remoting.netty.RequestTask;
import org.apache.rocketmq.remoting.protocol.RemotingCommand;
+import org.apache.rocketmq.remoting.protocol.heartbeat.SubscriptionData;
+import org.apache.rocketmq.store.ConsumeQueueExt;
+import org.apache.rocketmq.store.MessageFilter;
import static org.apache.rocketmq.broker.longpolling.PollingResult.NOT_POLLING;
import static
org.apache.rocketmq.broker.longpolling.PollingResult.POLLING_FULL;
@@ -147,39 +150,61 @@ public class PopLongPollingService extends ServiceThread {
}
public void notifyMessageArrivingWithRetryTopic(final String topic, final
int queueId) {
+this.notifyMessageArrivingWithRetryTopic(topic, queueId, null, 0L,
null, null);
+}
+
+public void notifyMessageArrivingWithRetryTopic(final String topic, final
int queueId,
+Long tagsCode, long msgStoreTime, byte[] filterBitMap, Map properties) {
String notifyTopic;
if (KeyBuilder.isPopRetryTopicV2(topic)) {
notifyTopic = KeyBuilder.parseNormalTopic(topic);
} else {
notifyTopic = topic;
}
-notifyMessageArriving(notifyTopic, queueId);
+notifyMessageArriving(notifyTopic, queueId, tagsCode, msgStoreTime,
filterBitMap, properties);
}
-public void notifyMessageArriving(final String topic, final int queueId) {
+public void notifyMessageArriving(final String topic, final int queueId,
+Long tagsCode, long msgStoreTime, byte[] filterBitMap, Map properties) {
ConcurrentHashMap cids = topicCidMap.get(topic);
if (cids == null) {
return;
}
for (Map.Entry cid : cids.entrySet()) {
if (queueId >= 0) {
-notifyMessageArriving(topic, cid.getKey(), -1);
+notifyMessageArriving(topic, -1, cid.getKey(), tagsCode,
msgStoreTime, fil
Re: [PR] [ISSUE #8269] Support pop consumption filter in long polling service [rocketmq]
lizhimins merged PR #8271: URL: https://github.com/apache/rocketmq/pull/8271 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Enhancement] Support pop consumption filter in long polling service [rocketmq]
lizhimins closed issue #8269: [Enhancement] Support pop consumption filter in long polling service URL: https://github.com/apache/rocketmq/issues/8269 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" is working again!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has succeeded. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: 254d3f47bc29bf44868790b6b12385aeb2f2a45e / lizhimins <707364...@qq.com> [ISSUE #8269] Support pop consumption filter in long polling service Report URL: https://github.com/apache/rocketmq/actions/runs/9398646039 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user drpmma (triggered by drpmma). Head commit for run: e05b3bfddfe0d7dc283c9a23ed372444f52e1852 / zhouxiang Fix pop orderly commitOffset when NO_MATCHED_MESSAGE Report URL: https://github.com/apache/rocketmq/actions/runs/9398185148 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user RongtongJin (triggered by RongtongJin). Head commit for run: b938258752d0ee6713a69d4d4e87c1e2cd2058b9 / RongtongJin Add a switch to allow reading uncommitted messages Report URL: https://github.com/apache/rocketmq/actions/runs/9398908451 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Coverage" failed!
The GitHub Actions job "Coverage" on rocketmq.git has failed. Run started by GitHub user RongtongJin (triggered by RongtongJin). Head commit for run: b938258752d0ee6713a69d4d4e87c1e2cd2058b9 / RongtongJin Add a switch to allow reading uncommitted messages Report URL: https://github.com/apache/rocketmq/actions/runs/9398908479 With regards, GitHub Actions via GitBox
Re: [PR] [ISSUE #8269] Support pop consumption filter in long polling service [rocketmq]
codecov-commenter commented on PR #8271: URL: https://github.com/apache/rocketmq/pull/8271#issuecomment-2151902214 ## [Codecov](https://app.codecov.io/gh/apache/rocketmq/pull/8271?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) Report Attention: Patch coverage is `20.40816%` with `39 lines` in your changes missing coverage. Please review. > Project coverage is 42.84%. Comparing base [(`155bcbf`)](https://app.codecov.io/gh/apache/rocketmq/commit/155bcbf23cf026d94a3233d5944571c3532a130d?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) to head [(`254d3f4`)](https://app.codecov.io/gh/apache/rocketmq/commit/254d3f47bc29bf44868790b6b12385aeb2f2a45e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache). | [Files](https://app.codecov.io/gh/apache/rocketmq/pull/8271?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | Patch % | Lines | |---|---|---| | [...etmq/broker/longpolling/PopLongPollingService.java](https://app.codecov.io/gh/apache/rocketmq/pull/8271?src=pr&el=tree&filepath=broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fbroker%2Flongpolling%2FPopLongPollingService.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvbG9uZ3BvbGxpbmcvUG9wTG9uZ1BvbGxpbmdTZXJ2aWNlLmphdmE=) | 6.25% | [15 Missing :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/8271?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | | [...rocketmq/broker/processor/PopMessageProcessor.java](https://app.codecov.io/gh/apache/rocketmq/pull/8271?src=pr&el=tree&filepath=broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fbroker%2Fprocessor%2FPopMessageProcessor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvcHJvY2Vzc29yL1BvcE1lc3NhZ2VQcm9jZXNzb3IuamF2YQ==) | 42.85% | [7 Missing and 1 partial :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/8271?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | | [...ker/longpolling/NotifyMessageArrivingListener.java](https://app.codecov.io/gh/apache/rocketmq/pull/8271?src=pr&el=tree&filepath=broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fbroker%2Flongpolling%2FNotifyMessageArrivingListener.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvbG9uZ3BvbGxpbmcvTm90aWZ5TWVzc2FnZUFycml2aW5nTGlzdGVuZXIuamF2YQ==) | 0.00% | [6 Missing :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/8271?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | | [...rocketmq/broker/processor/AckMessageProcessor.java](https://app.codecov.io/gh/apache/rocketmq/pull/8271?src=pr&el=tree&filepath=broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fbroker%2Fprocessor%2FAckMessageProcessor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvcHJvY2Vzc29yL0Fja01lc3NhZ2VQcm9jZXNzb3IuamF2YQ==) | 0.00% | [5 Missing :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/8271?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | | [...cketmq/broker/processor/NotificationProcessor.java](https://app.codecov.io/gh/apache/rocketmq/pull/8271?src=pr&el=tree&filepath=broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fbroker%2Fprocessor%2FNotificationProcessor.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvcHJvY2Vzc29yL05vdGlmaWNhdGlvblByb2Nlc3Nvci5qYXZh) | 0.00% | [3 Missing :warning: ](https://app.codecov.io/gh/apache/rocketmq/pull/8271?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache) | | [...apache/rocketmq/broker/longpolling/PopRequest.java](https://app.codecov.io/gh/apache/rocketmq/pull/8271?src=pr&el=tree&filepath=broker%2Fsrc%2Fmain%2Fjava%2Forg%2Fapache%2Frocketmq%2Fbroker%2Flongpolling%2FPopRequest.java&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=apache#diff-YnJva2VyL3NyYy9tYWluL2phdmEvb3JnL2FwYWNoZS9yb2NrZXRtcS9icm9rZXIvbG9uZ3BvbGxpbmcvUG9wUmVxdWVzdC5qYXZh)
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user RongtongJin (triggered by RongtongJin). Head commit for run: 155bcbf23cf026d94a3233d5944571c3532a130d / lizhimins <707364...@qq.com> [ISSUE #8197] Support fast filter message in tiered storage (#8198) Report URL: https://github.com/apache/rocketmq/actions/runs/9398926346 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "PR-CI" failed!
The GitHub Actions job "PR-CI" on rocketmq.git has failed. Run started by GitHub user RongtongJin (triggered by RongtongJin). Head commit for run: b938258752d0ee6713a69d4d4e87c1e2cd2058b9 / RongtongJin Add a switch to allow reading uncommitted messages Report URL: https://github.com/apache/rocketmq/actions/runs/9398908466 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: 155bcbf23cf026d94a3233d5944571c3532a130d / lizhimins <707364...@qq.com> [ISSUE #8197] Support fast filter message in tiered storage (#8198) Report URL: https://github.com/apache/rocketmq/actions/runs/9398683583 With regards, GitHub Actions via GitBox
[PR] [ISSUE #8269] Support pop consumption filter in long polling service [rocketmq]
lizhimins opened a new pull request, #8271: URL: https://github.com/apache/rocketmq/pull/8271 ### Which Issue(s) This PR Fixes Fixes #8269 ### Brief Description ### How Did You Test This Change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user drpmma (triggered by drpmma). Head commit for run: 155bcbf23cf026d94a3233d5944571c3532a130d / lizhimins <707364...@qq.com> [ISSUE #8197] Support fast filter message in tiered storage (#8198) Report URL: https://github.com/apache/rocketmq/actions/runs/9398226903 With regards, GitHub Actions via GitBox
[PR] [ISSUE #8268] Fix pop orderly commitOffset when NO_MATCHED_MESSAGE [rocketmq]
drpmma opened a new pull request, #8270: URL: https://github.com/apache/rocketmq/pull/8270 ### Which Issue(s) This PR Fixes Fixes #8268 ### Brief Description Fix pop orderly commitOffset when NO_MATCHED_MESSAGE ### How Did You Test This Change? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Enhancement] Support pop consumption filter in long polling service [rocketmq]
lizhimins opened a new issue, #8269: URL: https://github.com/apache/rocketmq/issues/8269 ### Before Creating the Enhancement Request - [X] I have confirmed that this should be classified as an enhancement rather than a bug/feature. ### Summary Support pop consumption filter in long polling service ### Motivation When a message is written, it wakes up long polling. In the current design, for pull message operations, only clients with matching filters are woken up, while for pop operations, all subscribed groups with pending long polling are woken up. This issue and pull request optimizes the notify mechanism, reducing CPU cost in scenarios where both pop consumption and tag filtering are present. ### Describe the Solution You'd Like Support pop consumption filter in long polling service ### Describe Alternatives You've Considered Support pop consumption filter in long polling service ### Additional Context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[I] [Bug] Fix pop orderly offset [rocketmq]
drpmma opened a new issue, #8268: URL: https://github.com/apache/rocketmq/issues/8268 ### Before Creating the Bug Report - [X] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions). - [X] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate. - [X] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ. ### Runtime platform environment Not related ### RocketMQ version 5.2.0 ### JDK Version _No response_ ### Describe the Bug When pop orderly returns NO_MATCHED_MESSAGE, it will invoke popBufferMergeService.addCkMock which is an async offset commit behavior leading to potential repeat offset. ### Steps to Reproduce Pop orderly with NO_MATCHED_MESSAGE, then pop message successfully. ### What Did You Expect to See? Repeat offset or repeat message consumption ### What Did You See Instead? No repeat ### Additional Context _No response_ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" is working again!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has succeeded. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: 155bcbf23cf026d94a3233d5944571c3532a130d / lizhimins <707364...@qq.com> [ISSUE #8197] Support fast filter message in tiered storage (#8198) Report URL: https://github.com/apache/rocketmq/actions/runs/9397108442 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user guyinyou (triggered by guyinyou). Head commit for run: 603ca1cdfa2b79284685bb7dec2a202adc49a590 / guyinyou fix codestyle Report URL: https://github.com/apache/rocketmq/actions/runs/9396670961 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "PUSH-CI" failed!
The GitHub Actions job "PUSH-CI" on rocketmq.git has failed. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: 155bcbf23cf026d94a3233d5944571c3532a130d / lizhimins <707364...@qq.com> [ISSUE #8197] Support fast filter message in tiered storage (#8198) Report URL: https://github.com/apache/rocketmq/actions/runs/9397108438 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" is working again!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has succeeded. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: 155bcbf23cf026d94a3233d5944571c3532a130d / lizhimins <707364...@qq.com> [ISSUE #8197] Support fast filter message in tiered storage (#8198) Report URL: https://github.com/apache/rocketmq/actions/runs/9397108454 With regards, GitHub Actions via GitBox
(rocketmq) branch develop updated (7558850df1 -> 155bcbf23c)
This is an automated email from the ASF dual-hosted git repository. lizhimin pushed a change to branch develop in repository https://gitbox.apache.org/repos/asf/rocketmq.git from 7558850df1 [ISSUE #8245]Fix typo in user_guide.md add 155bcbf23c [ISSUE #8197] Support fast filter message in tiered storage (#8198) No new revisions were added by this update. Summary of changes: .../tieredstore/core/MessageStoreFetcherImpl.java | 39 +--- .../core/MessageStoreFetcherImplTest.java | 102 - 2 files changed, 125 insertions(+), 16 deletions(-)
Re: [I] [Enhancement] Support fast filter message in tiered storage [rocketmq]
lizhimins closed issue #8197: [Enhancement] Support fast filter message in tiered storage URL: https://github.com/apache/rocketmq/issues/8197 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [ISSUE #8197] Support fast filter message by tag in tiered storage [rocketmq]
lizhimins merged PR #8198: URL: https://github.com/apache/rocketmq/pull/8198 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Coverage" is working again!
The GitHub Actions job "Coverage" on rocketmq.git has succeeded. Run started by GitHub user guyinyou (triggered by guyinyou). Head commit for run: 603ca1cdfa2b79284685bb7dec2a202adc49a590 / guyinyou fix codestyle Report URL: https://github.com/apache/rocketmq/actions/runs/9396670964 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" is working again!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has succeeded. Run started by GitHub user lizhimins (triggered by lizhimins). Head commit for run: ca775a6631c980346389c804498cf583a79670c6 / lizhimins <707364...@qq.com> [ISSUE #8197] Support fast filter message in tiered storage Report URL: https://github.com/apache/rocketmq/actions/runs/9396136777 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user guyinyou (triggered by guyinyou). Head commit for run: f6f37f7f1da1a87b0f19bc337f1cd7c4721a070a / gaoyang.cgy add mqadmin command for updating topic config in batch Report URL: https://github.com/apache/rocketmq/actions/runs/9396220683 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "E2E test for pull request" failed!
The GitHub Actions job "E2E test for pull request" on rocketmq.git has failed. Run started by GitHub user guyinyou (triggered by guyinyou). Head commit for run: 7558850df1773ffa67abae4f1b2ceaa7d5060e09 / liuzc9 <90489940+liu...@users.noreply.github.com> [ISSUE #8245]Fix typo in user_guide.md Report URL: https://github.com/apache/rocketmq/actions/runs/9396703818 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "Build and Run Tests by Bazel" failed!
The GitHub Actions job "Build and Run Tests by Bazel" on rocketmq.git has failed. Run started by GitHub user guyinyou (triggered by guyinyou). Head commit for run: 603ca1cdfa2b79284685bb7dec2a202adc49a590 / guyinyou fix codestyle Report URL: https://github.com/apache/rocketmq/actions/runs/9396670965 With regards, GitHub Actions via GitBox
[GH] (rocketmq): Workflow run "PR-CI" is working again!
The GitHub Actions job "PR-CI" on rocketmq.git has succeeded. Run started by GitHub user guyinyou (triggered by guyinyou). Head commit for run: 603ca1cdfa2b79284685bb7dec2a202adc49a590 / guyinyou fix codestyle Report URL: https://github.com/apache/rocketmq/actions/runs/9396670972 With regards, GitHub Actions via GitBox
Re: [PR] [ISSUE #116] feat: support client event notification [rocketmq-mqtt]
DongyuanPan commented on code in PR #283: URL: https://github.com/apache/rocketmq-mqtt/pull/283#discussion_r1628869462 ## mqtt-cs/src/main/java/org/apache/rocketmq/mqtt/cs/protocol/mqtt/handler/MqttConnectHandler.java: ## @@ -80,6 +85,9 @@ public void doHandler(ChannelHandlerContext ctx, MqttConnectMessage connectMessa ChannelInfo.setClientId(channel, connectMessage.payload().clientIdentifier()); ChannelInfo.setCleanSessionFlag(channel, variableHeader.isCleanSession()); +// add client online event +clientEventHookManager.putClientEvent(channel, ClientEventType.CONNECT); Review Comment: yes, should be created after connection success as save will message -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@rocketmq.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GH] (rocketmq): Workflow run "Build and Run Tests by Maven" failed!
The GitHub Actions job "Build and Run Tests by Maven" on rocketmq.git has failed. Run started by GitHub user zhiliatom (triggered by zhiliatom). Head commit for run: d6dc592def07ac5ed54959accc9a8aecbceda3f4 / zzl fix getBrokerConfig command prompt Report URL: https://github.com/apache/rocketmq/actions/runs/9396201457 With regards, GitHub Actions via GitBox
