zergduan opened a new issue, #9879: URL: https://github.com/apache/rocketmq/issues/9879
### Before Creating the Bug Report - [x] I found a bug, not just asking a question, which should be created in [GitHub Discussions](https://github.com/apache/rocketmq/discussions). - [x] I have searched the [GitHub Issues](https://github.com/apache/rocketmq/issues) and [GitHub Discussions](https://github.com/apache/rocketmq/discussions) of this repository and believe that this is not a duplicate. - [x] I have confirmed that this bug belongs to the current repository, not other repositories of RocketMQ. ### Runtime platform environment Oracle Linux 9 ### RocketMQ version Release 5.3.3 & Release 5.3.4 ### JDK Version OpenJDK 21 ### Describe the Bug 此问题在 https://github.com/apache/rocketmq/discussions/9831 中有具体描述 In a master-slave cluster, the Slave should support consuming backlogged messages when the Master is down. However, this functionality is not achievable in versions 5.3.3 and 5.3.4. ### Steps to Reproduce Test Case: Cluster Type: Master-Slave Cluster Topic: TP-FTEST-Normal Consumer Group: CG-FTEST-Normal Precondition: A large number of messages are backlogged in the cluster. Operation: Shut down the Master, then start a consumer (belonging to consumer group CG-FTEST-Normal) to attempt consuming the backlogged messages. Test Results: 1. In version 5.3.3: After the consumer connects to the Slave and attempts to consume, the consumption fails (the consumer does not receive any consumable messages). Meanwhile, the following error logs are recorded in the Slave's broker.log: > 2025-11-28 11:23:08 WARN AdminBrokerThread_10 - AdminBrokerProcessor#getConsumeStats: topic config does not exist, topic=TP-FTEST-Normal 2025-11-28 11:23:08 ERROR AdminBrokerThread_2 - No topic in this broker, client: /10.177.96.150:44658 topic: TP-FTEST-Normal 2025-11-28 11:23:08 WARN AdminBrokerThread_15 - AdminBrokerProcessor#getConsumeStats: topic config does not exist, topic=TP-FTEST-Normal 2025-11-28 11:23:08 ERROR AdminBrokerThread_1 - No topic in this broker, client: /10.177.96.150:44658 topic: TP-FTEST-Normal 2025-11-28 11:23:24 INFO brokerOutApi_thread_4 - Registering current broker to name server completed. TargetHost=10.177.96.143:19876 2025-11-28 11:23:24 INFO brokerOutApi_thread_3 - Registering current broker to name server completed. TargetHost=10.177.96.144:19876 2025-11-28 11:23:26 INFO HeartbeatThread_2 - create new topic TopicConfig [topicName=%RETRY%CG-FTEST-Normal, readQueueNums=1, writeQueueNums=1, perm=RW-, topicFilterType=SINGLE_TAG, topicSysFlag=0, order=false, attributes={}] 2025-11-28 11:23:26 INFO brokerOutApi_thread_1 - Registering current broker to name server completed. TargetHost=10.177.96.144:19876 2025-11-28 11:23:26 INFO brokerOutApi_thread_2 - Registering current broker to name server completed. TargetHost=10.177.96.143:19876 2025-11-28 11:23:26 INFO HeartbeatThread_2 - new consumer connected, group: CG-FTEST-Normal CONSUME_PASSIVELY CLUSTERING channel: ClientChannelInfo [channel=[id: 0x646bfb2b, L:/10.177.96.146:22922 - R:/10.177.96.149:42584], clientId=10.177.96.149@10697#2588002808595, language=JAVA, version=477, lastUpdateTimestamp=1764300206686] 2025-11-28 11:23:26 INFO HeartbeatThread_2 - subscription changed, add new topic, group: CG-FTEST-Normal SubscriptionData [classFilterMode=false, topic=%RETRY%CG-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764300206128, expressionType=TAG] 2025-11-28 11:23:26 INFO HeartbeatThread_2 - subscription changed, add new topic, group: CG-FTEST-Normal SubscriptionData [classFilterMode=false, topic=TP-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764300206119, expressionType=TAG] 2025-11-28 11:23:26 INFO HeartbeatThread_2 - ClientManageProcessor: registerConsumer info changed, SDK address=10.177.96.149:42584, consumerData=ConsumerData [groupName=CG-FTEST-Normal, consumeType=CONSUME_PASSIVELY, messageModel=CLUSTERING, consumeFromWhere=CONSUME_FROM_FIRST_OFFSET, unitMode=false, subscriptionDataSet=[SubscriptionData [classFilterMode=false, topic=%RETRY%CG-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764300206128, expressionType=TAG], SubscriptionData [classFilterMode=false, topic=TP-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764300206119, expressionType=TAG]]] 2025-11-28 11:23:26 INFO HeartbeatThread_2 - new producer connected, group: CLIENT_INNER_PRODUCER channel: ClientChannelInfo [channel=[id: 0x646bfb2b, L:/10.177.96.146:22922 - R:/10.177.96.149:42584], clientId=10.177.96.149@10697#2588002808595, language=JAVA, version=477, lastUpdateTimestamp=1764300206688] 2025-11-28 11:23:27 INFO HeartbeatThread_2 - subscription changed, group: CG-FTEST-Normal OLD: SubscriptionData [classFilterMode=false, topic=TP-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764300206119, expressionType=TAG] NEW: SubscriptionData [classFilterMode=false, topic=TP-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764300207437, expressionType=TAG] 2025-11-28 11:23:27 INFO HeartbeatThread_1 - subscription changed, group: CG-FTEST-Normal OLD: SubscriptionData [classFilterMode=false, topic=%RETRY%CG-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764300206128, expressionType=TAG] NEW: SubscriptionData [classFilterMode=false, topic=%RETRY%CG-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764300207449, expressionType=TAG] 2025-11-28 11:23:53 INFO BrokerControllerScheduledThread1 - Dispatch task fall behind commit log 0bytes 2025-11-28 11:23:54 INFO brokerOutApi_thread_3 - Registering current broker to name server completed. TargetHost=10.177.96.144:19876 2025-11-28 11:23:54 INFO brokerOutApi_thread_4 - Registering current broker to name server completed. TargetHost=10.177.96.143:19876 2. In version 5.3.4: After the consumer connects to the Slave and attempts to consume, the consumption also fails (the consumer does not receive any consumable messages). The relevant logs in the Slave's broker.log are as follows:Compared with version 5.3.3, the logs do not contain the entries: "AdminBrokerProcessor#getConsumeStats: topic config does not exist" "AdminBrokerThread_2 - No topic in this broker" However, the Slave node still fails to support consuming backlogged messages. > 2025-11-28 14:28:07 INFO brokerOutApi_thread_1 - Registering current broker to name server completed. TargetHost=10.177.96.143:19876 2025-11-28 14:28:07 INFO brokerOutApi_thread_2 - Registering current broker to name server completed. TargetHost=10.177.96.144:19876 2025-11-28 14:28:07 INFO BrokerControllerScheduledThread1 - Update master address from 10.177.96.145:22922 to null 2025-11-28 14:28:37 INFO BrokerControllerScheduledThread1 - Dispatch task fall behind commit log 0bytes 2025-11-28 14:28:37 INFO brokerOutApi_thread_3 - Registering current broker to name server completed. TargetHost=10.177.96.143:19876 2025-11-28 14:28:37 INFO brokerOutApi_thread_4 - Registering current broker to name server completed. TargetHost=10.177.96.144:19876 2025-11-28 14:28:45 INFO HeartbeatThread_1 - new consumer connected, group: CG-FTEST-Normal CONSUME_PASSIVELY CLUSTERING channel: ClientChannelInfo [channel=[id: 0x562632be, L:/10.177.96.146:22922 - R:/10.177.96.149:45634], clientId=10.177.96.149@23409#13706546187932, language=JAVA, version=477, lastUpdateTimestamp=1764311325161] 2025-11-28 14:28:45 INFO HeartbeatThread_1 - subscription changed, add new topic, group: CG-FTEST-Normal SubscriptionData [classFilterMode=false, topic=%RETRY%CG-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764311324672, expressionType=TAG] 2025-11-28 14:28:45 INFO HeartbeatThread_1 - subscription changed, add new topic, group: CG-FTEST-Normal SubscriptionData [classFilterMode=false, topic=TP-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764311324664, expressionType=TAG] 2025-11-28 14:28:45 INFO HeartbeatThread_1 - ClientManageProcessor: registerConsumer info changed, SDK address=10.177.96.149:45634, consumerData=ConsumerData [groupName=CG-FTEST-Normal, consumeType=CONSUME_PASSIVELY, messageModel=CLUSTERING, consumeFromWhere=CONSUME_FROM_FIRST_OFFSET, unitMode=false, subscriptionDataSet=[SubscriptionData [classFilterMode=false, topic=%RETRY%CG-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764311324672, expressionType=TAG], SubscriptionData [classFilterMode=false, topic=TP-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764311324664, expressionType=TAG]]] 2025-11-28 14:28:45 INFO HeartbeatThread_1 - new producer connected, group: CLIENT_INNER_PRODUCER channel: ClientChannelInfo [channel=[id: 0x562632be, L:/10.177.96.146:22922 - R:/10.177.96.149:45634], clientId=10.177.96.149@23409#13706546187932, language=JAVA, version=477, lastUpdateTimestamp=1764311325161] 2025-11-28 14:28:45 INFO HeartbeatThread_1 - subscription changed, group: CG-FTEST-Normal OLD: SubscriptionData [classFilterMode=false, topic=TP-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764311324664, expressionType=TAG] NEW: SubscriptionData [classFilterMode=false, topic=TP-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764311325958, expressionType=TAG] 2025-11-28 14:28:45 INFO HeartbeatThread_2 - subscription changed, group: CG-FTEST-Normal OLD: SubscriptionData [classFilterMode=false, topic=%RETRY%CG-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764311324672, expressionType=TAG] NEW: SubscriptionData [classFilterMode=false, topic=%RETRY%CG-FTEST-Normal, subString=, tagsSet=[], codeSet=[], subVersion=1764311325973, expressionType=TAG] 2025-11-28 14:29:07 INFO brokerOutApi_thread_2 - Registering current broker to name server completed. TargetHost=10.177.96.144:19876 2025-11-28 14:29:07 INFO brokerOutApi_thread_1 - Registering current broker to name server completed. TargetHost=10.177.96.143:19876 ### What Did You Expect to See? After the Master goes down, the Slave should be able to support consumers in consuming the backlogged messages of the consumer group. ### What Did You See Instead? After the Master goes down, consumers are unable to consume the backlogged messages of the consumer group from the Slave. ### Additional Context https://github.com/apache/rocketmq/discussions/9831 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
