SEPURI-SAI-KRISHNA commented on issue #12332: URL: https://github.com/apache/seatunnel/issues/12332#issuecomment-5711568064
Superseding the mechanism in the body above. The outcome the report describes is real and #12349 fixes it, but I had the mechanism wrong, and since two reviewers have since built on it I would rather correct it here than quietly edit the body. **Code 17 never refers to the requested topic.** `DefaultMQAdminExtImpl.examineConsumeStats(group, topic)` resolves its route solely from `MixAll.getRetryTopic(group)`, bytecode offsets 0 to 7, with no try/catch and no fallback. The requested topic is only a parameter to the later `getConsumeStats` broker call, which throws `MQBrokerException` rather than `MQClientException`, and `getTopicRouteInfoFromNameServer` always throws on code 17 instead of returning null. So every `MQClientException` carrying code 17 out of this call is `%RETRY%` plus the group name and nothing else. Wherever the body says `No topic route info in name server for the topic: <topic>`, read `%RETRY%<group>`. @DanielLeens found this on #12349. **Two supporting arguments in the body do not hold.** The "a job only has to be unlucky once" line assumes `partition.discovery.interval.millis` makes the enumerator re-enter this lookup; it does not. `setPartitionStartOffset()` has one call site, `RocketMqSourceSplitEnumerator.run():180`, immediately after `fetchPendingPartitionSplit():179` in the same `synchronized (lock)` block, and `discoverySplits():280` never calls it. The lookup runs once per job start. The 4m24s route gap I measured was on a data topic, and route removal in the name server is broker scoped rather than topic scoped: `RouteInfoManager.scanNotActiveBroker` (`BROKER_CHANNEL_EXPIRED_TIME = 120000L`) closes the channel and `onChannelDestroy` sweeps `topicQueueTable` keyed by brokerName. A gap that wide takes the retry topic and the data topic together, and `offsetTopics` runs first at line 179, so that job fails with `GET_MIN_AND_MAX_OFFSETS_ERROR` rather than rewinding. So the exposure is narrower than the body claims: the rewind needs a broker registration or the name server to die between lines 179 and 180 of one synchronized block, on a group that already has committed offsets. Rare, but real, and the outcome is unchanged: committed offsets exist, the lookup fails, and the caller reads the empty map as a cold start. **Superseding the "Possible fix" section.** @SEZ9 asked on #9940 that the fix distinguish a genuinely missing topic from a transient lookup failure. Agreed, with one adjustment: that is not the axis, since code 17 is never about the requested topic. The axis is "this group has never registered, so its retry topic does not exist yet" against "a route was lost". #12349 separates them by re-resolving the requested topic on the admin client already open: if it resolves, the name server is healthy and the empty map is legitimate; if it does not, the failure surfaces. That a group must register before it can commit is verifiable the same way, since `ClientManageProcessor.heartBeat` creates `%RETRY%` plus the group name before reaching `ConsumerManager.registerConsumer`, and `PullMessageProcessor` refuses a group with no subscription config outright. CI on #12349 is green: 87 tests with 0 failures and 0 errors on both the JDK 8 and JDK 11 legs, against 87 tests with 21 failures and 29 errors on the previous commit. The new WARN fires 7 times on each leg across 7 distinct fresh consumer groups, which is the cold start path this has to keep working. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
