SEZ9 commented on issue #9940: URL: https://github.com/apache/seatunnel/issues/9940#issuecomment-5707793831
Thanks @SEPURI-SAI-KRISHNA for laying this out so carefully, and for keeping it separate from the two conclusions already recorded here. To restate where this issue stands on the original symptom (`start.mode=CONSUME_FROM_GROUP_OFFSETS` starting from offset 0): - The checkpoint restore / failover path, where restored offsets were reset and the start mode reapplied, is addressed by #10778 and should be verified on `dev` or the next release containing it. - A cold start with `commit.on.checkpoint = false` and no broker-side committed group offset follows RocketMQ group-offset behavior, as @heye1005 confirmed, and is not treated as a bug here. The third path you describe — committed offsets exist on the broker, but `RocketMqAdminUtil.currentOffsets` swallows a `MQClientException` with code 17 because `TOPIC_NOT_EXIST` and "No topic route info in name server" share that code in 4.9.4, so the enumerator sees an empty map and falls back to the first offset — is a plausible explanation for a rewind on a plain start, and the 4m24s route gap you measured in CI makes the "unlucky once" argument credible with a 1s discovery interval. I agree it is best tracked in #12332 rather than reopening the discussion here, since it does not depend on restore or on a missing group offset. Concrete asks so we can close the loop on this issue: 1. @heye1005: if you can, re-run your scenario on current `dev` (with #10778) and confirm whether the restore-path duplicate consumption is gone. If so, we can close this issue and continue any remaining discussion in #12332. 2. If you still see consumption restarting from 0 on an ordinary start, please attach: the restart type, your checkpoint settings, whether the broker still shows committed offsets for `track_report_group` at that moment, and whether any `CODE: 17` / "No topic route info" warnings appear in the job log around startup. That would tell us whether it matches the path @SEPURI-SAI-KRISHNA described or something else. 3. @SEPURI-SAI-KRISHNA: in #12332, it would help to note that the fix should distinguish a genuinely missing topic from a transient route lookup failure rather than treating both as "no committed offsets", so we do not silently fall back to the first offset on a temporary name server gap. I'll leave this open until the `dev` verification comes back. <!-- streview-comment:1115 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
