AsperforMias opened a new issue, #3624: URL: https://github.com/apache/dubbo-go/issues/3624
## ✅ Verification Checklist - [x] 🔍 I have searched the [existing issues](https://github.com/apache/dubbo-go/issues) and confirmed this is not a duplicate --- ## 🔧 Environment Information ### 🚀 Go Version go1.26.1 ### 📦 Dubbo-go Version v3.3.2 (also present on `main` @ `48d6e696`) ### 🌐 Protocol Configuration Triple ### 📋 Registry Configuration Nacos v2.5.1, application-level service discovery (`service-discovery-registry://...`) ### 💾 Operating System 🐧 Linux --- ## 🐛 Issue Details ### 📝 Bug Description In application-level service discovery, installing the Nacos instance listener is fire-and-forget: `serviceDiscoveryRegistry.subscribeAndNotify` runs `serviceDiscovery.AddListener(listener)` in a goroutine, and if it fails the error is only logged once: https://github.com/apache/dubbo-go/blob/v3.3.2/registry/servicediscovery/service_discovery_registry.go ```go go func() { err = s.serviceDiscovery.AddListener(listener) ... if err != nil { logger.Errorf("[Registry][ServiceDiscovery] add instance listener catch error, ...") } }() ``` There is no retry, and `SubscribeURL` does not report the failure. From that moment the consumer never receives any instance-change push for those applications: the initial `GetInstances` snapshot has already been processed, so the current state looks fine, but every subsequent provider scale up/down or restart is invisible to this consumer permanently (until restart). Note that the interface-level Nacos registry already has exponential-backoff subscribe retry (#3178, `registry/nacos/registry.go`), but the application-level path (`service-discovery-registry://`) does not use it. This is the same "one-shot attempt, failure swallowed, permanent divergence" pattern as #3615 (metadata fetch without retry), just at the subscription-channel layer. Impact: a transient Nacos error at subscribe time (reconnect storm, leader election, brief unavailability) silently turns into a permanently stale consumer. Combined with #3615's masking scenario, it can affect a single consumer pod while others stay healthy, making it look like sporadic failures. ### 🔄 Steps to Reproduce 1. Consumer subscribes via application-level discovery (`registry.type=service`). 2. Make `nacosServiceDiscovery.AddListener` fail once (e.g. temporarily make Nacos reject the gRPC subscribe while still answering `GetInstances`, or inject the failure in a unit test around `service_discovery_registry.go`'s async `AddListener` call). 3. Restore Nacos. Add/remove a provider instance. 4. The consumer never receives the instance-change event; no further `AddListener` attempt is made. ### ✅ Expected Behavior A failed application-level `AddListener` is retried with backoff (with jitter, cancellation on unsubscribe/destroy, and dedup per listener) until it succeeds, similar to the interface-level retry added in #3178. While the subscription is not established, this state should be observable (log/metric, see #3356). ### ❌ Actual Behavior The failure is logged once (`add instance listener catch error`) and never retried; the consumer silently misses all future instance events. ### 💡 Possible Solution - Reuse the backoff/dedup/cancel machinery introduced for the metadata-fetch retry fix for #3615 (single timer per registry, exponential backoff with jitter and a capped interval, unlimited attempts while the subscriber is still active). - Cancel pending retries on `UnSubscribe`/`Destroy`. - Expose the "subscription not established" state via the observability work tracked in #3356 (e.g. a gauge/counter), so a single stuck consumer pod can be alerted on. - Regression test: first `AddListener` fails, second succeeds; assert the consumer eventually receives instance events without a restart. Also assert retries stop after `UnSubscribe`/`Destroy`. Related: #3615 (metadata fetch retry), #3356 (observability), #3178 (interface-level subscribe backoff). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
