Sirius-LiXiaoQi opened a new issue, #16397: URL: https://github.com/apache/dubbo/issues/16397
### Pre-check - [x] I am sure that all the content I provide is in English. ### Search before asking - [x] I had searched in the [issues](https://github.com/apache/dubbo/issues?q=is%3Aissue) and found no similar issues. ### Apache Dubbo Component Java SDK (apache/dubbo) ### Dubbo Version Dubbo 3.3.6, Triple protocol (`dubbo.protocol.name=tri`), instance mode , JDK 8. ### Steps to reproduce this issue Graceful shutdown tears down the Triple `PathResolver` mapping *before* in-flight requests have drained. A request that has already been accepted by the server, but has not yet resolved its invoker, will fail with `UNIMPLEMENTED` once `afterUnExport` runs. The race can be reproduced deterministically with two breakpoints on the **provider** side — no timing luck required. Use a **thread-level** (suspend-thread, not suspend-all) breakpoint so the shutdown thread and the request thread can be released independently. Provider service (anything works, the body is irrelevant): ```java @DubboService public class DemoServiceImpl implements DemoService { @Override public String sayHello(String name) { return "hello " + name; } } Consumer: a single call is enough. @DubboReference(timeout = 500000) private DemoService demoService; demoService.sayHello("dubbo"); Reproduction procedure (provider JVM, debugger attached): 1. Set a thread-suspending breakpoint at org.apache.dubbo.rpc.protocol.tri.h12.grpc.GrpcRequestHandlerMapping#getRequestHandler, on the line Invoker<?> invoker = pathResolver.resolve(path.getPath(), group, version); // GrpcRequestHandlerMapping.java:62 1. Issue the consumer call. The request thread is now suspended before the path is resolved — i.e. the request has already been accepted by a fully healthy provider. 2. Send TERM to the provider process to start graceful shutdown, with a second thread-suspending breakpoint at pathResolver.unregister(invoker); // TripleProtocol.java:104, in TripleProtocol$1#afterUnExport 2. Step over that line so the unregister actually executes, then leave the shutdown thread suspended. TriplePathResolver no longer contains the path mapping for the service. 3. Release the request thread from step 1 and step over line 62. resolve(...) now returns null, so control falls into if (invoker == null) { throw notFound(); // GrpcRequestHandlerMapping.java:64 } Result - Provider log: **Invoker for gRPC not found** - Consumer: **org.apache.dubbo.rpc.RpcException: ... UNIMPLEMENTED : Invoker for gRPC not found** Without the debugger, the same thing happens naturally on every shutdown that overlaps with concurrent traffic; the breakpoints only make the window deterministic. ### What you expected to happen The request accepted in step 1 was accepted while the service was still exported, and the configured graceful shutdown wait (dubbo.service.shutdown.wait) had not elapsed. It should have been served normally. Concretely: 1. PathResolver.unregister(invoker) should not take effect until in-flight requests have drained — the path mapping must outlive the request-draining phase rather than being removed at the start of it. 2. If the server has genuinely stopped accepting requests, the failure should be retriable: UNAVAILABLE, or an HTTP/2 GOAWAY that lets the consumer re-route to another provider instance. UNIMPLEMENTED is semantically wrong here — it means "this server does not implement the service", so consumer-side cluster fault tolerance treats it as a permanent definition error and does not fail over. Every rolling restart of a Triple provider therefore surfaces as business-visible errors on the consumer, which defeats the purpose of graceful shutdown. ### Anything else Code paths involved (3.3.6): - org.apache.dubbo.rpc.protocol.tri.TripleProtocol$1#afterUnExport — pathResolver.unregister(invoker) (TripleProtocol.java:111), registered by TripleProtocol#export - org.apache.dubbo.rpc.protocol.tri.h12.grpc.GrpcRequestHandlerMapping#getRequestHandler (GrpcRequestHandlerMapping.java:62 resolve, :64 throw notFound()) - org.apache.dubbo.rpc.PathResolver#unregister / #resolve, org.apache.dubbo.rpc.protocol.tri.TriplePathResolver The same window applies to any RequestHandlerMapping that resolves through PathResolver during shutdown, not only the gRPC one. ### Do you have a (mini) reproduction demo? - [x] Yes, I have a minimal reproduction demo to help resolve this issue more effectively! ### Are you willing to submit a pull request to fix on your own? - [ ] Yes I am willing to submit a pull request on my own! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
