nic-6443 opened a new issue, #13498:
URL: https://github.com/apache/apisix/issues/13498
### Current Behavior
When `proxy-mirror` is configured with a `grpc://` host, the mirrored
request arrives at the mirror target with `:path = /proxy_mirror_grpc` (the
name of the internal nginx location) instead of the real gRPC method path. A
gRPC server receiving such a call just rejects it with `UNIMPLEMENTED`, so gRPC
mirroring is effectively unusable — the mirror backend never sees a valid call.
The root cause is in the generated nginx config: the HTTP mirror location
forwards with `proxy_pass $upstream_mirror_uri` which carries the full URI, but
the gRPC mirror location only has `grpc_pass $upstream_mirror_host`.
`grpc_pass` cannot carry a URI, so the grpc module builds `:path` from the
subrequest's own URI, which is the mirror location name `/proxy_mirror_grpc`.
The existing test case in `t/plugin/proxy-mirror3.t` (added in #9388) only
asserts `Connection refused` against a closed port, so it never verified what
the mirrored gRPC request actually contains and this went unnoticed.
There is also a phase-timing aspect: plugins like `grpc-web` rewrite the
request URI in their access phase, while `proxy-mirror` captures the URI in its
rewrite phase, so the mirror path must be taken after all access phase plugins
have run.
### Expected Behavior
The mirrored gRPC request keeps the same method path as the upstream
request, including rewrites done by access phase plugins (e.g. `grpc-web`).
### Error Logs
No error on the APISIX side. The mirror target receives `POST
/proxy_mirror_grpc HTTP/2.0` (verified with an h2c server logging the incoming
`:path`).
### Steps to Reproduce
1. Create a route with a `grpc` scheme upstream and `proxy-mirror` pointing
to a `grpc://` mirror host:
```yaml
routes:
- uris:
- /helloworld.Greeter/SayHello
plugins:
proxy-mirror:
host: grpc://127.0.0.1:19797
upstream:
scheme: grpc
nodes:
"127.0.0.1:10051": 1
type: roundrobin
```
2. Run any h2c server on 19797 that logs the request path (the mirror
target).
3. Send a request: `grpcurl -plaintext -d '{"name":"apisix"}' 127.0.0.1:1984
helloworld.Greeter.SayHello`
4. The main upstream receives `POST /helloworld.Greeter/SayHello HTTP/2.0`,
while the mirror target receives `POST /proxy_mirror_grpc HTTP/2.0`.
### Environment
- APISIX version: reproduced on master and 3.14.1
- Operating system: Ubuntu 22.04
- OpenResty / Nginx version: apisix-runtime (openresty 1.29)
I have a fix ready: capture the mirror path into a new
`$upstream_mirror_grpc_path` variable in the `before_proxy` phase (which runs
after all access phase plugins have rewritten the URI), and rewrite the mirror
subrequest URI with it in the `proxy_mirror_grpc` location before `grpc_pass`.
Will submit a PR shortly.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
