viirya opened a new pull request, #58011: URL: https://github.com/apache/spark/pull/58011
### What changes were proposed in this pull request? Adds a section to the Spark Connect overview docs (`docs/spark-connect-overview.md`) explaining how to route a Spark Connect client to the right server when several services share one hostname behind a Kubernetes Ingress or reverse proxy. It documents routing on the HTTP/2 `:authority` (via the `grpc.default_authority` channel option) with proxy host-based routing, instead of a URL path prefix, and covers the TLS case (keeping the certificate-verification name and the routing tag separate via `grpc.ssl_target_name_override`). This documents an approach that came out of the discussion on #56933 (which proposed a path-prefix-based `PathAwareChannelBuilder`). Linking it here so others hitting the same "route Spark Connect behind a shared ingress" need can find the `:authority`-based approach without re-deriving it. ### Why are the changes needed? In gRPC the method name *is* the HTTP/2 `:path` (`/spark.connect.SparkConnectService/ExecutePlan`), so a URL path prefix cannot be used to route to different Spark Connect services — a prefixed request reaches the server as an unknown method and returns `UNIMPLEMENTED`. Routing on `:authority` is the mechanism the gRPC maintainers recommend for this scenario ([grpc/grpc#14900](https://github.com/grpc/grpc/issues/14900)), but it was not documented for Spark Connect. ### Does this PR introduce _any_ user-facing change? No. Documentation only. ### How was this patch tested? Documentation only. The routing approach was verified end-to-end against a local ingress-nginx setup, over both plaintext (h2c) and TLS. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
