hiboyang opened a new pull request, #56933: URL: https://github.com/apache/spark/pull/56933
### What changes were proposed in this pull request? This PR adds a new `PathAwareChannelBuilder` to the Spark Connect Python client to support a URL path in the Spark Connect connection string, e.g. `sc://host1/path1:15002`. The new builder accepts both the standard connection form (`sc://host[:port][/;params]`) and the path-routed form `sc://gateway/<prefix>:<port>`. When a path prefix is present, it is prepended to every gRPC method via a client interceptor, and TLS is enabled implicitly when the resolved port is 443 (unless `;use_ssl=` is set explicitly). Related discussion: https://github.com/apache/spark/issues/56816 JIRA: https://issues.apache.org/jira/browse/SPARK-57793 ### Why are the changes needed? In Kubernetes, users commonly set up an Ingress to expose a Spark Connect driver endpoint behind a matching URL, e.g. `http://host1/path1` routed to the driver endpoint. The existing channel builder cannot carry a path prefix, so the Spark Connect client needs to be updated to support this path-based routing scenario. ### Does this PR introduce _any_ user-facing change? No behavior change for existing connection strings — the existing host-based `sc://` connection strings continue to work unchanged. This PR adds a new opt-in `PathAwareChannelBuilder` that additionally supports a path inside the URL. ### How was this patch tested? Added unit tests in `python/pyspark/sql/tests/connect/test_connect_channel.py` covering path parsing, port extraction from the final path segment, implicit TLS on port 443, and the path-prefix interceptor. Also manually tested in a local environment. ### Was this patch authored or co-authored using generative AI tooling? Generated-by: Claude Code -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
