[
https://issues.apache.org/jira/browse/SPARK-61773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-61773:
-----------------------------------
Labels: pull-request-available (was: )
> Connect through the HTTP proxy of the environment, as grpc-python does
> ----------------------------------------------------------------------
>
> Key: SPARK-61773
> URL: https://issues.apache.org/jira/browse/SPARK-61773
> Project: Spark
> Issue Type: Improvement
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Kaicheng Zhou
> Assignee: Kaicheng Zhou
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> The Rust client (pyspark-client-rust and the native Rust API) always dials
> the Spark Connect server directly, so it cannot connect from behind an egress
> HTTP proxy. PySpark's client can: grpcio reads the proxy of the environment
> for every channel and opens an HTTP/1.1 CONNECT tunnel. A user whose PySpark
> connects through a proxy today only gets retries from the drop-in.
> Honor the proxy of the environment with grpcio's rules:
> - the proxy is the first of {{grpc_proxy}}, {{https_proxy}} and
> {{http_proxy}} that is set (an empty value means no proxy); only the
> lowercase names are read
> - {{no_grpc_proxy}}, or else {{no_proxy}}, lists the hosts that are not
> proxied: the host or a parent domain, or an IP in a CIDR range
> - only an {{http://}} proxy is supported; credentials in the URI are sent as
> Proxy-Authorization: Basic
> - TLS is negotiated with the Spark server inside the tunnel, so the proxy
> sees neither the token nor the data
> Two deliberate differences from grpcio: a loopback host is never proxied,
> because a token is sent to localhost without TLS (grpcio proxies it, which
> shows the token to the proxy in clear), and a proxy address with an invalid
> host or port is an error instead of a channel that never connects.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]