Hyukjin Kwon created SPARK-62030:
------------------------------------
Summary: Connect directly to the hosts that no_proxy excludes
despite an invalid proxy, end a connection whose server sends no HTTP/2
settings in the connect deadline, and write grpcio's errors of the proxy and
the TLS roots to stderr
Key: SPARK-62030
URL: https://issues.apache.org/jira/browse/SPARK-62030
Project: Spark
Issue Type: Bug
Components: Connect
Affects Versions: 5.0.0
Reporter: Hyukjin Kwon
>From a review of the documented differences of the transport against PySpark
>4.3.0 (grpcio 1.85.1):
- An invalid proxy address (such as grpc_proxy at port 80837) fails every
session, also of a host that no_proxy or no_grpc_proxy excludes, which grpcio
connects to directly.
- Against a server that accepts the connection and never answers, PySpark fails
at 20.3 s, as grpcio waits for the server's HTTP/2 SETTINGS within its connect
deadline; the drop-in waits for the keepalive, 80 s, and its first plan, of the
lookup of the plan compression and the execution, 160 s.
- grpcio's C core writes the errors of a proxy URI that is not a proxy and of a
TLS roots file that cannot be read to stderr ("E... http_proxy_mapper.cc:159]
'https' scheme not supported in proxy URI"), unless GRPC_VERBOSITY is NONE; the
drop-in sends them to Rust's tracing only, which the Python client never shows.
GRPC_DEFAULT_SSL_ROOTS_FILE_PATH is read once per process by grpcio.
- The docs lack the reasons of the user agent's read time, of the multi-address
racing and the TLS roots, and the CONNECT request's headers and PySpark's debug
logs (N6).
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]