Devin Petersohn created SPARK-56398:
---------------------------------------
Summary: [SPIP] Capability Discovery and Negotiation for Spark
Connect
Key: SPARK-56398
URL: https://issues.apache.org/jira/browse/SPARK-56398
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: 4.1.1
Reporter: Devin Petersohn
Spark Connect has no mechanism for a client to discover which operators,
functions, data types, or features a server supports. Clients discover
unsupported features only through runtime failures. With multiple server
implementations now targeting the protocol, this forces alternative backends to
either implement the full protocol surface or fail with opaque errors.
This SPIP proposes adding a GetServerCapabilities RPC to SparkConnectService
that returns the server's supported relation types, expression types, built-in
functions, data types, and coarse-grained feature flags.
Link to full SPIP design here:
https://docs.google.com/document/d/1lYPCJqh9ftOsDmM617SqtVpuSPaf5fNSD8uz1FHl_5A/edit?usp=sharing
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
