Hi all, Spinning this out of the RC1 vote thread. Alongside the Rust Spark Connect client, we now have pyspark-client-rust: a Python wrapper that installs the same pyspark surface as pyspark-client (existing PySpark Connect code runs unchanged), but with the native Rust client (tonic) as the engine instead of grpcio/py4j. It is checked against PySpark 4.2.0 (official test suite + call-by-call API parity), and the Rust crates work standalone too.
I personally thought that it does not need a SPIP - it is a separate feature/package, not a core change - but it is user-facing, so I wanted a dedicated thread for any thoughts. Not blocking the RC1 vote. Architecture: https://apache.github.io/spark-connect-rust/architecture/ Repo: https://github.com/apache/spark-connect-rust So the main question is, how do you guys like it? Thanks, Hyukjin Kwon
