🤣 El dom, 30 ago 2026 a las 3:53, Hyukjin Kwon (<[email protected]>) escribió:
> Let me -1 on my own and finish the discussion thread first for now. Will > start rc2 later. > > On Sun, Aug 30, 2026 at 10:29 AM Hyukjin Kwon <[email protected]> > wrote: > >> Ah that's fair. Let me raise a thread separately to see if there are more >> thoughts too but let me not pause this vote at the same time for now 👍 >> >> On Sun, 30 Aug 2026 at 10:28, Ángel Álvarez Pascua < >> [email protected]> wrote: >> >>> +1 (non-binding) >>> >>> Thanks, that makes sense. I don’t expect there to be much objection. I >>> was mainly surprised that there hadn’t been a separate discussion before >>> the release vote, given the addition of the Python wrapper and new package. >>> >>> >>> >>> El dom, 30 ago 2026 a las 3:23, Hyukjin Kwon (<[email protected]>) >>> escribió: >>> >>>> It is the Rust Client, and the new one is a Python wrapper of the Rust >>>> client, which conceptually thought did not require an SPIP but rather a >>>> separate feature. If you feel like it has to be discussed separately, I can >>>> bring it up and pause the release vote for now. >>>> >>>> On Sun, 30 Aug 2026 at 10:20, Ángel Álvarez Pascua < >>>> [email protected]> wrote: >>>> >>>>> Could you please point me to the previous dev@ discussion or SPIP for >>>>> this proposal? >>>>> >>>>> I found the earlier discussion about creating the Spark Connect Rust >>>>> client repository, but not one covering the ground-up Rust rewrite and the >>>>> new pyspark-client-rust package. Given the scope and user-facing >>>>> nature of these changes, I would have expected them to be discussed before >>>>> reaching the release vote, possibly through the SPIP process. >>>>> >>>>> El dom, 30 ago 2026 a las 0:37, Hyukjin Kwon (<[email protected]>) >>>>> escribió: >>>>> >>>>>> Please vote on releasing the following candidate as Apache Spark >>>>>> Connect Rust >>>>>> Client 4.2.0. >>>>>> >>>>>> This is the *first* release of the ground-up Rust rewrite of the >>>>>> Spark Connect >>>>>> Python client (published to PyPI as pyspark-client-rust): a drop-in >>>>>> replacement >>>>>> for pyspark-client <https://pypi.org/project/pyspark-client/>, >>>>>> backed by a native Rust core instead of py4j/grpcio. >>>>>> >>>>>> The vote is open until Thursday, September 4, 2026, 5:00 PM PDT and >>>>>> passes if a majority of +1 PMC votes are cast, with a minimum of >>>>>> 3 +1 votes. >>>>>> >>>>>> [ ] +1 Release this package as Apache Spark Connect Rust Client 4.2.0 >>>>>> [ ] -1 Do not release this package because … >>>>>> >>>>>> TAG: >>>>>> https://github.com/apache/spark-connect-rust/releases/tag/v4.2.0-rc1 >>>>>> (commit: f77df70) >>>>>> >>>>>> RELEASE FILES (wheels + sdist, built and attached by GitHub Actions): >>>>>> https://github.com/apache/spark-connect-rust/releases/tag/v4.2.0-rc1 >>>>>> >>>>>> DOCUMENTATION: >>>>>> https://apache.github.io/spark-connect-rust/ >>>>>> >>>>>> LIST OF ISSUES: >>>>>> https://issues.apache.org/jira/projects/SPARK/versions/12357484 >>>>>> HOW TO TRY IT — PYTHON (pip) >>>>>> >>>>>> The release attaches abi3 wheels (Linux x86_64/aarch64, macOS >>>>>> universal2, >>>>>> Windows x86_64; CPython 3.9+) plus a source distribution. Grab the >>>>>> wheel for >>>>>> your platform from the release page and install it directly, e.g.: >>>>>> >>>>>> pip install >>>>>> https://github.com/apache/spark-connect-rust/releases/download/v4.2.0-rc1/pyspark_client_rust-4.2.0-cp39-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl >>>>>> >>>>>> Then use it exactly like PySpark. See if your existing workloads work >>>>>> with this. >>>>>> HOW TO TEST IT — RUST (cargo) >>>>>> >>>>>> The Rust crates are intentionally NOT on crates.io for a release >>>>>> candidate >>>>>> (crates.io is immutable; the crates are published there only when >>>>>> the release is >>>>>> finalized). Test against the tagged source instead: >>>>>> >>>>>> # A) Pin a git dependency to the RC tag in your project's Cargo.toml: >>>>>> [dependencies] >>>>>> apache-spark-connect = { git = >>>>>> "https://github.com/apache/spark-connect-rust";, tag = "v4.2.0-rc1" } >>>>>> >>>>>> # B) Or build and run the crates' own tests from the tag: >>>>>> git clone --branch v4.2.0-rc1 >>>>>> https://github.com/apache/spark-connect-rust >>>>>> cd spark-connect-rust >>>>>> cargo build --release >>>>>> cargo test >>>>>> >>>>>> HIGHLIGHTS >>>>>> >>>>>> - *Drop-in PySpark Connect Python API*, verified against real >>>>>> PySpark 4.2.0 two >>>>>> ways: the official PySpark test suite runs against this client, >>>>>> and a >>>>>> behavioral API-parity sweep compares the Python API surface >>>>>> (functions and >>>>>> DataFrame / Column / SparkSession / GroupedData / Window methods) >>>>>> call-by-call >>>>>> to PySpark. >>>>>> - Spark Connect Rust API *full* coverage, including *Rust UDF*s >>>>>> (docs: https://apache.github.io/spark-connect-rust/). >>>>>> - *Native Rust transport* via tonic; cloudpickle vendored to >>>>>> match upstream, so >>>>>> no grpcio/py4j at runtime. and *native type conversion.* >>>>>> >>>>>> WHY DOES THE VERSION START AT 4.2.0? >>>>>> >>>>>> Even though this is the first release of the rewrite, the version >>>>>> starts at >>>>>> 4.2.0 to track the Apache Spark / PySpark version it targets. This >>>>>> (1) removes >>>>>> any ambiguity about which Spark version a given client is compatible >>>>>> with, and >>>>>> (2) lets PySpark users pip install pyspark-client-rust== exactly >>>>>> as they would pyspark-client, keeping the same versioning across both. >>>>>> >>>>>> Thanks, >>>>>> Hyukjin Kwon >>>>>> >>>>>>
