Thank you all for sharing great projects! Last week, I saw a POC that I guess is relevant to "Wasm with Python Spark Connect" and felt very cool. https://github.com/apache/spark/pull/56418
2026年6月16日(火) 1:45 Dongjoon Hyun <[email protected]>: > Thank you for sharing, Hyukjin, Jie, David. > > Dongjoon. > > On 2026/06/15 10:21:38 Yang Jie wrote: > > I also have a local project for Scala 3 support. > > > > - Source: https://github.com/LuciferYang/spark-connect-scala3 > > - Release: > https://central.sonatype.com/artifact/io.github.luciferyang/spark-connect-spark41_3 > > > > Jie Yang > > > > On 2026/06/15 09:59:29 Hyukjin Kwon wrote: > > > Oh did not know that exists. Let's leave it as a separate project for > now, > > > and discuss later when we actually think about putting this together > into > > > apache repo or sth :-). > > > > > > On Mon, 15 Jun 2026 at 18:34, David Li <[email protected]> wrote: > > > > > > > It's cool to see an ADBC driver for Spark! > > > > > > > > Just to jump on this, we've also put out an ADBC driver for Spark > (with a > > > > few orgs contributing), abstracting over Spark Connect/HiveServer2 > > > > Thrift/Apache Livy and testing against Spark 3.5/4 and Amazon EMR: > > > > https://github.com/adbc-drivers/spark > > > > > > > > Maybe there's interest in working together on a canonical driver? > [If the > > > > conversation turns to upstreaming, I think that would also be > interesting!] > > > > > > > > -David > > > > > > > > On Mon, Jun 15, 2026, at 14:44, Hyukjin Kwon wrote: > > > > > > > > I had some of my local work, and I decided to just release them as my > > > > personal repositories, and thought it is great to share to see the > > > > interest from the community, and even later we could hopefully > discuss > > > > whether to bring this into Apache repositories or not. > > > > > > > > Nothing is intended here. Just thought it's great to share my > > > > personal projects. > > > > > > > > *1. Arrow ADBC with Spark Connect* > > > > ADBC <https://arrow.apache.org/adbc/current/index.html> > implementation > > > > which will virtually support most of languages with Spark Connect > (with > > > > Arrow ADBC) > > > > > > > > Source: https://github.com/HyukjinKwon/adbc-driver-spark > > > > Docs: https://hyukjinkwon.github.io/adbc-driver-spark/ > > > > Release: > > > > - Go: https://pkg.go.dev/github.com/HyukjinKwon/adbc-driver-spark > > > > - Python: https://pypi.org/project/adbc-driver-spark/ > > > > - Other languages: > > > > https://github.com/HyukjinKwon/adbc-driver-spark/releases > > > > > > > > *2. Scala3 with Spark Connect* > > > > > > > > Scala 3 has been a long standing request. This opens up Scala 3 > support. > > > > > > > > Source: https://github.com/HyukjinKwon/spark-connect-scala3 > > > > Docs: https://hyukjinkwon.github.io/spark-connect-scala3/ > > > > Release (Maven Central): > > > > > https://central.sonatype.com/artifact/com.github.hyukjinkwon/spark-connect-scala3-client_3 > > > > > > > > *3. Ruby with Spark Connect* > > > > > > > > Opens Ruby with Spark Connect. > > > > > > > > Source: https://github.com/HyukjinKwon/spark-connect-ruby > > > > Docs: https://hyukjinkwon.github.io/spark-connect-ruby/ > > > > Release (Gem): https://rubygems.org/gems/spark-connect > > > > > > > > *4. Wasm with Python Spark Connect* > > > > > > > > Runs a native Python Spark Connect within a browser. > > > > > > > > Source: https://github.com/HyukjinKwon/pyspark-client-wasm > > > > Docs:https://hyukjinkwon.github.io/pyspark-client-wasm/ > > > > Release (PyPI): https://pypi.org/project/pyspark-connect-web/ > > > > > > > > > > > > > > > > > > > --------------------------------------------------------------------- > > To unsubscribe e-mail: [email protected] > > > > > > --------------------------------------------------------------------- > To unsubscribe e-mail: [email protected] > >
