Hi Spark Friends,

Anyone interested in using Golang to write Spark application? We
created a Spark
Connect Go Client library <https://github.com/apache/spark-connect-go>.
Would love to hear feedback/thoughts from the community.

Please see the quick start guide
<https://github.com/apache/spark-connect-go/blob/master/quick-start.md>
about how to use it. Following is a very short Spark Connect application in
Go:

func main() {
        spark, _ := 
sql.SparkSession.Builder.Remote("sc://localhost:15002").Build()
        defer spark.Stop()

        df, _ := spark.Sql("select 'apple' as word, 123 as count union all
select 'orange' as word, 456 as count")
        df.Show(100, false)
        df.Collect()

        df.Write().Mode("overwrite").
                Format("parquet").
                Save("file:///tmp/spark-connect-write-example-output.parquet")

        df = spark.Read().Format("parquet").
                Load("file:///tmp/spark-connect-write-example-output.parquet")
        df.Show(100, false)

        df.CreateTempView("view1", true, false)
        df, _ = spark.Sql("select count, word from view1 order by count")
}


Many thanks to Martin, Hyukjin, Ruifeng and Denny for creating and working
together on this repo! Welcome more people to contribute :)

Best,
Bo

Reply via email to