Hi, Our company is currently introducing the Spark Connect server to production.
Most of the issues have been solved yet I don't know how to configure authentication from a pySpark client to the Spark Connect server. I noticed that there is some interceptor configs at the Scala client side, where users can call the following codes: val spark = SparkSession.builder().remote(host).interceptor(...) > > to configure a client side interceptor and at the server side there is a config called spark.connect.grpc.interceptor.classes I'm wondering if there is any way to pass some authentication information at the pySpark side. If not, is there any on-road plans to support this ?
