spark can't connect to kafka via sasl_ssl

Hello, my spark client program is as following: import org.apache.spark.sql.SparkSession object Sparkafka { def main(args:Array[String]):Unit = { val spark = SparkSession.builder.appName("Mykafka").getOrCreate() val df = spark .readStream .format("kafka") .opt

Re: Spark Avro Java 17 Compatibility

See the documentation at spark.apache.org . Spark 2.4 definitely does not support versions after Java 8. Spark 3.3 supports 17. (General note to anyone mailing the list, don't use a ".invalid" reply-to address) On Wed, Jul 27, 2022 at 7:47 AM Shivaraj Sivasankaran wrote: > Gentle Reminder on the

RE: Spark Avro Java 17 Compatibility

Gentle Reminder on the below query. Regards, Shivaraj Sivasankaran. From: Shivaraj Sivasankaran Sent: Tuesday, March 29, 2022 11:54 AM To: user@spark.apache.org Cc: M Vasanthakumar Subject: Spark Avro Java 17 Compatibility Hi, Am Shivaraj from Ericsson India Global Private Limited, we have ou

[Spark thread pool configurations]: I would like to configure all ThreadPoolExecutor parameters for each thread pool started in Spark

Hi everyone, My name is Alex and I've been using Spark for the past 4 years to solve most, if not all, of my data processing challenges. From time to time I go a bit left field with this :). Like embedding Spark in my JVM based application running only in `local` mode and using it as a real-time