Re: Spark 3.1 with spark AVRO

2022-03-10 Thread Yong Zhang
me. Yong From: Artemis User Sent: Thursday, March 10, 2022 3:13 PM To: user Subject: Re: Spark 3.1 with spark AVRO It must be some misconfiguration in your environment. Do you perhaps have a hardwired $SPARK_HOME env variable in your shell? An easy test would be to place the spark-avro ja

Re: Spark 3.1 with spark AVRO

2022-03-10 Thread Artemis User
.  This will guarantee that the jar file is on the classpath of Spark driver and executors.. On 3/10/22 1:24 PM, Yong Zhang wrote: Hi, I am puzzled with this issue of Spark 3.1 version to read avro file. Everything is done on my local mac laptop so far, and I really don't know where the issue comes from

Spark 3.1 with spark AVRO

2022-03-10 Thread Yong Zhang
Hi, I am puzzled with this issue of Spark 3.1 version to read avro file. Everything is done on my local mac laptop so far, and I really don't know where the issue comes from, and I googled a lot and cannot find any clue. I am always using Spark 2.4 version, as it is really mature