AW: Issue after change to 3.0.2

@spark Betreff: Re: Issue after change to 3.0.2 That looks to me like you have two different versions of Spark in use somewhere here. Like the cluster and driver versions aren't quite the same. Check your classpaths? On Fri, Feb 26, 2021 at 2:53 AM Bode, Meikel, NMA-CFD mailto:meikel.b

Re: Issue after change to 3.0.2

That looks to me like you have two different versions of Spark in use somewhere here. Like the cluster and driver versions aren't quite the same. Check your classpaths? On Fri, Feb 26, 2021 at 2:53 AM Bode, Meikel, NMA-CFD < meikel.b...@bertelsmann.de> wrote: > Hi All, > > > > After changing to

Re: Issue after change to 3.0.2

So you have upgraded to Spark 3.0.2? How are you running your pyspark? Is this through python virtual env or spark-submit? Sounds like it cannot create executor Can you run it in local mode? spark-submit --master local[1] --deploy-mode client Check also values for PYSPARK_PYTHON and

Issue after change to 3.0.2

Hi All, After changing to 3.0.2 I face the following issue. Thanks for any hint on that issue. Best, Meikel df = self.spark.read.json(path_in) File "/opt/spark/python/lib/pyspark.zip/pyspark/sql/readwriter.py", line 300, in json File