Github user yaooqinn commented on the issue: https://github.com/apache/spark/pull/19840 i guess specifing `PYSPARK_PYTHON=~/anaconda3/envs/py3/bin/python` overwrites spark.executorEnv.PYSPARK_PYTHON by [context.py#L156](https://github.com/yaooqinn/spark/blob/8ff5663fe9a32eae79c8ee6bc310409170a8da64/python/pyspark/context.py#L156), where should use `setIfMissing`
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org