I'm not sure whether pyspark supports python 3.6 but pyspark and python 3.6 is working on my environment.
I found the following issue and it seems to be already resolved. https://issues.apache.org/jira/plugins/servlet/mobile#issue/SPARK-19019 2017/11/02 午前11:54 "Jun Shi" <junshi...@gmail.com>: Dear spark developers: It’s so exciting to send this email to you. I have encountered the problem that if pyspark supports python3.6? (I found some answer online is no.) Can you tell me the answer which python versions does pyspark support? I’m looking forward for your answer. Thank you very much! Best, Jun