Re: how to set up pyspark eclipse, pyDev, virtualenv? syntaxError: yield from walk(

2018-04-05 Thread Andy Davidson
ser @spark" <user@spark.apache.org> Subject: Re: how to set up pyspark eclipse, pyDev, virtualenv? syntaxError: yield from walk( > FYI, there is a PR and JIRA for virtualEnv support in PySpark > > https://issues.apache.org/jira/browse/SPARK-13587 > https://github.com/apache/spark/pull/1359

Re: how to set up pyspark eclipse, pyDev, virtualenv? syntaxError: yield from walk(

2018-04-05 Thread Hyukjin Kwon
spark-unit-test- > set-up-sparkcontext > > From: Andrew Davidson <a...@santacruzintegration.com> > Date: Wednesday, April 4, 2018 at 5:36 PM > To: "user @spark" <user@spark.apache.org> > Subject: how to set up pyspark eclipse, pyDev, virtualenv? syntaxErr

Re: how to set up pyspark eclipse, pyDev, virtualenv? syntaxError: yield from walk(

2018-04-05 Thread Andy Davidson
FYI http://www.learn4master.com/algorithms/pyspark-unit-test-set-up-sparkcontext From: Andrew Davidson <a...@santacruzintegration.com> Date: Wednesday, April 4, 2018 at 5:36 PM To: "user @spark" <user@spark.apache.org> Subject: how to set up pyspark eclipse, pyDev, v

how to set up pyspark eclipse, pyDev, virtualenv? syntaxError: yield from walk(

2018-04-04 Thread Andy Davidson
I am having a heck of a time setting up my development environment. I used pip to install pyspark. I also downloaded spark from apache. My eclipse pyDev intereperter is configured as a python3 virtualenv I have a simple unit test that loads a small dataframe. Df.show() generates the following