Thanks Bryn.
On Wed, Mar 5, 2014 at 9:00 PM, Bryn Keller wrote:
> Hi Christian,
>
> The PYSPARK_PYTHON environment variable specifies the python executable to
> use for pyspark. You can put the path to a virtualenv's python executable
> and it will work fine. Remember you have to have the same
Hi Christian,
The PYSPARK_PYTHON environment variable specifies the python executable to
use for pyspark. You can put the path to a virtualenv's python executable
and it will work fine. Remember you have to have the same installation at
the same path on each of your cluster nodes for pyspark to wo
Hello,
I usually create different python virtual environments for different
projects to avoid version conflicts and skip the requirement to be root to
install libs.
How can I specify to pyspark to activate a virtual environment before
executing the tasks ?
Further info on virtual envs:
http://vi
