Re: Scala vs Python for ETL with Spark

And you are an expert on python! Idiomatic... Please do everyone a favor and stop commenting on things you have no idea... I build ETL systems python that wiped java commercial stacks left and right. Pyspark was and is and will be a second class citizen in spark world. That has nothing to do with

Re: [discuss] dropping Python 2.6 support

+1 Companies that use stock python in redhat 2.6 will need to upgrade or install fresh version wich is total of 3.5 minutes so no issues ... On Tue, Jan 5, 2016 at 2:17 AM, Reynold Xin wrote: > Does anybody here care about us dropping support for Python 2.6 in Spark > 2.0?

Re: hadoop2.6.0 + spark1.4.1 + python2.7.10

orker, it works too. > > I am not sure if this will help or not for your use-case. > > > > Sincerely, > Ashish > > On Mon, Sep 7, 2015 at 11:04 PM, Sasha Kacanski <skacan...@gmail.com> > wrote: > >> Thanks Ashish, >> nice blog but does not cover my i

hadoop2.6.0 + spark1.4.1 + python2.7.10

Hi, I am successfully running python app via pyCharm in local mode setMaster("local[*]") When I turn on SparkConf().setMaster("yarn-client") and run via park-submit PysparkPandas.py I run into issue: Error from python worker: /cube/PY/Python27/bin/python: No module named pyspark PYTHONPATH

ZeroMQ and pyspark.streaming

Does pyspark supports zeroMQ? I see that java does it, but I am not sure for Python? regards -- Aleksandar Kacanski

Suitability for spark for master worker distributed patterns...

Hi, Did anyone tried to replace gigaspaces implementation of master worker with spark standalone or hadoop driven implementation... I guess I am looking to find out what are pros and cons and if people tried it on the production side (grid or hadoop) Regards, -- Aleksandar Kacanski