Re: Scala, Python or Java for Spark programming

2017-06-07 Thread Matt Tenenbaum
A lot depends on your context as well. If I'm using Spark _for analysis_, I frequently use python; it's a starting point, from which I can then leverage pandas, matplotlib/seaborn, and other powerful tools available on top of python. If the Spark outputs are the ends themselves, rather than the me

Re: spark-shell with different username

2016-04-02 Thread Matt Tenenbaum
; Thanks > > Dr Mich Talebzadeh > > > > LinkedIn * > https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw > <https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* > > > > http://talebzadehmich.

spark-shell with different username

2016-04-01 Thread Matt Tenenbaum
Hello all — tl;dr: I’m having an issue running spark-shell from my laptop (or other non-cluster-affiliated machine), and I think the issue boils down to usernames. Can I convince spark/scala that I’m someone other than $USER? A bit of background: our cluster is CDH 5.4.8, installed with Cloudera