Problem running Spark shell (1.0.0) on EMR

2014-08-05 Thread Omer Holzinger
I'm having similar problem to: http://mail-archives.apache.org/mod_mbox/spark-user/201407.mbox/browser I'm trying to follow the tutorial at: When I run: val file = sc.textFile("s3://bigdatademo/sample/wiki/") I get: WARN storage.BlockManager: Putting block broadcast_1 failed java.lang.NoSuchMet

Re: Problem running Spark shell (1.0.0) on EMR

2014-07-22 Thread Martin Goodson
I am also having exactly the same problem, calling using pyspark. Has anyone managed to get this script to work? -- Martin Goodson | VP Data Science (0)20 3397 1240 [image: Inline image 1] On Wed, Jul 16, 2014 at 2:10 PM, Ian Wilkinson wrote: > Hi, > > I’m trying to run the Spark (1.0.0) s

Problem running Spark shell (1.0.0) on EMR

2014-07-16 Thread Ian Wilkinson
Hi, I’m trying to run the Spark (1.0.0) shell on EMR and encountering a classpath issue. I suspect I’m missing something gloriously obviously, but so far it is eluding me. I launch the EMR Cluster (using the aws cli) with: aws emr create-cluster --name "Test Cluster" \ --ami-version 3