Spark permission denied error when invoking saveAsTextFile

Running a simple word count job in standalone mode as a non root user from spark-shell. The spark master, worker services are running as root user. The problem is the _temporary under /user/krajah/output2/_temporary/0 dir is being created with root permission even when running the job as non root

RE: --driver-memory parameter doesn't work for spark-submmit on yarn?

Shuai, What did ps aux | grep spark-submit reveal? When you compare using _JAVA_OPTIONS and without using it, where do you see the difference? Thanks Christopher -Original Message- From: Shuai Zheng [mailto:szheng.c...@gmail.com] Sent: Wednesday, April 01, 2015 11:12 AM To: 'Sean

Re: Spark on EC2

You're probably requesting more instances than allowed by your account, so the error gets generated for the extra instances. Try launching a smaller cluster. On Wed, Apr 1, 2015 at 12:41 PM, Vadim Bichutskiy vadim.bichuts...@gmail.com wrote: Hi all, I just tried launching a Spark cluster on

Re: Unable to run Spark application

You could try to run the 'make-distribution.sh' script with the proper options for your case. The script is in the extracted spark-1.2.1 folder.

Quick GraphX gutcheck

Hi Everyone, Quick (hopefully) and silly (likely) question--the VertexId can be used to join the VertexRDD generated from Graph.vertices with a transformed RDD where the keys are vertexIds from the original graph, correct? --John -- View this message in context:

Spark SQL does not read from cached table if table is renamed

Hi all, Noticed a bug in my current version of Spark 1.2.1. After a table is cached with cache table table command, query will not read from memory if SQL query renames the table. This query reads from in memory table i.e. select hivesampletable.country from default.hivesampletable group by

<    1   2