This is the command I am running: spark-submit --deploy-mode cluster --master yarn --class com.myorg.myApp s3://my-bucket/myapp-0.1.jar
On Wed, Mar 1, 2017 at 12:22 AM, Jonathan Kelly <[email protected]> wrote: > Prithish, > > It would be helpful for you to share the spark-submit command you are > running. > > ~ Jonathan > > On Sun, Feb 26, 2017 at 8:29 AM Prithish <[email protected]> wrote: > >> Thanks for the responses, I am running this on Amazon EMR which runs the >> Yarn cluster manager. >> >> On Sat, Feb 25, 2017 at 4:45 PM, [email protected] < >> [email protected]> wrote: >> >> Hi, >> I think you are using the local model of Spark. There >> are mainly four models, which are local, standalone, yarn >> and Mesos. Also, "blocks" is relative to hdfs, "partitions" >> is relative to spark. >> >> liangyihuai >> >> ---Original--- >> *From:* "Jacek Laskowski "<[email protected]> >> *Date:* 2017/2/25 02:45:20 >> *To:* "prithish"<[email protected]>; >> *Cc:* "user"<[email protected]>; >> *Subject:* Re: RDD blocks on Spark Driver >> >> Hi, >> >> Guess you're use local mode which has only one executor called driver. Is >> my guessing correct? >> >> Jacek >> >> On 23 Feb 2017 2:03 a.m., <[email protected]> wrote: >> >> Hello, >> >> Had a question. When I look at the executors tab in Spark UI, I notice >> that some RDD blocks are assigned to the driver as well. Can someone please >> tell me why? >> >> Thanks for the help. >> >> >>
