Prithish, It would be helpful for you to share the spark-submit command you are running.
~ Jonathan On Sun, Feb 26, 2017 at 8:29 AM Prithish <[email protected]> wrote: > Thanks for the responses, I am running this on Amazon EMR which runs the > Yarn cluster manager. > > On Sat, Feb 25, 2017 at 4:45 PM, [email protected] < > [email protected]> wrote: > > Hi, > > I think you are using the local model of Spark. There are mainly four > models, which are local, standalone, yarn and Mesos. Also, "blocks" is > relative to hdfs, "partitions" is relative to spark. > > liangyihuai > > ---Original--- > *From:* "Jacek Laskowski "<[email protected]> > *Date:* 2017/2/25 02:45:20 > *To:* "prithish"<[email protected]>; > *Cc:* "user"<[email protected]>; > *Subject:* Re: RDD blocks on Spark Driver > > Hi, > > Guess you're use local mode which has only one executor called driver. Is > my guessing correct? > > Jacek > > On 23 Feb 2017 2:03 a.m., <[email protected]> wrote: > > Hello, > > Had a question. When I look at the executors tab in Spark UI, I notice > that some RDD blocks are assigned to the driver as well. Can someone please > tell me why? > > Thanks for the help. > > >
