Connection to Presto via Spark

Hi, I am trying to connect to Presto via Spark shell using the following connection string, however ending up with exception *-bash-4.2$ spark-shell --driver-class-path com.facebook.presto.jdbc.PrestoDriver --jars presto-jdbc-0.221.jar* *scala> val presto_df =

Re: Running Spark on Yarn

RM NM logs traced below, RM --> 2016-03-30 14:59:15,498 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Setting up container Container: [ContainerId: container_1459326455972_0004_01_01, NodeId: myhost:60653, NodeHttpAddress: myhost:8042, Resource:

Re: Running Spark on Yarn

Report took 4ms On Wed, Mar 30, 2016 at 3:26 AM, Vineet Mishra <clearmido...@gmail.com> wrote: > Looks like still the same while the other MR application is working fine, > > > > On Wed, Mar 30, 2016 at 3:15 AM, Alexander Pivovarov <apivova...@gmail.com > > wrote: >

Re: Running Spark on Yarn

ng > applications. > > > > > Probably yarn can not allocate mem for AM container. dafault value is 0.1 > and Spark AM need 896 MB (and 0.1 gives just 393 MB which is not enough) > > On Tue, Mar 29, 2016 at 2:35 PM, Vineet Mishra <clearmido...@g

Re: Running Spark on Yarn

pps Completed is 0 on the UI* > > > On Tue, Mar 29, 2016 at 2:13 PM, Vineet Mishra <clearmido...@gmail.com> > wrote: > >> Hi Alex/Surendra, >> >> Hadoop is up and running fine and I am able to run example on the same. >> >> *Cluster Metrics* >> *Ap

Re: Running Spark on Yarn

nge accordingly. >> >> Regards, >> Surendra M >> >> -- Surendra Manchikanti >> >> On Tue, Mar 29, 2016 at 1:15 PM, Vineet Mishra <clearmido...@gmail.com> >> wrote: >> >>> Hi All, >>> >>> While starting Spark on Yarn o

Running Spark on Yarn

Hi All, While starting Spark on Yarn on local cluster(Single Node Hadoop 2.6 yarn) I am facing some issues. As I try to start the Spark Shell it keeps on iterating in a endless loop while initiating, *6/03/30 01:32:38 DEBUG ipc.Client: IPC Client (1782965120) connection to

Re: Running Spark on Gateway - Connecting to Resource Manager Retries

yarn service is running on 8032. Thanks Best Regards On Tue, Apr 14, 2015 at 12:35 PM, Vineet Mishra clearmido...@gmail.com wrote: Hi Team, I am running Spark Word Count example( https://github.com/sryza/simplesparkapp), if I go with master as local it works fine. But when I change

Running Spark on Gateway - Connecting to Resource Manager Retries

Hi Team, I am running Spark Word Count example( https://github.com/sryza/simplesparkapp), if I go with master as local it works fine. But when I change the master to yarn its end with retries connecting to resource manager(stack trace mentioned below), 15/04/14 11:31:57 INFO RMProxy: Connecting