Re: trying to understand yarn-client mode

2015-08-05 Thread nir
.n3.nabble.com/trying-to-understand-yarn-client-mode-tp7925p24145.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e

Re: trying to understand yarn-client mode

2014-12-29 Thread Fernando Otero
were you able to fix the issue? I'm facing a similar issue when trying to use yarn client from spark-jobserver -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/trying-to-understand-yarn-client-mode-tp7925p20888.html Sent from the Apache Spark User List

Re: trying to understand yarn-client mode

2014-12-29 Thread firemonk9
I am able to fix it by adding the the jars(in the spark distribution) to the classpath. In my sbt file I changed the scope to provided. Let me know if you need more details. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/trying-to-understand-yarn-client

Re: trying to understand yarn-client mode

2014-06-20 Thread Koert Kuipers
, Koert Kuipers ko...@tresata.com wrote: i am trying to understand how yarn-client mode works. i am not using Application application_1403117970283_0014 failed 2 times due to AM Container for appattempt_1403117970283_0014_02 exited with exitCode: -1000 due to: File file:/home/koert/test

Re: trying to understand yarn-client mode

2014-06-20 Thread Marcelo Vanzin
On Fri, Jun 20, 2014 at 8:22 AM, Koert Kuipers ko...@tresata.com wrote: thanks! i will try that. i guess what i am most confused about is why the executors are trying to retrieve the jars directly using the info i provided to add jars to my spark context. i mean, thats bound to fail no? i

Re: trying to understand yarn-client mode

2014-06-19 Thread Kevin Markey
location by exporting its location as SPARK_JAR. Kevin Markey On 06/19/2014 11:22 AM, Koert Kuipers wrote: i am trying to understand how yarn-client mode works. i am not using spark-submit, but instead launching a spark job

Re: trying to understand yarn-client mode

2014-06-19 Thread DB Tsai
to the application on the cluster. Best to copy it to HDFS and specify its location by exporting its location as SPARK_JAR. Kevin Markey On 06/19/2014 11:22 AM, Koert Kuipers wrote: i am trying to understand how yarn-client mode works. i am not using spark-submit, but instead launching a spark job from

Re: trying to understand yarn-client mode

2014-06-19 Thread Marcelo Vanzin
that it should tell executors to download that file from the driver. If you specify the jar with the file: scheme that should solve the problem. On Thu, Jun 19, 2014 at 10:22 AM, Koert Kuipers ko...@tresata.com wrote: i am trying to understand how yarn-client mode works. i am not using

Re: trying to understand yarn-client mode

2014-06-19 Thread Koert Kuipers
it to HDFS and specify its location by exporting its location as SPARK_JAR. Kevin Markey On 06/19/2014 11:22 AM, Koert Kuipers wrote: i am trying to understand how yarn-client mode works. i am not using spark-submit, but instead launching a spark job from within my own application

Re: trying to understand yarn-client mode

2014-06-19 Thread DB Tsai
location as SPARK_JAR. Kevin Markey On 06/19/2014 11:22 AM, Koert Kuipers wrote: i am trying to understand how yarn-client mode works. i am not using spark-submit, but instead launching a spark job from within my own application. i can see my application contacting yarn successfully

Re: trying to understand yarn-client mode

2014-06-19 Thread Koert Kuipers
Markey On 06/19/2014 11:22 AM, Koert Kuipers wrote: i am trying to understand how yarn-client mode works. i am not using spark-submit, but instead launching a spark job from within my own application. i can see my application contacting yarn successfully, but then in yarn