code hangs in local master mode

2016-01-14 Thread Kai Wei
from HDFS . I am using spark-1.5.2-bin-hadoop-2.6 on cloudera quickstart vm 5.4.2. -- Kai Wei Big Data Developer Pythian - love your data w...@pythian.com Tel: +1 613 565 8696 x1579 Mobile: +61 403 572 456 -- --

Re: code hangs in local master mode

2016-01-14 Thread Kai Wei
<yuzhih...@gmail.com> wrote: > Can you capture one or two stack traces of the local master process and > pastebin them ? > > Thanks > > On Thu, Jan 14, 2016 at 6:01 AM, Kai Wei <w...@pythian.com> wrote: > >> Hi list, >> >> I ran into an issue whic

Re: Packaging a jar for a jdbc connection using sbt assembly and scala.

2015-10-29 Thread Kai Wei
Submiting your app in client mode may help with your problem. -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Packaging-a-jar-for-a-jdbc-connection-using-sbt-assembly-and-scala-tp25225p25228.html Sent from the Apache Spark User List mailing list archive at

Re: No way to supply hive-site.xml in yarn client mode?

2015-10-29 Thread Kai Wei
hive-site.xml is being > picked up and I have no problem whatsoever. > > On Thu, Oct 29, 2015 at 10:48 AM, Zoltan Fedor <zoltan.0.fe...@gmail.com> > wrote: > >> Yes, I have the hive-site.xml in $SPARK_HOME/conf, also in yarn-conf, in >> /etc/hive/conf, etc >> >&

Re: No way to supply hive-site.xml in yarn client mode?

2015-10-29 Thread Kai Wei
Create /user/biapp in hdfs manually first. On 30 Oct 2015 1:36 am, "Zoltan Fedor" wrote: > Sure, I did it with spark-shell, which seems to be showing the same error > - not using the hive-site.xml > > > $ HADOOP_CONF_DIR=$SPARK_HOME/hadoop-conf >

Re: No way to supply hive-site.xml in yarn client mode?

2015-10-29 Thread Kai Wei
Did you try copy it to spark/conf dir? On 30 Oct 2015 1:42 am, "Zoltan Fedor" <zoltan.0.fe...@gmail.com> wrote: > There is /user/biapp in hdfs. The problem is that the hive-site.xml is > being ignored, so it is looking for it locally. > > On Thu, Oct 29, 2015 a