Hi Koert,
Could you provide more details? Job arguments, log messages, errors, etc.
On Fri, Jun 20, 2014 at 9:40 AM, Koert Kuipers ko...@tresata.com wrote:
i noticed that when i submit a job to yarn it mistakenly tries to upload
files to local filesystem instead of hdfs. what could cause this?
yeah sure see below. i strongly suspect its something i misconfigured
causing yarn to try to use local filesystem mistakenly.
*
[koert@cdh5-yarn ~]$ /usr/local/lib/spark/bin/spark-submit --class
org.apache.spark.examples.SparkPi --master yarn-cluster --num-executors 3
Koert, is there any chance that your fs.defaultFS isn't setup right?
On Fri, Jun 20, 2014 at 9:57 AM, Koert Kuipers ko...@tresata.com wrote:
yeah sure see below. i strongly suspect its something i misconfigured
causing yarn to try to use local filesystem mistakenly.
*
ok solved it. as it happened in spark/conf i also had a file called
core.site.xml (with some tachyone related stuff in it) so thats why it
ignored /etc/hadoop/conf/core-site.xml
On Fri, Jun 20, 2014 at 3:24 PM, Koert Kuipers ko...@tresata.com wrote:
i put some logging statements in
