Ewan,
What issue are you having with HDFS when only Spark is installed? I'm not aware
of any issue like this.
Thanks,
Jonathan
—
Sent from Mailbox
On Wed, Sep 9, 2015 at 11:48 PM, Ewan Leith
wrote:
> The last time I checked, if you launch EMR 4 with only Spark selected as an
> app
Thank you all for the comments, but my problem still exists.
@Dean,@Ewan yes, I do have hadoop file system installed and working
@Sujit: the last version of EMR (version 4) does not need manual copying
of jar file to the server. The blog that you pointed out refers to older
version (3.x) of EMR.
The last time I checked, if you launch EMR 4 with only Spark selected as an
application, HDFS isn't correctly installed.
Did you select another application like Hive at launch time as well as Spark?
If not, try that.
Thanks,
Ewan
-- Original message--
From: Dean Wampler
Date: Wed
If you log into the cluster, do you see the file if you type:
hdfs dfs
-ls
hdfs://ipx-x-x-x:8020/user/hadoop/.sparkStaging/application_123344567_0018/spark-assembly-1.4.1-hadoop2.6.0-amzn-0.jar
(with the correct server address for "ipx-x-x-x"). If not, is the server
address correct and routable
