you'll need a filesystem with
* consistency
* accessibility everywhere
* supports a binding through one of the hadoop fs connectors
NFS-style distributed filesystems work with file:// ; things like glusterfs
need their own connectors.
you can use azure's wasb:// as a drop in replacement for HDF
chang your fs.defaultFS to point to local file system and have a try
On Wed, Jun 21, 2017 at 4:50 PM, Alaa Zubaidi (PDF)
wrote:
> Hi,
>
> Can we run Spark on YARN with out installing HDFS?
> If yes, where would HADOOP_CONF_DIR point to?
>
> Regards,
>
> *This message may contain confidential and
Hi,
Can we run Spark on YARN with out installing HDFS?
If yes, where would HADOOP_CONF_DIR point to?
Regards,
--
*This message may contain confidential and privileged information. If it
has been sent to you in error, please reply to advise the sender of the
error and then immediately permanen