Hello,
I use Yarn client mode to submit my driver program to Hadoop, the dataset I
load is from the local file system, when i invoke load("file://path") Spark
complains about the csv file being not found, which i totally understand,
since the dataset is not in any of the workers or the applicationMaster but
only where the driver program resides.
I tried to share the file using the configurations:
> *spark.yarn.dist.files* OR *spark.files *
but both ain't working.
My question is how to share the csv dataset across the nodes at the
specified path?
Thanks.
