Hi Everyone,
I have some data located on the EdgeNode. Right
now, the process I follow to copy the data from Edgenode to HDFS is through
a shellscript which resides on Edgenode. In Oozie I am using a SSH action
to execute the shell script on Edgenode which copies the data to HDFS.
I was just wondering if there is any built in API
with in Spark to do this job. I want to read the data from Edgenode into
RDD using JavaSparkContext then do saveAsTextFile("hdfs://...").
JavaSparkContext does provide any method to pass Edgenode's access
credentials and read the data into an RDD ??
Thank you for your valuable time. Any pointers are appreciated.
Thank You,
Aj
