Hi Everyone, Any insights on this thread? Thank you.
On Friday, May 27, 2016, Ajay Chander <itsche...@gmail.com> wrote:
> Hi Everyone,
>
> I have some data located on the EdgeNode. Right
> now, the process I follow to copy the data from Edgenode to HDFS is through
> a shellscript which resides on Edgenode. In Oozie I am using a SSH action
> to execute the shell script on Edgenode which copies the data to HDFS.
>
> I was just wondering if there is any built in
> API with in Spark to do this job. I want to read the data from Edgenode
> into RDD using JavaSparkContext then do saveAsTextFile("hdfs://...").
> JavaSparkContext does provide any method to pass Edgenode's access
> credentials and read the data into an RDD ??
>
> Thank you for your valuable time. Any pointers are appreciated.
>
> Thank You,
> Aj
>
