Hi

Is it possible to write custom RDD in java?

Requirement is - I am having a list of Sqlserver tables  need to be dumped
in HDFS.

So I have a
List<String> tables = {dbname.tablename,dbname.tablename2......};

then
JavaRDD<String> rdd = javasparkcontext.parllelise(tables);

JavaRDDString> tablecontent = rdd.map(new
Function<String,Iterable<String>>){fetch table and return populate iterable}

tablecontent.storeAsTextFile("hffs path");


In rdd.map(new Function<String,>). I cannot keep complete table content in
memory , so I want to creat my own RDD to handle it.

Thanks
Shushant

Reply via email to