Thanks Ayan thats something similar to what I am looking at but trying the
same in Java is giving compile error:
JavaRDD<String> jRDD[] = new JavaRDD<String>[3];
//Error: Cannot create a generic array of JavaRDD<String>
Thanks
Amit
On Sat, Oct 31, 2015 at 5:46 PM, ayan guha <guha.a...@gmail.com> wrote:
> Corrected a typo...
>
> # In Driver
> fileList=["/file1.txt","/file2.txt"]
> rdds = []
> for f in fileList:
> rdd = jsc.textFile(f)
> rdds.append(rdd)
>
>
> On Sat, Oct 31, 2015 at 11:14 PM, ayan guha <guha.a...@gmail.com> wrote:
>
>> Yes, this can be done. quick python equivalent:
>>
>> # In Driver
>> fileList=["/file1.txt","/file2.txt"]
>> rdd = []
>> for f in fileList:
>> rdd = jsc.textFile(f)
>> rdds.append(rdd)
>>
>>
>>
>> On Sat, Oct 31, 2015 at 11:09 PM, amit tewari <amittewar...@gmail.com>
>> wrote:
>>
>>> Hi
>>>
>>> I need the ability to be able to create RDDs programatically inside my
>>> program (e.g. based on varaible number of input files).
>>>
>>> Can this be done?
>>>
>>> I need this as I want to run the following statement inside an iteration:
>>>
>>> JavaRDD<String> rdd1 = jsc.textFile("/file1.txt");
>>>
>>> Thanks
>>> Amit
>>>
>>
>>
>>
>> --
>> Best Regards,
>> Ayan Guha
>>
>
>
>
> --
> Best Regards,
> Ayan Guha
>
