My java knowledge is limited, but you may try with a hashmap and put RDDs
in it?
On Sun, Nov 1, 2015 at 4:34 AM, amit tewari <amittewar...@gmail.com> wrote:
> Thanks Ayan thats something similar to what I am looking at but trying the
> same in Java is giving compile error:
>
> JavaRDD<String> jRDD[] = new JavaRDD<String>[3];
>
> //Error: Cannot create a generic array of JavaRDD<String>
>
> Thanks
> Amit
>
>
>
> On Sat, Oct 31, 2015 at 5:46 PM, ayan guha <guha.a...@gmail.com> wrote:
>
>> Corrected a typo...
>>
>> # In Driver
>> fileList=["/file1.txt","/file2.txt"]
>> rdds = []
>> for f in fileList:
>> rdd = jsc.textFile(f)
>> rdds.append(rdd)
>>
>>
>> On Sat, Oct 31, 2015 at 11:14 PM, ayan guha <guha.a...@gmail.com> wrote:
>>
>>> Yes, this can be done. quick python equivalent:
>>>
>>> # In Driver
>>> fileList=["/file1.txt","/file2.txt"]
>>> rdd = []
>>> for f in fileList:
>>> rdd = jsc.textFile(f)
>>> rdds.append(rdd)
>>>
>>>
>>>
>>> On Sat, Oct 31, 2015 at 11:09 PM, amit tewari <amittewar...@gmail.com>
>>> wrote:
>>>
>>>> Hi
>>>>
>>>> I need the ability to be able to create RDDs programatically inside my
>>>> program (e.g. based on varaible number of input files).
>>>>
>>>> Can this be done?
>>>>
>>>> I need this as I want to run the following statement inside an
>>>> iteration:
>>>>
>>>> JavaRDD<String> rdd1 = jsc.textFile("/file1.txt");
>>>>
>>>> Thanks
>>>> Amit
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Ayan Guha
>>>
>>
>>
>>
>> --
>> Best Regards,
>> Ayan Guha
>>
>
>
--
Best Regards,
Ayan Guha
