Sure thanks. if you can guide me how to do this will be great help. On 17 April 2015 at 22:05, Ted Yu <yuzhih...@gmail.com> wrote:
> I have some assignments on hand at the moment. > > Will try to come up with sample code after I clear the assignments. > > FYI > > On Thu, Apr 16, 2015 at 2:00 PM, Jeetendra Gangele <gangele...@gmail.com> > wrote: > >> Can you please guide me how can I extend RDD and convert into this way >> you are suggesting. >> >> On 16 April 2015 at 23:46, Jeetendra Gangele <gangele...@gmail.com> >> wrote: >> >>> I type T i already have Object ... I have RDD<Object> and then I am >>> calling ZipWithIndex on this RDD and getting RDD<Object,Long> on this I am >>> running MapToPair and converting into RDD<Long,Object> so that i can use it >>> later for other operation like lookup and join. >>> >>> >>> On 16 April 2015 at 23:42, Ted Yu <yuzhih...@gmail.com> wrote: >>> >>>> The Long in RDD[(T, Long)] is type parameter. You can create RDD with >>>> Integer as the first type parameter. >>>> >>>> Cheers >>>> >>>> On Thu, Apr 16, 2015 at 11:07 AM, Jeetendra Gangele < >>>> gangele...@gmail.com> wrote: >>>> >>>>> Hi Ted. >>>>> This works for me. But since Long takes here 8 bytes. Can I reduce it >>>>> to 4 bytes. its just a index and I feel 4 bytes was more than >>>>> enough.is there any method which takes Integer or similar for Index? >>>>> >>>>> >>>>> On 13 April 2015 at 01:59, Ted Yu <yuzhih...@gmail.com> wrote: >>>>> >>>>>> bq. will return something like JavaPairRDD<Object, long> >>>>>> >>>>>> The long component of the pair fits your description of index. What >>>>>> other requirement does ZipWithIndex not provide you ? >>>>>> >>>>>> Cheers >>>>>> >>>>>> On Sun, Apr 12, 2015 at 1:16 PM, Jeetendra Gangele < >>>>>> gangele...@gmail.com> wrote: >>>>>> >>>>>>> Hi All I have an RDD JavaRDD<Object> and I want to convert it to >>>>>>> JavaPairRDD<Index,Object>.. Index should be unique and it should >>>>>>> maintain >>>>>>> the order. For first object It should have 1 and then for second 2 like >>>>>>> that. >>>>>>> >>>>>>> I tried using ZipWithIndex but it will return something like >>>>>>> JavaPairRDD<Object, long> >>>>>>> I wanted to use this RDD for lookup and join operation later in my >>>>>>> workflow so ordering is important. >>>>>>> >>>>>>> >>>>>>> Regards >>>>>>> jeet >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> >>>> >>> >>> >>> >> >> >> >> >
