Hi Ted Will you be able to guide me for any sample code for achieving this?
On 17 April 2015 at 02:30, Jeetendra Gangele <gangele...@gmail.com> wrote: > Can you please guide me how can I extend RDD and convert into this way you > are suggesting. > > On 16 April 2015 at 23:46, Jeetendra Gangele <gangele...@gmail.com> wrote: > >> I type T i already have Object ... I have RDD<Object> and then I am >> calling ZipWithIndex on this RDD and getting RDD<Object,Long> on this I am >> running MapToPair and converting into RDD<Long,Object> so that i can use it >> later for other operation like lookup and join. >> >> >> On 16 April 2015 at 23:42, Ted Yu <yuzhih...@gmail.com> wrote: >> >>> The Long in RDD[(T, Long)] is type parameter. You can create RDD with >>> Integer as the first type parameter. >>> >>> Cheers >>> >>> On Thu, Apr 16, 2015 at 11:07 AM, Jeetendra Gangele < >>> gangele...@gmail.com> wrote: >>> >>>> Hi Ted. >>>> This works for me. But since Long takes here 8 bytes. Can I reduce it >>>> to 4 bytes. its just a index and I feel 4 bytes was more than enough.is >>>> there any method which takes Integer or similar for Index? >>>> >>>> >>>> On 13 April 2015 at 01:59, Ted Yu <yuzhih...@gmail.com> wrote: >>>> >>>>> bq. will return something like JavaPairRDD<Object, long> >>>>> >>>>> The long component of the pair fits your description of index. What >>>>> other requirement does ZipWithIndex not provide you ? >>>>> >>>>> Cheers >>>>> >>>>> On Sun, Apr 12, 2015 at 1:16 PM, Jeetendra Gangele < >>>>> gangele...@gmail.com> wrote: >>>>> >>>>>> Hi All I have an RDD JavaRDD<Object> and I want to convert it to >>>>>> JavaPairRDD<Index,Object>.. Index should be unique and it should maintain >>>>>> the order. For first object It should have 1 and then for second 2 like >>>>>> that. >>>>>> >>>>>> I tried using ZipWithIndex but it will return something like >>>>>> JavaPairRDD<Object, long> >>>>>> I wanted to use this RDD for lookup and join operation later in my >>>>>> workflow so ordering is important. >>>>>> >>>>>> >>>>>> Regards >>>>>> jeet >>>>>> >>>>> >>>>> >>>> >>>> >>>> >>> >> >> >> > > > >
