Can you please guide me how can I extend RDD and convert into this way you are suggesting.
On 16 April 2015 at 23:46, Jeetendra Gangele <gangele...@gmail.com> wrote: > I type T i already have Object ... I have RDD<Object> and then I am > calling ZipWithIndex on this RDD and getting RDD<Object,Long> on this I am > running MapToPair and converting into RDD<Long,Object> so that i can use it > later for other operation like lookup and join. > > > On 16 April 2015 at 23:42, Ted Yu <yuzhih...@gmail.com> wrote: > >> The Long in RDD[(T, Long)] is type parameter. You can create RDD with >> Integer as the first type parameter. >> >> Cheers >> >> On Thu, Apr 16, 2015 at 11:07 AM, Jeetendra Gangele <gangele...@gmail.com >> > wrote: >> >>> Hi Ted. >>> This works for me. But since Long takes here 8 bytes. Can I reduce it to >>> 4 bytes. its just a index and I feel 4 bytes was more than enough.is >>> there any method which takes Integer or similar for Index? >>> >>> >>> On 13 April 2015 at 01:59, Ted Yu <yuzhih...@gmail.com> wrote: >>> >>>> bq. will return something like JavaPairRDD<Object, long> >>>> >>>> The long component of the pair fits your description of index. What >>>> other requirement does ZipWithIndex not provide you ? >>>> >>>> Cheers >>>> >>>> On Sun, Apr 12, 2015 at 1:16 PM, Jeetendra Gangele < >>>> gangele...@gmail.com> wrote: >>>> >>>>> Hi All I have an RDD JavaRDD<Object> and I want to convert it to >>>>> JavaPairRDD<Index,Object>.. Index should be unique and it should maintain >>>>> the order. For first object It should have 1 and then for second 2 like >>>>> that. >>>>> >>>>> I tried using ZipWithIndex but it will return something like >>>>> JavaPairRDD<Object, long> >>>>> I wanted to use this RDD for lookup and join operation later in my >>>>> workflow so ordering is important. >>>>> >>>>> >>>>> Regards >>>>> jeet >>>>> >>>> >>>> >>> >>> >>> >> > > >
