Thanks Guys !
On Thu, Apr 24, 2014 at 11:29 AM, Sourav Chandra < sourav.chan...@livestream.com> wrote: > Also same thing can be done using rdd.top(1)(reverseOrdering) > > > > On Thu, Apr 24, 2014 at 11:28 AM, Sourav Chandra < > sourav.chan...@livestream.com> wrote: > >> You can use rdd.takeOrdered(1)(reverseOrdrering) >> >> reverseOrdering is you Ordering[T] instance where you define the ordering >> logic. This you have to pass in the method >> >> >> >> On Thu, Apr 24, 2014 at 11:21 AM, Frank Austin Nothaft < >> fnoth...@berkeley.edu> wrote: >> >>> If you do this, you could simplify to: >>> >>> RDD.collect().last >>> >>> However, this has the problem of collecting all data to the driver. >>> >>> Is your data sorted? If so, you could reverse the sort and take the >>> first. Alternatively, a hackey implementation might involve a >>> mapPartitionsWithIndex that returns an empty iterator for all partitions >>> except for the last. For the last partition, you would filter all elements >>> except for the last element in your iterator. This should leave one >>> element, which is your last element. >>> >>> Frank Austin Nothaft >>> fnoth...@berkeley.edu >>> fnoth...@eecs.berkeley.edu >>> 202-340-0466 >>> >>> On Apr 23, 2014, at 10:44 PM, Adnan Yaqoob <nsyaq...@gmail.com> wrote: >>> >>> This function will return scala List, you can use List's last function >>> to get the last element. >>> >>> For example: >>> >>> RDD.take(RDD.count()).last >>> >>> >>> On Thu, Apr 24, 2014 at 10:28 AM, Sai Prasanna >>> <ansaiprasa...@gmail.com>wrote: >>> >>>> Adnan, but RDD.take(RDD.count()) returns all the elements of the RDD. >>>> >>>> I want only to access the last element. >>>> >>>> >>>> On Thu, Apr 24, 2014 at 10:33 AM, Sai Prasanna <ansaiprasa...@gmail.com >>>> > wrote: >>>> >>>>> Oh ya, Thanks Adnan. >>>>> >>>>> >>>>> On Thu, Apr 24, 2014 at 10:30 AM, Adnan Yaqoob <nsyaq...@gmail.com>wrote: >>>>> >>>>>> You can use following code: >>>>>> >>>>>> RDD.take(RDD.count()) >>>>>> >>>>>> >>>>>> On Thu, Apr 24, 2014 at 9:51 AM, Sai Prasanna < >>>>>> ansaiprasa...@gmail.com> wrote: >>>>>> >>>>>>> Hi All, Some help ! >>>>>>> RDD.first or RDD.take(1) gives the first item, is there a straight >>>>>>> forward way to access the last element in a similar way ? >>>>>>> >>>>>>> I coudnt fine a tail/last method for RDD. !! >>>>>>> >>>>>> >>>>>> >>>>> >>>> >>> >>> >> >> >> -- >> >> Sourav Chandra >> >> Senior Software Engineer >> >> · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · >> >> sourav.chan...@livestream.com >> >> o: +91 80 4121 8723 >> >> m: +91 988 699 3746 >> >> skype: sourav.chandra >> >> Livestream >> >> "Ajmera Summit", First Floor, #3/D, 68 Ward, 3rd Cross, 7th C Main, 3rd >> Block, Koramangala Industrial Area, >> >> Bangalore 560034 >> >> www.livestream.com >> > > > > -- > > Sourav Chandra > > Senior Software Engineer > > · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · > > sourav.chan...@livestream.com > > o: +91 80 4121 8723 > > m: +91 988 699 3746 > > skype: sourav.chandra > > Livestream > > "Ajmera Summit", First Floor, #3/D, 68 Ward, 3rd Cross, 7th C Main, 3rd > Block, Koramangala Industrial Area, > > Bangalore 560034 > > www.livestream.com >
