I thought so but doesn’t that read that into the driver? I need to keep piping it into other RDDs.
I have a huge table as the input and I need to do multiple transformations on the data so I just want to read the first N rows from that as an RDD and then keep doing my transformations. On Wed, Dec 31, 2014 at 7:09 PM, Ganelin, Ilya <ilya.gane...@capitalone.com> wrote: > You want to use take() or takeOrdered. > > > > Sent with Good (www.good.com) > > > > -----Original Message----- > *From: *Kevin Burton [bur...@spinn3r.com] > *Sent: *Wednesday, December 31, 2014 10:02 PM Eastern Standard Time > *To: *u...@spark.apache.org > *Subject: *limit vs sample for indexing a small amount of data quickly? > > Is there a limit function which just returns the first N records? > > Sample is nice but I’m trying to do this so it’s super fast and just to > test the functionality of an algorithm. > > With sample I’d have to compute the % that would yield 1000 results first… > > Kevin > > -- > > Founder/CEO Spinn3r.com > Location: *San Francisco, CA* > blog: http://burtonator.wordpress.com > … or check out my Google+ profile > <https://plus.google.com/102718274791889610666/posts> > <http://spinn3r.com> > > ------------------------------ > > The information contained in this e-mail is confidential and/or > proprietary to Capital One and/or its affiliates. The information > transmitted herewith is intended only for use by the individual or entity > to which it is addressed. If the reader of this message is not the > intended recipient, you are hereby notified that any review, > retransmission, dissemination, distribution, copying or other use of, or > taking of any action in reliance upon this information is strictly > prohibited. If you have received this communication in error, please > contact the sender and delete the material from your computer. > -- Founder/CEO Spinn3r.com Location: *San Francisco, CA* blog: http://burtonator.wordpress.com … or check out my Google+ profile <https://plus.google.com/102718274791889610666/posts> <http://spinn3r.com>
