It may involve access an element of an RDD from a remote machine and copying it back to the driver. That and the small overhead of job scheduling could be a millisecond.
You're comparing to just reading an entry from memory, which is of course faster. I don't think you should think of an RDD as something you query at scale in real-time. It's not a NoSQL store. On Mon, Feb 23, 2015 at 6:02 PM, shahab <shahab.mok...@gmail.com> wrote: > Hi, > > I just wonder what would be the access time to "take" one element from a > cached RDD? if I have understood correctly, access to RDD elements is not as > fast as accessing e.g. HashMap and it could take up to mili seconds compare > to nano seconds in HashMap, which is quite significant difference if you > plan for near real-time response from Spark ?! > > best, > > /Shahab > > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
