Re: Access time to an elemnt in cached RDD

2015-02-23 Thread Sean Owen
It may involve access an element of an RDD from a remote machine and copying it back to the driver. That and the small overhead of job scheduling could be a millisecond. You're comparing to just reading an entry from memory, which is of course faster. I don't think you should think of an RDD as

Access time to an elemnt in cached RDD

2015-02-23 Thread shahab
Hi, I just wonder what would be the access time to take one element from a cached RDD? if I have understood correctly, access to RDD elements is not as fast as accessing e.g. HashMap and it could take up to mili seconds compare to nano seconds in HashMap, which is quite significant difference if