Github user mateiz commented on the pull request:
https://github.com/apache/spark/pull/50#issuecomment-36483263
Hey Kyle, thanks for bringing this to the new repo. I looked through it and
made a few comments. Another concern though is that it would be good to make
this work for MEMORY_ONLY_SER storage as well, but note that that could be a
little trickier. In particular, for in-memory storage, the block store might
drop the block before you get to read it, which would then lead to an exception
here. It might be possible to take the return value of MemoryStore.put() and
pass that back from put() even if the memory store later decides to drop the
block, but you'll have to deal with replication and stuff like that. If you
want, we can open a separate issue for that and leave this for just the disk
case. But in that case you should add a comment to the code explaining why this
is only for disk.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
