Re: Some BlockManager Doubts

2015-07-09 Thread Shixiong Zhu
MEMORY_AND_DISK will use disk if there is no enough memory. If there is no enough memory when putting a MEMORY_AND_DISK block, BlockManager will store it to disk. And if a MEMORY_AND_DISK block is dropped from memory, MemoryStore will call BlockManager.dropFromMemory to store it to disk, see

Some BlockManager Doubts

2015-07-09 Thread Dibyendu Bhattacharya
Hi , Just would like to clarify few doubts I have how BlockManager behaves . This is mostly in regards to Spark Streaming Context . There are two possible cases Blocks may get dropped / not stored in memory Case 1. While writing the Block for MEMORY_ONLY settings , if Node's BlockManager does