wierd caching

2014-11-08 Thread Nathan Kronenfeld
RDD NameStorage LevelCached PartitionsFraction CachedSize in MemorySize in TachyonSize on Disk 8 http://hadoop-s1.oculus.guest:4042/storage/rdd?id=8 Memory Deserialized 1x Replicated 426 107% 59.7 GB 0.0 B 0.0 BAnyone understand what it means to have more than 100% of an rdd cached? Thanks,

Re: wierd caching

2014-11-08 Thread Matei Zaharia
It might mean that some partition was computed on two nodes, because a task for it wasn't able to be scheduled locally on the first node. Did the RDD really have 426 partitions total? You can click on it and see where there are copies of each one. Matei On Nov 8, 2014, at 10:16 PM, Nathan