wierd caching

2014-11-08 Thread Nathan Kronenfeld
RDD NameStorage LevelCached PartitionsFraction CachedSize in MemorySize in
TachyonSize on Disk   8
http://hadoop-s1.oculus.guest:4042/storage/rdd?id=8 Memory Deserialized
1x Replicated 426 107% 59.7 GB 0.0 B 0.0 BAnyone understand what it means
to have more than 100% of an rdd cached?

Thanks,
-Nathan


Re: wierd caching

2014-11-08 Thread Matei Zaharia
It might mean that some partition was computed on two nodes, because a task for 
it wasn't able to be scheduled locally on the first node. Did the RDD really 
have 426 partitions total? You can click on it and see where there are copies 
of each one.

Matei

 On Nov 8, 2014, at 10:16 PM, Nathan Kronenfeld nkronenf...@oculusinfo.com 
 wrote:
 
 RDD Name  Storage Level   Cached Partitions   Fraction Cached Size in 
 Memory  Size in Tachyon Size on Disk
 8 http://hadoop-s1.oculus.guest:4042/storage/rdd?id=8   Memory 
 Deserialized 1x Replicated   426 107%59.7 GB 0.0 B   0.0 B
 Anyone understand what it means to have more than 100% of an rdd cached?
 
 Thanks,
 -Nathan