Hi, I want to benchmark the memory savings by using the in-memory columnar storage for schemardds (using cacheTable) vs caching the SchemaRDD directly. It would be really helpful to be able to query this from the spark-shell or jobs directly. Could a dev point me to the way to do this? From what I understand i will need a reference to the block manager, or something like RDDInfo.fromRdd(rdd).memSize.
I could use reflection or whatever to override the private access modifiers. -- View this message in context: http://apache-spark-developers-list.1001551.n3.nabble.com/Get-size-of-rdd-in-memory-tp10366.html Sent from the Apache Spark Developers List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@spark.apache.org For additional commands, e-mail: dev-h...@spark.apache.org
