how to cache table with OFF_HEAP storage level in SparkSQL thriftserver

2015-03-23 Thread LiuZeshan
hi all: I got a spark on yarn cluster (spark-1.3.0, hadoop-2.2.0) with hive-0.12.0 and tachyon-0.6.1, and now I start SparkSQL thriftserver with start-thriftserver.sh, and use beeline to connect to thriftserver according to spark document. My question is: how to cache table with specified

OFF_HEAP storage level

2014-07-04 Thread Ajay Srivastava
Hi, I was checking different storage level of an RDD and found OFF_HEAP. Has anybody used this level ? If i use this level, where will data be stored ? If not in heap, does it mean that we can avoid GC ? How can I use this level ? I did not find anything in archive regarding this. Can someone

RE: OFF_HEAP storage level

2014-07-04 Thread Shao, Saisai
04, 2014 2:19 PM To: user@spark.apache.org Subject: OFF_HEAP storage level Hi, I was checking different storage level of an RDD and found OFF_HEAP. Has anybody used this level ? If i use this level, where will data be stored ? If not in heap, does it mean that we can avoid GC ? How can I use