hi, spark-developers, i am using shark/spark, and i am puzzled by such
question, and can not find any info from the web, so i ask you.
1. how spark partition data in memory when creating table when using
"create table a tblproperties("shark.cache"="memory") as select * from
table b " , in another words, how many rdds will be created ? how spark
decide the number of rdds ?
2. how spark partition data on tachyon when creating table when using
"create table a tblproperties("shark.cache"="tachyon") as select * from
table b ". in another words, how many files will be created ? how spark
decide the number of files?
i found this settings about tachyon "tachyon.user.default.block.size.byte"
, what it means? could i set it to control each file size ?
thanks for any guiding .
