Workload for spark testing

2014-09-13 Thread 牛兆捷
Hi All: We know some memory of spark are used for computing (e.g., spark.shuffle.memoryFraction) and some are used for caching RDD for future use (e.g., spark.storage.memoryFraction). Is there any existing workload which can utilize both of them during the running left cycle? I want to do some pe

workload for spark

2014-09-12 Thread 牛兆捷
We know some memory of spark are used for computing (e.g., shuffle buffer) and some are used for caching RDD for future use. Is there any existing workload which utilize both of them? I want to do some performance study by adjusting the ratio between them.