spark.speculation was not set, any speculative execution on tachyon side?
tachyon-env.sh only changed following
export TACHYON_MASTER_ADDRESS=test01.zala
#export TACHYON_UNDERFS_ADDRESS=$TACHYON_HOME/underfs
export TACHYON_UNDERFS_ADDRESS=hdfs://test01.zala:8020
export
more interesting is if spark-shell started on master node (test01)
then
parquetFile.saveAsParquetFile(tachyon://test01.zala:19998/parquet_tablex)
14/08/12 11:42:06 INFO : initialize(tachyon://...
...
...
14/08/12 11:42:06 INFO : File does not exist:
sharing /reusing RDDs is always useful for many use cases, is this possible
via persisting RDD on tachyon?
such as off heap persist a named RDD into a given path (instead of
/tmp_spark_tachyon/spark-xxx-xxx-xxx)
or
saveAsParquetFile on tachyon
i tried to save a SchemaRDD on tachyon,
val
Is the speculative execution enabled?
Best,
Haoyuan
On Mon, Aug 11, 2014 at 8:08 AM, chutium teng@gmail.com wrote:
sharing /reusing RDDs is always useful for many use cases, is this possible
via persisting RDD on tachyon?
such as off heap persist a named RDD into a given path (instead
