Re: off heap to alluxio/tachyon in Spark 2

Hi, If you are looking for how to run Spark on Alluxio (formerly Tachyon), here is the documentation from Alluxio doc site: http://www.alluxio.org/docs/master/en/Running-Spark-on-Alluxio.html It still works for Spark 2.x. Alluxio team also published articles on when and why running Spark (2.x)

Re: off heap to alluxio/tachyon in Spark 2

It backed the "OFF_HEAP" storage level for RDDs. That's not quite the same thing that off-heap Tungsten allocation refers to. It's also worth pointing out that things like HDFS also can put data into memory already. On Mon, Sep 19, 2016 at 7:48 PM, Richard Catlin

Re: off heap to alluxio/tachyon in Spark 2

Here is my understanding. Spark used Tachyon as an off-heap solution for RDDs. In certain situations, it would alleviate Garbage Collection or the RDDs. Tungsten, Spark 2’s off-heap (columnar format) is much more efficient and used as the default. Alluvio no longer makes sense for this use.

off heap to alluxio/tachyon in Spark 2

Hi folks, What has happened with Tachyon / Alluxio in Spark 2? Doc doesn't mention it no longer. -- Oleksiy Dyagilev