Here is my understanding. Spark used Tachyon as an off-heap solution for RDDs. In certain situations, it would alleviate Garbage Collection or the RDDs.
Tungsten, Spark 2’s off-heap (columnar format) is much more efficient and used as the default. Alluvio no longer makes sense for this use. You can still use Tachyon/Alluxio to bring your files into Memory, which is quicker for Spark to access than your DFS(HDFS or S3). Alluxio actually supports a “Tiered Filesystem”, and automatically brings the “hotter” files into the fastest storage (Memory, SSD). You can configure it with Memory, SSD, and/or HDDs with the DFS as the persistent store, called under-filesystem. Hope this helps. Richard Catlin > On Sep 19, 2016, at 7:56 AM, aka.fe2s <aka.f...@gmail.com> wrote: > > Hi folks, > > What has happened with Tachyon / Alluxio in Spark 2? Doc doesn't mention it > no longer. > > -- > Oleksiy Dyagilev
