It backed the "OFF_HEAP" storage level for RDDs. That's not quite the same thing that off-heap Tungsten allocation refers to.
It's also worth pointing out that things like HDFS also can put data into memory already. On Mon, Sep 19, 2016 at 7:48 PM, Richard Catlin <richard.m.cat...@gmail.com> wrote: > Here is my understanding. > > Spark used Tachyon as an off-heap solution for RDDs. In certain situations, > it would alleviate Garbage Collection or the RDDs. > > Tungsten, Spark 2’s off-heap (columnar format) is much more efficient and > used as the default. Alluvio no longer makes sense for this use. > > > You can still use Tachyon/Alluxio to bring your files into Memory, which is > quicker for Spark to access than your DFS(HDFS or S3). > > Alluxio actually supports a “Tiered Filesystem”, and automatically brings > the “hotter” files into the fastest storage (Memory, SSD). You can > configure it with Memory, SSD, and/or HDDs with the DFS as the persistent > store, called under-filesystem. > > Hope this helps. > > Richard Catlin > > On Sep 19, 2016, at 7:56 AM, aka.fe2s <aka.f...@gmail.com> wrote: > > Hi folks, > > What has happened with Tachyon / Alluxio in Spark 2? Doc doesn't mention it > no longer. > > -- > Oleksiy Dyagilev > > --------------------------------------------------------------------- To unsubscribe e-mail: user-unsubscr...@spark.apache.org
