I don't think the lineage thing is even turned on in Tachyon - it was mostly a research prototype, so I don't think it'd make sense for us to use that.
On Thu, Dec 11, 2014 at 3:51 PM, Andrew Ash <and...@andrewash.com> wrote: > I'm interested in understanding this as well. One of the main ways Tachyon > is supposed to realize performance gains without sacrificing durability is > by storing the lineage of data rather than full copies of it (similar to > Spark). But if Spark isn't sending lineage information into Tachyon, then > I'm not sure how this isn't a durability concern. > > On Wed, Dec 10, 2014 at 5:47 AM, Jun Feng Liu <liuj...@cn.ibm.com> wrote: > > > Dose Spark today really leverage Tachyon linage to process data? It seems > > like the application should call createDependency function in TachyonFS > > to create a new linage node. But I did not find any place call that in > > Spark code. Did I missed anything? > > > > Best Regards > > > > > > *Jun Feng Liu* > > IBM China Systems & Technology Laboratory in Beijing > > > > ------------------------------ > > [image: 2D barcode - encoded with contact information] *Phone: > *86-10-82452683 > > > > * E-mail:* *liuj...@cn.ibm.com* <liuj...@cn.ibm.com> > > [image: IBM] > > > > BLD 28,ZGC Software Park > > No.8 Rd.Dong Bei Wang West, Dist.Haidian Beijing 100193 > > China > > > > > > > > > > >
