No, I am thinking along lines of writing to an accelerator card or dedicated card with its own memory.
Regards, Mridul On Apr 6, 2014 5:19 AM, "Haoyuan Li" <haoyuan...@gmail.com> wrote: > Hi Mridul, > > Do you mean the scenario that different Spark applications need to read the > same raw data, which is stored in a remote cluster or machines. And the > goal is to load the remote raw data only once? > > Haoyuan > > > On Sat, Apr 5, 2014 at 4:30 PM, Mridul Muralidharan <mri...@gmail.com > >wrote: > > > Hi, > > > > We have a requirement to use a (potential) ephemeral storage, which > > is not within the VM, which is strongly tied to a worker node. So > > source of truth for a block would still be within spark; but to > > actually do computation, we would need to copy data to external device > > (where it might lie around for a while : so data locality really > > really helps if we can avoid a subsequent copy if it is already > > present on computations on same block again). > > > > I was wondering if the recently added storage level for tachyon would > > help in this case (note, tachyon wont help; just the storage level > > might). > > What sort of guarantees does it provide ? How extensible is it ? Or is > > it strongly tied to tachyon with only a generic name ? > > > > > > Thanks, > > Mridul > > > > > > -- > Haoyuan Li > Algorithms, Machines, People Lab, EECS, UC Berkeley > http://www.cs.berkeley.edu/~haoyuan/ >
