Thanks for sharing the doc. This is very much in line with the work we are doing since 6-7 years (https://github.com/zrlio, https://researcher.watson.ibm.com/researcher/view.php?person=zurich-stu). Please let us know if there is a specific aspect you would like to get involved in the context of Crail, ideally around the topics I proposed earlier. It would be great to get your help.
On Mon, Feb 19, 2018 at 1:37 AM, Bairen YI <[email protected]> wrote: > Hi Patrick, Yuval, > > I’m having a broader proposal to incorporate TensorFlow into Spark to enable > transparent e2e hardware acceleration of data flow pipeline. > > A zero copy distributed cache service such as Crail will play a central role > in this design. Without it, systems like Yahoo’s TensorFlow on Spark or > Databricks’ TensorFrame may incur large overheads transferring data across > system boundaries. > > Attached please find a deck of slides describing the central idea of such > design. >
