Andy Feng created SPARK-22658:
---------------------------------
Summary: SPIP: TeansorFlowOnSpark as a Scalable Deep Learning Lib
of Apache Spark
Key: SPARK-22658
URL: https://issues.apache.org/jira/browse/SPARK-22658
Project: Spark
Issue Type: New Feature
Components: ML
Affects Versions: 2.2.0
Reporter: Andy Feng
In Feburary 2017, TensorFlowOnSpark (TFoS) was released for distributed
TensorFlow training and inference on Apache Spark clusters. TFoS is designed to:
* Easily migrate all existing TensorFlow programs with minimum code change;
* Support all TensorFlow functionalities: synchronous/asynchronous training,
model/data parallelism, inference and TensorBoard;
* Easily integrate with your existing data processing pipelines (ex. Spark
SQL) and machine learning algorithms (ex. MLlib);
* Be easily deployed on cloud or on-premise: CPU & GPU, Ethernet and
Infiniband.
We propose to merge TFoS into Apache Spark as a scalable deep learning library
to:
* Make deep learning easy for Apache Spark community: Familiar pipeline API
for training and inference; Enable TensorFlow training/inference on existing
Spark clusters.
* Further simplify data scientist experience: Ensure compatibility b/w Apache
Spark and TFoS;
Reduce steps for installation.
* Help Apache Spark evolution on deep learning: Establish a design pattern for
additional frameworks (ex. Caffe, CNTK); Structured streaming for DL
training/inference.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
