[
https://issues.apache.org/jira/browse/SYSTEMML-540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866412#comment-15866412
]
Xin Wu commented on SYSTEMML-540:
---------------------------------
[~mwdus...@us.ibm.com] Is this Epic for post-0.13 version? Thanks!
> Deep Learning
> -------------
>
> Key: SYSTEMML-540
> URL: https://issues.apache.org/jira/browse/SYSTEMML-540
> Project: SystemML
> Issue Type: Epic
> Reporter: Mike Dusenberry
> Assignee: Mike Dusenberry
>
> This epic covers the addition of deep learning to SystemML, including:
> * Core DML layer abstractions for deep (convolutional, recurrent) neural
> nets, with simple forward/backward API: affine, convolution (start with 2D),
> max-pooling, non-linearities (relu, sigmoid, softmax), dropout, loss
> functions.
> * Modularized DML optimizers: (mini-batch, stochastic) gradient descent (w/
> momentum, etc.).
> * Additional DML language support as necessary (tensors, built-in functions
> such as convolution, function pointers, list structures, etc.).
> * Integration with other deep learning frameworks (Caffe, Torch, Theano,
> TensoFlow, etc.) via automatic DML code generation.
> * etc.
> ---
> *Plan*:
> \[*DONE*\] Phase 1: *MVPs*
> * Create mathematically correct DML deep learning library for running basic
> feed-forward and convolutional neural nets on a singlenode.
> * Create mathematically correct built-in operators for convolution and max
> pooling for singlenode operation.
> \[*CURRENT*\] Phase 2: *Singlenode*
> * Improve performance of DML deep learning library in singlenode operation.
> * Expand DML deep learning library to include additional commonly-used
> layers, such as RNNs and LSTMs, as well as additional optimizers.
> * Improve built-in operators for convolution and max pooling to be highly
> performant in singlenode operation.
> * Implement performant GPU acceleration for built-in operators (and
> end-to-end deep learning algorithms) in singlenode operation.
> * Add general engine improvements to improve bottlenecks, such as
> left-indexing within DML-bodied functions.
> * Add end-to-end deep learning algorithm examples, such as a "LeNet"
> convolutional neural net.
> Phase 3: *Distributed*
> * Expand deep learning support to include *distributed operations* with large
> models. This includes improvements to the DML deep learning library, the
> built-in operators, the GPU acceleration, and general engine improvements.
> Phase 4: *APIs/Wrappers*
> * Explore integration with Caffe, creating a SystemML interpreter for Caffe
> model definitions.
> * Explore integration with Keras, creating a SystemML backend for Keras.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
