[
https://issues.apache.org/jira/browse/SYSTEMML-540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Dusenberry updated SYSTEMML-540:
-------------------------------------
Description:
This epic covers the addition of deep learning to SystemML, including:
* Core DML layer abstractions for deep (convolutional, recurrent) neural nets,
with simple forward/backward API: affine, convolution (start with 2D),
max-pooling, non-linearities (relu, sigmoid, softmax), dropout, loss functions.
* Modularized DML optimizers: (mini-batch, stochastic) gradient descent (w/
momentum, etc.).
* Additional DML language support as necessary (tensors, built-in functions
such as convolution, function pointers, list structures, etc.).
* Integration with other deep learning frameworks (Caffe, Torch, Theano,
TensoFlow, etc.) via automatic DML code generation.
* etc.
\[*DONE*\] Phase 1: *MVPs*
* Create mathematically correct DML deep learning library for running basic
feed-forward and convolutional neural nets on a singlenode.
* Create mathematically correct built-in operators for convolution and max
pooling for singlenode operation.
\[*CURRENT*\] Phase 2: *Singlenode*
* Improve performance of DML deep learning library in singlenode operation.
* Expand DML deep learning library to include additional commonly-used layers,
such as RNNs and LSTMs, as well as additional optimizers.
* Improve built-in operators for convolution and max pooling to be highly
performant in singlenode operation.
* Implement performant GPU acceleration for built-in operators (and end-to-end
deep learning algorithms) in singlenode operation.
* Add general engine improvements to improve bottlenecks, such as left-indexing
within DML-bodied functions.
* Add end-to-end deep learning algorithm examples, such as a "LeNet"
convolutional neural net.
Phase 3: *Distributed*
* Expand deep learning support to include *distributed operations* with large
models. This includes improvements to the DML deep learning library, the
built-in operators, the GPU acceleration, and general engine improvements.
Phase 4: *APIs/Wrappers*
* Explore integration with Caffe, creating a SystemML interpreter for Caffe
model definitions.
* Explore integration with Keras, creating a SystemML backend for Keras.
was:
This epic covers the addition of deep learning to SystemML, including:
* Core DML layer abstractions for deep (convolutional, recurrent) neural nets,
with simple forward/backward API: affine, convolution (start with 2D),
max-pooling, non-linearities (relu, sigmoid, softmax), dropout, loss functions.
* Modularized DML optimizers: (mini-batch, stochastic) gradient descent (w/
momentum, etc.).
* Additional DML language support as necessary (tensors, built-in functions
such as convolution, function pointers, list structures, etc.).
* Integration with other deep learning frameworks (Caffe, Torch, Theano,
TensoFlow, etc.) via automatic DML code generation.
* etc.
[**DONE**] Phase 1: *MVPs*
* Create mathematically correct DML deep learning library for running basic
feed-forward and convolutional neural nets on a singlenode.
* Create mathematically correct built-in operators for convolution and max
pooling for singlenode operation.
[**CURRENT**] Phase 2: *Singlenode*
* Improve performance of DML deep learning library in singlenode operation.
* Expand DML deep learning library to include additional commonly-used layers,
such as RNNs and LSTMs, as well as additional optimizers.
* Improve built-in operators for convolution and max pooling to be highly
performant in singlenode operation.
* Implement performant GPU acceleration for built-in operators (and end-to-end
deep learning algorithms) in singlenode operation.
* Add general engine improvements to improve bottlenecks, such as left-indexing
within DML-bodied functions.
* Add end-to-end deep learning algorithm examples, such as a "LeNet"
convolutional neural net.
Phase 3: *Distributed*
* Expand deep learning support to include *distributed operations* with large
models. This includes improvements to the DML deep learning library, the
built-in operators, the GPU acceleration, and general engine improvements.
Phase 4: *APIs/Wrappers*
* Explore integration with Caffe, creating a SystemML interpreter for Caffe
model definitions.
* Explore integration with Keras, creating a SystemML backend for Keras.
> Deep Learning
> -------------
>
> Key: SYSTEMML-540
> URL: https://issues.apache.org/jira/browse/SYSTEMML-540
> Project: SystemML
> Issue Type: Epic
> Reporter: Mike Dusenberry
> Assignee: Mike Dusenberry
>
> This epic covers the addition of deep learning to SystemML, including:
> * Core DML layer abstractions for deep (convolutional, recurrent) neural
> nets, with simple forward/backward API: affine, convolution (start with 2D),
> max-pooling, non-linearities (relu, sigmoid, softmax), dropout, loss
> functions.
> * Modularized DML optimizers: (mini-batch, stochastic) gradient descent (w/
> momentum, etc.).
> * Additional DML language support as necessary (tensors, built-in functions
> such as convolution, function pointers, list structures, etc.).
> * Integration with other deep learning frameworks (Caffe, Torch, Theano,
> TensoFlow, etc.) via automatic DML code generation.
> * etc.
> \[*DONE*\] Phase 1: *MVPs*
> * Create mathematically correct DML deep learning library for running basic
> feed-forward and convolutional neural nets on a singlenode.
> * Create mathematically correct built-in operators for convolution and max
> pooling for singlenode operation.
> \[*CURRENT*\] Phase 2: *Singlenode*
> * Improve performance of DML deep learning library in singlenode operation.
> * Expand DML deep learning library to include additional commonly-used
> layers, such as RNNs and LSTMs, as well as additional optimizers.
> * Improve built-in operators for convolution and max pooling to be highly
> performant in singlenode operation.
> * Implement performant GPU acceleration for built-in operators (and
> end-to-end deep learning algorithms) in singlenode operation.
> * Add general engine improvements to improve bottlenecks, such as
> left-indexing within DML-bodied functions.
> * Add end-to-end deep learning algorithm examples, such as a "LeNet"
> convolutional neural net.
> Phase 3: *Distributed*
> * Expand deep learning support to include *distributed operations* with large
> models. This includes improvements to the DML deep learning library, the
> built-in operators, the GPU acceleration, and general engine improvements.
> Phase 4: *APIs/Wrappers*
> * Explore integration with Caffe, creating a SystemML interpreter for Caffe
> model definitions.
> * Explore integration with Keras, creating a SystemML backend for Keras.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
