Repository: systemml Updated Branches: refs/heads/master 6b25b3bf2 -> cbde17bab
[MINOR] Add clarity to depthwise convolution documentation. Project: http://git-wip-us.apache.org/repos/asf/systemml/repo Commit: http://git-wip-us.apache.org/repos/asf/systemml/commit/cbde17ba Tree: http://git-wip-us.apache.org/repos/asf/systemml/tree/cbde17ba Diff: http://git-wip-us.apache.org/repos/asf/systemml/diff/cbde17ba Branch: refs/heads/master Commit: cbde17babd2452c4f265e0b6c56296f246c007a4 Parents: 6b25b3b Author: Mike Dusenberry <mwdus...@us.ibm.com> Authored: Fri Jun 30 12:05:35 2017 -0700 Committer: Mike Dusenberry <mwdus...@us.ibm.com> Committed: Fri Jun 30 12:05:35 2017 -0700 ---------------------------------------------------------------------- scripts/nn/layers/conv2d_depthwise.dml | 4 ++++ scripts/nn/layers/conv2d_transpose_depthwise.dml | 14 ++++++++++++++ 2 files changed, 18 insertions(+) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/systemml/blob/cbde17ba/scripts/nn/layers/conv2d_depthwise.dml ---------------------------------------------------------------------- diff --git a/scripts/nn/layers/conv2d_depthwise.dml b/scripts/nn/layers/conv2d_depthwise.dml index 0e9abb5..ff36ea1 100644 --- a/scripts/nn/layers/conv2d_depthwise.dml +++ b/scripts/nn/layers/conv2d_depthwise.dml @@ -42,6 +42,10 @@ forward = function(matrix[double] X, matrix[double] W, matrix[double] b, * depth 1 that expand each input channel to M output channels, where * M is a "depth multiplier". * + * Although there are C*M filters of depth 1, instead of storing W as + * shape `(C*M, 1*Hf*Wf)`, we reshape it to `(C, M*Hf*Wf)` for + * performance reasons. + * * Inputs: * - X: Inputs, of shape (N, C*Hin*Win). * - W: Weights, of shape (C, M*Hf*Wf). http://git-wip-us.apache.org/repos/asf/systemml/blob/cbde17ba/scripts/nn/layers/conv2d_transpose_depthwise.dml ---------------------------------------------------------------------- diff --git a/scripts/nn/layers/conv2d_transpose_depthwise.dml b/scripts/nn/layers/conv2d_transpose_depthwise.dml index fdd7c10..b3e798c 100644 --- a/scripts/nn/layers/conv2d_transpose_depthwise.dml +++ b/scripts/nn/layers/conv2d_transpose_depthwise.dml @@ -41,6 +41,20 @@ forward = function(matrix[double] X, matrix[double] W, matrix[double] b, * The resulting C/M separate output channels are then concatenated * together channel-wise into a single volume of C/M output channels. * + * For clarity, if we were to use the same terminology as a regular + * depthwise convolution, a depthwise transpose convolution has the + * ability to contract each group of M input channels (from a total of + * C*M input channels) back to a single output channel, thus leading + * to C output channels. Thus, this is the "transpose" of the regular + * depthwise convolution. To keep the convention of always referring + * to the number of input channels as C, in this depthwise transpose + * layer we can reformulate the above by dividing by M. With this + * reformulation, we can now state that there are C input channels, + * and for each group of M inputs we output a single output channel, + * for a total of C/M output channels. For this, we use 1 filter of + * depth M for each group of M input channels, and we store W as + * `(C/M, M*Hf*Wf)`. + * * Inputs: * - X: Inputs, of shape (N, C*Hin*Win). * - W: Weights, of shape (C/M, M*Hf*Wf).