[
https://issues.apache.org/jira/browse/SINGA-58?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
wangwei resolved SINGA-58.
--------------------------
Resolution: Fixed
Assignee: wangwei
> Fix fan-in dimension of weight matrix
> -------------------------------------
>
> Key: SINGA-58
> URL: https://issues.apache.org/jira/browse/SINGA-58
> Project: Singa
> Issue Type: Improvement
> Reporter: wangwei
> Assignee: wangwei
>
> There are many layers using weight matrix, e.g., convolution layer and
> innerproduct layer. Some initialization methods of the weight matrix are
> based on the fan-in of the matrix, i.e., the dimension interacts with the
> other operator. E.g., for v*W, W's fan-in is the number of rows; For v*W^T,
> W's fan-in is the number of columns.
> However, the Param::InitValues() method does not know the position (and
> transposition) of the matrix in the multiplication operation. We have to fix
> the fan-in dimension. Particularly, we fix it to the second dimension (i.e.,
> number of columns) and assume users use the weight matrix correctly.
> The current implementation of convolution layer and innerproduct layer have
> different fan-in dimension, which should be unified to the second dimension.
> For the convolution layer, its operation is W*v. W's fan-in is already the
> number of columns. For the innerproduct layer, the current operation is v*W,
> which should be updated to v*W^T, then the fan-in is the number of columns.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
