[
https://issues.apache.org/jira/browse/FLINK-3330?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15132411#comment-15132411
]
Till Rohrmann commented on FLINK-3330:
--------------------------------------
Hi [~chiwanpark], yes the problem is that the {{GradientDescent}}
implementation expects the weight vector to be dense. There is a function
{{createInitialWeightsDS}} which will even convert sparse vectors into dense
vectors. The problem with MLR is now that the gradient is basically the feature
vector. If this vector is sparse, then the gradient will also be sparse. Thus,
in order to make it work, we either extend {{BLAS.axpy}} so that it also
supports sparse vectors or we make sure that the gradients are always dense, at
least the right side.
I think in most cases the gradient will also be dense. Thus it would make sense
to transform the first gradient into a dense vector and then add the following
sparse gradients to it.
> Add SparseVector support to BLAS library in FlinkML
> ---------------------------------------------------
>
> Key: FLINK-3330
> URL: https://issues.apache.org/jira/browse/FLINK-3330
> Project: Flink
> Issue Type: Improvement
> Components: Machine Learning Library
> Affects Versions: 1.0.0
> Reporter: Chiwan Park
> Assignee: Chiwan Park
>
> A user reported the problem using {{GradientDescent}} algorithm with
> {{SparseVector}}.
> (http://mail-archives.apache.org/mod_mbox/flink-user/201602.mbox/%3CCAMJxVsiNRy_B349tuRpC%2BY%2BfyW7j2SHcyVfhqnz3BGOwEHXHpg%40mail.gmail.com%3E)
> It seems lack of SparseVector support in {{BLAS.axpy}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
