[
https://issues.apache.org/jira/browse/MAHOUT-1346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13788820#comment-13788820
]
Dmitriy Lyubimov commented on MAHOUT-1346:
------------------------------------------
One inconvenience that i seem to be running into quite a bit when i am working
on this is that there's no efficient matrix block implementation that would
just take hanging vectors and put them into hashmap while retaining original
matrix geometry configuration (e.g. if I cut out a block
(3001:3010,3001:3010), it doesn't allocate hanging vector tables 1:m to
represent this).
SparseRow/ColumnMatrix does almost what i need; except it needs to have the
"Vector[] rows" replaced with some sort of HashMap..
maybe i need a new type, something a SparseBlockRowMatrix. that does that.
Unfortunately i feel it creates a little overload in the in-core world of
types. I would say SparseRow/ColumnMatrix rather should be modified into this.
Seems like a simple change, but of course it has implications for use patterns
since it would not provide good sequential row-wise iteration speed.
without this, i am forced manipulating with hash maps of index-> vector things,
but that does not make a nice abstraction and disallows in-core DSL.
> Spark Bindings (DRM)
> --------------------
>
> Key: MAHOUT-1346
> URL: https://issues.apache.org/jira/browse/MAHOUT-1346
> Project: Mahout
> Issue Type: Improvement
> Affects Versions: 0.8
> Reporter: Dmitriy Lyubimov
> Assignee: Dmitriy Lyubimov
> Fix For: 0.9
>
>
> Spark bindings for Mahout DRM.
> DRM DSL.
> Disclaimer. This will all be experimental at this point.
> The idea is to wrap DRM by Spark RDD with support of some basic
> functionality, perhaps some humble beginning of Cost-based optimizer
> (0) Spark serialization support for Vector, Matrix
> (1) Bagel transposition
> (2) slim X'X
> (2a) not-so-slim X'X
> (3) blockify() (compose RDD containing vertical blocks of original input)
> (4) read/write Mahout DRM off HDFS
> (5) A'B
> ...
--
This message was sent by Atlassian JIRA
(v6.1#6144)
