[ https://issues.apache.org/jira/browse/MAHOUT-1837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15270400#comment-15270400 ]
ASF GitHub Bot commented on MAHOUT-1837: ---------------------------------------- Github user andrewpalumbo commented on the pull request: https://github.com/apache/mahout/pull/228#issuecomment-216814874 See the Spreadsheet attached to the Jira: https://issues.apache.org/jira/secure/attachment/12802153/compareDensityTest.ods for performance comparison. > Sparse/Dense Matrix analysis for Matrix Multiplication > ------------------------------------------------------ > > Key: MAHOUT-1837 > URL: https://issues.apache.org/jira/browse/MAHOUT-1837 > Project: Mahout > Issue Type: Improvement > Components: Math > Affects Versions: 0.12.0 > Reporter: Andrew Palumbo > Assignee: Andrew Palumbo > Fix For: 0.12.1 > > Attachments: compareDensityTest.ods > > > In matrix multiplication, Sparse Matrices can easily turn dense and bloat > memory, one fully dense column and one fully dense row can cause a sparse > %*% sparse operation have a dense result. > There are two issues here one with a quick Fix and one a bit more involved: > # in {{ABt.Scala}} use check the `MatrixFlavor` of the combiner and use > the flavor of the Block as the resulting Sparse or Dense matrix type: > {code} > val comb = if (block.getFlavor == MatrixFlavor.SPARSELIKE) { > new SparseMatrix(prodNCol, block.nrow).t > } else { > new DenseMatrix(prodNCol, block.nrow).t > } > {code} > a simlar check needs to be made in the {{blockify}} transformation. > > # More importantly, and more involved is to do an actual analysis of the > resulting matrix data in the in-core {{mmul}} class and use a matrix of the > appropriate Structure as a result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)