[ https://issues.apache.org/jira/browse/SPARK-20687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16004372#comment-16004372 ]
Sean Owen commented on SPARK-20687: ----------------------------------- This doesn't say what the problem is. What goes wrong? > mllib.Matrices.fromBreeze may crash when converting breeze CSCMatrix > -------------------------------------------------------------------- > > Key: SPARK-20687 > URL: https://issues.apache.org/jira/browse/SPARK-20687 > Project: Spark > Issue Type: Bug > Components: MLlib > Affects Versions: 2.1.1 > Reporter: Ignacio Bermudez Corrales > Priority: Critical > > Conversion of Breeze sparse matrices to Matrix is broken when matrices are > product of certain operations. This problem I think is caused by the update > method in Breeze CSCMatrix when they add provisional zeros to the data for > efficiency. > This bug is serious and may affect at least BlockMatrix addition and > substraction > http://stackoverflow.com/questions/33528555/error-thrown-when-using-blockmatrix-add/43883458#43883458 > The following code, reproduces the bug (Check test("breeze conversion bug")) > https://github.com/ghoto/spark/blob/test-bug/CSCMatrixBreeze/mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala > {code:title=MatricesSuite.scala|borderStyle=solid} > test("breeze conversion bug") { > // (2, 0, 0) > // (2, 0, 0) > val mat1Brz = Matrices.sparse(2, 3, Array(0, 2, 2, 2), Array(0, 1), > Array(2, 2)).asBreeze > // (2, 1E-15, 1E-15) > // (2, 1E-15, 1E-15 > val mat2Brz = Matrices.sparse(2, 3, Array(0, 2, 4, 6), Array(0, 0, 0, 1, > 1, 1), Array(2, 1E-15, 1E-15, 2, 1E-15, 1E-15)).asBreeze > // The following shouldn't break > val t01 = mat1Brz - mat1Brz > val t02 = mat2Brz - mat2Brz > val t02Brz = Matrices.fromBreeze(t02) > val t01Brz = Matrices.fromBreeze(t01) > val t1Brz = mat1Brz - mat2Brz > val t2Brz = mat2Brz - mat1Brz > // The following ones should break > val t1 = Matrices.fromBreeze(t1Brz) > val t2 = Matrices.fromBreeze(t2Brz) > } > {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org