Ignacio Bermudez Corrales created SPARK-20687: -------------------------------------------------
Summary: mllib.Matrices.fromBreeze may crash when converting breeze CSCMatrix Key: SPARK-20687 URL: https://issues.apache.org/jira/browse/SPARK-20687 Project: Spark Issue Type: Bug Components: MLlib Affects Versions: 2.1.1 Reporter: Ignacio Bermudez Corrales Priority: Critical Conversion of Breeze sparse matrices to Matrix is broken when matrices are product of certain operations. This problem I think is caused by the update method in Breeze CSCMatrix when they add provisional zeros to the data for efficiency. This bug is serious and may affect at least BlockMatrix addition and substraction http://stackoverflow.com/questions/33528555/error-thrown-when-using-blockmatrix-add/43883458#43883458 The following code, reproduces the bug. test("breeze conversion bug") { // (2, 0, 0) // (2, 0, 0) val mat1Brz = Matrices.sparse(2, 3, Array(0, 2, 2, 2), Array(0, 1), Array(2, 2)).asBreeze // (2, 1E-15, 1E-15) // (2, 1E-15, 1E-15 val mat2Brz = Matrices.sparse(2, 3, Array(0, 2, 4, 6), Array(0, 0, 0, 1, 1, 1), Array(2, 1E-15, 1E-15, 2, 1E-15, 1E-15)).asBreeze // The following shouldn't break val t01 = mat1Brz - mat1Brz val t02 = mat2Brz - mat2Brz val t02Brz = Matrices.fromBreeze(t02) val t01Brz = Matrices.fromBreeze(t01) val t1Brz = mat1Brz - mat2Brz val t2Brz = mat2Brz - mat1Brz // The following ones should break val t1 = Matrices.fromBreeze(t1Brz) val t2 = Matrices.fromBreeze(t2Brz) } -- This message was sent by Atlassian JIRA (v6.3.15#6346) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org