[jira] [Updated] (SPARK-20687) mllib.Matrices.fromBreeze may crash when converting breeze CSCMatrix
[
https://issues.apache.org/jira/browse/SPARK-20687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean Owen updated SPARK-20687:
--
Priority: Minor (was: Critical)
> mllib.Matrices.fromBreeze may crash when converting breeze CSCMatrix
>
>
> Key: SPARK-20687
> URL: https://issues.apache.org/jira/browse/SPARK-20687
> Project: Spark
> Issue Type: Bug
> Components: MLlib
>Affects Versions: 2.1.1
>Reporter: Ignacio Bermudez Corrales
>Priority: Minor
>
> Conversion of Breeze sparse matrices to Matrix is broken when matrices are
> product of certain operations. This problem I think is caused by the update
> method in Breeze CSCMatrix when they add provisional zeros to the data for
> efficiency.
> This bug is serious and may affect at least BlockMatrix addition and
> substraction
> http://stackoverflow.com/questions/33528555/error-thrown-when-using-blockmatrix-add/43883458#43883458
> The following code, reproduces the bug (Check test("breeze conversion bug"))
> https://github.com/ghoto/spark/blob/test-bug/CSCMatrixBreeze/mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala
> {code:title=MatricesSuite.scala|borderStyle=solid}
> test("breeze conversion bug") {
> // (2, 0, 0)
> // (2, 0, 0)
> val mat1Brz = Matrices.sparse(2, 3, Array(0, 2, 2, 2), Array(0, 1),
> Array(2, 2)).asBreeze
> // (2, 1E-15, 1E-15)
> // (2, 1E-15, 1E-15
> val mat2Brz = Matrices.sparse(2, 3, Array(0, 2, 4, 6), Array(0, 0, 0, 1,
> 1, 1), Array(2, 1E-15, 1E-15, 2, 1E-15, 1E-15)).asBreeze
> // The following shouldn't break
> val t01 = mat1Brz - mat1Brz
> val t02 = mat2Brz - mat2Brz
> val t02Brz = Matrices.fromBreeze(t02)
> val t01Brz = Matrices.fromBreeze(t01)
> val t1Brz = mat1Brz - mat2Brz
> val t2Brz = mat2Brz - mat1Brz
> // The following ones should break
> val t1 = Matrices.fromBreeze(t1Brz)
> val t2 = Matrices.fromBreeze(t2Brz)
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-20687) mllib.Matrices.fromBreeze may crash when converting breeze CSCMatrix
[
https://issues.apache.org/jira/browse/SPARK-20687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ignacio Bermudez Corrales updated SPARK-20687:
--
Description:
Conversion of Breeze sparse matrices to Matrix is broken when matrices are
product of certain operations. This problem I think is caused by the update
method in Breeze CSCMatrix when they add provisional zeros to the data for
efficiency.
This bug is serious and may affect at least BlockMatrix addition and
substraction
http://stackoverflow.com/questions/33528555/error-thrown-when-using-blockmatrix-add/43883458#43883458
The following code, reproduces the bug (Check test("breeze conversion bug"))
https://github.com/ghoto/spark/blob/test-bug/CSCMatrixBreeze/mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala
{code:title=MatricesSuite.scala|borderStyle=solid}
test("breeze conversion bug") {
// (2, 0, 0)
// (2, 0, 0)
val mat1Brz = Matrices.sparse(2, 3, Array(0, 2, 2, 2), Array(0, 1),
Array(2, 2)).asBreeze
// (2, 1E-15, 1E-15)
// (2, 1E-15, 1E-15
val mat2Brz = Matrices.sparse(2, 3, Array(0, 2, 4, 6), Array(0, 0, 0, 1, 1,
1), Array(2, 1E-15, 1E-15, 2, 1E-15, 1E-15)).asBreeze
// The following shouldn't break
val t01 = mat1Brz - mat1Brz
val t02 = mat2Brz - mat2Brz
val t02Brz = Matrices.fromBreeze(t02)
val t01Brz = Matrices.fromBreeze(t01)
val t1Brz = mat1Brz - mat2Brz
val t2Brz = mat2Brz - mat1Brz
// The following ones should break
val t1 = Matrices.fromBreeze(t1Brz)
val t2 = Matrices.fromBreeze(t2Brz)
}
{code}
was:
Conversion of Breeze sparse matrices to Matrix is broken when matrices are
product of certain operations. This problem I think is caused by the update
method in Breeze CSCMatrix when they add provisional zeros to the data for
efficiency.
This bug is serious and may affect at least BlockMatrix addition and
substraction
http://stackoverflow.com/questions/33528555/error-thrown-when-using-blockmatrix-add/43883458#43883458
The following code, reproduces the bug.
test("breeze conversion bug") {
// (2, 0, 0)
// (2, 0, 0)
val mat1Brz = Matrices.sparse(2, 3, Array(0, 2, 2, 2), Array(0, 1),
Array(2, 2)).asBreeze
// (2, 1E-15, 1E-15)
// (2, 1E-15, 1E-15
val mat2Brz = Matrices.sparse(2, 3, Array(0, 2, 4, 6), Array(0, 0, 0, 1, 1,
1), Array(2, 1E-15, 1E-15, 2, 1E-15, 1E-15)).asBreeze
// The following shouldn't break
val t01 = mat1Brz - mat1Brz
val t02 = mat2Brz - mat2Brz
val t02Brz = Matrices.fromBreeze(t02)
val t01Brz = Matrices.fromBreeze(t01)
val t1Brz = mat1Brz - mat2Brz
val t2Brz = mat2Brz - mat1Brz
// The following ones should break
val t1 = Matrices.fromBreeze(t1Brz)
val t2 = Matrices.fromBreeze(t2Brz)
}
> mllib.Matrices.fromBreeze may crash when converting breeze CSCMatrix
>
>
> Key: SPARK-20687
> URL: https://issues.apache.org/jira/browse/SPARK-20687
> Project: Spark
> Issue Type: Bug
> Components: MLlib
>Affects Versions: 2.1.1
>Reporter: Ignacio Bermudez Corrales
>Priority: Critical
>
> Conversion of Breeze sparse matrices to Matrix is broken when matrices are
> product of certain operations. This problem I think is caused by the update
> method in Breeze CSCMatrix when they add provisional zeros to the data for
> efficiency.
> This bug is serious and may affect at least BlockMatrix addition and
> substraction
> http://stackoverflow.com/questions/33528555/error-thrown-when-using-blockmatrix-add/43883458#43883458
> The following code, reproduces the bug (Check test("breeze conversion bug"))
> https://github.com/ghoto/spark/blob/test-bug/CSCMatrixBreeze/mllib/src/test/scala/org/apache/spark/mllib/linalg/MatricesSuite.scala
> {code:title=MatricesSuite.scala|borderStyle=solid}
> test("breeze conversion bug") {
> // (2, 0, 0)
> // (2, 0, 0)
> val mat1Brz = Matrices.sparse(2, 3, Array(0, 2, 2, 2), Array(0, 1),
> Array(2, 2)).asBreeze
> // (2, 1E-15, 1E-15)
> // (2, 1E-15, 1E-15
> val mat2Brz = Matrices.sparse(2, 3, Array(0, 2, 4, 6), Array(0, 0, 0, 1,
> 1, 1), Array(2, 1E-15, 1E-15, 2, 1E-15, 1E-15)).asBreeze
> // The following shouldn't break
> val t01 = mat1Brz - mat1Brz
> val t02 = mat2Brz - mat2Brz
> val t02Brz = Matrices.fromBreeze(t02)
> val t01Brz = Matrices.fromBreeze(t01)
> val t1Brz = mat1Brz - mat2Brz
> val t2Brz = mat2Brz - mat1Brz
> // The following ones should break
> val t1 = Matrices.fromBreeze(t1Brz)
> val t2 = Matrices.fromBreeze(t2Brz)
> }
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For addit
[jira] [Updated] (SPARK-20687) mllib.Matrices.fromBreeze may crash when converting breeze CSCMatrix
[
https://issues.apache.org/jira/browse/SPARK-20687?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ignacio Bermudez Corrales updated SPARK-20687:
--
test("breeze conversion bug") {
// (2, 0, 0)
// (2, 0, 0)
val mat1Brz = Matrices.sparse(2, 3, Array(0, 2, 2, 2), Array(0, 1),
Array(2, 2)).asBreeze
// (2, 1E-15, 1E-15)
// (2, 1E-15, 1E-15
val mat2Brz = Matrices.sparse(2, 3, Array(0, 2, 4, 6), Array(0, 0, 0, 1, 1,
1), Array(2, 1E-15, 1E-15, 2, 1E-15, 1E-15)).asBreeze
// The following shouldn't break
val t01 = mat1Brz - mat1Brz
val t02 = mat2Brz - mat2Brz
val t02Brz = Matrices.fromBreeze(t02)
val t01Brz = Matrices.fromBreeze(t01)
val t1Brz = mat1Brz - mat2Brz
val t2Brz = mat2Brz - mat1Brz
// The following ones should break
val t1 = Matrices.fromBreeze(t1Brz)
val t2 = Matrices.fromBreeze(t2Brz)
}
> mllib.Matrices.fromBreeze may crash when converting breeze CSCMatrix
>
>
> Key: SPARK-20687
> URL: https://issues.apache.org/jira/browse/SPARK-20687
> Project: Spark
> Issue Type: Bug
> Components: MLlib
>Affects Versions: 2.1.1
>Reporter: Ignacio Bermudez Corrales
>Priority: Critical
>
> Conversion of Breeze sparse matrices to Matrix is broken when matrices are
> product of certain operations. This problem I think is caused by the update
> method in Breeze CSCMatrix when they add provisional zeros to the data for
> efficiency.
> This bug is serious and may affect at least BlockMatrix addition and
> substraction
> http://stackoverflow.com/questions/33528555/error-thrown-when-using-blockmatrix-add/43883458#43883458
> The following code, reproduces the bug.
> test("breeze conversion bug") {
> // (2, 0, 0)
> // (2, 0, 0)
> val mat1Brz = Matrices.sparse(2, 3, Array(0, 2, 2, 2), Array(0, 1),
> Array(2, 2)).asBreeze
> // (2, 1E-15, 1E-15)
> // (2, 1E-15, 1E-15
> val mat2Brz = Matrices.sparse(2, 3, Array(0, 2, 4, 6), Array(0, 0, 0, 1,
> 1, 1), Array(2, 1E-15, 1E-15, 2, 1E-15, 1E-15)).asBreeze
> // The following shouldn't break
> val t01 = mat1Brz - mat1Brz
> val t02 = mat2Brz - mat2Brz
> val t02Brz = Matrices.fromBreeze(t02)
> val t01Brz = Matrices.fromBreeze(t01)
> val t1Brz = mat1Brz - mat2Brz
> val t2Brz = mat2Brz - mat1Brz
> // The following ones should break
> val t1 = Matrices.fromBreeze(t1Brz)
> val t2 = Matrices.fromBreeze(t2Brz)
> }
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
