[
https://issues.apache.org/jira/browse/SPARK-9750?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696101#comment-14696101
]
Xiangrui Meng edited comment on SPARK-9750 at 8/13/15 10:50 PM:
----------------------------------------------------------------
This is really messy ... Using your terminology, this is what I found:
In 1.4 (which is the same as before this PR):
* Python
* DenseVector: Semantic eq but only with `DenseVector`. Default hash. -> *bug*
* SparseVector: Semantic eq but *wrong* (only with `SparseVector` and not
handling explicit zeros). Default hash. -> *bug*
* DenseMatrix: Semantic eq but only with `DenseMatrix`. Default hash. -> *bug*
* SparseMatrix: Semantic eq but converting to dense. Default hash. -> *bug*
* Scala
* Vectors: Semantic eq. Hash uses first 16 entries.
* DenseMatrix: Semantic eq but *wrong* (only with `DenseMatrix` and not
handling transpose). Guava Objects.hashCode (similar to default hash). -> *bug*
* SparseMatrix: Literal eq. Default hash.
was (Author: mengxr):
This is really messy ... Using your terminology, this is what I found:
In 1.4 (which is the same as before this PR):
* Python
* DenseVector: Semantic eq but only with `DenseVector`. Default hash.
* SparseVector: Semantic eq but *wrong* (only with `SparseVector` and not
handling explicit zeros). Default hash.
* DenseMatrix: Semantic eq but only with `DenseMatrix`. Default hash.
* SparseMatrix: Semantic eq but converting to dense. Default hash.
* Scala
* Vectors: Semantic eq. Hash uses first 16 entries.
* DenseMatrix: Semantic eq but *wrong* (only with `DenseMatrix` and not
handling transpose). Guava Objects.hashCode (similar to default hash).
* SparseMatrix: Literal eq. Default hash.
> SparseMatrix should override equals
> -----------------------------------
>
> Key: SPARK-9750
> URL: https://issues.apache.org/jira/browse/SPARK-9750
> Project: Spark
> Issue Type: Bug
> Components: MLlib
> Reporter: Feynman Liang
> Assignee: Feynman Liang
> Priority: Critical
> Fix For: 1.5.0
>
>
> [SparseMatrix|https://github.com/apache/spark/blob/9897cc5e3d6c70f7e45e887e2c6fc24dfa1adada/mllib/src/main/scala/org/apache/spark/mllib/linalg/Matrices.scala#L479]
> should override equals to ensure that two instances of the same matrix are
> equal.
> This implementation should take into account the {{isTransposed}} flag and
> {{values}} may not be in the same order.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
