Repository: spark Updated Branches: refs/heads/master 8cfa218f4 -> c48f2a3a5
[SPARK-7615][MLLIB] MLLIB Word2Vec wordVectors divided by Euclidean Norm equals to zero Cosine similarity with 0 vector should be 0 Related to https://github.com/apache/spark/pull/10152 Author: Sean Owen <so...@cloudera.com> Closes #10696 from srowen/SPARK-7615. Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/c48f2a3a Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/c48f2a3a Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/c48f2a3a Branch: refs/heads/master Commit: c48f2a3a5fd714ad2ff19b29337e55583988431e Parents: 8cfa218 Author: Sean Owen <so...@cloudera.com> Authored: Tue Jan 12 11:50:33 2016 +0000 Committer: Sean Owen <so...@cloudera.com> Committed: Tue Jan 12 11:50:33 2016 +0000 ---------------------------------------------------------------------- .../main/scala/org/apache/spark/mllib/feature/Word2Vec.scala | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/c48f2a3a/mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala ---------------------------------------------------------------------- diff --git a/mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala b/mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala index dc5d070..dee8988 100644 --- a/mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala +++ b/mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala @@ -543,7 +543,12 @@ class Word2VecModel private[spark] ( val cosVec = cosineVec.map(_.toDouble) var ind = 0 while (ind < numWords) { - cosVec(ind) /= wordVecNorms(ind) + val norm = wordVecNorms(ind) + if (norm == 0.0) { + cosVec(ind) = 0.0 + } else { + cosVec(ind) /= norm + } ind += 1 } wordList.zip(cosVec) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org