Repository: spark Updated Branches: refs/heads/branch-1.6 4c67d55c0 -> 94b39f777
[SPARK-7615][MLLIB] MLLIB Word2Vec wordVectors divided by Euclidean Norm equals to zero Cosine similarity with 0 vector should be 0 Related to https://github.com/apache/spark/pull/10152 Author: Sean Owen <so...@cloudera.com> Closes #10696 from srowen/SPARK-7615. (cherry picked from commit c48f2a3a5fd714ad2ff19b29337e55583988431e) Signed-off-by: Sean Owen <so...@cloudera.com> Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/94b39f77 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/94b39f77 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/94b39f77 Branch: refs/heads/branch-1.6 Commit: 94b39f7777ecff3794727c186bd681fa4c6af4fd Parents: 4c67d55 Author: Sean Owen <so...@cloudera.com> Authored: Tue Jan 12 11:50:33 2016 +0000 Committer: Sean Owen <so...@cloudera.com> Committed: Tue Jan 12 13:27:44 2016 +0000 ---------------------------------------------------------------------- .../main/scala/org/apache/spark/mllib/feature/Word2Vec.scala | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/94b39f77/mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala ---------------------------------------------------------------------- diff --git a/mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala b/mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala index a010775..1dbedaa 100644 --- a/mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala +++ b/mllib/src/main/scala/org/apache/spark/mllib/feature/Word2Vec.scala @@ -539,7 +539,12 @@ class Word2VecModel private[spark] ( val cosVec = cosineVec.map(_.toDouble) var ind = 0 while (ind < numWords) { - cosVec(ind) /= wordVecNorms(ind) + val norm = wordVecNorms(ind) + if (norm == 0.0) { + cosVec(ind) = 0.0 + } else { + cosVec(ind) /= norm + } ind += 1 } wordList.zip(cosVec) --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org
