[
https://issues.apache.org/jira/browse/SPARK-32315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Huaxin Gao resolved SPARK-32315.
--------------------------------
Fix Version/s: 3.1.0
Resolution: Fixed
Issue resolved by pull request 29115
[https://github.com/apache/spark/pull/29115]
> Provide an explanation error message when calling require
> ---------------------------------------------------------
>
> Key: SPARK-32315
> URL: https://issues.apache.org/jira/browse/SPARK-32315
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 3.0.0
> Reporter: dzlabs
> Priority: Trivial
> Fix For: 3.1.0
>
>
> When requirement checks inside `MLibUtils.fastSquaredDistance` are not met a
> non useful exception is thrown
> ```
> Caused by: java.lang.IllegalArgumentException: requirement failedCaused by:
> java.lang.IllegalArgumentException: requirement failed at
> scala.Predef$.require(Predef.scala:212) at
> org.apache.spark.mllib.util.MLUtils$.fastSquaredDistance(MLUtils.scala:508)
> at
> org.apache.spark.mllib.clustering.EuclideanDistanceMeasure$.fastSquaredDistance(DistanceMeasure.scala:232)
> at
> org.apache.spark.mllib.clustering.EuclideanDistanceMeasure.isCenterConverged(DistanceMeasure.scala:190)
> at
> org.apache.spark.mllib.clustering.KMeans$$anonfun$runAlgorithm$4.apply(KMeans.scala:336)
> at
> org.apache.spark.mllib.clustering.KMeans$$anonfun$runAlgorithm$4.apply(KMeans.scala:334)
> at
> scala.collection.MapLike$MappedValues$$anonfun$foreach$3.apply(MapLike.scala:245)
> at
> scala.collection.MapLike$MappedValues$$anonfun$foreach$3.apply(MapLike.scala:245)
> at
> scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:733)
> at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)
> at
> scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:130)
> at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:236)
> at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:40) at
> scala.collection.mutable.HashMap.foreach(HashMap.scala:130) at
> scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:732)
> at scala.collection.MapLike$MappedValues.foreach(MapLike.scala:245) at
> org.apache.spark.mllib.clustering.KMeans.runAlgorithm(KMeans.scala:334) at
> org.apache.spark.mllib.clustering.KMeans.run(KMeans.scala:251) at
> org.apache.spark.mllib.clustering.KMeans.run(KMeans.scala:233)
> ```
> This is happening in the following lines. Please add more useful information
> to the error message when the requirements are not met
> [https://github.com/apache/spark/blob/master/mllib/src/main/scala/org/apache/spark/mllib/util/MLUtils.scala#L537-L538]
>
> Example:
> ```
> require(v2.size == n, s"Both vectors should have same length, found v1 is $n
> while v2 is ${v2.size}")
> require(norm1 >= 0.0 && norm2 >= 0.0, "Both norms should be greater or equal
> to 0.0, found norm1=${norm1}, norm2=${norm2}")
> ```
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
