Github user viirya commented on a diff in the pull request:
https://github.com/apache/spark/pull/21501#discussion_r193954052
--- Diff:
mllib/src/main/scala/org/apache/spark/ml/feature/StopWordsRemover.scala ---
@@ -95,8 +121,7 @@ class StopWordsRemover @Since("1.5.0") (@Since("1.5.0")
override val uid: String
terms.filter(s => !stopWordsSet.contains(s))
}
} else {
- // TODO: support user locale (SPARK-15064)
- val toLower = (s: String) => if (s != null) s.toLowerCase else s
+ val toLower = (s: String) => if (s != null) s.toLowerCase(new
Locale($(locale))) else s
--- End diff --
Maybe move `new Locale($(locale))` out of `toLower`, so we don't need to
create it every time.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
