HyukjinKwon edited a comment on issue #25133: [SPARK-28365][ML] Fallback locale to en_US in StopWordsRemover if system default locale isn't in available locales in JVM URL: https://github.com/apache/spark/pull/25133#issuecomment-513056938 > Specifying the en-US locale directly in StopWordsRemover This isn't possible because the error is thrown in its constructor of `StopWordsRemover`. This PR actually targets to allow to set different locale (vis `StopWordsRemover.setLocale`). Otherwise, the locale should be set into JVM or OS only to use this API. Here's an example full stack trace: ``` Py4JJavaError: An error occurred while calling None.org.apache.spark.ml.feature.StopWordsRemover. : java.lang.IllegalArgumentException: StopWordsRemover_daf8924a73f7 parameter locale given invalid value pl_US. at org.apache.spark.ml.param.Param.validate(params.scala:77) at org.apache.spark.ml.param.ParamPair.<init>(params.scala:656) at org.apache.spark.ml.param.Param.$minus$greater(params.scala:87) at org.apache.spark.ml.feature.StopWordsRemover.<init>(StopWordsRemover.scala:109) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at py4j.reflection.MethodInvoker.invoke(MethodInvoker.java:247) at py4j.reflection.ReflectionEngine.invoke(ReflectionEngine.java:357) at py4j.Gateway.invoke(Gateway.java:238) at py4j.commands.ConstructorCommand.invokeConstructor(ConstructorCommand.java:80) at py4j.commands.ConstructorCommand.execute(ConstructorCommand.java:69) at py4j.GatewayConnection.run(GatewayConnection.java:238) at java.lang.Thread.run(Thread.java:748) ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
