The example works for me, please check your environment and ensure you are using Spark 2.3.0 where OneHotEncoderEstimator was introduced.
On Fri, May 18, 2018 at 12:57 AM, Matteo Cossu <elco...@gmail.com> wrote: > Hi, > > are you sure Dataset has a method withColumns? > > On 15 May 2018 at 16:58, Mina Aslani <aslanim...@gmail.com> wrote: > >> Hi, >> >> I get below error when I try to run oneHotEncoderEstimator example. >> https://github.com/apache/spark/blob/b74366481cc87490adf4e69 >> d26389ec737548c15/examples/src/main/java/org/apache/ >> spark/examples/ml/JavaOneHotEncoderEstimatorExample.java#L67 >> >> Which is this line of the code: >> https://github.com/apache/spark/blob/master/mllib/src/main/ >> scala/org/apache/spark/ml/feature/OneHotEncoderEstimator.scala#L348 >> >> Exception in thread "streaming-job-executor-0" java.lang.NoSuchMethodError: >> org.apache.spark.sql.Dataset.withColumns(Lscala/collection/Seq;Lscala/collection/Seq;)Lorg/apache/spark/sql/Dataset; >> at >> org.apache.spark.ml.feature.OneHotEncoderModel.transform(OneHotEncoderEstimator.scala:348) >> >> >> Can you please let me know, what is the cause? Any workaround? >> >> Seeing the example in the repo, looks like that at some point it used be >> running fine. And, now it's not working. Also, oneHotEncoder is deprecated. >> >> I really appreciate your quick response. >> >> Regards, >> Mina >> >> >
