Re: Using Custom Scala Spark ML Estimator in PySpark

2021-02-16 Thread HARSH TAKKAR
Hello Sean, Thanks for the advice, can you please point me to an example where i can find a custom wrapper for python. Kind Regards Harsh Takkar On Tue, 16 Feb, 2021, 8:25 pm Sean Owen, wrote: > You won't be able to use it in python if it is implemented in Java - needs > a python wrapper

Re: Using Custom Scala Spark ML Estimator in PySpark

2021-02-16 Thread Sean Owen
You won't be able to use it in python if it is implemented in Java - needs a python wrapper too. On Mon, Feb 15, 2021, 11:29 PM HARSH TAKKAR wrote: > Hi , > > I have created a custom Estimator in scala, which i can use successfully > by creating a pipeline model in Java and scala, But when i

Re: Using Custom Scala Spark ML Estimator in PySpark

2021-02-16 Thread Mich Talebzadeh
Hi, Specifically is this a run time or compilation error. I gather by class path you mean something like below spark-submit --master yarn --deploy-mode client --driver-class-path --jars .. HTH LinkedIn *

Using Custom Scala Spark ML Estimator in PySpark

2021-02-15 Thread HARSH TAKKAR
Hi , I have created a custom Estimator in scala, which i can use successfully by creating a pipeline model in Java and scala, But when i try to load the pipeline model saved using scala api in pyspark, i am getting an error saying module not found. I have included my custom model jar in the