Hi , I have created a custom Estimator in scala, which i can use successfully by creating a pipeline model in Java and scala, But when i try to load the pipeline model saved using scala api in pyspark, i am getting an error saying module not found.
I have included my custom model jar in the class pass using "spark.jars" Can you please help, if i am missing something. Kind Regards Harsh Takkar