Calibration Methods
Hi, I am looking for calibrating the output of a pyspark model. I looked for possible implementations in Spark but didn't find any. Sklearn has CalibratedClassifierCV https://scikit-learn.org/stable/modules/generated/sklearn.calibration.CalibratedClassifierCV.html Could anyone point if there is any/similar implementation of the same in Spark? Regards, Praneeth -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
Calibration Methods
Hi All, I am looking for ways to calibrate the output of a pyspark ML model. Could anyone share if there are any implementations around of the same available in spark/pyspark? Here is the implementation available in sklearn: https://scikit-learn.org/stable/modules/generated/sklearn.calibration.CalibratedClassifierCV.html Please feel to share your experience/ideas around the calibration methods with spark. Regards, Praneeth -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ - To unsubscribe e-mail: user-unsubscr...@spark.apache.org
[Spark Streaming] unsubscribe and subscribe kafka topic without stopping context
Hi All, I am using spark streaming with kafka, I am working on an application which requires me to process data based on user requests. I have created 2 dstream i.e. one per topic but I am unable to stop processing of data on user request or re-enable processing of data. I have asked question on stackoverflow as well .. https://stackoverflow.com/questions/65285184/spark-subscribe-and-unsubscribe-topic-from-spark-streaming-without-requiring-res Any help here will be appreciated. Thanks Sanjay
