Calibration Methods

2020-12-14 Thread Praneeth Shishtla
Hi, I am looking for calibrating the output of a pyspark model. I looked for possible implementations in Spark but didn't find any. Sklearn has CalibratedClassifierCV https://scikit-learn.org/stable/modules/generated/sklearn.calibration.CalibratedClassifierCV.html Could anyone point if there

Calibration Methods

2020-12-14 Thread Praneeth Shishtla
Hi All, I am looking for ways to calibrate the output of a pyspark ML model. Could anyone share if there are any implementations around of the same available in spark/pyspark? Here is the implementation available in sklearn:

[Spark Streaming] unsubscribe and subscribe kafka topic without stopping context

2020-12-14 Thread Sanjay Tiwari
Hi All, I am using spark streaming with kafka, I am working on an application which requires me to process data based on user requests. I have created 2 dstream i.e. one per topic but I am unable to stop processing of data on user request or re-enable processing of data. I have asked question on