SOLVED.
The rawPredictionCol input to BinaryClassificationEvaluator is a
vector specifying the prediction confidence for each class. Since we
are talking about binary classification the prediction for class 0 is
simply (1 - y_pred), where y_pred is the prediction for class 1.
So this can be appli
pyspark.ml.evaluation.BinaryClassificationEvaluator expects
predictions in the form of vectors (apparently designating confidence
intervals), as described in
https://spark.apache.org/docs/latest/api/python/pyspark.ml.html#pyspark.ml.evaluation.BinaryClassificationEvaluator
However, I am trying to