I am building a ML pipeline for logistic regression. val lr = new LogisticRegression()
lr.setMaxIter(100).setRegParam(0.001) val pipeline = new Pipeline().setStages(Array(geoDimEncoder,clientTypeEncoder, devTypeDimIdEncoder,pubClientIdEncoder,tmpltIdEncoder, hourEncoder,assembler,lr)) val model = pipeline.fit(trainingDF) Now, when the model is trained, I want to see the value the probabilities for the training set and compute certain validation parameters like log-loss. But, I am unable to find this using "model". The only thing I could find is model.transform(testDF).select(....) Cannot I get the metrics using the trained set for training set validation ? Thanks !!