'requirement failed: OneHotEncoderModel expected x categorical values for input column label, but the input column had metadata specifying n values.'

[Mina Aslani]

Hi,

I am getting the following exception when I am using OneHotEncoderEstimator
with MultilayerPerceptronClassifier in Pyspark. (using version 2.4.4)

*'requirement failed: OneHotEncoderModel expected x categorical values for
input column label, but the input column had metadata specifying n values.'*

Using LogisticRegression, RandomForestClassifier or LinearRegression works
fine for the same data and OneHotEncoderEstimator.

Any insight on how to resolve this?

Regards,
Mina