[ https://issues.apache.org/jira/browse/SPARK-23244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357504#comment-16357504 ]
Bryan Cutler commented on SPARK-23244: -------------------------------------- This is same issue as SPARK-21685 caused by pyspark not differentiating from defaults and set params when transferring values to Java. I think we can close this and SPARK-23234 also. > Incorrect handling of default values when deserializing python wrappers of > scala transformers > --------------------------------------------------------------------------------------------- > > Key: SPARK-23244 > URL: https://issues.apache.org/jira/browse/SPARK-23244 > Project: Spark > Issue Type: Bug > Components: MLlib > Affects Versions: 2.2.1 > Reporter: Tomas Nykodym > Priority: Minor > > Default values are not handled properly when serializing/deserializing python > trasnformers which are wrappers around scala objects. It looks like that > after deserialization the default values which were based on uid do not get > properly restored and values which were not set are set to their (original) > default values. > Here's a simple code example using Bucketizer: > {code:python} > >>> from pyspark.ml.feature import Bucketizer > >>> a = Bucketizer() > >>> a.save("bucketizer0") > >>> b = load("bucketizer0") > >>> a._defaultParamMap[a.outputCol] > u'Bucketizer_440bb49206c148989db7__output' > >>> b._defaultParamMap[b.outputCol] > u'Bucketizer_41cf9afbc559ca2bfc9a__output' > >>> a.isSet(a.outputCol) > False > >>> b.isSet(b.outputCol) > True > >>> a.getOutputCol() > u'Bucketizer_440bb49206c148989db7__output' > >>> b.getOutputCol() > u'Bucketizer_440bb49206c148989db7__output' > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org