[ 
https://issues.apache.org/jira/browse/SPARK-61923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated SPARK-61923:
-----------------------------------
    Labels: pull-request-available  (was: )

> Send and save the ML params as PySpark's serialize_ml_params does: params of 
> no converter, params of subclasses, and the class of subclasses of 
> meta-algorithms
> ---------------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-61923
>                 URL: https://issues.apache.org/jira/browse/SPARK-61923
>             Project: Spark
>          Issue Type: Bug
>          Components: Connect
>    Affects Versions: connect-rust-4.4.0
>            Reporter: Hyukjin Kwon
>            Priority: Major
>              Labels: pull-request-available
>             Fix For: connect-rust-4.4.0
>
>
> The saves of the native ML classes of the drop-in differ from PySpark 4.3.0's:
> - RegexTokenizer.gaps, a param of no converter, is never written by save, so 
> a saved tokenizer (also inside a Pipeline or PipelineModel) loads with 
> gaps=True and gives other results; values that PySpark's save refuses 
> (gaps=(1,), 2**63, 1, "yes", [True]) save silently.
> - The params of a Python subclass that are set are not sent, at a fit, 
> transform, evaluate or save, where PySpark's serialize_ml_params sends every 
> item of _paramMap, which the server refuses.
> - Subclasses of OneVsRest, Pipeline, CrossValidator and TrainValidationSplit 
> save a constant class name, where PySpark writes the instance's own module 
> and class name.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to