Hyukjin Kwon created SPARK-61697:
------------------------------------

             Summary: Add PySpark's Params API, typed models, summaries and 
persistence to the native ML classes
                 Key: SPARK-61697
                 URL: https://issues.apache.org/jira/browse/SPARK-61697
             Project: Spark
          Issue Type: Improvement
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


The native ML classes of the Python client (pyspark.ml.connect) are to follow 
the API of pyspark.ml 4.3.0 over Spark Connect, class by class. This adds what 
each class builds on, and completes LinearSVC and KMeans with their models and 
summaries as the worked examples:
- PySpark's Params API (params, getParam, hasParam, isSet, isDefined, 
hasDefault, getOrDefault, explainParam(s), extractParamMap, set, clear, copy), 
of PySpark's own Param objects, with the docs, converters and defaults of each 
official class (captured from pyspark-client 4.3.0), and the defaults of each 
instance as PySpark keeps them (seed, outputCol, the defaults that a fit copies 
from the estimator to its model, and those that a load keeps);
- typed model and summary classes, whose members read the server's model 
objects;
- persistence (write/save/read/load, MLWriter, MLReader), fits of param maps 
and fitMultiple;
- in the Rust API, ObjectModel, training_summary, summary_transform and 
evaluated_summary.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to