[
https://issues.apache.org/jira/browse/SPARK-61697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-61697.
----------------------------------
Resolution: Fixed
Issue resolved by pull request 681
[https://github.com/apache/spark-connect-rust/pull/681]
> Add PySpark's Params API, typed models, summaries and persistence to the
> native ML classes
> ------------------------------------------------------------------------------------------
>
> Key: SPARK-61697
> URL: https://issues.apache.org/jira/browse/SPARK-61697
> Project: Spark
> Issue Type: Improvement
> Components: Connect
> Affects Versions: connect-rust-4.3.0
> Reporter: Hyukjin Kwon
> Assignee: Hyukjin Kwon
> Priority: Major
> Labels: pull-request-available
> Fix For: connect-rust-4.3.0
>
>
> The native ML classes of the Python client (pyspark.ml.connect) are to follow
> the API of pyspark.ml 4.3.0 over Spark Connect, class by class. This adds
> what each class builds on, and completes LinearSVC and KMeans with their
> models and summaries as the worked examples:
> - PySpark's Params API (params, getParam, hasParam, isSet, isDefined,
> hasDefault, getOrDefault, explainParam(s), extractParamMap, set, clear,
> copy), of PySpark's own Param objects, with the docs, converters and defaults
> of each official class (captured from pyspark-client 4.3.0), and the defaults
> of each instance as PySpark keeps them (seed, outputCol, the defaults that a
> fit copies from the estimator to its model, and those that a load keeps);
> - typed model and summary classes, whose members read the server's model
> objects;
> - persistence (write/save/read/load, MLWriter, MLReader), fits of param maps
> and fitMultiple;
> - in the Rust API, ObjectModel, training_summary, summary_transform and
> evaluated_summary.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]