Hyukjin Kwon created SPARK-61697:
------------------------------------
Summary: Add PySpark's Params API, typed models, summaries and
persistence to the native ML classes
Key: SPARK-61697
URL: https://issues.apache.org/jira/browse/SPARK-61697
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
The native ML classes of the Python client (pyspark.ml.connect) are to follow
the API of pyspark.ml 4.3.0 over Spark Connect, class by class. This adds what
each class builds on, and completes LinearSVC and KMeans with their models and
summaries as the worked examples:
- PySpark's Params API (params, getParam, hasParam, isSet, isDefined,
hasDefault, getOrDefault, explainParam(s), extractParamMap, set, clear, copy),
of PySpark's own Param objects, with the docs, converters and defaults of each
official class (captured from pyspark-client 4.3.0), and the defaults of each
instance as PySpark keeps them (seed, outputCol, the defaults that a fit copies
from the estimator to its model, and those that a load keeps);
- typed model and summary classes, whose members read the server's model
objects;
- persistence (write/save/read/load, MLWriter, MLReader), fits of param maps
and fitMultiple;
- in the Rust API, ObjectModel, training_summary, summary_transform and
evaluated_summary.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]