[ 
https://issues.apache.org/jira/browse/SPARK-61748?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Hyukjin Kwon updated SPARK-61748:
---------------------------------
    Fix Version/s: connect-rust-4.3.0

> Construct, fit and copy a Python subclass of the native ML classes as a 
> subclass of PySpark's classes
> -----------------------------------------------------------------------------------------------------
>
>                 Key: SPARK-61748
>                 URL: https://issues.apache.org/jira/browse/SPARK-61748
>             Project: Spark
>          Issue Type: Improvement
>          Components: Connect
>    Affects Versions: connect-rust-4.3.0
>            Reporter: Hyukjin Kwon
>            Priority: Major
>             Fix For: connect-rust-4.3.0
>
>
> The public classes of the native ML classes of spark-connect-rust 
> (pyspark.ml.connect.*) are subclasses of PySpark's classes, but a Python 
> subclass of one did not behave as a subclass of PySpark's class:
> - the keywords of super().__init__(...) of a subclass's __init__ were 
> dropped, and a subclass of positional arguments of its own raised "Method 
> __init__ forces keyword arguments.";
> - fit, transform and evaluate never ran a subclass's _fit, _transform or 
> _evaluate;
> - copy() returned an object of the public class, without the object's 
> attributes;
> - uid was read-only, and an object of a subclass had a uid of the native 
> class's name.
> The public classes should construct (PySpark's __init__ signature and 
> keyword_only), dispatch (PySpark's own fit, fitMultiple, transform, evaluate, 
> save and load over native _fit, _transform and _evaluate), copy (of the 
> object's class and attributes, as copy.copy) and name their objects as 
> PySpark's classes do.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to