Hyukjin Kwon created SPARK-61748:
------------------------------------

             Summary: Construct, fit and copy a Python subclass of the native 
ML classes as a subclass of PySpark's classes
                 Key: SPARK-61748
                 URL: https://issues.apache.org/jira/browse/SPARK-61748
             Project: Spark
          Issue Type: Improvement
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


The public classes of the native ML classes of spark-connect-rust 
(pyspark.ml.connect.*) are subclasses of PySpark's classes, but a Python 
subclass of one did not behave as a subclass of PySpark's class:
- the keywords of super().__init__(...) of a subclass's __init__ were dropped, 
and a subclass of positional arguments of its own raised "Method __init__ 
forces keyword arguments.";
- fit, transform and evaluate never ran a subclass's _fit, _transform or 
_evaluate;
- copy() returned an object of the public class, without the object's 
attributes;
- uid was read-only, and an object of a subclass had a uid of the native 
class's name.

The public classes should construct (PySpark's __init__ signature and 
keyword_only), dispatch (PySpark's own fit, fitMultiple, transform, evaluate, 
save and load over native _fit, _transform and _evaluate), copy (of the 
object's class and attributes, as copy.copy) and name their objects as 
PySpark's classes do.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to