Hyukjin Kwon created SPARK-61711:
------------------------------------

             Summary: Fetch the server defaults of the native ML classes as 
PySpark's constructors do, and refuse Column arguments of their members
                 Key: SPARK-61711
                 URL: https://issues.apache.org/jira/browse/SPARK-61711
             Project: Spark
          Issue Type: Improvement
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


Part of completing the native ML classes of pyspark.ml.connect to the full 
pyspark.ml 4.3.0 surface (fixes of the PyO3 base, SPARK-61697, that the area 
conversions found):
- The defaults that PySpark's constructor fetches of the server, by the ML 
helper (StopWordsRemover's default stop words and locale), are recorded as the 
fetch in the API golden, and the native constructors fetch them of the active 
session, raising PySpark's AssertionError without one.
- An argument of a member of a native model that is a Column, even of a 
literal, raises PySpark's UNSUPPORTED_DATA_TYPE, as 
pyspark.ml.connect.serialize raises it.
- The native classes of Params that neither fit, transform nor evaluate 
(PrefixSpan) are virtual subclasses of pyspark.ml.param.Params.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to