Hyukjin Kwon created SPARK-61399:
------------------------------------

             Summary: Raise in fit of the native Pipeline and CrossValidator 
instead of returning a model that the server cannot use
                 Key: SPARK-61399
                 URL: https://issues.apache.org/jira/browse/SPARK-61399
             Project: Spark
          Issue Type: Bug
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


Found in a post-merge review of SPARK-61324 and SPARK-61334. The fit of the 
native pyspark.ml.connect Pipeline and CrossValidator (Pipeline::fit_impl and 
CrossValidator::fit_impl in the Rust API) ignores the DataFrame and returns a 
local model without a server call, so fit silently succeeds and the model's 
transform then fails on the server with java.lang.AssertionError. The native 
Pipeline takes stage names, not stages, and the native CrossValidator has no 
estimator, evaluator or param grid, so neither can be fitted.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to