Hyukjin Kwon created SPARK-61679:
------------------------------------

             Summary: Add all native ML statistics
                 Key: SPARK-61679
                 URL: https://issues.apache.org/jira/browse/SPARK-61679
             Project: Spark
          Issue Type: Improvement
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


Add native Rust APIs and Python bindings for the public classes of standard 
pyspark.ml.stat: ChiSquareTest, Correlation, KolmogorovSmirnovTest, Summarizer, 
SummaryBuilder and MultivariateGaussian. The statistical tests use Spark's ML 
helper relations, and Summarizer builds aggregate_metrics expressions for its 
ten metrics, with optional weights. Python's pyspark.ml.stat delegates to the 
native bindings.

PR: https://github.com/apache/spark-connect-rust/pull/653




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to