Hyukjin Kwon created SPARK-61732:
------------------------------------

             Summary: Add PySpark's LDAModel, the base class of the native 
LocalLDAModel and DistributedLDAModel
                 Key: SPARK-61732
                 URL: https://issues.apache.org/jira/browse/SPARK-61732
             Project: Spark
          Issue Type: Improvement
          Components: Connect
    Affects Versions: connect-rust-4.3.0
            Reporter: Hyukjin Kwon


PySpark's pyspark.ml.clustering has LDAModel, the base class of LocalLDAModel 
and DistributedLDAModel (both are (LDAModel, JavaMLReadable, JavaMLWritable)), 
with the members of an LDA model (describeTopics, estimatedDocConcentration, 
isDistributed, logLikelihood, logPerplexity, topicsMatrix, vocabSize, the 
setters, the getters of its 14 params, the Params API and transform) and no 
persistence of its own.

The native pyspark.ml.connect.clustering of Spark Connect Rust has no LDAModel: 
its two LDA models are unrelated classes, each with all those members, so 
isinstance(model, LDAModel) has no native class to hold of, and LDAModel is the 
one clustering class of pyspark.ml 4.3.0 that has no native class.




--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to