Hyukjin Kwon created SPARK-61732:
------------------------------------
Summary: Add PySpark's LDAModel, the base class of the native
LocalLDAModel and DistributedLDAModel
Key: SPARK-61732
URL: https://issues.apache.org/jira/browse/SPARK-61732
Project: Spark
Issue Type: Improvement
Components: Connect
Affects Versions: connect-rust-4.3.0
Reporter: Hyukjin Kwon
PySpark's pyspark.ml.clustering has LDAModel, the base class of LocalLDAModel
and DistributedLDAModel (both are (LDAModel, JavaMLReadable, JavaMLWritable)),
with the members of an LDA model (describeTopics, estimatedDocConcentration,
isDistributed, logLikelihood, logPerplexity, topicsMatrix, vocabSize, the
setters, the getters of its 14 params, the Params API and transform) and no
persistence of its own.
The native pyspark.ml.connect.clustering of Spark Connect Rust has no LDAModel:
its two LDA models are unrelated classes, each with all those members, so
isinstance(model, LDAModel) has no native class to hold of, and LDAModel is the
one clustering class of pyspark.ml 4.3.0 that has no native class.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]