[
https://issues.apache.org/jira/browse/SPARK-2199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14037659#comment-14037659
]
Valeriy Avanesov commented on SPARK-2199:
-----------------------------------------
Here is the implementation we currently have. https://github.com/akopich/dplsa
Robust and non robust PLSA are implemented but no regularizers are currently
supported.
> Distributed probabilistic latent semantic analysis in MLlib
> -----------------------------------------------------------
>
> Key: SPARK-2199
> URL: https://issues.apache.org/jira/browse/SPARK-2199
> Project: Spark
> Issue Type: Improvement
> Components: MLlib
> Affects Versions: 1.1.0
> Reporter: Denis Turdakov
> Labels: features
>
> Probabilistic latent semantic analysis (PLSA) is a topic model which extracts
> topics from text corpus. PLSA was historically a predecessor of LDA. However
> recent research shows that modifications of PLSA sometimes performs better
> then LDA[1]. Furthermore, the most recent paper by same authors shows that
> there is a clear way to extend PLSA to LDA and beyond[2].
> We should implement distributed versions of PLSA. In addition it should be
> possible to easily add user defined regularizers or combination of them. We
> will implement regularizers that allows
> * extract sparse topics
> * extract human interpretable topics
> * perform semi-supervised training
> * sort out non-topic specific terms.
> [1] Potapenko, K. Vorontsov. 2013. Robust PLSA performs better than LDA. In
> Proceedings of ECIR'13.
> [2] Vorontsov, Potapenko. Tutorial on Probabilistic Topic Modeling: Additive
> Regularization for Stochastic Matrix Factorization.
> http://www.machinelearning.ru/wiki/images/1/1f/Voron14aist.pdf
--
This message was sent by Atlassian JIRA
(v6.2#6252)
