Hello, We just published the Spectral LDA model on Spark Packages. It’s an alternative approach to the LDA modelling based on tensor decompositions. We first build the 2nd, 3rd-moment tensors from empirical word counts, then orthogonalise them and perform decomposition on the 3rd-moment tensor. The convergence is guaranteed by theory, in contrast to most current approaches. We achieve comparable log-perplexity in much shorter running time.
You could find the package at https://spark-packages.org/package/FurongHuang/SpectralLDA-TensorSpark <https://spark-packages.org/package/FurongHuang/SpectralLDA-TensorSpark> We’d welcome any thoughts or feedback on it. Thanks very much, Furong Huang Jencir Lee Anima Anandkumar
