No topicDistributions(..) method in ml.clustering.LocalLDAModel

2017-02-07 Thread sachintyagi22
Hi, I was using ml.clustering.LDA for topic modelling (with online optimizer) and it returns ml.clustering.LocalLDAModel. However, using this model there doesn't seem to be any way to get the topic distribution over documents. While older mllib API (mllib.clustering.LocalLDAModel ) does have the

Checkpointing in Iterative Graph Computation

2015-08-25 Thread sachintyagi22
Hi, I have stumbled upon an issue with iterative Graphx computation (using v 1.4.1). It goes thusly -- Setup 1. Construct a graph. 2. Validate that the graph satisfies certain conditions. Here I do some assert(*conditions*) within graph.triplets.foreach(). [Notice that this materializes the