Hey, I just reread your question and saw I overlooked some crucial information. Here's a solution:
val data = model.asInstanceOf[DistributedLDAModel].topicDistributions.sortByKey().collect() val tpdist = data.map(doc => doc._2.toArray) hope it works this time On 11 January 2016 at 17:14, Jakob Odersky <joder...@gmail.com> wrote: > Hi Daniel, > > You're actually not modifying the original array: `array :+ x ` will give > you a new array with `x` appended to it. > In your case the fix is simple: collect() already returns an array, use it > as the assignment value to your val. > > In case you ever want to append values iteratively, search for how to use > scala "ArrayBuffer"s. Also, keep in mind that RDDs have a foreach method, > so no need to call collect followed by foreach. > > regards, > --Jakob > > > > On 11 January 2016 at 16:55, Daniel Valdivia <h...@danielvaldivia.com> > wrote: > >> Hello, >> >> I'm trying to put all the values in pair rdd into an array (or list) for >> later storing, however even if I'm collecting the data then pushing it to >> the array the array size after the run is 0. >> >> Any idea on what I'm missing? >> >> Thanks in advance >> >> scala> val tpdist: Array[Array[Double]] = Array() >> tpdist: Array[Array[Double]] = Array() >> >> scala> >> ldaModel.asInstanceOf[DistributedLDAModel].topicDistributions.sortByKey().collect().foreach(doc >> => tpdist :+ doc._2.toArray ) >> >> >> >> scala> tpdist.size >> res27: Int = 0 >> > >
