The partitioner is not saved with the RDD. So when you load the model back, we lose the partitioner information. You can call repartition on the user/product factors and then create a new MatrixFactorizationModel object using the repartitioned RDDs. It would be useful to create a utility method for this, e.g., `MatrixFactorizationModel.repartition(num: Int): MatrixFactorizationModel`. -Xiangrui
On Wed, Jul 22, 2015 at 4:34 AM, PShestov <pshes...@nvidia.com> wrote: > Hi all! > I have MatrixFactorizationModel object. If I'm trying to recommend products > to single user right after constructing model through ALS.train(...) then it > takes 300ms (for my data and hardware). But if I save model to disk and load > it back then recommendation takes almost 2000ms. Also Spark warns: > 15/07/17 11:05:47 WARN MatrixFactorizationModel: User factor does not have a > partitioner. Prediction on individual records could be slow. > 15/07/17 11:05:47 WARN MatrixFactorizationModel: User factor is not cached. > Prediction could be slow. > 15/07/17 11:05:47 WARN MatrixFactorizationModel: Product factor does not > have a partitioner. Prediction on individual records could be slow. > 15/07/17 11:05:47 WARN MatrixFactorizationModel: Product factor is not > cached. Prediction could be slow. > How can I create/set partitioner and cache user and product factors after > loading model? Following approach didn't help: > model.userFeatures().cache(); > model.productFeatures().cache(); > Also I was trying to repartition those rdds and create new model from > repartitioned versions but that also didn't help. > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/Proper-saving-loading-of-MatrixFactorizationModel-tp23952.html > Sent from the Apache Spark User List mailing list archive at Nabble.com. > > --------------------------------------------------------------------- > To unsubscribe, e-mail: user-unsubscr...@spark.apache.org > For additional commands, e-mail: user-h...@spark.apache.org > --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
