[ https://issues.apache.org/jira/browse/SPARK-3066?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14351945#comment-14351945 ]
Debasish Das commented on SPARK-3066: ------------------------------------- [~josephkb] do you mean knn ? For recommendation until you do the dot product I am not sure how can you find topk..level 3 BLAS will definite give a big boost since it's all blocked dense with dense multiplication...For https://issues.apache.org/jira/browse/SPARK-4823 I am looking into dense dense BLAS and dense sparse BLAS..ideally there we can add in a knn based optimization followed by row similarity calculation > Support recommendAll in matrix factorization model > -------------------------------------------------- > > Key: SPARK-3066 > URL: https://issues.apache.org/jira/browse/SPARK-3066 > Project: Spark > Issue Type: New Feature > Components: MLlib > Reporter: Xiangrui Meng > Assignee: Debasish Das > > ALS returns a matrix factorization model, which we can use to predict ratings > for individual queries as well as small batches. In practice, users may want > to compute top-k recommendations offline for all users. It is very expensive > but a common problem. We can do some optimization like > 1) collect one side (either user or product) and broadcast it as a matrix > 2) use level-3 BLAS to compute inner products > 3) use Utils.takeOrdered to find top-k -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org