Github user debasish83 commented on the pull request: https://github.com/apache/spark/pull/1290#issuecomment-62247730 Our sparse data very often has ~10M users and 1M features...for 1 hidden layer net with 10K nodes, we need 10M x 10K + 1M x 10K doubles...if the model is not distributed (like ALS design) it's not possible to put such complicated model into memory of one node...having said that it makes sense to have a baseline implementation so that it can be used as a reference for further enhancements...also as a auto encoder, neural nets should generate better results than sparse coding (which we can do in mllib through the PR) due to the non-linearity of hidden units...
--- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. --- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org