Just wanted to add a comment to the Jira ticket but I don't think I have
permission to do so, so answering here instead. I am encountering the same
issue with a stackOverflow Exception.
I would like to point out that there is a
localCheckpoint
.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/StackOverflow-Error-when-run-ALS-with-100-iterations-tp4296p22619.html
Sent from the Apache Spark User List mailing list archive at Nabble.com
Hi, would you please how to checkpoint the training set rdd since all things
are done in ALS.train method.
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/StackOverflow-Error-when-run-ALS-with-100-iterations-tp4296p22619.html
Sent from the Apache Spark User
I'd also say that running for 100 iterations is a waste of resources, as
ALS will typically converge pretty quickly, as in within 10-20 iterations.
On Wed, Apr 16, 2014 at 3:54 AM, Xiaoli Li lixiaolima...@gmail.com wrote:
Thanks a lot for your information. It really helps me.
On Tue, Apr
Hi,
I am testing ALS using 7 nodes. Each node has 4 cores and 8G memeory. ALS
program cannot run even with a very small size of training data (about 91
lines) due to StackVverFlow error when I set the number of iterations to
100. I think the problem may be caused by updateFeatures method which
Probably this JIRA
issuehttps://spark-project.atlassian.net/browse/SPARK-1006solves
your problem. When running with large iteration number, the lineage
DAG of ALS becomes very deep, both DAGScheduler and Java serializer may
overflow because they are implemented in a recursive way. You may resort
Thanks a lot for your information. It really helps me.
On Tue, Apr 15, 2014 at 7:57 PM, Cheng Lian lian.cs@gmail.com wrote:
Probably this JIRA
issuehttps://spark-project.atlassian.net/browse/SPARK-1006solves your
problem. When running with large iteration number, the lineage
DAG of
