Hello, I am new to spark and I am currently learning how to use classification algorithm with it. For now on I am playing with a rather small dataset and training a decision tree on my laptop (running with --master local[1]). However, systematically I see that my jobs are hanging forever at the training stage. Looking at the application UI, I see that my ongoing stage as still 15 out of 16 task to do. But none are scheduled and looking at the executors tab I see that my single executor is indeed not processing anything (even if executor memory is still fine). <http://apache-spark-user-list.1001560.n3.nabble.com/file/n26922/Stage.png> <http://apache-spark-user-list.1001560.n3.nabble.com/file/n26922/Executor.png>
If I ask the details of the hanging job I see: org.apache.spark.rdd.RDD.count(RDD.scala:1125) org.apache.spark.mllib.tree.impl.DecisionTreeMetadata$.buildMetadata(DecisionTreeMetadata.scala:114) org.apache.spark.ml.tree.impl.RandomForest$.run(RandomForest.scala:65) org.apache.spark.ml.classification.DecisionTreeClassifier.train(DecisionTreeClassifier.scala:77) org.apache.spark.ml.classification.DecisionTreeClassifier.train(DecisionTreeClassifier.scala:40) Any idea of what my problem could be? My code is compiled with spark 1.6.1 Thanks in advance, Loic -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-hanging-forever-when-doing-decision-tree-training-tp26922.html Sent from the Apache Spark User List mailing list archive at Nabble.com. --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
