Why so many tasks?

2014-12-16 Thread bethesda
://apache-spark-user-list.1001560.n3.nabble.com/Why-so-many-tasks-tp20712.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional

Re: Why so many tasks?

2014-12-16 Thread Koert Kuipers
) 14/12/16 07:40:49 INFO TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 1975 ms on dev-myserver-1.abc.cloud (3/1000) -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-so-many-tasks-tp20712.html Sent from the Apache Spark User List

Re: Why so many tasks?

2014-12-16 Thread Akhil Das
.abc.cloud (3/1000) -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-so-many-tasks-tp20712.html Sent from the Apache Spark User List mailing list archive at Nabble.com

Re: Why so many tasks?

2014-12-16 Thread Gerard Maas
) -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-so-many-tasks-tp20712.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr

Re: Why so many tasks?

2014-12-16 Thread Gen
) 14/12/16 07:40:49 INFO TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 1975 ms on dev-myserver-1.abc.cloud (3/1000) -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-so-many-tasks-tp20712p20715.html Sent from the Apache Spark User List

Re: Why so many tasks?

2014-12-16 Thread bethesda
Thank you! I had known about the small-files problem in HDFS but didn't realize that it affected sc.textFile(). -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-so-many-tasks-tp20712p20717.html Sent from the Apache Spark User List mailing list archive

Re: Why so many tasks?

2014-12-16 Thread Ashish Rangole
this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Why-so-many-tasks-tp20712p20717.html Sent from the Apache Spark User List mailing list archive at Nabble.com. - To unsubscribe, e-mail: user-unsubscr