Re: takeSample triggers 2 jobs

2015-03-06 Thread Denny Lee
Hi Rares, If you dig into the descriptions for the two jobs, it will probably return something like: Job ID: 1 org.apache.spark.rdd.RDD.takeSample(RDD.scala:447) $line41.$read$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC$$iwC.init(console:22) ... Job ID: 0

takeSample triggers 2 jobs

2015-03-06 Thread Rares Vernica
Hello, I am using takeSample from the Scala Spark 1.2.1 shell: scala sc.textFile(README.md).takeSample(false, 3) and I notice that two jobs are generated on the Spark Jobs page: Job Id Description 1 takeSample at console:13 0 takeSample at console:13 Any ideas why the two jobs are needed?