Hi,
I have some iterative program written in Spark and have been tested under a
snapshot version of spark 0.8 before. After I ported it to the 0.8 release
version, I see performance drops in large datasets. I am wondering if
there is any clue?
I monitored the number of partitions on each machine
I don't even think task stealing / speculative execution is turned on by
default. Do you know what snapshot version you used for 0.8 previously?
On Mon, Nov 4, 2013 at 12:03 PM, Wenlei Xie wrote:
> Hi,
>
> I have some iterative program written in Spark and have been tested under
> a snapshot ve
Hi,
I have all the code for the previous 0.8 version. But how I can find out
the SNAPSHOT version there? (in project/SparkBuild.scala it just
says version := "0.8.0-SNAPSHOT)
Best,
Wenlei
On Wed, Nov 6, 2013 at 12:09 AM, Reynold Xin wrote:
> I don't even think task stealing / speculative exec
