The problem is that SparkPi uses Math.random(), which is a synchronized method,
so it can’t scale to multiple cores. In fact it will be slower on multiple
cores due to lock contention. Try another example and you’ll see better
scaling. I think we’ll have to update SparkPi to create a new Random in each
task to avoid this.
Matei
On Apr 24, 2014, at 4:43 AM, Adnan wrote:
> Hi,
>
> Relatively new on spark and have tried running SparkPi example on a
> standalone 12 core three machine cluster. What I'm failing to understand is,
> that running this example with a single slice gives better performance as
> compared to using 12 slices. Same was the case when I was using parallelize
> function. The time is scaling almost linearly with adding each slice. Please
> let me know if I'm doing anything wrong. The code snippet is given below:
>
>
>
> Regards,
>
> Ahsan Ijaz
>
>
>
> --
> View this message in context:
> http://apache-spark-user-list.1001560.n3.nabble.com/SparkPi-performance-3-cluster-standalone-mode-tp4530p4751.html
> Sent from the Apache Spark User List mailing list archive at Nabble.com.
