Re: How to tunning my spark application.

2016-01-17 Thread
unt of computation in sampleArray() can be reduced. > > Which Spark release are you using ? > > Thanks > >> On Sun, Jan 17, 2016 at 6:22 AM, 张峻 <julian_do...@me.com> wrote: >> Dear All >> >> I used jProfiler

How to tunning my spark application.

2016-01-17 Thread
Dear All I used jProfiler to profiling my spark application. And I had find more than 70% cpu is used by the org.apache.spark.util.SizeEstimator class. There call tree is as blow. java.lang.Thread.run --scala.collection.immutable.Range.foreach$mVc$sp