Dear All I used jProfiler to profiling my spark application. And I had find more than 70% cpu is used by the org.apache.spark.util.SizeEstimator class.
There call tree is as blow. java.lang.Thread.run --scala.collection.immutable.Range.foreach$mVc$sp ----org.apache.spark.util.SizeEstimator$$anonfun$sampleArray$1.apply$mcVI$sp ------scala.collection.immutable.List.foreach --------org.apache.spark.util.SizeEstimator$$anonfun$visitSingleObject$1.apply --scala.collection.immutable.List.foreach ----org.apache.spark.util.SizeEstimator$$anonfun$visitSingleObject$1.apply My code don’t show in this two biggest branch of the call tree. I want to know what will cause spark to spend so many time in “Range.foreach” or “.List.foreach” Any one can give me some tips? BR Julian Zhang --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscr...@spark.apache.org For additional commands, e-mail: user-h...@spark.apache.org
