Thanks Ted Yu.
Following are the error message: 1. The exception that is shown on the UI is : Exception in thread "Thread-113" Exception in thread "Thread-126" Exception in thread "Thread-64" Exception in thread "Thread-90" Exception in thread "Thread-117" Exception in thread "Thread-80" Exception in thread "Thread-115" Exception in thread "ResponseProcessor for block BP-1564562096-172.18.149.132-1435294011279:blk_1073846767_105984" Exception in thread "qtp1270119920-57" Exception in thread "Thread-77" Exception in thread "Thread-132" Exception in thread "Thread-68" Exception in thread "Thread-61" Exception in thread "Thread-70" Exception in thread "qtp1270119920-52" Exception in thread "Thread-88" Exception in thread "qtp318933312-47" Exception in thread "qtp1270119920-56" 2. jstack the process, I see bunch of following message: Thread 31258: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - java.lang.Object.wait() @bci=2, line=503 (Interpreted frame) - java.lang.UNIXProcess.waitFor() @bci=8, line=263 (Interpreted frame) - scala.sys.process.ProcessImpl$SimpleProcess.exitValue() @bci=4, line=218 (Interpreted frame) - scala.sys.process.ProcessBuilderImpl$AbstractBuilder$$anonfun$lines$1.apply$mcV$sp() @bci=11, line=142 (Interpreted frame) - scala.sys.process.ProcessImpl$Spawn$$anon$1.run() @bci=4, line=22 (Interpreted frame) Thread 31257: (state = BLOCKED) - java.lang.Object.wait(long) @bci=0 (Interpreted frame) - java.lang.Object.wait() @bci=2, line=503 (Interpreted frame) - java.lang.UNIXProcess.waitFor() @bci=8, line=263 (Interpreted frame) - scala.sys.process.ProcessImpl$SimpleProcess.exitValue() @bci=4, line=218 (Interpreted frame) - scala.sys.process.ProcessBuilderImpl$AbstractBuilder$$anonfun$lines$1.apply$mcV$sp() @bci=11, line=142 (Interpreted frame) - scala.sys.process.ProcessImpl$Spawn$$anon$1.run() @bci=4, line=22 (Interpreted frame) At 2015-08-25 19:32:56, "Ted Yu" <yuzhih...@gmail.com> wrote: Looks like you were attaching images to your email which didn't go through. Consider using third party site for images - or paste error in text. Cheers On Tue, Aug 25, 2015 at 4:22 AM, Todd <bit1...@163.com> wrote: Hi， The spark sql perf itself contains benchmark data generation. I am using spark shell to run the spark sql perf to generate the data with 10G memory for both driver and executor. When I increase the scalefactor to be 30,and run the job, Then I got the following error: When I jstack it to see the status of the thread. I see the following: looks it is waiting for the process that the spark job kicks off.
