Hi, The spark sql perf itself contains benchmark data generation. I am using spark shell to run the spark sql perf to generate the data with 10G memory for both driver and executor. When I increase the scalefactor to be 30,and run the job, Then I got the following error:
When I jstack it to see the status of the thread. I see the following: looks it is waiting for the process that the spark job kicks off.