Hi thanks for the response. Each Job is processing around 5gb of skewed data does group by multiple fields and does aggregation and does coalesce(1) and saves csv file in gzip format. I think coalesce is causing problem but data is not that huge I don't understand why it keeps on running for an hour and avoiding other jobs to run. Please guide. On Jan 7, 2016 3:58 AM, "Jakob Odersky" <joder...@gmail.com> wrote:
> What is the job doing? How much data are you processing? > > On 6 January 2016 at 10:33, unk1102 <umesh.ka...@gmail.com> wrote: > >> Hi I have one main Spark job which spawns multiple child spark jobs. One >> of >> the child spark job is running for an hour and it keeps on hanging there I >> have taken snap shot please see >> < >> http://apache-spark-user-list.1001560.n3.nabble.com/file/n25899/Screen_Shot_2016-01-06_at_11.jpg >> > >> >> >> >> -- >> View this message in context: >> http://apache-spark-user-list.1001560.n3.nabble.com/Why-is-this-job-running-since-one-hour-tp25899.html >> Sent from the Apache Spark User List mailing list archive at Nabble.com. >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: user-unsubscr...@spark.apache.org >> For additional commands, e-mail: user-h...@spark.apache.org >> >> >
