Re: How to deal with tasks running too long?

2016-06-16 Thread Utkarsh Sengar
laskowski/ > Mastering Apache Spark http://bit.ly/mastering-apache-spark > Follow me at https://twitter.com/jaceklaskowski > > > On Thu, Jun 16, 2016 at 6:45 AM, Utkarsh Sengar <utkarsh2...@gmail.com> > wrote: > > This SO question was asked about 1yr ago. > > > http:

Re: How to deal with tasks running too long?

2016-06-16 Thread Jacek Laskowski
sh2...@gmail.com> wrote: > This SO question was asked about 1yr ago. > http://stackoverflow.com/questions/31799755/how-to-deal-with-tasks-running-too-long-comparing-to-others-in-job-in-yarn-cli > > I answered this question with a suggestion to try speculation but it doesn't > quite do what t

Re: How to deal with tasks running too long?

2016-06-16 Thread Jeff Zhang
This may be due to data skew On Thu, Jun 16, 2016 at 12:45 PM, Utkarsh Sengar <utkarsh2...@gmail.com> wrote: > This SO question was asked about 1yr ago. > > http://stackoverflow.com/questions/31799755/how-to-deal-with-tasks-running-too-long-comparing-to-others-in-job-in-yarn-cli

How to deal with tasks running too long?

2016-06-15 Thread Utkarsh Sengar
This SO question was asked about 1yr ago. http://stackoverflow.com/questions/31799755/how-to-deal-with-tasks-running-too-long-comparing-to-others-in-job-in-yarn-cli I answered this question with a suggestion to try speculation but it doesn't quite do what the OP expects. I have been running