Re: Spark job is running infinitely

2015-10-12 Thread Ted Yu
Each vendor provides own monitoring tool. There is: http://ganglia.sourceforge.net/ There is: http://bosun.org/ which I haven't used FYI On Mon, Oct 12, 2015 at 11:25 AM, Saurav Sinha wrote: > Hi Ted, > > Which would you suggest for monitoring service for me. > > Thanks, > Saurav > > On Mo

Re: Spark job is running infinitely

2015-10-12 Thread Saurav Sinha
Hi Ted, Which monitoring service would you suggest for me. Thanks, Saurav On Mon, Oct 12, 2015 at 11:55 PM, Saurav Sinha wrote: > Hi Ted, > > Which would you suggest for monitoring service for me. > > Thanks, > Saurav > > On Mon, Oct 12, 2015 at 11:47 PM, Ted Yu wrote: > >> I would suggest

Re: Spark job is running infinitely

2015-10-12 Thread Saurav Sinha
Hi Ted, Which would you suggest for monitoring service for me. Thanks, Saurav On Mon, Oct 12, 2015 at 11:47 PM, Ted Yu wrote: > I would suggest you install monitoring service. > 'no space left' condition would affect other services, not just Spark. > > For the second part, Spark experts may h

Re: Spark job is running infinitely

2015-10-12 Thread Ted Yu
I would suggest you install monitoring service. 'no space left' condition would affect other services, not just Spark. For the second part, Spark experts may have answer for you. On Mon, Oct 12, 2015 at 11:09 AM, Saurav Sinha wrote: > Hi Ted, > > *Do you have monitoring put in place to detect '

Re: Spark job is running infinitely

2015-10-12 Thread Saurav Sinha
Hi Ted, *Do you have monitoring put in place to detect 'no space left' scenario ?* No, I don't have any monitoring in place. *By 'way to kill job', do you mean automatic kill ?* Yes, I need some way by which my job will detect this failure and kill itself. Thanks, Saurav On Mon, Oct 12, 2015

Re: Spark job is running infinitely

2015-10-12 Thread Ted Yu
Do you have monitoring put in place to detect 'no space left' scenario ? By 'way to kill job', do you mean automatic kill ? Please include the release of Spark, command line for 'spark-submit' in your reply. Thanks On Mon, Oct 12, 2015 at 10:07 AM, Saurav Sinha wrote: > Hi Experts, > > I am f