The problem could be not only the resource, but the session. If you run a chunk of spark code and you should see the a running application in the spark UI, but in your code if you shut it down after the job is finished, then on the spark UI you will see the hob is finished. Within zeppelin, each job will start the spark session only once(different interpreter mode could be set if you want notebooks to share the session or not), if you closed it ,it will never restart it again. The only way to get the same code work again is to restat the interpreter or restart zeppelin. I`m not sure if I explain clearly, but hope it could help From: Paul Brenner Date: 2017-02-17 12:14 To: users Subject: Re: Re: Zeppelin unable to respond after some time I’ve definitely had this problem with jobs that don’t take all the resources on the cluster. Also, my experience matches what others have reported: just restarting zeppelin and re-runing the stuck paragraph solves the issue.
I’ve also experienced this problem with for loops. Some for loops which write to disk but absolutely don’t have any variables that are increasing in size will hang in Zeppelin. If I run the exact same code in the scala REPL it goes through without problem. Paul Brenner DATA SCIENTIST (217) 390-3033 On Fri, Feb 17, 2017 at 2:12 PM "xyun...@simuwell.com" <">"xyun...@simuwell.com" > wrote: I have solved the similar issue before. You should check on spark UI and probably you will see your single job is taking all the resources. Therefore further job that submitting to the same cluster will just hang on there. When you restart zeppelin then the old job is killed and all the resource it took will be released xyun...@simuwell.com From: RUSHIKESH RAUT Date: 2017-02-17 02:29 To: users Subject: Re: Zeppelin unable to respond after some time Yes happens with r and spark codes frequently On Feb 17, 2017 3:25 PM, "小野圭二" <onoke...@gmail.com> wrote: yes, almost every time. There are not any special operations. Just run the tutorial demos. From my feeling, it happens in R demo frequently. 2017-02-17 18:50 GMT+09:00 Jeff Zhang <zjf...@gmail.com>: Is it easy to reproduce it ? 小野圭二 <onoke...@gmail.com>于2017年2月17日周五 下午5:47写道： I am facing on the same issue now. 2017-02-17 18:25 GMT+09:00 RUSHIKESH RAUT <rushikeshraut...@gmail.com>: Hi all, I am facing a issue while using Zeppelin. I am trying to load some data(not that big data) into Zeppelin and then build some visualization on it. The problem is that when I try to run the code first time it's working but after some time the same code doesn't work. It remains in running state on gui, but no logs are generated in Zeppelin logs. Also all further tasks are hanging in pending state. As soon as I restart Zeppelin it works. So I am guessing it's some memory issue. I have read that Zeppelin stores the data in memory so it is possible that it runs out of memory after some time. How do I debug this issue? How much is the default memory that Zeppelin takes at start? Also is there any way that I can run Zeppelin with specified memory so that I can start the process with more memory. Because it doesn't make sense to restart Zeppelin after every half hour Thanks, Rushikesh Raut
