Hi Florian, In some cases where the paragraphs generate JDBC queries, I have managed to > unblock threads by killing the queries in the database.
These paragraphs were using the JDBC interpreter. Can I do anything in my code to close the jdbc connection? I use the normal > spark module to query the jdbc data. It seems like you are using the Spark interpreter. Can you send an example paragraph? How many paragraphs do you have in your note? Do you see any that are not terminated (ready, running or pending status)? Here's <https://github.com/apache/zeppelin/blob/b8755ebb25ad793daa6873acc2e00b8d69588188/zeppelin-interpreter/src/main/java/org/apache/zeppelin/scheduler/Job.java#L162> the code for isTerminated. On Thu, Nov 10, 2016 at 2:37 AM, Florian Schulz <flo_schul...@web.de> wrote: > Hi Prasad, > > thanks for your fast response! I use a lot of JDBC-Queries, so your > assumption is right. Here is the jstack output regarding the scheduler: > > > > "DefaultQuartzScheduler_QuartzSchedulerThread" #32 prio=5 os_prio=31 > tid=0x00007ffee8cd1000 nid=0x7403 in Object.wait() [0x0000700002502000] > > java.lang.Thread.State: TIMED_WAITING (on object monitor) > > at java.lang.Object.wait(Native Method) > > at org.quartz.simpl.SimpleThreadPool.blockForAvailableThreads(S > impleThreadPool.java:452) > > - locked <0x0000000780420438> (a java.lang.Object) > > at org.quartz.core.QuartzSchedulerThread.run(QuartzSchedulerThr > ead.java:263) > > > > "DefaultQuartzScheduler_Worker-10" #31 prio=5 os_prio=31 > tid=0x00007ffee9dee800 nid=0x7203 waiting on condition [0x00007000023ff000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebo > ok.java:645) > > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run( > SimpleThreadPool.java:573) > > - locked <0x0000000780234e60> (a java.lang.Object) > > > > "DefaultQuartzScheduler_Worker-9" #30 prio=5 os_prio=31 > tid=0x00007ffee8cce000 nid=0x7003 waiting on condition [0x00007000022fc000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebo > ok.java:645) > > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run( > SimpleThreadPool.java:573) > > - locked <0x000000078023cbc0> (a java.lang.Object) > > > > "DefaultQuartzScheduler_Worker-8" #29 prio=5 os_prio=31 > tid=0x00007ffee91e5000 nid=0x6e03 waiting on condition [0x00007000021f9000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebo > ok.java:645) > > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run( > SimpleThreadPool.java:573) > > - locked <0x00000007802441b8> (a java.lang.Object) > > > > "DefaultQuartzScheduler_Worker-7" #28 prio=5 os_prio=31 > tid=0x00007ffee91e4000 nid=0x6c03 waiting on condition [0x00007000020f6000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebo > ok.java:645) > > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run( > SimpleThreadPool.java:573) > > - locked <0x0000000780235000> (a java.lang.Object) > > > > "DefaultQuartzScheduler_Worker-6" #27 prio=5 os_prio=31 > tid=0x00007ffee9dfc800 nid=0x6a03 waiting on condition [0x0000700001ff3000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebo > ok.java:645) > > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run( > SimpleThreadPool.java:573) > > - locked <0x00000007802351a0> (a java.lang.Object) > > > > "DefaultQuartzScheduler_Worker-5" #26 prio=5 os_prio=31 > tid=0x00007ffee92f4800 nid=0x6803 waiting on condition [0x0000700001ef0000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebo > ok.java:645) > > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run( > SimpleThreadPool.java:573) > > - locked <0x0000000780244358> (a java.lang.Object) > > > > "DefaultQuartzScheduler_Worker-4" #25 prio=5 os_prio=31 > tid=0x00007ffee92ef000 nid=0x6603 waiting on condition [0x0000700001ded000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebo > ok.java:645) > > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run( > SimpleThreadPool.java:573) > > - locked <0x0000000780235340> (a java.lang.Object) > > > > "DefaultQuartzScheduler_Worker-3" #24 prio=5 os_prio=31 > tid=0x00007ffee8ce7000 nid=0x6403 waiting on condition [0x0000700001cea000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebo > ok.java:645) > > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run( > SimpleThreadPool.java:573) > > - locked <0x00000007802444f8> (a java.lang.Object) > > > > "DefaultQuartzScheduler_Worker-2" #23 prio=5 os_prio=31 > tid=0x00007ffee8cee800 nid=0x6203 waiting on condition [0x0000700001be7000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebo > ok.java:645) > > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run( > SimpleThreadPool.java:573) > > - locked <0x0000000780235548> (a java.lang.Object) > > > > "DefaultQuartzScheduler_Worker-1" #22 prio=5 os_prio=31 > tid=0x00007ffee8c69000 nid=0x6003 waiting on condition [0x0000700001ae4000] > > java.lang.Thread.State: TIMED_WAITING (sleeping) > > at java.lang.Thread.sleep(Native Method) > > at org.apache.zeppelin.notebook.Notebook$CronJob.execute(Notebo > ok.java:645) > > at org.quartz.core.JobRunShell.run(JobRunShell.java:202) > > at org.quartz.simpl.SimpleThreadPool$WorkerThread.run( > SimpleThreadPool.java:573) > > - locked <0x000000078023cd60> (a java.lang.Object) > > > > > > Can I do anything in my code to close the jdbc connection? I use the > normal spark module to query the jdbc data. > > > > > > Best, > > Florian > > > > > > On 2016-11-09 21:12 (+0100), Prasad Wagle <p...@gmail.com> wrote: > > > Hi Florian,> > > > > > > We have seen this behavior when the 10 quartz scheduler worker threads > are> > > > waiting for some note paragraph to complete at this statement> > > > <https://github.com/apache/zeppelin/blob/v0.6.2/zeppelin-zen > gine/src/main/java/org/apache/zeppelin/notebook/Notebook.java#L645>.> > > > You can see what the quartz scheduler worker threads are doing in the> > > > jstack output of the ZeppelinServer process (look for> > > > DefaultQuartzScheduler_Worker). Can you post that part of the jstack> > > > output?> > > > > > > In some cases where the paragraphs generate JDBC queries, I have managed > to> > > > unblock threads by killing the queries in the database.> > > > > > > Regards,> > > > Prasad> > > > > > > > > > > > > On Wed, Nov 9, 2016 at 1:41 AM, Florian Schulz <fl...@web.de> wrote:> > > > > > > > Hi,> > > > >> > > > > I have a problem with the scheduler. I have a notebook and I execute > the> > > > > spark code in it every minute (with the cron scheduler). The > scheduler> > > > > starts ten times and then it stops and do nothing anymore (in no > notebook> > > > > at all). I use version 0.6.2 of Apache Zeppelin. Do you have any idea, > why> > > > > this happens? I can reproduce this without a problem, but I can't > find> > > > > anything in the logs. Hope you can help me.> > > > >> > > > >> > > > > Best regards> > > > > Florian> > > > >> > > > >
