RE: Channel closed exception
Ohh..Sorry about that, Could you please try again? Or I'll put it in github https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ Thanks, Sudip -Original Message- From: Sudheesh Katkam [mailto:skat...@maprtech.com] Sent: 02 October 2015 AM 02:04 To: dev@drill.apache.org Subject: Re: Channel closed exception I cannot open the shared documents (“Whoops! There was a problem previewing this document.”) nor can I download them ("We're sorry, but you do not have access to this page.”). The folder link in this thread is also broken (“The open folder can no longer be viewed”). - Sudheesh > On Oct 1, 2015, at 12:43 PM, Sudip Mukherjee <smukher...@commvault.com> wrote: > > I uploaded to google drive. Should it be okay? > > https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ > > Thanks, > Sudip > > -Original Message- > From: Sudheesh Katkam [mailto:skat...@maprtech.com] > Sent: 01 October 2015 AM 10:01 > To: dev@drill.apache.org > Subject: Re: Channel closed exception > > Ah there, I assumed you were not on Windows. We do not have the drillbit.out > file for Windows (I think). > > Anyway, the error messages you see are symptoms of the real issue, which is > most likely an “out of memory” scenario. I think that the drillbit did not > crash either (in which case you might see a JVM core dump). Instead the > drillbit gracefully shutdown unable to handle the error. Could you share the > query (if possible), data set details, and configuration? > > It would also help if you put the entire log files somewhere (Github gist or > Dropbox) and shared the links. > > - Sudheesh > >> On Sep 30, 2015, at 9:10 PM, Sudip Mukherjee <smukher...@commvault.com> >> wrote: >> >> Yes. I was running drill in embedded mode to try it out with 3.9 >> lakhs records , near about 50+ columns.[ sqlline.bat -u >> "jdbc:drill:zk=local" ] I see sqlline.log and sqlline_queries.log under log >> folder. Did I miss some steps? >> >> But was able run queries from the command prompt after that >> >> Thanks, >> Sudip >> >> -Original Message- >> From: Sudheesh Katkam [mailto:skat...@maprtech.com] >> Sent: 01 October 2015 AM 09:24 >> To: dev@drill.apache.org >> Subject: Re: Channel closed exception >> >> Are you running drill in embedded mode? The directory containing the >> drillbit.log file contains the drillbit.out file too. There might be a JVM >> core dump in the /tmp directory. >> >> Usually "Received shutdown request” message means that’s the drillbit ran >> out of memory. Given that you mentioned the query is on a large document >> set, this scenario is likely. >> >> - Sudheesh >> >>> On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee <smukher...@commvault.com> >>> wrote: >>> >>> Hi, >>> This is the excerpt of exception in sqlline.log. Could you please refer me >>> to the location of drillbit.out file? >>> >>> >>> 2015-09-30 22:23:36,286 [BitServer-4] WARN >>> o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type >>> 6 took longer than 500ms. Actual duration was 16047ms. >>> 2015-09-30 22:24:13,581 >>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO >>> o.a.d.e.w.fragment.FragmentExecutor - >>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested >>> CANCELLATION_REQUESTED --> FINISHED >>> 2015-09-30 22:24:13,581 >>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO >>> o.a.d.e.w.f.FragmentStatusReporter - >>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: CANCELLED >>> 2015-09-30 22:24:13,581 [BitServer-4] INFO >>> o.a.drill.exec.work.foreman.Foreman - State change requested. >>> CANCELLATION_REQUESTED --> COMPLETED >>> 2015-09-30 22:24:13,581 [BitServer-4] INFO >>> o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. >>> 2015-09-30 22:24:13,597 [UserServer-1] INFO >>> o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate >>> query result to initiating client. This would happen if a client is >>> disconnected before response notice can be sent. >>> org.apache.drill.exec.rpc.ChannelClosedException: null >>> at >>> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89) >>> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] >>> at >>> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComp
RE: Channel closed exception
I uploaded to google drive. Should it be okay? https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ Thanks, Sudip -Original Message- From: Sudheesh Katkam [mailto:skat...@maprtech.com] Sent: 01 October 2015 AM 10:01 To: dev@drill.apache.org Subject: Re: Channel closed exception Ah there, I assumed you were not on Windows. We do not have the drillbit.out file for Windows (I think). Anyway, the error messages you see are symptoms of the real issue, which is most likely an “out of memory” scenario. I think that the drillbit did not crash either (in which case you might see a JVM core dump). Instead the drillbit gracefully shutdown unable to handle the error. Could you share the query (if possible), data set details, and configuration? It would also help if you put the entire log files somewhere (Github gist or Dropbox) and shared the links. - Sudheesh > On Sep 30, 2015, at 9:10 PM, Sudip Mukherjee <smukher...@commvault.com> wrote: > > Yes. I was running drill in embedded mode to try it out with 3.9 > lakhs records , near about 50+ columns.[ sqlline.bat -u "jdbc:drill:zk=local" > ] I see sqlline.log and sqlline_queries.log under log folder. Did I miss some > steps? > > But was able run queries from the command prompt after that > > Thanks, > Sudip > > -Original Message- > From: Sudheesh Katkam [mailto:skat...@maprtech.com] > Sent: 01 October 2015 AM 09:24 > To: dev@drill.apache.org > Subject: Re: Channel closed exception > > Are you running drill in embedded mode? The directory containing the > drillbit.log file contains the drillbit.out file too. There might be a JVM > core dump in the /tmp directory. > > Usually "Received shutdown request” message means that’s the drillbit ran out > of memory. Given that you mentioned the query is on a large document set, > this scenario is likely. > > - Sudheesh > >> On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee <smukher...@commvault.com> >> wrote: >> >> Hi, >> This is the excerpt of exception in sqlline.log. Could you please refer me >> to the location of drillbit.out file? >> >> >> 2015-09-30 22:23:36,286 [BitServer-4] WARN >> o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type 6 >> took longer than 500ms. Actual duration was 16047ms. >> 2015-09-30 22:24:13,581 >> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO >> o.a.d.e.w.fragment.FragmentExecutor - >> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested >> CANCELLATION_REQUESTED --> FINISHED >> 2015-09-30 22:24:13,581 >> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO >> o.a.d.e.w.f.FragmentStatusReporter - >> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: CANCELLED >> 2015-09-30 22:24:13,581 [BitServer-4] INFO >> o.a.drill.exec.work.foreman.Foreman - State change requested. >> CANCELLATION_REQUESTED --> COMPLETED >> 2015-09-30 22:24:13,581 [BitServer-4] INFO >> o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. >> 2015-09-30 22:24:13,597 [UserServer-1] INFO >> o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate query >> result to initiating client. This would happen if a client is disconnected >> before response notice can be sent. >> org.apache.drill.exec.rpc.ChannelClosedException: null >> at >> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89) >> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] >> at >> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67) >> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] >> at >> io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680) >> [netty-common-4.0.27.Final.jar:4.0.27.Final] >> at >> io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603) >> [netty-common-4.0.27.Final.jar:4.0.27.Final] >> at >> io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563) >> [netty-common-4.0.27.Final.jar:4.0.27.Final] >> at >> io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.jav >> a >> :424) [netty-common-4.0.27.Final.jar:4.0.27.Final] >> >> Thanks, >> Sudip >> >> -Original Message- >> From: Sudheesh Katkam [mailto:skat...@maprtech.com] >> Sent: 01 October 2015 AM 09:09 >> To: dev@drill.apache.org >> Subject: Re: Channel closed exception >> >> Can you check if there are any error messages in the drillbit.out f
Re: Channel closed exception
I cannot open the shared documents (“Whoops! There was a problem previewing this document.”) nor can I download them ("We're sorry, but you do not have access to this page.”). The folder link in this thread is also broken (“The open folder can no longer be viewed”). - Sudheesh > On Oct 1, 2015, at 12:43 PM, Sudip Mukherjee <smukher...@commvault.com> wrote: > > I uploaded to google drive. Should it be okay? > > https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ > > Thanks, > Sudip > > -Original Message- > From: Sudheesh Katkam [mailto:skat...@maprtech.com] > Sent: 01 October 2015 AM 10:01 > To: dev@drill.apache.org > Subject: Re: Channel closed exception > > Ah there, I assumed you were not on Windows. We do not have the drillbit.out > file for Windows (I think). > > Anyway, the error messages you see are symptoms of the real issue, which is > most likely an “out of memory” scenario. I think that the drillbit did not > crash either (in which case you might see a JVM core dump). Instead the > drillbit gracefully shutdown unable to handle the error. Could you share the > query (if possible), data set details, and configuration? > > It would also help if you put the entire log files somewhere (Github gist or > Dropbox) and shared the links. > > - Sudheesh > >> On Sep 30, 2015, at 9:10 PM, Sudip Mukherjee <smukher...@commvault.com> >> wrote: >> >> Yes. I was running drill in embedded mode to try it out with 3.9 >> lakhs records , near about 50+ columns.[ sqlline.bat -u >> "jdbc:drill:zk=local" ] I see sqlline.log and sqlline_queries.log under log >> folder. Did I miss some steps? >> >> But was able run queries from the command prompt after that >> >> Thanks, >> Sudip >> >> -Original Message- >> From: Sudheesh Katkam [mailto:skat...@maprtech.com] >> Sent: 01 October 2015 AM 09:24 >> To: dev@drill.apache.org >> Subject: Re: Channel closed exception >> >> Are you running drill in embedded mode? The directory containing the >> drillbit.log file contains the drillbit.out file too. There might be a JVM >> core dump in the /tmp directory. >> >> Usually "Received shutdown request” message means that’s the drillbit ran >> out of memory. Given that you mentioned the query is on a large document >> set, this scenario is likely. >> >> - Sudheesh >> >>> On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee <smukher...@commvault.com> >>> wrote: >>> >>> Hi, >>> This is the excerpt of exception in sqlline.log. Could you please refer me >>> to the location of drillbit.out file? >>> >>> >>> 2015-09-30 22:23:36,286 [BitServer-4] WARN >>> o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type >>> 6 took longer than 500ms. Actual duration was 16047ms. >>> 2015-09-30 22:24:13,581 >>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO >>> o.a.d.e.w.fragment.FragmentExecutor - >>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested >>> CANCELLATION_REQUESTED --> FINISHED >>> 2015-09-30 22:24:13,581 >>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO >>> o.a.d.e.w.f.FragmentStatusReporter - >>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: CANCELLED >>> 2015-09-30 22:24:13,581 [BitServer-4] INFO >>> o.a.drill.exec.work.foreman.Foreman - State change requested. >>> CANCELLATION_REQUESTED --> COMPLETED >>> 2015-09-30 22:24:13,581 [BitServer-4] INFO >>> o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. >>> 2015-09-30 22:24:13,597 [UserServer-1] INFO >>> o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate >>> query result to initiating client. This would happen if a client is >>> disconnected before response notice can be sent. >>> org.apache.drill.exec.rpc.ChannelClosedException: null >>> at >>> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89) >>> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] >>> at >>> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67) >>> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] >>> at >>> io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680) >>> [netty-common-4.0.27.Final.jar:4.0.27.Final] >>> at >>> io.netty.util.concurrent.Defau
RE: Channel closed exception
Uploaded the entire log. Thanks, Sudip -Original Message- From: Sudheesh Katkam [mailto:skat...@maprtech.com] Sent: 02 October 2015 AM 02:40 To: dev@drill.apache.org Subject: Re: Channel closed exception Could you share the entire log file? - Sudheesh > On Oct 1, 2015, at 1:57 PM, Sudip Mukherjee <smukher...@commvault.com> wrote: > > Ohh..Sorry about that, > Could you please try again? Or I'll put it in github > https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ > > Thanks, > Sudip > > -Original Message- > From: Sudheesh Katkam [mailto:skat...@maprtech.com] > Sent: 02 October 2015 AM 02:04 > To: dev@drill.apache.org > Subject: Re: Channel closed exception > > I cannot open the shared documents (“Whoops! There was a problem previewing > this document.”) nor can I download them ("We're sorry, but you do not have > access to this page.”). The folder link in this thread is also broken (“The > open folder can no longer be viewed”). > > - Sudheesh > >> On Oct 1, 2015, at 12:43 PM, Sudip Mukherjee <smukher...@commvault.com> >> wrote: >> >> I uploaded to google drive. Should it be okay? >> >> https://drive.google.com/drive/folders/0ByRlB6iLvhRvSm9UQWJ4b2ZjVDQ >> >> Thanks, >> Sudip >> >> -Original Message- >> From: Sudheesh Katkam [mailto:skat...@maprtech.com] >> Sent: 01 October 2015 AM 10:01 >> To: dev@drill.apache.org >> Subject: Re: Channel closed exception >> >> Ah there, I assumed you were not on Windows. We do not have the drillbit.out >> file for Windows (I think). >> >> Anyway, the error messages you see are symptoms of the real issue, which is >> most likely an “out of memory” scenario. I think that the drillbit did not >> crash either (in which case you might see a JVM core dump). Instead the >> drillbit gracefully shutdown unable to handle the error. Could you share the >> query (if possible), data set details, and configuration? >> >> It would also help if you put the entire log files somewhere (Github gist or >> Dropbox) and shared the links. >> >> - Sudheesh >> >>> On Sep 30, 2015, at 9:10 PM, Sudip Mukherjee <smukher...@commvault.com> >>> wrote: >>> >>> Yes. I was running drill in embedded mode to try it out with 3.9 >>> lakhs records , near about 50+ columns.[ sqlline.bat -u >>> "jdbc:drill:zk=local" ] I see sqlline.log and sqlline_queries.log under log >>> folder. Did I miss some steps? >>> >>> But was able run queries from the command prompt after that >>> >>> Thanks, >>> Sudip >>> >>> -Original Message- >>> From: Sudheesh Katkam [mailto:skat...@maprtech.com] >>> Sent: 01 October 2015 AM 09:24 >>> To: dev@drill.apache.org >>> Subject: Re: Channel closed exception >>> >>> Are you running drill in embedded mode? The directory containing the >>> drillbit.log file contains the drillbit.out file too. There might be a JVM >>> core dump in the /tmp directory. >>> >>> Usually "Received shutdown request” message means that’s the drillbit ran >>> out of memory. Given that you mentioned the query is on a large document >>> set, this scenario is likely. >>> >>> - Sudheesh >>> >>>> On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee <smukher...@commvault.com> >>>> wrote: >>>> >>>> Hi, >>>> This is the excerpt of exception in sqlline.log. Could you please refer me >>>> to the location of drillbit.out file? >>>> >>>> >>>> 2015-09-30 22:23:36,286 [BitServer-4] WARN >>>> o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type >>>> 6 took longer than 500ms. Actual duration was 16047ms. >>>> 2015-09-30 22:24:13,581 >>>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO >>>> o.a.d.e.w.fragment.FragmentExecutor - >>>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested >>>> CANCELLATION_REQUESTED --> FINISHED >>>> 2015-09-30 22:24:13,581 >>>> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO >>>> o.a.d.e.w.f.FragmentStatusReporter - >>>> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: >>>> CANCELLED >>>> 2015-09-30 22:24:13,581 [BitServer-4] INFO >>>> o.a.drill.exec.work.foreman.Foreman - State change req
Re: Channel closed exception
Can you check if there are any error messages in the drillbit.out file? - Sudheesh > On Sep 30, 2015, at 8:35 PM, Sudip Mukherjeewrote: > > Hi Devs, > > Getting into a channel closed exception when I tried to connect drill to > tableau and query a large document set. Could it be because tableau dropped > the connection due to datasize. Any thoughts? > > 2015-09-30 21:13:43,716 [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State change requested > AWAITING_ALLOCATION --> RUNNING > 2015-09-30 21:13:43,716 [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO > o.a.d.e.w.f.FragmentStatusReporter - > 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State to report: RUNNING > 2015-09-30 21:14:36,875 [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State change requested RUNNING --> > FINISHED > 2015-09-30 21:14:36,876 [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO > o.a.d.e.w.f.FragmentStatusReporter - > 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State to report: FINISHED > 2015-09-30 21:14:36,877 [BitServer-4] INFO > o.a.drill.exec.work.foreman.Foreman - State change requested. RUNNING --> > COMPLETED > 2015-09-30 21:14:36,877 [BitServer-4] INFO > o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. > 2015-09-30 21:33:09,745 [Client-1] INFO o.a.drill.exec.rpc.user.UserClient - > Channel closed /172.19.77.101:57162 <--> > DEVSTAGING2.testlab.commvault.com/172.19.77.101:31010. > 2015-09-30 21:33:12,859 [BitServer-3] INFO > o.a.d.exec.rpc.control.ControlClient - Channel closed /172.19.77.101:57207 > <--> DEVSTAGING2.testlab.commvault.com/172.19.77.101:31011. > 2015-09-30 21:33:16,077 [main] INFO o.apache.drill.exec.server.Drillbit - > Shutdown completed (6331 ms). > 2015-09-30 21:33:16,102 [Drillbit-ShutdownHook#0] INFO > o.apache.drill.exec.server.Drillbit - Received shutdown request. > > Thanks, > Sudip > > > > ***Legal Disclaimer*** > "This communication may contain confidential and privileged material for the > sole use of the intended recipient. Any unauthorized review, use or > distribution > by others is strictly prohibited. If you have received the message by mistake, > please advise the sender by reply email and delete the message. Thank you." > **
RE: Channel closed exception
Hi, This is the excerpt of exception in sqlline.log. Could you please refer me to the location of drillbit.out file? 2015-09-30 22:23:36,286 [BitServer-4] WARN o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type 6 took longer than 500ms. Actual duration was 16047ms. 2015-09-30 22:24:13,581 [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO o.a.d.e.w.fragment.FragmentExecutor - 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested CANCELLATION_REQUESTED --> FINISHED 2015-09-30 22:24:13,581 [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO o.a.d.e.w.f.FragmentStatusReporter - 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: CANCELLED 2015-09-30 22:24:13,581 [BitServer-4] INFO o.a.drill.exec.work.foreman.Foreman - State change requested. CANCELLATION_REQUESTED --> COMPLETED 2015-09-30 22:24:13,581 [BitServer-4] INFO o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. 2015-09-30 22:24:13,597 [UserServer-1] INFO o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate query result to initiating client. This would happen if a client is disconnected before response notice can be sent. org.apache.drill.exec.rpc.ChannelClosedException: null at org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89) [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67) [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680) [netty-common-4.0.27.Final.jar:4.0.27.Final] at io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603) [netty-common-4.0.27.Final.jar:4.0.27.Final] at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563) [netty-common-4.0.27.Final.jar:4.0.27.Final] at io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424) [netty-common-4.0.27.Final.jar:4.0.27.Final] Thanks, Sudip -Original Message- From: Sudheesh Katkam [mailto:skat...@maprtech.com] Sent: 01 October 2015 AM 09:09 To: dev@drill.apache.org Subject: Re: Channel closed exception Can you check if there are any error messages in the drillbit.out file? - Sudheesh > On Sep 30, 2015, at 8:35 PM, Sudip Mukherjee <smukher...@commvault.com> wrote: > > Hi Devs, > > Getting into a channel closed exception when I tried to connect drill to > tableau and query a large document set. Could it be because tableau dropped > the connection due to datasize. Any thoughts? > > 2015-09-30 21:13:43,716 > [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State change requested > AWAITING_ALLOCATION --> RUNNING > 2015-09-30 21:13:43,716 > [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO > o.a.d.e.w.f.FragmentStatusReporter - > 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State to report: RUNNING > 2015-09-30 21:14:36,875 > [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State change requested > RUNNING --> FINISHED > 2015-09-30 21:14:36,876 > [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO > o.a.d.e.w.f.FragmentStatusReporter - > 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State to report: FINISHED > 2015-09-30 21:14:36,877 [BitServer-4] INFO > o.a.drill.exec.work.foreman.Foreman - State change requested. RUNNING > --> COMPLETED > 2015-09-30 21:14:36,877 [BitServer-4] INFO > o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. > 2015-09-30 21:33:09,745 [Client-1] INFO o.a.drill.exec.rpc.user.UserClient - > Channel closed /172.19.77.101:57162 <--> > DEVSTAGING2.testlab.commvault.com/172.19.77.101:31010. > 2015-09-30 21:33:12,859 [BitServer-3] INFO > o.a.d.exec.rpc.control.ControlClient - Channel closed /172.19.77.101:57207 > <--> DEVSTAGING2.testlab.commvault.com/172.19.77.101:31011. > 2015-09-30 21:33:16,077 [main] INFO o.apache.drill.exec.server.Drillbit - > Shutdown completed (6331 ms). > 2015-09-30 21:33:16,102 [Drillbit-ShutdownHook#0] INFO > o.apache.drill.exec.server.Drillbit - Received shutdown request. > > Thanks, > Sudip > > > > ***Legal Disclaimer*** > "This communication may contain confidential and privileged material > for the sole use of the intended recipient. Any unauthorized review, > use or distribution by others is strictly prohibited. If you have > received the message by mistake, please advise the sender by reply email and > delete the mess
Re: Channel closed exception
Are you running drill in embedded mode? The directory containing the drillbit.log file contains the drillbit.out file too. There might be a JVM core dump in the /tmp directory. Usually "Received shutdown request” message means that’s the drillbit ran out of memory. Given that you mentioned the query is on a large document set, this scenario is likely. - Sudheesh > On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee <smukher...@commvault.com> wrote: > > Hi, > This is the excerpt of exception in sqlline.log. Could you please refer me to > the location of drillbit.out file? > > > 2015-09-30 22:23:36,286 [BitServer-4] WARN > o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type 6 > took longer than 500ms. Actual duration was 16047ms. > 2015-09-30 22:24:13,581 [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested > CANCELLATION_REQUESTED --> FINISHED > 2015-09-30 22:24:13,581 [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO > o.a.d.e.w.f.FragmentStatusReporter - > 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: CANCELLED > 2015-09-30 22:24:13,581 [BitServer-4] INFO > o.a.drill.exec.work.foreman.Foreman - State change requested. > CANCELLATION_REQUESTED --> COMPLETED > 2015-09-30 22:24:13,581 [BitServer-4] INFO > o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. > 2015-09-30 22:24:13,597 [UserServer-1] INFO > o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate query > result to initiating client. This would happen if a client is disconnected > before response notice can be sent. > org.apache.drill.exec.rpc.ChannelClosedException: null > at > org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89) > [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67) > [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:424) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > > Thanks, > Sudip > > -Original Message- > From: Sudheesh Katkam [mailto:skat...@maprtech.com] > Sent: 01 October 2015 AM 09:09 > To: dev@drill.apache.org > Subject: Re: Channel closed exception > > Can you check if there are any error messages in the drillbit.out file? > > - Sudheesh > >> On Sep 30, 2015, at 8:35 PM, Sudip Mukherjee <smukher...@commvault.com> >> wrote: >> >> Hi Devs, >> >> Getting into a channel closed exception when I tried to connect drill to >> tableau and query a large document set. Could it be because tableau dropped >> the connection due to datasize. Any thoughts? >> >> 2015-09-30 21:13:43,716 >> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO >> o.a.d.e.w.fragment.FragmentExecutor - >> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State change requested >> AWAITING_ALLOCATION --> RUNNING >> 2015-09-30 21:13:43,716 >> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO >> o.a.d.e.w.f.FragmentStatusReporter - >> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State to report: RUNNING >> 2015-09-30 21:14:36,875 >> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO >> o.a.d.e.w.fragment.FragmentExecutor - >> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State change requested >> RUNNING --> FINISHED >> 2015-09-30 21:14:36,876 >> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO >> o.a.d.e.w.f.FragmentStatusReporter - >> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State to report: FINISHED >> 2015-09-30 21:14:36,877 [BitServer-4] INFO >> o.a.drill.exec.work.foreman.Foreman - State change requested. RUNNING >> --> COMPLETED >> 2015-09-30 21:14:36,877 [BitServer-4] INFO >> o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. >> 2015-09-30 21:33:09,745 [Client-1] INFO o.a.drill.exec.rpc.user.UserClient >> - Channel closed /172.19.77.101:57162 <--> >> DEVSTAGING2.testlab.commvault.com/172.19.77.101:31010. >
RE: Channel closed exception
Yes. I was running drill in embedded mode to try it out with 3.9 lakhs records , near about 50+ columns.[ sqlline.bat -u "jdbc:drill:zk=local" ] I see sqlline.log and sqlline_queries.log under log folder. Did I miss some steps? But was able run queries from the command prompt after that Thanks, Sudip -Original Message- From: Sudheesh Katkam [mailto:skat...@maprtech.com] Sent: 01 October 2015 AM 09:24 To: dev@drill.apache.org Subject: Re: Channel closed exception Are you running drill in embedded mode? The directory containing the drillbit.log file contains the drillbit.out file too. There might be a JVM core dump in the /tmp directory. Usually "Received shutdown request” message means that’s the drillbit ran out of memory. Given that you mentioned the query is on a large document set, this scenario is likely. - Sudheesh > On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee <smukher...@commvault.com> wrote: > > Hi, > This is the excerpt of exception in sqlline.log. Could you please refer me to > the location of drillbit.out file? > > > 2015-09-30 22:23:36,286 [BitServer-4] WARN > o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type 6 > took longer than 500ms. Actual duration was 16047ms. > 2015-09-30 22:24:13,581 > [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested > CANCELLATION_REQUESTED --> FINISHED > 2015-09-30 22:24:13,581 > [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO > o.a.d.e.w.f.FragmentStatusReporter - > 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: CANCELLED > 2015-09-30 22:24:13,581 [BitServer-4] INFO > o.a.drill.exec.work.foreman.Foreman - State change requested. > CANCELLATION_REQUESTED --> COMPLETED > 2015-09-30 22:24:13,581 [BitServer-4] INFO > o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. > 2015-09-30 22:24:13,597 [UserServer-1] INFO > o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate query > result to initiating client. This would happen if a client is disconnected > before response notice can be sent. > org.apache.drill.exec.rpc.ChannelClosedException: null > at > org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89) > [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67) > [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] > at > io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563) > [netty-common-4.0.27.Final.jar:4.0.27.Final] > at > io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java > :424) [netty-common-4.0.27.Final.jar:4.0.27.Final] > > Thanks, > Sudip > > -Original Message- > From: Sudheesh Katkam [mailto:skat...@maprtech.com] > Sent: 01 October 2015 AM 09:09 > To: dev@drill.apache.org > Subject: Re: Channel closed exception > > Can you check if there are any error messages in the drillbit.out file? > > - Sudheesh > >> On Sep 30, 2015, at 8:35 PM, Sudip Mukherjee <smukher...@commvault.com> >> wrote: >> >> Hi Devs, >> >> Getting into a channel closed exception when I tried to connect drill to >> tableau and query a large document set. Could it be because tableau dropped >> the connection due to datasize. Any thoughts? >> >> 2015-09-30 21:13:43,716 >> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO >> o.a.d.e.w.fragment.FragmentExecutor - >> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State change requested >> AWAITING_ALLOCATION --> RUNNING >> 2015-09-30 21:13:43,716 >> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO >> o.a.d.e.w.f.FragmentStatusReporter - >> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State to report: RUNNING >> 2015-09-30 21:14:36,875 >> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO >> o.a.d.e.w.fragment.FragmentExecutor - >> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State change requested >> RUNNING --> FINISHED >> 2015-09-30 21:14:36,876 >> [29f3777a-d407-295a-d45e-243fb70efc07:frag:0:0] INFO >> o.a.d.e.w.f.FragmentStatusReporter - >> 29f3777a-d407-295a-d45e-243fb70efc07:0:0: State to report: FINISHED >>
Re: Channel closed exception
Ah there, I assumed you were not on Windows. We do not have the drillbit.out file for Windows (I think). Anyway, the error messages you see are symptoms of the real issue, which is most likely an “out of memory” scenario. I think that the drillbit did not crash either (in which case you might see a JVM core dump). Instead the drillbit gracefully shutdown unable to handle the error. Could you share the query (if possible), data set details, and configuration? It would also help if you put the entire log files somewhere (Github gist or Dropbox) and shared the links. - Sudheesh > On Sep 30, 2015, at 9:10 PM, Sudip Mukherjee <smukher...@commvault.com> wrote: > > Yes. I was running drill in embedded mode to try it out with 3.9 lakhs > records , near about 50+ columns.[ sqlline.bat -u "jdbc:drill:zk=local" ] > I see sqlline.log and sqlline_queries.log under log folder. Did I miss some > steps? > > But was able run queries from the command prompt after that > > Thanks, > Sudip > > -Original Message- > From: Sudheesh Katkam [mailto:skat...@maprtech.com] > Sent: 01 October 2015 AM 09:24 > To: dev@drill.apache.org > Subject: Re: Channel closed exception > > Are you running drill in embedded mode? The directory containing the > drillbit.log file contains the drillbit.out file too. There might be a JVM > core dump in the /tmp directory. > > Usually "Received shutdown request” message means that’s the drillbit ran out > of memory. Given that you mentioned the query is on a large document set, > this scenario is likely. > > - Sudheesh > >> On Sep 30, 2015, at 8:46 PM, Sudip Mukherjee <smukher...@commvault.com> >> wrote: >> >> Hi, >> This is the excerpt of exception in sqlline.log. Could you please refer me >> to the location of drillbit.out file? >> >> >> 2015-09-30 22:23:36,286 [BitServer-4] WARN >> o.a.d.exec.rpc.control.ControlServer - Message of mode REQUEST of rpc type 6 >> took longer than 500ms. Actual duration was 16047ms. >> 2015-09-30 22:24:13,581 >> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO >> o.a.d.e.w.fragment.FragmentExecutor - >> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State change requested >> CANCELLATION_REQUESTED --> FINISHED >> 2015-09-30 22:24:13,581 >> [29f36725-9de3-b370-7f38-50ebbe677bf7:frag:0:0] INFO >> o.a.d.e.w.f.FragmentStatusReporter - >> 29f36725-9de3-b370-7f38-50ebbe677bf7:0:0: State to report: CANCELLED >> 2015-09-30 22:24:13,581 [BitServer-4] INFO >> o.a.drill.exec.work.foreman.Foreman - State change requested. >> CANCELLATION_REQUESTED --> COMPLETED >> 2015-09-30 22:24:13,581 [BitServer-4] INFO >> o.a.drill.exec.work.foreman.Foreman - foreman cleaning up. >> 2015-09-30 22:24:13,597 [UserServer-1] INFO >> o.a.drill.exec.work.foreman.Foreman - Failure while trying communicate query >> result to initiating client. This would happen if a client is disconnected >> before response notice can be sent. >> org.apache.drill.exec.rpc.ChannelClosedException: null >> at >> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:89) >> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] >> at >> org.apache.drill.exec.rpc.CoordinationQueue$RpcListener.operationComplete(CoordinationQueue.java:67) >> [drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] >> at >> io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:680) >> [netty-common-4.0.27.Final.jar:4.0.27.Final] >> at >> io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:603) >> [netty-common-4.0.27.Final.jar:4.0.27.Final] >> at >> io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:563) >> [netty-common-4.0.27.Final.jar:4.0.27.Final] >> at >> io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java >> :424) [netty-common-4.0.27.Final.jar:4.0.27.Final] >> >> Thanks, >> Sudip >> >> -Original Message- >> From: Sudheesh Katkam [mailto:skat...@maprtech.com] >> Sent: 01 October 2015 AM 09:09 >> To: dev@drill.apache.org >> Subject: Re: Channel closed exception >> >> Can you check if there are any error messages in the drillbit.out file? >> >> - Sudheesh >> >>> On Sep 30, 2015, at 8:35 PM, Sudip Mukherjee <smukher...@commvault.com> >>> wrote: >>> >>> Hi Devs, >>> >>> Getting into a channel closed exception when I tried to connect drill to >>> table