[jira] [Updated] (DRILL-6252) Foreman node is going down when the non foreman node is stopped
[
https://issues.apache.org/jira/browse/DRILL-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-6252:
-
Attachment: foreman_drillbit.log
> Foreman node is going down when the non foreman node is stopped
> ---
>
> Key: DRILL-6252
> URL: https://issues.apache.org/jira/browse/DRILL-6252
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Venkata Jyothsna Donapati
>Assignee: Vlad Rozov
>Priority: Major
> Fix For: 1.14.0
>
> Attachments: foreman_drillbit.log, nonforeman_drillbit.log
>
>
> Two drillbits are running. I'm running a join query over parquet and tried to
> stop the non-foreman node using drillbit.sh stop. The query fails with
> *"Error: DATA_READ ERROR: Exception occurred while reading from disk".* The
> non-foreman node goes down. The foreman node also goes down. When I looked at
> the drillbit.log of both foreman and non-foreman I found that there is memory
> leak "Memory was leaked by query. Memory leaked:
> (2097152)\nAllocator(op:2:0:0:HashPartitionSender)
> 100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are
> the stack traces for memory leaks
> {noformat}
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalStateException: Memory was leaked by query. Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
>
>
> Fragment 2:1
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> Caused by: java.lang.IllegalStateException: Memory was leaked by query.
> Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
> {noformat}
>
> Ping me for the logs and more information.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6252) Foreman node is going down when the non foreman node is stopped
[
https://issues.apache.org/jira/browse/DRILL-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Venkata Jyothsna Donapati updated DRILL-6252:
-
Attachment: nonforeman_drillbit.log
> Foreman node is going down when the non foreman node is stopped
> ---
>
> Key: DRILL-6252
> URL: https://issues.apache.org/jira/browse/DRILL-6252
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Venkata Jyothsna Donapati
>Assignee: Vlad Rozov
>Priority: Major
> Fix For: 1.14.0
>
> Attachments: foreman_drillbit.log, nonforeman_drillbit.log
>
>
> Two drillbits are running. I'm running a join query over parquet and tried to
> stop the non-foreman node using drillbit.sh stop. The query fails with
> *"Error: DATA_READ ERROR: Exception occurred while reading from disk".* The
> non-foreman node goes down. The foreman node also goes down. When I looked at
> the drillbit.log of both foreman and non-foreman I found that there is memory
> leak "Memory was leaked by query. Memory leaked:
> (2097152)\nAllocator(op:2:0:0:HashPartitionSender)
> 100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are
> the stack traces for memory leaks
> {noformat}
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalStateException: Memory was leaked by query. Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
>
>
> Fragment 2:1
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> Caused by: java.lang.IllegalStateException: Memory was leaked by query.
> Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
> {noformat}
>
> Ping me for the logs and more information.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6252) Foreman node is going down when the non foreman node is stopped
[
https://issues.apache.org/jira/browse/DRILL-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vlad Rozov updated DRILL-6252:
--
Priority: Major (was: Critical)
> Foreman node is going down when the non foreman node is stopped
> ---
>
> Key: DRILL-6252
> URL: https://issues.apache.org/jira/browse/DRILL-6252
> Project: Apache Drill
> Issue Type: Bug
>Reporter: Venkata Jyothsna Donapati
>Assignee: Vlad Rozov
>Priority: Major
> Fix For: 1.14.0
>
>
> Two drillbits are running. I'm running a join query over parquet and tried to
> stop the non-foreman node using drillbit.sh stop. The query fails with
> *"Error: DATA_READ ERROR: Exception occurred while reading from disk".* The
> non-foreman node goes down. The foreman node also goes down. When I looked at
> the drillbit.log of both foreman and non-foreman I found that there is memory
> leak "Memory was leaked by query. Memory leaked:
> (2097152)\nAllocator(op:2:0:0:HashPartitionSender)
> 100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are
> the stack traces for memory leaks
> {noformat}
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
> IllegalStateException: Memory was leaked by query. Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
>
>
> Fragment 2:1
> [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
> at
> org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
> ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266)
> [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
> [drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> [na:1.8.0_161]
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> [na:1.8.0_161]
> at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
> Caused by: java.lang.IllegalStateException: Memory was leaked by query.
> Memory leaked: (3145728)
> Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
> (res/actual/peak/limit)
> {noformat}
>
> Ping me for the logs and more information.
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (DRILL-6252) Foreman node is going down when the non foreman node is stopped
[
https://issues.apache.org/jira/browse/DRILL-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vlad Rozov updated DRILL-6252:
--
Description:
Two drillbits are running. I'm running a join query over parquet and tried to
stop the non-foreman node using drillbit.sh stop. The query fails with *"Error:
DATA_READ ERROR: Exception occurred while reading from disk".* The non-foreman
node goes down. The foreman node also goes down. When I looked at the
drillbit.log of both foreman and non-foreman I found that there is memory leak
"Memory was leaked by query. Memory leaked:
(2097152)\nAllocator(op:2:0:0:HashPartitionSender)
100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are
the stack traces for memory leaks
{noformat}
[Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
IllegalStateException: Memory was leaked by query. Memory leaked: (3145728)
Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
(res/actual/peak/limit)
Fragment 2:1
[Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297)
[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266)
[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[na:1.8.0_161]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[na:1.8.0_161]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
Caused by: java.lang.IllegalStateException: Memory was leaked by query. Memory
leaked: (3145728)
Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
(res/actual/peak/limit)
{noformat}
Ping me for the logs and more information.
was:
Two drillbits are running. I'm running a join query over parquet and tried to
stop the non-foreman node using drillbit.sh stop. The query fails with *"Error:
DATA_READ ERROR: Exception occurred while reading from disk".* The non-foreman
node goes down. The foreman node also goes down. When I looked at the
drillbit.log of both foreman and non-foreman I found that there is memory leak
"Memory was leaked by query. Memory leaked:
(2097152)\nAllocator(op:2:0:0:HashPartitionSender)
100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are
the stack traces for memory leaks
[Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR:
IllegalStateException: Memory was leaked by query. Memory leaked: (3145728)
Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
(res/actual/peak/limit)
Fragment 2:1
[Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010]
at
org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633)
~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297)
[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160)
[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266)
[drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[na:1.8.0_161]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[na:1.8.0_161]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161]
Caused by: java.lang.IllegalStateException: Memory was leaked by query. Memory
leaked: (3145728)
Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100
(res/actual/peak/limit)
Ping me for the logs and more information.
> Foreman node is going down when the non foreman node is stopped
>
[jira] [Updated] (DRILL-6252) Foreman node is going down when the non foreman node is stopped
[ https://issues.apache.org/jira/browse/DRILL-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pritesh Maker updated DRILL-6252: - Fix Version/s: 1.14.0 > Foreman node is going down when the non foreman node is stopped > --- > > Key: DRILL-6252 > URL: https://issues.apache.org/jira/browse/DRILL-6252 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Assignee: Vlad Rozov >Priority: Critical > Fix For: 1.14.0 > > > Two drillbits are running. I'm running a join query over parquet and tried to > stop the non-foreman node using drillbit.sh stop. The query fails with > *"Error: DATA_READ ERROR: Exception occurred while reading from disk".* The > non-foreman node goes down. The foreman node also goes down. When I looked at > the drillbit.log of both foreman and non-foreman I found that there is memory > leak "Memory was leaked by query. Memory leaked: > (2097152)\nAllocator(op:2:0:0:HashPartitionSender) > 100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are > the stack traces for memory leaks > > [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010] > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > IllegalStateException: Memory was leaked by query. Memory leaked: (3145728) > Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100 > (res/actual/peak/limit) > > > Fragment 2:1 > [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) > ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297) > [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160) > [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266) > [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [na:1.8.0_161] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [na:1.8.0_161] > at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161] > Caused by: java.lang.IllegalStateException: Memory was leaked by query. > Memory leaked: (3145728) > Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100 > (res/actual/peak/limit) > > Ping me for the logs and more information. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (DRILL-6252) Foreman node is going down when the non foreman node is stopped
[ https://issues.apache.org/jira/browse/DRILL-6252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Khurram Faraaz updated DRILL-6252: -- Priority: Critical (was: Major) > Foreman node is going down when the non foreman node is stopped > --- > > Key: DRILL-6252 > URL: https://issues.apache.org/jira/browse/DRILL-6252 > Project: Apache Drill > Issue Type: Bug >Reporter: Venkata Jyothsna Donapati >Priority: Critical > > Two drillbits are running. I'm running a join query over parquet and tried to > stop the non-foreman node using drillbit.sh stop. The query fails with > *"Error: DATA_READ ERROR: Exception occurred while reading from disk".* The > non-foreman node goes down. The foreman node also goes down. When I looked at > the drillbit.log of both foreman and non-foreman I found that there is memory > leak "Memory was leaked by query. Memory leaked: > (2097152)\nAllocator(op:2:0:0:HashPartitionSender) > 100/6291456/6832128/100 (res/actual/peak/limit)\n". Following are > the stack traces for memory leaks > > [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010] > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > IllegalStateException: Memory was leaked by query. Memory leaked: (3145728) > Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100 > (res/actual/peak/limit) > > > Fragment 2:1 > [Error Id: 0d9a2799-7e97-46b3-953b-1f8d0dd87a04 on qa102-34.qa.lab:31010] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:633) > ~[drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:297) > [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:160) > [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:266) > [drill-java-exec-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.13.0-SNAPSHOT.jar:1.13.0-SNAPSHOT] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) > [na:1.8.0_161] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) > [na:1.8.0_161] > at java.lang.Thread.run(Thread.java:748) [na:1.8.0_161] > Caused by: java.lang.IllegalStateException: Memory was leaked by query. > Memory leaked: (3145728) > Allocator(op:2:1:0:HashPartitionSender) 100/6291456/6291456/100 > (res/actual/peak/limit) > > Ping me for the logs and more information. > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
