[ https://issues.apache.org/jira/browse/DRILL-3095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Khurram Faraaz closed DRILL-3095. --------------------------------- > Memory Leak : Failure while closing accountor. > ---------------------------------------------- > > Key: DRILL-3095 > URL: https://issues.apache.org/jira/browse/DRILL-3095 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow > Affects Versions: 1.0.0 > Environment: f7f6efc525cd833ce1530deae32eb9ccb20b664a > Reporter: Khurram Faraaz > Assignee: Chris Westin > Fix For: 1.2.0 > > > I am seeing a memory leak when i Cancel a long running query on sqlline. I am > re running the query with assertion enabled, will add details after the > second run is complete with assertions. > Long running query was, > {code} > select key1, key2 from `twoKeyJsn.json`; > {code} > I did Ctrl-C when the above query was running on sqlline, and then issued the > below query that returned correct results. And then I see there is a memory > leak message in drillbit.log > {code} > 0: jdbc:drill:schema=dfs.tmp> select count(*) from `twoKeyJsn.json`; > +------------+ > | EXPR$0 | > +------------+ > | 26212355 | > +------------+ > 1 row selected (14.734 seconds) > {code} > Stack trace from drillbit.log > {code} > 2015-05-15 00:59:01,951 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] WARN > o.a.drill.exec.ops.SendingAccountor - Interrupted while waiting for send > complete. Continuing to wait. > java.lang.InterruptedException: null > at > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1301) > ~[na:1.7.0_45] > at java.util.concurrent.Semaphore.acquire(Semaphore.java:472) > ~[na:1.7.0_45] > at > org.apache.drill.exec.ops.SendingAccountor.waitForSendComplete(SendingAccountor.java:48) > ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.ops.FragmentContext.waitForSendComplete(FragmentContext.java:436) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.BaseRootExec.close(BaseRootExec.java:112) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.physical.impl.ScreenCreator$ScreenRoot.close(ScreenCreator.java:141) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:333) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:278) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_45] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_45] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] > 2015-05-15 00:59:01,952 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 2aaabb50-3afd-2906-3f48-eb86a315a1f5:0:0: State change requested from > CANCELLATION_REQUESTED --> FAILED for > 2015-05-15 00:59:01,952 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 2aaabb50-3afd-2906-3f48-eb86a315a1f5:0:0: State change requested from FAILED > --> FAILED for > 2015-05-15 00:59:01,952 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO > o.a.d.e.w.fragment.FragmentExecutor - > 2aaabb50-3afd-2906-3f48-eb86a315a1f5:0:0: State change requested from FAILED > --> FINISHED for > 2015-05-15 00:59:01,956 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] ERROR > o.a.d.c.exceptions.UserException - SYSTEM ERROR: > java.lang.IllegalStateException: Failure while closing accountor. Expected > private and shared pools to be set to initial values. However, one or more > were not. Stats are > zone init allocated delta > private 1000000 918080 81920 > shared 9999000000 9999000000 0. > Fragment 0:0 > [Error Id: 90ced8b1-b6db-438f-b193-b7634de31b81 on centos-03.qa.lab:31010] > org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: > java.lang.IllegalStateException: Failure while closing accountor. Expected > private and shared pools to be set to initial values. However, one or more > were not. Stats are > zone init allocated delta > private 1000000 918080 81920 > shared 9999000000 9999000000 0. > Fragment 0:0 > [Error Id: 90ced8b1-b6db-438f-b193-b7634de31b81 on centos-03.qa.lab:31010] > at > org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:522) > ~[drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:315) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:283) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_45] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_45] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] > Caused by: java.lang.IllegalStateException: Failure while closing accountor. > Expected private and shared pools to be set to initial values. However, one > or more were not. Stats are > zone init allocated delta > private 1000000 918080 81920 > shared 9999000000 9999000000 0. > at > org.apache.drill.exec.memory.AtomicRemainder.close(AtomicRemainder.java:200) > ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at org.apache.drill.exec.memory.Accountor.close(Accountor.java:386) > ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.memory.TopLevelAllocator$ChildAllocator.close(TopLevelAllocator.java:325) > ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.ops.OperatorContextImpl.close(OperatorContextImpl.java:116) > ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.ops.FragmentContext.suppressingClose(FragmentContext.java:405) > ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.ops.FragmentContext.close(FragmentContext.java:394) > ~[drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:340) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:278) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > ... 4 common frames omitted > 2015-05-15 00:59:01,963 [2aaabb50-3afd-2906-3f48-eb86a315a1f5:frag:0:0] INFO > o.a.drill.exec.work.foreman.Foreman - State change requested. > CANCELLATION_REQUESTED --> FAILED > org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: > java.lang.IllegalStateException: Failure while closing accountor. Expected > private and shared pools to be set to initial values. However, one or more > were not. Stats are > zone init allocated delta > private 1000000 918080 81920 > shared 9999000000 9999000000 0. > Fragment 0:0 > [Error Id: 90ced8b1-b6db-438f-b193-b7634de31b81 on centos-03.qa.lab:31010] > at > org.apache.drill.exec.work.foreman.QueryManager$1.statusUpdate(QueryManager.java:481) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.work.foreman.QueryManager$RootStatusReporter.statusChange(QueryManager.java:461) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:90) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.AbstractStatusReporter.fail(AbstractStatusReporter.java:86) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:319) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:283) > [drill-java-exec-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > [drill-common-1.0.0-SNAPSHOT-rebuffed.jar:1.0.0-SNAPSHOT] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_45] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_45] > at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45] > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)