[ https://issues.apache.org/jira/browse/DRILL-6914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16733347#comment-16733347 ]
Abhishek Ravi commented on DRILL-6914: -------------------------------------- Yes, as mentioned earlier, the issue is seen with SF 100 and ONLY when both \{{planner.enable_semijoin}} and \{{exec.hashjoin.enable.runtime_filter}} options are set to \{{true}}. The issue is not seen if either one is disabled. > Query with RuntimeFilter and SemiJoin fails with IllegalStateException: > Memory was leaked by query > -------------------------------------------------------------------------------------------------- > > Key: DRILL-6914 > URL: https://issues.apache.org/jira/browse/DRILL-6914 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Flow > Affects Versions: 1.15.0 > Reporter: Abhishek Ravi > Assignee: Boaz Ben-Zvi > Priority: Major > Fix For: 1.16.0 > > > Following query fails on TPC-H SF 100 dataset when > exec.hashjoin.enable.runtime_filter = true AND planner.enable_semijoin = true. > Note that the query does not fail if any one of them or both are disabled. > {code:sql} > set `exec.hashjoin.enable.runtime_filter` = true; > set `exec.hashjoin.runtime_filter.max.waiting.time` = 10000; > set `planner.enable_broadcast_join` = false; > set `planner.enable_semijoin` = true; > select > count(*) as row_count > from > lineitem l1 > where > l1.l_shipdate IN ( > select > distinct(cast(l2.l_shipdate as date)) > from > lineitem l2); > reset `exec.hashjoin.enable.runtime_filter`; > reset `exec.hashjoin.runtime_filter.max.waiting.time`; > reset `planner.enable_broadcast_join`; > reset `planner.enable_semijoin`; > {code} > > {noformat} > Error: SYSTEM ERROR: IllegalStateException: Memory was leaked by query. > Memory leaked: (134217728) > Allocator(frag:1:0) 8000000/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > (state=,code=0) > java.sql.SQLException: SYSTEM ERROR: IllegalStateException: Memory was leaked > by query. Memory leaked: (134217728) > Allocator(frag:1:0) 8000000/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.jdbc.impl.DrillCursor.nextRowInternally(DrillCursor.java:536) > at org.apache.drill.jdbc.impl.DrillCursor.next(DrillCursor.java:640) > at org.apache.calcite.avatica.AvaticaResultSet.next(AvaticaResultSet.java:217) > at > org.apache.drill.jdbc.impl.DrillResultSetImpl.next(DrillResultSetImpl.java:151) > at sqlline.BufferedRows.<init>(BufferedRows.java:37) > at sqlline.SqlLine.print(SqlLine.java:1716) > at sqlline.Commands.execute(Commands.java:949) > at sqlline.Commands.sql(Commands.java:882) > at sqlline.SqlLine.dispatch(SqlLine.java:725) > at sqlline.SqlLine.runCommands(SqlLine.java:1779) > at sqlline.Commands.run(Commands.java:1485) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at sqlline.ReflectiveCommandHandler.execute(ReflectiveCommandHandler.java:38) > at sqlline.SqlLine.dispatch(SqlLine.java:722) > at sqlline.SqlLine.initArgs(SqlLine.java:458) > at sqlline.SqlLine.begin(SqlLine.java:514) > at sqlline.SqlLine.start(SqlLine.java:264) > at sqlline.SqlLine.main(SqlLine.java:195) > Caused by: org.apache.drill.common.exceptions.UserRemoteException: SYSTEM > ERROR: IllegalStateException: Memory was leaked by query. Memory leaked: > (134217728) > Allocator(frag:1:0) 8000000/134217728/172453568/70126322567 > (res/actual/peak/limit) > Fragment 1:0 > Please, refer to logs for more information. > [Error Id: ccee18b3-c3ff-4fdb-b314-23a6cfed0a0e on qa-node185.qa.lab:31010] > at > org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:123) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:422) > at org.apache.drill.exec.rpc.user.UserClient.handle(UserClient.java:96) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:273) > at org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:243) > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:88) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) > at > io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:287) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) > at > io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) > at > io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:312) > at > io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:286) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) > at > io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) > at > io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:335) > at > io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1294) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:356) > at > io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:342) > at > io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:911) > at > io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131) > at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:645) > at > io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580) > at > io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497) > at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459) > at > io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:131) > at java.lang.Thread.run(Thread.java:748) > Caused by: java.lang.IllegalStateException: Memory was leaked by query. > Memory leaked: (134217728) > Allocator(frag:1:0) 8000000/134217728/172453568/70126322567 > (res/actual/peak/limit) > at org.apache.drill.exec.memory.BaseAllocator.close(BaseAllocator.java:520) > at > org.apache.drill.exec.ops.FragmentContextImpl.suppressingClose(FragmentContextImpl.java:545) > at > org.apache.drill.exec.ops.FragmentContextImpl.close(FragmentContextImpl.java:539) > at > org.apache.drill.exec.work.fragment.FragmentExecutor.closeOutResources(FragmentExecutor.java:387) > at > org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:214) > at > org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:330) > at > org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38) > at .......(:0){noformat} -- This message was sent by Atlassian JIRA (v7.6.3#76005)