[jira] [Commented] (DRILL-4274) ExternalSort doesn't always handle low memory condition well, failing execution instead of spilling in some cases

[Khurram Faraaz (JIRA)]

    [ 
https://issues.apache.org/jira/browse/DRILL-4274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15111993#comment-15111993
 ] 

Khurram Faraaz commented on DRILL-4274:
---------------------------------------

This is also seen on Drill 1.5.0 (commit ID : 2f0e3f27) with JDK8

{noformat}
framework/framework/resources/Functional/tpcds/original/json/query56.sql 
(connection: 1117749405)
[#2158] Query failed: 
oadd.org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: 
DrillRuntimeException: Failed to pre-allocate memory for SV. Existing 
recordCount*4 = 0, incoming batch recordCount*4 = 3348

Fragment 0:0

[Error Id: f50dcc75-0f83-401b-8249-599c6a1d0d8c on drill-democ1:31010]
        at 
oadd.org.apache.drill.exec.rpc.user.QueryResultHandler.resultArrived(QueryResultHandler.java:119)
        at 
oadd.org.apache.drill.exec.rpc.user.UserClient.handleReponse(UserClient.java:113)
        at 
oadd.org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:46)
        at 
oadd.org.apache.drill.exec.rpc.BasicClientWithConnection.handle(BasicClientWithConnection.java:31)
        at oadd.org.apache.drill.exec.rpc.RpcBus.handle(RpcBus.java:67)
        at 
oadd.org.apache.drill.exec.rpc.RpcBus$RequestEvent.run(RpcBus.java:374)
        at 
oadd.org.apache.drill.common.SerializedExecutor$RunnableProcessor.run(SerializedExecutor.java:89)
        at 
oadd.org.apache.drill.exec.rpc.RpcBus$SameExecutor.execute(RpcBus.java:252)
        at 
oadd.org.apache.drill.common.SerializedExecutor.execute(SerializedExecutor.java:123)
        at 
oadd.org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:285)
        at 
oadd.org.apache.drill.exec.rpc.RpcBus$InboundHandler.decode(RpcBus.java:257)
        at 
oadd.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:89)
        at 
oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
        at 
oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
        at 
oadd.io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:254)
        at 
oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
        at 
oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
        at 
oadd.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103)
        at 
oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
        at 
oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
        at 
oadd.io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:242)
        at 
oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
        at 
oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
        at 
oadd.io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86)
        at 
oadd.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:339)
        at 
oadd.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:324)
        at 
oadd.io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:847)
        at 
oadd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:131)
        at 
oadd.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:511)
        at 
oadd.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:468)
        at 
oadd.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:382)
        at oadd.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:354)
        at 
oadd.io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:111)
        at java.lang.Thread.run(Thread.java:745)
{noformat}

> ExternalSort doesn't always handle low memory condition well, failing 
> execution instead of spilling in some cases
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-4274
>                 URL: https://issues.apache.org/jira/browse/DRILL-4274
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Execution - Relational Operators
>            Reporter: Jacques Nadeau
>            Assignee: Jacques Nadeau
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)