[
https://issues.apache.org/jira/browse/DRILL-5503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16052613#comment-16052613
]
Paul Rogers commented on DRILL-5503:
------------------------------------
One change that was made is to log an error message in the case of
exceptionally low memory to help diagnose this problem if it occurs in the
field.
{code}
Insufficient memory to merge two batches. \n
Incoming batch size: 269,082,624, \n
available memory: 62,600,000
{code}
> Disabling exchanges results in "Unable to allocate sv2 buffer" error within
> the managed external sort code
> ----------------------------------------------------------------------------------------------------------
>
> Key: DRILL-5503
> URL: https://issues.apache.org/jira/browse/DRILL-5503
> Project: Apache Drill
> Issue Type: Bug
> Components: Execution - Flow
> Affects Versions: 1.10.0
> Reporter: Rahul Challapalli
> Assignee: Paul Rogers
> Attachments: drill5503.log, failure.sys.drill, success.sys.drill
>
>
> Setup :
> {code}
> git.commit.id.abbrev=1e0a14c
> No of drillbits : 1
> DRILL_MAX_DIRECT_MEMORY="32G"
> DRILL_MAX_HEAP="4G"
> {code}
> The below successfully completes
> {code}
> ALTER SESSION SET `exec.sort.disable_managed` = false;
> alter session set `planner.width.max_per_node` = 1;
> alter session set `planner.memory.max_query_memory_per_node` = 62600000;
> alter session set `planner.width.max_per_query` = 17;
> select count(*) from (select * from
> dfs.`/drill/testdata/resource-manager/5kwidecolumns_500k.tbl` order by
> columns[0]) d where d.columns[0] = '4041054511';
> +---------+
> | EXPR$0 |
> +---------+
> | 0 |
> +---------+
> 1 row selected (814.104 seconds)
> {code}
> However if I disable exchanges, I get the following error
> {code}
> alter session set `planner.disable_exchanges` = false;
> select count(*) from (select * from
> dfs.`/drill/testdata/resource-manager/5kwidecolumns_500k.tbl` order by
> columns[0]) d where d.columns[0] = '4041054511';
> +---------+
> | EXPR$0 |
> +---------+
> | 0 |
> +---------+
> 1 row selected (814.104 seconds)
> {code}
> I attached the profile and the log file. The data set used is too large to
> attach here.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
