shihuafeng created DRILL-8479:
---------------------------------
Summary: mergejion memory leak when exception
Key: DRILL-8479
URL: https://issues.apache.org/jira/browse/DRILL-8479
Project: Apache Drill
Issue Type: Bug
Components: Execution - Relational Operators
Affects Versions: 1.21.1
Reporter: shihuafeng
Attachments: 0001-mergejoin-leak.patch
*Describe the bug*
megerjoin leak when RecordIterator allocate memory exception with
OutOfMemoryException{*}{*}
{*}Steps to reproduce the behavior{*}:
# prepare data for tpch 1s
# set direct memory 5g
# set planner.enable_hashjoin =false to ensure use mergejoin operator。
# set drill.memory.debug.allocator =true (Check for memory leaks )
# 20 concurrent for tpch sql8
# when it had OutOfMemoryException or null EXCEPTION , stopped all sql.
# finding memory leak
*Expected behavior*
when all sql sop , we should find direct memory is 0 AND could not find
leak log like following.
{code:java}
Allocator(op:2:0:11:MergeJoinPOP) 1000000/73728/4874240/10000000000
(res/actual/peak/limit){code}
*Error detail, log output or screenshots*
{code:java}
Unable to allocate buffer of size XX (rounded from XX) due to memory limit ().
Current allocation: xx{code}
[^0001-mergejoin-leak.patch]
sql
{code:java}
// code placeholder
select o_year, sum(case when nation = 'CHINA' then volume else 0 end) /
sum(volume) as mkt_share from ( select extract(year from o_orderdate) as
o_year, l_extendedprice * 1.0 as volume, n2.n_name as nation from
hive.tpch1s.part, hive.tpch1s.supplier, hive.tpch1s.lineitem,
hive.tpch1s.orders, hive.tpch1s.customer, hive.tpch1s.nation n1,
hive.tpch1s.nation n2, hive.tpch1s.region where p_partkey = l_partkey and
s_suppkey = l_suppkey and l_orderkey = o_orderkey and o_custkey = c_custkey and
c_nationkey = n1.n_nationkey and n1.n_regionkey = r_regionkey and r_name =
'ASIA' and s_nationkey = n2.n_nationkey and o_orderdate between date
'1995-01-01' and date '1996-12-31' and p_type = 'LARGE BRUSHED BRASS') as
all_nations group by o_year order by o_year
{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
