shihuafeng created DRILL-8479: --------------------------------- Summary: mergejion memory leak when exception Key: DRILL-8479 URL: https://issues.apache.org/jira/browse/DRILL-8479 Project: Apache Drill Issue Type: Bug Components: Execution - Relational Operators Affects Versions: 1.21.1 Reporter: shihuafeng Attachments: 0001-mergejoin-leak.patch
*Describe the bug* megerjoin leak when RecordIterator allocate memory exception with OutOfMemoryException{*}{*} {*}Steps to reproduce the behavior{*}: # prepare data for tpch 1s # set direct memory 5g # set planner.enable_hashjoin =false to ensure use mergejoin operator。 # set drill.memory.debug.allocator =true (Check for memory leaks ) # 20 concurrent for tpch sql8 # when it had OutOfMemoryException or null EXCEPTION , stopped all sql. # finding memory leak *Expected behavior* when all sql sop , we should find direct memory is 0 AND could not find leak log like following. {code:java} Allocator(op:2:0:11:MergeJoinPOP) 1000000/73728/4874240/10000000000 (res/actual/peak/limit){code} *Error detail, log output or screenshots* {code:java} Unable to allocate buffer of size XX (rounded from XX) due to memory limit (). Current allocation: xx{code} [^0001-mergejoin-leak.patch] sql {code:java} // code placeholder select o_year, sum(case when nation = 'CHINA' then volume else 0 end) / sum(volume) as mkt_share from ( select extract(year from o_orderdate) as o_year, l_extendedprice * 1.0 as volume, n2.n_name as nation from hive.tpch1s.part, hive.tpch1s.supplier, hive.tpch1s.lineitem, hive.tpch1s.orders, hive.tpch1s.customer, hive.tpch1s.nation n1, hive.tpch1s.nation n2, hive.tpch1s.region where p_partkey = l_partkey and s_suppkey = l_suppkey and l_orderkey = o_orderkey and o_custkey = c_custkey and c_nationkey = n1.n_nationkey and n1.n_regionkey = r_regionkey and r_name = 'ASIA' and s_nationkey = n2.n_nationkey and o_orderdate between date '1995-01-01' and date '1996-12-31' and p_type = 'LARGE BRUSHED BRASS') as all_nations group by o_year order by o_year {code} -- This message was sent by Atlassian Jira (v8.20.10#820010)