[I] mergejoin memory leak when Depleting incoming batches throw exception (drill)
shfshihuafeng opened a new issue, #2876: URL: https://github.com/apache/drill/issues/2876 Before submitting a bug report, please verify that you are using the most current version of Drill. **Describe the bug** mergejoin memory leak when Depleting incoming batches throw exception. because we could close rightIterator when leftIterator throw exception **To Reproduce** Steps to reproduce the behavior: 1. prepare data for tpch 1s 2. 20 concurrent for tpch sql8 3. set direct memory 5g 4. when it had OutOfMemoryException , stopped all sql. 5.finding memory leak **Expected behavior** when all sql sop , we should find direct memory is 0 AND could not find leak log like following. `Allocator(op:2:0:11:MergeJoinPOP) 100/73728/4874240/100 (res/actual/peak/limit)` **Error detail, log output or screenshots** `Unable to allocate buffer of size XX (rounded from XX) due to memory limit (). Current allocation: xx` **Drill version** The version of Drill you encountered the issue in. **Additional context** // code placeholder `select o_year, sum(case when nation = 'CHINA' then volume else 0 end) / sum(volume) as mkt_share from ( select extract(year from o_orderdate) as o_year, l_extendedprice * 1.0 as volume, n2.n_name as nation from hive.tpch1s.part, hive.tpch1s.supplier, hive.tpch1s.lineitem, hive.tpch1s.orders, hive.tpch1s.customer, hive.tpch1s.nation n1, hive.tpch1s.nation n2, hive.tpch1s.region where p_partkey = l_partkey and s_suppkey = l_suppkey and l_orderkey = o_orderkey and o_custkey = c_custkey and c_nationkey = n1.n_nationkey and n1.n_regionkey = r_regionkey and r_name = 'ASIA' and s_nationkey = n2.n_nationkey and o_orderdate between date '1995-01-01' and date '1996-12-31' and p_type = 'LARGE BRUSHED BRASS') as all_nations group by o_year order by o_year` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@drill.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] mergejoin memory leak when Depleting incoming batches throw exception (drill)
shfshihuafeng commented on issue #2876: URL: https://github.com/apache/drill/issues/2876#issuecomment-1907419612 i fixed it see attachment [0001-mergejoin-leak.patch](https://github.com/apache/drill/files/14033613/0001-mergejoin-leak.patch) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@drill.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] mergejoin memory leak when Depleting incoming batches throw exception (drill)
shfshihuafeng commented on issue #2876: URL: https://github.com/apache/drill/issues/2876#issuecomment-1907421433 [https://github.com/apache/drill/issues/2871](url) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@drill.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] mergejoin memory leak when Depleting incoming batches throw exception (drill)
shfshihuafeng commented on issue #2876: URL: https://github.com/apache/drill/issues/2876#issuecomment-1907424021 scenario is reliably repeated by above test ,i fixed 。i can not find leak -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@drill.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] mergejoin memory leak when Depleting incoming batches throw exception (drill)
cgivre closed issue #2876: mergejoin memory leak when Depleting incoming batches throw exception URL: https://github.com/apache/drill/issues/2876 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@drill.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
