[jira] [Commented] (DRILL-8489) Sender memory leak when rpc encode exception

[ASF GitHub Bot (Jira)]

    [ 
https://issues.apache.org/jira/browse/DRILL-8489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17837949#comment-17837949
 ] 

ASF GitHub Bot commented on DRILL-8489:
---------------------------------------

shfshihuafeng opened a new pull request, #2901:
URL: https://github.com/apache/drill/pull/2901

   # [DRILL-8489](https://issues.apache.org/jira/browse/DRILL-8489): Sender 
memory leak when rpc encode exception
   
   ## Description
   
   When encode throw Exception, if encode msg  instanceof ReferenceCounted, 
netty can release msg, but drill convert msg to OutboundRpcMessage, so netty 
can not release msg. 
   
   ## Documentation
   (Please describe user-visible changes similar to what should appear in the 
Drill documentation.)
   
   ## Testing
   1. export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"2G"}
   2. tpch 1s
   3. tpch sql 8
   
   ```
   select
   o_year,
   sum(case when nation = 'CHINA' then volume else 0 end) / sum(volume) as 
mkt_share
   from (
   select
   extract(year from o_orderdate) as o_year,
   l_extendedprice * 1.0 as volume,
   n2.n_name as nation
   from hive.tpch1s.part, hive.tpch1s.supplier, hive.tpch1s.lineitem, 
hive.tpch1s.orders, hive.tpch1s.customer, hive.tpch1s.nation n1, 
hive.tpch1s.nation n2, hive.tpch1s.region
   where
   p_partkey = l_partkey
   and s_suppkey = l_suppkey
   and l_orderkey = o_orderkey
   and o_custkey = c_custkey
   and c_nationkey = n1.n_nationkey
   and n1.n_regionkey = r_regionkey
   and r_name = 'ASIA'
   and s_nationkey = n2.n_nationkey
   and o_orderdate between date '1995-01-01'
   and date '1996-12-31'
   and p_type = 'LARGE BRUSHED BRASS') as all_nations
   group by o_year
   order by o_year;   
   ```
   
   5. This scenario is relatively easy to Reproduce by running the following 
script
   ```
   drill_home=/data/shf/apache-drill-1.22.0-SNAPSHOT/bin
   fileName=/data/shf/1s/shf.txt
   
   random_sql(){
   #for i in `seq 1 3`
   while true
   do
     num=$((RANDOM%22+1))
     if [ -f $fileName ]; then
     echo "$fileName" " is exit"
     exit 0
     else
             $drill_home/sqlline -u 
\"jdbc:drill:zk=jupiter-2:2181/drill_shf/jupiterbits_shf1\" -f tpch_sql8.sql >> 
sql8.log 2>&1
     fi
   done
   }
   main(){
   unset HADOOP_CLASSPATH
   #TPCH power test
   for i in `seq 1 25`
   do
           random_sql &
   done
   
   
   }
   
   ```




> Sender memory leak when rpc encode exception
> --------------------------------------------
>
>                 Key: DRILL-8489
>                 URL: https://issues.apache.org/jira/browse/DRILL-8489
>             Project: Apache Drill
>          Issue Type: Bug
>          Components:  Server
>    Affects Versions: 1.21.1
>            Reporter: shihuafeng
>            Priority: Major
>             Fix For: 1.22.0
>
>




--
This message was sent by Atlassian Jira
(v8.20.10#820010)