[PR] DRILL-8489: Sender memory leak when rpc encode exception (drill)

[via GitHub]

shfshihuafeng opened a new pull request, #2901:
URL: https://github.com/apache/drill/pull/2901

   # [DRILL-8489](https://issues.apache.org/jira/browse/DRILL-8489): Sender 
memory leak when rpc encode exception
   
   ## Description
   
   When encode throw Exception, if encode msg  instanceof ReferenceCounted, 
netty can release msg, but drill convert msg to OutboundRpcMessage, so netty 
can not release msg. 
   
   ## Documentation
   (Please describe user-visible changes similar to what should appear in the 
Drill documentation.)
   
   ## Testing
   1. export DRILL_MAX_DIRECT_MEMORY=${DRILL_MAX_DIRECT_MEMORY:-"2G"}
   2. tpch 1s
   3. tpch sql 8
   
   ```
   select
   o_year,
   sum(case when nation = 'CHINA' then volume else 0 end) / sum(volume) as 
mkt_share
   from (
   select
   extract(year from o_orderdate) as o_year,
   l_extendedprice * 1.0 as volume,
   n2.n_name as nation
   from hive.tpch1s.part, hive.tpch1s.supplier, hive.tpch1s.lineitem, 
hive.tpch1s.orders, hive.tpch1s.customer, hive.tpch1s.nation n1, 
hive.tpch1s.nation n2, hive.tpch1s.region
   where
   p_partkey = l_partkey
   and s_suppkey = l_suppkey
   and l_orderkey = o_orderkey
   and o_custkey = c_custkey
   and c_nationkey = n1.n_nationkey
   and n1.n_regionkey = r_regionkey
   and r_name = 'ASIA'
   and s_nationkey = n2.n_nationkey
   and o_orderdate between date '1995-01-01'
   and date '1996-12-31'
   and p_type = 'LARGE BRUSHED BRASS') as all_nations
   group by o_year
   order by o_year;   
   ```
   
   5. This scenario is relatively easy to Reproduce by running the following 
script
   ```
   drill_home=/data/shf/apache-drill-1.22.0-SNAPSHOT/bin
   fileName=/data/shf/1s/shf.txt
   
   random_sql(){
   #for i in `seq 1 3`
   while true
   do
     num=$((RANDOM%22+1))
     if [ -f $fileName ]; then
     echo "$fileName" " is exit"
     exit 0
     else
             $drill_home/sqlline -u 
\"jdbc:drill:zk=jupiter-2:2181/drill_shf/jupiterbits_shf1\" -f tpch_sql8.sql >> 
sql8.log 2>&1
     fi
   done
   }
   main(){
   unset HADOOP_CLASSPATH
   #TPCH power test
   for i in `seq 1 25`
   do
           random_sql &
   done
   
   
   }
   
   ```


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@drill.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org