Volodymyr Vysotskyi created DRILL-7372: ------------------------------------------
Summary: MethodAnalyzer consumes too many memory Key: DRILL-7372 URL: https://issues.apache.org/jira/browse/DRILL-7372 Project: Apache Drill Issue Type: Bug Affects Versions: 1.16.0 Reporter: Volodymyr Vysotskyi Assignee: Volodymyr Vysotskyi Fix For: 1.17.0 In the scope of DRILL-6524 was added logic for determining whether a variable is assigned in conditional block to prevent incorrect scalar replacement for such cases. But for some queries, this logic consumes too many memory, for example, for the following query: {code:sql} SELECT * FROM cp.`employee.json` WHERE employee_id+0 < employee_id OR employee_id+1 < employee_id AND employee_id+2 < employee_id OR employee_id+3 < employee_id AND employee_id+4 < employee_id OR employee_id+5 < employee_id AND employee_id+6 < employee_id OR employee_id+7 < employee_id AND employee_id+8 < employee_id OR employee_id+9 < employee_id AND employee_id+10 < employee_id OR employee_id+11 < employee_id AND employee_id+12 < employee_id OR employee_id+13 < employee_id AND employee_id+14 < employee_id OR employee_id+15 < employee_id AND employee_id+16 < employee_id OR employee_id+17 < employee_id AND employee_id+18 < employee_id OR employee_id+19 < employee_id AND employee_id+20 < employee_id OR employee_id+21 < employee_id AND employee_id+22 < employee_id OR employee_id+23 < employee_id AND employee_id+24 < employee_id OR employee_id+25 < employee_id AND employee_id+26 < employee_id OR employee_id+27 < employee_id AND employee_id+28 < employee_id OR employee_id+29 < employee_id AND employee_id+30 < employee_id OR employee_id+31 < employee_id AND employee_id+32 < employee_id OR employee_id+33 < employee_id AND employee_id+34 < employee_id OR employee_id+35 < employee_id AND employee_id+36 < employee_id OR employee_id+37 < employee_id AND employee_id+38 < employee_id OR employee_id+39 < employee_id AND employee_id+40 < employee_id OR employee_id+41 < employee_id AND employee_id+42 < employee_id OR employee_id+43 < employee_id AND employee_id+44 < employee_id OR employee_id+45 < employee_id AND employee_id+46 < employee_id OR employee_id+47 < employee_id AND employee_id+48 < employee_id OR employee_id+49 < employee_id AND TRUE; {code} Drill consumes more than 6 GB memory. One of the issues to fix is to replace {{Deque<Set<Integer>> localVariablesSet;}} with {{Deque<BitSet>}}, it will reduce memory usage significantly. Additionally should be investigated why these objects cannot be collected by GC. -- This message was sent by Atlassian Jira (v8.3.2#803003)