Kunal Khatua created DRILL-5741: ----------------------------------- Summary: Drillbit during startup should not exceed the available memory on a node Key: DRILL-5741 URL: https://issues.apache.org/jira/browse/DRILL-5741 Project: Apache Drill Issue Type: Improvement Components: Server Affects Versions: 1.11.0 Reporter: Kunal Khatua Fix For: 1.12.0
Currently, during startup, a Drillbit can be assigned large values for the following: * Xmx (Heap) * XX:MaxDirectMemorySize * XX:ReservedCodeCacheSize * XX:MaxPermSize All of this, potentially, can exceed the available memory on a system when a Drillbit is under heavy load. It would be good to have the Drillbit ensure during startup itself that the cumulative value of these parameters does not exceed a pre-defined upper limit for the Drill process. The proposal is to have the [runbit|https://github.com/apache/drill/blob/master/distribution/src/resources/runbit] script look for an additional environment variable: {{DRILLBIT_MAX_PROC_MEM}} The parameter can specify the maximum in GB/MB (similar in syntax to how the Java's MaxHeap is defined), or in terms of percentage of available memory (not to exceed 95%). The [runbit|https://github.com/apache/drill/blob/master/distribution/src/resources/runbit] script will perform the calculation of the sum of memory required by the memory spaces (heap, direct, etc) and ensure that it is within the limit defined by the {{DRILLBIT_MAX_PROC_MEM}} env variable. In the absence of this parameter, there will be no restriction. A node admin can then define this variable in the default terminal's environment (e.g. {{/root/.bashrc}} ) files. -- This message was sent by Atlassian JIRA (v6.4.14#64029)