GitHub user kkhatua opened a pull request: https://github.com/apache/drill/pull/922
DRILL-5741: During startup Drill should not exceed the available memory Providing an environment variable - DRILLBIT_MAX_PROC_MEM to ensure that a Drillbit's max memory parameters, cumulatively, don't exceed the value specified. The variable can be defined in KB, MB, or GB; similar in syntax to how the JVM MaxHeap is specified. e.g. ``` DRILLBIT_MAX_PROC_MEM=13G DRILLBIT_MAX_PROC_MEM=8192m DRILLBIT_MAX_PROC_MEM=4194304K ``` In addition, you can specify it as a percent of the available free memory prior to the Drillbit starting up: `DRILLBIT_MAX_PROC_MEM=40% ` For a system with with 28GB free memory, when set to 40% the Drillbit (with settings defined in drill-env.sh) fails startup with the following message: ``` 2017-08-25 14:58:57 [ERROR] Unable to start Drillbit due to memory constraint violations Max Memory Permitted : 40% (11 of 28 GB free memory) Total Memory Requested : 26 GB Check the following settings to possibly modify (or increase the Max Memory Permitted): -Xmx8g -XX:MaxDirectMemorySize=16g -XX:ReservedCodeCacheSize=1G -XX:MaxPermSize=512M ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/kkhatua/drill DRILL-5741 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/drill/pull/922.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #922 ---- commit 04b04d8e4b0c24fef053bbbd6ecebfaebad99d9c Author: Kunal Khatua <kkha...@maprtech.com> Date: 2017-08-25T23:35:37Z DRILL-5741: During startup Drill should not exceed the available memory Providing an environment variable - DRILLBIT_MAX_PROC_MEM to ensure that a Drillbit's max memory parameters, cumulatively, don't exceed the value specified. The variable can be defined in KB, MB, or GB; similar in syntax to how the JVM MaxHeap is specified. e.g. ``` DRILLBIT_MAX_PROC_MEM=13G DRILLBIT_MAX_PROC_MEM=8192m DRILLBIT_MAX_PROC_MEM=4194304K ``` In addition, you can specify it as a percent of the available free memory prior to the Drillbit starting up: `DRILLBIT_MAX_PROC_MEM=40% ` For a system with with 28GB free memory, when set to 40% the Drillbit (with settings defined in drill-env.sh) fails startup with the following message: ``` 2017-08-25 14:58:57 [ERROR] Unable to start Drillbit due to memory constraint violations Max Memory Permitted : 40% (11 of 28 GB free memory) Total Memory Requested : 26 GB Check the following settings to possibly modify (or increase the Max Memory Permitted): -Xmx8g -XX:MaxDirectMemorySize=16g -XX:ReservedCodeCacheSize=1G -XX:MaxPermSize=512M ``` ---- --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---