[ https://issues.apache.org/jira/browse/IMPALA-9754?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Qifan Chen resolved IMPALA-9754. -------------------------------- Fix Version/s: Impala 4.0 Resolution: Fixed > buffer_pool_limit error message is confusing > -------------------------------------------- > > Key: IMPALA-9754 > URL: https://issues.apache.org/jira/browse/IMPALA-9754 > Project: IMPALA > Issue Type: Improvement > Components: Backend > Reporter: Sahil Takiar > Assignee: Qifan Chen > Priority: Minor > Labels: newbie, ramp-up > Fix For: Impala 4.0 > > > The error message for an invalid value of {{buffer_pool_limit}} is confusing. > For example: > {code:java} > I0515 18:30:20.143811 1 exec-env.cc:480] System memory available: 30.63 > GB (from physical mem) > I0515 18:30:20.144037 1 exec-env.cc:487] CGroup memory limit for this > process reduces physical memory available to: 9.31 GB > I0515 18:30:20.144050 1 exec-env.cc:503] Using process memory limit: 7.45 > GB (--mem_limit=80% of 9.31 GB) > I0515 18:30:20.150154 1 status.cc:129] Invalid --buffer_pool_limit value, > must be a percentage or positive bytes value or percentage: 85% > @ 0xc33b79 > @ 0x111afb8 > @ 0x1251256 > @ 0xb730b3 > @ 0x7f60da4c9554 > @ 0xbf3e86 > F0515 18:30:20.150177 1 impalad-main.cc:73] Invalid --buffer_pool_limit > value, must be a percentage or positive bytes value or percentage: 85% > . Impalad exiting. {code} > 1: "must be a percentage or positive bytes value or percentage" needs to be > revised to something like "must be a positive bytes value or percentage" > 2: The error message makes it sound like the value of 85% is invalid, when > in reality, it is a valid value - Impala does some internal calculations of > what the actual {{buffer_pool_limit}} should be internally (see > ExecEnv::Init) - these calculations first compute the mem limit for the > process (ChooseProcessMemLimit) based on the physical machines memory + > cgroup settings, and then if mem_limit_includes_jvm == true it subtracts the > value of Xmx from the amount of available memory - we don't have logic to > capture if the results of that subtraction becomes a negative number, which > causes the buffer_pool_limit parsing to fail in a weird way. We should add a > check to see if {{admit_mem_limit_ \-= > JvmMemoryMetric::HEAP_MAX_USAGE->GetValue()}} is negative, and then exit with > an informative status message -- This message was sent by Atlassian Jira (v8.3.4#803005)