Joe McDonnell created IMPALA-11370:
--------------------------------------
Summary: Run make with --load-average for large toolchain
components
Key: IMPALA-11370
URL: https://issues.apache.org/jira/browse/IMPALA-11370
Project: IMPALA
Issue Type: Improvement
Components: Infrastructure
Affects Versions: Impala 4.2.0
Reporter: Joe McDonnell
Several components of the toolchain can use a significant amount of parallelism
and resources (e.g. LLVM, Kudu). When using the docker build infrastructures,
multiple of these builds can happen simultaneously. This can result in
overloading the machine and high memory usage / memory pressure.
One way to deal with it is to run those builds with make's --load-average
command so that the total load doesn't go too high. This can result in a
sawtooth load pattern as the make commands cut off and the load drifts down, so
this may need some tuning of the value.
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
