Laszlo Gaal has uploaded this change for review. ( http://gerrit.cloudera.org:8080/17331
Change subject: IMPALA-10669: Limit memory during dataload in Docker-based tests ...................................................................... IMPALA-10669: Limit memory during dataload in Docker-based tests When Docker-based tests are run on machines with lots of RAM (e.g AWS m5.12xlarge with 192GB of RAM), creating ORC files with complex types often caused Tez containers to attempt to exceed their memory limits. This resulted in Yarn terminating such containers, making the dataload phase and the shole build fail. Since this type of failure was only observed on hosts with very high amounts of memory, the workaround is to put an artificial constrain on the memory available for the build container (this is the Docker container in which Impala is built and the initial dataload is performed). Memory for this container is limited to 128GB, which is still much more than dataload requires. It also matches the native memory size of an AWS r5.4xlarge instance, which routinely runs Impala builds in our downstream environment. Change-Id: I5100589ffc54184b6ca13db139cb983175c934eb --- M docker/test-with-docker.py 1 file changed, 5 insertions(+), 0 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/31/17331/1 -- To view, visit http://gerrit.cloudera.org:8080/17331 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5100589ffc54184b6ca13db139cb983175c934eb Gerrit-Change-Number: 17331 Gerrit-PatchSet: 1 Gerrit-Owner: Laszlo Gaal <laszlo.g...@cloudera.com>
