Re: Building Flink on VirtualBox VM failing

2020-10-28 Thread Khachatryan Roman
The values printed by the OOM killer seem indeed strange. But from the line above the memory usage seems fine: rss=2440960. Running the given command I see only one forked process. Probably, this is an issue of OOM killer running in VM on Wwindows host. Can you try with OOM killer disabled? Regard

Re: Building Flink on VirtualBox VM failing

2020-10-23 Thread Juha Mynttinen
I'm trying again running the tests, now I have four cores (previously five) and 12 GB RAM (previously 8 GB). I'm still hit by the OOM killer. The command I'm running is: mvn -Dflink.forkCount=1 -Dflink.forkCountTestPackage=1 clean verify [INFO] BUILD FAILURE [INFO] --

Re: Building Flink on VirtualBox VM failing

2020-10-21 Thread Juha Mynttinen
Hmm Even when setting the forkcounts to 1 things fail. I wonder why there seem to be five of these JVM crashes. There should be one JVM at time. And Maven should fail after the 1st fail? ~/apache-maven-3.2.5/bin/mvn -Dflink.forkCount=1 -Dflink.forkCountTestPackage=1 clean verify [INFO]

Re: Building Flink on VirtualBox VM failing

2020-10-21 Thread Juha Mynttinen
Hi, You're right, I thought about this also after writing the last comment - for example on Linux, the Kernel by default overcommits memory allocations and this approach doesn't work (doesn't make JVM crash right when it starts). I dug a little deeper. It seems that for ci-environments there are

Re: Building Flink on VirtualBox VM failing

2020-10-20 Thread Khachatryan Roman
I think you are right and I like the idea of failing the build fast. However, when trying this approach on my local machine it didn't help: the build didn't crash (probably, because of overcommit). Did you try this approach in your VM? Regards, Roman On Tue, Oct 20, 2020 at 12:12 PM Juha Mynttin

Re: Building Flink on VirtualBox VM failing

2020-10-20 Thread Juha Mynttinen
Hey, > Currently, tests do not run in parallel I don't think this is true, at least 100%. In 'top' it's clearly visible that there are multiple JVMs. If not running tests in parallel, what are these doing? In the main pom.xml there's configuration for the plug-in 'maven-surefire-plugin'. I'm not

Re: Building Flink on VirtualBox VM failing

2020-10-20 Thread Khachatryan Roman
Thanks for sharing this, I think the activity of OOM-Killer means high memory pressure (it just kills a process with the highest score of memory consumption). High CPU usage can only be a consequence of it, being constant GC. Currently, tests do not run in parallel, but high memory usage can be ca

Re: Building Flink on VirtualBox VM failing

2020-10-19 Thread Juha Mynttinen
Hey, Good hint that /var/log/kern.log. This time I can see this: Oct 20 09:44:48 ubuntu kernel: [ 1925.651551] oom-kill:constraint=CONSTRAINT_NONE,nodemask=(null),cpuset=/,mems_allowed=0,global_oom,task_memcg=/user.slice/user-1000.slice/user@1000.service ,task=java,pid=270024,uid=1000 Oct 20 09:4

Re: Building Flink on VirtualBox VM failing

2020-10-19 Thread Khachatryan Roman
Hey, One reason could be that a resource-intensive test was killed by oom killer. You can inspect /var/log/kern.log for the related messages in your VM. Regards, Roman On Mon, Oct 19, 2020 at 5:57 PM Juha Mynttinen wrote: > > Hey, > > I'm trying to build Flink and failing. I'm running Ubuntu

Building Flink on VirtualBox VM failing

2020-10-19 Thread Juha Mynttinen
Hey, I'm trying to build Flink and failing. I'm running Ubuntu 20.04.1 in a virtual machine on Windows 10. I'm using OpenJDK 11.0.8. I'm on the master branch, commit 9eae578ae592254d54bc51c679644e8e84c65152. The command I'm using: apache-maven-3.2.5/bin/mvn clean verify The output: [INFO] Flin