Hi Neale, Yea they do require a lot of memory - the same is true for x86. Is there a way to specify the max number of these? Or is that done with -j?
Would it be worthwhile to investigate if it's possible to reduce the memory requirements of these? Is there a way to clear the cache so that I could run make verify back to back without deleting and recloning the vpp repo? ccache -C didn't work for me. Thanks, Juraj From: Neale Ranns (nranns) [mailto:nra...@cisco.com] Sent: Thursday, August 2, 2018 11:11 AM To: Juraj Linkeš <juraj.lin...@pantheon.tech>; vpp-dev@lists.fd.io Subject: Re: [vpp-dev] Large memory spike during make verify on ARM machine ThunderX Hi Juraj, I couldn’t say how much each compile ‘should’ use, but it has been noted in the past that these template heavy C++ files do require a lot of memory to compile. With the many cores you have, then that’s a lot in total. ‘make wipe’ does not clear the ccache, so any subsequent builds will require less memory because the compile is skipped. /neale From: <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> on behalf of Juraj Linkeš <juraj.lin...@pantheon.tech<mailto:juraj.lin...@pantheon.tech>> Date: Thursday, 2 August 2018 at 10:10 To: "Neale Ranns (nranns)" <nra...@cisco.com<mailto:nra...@cisco.com>>, "vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>" <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> Subject: Re: [vpp-dev] Large memory spike during make verify on ARM machine ThunderX Hi Neale, I'm not specifying -j, but I see a lot of processes running in parallel when the spike is happening. The processes are attached. They utilized most of 96 available cores and most of them used more than 400MB - is that how much they should be using? Also, here's the gcc version on the box: gcc -v Using built-in specs. COLLECT_GCC=gcc COLLECT_LTO_WRAPPER=/usr/lib/gcc/aarch64-linux-gnu/5/lto-wrapper Target: aarch64-linux-gnu Configured with: ../src/configure -v --with-pkgversion='Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.4' --with-bugurl=file:///usr/share/doc/gcc-5/README.Bugs --enable-languages=c,ada,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-5 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-libquadmath --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-5-arm64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-5-arm64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-5-arm64 --with-arch-directory=aarch64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-multiarch --enable-fix-cortex-a53-843419 --disable-werror --enable-checking=release --build=aarch64-linux-gnu --host=aarch64-linux-gnu --target=aarch64-linux-gnu Thread model: posix gcc version 5.4.0 20160609 (Ubuntu/Linaro 5.4.0-6ubuntu1~16.04.4) Thanks, Juraj From: Neale Ranns (nranns) [mailto:nra...@cisco.com] Sent: Wednesday, August 1, 2018 5:09 PM To: Juraj Linkeš <juraj.lin...@pantheon.tech<mailto:juraj.lin...@pantheon.tech>>; vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> Subject: Re: [vpp-dev] Large memory spike during make verify on ARM machine ThunderX Hi Juraj, How many parallel compiles do you have? What’s the j factor /neale From: <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> on behalf of Juraj Linkeš <juraj.lin...@pantheon.tech<mailto:juraj.lin...@pantheon.tech>> Date: Wednesday, 1 August 2018 at 16:59 To: "vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>" <vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>> Subject: [vpp-dev] Large memory spike during make verify on ARM machine ThunderX Hi vpp-devs, I noticed that during a specific portion of make verify build on an ARM ThunderX machine the build consumes a lot of memory - around 25GB. I can identify the spot in the logs: Jul 31 03:12:48 CXX gbp_contract.lo 25GB memory hog Jul 31 03:16:13 CXXLD libvom.la but not much else. I created a ticket<https://jira.fd.io/browse/VPP-1371> which contains some more information. I didn't see this memory spike when trying to reproducing the behavior on my x86 laptop. Does anyone has any idea what could be the cause or how to debug this? Thanks, Juraj
-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#10029): https://lists.fd.io/g/vpp-dev/message/10029 Mute This Topic: https://lists.fd.io/mt/24005970/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
