Hi
Op 29-11-2023 om 10:45 schreef Martin Jansa:
I was experimenting quite a bit with this in:
https://github.com/shr-project/test-oe-build-time/commits/results
<https://github.com/shr-project/test-oe-build-time/commits/results> e.g.:
https://github.com/shr-project/test-oe-build-time/commit/d5111f4472ac397c0f1197eb6366ac7d2e56453f
<https://github.com/shr-project/test-oe-build-time/commit/d5111f4472ac397c0f1197eb6366ac7d2e56453f>
https://github.com/shr-project/test-oe-build-time/commit/e97d8fe55fcf848416a8494a90da4ae6418b19fe
<https://github.com/shr-project/test-oe-build-time/commit/e97d8fe55fcf848416a8494a90da4ae6418b19fe>
but be aware that reasonable BB_PRESSURE_MAX_ values are different for
kirkstone and master, see:
https://lists.openembedded.org/g/bitbake-devel/message/15499
<https://lists.openembedded.org/g/bitbake-devel/message/15499>
which wasn't merged yet.
And for my system and typical builds BB_PRESSURE_MAX_* isn't able to
prevent OOMK, because pressure is still fine when bitbake is starting
chromium.do_compile and nodejs.do_compile tasks at the same time and
OOMK gets triggered later even when only these 2 tasks are running in
parallel with other tasks not started thanks to BB_PRESSURE_MAX_*.
So I'm setting a bit lower PARALLEL_MAKE for chromium instead as in:
BB_PRESSURE_MAX_CPU = "200000"
BB_NUMBER_THREADS = "8"
BB_NUMBER_PARSE_THREADS = "64"
PARALLEL_MAKE = "-j 70 -l 140"
PARALLEL_MAKE:pn-qtwebengine = "-j 40"
PARALLEL_MAKE:pn-webruntime = "-j 40"
which works well for my system and my typical builds, YMMV for yours.
Regards,
On Wed, Nov 29, 2023 at 10:21 AM Robert P. J. Day
<rpj...@crashcourse.ca
<mailto:rpj...@crashcourse.ca>> wrote:
a colleague is using Wind River Linux and is getting occasional OOM
errors when taking advantage of maximum parallelism, so he took the
quick way out and dialed things back using BB_NUMBER_THREADS and
PARALLEL_MAKE.
however, i am aware of the bitbake variables
BB_PRESSURE_MAX_{IO,CPU,MEMORY} and was wondering if there is a decent
writeup on how to use, perhaps, BB_PRESSURE_MAX_MEMORY, to dynamically
detect imminent OOM and adjust things. that is, how to calculate a
sane value for that as i haven't yet dug into the values under
/proc/pressure and how to interpret them?
i've seen mentions of combining the above with the "memstat"
command, and other approaches. thoughts?
rday
p.s. is there a docs page that expands on this stuff?
There was an older thread "bitbake controlling memory use", maybe that
might help.
Myself, I found these pressure things don't help. And even though we can
set the number of make threads we still can have a number of makes in
parallel. I patched bitbake to have make and ninja use a jobserver to
limit the total number of compile threads to the number of cores.
See https://github.com/htot/meta-intel-edison/tree/kirkstone
So, if you have 1GB ram x #cores you should be fine except for 1 thing:
nodejs is linking 5 executables that require 4GB ram simultaneously. So
if you don't have 20GB you will start thrashing up to the point that you
have no mouse movement. This I solved by modifying the recipe to
serialize the linking.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#61790): https://lists.yoctoproject.org/g/yocto/message/61790
Mute This Topic: https://lists.yoctoproject.org/mt/102868689/21656
Group Owner: yocto+ow...@lists.yoctoproject.org
Unsubscribe: https://lists.yoctoproject.org/g/yocto/unsub
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
