perhaps you should look at how ceph has dealt with a similar issue, they set the max number of cpus based on the system ram. https://src.fedoraproject.org/rpms/ceph/blob/rawhide/f/ceph.spec#_1246
Dennis On Fri, Mar 26, 2021 at 7:49 PM Michel Alexandre Salim <mic...@michel-slm.name> wrote: > > Hi all, > > This idea came about when I'm debugging build issues with mcrouter, > which turns out to be due to build jobs failing to allocate memory and > getting terminated without aborting the entire compilation, causing > link issues when empty or corrupted objects are encountered: > > https://src.fedoraproject.org/rpms/mcrouter/blob/rawhide/f/mcrouter.spec#_4-8 > > As a rough estimate it seems like each of the CPU core passed with > %{_smp_build_ncpus} ended up consuming close to 8 GB of RAM. And that's > with LTO disabled (yeah, it's not a good situation to be in). > > Right now I'm just overriding _smp_build_ncpus to 1, but there is a > more elegant solution I'd like to propose: > > What if one can declaratively set the required RAM per build job -- > either with a single macro, or maybe two if the LTO usecase requires > even more RAM. e.g. to declare each core might take up to 8 GB: > > %global _smp_build_ram_per_cpu 8192 > > then in case this is run on our aarch64 builder with 40GB RAM, > dynamically take the minimum of the existing _smp_build_ncpus (which > AIUI is determined by the number of cores on the machine) and (amount > of RAM / _smp_build_ram_per_cpu), in this case capping the actual > number passed to -j to 5. > > Is there interest in having this be available? I could imagine it might > be useful for other resource-intensive package builds e.g. for > Chromium. > > Best regards, > > -- > Michel Alexandre Salim > profile: https://keyoxide.org/mic...@michel-slm.name > _______________________________________________ > devel mailing list -- devel@lists.fedoraproject.org > To unsubscribe send an email to devel-le...@lists.fedoraproject.org > Fedora Code of Conduct: > https://docs.fedoraproject.org/en-US/project/code-of-conduct/ > List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines > List Archives: > https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org > Do not reply to spam on the list, report it: > https://pagure.io/fedora-infrastructure _______________________________________________ devel mailing list -- devel@lists.fedoraproject.org To unsubscribe send an email to devel-le...@lists.fedoraproject.org Fedora Code of Conduct: https://docs.fedoraproject.org/en-US/project/code-of-conduct/ List Guidelines: https://fedoraproject.org/wiki/Mailing_list_guidelines List Archives: https://lists.fedoraproject.org/archives/list/devel@lists.fedoraproject.org Do not reply to spam on the list, report it: https://pagure.io/fedora-infrastructure
