[gentoo-user] Re: Installing grub2 in ext4 partition
On 2023-02-16, Grant Edwards wrote: > A while back, I stumbled across a web page or blog entry that > explained how to safely use grub2 with it installed in a partition > instead of a device (e.g. grub is installed in /dev/sda10 instead of > /dev/sda). And of course 20 minutes after I posted this I managed to find it: https://github.com/alberthdev/alberthdev-misc/wiki/Installing-GRUB2-to-a-Partition
[gentoo-user] Installing grub2 in ext4 partition
A while back, I stumbled across a web page or blog entry that explained how to safely use grub2 with it installed in a partition instead of a device (e.g. grub is installed in /dev/sda10 instead of /dev/sda). IIRC, it involved locking a couple key files in the /boot/grub directory so that they can't be moved or modified. This is required because the grub code installed in the boot sector of the partition uses a raw block list to find those key files (grub doesn't actually understand the filesystem at that point). I thought I had saved that info, but now I need it I can't find it — and Google is unable to find the page again (perhaps it's gone). Does this ring a bell with anybody? -- Grant
RE: [gentoo-user] Jobs and load-average
> -Original Message- > From: Rich Freeman > Sent: Thursday, February 16, 2023 6:24 AM > To: gentoo-user@lists.gentoo.org > Subject: Re: [gentoo-user] Jobs and load-average > > On Thu, Feb 16, 2023 at 8:39 AM Peter Humphrey wrote: > > > > I've just looked at 'man make', from which it's clear that -j = > > --jobs, and that both those and --load-average are passed to > > /usr/bin/make, presumably untouched unless portage itself has > > identically named variables. So I wonder how feasible it might be for > > make to incorporate its own checks to ensure that the load average is > > not exceeded. I am not a programmer (not for at least 35 years, anyway), so > > I have to leave any such suggestion to the experts. > > > > Well, if we just want to have a fun discussion here are my thoughts. > However, the complexity vs usefulness outside of Gentoo is such that I don't > see it happening. > > For the most typical use case - a developer building the same thing over and > over (which isn't Gentoo), then make could cache info on resources consumed, > and use that to make more educated decisions about how many tasks to launch. > That wouldn't help us at all, but it would help the typical make user. > However, the typical make user can just tune things in other ways. > > It isn't going to be possible for make to estimate build complexity in any > practical way. Halting problem aside maybe you could build in some smarts > looking at the program being executed and its arguments, but it would be a > big mess. > > Something make could do is tune the damping a bit. It could gradually > increase the number of jobs it runs and watch the load average, and gradually > scale it up appropriately, and gradually scale down if CPU is the issue, or > rapidly scale down if swap is the issue. If swapping is detected it could > even suspend most of the tasks it has spawned and then gradually continue > them as other tasks finish to recover from this condition. However, this > isn't going to work as well if portage is itself spawning parallel instances > of make - they'd have to talk to each other or portage would somehow need to > supervise things. > > A way of thinking about it is that when you have portage spawning multiple > instances of make, that is a bit like adding gain to the --load-average > MAKEOPTS. So each instance of make independently looks at load average and > takes action. So you have an output (compilers that create load), then you > sample that load with a time-weighted average, and then you apply gain to > this average, and then use that as feedback. That's basically a recipe for > out of control oscillation. > You need to add damping and get rid of the gain. > > Disclaimer: I'm not an engineer and I suspect a real engineer would be able > to add a bit more insight. > > Really though the issue is that this is the sort of thing that only impacts > Gentoo and so nobody else is likely to solve this problem for us. > > -- > Rich Expanding the capabilities of app-admin/cpulimit to tally up memory usage and suspend new compile jobs temporarily when it gets too high probably wouldn't be too horribly difficult... LMP
RE: [gentoo-user] Re: my 5.15.93 kernel keeps rebooting
>-Original Message- >From: John Covici >Sent: Wednesday, February 15, 2023 7:20 AM >To: gentoo-user@lists.gentoo.org >Subject: Re: [gentoo-user] Re: my 5.15.93 kernel keeps rebooting > >On Wed, 15 Feb 2023 09:50:27 -0500, >Grant Edwards wrote: >> >> On 2023-02-14, Rich Freeman wrote: >> >> > Where are you getting this from, the system log/journal? This >> > doesn't seem like a clean shutdown, so if it is a kernel PANIC I >> > wouldn't expect the most critical info to be in the log (since it >> > will stop syncing to protect the filesystem). The details you need >> > probably will be displayed on the console briefly. You can also >> > enable a network console, which will send the dmesg output >> > continuously over UDP to another device. This won't be interrupted >> > by a PANIC unless there is some issue with the hardware or networking >> > stack. >> >> If you've got a serial port[1], you could also set up serial logging. >> Though using serial ports have become a bit of a lost art, the serial >> console code in the kernel is pretty carefully designed to be the last >> man standing when things start to die. It's possible (though I >> wouldn't say probable) that a serial console will be able to show you >> stuff closer to the event horizon than a network console can. >> >> Anyway, since still I'm in the serial port business (yes, there are >> still plenty of people using serial ports in industrial settings) I >> had to mention it... >> >> [1] For this purpose you want a plain old UART on the motherboard type >> seial port. You'd be surprised how many motherboards still have >> them. Even though they're never brought out to a DB9 connector on >> the back panel, there's often an 8-pin header on the edge of the >> board somewhere, so you'd need one of these: >> >> >> https://www.amazon.com/C2G-27550-Adapter-Bracket-Motherboards/dp/B0002 >> J27R8/ > >I do have one which I use for my speech synthesizer. I also have one on my >other box which I could hook up -- if I can find my null modem cable. I think >I will try the netconsole first and the serial console if that does not work. > >Thanks for the hint. > > https://wiki.gentoo.org/wiki/Kernel_Crash_Dumps is another option if you're somehow not getting enough information out of the console. More complex to set up, but you can take an actual debugger to the result and hopefully find out exactly what's going on. LMP
Re: [gentoo-user] Jobs and load-average
On Thu, 16 Feb 2023 09:24:08 -0500 Rich Freeman wrote: > On Thu, Feb 16, 2023 at 8:39 AM Peter Humphrey wrote: > > > > I've just looked at 'man make', from which it's clear that -j = --jobs, and > > that both those and --load-average are passed to /usr/bin/make, presumably > > untouched unless portage itself has identically named variables. So I wonder > > how feasible it might be for make to incorporate its own checks to ensure > > that > > the load average is not exceeded. I am not a programmer (not for at least 35 > > years, anyway), so I have to leave any such suggestion to the experts. > > > > Well, if we just want to have a fun discussion here are my thoughts. > However, the complexity vs usefulness outside of Gentoo is such that I > don't see it happening. > > For the most typical use case - a developer building the same thing > over and over (which isn't Gentoo), then make could cache info on > resources consumed, and use that to make more educated decisions about > how many tasks to launch. That wouldn't help us at all, but it would > help the typical make user. However, the typical make user can just > tune things in other ways. > > It isn't going to be possible for make to estimate build complexity in > any practical way. Halting problem aside maybe you could build in > some smarts looking at the program being executed and its arguments, > but it would be a big mess. > > Something make could do is tune the damping a bit. It could gradually > increase the number of jobs it runs and watch the load average, and > gradually scale it up appropriately, and gradually scale down if CPU > is the issue, or rapidly scale down if swap is the issue. If swapping > is detected it could even suspend most of the tasks it has spawned and > then gradually continue them as other tasks finish to recover from > this condition. However, this isn't going to work as well if portage > is itself spawning parallel instances of make - they'd have to talk to > each other or portage would somehow need to supervise things. > > A way of thinking about it is that when you have portage spawning > multiple instances of make, that is a bit like adding gain to the > --load-average MAKEOPTS. So each instance of make independently looks > at load average and takes action. So you have an output (compilers > that create load), then you sample that load with a time-weighted > average, and then you apply gain to this average, and then use that as > feedback. That's basically a recipe for out of control oscillation. > You need to add damping and get rid of the gain. > > Disclaimer: I'm not an engineer and I suspect a real engineer would be > able to add a bit more insight. > > Really though the issue is that this is the sort of thing that only > impacts Gentoo and so nobody else is likely to solve this problem for > us. > Given all your explenation and my annoyance a couple of years ago, I hacked a little helper that sits between make and spawned build jobs. Basically what annoyed me is the fact that chromium would compile for hours and then fail, because it would need more memory than memory available, and this would fail the whole build. One possible solution is to reduce the number of build jobs to e.g. -j1 for chromium, but this is stupid because 99% of the time -j16 would work just fine. So I hacked a bit around, and came up with little helper&watcher. The helper would limit spawning new jobs to SOME_LIMIT, and when load is too high (e.g.g I am doing other work on the PC, that's not under emerge's control). The watcher kills memory hungry build jobs, once memory usage higher than 90%, tells the helper to stop spawning new jobs, waits until the helper reports that no more build jobs are running and then respawns the memory hungry build job (i.e. the memory hungry build job will run essentially as if -j1 was specified) This way I can mix emerge --jobs=HIGH_NUMBER and make -jOTHER_HIGH_NUMBER, and it wouldn't affect the system, because the total number of actual build jobs is controlled by the helper, and would never go beyond SOME_LIMIT, even if HIGH_NUMBER*OTHER_HIGH_NUMBER > SOME_LIMIT. I never published this anywhere, but if there's interest in it, I can probably upload it somewhere, but I had the feeling that it's quite hacky and not worth publishing. Also I was never sure if I break emerge in some way, because it's very low-level, but now it's running since more than a year without any emerge failure due to this hijacking.
Re: [gentoo-user] Re: my 5.15.93 kernel keeps rebooting
On Thu, Feb 16, 2023 at 9:08 AM John Covici wrote: > > hmmm, but what should I use for the source ip, I only assign those > when I bring the interface up when I start the interface -- I have > something like this: > [Unit] > Description=Network Connectivity for %i > ... > So, before I run this, I don't think the card has any ip address, does > it? So, "cards" don't have an IP address. The kernel assigns an IP address to an interface, which is entirely a software construct. It happens to be a software construct that the network console feature largely ignores anyway. I didn't go reading the source code, but I'm guessing it is just constructing raw UDP packets and it will happily set the IP to whatever you want it to be. After all, it is just a field on the packet. So you can make the source IP whatever you want it to be. Just expect the packets to show up with the IP you set on them. There is no connection, so the IP doesn't need to be reachable by anything else. You could stick literally anything in there as long as some firewall isn't going to object and drop the packet. The destination IP matters because that is where it is going to go, and the interface matters because if it gets sent out on the wrong interface then obviously it won't make it there. I have no idea if the netconsole packets get seen by netfilter, but if this is on some kind of router that might be something you need to check, because if netfilter is configured to drop unassociated UDP from the firewall to the LAN that could be an issue. However, it is possible this just bypasses netfilter entirely. If you have the dynamic netconsole option enabled you could have a script update the settings after your network is up to set the source IP to the one assigned by DHCP and make sure it is on the right interface. As you point out though at boot time the interface won't have an IP. It won't even be "up," not that this is likely to bother the kernel. -- Rich
Re: [gentoo-user] Jobs and load-average
On Thu, Feb 16, 2023 at 8:39 AM Peter Humphrey wrote: > > I've just looked at 'man make', from which it's clear that -j = --jobs, and > that both those and --load-average are passed to /usr/bin/make, presumably > untouched unless portage itself has identically named variables. So I wonder > how feasible it might be for make to incorporate its own checks to ensure that > the load average is not exceeded. I am not a programmer (not for at least 35 > years, anyway), so I have to leave any such suggestion to the experts. > Well, if we just want to have a fun discussion here are my thoughts. However, the complexity vs usefulness outside of Gentoo is such that I don't see it happening. For the most typical use case - a developer building the same thing over and over (which isn't Gentoo), then make could cache info on resources consumed, and use that to make more educated decisions about how many tasks to launch. That wouldn't help us at all, but it would help the typical make user. However, the typical make user can just tune things in other ways. It isn't going to be possible for make to estimate build complexity in any practical way. Halting problem aside maybe you could build in some smarts looking at the program being executed and its arguments, but it would be a big mess. Something make could do is tune the damping a bit. It could gradually increase the number of jobs it runs and watch the load average, and gradually scale it up appropriately, and gradually scale down if CPU is the issue, or rapidly scale down if swap is the issue. If swapping is detected it could even suspend most of the tasks it has spawned and then gradually continue them as other tasks finish to recover from this condition. However, this isn't going to work as well if portage is itself spawning parallel instances of make - they'd have to talk to each other or portage would somehow need to supervise things. A way of thinking about it is that when you have portage spawning multiple instances of make, that is a bit like adding gain to the --load-average MAKEOPTS. So each instance of make independently looks at load average and takes action. So you have an output (compilers that create load), then you sample that load with a time-weighted average, and then you apply gain to this average, and then use that as feedback. That's basically a recipe for out of control oscillation. You need to add damping and get rid of the gain. Disclaimer: I'm not an engineer and I suspect a real engineer would be able to add a bit more insight. Really though the issue is that this is the sort of thing that only impacts Gentoo and so nobody else is likely to solve this problem for us. -- Rich
Re: [gentoo-user] Re: my 5.15.93 kernel keeps rebooting
address=192.168.0.1 netmask=24 broadcast=192.168.0.255 So, before I run this, I don't think the card has any ip address, does it? >From what you hope the receiving machine arp -a ? HTH, Mark
Re: [gentoo-user] Re: my 5.15.93 kernel keeps rebooting
On Thu, 16 Feb 2023 07:11:12 -0500,
Rich Freeman wrote:
>
> On Thu, Feb 16, 2023 at 6:50 AM John Covici wrote:
> >
> > The sending computer has two nics, eno1 for the internal network and
> > eno2 is on the internet. So, my netconsole stanza said
> > netconsole=@192.168.0.1/eno1,@192.168.0.2
>
> Is CONFIG_NETCONSOLE enabled for your kernel?
>
> I'm not sure if the kernel will assign the names eno1/2 to interfaces
> - I think those might be assigned by udev, which probably won't have
> run before the kernel parses this instruction. You might need to use
> eth0/1 - and your guess is as good as mine which one corresponds to
> which.
>
> If it isn't one of those it might not hurt to put the target mac
> address in there just to be safe. I haven't needed that but maybe
> there are situations where ARP won't work (it would be needed if you
> are crossing subnets, in which case you'd need the gateway MAC). Keep
> in mind that this is a low-level function that doesn't use any
> routing/userspace/etc. It was designed to be robust in the event of a
> PANIC and to be able to be enabled fairly early during boot, so it
> can't rely on the sorts of things we just take for granted with
> networking.
>
> >
> > The box which is at 192.168.0.2 has netcat (windows version) and I
> > tried the following:
> > netcat -u -v -l 192.168.0.2 and I also tried 192.168.0.1
> > which is the ip address of the linux console which I am trying to
> > debug.
> >
> > I also tried 0.0.0.0 which did not work either, but I think the
> > windows firewall was blocking, and I did fix that, but did not try the
> > 0.0.0.0 after that.
> >
>
> So I'm pretty sure that netcat requires listing the destination IP,
> since it has to open a socket to listen on that IP. You can
> optionally set a source address/port in which case it will ignore
> anything else, but by default it will accept packets from any source.
>
> I was definitely going to suggest making sure that a windows firewall
> wasn't blocking the inbound connections. That's fairly default
> behavior on windows.
hmmm, but what should I use for the source ip, I only assign those
when I bring the interface up when I start the interface -- I have
something like this:
[Unit]
Description=Network Connectivity for %i
Documentation=man:ip
Before=network.target
Wants=network.target
BindsTo=sys-subsystem-net-devices-%i.device
After=sys-subsystem-net-devices-%i.device
[Service]
Type=oneshot
RemainAfterExit=yes
EnvironmentFile=/etc/conf.d/network@%i
ExecStart=/bin/ip link set dev %i up
ExecStart=/bin/ip addr add ${address}/${netmask} broadcast ${broadcast} dev %i
ExecStart=-/bin/bash -c "test -n ${gateway} && /bin/ip route add default via
${gateway}"
ExecStart=-/bin/bash -c "test -f /etc/conf.d/postup@%i.sh&&/bin/bash -c
/etc/conf.d/postup@%i.sh"
ExecStop=/bin/ip addr flush dev %i
ExecStop=/bin/ip link set dev %i down
ExecStop=-/bin/bash -c "test -f /etc/conf.d/postdown@%i.sh&&/bin/bash -c
/etc/conf.d/postdown@%i.sh"
[Install]
WantedBy=multi-user.target
and the /etc/conf.d/network@eno1 is
address=192.168.0.1
netmask=24
broadcast=192.168.0.255
So, before I run this, I don't think the card has any ip address, does
it?
--
Your life is like a penny. You're going to lose it. The question is:
How do
you spend it?
John Covici wb2una
cov...@ccs.covici.com
Re: [gentoo-user] Jobs and load-average
On Thursday, 16 February 2023 12:23:52 GMT Rich Freeman wrote: --->8 Much useful detail. That all makes perfect sense, and is what I'd assumed, but it's good to have it confirmed. > The load average setting is definitely useful and I would definitely > set it, but when the issue is swapping it doesn't go far enough. Make > has no idea how much memory a gcc process will require. Since that is > the resource likely causing problems it is hard to efficiently max out > your cores without actually accounting for memory use. The best I've > been able to do is just set things conservatively so it never gets out > of control, and underutilizes CPU in the process. Often it is only > parts of a build that even have issues - something big like chromium > might have 10,000 tasks that would run fine with -j16 or whatever, but > then there is this one part where the jobs all want a ton of RAM and > you need to run just that one part at a lower setting. I've just looked at 'man make', from which it's clear that -j = --jobs, and that both those and --load-average are passed to /usr/bin/make, presumably untouched unless portage itself has identically named variables. So I wonder how feasible it might be for make to incorporate its own checks to ensure that the load average is not exceeded. I am not a programmer (not for at least 35 years, anyway), so I have to leave any such suggestion to the experts. -- Regards, Peter.
Re: [gentoo-user] Jobs and load-average
On Thu, Feb 16, 2023 at 5:32 AM Andreas Fink wrote: > > On Thu, 16 Feb 2023 09:53:30 + > Peter Humphrey wrote: > > > Yes, I was aware of that, but why didn't --load-average=32 take precedence? > This only means that emerge would not schedule additional package job > (where a package job means something like `emerge gcc`) when load > average > 32, howwever if a job is scheduled it's running, independently > of the current load. > While having it in MAKEOPTS, it would be handled by the make system, > which schedules single build jobs, and would stop scheduling additional > jobs, when the load is too high. > > Extreme case: > emerge chromium firefox qtwebengine > --> your load when you do this is pretty much close to 0, i.e. all 3 > packages are being merged simultaneously and each will be built with > -j16. > I.e. for a long time you will have about 3*16=48 single build jobs > running in parallel, i.e. you should see a load going towards 48, when > you do not have anything in your MAKEOPTS. TL;DR - the load-average option results in underdamping, as a result of the delay with the measurement of load average. Keep in mind that load averages are averages and have a time lag, and compilers that are swapping like crazy can run for a fairly long time. So you will probably have fairly severe oscillation in the load if swapping is happening. If your load is under 32, each of your 16 parallel makes, even if running with the limit in MAKEOPTS, will feel free to launch another 256 jobs, because it will take seconds for the 1 minute load average to creep above 32. At that point you have WAY more than 32 tasks running and if they're swapping then half of the processes on your system are probably going to start blocking. So now make (if configured in MAKEOPTS) will hold off on launching anything, but it could take minutes for those swapping compiler jobs to complete the amount of work that would normally take a few seconds. Then as those processes eventually start terminating (assuming you don't get OOM killing or PANICs) your load will start dropping, until eventually it gets back below 32, at which point all those make processes that are just sitting around will wake up and fire off another 50 gcc instances or whatever they get up to before the brakes come back on. The load average setting is definitely useful and I would definitely set it, but when the issue is swapping it doesn't go far enough. Make has no idea how much memory a gcc process will require. Since that is the resource likely causing problems it is hard to efficiently max out your cores without actually accounting for memory use. The best I've been able to do is just set things conservatively so it never gets out of control, and underutilizes CPU in the process. Often it is only parts of a build that even have issues - something big like chromium might have 10,000 tasks that would run fine with -j16 or whatever, but then there is this one part where the jobs all want a ton of RAM and you need to run just that one part at a lower setting. -- Rich
Re: [gentoo-user] Re: my 5.15.93 kernel keeps rebooting
On Thu, Feb 16, 2023 at 6:50 AM John Covici wrote: > > The sending computer has two nics, eno1 for the internal network and > eno2 is on the internet. So, my netconsole stanza said > netconsole=@192.168.0.1/eno1,@192.168.0.2 Is CONFIG_NETCONSOLE enabled for your kernel? I'm not sure if the kernel will assign the names eno1/2 to interfaces - I think those might be assigned by udev, which probably won't have run before the kernel parses this instruction. You might need to use eth0/1 - and your guess is as good as mine which one corresponds to which. If it isn't one of those it might not hurt to put the target mac address in there just to be safe. I haven't needed that but maybe there are situations where ARP won't work (it would be needed if you are crossing subnets, in which case you'd need the gateway MAC). Keep in mind that this is a low-level function that doesn't use any routing/userspace/etc. It was designed to be robust in the event of a PANIC and to be able to be enabled fairly early during boot, so it can't rely on the sorts of things we just take for granted with networking. > > The box which is at 192.168.0.2 has netcat (windows version) and I > tried the following: > netcat -u -v -l 192.168.0.2 and I also tried 192.168.0.1 > which is the ip address of the linux console which I am trying to > debug. > > I also tried 0.0.0.0 which did not work either, but I think the > windows firewall was blocking, and I did fix that, but did not try the > 0.0.0.0 after that. > So I'm pretty sure that netcat requires listing the destination IP, since it has to open a socket to listen on that IP. You can optionally set a source address/port in which case it will ignore anything else, but by default it will accept packets from any source. I was definitely going to suggest making sure that a windows firewall wasn't blocking the inbound connections. That's fairly default behavior on windows. -- Rich
Re: [gentoo-user] Re: my 5.15.93 kernel keeps rebooting
On Wed, 15 Feb 2023 09:50:27 -0500, Grant Edwards wrote: > > On 2023-02-14, Rich Freeman wrote: > > > Where are you getting this from, the system log/journal? This doesn't > > seem like a clean shutdown, so if it is a kernel PANIC I wouldn't > > expect the most critical info to be in the log (since it will stop > > syncing to protect the filesystem). The details you need probably > > will be displayed on the console briefly. You can also enable a > > network console, which will send the dmesg output continuously over > > UDP to another device. This won't be interrupted by a PANIC unless > > there is some issue with the hardware or networking stack. > > If you've got a serial port[1], you could also set up serial > logging. Though using serial ports have become a bit of a lost art, > the serial console code in the kernel is pretty carefully designed to > be the last man standing when things start to die. It's possible > (though I wouldn't say probable) that a serial console will be able to > show you stuff closer to the event horizon than a network console can. > > Anyway, since still I'm in the serial port business (yes, there are > still plenty of people using serial ports in industrial settings) I > had to mention it... > > [1] For this purpose you want a plain old UART on the motherboard type > seial port. You'd be surprised how many motherboards still have > them. Even though they're never brought out to a DB9 connector on > the back panel, there's often an 8-pin header on the edge of the > board somewhere, so you'd need one of these: > > > https://www.amazon.com/C2G-27550-Adapter-Bracket-Motherboards/dp/B0002J27R8/ > > Still having problems with the netconsole -- I am determined to get this working,so let me explain a bit more. The sending computer has two nics, eno1 for the internal network and eno2 is on the internet. So, my netconsole stanza said netconsole=@192.168.0.1/eno1,@192.168.0.2 The box which is at 192.168.0.2 has netcat (windows version) and I tried the following: netcat -u -v -l 192.168.0.2 and I also tried 192.168.0.1 which is the ip address of the linux console which I am trying to debug. I also tried 0.0.0.0 which did not work either, but I think the windows firewall was blocking, and I did fix that, but did not try the 0.0.0.0 after that. So, what am I doing wrong here? -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici wb2una cov...@ccs.covici.com
Re: [gentoo-user] Jobs and load-average
On Thu, 16 Feb 2023 09:53:30 + Peter Humphrey wrote: > On Wednesday, 15 February 2023 13:18:24 GMT Rich Freeman wrote: > > > First, keep in mind that --jobs=16 + -j16 can result in up to 256 > > (16*16) tasks running at once. Of course, that is worst case and most > > of the time you'll have way less than that. > > Yes, I was aware of that, but why didn't --load-average=32 take precedence? This only means that emerge would not schedule additional package job (where a package job means something like `emerge gcc`) when load average > 32, howwever if a job is scheduled it's running, independently of the current load. While having it in MAKEOPTS, it would be handled by the make system, which schedules single build jobs, and would stop scheduling additional jobs, when the load is too high. Extreme case: emerge chromium firefox qtwebengine --> your load when you do this is pretty much close to 0, i.e. all 3 packages are being merged simultaneously and each will be built with -j16. I.e. for a long time you will have about 3*16=48 single build jobs running in parallel, i.e. you should see a load going towards 48, when you do not have anything in your MAKEOPTS. Cheers Andreas
Re: [gentoo-user] Jobs and load-average
On Wednesday, 15 February 2023 13:18:24 GMT Rich Freeman wrote: > First, keep in mind that --jobs=16 + -j16 can result in up to 256 > (16*16) tasks running at once. Of course, that is worst case and most > of the time you'll have way less than that. Yes, I was aware of that, but why didn't --load-average=32 take precedence? --->8 > Some packages that I build with either a greatly reduced -j setting or > a non-tmpfs build directory are: > sys-cluster/ceph > dev-python/scipy > dev-python/pandas > app-office/calligra > net-libs/nodejs > dev-qt/qtwebengine > dev-qt/qtwebkit > dev-lang/spidermonkey > www-client/chromium > app-office/libreoffice > sys-devel/llvm > dev-lang/rust (I use the rust binary these days as this has gotten > really out of hand) > x11-libs/gtk+ Thanks for the list, Rich. -- Regards, Peter.
