Hi Adrian, On Fri, Oct 09, 2026 at 05:31:45PM +0300, Adrian Bunk wrote: > Let me try to explain the severity of the problem caused by > guess-concurrency.
guess-concurrency is not breaking anything. It merely collects a pattern that was vendored into several source packages into a binary package for easier reuse. The guesswork (that people seem to be getting wrong) remains with them. > IMHO the most important fix would be to make it clear in the > documentation that guess-concurrency should only be used when > ***really*** necessary. Expanding the documentation likely is a good thing. Unfortunately, we first have to agree what guidance we want to give. What it means to be really necessary is not that obvious. From my pov, it is necessary when the build OOMs the machine. The tricky part is figuring out what the lowest machine specs are that we want to support. Generally though, I agree that using guess-concurrency is more of an exception as most packages should work with the default concurrency. > Sometimes guess-concurrency is used with an insanely high value > (#1149937), people guessing numbers is just harmful. This is sad indeed. > From a decade of experience keeping packages alive on 32-bit I can tell > that it is usually exactly one ld call or the one compilation of the > swig-generated Python bindings for a C++ library that is the memory hog. I know about this behavior and took care of making guess-concurrency somewhat support this common case. If you have a single large (e.g. 3GB) link job and expect your other compiler invocations to not typically exceed 1GB, you could pass --require-mem 3G,1G. > Using guess-concurrency does not make much sense in this typical case. That's your view. My view is that suing guess-concurrency makes more sense than having to manually retry your build several times due to it being OOM killed. > Packages where the maximum memory usage scales with the number of cores > do exist, but they are rare. In my experience it is most packages that scale with the number of cores. I think what you meant to imply here is "scales linearly". I agree that a linear approximation often is a bad match, but an affine approximation is accurate more often. Effectively, guess-concurrency supports piecewise linear approximation - a generalization of affine functions. > On 32-bit architectures, requiring more memory per process than the > address space available for a process is obviously wrong. This is a different problem not addressed by guess-concurrency in any way. > In [1] the removal of guess-concurrency in my 9.6.2+ds-2+test1 package > uses 6 instead of 2 cores on the buildd, resulting in a 2.76x build time > difference. A 2.76x speedup for a 3x increase in the number of cores is > as close to perfect scaling as might happen practice. That was using --require-mem=4G. And then to me there is a difference between it building slowly and it requiring human intervention to build at all. > In practice we have to pick one of the following: > 1. treat build time increases due to guess-concurrency as RC bugs, or > 2. drop riscv64 as release architecture I don't follow how you reach this conclusion. While debhelper implements concurrent building by default, doing so still is not mandatory. For instance, sqlite3 still is annoyingly sequential and I haven't seen a bug about this. Much less an rc one. Nor is sqlite3 being used as a reason to demote riscv64. What you probably see as a problem here is that pessimistic use of guess-concurrency tends to clog the build queue. This has been a problem before when guess-concurrency was open coded into debian/rules in various ways. In my view, the fix is not to stop using it but to pass more sensible memory guesses. > And I mean 2. seriously, [2] increases the build time of a package > currently involved in a transition from 4 to 6 days on the slower > buildds due to only using 2 instead of 4 cores. Even on the new faster > buildds[3] it would likely also push the build time above 2 days. The alternative that you do not highlight here is failing the build after two days to have some developer give it back hoping that it does not fail again. Not only does this waste buildd time, but also developer time. > And riscv64 is just the tip of the iceberg: > We spent a decade getting parallel builds working in nearly all packages, > we don't want to lose a large part of that when people start modifying > packages on a large scale with super-high guessed limits to > guess-concurrency - requiring in the need for more buildds on multiple > architectures. The vast majority of packages should not be using guess-concurrency. Unfortunately, the ones where it does make sense are the ones that typically take several days so any misuse delays builds a lot. Figuring out what limits are sensible is a task that arises independently of whether you implement it using guess-concurrency or not. > And you also want to be conservative regarding how to calculate the > available memory. Not using all cores on a 4-core riscv64 buildd with > 16 GB RAM because your proposed formula would use only 3 cores when > an uploader used 4 GB as guessed value[4] - that's just bad. That's your view. My view is that less builds fail by OOMing the buildd and thus *save* a lot of time. > Can we agree that guess-concurrency increasing build times is a bad > thing that should usually result in an RC bug, or do you insist on > killing riscv64? No, we actually disagree about this. Can we agree that the way guess-concurrency is presently being used in a few packages is less than ideal? Note that the forky/sid manual page is expanded compared to trixie. It now includes a ready example for the linker behavior. I guess that guess that some users read the trixie one. Helmut
