Hello, On Mon 04 Aug 2025 at 05:19pm +01, Ian Jackson wrote:
> Package: dgit-infrastructure > Version: 13.10 > > Currently we have only one worker. We may want to increase this, > even though we have only one vcpu. > > Each worker will do some combination of local "disk" IO, network > activitym and CPU use. Effective use of these resources probably > means greater parallelism than the vcpu count. (I think very little > of our processing code makes effective use of multicore pallelism.) > > My experience is that maximum performance with a mixed workload on a > desktop-sized machine occurs at about 1.5x cpu "overcommitment". > > This analysis depends stronly on the situation on the underlying > physical host, which presuably has more than one pcpu and is > presumably doing other work too. > > Another argument in favour of greater parallelism is that it can > significantly reduce maximum latency especially for small packages > mixed in with a few big ones. > > My guess is that we should: > > - Plan to change tag2upload-oracled to default the number of workers > to `ceil(nproc * 1.5)`, for a minimum of 2. (Not sure if > 1.5 should be configurable.) > > - Consider the RAM and disk space implications and talk to DSA about > that. (We'd almost certainly need more disk space both for the > additional running containers, and for working space for multiple > possibly-large source packages.) > > We can this put off until we start to see nontrivial queueing. Given that we have to ensure there is enough RAM and disk space before doing any increases, I would have thought that leaving the default as 1 and just passing --workers=2 in the service file would be better than calculating it based on nproc. For example, if one day DSA were to increase the cores but not the other resources (perhaps as a side effect of some other maintenance they were doing) the service would then try to start more workers than the host could really handle, and things would go wrong. -- Sean Whitton
signature.asc
Description: PGP signature
