On Mon, Jan 22, 2024 at 12:25:58PM +0000, George Dunlap wrote: > On Mon, Jan 22, 2024 at 12:17 PM Marek Marczykowski-Górecki > <marma...@invisiblethingslab.com> wrote: > > > > On Mon, Jan 22, 2024 at 11:54:14AM +0000, George Dunlap wrote: > > > The other issue I have with this (and essentially where I got stuck > > > developing credit2 in the first place) is testing: how do you ensure > > > that it has the properties that you expect? > > > > Audio is actually quite nice use case at this, since it's quite > > sensitive for scheduling jitter. I think even a simple "PCI passthrough a > > sound card and play/record something" should show results. Especially > > you can measure how hard you can push the system (for example artificial > > load in other domains) until it breaks. > > Are we going have a gitlab runner which says, "Marek sits in front of > his test machine and listens to audio for pops"? :-)
Kinda ;) We have already audio tests in qubes CI. They do more or less the above, but using our audio virtualization. Play something, record in another domain, and compare. Running the very same thing in gitlab-ci may be too complicated (require bringing in some qubes infrastructure to make PV audio work), but maybe similar test can be done based on qemu-emulated audio or other pv audio solution? > > > How do you develop a > > > "regression test" to make sure that server-based workloads don't have > > > issues in this sort of case? > > > > For this I believe there are several benchmarking methods already, > > starting with old trusty "Linux kernel build time". > > First of all, AFAICT "Linux kernel bulid time" is not representative > of almost any actual server workload; and the end-to-end throughput > completely misses what most server workloads will actually care about, > like latency. > > Secondly, what you're testing isn't the performance of a single > workload on an empty system; you're testing how workloads *interact*. > If you want ideal throughput for a single workload on an empty system, > use the null scheduler; more complex schedulers are only necessary > when multiple different workloads interact. I should have clarified I meant `make -jNN`. But still, that's the same workload on multiple vCPUs. > FWIW this was my first stab at trying to be systematic about testing > the scheduler: > > https://github.com/gwd/schedbench > > The rump kernel project has basically died AFAIK, so anyone trying to > resurrect this would probably have to try to rebase that bit of it > against something like XTF or unikernels. > > -George -- Best Regards, Marek Marczykowski-Górecki Invisible Things Lab
signature.asc
Description: PGP signature
