On Mon, 31 Aug 2026, Ludovic Courtès <[email protected]> wrote: > Hello! > > Olivier Dion <[email protected]> skribis: > >> For example, I often find myself modifying the compiler. I pay close >> attention to the size of the emitted code on disk like so: >> >> $ git checkout main >> $ make >> $ find stage2 -iname "*.go" -printf "%f %s\n" > then >> $ git cherry-pick ... >> $ make >> $ find stage2 -iname "*.go" -printf "%f %s\n" > now >> $ sdiff -s then now | awk '{ printf("%s\t%+d\n", $1, $5 - $2) }' | column >> -t >> >> This gives me something like: >> >> ice-9/boot-9.go +272 >> ice-9/i18n.go -72 >> ice-9/local-eval.go +160 >> ... > > Nice trick! > >> There are other kinds of metrics which are interesting to have. For >> example: >> >> $ hyperfine --export-csv load-time.csv "guile -c '(use-modules (rnrs >> base))'" > dev/null >> Benchmark 1: guile -c '(use-modules (rnrs base))' >> Time (mean ± σ): 9.6 ms ± 0.3 ms [User: 5.9 ms, System: 4.3 >> ms] >> Range (min … max): 8.6 ms … 11.5 ms 241 runs >> >> $ cat load-tim.csv >> command,mean,stddev,median,user,system,min,max >> guile -c '(use-modules (rnrs >> base))',0.009666700473662545,0.00034979357393589316,0.009594730700000002,0.005276396707818929,0.004960339588477369,0.0088011947,0.0111349187 >> >> I think it would be nice to track these kind of metrics and expose them >> through a web UI. I know that the Zig language does that [0]. What >> would also be nice is for the maintainers of Guile to be notified about >> potential performance regressions. > > I agree it would be nice to have. > > In terms of existing infrastructure, Cuirass is not customizable right > now: you could not have it draw plots of individual .go file sizes. It > could keep track of package sizes (Hydra, the inspiration for Cuirass, > does this), but that’s probably too coarse-grain for your needs.
I don't think we need Cuirass for this. We only need to poll a git source of Guile for changes. Then we can list all commits sinces last poll and generate metrics for each of them (we could put some heuristics to avoid generating metrics when only documentation changes are made). >From there, we can generate Guix profiles and run some scripts in them to gather the needed metrics. Finally, put all of that in a DB and share it to a web-server. > As for run-time performance, I think it needs a dedicated machine (you > always want to run your benchmarks on the same machine, without anything > else running on it at the same time) Yes and you also need some tweaking of the hardware for reproducibility. For some μ-benchmark of tracers, I go with: - SMT disabled - Frequency boosting disabled - Kernel boot parameters: idle=poll processor.max_cstate=0 It is also important to know the Kernel version and the possible security mitigations that are enabled (e.g. Retpolines), which can impact system calls performance. [...] > I would suggest asking Guix Foundation Is there an official channel for this? > for a dedicated machine to do that. It doesn’t need to be powerful or > anything; it just needs to be stable and kept around long enough (a > few years at least). There are even laptops that were donated to Guix > Foundation that would be good enough for the job. It does not need to be very powerful indeed. The important factors here are the heat and the battery. If the laptop is heating too much, the results will be skewed due to hardware trottling. Same goes if the battery is too low. Also, who is going to maintain that hardware? [...] Thanks, Olivier -- Olivier Dion
