Measuring compiler performance
Hi devs! I've recently started working on a few compiler perf related tickets, and wondered how to verify that a change actually has a positive impact on compiler performance. I first looked at the wiki for information on this, but didn't find much: https://gitlab.haskell.org/ghc/ghc/-/wikis/performance/compiler doesn't contain any information on how to measure compiler performance The best tip I found was the recommendation in https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-no-fib to compile nofib/spectral/simple/Main.hs. With -O0 and -O that takes respectively about 1.5s and 5s for me, so the effort is manageable. I've also done full nofib runs, but they take a very long time. A problem in this context is that reliable performance measurements require a quiet machine. Closing my browser, and turning off other programs is – in my perception – rather inconvenient, particularly when I have to do it for a prolonged time. Ideally I wouldn't have to perform these measurements on my local machine at all! Do you usually use a separate machine for this? _Very_ convenient would be some kind of bot whom I could tell e.g. @perf-bot compiler perf …or more concretely @perf-bot compile nofib/spectral/simple/Main.hs …or just @nofib-bot run … or something like that. I've noticed that CI now includes a perf-nofib job. But since it appears to run on a different machine each time, I'm not sure whether it's actually useful for comparing performance. Could it be made more useful by running it consistently on the same dedicated machine? Another question regarding performing compiler perf measurements locally is which build flavour to use: So far I have used the "perf" flavour. A problem here is that a full build seems to take close to an hour. A rebuild with --freeze1 takes ~15 minutes on my machine. Is this the right flavour to use? BTW what's the purpose of the profiled GHC modules built with this flavour which just seem to additionally prolong compile time? I don't see a ghc-prof binary or similar in _build/stage1/bin. Also, what's the status of gipeda? The most recent commit at https://perf.haskell.org/ghc/ is from "about a year ago"? Sorry for this load of questions and complaints! I do believe though that if work on compiler performance was a bit better documented and more convenient, we might see even more progress on that front. :) Cheers, Simon ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Measuring compiler performance
Because of the fragility of time-based performance tests, I often use allocations as a proxy for time. The number of allocations is very stable. On the down side, it is sometimes possible to reduce allocations while increasing time, so one shouldn't try too hard to blindly optimize allocations. Separately, in order to get measurements, I build Cabal with ghc --make. Cabal is convenient because it is sizeable, real-world, updated, and needs no dependencies. Compiling the full library definitely takes more than 5 seconds, though. I agree that there is plenty of room for improvement here! Richard > On Apr 5, 2020, at 12:24 AM, Simon Jakobi via ghc-devs > wrote: > > Hi devs! > > I've recently started working on a few compiler perf related tickets, > and wondered how to verify that a change actually has a positive > impact on compiler performance. > > I first looked at the wiki for information on this, but didn't find much: > > https://gitlab.haskell.org/ghc/ghc/-/wikis/performance/compiler > doesn't contain any information on how to measure compiler performance > > The best tip I found was the recommendation in > https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-no-fib to > compile nofib/spectral/simple/Main.hs. With -O0 and -O that takes > respectively about 1.5s and 5s for me, so the effort is manageable. > I've also done full nofib runs, but they take a very long time. > > A problem in this context is that reliable performance measurements > require a quiet machine. Closing my browser, and turning off other > programs is – in my perception – rather inconvenient, particularly > when I have to do it for a prolonged time. > > Ideally I wouldn't have to perform these measurements on my local > machine at all! Do you usually use a separate machine for this? _Very_ > convenient would be some kind of bot whom I could tell e.g. > > @perf-bot compiler perf > > …or more concretely > > @perf-bot compile nofib/spectral/simple/Main.hs > > …or just > > @nofib-bot run > > … or something like that. > > I've noticed that CI now includes a perf-nofib job. But since it > appears to run on a different machine each time, I'm not sure whether > it's actually useful for comparing performance. Could it be made more > useful by running it consistently on the same dedicated machine? > > Another question regarding performing compiler perf measurements > locally is which build flavour to use: So far I have used the "perf" > flavour. A problem here is that a full build seems to take close to an > hour. A rebuild with --freeze1 takes ~15 minutes on my machine. Is > this the right flavour to use? > > BTW what's the purpose of the profiled GHC modules built with this > flavour which just seem to additionally prolong compile time? I don't > see a ghc-prof binary or similar in _build/stage1/bin. > > Also, what's the status of gipeda? The most recent commit at > https://perf.haskell.org/ghc/ is from "about a year ago"? > > Sorry for this load of questions and complaints! I do believe though > that if work on compiler performance was a bit better documented and > more convenient, we might see even more progress on that front. :) > > Cheers, > Simon > ___ > ghc-devs mailing list > ghc-devs@haskell.org > http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Measuring compiler performance
Hi Simon, things I do to measure performance: * compile nofib/spectral/simple/Main.hs, look at instructions (perf) and allocations/time (+RTS -s) * compile nofib as a whole (Use NoFibRuns=0 to avoid running the benchmarks). Look at compile time/allocations. * compile Cabal the library (cd cabal-head/Cabal && ghc Setup.hs -fforce-recomp). Look at allocations time via +RTS -s or instructions using perf. * compile a particular files triggering the case I want to optimize In general: Adjust depending on flags you want to look at. If you optimize the simplifier -O0 will be useless. If you optimize type-checking -O2 will be pointless. And so on. In general I only compile as linking adds overhead which isn't really part of GHC. Another question regarding performing compiler perf measurements locally is which build flavour to use: So far I have used the "perf" flavour. A problem here is that a full build seems to take close to an hour. A rebuild with --freeze1 takes ~15 minutes on my machine. Is this the right flavour to use? Personally I use the quick flavour, freeze stage 1 and configure hadrian to pass -O to stage2 unless I know the thing I'm working on will benefit significantly from -O2. That is if I optimize an algorithm -O2 won't really make a difference so I use -O. If I optimize a particular hotspot in the implementation of an algorithm by using bangs it's worthwhile to look at -O2 as well. You can also set particular flags for only specific files using OPTIONS_GHC pragmas. This way you can avoid compiling the whole of GHC with -O/-O2. Ideally I wouldn't have to perform these measurements on my local machine at all! Do you usually use a separate machine for this? _Very_ convenient would be some kind of bot whom I could tell e.g. I use another machine. Others only look at metrics which are less affected by system load like allocations. Ideally I wouldn't have to perform these measurements on my local machine at all! Do you usually use a separate machine for this? _Very_ convenient would be some kind of bot whom I could tell e.g. Various people have come up with scripts to automate the measurements on nofib which get's you closer to this. I discussed with ben and others a few times in the past having a wider framework for collecting compiler performance indicators. But it's a lot of work to get right and once the immediate need is gone those ideas usually get shelved again. BTW what's the purpose of the profiled GHC modules built with this flavour which just seem to additionally prolong compile time? I don't see a ghc-prof binary or similar in _build/stage1/bin. As far as I know if you compile (non-ghc) code using -prof then you will need the ghc library available in the prof way. But it would be good to have the option to disable this. Also, what's the status of gipeda? The most recent commit at https://perf.haskell.org/ghc/ is from "about a year ago"? I think the author stopped maintaining it after he switched jobs. So it's currently not useful for investigating performance. But I'm sure he wouldn't object if anyone were to pick it up. Cheers Andreas ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Measuring compiler performance
Hi, Am Montag, den 06.04.2020, 11:03 +0200 schrieb Andreas Klebinger: > > Also, what's the status of gipeda? The most recent commit at > > https://perf.haskell.org/ghc/ is from "about a year ago"? > > > I think the author stopped maintaining it after he switched jobs. So > it's currently not useful > for investigating performance. I think a few things happened. For a while, the dedicated builder (which was a machine on Richard’s university) was down. Then there was a phase where data was collected and the page was generated, but somehow the upload to the web hosting stopped working. Ben started to work on running these things “more properly” as part of Circle CI (but that never finished). And then there is the ways of recording performance data via git notes, which maybe was meant to be fed into gipeda somehow? Somewhere in this confusion (and the fact that there wasn’t much complaining about the lack of this service) it fell off the table. > But I'm sure he wouldn't object if anyone were to pick it up. Definitey not! It seems that “the other” solutions didn’t materialize as quickly as we hoped for, so reactivating perf.haskell.org might be useful. Anybody willing to drive this? Happy to advise! Cheers, Joachim -- Joachim Breitner m...@joachim-breitner.de http://www.joachim-breitner.de/ ___ ghc-devs mailing list ghc-devs@haskell.org http://mail.haskell.org/cgi-bin/mailman/listinfo/ghc-devs
Re: Measuring compiler performance
Simon Jakobi via ghc-devs writes: > Hi devs! > Hi Simon! > I've recently started working on a few compiler perf related tickets, > and wondered how to verify that a change actually has a positive > impact on compiler performance. > > I first looked at the wiki for information on this, but didn't find much: > > https://gitlab.haskell.org/ghc/ghc/-/wikis/performance/compiler > doesn't contain any information on how to measure compiler performance > > The best tip I found was the recommendation in > https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-no-fib to > compile nofib/spectral/simple/Main.hs. With -O0 and -O that takes > respectively about 1.5s and 5s for me, so the effort is manageable. > I've also done full nofib runs, but they take a very long time. > As Joachim points out, progress on this proceeds in fits and starts. I have a number of pointers to share: * When performing compiler measurements myself I generally use a combination of the following: * nofib * the GHC performance testsuite * what I call the "Cabal test"; namely: $ _build/stage1/bin/ghc -O -ilibraries/Cabal/Cabal \ libraries/Cabal/Cabal/Setup.hs +RTS -s * My WIP nofib branch [1] makes nofib much faster and easier to work with and adds the ability to measure perf counters, in addition to the usual RTS and cachegrind statistics. * My nofib branch produces output in a uniform, easy to consume format and provides a tool for comparing sets of measurements in this format. * My ghc_perf tool [2] is very useful for extracting runtime and perf statistics from Haskell program runs; furthermore, it produces output in the same format as expected by the aforementioned nofib-compare utility. * I have a utility [3] which I use to reproducibly build a set of branches, run the testsuite, nofib, and the Cabal test on each of them. Admittedly it could use a bit of cleanup but it does its job reasonably well, making performance measurement a "set it and forget it" sort of task. * We collect and record a complete set of testsuite statistics (saved to git notes 43]); however, we currently do not import these into gipeda. * We don't currently have a box which can measure reliable timings (since our builders are nearly all virtualised cloud instances). I'm going to need to do some shuffling to change this. * One potentially useful source of performance information (which sadly we currently do not exploit) is the -ddump-timing output produced during head.hackage runs. [1] https://gitlab.haskell.org/ghc/nofib/merge_requests/24 [2] https://gitlab.haskell.org/bgamari/ghc-utils/blob/master/ghc_perf.py [3] https://gitlab.haskell.org/bgamari/ghc-utils/-/tree/master/build-all [4] https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-tests/performance-tests > A problem in this context is that reliable performance measurements > require a quiet machine. Closing my browser, and turning off other > programs is – in my perception – rather inconvenient, particularly > when I have to do it for a prolonged time. > > Ideally I wouldn't have to perform these measurements on my local > machine at all! Do you usually use a separate machine for this? _Very_ > convenient would be some kind of bot whom I could tell e.g. > Indeed it is inconvenient. I am in the lucky situation that I have another machine locally that can be made reasonably quiet without interfering with my worflow. However, in general > @perf-bot compiler perf > > …or more concretely > > @perf-bot compile nofib/spectral/simple/Main.hs > > …or just > > @nofib-bot run > > … or something like that. > > I've noticed that CI now includes a perf-nofib job. But since it > appears to run on a different machine each time, I'm not sure whether > it's actually useful for comparing performance. Could it be made more > useful by running it consistently on the same dedicated machine? > Indeed, we currently don't have a dedicated machine for timings. However, allocations and executable sizes are still useful. Nevertheless, as noted above I think that we should make more of an effort to measure time. I need to do some shuffling of our runners so we have a quiet bare-metal which can be dedicated to performance measurement. I'll try to get to this in the next day or so. > Another question regarding performing compiler perf measurements > locally is which build flavour to use: So far I have used the "perf" > flavour. A problem here is that a full build seems to take close to an > hour. A rebuild with --freeze1 takes ~15 minutes on my machine. Is > this the right flavour to use? > I think perf is the best option for performance measurement (afterall, we want to know what users would see). However, it is indeed a bit painful. > BTW what's the purpose of the profiled GHC modules built with this > flavour which just seem to additionally prolong compile time? I don't > see a ghc-prof binary or similar in _build/sta
Re: Measuring compiler performance
Many thanks, Richard, Andreas, Joachim, and Ben, for your responses! I have a few things to try now. :) > * what I call the "Cabal test"; namely: > > $ _build/stage1/bin/ghc -O -ilibraries/Cabal/Cabal \ > libraries/Cabal/Cabal/Setup.hs +RTS -s Thanks for spelling it out like that, Ben! I'm slightly embarrassed to say that I hadn't been aware that I could use GHC directly in this way to build a package! Andreas, you wrote: > In general I only compile as linking adds overhead which isn't really part of > GHC. How do I tell GHC to build e.g. nofib/spectral/simple/Main.hs or Cabal without linking? I'll eventually try to distill a wiki page from all this! Cheers, Simon > > * My WIP nofib branch [1] makes nofib much faster and easier to work >with and adds the ability to measure perf counters, in addition to >the usual RTS and cachegrind statistics. > > * My nofib branch produces output in a uniform, easy to consume format >and provides a tool for comparing sets of measurements in this format. > > * My ghc_perf tool [2] is very useful for extracting runtime and perf >statistics from Haskell program runs; furthermore, it produces output >in the same format as expected by the aforementioned nofib-compare >utility. > > * I have a utility [3] which I use to reproducibly build a set of >branches, run the testsuite, nofib, and the Cabal test on each of >them. Admittedly it could use a bit of cleanup but it does its job >reasonably well, making performance measurement a "set it and forget >it" sort of task. > > * We collect and record a complete set of testsuite statistics (saved >to git notes 43]); however, we currently do not import these into >gipeda. > > * We don't currently have a box which can measure reliable timings >(since our builders are nearly all virtualised cloud instances). I'm >going to need to do some shuffling to change this. > > * One potentially useful source of performance information (which sadly >we currently do not exploit) is the -ddump-timing output produced >during head.hackage runs. > > [1] https://gitlab.haskell.org/ghc/nofib/merge_requests/24 > [2] https://gitlab.haskell.org/bgamari/ghc-utils/blob/master/ghc_perf.py > [3] https://gitlab.haskell.org/bgamari/ghc-utils/-/tree/master/build-all > [4] > https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-tests/performance-tests > > > > A problem in this context is that reliable performance measurements > > require a quiet machine. Closing my browser, and turning off other > > programs is – in my perception – rather inconvenient, particularly > > when I have to do it for a prolonged time. > > > > Ideally I wouldn't have to perform these measurements on my local > > machine at all! Do you usually use a separate machine for this? _Very_ > > convenient would be some kind of bot whom I could tell e.g. > > > > Indeed it is inconvenient. I am in the lucky situation that I have > another machine locally that can be made reasonably quiet without > interfering with my worflow. However, in general > > > @perf-bot compiler perf > > > > …or more concretely > > > > @perf-bot compile nofib/spectral/simple/Main.hs > > > > …or just > > > > @nofib-bot run > > > > … or something like that. > > > > I've noticed that CI now includes a perf-nofib job. But since it > > appears to run on a different machine each time, I'm not sure whether > > it's actually useful for comparing performance. Could it be made more > > useful by running it consistently on the same dedicated machine? > > > Indeed, we currently don't have a dedicated machine for timings. > However, allocations and executable sizes are still useful. > > Nevertheless, as noted above I think that we should make more of an > effort to measure time. I need to do some shuffling of our runners so we > have a quiet bare-metal which can be dedicated to performance > measurement. I'll try to get to this in the next day or so. > > > Another question regarding performing compiler perf measurements > > locally is which build flavour to use: So far I have used the "perf" > > flavour. A problem here is that a full build seems to take close to an > > hour. A rebuild with --freeze1 takes ~15 minutes on my machine. Is > > this the right flavour to use? > > > I think perf is the best option for performance measurement (afterall, > we want to know what users would see). However, it is indeed a bit > painful. > > > BTW what's the purpose of the profiled GHC modules built with this > > flavour which just seem to additionally prolong compile time? I don't > > see a ghc-prof binary or similar in _build/stage1/bin. > > > Indeed; there is little sense in building profiled modules just for > performance measurement. However, I don't believe we currently have a > build flavour which provides comparable optimisation but without the > profiled way. Perhaps we should add one. > > > Also, what's the status of gipeda? The most
Re: Measuring compiler performance
Simon, I assume the `-no-link` flag does this. ``` > ghc --show-options | grep link -copy-libs-when-linking -no-link -no-auto-link-packages --print-c-compiler-link-flags ``` Cheers, Matt On Wed, Apr 8, 2020 at 4:15 PM Simon Jakobi via ghc-devs wrote: > > Many thanks, Richard, Andreas, Joachim, and Ben, for your responses! I > have a few things to try now. :) > > > * what I call the "Cabal test"; namely: > > > > $ _build/stage1/bin/ghc -O -ilibraries/Cabal/Cabal \ > > libraries/Cabal/Cabal/Setup.hs +RTS -s > > Thanks for spelling it out like that, Ben! I'm slightly embarrassed to > say that I hadn't been aware that I could use GHC directly in this way > to build a package! > > Andreas, you wrote: > > > In general I only compile as linking adds overhead which isn't really part > > of GHC. > > How do I tell GHC to build e.g. nofib/spectral/simple/Main.hs or Cabal > without linking? > > I'll eventually try to distill a wiki page from all this! > > Cheers, > Simon > > > > > > > > > * My WIP nofib branch [1] makes nofib much faster and easier to work > >with and adds the ability to measure perf counters, in addition to > >the usual RTS and cachegrind statistics. > > > > * My nofib branch produces output in a uniform, easy to consume format > >and provides a tool for comparing sets of measurements in this format. > > > > * My ghc_perf tool [2] is very useful for extracting runtime and perf > >statistics from Haskell program runs; furthermore, it produces output > >in the same format as expected by the aforementioned nofib-compare > >utility. > > > > * I have a utility [3] which I use to reproducibly build a set of > >branches, run the testsuite, nofib, and the Cabal test on each of > >them. Admittedly it could use a bit of cleanup but it does its job > >reasonably well, making performance measurement a "set it and forget > >it" sort of task. > > > > * We collect and record a complete set of testsuite statistics (saved > >to git notes 43]); however, we currently do not import these into > >gipeda. > > > > * We don't currently have a box which can measure reliable timings > >(since our builders are nearly all virtualised cloud instances). I'm > >going to need to do some shuffling to change this. > > > > * One potentially useful source of performance information (which sadly > >we currently do not exploit) is the -ddump-timing output produced > >during head.hackage runs. > > > > [1] https://gitlab.haskell.org/ghc/nofib/merge_requests/24 > > [2] https://gitlab.haskell.org/bgamari/ghc-utils/blob/master/ghc_perf.py > > [3] https://gitlab.haskell.org/bgamari/ghc-utils/-/tree/master/build-all > > [4] > > https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-tests/performance-tests > > > > > > > A problem in this context is that reliable performance measurements > > > require a quiet machine. Closing my browser, and turning off other > > > programs is – in my perception – rather inconvenient, particularly > > > when I have to do it for a prolonged time. > > > > > > Ideally I wouldn't have to perform these measurements on my local > > > machine at all! Do you usually use a separate machine for this? _Very_ > > > convenient would be some kind of bot whom I could tell e.g. > > > > > > > Indeed it is inconvenient. I am in the lucky situation that I have > > another machine locally that can be made reasonably quiet without > > interfering with my worflow. However, in general > > > > > @perf-bot compiler perf > > > > > > …or more concretely > > > > > > @perf-bot compile nofib/spectral/simple/Main.hs > > > > > > …or just > > > > > > @nofib-bot run > > > > > > … or something like that. > > > > > > I've noticed that CI now includes a perf-nofib job. But since it > > > appears to run on a different machine each time, I'm not sure whether > > > it's actually useful for comparing performance. Could it be made more > > > useful by running it consistently on the same dedicated machine? > > > > > Indeed, we currently don't have a dedicated machine for timings. > > However, allocations and executable sizes are still useful. > > > > Nevertheless, as noted above I think that we should make more of an > > effort to measure time. I need to do some shuffling of our runners so we > > have a quiet bare-metal which can be dedicated to performance > > measurement. I'll try to get to this in the next day or so. > > > > > Another question regarding performing compiler perf measurements > > > locally is which build flavour to use: So far I have used the "perf" > > > flavour. A problem here is that a full build seems to take close to an > > > hour. A rebuild with --freeze1 takes ~15 minutes on my machine. Is > > > this the right flavour to use? > > > > > I think perf is the best option for performance measurement (afterall, > > we want to know what users would see). However, it is indeed a bit > > painful. > > > > > BTW what's the purpose of the
Re: Measuring compiler performance
Thanks, Matt! That works! Am Mi., 8. Apr. 2020 um 17:21 Uhr schrieb Matthew Pickering : > > Simon, I assume the `-no-link` flag does this. > > ``` > > ghc --show-options | grep link > -copy-libs-when-linking > -no-link > -no-auto-link-packages > --print-c-compiler-link-flags > ``` > > Cheers, > > Matt > > On Wed, Apr 8, 2020 at 4:15 PM Simon Jakobi via ghc-devs > wrote: > > > > Many thanks, Richard, Andreas, Joachim, and Ben, for your responses! I > > have a few things to try now. :) > > > > > * what I call the "Cabal test"; namely: > > > > > > $ _build/stage1/bin/ghc -O -ilibraries/Cabal/Cabal \ > > > libraries/Cabal/Cabal/Setup.hs +RTS -s > > > > Thanks for spelling it out like that, Ben! I'm slightly embarrassed to > > say that I hadn't been aware that I could use GHC directly in this way > > to build a package! > > > > Andreas, you wrote: > > > > > In general I only compile as linking adds overhead which isn't really > > > part of GHC. > > > > How do I tell GHC to build e.g. nofib/spectral/simple/Main.hs or Cabal > > without linking? > > > > I'll eventually try to distill a wiki page from all this! > > > > Cheers, > > Simon > > > > > > > > > > > > > > > > * My WIP nofib branch [1] makes nofib much faster and easier to work > > >with and adds the ability to measure perf counters, in addition to > > >the usual RTS and cachegrind statistics. > > > > > > * My nofib branch produces output in a uniform, easy to consume format > > >and provides a tool for comparing sets of measurements in this format. > > > > > > * My ghc_perf tool [2] is very useful for extracting runtime and perf > > >statistics from Haskell program runs; furthermore, it produces output > > >in the same format as expected by the aforementioned nofib-compare > > >utility. > > > > > > * I have a utility [3] which I use to reproducibly build a set of > > >branches, run the testsuite, nofib, and the Cabal test on each of > > >them. Admittedly it could use a bit of cleanup but it does its job > > >reasonably well, making performance measurement a "set it and forget > > >it" sort of task. > > > > > > * We collect and record a complete set of testsuite statistics (saved > > >to git notes 43]); however, we currently do not import these into > > >gipeda. > > > > > > * We don't currently have a box which can measure reliable timings > > >(since our builders are nearly all virtualised cloud instances). I'm > > >going to need to do some shuffling to change this. > > > > > > * One potentially useful source of performance information (which sadly > > >we currently do not exploit) is the -ddump-timing output produced > > >during head.hackage runs. > > > > > > [1] https://gitlab.haskell.org/ghc/nofib/merge_requests/24 > > > [2] https://gitlab.haskell.org/bgamari/ghc-utils/blob/master/ghc_perf.py > > > [3] https://gitlab.haskell.org/bgamari/ghc-utils/-/tree/master/build-all > > > [4] > > > https://gitlab.haskell.org/ghc/ghc/-/wikis/building/running-tests/performance-tests > > > > > > > > > > A problem in this context is that reliable performance measurements > > > > require a quiet machine. Closing my browser, and turning off other > > > > programs is – in my perception – rather inconvenient, particularly > > > > when I have to do it for a prolonged time. > > > > > > > > Ideally I wouldn't have to perform these measurements on my local > > > > machine at all! Do you usually use a separate machine for this? _Very_ > > > > convenient would be some kind of bot whom I could tell e.g. > > > > > > > > > > Indeed it is inconvenient. I am in the lucky situation that I have > > > another machine locally that can be made reasonably quiet without > > > interfering with my worflow. However, in general > > > > > > > @perf-bot compiler perf > > > > > > > > …or more concretely > > > > > > > > @perf-bot compile nofib/spectral/simple/Main.hs > > > > > > > > …or just > > > > > > > > @nofib-bot run > > > > > > > > … or something like that. > > > > > > > > I've noticed that CI now includes a perf-nofib job. But since it > > > > appears to run on a different machine each time, I'm not sure whether > > > > it's actually useful for comparing performance. Could it be made more > > > > useful by running it consistently on the same dedicated machine? > > > > > > > Indeed, we currently don't have a dedicated machine for timings. > > > However, allocations and executable sizes are still useful. > > > > > > Nevertheless, as noted above I think that we should make more of an > > > effort to measure time. I need to do some shuffling of our runners so we > > > have a quiet bare-metal which can be dedicated to performance > > > measurement. I'll try to get to this in the next day or so. > > > > > > > Another question regarding performing compiler perf measurements > > > > locally is which build flavour to use: So far I have used the "perf" > > > > flavour. A problem here is that a full build se
