> On 27 Sep 2021, at 19:02, Andrew MacLeod <amacl...@redhat.com> wrote: > > On 9/27/21 11:39 AM, Maxim Kuvyrkov via Gcc wrote: >>> On 27 Sep 2021, at 16:52, Aldy Hernandez <al...@redhat.com> wrote: >>> >>> [CCing Jeff and list for broader audience] >>> >>> On 9/27/21 2:53 PM, Maxim Kuvyrkov wrote: >>>> Hi Aldy, >>>> Your patch seems to slow down 471.omnetpp by 8% at -O3. Could you please >>>> take a look if this is something that could be easily fixed? >>> First of all, thanks for chasing this down. It's incredibly useful to have >>> these types of bug reports. >> Thanks, Aldy, this is music to my ears :-). >> >> We have built this automated benchmarking CI that bisects code-speed and >> code-size regressions down to a single commit. It is still >> work-in-progress, and I’m forwarding these reports to patch authors, whose >> patches caused regressions. If GCC community finds these useful, we can >> also setup posting to one of GCC’s mailing lists. > > I second that this sort of thing is incredibly useful. I don't suppose its > easy to do the reverse?... let patch authors know when they've caused a > significant improvement? :-) That would be much less common I suspect, so > perhaps not worth it :-)
We do this occasionally, when identifying a regression in a patch revert commit :-). Seriously, though, it’s an easy enough code-change to the metric, but we are maxing out our benchmarking capacity with current configuration matrix. > > Its certainly very useful when we are making a wholesale change to a pass > which we think is beneficial, but aren't sure. > > And a followup question... Sometimes we have no good way of determining the > widespread run-time effects of a change. You seem to be running SPEC/other > things continuously then? We continuously run SPEC CPU2006 on {arm,aarch64}-{-Os/-O2/-O3}-{no LTO/LTO} matrix for GNU and LLVM toolchains. In the GNU toolchain we track master branches and latest-release branches of Binutils, GCC and Glibc — and detect code-speed and code-size regressions across all toolchain components. > Does it run like once a day/some-time-period, and if you note a regression, > narrow it down? Configurations that track master branches have 3-day intervals. Configurations that track release branches — 6 days. If a regression is detected it is narrowed down to component first — binutils, gcc or glibc — and then the commit range of the component is bisected down to a specific commit. All. Done. Automatically. I will make a presentation on this CI at the next GNU Tools Cauldron. > Regardless, I think it could be very useful to be able to see the results of > anything you do run at whatever frequency it happens. Thanks! -- Maxim Kuvyrkov https://www.linaro.org _______________________________________________ linaro-toolchain mailing list linaro-toolchain@lists.linaro.org https://lists.linaro.org/mailman/listinfo/linaro-toolchain