I don't have any data to back this up, but my suspicion is that a large percentage of backouts had try runs, but said try runs didn't run the jobs that failed and caused the backout. Imo, these kinds of backouts are (more) acceptable because it means the developer was trying to avoid doing a full try run, a practice that should be cheaper overall in the long run (and if done properly).
For example, you could either: A) Do a full try run then push, almost guaranteeing you won't be backed out. But then you run every job twice and take longer to complete your bug, a significant cost. B) Do a partial try run, running X% of the jobs yielding a Y% chance of being backed out. There's some sweet spot between no try run, and try: -all which is the most cost effective (both in terms of AWS bill and developer time). That being said, I think this is an issue of tools rather than policy. Things like being smarter about running jobs based on files changed and improving interfaces to selecting jobs on try, should help with backout rates. But the single biggest thing we could do is getting rid of integration branches altogether (and instead get autoland to merge directly from try). In this world, backouts would hardly even exist anymore. I believe we're already headed in this direction, albeit slowly. -Andrew On Fri, Mar 10, 2017 at 8:55 AM, David Burns <dbu...@mozilla.com> wrote: > I went back and did some checks with autoland to servo and the results are > negligible. So from 01 February 2017 to 10 March 2017 (as of sending this > email). I have removed merge commits from the numbers. > > Autoland: > Total Servo Sync Pushes: 152 > Total Pushes: 1823 > Total Backouts: 144 > Percentage of backouts: 7.8990674712 > Percentage of backouts without Servo: 8.61759425494 > > Mozilla-Inbound: > Total Pushes: 1472 > Total Backouts: 166 > Percentage of backouts: 11.277173913 > > > I will look into why, with more pushes, is resulting in fewer backouts. The > thing to note is that autoland, by its nature, does not allow us to fail > forward like inbound without having to get a sheriff to land the code. > > I think, and this is my next area to investigate, is the 1 bug per push > (the autoland model) could be helping with the percentage of backouts being > lower. > > David > > On 7 March 2017 at 21:29, Chris Peterson <cpeter...@mozilla.com> wrote: > > > On 3/7/2017 3:38 AM, Joel Maher wrote: > > > >> One large difference I see between autoland and mozilla-inbound is that > on > >> autoland we have many single commits/push whereas mozilla-inbound it is > >> fewer. I see the Futurama data showing pushes and the sheriff report > >> showing total commits. > >> > > > > autoland also includes servo commits imported from GitHub that won't > break > > Gecko. (They might break the linux64-stylo builds, but they won't be > backed > > out for that.) > > > > _______________________________________________ > > dev-platform mailing list > > dev-platform@lists.mozilla.org > > https://lists.mozilla.org/listinfo/dev-platform > > > _______________________________________________ > dev-platform mailing list > dev-platform@lists.mozilla.org > https://lists.mozilla.org/listinfo/dev-platform > _______________________________________________ dev-platform mailing list dev-platform@lists.mozilla.org https://lists.mozilla.org/listinfo/dev-platform
