Neat, thanks Sven! Here are the nova stats:
http://paste.openstack.org/show/592551/ --diana On Thu, Dec 15, 2016 at 12:22 PM, Sven Anderson <s...@redhat.com> wrote: > Hi all, > > while I was waiting again for the CI to be fixed and didn't want to > torture it with additional rechecks, I wanted to find out, how much of > our CI infrastructure we waste with rechecks. My assumption was that > every recheck is a waste of resources based on a false negative, because > it renders the previous build useless. So I wrote a small script[1] to > calculate how many rechecks are made on average per built patch-set. It > calculates the number of patch-sets of merged changes that CI was > testing (some patch-sets are not, because they were updated before CI > started testing), the number of rechecks issued on these patch-sets, and > a value "CI-factor", which is the factor by which the rechecks increased > the the CI runs, that is, without rechecks it would be 1, if every > tested patch-set would have exactly one recheck it would be 2. > > The results were not as bad as my feeling, we are below 2 for most of > the projects I tested. :-) But still, on THT for instance we use 71% > more resources because of the false negatives. I made monthly > breakdowns, so you can see a positive trend at least. > > > Here the results: > > Project: tripleo-heat-templates > > month patches rechecks CI-factor > 1 221 102 1.46 > 2 282 300 2.06 > 3 588 567 1.96 > 4 220 253 2.15 > 5 333 242 1.73 > 6 459 325 1.71 > 7 612 390 1.64 > 8 694 442 1.64 > 9 717 440 1.61 > 10 474 316 1.67 > 11 358 189 1.53 > 12 168 80 1.48 > total 5126 3646 1.71 > > Project: tripleo-common > > month patches rechecks CI-factor > 1 73 29 1.4 > 2 59 48 1.81 > 3 92 101 2.1 > 4 17 19 2.12 > 5 47 27 1.57 > 6 83 46 1.55 > 7 66 26 1.39 > 8 209 102 1.49 > 9 261 129 1.49 > 10 110 51 1.46 > 11 121 47 1.39 > 12 40 19 1.48 > total 1178 644 1.55 > > Project: tripleo-puppet-elements > > month patches rechecks CI-factor > 1 24 9 1.38 > 2 9 20 3.22 > 3 7 16 3.29 > 4 9 24 3.67 > 5 14 17 2.21 > 6 17 33 2.94 > 7 12 16 2.33 > 8 15 21 2.4 > 9 10 14 2.4 > 10 12 5 1.42 > 11 34 25 1.74 > 12 10 13 2.3 > total 173 213 2.23 > > Project: puppet-tripleo > > month patches rechecks CI-factor > 1 29 23 1.79 > 2 36 68 2.89 > 3 40 44 2.1 > 4 68 74 2.09 > 5 129 43 1.33 > 6 265 206 1.78 > 7 235 118 1.5 > 8 193 130 1.67 > 9 147 123 1.84 > 10 233 159 1.68 > 11 137 86 1.63 > 12 20 5 1.25 > total 1532 1079 1.7 > > > [1] https://gist.github.com/ansiwen/e139cbf25bc243d30629e0157fc753ff > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev