Hi all, while I was waiting again for the CI to be fixed and didn't want to torture it with additional rechecks, I wanted to find out, how much of our CI infrastructure we waste with rechecks. My assumption was that every recheck is a waste of resources based on a false negative, because it renders the previous build useless. So I wrote a small script[1] to calculate how many rechecks are made on average per built patch-set. It calculates the number of patch-sets of merged changes that CI was testing (some patch-sets are not, because they were updated before CI started testing), the number of rechecks issued on these patch-sets, and a value "CI-factor", which is the factor by which the rechecks increased the the CI runs, that is, without rechecks it would be 1, if every tested patch-set would have exactly one recheck it would be 2.
The results were not as bad as my feeling, we are below 2 for most of the projects I tested. :-) But still, on THT for instance we use 71% more resources because of the false negatives. I made monthly breakdowns, so you can see a positive trend at least. Here the results: Project: tripleo-heat-templates month patches rechecks CI-factor 1 221 102 1.46 2 282 300 2.06 3 588 567 1.96 4 220 253 2.15 5 333 242 1.73 6 459 325 1.71 7 612 390 1.64 8 694 442 1.64 9 717 440 1.61 10 474 316 1.67 11 358 189 1.53 12 168 80 1.48 total 5126 3646 1.71 Project: tripleo-common month patches rechecks CI-factor 1 73 29 1.4 2 59 48 1.81 3 92 101 2.1 4 17 19 2.12 5 47 27 1.57 6 83 46 1.55 7 66 26 1.39 8 209 102 1.49 9 261 129 1.49 10 110 51 1.46 11 121 47 1.39 12 40 19 1.48 total 1178 644 1.55 Project: tripleo-puppet-elements month patches rechecks CI-factor 1 24 9 1.38 2 9 20 3.22 3 7 16 3.29 4 9 24 3.67 5 14 17 2.21 6 17 33 2.94 7 12 16 2.33 8 15 21 2.4 9 10 14 2.4 10 12 5 1.42 11 34 25 1.74 12 10 13 2.3 total 173 213 2.23 Project: puppet-tripleo month patches rechecks CI-factor 1 29 23 1.79 2 36 68 2.89 3 40 44 2.1 4 68 74 2.09 5 129 43 1.33 6 265 206 1.78 7 235 118 1.5 8 193 130 1.67 9 147 123 1.84 10 233 159 1.68 11 137 86 1.63 12 20 5 1.25 total 1532 1079 1.7 [1] https://gist.github.com/ansiwen/e139cbf25bc243d30629e0157fc753ff __________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev