On 4/10/2012 1:49 PM, Garrett Wollman wrote: > <<On Tue, 10 Apr 2012 13:45:30 -0400, Jeffrey Altman > <[email protected]> said: > >> I don't see how requiring that builder owners demonstrate that "make >> check" works on their system prior to adding "make check" to that system >> in the buildbot is contrary to testing the test harnesses. > > You are failing to make the fundamental distinction between testing > that the test harnesses build, which is simple, takes very little > time, and should just be done as a part of the normal build process, > and testing that the tests pass, which is not necessarily simple, > quick, or appropriate to do for every change.
The distinction is irrelevant to my point. Do not add new requirements to the builders if it is not known that the requirements are met at the time they are added. Perhaps you do not understand the impact that such a change has on the review process because you are not actively involved in reviewing patchsets. All of the buildbot builders to which a patchset is assigned must successfully complete the build script without errors in order for the patchset to be marked "verified" by the buildbot coordinator. If the patchset is not marked verified, the gatekeepers (or other reviewers) must go through the buildbot output manually to determine if the failure is of consequence to the patchset being reviewed. This additional workload is a significant burden. If you would be willing to take on this burden, please register with gerrit to receive e-mail notification of all patchset submissions and updates. Then when a buildbot update is posted, check to see whether it verified the patchset or not, and if not, perform a manual review of the failed builder output. Jeffrey Altman
signature.asc
Description: OpenPGP digital signature
