On Thu, Jun 14, 2018 at 11:27:03AM -0700, Junio C Hamano wrote: > Jeff King <p...@peff.net> writes: > > >> Another alternative is to simply accept that these tests are racy, and > >> that the resulting test failures are rare enough that it isn't worth > >> the complexity of the workaround, but adding a comment to the affected > >> tests warning about the raciness is sufficient. (But I wrote this > >> when I first saw and tracked down this failure; since then I observed > >> it four more times... :) > > > > It's definitely bugged me. I'd be happy to see some solution. I've been > > close to suggesting that reading apache logs is simply not robust, and > > we should focus our tests on the git-visible state changes (e.g., seeing > > successful requests, updated refs, etc). > > Hmph, that certainly is "checking only the things that matter", > which is desirable.
The existing tests that look at the apache logs are very white-box, and come from Shawn's original smart-http series. I suspect it was as much about sanity-checking the implementation then as it was about protecting against regressions. So it's possible that these tests have simply outlived their usefulness. OTOH, they might catch non-functional problems, like if we started sending redundant requests. But then, they are not very comprehensive, as they only cover a few basic situations (so for example, for a while we were sending extra auth-related requests, but I don't think these tests detected that). -Peff
