Ævar Arnfjörð Bjarmason <ava...@gmail.com> writes: >> Well, it is one thing to place git-annex under CI to make sure its >> latest and greatest works together well with our latest and greatest >> (and it may be something we want to see happen), but driving its >> tests from our testsuite sounds like a tail wagging the dog, at >> least to me. > > To me this is just a question of: > > * Is it the case that git-annex tests for a lot of edge cases we don't > test for: Yes, probably. As evidenced by them spotting this > regression, and not us.
And I'd encourage them to keep doing so. > * We can (and should) add a test for the specific breakage we caused > in 2.13.0, but that's no replacement for other things annex may be > covering & we may be missing which'll catch future breakages. > > * It's a pretty established practice to test a library (git) along > with its consumers (e.g. annex) before a major release. I am not so sure about the division of labor. What you are advocating would work _ONLY_ if we test with a perfect & bug-free version of the consumers. If they are also a moving target, then I do not think it is worth it. After all, we are *not* in the business of testing these consumers. Unless I misunderstood you and you were saying that we freeze a version, or a set of versions, of customer that is/are known to pass their own tests, and test the combination of that frozen version of the customer with our daily development. If that is the case, then I would agree that we are using their test to test us, not them. But I somehow didn't get that impression, hence my reaction.
