On Tue, 2015-06-23 at 17:23 +0700, Duy Nguyen wrote: > On Tue, Jun 23, 2015 at 7:50 AM, David Turner <dtur...@twopensource.com> wrote: > > To test this backend's correctness, I hacked test-lib.sh and > > test-lib-functions.sh to run all tests under the refs backend. > > Now we have two. split-index also benefits from running through full > test suite like this. I propose we make "make test" run the test suite > twice. The first run is with default configuration, no split index, no > fancy ref backend. The second run enables split-index and switches to > new backend, running through all test cases. In future we can also > enable packv4 in this second run. There won't be a third run. > > When the second ref backend comes, we can switch between the two > backends using a random number generator where we control both > algorithm and seed, so that when a test fails, the user can give us > their seed and we can re-run with the same configuration.
I'm not in love with this idea, because it makes it hard to do exhaustive testing efficiently. I would rather have make test run through all tests under all combinations -- or at least all relevant tests. We could perhaps mark tests with a list of features that they exercise, so that we don't have to run e.g. t8xxx with alternate refs backends. > Dozens of tests use manual ref/reflog reading/writing, or create submodules > > without passing --refs-backend-type to git init. If those tests are > > changed to use the update-ref machinery or test-refs-be-db (or, in the > > case of packed-refs, corrupt refs, and dumb fetch tests, are skipped), > > the only remaining failing tests are the git-new-workdir tests and the > > gitweb tests. > > I haven't read the series, but I guess you should also add a few tests > to run on the first run, so new code is exercised a bit even if people > skip the second run. I did this already, yes. -- To unsubscribe from this list: send the line "unsubscribe git" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
