On tor, 2009-08-27 at 09:58 -0400, Robert Haas wrote: > To get positive results in which you can have confidence, you have to > know that the testing which was done actually did a reasonably good > job exercising the code in a way that would have flushed out bugs, had > any been present. That sounds a lot like the definition of a > regression test suite. Of course, we have that already, but it's > nowhere near comprehensive. Maybe we should be looking at an expanded > test suite that runs on a time scale of hours rather than seconds. > Actually, didn't Peter talk about something like this at PGCon?
Let's look at it this way: If I were writing a compiler, then I would have two main test approaches. First, I would have an in-tree test suite that compiles a bunch of example code snippets and checks that the results are reasonable. Call that a regression test. It would be informed by code coverage analysis and previously reported bugs. Second, as part of my release cycle, I would have an agenda to try to compile a large set of real programs against my new compiler version. I would do that during the beta period. You will notice that GCC pretty much operates that way. We have regression tests. They could and should be expanded. That's a developer job, and we can start working on that now. But this discussion was about what to do during beta. And I think during beta you want to test PostgreSQL against a large set of real applications. But we could try to clarify how to actually do that in an organized way. Now, if you want to improve the regression tests, I would suggest going through the commits since 8.4beta and since 8.4.0 final release and ask how these problems could have been prevented or caught earlier. I suppose a test suite for WAL might be part of the answer, but a closer analysis might be insightful. -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers