Martijn van Oosterhout napsal(a):
On Tue, Apr 22, 2008 at 10:31:53AM +0200, Zdenek Kotala wrote:
When you are able detect ordering difference you are able also check if it
is important for the test or not without any extra effort. Only what we
need is put some flag to test that order is not important.
Not true. Sorting the file is going jumble all the results together.
Since we perform many tests in one file, you're not going to be able to
seperate them.
Each statement result must be sort separately, otherwise it should hide
problems. For example one statement return A instead of B and second returns B
instead of A. When sort will be used on whole file then it will be reported as a
ordering problem.
Regression test MUST BE bulletproof. If you get a error you must know that
it is really error (in postgresql or regtest) and must be fixed. When you
start to ignore some errors because it can happen sometimes you fall in the
trap soon.
I think people are misunderstanding. You posted a bunch of diffs with
that comment that they *appeared* to only be ordering differences. How
good did you check? If an 8 become a 9 chances are you'd miss it.
Having a second test checking the sorted results would at least
preclude the chance that there really is something wrong.
It was a guide, not a way of getting out of tests.
Have a nice day,
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
