Re: bailing out in tap tests nearly always a bad idea

2022-02-23 Thread Andrew Dunstan
On 2/23/22 11:40, Andres Freund wrote: > Hi, > > On 2022-02-23 08:43:38 -0500, Andrew Dunstan wrote: >> On 2/22/22 15:54, Andres Freund wrote: >>> On 2022-02-22 15:10:30 -0500, Andrew Dunstan wrote: I'll be surprised if we can't come up with something cleaner than that. >>> Suggestions? >>

Re: bailing out in tap tests nearly always a bad idea

2022-02-23 Thread Andres Freund
Hi, On 2022-02-23 08:43:38 -0500, Andrew Dunstan wrote: > > On 2/22/22 15:54, Andres Freund wrote: > > On 2022-02-22 15:10:30 -0500, Andrew Dunstan wrote: > >> I'll be surprised if we can't come up with something cleaner than that. > > Suggestions? > > > If we just have the sig handler actions

Re: bailing out in tap tests nearly always a bad idea

2022-02-23 Thread Andrew Dunstan
On 2/22/22 15:54, Andres Freund wrote: > On 2022-02-22 15:10:30 -0500, Andrew Dunstan wrote: >> I'll be surprised if we can't come up with something cleaner than that. > Suggestions? If we just have the sig handler actions as:     diag("died: $_[0]");     done_testing();    we get:     ok 1

Re: bailing out in tap tests nearly always a bad idea

2022-02-22 Thread Andres Freund
On 2022-02-22 15:10:30 -0500, Andrew Dunstan wrote: > I'll be surprised if we can't come up with something cleaner than that. Suggestions?

Re: bailing out in tap tests nearly always a bad idea

2022-02-22 Thread Andrew Dunstan
On 2/22/22 13:19, Andres Freund wrote: > Hi, > > On 2022-02-22 09:28:37 -0800, Andres Freund wrote: >> On 2022-02-14 09:46:20 -0800, Andres Freund wrote: t/die-immediately.t aieee at t/die-immediately.t line 1. t/die-immediately.t Dubious, test returned 255 (wstat 65280,

Re: bailing out in tap tests nearly always a bad idea

2022-02-22 Thread Andres Freund
Hi, On 2022-02-22 09:28:37 -0800, Andres Freund wrote: > On 2022-02-14 09:46:20 -0800, Andres Freund wrote: > > > t/die-immediately.t aieee at t/die-immediately.t line 1. > > > t/die-immediately.t Dubious, test returned 255 (wstat 65280, 0xff00) > > > No subtests run > > > > Hm. Looks

Re: bailing out in tap tests nearly always a bad idea

2022-02-22 Thread Andres Freund
Hi, On 2022-02-14 09:46:20 -0800, Andres Freund wrote: > > t/die-immediately.t aieee at t/die-immediately.t line 1. > > t/die-immediately.t Dubious, test returned 255 (wstat 65280, 0xff00) > > No subtests run > > Hm. Looks different when a test is including our test helpers. > >

Re: bailing out in tap tests nearly always a bad idea

2022-02-16 Thread Peter Eisentraut
On 14.02.22 00:22, Andres Freund wrote: Does anybody want to defend / explain the use of BAIL_OUT? I think this was my mistake when this was originally added. I agree it should be changed.

Re: bailing out in tap tests nearly always a bad idea

2022-02-15 Thread Andrew Dunstan
On 2/14/22 12:58, Tom Lane wrote: > Andres Freund writes: >> Even just getting rid of the "Tests were run but no plan was declared and >> done_testing() was not seen." noise would be helpful. So I think using a >> fatal >> error routine that forced a failure to be recognized via ok(0, 'fatal

Re: bailing out in tap tests nearly always a bad idea

2022-02-14 Thread Tom Lane
Andres Freund writes: > Even just getting rid of the "Tests were run but no plan was declared and > done_testing() was not seen." noise would be helpful. So I think using a fatal > error routine that forced a failure to be recognized via ok(0, 'fatal error') > and then does done_testing() would

Re: bailing out in tap tests nearly always a bad idea

2022-02-14 Thread Robert Haas
On Sun, Feb 13, 2022 at 6:33 PM Tom Lane wrote: > Huh, doesn't Test::More already provide a sane way to do this? > If not, why isn't die() good enough? (I don't think you can > realistically expect to prohibit die() anywhere in the TAP tests.) +1 for die. There's very little reason to use

Re: bailing out in tap tests nearly always a bad idea

2022-02-14 Thread Andres Freund
Hi, On 2022-02-14 16:08:42 +, Dagfinn Ilmari Mannsåker wrote: > die()-ing is the correct way to abort a single test script. There's really no way to nice way to abort without the "Dubious, test returned 255 (wstat 65280, 0xff00)" stuff? Even just getting rid of the "Tests were run but no

Re: bailing out in tap tests nearly always a bad idea

2022-02-14 Thread Dagfinn Ilmari Mannsåker
Andres Freund writes: > Hi, > > On 2022-02-13 18:32:59 -0500, Tom Lane wrote: >> Andres Freund writes: >> > Best with a >> > central function signalling fatal error, rather than individual uses of die >> > or such. >> >> Huh, doesn't Test::More already provide a sane way to do this? > > I

Re: bailing out in tap tests nearly always a bad idea

2022-02-13 Thread Andres Freund
Hi, On 2022-02-13 18:32:59 -0500, Tom Lane wrote: > Andres Freund writes: > > Best with a > > central function signalling fatal error, rather than individual uses of die > > or such. > > Huh, doesn't Test::More already provide a sane way to do this? I looked, and didn't see anything. But I'm

Re: bailing out in tap tests nearly always a bad idea

2022-02-13 Thread Tom Lane
Andres Freund writes: > Does anybody want to defend / explain the use of BAIL_OUT? If not, I think we > should consider doing a global replace of the use of bailing. +1 > Best with a > central function signalling fatal error, rather than individual uses of die > or such. Huh, doesn't

bailing out in tap tests nearly always a bad idea

2022-02-13 Thread Andres Freund
Hi, A recent cfbot failure reminded me of a topic I wanted to raise: The use of BAIL in tap tests. When a tap test exits with BAIL it doesn't just consider that test failed, but it also stops running further tap tests in the same group: https://metacpan.org/pod/Test::More "Indicates to the