Re: TAP output format in pg_regress

2023-03-31 Thread Daniel Gustafsson
> On 29 Mar 2023, at 22:08, Daniel Gustafsson wrote: > >> On 29 Mar 2023, at 21:14, Daniel Gustafsson wrote: > >> Ugh, I clearly should've stayed on the couch yesterday. > > Maybe today as well, just as I had sent this I realized there is mention of > the > output format in the docs that I ha

Re: TAP output format in pg_regress

2023-03-29 Thread Daniel Gustafsson
> On 29 Mar 2023, at 21:14, Daniel Gustafsson wrote: > Ugh, I clearly should've stayed on the couch yesterday. Maybe today as well, just as I had sent this I realized there is mention of the output format in the docs that I had missed. The attached changes that as well to match the new format.

Re: TAP output format in pg_regress

2023-03-29 Thread Daniel Gustafsson
> On 29 Mar 2023, at 09:08, Peter Eisentraut > wrote: > > On 28.03.23 15:56, Daniel Gustafsson wrote: >>> On 28 Mar 2023, at 15:26, Daniel Gustafsson wrote: >>> I think the attached is a good candidate for going in, but I would like to >>> see it >>> for another spin in the CF bot first. >> An

Re: TAP output format in pg_regress

2023-03-29 Thread Peter Eisentraut
On 28.03.23 15:56, Daniel Gustafsson wrote: On 28 Mar 2023, at 15:26, Daniel Gustafsson wrote: I think the attached is a good candidate for going in, but I would like to see it for another spin in the CF bot first. Another candidate due to a thinko which raised a compiler warning. This i

Re: TAP output format in pg_regress

2023-03-28 Thread Daniel Gustafsson
> On 28 Mar 2023, at 15:26, Daniel Gustafsson wrote: > I think the attached is a good candidate for going in, but I would like to > see it > for another spin in the CF bot first. Another candidate due to a thinko which raised a compiler warning. -- Daniel Gustafsson v19-0001-pg_regress-Emit

Re: TAP output format in pg_regress

2023-03-28 Thread Daniel Gustafsson
> On 15 Mar 2023, at 11:36, Peter Eisentraut > wrote: > > On 24.02.23 10:49, Daniel Gustafsson wrote: >> Another rebase on top of 337903a16f. Unless there are conflicting reviews, I >> consider this patch to be done and ready for going in during the next CF. > > I think this is just about as g

Re: TAP output format in pg_regress

2023-03-15 Thread Peter Eisentraut
On 24.02.23 10:49, Daniel Gustafsson wrote: Another rebase on top of 337903a16f. Unless there are conflicting reviews, I consider this patch to be done and ready for going in during the next CF. I think this is just about as good as it's going to get, so I think we can consider committing thi

Re: TAP output format in pg_regress

2023-02-24 Thread Daniel Gustafsson
Another rebase on top of 337903a16f. Unless there are conflicting reviews, I consider this patch to be done and ready for going in during the next CF. -- Daniel Gustafsson v17-0001-Emit-TAP-compliant-output-from-pg_regress.patch Description: Binary data

Re: TAP output format in pg_regress

2023-01-23 Thread Daniel Gustafsson
> On 23 Jan 2023, at 12:42, Daniel Gustafsson wrote: > >> On 19 Jan 2023, at 12:14, vignesh C wrote: > >> The patch does not apply on top of HEAD as in [1], please post a rebased >> patch: > > The attached v16 is a rebase on top of current master which resolves the > conflict which came fro

Re: TAP output format in pg_regress

2023-01-23 Thread Daniel Gustafsson
> On 19 Jan 2023, at 12:14, vignesh C wrote: > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: The attached v16 is a rebase on top of current master which resolves the conflict which came from the recent commit removing the "ignore" functionality. It also fixes

Re: TAP output format in pg_regress

2023-01-19 Thread Daniel Gustafsson
> On 19 Jan 2023, at 12:14, vignesh C wrote: > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: Sorry for the silence, and thanks for your previous rebase, $life has kept me too busy lately. I'll post a rebased version shortly. -- Daniel Gustafsson

Re: TAP output format in pg_regress

2023-01-19 Thread vignesh C
On Fri, 6 Jan 2023 at 11:20, vignesh C wrote: > > On Tue, 3 Jan 2023 at 16:01, vignesh C wrote: > > > > On Tue, 29 Nov 2022 at 00:57, Daniel Gustafsson wrote: > > > > > > > On 28 Nov 2022, at 20:02, Nikolay Shaplov wrote: > > > > > > > From my reviewer's point of view patch is ready for commit.

Re: TAP output format in pg_regress

2023-01-05 Thread vignesh C
On Tue, 3 Jan 2023 at 16:01, vignesh C wrote: > > On Tue, 29 Nov 2022 at 00:57, Daniel Gustafsson wrote: > > > > > On 28 Nov 2022, at 20:02, Nikolay Shaplov wrote: > > > > > From my reviewer's point of view patch is ready for commit. > > > > > > Thank you for your patience :-) > > > > Thanks for

Re: TAP output format in pg_regress

2023-01-03 Thread vignesh C
On Tue, 29 Nov 2022 at 00:57, Daniel Gustafsson wrote: > > > On 28 Nov 2022, at 20:02, Nikolay Shaplov wrote: > > > From my reviewer's point of view patch is ready for commit. > > > > Thank you for your patience :-) > > Thanks for review. > > The attached tweaks a few comments and attempts to add

Re: TAP output format in pg_regress

2022-11-28 Thread Daniel Gustafsson
> On 28 Nov 2022, at 20:02, Nikolay Shaplov wrote: > From my reviewer's point of view patch is ready for commit. > > Thank you for your patience :-) Thanks for review. The attached tweaks a few comments and attempts to address the compiler warning error in the CFBot CI. Not sure I entirely ag

Re: TAP output format in pg_regress

2022-11-28 Thread Nikolay Shaplov
В письме от понедельник, 28 ноября 2022 г. 21:28:48 MSK пользователь Andres Freund написал: > On 2022-11-28 14:13:16 +0100, Daniel Gustafsson wrote: > > > On 27 Nov 2022, at 11:22, Nikolay Shaplov wrote: > > > В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel > > > Gustafsso

Re: TAP output format in pg_regress

2022-11-28 Thread Andres Freund
On 2022-11-28 14:13:16 +0100, Daniel Gustafsson wrote: > > On 27 Nov 2022, at 11:22, Nikolay Shaplov wrote: > > В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel > > Gustafsson написал: > > > I wold suggest to use word immediate instead of noatexit. This will do the > > code

Re: TAP output format in pg_regress

2022-11-28 Thread Daniel Gustafsson
> On 27 Nov 2022, at 11:22, Nikolay Shaplov wrote: > В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel > Gustafsson написал: > I wold suggest to use word immediate instead of noatexit. This will do the > code much more sensible for me. I think noatexit is clearer since t

Re: TAP output format in pg_regress

2022-11-27 Thread Nikolay Shaplov
В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel Gustafsson написал: > > + #define bail_noatexit(...) bail_out(true, __VA_ARGS__) > > > > BTW what does "noat" stands for? I thought it is typo too :-) and > > originally meant to be "not". > > Calling _exit() will cause exit

Re: TAP output format in pg_regress

2022-11-26 Thread Daniel Gustafsson
> On 26 Nov 2022, at 20:37, Nikolay Shaplov wrote: > В письме от пятница, 25 ноября 2022 г. 00:20:01 MSK пользователь Daniel > Gustafsson написал: > "Thius" seems to be a typo :-) Correct, fixed in the attached. > + #define bail_noatexit(...) bail_out(true, __VA_ARGS__) > > BTW what does "noa

Re: TAP output format in pg_regress

2022-11-26 Thread Nikolay Shaplov
В письме от пятница, 25 ноября 2022 г. 00:20:01 MSK пользователь Daniel Gustafsson написал: + /* + * The width of the testname field when printing to ensure vertical alignment + * of test runtimes. Thius number i

Re: TAP output format in pg_regress

2022-11-24 Thread Daniel Gustafsson
> On 24 Nov 2022, at 20:32, Andres Freund wrote: > > On November 24, 2022 11:07:43 AM PST, Daniel Gustafsson > wrote: >>> On 24 Nov 2022, at 18:07, Nikolay Shaplov wrote: >> One option could be to redefine bail() to take the exit function as a >> parameter >> and have the caller pass the pref

Re: TAP output format in pg_regress

2022-11-24 Thread Andres Freund
On November 24, 2022 11:07:43 AM PST, Daniel Gustafsson wrote: >> On 24 Nov 2022, at 18:07, Nikolay Shaplov wrote: >One option could be to redefine bail() to take the exit function as a parameter >and have the caller pass the preferred exit handler. > >-bail_out(bool non_rec, const char *fmt,.

Re: TAP output format in pg_regress

2022-11-24 Thread Daniel Gustafsson
> On 24 Nov 2022, at 18:07, Nikolay Shaplov wrote: > > You guys are really fast... I only think about problem, it is already > mentioned here... Most issues I've noticed are already fixed before I was > able > to say something. Thanks for looking and reviewing! > /*

Re: TAP output format in pg_regress

2022-11-24 Thread Nikolay Shaplov
You guys are really fast... I only think about problem, it is already mentioned here... Most issues I've noticed are already fixed before I was able to say something. Nevertheless... /* * Bailing out is for unrec

Re: TAP output format in pg_regress

2022-11-24 Thread Daniel Gustafsson
> On 23 Nov 2022, at 00:56, Andres Freund wrote: > On 2022-11-22 23:17:44 +0100, Daniel Gustafsson wrote: >> The attached v10 attempts to address the points raised above. Notes and >> diagnostics are printed to stdout/stderr respectively and the TAP emitter is >> changed to move more of the synta

Re: TAP output format in pg_regress

2022-11-22 Thread Andres Freund
Hi, On 2022-11-22 23:17:44 +0100, Daniel Gustafsson wrote: > The attached v10 attempts to address the points raised above. Notes and > diagnostics are printed to stdout/stderr respectively and the TAP emitter is > changed to move more of the syntax into it making it less painful on the > transla

Re: TAP output format in pg_regress

2022-11-22 Thread Daniel Gustafsson
> On 21 Nov 2022, at 14:42, Dagfinn Ilmari Mannsåker wrote: > > Andres Freund writes: > >> But either way, it seems nicer to output the # inside a helper function? > > Note that the helper function should inject '# ' at the start of every > line in the message, not just the first line. It migh

Re: TAP output format in pg_regress

2022-11-21 Thread Dagfinn Ilmari Mannsåker
Andres Freund writes: > But either way, it seems nicer to output the # inside a helper function? Note that the helper function should inject '# ' at the start of every line in the message, not just the first line. It might also be worth having two separate functions: one that prints to stdout,

Re: TAP output format in pg_regress

2022-11-17 Thread Andres Freund
Hi, On 2022-11-17 11:36:13 +0100, Daniel Gustafsson wrote: > > On 10 Nov 2022, at 11:44, Nikolay Shaplov wrote: > > Did not found quick way to include prove TAP harness right into Makefile, > > so I > > check dumped output, but it is not really important for now, I guess. > > I think we'll start

Re: TAP output format in pg_regress

2022-11-17 Thread Daniel Gustafsson
> On 10 Nov 2022, at 11:44, Nikolay Shaplov wrote: > I've checked new output, if is conform TAP specification. Checked that prove > consumes new pg_regress output well. Great! > Did not found quick way to include prove TAP harness right into Makefile, so > I > check dumped output, but it is

Re: TAP output format in pg_regress

2022-11-10 Thread Nikolay Shaplov
Hi! Do this patch still need reviewer, or reviewer field in commit-fest have been left empty by mistake? I am fan of TAP-tests, so I am quite happy that pg_regress output changed to TAP tests... I've checked new output, if is conform TAP specification. Checked that prove consumes new pg_regres

Re: TAP output format in pg_regress

2022-10-01 Thread Andres Freund
Hi, On 2022-09-01 14:21:18 +0200, Daniel Gustafsson wrote: > Attached is a v8 which fixes a compiler warning detected by the CFBot. cfbot at the moment does show a warning. A bit surprised to see this warning enabled by default in gcc, but it seems correct here: [20:57:02.892] make -s -j${BUILD_

Re: TAP output format in pg_regress

2022-09-01 Thread Daniel Gustafsson
> On 18 Aug 2022, at 16:40, Andrew Dunstan wrote: > > On 2022-08-18 Th 07:24, Daniel Gustafsson wrote: >> >> One thing I haven't researched yet is if the Buildfarm or CFBot parses the >> current output in any way or just check the exit status of the testrun. > > Buildfarm: just the status. Tha

Re: TAP output format in pg_regress

2022-08-18 Thread Andrew Dunstan
On 2022-08-18 Th 07:24, Daniel Gustafsson wrote: > > One thing I haven't researched yet is if the Buildfarm or CFBot parses the > current output in any way or just check the exit status of the testrun. > > Buildfarm: just the status. cheers andrew -- Andrew Dunstan EDB: https://www.enterpri

Re: TAP output format in pg_regress

2022-08-18 Thread Daniel Gustafsson
> On 18 Aug 2022, at 00:49, Andres Freund wrote: >> while still be TAP compliant enough that running it with prove (with a tiny >> Perl wrapper) works. > > Wonder if we could utilize that for making failures of 002_pg_upgrade.pl and > 027_stream_regress.pl easier to understand, by reporting pg_r

Re: TAP output format in pg_regress

2022-08-17 Thread Andres Freund
Hi, On 2022-08-17 23:04:20 +0200, Daniel Gustafsson wrote: > Attached is a new version of the pg_regress TAP patch which - per reviewer > feedback - does away with dual output formats and just converts the existing > output to be TAP complaint. Cool. > while still be TAP compliant enough that r

Re: TAP output format in pg_regress

2022-08-17 Thread Daniel Gustafsson
Attached is a new version of the pg_regress TAP patch which - per reviewer feedback - does away with dual output formats and just converts the existing output to be TAP complaint. The idea was to keep it fairly human readable while still be TAP compliant enough that running it with prove (with a t

Re: TAP output format in pg_regress

2022-08-02 Thread Jacob Champion
This entry has been waiting on author input for a while (our current threshold is roughly two weeks), so I've marked it Returned with Feedback. Once you think the patchset is ready for review again, you (or any interested party) can resurrect the patch entry by visiting https://commitfest.pos

Re: TAP output format in pg_regress

2022-07-04 Thread Andres Freund
Hi, On 2022-07-04 21:56:24 -0400, Tom Lane wrote: > > For non-parallel tests I think we currently print the test name before > > running > > the test, which obviously doesn't work well when needing to print the 'ok' > > 'not ok' first. > > Is this still a consideration? We got rid of serial_sch

Re: TAP output format in pg_regress

2022-07-04 Thread Tom Lane
Andres Freund writes: > I think with a bit of care the tap output could be nearly the same > "quality". It might not be the absolute "purest" tap output, but who cares. +1 > For non-parallel tests I think we currently print the test name before running > the test, which obviously doesn't work we

Re: TAP output format in pg_regress

2022-07-04 Thread Andres Freund
Hi, On 2022-07-05 00:06:04 +0200, Daniel Gustafsson wrote: > > On 4 Jul 2022, at 16:27, Peter Eisentraut > > wrote: > > > > On 29.06.22 21:50, Daniel Gustafsson wrote: > >> Attached is a new version of this patch, which completes the TAP output > >> format > >> option such that all codepaths em

Re: TAP output format in pg_regress

2022-07-04 Thread Daniel Gustafsson
> On 4 Jul 2022, at 16:39, Tom Lane wrote: > > Peter Eisentraut writes: >> I'm not sure what to make of all these options. I think providing a TAP >> output for pg_regress is a good idea. But then do we still need the old >> output? Is it worth maintaining two output formats that display ex

Re: TAP output format in pg_regress

2022-07-04 Thread Tom Lane
Andres Freund writes: >> Both of those things are fairly critical for test development. You >> need to know what else might be running in parallel with a test case, >> and you need to know whether you just bloated the runtime unreasonably. > That should be doable with tap as well - afaics the ou

Re: TAP output format in pg_regress

2022-07-04 Thread Daniel Gustafsson
> On 4 Jul 2022, at 16:27, Peter Eisentraut > wrote: > > On 29.06.22 21:50, Daniel Gustafsson wrote: >> Attached is a new version of this patch, which completes the TAP output >> format >> option such that all codepaths emitting output are TAP compliant. The >> verbose >> option is fixed to t

Re: TAP output format in pg_regress

2022-07-04 Thread Daniel Gustafsson
> On 4 Jul 2022, at 22:13, Andres Freund wrote: > > Hi, > > On 2022-06-29 21:50:45 +0200, Daniel Gustafsson wrote: >> @@ -279,8 +648,7 @@ stop_postmaster(void) >> r = system(buf); >> if (r != 0) >> { >> -fprintf(stderr, _("\n%s: could no

Re: TAP output format in pg_regress

2022-07-04 Thread Andres Freund
Hi, On 2022-06-29 21:50:45 +0200, Daniel Gustafsson wrote: > @@ -279,8 +648,7 @@ stop_postmaster(void) > r = system(buf); > if (r != 0) > { > - fprintf(stderr, _("\n%s: could not stop postmaster: > exit code was %d\n"), > -

Re: TAP output format in pg_regress

2022-07-04 Thread Andres Freund
Hi, > Peter Eisentraut writes: > > I'm not sure what to make of all these options. I think providing a TAP > > output for pg_regress is a good idea. But then do we still need the old > > output? Is it worth maintaining two output formats that display exactly > > the same thing in slightly diff

Re: TAP output format in pg_regress

2022-07-04 Thread Peter Eisentraut
On 04.07.22 16:39, Tom Lane wrote: Probably is, because this is bad: ... The proposed default format now hides the fact that some tests are started in parallel. I remember the last time I wanted to tweak the output of the parallel tests, people were very attached to the particular timing and s

Re: TAP output format in pg_regress

2022-07-04 Thread Tom Lane
Peter Eisentraut writes: > I'm not sure what to make of all these options. I think providing a TAP > output for pg_regress is a good idea. But then do we still need the old > output? Is it worth maintaining two output formats that display exactly > the same thing in slightly different ways?

Re: TAP output format in pg_regress

2022-07-04 Thread Peter Eisentraut
On 29.06.22 21:50, Daniel Gustafsson wrote: Attached is a new version of this patch, which completes the TAP output format option such that all codepaths emitting output are TAP compliant. The verbose option is fixed to to not output extraneous newlines which the previous PoC did. The output it

Re: TAP output format in pg_regress

2022-06-29 Thread Daniel Gustafsson
Attached is a new version of this patch, which completes the TAP output format option such that all codepaths emitting output are TAP compliant. The verbose option is fixed to to not output extraneous newlines which the previous PoC did. The output it made to conform to the original TAP spec sinc

Re: TAP output format in pg_regress

2022-03-24 Thread Daniel Gustafsson
> On 22 Mar 2022, at 00:49, Andres Freund wrote: > This is failing with segmentation faults on cfbot: > https://cirrus-ci.com/task/4879227009892352?logs=test_world#L21 > > Looks like something around isolationtester is broken? It could be triggered by plpgsql tests as well, and was (as usual) a

Re: TAP output format in pg_regress

2022-03-21 Thread Andres Freund
Hi, On 2022-02-22 15:10:11 +0100, Daniel Gustafsson wrote: > > On 22 Feb 2022, at 00:08, Daniel Gustafsson wrote: > > > I'll fix that. > > The attached v3 fixes that thinko, and cleans up a lot of the output which > isn't diagnostic per the TAP spec to make it less noisy. It also fixes tag > s

Re: TAP output format in pg_regress

2022-02-22 Thread Daniel Gustafsson
> On 22 Feb 2022, at 18:13, Andres Freund wrote: > On 2022-02-22 15:10:11 +0100, Daniel Gustafsson wrote: >> The errorpaths that exit(2) the testrun should be converted to "bail out" >> lines >> when running with TAP output, but apart from that I think it's fairly spec >> compliant. > > I'd muc

Re: TAP output format in pg_regress

2022-02-22 Thread Andres Freund
Hi, Thanks for the updated version! On 2022-02-22 15:10:11 +0100, Daniel Gustafsson wrote: > The errorpaths that exit(2) the testrun should be converted to "bail out" > lines > when running with TAP output, but apart from that I think it's fairly spec > compliant. I'd much rather not use BAIL -

Re: TAP output format in pg_regress

2022-02-22 Thread Daniel Gustafsson
> On 22 Feb 2022, at 00:08, Daniel Gustafsson wrote: > I'll fix that. The attached v3 fixes that thinko, and cleans up a lot of the output which isn't diagnostic per the TAP spec to make it less noisy. It also fixes tag support used in the ECPG tests and a few small cleanups. There is a blank