Re: TAP output format in pg_regress

2023-03-31 Thread Daniel Gustafsson
> On 29 Mar 2023, at 22:08, Daniel Gustafsson wrote: > >> On 29 Mar 2023, at 21:14, Daniel Gustafsson wrote: > >> Ugh, I clearly should've stayed on the couch yesterday. > > Maybe today as well, just as I had sent this I realized there is mention of > the > output format in the docs that I

Re: TAP output format in pg_regress

2023-03-29 Thread Daniel Gustafsson
> On 29 Mar 2023, at 21:14, Daniel Gustafsson wrote: > Ugh, I clearly should've stayed on the couch yesterday. Maybe today as well, just as I had sent this I realized there is mention of the output format in the docs that I had missed. The attached changes that as well to match the new format.

Re: TAP output format in pg_regress

2023-03-29 Thread Daniel Gustafsson
> On 29 Mar 2023, at 09:08, Peter Eisentraut > wrote: > > On 28.03.23 15:56, Daniel Gustafsson wrote: >>> On 28 Mar 2023, at 15:26, Daniel Gustafsson wrote: >>> I think the attached is a good candidate for going in, but I would like to >>> see it >>> for another spin in the CF bot first. >>

Re: TAP output format in pg_regress

2023-03-29 Thread Peter Eisentraut
On 28.03.23 15:56, Daniel Gustafsson wrote: On 28 Mar 2023, at 15:26, Daniel Gustafsson wrote: I think the attached is a good candidate for going in, but I would like to see it for another spin in the CF bot first. Another candidate due to a thinko which raised a compiler warning. This

Re: TAP output format in pg_regress

2023-03-28 Thread Daniel Gustafsson
> On 28 Mar 2023, at 15:26, Daniel Gustafsson wrote: > I think the attached is a good candidate for going in, but I would like to > see it > for another spin in the CF bot first. Another candidate due to a thinko which raised a compiler warning. -- Daniel Gustafsson

Re: TAP output format in pg_regress

2023-03-28 Thread Daniel Gustafsson
> On 15 Mar 2023, at 11:36, Peter Eisentraut > wrote: > > On 24.02.23 10:49, Daniel Gustafsson wrote: >> Another rebase on top of 337903a16f. Unless there are conflicting reviews, I >> consider this patch to be done and ready for going in during the next CF. > > I think this is just about as

Re: TAP output format in pg_regress

2023-03-15 Thread Peter Eisentraut
On 24.02.23 10:49, Daniel Gustafsson wrote: Another rebase on top of 337903a16f. Unless there are conflicting reviews, I consider this patch to be done and ready for going in during the next CF. I think this is just about as good as it's going to get, so I think we can consider committing

Re: TAP output format in pg_regress

2023-02-24 Thread Daniel Gustafsson
Another rebase on top of 337903a16f. Unless there are conflicting reviews, I consider this patch to be done and ready for going in during the next CF. -- Daniel Gustafsson v17-0001-Emit-TAP-compliant-output-from-pg_regress.patch Description: Binary data

Re: TAP output format in pg_regress

2023-01-23 Thread Daniel Gustafsson
esolves the > conflict which came from the recent commit removing the "ignore" > functionality. > It also fixes a few small things pointed out off-list. And now with the missing attachment.. -- Daniel Gustafsson v16-0001-Change-pg_regress-output-format-to-be-TAP-compli.patch Description: Binary data

Re: TAP output format in pg_regress

2023-01-23 Thread Daniel Gustafsson
> On 19 Jan 2023, at 12:14, vignesh C wrote: > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: The attached v16 is a rebase on top of current master which resolves the conflict which came from the recent commit removing the "ignore" functionality. It also

Re: TAP output format in pg_regress

2023-01-19 Thread Daniel Gustafsson
> On 19 Jan 2023, at 12:14, vignesh C wrote: > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: Sorry for the silence, and thanks for your previous rebase, $life has kept me too busy lately. I'll post a rebased version shortly. -- Daniel Gustafsson

Re: TAP output format in pg_regress

2023-01-19 Thread vignesh C
> > > > > > > The patch does not apply on top of HEAD as in [1], please post a rebased > > patch: > > === Applying patches on top of PostgreSQL commit ID > > 92957ed98c5c565362ce665266132a7f08f6b0c0 === > > === applying patch > > ./v14-0001-Change-pg_regre

Re: TAP output format in pg_regress

2023-01-05 Thread vignesh C
gt; > PS Should I change commitfest status? > > > > Sure, go ahead. > > > > The patch does not apply on top of HEAD as in [1], please post a rebased > patch: > === Applying patches on top of PostgreSQL commit ID > 92957ed98c5c565362ce665266132a7f08f6b0c0 === >

Re: TAP output format in pg_regress

2023-01-03 Thread vignesh C
please post a rebased patch: === Applying patches on top of PostgreSQL commit ID 92957ed98c5c565362ce665266132a7f08f6b0c0 === === applying patch ./v14-0001-Change-pg_regress-output-format-to-be-TAP-compli.patch patching file meson.build Hunk #1 FAILED at 2968. 1 out of 1 hunk FAILED -- saving rejects to file meson.build.rej [1] - http://cfbot.cputube.org/patch_41_3837.log Regards, Vignesh

Re: TAP output format in pg_regress

2022-11-28 Thread Daniel Gustafsson
gt; PS Should I change commitfest status? Sure, go ahead. -- Daniel Gustafsson https://vmware.com/ v14-0001-Change-pg_regress-output-format-to-be-TAP-compli.patch Description: Binary data

Re: TAP output format in pg_regress

2022-11-28 Thread Nikolay Shaplov
В письме от понедельник, 28 ноября 2022 г. 21:28:48 MSK пользователь Andres Freund написал: > On 2022-11-28 14:13:16 +0100, Daniel Gustafsson wrote: > > > On 27 Nov 2022, at 11:22, Nikolay Shaplov wrote: > > > В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel > > >

Re: TAP output format in pg_regress

2022-11-28 Thread Andres Freund
On 2022-11-28 14:13:16 +0100, Daniel Gustafsson wrote: > > On 27 Nov 2022, at 11:22, Nikolay Shaplov wrote: > > В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel > > Gustafsson написал: > > > I wold suggest to use word immediate instead of noatexit. This will do the > > code

Re: TAP output format in pg_regress

2022-11-28 Thread Daniel Gustafsson
> On 27 Nov 2022, at 11:22, Nikolay Shaplov wrote: > В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel > Gustafsson написал: > I wold suggest to use word immediate instead of noatexit. This will do the > code much more sensible for me. I think noatexit is clearer since

Re: TAP output format in pg_regress

2022-11-27 Thread Nikolay Shaplov
В письме от суббота, 26 ноября 2022 г. 23:35:45 MSK пользователь Daniel Gustafsson написал: > > + #define bail_noatexit(...) bail_out(true, __VA_ARGS__) > > > > BTW what does "noat" stands for? I thought it is typo too :-) and > > originally meant to be "not". > > Calling _exit() will cause

Re: TAP output format in pg_regress

2022-11-26 Thread Daniel Gustafsson
ls 39 ms ok 150 stats_ext 2578 ms ok 151 collate.linux.utf8 27 ms ok 152 test select_parallel 668 ms ok 153 test write_parallel84 ms ok 154 test

Re: TAP output format in pg_regress

2022-11-26 Thread Nikolay Shaplov
В письме от пятница, 25 ноября 2022 г. 00:20:01 MSK пользователь Daniel Gustafsson написал: + /* + * The width of the testname field when printing to ensure vertical alignment + * of test runtimes. Thius number

Re: TAP output format in pg_regress

2022-11-24 Thread Daniel Gustafsson
the line will make it less readable instead. > > I don't think it's terrible either. I do think it'd also be ok to switch > between ok / not ok within a single printf, making it easier to keep them in > sync. I made it into a single printf to see what it would look like, with some additi

Re: TAP output format in pg_regress

2022-11-24 Thread Andres Freund
On November 24, 2022 11:07:43 AM PST, Daniel Gustafsson wrote: >> On 24 Nov 2022, at 18:07, Nikolay Shaplov wrote: >One option could be to redefine bail() to take the exit function as a parameter >and have the caller pass the preferred exit handler. > >-bail_out(bool non_rec, const char

Re: TAP output format in pg_regress

2022-11-24 Thread Daniel Gustafsson
> On 24 Nov 2022, at 18:07, Nikolay Shaplov wrote: > > You guys are really fast... I only think about problem, it is already > mentioned here... Most issues I've noticed are already fixed before I was > able > to say something. Thanks for looking and reviewing! > /*

Re: TAP output format in pg_regress

2022-11-24 Thread Nikolay Shaplov
You guys are really fast... I only think about problem, it is already mentioned here... Most issues I've noticed are already fixed before I was able to say something. Nevertheless... /* * Bailing out is for

Re: TAP output format in pg_regress

2022-11-24 Thread Daniel Gustafsson
ally think thats fine, but it can be tweaked to print "# tags: stdout, source" instead if we want to keep it closer to the current format. > While the above may sound like a fair bit of work, I think it's actually not > that much work, and we should strive to get this committed pretty soon! The attached v11 fixes the above and have had a pgindent run on it as it's hopefully getting close to done. Thanks again for reviewing! -- Daniel Gustafsson https://vmware.com/ v11-0001-Change-pg_regress-output-format-to-be-TAP-compli.patch Description: Binary data

Re: TAP output format in pg_regress

2022-11-22 Thread Andres Freund
Hi, On 2022-11-22 23:17:44 +0100, Daniel Gustafsson wrote: > The attached v10 attempts to address the points raised above. Notes and > diagnostics are printed to stdout/stderr respectively and the TAP emitter is > changed to move more of the syntax into it making it less painful on the >

Re: TAP output format in pg_regress

2022-11-22 Thread Daniel Gustafsson
the most >> important >> detail is immediately visible when looking at the failed test result. > > Agreed on all three points. The attached v10 attempts to address the points raised above. Notes and diagnostics are printed to stdout/stderr respectively and the TAP emitter is changed to move more of the syntax into it making it less painful on the translators. -- Daniel Gustafsson https://vmware.com/ v10-0002-Experimental-meson-treat-regress-tests-as-TAP.patch Description: Binary data v10-0001-Make-pg_regress-output-format-TAP-compliant.patch Description: Binary data

Re: TAP output format in pg_regress

2022-11-21 Thread Dagfinn Ilmari Mannsåker
Andres Freund writes: > But either way, it seems nicer to output the # inside a helper function? Note that the helper function should inject '# ' at the start of every line in the message, not just the first line. It might also be worth having two separate functions: one that prints to stdout,

Re: TAP output format in pg_regress

2022-11-17 Thread Andres Freund
Hi, On 2022-11-17 11:36:13 +0100, Daniel Gustafsson wrote: > > On 10 Nov 2022, at 11:44, Nikolay Shaplov wrote: > > Did not found quick way to include prove TAP harness right into Makefile, > > so I > > check dumped output, but it is not really important for now, I guess. > > I think we'll

Re: TAP output format in pg_regress

2022-11-17 Thread Daniel Gustafsson
ecpg/ecpg OK 23.68s 62 subtests passed I'm not sure if that's the right way to go about configuring regress tests as TAP emitting, but it at least shows that meson is properly parsing the output AFAICT. -- Daniel Gustafsson https://vmware.com/ v9-0001-Make-pg_regress-output-format-TAP

Re: TAP output format in pg_regress

2022-11-10 Thread Nikolay Shaplov
Hi! Do this patch still need reviewer, or reviewer field in commit-fest have been left empty by mistake? I am fan of TAP-tests, so I am quite happy that pg_regress output changed to TAP tests... I've checked new output, if is conform TAP specification. Checked that prove consumes new

Re: TAP output format in pg_regress

2022-10-01 Thread Andres Freund
Hi, On 2022-09-01 14:21:18 +0200, Daniel Gustafsson wrote: > Attached is a v8 which fixes a compiler warning detected by the CFBot. cfbot at the moment does show a warning. A bit surprised to see this warning enabled by default in gcc, but it seems correct here: [20:57:02.892] make -s

Re: TAP output format in pg_regress

2022-09-01 Thread Daniel Gustafsson
> Buildfarm: just the status. Thanks for confirming, the same is true for CFBot AFAICT. Attached is a v8 which fixes a compiler warning detected by the CFBot. -- Daniel Gustafsson https://vmware.com/ v8-0001-Make-pg_regress-output-format-TAP-compliant.patch Description: Binary data

Re: TAP output format in pg_regress

2022-08-18 Thread Andrew Dunstan
On 2022-08-18 Th 07:24, Daniel Gustafsson wrote: > > One thing I haven't researched yet is if the Buildfarm or CFBot parses the > current output in any way or just check the exit status of the testrun. > > Buildfarm: just the status. cheers andrew -- Andrew Dunstan EDB:

Re: TAP output format in pg_regress

2022-08-18 Thread Daniel Gustafsson
oins 194 ms ok 214 fast_default 178 ms 1..214 I think that's fairly readable, and not that much different from today. -- Daniel Gustafsson https://vmware.com/ v7-0001-Make-pg_regress-output-format-TAP-compliant.patch Description: Binary data

Re: TAP output format in pg_regress

2022-08-17 Thread Andres Freund
Hi, On 2022-08-17 23:04:20 +0200, Daniel Gustafsson wrote: > Attached is a new version of the pg_regress TAP patch which - per reviewer > feedback - does away with dual output formats and just converts the existing > output to be TAP complaint. Cool. > while still be TAP compliant enough that

Re: TAP output format in pg_regress

2022-08-17 Thread Daniel Gustafsson
perhaps be handled by marking the test name in some way like "tablespace (serial) or prefixing with a symbol or somerthing. I can take a stab at that in case we think that level of detail is important to preserve. -- Daniel Gustafsson https://vmware.com/ v6-0001-Make-pg_regress-output-format-TAP-compliant.patch Description: Binary data

Re: TAP output format in pg_regress

2022-08-02 Thread Jacob Champion
This entry has been waiting on author input for a while (our current threshold is roughly two weeks), so I've marked it Returned with Feedback. Once you think the patchset is ready for review again, you (or any interested party) can resurrect the patch entry by visiting

Re: TAP output format in pg_regress

2022-07-04 Thread Andres Freund
Hi, On 2022-07-04 21:56:24 -0400, Tom Lane wrote: > > For non-parallel tests I think we currently print the test name before > > running > > the test, which obviously doesn't work well when needing to print the 'ok' > > 'not ok' first. > > Is this still a consideration? We got rid of

Re: TAP output format in pg_regress

2022-07-04 Thread Tom Lane
Andres Freund writes: > I think with a bit of care the tap output could be nearly the same > "quality". It might not be the absolute "purest" tap output, but who cares. +1 > For non-parallel tests I think we currently print the test name before running > the test, which obviously doesn't work

Re: TAP output format in pg_regress

2022-07-04 Thread Andres Freund
Hi, On 2022-07-05 00:06:04 +0200, Daniel Gustafsson wrote: > > On 4 Jul 2022, at 16:27, Peter Eisentraut > > wrote: > > > > On 29.06.22 21:50, Daniel Gustafsson wrote: > >> Attached is a new version of this patch, which completes the TAP output > >> format > >> option such that all codepaths

Re: TAP output format in pg_regress

2022-07-04 Thread Daniel Gustafsson
> On 4 Jul 2022, at 16:39, Tom Lane wrote: > > Peter Eisentraut writes: >> I'm not sure what to make of all these options. I think providing a TAP >> output for pg_regress is a good idea. But then do we still need the old >> output? Is it worth maintaining two output formats that display

Re: TAP output format in pg_regress

2022-07-04 Thread Tom Lane
Andres Freund writes: >> Both of those things are fairly critical for test development. You >> need to know what else might be running in parallel with a test case, >> and you need to know whether you just bloated the runtime unreasonably. > That should be doable with tap as well - afaics the

Re: TAP output format in pg_regress

2022-07-04 Thread Daniel Gustafsson
> On 4 Jul 2022, at 16:27, Peter Eisentraut > wrote: > > On 29.06.22 21:50, Daniel Gustafsson wrote: >> Attached is a new version of this patch, which completes the TAP output >> format >> option such that all codepaths emitting output are TAP compliant. The >> verbose >> option is fixed to

Re: TAP output format in pg_regress

2022-07-04 Thread Daniel Gustafsson
> On 4 Jul 2022, at 22:13, Andres Freund wrote: > > Hi, > > On 2022-06-29 21:50:45 +0200, Daniel Gustafsson wrote: >> @@ -279,8 +648,7 @@ stop_postmaster(void) >> r = system(buf); >> if (r != 0) >> { >> -fprintf(stderr, _("\n%s: could

Re: TAP output format in pg_regress

2022-07-04 Thread Andres Freund
Hi, On 2022-06-29 21:50:45 +0200, Daniel Gustafsson wrote: > @@ -279,8 +648,7 @@ stop_postmaster(void) > r = system(buf); > if (r != 0) > { > - fprintf(stderr, _("\n%s: could not stop postmaster: > exit code was %d\n"), > -

Re: TAP output format in pg_regress

2022-07-04 Thread Andres Freund
Hi, > Peter Eisentraut writes: > > I'm not sure what to make of all these options. I think providing a TAP > > output for pg_regress is a good idea. But then do we still need the old > > output? Is it worth maintaining two output formats that display exactly > > the same thing in slightly

Re: TAP output format in pg_regress

2022-07-04 Thread Peter Eisentraut
On 04.07.22 16:39, Tom Lane wrote: Probably is, because this is bad: ... The proposed default format now hides the fact that some tests are started in parallel. I remember the last time I wanted to tweak the output of the parallel tests, people were very attached to the particular timing and

Re: TAP output format in pg_regress

2022-07-04 Thread Tom Lane
Peter Eisentraut writes: > I'm not sure what to make of all these options. I think providing a TAP > output for pg_regress is a good idea. But then do we still need the old > output? Is it worth maintaining two output formats that display exactly > the same thing in slightly different ways?

Re: TAP output format in pg_regress

2022-07-04 Thread Peter Eisentraut
On 29.06.22 21:50, Daniel Gustafsson wrote: Attached is a new version of this patch, which completes the TAP output format option such that all codepaths emitting output are TAP compliant. The verbose option is fixed to to not output extraneous newlines which the previous PoC did. The output

Re: TAP output format in pg_regress

2022-06-29 Thread Daniel Gustafsson
Attached is a new version of this patch, which completes the TAP output format option such that all codepaths emitting output are TAP compliant. The verbose option is fixed to to not output extraneous newlines which the previous PoC did. The output it made to conform to the original TAP spec

Re: TAP output format in pg_regress

2022-03-24 Thread Daniel Gustafsson
> On 22 Mar 2022, at 00:49, Andres Freund wrote: > This is failing with segmentation faults on cfbot: > https://cirrus-ci.com/task/4879227009892352?logs=test_world#L21 > > Looks like something around isolationtester is broken? It could be triggered by plpgsql tests as well, and was (as usual)

Re: TAP output format in pg_regress

2022-03-21 Thread Andres Freund
Hi, On 2022-02-22 15:10:11 +0100, Daniel Gustafsson wrote: > > On 22 Feb 2022, at 00:08, Daniel Gustafsson wrote: > > > I'll fix that. > > The attached v3 fixes that thinko, and cleans up a lot of the output which > isn't diagnostic per the TAP spec to make it less noisy. It also fixes tag >

Re: TAP output format in pg_regress

2022-02-22 Thread Daniel Gustafsson
> On 22 Feb 2022, at 18:13, Andres Freund wrote: > On 2022-02-22 15:10:11 +0100, Daniel Gustafsson wrote: >> The errorpaths that exit(2) the testrun should be converted to "bail out" >> lines >> when running with TAP output, but apart from that I think it's fairly spec >> compliant. > > I'd

Re: TAP output format in pg_regress

2022-02-22 Thread Andres Freund
Hi, Thanks for the updated version! On 2022-02-22 15:10:11 +0100, Daniel Gustafsson wrote: > The errorpaths that exit(2) the testrun should be converted to "bail out" > lines > when running with TAP output, but apart from that I think it's fairly spec > compliant. I'd much rather not use BAIL

Re: TAP output format in pg_regress

2022-02-22 Thread Daniel Gustafsson
> On 22 Feb 2022, at 00:08, Daniel Gustafsson wrote: > I'll fix that. The attached v3 fixes that thinko, and cleans up a lot of the output which isn't diagnostic per the TAP spec to make it less noisy. It also fixes tag support used in the ECPG tests and a few small cleanups. There is a blank

TAP output format in pg_regress

2022-02-21 Thread Daniel Gustafsson
Starting a new thread on the TAP patch from the "[RFC] building postgres with meson" thread at 20220221165228.aqnfg45mceym7...@alap3.anarazel.de to have somewhere to discuss this patch. > On 21 Feb 2022, at 17:52, Andres Freund wrote: > On 2021-10-13 13:54:10 +0200, Daniel Gustafsson wrote: >>