Re: CI and test improvements

2024-04-10 Thread Michael Paquier
On Mon, Apr 08, 2024 at 05:54:10PM +0300, Andrey M. Borodin wrote: > Justin, Peter, I can't determine actual status of the CF entry > [0]. May I ask someone of you to move patch to next CF or close as > committed? 0002 is the only thing committed as of 21a71648d39f. I can see the value in 0001,

Re: CI and test improvements

2024-04-08 Thread Andrey M. Borodin
> On 19 Feb 2024, at 11:33, Peter Eisentraut wrote: > > Ok, I didn't see that my feedback had been addressed. I have committed this > patch. Justin, Peter, I can't determine actual status of the CF entry [0]. May I ask someone of you to move patch to next CF or close as committed? Thanks!

Re: CI and test improvements

2024-02-19 Thread Peter Eisentraut
On 13.02.24 20:10, Justin Pryzby wrote: On Mon, Mar 13, 2023 at 07:39:52AM +0100, Peter Eisentraut wrote: On 03.02.23 15:26, Justin Pryzby wrote: 9baf41674ad pg_upgrade: tap test: exercise --link and --clone This seems like a good idea. On Wed, Mar 15, 2023 at 10:58:41AM +0100, Peter

Re: CI and test improvements

2024-02-13 Thread Justin Pryzby
On Wed, Jan 31, 2024 at 11:59:21AM +0100, Alvaro Herrera wrote: > On 2024-Jan-31, vignesh C wrote: > > > Are we planning to do anything more on this? I was not sure if we > > should move this to next commitfest or return it. > > Well, the patches don't apply anymore since .cirrus.tasks.yml has

Re: CI and test improvements

2024-01-31 Thread Alvaro Herrera
On 2024-Jan-31, vignesh C wrote: > Are we planning to do anything more on this? I was not sure if we > should move this to next commitfest or return it. Well, the patches don't apply anymore since .cirrus.tasks.yml has been created. However, I'm sure we still want [some of] the improvements to

Re: CI and test improvements

2024-01-31 Thread vignesh C
On Thu, 18 Jan 2024 at 06:46, Michael Paquier wrote: > > On Wed, Jan 17, 2024 at 05:34:00PM +0530, vignesh C wrote: > > Are we planning to do anything more in this thread, the thread has > > been idle for more than 7 months. If nothing more is planned for this, > > I'm planning to close this

Re: CI and test improvements

2024-01-17 Thread Michael Paquier
On Wed, Jan 17, 2024 at 05:34:00PM +0530, vignesh C wrote: > Are we planning to do anything more in this thread, the thread has > been idle for more than 7 months. If nothing more is planned for this, > I'm planning to close this commitfest entry in this commitfest. Oops, this went through the

Re: CI and test improvements

2024-01-17 Thread vignesh C
On Wed, 12 Jul 2023 at 11:38, Michael Paquier wrote: > > On Wed, Jul 12, 2023 at 12:56:17AM -0500, Justin Pryzby wrote: > > And, since 681d9e462: > > > > security is missing from contrib/seg/meson.build > > Ugh. Good catch! Are we planning to do anything more in this thread, the thread has been

Re: CI and test improvements

2023-07-12 Thread Michael Paquier
On Wed, Jul 12, 2023 at 12:56:17AM -0500, Justin Pryzby wrote: > And, since 681d9e462: > > security is missing from contrib/seg/meson.build Ugh. Good catch! -- Michael signature.asc Description: PGP signature

Re: CI and test improvements

2023-07-11 Thread Justin Pryzby
On Tue, Jan 17, 2023 at 11:35:09AM -0600, Justin Pryzby wrote: > However, this finds two real problems and one false-positive with > missing regress/isolation tests: > > $ for makefile in `find src contrib -name Makefile`; do for testname in `sed > -r '/^(REGRESS|ISOLATION) =/!d; s///; :l;

Re: CI and test improvements

2023-07-03 Thread Peter Eisentraut
On 12.04.23 03:05, Justin Pryzby wrote: The patch is rebased now that meson is updated to avoid the windows python warnings (thanks Andres). To keep this moving along, I have committed [PATCH 3/8] cirrus/freebsd: define ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS

Re: CI and test improvements

2023-04-11 Thread Justin Pryzby
On Wed, Mar 15, 2023 at 04:57:34PM +0100, Peter Eisentraut wrote: > On 15.03.23 15:56, Justin Pryzby wrote: > > I'm surprised if there's any question about the merits of making > > documentation easily available for review. Several people have agreed; > > one person mailed me privately

Re: CI and test improvements

2023-03-15 Thread Peter Eisentraut
On 15.03.23 15:56, Justin Pryzby wrote: I'm surprised if there's any question about the merits of making documentation easily available for review. Several people have agreed; one person mailed me privately specifically to ask how to show HTML docs on cirrusci. Anyway, all this stuff is best

Re: CI and test improvements

2023-03-15 Thread Justin Pryzby
On Wed, Mar 15, 2023 at 10:58:41AM +0100, Peter Eisentraut wrote: > On 14.03.23 05:56, Justin Pryzby wrote: > > I'm soliticing feedback on those patches that I've sent recently - I've > > elided patches if they have some unresolved issue. > > > [PATCH 1/8] cirrus/windows: add

Re: CI and test improvements

2023-03-15 Thread Peter Eisentraut
On 14.03.23 05:56, Justin Pryzby wrote: I'm soliticing feedback on those patches that I've sent recently - I've elided patches if they have some unresolved issue. > [PATCH 1/8] cirrus/windows: add compiler_warnings_script Needs a better description of what it actually does. (And fewer "I'm

Re: CI and test improvements

2023-03-13 Thread Justin Pryzby
On Mon, Mar 13, 2023 at 07:39:52AM +0100, Peter Eisentraut wrote: > On 03.02.23 15:26, Justin Pryzby wrote: > > rebased, and re-including a patch to show code coverage of changed > > files. > > This constant flow of patches under one subject doesn't lend itself well to > the commit fest model of

Re: CI and test improvements

2023-03-13 Thread Peter Eisentraut
On 03.02.23 15:26, Justin Pryzby wrote: rebased, and re-including a patch to show code coverage of changed files. This constant flow of patches under one subject doesn't lend itself well to the commit fest model of trying to finish things up. I can't quite tell which of these patches are

Re: CI and test improvements

2023-02-03 Thread Justin Pryzby
rebased, and re-including a patch to show code coverage of changed files. a5b3e50d922 cirrus/windows: add compiler_warnings_script 4c98dcb0e03 cirrus/freebsd: run with more CPUs+RAM and do not repartition aaeef938ed4 cirrus/freebsd: define ENFORCE_REGRESSION_TEST_NAME_RESTRICTIONS 9baf41674ad

Re: CI and test improvements

2023-02-02 Thread Thomas Munro
On Thu, Feb 2, 2023 at 2:12 PM Tom Lane wrote: > Thomas Munro writes: > > Some observations: > > So what should our policy be on when to roll the CI image forward? I > > guess around New Year/now (~6 months after release) is a good time and > > we should just do it. Anyone got a reason why we

Re: CI and test improvements

2023-02-01 Thread Tom Lane
Thomas Munro writes: > Some observations: > * macOS has a new release every year in June[1] > * updates cease after three years[1] > * thus three releases are in support (by that definition) at a time > * we need an image on Cirrus; 13 appeared ~1 month later[2] > * we need Homebrew support; 13

Re: CI and test improvements

2023-02-01 Thread Thomas Munro
On Fri, Dec 30, 2022 at 4:59 PM Thomas Munro wrote: > On Wed, Nov 23, 2022 at 11:57 AM Justin Pryzby wrote: > > [PATCH 03/10] cirrus/macos: update to macos ventura > > I don't know any reason not to push this one too, but it's not time critical. Some observations: * macOS has a new release

Re: CI and test improvements

2023-01-17 Thread Justin Pryzby
On Tue, Jan 17, 2023 at 11:56:42AM -0800, Andres Freund wrote: > > $(CF_PGP_TESTS) is missing from contrib/pgcrypto/meson.build > > Assume that's the false positive? Yes > > I also tried but failed to write something to warn if "meson test" was > > run with a list of tests but without

Re: CI and test improvements

2023-01-17 Thread Andres Freund
Hi, On 2023-01-17 11:35:09 -0600, Justin Pryzby wrote: > The autoconf system runs all tap tests in t/*.pl, but meson requires > enumerating them in ./meson.build. Yes. It was a mistake that we ever used t/*.pl for make. For one, it means that make can't control concurrency meaningfully, due to

Re: CI and test improvements

2023-01-17 Thread Justin Pryzby
The autoconf system runs all tap tests in t/*.pl, but meson requires enumerating them in ./meson.build. This checks for and finds no missing tests in the current tree: $ for pl in `find src contrib -path '*/t/*.pl'`; do base=${pl##*/}; dir=${pl%/*}; meson=${dir%/*}/meson.build; grep "$base"

Re: CI and test improvements

2023-01-04 Thread Justin Pryzby
On Tue, Nov 22, 2022 at 04:57:44PM -0600, Justin Pryzby wrote: > I shuffled my branch around and sending now the current "docs" patches, > but I suppose this is waiting on the "convert CompilerWarnings task to > meson" patch. In case it's not, here's a version to do that now. >From

Re: CI and test improvements

2023-01-04 Thread Justin Pryzby
On Mon, Nov 21, 2022 at 02:45:42PM -0800, Andres Freund wrote: > On 2022-11-13 17:53:04 -0600, Justin Pryzby wrote: > > > > From: Justin Pryzby > > > > Date: Tue, 26 Jul 2022 20:30:02 -0500 > > > > Subject: [PATCH 6/8] cirrus/ccache: add explicit cache keys.. > > > > > > > > Since otherwise,

Re: CI and test improvements

2023-01-03 Thread vignesh C
On Fri, 30 Dec 2022 at 09:29, Thomas Munro wrote: > > On Wed, Nov 23, 2022 at 11:57 AM Justin Pryzby wrote: > > [PATCH 02/10] cirrus/macos: switch to "macos_instance" / M1.. > > Duelling patches. > > Bilal's patch[1] uses the matrix feature to run the tests on both > Intel and ARM, which made

Re: CI and test improvements

2022-12-29 Thread Thomas Munro
On Wed, Nov 23, 2022 at 11:57 AM Justin Pryzby wrote: > [PATCH 02/10] cirrus/macos: switch to "macos_instance" / M1.. Duelling patches. Bilal's patch[1] uses the matrix feature to run the tests on both Intel and ARM, which made sense when he proposed it, but according to Cirrus CI warnings, the

Re: CI and test improvements

2022-11-26 Thread Andrew Dunstan
[ resending to -hackers because of list issues ] On 2022-05-28 Sa 11:37, Justin Pryzby wrote: > I'm "joining" a bunch of unresolved threads hoping to present them better > since > they're related and I'm maintaining them together anyway. > >

Re: CI and test improvements

2022-11-22 Thread Justin Pryzby
On Mon, Nov 21, 2022 at 02:45:42PM -0800, Andres Freund wrote: > > > > +ninja -C build |tee build/meson-logs/build.txt > > > > +REM Since pipes lose exit status of the preceding command, rerun > > > > compilation, > > > > +REM without the pipe exiting now if it fails, rather than

Re: CI and test improvements

2022-11-21 Thread Andres Freund
Hi, On 2022-11-13 17:53:04 -0600, Justin Pryzby wrote: > On Fri, Nov 04, 2022 at 06:59:46PM -0700, Andres Freund wrote: > > > diff --git a/.cirrus.yml b/.cirrus.yml > > > index 9f2282471a9..99ac09dc679 100644 > > > --- a/.cirrus.yml > > > +++ b/.cirrus.yml > > > @@ -451,12 +451,20 @@ task: > > >

Re: CI and test improvements

2022-11-21 Thread Andres Freund
Hi, On 2022-11-19 13:18:54 -0800, Andres Freund wrote: > I'll try to repost a version of the ubsan/asan patch together with the > sanitycheck patch and see how that looks. I just pushed the prerequisite patch making UBSAN_OPTIONS work. Attached is 1) addition of SanityCheck 2) use of asan and

Re: CI and test improvements

2022-11-19 Thread Andres Freund
Hi, On 2022-11-19 15:45:06 -0600, Justin Pryzby wrote: > What do you mean "temporarily" ? I think you're implying that the > Warnings task is fast but (at least right now) it is not. In the sense that we don't need all CPUs until the whole commit has finished testing (none of the tasks are the

Re: CI and test improvements

2022-11-19 Thread Justin Pryzby
On Sat, Nov 19, 2022 at 01:18:54PM -0800, Andres Freund wrote: > > Also, if CompilerWarnings doesn't depend on Linux, that means those two > > tasks will normally start and run simultaneously, which means a single > > branch will use all 8 of the linux CPUs available from cirrus. Is that > >

Re: CI and test improvements

2022-11-19 Thread Andres Freund
Hi, On 2022-11-19 13:18:54 -0800, Andres Freund wrote: > > Also, if CompilerWarnings doesn't depend on Linux, that means those two > > tasks will normally start and run simultaneously, which means a single > > branch will use all 8 of the linux CPUs available from cirrus. Is that > >

Re: CI and test improvements

2022-11-19 Thread Andres Freund
Hi, On 2022-11-19 14:22:20 -0600, Justin Pryzby wrote: > On Sun, Oct 02, 2022 at 01:52:01PM -0700, Andres Freund wrote: > > > > +# To avoid unnecessarily spinning up a lot of VMs / containers for entirely > > +# broken commits, have a very minimal test that all others depend on. > > +task: > >

Re: CI and test improvements

2022-11-19 Thread Justin Pryzby
On Sun, Oct 02, 2022 at 01:52:01PM -0700, Andres Freund wrote: > > +# To avoid unnecessarily spinning up a lot of VMs / containers for entirely > +# broken commits, have a very minimal test that all others depend on. > +task: > + name: SanityCheck Maybe this should be named 00-SanityCheck, so

Re: CI and test improvements

2022-11-16 Thread Justin Pryzby
On Wed, Nov 16, 2022 at 08:08:32PM -0800, Andres Freund wrote: > I also don't like my "cores_script". Not quite sure yet how to do that > more cleanly. I don't know which is cleaner: ls /core* && mv /tmp/core* /tmp/cores find / -maxdepth 1 -type f -name 'core*' -print0 | xargs -r0 mv

Re: CI and test improvements

2022-11-16 Thread Andres Freund
Hi, On 2022-11-16 21:58:39 -0600, Justin Pryzby wrote: > On Wed, Nov 16, 2022 at 07:48:14PM -0800, Andres Freund wrote: > > I've used this a bunch on personal branches, and I think it's the way to > > go. It doesn't take long, saves a lot of cycles when one pushes something > > broken. Starts to

Re: CI and test improvements

2022-11-16 Thread Justin Pryzby
On Wed, Nov 16, 2022 at 07:48:14PM -0800, Andres Freund wrote: > I've used this a bunch on personal branches, and I think it's the way to > go. It doesn't take long, saves a lot of cycles when one pushes something > broken. Starts to runs the CompilerWarnings task after a minimal amount of >

Re: CI and test improvements

2022-11-16 Thread Andres Freund
Hi, On 2022-10-02 14:54:21 -0700, Andres Freund wrote: > On 2022-10-02 16:35:06 -0500, Justin Pryzby wrote: > > On Sun, Oct 02, 2022 at 01:52:01PM -0700, Andres Freund wrote: > > > On 2022-10-01 18:36:41 -0700, Andres Freund wrote: > > > > I am wondering if we should instead introduce a new

Re: CI and test improvements

2022-11-13 Thread Justin Pryzby
On Fri, Nov 04, 2022 at 06:59:46PM -0700, Andres Freund wrote: > Hi, > > On 2022-11-04 18:54:12 -0500, Justin Pryzby wrote: > > Subject: [PATCH 1/8] meson: PROVE is not required > > Subject: [PATCH 3/8] meson: rename 'main' tasks to 'regress' and 'isolation' > > Pushed, thanks for the patches.

Re: CI and test improvements

2022-11-04 Thread Andres Freund
Hi, On 2022-11-04 18:54:12 -0500, Justin Pryzby wrote: > Subject: [PATCH 1/8] meson: PROVE is not required > Subject: [PATCH 3/8] meson: rename 'main' tasks to 'regress' and 'isolation' Pushed, thanks for the patches. > Subject: [PATCH 2/8] meson: other fixes for cygwin > > XXX: what about

Re: CI and test improvements

2022-11-04 Thread Justin Pryzby
On Sat, Sep 10, 2022 at 03:05:42PM -0500, Justin Pryzby wrote: > On Sun, Aug 28, 2022 at 02:28:02PM -0700, Andres Freund wrote: > > On 2022-08-28 12:10:29 -0500, Justin Pryzby wrote: > > > On Sun, Aug 28, 2022 at 09:07:52AM -0700, Andres Freund wrote: > > > > > --- /dev/null > > > > > +++

Re: CI and test improvements

2022-10-02 Thread Justin Pryzby
On Sun, Oct 02, 2022 at 02:54:21PM -0700, Andres Freund wrote: > the clang-only memory sanitizer there (if it works on freebsd)... Have you looked at this much ? I think it'll require a bunch of exclusions, right ? -- Justin

Re: CI and test improvements

2022-10-02 Thread Andres Freund
Hi, On 2022-10-02 16:35:06 -0500, Justin Pryzby wrote: > On Sun, Oct 02, 2022 at 01:52:01PM -0700, Andres Freund wrote: > > On 2022-10-01 18:36:41 -0700, Andres Freund wrote: > > > I am wondering if we should instead introduce a new "quickcheck" task that > > > just compiles and runs maybe one

Re: CI and test improvements

2022-10-02 Thread Andres Freund
Hi, On 2022-10-02 16:35:06 -0500, Justin Pryzby wrote: > Maybe - that would avoid waiting 4 minutes for a windows instance to > start in the (hopefully atypical) case of a patch that fails in 1-2 > minutes under linux/freebsd. > > If the patch were completely broken, the windows task would take

Re: CI and test improvements

2022-10-02 Thread Justin Pryzby
On Sun, Oct 02, 2022 at 01:52:01PM -0700, Andres Freund wrote: > Hi, > > On 2022-10-01 18:36:41 -0700, Andres Freund wrote: > > I am wondering if we should instead introduce a new "quickcheck" task that > > just compiles and runs maybe one test and have *all* other tests depend on > > that.

Re: CI and test improvements

2022-10-02 Thread Andres Freund
Hi, On 2022-10-01 18:36:41 -0700, Andres Freund wrote: > I am wondering if we should instead introduce a new "quickcheck" task that > just compiles and runs maybe one test and have *all* other tests depend on > that. Wasting a precious available windows instance to just fail to build or >

Re: CI and test improvements

2022-10-01 Thread Andres Freund
Hi, On 2022-10-01 19:58:01 -0500, Justin Pryzby wrote: > One other thing is that your -m32 changes caused the linux/meson task to > take an additional 3+ minutes (total ~8). That's no issue, except that > the Warnings task depends on the linux/mason task, and itself can take > up to 15 minutes.

Re: CI and test improvements

2022-10-01 Thread Justin Pryzby
On Sat, Oct 01, 2022 at 05:45:01PM -0700, Andres Freund wrote: > Hi, > > On 2022-09-10 15:05:42 -0500, Justin Pryzby wrote: > > From 4ed5eb427de4508a4c3422e60891b45c8512814a Mon Sep 17 00:00:00 2001 > > From: Justin Pryzby > > Date: Sun, 3 Apr 2022 00:10:20 -0500 > > Subject: [PATCH 03/23]

Re: CI and test improvements

2022-10-01 Thread Andres Freund
Hi, On 2022-09-10 15:05:42 -0500, Justin Pryzby wrote: > From 4ed5eb427de4508a4c3422e60891b45c8512814a Mon Sep 17 00:00:00 2001 > From: Justin Pryzby > Date: Sun, 3 Apr 2022 00:10:20 -0500 > Subject: [PATCH 03/23] cirrus/ccache: disable compression and show stats > > Since v4.0, ccache enables

Re: CI and test improvements

2022-09-22 Thread Thomas Munro
On Fri, Sep 23, 2022 at 9:07 AM Justin Pryzby wrote: > On Sun, Aug 28, 2022 at 02:28:02PM -0700, Andres Freund wrote: > > > > > - # Workaround around performance issues due to 32KB block size > > > > > - repartition_script: src/tools/ci/gcp_freebsd_repartition.sh > > > > >

Re: CI and test improvements

2022-09-22 Thread Andres Freund
Hi, On 2022-09-22 16:07:02 -0500, Justin Pryzby wrote: > On Sun, Aug 28, 2022 at 02:28:02PM -0700, Andres Freund wrote: > > > > > @@ -71,8 +69,6 @@ task: > > > > > fingerprint_key: ccache/freebsd > > > > > reupload_on_changes: true > > > > > > > > > > - # Workaround around performance

Re: CI and test improvements

2022-09-22 Thread Justin Pryzby
On Sun, Aug 28, 2022 at 02:28:02PM -0700, Andres Freund wrote: > > > > @@ -71,8 +69,6 @@ task: > > > > fingerprint_key: ccache/freebsd > > > > reupload_on_changes: true > > > > > > > > - # Workaround around performance issues due to 32KB block size > > > > - repartition_script:

Re: CI and test improvements

2022-09-10 Thread Justin Pryzby
On Sun, Aug 28, 2022 at 02:28:02PM -0700, Andres Freund wrote: > On 2022-08-28 12:10:29 -0500, Justin Pryzby wrote: > > On Sun, Aug 28, 2022 at 09:07:52AM -0700, Andres Freund wrote: > > > > --- /dev/null > > > > +++ b/src/tools/ci/windows-compiler-warnings > > > > > > Wouldn't that be doable as

Re: CI and test improvements

2022-08-28 Thread Andres Freund
Hi, On 2022-08-28 12:10:29 -0500, Justin Pryzby wrote: > On Sun, Aug 28, 2022 at 09:07:52AM -0700, Andres Freund wrote: > > > --- /dev/null > > > +++ b/src/tools/ci/windows-compiler-warnings > > > > Wouldn't that be doable as something like > > sh -c 'if test -s file; then cat file;exit 1; fi" >

Re: CI and test improvements

2022-08-28 Thread Justin Pryzby
On Sun, Aug 28, 2022 at 09:07:52AM -0700, Andres Freund wrote: > > --- /dev/null > > +++ b/src/tools/ci/windows-compiler-warnings > > Wouldn't that be doable as something like > sh -c 'if test -s file; then cat file;exit 1; fi" > inside .cirrus.yml? I had written it inline in a couple ways, like

Re: CI and test improvements

2022-08-28 Thread Andres Freund
Hi, On 2022-08-28 09:44:47 -0500, Justin Pryzby wrote: > On Sat, May 28, 2022 at 10:37:41AM -0500, Justin Pryzby wrote: > > I'm anticipating the need to further re-arrange the patch set - it's not > > clear > > which patches should go first. Maybe some patches should be dropped in > > favour >

Re: CI and test improvements

2022-08-28 Thread Justin Pryzby
Resending with a problematic address removed... On Sat, May 28, 2022 at 10:37:41AM -0500, Justin Pryzby wrote: > I'm anticipating the need to further re-arrange the patch set - it's not clear > which patches should go first. Maybe some patches should be dropped in favour > of the meson project.

Re: CI and test improvements

2022-07-07 Thread Justin Pryzby
rebased on b6a5158f9 and 054325c5e Also, cirrus/freebsd task can run 3x faster with more CPUs. Subject: [PATCH 21/21] cirrus: run freebsd with more CPUs+RAM

Re: CI and test improvements

2022-06-23 Thread Thomas Munro
[Resending -- sorry if you receive this twice. Jacob's mail server has apparently offended the list management software so emails bounce if he's in CC.] On Fri, Jun 24, 2022 at 7:23 AM Justin Pryzby wrote: > Freebsd uploads an artifact 3x smaller than the size ccache reports, because > its

Re: CI and test improvements

2022-06-23 Thread Justin Pryzby
On Sat, May 28, 2022 at 10:37:41AM -0500, Justin Pryzby wrote: > I'm "joining" a bunch of unresolved threads hoping to present them better > since > they're related and I'm maintaining them together anyway. This resolves an error with libpq tests in an intermediate commit, probably caused by

CI and test improvements

2022-05-28 Thread Justin Pryzby
I'm "joining" a bunch of unresolved threads hoping to present them better since they're related and I'm maintaining them together anyway. https://www.postgresql.org/message-id/flat/20220219234148.GC9008%40telsasoft.com - set TESTDIR from perl rather than Makefile