Re: pgsql: scripts: add Perl script to add links to release notes

2024-09-17 Thread Andrew Dunstan


On 2024-09-16 Mo 1:27 PM, Bruce Momjian wrote:

scripts:  add Perl script to add links to release notes

Reported-by: jian he

Discussion:https://postgr.es/m/zuyss5xda7hvc...@momjian.us



I realize I'm late to the party on this, but here's a review patch for 
this script. It's mostly stylistic changes (simpler use of print, use of 
POSIX xdigit class for hex digits), but it does correct one actual 
error: AFAIK "die" doesn't do printf style substitutions.



cheers


andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com
diff --git a/src/tools/add_commit_links.pl b/src/tools/add_commit_links.pl
index ebfc97ea32..64a5783297 100755
--- a/src/tools/add_commit_links.pl
+++ b/src/tools/add_commit_links.pl
@@ -51,9 +51,8 @@ sub process_file
 	$file =~ m/-(\d+)\./;
 	my $major_version = $1;
 
-	open(my $fh, '<', $file) || die "could not open file %s: $!\n", $file;
-	open(my $tfh, '>', $tmpfile) || die "could not open file %s: $!\n",
-	  $tmpfile;
+	open(my $fh, '<', $file) || die "could not open file $file: $!\n";
+	open(my $tfh, '>', $tmpfile) || die "could not open file $tmpfile: $!\n";
 
 	while (<$fh>)
 	{
@@ -64,9 +63,9 @@ sub process_file
 		# skip over commit links because we will add them below
 		next
 		  if (!$in_comment &&
-			m{^\s*§\s*$});
+			m{^\s*§\s*$});
 
-		if ($in_comment && m/\[([\da-f]+)\]/)
+		if ($in_comment && m/\[([[:xdigit:]]+)\]/)
 		{
 			my $hash = $1;
 
@@ -88,23 +87,21 @@ sub process_file
 {
 	for my $hash (@hashes)
 	{
-		print({$tfh}
-			  "$prev_leading_space§\n"
-		);
+		print $tfh
+		  "$prev_leading_space§\n";
 	}
 	@hashes = ();
 }
 else
 {
-	printf(
-		"hashes found but no matching text found for placement on line %s\n",
-		$lineno);
+	print
+	  "hashes found but no matching text found for placement on line $lineno\n";
 	exit(1);
 }
 			}
 		}
 
-		print({$tfh} $_);
+		print $tfh $_;
 
 		$prev_line_ended_with_paren = m/\)\s*$/;
 


Re: pgsql: Perl scripts: eliminate "Useless interpolation" warnings

2024-09-16 Thread Andrew Dunstan


On 2024-09-15 Su 9:28 PM, Bruce Momjian wrote:

On Sun, Sep 15, 2024 at 06:07:21PM -0400, Andrew Dunstan wrote:

On 2024-09-15 Su 4:36 PM, Bruce Momjian wrote:

On Sun, Sep 15, 2024 at 04:33:49PM -0400, Andrew Dunstan wrote:

I understand perfectly what the warning is about.

But the project's perlcritic policy is expressed at src/tools/perlcheck/
perlcriticrc. It's basically severity 5 plus some additions and one exception.
We shouldn't be imposing our personal perlcritic policies on the project.

The change isn't bad in itself, but there shouldn't be any expectation that we
will keep to this policy, as it's not required by project policy.

There is a huge mass of perlcritic issues in our perl code at severity 1 (over
13000 by my count). If we're going to clean them up (which I would oppose) we
should do it in a systematic way. It's hard to see why this one is special.

I guess it is special only because my policy is more strict so I wanted
my new code to match.  Should I revert it?


Yes I think so.

Okay, done.


Is the very tiny improvement
not worth the churn in the code?


Yeah, I don't think it is.

FYI, the line that got me started was:

my ($tmpfilename) = $filename . ".tmp";

while I would write that with single quotes.  Because mine uses single
quotes and version_stamp.pl uses double-quotes, I started to try to
unify the style.



I would normally write this as

    my $tempfile = "$filename.tmp";

or possibly

    my $tempfile = "${filename}.tmp";

There's a perl mantra that says There Is More Than One Way To Do It, 
usually abbreviated to TIMTOWTDI, which perlcritic fights against to 
some extent. I think the idea of unifying style beyond the official 
project policy is an effort doomed to failure. Nobody is going to 
maintain that consistency, and the project has rejected attempts to have 
a stricter set of rules in the past.





I agree severity=1 generates many false warnings, and changing code
based on them can actually produce errors, but I do think there are some
safe ones like the single/double quote item we can improve.

Attached is my Perl critic file, where I do use severity=1, but turn off
various warnings.



Sure, or for something in between, there's the buildfarm policy at 
<https://github.com/PGBuildFarm/client-code/blob/main/.perlcriticrc>


cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com


Re: pgsql: Perl scripts: eliminate "Useless interpolation" warnings

2024-09-15 Thread Andrew Dunstan



On 2024-09-15 Su 4:36 PM, Bruce Momjian wrote:

On Sun, Sep 15, 2024 at 04:33:49PM -0400, Andrew Dunstan wrote:

I understand perfectly what the warning is about.

But the project's perlcritic policy is expressed at src/tools/perlcheck/
perlcriticrc. It's basically severity 5 plus some additions and one exception.
We shouldn't be imposing our personal perlcritic policies on the project.

The change isn't bad in itself, but there shouldn't be any expectation that we
will keep to this policy, as it's not required by project policy.

There is a huge mass of perlcritic issues in our perl code at severity 1 (over
13000 by my count). If we're going to clean them up (which I would oppose) we
should do it in a systematic way. It's hard to see why this one is special.

I guess it is special only because my policy is more strict so I wanted
my new code to match.  Should I revert it?



Yes I think so.



Is the very tiny improvement
not worth the churn in the code?



Yeah, I don't think it is.


cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: pgsql: Perl scripts: eliminate "Useless interpolation" warnings

2024-09-15 Thread Andrew Dunstan


On 2024-09-15 Su 4:16 PM, Bruce Momjian wrote:

On Sun, Sep 15, 2024 at 02:30:47PM -0400, Andrew Dunstan wrote:


On 2024-09-15 Su 10:56 AM, Bruce Momjian wrote:

 Perl scripts:  eliminate "Useless interpolation" warnings

 Eliminate warnings of Perl Critic from src/tools.

 Backpatch-through: master



I don't understand this commit. The buildfarm members crake and koel regularly
run the perl critic checks and have not complained. See for example from before
this change:<https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm= 
koel&dt=2024-09-15%2003%3A34%3A02&stg=perl-check>


The change doesn't seem to have had any discussion either. This particular
warning is surely a very low level (i.e. fairly unimportant) one, of the type
we normally ignore. If some version of perlcritic has raised it to severity 5
then the correct action IMNSHO would be to add an exception for it to the
perlcriticrc, like we do for ProhibitLeadingZeros. If not, then perhaps you can
explain how you got the warnings.

So, the warning is about the use of double-quotes when single-quotes
will work just fine.  I wrote a new script and changed it to single
quotes, so for consistency, I looked at other Perl scripts that might
have the issue.  The message I got was:

/root/add_commit_links.pl: Useless interpolation of literal string at line 51 near 
'my $tmpfile = $file . ".tmp";'.  (Severity: 1)

My $HOME/.perlcritic has:

severity = 1

and that is why I saw it.  Is it a bad change?



I understand perfectly what the warning is about.

But the project's perlcritic policy is expressed at 
src/tools/perlcheck/perlcriticrc. It's basically severity 5 plus some 
additions and one exception. We shouldn't be imposing our personal 
perlcritic policies on the project.


The change isn't bad in itself, but there shouldn't be any expectation 
that we will keep to this policy, as it's not required by project policy.


There is a huge mass of perlcritic issues in our perl code at severity 1 
(over 13000 by my count). If we're going to clean them up (which I would 
oppose) we should do it in a systematic way. It's hard to see why this 
one is special.


cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com


Re: pgsql: Perl scripts: eliminate "Useless interpolation" warnings

2024-09-15 Thread Andrew Dunstan



On 2024-09-15 Su 10:56 AM, Bruce Momjian wrote:

Perl scripts:  eliminate "Useless interpolation" warnings

Eliminate warnings of Perl Critic from src/tools.

Backpatch-through: master




I don't understand this commit. The buildfarm members crake and koel 
regularly run the perl critic checks and have not complained. See for 
example from before this change: 
<https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=koel&dt=2024-09-15%2003%3A34%3A02&stg=perl-check>


The change doesn't seem to have had any discussion either. This 
particular warning is surely a very low level (i.e. fairly unimportant) 
one, of the type we normally ignore. If some version of perlcritic has 
raised it to severity 5 then the correct action IMNSHO would be to add 
an exception for it to the perlcriticrc, like we do for 
ProhibitLeadingZeros. If not, then perhaps you can explain how you got 
the warnings.



cheers


andrew



--
Andrew Dunstan
EDB:https://www.enterprisedb.com


pgsql: Improve meson's detection of perl build flags

2024-09-14 Thread Andrew Dunstan
Improve meson's detection of perl build flags

The current method of detecting perl build flags breaks if the path to
perl contains a space. This change makes two improvements. First,
instead of getting a list of ldflags and ccdlflags and then trying to
filter those out of the reported ldopts, we tell perl to suppress
reporting those in the first instance. Second, it tells perl to parse
those and output them, one per line. Thus any space on the option in a
file name, for example, is preserved.

Issue reported off-list by Muralikrishna Bandaru

Discussion: https://postgr.es/01117f88-f465-bf6c-9362-083bd72ca...@dunslane.net

Backpatch to release 16.

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/dc2a660bd983f2675f357d279253256a2aef9836

Modified Files
--
meson.build | 25 -
1 file changed, 12 insertions(+), 13 deletions(-)



pgsql: Improve meson's detection of perl build flags

2024-09-14 Thread Andrew Dunstan
Improve meson's detection of perl build flags

The current method of detecting perl build flags breaks if the path to
perl contains a space. This change makes two improvements. First,
instead of getting a list of ldflags and ccdlflags and then trying to
filter those out of the reported ldopts, we tell perl to suppress
reporting those in the first instance. Second, it tells perl to parse
those and output them, one per line. Thus any space on the option in a
file name, for example, is preserved.

Issue reported off-list by Muralikrishna Bandaru

Discussion: https://postgr.es/01117f88-f465-bf6c-9362-083bd72ca...@dunslane.net

Backpatch to release 16.

Branch
--
REL_16_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/cb52d1cdd1031d3b0d66a001b7b153990a0497c8

Modified Files
--
meson.build | 25 -
1 file changed, 12 insertions(+), 13 deletions(-)



pgsql: Improve meson's detection of perl build flags

2024-09-14 Thread Andrew Dunstan
Improve meson's detection of perl build flags

The current method of detecting perl build flags breaks if the path to
perl contains a space. This change makes two improvements. First,
instead of getting a list of ldflags and ccdlflags and then trying to
filter those out of the reported ldopts, we tell perl to suppress
reporting those in the first instance. Second, it tells perl to parse
those and output them, one per line. Thus any space on the option in a
file name, for example, is preserved.

Issue reported off-list by Muralikrishna Bandaru

Discussion: https://postgr.es/01117f88-f465-bf6c-9362-083bd72ca...@dunslane.net

Backpatch to release 16.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/76f2a0e5479618d48161549a148a37251b4b3d4b

Modified Files
--
meson.build | 25 -
1 file changed, 12 insertions(+), 13 deletions(-)



pgsql: Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

2024-09-14 Thread Andrew Dunstan
Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

Latest versions of Strawberry Perl define USE_THREAD_SAFE_LOCALE, and we
therefore get a handshake error when building against such instances.
The solution is to perform a test to see if USE_THREAD_SAFE_LOCALE is
defined and only define NO_THREAD_SAFE_LOCALE if it isn't.

Backpatch the meson.build fix back to release 16 and apply the same
logic to Mkvcbuild.pm in releases 12 through 16.

Original report of the issue from Muralikrishna Bandaru.

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/9f77494647ded10c5ffd67cea92abcfdaea7

Modified Files
--
src/tools/msvc/Mkvcbuild.pm | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)



pgsql: Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

2024-09-14 Thread Andrew Dunstan
Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

Latest versions of Strawberry Perl define USE_THREAD_SAFE_LOCALE, and we
therefore get a handshake error when building against such instances.
The solution is to perform a test to see if USE_THREAD_SAFE_LOCALE is
defined and only define NO_THREAD_SAFE_LOCALE if it isn't.

Backpatch the meson.build fix back to release 16 and apply the same
logic to Mkvcbuild.pm in releases 12 through 16.

Original report of the issue from Muralikrishna Bandaru.

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/d94e3b33eaf6c57109a21115572ac78124da5939

Modified Files
--
src/tools/msvc/Mkvcbuild.pm | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)



pgsql: Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

2024-09-14 Thread Andrew Dunstan
Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

Latest versions of Strawberry Perl define USE_THREAD_SAFE_LOCALE, and we
therefore get a handshake error when building against such instances.
The solution is to perform a test to see if USE_THREAD_SAFE_LOCALE is
defined and only define NO_THREAD_SAFE_LOCALE if it isn't.

Backpatch the meson.build fix back to release 16 and apply the same
logic to Mkvcbuild.pm in releases 12 through 16.

Original report of the issue from Muralikrishna Bandaru.

Branch
--
REL_15_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/17c35ab236980fce2989f7fac7cee42ca4d5ca04

Modified Files
--
src/tools/msvc/Mkvcbuild.pm | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)



pgsql: Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

2024-09-14 Thread Andrew Dunstan
Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

Latest versions of Strawberry Perl define USE_THREAD_SAFE_LOCALE, and we
therefore get a handshake error when building against such instances.
The solution is to perform a test to see if USE_THREAD_SAFE_LOCALE is
defined and only define NO_THREAD_SAFE_LOCALE if it isn't.

Backpatch the meson.build fix back to release 16 and apply the same
logic to Mkvcbuild.pm in releases 12 through 16.

Original report of the issue from Muralikrishna Bandaru.

Branch
--
REL_13_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/f40d9e9f1b5892a340d5ed5d650c9dc3cf72e6e9

Modified Files
--
src/tools/msvc/Mkvcbuild.pm | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)



pgsql: Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

2024-09-14 Thread Andrew Dunstan
Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

Latest versions of Strawberry Perl define USE_THREAD_SAFE_LOCALE, and we
therefore get a handshake error when building against such instances.
The solution is to perform a test to see if USE_THREAD_SAFE_LOCALE is
defined and only define NO_THREAD_SAFE_LOCALE if it isn't.

Backpatch the meson.build fix back to release 16 and apply the same
logic to Mkvcbuild.pm in releases 12 through 16.

Original report of the issue from Muralikrishna Bandaru.

Branch
--
REL_16_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/0a0db46313749bb379db65eb987af6bf29470300

Modified Files
--
meson.build | 5 -
src/tools/msvc/Mkvcbuild.pm | 4 +++-
2 files changed, 7 insertions(+), 2 deletions(-)



pgsql: Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

2024-09-14 Thread Andrew Dunstan
Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

Latest versions of Strawberry Perl define USE_THREAD_SAFE_LOCALE, and we
therefore get a handshake error when building against such instances.
The solution is to perform a test to see if USE_THREAD_SAFE_LOCALE is
defined and only define NO_THREAD_SAFE_LOCALE if it isn't.

Backpatch the meson.build fix back to release 16 and apply the same
logic to Mkvcbuild.pm in releases 12 through 16.

Original report of the issue from Muralikrishna Bandaru.

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/648397b1d409f15612b5bb6f95c5527f94e69807

Modified Files
--
meson.build | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)



pgsql: Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

2024-09-14 Thread Andrew Dunstan
Only define NO_THREAD_SAFE_LOCALE for MSVC plperl when required

Latest versions of Strawberry Perl define USE_THREAD_SAFE_LOCALE, and we
therefore get a handshake error when building against such instances.
The solution is to perform a test to see if USE_THREAD_SAFE_LOCALE is
defined and only define NO_THREAD_SAFE_LOCALE if it isn't.

Backpatch the meson.build fix back to release 16 and apply the same
logic to Mkvcbuild.pm in releases 12 through 16.

Original report of the issue from Muralikrishna Bandaru.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/bc46104fc9aa5254f98250cf2756552f92095ae9

Modified Files
--
meson.build | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)



pgsql: Preserve tz when converting to jsonb timestamptz

2024-07-30 Thread Andrew Dunstan
Preserve tz when converting to jsonb timestamptz

This removes an inconsistency in the treatment of different datatypes by
the jsonpath timestamp_tz() function. Conversions from data types that
are not timestamp-aware, such as date and timestamp, are now treated
consistently with conversion from those that are such as timestamptz.

Author: David Wheeler
Reviewed-by: Junwang Zhao and Jeevan Chalke

Discussion: 
https://postgr.es/m/7DE080CE-6D8C-4794-9BD1-7D9699172FAB%40justatheory.com

Backpatch to release 17.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/524d490a9f4ab81d86bbedc6e429fbc27776351c

Modified Files
--
src/backend/utils/adt/jsonpath_exec.c| 25 +
src/test/regress/expected/jsonb_jsonpath.out |  4 ++--
2 files changed, 27 insertions(+), 2 deletions(-)



pgsql: Preserve tz when converting to jsonb timestamptz

2024-07-30 Thread Andrew Dunstan
Preserve tz when converting to jsonb timestamptz

This removes an inconsistency in the treatment of different datatypes by
the jsonpath timestamp_tz() function. Conversions from data types that
are not timestamp-aware, such as date and timestamp, are now treated
consistently with conversion from those that are such as timestamptz.

Author: David Wheeler
Reviewed-by: Junwang Zhao and Jeevan Chalke

Discussion: 
https://postgr.es/m/7DE080CE-6D8C-4794-9BD1-7D9699172FAB%40justatheory.com

Backpatch to release 17.

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/0d57dc2f9137de47534cc4db115e182e4093b945

Modified Files
--
src/backend/utils/adt/jsonpath_exec.c| 25 +
src/test/regress/expected/jsonb_jsonpath.out |  4 ++--
2 files changed, 27 insertions(+), 2 deletions(-)



pgsql: Stabilize xid_wraparound tests

2024-07-30 Thread Andrew Dunstan
Stabilize xid_wraparound tests

The tests had a race condition if autovacuum was set to off. Instead we
create all the tables we are interested in with autovacuum disabled, so
they are only ever touched when in danger of wraparound.

Discussion: 
https://postgr.es/m/3e2cbd24-f45e-4b2b-ba83-8149214f0...@dunslane.net

Masahiko Sawada (slightly tweaked by me)

Backpatch to release 17 where these tests were introduced.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/800cd3e923597172a29aba49da45753f52996ee8

Modified Files
--
.../xid_wraparound/t/001_emergency_vacuum.pl   | 28 ++
src/test/modules/xid_wraparound/t/002_limits.pl|  6 ++---
.../modules/xid_wraparound/t/003_wraparounds.pl|  6 ++---
3 files changed, 19 insertions(+), 21 deletions(-)



pgsql: Stabilize xid_wraparound tests

2024-07-30 Thread Andrew Dunstan
Stabilize xid_wraparound tests

The tests had a race condition if autovacuum was set to off. Instead we
create all the tables we are interested in with autovacuum disabled, so
they are only ever touched when in danger of wraparound.

Discussion: 
https://postgr.es/m/3e2cbd24-f45e-4b2b-ba83-8149214f0...@dunslane.net

Masahiko Sawada (slightly tweaked by me)

Backpatch to release 17 where these tests were introduced.

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/787ea8c9d812e5ae76b7a8981e5f93cee80c8a36

Modified Files
--
.../xid_wraparound/t/001_emergency_vacuum.pl   | 28 ++
src/test/modules/xid_wraparound/t/002_limits.pl|  6 ++---
.../modules/xid_wraparound/t/003_wraparounds.pl|  6 ++---
3 files changed, 19 insertions(+), 21 deletions(-)



pgsql: Avoid error in recovery test if history file is not yet present

2024-07-17 Thread Andrew Dunstan
Avoid error in recovery test if history file is not yet present

Error was detected when testing use of libpq sessions instead of psql
for polling queries.

Discussion: 
https://postgr.es/m/e86b6d2d-20d8-4ac9-9a98-165fff7db...@dunslane.net

Backpatch to all live branches

Branch
--
REL_13_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/5ee78e303821150658453c7671f124c44ec66b48

Modified Files
--
src/test/recovery/t/002_archiving.pl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



pgsql: Avoid error in recovery test if history file is not yet present

2024-07-17 Thread Andrew Dunstan
Avoid error in recovery test if history file is not yet present

Error was detected when testing use of libpq sessions instead of psql
for polling queries.

Discussion: 
https://postgr.es/m/e86b6d2d-20d8-4ac9-9a98-165fff7db...@dunslane.net

Backpatch to all live branches

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/f2a0d5808c246e556efbe3df0fb2be7841e3c988

Modified Files
--
src/test/recovery/t/002_archiving.pl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



pgsql: Avoid error in recovery test if history file is not yet present

2024-07-17 Thread Andrew Dunstan
Avoid error in recovery test if history file is not yet present

Error was detected when testing use of libpq sessions instead of psql
for polling queries.

Discussion: 
https://postgr.es/m/e86b6d2d-20d8-4ac9-9a98-165fff7db...@dunslane.net

Backpatch to all live branches

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/b06fe880da1423063104ac42214b04245c3277a4

Modified Files
--
src/test/recovery/t/002_archiving.pl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



pgsql: Avoid error in recovery test if history file is not yet present

2024-07-17 Thread Andrew Dunstan
Avoid error in recovery test if history file is not yet present

Error was detected when testing use of libpq sessions instead of psql
for polling queries.

Discussion: 
https://postgr.es/m/e86b6d2d-20d8-4ac9-9a98-165fff7db...@dunslane.net

Backpatch to all live branches

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/5e578acca54b47c0fe195458f3fe57a5a42f4f01

Modified Files
--
src/test/recovery/t/002_archiving.pl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



pgsql: Avoid error in recovery test if history file is not yet present

2024-07-17 Thread Andrew Dunstan
Avoid error in recovery test if history file is not yet present

Error was detected when testing use of libpq sessions instead of psql
for polling queries.

Discussion: 
https://postgr.es/m/e86b6d2d-20d8-4ac9-9a98-165fff7db...@dunslane.net

Backpatch to all live branches

Branch
--
REL_15_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/bc9d706a19629a03180887bfa061a9a0ec6f5d7e

Modified Files
--
src/test/recovery/t/002_archiving.pl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



pgsql: Avoid error in recovery test if history file is not yet present

2024-07-17 Thread Andrew Dunstan
Avoid error in recovery test if history file is not yet present

Error was detected when testing use of libpq sessions instead of psql
for polling queries.

Discussion: 
https://postgr.es/m/e86b6d2d-20d8-4ac9-9a98-165fff7db...@dunslane.net

Backpatch to all live branches

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/5d797c896fcf0ef04e4fc3354bc94512e8886cb2

Modified Files
--
src/test/recovery/t/002_archiving.pl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



pgsql: Avoid error in recovery test if history file is not yet present

2024-07-17 Thread Andrew Dunstan
Avoid error in recovery test if history file is not yet present

Error was detected when testing use of libpq sessions instead of psql
for polling queries.

Discussion: 
https://postgr.es/m/e86b6d2d-20d8-4ac9-9a98-165fff7db...@dunslane.net

Backpatch to all live branches

Branch
--
REL_16_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/071ed76a06cfc42107bb3126cb6075c962879569

Modified Files
--
src/test/recovery/t/002_archiving.pl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



pgsql: Adjust recently added test for pg_signal_autovacuum role

2024-07-16 Thread Andrew Dunstan
Adjust recently added test for pg_signal_autovacuum role

This test was added by commit d2b74882ca, but fails if
log_error_verbosity is set to verbose. Adjust the regex that checks the
error message to allow for it containing an SQL status code.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/49546ae9c74f02dc2a0d6b5469ba0bb025ae25c2

Modified Files
--
src/test/modules/test_misc/t/006_signal_autovacuum.pl | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



pgsql: Use correct collate.windows.win1252.out

2024-07-13 Thread Andrew Dunstan
Use correct collate.windows.win1252.out

I inadvertently missed backporting this to Release 17 from commit 291c420747

per offlist reminder from Alexander Lakhin.

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/5ea9f66616801d0b4be6ce49c74e45bb4f325359

Modified Files
--
.../regress/expected/collate.windows.win1252.out   | 194 ++---
1 file changed, 97 insertions(+), 97 deletions(-)



pgsql: Make sure to run pg_isready on correct port

2024-07-13 Thread Andrew Dunstan
Make sure to run pg_isready on correct port

The current code can have pg_isready unexpectedly succeed if there is a
server running on the default port. To avoid this we delay running the
test until after a node has been created but before it starts, and then
use that node's port, so we are fairly sure there is nothing running on
the port.

Backpatch to all live branches.

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/c5b15c417140b2b18b590d201b2fdb7b587214fd

Modified Files
--
src/bin/scripts/t/080_pg_isready.pl | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)



pgsql: Make sure to run pg_isready on correct port

2024-07-13 Thread Andrew Dunstan
Make sure to run pg_isready on correct port

The current code can have pg_isready unexpectedly succeed if there is a
server running on the default port. To avoid this we delay running the
test until after a node has been created but before it starts, and then
use that node's port, so we are fairly sure there is nothing running on
the port.

Backpatch to all live branches.

Branch
--
REL_15_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/6f55b4f112ab72ad392b0acfd5bf9b827e9bb458

Modified Files
--
src/bin/scripts/t/080_pg_isready.pl | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)



pgsql: Make sure to run pg_isready on correct port

2024-07-13 Thread Andrew Dunstan
Make sure to run pg_isready on correct port

The current code can have pg_isready unexpectedly succeed if there is a
server running on the default port. To avoid this we delay running the
test until after a node has been created but before it starts, and then
use that node's port, so we are fairly sure there is nothing running on
the port.

Backpatch to all live branches.

Branch
--
REL_16_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/75f4ae2b8065ca7d29bec55f261828f794fbba1e

Modified Files
--
src/bin/scripts/t/080_pg_isready.pl | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)



pgsql: Make sure to run pg_isready on correct port

2024-07-13 Thread Andrew Dunstan
Make sure to run pg_isready on correct port

The current code can have pg_isready unexpectedly succeed if there is a
server running on the default port. To avoid this we delay running the
test until after a node has been created but before it starts, and then
use that node's port, so we are fairly sure there is nothing running on
the port.

Backpatch to all live branches.

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/dd12eb33aff94dabee03d32a98d80dfa434805a6

Modified Files
--
src/bin/scripts/t/080_pg_isready.pl | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)



pgsql: Make sure to run pg_isready on correct port

2024-07-13 Thread Andrew Dunstan
Make sure to run pg_isready on correct port

The current code can have pg_isready unexpectedly succeed if there is a
server running on the default port. To avoid this we delay running the
test until after a node has been created but before it starts, and then
use that node's port, so we are fairly sure there is nothing running on
the port.

Backpatch to all live branches.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/055891f374a347aedd53c5b4fc79caf0a5522e18

Modified Files
--
src/bin/scripts/t/080_pg_isready.pl | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)



pgsql: Make sure to run pg_isready on correct port

2024-07-13 Thread Andrew Dunstan
Make sure to run pg_isready on correct port

The current code can have pg_isready unexpectedly succeed if there is a
server running on the default port. To avoid this we delay running the
test until after a node has been created but before it starts, and then
use that node's port, so we are fairly sure there is nothing running on
the port.

Backpatch to all live branches.

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/53a14ec29024e8060755e462c384bc6c323dee14

Modified Files
--
src/bin/scripts/t/080_pg_isready.pl | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)



pgsql: Make sure to run pg_isready on correct port

2024-07-13 Thread Andrew Dunstan
Make sure to run pg_isready on correct port

The current code can have pg_isready unexpectedly succeed if there is a
server running on the default port. To avoid this we delay running the
test until after a node has been created but before it starts, and then
use that node's port, so we are fairly sure there is nothing running on
the port.

Backpatch to all live branches.

Branch
--
REL_13_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/2423c84d098ab47ae540bc88a29bcb6fb3812e60

Modified Files
--
src/bin/scripts/t/080_pg_isready.pl | 5 +++--
1 file changed, 3 insertions(+), 2 deletions(-)



pgsql: Use diff --strip-trailing-cr in pg_regress.c

2024-07-12 Thread Andrew Dunstan
Use diff --strip-trailing-cr in pg_regress.c

This was reverted in commit c194de0713. However with a correct
collate.windows.win1252.out we can now re-enable it.

Discussion: 
https://postgr.es/m/can55fz1objlz3vn5afu4ojnesmqpxjxkcp2q18yrkf4ekml...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/291c420747211ffa04678434b43eac26b353f470

Modified Files
--
.../regress/expected/collate.windows.win1252.out   | 194 ++---
src/test/regress/pg_regress.c  |   4 +-
2 files changed, 99 insertions(+), 99 deletions(-)



pgsql: Change pg_regress.c back to using diff -w on Windows

2024-07-11 Thread Andrew Dunstan
Change pg_regress.c back to using diff -w on Windows

This partially reverts commit 628c1d1f2c.

It appears that there are non line-end differences in some regression
tests on Windows. To keep the buildfarm and CI clients happy, change
this back for now, pending further investigation.

Per reports from Tatsuo Ishii and Nazir Bilal Yavuz.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/c194de0713ebe71aaeeb5ebed4af2390cc1b521c

Modified Files
--
src/test/regress/pg_regress.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)



pgsql: Use diff's --strip-trailing-cr flag where appropriate on Windows

2024-07-10 Thread Andrew Dunstan
Use diff's --strip-trailing-cr flag where appropriate on Windows

Test result files might be checked out using Unix or Windows style line
endings, depening on git flags, so on Windows we use the
--strip-trailing-cr flag to tell diff to ignore line endings
differences.

The flag is added to the diff invocation for the test_json_parser module
tests and the pg_bsd_indent tests. in pg_regress.c we replace the
current use of the "-w" flag, which ignore all white space differences,
with this one which only ignores line end differences.

Discussion: 
https://postgr.es/m/20240707052030.r77hbdkid3mwk...@awork3.anarazel.de

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/4506d18a9891c70ce977047e2b4ffc05bef65165

Modified Files
--
src/test/modules/test_json_parser/t/003_test_semantic.pl | 4 +++-
src/test/regress/pg_regress.c| 4 ++--
src/tools/pg_bsd_indent/t/001_pg_bsd_indent.pl   | 6 +-
3 files changed, 10 insertions(+), 4 deletions(-)



pgsql: Use diff's --strip-trailing-cr flag where appropriate on Windows

2024-07-10 Thread Andrew Dunstan
Use diff's --strip-trailing-cr flag where appropriate on Windows

Test result files might be checked out using Unix or Windows style line
endings, depening on git flags, so on Windows we use the
--strip-trailing-cr flag to tell diff to ignore line endings
differences.

The flag is added to the diff invocation for the test_json_parser module
tests and the pg_bsd_indent tests. in pg_regress.c we replace the
current use of the "-w" flag, which ignore all white space differences,
with this one which only ignores line end differences.

Discussion: 
https://postgr.es/m/20240707052030.r77hbdkid3mwk...@awork3.anarazel.de

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/628c1d1f2c82f1983e5248b5dfe53170dc23d90a

Modified Files
--
src/test/modules/test_json_parser/t/003_test_semantic.pl | 4 +++-
src/test/regress/pg_regress.c| 4 ++--
src/tools/pg_bsd_indent/t/001_pg_bsd_indent.pl   | 6 +-
3 files changed, 10 insertions(+), 4 deletions(-)



pgsql: Prevent CRLF conversion of inputs in json_parser test module

2024-07-09 Thread Andrew Dunstan
Prevent CRLF conversion of inputs in json_parser test module

Do this by opening the file in PG_BINARY_R mode. This prevents us from
getting wrong byte count from stat().

Per complaint from Andres Freund

Discussion: 
https://postgr.es/m/20240707052030.r77hbdkid3mwk...@awork3.anarazel.de

Backpatch to rlease 17 where this code was introduced

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/e72f787ea6b287ed624c1f5be71eb13e405b1577

Modified Files
--
src/test/modules/test_json_parser/test_json_parser_incremental.c | 2 +-
src/test/modules/test_json_parser/test_json_parser_perf.c| 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)



pgsql: Prevent CRLF conversion of inputs in json_parser test module

2024-07-09 Thread Andrew Dunstan
Prevent CRLF conversion of inputs in json_parser test module

Do this by opening the file in PG_BINARY_R mode. This prevents us from
getting wrong byte count from stat().

Per complaint from Andres Freund

Discussion: 
https://postgr.es/m/20240707052030.r77hbdkid3mwk...@awork3.anarazel.de

Backpatch to rlease 17 where this code was introduced

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/f7bd0a381de12a5c309f3c19d8c966b77e87cff3

Modified Files
--
src/test/modules/test_json_parser/test_json_parser_incremental.c | 2 +-
src/test/modules/test_json_parser/test_json_parser_perf.c| 2 +-
2 files changed, 2 insertions(+), 2 deletions(-)



pgsql: Unrevert "Force nodes for SSL tests to start in TCP mode"

2024-07-09 Thread Andrew Dunstan
Unrevert "Force nodes for SSL tests to start in TCP mode"

This reverts commit 8b03d743a85fed4930645382a219cc75bdf7ab73.

Release 12 required a little extra sauce to make it work, but this has
been tested now.

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/d1c30fd884b60c6f342e728df046cf2f701e79d7

Modified Files
--
src/test/ssl/t/SSLServer.pm | 7 +++
1 file changed, 7 insertions(+)



pgsql: Revert "Force nodes for SSL tests to start in TCP mode"

2024-07-08 Thread Andrew Dunstan
Revert "Force nodes for SSL tests to start in TCP mode"

This reverts commit 198088dc63de4f89835419969c7b5d1640be3441.

This is causing errors on Release 12, so revert for now to keep the
buildfarm green.

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/8b03d743a85fed4930645382a219cc75bdf7ab73

Modified Files
--
src/test/ssl/t/SSLServer.pm | 7 ---
1 file changed, 7 deletions(-)



pgsql: Symlink pg_replslot robustly on Windows in pg_basebackup test

2024-07-08 Thread Andrew Dunstan
Symlink pg_replslot robustly on Windows in pg_basebackup test

This reverts commit e9f15bc9. Instead of a hacky solution that didn't
work on Windows, we avoid trying to move the directory possibly across
drives, and instead remove it and recreate it in the new location.

Discussion: 
https://postgr.es/m/20240707070243.sb77kp4ubowau...@awork3.anarazel.de

Backpatch to release 14 like the previous patch.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/5193ca8e155ac0140d20361bdafa27744c016252

Modified Files
--
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)



pgsql: Symlink pg_replslot robustly on Windows in pg_basebackup test

2024-07-08 Thread Andrew Dunstan
Symlink pg_replslot robustly on Windows in pg_basebackup test

This reverts commit e9f15bc9. Instead of a hacky solution that didn't
work on Windows, we avoid trying to move the directory possibly across
drives, and instead remove it and recreate it in the new location.

Discussion: 
https://postgr.es/m/20240707070243.sb77kp4ubowau...@awork3.anarazel.de

Backpatch to release 14 like the previous patch.

Branch
--
REL_16_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/4ac08251eb3db0de3924c788d81a776cc6008edd

Modified Files
--
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)



pgsql: Symlink pg_replslot robustly on Windows in pg_basebackup test

2024-07-08 Thread Andrew Dunstan
Symlink pg_replslot robustly on Windows in pg_basebackup test

This reverts commit e9f15bc9. Instead of a hacky solution that didn't
work on Windows, we avoid trying to move the directory possibly across
drives, and instead remove it and recreate it in the new location.

Discussion: 
https://postgr.es/m/20240707070243.sb77kp4ubowau...@awork3.anarazel.de

Backpatch to release 14 like the previous patch.

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/4b719e15beed7ceb49d0bd4047aa08001c7fbcf6

Modified Files
--
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)



pgsql: Symlink pg_replslot robustly on Windows in pg_basebackup test

2024-07-08 Thread Andrew Dunstan
Symlink pg_replslot robustly on Windows in pg_basebackup test

This reverts commit e9f15bc9. Instead of a hacky solution that didn't
work on Windows, we avoid trying to move the directory possibly across
drives, and instead remove it and recreate it in the new location.

Discussion: 
https://postgr.es/m/20240707070243.sb77kp4ubowau...@awork3.anarazel.de

Backpatch to release 14 like the previous patch.

Branch
--
REL_15_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/468b2367d2fe29fe1a53bcba408b472dc8f5051b

Modified Files
--
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)



pgsql: Symlink pg_replslot robustly on Windows in pg_basebackup test

2024-07-08 Thread Andrew Dunstan
Symlink pg_replslot robustly on Windows in pg_basebackup test

This reverts commit e9f15bc9. Instead of a hacky solution that didn't
work on Windows, we avoid trying to move the directory possibly across
drives, and instead remove it and recreate it in the new location.

Discussion: 
https://postgr.es/m/20240707070243.sb77kp4ubowau...@awork3.anarazel.de

Backpatch to release 14 like the previous patch.

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/d3e213ae1359f0bcfa9c446166d187811a7db8bc

Modified Files
--
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)



pgsql: Force nodes for SSL tests to start in TCP mode

2024-07-08 Thread Andrew Dunstan
Force nodes for SSL tests to start in TCP mode

Currently they are started in unix socket mode in ost cases, and then
converted to run in TCP mode. This can result in port collisions, and
there is no virtue in startng in unix socket mode, so start as we will
be going on.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up).

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/e4f4c5424cf9564b934ac6f632c6c90a2edddb77

Modified Files
--
src/test/ssl/t/SSL/Server.pm | 6 ++
1 file changed, 6 insertions(+)



pgsql: Choose ports for test servers less likely to result in conflicts

2024-07-08 Thread Andrew Dunstan
Choose ports for test servers less likely to result in conflicts

If we choose ports in the range typically used for ephemeral ports there
is a danger of encountering a port conflict due to a race condition
between the time we choose the port in a range below that typically used
to allocate ephemeral ports, but higher than the range typically used by
well known services.

Author: Jelte Fenema-Nio, with some editing by me.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up)

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/b7d3be64e4378ad85e180838912c774825b02b8c

Modified Files
--
src/test/perl/PostgresNode.pm | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)



pgsql: Force nodes for SSL tests to start in TCP mode

2024-07-08 Thread Andrew Dunstan
Force nodes for SSL tests to start in TCP mode

Currently they are started in unix socket mode in ost cases, and then
converted to run in TCP mode. This can result in port collisions, and
there is no virtue in startng in unix socket mode, so start as we will
be going on.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up).

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/81749aa39ec32a8dcb008448fa245b24639861f7

Modified Files
--
src/test/ssl/t/SSLServer.pm | 6 ++
1 file changed, 6 insertions(+)



pgsql: Force nodes for SSL tests to start in TCP mode

2024-07-08 Thread Andrew Dunstan
Force nodes for SSL tests to start in TCP mode

Currently they are started in unix socket mode in ost cases, and then
converted to run in TCP mode. This can result in port collisions, and
there is no virtue in startng in unix socket mode, so start as we will
be going on.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up).

Branch
--
REL_15_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/e28850111eebdbfa67cd07887adfe7f7629f3c39

Modified Files
--
src/test/ssl/t/SSL/Server.pm | 6 ++
1 file changed, 6 insertions(+)



pgsql: Force nodes for SSL tests to start in TCP mode

2024-07-08 Thread Andrew Dunstan
Force nodes for SSL tests to start in TCP mode

Currently they are started in unix socket mode in ost cases, and then
converted to run in TCP mode. This can result in port collisions, and
there is no virtue in startng in unix socket mode, so start as we will
be going on.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up).

Branch
--
REL_16_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/85242f9cbce2f9b8c728423a863037aa455e980f

Modified Files
--
src/test/ssl/t/SSL/Server.pm | 6 ++
1 file changed, 6 insertions(+)



pgsql: Choose ports for test servers less likely to result in conflicts

2024-07-08 Thread Andrew Dunstan
Choose ports for test servers less likely to result in conflicts

If we choose ports in the range typically used for ephemeral ports there
is a danger of encountering a port conflict due to a race condition
between the time we choose the port in a range below that typically used
to allocate ephemeral ports, but higher than the range typically used by
well known services.

Author: Jelte Fenema-Nio, with some editing by me.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up)

Branch
--
REL_13_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/c774d31f8493a2443cd9f5f82b6266347997c1d4

Modified Files
--
src/test/perl/PostgresNode.pm | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)



pgsql: Choose ports for test servers less likely to result in conflicts

2024-07-08 Thread Andrew Dunstan
Choose ports for test servers less likely to result in conflicts

If we choose ports in the range typically used for ephemeral ports there
is a danger of encountering a port conflict due to a race condition
between the time we choose the port in a range below that typically used
to allocate ephemeral ports, but higher than the range typically used by
well known services.

Author: Jelte Fenema-Nio, with some editing by me.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up)

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/8398485957b630a6686cb83b67a17c75dc6ec54b

Modified Files
--
src/test/perl/PostgresNode.pm | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)



pgsql: Force nodes for SSL tests to start in TCP mode

2024-07-08 Thread Andrew Dunstan
Force nodes for SSL tests to start in TCP mode

Currently they are started in unix socket mode in ost cases, and then
converted to run in TCP mode. This can result in port collisions, and
there is no virtue in startng in unix socket mode, so start as we will
be going on.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up).

Branch
--
REL_13_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/94c2386878f01404ec9f4a7d5bcbbb21bee6b550

Modified Files
--
src/test/ssl/t/SSLServer.pm | 6 ++
1 file changed, 6 insertions(+)



pgsql: Choose ports for test servers less likely to result in conflicts

2024-07-08 Thread Andrew Dunstan
Choose ports for test servers less likely to result in conflicts

If we choose ports in the range typically used for ephemeral ports there
is a danger of encountering a port conflict due to a race condition
between the time we choose the port in a range below that typically used
to allocate ephemeral ports, but higher than the range typically used by
well known services.

Author: Jelte Fenema-Nio, with some editing by me.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up)

Branch
--
REL_16_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/2e9dfa5e02e63732937fca270bbee77660401b04

Modified Files
--
src/test/perl/PostgreSQL/Test/Cluster.pm | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)



pgsql: Choose ports for test servers less likely to result in conflicts

2024-07-08 Thread Andrew Dunstan
Choose ports for test servers less likely to result in conflicts

If we choose ports in the range typically used for ephemeral ports there
is a danger of encountering a port conflict due to a race condition
between the time we choose the port in a range below that typically used
to allocate ephemeral ports, but higher than the range typically used by
well known services.

Author: Jelte Fenema-Nio, with some editing by me.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up)

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/4b4b931bcdf23f5facd49809278a3048c4fdba1f

Modified Files
--
src/test/perl/PostgreSQL/Test/Cluster.pm | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)



pgsql: Choose ports for test servers less likely to result in conflicts

2024-07-08 Thread Andrew Dunstan
Choose ports for test servers less likely to result in conflicts

If we choose ports in the range typically used for ephemeral ports there
is a danger of encountering a port conflict due to a race condition
between the time we choose the port in a range below that typically used
to allocate ephemeral ports, but higher than the range typically used by
well known services.

Author: Jelte Fenema-Nio, with some editing by me.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up)

Branch
--
REL_15_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/33470429f2bdb967d945b72a5b553fbc6d1d8f96

Modified Files
--
src/test/perl/PostgreSQL/Test/Cluster.pm | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)



pgsql: Force nodes for SSL tests to start in TCP mode

2024-07-08 Thread Andrew Dunstan
Force nodes for SSL tests to start in TCP mode

Currently they are started in unix socket mode in ost cases, and then
converted to run in TCP mode. This can result in port collisions, and
there is no virtue in startng in unix socket mode, so start as we will
be going on.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up).

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/e4754f780f5e8917dbddc1c92036deb259143433

Modified Files
--
src/test/ssl/t/SSL/Server.pm | 6 ++
1 file changed, 6 insertions(+)



pgsql: Choose ports for test servers less likely to result in conflicts

2024-07-08 Thread Andrew Dunstan
Choose ports for test servers less likely to result in conflicts

If we choose ports in the range typically used for ephemeral ports there
is a danger of encountering a port conflict due to a race condition
between the time we choose the port in a range below that typically used
to allocate ephemeral ports, but higher than the range typically used by
well known services.

Author: Jelte Fenema-Nio, with some editing by me.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up)

Branch
--
REL_17_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/e68cf81bfea4561518d6b890f75e7aad70b6fbb1

Modified Files
--
src/test/perl/PostgreSQL/Test/Cluster.pm | 13 +++--
1 file changed, 11 insertions(+), 2 deletions(-)



pgsql: Force nodes for SSL tests to start in TCP mode

2024-07-08 Thread Andrew Dunstan
Force nodes for SSL tests to start in TCP mode

Currently they are started in unix socket mode in ost cases, and then
converted to run in TCP mode. This can result in port collisions, and
there is no virtue in startng in unix socket mode, so start as we will
be going on.

Discussion: https://postgr.es/m/d6ee8761-39d1-0033-1afb-d5a57ee05...@gmail.com

Backpatch to all live branches (12 and up).

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/198088dc63de4f89835419969c7b5d1640be3441

Modified Files
--
src/test/ssl/t/SSLServer.pm | 7 +++
1 file changed, 7 insertions(+)



pgsql: Remove redundant perl version checks

2024-06-26 Thread Andrew Dunstan
Remove redundant perl version checks

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl
scripts, including in plperl's plc_perlboot.pl. Because it made other
changes it wasn't backpatched. However, now this is causing a failure on
back branches when built with bleeding edge perl. Therefore, backpatch
just that part of it which removed those uses, from 15 all the way down
to 9.2, which is the earliest version currently built in the buildfarm.

per report from Alexander Lakhin

Discussion: https://postgr.es/m/4cc2ee93-e03c-8e13-61ed-412e7e6ff...@gmail.com

Branch
--
REL_10_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/320534f8f25bf5aa2de4ad1f561b71219cb4e5ff

Modified Files
--
src/pl/plperl/plc_perlboot.pl | 1 -
src/tools/pgindent/pgindent   | 1 -
2 files changed, 2 deletions(-)



pgsql: Remove redundant perl version checks

2024-06-26 Thread Andrew Dunstan
Remove redundant perl version checks

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl
scripts, including in plperl's plc_perlboot.pl. Because it made other
changes it wasn't backpatched. However, now this is causing a failure on
back branches when built with bleeding edge perl. Therefore, backpatch
just that part of it which removed those uses, from 15 all the way down
to 9.2, which is the earliest version currently built in the buildfarm.

per report from Alexander Lakhin

Discussion: https://postgr.es/m/4cc2ee93-e03c-8e13-61ed-412e7e6ff...@gmail.com

Branch
--
REL_11_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/e1541d518821e88344790c2d046e4f9d770fbf66

Modified Files
--
src/pl/plperl/plc_perlboot.pl | 1 -
src/tools/pgindent/pgindent   | 1 -
2 files changed, 2 deletions(-)



pgsql: Remove redundant perl version checks

2024-06-26 Thread Andrew Dunstan
Remove redundant perl version checks

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl
scripts, including in plperl's plc_perlboot.pl. Because it made other
changes it wasn't backpatched. However, now this is causing a failure on
back branches when built with bleeding edge perl. Therefore, backpatch
just that part of it which removed those uses, from 15 all the way down
to 9.2, which is the earliest version currently built in the buildfarm.

per report from Alexander Lakhin

Discussion: https://postgr.es/m/4cc2ee93-e03c-8e13-61ed-412e7e6ff...@gmail.com

Branch
--
REL9_6_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/12c8faaa7b59b99d56ddb730480645725bf4fa76

Modified Files
--
src/pl/plperl/plc_perlboot.pl | 1 -
src/tools/pgindent/pgindent   | 1 -
2 files changed, 2 deletions(-)



pgsql: Remove redundant perl version checks

2024-06-26 Thread Andrew Dunstan
Remove redundant perl version checks

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl
scripts, including in plperl's plc_perlboot.pl. Because it made other
changes it wasn't backpatched. However, now this is causing a failure on
back branches when built with bleeding edge perl. Therefore, backpatch
just that part of it which removed those uses, from 15 all the way down
to 9.2, which is the earliest version currently built in the buildfarm.

per report from Alexander Lakhin

Discussion: https://postgr.es/m/4cc2ee93-e03c-8e13-61ed-412e7e6ff...@gmail.com

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/ab46e132fa82cfcdd33accaa6eb18c887f3cd2a3

Modified Files
--
src/pl/plperl/plc_perlboot.pl | 1 -
src/tools/pgindent/pgindent   | 1 -
2 files changed, 2 deletions(-)



pgsql: Remove redundant perl version checks

2024-06-26 Thread Andrew Dunstan
Remove redundant perl version checks

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl
scripts, including in plperl's plc_perlboot.pl. Because it made other
changes it wasn't backpatched. However, now this is causing a failure on
back branches when built with bleeding edge perl. Therefore, backpatch
just that part of it which removed those uses, from 15 all the way down
to 9.2, which is the earliest version currently built in the buildfarm.

per report from Alexander Lakhin

Discussion: https://postgr.es/m/4cc2ee93-e03c-8e13-61ed-412e7e6ff...@gmail.com

Branch
--
REL9_4_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/8851d5c3ad0487edec9b91d8267d1f1acf134e02

Modified Files
--
src/pl/plperl/plc_perlboot.pl | 1 -
src/tools/pgindent/pgindent   | 1 -
2 files changed, 2 deletions(-)



pgsql: Remove redundant perl version checks

2024-06-26 Thread Andrew Dunstan
Remove redundant perl version checks

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl
scripts, including in plperl's plc_perlboot.pl. Because it made other
changes it wasn't backpatched. However, now this is causing a failure on
back branches when built with bleeding edge perl. Therefore, backpatch
just that part of it which removed those uses, from 15 all the way down
to 9.2, which is the earliest version currently built in the buildfarm.

per report from Alexander Lakhin

Discussion: https://postgr.es/m/4cc2ee93-e03c-8e13-61ed-412e7e6ff...@gmail.com

Branch
--
REL_15_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/f853e23bf8276d5509ebb96395e6dc95fac0ec0a

Modified Files
--
src/pl/plperl/plc_perlboot.pl | 1 -
src/tools/pgindent/pgindent   | 1 -
2 files changed, 2 deletions(-)



pgsql: Remove redundant perl version checks

2024-06-26 Thread Andrew Dunstan
Remove redundant perl version checks

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl
scripts, including in plperl's plc_perlboot.pl. Because it made other
changes it wasn't backpatched. However, now this is causing a failure on
back branches when built with bleeding edge perl. Therefore, backpatch
just that part of it which removed those uses, from 15 all the way down
to 9.2, which is the earliest version currently built in the buildfarm.

per report from Alexander Lakhin

Discussion: https://postgr.es/m/4cc2ee93-e03c-8e13-61ed-412e7e6ff...@gmail.com

Branch
--
REL9_5_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/0536f8e2cca21aad53eb11b39513e96e5f0f0830

Modified Files
--
src/pl/plperl/plc_perlboot.pl | 1 -
src/tools/pgindent/pgindent   | 1 -
2 files changed, 2 deletions(-)



pgsql: Remove redundant perl version checks

2024-06-26 Thread Andrew Dunstan
Remove redundant perl version checks

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl
scripts, including in plperl's plc_perlboot.pl. Because it made other
changes it wasn't backpatched. However, now this is causing a failure on
back branches when built with bleeding edge perl. Therefore, backpatch
just that part of it which removed those uses, from 15 all the way down
to 9.2, which is the earliest version currently built in the buildfarm.

per report from Alexander Lakhin

Discussion: https://postgr.es/m/4cc2ee93-e03c-8e13-61ed-412e7e6ff...@gmail.com

Branch
--
REL9_3_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/8f3be9661f7681624b3fa5454df0bd2ccf708351

Modified Files
--
src/pl/plperl/plc_perlboot.pl | 1 -
src/tools/pgindent/pgindent   | 1 -
2 files changed, 2 deletions(-)



pgsql: Remove redundant perl version checks

2024-06-26 Thread Andrew Dunstan
Remove redundant perl version checks

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl
scripts, including in plperl's plc_perlboot.pl. Because it made other
changes it wasn't backpatched. However, now this is causing a failure on
back branches when built with bleeding edge perl. Therefore, backpatch
just that part of it which removed those uses, from 15 all the way down
to 9.2, which is the earliest version currently built in the buildfarm.

per report from Alexander Lakhin

Discussion: https://postgr.es/m/4cc2ee93-e03c-8e13-61ed-412e7e6ff...@gmail.com

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/20f22e6a67bcb90fbc4b40d7368ed309377c06b4

Modified Files
--
src/pl/plperl/plc_perlboot.pl | 1 -
src/tools/pgindent/pgindent   | 1 -
2 files changed, 2 deletions(-)



pgsql: Remove redundant perl version checks

2024-06-26 Thread Andrew Dunstan
Remove redundant perl version checks

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl
scripts, including in plperl's plc_perlboot.pl. Because it made other
changes it wasn't backpatched. However, now this is causing a failure on
back branches when built with bleeding edge perl. Therefore, backpatch
just that part of it which removed those uses, from 15 all the way down
to 9.2, which is the earliest version currently built in the buildfarm.

per report from Alexander Lakhin

Discussion: https://postgr.es/m/4cc2ee93-e03c-8e13-61ed-412e7e6ff...@gmail.com

Branch
--
REL_13_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/b7374f15bd2c01e8ca6192648e964a950e5c6833

Modified Files
--
src/pl/plperl/plc_perlboot.pl | 1 -
src/tools/pgindent/pgindent   | 1 -
2 files changed, 2 deletions(-)



pgsql: Remove redundant perl version checks

2024-06-26 Thread Andrew Dunstan
Remove redundant perl version checks

Commit 4c1532763a removed some redundant uses of 'use 5.008001;' in perl
scripts, including in plperl's plc_perlboot.pl. Because it made other
changes it wasn't backpatched. However, now this is causing a failure on
back branches when built with bleeding edge perl. Therefore, backpatch
just that part of it which removed those uses, from 15 all the way down
to 9.2, which is the earliest version currently built in the buildfarm.

per report from Alexander Lakhin

Discussion: https://postgr.es/m/4cc2ee93-e03c-8e13-61ed-412e7e6ff...@gmail.com

Branch
--
REL9_2_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/1c417311692a15aca0c295e1570e93b6f84ae82a

Modified Files
--
src/pl/plperl/plc_perlboot.pl | 1 -
1 file changed, 1 deletion(-)



Re: pgsql: Skip some permissions checks on Cygwin

2024-06-25 Thread Andrew Dunstan



On 2024-06-24 Mo 6:43 PM, Marco Atzeri wrote:

On 24/06/2024 22:58, Andrew Dunstan wrote:


On 2024-06-24 Mo 4:42 PM, Noah Misch wrote:

On Mon, Jun 24, 2024 at 10:17:32AM -0400, Andrew Dunstan wrote:

On 2024-06-23 Su 9:03 PM, Noah Misch wrote:

On Thu, Jun 13, 2024 at 12:12:29PM +, Andrew Dunstan wrote:

Skip some permissions checks on Cygwin

These are checks that are already skipped on other Windows systems.
   skip "unix-style permissions not supported on Windows", 2
-  if ($windows_os);
+  if ($windows_os || $Config::Config{osname} eq 'cygwin');
-    skip "group access not supported on Windows", 3 if 
($windows_os);

+    skip "group access not supported on Windows", 3
+  if ($windows_os || $Config::Config{osname} eq 'cygwin');
Cygwin does support Unix-style permissions, so I'm not following 
the rationale

for this change.  Can you say more?
It seems to have some difficulties with group permissions in some 
cases. I
improved the error message a bit, and got a bunch that all look 
like this:


build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/global/6303 


mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/global/pg_control 


mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/global/pg_filenode.map 


mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5 


mode must be 0750, not 0700
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/112 


mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/113 


mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1247 


mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1247_fsm 


mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1247_vm 


mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1249 


mode must be 0640, not 0600


Apparently there's some issue with the group None. I don't mind 
digging
further if you think it might be worthwhile, but it seemed to me 
that since
I'll guess digging further is not worthwhile.  Even if it is a 
PostgreSQL bug,
the consequences seem limited to unintentionally withholding group 
access.
There's a remote chance you're using a "noacl" mount, where this 
result could

fit.



mount doesn't show noacl being used




we didn't mind too much skipping these checks on native Windows we 
wouldn't

mind on Cygwin either.
The reasons for skipping the test are different on native Windows 
(does not

provide POSIX file modes) vs. Cygwin (has POSIX file modes, but the
combination of PostgreSQL and Cygwin somehow reaches the wrong mode).



True.


I wonder if Marco can shed any light on what's going on here.


cheers


andrew



are you referring to a specific version or is an old issue ?





This is probably an old issue. We haven't been running TAP tests on 
Cygwin (at least in the buildfarm) so the issue hasn't surfaced before now.



cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: pgsql: Skip some permissions checks on Cygwin

2024-06-24 Thread Andrew Dunstan



On 2024-06-24 Mo 4:42 PM, Noah Misch wrote:

On Mon, Jun 24, 2024 at 10:17:32AM -0400, Andrew Dunstan wrote:

On 2024-06-23 Su 9:03 PM, Noah Misch wrote:

On Thu, Jun 13, 2024 at 12:12:29PM +, Andrew Dunstan wrote:

Skip some permissions checks on Cygwin

These are checks that are already skipped on other Windows systems.
skip "unix-style permissions not supported on Windows", 2
- if ($windows_os);
+ if ($windows_os || $Config::Config{osname} eq 'cygwin');
-   skip "group access not supported on Windows", 3 if ($windows_os);
+   skip "group access not supported on Windows", 3
+ if ($windows_os || $Config::Config{osname} eq 'cygwin');

Cygwin does support Unix-style permissions, so I'm not following the rationale
for this change.  Can you say more?

It seems to have some difficulties with group permissions in some cases. I
improved the error message a bit, and got a bunch that all look like this:

build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/global/6303
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/global/pg_control
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/global/pg_filenode.map
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5
mode must be 0750, not 0700
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/112
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/113
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1247
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1247_fsm
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1247_vm
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1249
mode must be 0640, not 0600


Apparently there's some issue with the group None. I don't mind digging
further if you think it might be worthwhile, but it seemed to me that since

I'll guess digging further is not worthwhile.  Even if it is a PostgreSQL bug,
the consequences seem limited to unintentionally withholding group access.
There's a remote chance you're using a "noacl" mount, where this result could
fit.



mount doesn't show noacl being used





we didn't mind too much skipping these checks on native Windows we wouldn't
mind on Cygwin either.

The reasons for skipping the test are different on native Windows (does not
provide POSIX file modes) vs. Cygwin (has POSIX file modes, but the
combination of PostgreSQL and Cygwin somehow reaches the wrong mode).



True.


I wonder if Marco can shed any light on what's going on here.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: pgsql: Skip some permissions checks on Cygwin

2024-06-24 Thread Andrew Dunstan



On 2024-06-23 Su 9:03 PM, Noah Misch wrote:

On Thu, Jun 13, 2024 at 12:12:29PM +, Andrew Dunstan wrote:

Skip some permissions checks on Cygwin

These are checks that are already skipped on other Windows systems.
skip "unix-style permissions not supported on Windows", 2
- if ($windows_os);
+ if ($windows_os || $Config::Config{osname} eq 'cygwin');
-   skip "group access not supported on Windows", 3 if ($windows_os);
+   skip "group access not supported on Windows", 3
+ if ($windows_os || $Config::Config{osname} eq 'cygwin');

Cygwin does support Unix-style permissions, so I'm not following the rationale
for this change.  Can you say more?



It seems to have some difficulties with group permissions in some cases. 
I improved the error message a bit, and got a bunch that all look like this:


build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/global/6303 
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/global/pg_control 
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/global/pg_filenode.map 
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5 
mode must be 0750, not 0700
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/112 
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/113 
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1247 
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1247_fsm 
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1247_vm 
mode must be 0640, not 0600
build/testrun/initdb/001_initdb/log/regress_log_001_initdb:*main::STDERR/home/Administrator/postgresql/build/testrun/initdb/001_initdb/data/tmp_test_qlhf/data_group/base/5/1249 
mode must be 0640, not 0600



Apparently there's some issue with the group None. I don't mind digging 
further if you think it might be worthwhile, but it seemed to me that 
since we didn't mind too much skipping these checks on native Windows we 
wouldn't mind on Cygwin either.


Up to now we haven't been covering Cygwin with TAP tests, and I'd like 
to change that.



cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com





pgsql: Teach jsonpath string() to unwrap in lax mode

2024-06-17 Thread Andrew Dunstan
Teach jsonpath string() to unwrap in lax mode

This was an ommission in commit 66ea94e, and brings it into compliance
with both other methods and the standard.

Per complaint from David Wheeler.

Author: David Wheeler, Jeevan Chalke
Reviewed-by: Chapman Flack

Discussion: 
https://postgr.es/m/a64ae04f-4410-42b7-a141-7a7349260...@justatheory.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/653d3969bb013f14c4a6884a253ad9676caf8166

Modified Files
--
doc/src/sgml/func.sgml   |  5 -
src/backend/utils/adt/jsonpath_exec.c|  3 +++
src/test/regress/expected/jsonb_jsonpath.out | 12 +++-
src/test/regress/sql/jsonb_jsonpath.sql  |  1 +
4 files changed, 19 insertions(+), 2 deletions(-)



pgsql: Skip some permissions checks on Cygwin

2024-06-13 Thread Andrew Dunstan
Skip some permissions checks on Cygwin

These are checks that are already skipped on other Windows systems.

Backpatch to all live branches, as appropriate.

Branch
--
REL_14_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/0e51485392c0c457de0709181a3d2bfdba6f3fb6

Modified Files
--
src/bin/initdb/t/001_initdb.pl   | 2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 2 +-
src/bin/pg_ctl/t/001_start_stop.pl   | 3 ++-
src/bin/pg_rewind/t/002_databases.pl | 2 +-
src/bin/pg_verifybackup/t/003_corruption.pl  | 3 ++-
5 files changed, 7 insertions(+), 5 deletions(-)



pgsql: Skip some permissions checks on Cygwin

2024-06-13 Thread Andrew Dunstan
Skip some permissions checks on Cygwin

These are checks that are already skipped on other Windows systems.

Backpatch to all live branches, as appropriate.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/f83908798f78c4cafda217ca875602c88ea2ae28

Modified Files
--
src/bin/initdb/t/001_initdb.pl   | 2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 2 +-
src/bin/pg_ctl/t/001_start_stop.pl   | 3 ++-
src/bin/pg_rewind/t/002_databases.pl | 2 +-
src/bin/pg_verifybackup/t/003_corruption.pl  | 3 ++-
5 files changed, 7 insertions(+), 5 deletions(-)



pgsql: Skip some permissions checks on Cygwin

2024-06-13 Thread Andrew Dunstan
Skip some permissions checks on Cygwin

These are checks that are already skipped on other Windows systems.

Backpatch to all live branches, as appropriate.

Branch
--
REL_15_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/c38518fa978075d08d64a34e1b4504da13a81e02

Modified Files
--
src/bin/initdb/t/001_initdb.pl   | 2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 2 +-
src/bin/pg_ctl/t/001_start_stop.pl   | 3 ++-
src/bin/pg_rewind/t/002_databases.pl | 2 +-
src/bin/pg_verifybackup/t/003_corruption.pl  | 3 ++-
5 files changed, 7 insertions(+), 5 deletions(-)



pgsql: Skip some permissions checks on Cygwin

2024-06-13 Thread Andrew Dunstan
Skip some permissions checks on Cygwin

These are checks that are already skipped on other Windows systems.

Backpatch to all live branches, as appropriate.

Branch
--
REL_12_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/ca947649208ce7343c54c8f92e67ea9274be812c

Modified Files
--
src/bin/initdb/t/001_initdb.pl   |  2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 15 +++
src/bin/pg_ctl/t/001_start_stop.pl   |  3 ++-
src/bin/pg_rewind/t/002_databases.pl |  2 +-
4 files changed, 15 insertions(+), 7 deletions(-)



pgsql: Skip some permissions checks on Cygwin

2024-06-13 Thread Andrew Dunstan
Skip some permissions checks on Cygwin

These are checks that are already skipped on other Windows systems.

Backpatch to all live branches, as appropriate.

Branch
--
REL_13_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/57625308f62da2d4f875478a39158a23e41a3dbe

Modified Files
--
src/bin/initdb/t/001_initdb.pl   |  2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 15 +++
src/bin/pg_ctl/t/001_start_stop.pl   |  3 ++-
src/bin/pg_rewind/t/002_databases.pl |  2 +-
src/bin/pg_verifybackup/t/003_corruption.pl  |  3 ++-
5 files changed, 17 insertions(+), 8 deletions(-)



pgsql: Skip some permissions checks on Cygwin

2024-06-13 Thread Andrew Dunstan
Skip some permissions checks on Cygwin

These are checks that are already skipped on other Windows systems.

Backpatch to all live branches, as appropriate.

Branch
--
REL_16_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/32c5dc0ebe83e927c953c42ad79148a03933c6ba

Modified Files
--
src/bin/initdb/t/001_initdb.pl   | 2 +-
src/bin/pg_basebackup/t/010_pg_basebackup.pl | 2 +-
src/bin/pg_ctl/t/001_start_stop.pl   | 3 ++-
src/bin/pg_rewind/t/002_databases.pl | 2 +-
src/bin/pg_verifybackup/t/003_corruption.pl  | 3 ++-
5 files changed, 7 insertions(+), 5 deletions(-)



pgsql: Add postgres_inc to meson check for Python.h

2024-06-13 Thread Andrew Dunstan
Add postgres_inc to meson check for Python.h

Required for Cygwin.

Backpatch to release 16.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/11b9b8ce44a8cc80cbef6ade2b5ae7438227da79

Modified Files
--
meson.build | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



pgsql: Add postgres_inc to meson check for Python.h

2024-06-13 Thread Andrew Dunstan
Add postgres_inc to meson check for Python.h

Required for Cygwin.

Backpatch to release 16.

Branch
--
REL_16_STABLE

Details
---
https://git.postgresql.org/pg/commitdiff/a13c7ee87b0ab31ecbf1a9e9302ee48b52d1cf9a

Modified Files
--
meson.build | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)



Re: pgsql: Fix potential NULL pointer dereference in getIdentitySequence()

2024-05-26 Thread Andrew Dunstan


On 2024-05-26 Su 17:51, Tom Lane wrote:

Andrew Dunstan  writes:

On 2024-05-26 Su 07:58, Michael Paquier wrote:

Fix potential NULL pointer dereference in getIdentitySequence()

This appears to have upset a number of buildfarm members

It's hard to see how that patch would have broken "configure",
and besides that the failures started earlier.

What it looks like from here is that Andres messed up the
installed-packages set on a lot of his animals.





Oops, sorry for the noise. I just looked at a few failures and this was 
what I saw.


cheers

andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com


pgsql: Fix meson uuid header check so it works with MSVC

2024-05-26 Thread Andrew Dunstan
Fix meson uuid header check so it works with MSVC

The OSSP uuid.h file includes unistd.h, so to use it with MSVC we need to
include the postgres include directories so it picks up our version of
that in src/include/port/win32_msvc. Adjust the meson test accordingly.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/7c655a04a2dc84b59ed6dce97bd38b79e734ecca

Modified Files
--
meson.build | 5 -
1 file changed, 4 insertions(+), 1 deletion(-)



Re: pgsql: Fix potential NULL pointer dereference in getIdentitySequence()

2024-05-26 Thread Andrew Dunstan



On 2024-05-26 Su 07:58, Michael Paquier wrote:

Fix potential NULL pointer dereference in getIdentitySequence()

The function invokes SearchSysCacheAttNum() and SearchSysCacheAttName().
They may respectively return 0 for the attribute number or NULL for
the attribute name if the attribute does not exist, without any kind of
error handling.  The common practice is to check that the data retrieved
from the syscache is valid.  There is no risk of NULL pointer
dereferences currently, but let's stick to the practice of making sure
that this data is always valid, to catch future inconsistency mistakes.
The code is switched to use get_attnum() and get_attname(), and adds
some error handling.

Oversight in 509199587df7.

Reported-by: Ranier Vilela
Author: Ashutosh Bapat
Discussion: 
https://postgr.es/m/caeudqaqh_rzqofcykso5d9vhf-vd64_zodfq_2zsusszkem...@mail.gmail.com



This appears to have upset a number of buildfarm members


cheers


andrew

--

Andrew Dunstan
EDB: https://www.enterprisedb.com





pgsql: Remove redundant JSON parser typedefs

2024-04-27 Thread Andrew Dunstan
Remove redundant JSON parser typedefs

JsonNonTerminal and JsonParserSem were added in commit 3311ea86ed

These names of these two enums are not actually used, so there is no
need for typedefs. Instead use plain enums to declare the constants.

Noticed by Alvaro Herera.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/e00b4f79e7367272b436dea6f15d6f421f3d6e1a

Modified Files
--
src/common/jsonapi.c | 8 
src/tools/pgindent/typedefs.list | 2 --
2 files changed, 4 insertions(+), 6 deletions(-)



pgsql: Add pg_logging_init() calls missing in commit ba3e6e2bca

2024-04-24 Thread Andrew Dunstan
Add pg_logging_init() calls missing in commit ba3e6e2bca

As noticed by Michael Paquier.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/cc893b82370b5d68e0a72f5bb1663644a3a0351d

Modified Files
--
src/test/modules/test_json_parser/test_json_parser_incremental.c | 2 ++
src/test/modules/test_json_parser/test_json_parser_perf.c| 2 ++
2 files changed, 4 insertions(+)



pgsql: Post review fixes for test_json_parser test module

2024-04-23 Thread Andrew Dunstan
Post review fixes for test_json_parser test module

. Add missing copytight notices
. improve code coverage
. put work files in a temp directory in the standard location
. improve error checking in C code
. indent perl files with perltidy
. add some comments

per comments from Michael Paquier

Discussion: https://postgr.es/m/zic3-cdfys4-6...@paquier.xyz

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/ba3e6e2bca97df14920375b0a1ebf4eab95b78b5

Modified Files
--
.../t/001_test_json_parser_incremental.pl  | 15 +++-
src/test/modules/test_json_parser/t/002_inline.pl  | 93 --
.../test_json_parser/t/003_test_semantic.pl| 15 +++-
.../test_json_parser/t/004_test_parser_perf.pl | 18 +++--
.../test_json_parser_incremental.c | 25 --
.../test_json_parser/test_json_parser_perf.c   | 13 ++-
src/test/modules/test_json_parser/tiny.json|  1 +
src/test/modules/test_json_parser/tiny.out |  1 +
8 files changed, 133 insertions(+), 48 deletions(-)



Re: pgsql: Fix assorted bugs in ecpg's macro mechanism.

2024-04-18 Thread Andrew Dunstan



On 2024-04-16 Tu 12:31, Tom Lane wrote:

Fix assorted bugs in ecpg's macro mechanism.



Buildfarm animals running the old MSVC build system don't like this.

It looks to me like we'll need an adjustment in 
src/tools/msvc/ecpg_regression.proj to replicate the special rules 
surrounding define.c and define_prelim.c. Not sure who has the requisite 
knowledge to do that - I don't.



cheers


andrew

--

Andrew Dunstan
EDB: https://www.enterprisedb.com





pgsql: Shrink test file for test_json_parser module

2024-04-12 Thread Andrew Dunstan
Shrink test file for test_json_parser module

Also delete live URLs

Jacob Champion

Discussion: 
https://postgr.es/m/CAOYmi+mtH=v1wzkaoaucd5qqqwr61hnxmjbj9h-czxaa1jx...@mail.gmail.com

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/b8a7bfa33324bc40c7180cb946a39378fb48309c

Modified Files
--
src/test/modules/test_json_parser/tiny.json |   515 +-
src/test/modules/test_json_parser/tiny.out  | 19346 +-
2 files changed, 192 insertions(+), 19669 deletions(-)



pgsql: Assorted minor cleanups in the test_json_parser module

2024-04-12 Thread Andrew Dunstan
Assorted minor cleanups in the test_json_parser module

Per gripes from Michael Paquier

Discussion: https://postgr.es/m/zhtq6_w1vwohq...@paquier.xyz

Along the way, also clean up a handful of typos in 3311ea86ed and
ea7b4e9a2a, found by Alexander Lakhin, and a couple of stylistic
snafus noted by Daniel Westermann and Daniel Gustafsson.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/42fa4b660143b66bea1fb90793ec90054e170c93

Modified Files
--
src/backend/backup/basebackup_incremental.c|  6 ++---
src/common/jsonapi.c   |  6 ++---
src/common/parse_manifest.c|  2 +-
src/test/modules/test_json_parser/README   | 19 +++
.../test_json_parser_incremental.c | 28 --
.../test_json_parser/test_json_parser_perf.c   | 11 +
6 files changed, 43 insertions(+), 29 deletions(-)



pgsql: Add a TAP test for test_json_parser_perf

2024-04-12 Thread Andrew Dunstan
Add a TAP test for test_json_parser_perf

This just makes sure the test can run with a single iteration. A real
performance test would test with many more.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/daf554dbeabf0957a25c9e37488d42c047c0ce23

Modified Files
--
src/test/modules/test_json_parser/meson.build  |  3 +-
.../test_json_parser/t/004_test_parser_perf.pl | 35 ++
2 files changed, 37 insertions(+), 1 deletion(-)



pgsql: Don't allocate large buffer on the stack in pg_verifybackup

2024-04-12 Thread Andrew Dunstan
Don't allocate large buffer on the stack in pg_verifybackup

Per complaint from Andres Freund. Follow his suggestion to allocate the
buffer once in the calling routine instead.

Also make a tiny indentation improvement.

Discussion: 
https://postgr.es/m/20240411190147.a3yries632olf...@awork3.anarazel.de

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/929c05774b512bdf7ea4a5912fa2f4d63fc90712

Modified Files
--
src/bin/pg_verifybackup/pg_verifybackup.c | 17 +++--
1 file changed, 11 insertions(+), 6 deletions(-)



pgsql: Fix some memory leaks associated with parsing json and manifests

2024-04-12 Thread Andrew Dunstan
Fix some memory leaks associated with parsing json and manifests

Coverity complained about not freeing some memory associated with
incrementally parsing backup manifests. To fix that, provide and use a new
shutdown function for the JsonManifestParseIncrementalState object, in
line with a suggestion from Tom Lane.

While analysing the problem, I noticed a buglet in freeing memory for
incremental json lexers. To fix that remove a bogus condition on
freeing the memory allocated for them.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/661ab4e185784db79c194b5758555b1db3f30483

Modified Files
--
src/backend/backup/basebackup_incremental.c |  3 +++
src/bin/pg_combinebackup/load_manifest.c|  3 +++
src/bin/pg_verifybackup/pg_verifybackup.c   |  3 +++
src/common/jsonapi.c| 21 ++---
src/common/parse_manifest.c | 13 -
src/include/common/parse_manifest.h |  1 +
6 files changed, 32 insertions(+), 12 deletions(-)



pgsql: Silence some compiler warnings in commit 3311ea86ed

2024-04-05 Thread Andrew Dunstan
Silence some compiler warnings in commit 3311ea86ed

Per report from Nathan Bossart

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/c3e60f3d7e4268c6448ec592053b3b893899867d

Modified Files
--
src/common/jsonapi.c | 7 +++
1 file changed, 7 insertions(+)



pgsql: Tidy up after incremental JSON parser patch

2024-04-04 Thread Andrew Dunstan
Tidy up after incremental JSON parser patch

Remove junk left over from non-vpath builds.

Try to remedy gettext error on some platforms.

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/88620824c2a62376e224c4b595b9fe69fb858978

Modified Files
--
src/common/jsonapi.c | 4 ++--
src/test/modules/test_json_parser/.gitignore | 3 +++
src/test/modules/test_json_parser/Makefile   | 5 +
3 files changed, 10 insertions(+), 2 deletions(-)



pgsql: Fix warnings re typedef redefinition in ea7b4e9a2a and 3311ea86e

2024-04-04 Thread Andrew Dunstan
Fix warnings re typedef redefinition in ea7b4e9a2a and 3311ea86ed

Per gripe from Tom Lane and the buildfarm

Branch
--
master

Details
---
https://git.postgresql.org/pg/commitdiff/1b00fe30a67774e55c5fc776096a3c96f1a147d2

Modified Files
--
src/common/jsonapi.c| 12 
src/common/parse_manifest.c |  5 +++--
2 files changed, 11 insertions(+), 6 deletions(-)



  1   2   3   4   5   6   7   8   9   10   >