Re: Remove MSVC scripts from the tree

2023-12-23 Thread Michael Paquier
On Fri, Dec 22, 2023 at 09:07:21AM -0500, Andrew Dunstan wrote:
> I've done it a bit differently, but the same idea. I have tested that what I
> committed passes checks on Unix and works on Windows.

Sounds fine by me.  Thanks for the quick turnaround!
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-12-22 Thread Andrew Dunstan



On 2023-12-21 Th 18:20, Michael Paquier wrote:

On Thu, Dec 21, 2023 at 03:43:32PM -0500, Andrew Dunstan wrote:

On 2023-12-21 Th 03:01, Michael Paquier wrote:

Andrew, was the original target of pgperlsyncheck committers and
hackers who played with the MSVC scripts but could not run sanity
checks on Windows (see [1])?


yes.

Okay, thanks.  Wouldn't it be better to remove it at the end?  With
the main use case behind its introduction being gone, it is less
attractive to keep maintaining it.  If some people have been using it
in their workflows, I'm OK to keep it but the rest of the tree can be
checked at runtime as well.


I'm actually a bit dubious about win32tzlist.pl. Win32::Registry is not
present in a recent Strawberry Perl installation, and its latest version
says it is obsolete, although it's still included in the cpan bundle
libwin32.

I wonder who has actually run the script any time recently?

Hmm...  I've never run it with meson on Win32.



Turns out I was wrong - Windows sometimes doesn't find files nicely. It 
is present in my Strawberry installation.






In any case, we can probably work around the syncheck issue by making the
module a runtime requirement rather than a compile time requirement, by
using "require" instead of "use".

Interesting.  Another trick would be needed for HKEY_LOCAL_MACHINE,
like what the dummylib but local to win32tzlist.pl.  Roughly among
these lines:
-use Win32::Registry;
+use Config;
+
+require Win32::Registry;
  
  my $tzfile = 'src/bin/initdb/findtimezone.c';
  
+if ($Config{osname} ne 'MSWin32' && $Config{osname} ne 'msys')

+{
+   use vars qw($HKEY_LOCAL_MACHINE);
+}



I've done it a bit differently, but the same idea. I have tested that 
what I committed passes checks on Unix and works on Windows.



cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: Remove MSVC scripts from the tree

2023-12-21 Thread Michael Paquier
On Thu, Dec 21, 2023 at 03:43:32PM -0500, Andrew Dunstan wrote:
> On 2023-12-21 Th 03:01, Michael Paquier wrote:
>> Andrew, was the original target of pgperlsyncheck committers and
>> hackers who played with the MSVC scripts but could not run sanity
>> checks on Windows (see [1])?
> 
> 
> yes.

Okay, thanks.  Wouldn't it be better to remove it at the end?  With
the main use case behind its introduction being gone, it is less
attractive to keep maintaining it.  If some people have been using it
in their workflows, I'm OK to keep it but the rest of the tree can be
checked at runtime as well.

> I'm actually a bit dubious about win32tzlist.pl. Win32::Registry is not
> present in a recent Strawberry Perl installation, and its latest version
> says it is obsolete, although it's still included in the cpan bundle
> libwin32.
> 
> I wonder who has actually run the script any time recently?

Hmm...  I've never run it with meson on Win32.

> In any case, we can probably work around the syncheck issue by making the
> module a runtime requirement rather than a compile time requirement, by
> using "require" instead of "use".

Interesting.  Another trick would be needed for HKEY_LOCAL_MACHINE,
like what the dummylib but local to win32tzlist.pl.  Roughly among
these lines:
-use Win32::Registry;
+use Config;
+
+require Win32::Registry;
 
 my $tzfile = 'src/bin/initdb/findtimezone.c';
 
+if ($Config{osname} ne 'MSWin32' && $Config{osname} ne 'msys')
+{
+   use vars qw($HKEY_LOCAL_MACHINE);
+}
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-12-21 Thread Andrew Dunstan



On 2023-12-21 Th 03:01, Michael Paquier wrote:

On Wed, Dec 20, 2023 at 11:39:15PM -0800, Andres Freund wrote:

Can't we teach the tool that it should not validate src/tools/win32tzlist.pl
on !windows? It's obviously windows specific code, and it's special case
enough that there doesn't seem like a need to develop it on !windows.

I am not really excited about keeping a dummy library for the sake of
a script checking if this WIN32-only file is correctly written, and
I've never used pgperlsyncheck, TBH, since it exists in af616ce48347.
Anyway, we could just tweak the list of files returned by
find_perl_files as win32tzlist.pl is valid for perltidy and
perlcritic.

Andrew, was the original target of pgperlsyncheck committers and
hackers who played with the MSVC scripts but could not run sanity
checks on Windows (see [1])?



yes.



There are a few more cases like the
Unicode scripts or some of the stuff in src/tools/ where that can be
useful still these are not touched on a daily basis.  The rest of the
pm files are for TAP tests, one for Unicode.  I'm OK to tweak the
script, still, if its main purpose is gone..

[1]: 
https://www.postgresql.org/message-id/f3c12e2c-618f-cb6f-082b-a2f604dbe...@2ndquadrant.com



I'm actually a bit dubious about win32tzlist.pl. Win32::Registry is not 
present in a recent Strawberry Perl installation, and its latest version 
says it is obsolete, although it's still included in the cpan bundle 
libwin32.


I wonder who has actually run the script any time recently?

In any case, we can probably work around the syncheck issue by making 
the module a runtime requirement rather than a compile time requirement, 
by using "require" instead of "use".



cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: Remove MSVC scripts from the tree

2023-12-21 Thread Peter Eisentraut

On 20.12.23 16:43, Peter Eisentraut wrote:

On 20.12.23 12:40, Andres Freund wrote:
Hm, or perhaps we should just get rid of sed use altogether. The 
sepgsql case

is trivially translateable to perl, and postprocess_dtrace.sed isn't
much harder.


Maybe yeah, but also it seems fine as is and we can easily fix the 
present issue ...



OTOH, I actually don't think it's valid to not have sed when you have
dtrace. Erroring out in a weird way in such an artificially 
constructed test

doesn't really seem like a problem.


Agreed.  So let's just make it not-required, and that should work.

Updated patch set attached.


I have committed these two.




Re: Remove MSVC scripts from the tree

2023-12-21 Thread Michael Paquier
On Wed, Dec 20, 2023 at 11:39:15PM -0800, Andres Freund wrote:
> Can't we teach the tool that it should not validate src/tools/win32tzlist.pl
> on !windows? It's obviously windows specific code, and it's special case
> enough that there doesn't seem like a need to develop it on !windows.

I am not really excited about keeping a dummy library for the sake of
a script checking if this WIN32-only file is correctly written, and
I've never used pgperlsyncheck, TBH, since it exists in af616ce48347.
Anyway, we could just tweak the list of files returned by
find_perl_files as win32tzlist.pl is valid for perltidy and
perlcritic.

Andrew, was the original target of pgperlsyncheck committers and
hackers who played with the MSVC scripts but could not run sanity
checks on Windows (see [1])?  There are a few more cases like the
Unicode scripts or some of the stuff in src/tools/ where that can be
useful still these are not touched on a daily basis.  The rest of the
pm files are for TAP tests, one for Unicode.  I'm OK to tweak the
script, still, if its main purpose is gone..

[1]: 
https://www.postgresql.org/message-id/f3c12e2c-618f-cb6f-082b-a2f604dbe...@2ndquadrant.com
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andres Freund
Hi,

On 2023-12-21 08:31:57 +0100, Peter Eisentraut wrote:
> On 20.12.23 18:22, Andrew Dunstan wrote:
> > 
> > On 2023-12-20 We 11:32, Robert Haas wrote:
> > > On Wed, Dec 20, 2023 at 11:03 AM Andrew Dunstan
> > >  wrote:
> > > > > This could perhaps be related to this patch removing the module in
> > > > > src/tools/msvc/dummylib/Win32/Registry.pm ?
> > > > It is. I've fixed the buildfarm to stop checking this script.
> > > Thanks! But I wonder whether the script itself also needs to be
> > > changed? Are we expecting that the 'use Win32::Registry' in
> > > win32tzlist.pl would be satisfied externally in some case?
> > > 
> > 
> > Yes, the module will normally be present on a Windows perl. The only
> > reason we had dummylib was so we could check the perl scripts on Unix.
> 
> But this use case still exists.  Right now, running
> 
> ./src/tools/perlcheck/pgperlsyncheck .
> 
> fails because this module is missing.  So I think we need to put the dummy
> module back somehow.

Can't we teach the tool that it should not validate src/tools/win32tzlist.pl
on !windows? It's obviously windows specific code, and it's special case
enough that there doesn't seem like a need to develop it on !windows.

Greetings,

Andres Freund




Re: Remove MSVC scripts from the tree

2023-12-20 Thread Peter Eisentraut

On 21.12.23 07:35, vignesh C wrote:

I noticed this is because the lex file getting added without escape
characters in the C file:
#line 2 "D:\postgres\pg_meson\src\backend\utils\adt\jsonpath_scan.l"

There were no warnings when I used flex 2.6.4.

Did anyone else get these warnings with the flex 2.5.35 version?


It appears that this is an issue related to building in a separate build 
directory, not something specific to meson.  The solution would be to 
use an appropriately new flex, as you have done.






Re: Remove MSVC scripts from the tree

2023-12-20 Thread Peter Eisentraut

On 20.12.23 18:22, Andrew Dunstan wrote:


On 2023-12-20 We 11:32, Robert Haas wrote:
On Wed, Dec 20, 2023 at 11:03 AM Andrew Dunstan  
wrote:

This could perhaps be related to this patch removing the module in
src/tools/msvc/dummylib/Win32/Registry.pm ?

It is. I've fixed the buildfarm to stop checking this script.

Thanks! But I wonder whether the script itself also needs to be
changed? Are we expecting that the 'use Win32::Registry' in
win32tzlist.pl would be satisfied externally in some case?



Yes, the module will normally be present on a Windows perl. The only 
reason we had dummylib was so we could check the perl scripts on Unix.


But this use case still exists.  Right now, running

./src/tools/perlcheck/pgperlsyncheck .

fails because this module is missing.  So I think we need to put the 
dummy module back somehow.






Re: Remove MSVC scripts from the tree

2023-12-20 Thread vignesh C
On Wed, 20 Dec 2023 at 21:13, Peter Eisentraut  wrote:
>
> On 20.12.23 12:40, Andres Freund wrote:
> > Hm, or perhaps we should just get rid of sed use altogether. The sepgsql 
> > case
> > is trivially translateable to perl, and postprocess_dtrace.sed isn't
> > much harder.
>
> Maybe yeah, but also it seems fine as is and we can easily fix the
> present issue ...
>
> > OTOH, I actually don't think it's valid to not have sed when you have
> > dtrace. Erroring out in a weird way in such an artificially constructed test
> > doesn't really seem like a problem.
>
> Agreed.  So let's just make it not-required, and that should work.
>
> Updated patch set attached.

Thanks for the patches.
I noticed one issue about the flex 2.5.35 version mentioned.
I noticed some warning with meson build in windows with flex 2.5.35
for several files:
 python = find_program(get_option('PYTHON'), required: true, native: true)
 flex = find_program(get_option('FLEX'), native: true, version: '>= 2.5.35')
 bison = find_program(get_option('BISON'), native: true, version: '>= 2.3')
-sed = find_program(get_option('SED'), 'sed', native: true)
+sed = find_program(get_option('SED'), 'sed', native: true, required: false)
 prove = find_program(get_option('PROVE'), native: true, required: false)
 tar = find_program(get_option('TAR'), native: true)


Compiling C object
src/test/isolation/isolationtester.exe.p/meson-generated_.._specscanner.c.obj
src/test/isolation/specscanner.c(2): warning C4129: 'W': unrecognized
character escape sequence
src/test/isolation/specscanner.c(2): warning C4129: 'P': unrecognized
character escape sequence
src/test/isolation/specscanner.c(2): warning C4129: 'p': unrecognized
character escape sequence
src/test/isolation/specscanner.c(2): warning C4129: 's': unrecognized
character escape sequence
src/test/isolation/specscanner.c(2): warning C4129: 'i': unrecognized
character escape sequence

I noticed this is because the lex file getting added without escape
characters in the C file:
#line 2 "D:\postgres\pg_meson\src\backend\utils\adt\jsonpath_scan.l"

There were no warnings when I used flex 2.6.4.

Did anyone else get these warnings with the flex 2.5.35 version?

Regards,
Vignesh




Re: Remove MSVC scripts from the tree

2023-12-20 Thread Michael Paquier
On Wed, Dec 20, 2023 at 07:00:50PM +0100, Daniel Gustafsson wrote:
> On 20 Dec 2023, at 18:22, Andrew Dunstan  wrote:
>> Yes, the module will normally be present on a Windows perl. The
>> only reason we had dummylib was so we could check the perl scripts on
>> Unix.
> 
> Thanks for taking care of it!

Thanks!
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-12-20 Thread Daniel Gustafsson
> On 20 Dec 2023, at 18:22, Andrew Dunstan  wrote:
> On 2023-12-20 We 11:32, Robert Haas wrote:
>> On Wed, Dec 20, 2023 at 11:03 AM Andrew Dunstan  wrote:
 This could perhaps be related to this patch removing the module in
 src/tools/msvc/dummylib/Win32/Registry.pm ?
>>> It is. I've fixed the buildfarm to stop checking this script.
>> Thanks! But I wonder whether the script itself also needs to be
>> changed? Are we expecting that the 'use Win32::Registry' in
>> win32tzlist.pl would be satisfied externally in some case?
> 
> Yes, the module will normally be present on a Windows perl. The only reason 
> we had dummylib was so we could check the perl scripts on Unix.

Thanks for taking care of it!

--
Daniel Gustafsson





Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andrew Dunstan



On 2023-12-20 We 11:32, Robert Haas wrote:

On Wed, Dec 20, 2023 at 11:03 AM Andrew Dunstan  wrote:

This could perhaps be related to this patch removing the module in
src/tools/msvc/dummylib/Win32/Registry.pm ?

It is. I've fixed the buildfarm to stop checking this script.

Thanks! But I wonder whether the script itself also needs to be
changed? Are we expecting that the 'use Win32::Registry' in
win32tzlist.pl would be satisfied externally in some case?



Yes, the module will normally be present on a Windows perl. The only 
reason we had dummylib was so we could check the perl scripts on Unix.



cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: Remove MSVC scripts from the tree

2023-12-20 Thread Robert Haas
On Wed, Dec 20, 2023 at 11:03 AM Andrew Dunstan  wrote:
> > This could perhaps be related to this patch removing the module in
> > src/tools/msvc/dummylib/Win32/Registry.pm ?
>
> It is. I've fixed the buildfarm to stop checking this script.

Thanks! But I wonder whether the script itself also needs to be
changed? Are we expecting that the 'use Win32::Registry' in
win32tzlist.pl would be satisfied externally in some case?

-- 
Robert Haas
EDB: http://www.enterprisedb.com




Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andrew Dunstan



On 2023-12-20 We 08:31, Daniel Gustafsson wrote:

On 20 Dec 2023, at 01:48, Michael Paquier  wrote:

On Mon, Nov 20, 2023 at 05:03:28PM +0900, Michael Paquier wrote:

Your suggestion to create a new sect2 for "Windows" as much as Andres'
suggestion are OK by as an intermediate step, and I suspect that the
end result will likely not be that.

It took me some time to get back to this one, and just applied the
patch removing the scripts.

The Buildfarm complains that Win32::Registry can't be found:

Can't locate Win32/Registry.pm in @INC (you may need to install the 
Win32::Registry module) (@INC entries checked: src/test/perl src/tools/msvc 
src/backend/catalog src/backend/utils/mb/Unicode src/bin/pg_rewind 
src/test/ssl/t src/tools/msvc/dummylib /usr/local/lib64/perl5/5.38 
/usr/local/share/perl5/5.38 /usr/lib64/perl5/vendor_perl 
/usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at 
./src/tools/win32tzlist.pl line 21.
BEGIN failed--compilation aborted at ./src/tools/win32tzlist.pl line 21.

https://brekka.postgresql.org/cgi-bin/show_log.pl?nm=koel&dt=2023-12-20%2013%3A19%3A04

This could perhaps be related to this patch removing the module in
src/tools/msvc/dummylib/Win32/Registry.pm ?




It is. I've fixed the buildfarm to stop checking this script.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: Remove MSVC scripts from the tree

2023-12-20 Thread Peter Eisentraut

On 20.12.23 12:40, Andres Freund wrote:

Hm, or perhaps we should just get rid of sed use altogether. The sepgsql case
is trivially translateable to perl, and postprocess_dtrace.sed isn't
much harder.


Maybe yeah, but also it seems fine as is and we can easily fix the 
present issue ...



OTOH, I actually don't think it's valid to not have sed when you have
dtrace. Erroring out in a weird way in such an artificially constructed test
doesn't really seem like a problem.


Agreed.  So let's just make it not-required, and that should work.

Updated patch set attached.
From ade66f6cee93ba070e6019bc7368d89884f7933c Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 20 Dec 2023 16:36:12 +0100
Subject: [PATCH v2 1/2] meson: Make sed optional

sed is used only if dtrace or selinux are enabled.  Those options are
only used on Unix platforms, which should have sed.  But we don't want
to make sed a hard requirement on Windows, which was the case in meson
until now.

This just changes sed to be not-required by meson.  If you happen to
use a system with, say, dtrace but without sed, you might get a
slightly complicated error from meson during the build, but that seems
better than making the requiredness a complicated conditional that
will need to be maintained.
---
 meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meson.build b/meson.build
index f816283301..3162d0343b 100644
--- a/meson.build
+++ b/meson.build
@@ -331,7 +331,7 @@ perl = find_program(get_option('PERL'), required: true, 
native: true)
 python = find_program(get_option('PYTHON'), required: true, native: true)
 flex = find_program(get_option('FLEX'), native: true, version: '>= 2.5.35')
 bison = find_program(get_option('BISON'), native: true, version: '>= 2.3')
-sed = find_program(get_option('SED'), 'sed', native: true)
+sed = find_program(get_option('SED'), 'sed', native: true, required: false)
 prove = find_program(get_option('PROVE'), native: true, required: false)
 tar = find_program(get_option('TAR'), native: true)
 gzip = find_program(get_option('GZIP'), native: true)
-- 
2.43.0

From 2ba06a6414be2654b5a4f1cfae2b28e36cd1bcab Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Tue, 19 Dec 2023 16:20:41 +0100
Subject: [PATCH v2 2/2] meson: Make gzip and tar optional

They are only used for some tests.  The tests are already set to skip
as appropriate if they are not available.
---
 contrib/basebackup_to_shell/meson.build | 4 ++--
 meson.build | 4 ++--
 src/bin/pg_basebackup/meson.build   | 4 ++--
 src/bin/pg_dump/meson.build | 2 +-
 src/bin/pg_verifybackup/meson.build | 4 ++--
 5 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/contrib/basebackup_to_shell/meson.build 
b/contrib/basebackup_to_shell/meson.build
index a5488c3023..331ee1c9be 100644
--- a/contrib/basebackup_to_shell/meson.build
+++ b/contrib/basebackup_to_shell/meson.build
@@ -24,7 +24,7 @@ tests += {
 'tests': [
   't/001_basic.pl',
 ],
-'env': {'GZIP_PROGRAM': gzip.path(),
-'TAR': tar.path()},
+'env': {'GZIP_PROGRAM': gzip.found() ? gzip.path() : '',
+'TAR': tar.found() ? tar.path() : '' },
   },
 }
diff --git a/meson.build b/meson.build
index 3162d0343b..4e98cbefe4 100644
--- a/meson.build
+++ b/meson.build
@@ -333,8 +333,8 @@ flex = find_program(get_option('FLEX'), native: true, 
version: '>= 2.5.35')
 bison = find_program(get_option('BISON'), native: true, version: '>= 2.3')
 sed = find_program(get_option('SED'), 'sed', native: true, required: false)
 prove = find_program(get_option('PROVE'), native: true, required: false)
-tar = find_program(get_option('TAR'), native: true)
-gzip = find_program(get_option('GZIP'), native: true)
+tar = find_program(get_option('TAR'), native: true, required: false)
+gzip = find_program(get_option('GZIP'), native: true, required: false)
 program_lz4 = find_program(get_option('LZ4'), native: true, required: false)
 openssl = find_program(get_option('OPENSSL'), native: true, required: false)
 program_zstd = find_program(get_option('ZSTD'), native: true, required: false)
diff --git a/src/bin/pg_basebackup/meson.build 
b/src/bin/pg_basebackup/meson.build
index c426173db3..5445903a5b 100644
--- a/src/bin/pg_basebackup/meson.build
+++ b/src/bin/pg_basebackup/meson.build
@@ -80,8 +80,8 @@ tests += {
   'sd': meson.current_source_dir(),
   'bd': meson.current_build_dir(),
   'tap': {
-'env': {'GZIP_PROGRAM': gzip.path(),
-'TAR': tar.path(),
+'env': {'GZIP_PROGRAM': gzip.found() ? gzip.path() : '',
+'TAR': tar.found() ? tar.path() : '',
 'LZ4': program_lz4.found() ? program_lz4.path() : '',
 },
 'tests': [
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index b6603e26a5..77d162cad4 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -90,7 +90,7 @@ tests += {
   'bd': meson.current_build_dir(),
   'tap': {
 'env': {

Re: Remove MSVC scripts from the tree

2023-12-20 Thread Daniel Gustafsson
> On 20 Dec 2023, at 01:48, Michael Paquier  wrote:
> 
> On Mon, Nov 20, 2023 at 05:03:28PM +0900, Michael Paquier wrote:
>> Your suggestion to create a new sect2 for "Windows" as much as Andres'
>> suggestion are OK by as an intermediate step, and I suspect that the
>> end result will likely not be that.
> 
> It took me some time to get back to this one, and just applied the
> patch removing the scripts.

The Buildfarm complains that Win32::Registry can't be found:

Can't locate Win32/Registry.pm in @INC (you may need to install the 
Win32::Registry module) (@INC entries checked: src/test/perl src/tools/msvc 
src/backend/catalog src/backend/utils/mb/Unicode src/bin/pg_rewind 
src/test/ssl/t src/tools/msvc/dummylib /usr/local/lib64/perl5/5.38 
/usr/local/share/perl5/5.38 /usr/lib64/perl5/vendor_perl 
/usr/share/perl5/vendor_perl /usr/lib64/perl5 /usr/share/perl5) at 
./src/tools/win32tzlist.pl line 21.
BEGIN failed--compilation aborted at ./src/tools/win32tzlist.pl line 21.

https://brekka.postgresql.org/cgi-bin/show_log.pl?nm=koel&dt=2023-12-20%2013%3A19%3A04

This could perhaps be related to this patch removing the module in
src/tools/msvc/dummylib/Win32/Registry.pm ?

--
Daniel Gustafsson





Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andres Freund
On 2023-12-20 16:15:55 +0530, vignesh C wrote:
> On Tue, 19 Dec 2023 at 20:54, Peter Eisentraut  wrote:
> >
> > On 18.12.23 14:52, Peter Eisentraut wrote:
> > >> 2) I had seen that if sed/gzip is not available meson build will fail:
> > >> 2.a)
> > >> Program gsed sed found: NO
> > >> meson.build:334:6: ERROR: Program 'gsed sed' not found or not executable
> > >
> > > Yes, this would need to be improved.  Currently, sed is only required if
> > > either selinux or dtrace is enabled, which isn't supported on Windows.
> > > But we should adjust the build scripts to not fail the top-level setup
> > > run unless those options are enabled.
> > >
> > >> 2.b)
> > >> Program gzip found: NO
> > >> meson.build:337:7: ERROR: Program 'gzip' not found or not executable
> > >
> > > gzip is only required for certain test suites, so again we should adjust
> > > the build scripts to not fail the build but instead skip the tests as
> > > appropriate.
> >
> > Here are patches for these two issues.  More testing would be appreciated.
> 
> Thanks for the patches, Windows build is successful without these binaries.
> In linux when I try with Dtrace enabled, it throws the following error:
> Compiler for C supports arguments -fPIC: YES
> Compiler for C supports link arguments -Wl,--as-needed: YES
> Configuring pg_config_paths.h using configuration
> 
> src/include/utils/meson.build:39:2: ERROR: Tried to use not-found
> external program in "command"
> 
> With Dtrace enabled we should throw the original error that we were
> getting i.e.:
> ERROR: Program sed not found or not executable

I think the problem is that the current formulation in the patches doesn't
with deal with dtrace=auto. I think we ought to make that in a proper feature
checking block instead of just checking the presence of the dtrace binary.

Hm, or perhaps we should just get rid of sed use altogether. The sepgsql case
is trivially translateable to perl, and postprocess_dtrace.sed isn't
much harder.


OTOH, I actually don't think it's valid to not have sed when you have
dtrace. Erroring out in a weird way in such an artificially constructed test
doesn't really seem like a problem.


> Another observation is that we could include the executable name in
> this case something like:
> ERROR: Tried to use not-found external program "sed" in "command"

It's a meson generated message, so you'd need to open a bug report / feature
request for it.

Greetings,

Andres Freund




Re: Remove MSVC scripts from the tree

2023-12-20 Thread vignesh C
On Tue, 19 Dec 2023 at 20:54, Peter Eisentraut  wrote:
>
> On 18.12.23 14:52, Peter Eisentraut wrote:
> >> 2) I had seen that if sed/gzip is not available meson build will fail:
> >> 2.a)
> >> Program gsed sed found: NO
> >> meson.build:334:6: ERROR: Program 'gsed sed' not found or not executable
> >
> > Yes, this would need to be improved.  Currently, sed is only required if
> > either selinux or dtrace is enabled, which isn't supported on Windows.
> > But we should adjust the build scripts to not fail the top-level setup
> > run unless those options are enabled.
> >
> >> 2.b)
> >> Program gzip found: NO
> >> meson.build:337:7: ERROR: Program 'gzip' not found or not executable
> >
> > gzip is only required for certain test suites, so again we should adjust
> > the build scripts to not fail the build but instead skip the tests as
> > appropriate.
>
> Here are patches for these two issues.  More testing would be appreciated.

Thanks for the patches, Windows build is successful without these binaries.
In linux when I try with Dtrace enabled, it throws the following error:
Compiler for C supports arguments -fPIC: YES
Compiler for C supports link arguments -Wl,--as-needed: YES
Configuring pg_config_paths.h using configuration

src/include/utils/meson.build:39:2: ERROR: Tried to use not-found
external program in "command"

With Dtrace enabled we should throw the original error that we were
getting i.e.:
ERROR: Program sed not found or not executable

Another observation is that we could include the executable name in
this case something like:
ERROR: Tried to use not-found external program "sed" in "command"

Regards,
Vignesh




Re: Remove MSVC scripts from the tree

2023-12-20 Thread Andres Freund
On 2023-12-20 09:48:37 +0900, Michael Paquier wrote:
> On Mon, Nov 20, 2023 at 05:03:28PM +0900, Michael Paquier wrote:
> > Your suggestion to create a new sect2 for "Windows" as much as Andres'
> > suggestion are OK by as an intermediate step, and I suspect that the
> > end result will likely not be that.
> 
> It took me some time to get back to this one, and just applied the
> patch removing the scripts.

Wohooo!




Re: Remove MSVC scripts from the tree

2023-12-19 Thread Peter Eisentraut

On 20.12.23 02:14, Michael Paquier wrote:

Hmm.  Interesting.  So this basically comes down to the fact that GZIP
and TAR are required in ./configure because distcheck has a hard
dependency on both, but we don't support this target in meson.  Is
that right?


No, the issue is that gzip and tar are not required by configure (it 
will proceed if they are not found), but they are currently required by 
meson.build (it will error if they are not found).


They are used in two different areas.  One is for "make dist", but that 
doesn't affect meson anyway.


The other is various test suites.  The test suites are already set up to 
skip tests when gzip and tar are not found.






Re: Remove MSVC scripts from the tree

2023-12-19 Thread Peter Eisentraut

On 19.12.23 17:44, Nazir Bilal Yavuz wrote:

I think we need to require sed when dtrace or selinux is found, not by
looking at the return value of the get_option().enabled().


Right.  I think the correct condition would be

sed = find_program(get_option('SED'), 'sed', native: true,
   required: dtrace.found() or selinux.found())

I was trying to avoid that because it would require moving the 
find_program() to somewhere later in the top-level meson.build, but I 
suppose we have to do it that way.






Re: Remove MSVC scripts from the tree

2023-12-19 Thread Michael Paquier
On Tue, Dec 19, 2023 at 04:24:02PM +0100, Peter Eisentraut wrote:
> Here are patches for these two issues.  More testing would be appreciated.
>
> --- a/contrib/basebackup_to_shell/meson.build
> +++ b/contrib/basebackup_to_shell/meson.build
> @@ -24,7 +24,7 @@ tests += {
>  'tests': [
>'t/001_basic.pl',
>  ],
> -'env': {'GZIP_PROGRAM': gzip.path(),
> -'TAR': tar.path()},
> +'env': {'GZIP_PROGRAM': gzip.found() ? gzip.path() : '',
> +'TAR': tar.found() ? tar.path() : '' },
>},

Hmm.  Interesting.  So this basically comes down to the fact that GZIP
and TAR are required in ./configure because distcheck has a hard
dependency on both, but we don't support this target in meson.  Is
that right?
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-12-19 Thread Michael Paquier
On Mon, Nov 20, 2023 at 05:03:28PM +0900, Michael Paquier wrote:
> Your suggestion to create a new sect2 for "Windows" as much as Andres'
> suggestion are OK by as an intermediate step, and I suspect that the
> end result will likely not be that.

It took me some time to get back to this one, and just applied the
patch removing the scripts.

At the end, I have gone with the addition of a subsection named
"Visual" for now in the platform-specific notes, keeping all the
information originally in install-windows.sgml the same.  A proposal
of patch to clean up the docs is on my TODO list for the next CF.
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-12-19 Thread Michael Paquier
On Mon, Dec 18, 2023 at 02:52:41PM +0100, Peter Eisentraut wrote:
> On 18.12.23 11:49, vignesh C wrote:
>> Few comments:
>> 1) Now that the MSVC build scripts are removed, should we have the
>> reference to "MSVC build scripts" here?
>> ltree.h:
> 
> I think this note is correct and can be kept, as it explains the historical
> context.

Yeah, that's something I was pondering about for a bit a few weeks ago
but keeping the historical context is still the most important piece
to me.

>> 2) I had seen that if sed/gzip is not available meson build will fail:
>> 2.a)
>> Program gsed sed found: NO
>> meson.build:334:6: ERROR: Program 'gsed sed' not found or not executable
> 
> Yes, this would need to be improved.  Currently, sed is only required if
> either selinux or dtrace is enabled, which isn't supported on Windows. But
> we should adjust the build scripts to not fail the top-level setup run
> unless those options are enabled.
> 
>> 2.b)
>> Program gzip found: NO
>> meson.build:337:7: ERROR: Program 'gzip' not found or not executable
> 
> gzip is only required for certain test suites, so again we should adjust the
> build scripts to not fail the build but instead skip the tests as
> appropriate.

Oops.
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-12-19 Thread Nazir Bilal Yavuz
Hi,

On Tue, 19 Dec 2023 at 18:24, Peter Eisentraut  wrote:
>
> On 18.12.23 14:52, Peter Eisentraut wrote:
> >> 2) I had seen that if sed/gzip is not available meson build will fail:
> >> 2.a)
> >> Program gsed sed found: NO
> >> meson.build:334:6: ERROR: Program 'gsed sed' not found or not executable
> >
> > Yes, this would need to be improved.  Currently, sed is only required if
> > either selinux or dtrace is enabled, which isn't supported on Windows.
> > But we should adjust the build scripts to not fail the top-level setup
> > run unless those options are enabled.
> >
> >> 2.b)
> >> Program gzip found: NO
> >> meson.build:337:7: ERROR: Program 'gzip' not found or not executable
> >
> > gzip is only required for certain test suites, so again we should adjust
> > the build scripts to not fail the build but instead skip the tests as
> > appropriate.
>
> Here are patches for these two issues.  More testing would be appreciated.

0001-meson-Require-sed-only-when-needed:

+sed = find_program(get_option('SED'), 'sed', native: true,
+   required: get_option('dtrace').enabled() or
get_option('selinux').enabled())

dtrace is disabled as default but selinux is set to auto. So, meson
could find selinux ( because of the auto ) and fail to find sed, then
compilation will fail with:
contrib/sepgsql/meson.build:34:19: ERROR: Tried to use not-found
external program in "command"

I think we need to require sed when dtrace or selinux is found, not by
looking at the return value of the get_option().enabled().

Second patch looks good to me.

-- 
Regards,
Nazir Bilal Yavuz
Microsoft




Re: Remove MSVC scripts from the tree

2023-12-19 Thread Tristan Partin

On Tue Dec 19, 2023 at 9:24 AM CST, Peter Eisentraut wrote:

On 18.12.23 14:52, Peter Eisentraut wrote:
>> 2) I had seen that if sed/gzip is not available meson build will fail:
>> 2.a)
>> Program gsed sed found: NO
>> meson.build:334:6: ERROR: Program 'gsed sed' not found or not executable
> 
> Yes, this would need to be improved.  Currently, sed is only required if 
> either selinux or dtrace is enabled, which isn't supported on Windows. 
> But we should adjust the build scripts to not fail the top-level setup 
> run unless those options are enabled.
> 
>> 2.b)

>> Program gzip found: NO
>> meson.build:337:7: ERROR: Program 'gzip' not found or not executable
> 
> gzip is only required for certain test suites, so again we should adjust 
> the build scripts to not fail the build but instead skip the tests as 
> appropriate.


Here are patches for these two issues.  More testing would be appreciated.


Meson looks good to me!

--
Tristan Partin
Neon (https://neon.tech)




Re: Remove MSVC scripts from the tree

2023-12-19 Thread Peter Eisentraut

On 18.12.23 14:52, Peter Eisentraut wrote:

2) I had seen that if sed/gzip is not available meson build will fail:
2.a)
Program gsed sed found: NO
meson.build:334:6: ERROR: Program 'gsed sed' not found or not executable


Yes, this would need to be improved.  Currently, sed is only required if 
either selinux or dtrace is enabled, which isn't supported on Windows. 
But we should adjust the build scripts to not fail the top-level setup 
run unless those options are enabled.



2.b)
Program gzip found: NO
meson.build:337:7: ERROR: Program 'gzip' not found or not executable


gzip is only required for certain test suites, so again we should adjust 
the build scripts to not fail the build but instead skip the tests as 
appropriate.


Here are patches for these two issues.  More testing would be appreciated.From 8c51d31676b8a5b9ee17ab3a04be142076f2a48f Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Tue, 19 Dec 2023 16:05:54 +0100
Subject: [PATCH 1/2] meson: Require sed only when needed

sed is required only if dtrace or selinux is enabled.  Otherwise, we
don't need it.  This avoids making sed a hard requirement on Windows.
---
 meson.build | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meson.build b/meson.build
index d3771690f0..b8c7ed8bf2 100644
--- a/meson.build
+++ b/meson.build
@@ -331,7 +331,8 @@ perl = find_program(get_option('PERL'), required: true, 
native: true)
 python = find_program(get_option('PYTHON'), required: true, native: true)
 flex = find_program(get_option('FLEX'), native: true, version: '>= 2.5.35')
 bison = find_program(get_option('BISON'), native: true, version: '>= 2.3')
-sed = find_program(get_option('SED'), 'sed', native: true)
+sed = find_program(get_option('SED'), 'sed', native: true,
+   required: get_option('dtrace').enabled() or 
get_option('selinux').enabled())
 prove = find_program(get_option('PROVE'), native: true, required: false)
 tar = find_program(get_option('TAR'), native: true)
 gzip = find_program(get_option('GZIP'), native: true)
-- 
2.43.0

From d95033439f60ea19c5a316f658f0016b6691a232 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Tue, 19 Dec 2023 16:20:41 +0100
Subject: [PATCH 2/2] meson: Make gzip and tar optional

They are only used for some tests.  The tests are already set to skip
as appropriate if they are not available.
---
 contrib/basebackup_to_shell/meson.build | 4 ++--
 meson.build | 4 ++--
 src/bin/pg_basebackup/meson.build   | 4 ++--
 src/bin/pg_dump/meson.build | 2 +-
 src/bin/pg_verifybackup/meson.build | 4 ++--
 5 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/contrib/basebackup_to_shell/meson.build 
b/contrib/basebackup_to_shell/meson.build
index a5488c3023..331ee1c9be 100644
--- a/contrib/basebackup_to_shell/meson.build
+++ b/contrib/basebackup_to_shell/meson.build
@@ -24,7 +24,7 @@ tests += {
 'tests': [
   't/001_basic.pl',
 ],
-'env': {'GZIP_PROGRAM': gzip.path(),
-'TAR': tar.path()},
+'env': {'GZIP_PROGRAM': gzip.found() ? gzip.path() : '',
+'TAR': tar.found() ? tar.path() : '' },
   },
 }
diff --git a/meson.build b/meson.build
index b8c7ed8bf2..c8a2630db8 100644
--- a/meson.build
+++ b/meson.build
@@ -334,8 +334,8 @@ bison = find_program(get_option('BISON'), native: true, 
version: '>= 2.3')
 sed = find_program(get_option('SED'), 'sed', native: true,
required: get_option('dtrace').enabled() or 
get_option('selinux').enabled())
 prove = find_program(get_option('PROVE'), native: true, required: false)
-tar = find_program(get_option('TAR'), native: true)
-gzip = find_program(get_option('GZIP'), native: true)
+tar = find_program(get_option('TAR'), native: true, required: false)
+gzip = find_program(get_option('GZIP'), native: true, required: false)
 program_lz4 = find_program(get_option('LZ4'), native: true, required: false)
 openssl = find_program(get_option('OPENSSL'), native: true, required: false)
 program_zstd = find_program(get_option('ZSTD'), native: true, required: false)
diff --git a/src/bin/pg_basebackup/meson.build 
b/src/bin/pg_basebackup/meson.build
index c426173db3..5445903a5b 100644
--- a/src/bin/pg_basebackup/meson.build
+++ b/src/bin/pg_basebackup/meson.build
@@ -80,8 +80,8 @@ tests += {
   'sd': meson.current_source_dir(),
   'bd': meson.current_build_dir(),
   'tap': {
-'env': {'GZIP_PROGRAM': gzip.path(),
-'TAR': tar.path(),
+'env': {'GZIP_PROGRAM': gzip.found() ? gzip.path() : '',
+'TAR': tar.found() ? tar.path() : '',
 'LZ4': program_lz4.found() ? program_lz4.path() : '',
 },
 'tests': [
diff --git a/src/bin/pg_dump/meson.build b/src/bin/pg_dump/meson.build
index b6603e26a5..77d162cad4 100644
--- a/src/bin/pg_dump/meson.build
+++ b/src/bin/pg_dump/meson.build
@@ -90,7 +90,7 @@ tests += {
   'bd': meson.current_build_dir(),
   'tap': {
 'env': {
-  'GZIP_PROGRAM': gzip.path(),
+  'GZIP_P

Re: Remove MSVC scripts from the tree

2023-12-18 Thread Peter Eisentraut

On 18.12.23 11:49, vignesh C wrote:

Few comments:
1) Now that the MSVC build scripts are removed, should we have the
reference to "MSVC build scripts" here?
ltree.h:


I think this note is correct and can be kept, as it explains the 
historical context.



2) I had seen that if sed/gzip is not available meson build will fail:
2.a)
Program gsed sed found: NO
meson.build:334:6: ERROR: Program 'gsed sed' not found or not executable


Yes, this would need to be improved.  Currently, sed is only required if 
either selinux or dtrace is enabled, which isn't supported on Windows. 
But we should adjust the build scripts to not fail the top-level setup 
run unless those options are enabled.



2.b)
Program gzip found: NO
meson.build:337:7: ERROR: Program 'gzip' not found or not executable


gzip is only required for certain test suites, so again we should adjust 
the build scripts to not fail the build but instead skip the tests as 
appropriate.






Re: Remove MSVC scripts from the tree

2023-12-18 Thread vignesh C
On Wed, 6 Dec 2023 at 12:59, Michael Paquier  wrote:
>
> On Wed, Dec 06, 2023 at 12:15:50PM +0530, Shubham Khanna wrote:
> > Patch is not applying. Please share the Rebased Version. Please find the 
> > error:
>
> Thanks.  Here you go with a v6.

Few comments:
1) Now that the MSVC build scripts are removed, should we have the
reference to "MSVC build scripts" here?
ltree.h:
.
/*
 * LOWER_NODE used to be defined in the Makefile via the compile flags.
 * However the MSVC build scripts neglected to do the same which resulted in
 * MSVC builds not using LOWER_NODE.  Since then, the MSVC scripts have been
 * modified to look for -D compile flags in Makefiles, so here, in order to
 * get the historic behavior of LOWER_NODE not being defined on MSVC, we only
 * define it when not building in that environment.  This is important as we
 * want to maintain the same LOWER_NODE behavior after a pg_upgrade.
 */
#ifndef _MSC_VER
#define LOWER_NODE
#endif
.

2) I had seen that if sed/gzip is not available meson build will fail:
2.a)
Program gsed sed found: NO
meson.build:334:6: ERROR: Program 'gsed sed' not found or not executable

2.b)
Program gzip found: NO
meson.build:337:7: ERROR: Program 'gzip' not found or not executable

Should we mention sed and gzip here?
+  
+   Bison and
+Flex
+   
+   
+Bison and
Flex are
+required.  Only Bison versions 2.3 and later
+will work. Flex must be version
2.5.35 or later.
+   

Regards,
Vignesh




Re: Remove MSVC scripts from the tree

2023-12-16 Thread Michael Paquier
On Thu, Dec 14, 2023 at 11:43:14AM +0900, NINGWEI CHEN wrote:
> Sorry for the delayed response. 
> We are currently working on transitioning to meson build at hamerkop and 
> anticipating that this can be accomplished by no later than January.
> 
> If the old build scripts are removed before that, hamerkop will be 
> temporarily 
> taken off the master branch, and will rejoin once the adjustment is done.

Thanks for the update.  Let's move on with that on HEAD then.  I've
wanted some room to work on improving the set of docs for v17.
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-12-13 Thread NINGWEI CHEN
On Mon, 4 Dec 2023 17:05:24 +0900
Michael Paquier  wrote:

> On Tue, Sep 26, 2023 at 12:17:04PM -0400, Andrew Dunstan wrote:
> > On 2023-09-26 Tu 01:25, NINGWEI CHEN wrote:
> >> hamerkop is not yet prepared for Meson builds, but we plan to work on this 
> >> support soon.
> >> If we go with Meson builds exclusively right now, we will have to 
> >> temporarily remove the master/HEAD for a while.
> > 
> > I don't think we should switch to that until you're ready.
> 
> Agreed that it would just be breaking a build for the sake of breaking
> it.  Saying that, the last exchange that we had about hamerkop
> switching to meson was two months ago.  Are there any plans to do the
> switch?
> --
> Michael


Sorry for the delayed response. 
We are currently working on transitioning to meson build at hamerkop and 
anticipating that this can be accomplished by no later than January.

If the old build scripts are removed before that, hamerkop will be temporarily 
taken off the master branch, and will rejoin once the adjustment is done.


Best Regards.
-- 
SRA OSS LLC
Chen Ningwei




Re: Remove MSVC scripts from the tree

2023-12-13 Thread Andrew Dunstan



On 2023-12-13 We 09:23, Michael Paquier wrote:

On Tue, Dec 05, 2023 at 07:29:59AM +0900, Michael Paquier wrote:

Okay.  Thanks for the update.

While in Prague, Andres and Peter E. have mentioned me that we perhaps
had better move on with this patch sooner than later, without waiting
for the two buildfarm members to do the switch because much more
cleanup is required for the documentation once the scripts are
removed.

So, any objections with the patch as presented to remove the scripts
while moving the existing doc blocks from install-windows.sgml that
still need more discussion?




TBH I'd prefer to wait. But I have had a couple more urgent things on my 
plate. I hope to get back to it before New Year. In the meantime I have 
switched bowerbird to building only STABLE branches.



cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: Remove MSVC scripts from the tree

2023-12-13 Thread Michael Paquier
On Tue, Dec 05, 2023 at 07:29:59AM +0900, Michael Paquier wrote:
> Okay.  Thanks for the update.

While in Prague, Andres and Peter E. have mentioned me that we perhaps
had better move on with this patch sooner than later, without waiting
for the two buildfarm members to do the switch because much more
cleanup is required for the documentation once the scripts are
removed.

So, any objections with the patch as presented to remove the scripts
while moving the existing doc blocks from install-windows.sgml that
still need more discussion?
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-12-07 Thread Michael Paquier
On Fri, Dec 08, 2023 at 08:50:47AM +1300, Thomas Munro wrote:
> . o O { I wish master would systematically drop support for compilers
> that were out of 'mainstream' vendor support. }

Calling for a patch once, twice ;p 

FWIW, I would not mind marking VS 2019 as the minimum requirement on
HEAD once the MSVC scripts are gone.  The oldest VS version tested in
the buildfarm is hamerkop with VS2017, still under the MSVC scripts.
I'd like to believe that a switch to meson implies a newer version of
VS installed there.
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-12-07 Thread Thomas Munro
On Thu, Dec 7, 2023 at 5:27 AM Andrew Dunstan  wrote:
> But the VS2019 compiler implements enough of C99 to pass our meson test,
> unlike VS2017. Maybe the test is too strict. After all, we know we can
> in fact build with the earlier versions.

. o O { I wish master would systematically drop support for compilers
that were out of 'mainstream' vendor support. }




Re: Remove MSVC scripts from the tree

2023-12-07 Thread Andres Freund
Hi,

On 2023-12-07 13:49:44 -0500, Andrew Dunstan wrote:
> On 2023-12-07 Th 12:34, Andres Freund wrote:
> > However: I used godbolt to compile the test code on msvc, and it seems to
> > build with 19.15 (which is the version Andrew referenced upthread), with a
> > warning that's triggered independent of the structfunc bit.
> > 
> > https://godbolt.org/z/j99E9MeEK
> > 
> > 
> > Andrew, could you attach meson.log from the failed build?
> > 
> > 
> 
> The odd thing is I tried to reproduce the issue and instead it's now
> compiling with VS2017. The only thing I have changed on the machine was to
> install VS2022 alongside VS2017, as well as switching which perl to link to,
> which should have no effect on this.

The error might have been due to an older C runtime. I think installing visual
studio 2022 might have lead to updating the C runtime associated with VS 2017
to a newer release of VS 2017. Or even updated the version of VS 2017 - I find
the version numbers of msvc vs visual studio incomprehensible, but I think
19.15.26726 is from 2017, missing a lot of bugfixes that were made to VS 2017
since.

Greetings,

Andres Freund




Re: Remove MSVC scripts from the tree

2023-12-07 Thread Andrew Dunstan



On 2023-12-07 Th 12:34, Andres Freund wrote:

Hi,

On 2023-12-07 12:33:35 +0100, Alvaro Herrera wrote:

Well, We have things like these

typedef struct _archiveOpts
{
...
} ArchiveOpts;
#define ARCHIVE_OPTS(...) &(ArchiveOpts){__VA_ARGS__}

XL_ROUTINE is quite similar.

These are then used like
  ARCHIVE_OPTS(.tag = "pg_largeobject",
   .description = "pg_largeobject",
   .section = SECTION_PRE_DATA,
   .createStmt = loOutQry->data));

so the difference is that we're passing a pointer to a struct, not
the struct bare, which is what c99_test is doing:

struct named_init_test {
   int a;
   int b;
};

int main() {
   ...
   structfunc((struct named_init_test){1, 0});
}

Maybe this would work if the function received the pointer too?

extern void structfunc(struct named_init_test *);

   structfunc(&(struct named_init_test){1, 0});

The fact that this is called "structfunc" makes me wonder if the author
did indeed want to test passing a struct to the function.  That'd be
odd, since the interesting thing in this line is the expression used to
initialize the struct argument.  (We do pass structs, eg. ObjectAddress
to check_object_ownership; old code.)

It seems like both might be interesting?  But I think there's no reason to not
evolve this test if we need to. I think I wrote it testing with a few old *nix
compilers to see where -std=c99 was needed, not more. It's not too surprising
that it might need some massaging for older msvc...


However: I used godbolt to compile the test code on msvc, and it seems to
build with 19.15 (which is the version Andrew referenced upthread), with a
warning that's triggered independent of the structfunc bit.

https://godbolt.org/z/j99E9MeEK


Andrew, could you attach meson.log from the failed build?




The odd thing is I tried to reproduce the issue and instead it's now 
compiling with VS2017. The only thing I have changed on the machine was 
to install VS2022 alongside VS2017, as well as switching which perl to 
link to, which should have no effect on this.



So never mind, we make progress.


Not sure about VS 2015 though.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: Remove MSVC scripts from the tree

2023-12-07 Thread Andres Freund
Hi,

On 2023-12-07 12:33:35 +0100, Alvaro Herrera wrote:
> Well, We have things like these
> 
> typedef struct _archiveOpts
> {
>   ...
> } ArchiveOpts;
> #define ARCHIVE_OPTS(...) &(ArchiveOpts){__VA_ARGS__}
> 
> XL_ROUTINE is quite similar.
> 
> These are then used like
>  ARCHIVE_OPTS(.tag = "pg_largeobject",
>   .description = "pg_largeobject",
>   .section = SECTION_PRE_DATA,
>   .createStmt = loOutQry->data));
> 
> so the difference is that we're passing a pointer to a struct, not
> the struct bare, which is what c99_test is doing:
> 
> struct named_init_test {
>   int a;
>   int b;
> };
> 
> int main() {
>   ...
>   structfunc((struct named_init_test){1, 0});
> }
> 
> Maybe this would work if the function received the pointer too?
> 
> extern void structfunc(struct named_init_test *);
> 
>   structfunc(&(struct named_init_test){1, 0});
> 
> The fact that this is called "structfunc" makes me wonder if the author
> did indeed want to test passing a struct to the function.  That'd be
> odd, since the interesting thing in this line is the expression used to
> initialize the struct argument.  (We do pass structs, eg. ObjectAddress
> to check_object_ownership; old code.)

It seems like both might be interesting?  But I think there's no reason to not
evolve this test if we need to. I think I wrote it testing with a few old *nix
compilers to see where -std=c99 was needed, not more. It's not too surprising
that it might need some massaging for older msvc...


However: I used godbolt to compile the test code on msvc, and it seems to
build with 19.15 (which is the version Andrew referenced upthread), with a
warning that's triggered independent of the structfunc bit.

https://godbolt.org/z/j99E9MeEK


Andrew, could you attach meson.log from the failed build?


Greetings,

Andres Freund




Re: Remove MSVC scripts from the tree

2023-12-07 Thread Andres Freund
Hi,

On 2023-12-04 15:11:47 -0500, Andrew Dunstan wrote:
> I just had a look at shifting bowerbird to use meson, and it got stymied at
> the c99 test, which apparently doesn't compile with anything less than
> VS2019.

What error or warning is being raised by msvc?

Andres




Re: Remove MSVC scripts from the tree

2023-12-07 Thread Alvaro Herrera
On 2023-Dec-07, Peter Eisentraut wrote:

> On 06.12.23 21:52, Andrew Dunstan wrote:

> > Yes, if I comment out the call to structfunc() the test passes on VS2017
> > (compiler version 19.15.26726)
> 
> This is strange, because we use code like that in the tree.  There must be
> some small detail that trips it up here.

Well, We have things like these

typedef struct _archiveOpts
{
...
} ArchiveOpts;
#define ARCHIVE_OPTS(...) &(ArchiveOpts){__VA_ARGS__}

XL_ROUTINE is quite similar.

These are then used like
 ARCHIVE_OPTS(.tag = "pg_largeobject",
  .description = "pg_largeobject",
  .section = SECTION_PRE_DATA,
  .createStmt = loOutQry->data));

so the difference is that we're passing a pointer to a struct, not
the struct bare, which is what c99_test is doing:

struct named_init_test {
  int a;
  int b;
};

int main() {
  ...
  structfunc((struct named_init_test){1, 0});
}

Maybe this would work if the function received the pointer too?


extern void structfunc(struct named_init_test *);

  structfunc(&(struct named_init_test){1, 0});

The fact that this is called "structfunc" makes me wonder if the author
did indeed want to test passing a struct to the function.  That'd be
odd, since the interesting thing in this line is the expression used to
initialize the struct argument.  (We do pass structs, eg. ObjectAddress
to check_object_ownership; old code.)

-- 
Álvaro HerreraBreisgau, Deutschland  —  https://www.EnterpriseDB.com/
"No renuncies a nada. No te aferres a nada."




Re: Remove MSVC scripts from the tree

2023-12-06 Thread Peter Eisentraut

On 06.12.23 21:52, Andrew Dunstan wrote:


On 2023-12-06 We 12:24, Peter Eisentraut wrote:

On 06.12.23 17:27, Andrew Dunstan wrote:
But the VS2019 compiler implements enough of C99 to pass our meson 
test, unlike VS2017. Maybe the test is too strict. After all, we know 
we can in fact build with the earlier versions.


I just realized that the C99 test is actually our own, not provided by 
meson.  (See "c99_test" in meson.build.)


Can you try disabling a few bits of that to see what makes it pass for 
you?  I suspect it's the structfunc() call.



Yes, if I comment out the call to structfunc() the test passes on VS2017 
(compiler version 19.15.26726)


This is strange, because we use code like that in the tree.  There must 
be some small detail that trips it up here.


Perhaps try moving the definition of struct named_init_test outside of 
the function, or make it a typedef.





Re: Remove MSVC scripts from the tree

2023-12-06 Thread Andrew Dunstan



On 2023-12-06 We 12:24, Peter Eisentraut wrote:

On 06.12.23 17:27, Andrew Dunstan wrote:
But the VS2019 compiler implements enough of C99 to pass our meson 
test, unlike VS2017. Maybe the test is too strict. After all, we know 
we can in fact build with the earlier versions.


I just realized that the C99 test is actually our own, not provided by 
meson.  (See "c99_test" in meson.build.)


Can you try disabling a few bits of that to see what makes it pass for 
you?  I suspect it's the structfunc() call.



Yes, if I comment out the call to structfunc() the test passes on VS2017 
(compiler version 19.15.26726)



cheers


andrew


--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: Remove MSVC scripts from the tree

2023-12-06 Thread Peter Eisentraut

On 06.12.23 17:27, Andrew Dunstan wrote:
But the VS2019 compiler implements enough of C99 to pass our meson test, 
unlike VS2017. Maybe the test is too strict. After all, we know we can 
in fact build with the earlier versions.


I just realized that the C99 test is actually our own, not provided by 
meson.  (See "c99_test" in meson.build.)


Can you try disabling a few bits of that to see what makes it pass for 
you?  I suspect it's the structfunc() call.






Re: Remove MSVC scripts from the tree

2023-12-06 Thread Andrew Dunstan



On 2023-12-06 We 01:18, Peter Eisentraut wrote:

On 04.12.23 21:11, Andrew Dunstan wrote:
I just had a look at shifting bowerbird to use meson, and it got 
stymied at the c99 test, which apparently doesn't compile with 
anything less than VS2019.


If that is the case, then wouldn't that invalidate the documented 
claim that you can build with VS2015 or newer?



Indeed it would.

Here's what the Microsoft site says at 
:



You can invoke the Microsoft C compiler by using the /TC or /Tc 
compiler option. It's used by default for code that has a .c file 
extension, unless overridden by a /TP or /Tp option. The default C 
compiler (that is, the compiler when /std:c11 or /std:c17 isn't 
specified) implements ANSI C89, but includes several Microsoft 
extensions, some of which are part of ISO C99. Some Microsoft 
extensions to C89 can be disabled by using the /Za compiler option, 
but others remain in effect. It isn't possible to specify strict C89 
conformance. The compiler doesn't implement several required features 
of C99, so it isn't possible to specify C99 conformance, either.


But the VS2019 compiler implements enough of C99 to pass our meson test, 
unlike VS2017. Maybe the test is too strict. After all, we know we can 
in fact build with the earlier versions.



cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: Remove MSVC scripts from the tree

2023-12-05 Thread Shubham Khanna
On Fri, Nov 17, 2023 at 6:31 AM Michael Paquier  wrote:
>
> On Wed, Nov 15, 2023 at 05:07:03PM -0800, Andres Freund wrote:
> > On 2023-11-15 13:49:06 +0900, Michael Paquier wrote:
> > Where "Windows" actually seems to solely describe visual studio? That seems
> > confusing.
>
> Yeah, switch that to Visual.
>
> > Huh, so this was wrong since the code was added? For a moment I thought I'd
> > unintentionally promoted it to be built by default, but ...
>
> Yes, I was wondering if there could be an argument for simplifying
> some code here by pushing more logic into this wrapper, but I'm
> finding that a bit unappealing, and building it under Visual has no
> actual consequence: it seems that we never call pg_strsignal() under
> WIN32.
>
> >> -# FIXME: copied from Mkvcbuild.pm, but I don't think that's the right 
> >> approach
> >>  pgevent_link_args = []
> >>  if cc.get_id() == 'msvc'
> >>pgevent_link_args += '/ignore:4104'
> >
> > I think it's worth leaving a trail indicating that adding this
> > warning-suppression is dubious at best.  It seems to pretty obviously paper
> > over us exporting the symbols the wrong way:
> > https://learn.microsoft.com/en-us/cpp/error-messages/tool-errors/linker-tools-warning-lnk4104?view=msvc-170
> >
> > Which pretty clearly explains that pgevent.def is wrong.
> >
> > I just can't really test it, nor does it have test. Otherwise I might have
> > fixed it.
>
> Agreed that there is a good argument for removing it at some point,
> with a separate investigation.  I've just added a XXX comment for now.
>
> >> @@ -53,10 +53,25 @@ AC_DEFUN([PGAC_CHECK_PERL_CONFIGS],
> >>  # would be fatal to try to compile PL/Perl to a different libc ABI than 
> >> core
> >>  # Postgres uses.  The available information says that most symbols that 
> >> affect
> >>  # Perl's own ABI begin with letters, so it's almost sufficient to adopt -D
> >> -# switches for symbols not beginning with underscore.  Some exceptions 
> >> are the
> >> -# Windows-specific -D_USE_32BIT_TIME_T and -D__MINGW_USE_VC2005_COMPAT; 
> >> see
> >> -# Mkvcbuild.pm for details.  We absorb the former when Perl reports it.  
> >> Perl
> >> -# never reports the latter, and we don't attempt to deduce when it's 
> >> needed.
> >> +# switches for symbols not beginning with underscore.
> >> +
> >> +# Some exceptions are the Windows-specific -D_USE_32BIT_TIME_T and
> >> +# -D__MINGW_USE_VC2005_COMPAT. To be exact, Windows offers several 32-bit 
> >> ABIs.
> >> +# Perl is sensitive to sizeof(time_t), one of the ABI dimensions.  To get
> >> +# 32-bit time_t, use "cl -D_USE_32BIT_TIME_T" or plain "gcc".  For 64-bit
> >> +# time_t, use "gcc -D__MINGW_USE_VC2005_COMPAT" or plain "cl".  Before 
> >> MSVC
> >> +# 2005, plain "cl" chose 32-bit time_t.  PostgreSQL doesn't support 
> >> building
> >> +# with pre-MSVC-2005 compilers, but it does support linking to Perl built 
> >> with
> >> +# such a compiler.  MSVC-built Perl 5.13.4 and later report 
> >> -D_USE_32BIT_TIME_T
> >> +# in $Config{ccflags} if applicable, but MinGW-built Perl never reports
> >> +# -D_USE_32BIT_TIME_T despite typically needing it.
> >
> > Hm, it's pretty odd to have comments about cl.exe here, given that it can't
> > even be used with msvc.
> >
> > My impression from testing this is that absorbing the flag from perl 
> > suffices
> > with strawberry perl and mingw perl, both when building with mingw and msvc.
>
> I was a bit uncomfortable with removing these references, but I
> suspect that you are right and that they're outdated artifacts of the
> past.  So I'm OK to remove the cl and gcc parts as the flags come from
> $PERL.
>
> >> +# Ignore the $Config{ccflags} opinion about -D_USE_32BIT_TIME_T, and use a
> >> +# runtime test to deduce the ABI Perl expects.  Specifically, test use of
> >> +# PL_modglobal, which maps to a PerlInterpreter field whose position 
> >> depends
> >> +# on sizeof(time_t).  We absorb the former when Perl reports it.  Perl 
> >> never
> >> +# reports the latter, and we don't attempt to deduce when it's needed.
> >
> > I don't think this is implemented anywhere now?
>
> Indeed, that's now gone.
>
> >> +   
> >> +PostgreSQL for Windows can be built using meson, as described
> >> +in .
> >> +The native Windows port requires a 32 or 64-bit version of Windows
> >> +2000 or later. Earlier operating systems do
> >> +not have sufficient infrastructure (but Cygwin may be used on
> >> +those).
> >> +   
> >
> > Is this actually true? I don't think we build on win2k...
>
> Nah, this is a reference outdated for ages.  495ed0ef2d72 has even
> bumped _WIN32_WINNT to require Windows 10 as the minimal runtime
> version supported, so this needs to be updated and backpatched.  The
> first two sentences can be simplified like that:
> -The native Windows port requires a 32 or 64-bit version of Windows
> -2000 or later. Earlier operating systems do
> -not have sufficient infrastructure (but Cygwin may be used on
> -those)

Re: Remove MSVC scripts from the tree

2023-12-05 Thread Peter Eisentraut

On 04.12.23 21:11, Andrew Dunstan wrote:
I just had a look at shifting bowerbird to use meson, and it got stymied 
at the c99 test, which apparently doesn't compile with anything less 
than VS2019.


If that is the case, then wouldn't that invalidate the documented claim 
that you can build with VS2015 or newer?






Re: Remove MSVC scripts from the tree

2023-12-04 Thread Michael Paquier
On Mon, Dec 04, 2023 at 03:11:47PM -0500, Andrew Dunstan wrote:
> I just had a look at shifting bowerbird to use meson, and it got stymied at
> the c99 test, which apparently doesn't compile with anything less than
> VS2019.
> 
> I can upgrade bowerbird, but that will take rather longer. It looks like
> hamerkop is in th4e same boat.

Okay.  Thanks for the update.
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-12-04 Thread Andrew Dunstan



On 2023-12-04 Mo 03:05, Michael Paquier wrote:

On Tue, Sep 26, 2023 at 12:17:04PM -0400, Andrew Dunstan wrote:

On 2023-09-26 Tu 01:25, NINGWEI CHEN wrote:

hamerkop is not yet prepared for Meson builds, but we plan to work on this 
support soon.
If we go with Meson builds exclusively right now, we will have to temporarily 
remove the master/HEAD for a while.

I don't think we should switch to that until you're ready.

Agreed that it would just be breaking a build for the sake of breaking
it.  Saying that, the last exchange that we had about hamerkop
switching to meson was two months ago.  Are there any plans to do the
switch?



I just had a look at shifting bowerbird to use meson, and it got stymied 
at the c99 test, which apparently doesn't compile with anything less 
than VS2019.


I can upgrade bowerbird, but that will take rather longer. It looks like 
hamerkop is in th4e same boat.



cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: Remove MSVC scripts from the tree

2023-12-04 Thread Michael Paquier
On Tue, Sep 26, 2023 at 12:17:04PM -0400, Andrew Dunstan wrote:
> On 2023-09-26 Tu 01:25, NINGWEI CHEN wrote:
>> hamerkop is not yet prepared for Meson builds, but we plan to work on this 
>> support soon.
>> If we go with Meson builds exclusively right now, we will have to 
>> temporarily remove the master/HEAD for a while.
> 
> I don't think we should switch to that until you're ready.

Agreed that it would just be breaking a build for the sake of breaking
it.  Saying that, the last exchange that we had about hamerkop
switching to meson was two months ago.  Are there any plans to do the
switch?
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-11-20 Thread Michael Paquier
On Mon, Nov 20, 2023 at 08:00:09AM +0100, Peter Eisentraut wrote:
> I had suggested this arrangement as a way to reduce churn in this patch set.
> We'd just move over the existing separate chapter into a new section, and
> then later consider further rearrangements.
>
> It's not always clear where all of these things should go, as there are so
> many dimensions.  For example, the existing sentence "After you have
> everything installed, it is suggested that you run psql under CMD.EXE, as
> the MSYS console has buffering issues.", does that apply to MinGW, or really
> MSYS, or does it also apply if you build with Visual Something?

Even for this specific one, are you sure that it still applies?  :D

> Ultimately, I don't think MinGW needs to be its own section.

Yes, agreed.  The end result should be one single sect2 for Windows
divided into multiple sect3, perhaps themselves divided into more
sect4 for each build method.  As a whole, before refactoring all that,
I'd be in favor of a slightly different strategy once the MSVC scripts
and install-windows.sgml with its stuff specific to src/tools/msvc are
gone:
- Review all this section from the docs and trim them from everything
that we think is now irrelevant.
- Look at the rest and see how it can be efficiently refactored into
balanced sections.

Your suggestion to create a new sect2 for "Windows" as much as Andres'
suggestion are OK by as an intermediate step, and I suspect that the
end result will likely not be that.
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-11-19 Thread Peter Eisentraut

On 16.11.23 02:07, Andres Freund wrote:

It doesn't seem like your patch has it quite that way? I see

   
MinGW
...
   
Windows

Where "Windows" actually seems to solely describe visual studio? That seems
confusing.


I had suggested this arrangement as a way to reduce churn in this patch 
set.  We'd just move over the existing separate chapter into a new 
section, and then later consider further rearrangements.


It's not always clear where all of these things should go, as there are 
so many dimensions.  For example, the existing sentence "After you have 
everything installed, it is suggested that you run psql under CMD.EXE, 
as the MSYS console has buffering issues.", does that apply to MinGW, or 
really MSYS, or does it also apply if you build with Visual Something?


Ultimately, I don't think MinGW needs to be its own section.




Re: Remove MSVC scripts from the tree

2023-11-15 Thread Michael Paquier
On Wed, Nov 15, 2023 at 11:27:13AM +0100, Peter Eisentraut wrote:
> (Note, however, that your rebase didn't pick up commits e7814b40d0 and
> b41b1a7f49 that I did yesterday.  Please check that again.)

Indeed.  I need to absorb that properly.
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-11-15 Thread Peter Eisentraut

On 15.11.23 05:49, Michael Paquier wrote:

Attached is a v4.


I'm happy with that.

(Note, however, that your rebase didn't pick up commits e7814b40d0 and 
b41b1a7f49 that I did yesterday.  Please check that again.)






Re: Remove MSVC scripts from the tree

2023-11-14 Thread Peter Eisentraut
Other than the documentation details and the business about 
Gen_dummy_probes, which has been dealt with separately, this patch looks 
solid to me.


On 12.10.23 07:23, Michael Paquier wrote:

On Thu, Oct 05, 2023 at 09:38:51AM +0200, Peter Eisentraut wrote:

- I think we can't just delete install-windows.sgml.  Some of that content
needs to be moved over to installation.sgml.  As a simple example,
install-windows.sgml shows which MSVC versions are supported. That
information should surely be kept.


I've been thinking about the whole structure for a bit, but with the
MSVC scripts gone and the fact that we would rely on meson, moving
this information to the section under the platform-specific notes is
feeling kind of natural here.  Here is a possible split of the
information across several sections:
- The requirements:
-- ActiveState Perl could be entirely removed, IMO.  Perhaps we should
replace that to a reference to raspberry-perl, chocolatey or similar?
I am not sure about the best approach here, so for now I've kept the
bits about active perl.
-- bison and flex, which would become hard requirements on Windows
with Visual Studio now.  Perhaps this could be unified with the patch
for distprep later on, but here we have specifics for Windows.
-- All the other optional requirements, tcl, etc.
- MinGW notes.
- Visual Studio notes, with the versions of visual supported, download
links, and a bit more.
- Notes specific about 64b builds.

The attached is a bit crude and requires adjustments, but it shows the
idea.


It's tricky.  Eventually, we would like to reduce some of the 
duplication, like the whole list of requirements.  But there are some 
Windows-specific details in there, so I don't know.


My suggestion would be:

Make a new  titled "Windows" at 
the end of installation.sgml (after the Solaris section).  Dump most of 
the content from install-windows.sgml in there (except the stuff about 
the old build system).  Rename the existing section "MinGW/Native 
Windows" to just "MinGW" and make some minor adjustments, similar to 
your patch.


That way, we can move forward, and we can adjust and trim the details of 
the documentation over time.





Re: Remove MSVC scripts from the tree

2023-11-14 Thread Peter Eisentraut

On 13.11.23 06:30, Michael Paquier wrote:

On Fri, Nov 10, 2023 at 08:38:21AM +0100, Peter Eisentraut wrote:

How about this patch as a comprehensive solution?
  8 files changed, 26 insertions(+), 339 deletions(-)


Thanks for the patch.  The numbers are here, and the patch looks
sensible.

The contents of probes.h without --enable-trace are exactly the same
before and after the patch.

In short, +1 to switch to what you are proposing here.


done





Re: Remove MSVC scripts from the tree

2023-11-12 Thread Michael Paquier
On Fri, Nov 10, 2023 at 08:38:21AM +0100, Peter Eisentraut wrote:
> How about this patch as a comprehensive solution?
>  8 files changed, 26 insertions(+), 339 deletions(-)

Thanks for the patch.  The numbers are here, and the patch looks
sensible.

The contents of probes.h without --enable-trace are exactly the same
before and after the patch.

In short, +1 to switch to what you are proposing here.
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-11-09 Thread Peter Eisentraut

On 09.11.23 00:05, Michael Paquier wrote:

Attached is a Perl version of the sed script, converted by hand (so not the
super-verbose s2p thing).  It's basically just the sed script with
semicolons added and the backslashes in the regular expressions moved
around.  I think we could use something like that for all platforms now.


Sounds like a good idea to me now that perl is a hard requirement.
+1.


How about this patch as a comprehensive solution?
From 47731e486d7356462ce610a6a76c21a0b3619823 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Fri, 10 Nov 2023 08:07:11 +0100
Subject: [PATCH v1] Replace Gen_dummy_probes.sed with Gen_dummy_probes.pl

---
 .gitattributes   |   1 -
 src/backend/utils/Gen_dummy_probes.pl| 275 ++-
 src/backend/utils/Gen_dummy_probes.pl.prolog |  19 --
 src/backend/utils/Gen_dummy_probes.sed   |  24 --
 src/backend/utils/Makefile   |  15 +-
 src/backend/utils/README.Gen_dummy_probes|  27 --
 src/include/utils/meson.build|   2 +-
 src/tools/msvc/Solution.pm   |   2 +-
 8 files changed, 26 insertions(+), 339 deletions(-)
 delete mode 100644 src/backend/utils/Gen_dummy_probes.pl.prolog
 delete mode 100644 src/backend/utils/Gen_dummy_probes.sed
 delete mode 100644 src/backend/utils/README.Gen_dummy_probes

diff --git a/.gitattributes b/.gitattributes
index 2384956d88..55e6060405 100644
--- a/.gitattributes
+++ b/.gitattributes
@@ -14,7 +14,6 @@ README.*  conflict-marker-size=32
 *.data -whitespace
 contrib/pgcrypto/sql/pgp-armor.sql whitespace=-blank-at-eol
 src/backend/catalog/sql_features.txt   
whitespace=space-before-tab,blank-at-eof,-blank-at-eol
-src/backend/utils/Gen_dummy_probes.pl.prolog   whitespace=-blank-at-eof
 
 # Test output files that contain extra whitespace
 *.out  -whitespace
diff --git a/src/backend/utils/Gen_dummy_probes.pl 
b/src/backend/utils/Gen_dummy_probes.pl
index f289b19344..f6df82baa5 100644
--- a/src/backend/utils/Gen_dummy_probes.pl
+++ b/src/backend/utils/Gen_dummy_probes.pl
@@ -1,259 +1,28 @@
-#! /usr/bin/perl -w
 #-
+# Perl script to create dummy probes.h file when dtrace is not available
 #
-# Gen_dummy_probes.pl
-#Perl script that generates probes.h file when dtrace is not available
-#
-# Portions Copyright (c) 2008-2023, PostgreSQL Global Development Group
-#
-#
-# IDENTIFICATION
-#src/backend/utils/Gen_dummy_probes.pl
-#
-# This program was generated by running perl's s2p over Gen_dummy_probes.sed
+# Copyright (c) 2008-2023, PostgreSQL Global Development Group
 #
+# src/backend/utils/Gen_dummy_probes.pl
 #-
 
-# turn off perlcritic for autogenerated code
-## no critic
-
-$0 =~ s/^.*?(\w+)[\.\w+]*$/$1/;
-
 use strict;
-use Symbol;
-use vars qw{ $isEOF $Hold %wFiles @Q $CondReg
-  $doAutoPrint $doOpenWrite $doPrint };
-$doAutoPrint = 1;
-$doOpenWrite = 1;
-
-# prototypes
-sub openARGV();
-sub getsARGV(;\$);
-sub eofARGV();
-sub printQ();
-
-# Run: the sed loop reading input and applying the script
-#
-sub Run()
-{
-   my ($h, $icnt, $s, $n);
-
-   # hack (not unbreakable :-/) to avoid // matching an empty string
-   my $z = "\000";
-   $z =~ /$z/;
-
-   # Initialize.
-   openARGV();
-   $Hold = '';
-   $CondReg = 0;
-   $doPrint = $doAutoPrint;
-  CYCLE:
-   while (getsARGV())
-   {
-   chomp();
-   $CondReg = 0;# cleared on t
- BOS:;
-
-   # /^[   ]*probe /!d
-   unless (m /^[ \t]*probe /s)
-   {
-   $doPrint = 0;
-   goto EOS;
-   }
-
-   # s/^[  ]*probe \([^(]*\)\(.*\);/\1\2/
-   {
-   $s = s /^[ \t]*probe ([^(]*)(.*);/${1}${2}/s;
-   $CondReg ||= $s;
-   }
-
-   # s/__/_/g
-   {
-   $s = s /__/_/sg;
-   $CondReg ||= $s;
-   }
-
-   # y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/
-   { y{abcdefghijklmnopqrstuvwxyz}{ABCDEFGHIJKLMNOPQRSTUVWXYZ}; }
-
-   # s/^/#define TRACE_POSTGRESQL_/
-   {
-   $s = s /^/#define TRACE_POSTGRESQL_/s;
-   $CondReg ||= $s;
-   }
-
-   # s/([^,)]\{1,\})/(INT1)/
-   {
-   $s = s /\([^,)]+\)/(INT1)/s;
-   $CondReg ||= $s;
-   }
-
-   # s/([^,)]\{1,\}, [^,)]\{1,\})/(INT1, INT2)/
-   {
-   $s = s /\([^,)]+, [^,)]+\)/(INT1, INT2)/s;
-   $CondReg ||= $s;
-   }
-
-   # s/([^,)]\{1,\}, 

Re: Remove MSVC scripts from the tree

2023-11-08 Thread Michael Paquier
On Wed, Nov 08, 2023 at 09:41:19AM +0100, Peter Eisentraut wrote:
> I don't think we should rely on sed being there on Windows.  Maybe it's true
> now on the handful of buildfarm/CI machines and early adopters, but do we
> have any indication that that is systematic or just an accident?

Or both?  When doing builds based on MinGW in the past I vaguely
recall getting annoyed that I needed to look for sed as one thing, so
your suggestion could simplify the experience a bit.

> Since we definitely require Perl now, we could just as well use the Perl
> script and avoid this issue.
>
> Attached is a Perl version of the sed script, converted by hand (so not the
> super-verbose s2p thing).  It's basically just the sed script with
> semicolons added and the backslashes in the regular expressions moved
> around.  I think we could use something like that for all platforms now.

Sounds like a good idea to me now that perl is a hard requirement.
+1.
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-11-08 Thread Andrew Dunstan



On 2023-11-08 We 03:41, Peter Eisentraut wrote:

On 12.10.23 07:23, Michael Paquier wrote:
- Is src/backend/utils/README.Gen_dummy_probes still correct after 
this?
AFAICT, the Perl-based MSVC build system uses Gen_dummy_probes.pl, 
but the
meson build uses Gen_dummy_probes.sed even on Windows.  Is that 
correct,

intended?

Interesting point.  This may depend on the environment at the end?  As
far as I can see, sed is currently a hard requirement in the meson
build and we'd fail if the command cannot be used.  The buildfarm
machines that test meson are able to find sed, making
Gen_dummy_probes.pl not necessary:
https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=drongo&dt=2023-10-11%2020%3A21%3A17&stg=configure 



So the $1000 question is: could there be a point in keeping the perl
script around if sed cannot be found?  The buildfarm coverage is
currently saying no thanks to chocolatey, at least.  The VM images
compiled by Andres for the CI seem to have the same opinion.


I don't think we should rely on sed being there on Windows.  Maybe 
it's true now on the handful of buildfarm/CI machines and early 
adopters, but do we have any indication that that is systematic or 
just an accident?


Since we definitely require Perl now, we could just as well use the 
Perl script and avoid this issue.


Attached is a Perl version of the sed script, converted by hand (so 
not the super-verbose s2p thing).  It's basically just the sed script 
with semicolons added and the backslashes in the regular expressions 
moved around.  I think we could use something like that for all 
platforms now.




I think it's alright, but please don't use literal tabs, use \t, even in 
a character class.



cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com





Re: Remove MSVC scripts from the tree

2023-11-08 Thread Peter Eisentraut

On 12.10.23 07:23, Michael Paquier wrote:

- Is src/backend/utils/README.Gen_dummy_probes still correct after this?
AFAICT, the Perl-based MSVC build system uses Gen_dummy_probes.pl, but the
meson build uses Gen_dummy_probes.sed even on Windows.  Is that correct,
intended?

Interesting point.  This may depend on the environment at the end?  As
far as I can see, sed is currently a hard requirement in the meson
build and we'd fail if the command cannot be used.  The buildfarm
machines that test meson are able to find sed, making
Gen_dummy_probes.pl not necessary:
https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=drongo&dt=2023-10-11%2020%3A21%3A17&stg=configure

So the $1000 question is: could there be a point in keeping the perl
script around if sed cannot be found?  The buildfarm coverage is
currently saying no thanks to chocolatey, at least.  The VM images
compiled by Andres for the CI seem to have the same opinion.


I don't think we should rely on sed being there on Windows.  Maybe it's 
true now on the handful of buildfarm/CI machines and early adopters, but 
do we have any indication that that is systematic or just an accident?


Since we definitely require Perl now, we could just as well use the Perl 
script and avoid this issue.


Attached is a Perl version of the sed script, converted by hand (so not 
the super-verbose s2p thing).  It's basically just the sed script with 
semicolons added and the backslashes in the regular expressions moved 
around.  I think we could use something like that for all platforms now.


#-
# Perl script to create dummy probes.h file when dtrace is not available
#
# Copyright (c) 2008-2023, PostgreSQL Global Development Group
#
# src/backend/utils/Gen_dummy_probes.pl
#-

use strict;
use warnings;

m/^[ 	]*probe / || next;
s/^[ 	]*probe ([^(]*)(.*);/$1$2/;
s/__/_/g;
y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/;
s/^/#define TRACE_POSTGRESQL_/;
s/\([^,)]{1,}\)/(INT1)/;
s/\([^,)]{1,}, [^,)]{1,}\)/(INT1, INT2)/;
s/\([^,)]{1,}, [^,)]{1,}, [^,)]{1,}\)/(INT1, INT2, INT3)/;
s/\([^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}\)/(INT1, INT2, INT3, INT4)/;
s/\([^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}\)/(INT1, INT2, INT3, INT4, INT5)/;
s/\([^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}\)/(INT1, INT2, INT3, INT4, INT5, INT6)/;
s/\([^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}\)/(INT1, INT2, INT3, INT4, INT5, INT6, INT7)/;
s/\([^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}, [^,)]{1,}\)/(INT1, INT2, INT3, INT4, INT5, INT6, INT7, INT8)/;
s/$/ do {} while (0)/;
print;
s/\(.*$/_ENABLED() (0)/;
print;


Re: Remove MSVC scripts from the tree

2023-10-05 Thread Peter Eisentraut

On 02.10.23 09:38, Michael Paquier wrote:

Attached is a v2 with these adjustments, for now.


General comments:

- I think we can't just delete install-windows.sgml.  Some of that 
content needs to be moved over to installation.sgml.  As a simple 
example, install-windows.sgml shows which MSVC versions are supported. 
That information should surely be kept.


- Is src/backend/utils/README.Gen_dummy_probes still correct after this? 
 AFAICT, the Perl-based MSVC build system uses Gen_dummy_probes.pl, but 
the meson build uses Gen_dummy_probes.sed even on Windows.  Is that 
correct, intended?


- src/port/pgstrsignal.c contains a comment that it is not "built in 
MSVC builds", but AFAICT, that is only correct for the legacy Perl-based 
build system, not for meson.  Again, is that correct, intended?



Detail comments:

(Btw., whatever orderfile you used for the diff, I find that confusing.)

* config/perl.m4: This now contains all the required information, but 
maybe break the text into paragraphs a bit?



* doc/src/sgml/installation.sgml:

I think this paragraph should just be removed altogether:

  
   If you are building PostgreSQL for Microsoft
-  Windows, read this chapter if you intend to build with MinGW or Cygwin;
-  but if you intend to build with Microsoft's Visual
-  C++, see  instead.
+  Windows, read this chapter if you intend to build with Meson, MinGW or
+  Cygwin.
  

Here


 PostgreSQL can be built using Cygwin, a Linux-like environment for
 Windows, but that method is inferior to the native Windows build
-(see linkend="install-windows"/>) and

-running a server under Cygwin is no longer recommended.
+with Meson, and running a server under Cygwin is no longer recommended.


I think "with Meson" should be removed.  The tradeoff is Cygwin vs. 
native, it doesn't have anything to do with Meson.


Also, I think this paragraph needs a complete revision, along with 
however install-windows.sgml gets integrated:



-PostgreSQL for Windows can be built using MinGW, a Unix-like build
 [...]


* meson.build:  I think these comments are unnecessary and can be removed:

-# From Project.pm
+# MSVC flags

+  # Preprocessor definitions.


* src/bin/pgevent/meson.build:  After consideration, I think this 
comment should just be removed:


-# FIXME: copied from Mkvcbuild.pm, but I don't think that's the right 
approach

+# FIXME: this may not not the right approach..

The original site in Mkvcbuild.pm does not contain a comment, so we 
should accept that as canonical.  It doesn't help much if we carry 
around a comment like "this might be wrong" indefinitely without any 
further supporting material.



* src/common/Makefile and src/common/meson.build:  This change is losing 
the period at the end of the first sentence:


 # A few files are currently only built for frontend, not server
-# (Mkvcbuild.pm has a copy of this list, too).  logging.c is excluded
-# from OBJS_FRONTEND_SHLIB (shared library) as a matter of policy,
-# because it is not appropriate for general purpose libraries such
-# as libpq to report errors directly.
+# logging.c is excluded from OBJS_FRONTEND_SHLIB (shared library) as
+# a matter of policy, because it is not appropriate for general purpose
+# libraries such as libpq to report errors directly.





Re: Remove MSVC scripts from the tree

2023-09-29 Thread Peter Eisentraut

On 22.09.23 03:12, Michael Paquier wrote:

It has been mentioned a few times now that, as Meson has been
integrated in the tree, the next step would be to get rid of the
custom scripts in src/tools/msvc/ and moving forward only support
Meson when building with VS compilers.  As far as I understand, nobody
has sent a patch to do that yet, so here is one.

Please find attached a patch to move the needle in this sense.


Your patch still leaves various mentions of Mkvcbuild.pm and Project.pm 
in other files, including in


config/perl.m4
meson.build
src/bin/pg_basebackup/Makefile
src/bin/pgevent/meson.build
src/common/Makefile
src/common/meson.build
src/interfaces/libpq/Makefile
src/port/Makefile

A few of these comments are like "see $otherfile for the reason", which 
means that if we delete $otherfile, we should move that information to a 
new site somehow.






Re: Remove MSVC scripts from the tree

2023-09-26 Thread Andrew Dunstan


On 2023-09-26 Tu 01:25, NINGWEI CHEN wrote:

On Fri, 22 Sep 2023 10:12:29 +0900
Michael Paquier  wrote:


As of today, I can see that the only buildfarm members relying on
these scripts are bowerbird and hamerkop, so these two would fail if
the patch attached were to be applied today.  I am adding Andrew
D. and the maintainers of hamerkop (SRA-OSS) in CC for comments.

hamerkop is not yet prepared for Meson builds, but we plan to work on this 
support soon.
If we go with Meson builds exclusively right now, we will have to temporarily 
remove the master/HEAD for a while.

Best Regards.




I don't think we should switch to that until you're ready.


cheers


andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com


Re: Remove MSVC scripts from the tree

2023-09-25 Thread NINGWEI CHEN
On Fri, 22 Sep 2023 10:12:29 +0900
Michael Paquier  wrote:

> As of today, I can see that the only buildfarm members relying on
> these scripts are bowerbird and hamerkop, so these two would fail if
> the patch attached were to be applied today.  I am adding Andrew
> D. and the maintainers of hamerkop (SRA-OSS) in CC for comments.

hamerkop is not yet prepared for Meson builds, but we plan to work on this 
support soon. 
If we go with Meson builds exclusively right now, we will have to temporarily 
remove the master/HEAD for a while.

Best Regards.
-- 
SRA OSS LLC
Chen Ningwei





Re: Remove MSVC scripts from the tree

2023-09-25 Thread Andrew Dunstan


On 2023-09-21 Th 21:12, Michael Paquier wrote:


As of today, I can see that the only buildfarm members relying on
these scripts are bowerbird and hamerkop, so these two would fail if
the patch attached were to be applied today.  I am adding Andrew
D. and the maintainers of hamerkop (SRA-OSS) in CC for comments.



Changing bowerbird to use meson should not be difficult, just needs some 
TUITs.



cheers


andrew

--
Andrew Dunstan
EDB:https://www.enterprisedb.com


Re: Remove MSVC scripts from the tree

2023-09-24 Thread Michael Paquier
On Fri, Sep 22, 2023 at 08:06:57AM +0200, Peter Eisentraut wrote:
> First we need to fix things so that we can build using meson from a
> distribution tarball, which is the subject of
> .

Thanks, missed this one.
--
Michael


signature.asc
Description: PGP signature


Re: Remove MSVC scripts from the tree

2023-09-21 Thread Peter Eisentraut

On 22.09.23 03:12, Michael Paquier wrote:

It has been mentioned a few times now that, as Meson has been
integrated in the tree, the next step would be to get rid of the
custom scripts in src/tools/msvc/ and moving forward only support
Meson when building with VS compilers.


First we need to fix things so that we can build using meson from a 
distribution tarball, which is the subject of 
.