Re: [RFC] building postgres with meson - v11

[samay sharma]

On Thu, Sep 1, 2022 at 4:12 PM samay sharma  wrote:

> Hi,
>
> On Wed, Aug 31, 2022 at 1:42 AM Peter Eisentraut <
> peter.eisentr...@enterprisedb.com> wrote:
>
>> On 24.08.22 17:30, Andres Freund wrote:
>> >> 0545eec895 meson: Add docs
>> >>
>> >> We should think more about how to arrange the documentation.  We
>> >> probably don't want to copy-and-paste all the introductory and
>> >> requirements information.  I think we can make this initially much
>> >> briefer, like the Windows installation chapter.  For example, instead
>> >> of documenting each setup option again, just mention which ones exist
>> >> and then point (link) to the configure chapter for details.
>> > The current docs, including the windows ones, are already hard to
>> follow. I
>> > think we should take some care to not make the meson bits even more
>> > confusing. Cross referencing left and right seems problematic from that
>> angle.
>>
>> If you look at the current structure of the installation chapter
>>
>> 17.1. Short Version
>> 17.2. Requirements
>> 17.3. Getting the Source
>> 17.4. Installation Procedure
>> 17.5. Post-Installation Setup
>> 17.6. Supported Platforms
>> 17.7. Platform-Specific Notes
>>
>> only 17.1, small parts of 12.2, and 17.4 should differ between make and
>> meson.  There is no conceivable reason why the meson installation
>> chapter should have a different "Getting the Source" section.  And some
>> of the post-installation and platform-specific information doesn't
>> appear at all on the meson chapter.
>>
>> I think we can try to be a bit more ingenious in how we weave this
>> together in the best way.  What I really wouldn't want is two separate
>> chapters that duplicate the entire process.  I think we could do one
>> chapter, like
>>
>> - Short Version
>> - Requirements
>> - Getting the Source
>> - Installation Procedure
>> - Installation Procedure using Meson
>> - Post-Installation Setup
>> - Supported Platforms
>> - Platform-Specific Notes
>>
>
> I spent some more time thinking about the structure of the docs. The
> getting the source, supported platforms, post installation setup and
> platform specific notes sections are going to be mostly common. We do
> expect some differences in supported platforms and platform specific notes
> but I think they should be manageable without confusing readers.
>
> The others; short version, requirements, and installation procedure are
> pretty different and I feel combining them will end up confusing readers or
> require creating autoconf / make and meson versions of many things at many
> different places. Also, if we keep it separate, it'll be easier to remove
> make / autoconf specific sections if (when?) we want to do that.
>
> So, I was thinking of the following structure:
> - Supported Platforms
> - Getting the Source
> - Building with make and autoconf
>   -- Short version
>   -- Requirements
>   -- Installation Procedure and it's subsections
> - Building with Meson
>   -- Short version
>   -- Requirements
>   -- Installation Procedure and it's subsections
> - Post-installation Setup
> - Platform specific notes
>
> It has the disadvantage of short version moving to a bit later in the
> chapter but I think it's a good structure to reduce duplication and also
> keep sections which are different separate. Thoughts on this approach? If
> this looks good, I can submit a patch rearranging things this way.
>

Another thing I'd like input on. A common question I've heard from people
who've tried out the docs is How do we do the equivalent of X in make with
meson. As meson will be new for a bunch of people who are fluent with make,
I won't be surprised if this is a common ask. To address that, I was
planning to add a page to specify the key things one needs to keep in mind
while "migrating" from make to meson and having a translation table of
commonly used commands.

I was planning to add it in the meson section, but if we go ahead with the
structure proposed above, it doesn't fit it into one as cleanly. Maybe, it
still goes in the meson section? Thoughts?

Regards,
Samay


>
> As a follow up patch, we could also try to fit the Windows part into this
> model. We could add a Building with visual C++ or Microsoft windows SDK
> section. It doesn't have a short version but follows the remaining template
> of requirements and installation procedure subsections (Building, Cleaning
> and Installing and Running Regression tests) well.
>
> Regards,
> Samay
>
>>
>> Alternatively, if people prefer two separate chapters, let's think about
>> some source-code level techniques to share the common contents.
>>
>

Re: [RFC] building postgres with meson - v12

[Tom Lane]

Andres Freund  writes:
> On 2022-09-02 14:17:26 +0700, John Naylor wrote:
>> +  # FIXME: -output option is only available in perl 5.9.3 - but that's
>> +  # probably a fine minimum requirement?
>> 
>> Since we've retired some buildfarm animals recently, it seems the
>> oldest perl there is 5.14? ... which came out in 2011, so it seems
>> later on we could just set that as the minimum.

> At the moment we document 5.8.3 as our minimum, supposedly based on some
> buildfarm animal - but that's probably outdated.

Yeah, definitely.  prairiedog was the only animal running such an old
version, and it's gone.  I don't think we have anything testing ancient
bison or flex anymore, either.  I'm a fan of actually testing whatever
we claim as the minimum supported version of any tool, so there's some
work to be done here, on buildfarm config or docs or both.

regards, tom lane

Re: [RFC] building postgres with meson - v12

[Andres Freund]

Hi,

On 2022-09-02 09:35:15 -0700, Andres Freund wrote:
> Part of that is due to some ugly dependencies of src/common on backend headers
> that IMO probably shouldn't exist (e.g. src/common/relpath.c includes
> catalog/pg_tablespace_d.h). Looks like it'd not be hard to get at least the
> _shlib version of src/common and libpq build without waiting for that. But for
> all the backend code I don't really see a way, so it'd be nice to make genbki
> faster at some point.

This reminded me of a question I had. Requires a bit of an intro...


Because ninja's build specification ends up as a fairly clean DAG, and because
it also collects compiler generated dependencies as a DAG, it's possible to
check whether the build specification contains sufficient dependencies.

Before ninja 1.11 there was
https://github.com/llvm/llvm-project/blob/main/llvm/utils/check_ninja_deps.py
and ninja 1.11 has "ninja -t missingdeps" built in.

Intentionally removing some of the dependencies to show the output:

$ ninja -t missingdeps
Missing dep: src/interfaces/ecpg/preproc/ecpg.p/.._ecpglib_typename.c.o uses 
src/include/catalog/pg_type_d.h (generated by CUSTOM_COMMAND)
...
Missing dep: src/bin/scripts/reindexdb.p/reindexdb.c.o uses 
src/include/catalog/pg_class_d.h (generated by CUSTOM_COMMAND)
Missing dep: contrib/oid2name/oid2name.p/oid2name.c.o uses 
src/include/catalog/pg_class_d.h (generated by CUSTOM_COMMAND)
Missing dep: contrib/vacuumlo/vacuumlo.p/vacuumlo.c.o uses 
src/include/catalog/pg_class_d.h (generated by CUSTOM_COMMAND)
Missing dep: 
src/test/modules/libpq_pipeline/libpq_pipeline.p/libpq_pipeline.c.o uses 
src/include/catalog/pg_type_d.h (generated by CUSTOM_COMMAND)
Processed 2299 nodes.
Error: There are 62 missing dependency paths.
62 targets had depfile dependencies on 25 distinct generated inputs (from 1 
rules)  without a non-depfile dep path to the generator.
There might be build flakiness if any of the targets listed above are built 
alone, or not late enough, in a clean output directory.

Obviously that can only work after building, as the compiler generated
dependencies are needed.

I find this exceedingly helpful, because it supplies a very high guarantee
that the build specification will not fail on a different machine due to
different performance characteristics.

The question:

Is it worth running ninja -t missingdeps as a test? At the time we run tests
we'll obviously have built and thus collected "real" dependencies, so we would
have the necessary information to determine whether dependencies are missing.
I think it'd be fine to do so only for ninja >= 1.11, rather than falling back
to the llvm python implementation, which is much slower (0.068s vs
3.760s). And also because it's not as obvious how to include the python script.

Alternatively, we could just document that ninja -t missingdeps is worth
running. Perhaps at the top of the toplevel build.meson file?

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v12

[John Naylor]

On Thu, Sep 1, 2022 at 1:12 AM Andres Freund  wrote:
> [v12]

+# Build a small utility static lib for the parser. This makes it easier to not
+# depend on gram.h already having been generated for most of the other code
+# (which depends on generated headers having been generated). The generation
+# of the parser is slow...

It's not obvious whether this is intended to be a Meson-only
optimization or a workaround for something awkward to specify.

+  # FIXME: -output option is only available in perl 5.9.3 - but that's
+  # probably a fine minimum requirement?

Since we've retired some buildfarm animals recently, it seems the
oldest perl there is 5.14? ... which came out in 2011, so it seems
later on we could just set that as the minimum.

-- 
John Naylor
EDB: http://www.enterprisedb.com

Re: [RFC] building postgres with meson - v11

[samay sharma]

Hi,

On Wed, Aug 31, 2022 at 1:42 AM Peter Eisentraut <
peter.eisentr...@enterprisedb.com> wrote:

> On 24.08.22 17:30, Andres Freund wrote:
> >> 0545eec895 meson: Add docs
> >>
> >> We should think more about how to arrange the documentation.  We
> >> probably don't want to copy-and-paste all the introductory and
> >> requirements information.  I think we can make this initially much
> >> briefer, like the Windows installation chapter.  For example, instead
> >> of documenting each setup option again, just mention which ones exist
> >> and then point (link) to the configure chapter for details.
> > The current docs, including the windows ones, are already hard to
> follow. I
> > think we should take some care to not make the meson bits even more
> > confusing. Cross referencing left and right seems problematic from that
> angle.
>
> If you look at the current structure of the installation chapter
>
> 17.1. Short Version
> 17.2. Requirements
> 17.3. Getting the Source
> 17.4. Installation Procedure
> 17.5. Post-Installation Setup
> 17.6. Supported Platforms
> 17.7. Platform-Specific Notes
>
> only 17.1, small parts of 12.2, and 17.4 should differ between make and
> meson.  There is no conceivable reason why the meson installation
> chapter should have a different "Getting the Source" section.  And some
> of the post-installation and platform-specific information doesn't
> appear at all on the meson chapter.
>
> I think we can try to be a bit more ingenious in how we weave this
> together in the best way.  What I really wouldn't want is two separate
> chapters that duplicate the entire process.  I think we could do one
> chapter, like
>
> - Short Version
> - Requirements
> - Getting the Source
> - Installation Procedure
> - Installation Procedure using Meson
> - Post-Installation Setup
> - Supported Platforms
> - Platform-Specific Notes
>

I spent some more time thinking about the structure of the docs. The
getting the source, supported platforms, post installation setup and
platform specific notes sections are going to be mostly common. We do
expect some differences in supported platforms and platform specific notes
but I think they should be manageable without confusing readers.

The others; short version, requirements, and installation procedure are
pretty different and I feel combining them will end up confusing readers or
require creating autoconf / make and meson versions of many things at many
different places. Also, if we keep it separate, it'll be easier to remove
make / autoconf specific sections if (when?) we want to do that.

So, I was thinking of the following structure:
- Supported Platforms
- Getting the Source
- Building with make and autoconf
  -- Short version
  -- Requirements
  -- Installation Procedure and it's subsections
- Building with Meson
  -- Short version
  -- Requirements
  -- Installation Procedure and it's subsections
- Post-installation Setup
- Platform specific notes

It has the disadvantage of short version moving to a bit later in the
chapter but I think it's a good structure to reduce duplication and also
keep sections which are different separate. Thoughts on this approach? If
this looks good, I can submit a patch rearranging things this way.

As a follow up patch, we could also try to fit the Windows part into this
model. We could add a Building with visual C++ or Microsoft windows SDK
section. It doesn't have a short version but follows the remaining template
of requirements and installation procedure subsections (Building, Cleaning
and Installing and Running Regression tests) well.

Regards,
Samay

>
> Alternatively, if people prefer two separate chapters, let's think about
> some source-code level techniques to share the common contents.
>

Re: [RFC] building postgres with meson - v12

[Andres Freund]

Hi,

On 2022-08-31 10:28:05 +0200, Peter Eisentraut wrote:
> I found that the perl test modules are not installed.  See attached patch to
> correct this.
>
> To the patches:
>
> 4e15ee0e24 Don't hardcode tmp_check/ as test directory for tap tests
> 1a3169bc3f Split TESTDIR into TESTLOGDIR and TESTDATADIR
>
> It's a bit weird that the first patch changes the meaning of TESTDIR
> and the second patch removes it.  Maybe these patches should be
> squashed together?

Hm, to me they seem topically separate enough, but I don't have a strong
opinion on it.


> 581721fa99 meson: prereq: Add src/tools/gen_export.pl
>
> Still wondering about the whitespace changes I reported recently, but
> that can also be fine-tuned later.

I'll look into it in a bit.


> a1fb97a81b meson: Add meson based buildsystem
>
> I'm not a fan of all this business to protect the two build systems
> from each other.  I don't like the build process touching a file under
> version control every time.  How necessary is this?  What happens
> otherwise?

I added it after just about everyone trying meson hit problems due to
conflicts between (past) in-tree configure builds and meson, due to files left
in tree (picking up the wrong .h files, cannot entirely be fixed with -I
arguments, due to the "" includes). By adding the relevant check to the meson
configure phase, and by triggering meson re-configure whenever an in-tree
configure build is done, these issues can be detected.

It'd of course be nicer to avoid the potential for such conflicts, but that
appears to be a huge chunk of work, see the bison/flex subthread.

So I don't really see an alternative.


> conversion_helpers.txt: should probably be removed now.

Done.


> doc/src/sgml/resolv.xsl: I don't understand what this is doing.  Maybe
> at least add a comment in the file.

It's only used for building epubs. Perhaps I should extract that into a
separate patch as well? The relevant section is:

> #
> # epub
> #
>
> # This was previously implemented using dbtoepub - but that doesn't seem to
> # support running in build != source directory (i.e. VPATH builds already
> # weren't supported).
> if pandoc.found() and xsltproc.found()
>   # XXX: Wasn't able to make pandoc successfully resolve entities
>   # XXX: Perhaps we should just make all targets use this, to avoid repeatedly
>   # building whole thing? It's comparatively fast though.
>   postgres_full_xml = custom_target('postgres-full.xml',
> input: ['resolv.xsl', 'postgres.sgml'],
> output: ['postgres-full.xml'],
> depends: doc_generated + [postgres_sgml_valid],
> command: [xsltproc, '--path', '@OUTDIR@/', xsltproc_flags,
>   '-o', '@OUTPUT@', '@INPUT@'],
> build_by_default: false,
>   )

A noted, I couldn't make pandoc resolve our entities, so I used resolv.xsl
them, before calling pandoc.

I'll rename it to resolve-entities.xsl and add a comment.


> src/common/unicode/meson.build: The comment at the top of the file
> should be moved next to the files it is describing (similar to how it
> is in the makefile).

Done.


> I don't see CLDR_VERSION set anywhere.  Is that part implemented?

No, I didn't implement the generation parts of contrib/unaccent. I started
tackling the src/common/unicode bits after John Naylor asked whether that
could be done, but considered that good enough...


> src/port/win32ver.rc.in: This is redundant with src/port/win32ver.rc.
> (Note that the latter is also used as an input file for text
> substitution.  So having another file named *.in next to it would be
> super confusing.)

Yea, this stuff isn't great. I think the better solution, both for meson and
for configure, would be to move to do all the substitution to the C
preprocessor.


> src/tools/find_meson: Could use a brief comment what it does.

Added.


> src/tools/pgflex: Could use a not-brief comment about what it does,
> why it's needed.  Also a comment where it's used.  Also run this
> through pycodestyle.

Working on that.


> cd193eb3e8 meson: ci: Build both with meson and as before
>
> I haven't reviewed this one in detail.  Maybe add a summary in the
> commit message, like these are the new jobs, these are the changes to
> existing jobs.  It looks like there is more in there than just adding
> a few meson jobs.

I don't think we want to commit this as-is. It contains CI for a lot of
platforms - that's very useful for working on meson, but too much for
in-tree. I guess I'll split it into two, one patch for converting a reasonable
subset of the current CI tasks to meson and another to add (back) the current
array of tested platforms.


> If the above are addressed, I think this will be just about at the
> point where the above patches can be committed.

Woo!


> Everything past these patches I'm mentally postponing right now.

Makes sense.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v11

[Peter Eisentraut]

On 24.08.22 17:30, Andres Freund wrote:

0545eec895 meson: Add docs

We should think more about how to arrange the documentation.  We
probably don't want to copy-and-paste all the introductory and
requirements information.  I think we can make this initially much
briefer, like the Windows installation chapter.  For example, instead
of documenting each setup option again, just mention which ones exist
and then point (link) to the configure chapter for details.

The current docs, including the windows ones, are already hard to follow. I
think we should take some care to not make the meson bits even more
confusing. Cross referencing left and right seems problematic from that angle.


If you look at the current structure of the installation chapter

17.1. Short Version
17.2. Requirements
17.3. Getting the Source
17.4. Installation Procedure
17.5. Post-Installation Setup
17.6. Supported Platforms
17.7. Platform-Specific Notes

only 17.1, small parts of 12.2, and 17.4 should differ between make and 
meson.  There is no conceivable reason why the meson installation 
chapter should have a different "Getting the Source" section.  And some 
of the post-installation and platform-specific information doesn't 
appear at all on the meson chapter.


I think we can try to be a bit more ingenious in how we weave this 
together in the best way.  What I really wouldn't want is two separate 
chapters that duplicate the entire process.  I think we could do one 
chapter, like


- Short Version
- Requirements
- Getting the Source
- Installation Procedure
- Installation Procedure using Meson
- Post-Installation Setup
- Supported Platforms
- Platform-Specific Notes

Alternatively, if people prefer two separate chapters, let's think about 
some source-code level techniques to share the common contents.

Re: [RFC] building postgres with meson - v12

[Peter Eisentraut]

I found that the perl test modules are not installed.  See attached 
patch to correct this.


To the patches:

4e15ee0e24 Don't hardcode tmp_check/ as test directory for tap tests
1a3169bc3f Split TESTDIR into TESTLOGDIR and TESTDATADIR

It's a bit weird that the first patch changes the meaning of TESTDIR
and the second patch removes it.  Maybe these patches should be
squashed together?


96d1d0a0cf meson: prereq: Extend gendef.pl in preparation for meson

ok


581721fa99 meson: prereq: Add src/tools/gen_export.pl

Still wondering about the whitespace changes I reported recently, but
that can also be fine-tuned later.


4245cc888e meson: prereq: Refactor PG_TEST_EXTRA logic in autoconf build

ok


3afe803e0f meson: prereq: Fix warning compat_informix/rnull.pgc with msvc

ok


ae7733f46c meson: prereq: Move darwin sysroot determination into 
separate file


ok


a1fb97a81b meson: Add meson based buildsystem

I'm not a fan of all this business to protect the two build systems
from each other.  I don't like the build process touching a file under
version control every time.  How necessary is this?  What happens
otherwise?

conversion_helpers.txt: should probably be removed now.

doc/src/sgml/resolv.xsl: I don't understand what this is doing.  Maybe
at least add a comment in the file.

src/common/unicode/meson.build: The comment at the top of the file
should be moved next to the files it is describing (similar to how it
is in the makefile).  I don't see CLDR_VERSION set anywhere.  Is that
part implemented?

src/port/win32ver.rc.in: This is redundant with src/port/win32ver.rc.
(Note that the latter is also used as an input file for text
substitution.  So having another file named *.in next to it would be
super confusing.)

src/tools/find_meson: Could use a brief comment what it does.

src/tools/pgflex: Could use a not-brief comment about what it does,
why it's needed.  Also a comment where it's used.  Also run this
through pycodestyle.

src/tools/rcgen: This is connected with the comment on win32ver.rc.in
above.  We already have this equivalent code in
src/makefiles/Makefile.win32.  Let's figure out a way to share this
code.  (It could be a Perl script, which is already required on
Windows.)  Also pycodestyle.

src/tools/testwrap: also documentation/comments/pycodestyle


cd193eb3e8 meson: ci: Build both with meson and as before

I haven't reviewed this one in detail.  Maybe add a summary in the
commit message, like these are the new jobs, these are the changes to
existing jobs.  It looks like there is more in there than just adding
a few meson jobs.


If the above are addressed, I think this will be just about at the
point where the above patches can be committed.

Everything past these patches I'm mentally postponing right now.From 80d6f4f9a574ddb6250e06490c610969aacdb824 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 31 Aug 2022 09:16:04 +0200
Subject: [PATCH] meson: Install test perl modules

---
 src/test/meson.build  |  2 ++
 src/test/perl/meson.build | 12 
 2 files changed, 14 insertions(+)
 create mode 100644 src/test/perl/meson.build

diff --git a/src/test/meson.build b/src/test/meson.build
index b86a0f3889..241d9d48aa 100644
--- a/src/test/meson.build
+++ b/src/test/meson.build
@@ -21,3 +21,5 @@ endif
 if icu.found()
   subdir('icu')
 endif
+
+subdir('perl')
diff --git a/src/test/perl/meson.build b/src/test/perl/meson.build
new file mode 100644
index 00..901bae7a56
--- /dev/null
+++ b/src/test/perl/meson.build
@@ -0,0 +1,12 @@
+# could use install_data's preserve_path option in >=0.64.0
+
+install_data(
+  'PostgreSQL/Version.pm',
+  install_dir: dir_pgxs / 'src/test/perl/PostgreSQL')
+
+install_data(
+  'PostgreSQL/Test/Utils.pm',
+  'PostgreSQL/Test/SimpleTee.pm',
+  'PostgreSQL/Test/RecursiveCopy.pm',
+  'PostgreSQL/Test/Cluster.pm',
+  install_dir: dir_pgxs / 'src/test/perl/PostgreSQL/Test')
-- 
2.37.1

Re: [RFC] building postgres with meson - v12

[Andres Freund]

Hi,

On 2022-08-28 12:08:07 -0500, Justin Pryzby wrote:
> with_temp_install is repeated twice in prove_check:
>
> > Subject: [PATCH v12 02/15] Split TESTDIR into TESTLOGDIR and TESTDATADIR
> >
> > -   TESTDIR='$(CURDIR)/tmp_check' $(with_temp_install)
> > PGPORT='6$(DEF_PGPORT)' \
> > +   TESTLOGDIR='$(CURDIR)/tmp_check/log' $(with_temp_install) \
> > +   TESTDATADIR='$(CURDIR)/tmp_check' $(with_temp_install) \
> > +   PGPORT='6$(DEF_PGPORT)' \

Oops, must have screwed up resolving a conflict...


> Before running an individual test like "meson test recovery/017_shm",
> it's currently necessary to first manually run "meson test tmp_install".
> Is it possible to make that happen automatically ?

Not in a trivial way that I found. We don't want to reinstall all the time -
it's *quite* expensive on older machines. We could have a lock file in the
test setup so that the first test run installs it, with the others getting
stalled, but that has pretty obvious disadvantages too (like the test timing
being distorted).

Medium term I think we should consider simply not needing the temp install.

FWIW, if you can do the above as 'meson test tmp_install recovery/017_shm'.


> You're running tap tests via a python script.  There's no problem with
> that, but it's different from what's done by the existing makefiles.
> I was able to remove the python indirection - maybe that's better to
> talk about on the CI thread?  That moves some setup for TAP tests
> (TESTDIR, PATH, cd) from Makefile into the existing perl, which means
> less duplication.

I'm doubtful it's worth removing. You'd need to move removing the files from
the last run into both pg_regress and the tap test infrastructure. And I do
think it's nice to afterwards have markers which tests failed, so we can only
collect their logs.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v12

[Justin Pryzby]

with_temp_install is repeated twice in prove_check:

> Subject: [PATCH v12 02/15] Split TESTDIR into TESTLOGDIR and TESTDATADIR  
>
> 
> -   TESTDIR='$(CURDIR)/tmp_check' $(with_temp_install)
> PGPORT='6$(DEF_PGPORT)' \
> +   TESTLOGDIR='$(CURDIR)/tmp_check/log' $(with_temp_install) \
> +   TESTDATADIR='$(CURDIR)/tmp_check' $(with_temp_install) \
> +   PGPORT='6$(DEF_PGPORT)' \

Before running an individual test like "meson test recovery/017_shm",
it's currently necessary to first manually run "meson test tmp_install".
Is it possible to make that happen automatically ?

You're running tap tests via a python script.  There's no problem with
that, but it's different from what's done by the existing makefiles.
I was able to remove the python indirection - maybe that's better to
talk about on the CI thread?  That moves some setup for TAP tests
(TESTDIR, PATH, cd) from Makefile into the existing perl, which means
less duplication.

Re: [RFC] building postgres with meson - v12

[Thomas Munro]

On Sun, Aug 28, 2022 at 1:39 PM Andres Freund  wrote:
> On 2022-08-27 18:02:40 -0700, Andres Freund wrote:
> > FWIW, I did notice that netbsd does have working unnamed semaphores. I don't
> > know how long ago they were added, but they apparently didn't work quite 
> > right
> > in 2018 [1]. No meaningful performance chance in the main regression tests,
> > I'll run a concurrent check world comparison in the background...
>
> Unnamed ones are substantially worse unfortunately. On an 8 core netbsd 9.3
> VM:

I could update my experimental patch to add home made semaphores using
atomics and futexes.  It needs NetBSD 10, though.  Also works on
OpenBSD and macOS.

Re: [RFC] building postgres with meson - v12

[Andres Freund]

Hi,

On 2022-08-27 18:02:40 -0700, Andres Freund wrote:
> FWIW, I did notice that netbsd does have working unnamed semaphores. I don't
> know how long ago they were added, but they apparently didn't work quite right
> in 2018 [1]. No meaningful performance chance in the main regression tests,
> I'll run a concurrent check world comparison in the background...

Unnamed ones are substantially worse unfortunately. On an 8 core netbsd 9.3
VM:

sysv:
real4m39.777s
user7m35.534s
sys 7m33.831s

unnamed posix
real5m44.035s
user7m23.326s
sys 11m58.946s

The difference in system time is even more substantial than the wall clock
time. And repeated runs were even worse.

I also had the ecpg tests hang in one run with unnamed posix semas, until I
killed 'alloc'. Didn't reproduce since though.


So clearly we shouldn't go and start auto-detecting unnamed posix sema
support.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v12

[Andres Freund]

Hi,

On 2022-08-27 11:04:47 -0700, Andres Freund wrote:
> - choice of semaphore API needs to be cleaned up, that should be easy now, but
>   I thought that I needed to get a new version out first

Everytime I look at the existing selection code I get confused, which is part
of why I haven't tackled this yet.

If I understand the current logic right, we check for sem_open, sem_init if
and only if PREFERRED_SEMAPHORES is set to UNNAMED_POSIX or NAMED_POSIX (no
template defaults to named).  Which also means that we don't link in rt or
pthread if USE_*_POSIX_SEMAPHORES is set directly in the template, as darwin
does.

I read the configure.ac code combined with the templates as resulting in the
following precedence order:

1) windows uses windows semaphores

2) freebsd, linux use unnamed posix semaphores if available

3) macOS < 10.2 uses named semaphores, without linking in rt/pthread
4) macos >= 10.2 uses sysv semaphores

5) sysv semaphores are used

Correct?


Given that Mac OS 10.2 was released in 2002, I think we can safely consider
that unsupported - even prairiedog was a few years younger... :).  Given the
downsides of named semaphores and that we apparently haven't used that code in
years, I wonder if we should remove it?

However, there has been a thread about defaulting to named semas on openbsd,
but then Tom benchmarked out that that's not going to fly for performance
reasons ([1]).


FWIW, I did notice that netbsd does have working unnamed semaphores. I don't
know how long ago they were added, but they apparently didn't work quite right
in 2018 [1]. No meaningful performance chance in the main regression tests,
I'll run a concurrent check world comparison in the background...


Should the choice be configurable with meson? I'm inclined to say not for now.

Regards,

Andres

[1] https://postgr.es/m/3010886.1634950831%40sss.pgh.pa.us
[2] http://www.polarhome.com/service/man/?qf=sem_init=2=NetBSD=3

Re: [RFC] building postgres with meson - v11

[Peter Eisentraut]

On 24.08.22 17:30, Andres Freund wrote:

258f6dc0a7 Don't hardcode tmp_check/ as test directory for tap tests
8ecc33cf04 Split TESTDIR into TESTLOGDIR and TESTDATADIR

I think these patches are split up a bit incorrectly.  If you apply
the first patch by itself, then the output appears in tab_comp_dir/
directly under the source directory.  And then the second patch moves
it to tmp_check/tap_comp_dir/.  If there is an intent to apply these
patches separately somehow, this should be cleaned up.


How is that happening with that version of the patch? The test puts
tap_comp_dir under TESTDIR, and TESTDIR is $(CURDIR)/tmp_check. There was an
earlier version of the patch that was split one more time that did have that
problem, but I don't quite see how that version has it?


Ok, I see now how this works.  It's a bit weird since the meaning of 
TESTDIR is changed.  I'm not sure if this could create cross-branch 
confusion.



97a0b096e8 meson: prereq: Add src/tools/gen_export.pl

This produces leading whitespace in the output files that at least on
darwin wasn't there before.  See attached patch (0002).  This should
be checked again on other platforms as well.


Hm, to me the indentation as is makes more sense, but ...


Maybe for the 'gnu' format, but on darwin (and others) it's just a flat 
list, so indenting it is pointless.



I wonder if we should rewrite this in python - I chose perl because I thought
we could share it, but as you pointed out, that's not possible, because we
don't want to depend on perl during the autoconf build from a tarball.


Given that the code is already written, I wouldn't do it.  Introducing 
Python into the toolchain would require considerations of minimum 
versions, finding the right binaries, formatting, style checking, etc., 
which would probably be a distraction right now.  I also think that this 
script, whose purpose is to read an input file line by line and print it 
back out slightly differently, is not going to be done better in Python.

Re: [RFC] building postgres with meson - v11

[samay sharma]

Hi,

On Wed, Aug 24, 2022 at 8:30 AM Andres Freund  wrote:

> Hi,
>
> On 2022-08-24 11:39:06 +0200, Peter Eisentraut wrote:
> > I have looked at your branch at 0545eec895:
> >
> > 258f6dc0a7 Don't hardcode tmp_check/ as test directory for tap tests
> > 8ecc33cf04 Split TESTDIR into TESTLOGDIR and TESTDATADIR
> >
> > I think these patches are split up a bit incorrectly.  If you apply
> > the first patch by itself, then the output appears in tab_comp_dir/
> > directly under the source directory.  And then the second patch moves
> > it to tmp_check/tap_comp_dir/.  If there is an intent to apply these
> > patches separately somehow, this should be cleaned up.
>
> How is that happening with that version of the patch? The test puts
> tap_comp_dir under TESTDIR, and TESTDIR is $(CURDIR)/tmp_check. There was
> an
> earlier version of the patch that was split one more time that did have
> that
> problem, but I don't quite see how that version has it?
>
>
> > I haven't checked the second patch in detail yet, but it looks like
> > the thought was that the first patch is about ready to go.
> >
> > 834a40e609 meson: prereq: Extend gendef.pl in preparation for meson
> >
> > I'm not qualified to check that in detail, but it looks reasonable
> > enough to me.
> >
> > See attached patch (0001) for a perlcritic fix.
>
> Thanks.
>
>
> > 97a0b096e8 meson: prereq: Add src/tools/gen_export.pl
> >
> > This produces leading whitespace in the output files that at least on
> > darwin wasn't there before.  See attached patch (0002).  This should
> > be checked again on other platforms as well.
>
> Hm, to me the indentation as is makes more sense, but ...
>
> > Other than that this looks good.  Attached is a small cosmetic patch
> (0003).
>
> I wonder if we should rewrite this in python - I chose perl because I
> thought
> we could share it, but as you pointed out, that's not possible, because we
> don't want to depend on perl during the autoconf build from a tarball.
>
>
> > e0a8387660 solaris: Use versioning scripts instead of -Bsymbolic
> >
> > This looks like a good idea.  The documentation clearly states that
> > -Bsymbolic shouldn't be used, at least not in the way we have been
> > doing.  Might as well go ahead with this and give it a whirl on the
> > build farm.
>
> Cool. I looked at this because I was confused about getting warnings with
> autoconf that I wasn't getting with meson.
>
>
> > 0545eec895 meson: Add docs
> >
> > We should think more about how to arrange the documentation.  We
> > probably don't want to copy-and-paste all the introductory and
> > requirements information.  I think we can make this initially much
> > briefer, like the Windows installation chapter.  For example, instead
> > of documenting each setup option again, just mention which ones exist
> > and then point (link) to the configure chapter for details.
>
> The current docs, including the windows ones, are already hard to follow. I
> think we should take some care to not make the meson bits even more
> confusing. Cross referencing left and right seems problematic from that
> angle.
>

On Configure options:

To add to the above, very few sections are an exact copy paste. The
arguments and default behaviors of quite a few configure options are
different. The change in default behavior and arguments is primarily due to
"auto" features which get enabled if the dependencies are found. Whereas
with make, we have explicit --enable or --disable options which don't take
any arguments.

Also, a few instructions / commands which worked with make will need to be
done a bit differently due to environment variables etc. which also had to
be communicated.

Communicating these differences and nuances with cross referencing would
make it confusing as most of this information is in the explanation
paragraph.

On requirements:

They are also a bit different eg. readline is not a "required" thing
anymore, perl, flex, bison are required etc. Also, these are bullet points
with information inlined and not separate sections, so cross-referencing
here also would be hard.

Regards,
Samay


> > I spent a bit of time with the test suites.  I think there is a
> > problem in that selecting a test suite directly, like
> >
> > meson test -C _build --suite recovery
> >
> > doesn't update the tmp_install.  So if this is the first thing you run
> > after a build, everything will fail.  Also, if you run this later, the
> > tmp_install doesn't get updated, so you're not testing up-to-date
> > code.
>
> At the moment creation of the tmp_install is its own test suite. I don't
> know
> if that's the best way, or what the best way is, but that explains that
> fact. You can do the above without the issue by specifying
> --suite setup --suite recovery.
>
>
> Greetings,
>
> Andres Freund
>

Re: [RFC] building postgres with meson - v11

[Andres Freund]

Hi,

On 2022-08-17 14:53:17 -0700, Andres Freund wrote:
> > - In the top-level meson.build, the "renaming" of the Windows system
> >   name
> >
> >   host_system = host_machine.system() == 'windows' ? 'win32' :
> > host_machine.system()
> >   build_system = build_machine.system() == 'windows' ? 'win32' :
> > build_machine.system()
> >
> >   seems unnecessary to me.  Why not stick with the provided names?
> 
> Because right now we also use it for things like choosing the "source" for
> pg_config_os.h (i.e. include/port/{darwin,linux,win32,..}.h). And it seemed
> easier to just have one variable name for all of it.

I am now changing this so that there's an additional 'portname' variable for
this purpose. Otherwise the meson names are used.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v11

[Andres Freund]

Hi,

On 2022-08-24 11:39:06 +0200, Peter Eisentraut wrote:
> I have looked at your branch at 0545eec895:
> 
> 258f6dc0a7 Don't hardcode tmp_check/ as test directory for tap tests
> 8ecc33cf04 Split TESTDIR into TESTLOGDIR and TESTDATADIR
> 
> I think these patches are split up a bit incorrectly.  If you apply
> the first patch by itself, then the output appears in tab_comp_dir/
> directly under the source directory.  And then the second patch moves
> it to tmp_check/tap_comp_dir/.  If there is an intent to apply these
> patches separately somehow, this should be cleaned up.

How is that happening with that version of the patch? The test puts
tap_comp_dir under TESTDIR, and TESTDIR is $(CURDIR)/tmp_check. There was an
earlier version of the patch that was split one more time that did have that
problem, but I don't quite see how that version has it?


> I haven't checked the second patch in detail yet, but it looks like
> the thought was that the first patch is about ready to go.
> 
> 834a40e609 meson: prereq: Extend gendef.pl in preparation for meson
> 
> I'm not qualified to check that in detail, but it looks reasonable
> enough to me.
> 
> See attached patch (0001) for a perlcritic fix.

Thanks.


> 97a0b096e8 meson: prereq: Add src/tools/gen_export.pl
> 
> This produces leading whitespace in the output files that at least on
> darwin wasn't there before.  See attached patch (0002).  This should
> be checked again on other platforms as well.

Hm, to me the indentation as is makes more sense, but ...

> Other than that this looks good.  Attached is a small cosmetic patch (0003).

I wonder if we should rewrite this in python - I chose perl because I thought
we could share it, but as you pointed out, that's not possible, because we
don't want to depend on perl during the autoconf build from a tarball.


> e0a8387660 solaris: Use versioning scripts instead of -Bsymbolic
> 
> This looks like a good idea.  The documentation clearly states that
> -Bsymbolic shouldn't be used, at least not in the way we have been
> doing.  Might as well go ahead with this and give it a whirl on the
> build farm.

Cool. I looked at this because I was confused about getting warnings with
autoconf that I wasn't getting with meson.


> 0545eec895 meson: Add docs
> 
> We should think more about how to arrange the documentation.  We
> probably don't want to copy-and-paste all the introductory and
> requirements information.  I think we can make this initially much
> briefer, like the Windows installation chapter.  For example, instead
> of documenting each setup option again, just mention which ones exist
> and then point (link) to the configure chapter for details.

The current docs, including the windows ones, are already hard to follow. I
think we should take some care to not make the meson bits even more
confusing. Cross referencing left and right seems problematic from that angle.


> I spent a bit of time with the test suites.  I think there is a
> problem in that selecting a test suite directly, like
> 
> meson test -C _build --suite recovery
> 
> doesn't update the tmp_install.  So if this is the first thing you run
> after a build, everything will fail.  Also, if you run this later, the
> tmp_install doesn't get updated, so you're not testing up-to-date
> code.

At the moment creation of the tmp_install is its own test suite. I don't know
if that's the best way, or what the best way is, but that explains that
fact. You can do the above without the issue by specifying
--suite setup --suite recovery.


Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v11

[Peter Eisentraut]

I have looked at your branch at 0545eec895:

258f6dc0a7 Don't hardcode tmp_check/ as test directory for tap tests
8ecc33cf04 Split TESTDIR into TESTLOGDIR and TESTDATADIR

I think these patches are split up a bit incorrectly.  If you apply
the first patch by itself, then the output appears in tab_comp_dir/
directly under the source directory.  And then the second patch moves
it to tmp_check/tap_comp_dir/.  If there is an intent to apply these
patches separately somehow, this should be cleaned up.

I haven't checked the second patch in detail yet, but it looks like
the thought was that the first patch is about ready to go.

834a40e609 meson: prereq: Extend gendef.pl in preparation for meson

I'm not qualified to check that in detail, but it looks reasonable
enough to me.

See attached patch (0001) for a perlcritic fix.

97a0b096e8 meson: prereq: Add src/tools/gen_export.pl

This produces leading whitespace in the output files that at least on
darwin wasn't there before.  See attached patch (0002).  This should
be checked again on other platforms as well.

Other than that this looks good.  Attached is a small cosmetic patch (0003).

40e363b263 meson: prereq: Refactor PG_TEST_EXTRA logic in autoconf build

Since I last looked, this has been turned into a meson option.  Which
is probably the best solution.  But then we should probably make this
a configure option as well.  Otherwise, it could get a bit confusing.
For example, I just unset PG_TEST_EXTRA in my environment to test
something with the meson build, but I was unaware that meson captures
the value at setup time, so my unsetting had no effect.

In any case, maybe adjust the regular expressions to check for word
boundaries, to maintain the original "whitespace-separated"
specification.  For example,

elsif ($ENV{PG_TEST_EXTRA} !~ /\bssl\b/)

e0a8387660 solaris: Use versioning scripts instead of -Bsymbolic

This looks like a good idea.  The documentation clearly states that
-Bsymbolic shouldn't be used, at least not in the way we have been
doing.  Might as well go ahead with this and give it a whirl on the
build farm.

0545eec895 meson: Add docs

We should think more about how to arrange the documentation.  We
probably don't want to copy-and-paste all the introductory and
requirements information.  I think we can make this initially much
briefer, like the Windows installation chapter.  For example, instead
of documenting each setup option again, just mention which ones exist
and then point (link) to the configure chapter for details.


I spent a bit of time with the test suites.  I think there is a
problem in that selecting a test suite directly, like

meson test -C _build --suite recovery

doesn't update the tmp_install.  So if this is the first thing you run
after a build, everything will fail.  Also, if you run this later, the
tmp_install doesn't get updated, so you're not testing up-to-date
code.From 2f25b48271bceb7aa1551e015b03fc20b9aff162 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 24 Aug 2022 11:23:52 +0200
Subject: [PATCH 1/3] Fix for perlcritic

---
 src/tools/msvc/gendef.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/tools/msvc/gendef.pl b/src/tools/msvc/gendef.pl
index f08268e781..cfbdef9007 100644
--- a/src/tools/msvc/gendef.pl
+++ b/src/tools/msvc/gendef.pl
@@ -174,7 +174,7 @@ sub usage
 {
if (-d $in)
{
-   push @files, <$in/*.obj>;
+   push @files, glob "$in/*.obj";
}
else
{
-- 
2.37.1

From a64c90e756d6996b7d8d9d63d42e30c72d4ce098 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 24 Aug 2022 11:24:22 +0200
Subject: [PATCH 2/3] Fix whitespace in output of gen_export.pl

---
 src/tools/gen_export.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/tools/gen_export.pl b/src/tools/gen_export.pl
index 6a8b196ad8..1265564473 100644
--- a/src/tools/gen_export.pl
+++ b/src/tools/gen_export.pl
@@ -56,7 +56,7 @@
}
elsif ($format eq 'darwin')
{
-   print $output_handle "_$1\n";
+   print $output_handle "_$1\n";
}
elsif ($format eq 'gnu')
{
-- 
2.37.1

From 5b79baf6bbf4fa62f153a5e96e97f8d5a6345821 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 24 Aug 2022 11:24:36 +0200
Subject: [PATCH 3/3] Some Perl code simplification in gen_export.pl

---
 src/tools/gen_export.pl | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/src/tools/gen_export.pl b/src/tools/gen_export.pl
index 1265564473..727105ba08 100644
--- a/src/tools/gen_export.pl
+++ b/src/tools/gen_export.pl
@@ -48,7 +48,7 @@
{
# don't do anything with a comment
}
-   elsif (/^([^\s]+)\s+([^\s]+)/)
+   elsif (/^(\S+)\s+(\S+)/)
{
if ($format eq 'aix')
{
@@ -79,5 +79,3 @@
 };
 ";
 }
-
-exit(0);
-- 
2.37.1

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-08-09 08:37:16 -0400, Andrew Dunstan wrote:
> On 2022-08-09 Tu 03:10, Andres Freund wrote:
> > Hi,
> >
> > I was looking at re-unifying gendef2.pl that the meson patchset had 
> > introduced
> > for temporary ease during hacking with gendef.pl. Testing that I noticed 
> > that
> > either I and my machine is very confused, or gendef.pl's check whether it 
> > can
> > skip work is bogus.
> >
> > I noticed that, despite having code to avoid rerunning when the input files
> > are older than the .def file, it always runs.
> >
> > # if the def file exists and is newer than all input object files, skip
> > # its creation
> > if (-f $deffile
> > && (-M $deffile > max(map { -M } <$ARGV[0]/*.obj>)))
> > {
> > print "Not re-generating $defname.DEF, file already exists.\n";
> > exit(0);
> > }
> >
> > My understanding of -M is that it returns the time delta between the file
> > modification and the start of the script. Which makes the use of max() 
> > bogus,
> > since it'll return the oldest time any input has been modified, not the
> > newest. And the condition needs to be inverted, because we want to skip the
> > work if $deffile is *newer*, right?
> >
> > Am I missing something here?
> 
> 
> No, you're right, this is bogus. Reversing the test and using min
> instead of max is the obvious fix.
> 
> 
> > I'm tempted to just remove the not-regenerating logic - gendef.pl shouldn't
> > run if there's nothing to do, and it'll e.g. not notice if there's an
> > additional input that wasn't there during the last invocation of gendef.pl.
> >
> 
> Maybe, need to think about that more.

Any thoughts?

I'd like to commit 0003 in
https://postgr.es/m/20220811002012.ju3rrz47i2e5tdha%40awork3.anarazel.de
fairly soon.

I did fix the bogus "die" message I added during some debugging since posting
that...

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v11

[Andres Freund]

Hi,

On 2022-08-20 09:38:48 +0200, Peter Eisentraut wrote:
> On 17.08.22 23:53, Andres Freund wrote:
> > Any comment on the pg_regress_ecpg commit? I'd like to get that out of the
> > way, and it seems considerably cleaner than the hackery we do right now to
> > make VPATH builds work.
> 
> That one looks like a very good improvement.

Thanks for checking! Pushed.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v11

[Peter Eisentraut]

On 17.08.22 23:53, Andres Freund wrote:

Any comment on the pg_regress_ecpg commit? I'd like to get that out of the
way, and it seems considerably cleaner than the hackery we do right now to
make VPATH builds work.


That one looks like a very good improvement.

Re: [RFC] building postgres with meson - v11

[Andres Freund]

Hi,

On 2022-08-17 15:50:23 +0200, Peter Eisentraut wrote:
> - There are various references to "pch" (pre-compiled headers).  Is
>   there more discussion anywhere about this?  I don't know what this
>   would entail or whether there are any drawbacks to be aware of.  The
>   new *_pch.h files don't have any comments.  Maybe this should be a
>   separate patch later.

It's mainly to make windows builds a bit slower. I've no objection to
separating this out.


> - About relativize_shared_library_references: We have had several
>   patches over the years for working around SIP stuff, and some of
>   them did essentially this, but we decided not to go ahead with them.
>   We could revisit that, but it should be a separate patch, not mixed
>   in with this.

The prior approaches all had issues because they didn't support relative
references IIRC (and thus broke being able to relocate the installation),
which this does.

I just found it very annoying to work on macs without this. And there were at
least two "bug" reports of testers of the meson branch that were just due to
SIP.

I'm ok with splitting it out, but I also think it's a lower risk opportunity
to test that this works.


> - postgresql-extension.pc: Similarly, this ought to be a separate
>   patch.  If we want people to use this, we'll need it in the makefile
>   build system anyway.

Makes sense. I'd like to keep it in the same patch for a short while longer,
to deduplicate some of the code, but then will split it out.


> - -DFRONTEND is used somewhat differently from the makefiles.  For
>example, meson sets -DFRONTEND for pg_controldata, but the
>makefiles don't.  Conversely, the makefiles set -DFRONTEND for
>ecpglib, but meson does not.  This should be checked again to make
>sure it all matches up.

Yes, should sync that up.

FWIW, meson does add -DFRONTEND for ecpglib. There were a few places that did
add it twice, I'll push a cleanup of that in a bit.


> - Option name spelling should be make consistent about underscores
>   versus hyphens.  Built-in meson options use underscores, so we
>   should make the user-defined ones like that as well (some already
>   do).  (wal-blocksize krb-srvnam system-tzdata tap-tests bsd-auth)

No objection.


> - I have found the variable name "cdata" for configuration_data() to
>   be less than clear.  I see some GNOME projects to it that way, is
>   that where it's from?  systemd uses "conf", maybe that's better.

I don't know where it's from - I don't think I ever looked at gnome
buildsystem stuff. It seems to be the obvious abbreviation for
configuration_data()...  I don't object to conf, but it's not a clear
improvement to me.


> - In the top-level meson.build, the "renaming" of the Windows system
>   name
>
>   host_system = host_machine.system() == 'windows' ? 'win32' :
> host_machine.system()
>   build_system = build_machine.system() == 'windows' ? 'win32' :
> build_machine.system()
>
>   seems unnecessary to me.  Why not stick with the provided names?

Because right now we also use it for things like choosing the "source" for
pg_config_os.h (i.e. include/port/{darwin,linux,win32,..}.h). And it seemed
easier to just have one variable name for all of it.


> - The c99_test ought to be not needed if the c_std project option is
>   used.  Was there a problem with that?

We don't want to force -std=c99 when not necessary, I think. We sometimes use
features from newer (and from gnu) language versions after testing
availability, and if we hardcode the version those will either fail or elicit
warnings.


> - Is there a way to split up the top-level meson.build somehow?  Maybe
>   just throw some stuff into included files?  This might get out of
>   hand at some point.

We can put stuff into config/meson.build or such. But I don't think it's
clearly warranted at this point.


> - The PG_SYSROOT detection gives different results.  On my system,
>   configure produces
>
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk,
>   meson produces
>
> /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk.
>   src/template/darwin goes out of its way to get a version-specific
>   result, so we need to carry that over somehow.  (The difference does
>   result in differences in the built binaries.)

TBH, I don't really understand the SYSROOT stuff all that well, never having
used a mac in anger (well, only in anger, but ...).

What do you think about extracting the relevant portion of src/template/darwin
into a dedicated shell script that gets called by both?


> Then, some patches from me:
>
> 0001-Change-shared-library-installation-naming-on-macOS.patch
>
> This changes the makefiles to make the shared library file naming on
> macOS match what meson produces.  I don't know what the situation is
> on other platforms.

No opinion on the matter. Seems best to apply separately if we want to?


> 

Re: [RFC] building postgres with meson - v11

[Peter Eisentraut]

On 11.08.22 02:20, Andres Freund wrote:

Attached is a new version of the meson patchset. Plenty changes:


I have various bits of comments on this.

- There are various references to "pch" (pre-compiled headers).  Is
  there more discussion anywhere about this?  I don't know what this
  would entail or whether there are any drawbacks to be aware of.  The
  new *_pch.h files don't have any comments.  Maybe this should be a
  separate patch later.

- About relativize_shared_library_references: We have had several
  patches over the years for working around SIP stuff, and some of
  them did essentially this, but we decided not to go ahead with them.
  We could revisit that, but it should be a separate patch, not mixed
  in with this.

- postgresql-extension.pc: Similarly, this ought to be a separate
  patch.  If we want people to use this, we'll need it in the makefile
  build system anyway.

- -DFRONTEND is used somewhat differently from the makefiles.  For
   example, meson sets -DFRONTEND for pg_controldata, but the
   makefiles don't.  Conversely, the makefiles set -DFRONTEND for
   ecpglib, but meson does not.  This should be checked again to make
   sure it all matches up.

- Option name spelling should be make consistent about underscores
  versus hyphens.  Built-in meson options use underscores, so we
  should make the user-defined ones like that as well (some already
  do).  (wal-blocksize krb-srvnam system-tzdata tap-tests bsd-auth)

- I have found the variable name "cdata" for configuration_data() to
  be less than clear.  I see some GNOME projects to it that way, is
  that where it's from?  systemd uses "conf", maybe that's better.

- In the top-level meson.build, the "renaming" of the Windows system
  name

  host_system = host_machine.system() == 'windows' ? 'win32' : 
host_machine.system()
  build_system = build_machine.system() == 'windows' ? 'win32' : 
build_machine.system()


  seems unnecessary to me.  Why not stick with the provided names?

- The c99_test ought to be not needed if the c_std project option is
  used.  Was there a problem with that?

- Is there a way to split up the top-level meson.build somehow?  Maybe
  just throw some stuff into included files?  This might get out of
  hand at some point.

- The PG_SYSROOT detection gives different results.  On my system,
  configure produces

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk,
  meson produces

/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk.
  src/template/darwin goes out of its way to get a version-specific
  result, so we need to carry that over somehow.  (The difference does
  result in differences in the built binaries.)


Then, some patches from me:

0001-Change-shared-library-installation-naming-on-macOS.patch

This changes the makefiles to make the shared library file naming on
macOS match what meson produces.  I don't know what the situation is
on other platforms.

0002-meson-Fix-installation-name-of-libpgfeutils.patch

This was presumably an accidental mistake.

0003-meson-Libraries-need-to-be-built-with-DSO_MAJOR_VERS.patch

This is needed to make NLS work for the libraries.

0004-meson-Add-darwin_versions-argument-for-libraries.patch

This is to make the output match what Makefile.shlib produces.

0005-meson-Fix-link-order-of-support-libraries.patch
0006-meson-Make-link-order-of-external-libraries-match-ma.patch
0007-WIP-meson-Make-link-order-of-object-files-match-make.patch

I have analyzed the produced binaries between both build systems to
make sure they match.  If we link the files and libraries in different
orders, that becomes difficult.  So this fixes this up a bit.  0005 is
needed for correctness in general, I think.  0006 is mostly cosmetic.
You probably wanted to make the library order alphabetical in the
meson files, which I'd support, but then we should change the
makefiles to match.  Similarly, 0007, which is clearly a bit messy at
the moment, but we should try to sort that out either in the old or
the new build files.


And finally some comments on your patches:

meson: prereq: Don't add HAVE_LDAP_H HAVE_WINLDAP_H to pg_config.h.

This can go ahead.

meson: prereq: fix warning compat_informix/rnull.pgc with msvc

-   $float f = 3.71;
+   $float f = (float) 3.71;

This could use float literals like

+   $float f = 3.71f;From 054b24e9ae5fc859f13c05d8150ef1168a477ce4 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 17 Aug 2022 14:18:52 +0200
Subject: [PATCH 1/7] Change shared library installation naming on macOS

It is not customary to install a shared library with a minor version
number (libpq.5.16.dylib) on macOS.  We just need the file with the
major version number (libpq.5.dylib) and the one without version
number (libpq.dylib).  This also matches the installation layout used
by Meson.
---
 src/Makefile.shlib | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git 

Re: [RFC] building postgres with meson

[Tom Lane]

John Naylor  writes:
> I'll volunteer to work on this unless an easier solution happens to
> come along in the next couple days. (aside: guc-file.l doesn't have a
> grammar, so not yet sure if that makes the issue easier or harder...)

That one's probably mostly about the issue mentioned in the other
commit you identified.  Without %top, it's impossible to make a
standalone flex module honor the rule about thou-shalt-have-no-
other-includes-before-postgres.h.  So embedding it in some other
file was originally a necessity for that.  Now that we know how
to fix that, it's just a matter of making sure that any other stuff
the scanner needs is available from a .h file.

regards, tom lane

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-08-11 10:57:33 +0700, John Naylor wrote:
> I'll volunteer to work on this unless an easier solution happens to
> come along in the next couple days.

Cool!


> (aside: guc-file.l doesn't have a grammar, so not yet sure if that makes the
> issue easier or harder...)

I think we should consider compiling it separately from guc.c. guc.c already
compiles quite slowly (iirc beat only by ecpg and main grammar), and it's a
relatively commonly changed source file.

It might even be a good idea to split guc.c so it only contains the settings
arrays + direct dependencies...

Greetings,

Andres Freund

Re: [RFC] building postgres with meson

[John Naylor]

On Thu, Aug 11, 2022 at 10:37 AM Tom Lane  wrote:
>
> John Naylor  writes:
> > With a bit of work, we could probably get rid of those includes. See
> > 27199058d98ef7f for one example.
>
> Yeah --- it would mean creating gram.h files for all the bison grammars
> not just a few of them, but it's certainly do-able if there's motivation
> to make the changes.  Most of the files that are done that way date
> from before we knew about flex's %top.

I'll volunteer to work on this unless an easier solution happens to
come along in the next couple days. (aside: guc-file.l doesn't have a
grammar, so not yet sure if that makes the issue easier or harder...)

-- 
John Naylor
EDB: http://www.enterprisedb.com

Re: [RFC] building postgres with meson

[Tom Lane]

I wrote:
> John Naylor  writes:
>> With a bit of work, we could probably get rid of those includes. See
>> 27199058d98ef7f for one example.

> Yeah --- it would mean creating gram.h files for all the bison grammars
> not just a few of them, but it's certainly do-able if there's motivation
> to make the changes.  Most of the files that are done that way date
> from before we knew about flex's %top.

BTW, 72b1e3a21 is another useful precedent in this area.

regards, tom lane

Re: [RFC] building postgres with meson

[Tom Lane]

John Naylor  writes:
> With a bit of work, we could probably get rid of those includes. See
> 27199058d98ef7f for one example.

Yeah --- it would mean creating gram.h files for all the bison grammars
not just a few of them, but it's certainly do-able if there's motivation
to make the changes.  Most of the files that are done that way date
from before we knew about flex's %top.

regards, tom lane

Re: [RFC] building postgres with meson

[John Naylor]

On Thu, Aug 11, 2022 at 12:19 AM Andres Freund  wrote:
> I tried to ignore various generated files in the source tree, but I don't
> think it's doable for all of them. Consider
> e.g. src/backend/utils/misc/guc-file.c which is gets built via #include
> "guc-file.c" from gram.c

With a bit of work, we could probably get rid of those includes. See
27199058d98ef7f for one example.

-- 
John Naylor
EDB: http://www.enterprisedb.com

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-06-02 10:26:09 -0700, Andres Freund wrote:
> > Could we have the meson build check that, say, if gram.c exists it
> > is newer than gram.y?  Or get it to ignore an in-tree gram.c?
>
> I suspect the problem with ignoring is gram.h, that's probably a bit harder to
> ignore.

I tried to ignore various generated files in the source tree, but I don't
think it's doable for all of them. Consider
e.g. src/backend/utils/misc/guc-file.c which is gets built via #include
"guc-file.c" from gram.c

Because it's a "" include, the search path starts in the current directory and
only then -I is searched. To my knowledge there's no way of changing
that. Quoting the gcc manpage:

   -I dir
   -iquote dir
   -isystem dir
   -idirafter dir
   Add the directory dir to the list of directories to be searched for 
header files during preprocessing.  If dir begins with = or $SYSROOT, then
   the = or $SYSROOT is replaced by the sysroot prefix; see --sysroot 
and -isysroot.

   Directories specified with -iquote apply only to the quote form of 
the directive, "#include "file"".  Directories specified with -I, -isystem, or
   -idirafter apply to lookup for both the "#include "file"" and 
"#include " directives.

   You can specify any number or combination of these options on the 
command line to search for header files in several directories.  The lookup
   order is as follows:

   1.  For the quote form of the include directive, the directory of 
the current file is searched first.

   2.  For the quote form of the include directive, the directories 
specified by -iquote options are searched in left-to-right order, as they appear
   on the command line.

   3.  Directories specified with -I options are scanned in 
left-to-right order.
   [...]

Except for copying guc.c from source to build tree before building, I don't
see a way of ignoring the in-build-tree guc-file.c.

Not sure what a good way of dealing with this is. For now I'll make it just
error out if there's any known such file in the source tree, but that's not a
good solution forever.  If it were just "normal" build leftovers I'd propose
to (optionally) just remove them, but that's not good for tarballs.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v10

[Nazir Bilal Yavuz]

Hi,

On 8/8/22 18:53, Andres Freund wrote:

Bilal's version checked different directories for expected files, but I don't
think that's necessary. Bilal, do you remember why you added that?
This was for not breaking autoconf build. Autoconf wasn't using 
expecteddir, so I checked different directories.


Greetings,

Nazir Bilal Yavuz

Re: [RFC] building postgres with meson

[Andrew Dunstan]

On 2022-08-09 Tu 03:10, Andres Freund wrote:
> Hi,
>
> I was looking at re-unifying gendef2.pl that the meson patchset had introduced
> for temporary ease during hacking with gendef.pl. Testing that I noticed that
> either I and my machine is very confused, or gendef.pl's check whether it can
> skip work is bogus.
>
> I noticed that, despite having code to avoid rerunning when the input files
> are older than the .def file, it always runs.
>
> # if the def file exists and is newer than all input object files, skip
> # its creation
> if (-f $deffile
> && (-M $deffile > max(map { -M } <$ARGV[0]/*.obj>)))
> {
> print "Not re-generating $defname.DEF, file already exists.\n";
> exit(0);
> }
>
> My understanding of -M is that it returns the time delta between the file
> modification and the start of the script. Which makes the use of max() bogus,
> since it'll return the oldest time any input has been modified, not the
> newest. And the condition needs to be inverted, because we want to skip the
> work if $deffile is *newer*, right?
>
> Am I missing something here?


No, you're right, this is bogus. Reversing the test and using min
instead of max is the obvious fix.


> I'm tempted to just remove the not-regenerating logic - gendef.pl shouldn't
> run if there's nothing to do, and it'll e.g. not notice if there's an
> additional input that wasn't there during the last invocation of gendef.pl.
>

Maybe, need to think about that more.


cheers


andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

I was looking at re-unifying gendef2.pl that the meson patchset had introduced
for temporary ease during hacking with gendef.pl. Testing that I noticed that
either I and my machine is very confused, or gendef.pl's check whether it can
skip work is bogus.

I noticed that, despite having code to avoid rerunning when the input files
are older than the .def file, it always runs.

# if the def file exists and is newer than all input object files, skip
# its creation
if (-f $deffile
&& (-M $deffile > max(map { -M } <$ARGV[0]/*.obj>)))
{
print "Not re-generating $defname.DEF, file already exists.\n";
exit(0);
}

My understanding of -M is that it returns the time delta between the file
modification and the start of the script. Which makes the use of max() bogus,
since it'll return the oldest time any input has been modified, not the
newest. And the condition needs to be inverted, because we want to skip the
work if $deffile is *newer*, right?

Am I missing something here?


I'm tempted to just remove the not-regenerating logic - gendef.pl shouldn't
run if there's nothing to do, and it'll e.g. not notice if there's an
additional input that wasn't there during the last invocation of gendef.pl.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v10

[Andres Freund]

Hi,

On 2022-07-21 15:26:05 +0300, Bilal Yavuz wrote:
> > On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote:
> > > dff7b5a960 meson: prereq: regress: allow to specify director containing
> > > expected files.
> > >
> > > This could use a bit more explanation, but it doesn't look
> > > controversial so far.
> 
> While testing ECPG, C and exe files are generated by meson so these files
> are in the meson's build directory but expected files are in the source
> directory. However; there was no way to set different paths for inputs (C
> and exe files') and expected files' directory. So, I added `--expecteddir`
> to separately set expected files' directory.

Attached is a version of this patch that also removes the copying of these
files from ecpg's makefile.

Bilal's version checked different directories for expected files, but I don't
think that's necessary. Bilal, do you remember why you added that?


I'm somewhat tempted to rename ecpg's pg_regress to pg_regress_ecpg as part of
this, given the .c file is named pg_regress_ecpg.c and that pg_regress is a
pre-existing binary.

Greetings,

Andres Freund
>From 9645e0a01305ce42fdacaae73bd0d021a5a1c47e Mon Sep 17 00:00:00 2001
From: Andres Freund 
Date: Thu, 30 Jun 2022 18:28:17 -0700
Subject: [PATCH] meson: prereq: regress: allow to specify director containing
 expected files

The ecpg tests have their input directory in the build directory as the tests
need to be built. Until now that required copying the expected/ directory to
the build directory in VPATH builds. To avoid needing to implement the same
for the meson build, add support for specifying the expected directory/.

Remove the copying of ecpg's expected/ to the build dir in VPATH builds.

Author: Nazir Bilal Yavuz 
Author: Andres Freund 
Reviewed-By: Andres Freund 
Discussion: https://postgr.es/m/20220718202327.pspcqz5mwbi2y...@awork3.anarazel.de
---
 src/interfaces/ecpg/test/Makefile  | 28 --
 src/interfaces/ecpg/test/pg_regress_ecpg.c |  8 +++
 src/test/regress/pg_regress.c  |  7 ++
 src/test/regress/pg_regress.h  |  1 +
 4 files changed, 22 insertions(+), 22 deletions(-)

diff --git a/src/interfaces/ecpg/test/Makefile b/src/interfaces/ecpg/test/Makefile
index 10f53c708c8..d7a7d1d1ca5 100644
--- a/src/interfaces/ecpg/test/Makefile
+++ b/src/interfaces/ecpg/test/Makefile
@@ -59,24 +59,16 @@ pg_regress_ecpg.o: pg_regress_ecpg.c $(top_builddir)/src/port/pg_config_paths.h
 $(top_builddir)/src/port/pg_config_paths.h: $(top_builddir)/src/Makefile.global
 	$(MAKE) -C $(top_builddir)/src/port pg_config_paths.h
 
-# When doing a VPATH build, copy over the .pgc, .stdout and .stderr
-# files so that the driver script can find them.  We have to use an
-# absolute path for the targets, because otherwise make will try to
-# locate the missing files using VPATH, and will find them in
-# $(srcdir), but the point here is that we want to copy them from
-# $(srcdir) to the build directory.
-
-ifdef VPATH
-remaining_files_src := $(wildcard $(srcdir)/*/*.pgc) $(wildcard $(srcdir)/expected/*.c) $(wildcard $(srcdir)/expected/*.stdout) $(wildcard $(srcdir)/expected/*.stderr)
-remaining_files_build := $(patsubst $(srcdir)/%, $(abs_builddir)/%, $(remaining_files_src))
-
-all: $(remaining_files_build)
-$(remaining_files_build): $(abs_builddir)/%: $(srcdir)/%
-	ln -s $< $@
-endif
-
-# Common options for tests. Also pick up anything passed in EXTRA_REGRESS_OPTS
-REGRESS_OPTS = --dbname=ecpg1_regression,ecpg2_regression --create-role=regress_ecpg_user1,regress_ecpg_user2 $(EXTRA_REGRESS_OPTS)
+# Common options for tests
+#
+# Need to specify expecteddir explicitly, as the inputdir is located in the
+# build directory, because the files need to be compiled. Other pg_regress
+# style tests have the expecteddir in the source directory.
+#
+# Also pick up anything passed in EXTRA_REGRESS_OPTS.
+REGRESS_OPTS =  --expecteddir=$(srcdir) \
+  --dbname=ecpg1_regression,ecpg2_regression --create-role=regress_ecpg_user1,regress_ecpg_user2 \
+  $(EXTRA_REGRESS_OPTS)
 
 check: all
 	$(with_temp_install) ./pg_regress $(REGRESS_OPTS) --temp-instance=./tmp_check $(TEMP_CONF) --bindir= $(pg_regress_locale_flags) $(THREAD) --schedule=$(srcdir)/ecpg_schedule sql/twophase
diff --git a/src/interfaces/ecpg/test/pg_regress_ecpg.c b/src/interfaces/ecpg/test/pg_regress_ecpg.c
index f920af4560c..84e45ceebe0 100644
--- a/src/interfaces/ecpg/test/pg_regress_ecpg.c
+++ b/src/interfaces/ecpg/test/pg_regress_ecpg.c
@@ -164,7 +164,7 @@ ecpg_start_test(const char *testname,
 	char	   *appnameenv;
 
 	snprintf(inprg, sizeof(inprg), "%s/%s", inputdir, testname);
-	snprintf(insource, sizeof(insource), "%s.c", testname);
+	snprintf(insource, sizeof(insource), "%s/%s.c", inputdir, testname);
 
 	/* make a version of the test name that has dashes in place of slashes */
 	initStringInfo(_dash);
@@ -177,13 +177,13 @@ ecpg_start_test(const char *testname,
 
 	snprintf(expectfile_stdout, 

Re: [RFC] building postgres with meson -v

[Andres Freund]

Hi,

On 2021-10-31 16:24:48 -0700, Andres Freund wrote:
> - support for building docs.
>   I couldn't get dbtoepub work in a vpath style build, so I changed that
>   to also use pandoc. No idea if anybody uses the epub rules?

combing through various FIXMEs in the meson patch I started to look into docs
again. A few questions / noteworthy points:

- I still haven't gotten dbtoepub to work in vpath style builds (epub
  generation is instead using pandoc). Could somebody using these comment on
  the quality difference?

  We don't seem to offer these for download anywhere...

  Worth noting that dbtoepub takes approximately forever (>25min). Not that
  pandoc is fast, but ...

  Unfortunately pandoc doesn't understand  level stuff. That'd probably
  easy enough to tweak, but ...

- We have logic to change the man section (grep for sqlmansect) for some
  platforms. The only remaining platform is solaris. I'm inclined to not
  implement that.

- I've not implemented the texinfo targets - don't think they're really used?

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v10

[Bilal Yavuz]

Hi,

Sorry for the first email.

On Mon, 18 Jul 2022 at 23:23, Andres Freund  wrote:
>
> In
https://postgr.es/m/0e81e45c-c9a5-e95b-2782-ab2dfec8bf57%40enterprisedb.com
> On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote:
> > dff7b5a960 meson: prereq: regress: allow to specify director containing
> > expected files.
> >
> > This could use a bit more explanation, but it doesn't look
> > controversial so far.

While testing ECPG, C and exe files are generated by meson so these files
are in the meson's build directory but expected files are in the source
directory. However; there was no way to set different paths for inputs (C
and exe files') and expected files' directory. So, I added `--expecteddir`
to separately set expected files' directory.

Greetings,

Nazir Bilal Yavuz

Re: [RFC] building postgres with meson - v10

[Bilal Yavuz]

Hi,

On 2022-07-18 23:23:27 +0300, Andres Freund wrote:

> Bilal, Peter previously commented on the pg_regress change for ecpg,
> perhaps
> you can comment on that?
>
> In
> https://postgr.es/m/0e81e45c-c9a5-e95b-2782-ab2dfec8bf57%40enterprisedb.com
> On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote:
> > dff7b5a960 meson: prereq: regress: allow to specify director containing
> > expected files.
> >
> > This could use a bit more explanation, but it doesn't look
> > controversial so far


While testing ECPG, C and exe files are generated by meson so these files
are in the meson's build directory but expected files are in the source
directory. However; there was no way to set different paths for inputs (C
and exe files') and expected files' directory. So, I added `--expecteddir`
to separately set expected files' directory.

Greetings,

Nazir Bilal Yavuz

On Mon, 18 Jul 2022 at 23:23, Andres Freund  wrote:

> Hi,
>
> On 2022-07-18 11:33:09 +0200, Peter Eisentraut wrote:
> > The following patches are ok to commit IMO:
> >
> > a1c5542929 prereq: Deal with paths containing \ and spaces in
> basebackup_to_shell tests
> > e37951875d meson: prereq: psql: Output dir and dependency generation for
> sql_help
> > 18cc9fbd02 meson: prereq: ecpg: Add and use output directory argument
> for preproc/*.pl
> > 58a32694e9 meson: prereq: Move snowball_create.sql creation into perl
> file
> > 59b8bffdaf meson: prereq: Add output path arg in generate-lwlocknames.pl
> > 2db97b00d5 meson: prereq: generate-errcodes.pl: Accept output file
> > fb8f52f21d meson: prereq: unicode: Allow to specify output directory
> > 8f1e4410d6 meson: prereq: Refactor dtrace postprocessing make rules
> > 3d18a20b11 meson: prereq: Add --outdir option to gen_node_support.pl
>
> I pushed these. Thanks for the reviews and patches!
>
> The symbol export stuff has also been pushed (discussed in a separate
> thread).
>
> It's nice to see the meson patchset length reduced by this much.
>
> I pushed a rebased version of the remaining branches to git. I'll be on
> vacation for a bit, I'm not sure I can get a new version with further
> cleanups
> out before.
>
>
> Given that we can't use src/tools/gen_versioning_script.pl for the make
> build,
> due to not depending on perl for tarball builds, I'm inclined to rewrite it
> python (which we depend on via meson anyway) and consider it a meson
> specific
> wrapper?
>
>
> Bilal, Peter previously commented on the pg_regress change for ecpg,
> perhaps
> you can comment on that?
>
> In
> https://postgr.es/m/0e81e45c-c9a5-e95b-2782-ab2dfec8bf57%40enterprisedb.com
> On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote:
> > dff7b5a960 meson: prereq: regress: allow to specify director containing
> > expected files.
> >
> > This could use a bit more explanation, but it doesn't look
> > controversial so far.
>
> Greetings,
>
> Andres Freund
>

Re: [RFC] building postgres with meson - v10

[Andres Freund]

Hi,

On 2022-07-18 11:33:09 +0200, Peter Eisentraut wrote:
> The following patches are ok to commit IMO:
> 
> a1c5542929 prereq: Deal with paths containing \ and spaces in 
> basebackup_to_shell tests
> e37951875d meson: prereq: psql: Output dir and dependency generation for 
> sql_help
> 18cc9fbd02 meson: prereq: ecpg: Add and use output directory argument for 
> preproc/*.pl
> 58a32694e9 meson: prereq: Move snowball_create.sql creation into perl file
> 59b8bffdaf meson: prereq: Add output path arg in generate-lwlocknames.pl
> 2db97b00d5 meson: prereq: generate-errcodes.pl: Accept output file
> fb8f52f21d meson: prereq: unicode: Allow to specify output directory
> 8f1e4410d6 meson: prereq: Refactor dtrace postprocessing make rules
> 3d18a20b11 meson: prereq: Add --outdir option to gen_node_support.pl

I pushed these. Thanks for the reviews and patches!

The symbol export stuff has also been pushed (discussed in a separate thread).

It's nice to see the meson patchset length reduced by this much.

I pushed a rebased version of the remaining branches to git. I'll be on
vacation for a bit, I'm not sure I can get a new version with further cleanups
out before.


Given that we can't use src/tools/gen_versioning_script.pl for the make build,
due to not depending on perl for tarball builds, I'm inclined to rewrite it
python (which we depend on via meson anyway) and consider it a meson specific
wrapper?


Bilal, Peter previously commented on the pg_regress change for ecpg, perhaps
you can comment on that?

In https://postgr.es/m/0e81e45c-c9a5-e95b-2782-ab2dfec8bf57%40enterprisedb.com
On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote:
> dff7b5a960 meson: prereq: regress: allow to specify director containing
> expected files.
> 
> This could use a bit more explanation, but it doesn't look
> controversial so far.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v10

[Andres Freund]

Hi,

On 2022-07-18 11:12:10 +0300, Aleksander Alekseev wrote:
> > Just a quick question - is there a reason for changing the subject of
> > the emails?
> >
> > Not all email clients handle this well, e.g. Google Mail considers
> > this being 10 separate threads. The CF application and/or
> > pgsql-hackers@ archive also don't recognise this as a continuation of
> > the original thread. So all the discussions in -v8, -v9, -v9 ets
> > threads get lost.
> >
> > May I suggest using a single thread?
> 
> OK, the part about the archive is wrong - I scrolled right to the end
> of the thread, didn't notice v10 patch above and assumed it was lost.
> Sorry for the confusion. However, the part about various email clients
> is accurate.

For me the thread is too long to look through without some separation. I
wouldn't do the version in the subject for a small patchset / thread, but at
this size I think it's reasonable.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson - v10

[Peter Eisentraut]

On 15.07.22 07:08, Andres Freund wrote:

Attached is v10 of the meson patchset. Lots of small changes, I don't think
anything major. I tried to address most of Peter's feedback for the earlier
patches.


The following patches are ok to commit IMO:

a1c5542929 prereq: Deal with paths containing \ and spaces in 
basebackup_to_shell tests
e37951875d meson: prereq: psql: Output dir and dependency generation for 
sql_help
18cc9fbd02 meson: prereq: ecpg: Add and use output directory argument for 
preproc/*.pl
58a32694e9 meson: prereq: Move snowball_create.sql creation into perl file
59b8bffdaf meson: prereq: Add output path arg in generate-lwlocknames.pl
2db97b00d5 meson: prereq: generate-errcodes.pl: Accept output file
fb8f52f21d meson: prereq: unicode: Allow to specify output directory
8f1e4410d6 meson: prereq: Refactor dtrace postprocessing make rules
3d18a20b11 meson: prereq: Add --outdir option to gen_node_support.pl

Re: [RFC] building postgres with meson - v10

[Aleksander Alekseev]

Hi again,

> Just a quick question - is there a reason for changing the subject of
> the emails?
>
> Not all email clients handle this well, e.g. Google Mail considers
> this being 10 separate threads. The CF application and/or
> pgsql-hackers@ archive also don't recognise this as a continuation of
> the original thread. So all the discussions in -v8, -v9, -v9 ets
> threads get lost.
>
> May I suggest using a single thread?

OK, the part about the archive is wrong - I scrolled right to the end
of the thread, didn't notice v10 patch above and assumed it was lost.
Sorry for the confusion. However, the part about various email clients
is accurate.

-- 
Best regards,
Aleksander Alekseev

Re: [RFC] building postgres with meson - v10

[Aleksander Alekseev]

Hi Andres,

> Attached is v10 of the meson patchset. Lots of small changes, I don't think
> anything major. I tried to address most of Peter's feedback for the earlier
> patches.
>
> After this I plan to clean up the "export" patch, since that's I think the
> next bigger step, and an improvement on its own. The step after will be to
> discuss where we want the output of tests to reside, whether the naming scheme
> for tests is good etc.
>
> I did try to address Peter's criticism around inconsistency of the added
> parameters to perl scripts. I hope it's more consistent now. I used the
> opportunity to make src/tools/msvc use the "output directory" parameters,
> providing coverage for those paths (and removing a few unnecessary chdirs, but
> ...).

Thanks for continuing to work on this!

Just a quick question - is there a reason for changing the subject of
the emails?

Not all email clients handle this well, e.g. Google Mail considers
this being 10 separate threads. The CF application and/or
pgsql-hackers@ archive also don't recognise this as a continuation of
the original thread. So all the discussions in -v8, -v9, -v9 ets
threads get lost.

May I suggest using a single thread?

-- 
Best regards,
Aleksander Alekseev

Re: [RFC] building postgres with meson -v9

[Andres Freund]

Hi,

On 2022-07-13 08:39:45 +0200, Peter Eisentraut wrote:
> On 06.07.22 15:21, Andres Freund wrote:
> > > - This patch is for unifying the list of languages in NLS, as
> > >previously discussed:https://commitfest.postgresql.org/38/3737/
> > There seems little downside to doing so, so ...
> 
> This has been committed, so on the next rebase, the languages arguments can
> be removed from the i18n.gettext() calls.

That's done in v10 I posted yesterday.


On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote:
> 3bf5b317d5 meson: prereq: Specify output directory for psql sql help script.
> 2e5ed807f8 meson: prereq: ecpg: add and use output directory argument for
> parse.pl.
> 4e7fab01c5 meson: prereq: msvc: explicit output file for pgflex.pl
> cdcd3da4c4 meson: prereq: add output path arg in generate-lwlocknames.pl
> 1f655486e4 meson: prereq: generate-errcodes.pl: accept output file
> e834c48758 meson: prereq: unicode: allow to specify output directory.
> 
> You said you are still finalizing these.  I think we can move ahead with
> these once that is done.

I tried to address the pending things in these patches.


> 9f4a9b1749 meson: prereq: move snowball_create.sql creation into perl file.
> 
> This looks ready, except I think it needs to be hooked into the
> distprep target, since it's now a Perl script running at build time.

Done. I vacillated between doing the distprep rule in
src/backend/snowball/Makefile (as most things outside of the backend do) and
src/backend/Makefile (most backend things). Ended up with doing it in
snowball/Makefile.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v9

[Andres Freund]

Hi,

On 2022-07-07 12:09:32 +0200, Peter Eisentraut wrote:
> On 06.07.22 15:21, Andres Freund wrote:
> > > Here is my rough assessment of where we are with this patch set:
> > > 
> > > 08b4330ded prereq: deal with \ paths in basebackup_to_shell tests.
> > > 
> > > This still needs clarification, per my previous review.
> > Hm. I thought I had explained that bit, but apparently not. Well, it's 
> > pretty
> > simple - without this, the test fail on windows for me, as soon as one of 
> > the
> > binaries is in a directory with spaces (which is common on windows). 
> > Iimagine
> > what happens with e.g.
> >qq{$gzip --fast > "$escaped_backup_path%f.gz"}
> > if $gzip contains spaces.
> > 
> > 
> > This doesn't happen currently on CI because nothing runs these tests on
> > windows yet.
> 
> Hmm, maybe this patch looked different the last time I saw it.

Don't think it had changed since I wrote it first.


> The quoting of "$gzip" is clearly necessary.
> 
> What about the backslash replacements s{\\}{/}g ?  Is that also required for
> passing the path through the shell?

I don't recall the details, but it's definitely needed for embedding it into
postgresql.conf. We could double the escapes instead, but that doesn't seem an
improvement (and wouldn't work when passing it to the shell anymore).


> If so, the treatment of $tar in that
> way doesn't seem necessary, since that doesn't get called through an
> intermediate shell.  (That would then also explain why $gzip in the
> pg_basebackup tests doesn't require that treatment, which had previously
> confused me.)

Yea, it doesn't immediately look like it's needed, and the test passes without
it. I guess I might just have tried to be complete...

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v9

[Peter Eisentraut]

On 06.07.22 15:21, Andres Freund wrote:

bda6a45bae meson: prereq: Refactor PG_TEST_EXTRA logic in autoconf build

I understand the intention behind this, but I think it changes the
behavior in an undesirable way.  Before this patch, you can go into
src/test/ssl/ and run make check manually.  This was indeed the only
way to do it before PG_TEST_EXTRA.  With this patch, this would now
skip all the tests unless you set PG_TEST_EXTRA, even if you run the
specific test directly.

It's not a free lunch, I agree. But I think the downsides outweigh the upsides
by far. Not seeing that tests were skipped in the test output is quite
problematic imo. And with meson's testrunner we're going to need something
that deals with skipping these tests - and it's more important to have them
skipped, so that they show up in the test summary.

It's not like it's hard to set PG_TEST_EXTRA for a single command invocation?


It's probably ok.  I have it set in my environment all the time anyway, 
so it wouldn't affect me.  But it's the sort of thing people might be 
particular about, so I thought it was worth pointing out.

Re: [RFC] building postgres with meson -v9

[Peter Eisentraut]

On 06.07.22 15:21, Andres Freund wrote:

- This patch is for unifying the list of languages in NLS, as
   previously discussed:https://commitfest.postgresql.org/38/3737/

There seems little downside to doing so, so ...


This has been committed, so on the next rebase, the languages arguments 
can be removed from the i18n.gettext() calls.

Re: [RFC] building postgres with meson -v9

[Peter Eisentraut]

On 06.07.22 15:21, Andres Freund wrote:

Here is my rough assessment of where we are with this patch set:

08b4330ded prereq: deal with \ paths in basebackup_to_shell tests.

This still needs clarification, per my previous review.

Hm. I thought I had explained that bit, but apparently not. Well, it's pretty
simple - without this, the test fail on windows for me, as soon as one of the
binaries is in a directory with spaces (which is common on windows). Iimagine
what happens with e.g.
   qq{$gzip --fast > "$escaped_backup_path%f.gz"}
if $gzip contains spaces.


This doesn't happen currently on CI because nothing runs these tests on
windows yet.


Hmm, maybe this patch looked different the last time I saw it.  I see 
your point.


The quoting of "$gzip" is clearly necessary.

What about the backslash replacements s{\\}{/}g ?  Is that also required 
for passing the path through the shell?  If so, the treatment of $tar in 
that way doesn't seem necessary, since that doesn't get called through 
an intermediate shell.  (That would then also explain why $gzip in the 
pg_basebackup tests doesn't require that treatment, which had previously 
confused me.)


If my understanding of this is correct, then I suggest:

1. Add a comment to the $gzip =~ s{\\}{/}g ... line.
2. Remove the $tar =~ s{\\}{/}g ... line.
3. Backpatch to PG15.

Re: [RFC] building postgres with meson -v9

[Andres Freund]

Hi

On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote:
> On 01.07.22 11:33, Andres Freund wrote:
> > Attached is an updated version of the meson patchset. There has been a 
> > steady
> > stream of incremental work over the last month, with patches from Peter
> > Eisentraut and Nazir Yavuz.
> > 
> > I tried to address the review comments Peter had downthread about the prep
> > patches. The one that I know is still outstanding is that there's still
> > different ways of passing output directories as parameters to a bunch of
> > scripts added, will resolve that next (some have been fixed).
> 
> Here is my rough assessment of where we are with this patch set:
> 
> 08b4330ded prereq: deal with \ paths in basebackup_to_shell tests.
> 
> This still needs clarification, per my previous review.

Hm. I thought I had explained that bit, but apparently not. Well, it's pretty
simple - without this, the test fail on windows for me, as soon as one of the
binaries is in a directory with spaces (which is common on windows). Iimagine
what happens with e.g.
  qq{$gzip --fast > "$escaped_backup_path%f.gz"}
if $gzip contains spaces.


This doesn't happen currently on CI because nothing runs these tests on
windows yet.


> bda6a45bae meson: prereq: Refactor PG_TEST_EXTRA logic in autoconf build
> 
> I understand the intention behind this, but I think it changes the
> behavior in an undesirable way.  Before this patch, you can go into
> src/test/ssl/ and run make check manually.  This was indeed the only
> way to do it before PG_TEST_EXTRA.  With this patch, this would now
> skip all the tests unless you set PG_TEST_EXTRA, even if you run the
> specific test directly.

It's not a free lunch, I agree. But I think the downsides outweigh the upsides
by far. Not seeing that tests were skipped in the test output is quite
problematic imo. And with meson's testrunner we're going to need something
that deals with skipping these tests - and it's more important to have them
skipped, so that they show up in the test summary.

It's not like it's hard to set PG_TEST_EXTRA for a single command invocation?



> 243f99da38 wip: split TESTDIR into two.
> 
> This one has already caused a bit of confusion, but the explanation at
> 
> https://www.postgresql.org/message-id/flat/2022060122.td2ato4wjqf7afnv%40alap3.anarazel.de#1f250dee73cf0da29a6d2c020c3bde08
> 
> seems reasonable.  But it clearly needs further work.

Yea. I kind of want to get some of the preparatory stuff out of the way first.


> 88dd280835 meson: Add meson based buildsystem.
> 1ee3073a3c meson: ci: Build both with meson and as before.
> 
> These are for later. ;-)
> 
> In the meantime, also of interest to this effort:
> 
> - If we're planning to remove the postmaster symlink in PG16, maybe we
>   should start a discussion on that.

Yea.


> - This patch is for unifying the list of languages in NLS, as
>   previously discussed: https://commitfest.postgresql.org/38/3737/

There seems little downside to doing so, so ...

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v9

[Peter Eisentraut]

On 01.07.22 11:33, Andres Freund wrote:

Attached is an updated version of the meson patchset. There has been a steady
stream of incremental work over the last month, with patches from Peter
Eisentraut and Nazir Yavuz.

I tried to address the review comments Peter had downthread about the prep
patches. The one that I know is still outstanding is that there's still
different ways of passing output directories as parameters to a bunch of
scripts added, will resolve that next (some have been fixed).


Here is my rough assessment of where we are with this patch set:

08b4330ded prereq: deal with \ paths in basebackup_to_shell tests.

This still needs clarification, per my previous review.

3bf5b317d5 meson: prereq: Specify output directory for psql sql help script.
2e5ed807f8 meson: prereq: ecpg: add and use output directory argument 
for parse.pl.

4e7fab01c5 meson: prereq: msvc: explicit output file for pgflex.pl
cdcd3da4c4 meson: prereq: add output path arg in generate-lwlocknames.pl
1f655486e4 meson: prereq: generate-errcodes.pl: accept output file
e834c48758 meson: prereq: unicode: allow to specify output directory.

You said you are still finalizing these.  I think we can move ahead with 
these once that is done.


9f4a9b1749 meson: prereq: move snowball_create.sql creation into perl file.

This looks ready, except I think it needs to be hooked into the
distprep target, since it's now a Perl script running at build time.

8951a6721e meson: prereq: Refactor dtrace postprocessing make rules

This looks ready.

bda6a45bae meson: prereq: Refactor PG_TEST_EXTRA logic in autoconf build

I understand the intention behind this, but I think it changes the
behavior in an undesirable way.  Before this patch, you can go into
src/test/ssl/ and run make check manually.  This was indeed the only
way to do it before PG_TEST_EXTRA.  With this patch, this would now
skip all the tests unless you set PG_TEST_EXTRA, even if you run the
specific test directly.

I think this needs a different idea.

eb852cc023 meson: prereq: Can we get away with not export-all'ing libraries?

This is also at , which
hasn't seen any activity in a while.  I think this needs a resolution
one way or the other before we can proceed to the main act.

2cc276ced6 meson: prereq: add src/tools/gen_versioning_script.pl.

Note that in the make build system we can only use perl before
distprep.  So it's not clear whether a script like this would help
unify the code.  Of course, we could still use it with the
understanding that it will be separate.

351ac51a89 meson: prereq: remove LLVM_CONFIG from Makefile.global.in

This can be committed.  AFAICT, LLVM_CONFIG is only used within
configure.

dff7b5a960 meson: prereq: regress: allow to specify director containing 
expected files.


This could use a bit more explanation, but it doesn't look
controversial so far.

243f99da38 wip: split TESTDIR into two.

This one has already caused a bit of confusion, but the explanation at

https://www.postgresql.org/message-id/flat/2022060122.td2ato4wjqf7afnv%40alap3.anarazel.de#1f250dee73cf0da29a6d2c020c3bde08

seems reasonable.  But it clearly needs further work.

88dd280835 meson: Add meson based buildsystem.
1ee3073a3c meson: ci: Build both with meson and as before.

These are for later. ;-)

In the meantime, also of interest to this effort:

- If we're planning to remove the postmaster symlink in PG16, maybe we
  should start a discussion on that.

- This patch is for unifying the list of languages in NLS, as
  previously discussed: https://commitfest.postgresql.org/38/3737/

Re: [RFC] building postgres with meson -v9

[Andres Freund]

Hi,

On 2022-07-01 14:01:11 -0500, Justin Pryzby wrote:
> vcvarsall isn't needed in cirrus' "check_world" scripts.

E.g. for the ecpg tests it isn't, except that we don't currently build ecpg on
windows. But I plan to fix that.


> I'm missing any way to re/run cirrus only for msbuild OR ninja OR homegrown
> with something more granular than "ci-os-only: windows".  (The same thing
> applies to the mingw patch).

Not sure that's really worth adding - I don't forsee merging all those
tasks. But I'm open to proposals.


> I'll mail shortly about ccache.

There's a meson PR that should fix some of the issues, need to test it...

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v9

[Justin Pryzby]

vcvarsall isn't needed in cirrus' "check_world" scripts.

I'm missing any way to re/run cirrus only for msbuild OR ninja OR homegrown
with something more granular than "ci-os-only: windows".  (The same thing
applies to the mingw patch).

I'll mail shortly about ccache.

-- 
Justin

Re: [RFC] building postgres with meson -v8

[Andres Freund]

On 2022-06-14 20:47:59 +0200, Peter Eisentraut wrote:
> On 14.06.22 20:27, Andres Freund wrote:
> > One thing I'm not quite sure about: Why does the makefile need awareness of
> > the stop files, but Install.pm doesn't? I suspect currently the patch leads 
> > to
> > stopwords not being installed on windows anymore?
> 
> Install.pm contains this elsewhere:
> 
> GenerateTsearchFiles($target);
> CopySetOfFiles(
> 'Stopword files',
> [ glob("src\\backend\\snowball\\stopwords\\*.stop") ],
> $target . '/share/tsearch_data/');
> 
> It's a bit confusing that the "generate" function that we are patching also
> installs some of the files right away, while the rest is installed by the
> calling function.

Ugh, that's confusing indeed. Thanks for the explanation.

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

On 14.06.22 20:27, Andres Freund wrote:

One thing I'm not quite sure about: Why does the makefile need awareness of
the stop files, but Install.pm doesn't? I suspect currently the patch leads to
stopwords not being installed on windows anymore?


Install.pm contains this elsewhere:

GenerateTsearchFiles($target);
CopySetOfFiles(
'Stopword files',
[ glob("src\\backend\\snowball\\stopwords\\*.stop") ],
$target . '/share/tsearch_data/');

It's a bit confusing that the "generate" function that we are patching 
also installs some of the files right away, while the rest is installed 
by the calling function.

Re: [RFC] building postgres with meson -v8

[Andres Freund]

Hi,

On 2022-06-08 14:33:16 +0200, Peter Eisentraut wrote:
> Attached is a patch the finishes up the work to move the snowball SQL script
> generation into a separate script.

That looks good, merged. I did split the commit, because there's not yet a
meson.build "at the time" of the prereq: commits.

One thing I'm not quite sure about: Why does the makefile need awareness of
the stop files, but Install.pm doesn't? I suspect currently the patch leads to
stopwords not being installed on windows anymore?

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v8

[Andres Freund]

Hi,

On 2022-06-08 08:27:06 +0200, Peter Eisentraut wrote:
> I looked at some of the "prereq" patches again to see what state they are
> in:
> 
> commit 351a12f48e395b31cce4aca239b934174b36ea9d
> Author: Andres Freund 
> Date:   Wed Apr 20 22:46:54 2022
> 
> prereq: deal with \ paths in basebackup_to_shell tests.
> 
> This is a new component in PG15, so a fix might be in scope for PG15 too.

Yea, I should probably post that to the relevant thread. I think at that point
I was just trying to get a rebase not to fail anymore...


> But I don't know if this change is really necessary.  There are other tests
> that use the GZIP and TAR environment variables (e.g., pg_verifybackup).  If
> this is a problem there too, we should think of a general solution.  If not,
> it could use some explanation.

I got failures on windows without it - which we just don't see on windows
because currently nothing runs these tests :(. The pg_verifybackup case likely
is unproblematic because it uses the array form of building subcommands,
instead of string interpolation.


> commit c00642483a53f4ee6e351085c7628363c293ee61
> Author: Andres Freund 
> Date:   Fri Mar 25 21:44:48 2022
> 
> meson: prereq: unicode: allow to specify output directory.
> 
> OK with attached fixup (but see below).

Merged.


> commit 31313056e153e099f236a29b752f7610c4f7764f
> Author: Andres Freund 
> Date:   Thu Jan 20 08:36:50 2022
> 
> meson: prereq: generate-errcodes.pl: accept output file
> 
> This is ok, but seems unnecessary, since meson can capture the output of a
> single file.  (See also similar script generate-errcodes-table.pl in doc/,
> which uses capture.)

Not sure why I didn't do that. It might be because the meson capture stuff has
a noticable overhead, particularly on windows, because it starts up a python
interpreter. Since nearly the whole build depends on generate-errcodes.pl to
have run...


> commit e4e77c0e20f3532be4ed270a7cf8b965b7cafa49
> Author: Andres Freund 
> Date:   Thu Jan 20 08:36:50 2022
> 
> meson: prereq: add output path arg in generate-lwlocknames.pl
> 
> We should make the command-line interface here the same as the unicode
> script: Either make the output directory a positional argument or an option.
> I don't have a strong feeling about it either way, but perhaps the solution
> with the option is more elegant and would also not require changing the
> makefiles.

I don't really have an opinion what's better here, so I'll go with your
preference / the option.


> Also, we should decide on short or long option: The code
> declares a long option, but the build uses a short option. It's confusing
> that that even works.

Getopt::Long auto-generates short options afaict...


> commit 7866620afa65223f6e657da972f501615fd32d3b
> Author: Andres Freund 
> Date:   Wed Apr 20 21:01:31 2022
> 
> meson: prereq: output and depencency tracking work.
> 
> This could be split into multiple parts with more detailed explanations.  I
> see where you're going but not everything is fully clear to me (especially
> the guc-file.c.h stuff).

Will take a stab at doing so.


> From 51c6d3544ae9e652c7aac26102a8bf5a116fb182 Mon Sep 17 00:00:00 2001
> From: Peter Eisentraut 
> Date: Tue, 7 Jun 2022 22:54:26 +0200
> Subject: [PATCH] fixup! meson: prereq: unicode: allow to specify output
>  directory.

Merged.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

Attached is a patch the finishes up the work to move the snowball SQL 
script generation into a separate script.From 02ca51dfb918666dfde8e48499a4c73afae4e89e Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 8 Jun 2022 11:05:31 +0200
Subject: [PATCH] fixup! meson: prereq: move snowball_create.sql creation into
 perl file.

- Remove LANGUAGES list from Makefile, keep in Perl script.
- Add list of stopword files to install.
- Make depfile generation optional.
- Call new Perl script from Install.pm.
---
 src/backend/snowball/Makefile   | 81 -
 src/backend/snowball/meson.build|  3 +-
 src/backend/snowball/snowball_create.pl | 73 +-
 src/tools/msvc/Install.pm   | 36 +--
 4 files changed, 86 insertions(+), 107 deletions(-)

diff --git a/src/backend/snowball/Makefile b/src/backend/snowball/Makefile
index 259104f8eb..c12a77055d 100644
--- a/src/backend/snowball/Makefile
+++ b/src/backend/snowball/Makefile
@@ -72,40 +72,22 @@ OBJS += \
stem_UTF_8_turkish.o \
stem_UTF_8_yiddish.o
 
-# first column is language name and also name of dictionary for not-all-ASCII
-# words, second is name of dictionary for all-ASCII words
-# Note order dependency: use of some other language as ASCII dictionary
-# must come after creation of that language
-LANGUAGES=  \
-   arabic  arabic  \
-   armenianarmenian\
-   basque  basque  \
-   catalan catalan \
-   danish  danish  \
-   dutch   dutch   \
-   english english \
-   finnish finnish \
-   french  french  \
-   german  german  \
-   greek   greek   \
-   hindi   english \
-   hungarian   hungarian   \
-   indonesian  indonesian  \
-   irish   irish   \
-   italian italian \
-   lithuanian  lithuanian  \
-   nepali  nepali  \
-   norwegian   norwegian   \
-   portuguese  portuguese  \
-   romanianromanian\
-   russian english \
-   serbian serbian \
-   spanish spanish \
-   swedish swedish \
-   tamil   tamil   \
-   turkish turkish \
-   yiddish yiddish
-
+stop_files = \
+   danish.stop \
+   dutch.stop \
+   english.stop \
+   finnish.stop \
+   french.stop \
+   german.stop \
+   hungarian.stop \
+   italian.stop \
+   nepali.stop \
+   norwegian.stop \
+   portuguese.stop \
+   russian.stop \
+   spanish.stop \
+   swedish.stop \
+   turkish.stop
 
 SQLSCRIPT= snowball_create.sql
 DICTDIR=tsearch_data
@@ -119,35 +101,22 @@ all: all-shared-lib $(SQLSCRIPT)
 
 include $(top_srcdir)/src/Makefile.shlib
 
-$(SQLSCRIPT): snowball_create.pl Makefile snowball_func.sql.in snowball.sql.in
+$(SQLSCRIPT): snowball_create.pl snowball_func.sql.in snowball.sql.in
$(PERL) $< --input ${srcdir} --output .
 
 install: all installdirs install-lib
$(INSTALL_DATA) $(SQLSCRIPT) '$(DESTDIR)$(datadir)'
-   @set -e; \
-   set $(LANGUAGES) ; \
-   while [ "$$#" -gt 0 ] ; \
-   do \
-   lang=$$1; shift; shift; \
-   if [ -s $(srcdir)/stopwords/$${lang}.stop ] ; then \
-   $(INSTALL_DATA) $(srcdir)/stopwords/$${lang}.stop 
'$(DESTDIR)$(datadir)/$(DICTDIR)' ; \
-   fi \
-   done
+   $(INSTALL_DATA) $(addprefix $(srcdir)/stopwords/,$(stop_files)) 
'$(DESTDIR)$(datadir)/$(DICTDIR)'
 
 installdirs: installdirs-lib
$(MKDIR_P) '$(DESTDIR)$(datadir)' '$(DESTDIR)$(datadir)/$(DICTDIR)'
 
 uninstall: uninstall-lib
rm -f '$(DESTDIR)$(datadir)/$(SQLSCRIPT)'
-   @set -e; \
-   set $(LANGUAGES) ; \
-   while [ "$$#" -gt 0 ] ; \
-   do \
-   lang=$$1; shift; shift;  \
-   if [ -s $(srcdir)/stopwords/$${lang}.stop ] ; then \
-   rm -f '$(DESTDIR)$(datadir)/$(DICTDIR)/'$${lang}.stop ; \
-   fi \
-   done
-
-clean distclean maintainer-clean: clean-lib
-   rm -f $(OBJS) $(SQLSCRIPT) snowball_create.dep
+   rm -f $(addprefix '$(DESTDIR)$(datadir)/$(DICTDIR)/',$(stop_files))
+
+clean distclean: clean-lib
+   rm -f $(OBJS)
+
+maintainer-clean: distclean
+   rm -f $(SQLSCRIPT)
diff --git a/src/backend/snowball/meson.build b/src/backend/snowball/meson.build
index 30aed714dc..c6326380e1 100644
--- a/src/backend/snowball/meson.build
+++ b/src/backend/snowball/meson.build
@@ -70,12 +70,11 @@ snowball_create = custom_target('snowball_create',
   input: ['snowball_create.pl'],
   output: ['snowball_create.sql'],
   depfile: 'snowball_create.dep',
-  command: [perl, 

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

I looked at some of the "prereq" patches again to see what state they 
are in:


commit 351a12f48e395b31cce4aca239b934174b36ea9d
Author: Andres Freund 
Date:   Wed Apr 20 22:46:54 2022

prereq: deal with \ paths in basebackup_to_shell tests.

This is a new component in PG15, so a fix might be in scope for PG15 
too.  But I don't know if this change is really necessary.  There are 
other tests that use the GZIP and TAR environment variables (e.g., 
pg_verifybackup).  If this is a problem there too, we should think of a 
general solution.  If not, it could use some explanation.



commit c00642483a53f4ee6e351085c7628363c293ee61
Author: Andres Freund 
Date:   Fri Mar 25 21:44:48 2022

meson: prereq: unicode: allow to specify output directory.

OK with attached fixup (but see below).


commit 31313056e153e099f236a29b752f7610c4f7764f
Author: Andres Freund 
Date:   Thu Jan 20 08:36:50 2022

meson: prereq: generate-errcodes.pl: accept output file

This is ok, but seems unnecessary, since meson can capture the output of 
a single file.  (See also similar script generate-errcodes-table.pl in 
doc/, which uses capture.)



commit e4e77c0e20f3532be4ed270a7cf8b965b7cafa49
Author: Andres Freund 
Date:   Thu Jan 20 08:36:50 2022

meson: prereq: add output path arg in generate-lwlocknames.pl

We should make the command-line interface here the same as the unicode 
script: Either make the output directory a positional argument or an 
option. I don't have a strong feeling about it either way, but perhaps 
the solution with the option is more elegant and would also not require 
changing the makefiles.  Also, we should decide on short or long option: 
The code declares a long option, but the build uses a short option. 
It's confusing that that even works.



commit 7866620afa65223f6e657da972f501615fd32d3b
Author: Andres Freund 
Date:   Wed Apr 20 21:01:31 2022

meson: prereq: output and depencency tracking work.

This could be split into multiple parts with more detailed explanations. 
 I see where you're going but not everything is fully clear to me 
(especially the guc-file.c.h stuff).From 51c6d3544ae9e652c7aac26102a8bf5a116fb182 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Tue, 7 Jun 2022 22:54:26 +0200
Subject: [PATCH] fixup! meson: prereq: unicode: allow to specify output
 directory.

---
 src/common/unicode/generate-unicode_norm_table.pl | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/common/unicode/generate-unicode_norm_table.pl 
b/src/common/unicode/generate-unicode_norm_table.pl
index 3d0c3f0ebc..ed4181fce6 100644
--- a/src/common/unicode/generate-unicode_norm_table.pl
+++ b/src/common/unicode/generate-unicode_norm_table.pl
@@ -40,7 +40,7 @@
 my @characters = ();
 my %character_hash = ();
 open($FH, '<', "$directory/UnicodeData.txt")
-  or die "Could not open UnicodeData.txt: $!.";
+  or die "Could not open $directory/UnicodeData.txt: $!.";
 while (my $line = <$FH>)
 {
 
-- 
2.36.1

Re: [RFC] building postgres with meson -v8

[Michael Paquier]

On Tue, May 24, 2022 at 08:08:26PM +0200, Peter Eisentraut wrote:
> On 18.05.22 21:48, Andres Freund wrote:
>> - GETTIMEOFDAY_1ARG - test doesn't exist - I suspect it might not be 
>> necessary
> 
> Might be obsolete, consider removing.

I just came across this one independently of what you are doing for
meson, and based on a lookup of the buildfarm, I think that it can be
removed.  One reference about GETTIMEOFDAY_1ARG on the -hackers list
comes from here, from 20 years ago:
https://www.postgresql.org/message-id/a1eeu5$1koe$1...@news.tht.net
--
Michael


signature.asc
Description: PGP signature

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-06-03 12:35:45 +0300, Aleksander Alekseev wrote:
> > Part of the patch series is to use visibility attributes, and your #if
> > TS_EMPTY(PGDLLEXPORT) thing can't handle that.
> 
> Out of curiosity, how come a patchset that adds an alternative build system
> changes the visibility attributes?

It was the simplest path - on windows (and AIx) extension symbols need to be
explicitly exported. We did that by building the objects constituting
extension libraries, collecting the symbols, generating an export file with
all the symbols, which then is passed to the linker.  It was a lot less work
to just add the necessary PGDLLEXPORT annotations than make that export file
generation work for extensions.


> I would guess they should be the same for both Autotools and Meson.

It is, the patch adds it to both.


> Is it necessary in order to make Meson work?

Yes, or at least the simplest path.


> If not, maybe it should be a separate patch.

It is.

https://commitfest.postgresql.org/38/3396/

Greetings,

Andres Freund

Re: [RFC] building postgres with meson

[Aleksander Alekseev]

Hi hackers,

> See about line 300 in src/backend/Makefile. In any case, it's
> easy to show by experiment that it does.

`make maintainer-clean` did the trick, thanks. I suggest modifying meson.build
accordingly:

-run make distclean in the source tree.
+run `make maintainer-clean` in the source tree.

> I think this is timescale's issue. Why are you defining / undefining
> PGDLLEXPORT?

That's a great question.

As I understand some time ago the developers had a problem with a collision of
exported symbols on *nix platforms [1] and chose to solve it by re-defining
PGDLLEXPORT to __attribute__((visibility ("default"))) for GCC and CLang.
I agree that this is a questionable approach. Redefining a macro provided
by Postgres doesn't strike me as a good idea. I tried to remove this
re-definition, but it didn't go well [2]. So apparently it should be addressed
somehow differently.

> Part of the patch series is to use visibility attributes, and your #if
> TS_EMPTY(PGDLLEXPORT) thing can't handle that.

Out of curiosity, how come a patchset that adds an alternative build system
changes the visibility attributes? I would guess they should be the same
for both Autotools and Meson. Is it necessary in order to make Meson work?
If not, maybe it should be a separate patch.

[1]: 
https://github.com/timescale/timescaledb/commit/027b7b29420a742d7615c70d9f19b2b99c488c2c
[2]: https://github.com/timescale/timescaledb/pull/4413

-- 
Best regards,
Aleksander Alekseev

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-06-02 15:53:50 -0400, Tom Lane wrote:
> Andres Freund  writes:
> > Oh. I executed maintainer-clean inside src/backend/parser/, and thus didn't
> > see it getting cleaned up.
> 
> > It seems pretty darn grotty that src/backend/parser/Makefile explicitly 
> > states
> > that gram.c ... aren't cleaned "here", but then src/backend/Makefile does
> > clean them up.
> 
> I agree the factorization of this ain't great.  I'd think about improving
> it, were it not that we're trying to get rid of it.

+1. I think I just wanted to excuse my confusion...


> (But with meson, the whole idea of building or cleaning just part of the
> tree is out the window anyway, no?)

Cleaning parts of the tree isn't supported as far as I know (not that I've
needed it). You can build parts of the tree by specifying the target
(e.g. ninja src/backend/postgres) or by specifying meta-targets (e.g. ninja
contrib backend). I've thought about contributing a patch to meson to
automatically generate targets for each directory that has sub-targets - it's
just a few lines.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson

[Tom Lane]

Andres Freund  writes:
> Oh. I executed maintainer-clean inside src/backend/parser/, and thus didn't
> see it getting cleaned up.

> It seems pretty darn grotty that src/backend/parser/Makefile explicitly states
> that gram.c ... aren't cleaned "here", but then src/backend/Makefile does
> clean them up.

I agree the factorization of this ain't great.  I'd think about improving
it, were it not that we're trying to get rid of it.

(But with meson, the whole idea of building or cleaning just part of the
tree is out the window anyway, no?)

regards, tom lane

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-06-02 15:05:10 -0400, Tom Lane wrote:
> Andres Freund  writes:
> > On 2022-06-02 13:33:51 -0400, Tom Lane wrote:
> >> Andres Freund  writes:
> >>> (*) do we really not have a target that removes bison / flex output?
> 
> >> maintainer-clean
> 
> > Don't think so:
> 
> See about line 300 in src/backend/Makefile.  In any case, it's
> easy to show by experiment that it does.
> 
> $ make maintainer-clean
> ...
> $ git status --ignored
> On branch master
> Your branch is up to date with 'origin/master'.
> 
> nothing to commit, working tree clean

Oh. I executed maintainer-clean inside src/backend/parser/, and thus didn't
see it getting cleaned up.

It seems pretty darn grotty that src/backend/parser/Makefile explicitly states
that gram.c ... aren't cleaned "here", but then src/backend/Makefile does
clean them up.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson

[Tom Lane]

Andres Freund  writes:
> On 2022-06-02 13:33:51 -0400, Tom Lane wrote:
>> Andres Freund  writes:
>>> (*) do we really not have a target that removes bison / flex output?

>> maintainer-clean

> Don't think so:

See about line 300 in src/backend/Makefile.  In any case, it's
easy to show by experiment that it does.

$ make maintainer-clean
...
$ git status --ignored
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean

regards, tom lane

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-06-02 13:33:51 -0400, Tom Lane wrote:
> Andres Freund  writes:
> > (*) do we really not have a target that removes bison / flex output?
> 
> maintainer-clean

Don't think so:

# gram.c, gram.h, and scan.c are in the distribution tarball, so they
# are not cleaned here.
clean distclean maintainer-clean:
rm -f lex.backup

Greetings,

Andres Freund

Re: [RFC] building postgres with meson

[Tom Lane]

Andres Freund  writes:
> (*) do we really not have a target that removes bison / flex output?

maintainer-clean

regards, tom lane

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-06-02 13:08:49 -0400, Tom Lane wrote:
> Andres Freund  writes:
> > I'm not quite sure what the proper behaviour is when doing an out-of-tree
> > build with meson (all builds are out-of-tree), with a pre-existing flex /
> > bison output in the source tree that is out of date.
> 
> Definitely sounds like a gotcha.
> 
> On the one hand, there's been some discussion already of removing all
> derived files from tarballs and just insisting that users provide all
> needed tools when building from source.  If we did that, it could be
> sufficient for the meson build to check that no such files are present
> in the source tree.  (Checking a couple of them would be enough, likely.)

There already is a check for pg_config.h, so the most obvious source of this
is addressed. Just didn't think about the files that make clean doesn't remove
:/.


> On the other hand, I'm not sure that I want such a change to be forced
> by a toolchain change.  It definitely seems a bit contrary to the plan
> we'd formed of allowing meson and make-based builds to coexist for
> a few years, because we'd be breaking at least some make-based build
> processes.

Agreed. I think it'd be pretty reasonable to not include flex / bison
output. They're not hard to acquire. The docs are perhaps another story.

I think it might be fine to say that make reallyclean (*) is required if
there's some conflicting in-source tree file?


> Could we have the meson build check that, say, if gram.c exists it
> is newer than gram.y?  Or get it to ignore an in-tree gram.c?

I suspect the problem with ignoring is gram.h, that's probably a bit harder to
ignore. Right now I'm leaning towards either always erroring out if there's
bison/flex output in the source tree (with a hint towards make
reallyclean(*)), or erroring out if they're out of date (again with a hint
towards reallyclean)?

Alternatively we could just remove the generated .c/h files from the source
dir, as a part of regenerating them in the build dir? But I like the idea of
the source dir being readonly outside of explicit targets modifying sources
(e.g. update-unicode or such).

Greetings,

Andres Freund

(*) do we really not have a target that removes bison / flex output?

Re: [RFC] building postgres with meson

[Tom Lane]

Andres Freund  writes:
> I'm not quite sure what the proper behaviour is when doing an out-of-tree
> build with meson (all builds are out-of-tree), with a pre-existing flex /
> bison output in the source tree that is out of date.

Definitely sounds like a gotcha.

On the one hand, there's been some discussion already of removing all
derived files from tarballs and just insisting that users provide all
needed tools when building from source.  If we did that, it could be
sufficient for the meson build to check that no such files are present
in the source tree.  (Checking a couple of them would be enough, likely.)

On the other hand, I'm not sure that I want such a change to be forced
by a toolchain change.  It definitely seems a bit contrary to the plan
we'd formed of allowing meson and make-based builds to coexist for
a few years, because we'd be breaking at least some make-based build
processes.

Could we have the meson build check that, say, if gram.c exists it
is newer than gram.y?  Or get it to ignore an in-tree gram.c?

regards, tom lane

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-06-01 12:39:50 +0300, Aleksander Alekseev wrote:
> > > ```
> > > ../src/include/parser/kwlist.h:332:25: error: ‘PARAMETER’ undeclared here 
> > > (not
> > > in a function)
> > > 332 | PG_KEYWORD("parameter", PARAMETER, UNRESERVED_KEYWORD, BARE_LABEL)
> > >
> > > ../src/interfaces/ecpg/preproc/keywords.c:32:55: note: in definition of 
> > > macro
> > > ‘PG_KEYWORD’
> > > 32 | #define PG_KEYWORD(kwname, value, category, collabel) value,
> > > ```
> >
> > Huh. I've not seen this before - could you provide a bit more detail about
> > what you did? CI isn't testing ubuntu, but it is testing Debian, so I'd 
> > expect
> > this to work.
> 
> I used PIP to install Meson, since the default APT package is too old, v0.53:
> 
> $ pip3 install --user meson
> $ meson --version
> 0.62.1
> $ ninja --version
> 1.10.0
> 
> The branch was checked out as it was described in the first email.
> Then to reproduce the issue:
> 
> $ git status
> On branch meson
> Your branch is up to date with 'andres/meson'.
> $ git fetch andres
> $ git rebase -i andres/meson
> $ meson setup build --buildtype debug
> $ cd build
> $ ninja
> 
> This is pretty much the default Ubuntu 20.04.4 LTS system with all the
> recent updates installed, so it shouldn't be a problem to reproduce
> the issue with a VM.

Chatting with a colleague (who unbeknownst to me hit something similar in the
past) I think we figured it out. It's not due to Ubuntu 20.04 or such. It's
likely due to previously having an in-tree build with autoconf, doing make
clean, doing a git pull, then building with meson. The meson build doesn't yet
handle pre-existing flex / bison output.

I had tried to defend against conflicts with in-tree builds by detecting an
in-tree pg_config.h, but that doesn't help with files that aren't removed by
make clean. Like bison / flex output.

And I didn't notice this problem because it doesn't cause visible issues until
the lexer / grammar changes...


I'm not quite sure what the proper behaviour is when doing an out-of-tree
build with meson (all builds are out-of-tree), with a pre-existing flex /
bison output in the source tree that is out of date.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-06-02 15:34:23 +0300, Aleksander Alekseev wrote:
> Hi Andres,
> 
> > Cool. I think I pushed a fix/workaround for the issue now. Still can't 
> > decide
> > whether it's apple's or meson's fault.
> 
> Many thanks! The fix solved the problem, I can compile with -Dldap=enabled 
> now.
> The code passes the tests too.

Cool.


> > I suspect the errors might be due to CC/CPPFLAGS/... not being defined. I 
> > can
> > try to make it output something roughly compatible with the old output, for
> > most of those I already started to compute values for the PGXS compat stuff 
> > I
> > was hacking on recently.
> 
> Yes, that could explain the problem. Just for the record, I get several errors
> regarding src/export.h in TimescaleDB code [1]:
> 

I think this is timescale's issue. Why are you defining / undefining
PGDLLEXPORT?

Part of the patch series is to use visibility attributes, and your #if
TS_EMPTY(PGDLLEXPORT) thing can't handle that.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson

[Aleksander Alekseev]

Hi Andres,

> Cool. I think I pushed a fix/workaround for the issue now. Still can't decide
> whether it's apple's or meson's fault.

Many thanks! The fix solved the problem, I can compile with -Dldap=enabled now.
The code passes the tests too.

> > $ meson configure -Dldap=disabled
> > $ meson configure -Dssl=openssl
> > $ meson configure -Dprefix=/Users/eax/pginstall
>
> FYI, you can set multiple options in one go ;)

Thanks! ;)

> Makes sense. Currently we don't fill the --configure thing, because there
> configure wasn't used. We could try to generate something compatible from
> meson options, but I'm not sure that's a good plan.

If pg_config output was 100% backward compatible with Autotools one, that would
simplify the lives of the extension developers for sure. However, considering
that at PGCon we agreed that both Autotools and Meson will be maintained for
several releases, personally I wouldn't say that this compatibility is
necessary, nor is it realistically deliverable. Nevertheless, IMO there should
be a stable and documented way to determine the PostgreSQL version (this can be
done with `pg_config --version` for both Autotools and Meson), the build tool
used (no way to determine) and the build options (no way to determine
for Meson).

> I suspect the errors might be due to CC/CPPFLAGS/... not being defined. I can
> try to make it output something roughly compatible with the old output, for
> most of those I already started to compute values for the PGXS compat stuff I
> was hacking on recently.

Yes, that could explain the problem. Just for the record, I get several errors
regarding src/export.h in TimescaleDB code [1]:

```
/Users/eax/projects/c/timescaledb/src/export.h:26:5: error: pasting formed
  ')87628', an invalid preprocessing token [clang-diagnostic-error]
#if TS_EMPTY(PGDLLEXPORT)
 ^
/Users/eax/projects/c/timescaledb/src/export.h:17:22: note: expanded from
  macro 'TS_EMPTY'
#define TS_EMPTY(x) (TS_CAT(x, 87628) == 87628)
 ^
/Users/eax/projects/c/timescaledb/src/export.h:15:23: note: expanded from
  macro 'TS_CAT'
#define TS_CAT(x, y) x##y
  ^
/Users/eax/projects/c/timescaledb/src/export.h:26:14: error: function-like
  macro '__attribute__' is not defined [clang-diagnostic-error]
#if TS_EMPTY(PGDLLEXPORT)
 ^
/Users/eax/pginstall/include/postgresql/server/c.h:1339:21: note: expanded
  from macro 'PGDLLEXPORT'
#define PGDLLEXPORT __attribute__((visibility("default")))
^
/Users/eax/projects/c/timescaledb/src/export.h:30:2: error: "PGDLLEXPORT is
  already defined" [clang-diagnostic-error]
#error "PGDLLEXPORT is already defined"
 ^
1 warning and 3 errors generated.
Error while processing /Users/eax/projects/c/timescaledb/src/extension.c
```

[1]: https://github.com/timescale/timescaledb/blob/main/src/export.h

-- 
Best regards,
Aleksander Alekseev

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-06-01 12:39:50 +0300, Aleksander Alekseev wrote:
> > > ```
> > > ../src/include/parser/kwlist.h:332:25: error: ‘PARAMETER’ undeclared here 
> > > (not
> > > in a function)
> > > 332 | PG_KEYWORD("parameter", PARAMETER, UNRESERVED_KEYWORD, BARE_LABEL)
> > >
> > > ../src/interfaces/ecpg/preproc/keywords.c:32:55: note: in definition of 
> > > macro
> > > ‘PG_KEYWORD’
> > > 32 | #define PG_KEYWORD(kwname, value, category, collabel) value,
> > > ```
> >
> > Huh. I've not seen this before - could you provide a bit more detail about
> > what you did? CI isn't testing ubuntu, but it is testing Debian, so I'd 
> > expect
> > this to work.
> 
> I used PIP to install Meson, since the default APT package is too old, v0.53:
> 
> $ pip3 install --user meson
> $ meson --version
> 0.62.1
> $ ninja --version
> 1.10.0
> 
> The branch was checked out as it was described in the first email.
> Then to reproduce the issue:
> 
> $ git status
> On branch meson
> Your branch is up to date with 'andres/meson'.
> $ git fetch andres
> $ git rebase -i andres/meson
> $ meson setup build --buildtype debug
> $ cd build
> $ ninja
> 
> This is pretty much the default Ubuntu 20.04.4 LTS system with all the
> recent updates installed, so it shouldn't be a problem to reproduce
> the issue with a VM.

Will test.


> > > On MacOS I got multiple errors regarding LDAP:
> >
> > Ah, yes. Sorry, that's an open issue that I need to fix. -Dldap=disabled for
> > the rescue.  There's some crazy ordering dependency in macos framework
> > headers. The ldap framework contains an "ldap.h" header that includes
> > "ldap.h". So if you end up with the framework on the include path, you get
> > endless recursion.
> 
> Thanks, this helped.

Cool. I think I pushed a fix/workaround for the issue now. Still can't decide
whether it's apple's or meson's fault.


> I did the following:
> 
> $ meson configure -Dldap=disabled
> $ meson configure -Dssl=openssl
> $ meson configure -Dprefix=/Users/eax/pginstall

FYI, you can set multiple options in one go ;)


> ... and it terminated successfully. I was also able to configure and
> run Postgres instance using my regular scripts, with some
> modifications [1]

Cool.


> Then I decided to compile TimescaleDB against the newly installed
> Postgres. Turns out there is a slight problem.
> 
> The extension uses CMake and also requires PostgreSQL to be compiled
> with OpenSSL support. CMakeLists.txt looks for a
> "--with-(ssl=)?openssl" regular expression in the "pg_config
> --configure" output. The output is empty although Postgres was
> compiled with OpenSSL support.

Makes sense. Currently we don't fill the --configure thing, because there
configure wasn't used. We could try to generate something compatible from
meson options, but I'm not sure that's a good plan.



> The full output of pg_config looks like
> this:
> 
> ```
> CONFIGURE =
> CC = not recorded
> CPPFLAGS = not recorded
> CFLAGS = not recorded
> CFLAGS_SL = not recorded
> ... etc ...
> ```
> 
> I get a bunch of errors from the compiler if I remove this particular
> check from CMakeLists, but I have to investigate these a bit more
> since the branch is based on PG15 and we don't officially support PG15
> yet. It worked last time we checked a month or so ago, but the
> situation may have changed.

I suspect the errors might be due to CC/CPPFLAGS/... not being defined. I can
try to make it output something roughly compatible with the old output, for
most of those I already started to compute values for the PGXS compat stuff I
was hacking on recently.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v8

[Andres Freund]

Hi,

On 2022-06-01 06:55:06 +0200, Peter Eisentraut wrote:
> 
> On 06.05.22 23:27, Andres Freund wrote:
> > I added pkgconfig since then. They're not exactly the same, but pretty 
> > close,
> > except for one thing: Looks like some of the ecpg libraries really should 
> > link
> > to some other ecpg libs? I think we're missing something there... That then
> > leads to missing requirements in the .pc files.
> 
> I took a closer look at the generated pkgconfig files.  I think they are ok.
> There are a couple of insignificant textual differences that we could reduce
> by patching Makefile.shlib.  But technically they are ok.

Thanks for checking!


> There is one significant difference: the ecpg libraries now get a
> Requires.private for openssl, which I think is technically correct since
> both libpgcommon and libpgport require openssl.

Yea, I noticed those too. It's not great, somehow. But I don't really see a
better alternative for now.


> Attached is a tiny patch to make the description in one file backward
> consistent.

Applied.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson

[Aleksander Alekseev]

Hi Andres,

> What do those extensions use to build? Since the unconference I added some
> rudimentary PGXS compatibility, but it's definitely not complete yet.

We mostly use CMake and Cargo, the Rust package manager. So I don't
anticipate many problems here, just want to make sure it's going to
work as expected.

> > ```
> > ../src/include/parser/kwlist.h:332:25: error: ‘PARAMETER’ undeclared here 
> > (not
> > in a function)
> > 332 | PG_KEYWORD("parameter", PARAMETER, UNRESERVED_KEYWORD, BARE_LABEL)
> >
> > ../src/interfaces/ecpg/preproc/keywords.c:32:55: note: in definition of 
> > macro
> > ‘PG_KEYWORD’
> > 32 | #define PG_KEYWORD(kwname, value, category, collabel) value,
> > ```
>
> Huh. I've not seen this before - could you provide a bit more detail about
> what you did? CI isn't testing ubuntu, but it is testing Debian, so I'd expect
> this to work.

I used PIP to install Meson, since the default APT package is too old, v0.53:

$ pip3 install --user meson
$ meson --version
0.62.1
$ ninja --version
1.10.0

The branch was checked out as it was described in the first email.
Then to reproduce the issue:

$ git status
On branch meson
Your branch is up to date with 'andres/meson'.
$ git fetch andres
$ git rebase -i andres/meson
$ meson setup build --buildtype debug
$ cd build
$ ninja

This is pretty much the default Ubuntu 20.04.4 LTS system with all the
recent updates installed, so it shouldn't be a problem to reproduce
the issue with a VM.

> > On MacOS I got multiple errors regarding LDAP:
>
> Ah, yes. Sorry, that's an open issue that I need to fix. -Dldap=disabled for
> the rescue.  There's some crazy ordering dependency in macos framework
> headers. The ldap framework contains an "ldap.h" header that includes
> "ldap.h". So if you end up with the framework on the include path, you get
> endless recursion.

Thanks, this helped. I did the following:

$ meson configure -Dldap=disabled
$ meson configure -Dssl=openssl
$ meson configure -Dprefix=/Users/eax/pginstall
$ ninja
$ meson test
$ meson install

... and it terminated successfully. I was also able to configure and
run Postgres instance using my regular scripts, with some
modifications [1]

Then I decided to compile TimescaleDB against the newly installed
Postgres. Turns out there is a slight problem.

The extension uses CMake and also requires PostgreSQL to be compiled
with OpenSSL support. CMakeLists.txt looks for a
"--with-(ssl=)?openssl" regular expression in the "pg_config
--configure" output. The output is empty although Postgres was
compiled with OpenSSL support. The full output of pg_config looks like
this:

```
CONFIGURE =
CC = not recorded
CPPFLAGS = not recorded
CFLAGS = not recorded
CFLAGS_SL = not recorded
... etc ...
```

I get a bunch of errors from the compiler if I remove this particular
check from CMakeLists, but I have to investigate these a bit more
since the branch is based on PG15 and we don't officially support PG15
yet. It worked last time we checked a month or so ago, but the
situation may have changed.

[1]: https://github.com/afiskon/pgscripts/blob/master/single-install.sh

-- 
Best regards,
Aleksander Alekseev

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

On 06.05.22 23:27, Andres Freund wrote:

I added pkgconfig since then. They're not exactly the same, but pretty close,
except for one thing: Looks like some of the ecpg libraries really should link
to some other ecpg libs? I think we're missing something there... That then
leads to missing requirements in the .pc files.


I took a closer look at the generated pkgconfig files.  I think they are 
ok.  There are a couple of insignificant textual differences that we 
could reduce by patching Makefile.shlib.  But technically they are ok.


There is one significant difference: the ecpg libraries now get a 
Requires.private for openssl, which I think is technically correct since 
both libpgcommon and libpgport require openssl.


Attached is a tiny patch to make the description in one file backward 
consistent.From 8320c520e01b52933f6688e210ffa9cdfdefffe9 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Tue, 31 May 2022 20:46:20 +0200
Subject: [PATCH] fixup! meson: Add meson based buildsystem.

Fix description in pkg-config file for backward consistency.
---
 src/interfaces/ecpg/compatlib/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/interfaces/ecpg/compatlib/meson.build 
b/src/interfaces/ecpg/compatlib/meson.build
index 3cd61217e3..84cd42ccee 100644
--- a/src/interfaces/ecpg/compatlib/meson.build
+++ b/src/interfaces/ecpg/compatlib/meson.build
@@ -11,6 +11,6 @@ ecpg_compat = both_libraries('ecpg_compat',
 pkgconfig.generate(
   ecpg_compat.get_shared_lib(),
   name: 'libecpg_compat',
-  description: 'PostgreSQL ecpg_compat library',
+  description: 'PostgreSQL libecpg_compat library',
   url: pg_url,
 )
-- 
2.36.1

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-05-31 16:49:17 +0300, Aleksander Alekseev wrote:
> I tried the branch on GitHub on MacOS Monterey 12.3.1 and Ubuntu 20.04 LTS.
> I was going to test it against several third party extensions, but it looks 
> like
> it is a bit early for this. On Ubuntu I got the following error:

What do those extensions use to build? Since the unconference I added some
rudimentary PGXS compatibility, but it's definitely not complete yet.


> ```
> ../src/include/parser/kwlist.h:332:25: error: ‘PARAMETER’ undeclared here (not
> in a function)
> 332 | PG_KEYWORD("parameter", PARAMETER, UNRESERVED_KEYWORD, BARE_LABEL)
> 
> ../src/interfaces/ecpg/preproc/keywords.c:32:55: note: in definition of macro
> ‘PG_KEYWORD’
> 32 | #define PG_KEYWORD(kwname, value, category, collabel) value,
> ```

Huh. I've not seen this before - could you provide a bit more detail about
what you did? CI isn't testing ubuntu, but it is testing Debian, so I'd expect
this to work.


> On MacOS I got multiple errors regarding LDAP:

Ah, yes. Sorry, that's an open issue that I need to fix. -Dldap=disabled for
the rescue.  There's some crazy ordering dependency in macos framework
headers. The ldap framework contains an "ldap.h" header that includes
"ldap.h". So if you end up with the framework on the include path, you get
endless recursion.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson

[Aleksander Alekseev]

Hi Andres,

> Not having a ninja backend etc didn't strike me as great either - the builds
> with scons I've done weren't fast at all.

I must admit, personally I never used Scons, I just know that it was considered
(an / the only?) alternative to CMake for many years. The Scons 4.3.0 release
notes say that Ninja is supported [1], but according to the user guide [2]
Ninja support is considered experimental.

Don't get me wrong, I don't insist on using Scons. I was just curious if it was
considered. Actually, a friend of mine pointed out that the fact that Scons
build files are literally a Python code could be a disadvantage. There is less
control of this code, basically it can do anything. It could complicate the
diagnosis of certain issues, etc.

Since you invested so much effort into Meson already let's just focus on it.

I tried the branch on GitHub on MacOS Monterey 12.3.1 and Ubuntu 20.04 LTS.
I was going to test it against several third party extensions, but it looks like
it is a bit early for this. On Ubuntu I got the following error:

```
../src/include/parser/kwlist.h:332:25: error: ‘PARAMETER’ undeclared here (not
in a function)
332 | PG_KEYWORD("parameter", PARAMETER, UNRESERVED_KEYWORD, BARE_LABEL)

../src/interfaces/ecpg/preproc/keywords.c:32:55: note: in definition of macro
‘PG_KEYWORD’
32 | #define PG_KEYWORD(kwname, value, category, collabel) value,
```

On MacOS I got multiple errors regarding LDAP:

```
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/
LDAP.framework/Headers/ldap.h:1:10: error: #include nested too deeply
#include 

../src/interfaces/libpq/fe-connect.c:4816:2: error: use of undeclared
identifier 'LDAP'
LDAP  *ld = NULL;
^

../src/interfaces/libpq/fe-connect.c:4817:2: error: use of undeclared
identifier 'LDAPMessage'
LDAPMessage *res,
^
... etc...
```

I didn't invest much time into investigating these issues. For now I just
wanted to report them. Please let me know if you need any help with these
and/or additional information.

[1]: https://scons.org/scons-430-is-available.html
[2]: https://scons.org/doc/production/PDF/scons-user.pdf

-- 
Best regards,
Aleksander Alekseev

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-05-26 11:47:13 +0300, Aleksander Alekseev wrote:
> Thanks for working on this! I'm very enthusiastic about this effort and I was
> glad to see on PGCon Unconference that the majority of the community seems
> to be as well.
> 
> > The halfway decent list includes, I think:
> > 1) cmake
> > 2) bazel
> > 3) meson
> 
> Was SCons considered as an option?
> What pros and cons do you see that make Meson a better choice?

I looked at it and quickly discarded it. From what I could see there's not
been meaningful moves to it in the last couple years, if anything adoption has
been dropping. And I don't think we want to end up relying on yet another half
maintained tool.

Not having a ninja backend etc didn't strike me as great either - the builds
with scons I've done weren't fast at all.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson

[Aleksander Alekseev]

Hi Andres,

Thanks for working on this! I'm very enthusiastic about this effort and I was
glad to see on PGCon Unconference that the majority of the community seems
to be as well.

> The halfway decent list includes, I think:
> 1) cmake
> 2) bazel
> 3) meson

Was SCons considered as an option? It is widely adopted and written in Python
as well. Personally, I like the fact that with SCons you write config files
in pure Python, not some dialect you have to learn additionally. There is
a free e-book available [1].

What pros and cons do you see that make Meson a better choice?

[1]: https://scons.org/doc/production/PDF/scons-user.pdf

-- 
Best regards,
Aleksander Alekseev

Re: [RFC] building postgres with meson

[Tom Lane]

Andres Freund  writes:
> On 2022-05-25 21:38:33 -0400, Tom Lane wrote:
>> I think the main thing I didn't like about the way you have it in the
>> meson file is the loss of greppability.

> What would you want to grep for? Places that specify additional flags? Or just
> places using flex?

Well, the consistency of having a single name for "flags given to
flex" seems to me to be worth something.

regards, tom lane

Re: [RFC] building postgres with meson

[Andres Freund]

Hi,

On 2022-05-25 21:38:33 -0400, Tom Lane wrote:
> Andres Freund  writes:
> > in the meson unconference session you'd spotted flex flags for 
> > psqlscanslash.l
> > (I think) being "hardcoded". As far as I can tell that's largely just copied
> > from the Makefile):
> 
> > src/backend/parser/Makefile:scan.c: FLEXFLAGS = -CF -p -p
> > src/backend/utils/adt/Makefile:jsonpath_scan.c: FLEXFLAGS = -CF -p -p
> > src/bin/psql/Makefile:psqlscanslash.c: FLEXFLAGS = -Cfe -p -p
> > src/fe_utils/Makefile:psqlscan.c: FLEXFLAGS = -Cfe -p -p
> 
> Hmm, OK.  There *is* a FLEXFLAGS definition supplied by configure, and
> I believe many of our scanners do use it, but evidently we're just
> overriding it for the ones where we really care about using specific
> flags.  It also looks like the configure-supplied version is usually
> empty, so the fact that this variable exists may be mostly a holdover
> from Autoconf practice rather than something we ever cared about.

Yea, it looks like that.

ISTM that it'd still be good to have something like FLEXFLAGS. But it doesn't
look great, nor really intentional, that FLEXFLAGS is overwritten rather than
appended?


> I think the main thing I didn't like about the way you have it in the
> meson file is the loss of greppability.  I could investigate this
> question in a few seconds just now, but if we drop the use of
> FLEXFLAGS as a macro it'll become much harder to figure out which
> places use what.

I disliked a bunch of repetitiveness as I had it, so I'm polishing that part
just now.

What would you want to grep for? Places that specify additional flags? Or just
places using flex?

Greetings,

Andres Freund

Re: [RFC] building postgres with meson

[Tom Lane]

Andres Freund  writes:
> in the meson unconference session you'd spotted flex flags for psqlscanslash.l
> (I think) being "hardcoded". As far as I can tell that's largely just copied
> from the Makefile):

> src/backend/parser/Makefile:scan.c: FLEXFLAGS = -CF -p -p
> src/backend/utils/adt/Makefile:jsonpath_scan.c: FLEXFLAGS = -CF -p -p
> src/bin/psql/Makefile:psqlscanslash.c: FLEXFLAGS = -Cfe -p -p
> src/fe_utils/Makefile:psqlscan.c: FLEXFLAGS = -Cfe -p -p

Hmm, OK.  There *is* a FLEXFLAGS definition supplied by configure, and
I believe many of our scanners do use it, but evidently we're just
overriding it for the ones where we really care about using specific
flags.  It also looks like the configure-supplied version is usually
empty, so the fact that this variable exists may be mostly a holdover
from Autoconf practice rather than something we ever cared about.

I think the main thing I didn't like about the way you have it in the
meson file is the loss of greppability.  I could investigate this
question in a few seconds just now, but if we drop the use of
FLEXFLAGS as a macro it'll become much harder to figure out which
places use what.

regards, tom lane

Re: [RFC] building postgres with meson

[Andres Freund]

Hi Tom,

in the meson unconference session you'd spotted flex flags for psqlscanslash.l
(I think) being "hardcoded". As far as I can tell that's largely just copied
from the Makefile):

src/backend/parser/Makefile:scan.c: FLEXFLAGS = -CF -p -p
src/backend/utils/adt/Makefile:jsonpath_scan.c: FLEXFLAGS = -CF -p -p
src/bin/psql/Makefile:psqlscanslash.c: FLEXFLAGS = -Cfe -p -p
src/fe_utils/Makefile:psqlscan.c: FLEXFLAGS = -Cfe -p -p

note that it's not even FLEXFLAGS += or such.

I honestly don't know enough about the various flex flags to judge what a
better approach would be? Looks like these flags are case specific? Perhaps we
could group them, i.e. have centrally defined "do compress" "don't compress"
flex flags?

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

On 18.05.22 21:48, Andres Freund wrote:

- CONFIGURE_ARGS - empty in meson, not clear what to fill it with


Ok to leave empty for now.


- GETTIMEOFDAY_1ARG - test doesn't exist - I suspect it might not be necessary


Might be obsolete, consider removing.


- PACKAGE_STRING, PACKAGE_TARNAME - unclear if they should be implemented?


leave out for now


- AC_APPLE_UNIVERSAL_BUILD logic - which I don't think we need?


no


- "missing" a bunch of defines that don't appear to be referenced:


Yeah, looks like these are implicitly defined by some autoconf check but 
then the result is only used within configure.ac itself, so isn't needed 
afterwards.



- meson additional defines, seems harmless:
   HAVE_GETTIMEOFDAY - only defined on windows rn
   HAVE_SHM_UNLINK
   HAVE_SSL_NEW
   HAVE_STRTOQ
   HAVE_STRTOUQ
   HAVE_CRYPTO_NEW_EX_DATA


Yeah, that's the opposite of the previous.

I don't see any other issues in pg_config.h either.  Obviously, some 
niche platforms might uncover some issues, but it looks good for now.

Re: [RFC] building postgres with meson -v8

[Andres Freund]

Hi,

On 2022-05-18 10:30:12 +0200, Peter Eisentraut wrote:
> Here are some more patches that clean up various minor issues.

I rebased the meson tree, squashed a lot of the existing commits, merged your
changes, and fixed a few more differences between autoconf and meson.


For me the difference in defines now boils down to:

- CONFIGURE_ARGS - empty in meson, not clear what to fill it with
- GETTIMEOFDAY_1ARG - test doesn't exist - I suspect it might not be necessary
- PACKAGE_STRING, PACKAGE_TARNAME - unclear if they should be implemented?
- AC_APPLE_UNIVERSAL_BUILD logic - which I don't think we need?
- pg_restrict is defined in a simplistic way
- "missing" a bunch of defines that don't appear to be referenced:
  HAVE_FSEEKO
  HAVE_GSSAPI_GSSAPI_H
  HAVE_INTTYPES_H
  HAVE_LDAP_H
  HAVE_LIBCRYPTO
  HAVE_LIBLDAP
  HAVE_LIBM
  HAVE_LIBPAM
  HAVE_LIBSSL
  HAVE_LIBXML2
  HAVE_LIBXSLT
  HAVE_MEMORY_H
  HAVE_PTHREAD
  HAVE_PTHREAD_PRIO_INHERIT
  HAVE_STDINT_H
  HAVE_STDLIB_H
  HAVE_STRING_H
  HAVE_SYS_STAT_H
  HAVE_SYS_TYPES_H
  HAVE_UNISTD_H
  SIZEOF_BOOL
  SIZEOF_OFF_T
  STDC_HEADERS
- meson additional defines, seems harmless:
  HAVE_GETTIMEOFDAY - only defined on windows rn
  HAVE_SHM_UNLINK
  HAVE_SSL_NEW
  HAVE_STRTOQ
  HAVE_STRTOUQ
  HAVE_CRYPTO_NEW_EX_DATA
- a bunch of additional #undef's


Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

Here are some more patches that clean up various minor issues.From bc06fda7198d182dce73f39cfff8e4724e00b12d Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 4 May 2022 14:38:02 +0200
Subject: [PATCH 1/7] meson: Put genbki header files back into original order

The order affects the output order in postgres.bki and other files.
This makes it match the order in the makefiles.
---
 src/include/catalog/meson.build | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/src/include/catalog/meson.build b/src/include/catalog/meson.build
index b2acc3ce30..813a963493 100644
--- a/src/include/catalog/meson.build
+++ b/src/include/catalog/meson.build
@@ -30,7 +30,6 @@ catalog_headers = [
   'pg_conversion.h',
   'pg_depend.h',
   'pg_database.h',
-  'pg_parameter_acl.h',
   'pg_db_role_setting.h',
   'pg_tablespace.h',
   'pg_authid.h',
@@ -54,6 +53,7 @@ catalog_headers = [
   'pg_seclabel.h',
   'pg_shseclabel.h',
   'pg_collation.h',
+  'pg_parameter_acl.h',
   'pg_partitioned_table.h',
   'pg_range.h',
   'pg_transform.h',
-- 
2.35.1

From b4ffaa02f1eb132b011871415c94b84e84fd5245 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 18 May 2022 09:22:22 +0200
Subject: [PATCH 2/7] meson: Some capitalization adjustments in setup output

This makes the style more uniform with what meson produces by itself.
---
 meson.build | 10 +-
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/meson.build b/meson.build
index 7b1c8d47c4..3acae5c3e4 100644
--- a/meson.build
+++ b/meson.build
@@ -2583,10 +2583,10 @@ alias_target('backend', backend_targets)
 if meson.version().version_compare('>=0.57')
 
   summary({
-'Data Block Size' : cdata.get('BLCKSZ'),
-'WAL Block Size' : cdata.get('XLOG_BLCKSZ'),
-'Segment Size' : cdata.get('RELSEG_SIZE')
-}, section: 'Data Layout'
+'data block size' : cdata.get('BLCKSZ'),
+'WAL block size' : cdata.get('XLOG_BLCKSZ'),
+'segment size' : cdata.get('RELSEG_SIZE')
+}, section: 'Data layout'
   )
 
   summary(
@@ -2644,7 +2644,7 @@ if meson.version().version_compare('>=0.57')
   'zlib': zlib,
   'zstd': zstd,
 },
-section: 'External Libraries'
+section: 'External libraries'
   )
 
 endif
-- 
2.35.1

From 23a7e7ffb4db7554b2029ba3026d93f8e32b801f Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 18 May 2022 09:47:47 +0200
Subject: [PATCH 3/7] meson: Formatting tweaks for pg_config.h to match
 autoheader better

---
 meson.build | 26 +++---
 1 file changed, 11 insertions(+), 15 deletions(-)

diff --git a/meson.build b/meson.build
index 3acae5c3e4..20dea263a2 100644
--- a/meson.build
+++ b/meson.build
@@ -227,19 +227,14 @@ meson_bin = find_program(meson_binpath, native: true)
 
 cdata.set('USE_ASSERT_CHECKING', get_option('cassert') ? 1 : false)
 
-cdata.set('BLCKSZ', 8192, description: '''
- Size of a disk block --- this also limits the size of a tuple.  You
- can set it bigger if you need bigger tuples (although TOAST should
- reduce the need to have large tuples, since fields can be spread
- across multiple tuples).
-
- BLCKSZ must be a power of 2.  The maximum possible value of BLCKSZ
- is currently 2^15 (32768).  This is determined by the 15-bit widths
- of the lp_off and lp_len fields in ItemIdData (see
- include/storage/itemid.h).
-
- Changing BLCKSZ requires an initdb.
-''')
+cdata.set('BLCKSZ', 8192, description:
+'''Size of a disk block --- this also limits the size of a tuple. You can set
+   it bigger if you need bigger tuples (although TOAST should reduce the need
+   to have large tuples, since fields can be spread across multiple tuples).
+   BLCKSZ must be a power of 2. The maximum possible value of BLCKSZ is
+   currently 2^15 (32768). This is determined by the 15-bit widths of the
+   lp_off and lp_len fields in ItemIdData (see include/storage/itemid.h).
+   Changing BLCKSZ requires an initdb.''')
 
 cdata.set('XLOG_BLCKSZ', get_option('wal-blocksize') * 1024)
 cdata.set('RELSEG_SIZE', get_option('segsize') * 131072)
@@ -1000,7 +995,8 @@ if get_option('ssl') == 'openssl'
 description: 'Define to 1 to build with OpenSSL support. 
(-Dssl=openssl)')
 
   cdata.set('OPENSSL_API_COMPAT', 0x10001000,
-description: 'Define to the OpenSSL API version in use. This 
avoids deprecation warnings from newer OpenSSL versions.')
+description: '''Define to the OpenSSL API version in use. This 
avoids deprecation warnings
+   from newer OpenSSL versions.''')
 else
   ssl = dependency('', required : false)
 endif
@@ -1639,7 +1635,7 @@ foreach c : decl_checks
 kwargs: args)
   cdata.set10(varname, found, description:
 '''Define to 1 if you have the declaration of `@0@', and to 0 if you
-  don't.'''.format(func))
+   don't.'''.format(func))
 endforeach
 
 
-- 
2.35.1

From 1d2f4c3d7eb32a01e3ea75816f9ec09add6500db Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 18 May 2022 09:48:43 +0200
Subject: [PATCH 4/7] meson: Remove 

Re: [RFC] building postgres with meson -v8

[Andres Freund]

Hi,

On 2022-05-16 17:47:24 +0200, Peter Eisentraut wrote:
> On 12.05.22 21:30, Andres Freund wrote:
> > On 2022-05-11 12:18:58 +0200, Peter Eisentraut wrote:
> > > I fixed the Perl detection issue in my macOS environment that I had 
> > > reported
> > > a while ago.
> > 
> > Hm. I wonder if it's right to check with is_file() - perhaps there are
> > platforms that have split off the include directory?
> 
> The existing code uses "test -f", so using is_file() would keep it working
> the same way.

I merged it that way. Merged.


> > >  From 049b34b6a8dd949f0eb7987cad65f6682a6ec786 Mon Sep 17 00:00:00 2001
> > > From: Peter Eisentraut 
> > > Date: Wed, 11 May 2022 09:06:13 +0200
> > > Subject: [PATCH 3/9] meson: prereq: Refactor dtrace postprocessing make 
> > > rules
> > > 
> > > Move the dtrace postprocessing sed commands into a separate file so
> > > that it can be shared by meson.  Also split the rule into two for
> > > proper dependency declaration.
> > 
> > Hm. Using sed may be problematic on windows...
> 
> This code is only used when dtrace is enabled, which probably doesn't apply
> on Windows.

Fair point. Merged.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v8

[Andres Freund]

On 2022-05-16 17:48:08 +0200, Peter Eisentraut wrote:
> On 14.05.22 01:17, Andres Freund wrote:
> > On 2022-05-11 12:18:58 +0200, Peter Eisentraut wrote:
> > > This currently only works on macOS.  The dtrace -G calls needed on
> > > other platforms are not implemented yet.
> > 
> > I looked into that part. The make rule passes all the backend object files 
> > as
> > an option, but it's not clear to me where / why that's needed. On linux it
> > certainly works to not pass in the object files...
> > 
> > Maybe CI will show problems on freebsd or such...
> 
> Yes, it failed for me on freebsd.

Yep, I saw those shortly after... I've implemented that bit now, although it
needs a bit more cleanup.

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

On 14.05.22 01:17, Andres Freund wrote:

On 2022-05-11 12:18:58 +0200, Peter Eisentraut wrote:

This currently only works on macOS.  The dtrace -G calls needed on
other platforms are not implemented yet.


I looked into that part. The make rule passes all the backend object files as
an option, but it's not clear to me where / why that's needed. On linux it
certainly works to not pass in the object files...

Maybe CI will show problems on freebsd or such...


Yes, it failed for me on freebsd.


Therefore, the option is not auto by default.


It probably shouldn't be auto either way, there's some overhead associated
with the probes...


ok

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

On 12.05.22 21:30, Andres Freund wrote:

On 2022-05-11 12:18:58 +0200, Peter Eisentraut wrote:

I fixed the Perl detection issue in my macOS environment that I had reported
a while ago.


Hm. I wonder if it's right to check with is_file() - perhaps there are
platforms that have split off the include directory?


The existing code uses "test -f", so using is_file() would keep it 
working the same way.



After that, these configure options don't have an equivalent yet:

--disable-rpath
--enable-profiling
--disable-thread-safety
--with-libedit-preferred

The first three overlap with meson built-in functionality, so we would need
to check whether the desired functionality is available somehow.


Which builtin functionality does --enable-profiling overlap with? There's a
coverage option, perhaps you were thinking of that?


I saw an option about "profile guided optimization" (b_pgo), which seems 
possibly related.



I don't think we should add --disable-thread-safety, platforms without it also
aren't going to support ninja / meson... IIRC Thomas was planning to submit a
patch getting rid of it independently...


sure


 From 049b34b6a8dd949f0eb7987cad65f6682a6ec786 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 11 May 2022 09:06:13 +0200
Subject: [PATCH 3/9] meson: prereq: Refactor dtrace postprocessing make rules

Move the dtrace postprocessing sed commands into a separate file so
that it can be shared by meson.  Also split the rule into two for
proper dependency declaration.


Hm. Using sed may be problematic on windows...


This code is only used when dtrace is enabled, which probably doesn't 
apply on Windows.

Re: [RFC] building postgres with meson -v8

[Andres Freund]

Hi,

On 2022-05-11 12:18:58 +0200, Peter Eisentraut wrote:
> This currently only works on macOS.  The dtrace -G calls needed on
> other platforms are not implemented yet.

I looked into that part. The make rule passes all the backend object files as
an option, but it's not clear to me where / why that's needed. On linux it
certainly works to not pass in the object files...

Maybe CI will show problems on freebsd or such...


> Therefore, the option is not auto by default.

It probably shouldn't be auto either way, there's some overhead associated
with the probes...

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v8

[Andres Freund]

Hi,

On 2022-05-11 12:18:58 +0200, Peter Eisentraut wrote:
> I fixed the Perl detection issue in my macOS environment that I had reported
> a while ago.

Hm. I wonder if it's right to check with is_file() - perhaps there are
platforms that have split off the include directory?


> Then I added in support for all configure options that had not been ported
> over yet.  Some of these are rather trivial.

Thanks!

Some of these (extra version, krb srvname, ...) I just merged from a
colleague.

Will look at merging the others.


> After that, these configure options don't have an equivalent yet:
> 
> --disable-rpath
> --enable-profiling
> --disable-thread-safety
> --with-libedit-preferred
> 
> The first three overlap with meson built-in functionality, so we would need
> to check whether the desired functionality is available somehow.

Which builtin functionality does --enable-profiling overlap with? There's a
coverage option, perhaps you were thinking of that?

I don't think we should add --disable-thread-safety, platforms without it also
aren't going to support ninja / meson... IIRC Thomas was planning to submit a
patch getting rid of it independently...


> The last one we probably want to keep somehow; it would need a bit of fiddly
> work.

A colleague just finished that bit. Probably can be improved further, but it
works now...


> From 049b34b6a8dd949f0eb7987cad65f6682a6ec786 Mon Sep 17 00:00:00 2001
> From: Peter Eisentraut 
> Date: Wed, 11 May 2022 09:06:13 +0200
> Subject: [PATCH 3/9] meson: prereq: Refactor dtrace postprocessing make rules
> 
> Move the dtrace postprocessing sed commands into a separate file so
> that it can be shared by meson.  Also split the rule into two for
> proper dependency declaration.

Hm. Using sed may be problematic on windows...


> From fad02f1fb71ec8c64e47e5031726ffbee4a1dd84 Mon Sep 17 00:00:00 2001
> From: Peter Eisentraut 
> Date: Wed, 11 May 2022 09:53:01 +0200
> Subject: [PATCH 7/9] meson: Add system-tzdata option
> 
> ---
>  meson.build   | 3 +++
>  meson_options.txt | 3 +++
>  2 files changed, 6 insertions(+)
> 
> diff --git a/meson.build b/meson.build
> index 7c9c6e7f23..b33a51a35d 100644
> --- a/meson.build
> +++ b/meson.build
> @@ -246,6 +246,9 @@ cdata.set('RELSEG_SIZE', get_option('segsize') * 131072)
>  cdata.set('DEF_PGPORT', get_option('pgport'))
>  cdata.set_quoted('DEF_PGPORT_STR', get_option('pgport'))
>  cdata.set_quoted('PG_KRB_SRVNAM', 'postgres')
> +if get_option('system-tzdata') != ''
> +  cdata.set_quoted('SYSTEMTZDIR', get_option('system-tzdata'))
> +endif

This doesn't quite seem sufficient - we also need to prevent building &
installing our own timezone data...

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

More patches:

I fixed the Perl detection issue in my macOS environment that I had 
reported a while ago.


Then I added in support for all configure options that had not been 
ported over yet.  Some of these are rather trivial.


After that, these configure options don't have an equivalent yet:

--disable-rpath
--enable-profiling
--disable-thread-safety
--with-libedit-preferred

The first three overlap with meson built-in functionality, so we would 
need to check whether the desired functionality is available somehow.


The last one we probably want to keep somehow; it would need a bit of 
fiddly work.From fd05ad3f1582884aef2db162c2c7ba4897299078 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 11 May 2022 11:08:07 +0200
Subject: [PATCH 1/9] meson: Fix Perl include dir detection

Only use -iwithsysroot if required.  This aligns with the logic in
configure.ac.
---
 meson.build | 12 +++-
 1 file changed, 7 insertions(+), 5 deletions(-)

diff --git a/meson.build b/meson.build
index 5f8165df9c..7958e8eb19 100644
--- a/meson.build
+++ b/meson.build
@@ -591,8 +591,6 @@ else
 
 perl_inc_dir = '@0@/CORE'.format(archlibexp)
 
-perl_ccflags = []
-
 if useshrplib != 'true'
   perl_may_work = false
   perl_msg = 'need a shared perl'
@@ -606,9 +604,13 @@ else
   # et al seeing them. So we can't use include_directories(), as that checks
   # file existence.
   if perl_may_work
-perl_ccflags += ['-I@0@'.format(perl_inc_dir)]
-if host_machine.system() == 'darwin'
-  perl_ccflags += ['-iwithsysroot', perl_inc_dir]
+# On most platforms, archlibexp is also where the Perl include files live 
...
+perl_ccflags = ['-I@0@'.format(perl_inc_dir)]
+# ... but on newer macOS versions, we must use -iwithsysroot to look
+# under sysroot
+if not fs.is_file('@0@/perl.h'.format(perl_inc_dir)) and \
+   fs.is_file('@0@@1@/perl.h'.format(sysroot, perl_inc_dir))
+  perl_ccflags = ['-iwithsysroot', perl_inc_dir]
 endif
   endif
 
-- 
2.35.1

From 1429482c5d3e83a89915cc332be3b0d932c939cb Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 11 May 2022 07:31:30 +0200
Subject: [PATCH 2/9] meson: Add pam option

---
 meson.build   | 26 +-
 meson_options.txt |  3 +++
 2 files changed, 28 insertions(+), 1 deletion(-)

diff --git a/meson.build b/meson.build
index 7958e8eb19..a07e25c732 100644
--- a/meson.build
+++ b/meson.build
@@ -525,6 +525,29 @@ endif
 
 
 
+###
+# Library: pam
+###
+
+pamopt = get_option('pam')
+if not pamopt.disabled()
+  pam = cc.find_library('pam', dirs: g_c_lib, required: pamopt)
+  if pam.found()
+if cc.has_header('security/pam_appl.h', args: g_c_args, 
include_directories: g_c_inc)
+  cdata.set('HAVE_SECURITY_PAM_APPL_H', 1)
+elif cc.has_header('pam/pam_appl.h', args: g_c_args, include_directories: 
g_c_inc)
+  cdata.set('HAVE_PAM_PAM_APPL_H', 1)
+else
+  error('pam header not found')
+endif
+  endif
+else
+  pam = dependency('', required: false)
+endif
+cdata.set('USE_PAM', pam.found() ? 1 : false)
+
+
+
 ###
 # Library: Tcl (for pltcl)
 # tclConfig.sh
@@ -2143,7 +2166,7 @@ backend_code = declare_dependency(
   include_directories: [postgres_inc],
   link_with: [],
   sources: generated_headers + generated_backend_headers,
-  dependencies: [os_deps, libintl, ssl, lz4, zlib, icu, icu_i18n, ldap, 
gssapi, libxml, zstd, systemd],
+  dependencies: [os_deps, libintl, ssl, lz4, zlib, icu, icu_i18n, ldap, pam, 
gssapi, libxml, zstd, systemd],
 )
 
 # Note there's intentionally no dependency on pgport/common here - we want the
@@ -2485,6 +2508,7 @@ if meson.version().version_compare('>=0.57')
   'llvm': llvm,
   'lz4': lz4,
   'nls' : libintl,
+  'pam' : pam,
   'perl': perl_dep,
   'python3': python3_dep,
   'readline': readline,
diff --git a/meson_options.txt b/meson_options.txt
index e3e36678d1..f57410fa50 100644
--- a/meson_options.txt
+++ b/meson_options.txt
@@ -74,6 +74,9 @@ option('lz4', type : 'feature', value: 'auto',
 option('nls', type: 'feature', value: 'auto',
   description: 'national language support')
 
+option('pam', type : 'feature', value: 'auto',
+  description: 'PAM support')
+
 option('plperl', type : 'feature', value: 'auto',
   description: 'build Perl modules (PL/Perl)')
 
-- 
2.35.1

From 049b34b6a8dd949f0eb7987cad65f6682a6ec786 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 11 May 2022 09:06:13 +0200
Subject: [PATCH 3/9] meson: prereq: Refactor dtrace postprocessing make rules

Move the dtrace postprocessing sed commands into a separate file so
that it can be shared by meson.  Also split the rule into two for
proper dependency declaration.
---
 src/backend/utils/Makefile   | 21 +++--
 

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

On 06.05.22 23:27, Andres Freund wrote:

Re symlink: Do you have an opion about dropping the symlink vs implementing it
(likely via a small helper script?)?


I think the postmaster symlink could be dropped.  The postmaster man 
page has been saying that it's deprecated since 2006.

Re: [RFC] building postgres with meson -v8

[Andres Freund]

Hi,

On 2022-05-06 14:27:24 -0700, Andres Freund wrote:
> > 0003-meson-Install-all-server-headers.patch
> > 
> > With this, all the server headers installed by a makefile-based build are
> > installed.  I tried to strike a balance between using install_subdir() with
> > exclude list versus listing things explicitly. Different variations might be
> > possible, but this looked pretty sensible to me.
> 
> I locally had something similar, but I'm worried that this approach will be
> too fragile. Leads to e.g. editor temp files getting installed. I've merged it
> for now, but I think we need a different approach.

Meant to add potential alternatives here: The easiest likely would be to just
add an install script that globs *.h. Alternatively we could build a file list
at configure time, and then install that with install_header(). The advantage
would be that it be available for things like cpluspluscheck, the disadvantage
that something needs to trigger reconfiguration to update the file list.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v8

[Andres Freund]

Hi,

On 2022-04-21 17:34:47 -0400, Tom Lane wrote:
> FWIW, I don't think that either gaur or prairiedog need be factored into
> this conversation.  They cannot build ninja at all for lack of ,
> so whether they could run meson is pretty much beside the point.

Yea.


> (I wonder if we should stick in a configure test for ,
> just to see if anything else doesn't have it?)

Might be worth doing?


> We should worry a little more about Solaris and AIX, but even there I
> think it's largely up to the platform owner whether they've updated
> python to something modern.

Looks like "AIX toolbox" is at 3.7. Solaris 11.4 apparently has 3.5 (11.3 is
EOL January 2024).

I think it's worth caring about supporting 3.6 due to RHEL 7 for now.


> If it isn't, you need to move the goalposts
> back some more :-(.  As of today I see the following pre-3.6 pythons
> in the buildfarm, exclusive of mine:
>
> skate 3.2.3
> snapper   3.2.3

Debian wheezy, I feel ok with dropping that.


> topminnow 3.4.2

Debian jessie, similar.


> hornet3.4.3
> sungazer  3.4.3

Looks like a newer python version is available for AIX, without manually
compiling.


> wrasse3.4.3

Apparently solaris 11.4 has python 3.5 (still not great :/)


> shelduck  3.4.10

This animal seems to have retired.


> curculio  3.5.1

Supported versions of openbsd have modern versions of python.


> hoverfly  3.5.1

AIX


> batfish   3.5.2
> spurfowl  3.5.2
> cuon  3.5.2

Ubuntu 16.04 is EOL (since 2021-04), outside of paid extended support.


> ayu   3.5.3
> chimaera  3.5.3
> chipmunk  3.5.3
> grison3.5.3
> mussurana 3.5.3
> tadarida  3.5.3
> urocryon  3.5.3

These are all [variants of] debian stretch. I think we should be ok dropping
support for that, the extended "LTS" support for stretch ends June 30, 2022
(with the last non-extended update at July 18, 2020).

Greetings,

Andres Freund

[1] https://repology.org/project/python/versions

Re: [RFC] building postgres with meson -v8

[Andres Freund]

Hi,

On 2022-05-04 13:53:54 +0200, Peter Eisentraut wrote:
> 0001-meson-Assorted-compiler-test-tweaks.patch
> 
> I was going through a diff of pg_config.h between old and new build and
> found a few omissions and small differences.

Thanks, merged that.


> is of course annoying and can be removed eventually, but it's useful when
> analyzing the diff, and since it's already done in other places it seems
> reasonable to apply it consistently.

Yea, I'd tried to minimize the difference at some point, but haven't done so
in a while...


> 0002-meson-Add-pg_walinspect.patch
> 
> This was added more recently and was not ported yet.  Nothing too
> interesting here.

Merged that.


> 0003-meson-Install-all-server-headers.patch
> 
> With this, all the server headers installed by a makefile-based build are
> installed.  I tried to strike a balance between using install_subdir() with
> exclude list versus listing things explicitly. Different variations might be
> possible, but this looked pretty sensible to me.

I locally had something similar, but I'm worried that this approach will be
too fragile. Leads to e.g. editor temp files getting installed. I've merged it
for now, but I think we need a different approach.


> With these patches, the list of files installed with make versus meson match
> up, except for known open items (postmaster symlink, some library naming
> differences, pkgconfig, pgxs, test modules installed, documentation).

I added pkgconfig since then. They're not exactly the same, but pretty close,
except for one thing: Looks like some of the ecpg libraries really should link
to some other ecpg libs? I think we're missing something there... That then
leads to missing requirements in the .pc files.

Re symlink: Do you have an opion about dropping the symlink vs implementing it
(likely via a small helper script?)?

Re library naming: It'd obviously be easy to adjust the library names, but I
wonder if it'd not be worth keeping the _static.a suffix, right now unsuffixed
library name imo is quite confusing.

Re test modules: Not sure what the best fix for that is yet. Except that we
don't have a search path for server libs, I'd just install them to a dedicated
path or add the build dir to the search path. But we don't, so ...

Re docs: I think the best approach here would be to have a new
meson_options.txt option defining whether the docs should be built. But not
quite sure.

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

More patches:

0001-meson-Assorted-compiler-test-tweaks.patch

I was going through a diff of pg_config.h between old and new build and 
found a few omissions and small differences.


Some of the

blah ? 1 : false

is of course annoying and can be removed eventually, but it's useful 
when analyzing the diff, and since it's already done in other places it 
seems reasonable to apply it consistently.


Of course there is some more work left for some of the more complicated 
tests; this isn't meant to be complete.



0002-meson-Add-pg_walinspect.patch

This was added more recently and was not ported yet.  Nothing too 
interesting here.



0003-meson-Install-all-server-headers.patch

With this, all the server headers installed by a makefile-based build 
are installed.  I tried to strike a balance between using 
install_subdir() with exclude list versus listing things explicitly. 
Different variations might be possible, but this looked pretty sensible 
to me.



With these patches, the list of files installed with make versus meson 
match up, except for known open items (postmaster symlink, some library 
naming differences, pkgconfig, pgxs, test modules installed, documentation).From c3f4f4f8002e473284587d21f89cb66364365a26 Mon Sep 17 00:00:00 2001
From: Peter Eisentraut 
Date: Wed, 4 May 2022 09:54:36 +0200
Subject: [PATCH 1/3] meson: Assorted compiler test tweaks

---
 meson.build | 46 +-
 1 file changed, 37 insertions(+), 9 deletions(-)

diff --git a/meson.build b/meson.build
index 80af8e13da..27bc9dcd48 100644
--- a/meson.build
+++ b/meson.build
@@ -222,7 +222,7 @@ meson_bin = find_program(meson_binpath, native: true)
 # Option Handling
 ###
 
-cdata.set('USE_ASSERT_CHECKING', get_option('cassert'))
+cdata.set('USE_ASSERT_CHECKING', get_option('cassert') ? 1 : false)
 
 cdata.set('BLCKSZ', 8192, description: '''
  Size of a disk block --- this also limits the size of a tuple.  You
@@ -241,7 +241,7 @@ cdata.set('BLCKSZ', 8192, description: '''
 cdata.set('XLOG_BLCKSZ', get_option('wal-blocksize') * 1024)
 cdata.set('RELSEG_SIZE', get_option('segsize') * 131072)
 cdata.set('DEF_PGPORT', get_option('pgport'))
-cdata.set_quoted('DEF_PGPORT_STR', '5432')
+cdata.set_quoted('DEF_PGPORT_STR', get_option('pgport'))
 cdata.set_quoted('PG_KRB_SRVNAM', 'postgres')
 
 
@@ -870,8 +870,6 @@ if get_option('ssl') == 'openssl'
 ssl_int = [ssl]
   endif
 
-  cdata.set_quoted('WITH_SSL', get_option('ssl'))
-
   check_funcs = [
 ['CRYPTO_new_ex_data', {'required': true}],
 ['SSL_new', {'required': true}],
@@ -1381,7 +1379,7 @@ int main(void)
 cdata.set(check['name'],
   cc.links(test,
 name: check['desc'],
-args: g_c_args + ['-DINT64=@0@'.format(cdata.get('PG_INT64_TYPE'))])
+args: g_c_args + ['-DINT64=@0@'.format(cdata.get('PG_INT64_TYPE'))]) ? 
1 : false
 )
   endforeach
 
@@ -1609,7 +1607,8 @@ endif
 
 if cc.has_member('struct sockaddr_storage', 'ss_family',
 args: g_c_args, include_directories: g_c_inc,
-prefix: '''#include 
+prefix: '''
+#include 
 #include ''')
   cdata.set('HAVE_STRUCT_SOCKADDR_STORAGE_SS_FAMILY', 1)
 endif
@@ -1622,6 +1621,30 @@ if cc.has_member('struct sockaddr_storage', 
'__ss_family',
   cdata.set('HAVE_STRUCT_SOCKADDR_STORAGE___SS_FAMILY', 1)
 endif
 
+if cc.has_member('struct sockaddr_storage', 'ss_len',
+args: g_c_args, include_directories: g_c_inc,
+prefix: '''
+#include 
+#include ''')
+  cdata.set('HAVE_STRUCT_SOCKADDR_STORAGE_SS_LEN', 1)
+endif
+
+if cc.has_member('struct sockaddr_storage', '__ss_len',
+args: g_c_args, include_directories: g_c_inc,
+prefix: '''
+#include 
+#include ''')
+  cdata.set('HAVE_STRUCT_SOCKADDR_STORAGE___SS_LEN', 1)
+endif
+
+if cc.has_member('struct sockaddr', 'sa_len',
+args: g_c_args, include_directories: g_c_inc,
+prefix: '''
+#include 
+#include ''')
+  cdata.set('HAVE_STRUCT_SOCKADDR_SA_LEN', 1)
+endif
+
 if cc.has_type('struct sockaddr_un',
 args: g_c_args, include_directories: g_c_inc,
 prefix: '''
@@ -1701,10 +1724,10 @@ endif
 # needs xlocale.h; standard is locale.h, but glibc also has an
 # xlocale.h file that we should not use.
 if cc.has_type('locale_t', prefix: '#include ')
-  cdata.set('HAVE_LOCALE_T', true)
+  cdata.set('HAVE_LOCALE_T', 1)
 elif cc.has_type('locale_t', prefix: '#include ')
-  cdata.set('HAVE_LOCALE_T', true)
-  cdata.set('LOCALE_T_IN_XLOCALE', true)
+  cdata.set('HAVE_LOCALE_T', 1)
+  cdata.set('LOCALE_T_IN_XLOCALE', 1)
 endif
 
 # MSVC doesn't cope well with defining restrict to __restrict, the
@@ -1781,6 +1804,7 @@ func_checks = [
   ['getrusage'],
   ['gettimeofday'], # XXX: This seems to be in the autoconf case
   ['inet_aton'],
+  ['inet_pton'],
   ['kqueue'],
   ['link'],
   ['mbstowcs_l'],
@@ -1852,6 +1876,10 @@ foreach c : func_checks
 endforeach
 
 
+if cc.has_function('syslog', args: g_c_args) and cc.check_header('syslog.h', 

Re: [RFC] building postgres with meson -v8

[Andres Freund]

Hi,

On 2022-05-02 16:47:43 +0200, Peter Eisentraut wrote:
> On 29.04.22 19:46, Andres Freund wrote:
> > explicitly using shared_library() rather than library()
> 
> Why is that?  We do build static libraries right now, so using library()
> would seem more suitable for that.

When I wrote this I hadn't realized that we build both shared and static
libraries. I've since changed the respective ecpg libraries to use
both_libraries(). Same with libpq (I really hadn't realized we build a static
libpq...).

Greetings,

Andres Freund

Re: [RFC] building postgres with meson -v8

[Peter Eisentraut]

On 29.04.22 19:46, Andres Freund wrote:

explicitly using shared_library() rather than library()


Why is that?  We do build static libraries right now, so using library() 
would seem more suitable for that.