Re: [RFC] building postgres with meson - v10
Hi, On 8/8/22 18:53, Andres Freund wrote: Bilal's version checked different directories for expected files, but I don't think that's necessary. Bilal, do you remember why you added that? This was for not breaking autoconf build. Autoconf wasn't using expecteddir, so I checked different directories. Greetings, Nazir Bilal Yavuz
Re: [RFC] building postgres with meson - v10
Hi, On 2022-07-21 15:26:05 +0300, Bilal Yavuz wrote: > > On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote: > > > dff7b5a960 meson: prereq: regress: allow to specify director containing > > > expected files. > > > > > > This could use a bit more explanation, but it doesn't look > > > controversial so far. > > While testing ECPG, C and exe files are generated by meson so these files > are in the meson's build directory but expected files are in the source > directory. However; there was no way to set different paths for inputs (C > and exe files') and expected files' directory. So, I added `--expecteddir` > to separately set expected files' directory. Attached is a version of this patch that also removes the copying of these files from ecpg's makefile. Bilal's version checked different directories for expected files, but I don't think that's necessary. Bilal, do you remember why you added that? I'm somewhat tempted to rename ecpg's pg_regress to pg_regress_ecpg as part of this, given the .c file is named pg_regress_ecpg.c and that pg_regress is a pre-existing binary. Greetings, Andres Freund >From 9645e0a01305ce42fdacaae73bd0d021a5a1c47e Mon Sep 17 00:00:00 2001 From: Andres Freund Date: Thu, 30 Jun 2022 18:28:17 -0700 Subject: [PATCH] meson: prereq: regress: allow to specify director containing expected files The ecpg tests have their input directory in the build directory as the tests need to be built. Until now that required copying the expected/ directory to the build directory in VPATH builds. To avoid needing to implement the same for the meson build, add support for specifying the expected directory/. Remove the copying of ecpg's expected/ to the build dir in VPATH builds. Author: Nazir Bilal Yavuz Author: Andres Freund Reviewed-By: Andres Freund Discussion: https://postgr.es/m/20220718202327.pspcqz5mwbi2y...@awork3.anarazel.de --- src/interfaces/ecpg/test/Makefile | 28 -- src/interfaces/ecpg/test/pg_regress_ecpg.c | 8 +++ src/test/regress/pg_regress.c | 7 ++ src/test/regress/pg_regress.h | 1 + 4 files changed, 22 insertions(+), 22 deletions(-) diff --git a/src/interfaces/ecpg/test/Makefile b/src/interfaces/ecpg/test/Makefile index 10f53c708c8..d7a7d1d1ca5 100644 --- a/src/interfaces/ecpg/test/Makefile +++ b/src/interfaces/ecpg/test/Makefile @@ -59,24 +59,16 @@ pg_regress_ecpg.o: pg_regress_ecpg.c $(top_builddir)/src/port/pg_config_paths.h $(top_builddir)/src/port/pg_config_paths.h: $(top_builddir)/src/Makefile.global $(MAKE) -C $(top_builddir)/src/port pg_config_paths.h -# When doing a VPATH build, copy over the .pgc, .stdout and .stderr -# files so that the driver script can find them. We have to use an -# absolute path for the targets, because otherwise make will try to -# locate the missing files using VPATH, and will find them in -# $(srcdir), but the point here is that we want to copy them from -# $(srcdir) to the build directory. - -ifdef VPATH -remaining_files_src := $(wildcard $(srcdir)/*/*.pgc) $(wildcard $(srcdir)/expected/*.c) $(wildcard $(srcdir)/expected/*.stdout) $(wildcard $(srcdir)/expected/*.stderr) -remaining_files_build := $(patsubst $(srcdir)/%, $(abs_builddir)/%, $(remaining_files_src)) - -all: $(remaining_files_build) -$(remaining_files_build): $(abs_builddir)/%: $(srcdir)/% - ln -s $< $@ -endif - -# Common options for tests. Also pick up anything passed in EXTRA_REGRESS_OPTS -REGRESS_OPTS = --dbname=ecpg1_regression,ecpg2_regression --create-role=regress_ecpg_user1,regress_ecpg_user2 $(EXTRA_REGRESS_OPTS) +# Common options for tests +# +# Need to specify expecteddir explicitly, as the inputdir is located in the +# build directory, because the files need to be compiled. Other pg_regress +# style tests have the expecteddir in the source directory. +# +# Also pick up anything passed in EXTRA_REGRESS_OPTS. +REGRESS_OPTS = --expecteddir=$(srcdir) \ + --dbname=ecpg1_regression,ecpg2_regression --create-role=regress_ecpg_user1,regress_ecpg_user2 \ + $(EXTRA_REGRESS_OPTS) check: all $(with_temp_install) ./pg_regress $(REGRESS_OPTS) --temp-instance=./tmp_check $(TEMP_CONF) --bindir= $(pg_regress_locale_flags) $(THREAD) --schedule=$(srcdir)/ecpg_schedule sql/twophase diff --git a/src/interfaces/ecpg/test/pg_regress_ecpg.c b/src/interfaces/ecpg/test/pg_regress_ecpg.c index f920af4560c..84e45ceebe0 100644 --- a/src/interfaces/ecpg/test/pg_regress_ecpg.c +++ b/src/interfaces/ecpg/test/pg_regress_ecpg.c @@ -164,7 +164,7 @@ ecpg_start_test(const char *testname, char *appnameenv; snprintf(inprg, sizeof(inprg), "%s/%s", inputdir, testname); - snprintf(insource, sizeof(insource), "%s.c", testname); + snprintf(insource, sizeof(insource), "%s/%s.c", inputdir, testname); /* make a version of the test name that has dashes in place of slashes */ initStringInfo(&testname_dash); @@ -177,13 +177,13 @@ ecpg_start_test(const char *testname, snprintf(expectfile_stdout, sizeof(expectf
Re: [RFC] building postgres with meson - v10
Hi, Sorry for the first email. On Mon, 18 Jul 2022 at 23:23, Andres Freund wrote: > > In https://postgr.es/m/0e81e45c-c9a5-e95b-2782-ab2dfec8bf57%40enterprisedb.com > On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote: > > dff7b5a960 meson: prereq: regress: allow to specify director containing > > expected files. > > > > This could use a bit more explanation, but it doesn't look > > controversial so far. While testing ECPG, C and exe files are generated by meson so these files are in the meson's build directory but expected files are in the source directory. However; there was no way to set different paths for inputs (C and exe files') and expected files' directory. So, I added `--expecteddir` to separately set expected files' directory. Greetings, Nazir Bilal Yavuz
Re: [RFC] building postgres with meson - v10
Hi, On 2022-07-18 23:23:27 +0300, Andres Freund wrote: > Bilal, Peter previously commented on the pg_regress change for ecpg, > perhaps > you can comment on that? > > In > https://postgr.es/m/0e81e45c-c9a5-e95b-2782-ab2dfec8bf57%40enterprisedb.com > On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote: > > dff7b5a960 meson: prereq: regress: allow to specify director containing > > expected files. > > > > This could use a bit more explanation, but it doesn't look > > controversial so far While testing ECPG, C and exe files are generated by meson so these files are in the meson's build directory but expected files are in the source directory. However; there was no way to set different paths for inputs (C and exe files') and expected files' directory. So, I added `--expecteddir` to separately set expected files' directory. Greetings, Nazir Bilal Yavuz On Mon, 18 Jul 2022 at 23:23, Andres Freund wrote: > Hi, > > On 2022-07-18 11:33:09 +0200, Peter Eisentraut wrote: > > The following patches are ok to commit IMO: > > > > a1c5542929 prereq: Deal with paths containing \ and spaces in > basebackup_to_shell tests > > e37951875d meson: prereq: psql: Output dir and dependency generation for > sql_help > > 18cc9fbd02 meson: prereq: ecpg: Add and use output directory argument > for preproc/*.pl > > 58a32694e9 meson: prereq: Move snowball_create.sql creation into perl > file > > 59b8bffdaf meson: prereq: Add output path arg in generate-lwlocknames.pl > > 2db97b00d5 meson: prereq: generate-errcodes.pl: Accept output file > > fb8f52f21d meson: prereq: unicode: Allow to specify output directory > > 8f1e4410d6 meson: prereq: Refactor dtrace postprocessing make rules > > 3d18a20b11 meson: prereq: Add --outdir option to gen_node_support.pl > > I pushed these. Thanks for the reviews and patches! > > The symbol export stuff has also been pushed (discussed in a separate > thread). > > It's nice to see the meson patchset length reduced by this much. > > I pushed a rebased version of the remaining branches to git. I'll be on > vacation for a bit, I'm not sure I can get a new version with further > cleanups > out before. > > > Given that we can't use src/tools/gen_versioning_script.pl for the make > build, > due to not depending on perl for tarball builds, I'm inclined to rewrite it > python (which we depend on via meson anyway) and consider it a meson > specific > wrapper? > > > Bilal, Peter previously commented on the pg_regress change for ecpg, > perhaps > you can comment on that? > > In > https://postgr.es/m/0e81e45c-c9a5-e95b-2782-ab2dfec8bf57%40enterprisedb.com > On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote: > > dff7b5a960 meson: prereq: regress: allow to specify director containing > > expected files. > > > > This could use a bit more explanation, but it doesn't look > > controversial so far. > > Greetings, > > Andres Freund >
Re: [RFC] building postgres with meson - v10
Hi, On 2022-07-18 11:33:09 +0200, Peter Eisentraut wrote: > The following patches are ok to commit IMO: > > a1c5542929 prereq: Deal with paths containing \ and spaces in > basebackup_to_shell tests > e37951875d meson: prereq: psql: Output dir and dependency generation for > sql_help > 18cc9fbd02 meson: prereq: ecpg: Add and use output directory argument for > preproc/*.pl > 58a32694e9 meson: prereq: Move snowball_create.sql creation into perl file > 59b8bffdaf meson: prereq: Add output path arg in generate-lwlocknames.pl > 2db97b00d5 meson: prereq: generate-errcodes.pl: Accept output file > fb8f52f21d meson: prereq: unicode: Allow to specify output directory > 8f1e4410d6 meson: prereq: Refactor dtrace postprocessing make rules > 3d18a20b11 meson: prereq: Add --outdir option to gen_node_support.pl I pushed these. Thanks for the reviews and patches! The symbol export stuff has also been pushed (discussed in a separate thread). It's nice to see the meson patchset length reduced by this much. I pushed a rebased version of the remaining branches to git. I'll be on vacation for a bit, I'm not sure I can get a new version with further cleanups out before. Given that we can't use src/tools/gen_versioning_script.pl for the make build, due to not depending on perl for tarball builds, I'm inclined to rewrite it python (which we depend on via meson anyway) and consider it a meson specific wrapper? Bilal, Peter previously commented on the pg_regress change for ecpg, perhaps you can comment on that? In https://postgr.es/m/0e81e45c-c9a5-e95b-2782-ab2dfec8bf57%40enterprisedb.com On 2022-07-06 11:03:31 +0200, Peter Eisentraut wrote: > dff7b5a960 meson: prereq: regress: allow to specify director containing > expected files. > > This could use a bit more explanation, but it doesn't look > controversial so far. Greetings, Andres Freund
Re: [RFC] building postgres with meson - v10
Hi, On 2022-07-18 11:12:10 +0300, Aleksander Alekseev wrote: > > Just a quick question - is there a reason for changing the subject of > > the emails? > > > > Not all email clients handle this well, e.g. Google Mail considers > > this being 10 separate threads. The CF application and/or > > pgsql-hackers@ archive also don't recognise this as a continuation of > > the original thread. So all the discussions in -v8, -v9, -v9 ets > > threads get lost. > > > > May I suggest using a single thread? > > OK, the part about the archive is wrong - I scrolled right to the end > of the thread, didn't notice v10 patch above and assumed it was lost. > Sorry for the confusion. However, the part about various email clients > is accurate. For me the thread is too long to look through without some separation. I wouldn't do the version in the subject for a small patchset / thread, but at this size I think it's reasonable. Greetings, Andres Freund
Re: [RFC] building postgres with meson - v10
On 15.07.22 07:08, Andres Freund wrote: Attached is v10 of the meson patchset. Lots of small changes, I don't think anything major. I tried to address most of Peter's feedback for the earlier patches. The following patches are ok to commit IMO: a1c5542929 prereq: Deal with paths containing \ and spaces in basebackup_to_shell tests e37951875d meson: prereq: psql: Output dir and dependency generation for sql_help 18cc9fbd02 meson: prereq: ecpg: Add and use output directory argument for preproc/*.pl 58a32694e9 meson: prereq: Move snowball_create.sql creation into perl file 59b8bffdaf meson: prereq: Add output path arg in generate-lwlocknames.pl 2db97b00d5 meson: prereq: generate-errcodes.pl: Accept output file fb8f52f21d meson: prereq: unicode: Allow to specify output directory 8f1e4410d6 meson: prereq: Refactor dtrace postprocessing make rules 3d18a20b11 meson: prereq: Add --outdir option to gen_node_support.pl
Re: [RFC] building postgres with meson - v10
Hi again, > Just a quick question - is there a reason for changing the subject of > the emails? > > Not all email clients handle this well, e.g. Google Mail considers > this being 10 separate threads. The CF application and/or > pgsql-hackers@ archive also don't recognise this as a continuation of > the original thread. So all the discussions in -v8, -v9, -v9 ets > threads get lost. > > May I suggest using a single thread? OK, the part about the archive is wrong - I scrolled right to the end of the thread, didn't notice v10 patch above and assumed it was lost. Sorry for the confusion. However, the part about various email clients is accurate. -- Best regards, Aleksander Alekseev
Re: [RFC] building postgres with meson - v10
Hi Andres, > Attached is v10 of the meson patchset. Lots of small changes, I don't think > anything major. I tried to address most of Peter's feedback for the earlier > patches. > > After this I plan to clean up the "export" patch, since that's I think the > next bigger step, and an improvement on its own. The step after will be to > discuss where we want the output of tests to reside, whether the naming scheme > for tests is good etc. > > I did try to address Peter's criticism around inconsistency of the added > parameters to perl scripts. I hope it's more consistent now. I used the > opportunity to make src/tools/msvc use the "output directory" parameters, > providing coverage for those paths (and removing a few unnecessary chdirs, but > ...). Thanks for continuing to work on this! Just a quick question - is there a reason for changing the subject of the emails? Not all email clients handle this well, e.g. Google Mail considers this being 10 separate threads. The CF application and/or pgsql-hackers@ archive also don't recognise this as a continuation of the original thread. So all the discussions in -v8, -v9, -v9 ets threads get lost. May I suggest using a single thread? -- Best regards, Aleksander Alekseev
