Re: zlib detection in Meson on Windows broken?
On Wed, 22 May 2024 at 17:50, Nazir Bilal Yavuz wrote: > Hi, > > On Wed, 22 May 2024 at 17:21, Dave Page wrote: > > > > Hi > > > > On Wed, 22 May 2024 at 14:11, Nazir Bilal Yavuz > wrote: > >> > >> > >> I tried to install your latest zlib artifact (nmake one) to the > >> Windows CI images (not the official ones) [1]. Then, I used the > >> default meson.build file to build but meson could not find the zlib. > >> After that, I modified it like you suggested before; I used a > >> 'cc.find_library()' to find zlib as a fallback method and it seems it > >> worked [2]. Please see meson setup logs below [3], does something > >> similar to the attached solve your problem? > > > > > > That patch does solve my problem - thank you! > > I am glad that it worked! > > Do you think that we need to have this patch in the upstream Postgres? > I am not sure because: > - There is a case that meson is able to find zlib but tests fail. > - This might be a band-aid fix rather than a permanent fix. Yes I do: - This is the documented way to build/install zlib on Windows. - The behaviour with the patch matches <= PG16 - The behaviour with the patch is consistent with OpenSSL detection, and (from a quick, unrelated test), libxml2 detection. Thanks! >
Re: zlib detection in Meson on Windows broken?
Hi, On Wed, 22 May 2024 at 17:21, Dave Page wrote: > > Hi > > On Wed, 22 May 2024 at 14:11, Nazir Bilal Yavuz wrote: >> >> >> I tried to install your latest zlib artifact (nmake one) to the >> Windows CI images (not the official ones) [1]. Then, I used the >> default meson.build file to build but meson could not find the zlib. >> After that, I modified it like you suggested before; I used a >> 'cc.find_library()' to find zlib as a fallback method and it seems it >> worked [2]. Please see meson setup logs below [3], does something >> similar to the attached solve your problem? > > > That patch does solve my problem - thank you! I am glad that it worked! Do you think that we need to have this patch in the upstream Postgres? I am not sure because: - There is a case that meson is able to find zlib but tests fail. - This might be a band-aid fix rather than a permanent fix. -- Regards, Nazir Bilal Yavuz Microsoft
Re: zlib detection in Meson on Windows broken?
Hi On Wed, 22 May 2024 at 14:11, Nazir Bilal Yavuz wrote: > Hi, > > On Tue, 21 May 2024 at 18:24, Dave Page wrote: > > > > > > > > On Tue, 21 May 2024 at 16:04, Andres Freund wrote: > >> > >> Hi, > >> > >> On 2024-05-20 11:58:05 +0100, Dave Page wrote: > >> > I have very little experience with Meson, and even less interpreting > it's > >> > logs, but it seems to me that it's not including the extra lib and > include > >> > directories when it runs the test compile, given the command line it's > >> > reporting: > >> > > >> > cl > C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c > >> > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od > /Oi- > >> > > >> > Bug, or am I doing something silly? > >> > >> It's a buglet. We rely on meson's internal fallback detection of zlib, > if it's > >> not provided via pkg-config or cmake. But it doesn't know about our > >> extra_include_dirs parameter. We should probably fix that... > > > > > > Oh good, then I'm not going bonkers. I'm still curious about how it > works for Andrew but not me, however fixing that buglet should solve my > issue, and would be sensible behaviour. > > > > Thanks! > > I tried to install your latest zlib artifact (nmake one) to the > Windows CI images (not the official ones) [1]. Then, I used the > default meson.build file to build but meson could not find the zlib. > After that, I modified it like you suggested before; I used a > 'cc.find_library()' to find zlib as a fallback method and it seems it > worked [2]. Please see meson setup logs below [3], does something > similar to the attached solve your problem? > That patch does solve my problem - thank you! > > The interesting thing is, I also tried this 'cc.find_library' method > with your old artifact (cmake one). It was able to find zlib but all > tests failed [4]. > Very odd. Whilst I haven't used that particular build elsewhere, we've been building PostgreSQL and shipping client utilities with pgAdmin using cmake-built zlib for years. > > Experimental zlib meson.build diff is attached. > > [1] https://cirrus-ci.com/task/6736867247259648 > [2] https://cirrus-ci.com/build/5286228755480576 > [3] > Run-time dependency zlib found: NO (tried pkgconfig, cmake and system) > Has header "zlib.h" : YES > Library zlib found: YES > ... > External libraries > ... > zlib : YES > ... > [4] https://cirrus-ci.com/task/5208433811521536 > > -- > Regards, > Nazir Bilal Yavuz > Microsoft > -- Dave Page pgAdmin: https://www.pgadmin.org PostgreSQL: https://www.postgresql.org EDB: https://www.enterprisedb.com
Re: zlib detection in Meson on Windows broken?
Hi,
On Tue, 21 May 2024 at 18:24, Dave Page wrote:
>
>
>
> On Tue, 21 May 2024 at 16:04, Andres Freund wrote:
>>
>> Hi,
>>
>> On 2024-05-20 11:58:05 +0100, Dave Page wrote:
>> > I have very little experience with Meson, and even less interpreting it's
>> > logs, but it seems to me that it's not including the extra lib and include
>> > directories when it runs the test compile, given the command line it's
>> > reporting:
>> >
>> > cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
>> > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
>> >
>> > Bug, or am I doing something silly?
>>
>> It's a buglet. We rely on meson's internal fallback detection of zlib, if
>> it's
>> not provided via pkg-config or cmake. But it doesn't know about our
>> extra_include_dirs parameter. We should probably fix that...
>
>
> Oh good, then I'm not going bonkers. I'm still curious about how it works for
> Andrew but not me, however fixing that buglet should solve my issue, and
> would be sensible behaviour.
>
> Thanks!
I tried to install your latest zlib artifact (nmake one) to the
Windows CI images (not the official ones) [1]. Then, I used the
default meson.build file to build but meson could not find the zlib.
After that, I modified it like you suggested before; I used a
'cc.find_library()' to find zlib as a fallback method and it seems it
worked [2]. Please see meson setup logs below [3], does something
similar to the attached solve your problem?
The interesting thing is, I also tried this 'cc.find_library' method
with your old artifact (cmake one). It was able to find zlib but all
tests failed [4].
Experimental zlib meson.build diff is attached.
[1] https://cirrus-ci.com/task/6736867247259648
[2] https://cirrus-ci.com/build/5286228755480576
[3]
Run-time dependency zlib found: NO (tried pkgconfig, cmake and system)
Has header "zlib.h" : YES
Library zlib found: YES
...
External libraries
...
zlib : YES
...
[4] https://cirrus-ci.com/task/5208433811521536
--
Regards,
Nazir Bilal Yavuz
Microsoft
diff --git a/meson.build b/meson.build
index f5ca5cfed49..d89a7a3e277 100644
--- a/meson.build
+++ b/meson.build
@@ -1373,20 +1373,23 @@ endif
zlibopt = get_option('zlib')
zlib = not_found_dep
if not zlibopt.disabled()
- zlib_t = dependency('zlib', required: zlibopt)
+ zlib = dependency('zlib', required: false)
- if zlib_t.type_name() == 'internal'
-# if fallback was used, we don't need to test if headers are present (they
-# aren't built yet, so we can't test)
-zlib = zlib_t
- elif not zlib_t.found()
-warning('did not find zlib')
- elif not cc.has_header('zlib.h',
- args: test_c_args, include_directories: postgres_inc,
- dependencies: [zlib_t], required: zlibopt)
-warning('zlib header not found')
- else
-zlib = zlib_t
+ if zlib.found() and zlib.type_name() != 'internal'
+if not cc.has_header('zlib.h',
+args: test_c_args, include_directories: postgres_inc,
+dependencies: zlib, required: false)
+ zlib = not_found_dep
+endif
+ elif not zlib.found()
+zlib_lib = cc.find_library('zlib',
+ dirs: test_lib_d,
+ header_include_directories: postgres_inc,
+ has_headers: ['zlib.h'],
+ required: zlibopt)
+if zlib_lib.found()
+ zlib = declare_dependency(dependencies: zlib_lib, include_directories: postgres_inc)
+endif
endif
if zlib.found()
Re: zlib detection in Meson on Windows broken?
On Tue, 21 May 2024 at 20:54, Andrew Dunstan wrote: > > On 2024-05-21 Tu 11:04, Andres Freund wrote: > > Hi, > > > > On 2024-05-20 11:58:05 +0100, Dave Page wrote: > >> I have very little experience with Meson, and even less interpreting > it's > >> logs, but it seems to me that it's not including the extra lib and > include > >> directories when it runs the test compile, given the command line it's > >> reporting: > >> > >> cl > C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c > >> /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od > /Oi- > >> > >> Bug, or am I doing something silly? > > It's a buglet. We rely on meson's internal fallback detection of zlib, > if it's > > not provided via pkg-config or cmake. But it doesn't know about our > > extra_include_dirs parameter. We should probably fix that... > > > > Yeah. Meanwhile, what I got working on a totally fresh Windows + VS > install was instead of using extra_include_dirs etc to add the relevant > directories to the environment LIB and INCLUDE settings before calling > `"meson setup". > Yes, that works for me too. -- Dave Page pgAdmin: https://www.pgadmin.org PostgreSQL: https://www.postgresql.org EDB: https://www.enterprisedb.com
Re: zlib detection in Meson on Windows broken?
On Tue, 21 May 2024 at 18:00, Andres Freund wrote: > Hi, > > On 2024-05-20 11:58:05 +0100, Dave Page wrote: > > I then attempt to build PostgreSQL: > > > > meson setup build > > -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include > > -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl > > -Dzlib=enabled --prefix=c:/build64/pgsql > > > > Which results in the output in output.txt, indicating that OpenSSL was > > correctly found, but zlib was not. I've also attached the meson log. > > I forgot to mention that earlier: Assuming you're building something to be > distributed, I'd recommend --auto-features=enabled/disabled and specifying > specifically which dependencies you want to be used. > Good idea - thanks. -- Dave Page pgAdmin: https://www.pgadmin.org PostgreSQL: https://www.postgresql.org EDB: https://www.enterprisedb.com
Re: zlib detection in Meson on Windows broken?
On Tue May 21, 2024 at 10:04 AM CDT, Andres Freund wrote:
Hi,
On 2024-05-20 11:58:05 +0100, Dave Page wrote:
> I have very little experience with Meson, and even less interpreting it's
> logs, but it seems to me that it's not including the extra lib and include
> directories when it runs the test compile, given the command line it's
> reporting:
>
> cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c
> /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi-
>
> Bug, or am I doing something silly?
It's a buglet. We rely on meson's internal fallback detection of zlib, if it's
not provided via pkg-config or cmake. But it doesn't know about our
extra_include_dirs parameter. We should probably fix that...
Here is the relevant Meson code for finding zlib in the Postgres tree:
postgres_inc_d = ['src/include']
postgres_inc_d += get_option('extra_include_dirs')
...
postgres_inc = [include_directories(postgres_inc_d)]
...
zlibopt = get_option('zlib')
zlib = not_found_dep
if not zlibopt.disabled()
zlib_t = dependency('zlib', required: zlibopt)
if zlib_t.type_name() == 'internal'
# if fallback was used, we don't need to test if headers are
present (they
# aren't built yet, so we can't test)
zlib = zlib_t
elif not zlib_t.found()
warning('did not find zlib')
elif not cc.has_header('zlib.h',
args: test_c_args, include_directories: postgres_inc,
dependencies: [zlib_t], required: zlibopt)
warning('zlib header not found')
elif not cc.has_type('z_streamp',
dependencies: [zlib_t], prefix: '#include ',
args: test_c_args, include_directories: postgres_inc)
if zlibopt.enabled()
error('zlib version is too old')
else
warning('zlib version is too old')
endif
else
zlib = zlib_t
endif
if zlib.found()
cdata.set('HAVE_LIBZ', 1)
endif
endif
You can see that we do pass the include dirs to the has_header check.
Something seems to be going wrong here since your extra_include_dirs
isn't being properly translated to include arguments.
--
Tristan Partin
https://tristan.partin.io
Re: zlib detection in Meson on Windows broken?
On 2024-05-21 Tu 11:04, Andres Freund wrote: Hi, On 2024-05-20 11:58:05 +0100, Dave Page wrote: I have very little experience with Meson, and even less interpreting it's logs, but it seems to me that it's not including the extra lib and include directories when it runs the test compile, given the command line it's reporting: cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi- Bug, or am I doing something silly? It's a buglet. We rely on meson's internal fallback detection of zlib, if it's not provided via pkg-config or cmake. But it doesn't know about our extra_include_dirs parameter. We should probably fix that... Yeah. Meanwhile, what I got working on a totally fresh Windows + VS install was instead of using extra_include_dirs etc to add the relevant directories to the environment LIB and INCLUDE settings before calling `"meson setup". cheers andrew -- Andrew Dunstan EDB: https://www.enterprisedb.com
Re: zlib detection in Meson on Windows broken?
Hi, On 2024-05-20 11:58:05 +0100, Dave Page wrote: > I then attempt to build PostgreSQL: > > meson setup build > -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include > -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl > -Dzlib=enabled --prefix=c:/build64/pgsql > > Which results in the output in output.txt, indicating that OpenSSL was > correctly found, but zlib was not. I've also attached the meson log. I forgot to mention that earlier: Assuming you're building something to be distributed, I'd recommend --auto-features=enabled/disabled and specifying specifically which dependencies you want to be used. Greetings, Andres Freund
Re: zlib detection in Meson on Windows broken?
On Tue, 21 May 2024 at 16:04, Andres Freund wrote: > Hi, > > On 2024-05-20 11:58:05 +0100, Dave Page wrote: > > I have very little experience with Meson, and even less interpreting it's > > logs, but it seems to me that it's not including the extra lib and > include > > directories when it runs the test compile, given the command line it's > > reporting: > > > > cl > C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c > > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od > /Oi- > > > > Bug, or am I doing something silly? > > It's a buglet. We rely on meson's internal fallback detection of zlib, if > it's > not provided via pkg-config or cmake. But it doesn't know about our > extra_include_dirs parameter. We should probably fix that... > Oh good, then I'm not going bonkers. I'm still curious about how it works for Andrew but not me, however fixing that buglet should solve my issue, and would be sensible behaviour. Thanks! -- Dave Page pgAdmin: https://www.pgadmin.org PostgreSQL: https://www.postgresql.org EDB: https://www.enterprisedb.com
Re: zlib detection in Meson on Windows broken?
Hi, On 2024-05-20 11:58:05 +0100, Dave Page wrote: > I have very little experience with Meson, and even less interpreting it's > logs, but it seems to me that it's not including the extra lib and include > directories when it runs the test compile, given the command line it's > reporting: > > cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi- > > Bug, or am I doing something silly? It's a buglet. We rely on meson's internal fallback detection of zlib, if it's not provided via pkg-config or cmake. But it doesn't know about our extra_include_dirs parameter. We should probably fix that... Greetings, Andres Freund
Re: zlib detection in Meson on Windows broken?
Hi
On Tue, 21 May 2024 at 15:12, Sandeep Thakkar <
sandeep.thak...@enterprisedb.com> wrote:
> Hi Dave,
>
>
> On Tue, May 21, 2024 at 3:12 PM Dave Page wrote:
>
>> Hi Sandeep, Nazir,
>>
>> On Tue, 21 May 2024 at 10:14, Nazir Bilal Yavuz
>> wrote:
>>
>>> Hi,
>>>
>>> On Tue, 21 May 2024 at 10:20, Sandeep Thakkar
>>> wrote:
>>> >
>>> > Hi Dave,
>>> >
>>> > Is the .pc file generated after the successful build of zlib? If yes,
>>> then meson should be able to detect the installation ideally
>>>
>>> If meson is not able to find the .pc file automatically, using 'meson
>>> setup ... --pkg-config-path $ZLIB_PC_PATH' might help.
>>>
>>
>> The problem is that on Windows there are no standard locations for a
>> Unix-style development library installation such as this, so the chances
>> are that the .pc file will point to entirely the wrong location.
>>
>> For example, please see
>> https://github.com/dpage/winpgbuild/actions/runs/9172187335 which is a
>> Github action that builds a completely vanilla zlib using VC++. If you look
>> at the uploaded artefact containing the build output and example the .pc
>> file, you'll see it references /zlib as the location, which is simply where
>> I built it in that action. On a developer's machine that's almost certainly
>> not going to be where it actually ends up. For example, on the pgAdmin
>> build farm, the dependencies all end up in C:\build64\[whatever]. On the
>> similar Github action I'm building for PostgreSQL, that artefact will be
>> unpacked into /build/zlib.
>>
>>
> The above link returned 404. But I found a successful build at
> https://github.com/dpage/winpgbuild/actions/runs/9175426807. I downloaded
> the artifact but didn't find .pc file as I wanted to look into the content
> of that file.
>
Yeah, sorry - that was an old one.
Following some offline discussion with Andrew I realised I was building
zlib incorrectly - using cmake/msbuild instead of nmake and some manual
copying. Whilst the former does create a working library and a sane looking
installation, it's not the recommended method, which is documented in a
very non-obvious way - far less obvious than "oh look, there's a cmake
config - let's use that".
The new build is done using the recommended method, which works with PG16
and below out of the box with no need to rename any files.
>
> I had a word with Murali who mentioned he encountered a similar issue
> while building PG17 on windows. He worked-around is by using a template .pc
> file that includes these lines:
> --
> prefix=${pcfiledir}/../..
> exec_prefix=${prefix}
> libdir=${prefix}/lib
> sharedlibdir=${prefix}/lib
> includedir=${prefix}/include
> --
>
The issue here is that there is no .pc file created with the correct way of
building zlib, and even if there were (or I created a dummy one),
pkg-config isn't really a thing on Windows.
I'd also note that from what Andrew has shown me of the zlib installation
on the buildfarm member drongo, there is no .pc file there either, and yet
seems to work fine (and my zlib installation now has the exact same set of
files as his does).
>
> But in general I agree with you on the issue of Meson's dependency on
> pkgconfig files to detect the third party libraries.
>
> Of course, for my own builds I can easily make everything use consistent
>> directories, however most people who are likely to want to build PostgreSQL
>> may not want to also build all the dependencies themselves as well, as some
>> are a lot more difficult than zlib. So what tends to happen is people find
>> third party builds or upstream official builds.
>>
>> I would therefore argue that if the .pc file that's found doesn't provide
>> correct paths for us, then Meson should fall back to searching in the paths
>> specified on its command line for the appropriate libraries/headers (which
>> is what it does for OpenSSL for example, as that doesn't include a .pc
>> file). This is also what happens with PG16 and earlier.
>>
>> One other thing I will note is that PG16 and earlier try to use the wrong
>> filename for the import library. For years, it's been a requirement to do
>> something like this: "copy \zlib\lib\zlib.lib \zlib\lib\zdll.lib" to make a
>> build succeed against a "vanilla" zlib build. I haven't got as far as
>> figuring out if the same is true with Meson yet.
>>
>> --
>> Dave Page
>> pgAdmin: https://www.pgadmin.org
>> PostgreSQL: https://www.postgresql.org
>> EDB: https://www.enterprisedb.com
>>
>>
>
> --
> Sandeep Thakkar
>
>
>
--
Dave Page
pgAdmin: https://www.pgadmin.org
PostgreSQL: https://www.postgresql.org
EDB: https://www.enterprisedb.com
Re: zlib detection in Meson on Windows broken?
Hi Dave,
On Tue, May 21, 2024 at 3:12 PM Dave Page wrote:
> Hi Sandeep, Nazir,
>
> On Tue, 21 May 2024 at 10:14, Nazir Bilal Yavuz
> wrote:
>
>> Hi,
>>
>> On Tue, 21 May 2024 at 10:20, Sandeep Thakkar
>> wrote:
>> >
>> > Hi Dave,
>> >
>> > Is the .pc file generated after the successful build of zlib? If yes,
>> then meson should be able to detect the installation ideally
>>
>> If meson is not able to find the .pc file automatically, using 'meson
>> setup ... --pkg-config-path $ZLIB_PC_PATH' might help.
>>
>
> The problem is that on Windows there are no standard locations for a
> Unix-style development library installation such as this, so the chances
> are that the .pc file will point to entirely the wrong location.
>
> For example, please see
> https://github.com/dpage/winpgbuild/actions/runs/9172187335 which is a
> Github action that builds a completely vanilla zlib using VC++. If you look
> at the uploaded artefact containing the build output and example the .pc
> file, you'll see it references /zlib as the location, which is simply where
> I built it in that action. On a developer's machine that's almost certainly
> not going to be where it actually ends up. For example, on the pgAdmin
> build farm, the dependencies all end up in C:\build64\[whatever]. On the
> similar Github action I'm building for PostgreSQL, that artefact will be
> unpacked into /build/zlib.
>
>
The above link returned 404. But I found a successful build at
https://github.com/dpage/winpgbuild/actions/runs/9175426807. I downloaded
the artifact but didn't find .pc file as I wanted to look into the content
of that file.
I had a word with Murali who mentioned he encountered a similar issue while
building PG17 on windows. He worked-around is by using a template .pc file
that includes these lines:
--
prefix=${pcfiledir}/../..
exec_prefix=${prefix}
libdir=${prefix}/lib
sharedlibdir=${prefix}/lib
includedir=${prefix}/include
--
But in general I agree with you on the issue of Meson's dependency on
pkgconfig files to detect the third party libraries.
Of course, for my own builds I can easily make everything use consistent
> directories, however most people who are likely to want to build PostgreSQL
> may not want to also build all the dependencies themselves as well, as some
> are a lot more difficult than zlib. So what tends to happen is people find
> third party builds or upstream official builds.
>
> I would therefore argue that if the .pc file that's found doesn't provide
> correct paths for us, then Meson should fall back to searching in the paths
> specified on its command line for the appropriate libraries/headers (which
> is what it does for OpenSSL for example, as that doesn't include a .pc
> file). This is also what happens with PG16 and earlier.
>
> One other thing I will note is that PG16 and earlier try to use the wrong
> filename for the import library. For years, it's been a requirement to do
> something like this: "copy \zlib\lib\zlib.lib \zlib\lib\zdll.lib" to make a
> build succeed against a "vanilla" zlib build. I haven't got as far as
> figuring out if the same is true with Meson yet.
>
> --
> Dave Page
> pgAdmin: https://www.pgadmin.org
> PostgreSQL: https://www.postgresql.org
> EDB: https://www.enterprisedb.com
>
>
--
Sandeep Thakkar
Re: zlib detection in Meson on Windows broken?
Hi Sandeep, Nazir, On Tue, 21 May 2024 at 10:14, Nazir Bilal Yavuz wrote: > Hi, > > On Tue, 21 May 2024 at 10:20, Sandeep Thakkar > wrote: > > > > Hi Dave, > > > > Is the .pc file generated after the successful build of zlib? If yes, > then meson should be able to detect the installation ideally > > If meson is not able to find the .pc file automatically, using 'meson > setup ... --pkg-config-path $ZLIB_PC_PATH' might help. > The problem is that on Windows there are no standard locations for a Unix-style development library installation such as this, so the chances are that the .pc file will point to entirely the wrong location. For example, please see https://github.com/dpage/winpgbuild/actions/runs/9172187335 which is a Github action that builds a completely vanilla zlib using VC++. If you look at the uploaded artefact containing the build output and example the .pc file, you'll see it references /zlib as the location, which is simply where I built it in that action. On a developer's machine that's almost certainly not going to be where it actually ends up. For example, on the pgAdmin build farm, the dependencies all end up in C:\build64\[whatever]. On the similar Github action I'm building for PostgreSQL, that artefact will be unpacked into /build/zlib. Of course, for my own builds I can easily make everything use consistent directories, however most people who are likely to want to build PostgreSQL may not want to also build all the dependencies themselves as well, as some are a lot more difficult than zlib. So what tends to happen is people find third party builds or upstream official builds. I would therefore argue that if the .pc file that's found doesn't provide correct paths for us, then Meson should fall back to searching in the paths specified on its command line for the appropriate libraries/headers (which is what it does for OpenSSL for example, as that doesn't include a .pc file). This is also what happens with PG16 and earlier. One other thing I will note is that PG16 and earlier try to use the wrong filename for the import library. For years, it's been a requirement to do something like this: "copy \zlib\lib\zlib.lib \zlib\lib\zdll.lib" to make a build succeed against a "vanilla" zlib build. I haven't got as far as figuring out if the same is true with Meson yet. -- Dave Page pgAdmin: https://www.pgadmin.org PostgreSQL: https://www.postgresql.org EDB: https://www.enterprisedb.com
Re: zlib detection in Meson on Windows broken?
Hi, On Tue, 21 May 2024 at 10:20, Sandeep Thakkar wrote: > > Hi Dave, > > Is the .pc file generated after the successful build of zlib? If yes, then > meson should be able to detect the installation ideally If meson is not able to find the .pc file automatically, using 'meson setup ... --pkg-config-path $ZLIB_PC_PATH' might help. -- Regards, Nazir Bilal Yavuz Microsoft
Re: zlib detection in Meson on Windows broken?
Hi Dave, Is the .pc file generated after the successful build of zlib? If yes, then meson should be able to detect the installation ideally On Mon, May 20, 2024 at 4:28 PM Dave Page wrote: > Hi > > I'm working on updating the build of PostgreSQL that pgAdmin uses in its > Windows installers to use Meson ready for the v17 release. I'm using Visual > Studio 2022, on Windows Server 2022. > > I've been unable to persuade Meson to detect zlib, whilst OpenSSL seems to > be fine. > > The dependencies have been built and installed as follows: > > mkdir c:\build64 > > wget https://zlib.net/zlib-1.3.2.tar.gz > tar -zxvf zlib-1.3.2.tar.gz > cd zlib-1.3.2 > cmake -DCMAKE_INSTALL_PREFIX=C:/build64/zlib -G "Visual Studio 17 2022" . > msbuild ALL_BUILD.vcxproj /p:Configuration=Release > msbuild RUN_TESTS.vcxproj /p:Configuration=Release > msbuild INSTALL.vcxproj /p:Configuration=Release > cd .. > > wget https://www.openssl.org/source/openssl-3.0.13.tar.gz > tar -zxvf openssl-3.0.13.tar.gz > cd openssl-3.0.013 > perl Configure VC-WIN64A no-asm --prefix=C:\build64\openssl no-ssl3 > no-comp > nmake > nmake test > nmake install > cd .. > > This results in the following headers and libraries being installed for > zlib: > > C:\Users\dpage\git\postgresql>dir C:\build64\zlib\include > Volume in drive C has no label. > Volume Serial Number is 3AAD-5864 > > Directory of C:\build64\zlib\include > > 17/05/2024 15:56 . > 17/05/2024 15:56 .. > 17/05/2024 15:5417,096 zconf.h > 22/01/2024 19:3296,829 zlib.h >2 File(s)113,925 bytes >2 Dir(s) 98,842,726,400 bytes free > > C:\Users\dpage\git\postgresql>dir C:\build64\zlib\lib > Volume in drive C has no label. > Volume Serial Number is 3AAD-5864 > > Directory of C:\build64\zlib\lib > > 17/05/2024 17:01 . > 17/05/2024 15:56 .. > 17/05/2024 15:5516,638 zlib.lib > 17/05/2024 15:55 184,458 zlibstatic.lib >2 File(s)201,096 bytes >2 Dir(s) 98,842,726,400 bytes free > > I then attempt to build PostgreSQL: > > meson setup build > -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include > -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl > -Dzlib=enabled --prefix=c:/build64/pgsql > > Which results in the output in output.txt, indicating that OpenSSL was > correctly found, but zlib was not. I've also attached the meson log. > > I have very little experience with Meson, and even less interpreting it's > logs, but it seems to me that it's not including the extra lib and include > directories when it runs the test compile, given the command line it's > reporting: > > cl > C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c > /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi- > > Bug, or am I doing something silly? > > > -- > Dave Page > pgAdmin: https://www.pgadmin.org > PostgreSQL: https://www.postgresql.org > EDB: https://www.enterprisedb.com > > -- Sandeep Thakkar
Re: zlib detection in Meson on Windows broken?
On 2024-05-20 Mo 06:58, Dave Page wrote: Hi I'm working on updating the build of PostgreSQL that pgAdmin uses in its Windows installers to use Meson ready for the v17 release. I'm using Visual Studio 2022, on Windows Server 2022. I've been unable to persuade Meson to detect zlib, whilst OpenSSL seems to be fine. The dependencies have been built and installed as follows: mkdir c:\build64 wget https://zlib.net/zlib-1.3.2.tar.gz tar -zxvf zlib-1.3.2.tar.gz cd zlib-1.3.2 cmake -DCMAKE_INSTALL_PREFIX=C:/build64/zlib -G "Visual Studio 17 2022" . msbuild ALL_BUILD.vcxproj /p:Configuration=Release msbuild RUN_TESTS.vcxproj /p:Configuration=Release msbuild INSTALL.vcxproj /p:Configuration=Release cd .. wget https://www.openssl.org/source/openssl-3.0.13.tar.gz tar -zxvf openssl-3.0.13.tar.gz cd openssl-3.0.013 perl Configure VC-WIN64A no-asm --prefix=C:\build64\openssl no-ssl3 no-comp nmake nmake test nmake install cd .. This results in the following headers and libraries being installed for zlib: C:\Users\dpage\git\postgresql>dir C:\build64\zlib\include Volume in drive C has no label. Volume Serial Number is 3AAD-5864 Directory of C:\build64\zlib\include 17/05/2024 15:56 . 17/05/2024 15:56 .. 17/05/2024 15:54 17,096 zconf.h 22/01/2024 19:32 96,829 zlib.h 2 File(s) 113,925 bytes 2 Dir(s) 98,842,726,400 bytes free C:\Users\dpage\git\postgresql>dir C:\build64\zlib\lib Volume in drive C has no label. Volume Serial Number is 3AAD-5864 Directory of C:\build64\zlib\lib 17/05/2024 17:01 . 17/05/2024 15:56 .. 17/05/2024 15:55 16,638 zlib.lib 17/05/2024 15:55 184,458 zlibstatic.lib 2 File(s) 201,096 bytes 2 Dir(s) 98,842,726,400 bytes free I then attempt to build PostgreSQL: meson setup build -Dextra_include_dirs=C:/build64/openssl/include,C:/build64/zlib/include -Dextra_lib_dirs=C:/build64/openssl/lib,C:/build64/zlib/lib -Dssl=openssl -Dzlib=enabled --prefix=c:/build64/pgsql Which results in the output in output.txt, indicating that OpenSSL was correctly found, but zlib was not. I've also attached the meson log. I have very little experience with Meson, and even less interpreting it's logs, but it seems to me that it's not including the extra lib and include directories when it runs the test compile, given the command line it's reporting: cl C:\Users\dpage\git\postgresql\build\meson-private\tmpg_h4xcue\testfile.c /nologo /showIncludes /utf-8 /EP /nologo /showIncludes /utf-8 /EP /Od /Oi- Bug, or am I doing something silly? Hi Dave! Not sure ;-) But this works for the buildfarm animal drongo, so we should be able to make it work for you. I'll contact you offlist and see if I can help. cheers andrew -- Andrew Dunstan EDB:https://www.enterprisedb.com
