[
https://issues.apache.org/jira/browse/KUDU-3678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jeffrey Smith updated KUDU-3678:
--------------------------------
Description:
While trying to build Kudu's thirdparty dependencies on arm64, I came across
three different issues that affect MacOS and prevent building the thirdparty
dependencies:
Cmake 2.25:
{code:java}
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
error: expected identifier or '('
318 | FILE *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6,
__IPHONE_2_0, __DARWIN_ALIAS(fdopen));
| ^
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:16:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.c:10:
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/gzguts.h:21:
In file included from
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/stdio.h:61:
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
error: expected ')'
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:16:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
note: to match this '('
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:15:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.c:10:
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/gzguts.h:21:
In file included from
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/stdio.h:61:
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
error: expected ')'
318 | FILE *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6,
__IPHONE_2_0, __DARWIN_ALIAS(fdopen));
| ^
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:22:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
note: to match this '('
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:14:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
3 errors generated.
make[2]: *** [Utilities/cmzlib/CMakeFiles/cmzlib.dir/zutil.c.o] Error 1
make[1]: *** [Utilities/cmzlib/CMakeFiles/cmzlib.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....{code}
Postgres:
{code:java}
/Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:414:27:
error: 'strchrnul' is only available on macOS 15.4 or newer
[-Werror,-Wunguarded-availability-new]
414 | const char *next_pct = strchrnul(format + 1,
'%');
| ^~~~~~~~~
/Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:366:14:
note: 'strchrnul' has been marked as being introduced in macOS 15.4 here, but
the deployment target is macOS 15.0.0
366 | extern char *strchrnul(const char *s, int c);
| ^
/Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:414:27:
note: enclose 'strchrnul' in a __builtin_available check to silence this
warning
414 | const char *next_pct = strchrnul(format + 1,
'%');
| ^~~~~~~~~
415 |
416 | /* Dump literal data we just scanned over */
417 | dostr(format, next_pct - format, target);
418 | if (target->failed)
419 | break;
420 |
421 | if (*next_pct == '\0')
422 | break;
423 | format = next_pct;
|
1 error generated.
make[2]: *** [snprintf.o] Error 1
{code}
Crcutil:
{code:java}
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-crc32c_sse4.o] Error 1
make: *** Waiting for unfinished jobs....
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_128_64_gcc_amd64_sse2.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [examples/usage-interface.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_64_64_gcc_i386_mmx.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [examples/usage-usage.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_64_64_gcc_amd64_asm.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_64_64_cl_i386_mmx.o] Error 1
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.lo
-MD -MP -MF code/.deps/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.Tpo -c
code/multiword_128_64_gcc_amd64_sse2.cc -fno-common -DPIC -o
code/.libs/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.o
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-crc32c_sse4.lo -MD -MP -MF
code/.deps/libcrcutil_la-crc32c_sse4.Tpo -c code/crc32c_sse4.cc -fno-common
-DPIC -o code/.libs/libcrcutil_la-crc32c_sse4.o
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-multiword_64_64_cl_i386_mmx.lo -MD
-MP -MF code/.deps/libcrcutil_la-multiword_64_64_cl_i386_mmx.Tpo -c
code/multiword_64_64_cl_i386_mmx.cc -fno-common -DPIC -o
code/.libs/libcrcutil_la-multiword_64_64_cl_i386_mmx.o
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-multiword_64_64_gcc_amd64_asm.lo -MD
-MP -MF code/.deps/libcrcutil_la-multiword_64_64_gcc_amd64_asm.Tpo -c
code/multiword_64_64_gcc_amd64_asm.cc -fno-common -DPIC -o
code/.libs/libcrcutil_la-multiword_64_64_gcc_amd64_asm.o
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]clang++:
error: unsupported option '-msse2' for target 'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.lo] Error 1
make: *** [code/libcrcutil_la-crc32c_sse4.lo] Error 1
rm -f stamp-h1
touch config.h.in
clang++clang++: : error: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'clang++clang++: : warning: warning: argument unused
during compilation: '-msse4.2' [-Wunused-command-line-argument]argument unused
during compilation: '-msse4.2' [-Wunused-command-line-argument]make: ***
[code/libcrcutil_la-multiword_64_64_gcc_amd64_asm.lo] Error 1
make: *** [code/libcrcutil_la-multiword_64_64_cl_i386_mmx.lo] Error 1{code}
Cmake fails because it's trying to redefine fdopen, in zutil.h:
{code:java}
#if defined(MACOS) || defined(TARGET_OS_MAC)
# define OS_CODE 7
# ifndef Z_SOLO
# if defined(__MWERKS__) && __dest_os != __be_os && __dest_os != __win32_os
# include <unix.h> /* for fdopen */
# else
# ifndef fdopen
# define fdopen(fd,mode) NULL /* No fdopen() */
# endif
# endif
# endif
#endif {code}
Using the 3 attached patches, I was able to get them to build with no issues
and then compile and run Kudu.
For Cmake, this change is already present in newer versions of cmake (cmake
3.30 and later)
The Postgres patch is based off [this
commit|https://github.com/postgres/postgres/commit/6da2ba1d8a031984eb016fed6741bb2ac945f19d]
The crcutil patch removes the offending flags when we're compiling on Arm64 and
MacOS.
was:
While trying to build Kudu's thirdparty dependencies on arm64, I came across
three different issues that affect MacOS and prevent building the thirdparty
dependencies:
Cmake 2.25:
{code:java}
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
error: expected identifier or '('
318 | FILE *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6,
__IPHONE_2_0, __DARWIN_ALIAS(fdopen));
| ^
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:16:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.c:10:
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/gzguts.h:21:
In file included from
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/stdio.h:61:
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
error: expected ')'
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:16:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
note: to match this '('
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:15:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.c:10:
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/gzguts.h:21:
In file included from
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/stdio.h:61:
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
error: expected ')'
318 | FILE *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6,
__IPHONE_2_0, __DARWIN_ALIAS(fdopen));
| ^
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:22:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
note: to match this '('
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:14:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
3 errors generated.
make[2]: *** [Utilities/cmzlib/CMakeFiles/cmzlib.dir/zutil.c.o] Error 1
make[1]: *** [Utilities/cmzlib/CMakeFiles/cmzlib.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....{code}
Postgres:
{code:java}
/Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:414:27:
error: 'strchrnul' is only available on macOS 15.4 or newer
[-Werror,-Wunguarded-availability-new]
414 | const char *next_pct = strchrnul(format + 1,
'%');
| ^~~~~~~~~
/Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:366:14:
note: 'strchrnul' has been marked as being introduced in macOS 15.4 here, but
the deployment target is macOS 15.0.0
366 | extern char *strchrnul(const char *s, int c);
| ^
/Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:414:27:
note: enclose 'strchrnul' in a __builtin_available check to silence this
warning
414 | const char *next_pct = strchrnul(format + 1,
'%');
| ^~~~~~~~~
415 |
416 | /* Dump literal data we just scanned over */
417 | dostr(format, next_pct - format, target);
418 | if (target->failed)
419 | break;
420 |
421 | if (*next_pct == '\0')
422 | break;
423 | format = next_pct;
|
1 error generated.
make[2]: *** [snprintf.o] Error 1
{code}
Crcutil:
{code:java}
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-crc32c_sse4.o] Error 1
make: *** Waiting for unfinished jobs....
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_128_64_gcc_amd64_sse2.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [examples/usage-interface.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_64_64_gcc_i386_mmx.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [examples/usage-usage.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_64_64_gcc_amd64_asm.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_64_64_cl_i386_mmx.o] Error 1
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.lo
-MD -MP -MF code/.deps/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.Tpo -c
code/multiword_128_64_gcc_amd64_sse2.cc -fno-common -DPIC -o
code/.libs/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.o
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-crc32c_sse4.lo -MD -MP -MF
code/.deps/libcrcutil_la-crc32c_sse4.Tpo -c code/crc32c_sse4.cc -fno-common
-DPIC -o code/.libs/libcrcutil_la-crc32c_sse4.o
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-multiword_64_64_cl_i386_mmx.lo -MD
-MP -MF code/.deps/libcrcutil_la-multiword_64_64_cl_i386_mmx.Tpo -c
code/multiword_64_64_cl_i386_mmx.cc -fno-common -DPIC -o
code/.libs/libcrcutil_la-multiword_64_64_cl_i386_mmx.o
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-multiword_64_64_gcc_amd64_asm.lo -MD
-MP -MF code/.deps/libcrcutil_la-multiword_64_64_gcc_amd64_asm.Tpo -c
code/multiword_64_64_gcc_amd64_asm.cc -fno-common -DPIC -o
code/.libs/libcrcutil_la-multiword_64_64_gcc_amd64_asm.o
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]clang++:
error: unsupported option '-msse2' for target 'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.lo] Error 1
make: *** [code/libcrcutil_la-crc32c_sse4.lo] Error 1
rm -f stamp-h1
touch config.h.in
clang++clang++: : error: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'clang++clang++: : warning: warning: argument unused
during compilation: '-msse4.2' [-Wunused-command-line-argument]argument unused
during compilation: '-msse4.2' [-Wunused-command-line-argument]make: ***
[code/libcrcutil_la-multiword_64_64_gcc_amd64_asm.lo] Error 1
make: *** [code/libcrcutil_la-multiword_64_64_cl_i386_mmx.lo] Error 1{code}
Cmake fails because it's trying to redefine fdopen, in zutil.h:
{code:java}
#if defined(MACOS) || defined(TARGET_OS_MAC)
# define OS_CODE 7
# ifndef Z_SOLO
# if defined(__MWERKS__) && __dest_os != __be_os && __dest_os != __win32_os
# include <unix.h> /* for fdopen */
# else
# ifndef fdopen
# define fdopen(fd,mode) NULL /* No fdopen() */
# endif
# endif
# endif
#endif {code}
Using the 3 attached patches, I was able to get them to build with no issues
and then compile and run Kudu.
For Cmake, this change is already present in newer versions of cmake (cmake
3.30 and later)
The Postgres patch comes from [this
commit|https://github.com/postgres/postgres/commit/6da2ba1d8a031984eb016fed6741bb2ac945f19d]
The crcutil patch removes the offending flags when we're compiling on Arm64 and
MacOS.
> MacOS 15.4 thirdparty dependency build failures
> -----------------------------------------------
>
> Key: KUDU-3678
> URL: https://issues.apache.org/jira/browse/KUDU-3678
> Project: Kudu
> Issue Type: Bug
> Components: build
> Affects Versions: 1.18
> Environment: MacOS 15.4.1
> CPU: M3
> Reporter: Jeffrey Smith
> Priority: Major
> Attachments: cmake-fix-macos-compilation.patch,
> crcutil-fix-macos-arm64-flags.patch, postgres-fix-strchrnul-macos-check.patch
>
>
> While trying to build Kudu's thirdparty dependencies on arm64, I came across
> three different issues that affect MacOS and prevent building the thirdparty
> dependencies:
> Cmake 2.25:
> {code:java}
>
> /Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
> error: expected identifier or '('
> 318 | FILE *fdopen(int, const char *)
> __DARWIN_ALIAS_STARTING(__MAC_10_6, __IPHONE_2_0, __DARWIN_ALIAS(fdopen));
> | ^
> /Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
> note: expanded from macro 'fdopen'
> 147 | # define fdopen(fd,mode) NULL /* No fdopen() */
> | ^
> /Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:16:
> note: expanded from macro 'NULL'
> 26 | #define NULL ((void*)0)
> | ^
> In file included from
> /Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.c:10:
> In file included from
> /Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/gzguts.h:21:
> In file included from
> /Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/stdio.h:61:
> /Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
> error: expected ')'
> /Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
> note: expanded from macro 'fdopen'
> 147 | # define fdopen(fd,mode) NULL /* No fdopen() */
> | ^
> /Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:16:
> note: expanded from macro 'NULL'
> 26 | #define NULL ((void*)0)
> | ^
> /Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
> note: to match this '('
> /Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
> note: expanded from macro 'fdopen'
> 147 | # define fdopen(fd,mode) NULL /* No fdopen() */
> | ^
> /Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:15:
> note: expanded from macro 'NULL'
> 26 | #define NULL ((void*)0)
> | ^
> In file included from
> /Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.c:10:
> In file included from
> /Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/gzguts.h:21:
> In file included from
> /Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/stdio.h:61:
> /Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
> error: expected ')'
> 318 | FILE *fdopen(int, const char *)
> __DARWIN_ALIAS_STARTING(__MAC_10_6, __IPHONE_2_0, __DARWIN_ALIAS(fdopen));
> | ^
> /Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
> note: expanded from macro 'fdopen'
> 147 | # define fdopen(fd,mode) NULL /* No fdopen() */
> | ^
> /Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:22:
> note: expanded from macro 'NULL'
> 26 | #define NULL ((void*)0)
> | ^
> /Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
> note: to match this '('
> /Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
> note: expanded from macro 'fdopen'
> 147 | # define fdopen(fd,mode) NULL /* No fdopen() */
> | ^
> /Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:14:
> note: expanded from macro 'NULL'
> 26 | #define NULL ((void*)0)
> | ^
> 3 errors generated.
> make[2]: *** [Utilities/cmzlib/CMakeFiles/cmzlib.dir/zutil.c.o] Error 1
> make[1]: *** [Utilities/cmzlib/CMakeFiles/cmzlib.dir/all] Error 2
> make[1]: *** Waiting for unfinished jobs....{code}
> Postgres:
> {code:java}
> /Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:414:27:
> error: 'strchrnul' is only available on macOS 15.4 or newer
> [-Werror,-Wunguarded-availability-new]
> 414 | const char *next_pct = strchrnul(format + 1,
> '%');
> | ^~~~~~~~~
> /Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:366:14:
> note: 'strchrnul' has been marked as being introduced in macOS 15.4 here,
> but the deployment target is macOS 15.0.0
> 366 | extern char *strchrnul(const char *s, int c);
> | ^
> /Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:414:27:
> note: enclose 'strchrnul' in a __builtin_available check to silence this
> warning
> 414 | const char *next_pct = strchrnul(format + 1,
> '%');
> | ^~~~~~~~~
> 415 |
> 416 | /* Dump literal data we just scanned over */
> 417 | dostr(format, next_pct - format, target);
> 418 | if (target->failed)
> 419 | break;
> 420 |
> 421 | if (*next_pct == '\0')
> 422 | break;
> 423 | format = next_pct;
> |
> 1 error generated.
> make[2]: *** [snprintf.o] Error 1
> {code}
> Crcutil:
> {code:java}
> clang++: error: unsupported option '-msse2' for target
> 'arm64-apple-darwin24.4.0'
> clang++: warning: argument unused during compilation: '-msse4.2'
> [-Wunused-command-line-argument]
> make: *** [code/usage-crc32c_sse4.o] Error 1
> make: *** Waiting for unfinished jobs....
> clang++: error: unsupported option '-msse2' for target
> 'arm64-apple-darwin24.4.0'
> clang++: warning: argument unused during compilation: '-msse4.2'
> [-Wunused-command-line-argument]
> make: *** [code/usage-multiword_128_64_gcc_amd64_sse2.o] Error 1
> clang++: error: unsupported option '-msse2' for target
> 'arm64-apple-darwin24.4.0'
> clang++: warning: argument unused during compilation: '-msse4.2'
> [-Wunused-command-line-argument]
> make: *** [examples/usage-interface.o] Error 1
> clang++: error: unsupported option '-msse2' for target
> 'arm64-apple-darwin24.4.0'
> clang++: warning: argument unused during compilation: '-msse4.2'
> [-Wunused-command-line-argument]
> make: *** [code/usage-multiword_64_64_gcc_i386_mmx.o] Error 1
> clang++: error: unsupported option '-msse2' for target
> 'arm64-apple-darwin24.4.0'
> clang++: warning: argument unused during compilation: '-msse4.2'
> [-Wunused-command-line-argument]
> make: *** [examples/usage-usage.o] Error 1
> clang++: error: unsupported option '-msse2' for target
> 'arm64-apple-darwin24.4.0'
> clang++: warning: argument unused during compilation: '-msse4.2'
> [-Wunused-command-line-argument]
> make: *** [code/usage-multiword_64_64_gcc_amd64_asm.o] Error 1
> clang++: error: unsupported option '-msse2' for target
> 'arm64-apple-darwin24.4.0'
> clang++: warning: argument unused during compilation: '-msse4.2'
> [-Wunused-command-line-argument]
> make: *** [code/usage-multiword_64_64_cl_i386_mmx.o] Error 1
> libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
> -Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g
> -fno-omit-frame-pointer -O2 -stdlib=libc++ -MT
> code/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.lo -MD -MP -MF
> code/.deps/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.Tpo -c
> code/multiword_128_64_gcc_amd64_sse2.cc -fno-common -DPIC -o
> code/.libs/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.o
> libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
> -Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g
> -fno-omit-frame-pointer -O2 -stdlib=libc++ -MT
> code/libcrcutil_la-crc32c_sse4.lo -MD -MP -MF
> code/.deps/libcrcutil_la-crc32c_sse4.Tpo -c code/crc32c_sse4.cc -fno-common
> -DPIC -o code/.libs/libcrcutil_la-crc32c_sse4.o
> libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
> -Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g
> -fno-omit-frame-pointer -O2 -stdlib=libc++ -MT
> code/libcrcutil_la-multiword_64_64_cl_i386_mmx.lo -MD -MP -MF
> code/.deps/libcrcutil_la-multiword_64_64_cl_i386_mmx.Tpo -c
> code/multiword_64_64_cl_i386_mmx.cc -fno-common -DPIC -o
> code/.libs/libcrcutil_la-multiword_64_64_cl_i386_mmx.o
> libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
> -Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g
> -fno-omit-frame-pointer -O2 -stdlib=libc++ -MT
> code/libcrcutil_la-multiword_64_64_gcc_amd64_asm.lo -MD -MP -MF
> code/.deps/libcrcutil_la-multiword_64_64_gcc_amd64_asm.Tpo -c
> code/multiword_64_64_gcc_amd64_asm.cc -fno-common -DPIC -o
> code/.libs/libcrcutil_la-multiword_64_64_gcc_amd64_asm.o
> clang++: error: unsupported option '-msse2' for target
> 'arm64-apple-darwin24.4.0'
> clang++: warning: argument unused during compilation: '-msse4.2'
> [-Wunused-command-line-argument]clang++:
> error: unsupported option '-msse2' for target 'arm64-apple-darwin24.4.0'
> clang++: warning: argument unused during compilation: '-msse4.2'
> [-Wunused-command-line-argument]
> make: *** [code/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.lo] Error 1
> make: *** [code/libcrcutil_la-crc32c_sse4.lo] Error 1
> rm -f stamp-h1
> touch config.h.in
> clang++clang++: : error: error: unsupported option '-msse2' for target
> 'arm64-apple-darwin24.4.0'unsupported option '-msse2' for target
> 'arm64-apple-darwin24.4.0'clang++clang++: : warning: warning: argument unused
> during compilation: '-msse4.2' [-Wunused-command-line-argument]argument
> unused during compilation: '-msse4.2' [-Wunused-command-line-argument]make:
> *** [code/libcrcutil_la-multiword_64_64_gcc_amd64_asm.lo] Error 1
> make: *** [code/libcrcutil_la-multiword_64_64_cl_i386_mmx.lo] Error 1{code}
> Cmake fails because it's trying to redefine fdopen, in zutil.h:
> {code:java}
> #if defined(MACOS) || defined(TARGET_OS_MAC)
> # define OS_CODE 7
> # ifndef Z_SOLO
> # if defined(__MWERKS__) && __dest_os != __be_os && __dest_os != __win32_os
> # include <unix.h> /* for fdopen */
> # else
> # ifndef fdopen
> # define fdopen(fd,mode) NULL /* No fdopen() */
> # endif
> # endif
> # endif
> #endif {code}
> Using the 3 attached patches, I was able to get them to build with no issues
> and then compile and run Kudu.
> For Cmake, this change is already present in newer versions of cmake (cmake
> 3.30 and later)
> The Postgres patch is based off [this
> commit|https://github.com/postgres/postgres/commit/6da2ba1d8a031984eb016fed6741bb2ac945f19d]
> The crcutil patch removes the offending flags when we're compiling on Arm64
> and MacOS.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
