Jeffrey Smith created KUDU-3678:
-----------------------------------
Summary: MacOS 15.4 thirdparty dependency build failures
Key: KUDU-3678
URL: https://issues.apache.org/jira/browse/KUDU-3678
Project: Kudu
Issue Type: Bug
Components: build
Affects Versions: 1.18
Environment: MacOS 15.4.1
CPU: M3
Reporter: Jeffrey Smith
Attachments: cmake-fix-macos-compilation.patch,
crcutil-fix-macos-arm64-flags.patch, postgres-fix-strchrnul-macos-check.patch
While trying to build Kudu's thirdparty dependencies on arm64, I came across
three different issues that affect MacOS and prevent building the thirdparty
dependencies:
Cmake 2.25:
{code:java}
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
error: expected identifier or '('
318 | FILE *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6,
__IPHONE_2_0, __DARWIN_ALIAS(fdopen));
| ^
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:16:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.c:10:
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/gzguts.h:21:
In file included from
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/stdio.h:61:
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
error: expected ')'
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:16:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
note: to match this '('
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:15:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.c:10:
In file included from
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/gzguts.h:21:
In file included from
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/stdio.h:61:
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
error: expected ')'
318 | FILE *fdopen(int, const char *) __DARWIN_ALIAS_STARTING(__MAC_10_6,
__IPHONE_2_0, __DARWIN_ALIAS(fdopen));
| ^
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:22:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
/Library/Developer/CommandLineTools/SDKs/MacOSX15.5.sdk/usr/include/_stdio.h:318:7:
note: to match this '('
/Users/user/kudu-apache/thirdparty/src/cmake-3.25.3/Utilities/cmzlib/zutil.h:147:33:
note: expanded from macro 'fdopen'
147 | # define fdopen(fd,mode) NULL /* No fdopen() */
| ^
/Library/Developer/CommandLineTools/usr/lib/clang/17/include/__stddef_null.h:26:14:
note: expanded from macro 'NULL'
26 | #define NULL ((void*)0)
| ^
3 errors generated.
make[2]: *** [Utilities/cmzlib/CMakeFiles/cmzlib.dir/zutil.c.o] Error 1
make[1]: *** [Utilities/cmzlib/CMakeFiles/cmzlib.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....{code}
Postgres:
{code:java}
/Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:414:27:
error: 'strchrnul' is only available on macOS 15.4 or newer
[-Werror,-Wunguarded-availability-new]
414 | const char *next_pct = strchrnul(format + 1,
'%');
| ^~~~~~~~~
/Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:366:14:
note: 'strchrnul' has been marked as being introduced in macOS 15.4 here, but
the deployment target is macOS 15.0.0
366 | extern char *strchrnul(const char *s, int c);
| ^
/Users/user/kudu-apache/thirdparty/src/postgresql-17.2/src/port/snprintf.c:414:27:
note: enclose 'strchrnul' in a __builtin_available check to silence this
warning
414 | const char *next_pct = strchrnul(format + 1,
'%');
| ^~~~~~~~~
415 |
416 | /* Dump literal data we just scanned over */
417 | dostr(format, next_pct - format, target);
418 | if (target->failed)
419 | break;
420 |
421 | if (*next_pct == '\0')
422 | break;
423 | format = next_pct;
|
1 error generated.
make[2]: *** [snprintf.o] Error 1
{code}
Crcutil:
{code:java}
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-crc32c_sse4.o] Error 1
make: *** Waiting for unfinished jobs....
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_128_64_gcc_amd64_sse2.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [examples/usage-interface.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_64_64_gcc_i386_mmx.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [examples/usage-usage.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_64_64_gcc_amd64_asm.o] Error 1
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/usage-multiword_64_64_cl_i386_mmx.o] Error 1
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.lo
-MD -MP -MF code/.deps/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.Tpo -c
code/multiword_128_64_gcc_amd64_sse2.cc -fno-common -DPIC -o
code/.libs/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.o
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-crc32c_sse4.lo -MD -MP -MF
code/.deps/libcrcutil_la-crc32c_sse4.Tpo -c code/crc32c_sse4.cc -fno-common
-DPIC -o code/.libs/libcrcutil_la-crc32c_sse4.o
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-multiword_64_64_cl_i386_mmx.lo -MD
-MP -MF code/.deps/libcrcutil_la-multiword_64_64_cl_i386_mmx.Tpo -c
code/multiword_64_64_cl_i386_mmx.cc -fno-common -DPIC -o
code/.libs/libcrcutil_la-multiword_64_64_cl_i386_mmx.o
libtool: compile: g++ -DHAVE_CONFIG_H -I. -DCRCUTIL_USE_MM_CRC32=1 -Wall
-Icode -Iexamples -Itests -O3 -msse2 -msse4.2 -fPIC -g -fno-omit-frame-pointer
-O2 -stdlib=libc++ -MT code/libcrcutil_la-multiword_64_64_gcc_amd64_asm.lo -MD
-MP -MF code/.deps/libcrcutil_la-multiword_64_64_gcc_amd64_asm.Tpo -c
code/multiword_64_64_gcc_amd64_asm.cc -fno-common -DPIC -o
code/.libs/libcrcutil_la-multiword_64_64_gcc_amd64_asm.o
clang++: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]clang++:
error: unsupported option '-msse2' for target 'arm64-apple-darwin24.4.0'
clang++: warning: argument unused during compilation: '-msse4.2'
[-Wunused-command-line-argument]
make: *** [code/libcrcutil_la-multiword_128_64_gcc_amd64_sse2.lo] Error 1
make: *** [code/libcrcutil_la-crc32c_sse4.lo] Error 1
rm -f stamp-h1
touch config.h.in
clang++clang++: : error: error: unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'unsupported option '-msse2' for target
'arm64-apple-darwin24.4.0'clang++clang++: : warning: warning: argument unused
during compilation: '-msse4.2' [-Wunused-command-line-argument]argument unused
during compilation: '-msse4.2' [-Wunused-command-line-argument]make: ***
[code/libcrcutil_la-multiword_64_64_gcc_amd64_asm.lo] Error 1
make: *** [code/libcrcutil_la-multiword_64_64_cl_i386_mmx.lo] Error 1{code}
Cmake fails because it's trying to redefine fdopen, in zutil.h:
{code:java}
#if defined(MACOS) || defined(TARGET_OS_MAC)
# define OS_CODE 7
# ifndef Z_SOLO
# if defined(__MWERKS__) && __dest_os != __be_os && __dest_os != __win32_os
# include <unix.h> /* for fdopen */
# else
# ifndef fdopen
# define fdopen(fd,mode) NULL /* No fdopen() */
# endif
# endif
# endif
#endif {code}
Using the 3 attached patches, I was able to get them to build with no issues
and then compile and run Kudu.
For Cmake, this change is already present in newer versions of cmake (cmake
3.30 and later)
The Postgres patch comes from [this
commit|https://github.com/postgres/postgres/commit/6da2ba1d8a031984eb016fed6741bb2ac945f19d]
The crcutil patch removes the offending flags when we're compiling on Arm64 and
MacOS.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
