On 29/01/2021 16:03, Frederik Seiffert wrote:
Good news everyone: I’ve successfully built Base on Windows with Clang, libobjc2, and the MSVC ABI. This results in lib, DLL, and PDB files that should be usable in any Windows app without using the MinGW toolchain.
*Very* nice!
I’ve opened the following pull requests with the required changes: - tools-make: https://github.com/gnustep/tools-make/pull/14 - libs-base: https://github.com/gnustep/libs-base/pull/168 A couple of notes: - Linking subproject object files directly (instead of merging) As David had suspected, merging the subproject object files turned out to be an issue: `ld -r` is not available on Windows, and using `ar cr` resulted in a subproject.o that didn’t contain all the symbols (I’m guessing because ar doesn't correctly handle PE/COFF files). So I looked into how we could instead use the subproject object files directly when linking the project, and came up with this solution: instead of merging the object files into subproject.o, we simply write the list of object files to a subproject.txt file (in the same location), and read that to create SUBPROJECT_OBJ_FILES. The change is pretty minimal and seems to work fine on all platforms/configurations in CI: https://github.com/gnustep/tools-make/commit/434f957df0ad81b52a09e3f8c4a200734898b342 Given that this affects all platforms I’d appreciate everyone’s feedback on this, but given that issues with incremental linking with various setups has been talked about multiple times on the mailing list, I hope that this change will be an overall improvement.
ld -r is pretty flaky everywhere. LLD wasn't going to implement it at all and GNUstep was one of only two consumers so I think it's fair to say that it's pretty likely that it will continue to be intermittently broken by various linker versions. I personally like the concept but it's probably better to not depend on it anywhere.
- Building Building with this setup requires using a standard (non-MinGW) Clang that e.g. comes with Visual Studio or is available as pre-built binary from the LLVM website, and requires passing a host to configure like --host=x86_64-pc-windows. Invoking `clang -v` should show a target like "x86_64-pc-windows-msvc". It *might* be that using a MinGW Clang works too when invoked with --target x86_64-pc-windows-msvc, but I haven’t tried that.
The easiest way of installing clang on Windows is via Chocolatey: just run `choco install llvm` once it's installed. Choco is also the easiest way of installing cmake and Ninja for building the runtime.
The build is best done in an MSYS2 shell that does not have any additional *-devel packages installed that might get picked up by configure. Alternatively --disable-xxx flags can be used to prevent these dependencies to be picked up.
Have you tried with the bash.exe in C:\Windows\System32?
I’m currently building like this: # tools-make export CC=/C/LLVM/bin/clang export CXX=/C/LLVM/bin/clang++ export OBJCXX=/C/LLVM/bin/clang++ ./configure --host=x86_64-pc-windows --with-library-combo=ng-gnu-gnu --with-runtime-abi=gnustep-2.0 --prefix=/c/GNUstep/x64 LDFLAGS="-fuse-ld=lld" make install
Is the -fuse-ld=lld line required? You get LLD installed when you install clang, so it's not very important, but the runtime and its tests, at least, also work with LINK.EXE (though you may have to disable incremental linking if it's on by default).
# libs-base . /c/GNUstep/x64/share/GNUstep/Makefiles/GNUstep.sh ./configure --host=x86_64-pc-windows --disable-iconv --disable-tls --disable-icu --disable-xml make -j12 install - Dependencies While most dependencies should be available via NuGet, I have not been able to figure out yet how we might integrate NuGet packages into the build process. Downloading the packages results in individual folders per package with deeply nested folder structures containing the libraries for different architectures, debug/release targets, and Visual Studio versions. For now I have manually copied the headers and libffi.lib import library (renamed to ffi.lib) from the libffi package, and the libffi.dll from the libffi.redist package. For pthreads I have built http://www.sourceware.org/pthreads-win32/ from source by invoking "nmake clean VC" in a VS native tools command prompt, and manually copied the headers (pthread.h, sched.h, semaphore.h), pthreadVC2.lib (renamed to pthread.lib), and phtreadVC2.dll.
At some point it would be nice to lose this dependency. I don't know how much the attitude to C++ in GNUstep has changed recently, but C++11 now has a nice set of threading abstractions that provide everything that NSThread, NSLock and friends need. If we use those instead of pthreads then we'd have something that worked on any platform with a C++11 implementation (Linux, Windows, Fuschia, Haiku, and so on).
I haven’t build with any other dependencies so far but that will be my next endeavor. If anyone has suggestions on how the NuGet packages might be used directly I’d appreciate any input. This is also the main reason I haven’t set this up on CI yet, as we’d have to replicate the currently manual process somehow. I’m also thinking about writing some scripts similar to tools-android that could build all the dependencies and GNUstep itself, ideally also for the different architectures and debug/release targets. - NSFileManager support NSFileManager is currently non-functional with this setup due to dirent.h missing (MinGW has it, but Windows doesn’t). I found this MIT-licensed header-only implementation – would that be ok to add to the project? https://github.com/win32ports/dirent_h/blob/master/dirent.h
I think that looks easier than using Win32 ABIs directly.
- Autoconf GNUC / GCC / Clang detection When targeting the MSVC ABI, Clang does not define `__GNUC__`, which causes Autoconf to not define $GCC (i.e. "checking whether we are using the GNU C compiler" will be NO). I thus removed $GCC as a pre-requisite in GS_CHECK_CC_IS_CLANG(), so that Clang is still correctly detected in this setup.
I *believe* this is controlled by either a gnu dialect (e.g. -std=gnu11 for C) or -fgnu-extensions. That said, I'm not sure if you can #include Windows headers with gnu extensions enabled.
- Debug/Release CRT Just a heads up that on Windows the C runtime libraries (CRT) come in two versions for debug and release builds (e.g. msvcrt and msvcrtd). The libraries that are used seem to have to match between all DLLs – e.g. building libobjc2 for debug will cause all sorts of crashes when Base is build for release, e.g. when freeing memory allocated by libobjc2 in Base.
Fun.
- Linker issues I tried using the MS linker (link.exe), but while that seems to generally work fine linking ObjC files, it throws up at some configure tests (specifically when testing "whether objc really works"). Using LLD works though, so one must run Make configure with LDFLAGS="-fuse-ld=lld". I also had to bypass the config checks for objc_sync_enter(), objc_setProperty(), _Block_copy(), and non-fragile-abi support and assume these functions are there, as these checks result in the following linker errors. lld-link: error: relocation against symbol in discarded section: __start_.objcrt$SEL >>> referenced by C:\msys64\tmp\conftest-78a937.o:(.objc_init) This is reproducible when building a file that calls ObjC runtime functions but doesn’t actually contain any ObjC code. Not sure if that’s expected behavior or bug.
I think this is a bug, probably a clang bug (and probably my fault)
While 64-bit builds works fine with this setup, I have not been able to get 32-bit builds to work yet due to this linker error, but I’m probably just missing some flags somewhere: lld-link: error: libcmt.lib: machine type x64 conflicts with x86 Also, as David noted, this setup requires dllexport/dllimport on all ObjC classes, so I annotated all our ObjC class interfaces with GS_EXPORT_CLASS (and also added missing GS_DECLARE annotations in externs.m), in order for these symbols to be correctly exported in the DLL.
Awesome work, and it makes me much happier than trying to support MinGW. We have given up supporting MinGW for snmalloc and I am considering optionally supporting snmalloc for Objective-C object allocations since it is much faster than the system malloc on most platforms and is particularly good for fixed-size allocations as are common in Objective-C. It would be a shame for this to be an everywhere-except-Windows thing.
David