Re: [Chicken-users] Building the openssl egg on MacOS
I maintain the openssl egg these days. Very nice, thanks for your efforts! Ugh. I've always had the impression macOS gets worse with each release, but this is ridiculous, almost as if they expect everyone to use XCode for development... I find that the usability gets better with each release (still dreading the inevitable downfall) but from a programmer's point of view it does indeed get worse. Mojave locked down some files so even with sudo you can't touch them, deprecated other stuf, and retroactively made it so you need the App Store login to install the full XCode. The easiest workaround is to install a copy of OpenSSL or LibreSSL from the popular Homebrew package manager and build the egg using that copy: This is what I recommend to everyone who has to work on that kind of system. It's sad, but the least painful way of getting work done. Cool, I didn't know that. Homebrew is actually really nice to use, but your point stands about the state of MacOS itself. I hereby reiterate my point that doing development on macOS involves much sadness, such as creating a developer account to do development. I'm afraid there isn't much else you can do, unless you somehow get gcc and the rest of the toolchain working without that. The command-line compilers (clang, swiftc) are available without an App Store login as before ("sudo xcode-select --install"). But the XCode IDE is apparently not. As for the OpenSSL headers, I don't know whether they are intentionally gone for good or only available from the App Store (and can't tell which is worse :) would it make sense to add MacOS-specific checks to the build-openssl script? Before you do that, there is some work I've done on a few more eggs I maintain, I got fed up with writing user-unfriendly shell scripts that I rewrote the non-Windows version to use a Scheme program instead That sounds like a wise move. And for us who maintain Scheme-related infrastructure, it's nice to get the chance to actually write some Scheme code every now and then :) It so often happens that the C/shell parts take the most time. basic version detection, falling back to environment variables and finally bailing out with an error. You can find the latest version of it at the breadline repository [1]. Please let me know if that fulfills your wishes and if not, whether it can be made to do so. If yes, then I'd be willing to migrate the openssl egg towards such a script as well. The reason I haven't done so is because unlike the other eggs I maintain it's something I'd rather not touch unnecessarily, breakages to it will be far more annoying to handle than anything else. And honestly speaking, OpenSSL isn't nice to deal with either :> Thank you for inviting feedback! I'll definitely help work things out. I was thinking, even without considering OpenSSL at all, dealing with pkg-config by itself is complicated enough that it might be worth baking special pkg-config support into chicken-install (or whatever the right internal module is that chicken-install uses). Some of the BSD's re-implemented the original glib-based pkg-config as "pkgconf". Luckily, pkgconf installs a pkg-config command as a compatibility wrapper and it seems quite feature-complete (in the same way that libressl is with openssl, or clang is with gcc). I haven't had any problems with pkgconf. But several OSes don't ship with pkg-config by default, so users have to install it. In order to be user-friendly, it'd be nice if Chicken advised them on how to do it. (If Chicken has all the pkg-config business in one place, instead of separately in each egg, then it's a very modest amount of code to provide a user-friendly experience.) I would tentatively recommend supporting pkg-config as the only way to specify compiler and linker flags for C libraries. I've always thought that users should have the option to give flags manually instead of relying on pkg-config. But today I realized that pkg-config .pc files are basically simple .ini files that contain little more than those same flags. So if users want manual overrides, we could just ask them to write their own .pc file and set PKG_CONFIG_PATH to find it. Also .pc files specify the library version. E.g. the openssl egg requires openssl 1.0.2 or later, which it can check from the .pc file. (It's not reliable to check using the "openssl" shell command, since that may be a different version.) With user-supplied CFLAGS and LDFLAGS we can't easily make this version check. The .pc file is unlikely to have the wrong version number unless the user deliberately messes it up or makes up an arbitrary version number. But is stuff detailed enough that it should be discussed on the "hackers" mailing list? ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Building the openssl egg on MacOS
I spent a _little_ while digging into the details of this a few months ago, and my understanding is this: * Apple were suffering version-skew hell because (reportedly) the OpenSSL folk kept changing the ABI for the library. * So they lost patience and deprecated OpenSSL on macOS Sounds quite plausible. * ...to the extent that there are now compiler-visible deprecation markers in the relevant still-visible header files. * The doctrine is that one should use Apple's own encryption frameworks (as you noted, Vasilij) * (which work fine, in the pretty basic uses I've made of them, but they don't pretend to be OpenSSL) * ...and act as if there were no OpenSSL library at all on macOS. * As Lassi noted, there is still a LibreSSL library on the system in fact, but I believe it is intended to be strictly for legacy use. This is the impression I got as well. The thing is, not only are the Apple frameworks non-portable to any other OS but Apple keeps changing, deprecating and merging frameworks from release to release. They just deprecated OpenGL (and didn't adopt Vulkan) in favor of some brand new framework that doesn't work on any non-Apple OS! And Swift may hold the world record for the number of deprecation warnings in any tool. So Apple's own programming interfaces are some of the least stable ever :) So if you have a tool which depends on Open/LibreSSL, then you need to get it on your machine either from source, or using the package manager of your choice, and not even try using the system one. Based on the non-portability and difficulty of using Apple's frameworks, and their history of changing them endlessly, I suggest adopting a simple policy to direct all Mac users to Homebrew :) Of SSL libraries, LibreSSL sounds like it will be the most stable and secure choice. The OpenSSL compatibility makes it extra nice. ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Building the openssl egg on MacOS
Greetings. On 15 Jul 2019, at 19:53, Vasilij Schneidermann wrote: > And honestly > speaking, OpenSSL isn't nice to deal with either Indeed! I spent a _little_ while digging into the details of this a few months ago, and my understanding is this: * Apple were suffering version-skew hell because (reportedly) the OpenSSL folk kept changing the ABI for the library. * So they lost patience and deprecated OpenSSL on macOS * ...to the extent that there are now compiler-visible deprecation markers in the relevant still-visible header files. * The doctrine is that one should use Apple's own encryption frameworks (as you noted, Vasilij) * (which work fine, in the pretty basic uses I've made of them, but they don't pretend to be OpenSSL) * ...and act as if there were no OpenSSL library at all on macOS. * As Lassi noted, there is still a LibreSSL library on the system in fact, but I believe it is intended to be strictly for legacy use. So if you have a tool which depends on Open/LibreSSL, then you need to get it on your machine either from source, or using the package manager of your choice, and not even try using the system one. If I've misunderstood or misrepresented one of the steps here, I'm sure someone will cheerfully correct me. Best wishes, Norman -- Norman Gray : https://nxg.me.uk SUPA School of Physics and Astronomy, University of Glasgow, UK ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
Re: [Chicken-users] Building the openssl egg on MacOS
Hello Lassi, I maintain the openssl egg these days. > On MacOS Mojave, "chicken-install openssl" fails because the OS doesn't ship > any pkg-config definition file for its version of the openssl library. The > pkg-config definition is supposed to be in a file called 'openssl.pc' but > 'sudo find / -name openssl.pc 2>/dev/null' turns up no such file for the > OpenSSL that comes with the OS. > > (In this version of MacOS the openssl library is actually the OpenSSL > compatibility wrapper of the LibreSSL library: "/usr/bin/openssl version" > says "LibreSSL 2.6.5". Even though the library is LibreSSL, it's still > designed to ship with a file named 'openssl.pc' for compatibility with > OpenSSL. But MacOS doesn't have that file.) Ugh. I've always had the impression macOS gets worse with each release, but this is ridiculous, almost as if they expect everyone to use XCode for development... > The easiest workaround is to install a copy of OpenSSL or LibreSSL from the > popular Homebrew package manager and build the egg using that copy: > > brew install openssl > export PKG_CONFIG_PATH="$(brew --prefix openssl)/lib/pkgconfig" > chicken-install openssl > > Or: > > brew install libressl > export PKG_CONFIG_PATH="$(brew --prefix libressl)/lib/pkgconfig" > chicken-install openssl This is what I recommend to everyone who has to work on that kind of system. It's sad, but the least painful way of getting work done. > In principle one could use the openssl library that ships with MacOS to > build the openssl egg. However, on this OS version I can't find the > C header file anywhere in the file system, even though the > library itself is installed as . The reason for this is because on macOS you're supposed to use "frameworks" instead which contain all that information, much like an .app contains all the files associated with a program. There's even a `-framework` option for `csc` which might just make this work... I can't test this though because I've abandoned that OS many years ago. > I installed Apple's command-line developer tools using "sudo > xcode-select --install". As far as I can tell, I don't have the full > GUI version of XCode anymore with this OS upgrade. I think the GUI > version is still available free of charge but it may now require a Mac > App Store login to install. From Chicken's point of view, we > unfortunately can't assume that people who use Chicken have the full > version of XCode. I hereby reiterate my point that doing development on macOS involves much sadness, such as creating a developer account to do development. I'm afraid there isn't much else you can do, unless you somehow get gcc and the rest of the toolchain working without that. > Many/most MacOS users of intarweb might stumble onto this problem now that > HTTPS websites are everywhere, so would it make sense to add MacOS-specific > checks to the build-openssl script? Since it seems tricky to reliably find > the system OpenSSL header files, maybe it should suggest that people use > Homebrew as the easiest alternative. I can write and test a patch for the > 'build-openssl' shell script if it helps (well, I already wrote most of it > :-) Before you do that, there is some work I've done on a few more eggs I maintain, I got fed up with writing user-unfriendly shell scripts that I rewrote the non-Windows version to use a Scheme program instead doing basic version detection, falling back to environment variables and finally bailing out with an error. You can find the latest version of it at the breadline repository [1]. Please let me know if that fulfills your wishes and if not, whether it can be made to do so. If yes, then I'd be willing to migrate the openssl egg towards such a script as well. The reason I haven't done so is because unlike the other eggs I maintain it's something I'd rather not touch unnecessarily, breakages to it will be far more annoying to handle than anything else. And honestly speaking, OpenSSL isn't nice to deal with either :> Vasilij [1] https://raw.githubusercontent.com/wasamasa/breadline/master/build-breadline.scm signature.asc Description: PGP signature ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users
[Chicken-users] Building the openssl egg on MacOS
On MacOS Mojave, "chicken-install openssl" fails because the OS doesn't ship any pkg-config definition file for its version of the openssl library. The pkg-config definition is supposed to be in a file called 'openssl.pc' but 'sudo find / -name openssl.pc 2>/dev/null' turns up no such file for the OpenSSL that comes with the OS. (In this version of MacOS the openssl library is actually the OpenSSL compatibility wrapper of the LibreSSL library: "/usr/bin/openssl version" says "LibreSSL 2.6.5". Even though the library is LibreSSL, it's still designed to ship with a file named 'openssl.pc' for compatibility with OpenSSL. But MacOS doesn't have that file.) The error chicken-install gives is: OpenSSL >= 1.0.2 seems to be unavailable Using the helpful verbose mode of chicken-install I figured out that the source of the error is the pkg-config invocation in the shell script <~/.cache/chicken-install/openssl/build-openssl>. The easiest workaround is to install a copy of OpenSSL or LibreSSL from the popular Homebrew package manager and build the egg using that copy: brew install openssl export PKG_CONFIG_PATH="$(brew --prefix openssl)/lib/pkgconfig" chicken-install openssl Or: brew install libressl export PKG_CONFIG_PATH="$(brew --prefix libressl)/lib/pkgconfig" chicken-install openssl In principle one could use the openssl library that ships with MacOS to build the openssl egg. However, on this OS version I can't find the C header file anywhere in the file system, even though the library itself is installed as . I installed Apple's command-line developer tools using "sudo xcode-select --install". As far as I can tell, I don't have the full GUI version of XCode anymore with this OS upgrade. I think the GUI version is still available free of charge but it may now require a Mac App Store login to install. From Chicken's point of view, we unfortunately can't assume that people who use Chicken have the full version of XCode. Many/most MacOS users of intarweb might stumble onto this problem now that HTTPS websites are everywhere, so would it make sense to add MacOS-specific checks to the build-openssl script? Since it seems tricky to reliably find the system OpenSSL header files, maybe it should suggest that people use Homebrew as the easiest alternative. I can write and test a patch for the 'build-openssl' shell script if it helps (well, I already wrote most of it :-) ___ Chicken-users mailing list Chicken-users@nongnu.org https://lists.nongnu.org/mailman/listinfo/chicken-users