Re: [systemd-devel] [RFC 00/25] Compile against the musl libc
On Thu, 18.09.14 21:21, Emil Renner Berthing (syst...@esmil.dk) wrote: Hi Tom, On 18 September 2014 18:29, Tom Gundersen t...@jklm.no wrote: In general, I don't think we should add patches for the sole purpose of non-glibc compatibility. You would in most cases be much better served by adding the missing functionality to your libc, rather than to each of the project requiring the functionality. Thank you for looking through these. The problem is that musl is aiming at making a lean and efficient posix libc, not a glibc clone. So stuff like parse_printf_format and canonicalize_file_name is not a fix, but unnecessary bloat. So it seems I'm stuck between a rock and a hard place. Well, bloat is a misleading term here. canonicalize_file_name() you can implement via a macro hence costs pretty much nothing. And parse_printf_format() is genuinely useful, that's why we make use of it. It's also something that can only be reasonably implemented at the same place your printf() implementaton is maintained, since it needs to stay compatible with that. In general I think it's a very weird approach to have a non-useful libc and then claim things would be less bloated because every user of a variety of useful GNU APIs is supposed to reimplement everything from scracth, so that we have a multitude of different, crappy implementations in a variety of projects, instead of a single one in the libc. Your overall footprint will be larger. Also, the entire dance around POSIX is awesome and what we thrive for, and GNU is bad is just nonsense. It shows that your libc was created for religious reasons, not for practical ones. If you want to create something useful, create something useful, but don't cut off useful functionality people use just because it was introduced by GNU, not by POSIX. The POSIX API alone does not cover many many things anyway. POSIX doesn't know epoll, pivot_root(), memfd, fanotify, inotify and all those other Linux-specific syscalls, that solve real problems. Don't tell me musl refuses to support those too? Or is Linux good, just GNU bad, hence Linux extensions are well supported, just not the ones from GNU? We will not take part in this game. Yes, we tend to gravitate more towards POSIX interfaces, if both a GNU and a POSIX version exists and both are equivalent in their functionality. But also: we will make use of GNU extensions and of Linux extensions if they are useful. And since these ultimately are part of the Linux API, that is the right thing to do. Anyway, regardless what the goal of the patches people sent us are and whether we sympasize with them, we'll always merge the ones that make sense to us. We are not married to GNU, but we are married to the idea that we will rely on good APIs exposed in the generally accepted Linux API which is the one glibc exposes. Hope that makes sense, Lennart -- Lennart Poettering, Red Hat ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC 00/25] Compile against the musl libc
On Fri, Sep 19, 2014 at 01:38:44PM +0200, Umut Tezduyar Lindskog wrote: Axis Communications is running fairly compact version of systemd with some properties disabled with --disable--xyz. Files are a bit over 10 mb on MIPS ISA (stripped of binaries and conf files). Top 10 large files are (in kb) 276 ./Root/usr/lib/systemd/systemd-udevd 280 ./Root/usr/bin/udevadm 288 ./Root/usr/bin/busctl 296 ./Root/usr/lib/systemd/systemd-hostnamed 300 ./Root/usr/lib/systemd/systemd-timedated 304 ./Root/usr/lib/systemd/systemd-bus-proxyd 368 ./Root/usr/bin/systemd-nspawn 548 ./Root/usr/bin/systemctl 1152 ./Root/usr/bin/systemd-analyze 1220 ./Root/usr/lib/systemd/systemd Umut [1] Maybe it is time (again) to visit making /shared code part of libsystemd or libsystemd-internal. Has anyone made any calculation regarding the cost of using static library. If not, maybe I can take a stab. [2] I am quite surprised systemd-analyze is so big. systemd-analyze includes full systemd manager now (for 'verify'), so this is not surprising. Zbyszek ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC 00/25] Compile against the musl libc
On Thu, Sep 18, 2014 at 9:21 PM, Emil Renner Berthing syst...@esmil.dk wrote: On 18 September 2014 18:29, Tom Gundersen t...@jklm.no wrote: In general, I don't think we should add patches for the sole purpose of non-glibc compatibility. You would in most cases be much better served by adding the missing functionality to your libc, rather than to each of the project requiring the functionality. Thank you for looking through these. The problem is that musl is aiming at making a lean and efficient posix libc, not a glibc clone. So stuff like parse_printf_format and canonicalize_file_name is not a fix, but unnecessary bloat. So it seems I'm stuck between a rock and a hard place. I guess I had hoped the resistance in this camp would be more against doing the work rather than actually be able to work with a posix libc and not only glibc. We are not against being more POSIX compliant per se, but reimplementing functionality in systemd that arguably should be done in libc does not make much sense to me. If parse_printf_format (for instance) is too bloated for yor libc, then surely putting it in systemd gives the same amount of bloat overall (actually, it would be worse as it would probably get statically linked into most of the systemd-* binaries, whereas if you keep it in your libc it would only be around once, and if you have more than just systemd around that requires the functions to be reimplemented it gets worse still). Using a minimal libc may make sense with projects that explicitly only use minimal functionality (without reimplementing things), but avoiding useful features is not a game systemd wants to be in, so I don't think that applies here. I understand not wanting to include unused functionality in libc if you are trying to save space, but by not including stuff that is actually used on your system I don't see how you gain anything. I would rather try to compile glibc and systemd with unused functionality disabled, or perhaps add the missing functionality to your libc and make it optional for systems that don't require it. That said, we are of course happy to take patches that are the right thing to do regardless of which libc is being used, so I merged some of the patches you posted. Cool. I can resend updates of the ones you commented on. Btw, have you considered simply putting glibc on your router in the first place? If you can fit systemd, you can probably fit glibc too... Well, last time I tried this linux, musl, systemd, busybox, base, I had a Lua webserver running from linux + squashfs taking around 3MB out the 8MB available on the gadget. Maybe systemd has grown since then, I didn't get this far this time around. Also I think I might have made a mistake in mentioning OpenWRT. My point was more that I'd like systemd to run on hardware with the same kind of specs as OpenWRT is designed for, not necessarely replacing OpenWRT everywhere. I was thinking of gadgets like the VoCore [1], Arduino's Yun [2], Intel's Galileo boards[3] and Edison SOC [4] to name a few. Would be cool if you get this working. Good luck! Cheers, Tom [1] https://www.indiegogo.com/projects/vocore-a-coin-sized-linux-computer-with-wifi [2] http://arduino.cc/en/Main/ArduinoBoardYun [3] http://www.intel.com/content/www/us/en/do-it-yourself/galileo-maker-quark-board.html [4] http://www.youtube.com/watch?v=GY8kaaFzbTE ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC 00/25] Compile against the musl libc
Axis Communications is running fairly compact version of systemd with some properties disabled with --disable--xyz. Files are a bit over 10 mb on MIPS ISA (stripped of binaries and conf files). Top 10 large files are (in kb) 276 ./Root/usr/lib/systemd/systemd-udevd 280 ./Root/usr/bin/udevadm 288 ./Root/usr/bin/busctl 296 ./Root/usr/lib/systemd/systemd-hostnamed 300 ./Root/usr/lib/systemd/systemd-timedated 304 ./Root/usr/lib/systemd/systemd-bus-proxyd 368 ./Root/usr/bin/systemd-nspawn 548 ./Root/usr/bin/systemctl 1152 ./Root/usr/bin/systemd-analyze 1220 ./Root/usr/lib/systemd/systemd Umut [1] Maybe it is time (again) to visit making /shared code part of libsystemd or libsystemd-internal. Has anyone made any calculation regarding the cost of using static library. If not, maybe I can take a stab. [2] I am quite surprised systemd-analyze is so big. On 18 Sep 2014, at 05:19 pm, Jóhann B. Guðmundsson johan...@gmail.com wrote: On 09/18/2014 02:20 PM, Philippe De Swert wrote: Hi, On 18/09/14 17:13, Emil Renner Berthing wrote: On 18 September 2014 16:10, Jóhann B. Guðmundsson johan...@gmail.com wrote: On 09/18/2014 01:24 PM, Emil Renner Berthing wrote: The real reason is of course that I'd like to see systemd running on my router and other small devices that usually run some OpenWRT derivative. The openwrt community is still going forward with their (re)-invention of init system called procd since they were incapable of using systemd built with --disable-alots right? Or are you talking about you dont want to use OpenWRT but your own or something else and thus the request? Yes, indeed. Also I am hoping that having this option means other small devices will run systemd in the future. I think you are missing the point of musl So you have so little space you need a smaller libc but then you want to use a binary of ~1MB + tools to replace an init of a few Kb? Moreover systemd needs a lot more kernel options, so making smaller kernels gets hard. Not to mention for example cgroups etc is often very unstable in the earlier kernels that have the support for those devices. IOW more kernel crashes due to systemd requiring cgroups. The reason why OpenWrt would do their own init, is to keep it small. Often those routers etc have only a few Mb of flash. Even with compression just having a 2Mb just for the init system is a bit much. It would be interesting to get some how low can you go numbers from the embedded crowd regarding systemd and or systemd + udev-hwdb. I would think 16MiB target should be fairly easy to achieved but anything less than that might be well work... JBG ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] [RFC 00/25] Compile against the musl libc
Hi, Around the internet one can read statements such as systemd is designed with glibc in mind and the systemd developers' idea of a standard libc is one that has bug-for-bug compatibility with glibc. So in a fit of naivety I decided to test this out and see how much work it would take to make systemd work with musl. The real reason is of course that I'd like to see systemd running on my router and other small devices that usually run some OpenWRT derivative. This series makes systemd compile cleanly against musl 1.1.4 with a minimal configuration and also fixes a few runtime errors. It is only lightly tested and a few of patches could definitely be done in a nicer, more generic way. Also I'd love to get even more stuff working like eg. networkd and resolved. However I would like some early feedback and see if this is something that could one day make it upstream or I'm just wasting my time. The first two patches are not strictly needed, but it makes the compilation much quieter and they are correct according POSIX. /Emil Emil Renner Berthing (25): include poll.h rather than sys/poll.h include fcntl.h rather than sys/fcntl.h udev: link-config: remove unneded linux/netdevice.h include sd-rtnl: rtnl-message: remove unneeded linux includes shared/util.h: include sys/reg.h for __WORDSIZE shared/label.h: add missing stdio.h include shared/sparse-endian.h: add missing byteswap.h include don't fail if GLOB_BRACE is not defined shared/socket-util: don't fail if libc doesn't support IDN shared/pty: fall back to kernel header for TIOCSIG shared/missing.h: fall pack to insecure getenv shared/missing.h: check for missing canonicalize_file_name shared/missing.h: check for missing strndupa shared/missing.h: check for missing __compar_fn_t typedef shared/utmp-wtmp: don't fail if libc doesn't support utmpx/wtmpx add fallback parse_printf_format implementation dhcp-internal.h: add fallback IPTOS_CLASS_CS6 definition sd-ipv4all: fall back to random if no random_r mount-setup: skip relabelling when SELinux and SMACK not supported udevadm-hwdb: don't use glibc-specific qsort_r make sure basename that doesn't alter it's argument support POSIX strerror_r returning int mount: order options before other arguments to mount units: order options before other arguments add -z muldefs to linker flags Makefile.am| 4 + configure.ac | 10 +- man/sd_journal_get_fd.xml | 2 +- src/ask-password/ask-password.c| 2 +- src/bus-proxyd/bus-proxyd.c| 2 +- src/core/execute.c | 8 +- src/core/load-fragment.c | 2 +- src/core/manager.c | 4 +- src/core/mount-setup.c | 4 + src/core/mount.c | 8 +- src/core/unit.c| 6 +- src/delta/delta.c | 14 +- src/initctl/initctl.c | 2 +- src/journal/journal-send.c | 38 ++- src/journal/journalctl.c | 2 +- src/journal/sd-journal.c | 2 +- src/libsystemd-network/dhcp-internal.h | 4 + src/libsystemd-network/sd-ipv4ll.c | 25 +- src/libsystemd/sd-bus/bus-error.c | 14 +- src/libsystemd/sd-bus/bus-socket.c | 2 +- src/libsystemd/sd-bus/sd-bus.c | 2 +- src/libsystemd/sd-login/sd-login.c | 2 +- src/libsystemd/sd-login/test-login.c | 2 +- src/libsystemd/sd-network/sd-network.c | 2 +- src/libsystemd/sd-resolve/sd-resolve.c | 2 +- src/libsystemd/sd-rtnl/rtnl-message.c | 6 - src/libudev/libudev-monitor.c | 2 +- src/locale/localectl.c | 2 +- src/login/logind-inhibit.c | 2 +- src/login/logind-seat.c| 2 +- src/login/logind-session.c | 2 +- src/nspawn/nspawn.c| 2 +- src/readahead/readahead-collect.c | 2 +- src/reply-password/reply-password.c| 2 +- src/resolve/resolved-manager.c | 2 +- src/shared/ask-password-api.c | 2 +- src/shared/cgroup-show.c | 4 +- src/shared/conf-files.c| 4 +- src/shared/install.c | 14 +- src/shared/label.h | 1 + src/shared/log.c | 2 +- src/shared/logs-show.c | 2 +- src/shared/missing.h | 24 +-
Re: [systemd-devel] [RFC 00/25] Compile against the musl libc
On 09/18/2014 01:24 PM, Emil Renner Berthing wrote: The real reason is of course that I'd like to see systemd running on my router and other small devices that usually run some OpenWRT derivative. The openwrt community is still going forward with their (re)-invention of init system called procd since they were incapable of using systemd built with --disable-alots right? Or are you talking about you dont want to use OpenWRT but your own or something else and thus the request? JBG ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC 00/25] Compile against the musl libc
On 18 September 2014 16:10, Jóhann B. Guðmundsson johan...@gmail.com wrote: On 09/18/2014 01:24 PM, Emil Renner Berthing wrote: The real reason is of course that I'd like to see systemd running on my router and other small devices that usually run some OpenWRT derivative. The openwrt community is still going forward with their (re)-invention of init system called procd since they were incapable of using systemd built with --disable-alots right? Or are you talking about you dont want to use OpenWRT but your own or something else and thus the request? Yes, indeed. Also I am hoping that having this option means other small devices will run systemd in the future. /Emil ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC 00/25] Compile against the musl libc
Hi, On 18/09/14 17:13, Emil Renner Berthing wrote: On 18 September 2014 16:10, Jóhann B. Guðmundsson johan...@gmail.com wrote: On 09/18/2014 01:24 PM, Emil Renner Berthing wrote: The real reason is of course that I'd like to see systemd running on my router and other small devices that usually run some OpenWRT derivative. The openwrt community is still going forward with their (re)-invention of init system called procd since they were incapable of using systemd built with --disable-alots right? Or are you talking about you dont want to use OpenWRT but your own or something else and thus the request? Yes, indeed. Also I am hoping that having this option means other small devices will run systemd in the future. I think you are missing the point of musl So you have so little space you need a smaller libc but then you want to use a binary of ~1MB + tools to replace an init of a few Kb? Moreover systemd needs a lot more kernel options, so making smaller kernels gets hard. Not to mention for example cgroups etc is often very unstable in the earlier kernels that have the support for those devices. IOW more kernel crashes due to systemd requiring cgroups. The reason why OpenWrt would do their own init, is to keep it small. Often those routers etc have only a few Mb of flash. Even with compression just having a 2Mb just for the init system is a bit much. Regards, Philippe ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC 00/25] Compile against the musl libc
On 09/18/2014 02:20 PM, Philippe De Swert wrote: Hi, On 18/09/14 17:13, Emil Renner Berthing wrote: On 18 September 2014 16:10, Jóhann B. Guðmundsson johan...@gmail.com wrote: On 09/18/2014 01:24 PM, Emil Renner Berthing wrote: The real reason is of course that I'd like to see systemd running on my router and other small devices that usually run some OpenWRT derivative. The openwrt community is still going forward with their (re)-invention of init system called procd since they were incapable of using systemd built with --disable-alots right? Or are you talking about you dont want to use OpenWRT but your own or something else and thus the request? Yes, indeed. Also I am hoping that having this option means other small devices will run systemd in the future. I think you are missing the point of musl So you have so little space you need a smaller libc but then you want to use a binary of ~1MB + tools to replace an init of a few Kb? Moreover systemd needs a lot more kernel options, so making smaller kernels gets hard. Not to mention for example cgroups etc is often very unstable in the earlier kernels that have the support for those devices. IOW more kernel crashes due to systemd requiring cgroups. The reason why OpenWrt would do their own init, is to keep it small. Often those routers etc have only a few Mb of flash. Even with compression just having a 2Mb just for the init system is a bit much. It would be interesting to get some how low can you go numbers from the embedded crowd regarding systemd and or systemd + udev-hwdb. I would think 16MiB target should be fairly easy to achieved but anything less than that might be well work... JBG ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC 00/25] Compile against the musl libc
El 18/09/14 a las #4, Emil Renner Berthing escribió: Hi, Around the internet one can read statements such as systemd is designed with glibc in mind and the systemd developers' idea of a standard libc is one that has bug-for-bug compatibility with glibc. For all practical purposes this is correct. You not only need glibc but the behavior glibc has, documented or otherwise. If you need an init system supporting whatever other alternate or special libc libraries you have to look elsewhere. As far as I understand no code will be added to support other c libraries. ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC 00/25] Compile against the musl libc
On Thu, Sep 18, 2014 at 3:24 PM, Emil Renner Berthing syst...@esmil.dk wrote: Around the internet one can read statements such as systemd is designed with glibc in mind and the systemd developers' idea of a standard libc is one that has bug-for-bug compatibility with glibc. So in a fit of naivety I decided to test this out and see how much work it would take to make systemd work with musl. The real reason is of course that I'd like to see systemd running on my router and other small devices that usually run some OpenWRT derivative. This series makes systemd compile cleanly against musl 1.1.4 with a minimal configuration and also fixes a few runtime errors. It is only lightly tested and a few of patches could definitely be done in a nicer, more generic way. Also I'd love to get even more stuff working like eg. networkd and resolved. However I would like some early feedback and see if this is something that could one day make it upstream or I'm just wasting my time. The first two patches are not strictly needed, but it makes the compilation much quieter and they are correct according POSIX. In general, I don't think we should add patches for the sole purpose of non-glibc compatibility. You would in most cases be much better served by adding the missing functionality to your libc, rather than to each of the project requiring the functionality. That said, we are of course happy to take patches that are the right thing to do regardless of which libc is being used, so I merged some of the patches you posted. Btw, have you considered simply putting glibc on your router in the first place? If you can fit systemd, you can probably fit glibc too... Thanks! Tom ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] [RFC 00/25] Compile against the musl libc
Hi Tom, On 18 September 2014 18:29, Tom Gundersen t...@jklm.no wrote: In general, I don't think we should add patches for the sole purpose of non-glibc compatibility. You would in most cases be much better served by adding the missing functionality to your libc, rather than to each of the project requiring the functionality. Thank you for looking through these. The problem is that musl is aiming at making a lean and efficient posix libc, not a glibc clone. So stuff like parse_printf_format and canonicalize_file_name is not a fix, but unnecessary bloat. So it seems I'm stuck between a rock and a hard place. I guess I had hoped the resistance in this camp would be more against doing the work rather than actually be able to work with a posix libc and not only glibc. That said, we are of course happy to take patches that are the right thing to do regardless of which libc is being used, so I merged some of the patches you posted. Cool. I can resend updates of the ones you commented on. Btw, have you considered simply putting glibc on your router in the first place? If you can fit systemd, you can probably fit glibc too... Well, last time I tried this linux, musl, systemd, busybox, base, I had a Lua webserver running from linux + squashfs taking around 3MB out the 8MB available on the gadget. Maybe systemd has grown since then, I didn't get this far this time around. Also I think I might have made a mistake in mentioning OpenWRT. My point was more that I'd like systemd to run on hardware with the same kind of specs as OpenWRT is designed for, not necessarely replacing OpenWRT everywhere. I was thinking of gadgets like the VoCore [1], Arduino's Yun [2], Intel's Galileo boards[3] and Edison SOC [4] to name a few. [1] https://www.indiegogo.com/projects/vocore-a-coin-sized-linux-computer-with-wifi [2] http://arduino.cc/en/Main/ArduinoBoardYun [3] http://www.intel.com/content/www/us/en/do-it-yourself/galileo-maker-quark-board.html [4] http://www.youtube.com/watch?v=GY8kaaFzbTE ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org http://lists.freedesktop.org/mailman/listinfo/systemd-devel
