[RFC] Un-staticise the toolchain
I think it is time to stop building the toolchain static. I was told that
original reasoning for static linking was the fear of loosing the ability
to recompile if some problem appears with rtld and any required dynamic
library. Apparently, current dependencies are much more spread, e.g. /bin/sh
is dynamically linked, and statically linked make does not solve anything.
Patch below makes the dynamically linked toolchain a default, adding an
WITHOUT_SHARED_TOOLCHAIN build-time option for real conservators.
I did not looked in details why including bsd.own.mk makes NO_MAN
non-functional. Please see the diffs for gnu/usr.bin/cc1*/Makefile.
gnu/usr.bin/binutils/ar/Makefile |3 +++
gnu/usr.bin/binutils/as/Makefile |3 +++
gnu/usr.bin/binutils/ld/Makefile |3 +++
gnu/usr.bin/binutils/ranlib/Makefile |3 +++
gnu/usr.bin/cc/cc/Makefile|2 ++
gnu/usr.bin/cc/cc1/Makefile |5 -
gnu/usr.bin/cc/cc1plus/Makefile |5 -
share/mk/bsd.own.mk |3 ++-
tools/build/options/WITH_STATIC_TOOLCHAIN |6 ++
usr.bin/ar/Makefile |4
usr.bin/make/Makefile |4
11 files changed, 38 insertions(+), 3 deletions(-)
diff --git a/gnu/usr.bin/binutils/ar/Makefile b/gnu/usr.bin/binutils/ar/Makefile
index 464445e..e26be85 100644
--- a/gnu/usr.bin/binutils/ar/Makefile
+++ b/gnu/usr.bin/binutils/ar/Makefile
@@ -1,6 +1,7 @@
# $FreeBSD$
.include "../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/binutils ${SRCDIR}/binutils/doc
@@ -16,7 +17,9 @@ CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${.CURDIR}/${RELTOP}/libbinutils
CFLAGS+= -I${SRCDIR}/binutils
CFLAGS+= -I${SRCDIR}/bfd
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?= yes
+.endif
DPADD= ${RELTOP}/libbinutils/libbinutils.a
DPADD+=${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
diff --git a/gnu/usr.bin/binutils/as/Makefile b/gnu/usr.bin/binutils/as/Makefile
index bf8df81..a39918c 100644
--- a/gnu/usr.bin/binutils/as/Makefile
+++ b/gnu/usr.bin/binutils/as/Makefile
@@ -4,6 +4,7 @@
# BINDIR
.include "${.CURDIR}/../../Makefile.inc"
.include "${.CURDIR}/../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/gas ${SRCDIR}/gas/config
@@ -79,7 +80,9 @@ CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${SRCDIR}/gas -I${SRCDIR}/bfd -I${SRCDIR}/gas/config -I${SRCDIR}
CFLAGS+= -I${.CURDIR} -I${.CURDIR}/${TARGET_CPUARCH}-freebsd
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?=yes
+.endif
DPADD= ${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
diff --git a/gnu/usr.bin/binutils/ld/Makefile b/gnu/usr.bin/binutils/ld/Makefile
index d4420ed..50d88b5 100644
--- a/gnu/usr.bin/binutils/ld/Makefile
+++ b/gnu/usr.bin/binutils/ld/Makefile
@@ -1,6 +1,7 @@
# $FreeBSD$
.include "../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/ld
@@ -34,7 +35,9 @@ CFLAGS+= -DBINDIR=\"${BINDIR}\"
-DTARGET_SYSTEM_ROOT=\"${TOOLS_PREFIX}\"
CFLAGS+= -DTOOLBINDIR=\"${TOOLS_PREFIX}/${BINDIR}/libexec\"
CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${SRCDIR}/ld -I${SRCDIR}/bfd
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?= yes
+.endif
DPADD= ${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
LDADD= ${DPADD}
diff --git a/gnu/usr.bin/binutils/ranlib/Makefile
b/gnu/usr.bin/binutils/ranlib/Makefile
index 8679375..6d4b13e 100644
--- a/gnu/usr.bin/binutils/ranlib/Makefile
+++ b/gnu/usr.bin/binutils/ranlib/Makefile
@@ -1,6 +1,7 @@
# $FreeBSD$
.include "../Makefile.inc0"
+.include
.PATH: ${SRCDIR}/binutils ${SRCDIR}/binutils/doc
@@ -16,7 +17,9 @@ CFLAGS+= -D_GNU_SOURCE
CFLAGS+= -I${.CURDIR}/${RELTOP}/libbinutils
CFLAGS+= -I${SRCDIR}/binutils
CFLAGS+= -I${SRCDIR}/bfd
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?= yes
+.endif
DPADD= ${RELTOP}/libbinutils/libbinutils.a
DPADD+=${RELTOP}/libbfd/libbfd.a
DPADD+=${RELTOP}/libiberty/libiberty.a
diff --git a/gnu/usr.bin/cc/cc/Makefile b/gnu/usr.bin/cc/cc/Makefile
index 78c83a5..6a65d69 100644
--- a/gnu/usr.bin/cc/cc/Makefile
+++ b/gnu/usr.bin/cc/cc/Makefile
@@ -9,7 +9,9 @@ PROG= gcc
MAN= gcc.1
SRCS+= gccspec.c
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?=yes
+.endif
MLINKS=gcc.1 g++.1
.if ${MK_CLANG_IS_CC} == "no"
diff --git a/gnu/usr.bin/cc/cc1/Makefile b/gnu/usr.bin/cc/cc1/Makefile
index c65acd2..bb8fe19 100644
--- a/gnu/usr.bin/cc/cc1/Makefile
+++ b/gnu/usr.bin/cc/cc1/Makefile
@@ -1,14 +1,17 @@
# $FreeBSD$
.include "../Makefile.inc"
+.include
.PATH: ${GCCDIR}
PROG= cc1
SRCS= main.c c-parser.c c-lang.c
BINDIR=/usr/libexec
-NO_MAN=
+MAN=
+.if ${MK_SHARED_TOOLCHAIN} != "no"
NO_SHARED?=yes
+.endif
OBJS+= ${PROG}-checksum.o
DPADD= ${LIBBACKEND} ${LIBCPP} ${LIBDECNUMBER} ${LIBIBERTY}
diff --git a/gnu/usr.bin/cc/cc1plus/Makefile b/gnu/usr.bin/cc/cc1plus/Makefile
index 964d20f..0bcbe61 100644
--- a/gnu/usr.bin/cc/cc1plus/Makefile
+++ b/gnu/usr.bin/cc/cc1plu
Re: [RFC] Un-staticise the toolchain
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi, On 26 Apr 2012, at 10:35, Konstantin Belousov wrote: > I think it is time to stop building the toolchain static. I was told that > original reasoning for static linking was the fear of loosing the ability > to recompile if some problem appears with rtld and any required dynamic > library. Apparently, current dependencies are much more spread, e.g. /bin/sh > is dynamically linked [etc] That seems like a bad mistake, because it would prevent even booting single-user if rtld/libraries are broken. - -- Bob Bishop r...@gid.co.uk -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.4 (Darwin) iD8DBQFPmTOivMaT6aS3xb8RAi1NAJ4gyvwPgdqtZjAERJ0grBsZYo5xBQCgikdo P4LXwI1rAbA23+29XWVV8w4= =i8e3 -END PGP SIGNATURE- ___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: [RFC] Un-staticise the toolchain
On 26 Apr 2012, at 12:38, Bob Bishop wrote: > Hi, > > On 26 Apr 2012, at 10:35, Konstantin Belousov wrote: > >> I think it is time to stop building the toolchain static. I was told that >> original reasoning for static linking was the fear of loosing the ability >> to recompile if some problem appears with rtld and any required dynamic >> library. Apparently, current dependencies are much more spread, e.g. /bin/sh >> is dynamically linked [etc] > > That seems like a bad mistake, because it would prevent even booting > single-user if rtld/libraries are broken. That's what /rescue is for. You will find statically linked tools there that are just about to repair a broken system (a static nc would also be nice...). I did some benchmarks a little while ago, and there was, I think, about a 5% slowdown on buildworld with a dynamically linked clang vs a statically linked one on x86-64. Ideally, I'd want the bootstrap compiler to be statically linked but the installed one to be dynamic. The advantage of dynamic linking is small now, but as we add more LLVM and clang-based tools to the base system (e.g. LLVM-based firewall JIT, clang-based indent replacement) we're going to see lots of simple tools being 5-10MB if we enforce static linking. We'll probably get a much bigger speed win from clangd (hopefully appearing in time for LLVM 3.2) avoiding the need to spawn a new process for every compiler invocation than we'll save from static linking. As to compiling things when rtld is broken... clang in the base system currently dynamically links to lib[std]c++, libgcc_s, libm and libc, it only statically links to the clang and LLVM libraries. David___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: [RFC] Un-staticise the toolchain
Den 26/04/2012 kl. 11.35 skrev Konstantin Belousov: > I think it is time to stop building the toolchain static. I was told that > original reasoning for static linking was the fear of loosing the ability > to recompile if some problem appears with rtld and any required dynamic > library. Apparently, current dependencies are much more spread, e.g. /bin/sh > is dynamically linked, and statically linked make does not solve anything. What are the benefits, apart from using a bit less disk space overall? Apparently, toolchain bits aren't considered important enough to be included in /rescue. Maybe they need to be, if the assumption currently is that the compiler will (almost) always work. Erik___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: [RFC] Un-staticise the toolchain
On Thursday, April 26, 2012 7:38:03 am Bob Bishop wrote: > Hi, > > On 26 Apr 2012, at 10:35, Konstantin Belousov wrote: > > > I think it is time to stop building the toolchain static. I was told that > > original reasoning for static linking was the fear of loosing the ability > > to recompile if some problem appears with rtld and any required dynamic > > library. Apparently, current dependencies are much more spread, e.g. /bin/sh > > is dynamically linked [etc] > > That seems like a bad mistake, because it would prevent even booting single- user if rtld/libraries are broken. You could use /rescue/sh as your single-user shell. Of course, that would perhaps let you still be able to recompile things if you had a static toolchain. :) -- John Baldwin ___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: [RFC] Un-staticise the toolchain
On 2012-04-26 13:53, David Chisnall wrote: ... > I did some benchmarks a little while ago, and there was, I think, about a 5% > slowdown on buildworld with a dynamically linked clang vs a statically linked > one on x86-64. Ideally, I'd want the bootstrap compiler to be statically > linked but the installed one to be dynamic. This is already the case, all bootstrap toolchain components are statically linked: $ file /usr/obj/usr/src/tmp/usr/bin/* /usr/obj/usr/src/tmp/usr/bin/CC: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/addr2line: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/as: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/bugpoint:ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/c++: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/c++filt: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/cc: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/clang: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/clang++: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/clang-cpp: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/clang-tblgen:ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/cpp: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/g++: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/gcc: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/gcov:ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/gcpp:ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/gnu-ar: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/gnu-ranlib: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/ld: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/lint:ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/llc: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/lli: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/llvm-ar: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/llvm-as: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/llvm-bcanalyzer: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/llvm-diff: ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/llvm-dis:ELF 32-bit LSB executable, Intel 80386, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900505), not stripped /usr/obj/usr/src/tmp/usr/bin/llv
Re: [RFC] Un-staticise the toolchain
Oops, just replied privately before: On Apr 26, 2012 12:39 PM, "Chris Rees" wrote: > > On Apr 26, 2012 10:36 AM, "Konstantin Belousov" > wrote: > > > > I think it is time to stop building the toolchain static. I was told that > > original reasoning for static linking was the fear of loosing the ability > > to recompile if some problem appears with rtld and any required dynamic > > library. Apparently, current dependencies are much more spread, e.g. > /bin/sh > > is dynamically linked, and statically linked make does not solve > anything. > > > > Patch below makes the dynamically linked toolchain a default, adding an > > WITHOUT_SHARED_TOOLCHAIN build-time option for real conservators. > > Nice idea, but sh etc al are built statically as /rescue/sh. Will we see > /rescue/ar etc? > > Chris > ___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: [RFC] Un-staticise the toolchain
On Thu, Apr 26, 2012 at 07:52:01AM -0400, John Baldwin wrote: > On Thursday, April 26, 2012 7:38:03 am Bob Bishop wrote: > > Hi, > > ... > > You could use /rescue/sh as your single-user shell. Of course, that would > perhaps let you still be able to recompile things if you had a static > toolchain. :) Put a toolchain on a CD or memstick and use that instead? ;-) *runs* - Diane -- - d...@freebsd.org d...@db.net http://www.db.net/~db Why leave money to our children if we don't leave them the Earth? ___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: [RFC] Un-staticise the toolchain
On Thu, Apr 26, 2012 at 12:35:48PM +0300, Konstantin Belousov wrote: > I think it is time to stop building the toolchain static. I was told that > original reasoning for static linking was the fear of loosing the ability > to recompile if some problem appears with rtld and any required dynamic > library. Apparently, current dependencies are much more spread, e.g. /bin/sh > is dynamically linked, and statically linked make does not solve anything. r76801 | sobomax | 2001-05-18 13:05:56 +0400 (Fri, 18 May 2001) | 6 lines By default build make(1) as a static binary. It costs only 100k of additional disk space, buf provides measureable speed increase for make-intensive operations, such as pkg_version(1), `make world' and so on. MFC after: 1 week Have things changed enough that the above is not true anymore? > Patch below makes the dynamically linked toolchain a default, adding an > WITHOUT_SHARED_TOOLCHAIN build-time option for real conservators. > > I did not looked in details why including bsd.own.mk makes NO_MAN > non-functional. Please see the diffs for gnu/usr.bin/cc1*/Makefile. Because you include bsd.own.mk before NO_MAN is defined, and the way how .if works in make(1). ___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
Re: [RFC] Un-staticise the toolchain
On Apr 26, 2012 2:42 PM, "Ruslan Ermilov" wrote: > > On Thu, Apr 26, 2012 at 12:35:48PM +0300, Konstantin Belousov wrote: > > I think it is time to stop building the toolchain static. I was told that > > original reasoning for static linking was the fear of loosing the ability > > to recompile if some problem appears with rtld and any required dynamic > > library. Apparently, current dependencies are much more spread, e.g. /bin/sh > > is dynamically linked, and statically linked make does not solve anything. > > > r76801 | sobomax | 2001-05-18 13:05:56 +0400 (Fri, 18 May 2001) | 6 lines > > By default build make(1) as a static binary. It costs only 100k of additional > disk space, buf provides measureable speed increase for make-intensive > operations, such as pkg_version(1), `make world' and so on. > > MFC after: 1 week > > > > Have things changed enough that the above is not true anymore? Well, the most make(1)-intensive test I can think of is make index, so some results from a quick test: hydra# uname -a FreeBSD hydra.bayofrum.net 9.0-RELEASE FreeBSD 9.0-RELEASE #7: Sun Apr 22 21:02:48 BST 2012 r...@hydra.bayofrum.net:/usr/obj/usr/src/sys/HYDRA amd64 hydra# cd /usr/src/usr.bin/make && make NO_SHARED=yes && cp make ~/bin/make-static >& /dev/null hydra# rm make && make NO_SHARED=no && cp make ~/bin/make-dynamic >& /dev/null hydra# file ~/bin/make-* /home/chris/bin/make-dynamic: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked (uses shared libs), for FreeBSD 9.0 (900044), not stripped /home/chris/bin/make-static: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), statically linked, for FreeBSD 9.0 (900044), not stripped hydra# cd /usr/ports && time make MAKE=~crees/bin/make-static index Generating INDEX-9 - please wait.. Done. 729.770u 120.841s 7:45.10 182.8%920+2676k 5251+116484io 7750pf+0w hydra# time make MAKE=~crees/bin/make-dynamic index Generating INDEX-9 - please wait.. Done. 771.320u 133.540s 8:07.83 185.4%609+2918k 474+116484io 570pf+0w We have a 10% slowdown (or 11% speedup, depending on your figures) when using a dynamically loaded make. Chris ___ freebsd-toolchain@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
