Re: Updating kernel.org cross compilers?
On 05/09/2017 10:59 AM, Andre Przywara wrote: On 30/04/17 06:29, Segher Boessenkool wrote: On Wed, Apr 26, 2017 at 03:14:16PM +0100, Andre Przywara wrote: It seems that many people (even outside the Linux kernel community) use the cross compilers provided at kernel.org/pub/tools/crosstool. The latest compiler I find there is 4.9.0, which celebrated its third birthday at the weekend, also has been superseded by 4.9.4 meanwhile. So I took Segher's buildall scripts from [1] and threw binutils 2.28 and GCC 6.3.0 at them. I am belatedly catching up on this thread. It sounds like the tilegx/tilepro issues were sorted out -- as someone noted, you need to have the kernel headers available to build glibc. However, if there are any outstanding tile issues, please feel free to loop me in! -- Chris Metcalf, Mellanox Technologies http://www.mellanox.com
Re: Updating kernel.org cross compilers?
On Wed, May 10, 2017 at 3:40 PM, Segher Boessenkool wrote: > Hi Arnd, long time no see, > > On Wed, May 10, 2017 at 09:58:13AM +0200, Arnd Bergmann wrote: >> >> So in addition to GCC 7.1 I'd like to have at least GCC 6.3 around, >> >> which builds kernels without warnings today. >> > >> > If you don't want warnings, turn off the warnings or just don't look at >> > them... or fix the problems? Many of the new warnings point out actual >> > problems. >> > >> > Many of those sprintf problems in the kernel have already been fixed. >> >> I've been using gcc-7.0 for a long time and fixed a lot of bugs it found, >> along with more harmless warnings, but I had disabled a couple of >> warning options when I first installed gcc-7 and ended up ignoring >> those. >> >> The exact set of additional options I used is: >> >> -Wimplicit-fallthrough=0 -Wno-duplicate-decl-specifier >> -Wno-int-in-bool-context -Wno-bool-operation -Wno-format-truncation >> -Wno-format-overflow >> >> there were a couple of others that I sent kernel fixes for instead. >> I should probably revisit that list and for each of them either >> only enable it with "make W=1" or fix all known warnings. >> In the long run, I'd actually hope to fix all W=1 warnings too >> and enable them by default. > > Most of those usually point out actual problems (at least code that > isn't as clear as it should be). I do hate that first one though. My point is that we have others in W=1 some of which are equally useful: warning-1 := -Wextra -Wunused -Wno-unused-parameter warning-1 += -Wmissing-declarations warning-1 += -Wmissing-format-attribute warning-1 += $(call cc-option, -Wmissing-prototypes) warning-1 += -Wold-style-definition warning-1 += $(call cc-option, -Wmissing-include-dirs) warning-1 += $(call cc-option, -Wunused-but-set-variable) warning-1 += $(call cc-option, -Wunused-const-variable) warning-1 += $(call cc-disable-warning, missing-field-initializers) warning-1 += $(call cc-disable-warning, sign-compare) I've looked through arm and x86 gcc-7 allmodconfig builds (without my longish fixup series) again and found these added warnings compared to gcc-6.3.1 overall: 2 -Werror=bool-operation 4 -Werror=maybe-uninitialized 1 -Werror=parentheses 2 -Werror=stringop-overflow= 2 -Werror=tautological-compare I probably submitted patches for those in the past, will have another look to see if I need to resubmit them, or if some of them might be regressions. 148 -Werror=duplicate-decl-specifier Only a few files are affected, I can take care of fixing them all: | 1 arch/arm/mach-at91/pm.c | 1 arch/arm/mach-bcm/bcm_kona_smc.c | 1 arch/arm/mach-cns3xxx/core.c | 1 arch/arm/mach-omap2/prm_common.c | 1 arch/arm/mach-omap2/vc.c | 1 arch/arm/mach-spear/time.c | 2 drivers/input/keyboard/cros_ec_keyb.c | 4 sound/soc/codecs/rt5514.c |136 sound/soc/codecs/rt5665.c 89 -Werror=int-in-bool-context This showed up in 26 files in allmodconfig alone. I had started on some of them but given up at some point. I can certainly submit the ones I did already, but probably won't have the patience to address all of them myself. 54 -Werror=format-overflow= Same here. 494 -Werror=format-truncation= These are all over the place, in 187 files. The last one in particular seems less useful than -Wformat-security which we already disable (for all levels), and I'd rather have them both in "make W=1". For -Wint-in-bool-context and -Wformat-overflow= it's less obvious whether we should try to get them all fixed quickly, using >100 patches or put them into W=1 along with -Wformat-truncation. Arnd
Re: Updating kernel.org cross compilers?
Hi Arnd, long time no see, On Wed, May 10, 2017 at 09:58:13AM +0200, Arnd Bergmann wrote: > >> So in addition to GCC 7.1 I'd like to have at least GCC 6.3 around, > >> which builds kernels without warnings today. > > > > If you don't want warnings, turn off the warnings or just don't look at > > them... or fix the problems? Many of the new warnings point out actual > > problems. > > > > Many of those sprintf problems in the kernel have already been fixed. > > I've been using gcc-7.0 for a long time and fixed a lot of bugs it found, > along with more harmless warnings, but I had disabled a couple of > warning options when I first installed gcc-7 and ended up ignoring > those. > > The exact set of additional options I used is: > > -Wimplicit-fallthrough=0 -Wno-duplicate-decl-specifier > -Wno-int-in-bool-context -Wno-bool-operation -Wno-format-truncation > -Wno-format-overflow > > there were a couple of others that I sent kernel fixes for instead. > I should probably revisit that list and for each of them either > only enable it with "make W=1" or fix all known warnings. > In the long run, I'd actually hope to fix all W=1 warnings too > and enable them by default. Most of those usually point out actual problems (at least code that isn't as clear as it should be). I do hate that first one though. Segher
Re: Updating kernel.org cross compilers?
On Wed, May 10, 2017 at 12:18 AM, Segher Boessenkool wrote: > On Tue, May 09, 2017 at 03:59:27PM +0100, Andre Przywara wrote: >> >> I was able to build (bare-metal) toolchains for >> >> all architectures except arc, m68k, tilegx and tilepro. >> > >> > arc needs a more recent GCC; the other probably as well. GCC 7 should >> > be out very soon, you probably want to wait for that :-) >> >> Well, GCC 7 indeed builds better, but then again is a very new compiler. >> For instance in the moment it spits a lot of warnings when compiling the >> kernel (mostly due to some *printf analysis). It's not hard to fix, but >> this will take a while to trickle in and it's questionable whether this >> will be backported everywhere. >> So in addition to GCC 7.1 I'd like to have at least GCC 6.3 around, >> which builds kernels without warnings today. > > If you don't want warnings, turn off the warnings or just don't look at > them... or fix the problems? Many of the new warnings point out actual > problems. > > Many of those sprintf problems in the kernel have already been fixed. I've been using gcc-7.0 for a long time and fixed a lot of bugs it found, along with more harmless warnings, but I had disabled a couple of warning options when I first installed gcc-7 and ended up ignoring those. The exact set of additional options I used is: -Wimplicit-fallthrough=0 -Wno-duplicate-decl-specifier -Wno-int-in-bool-context -Wno-bool-operation -Wno-format-truncation -Wno-format-overflow there were a couple of others that I sent kernel fixes for instead. I should probably revisit that list and for each of them either only enable it with "make W=1" or fix all known warnings. In the long run, I'd actually hope to fix all W=1 warnings too and enable them by default. Arnd
Re: Updating kernel.org cross compilers?
Hello again, On Tue, May 09, 2017 at 03:59:27PM +0100, Andre Przywara wrote: > >> and adding --disable-multilib (for building x86-64 on a x86-64 > >> box without 32-bit libs) > > > > Why is this needed? What error are you seeing. > > It was something along the lines of not finding 32-bit compat libraries > (which I don't have on that build machine): > > configure: error: I suspect your system does not have 32-bit development > libraries (libc and headers). If you have them, rerun configure with > --enable-multilib. If you do not have them, and want to build a > 64-bit-only compiler, rerun configure with --disable-multilib. > > > I don't think we need multilib for a kernel build, also we have an i386 > compiler for 32-bit kernels. So adding this flags allows x86_64 to be > build on pure 64-bit systems. Oh hrm, so it is building a native compiler? I thought I got rid of that everywhere, always build crosses instead. I'll investigate. > >> I was able to build (bare-metal) toolchains for > >> all architectures except arc, m68k, tilegx and tilepro. > > > > arc needs a more recent GCC; the other probably as well. GCC 7 should > > be out very soon, you probably want to wait for that :-) > > Well, GCC 7 indeed builds better, but then again is a very new compiler. > For instance in the moment it spits a lot of warnings when compiling the > kernel (mostly due to some *printf analysis). It's not hard to fix, but > this will take a while to trickle in and it's questionable whether this > will be backported everywhere. > So in addition to GCC 7.1 I'd like to have at least GCC 6.3 around, > which builds kernels without warnings today. If you don't want warnings, turn off the warnings or just don't look at them... or fix the problems? Many of the new warnings point out actual problems. Many of those sprintf problems in the kernel have already been fixed. > For GCC 6.3 (and probably before) arc was breaking because missing a > (default) CPU type. Adding "--with-cpu=arc700" to EXTRA_GCC_CONF fixed > that, but GCC 7 indeed builds fine, even without it. There were other problems, also in binutils. > Also I removed documentation (share/ directory, which won't be in > MANPATH mostly anyway) from the tarball and stripped the (host) binaries > to get the tarball size down (to about 16 MB per arch) I used to make tarballs for everything together, which is about 200MB using lzma (which compresses this *much* better than e.g. bzip2). But that is a while ago, the compiler grows in size really fast. > I built both toolchains and kernels for almost all 31 supported > architectures. Some kernel builds fails (sparc, sparc64, arc), but not > due to toolchain issues, as it seems. Only sh4 complains: > sh4-linux-gcc: error: command line option '-m4-nofpu' is not supported > by this configuration. Everything builds for me. I do have a few local patches, also one for that SuperH problem, yeah. And my kernel trees are a few weeks old, who knows what snuck in :-) Segher
Re: Updating kernel.org cross compilers?
On Tue, May 09, 2017 at 03:59:27PM +0100, Andre Przywara wrote: > On 30/04/17 06:29, Segher Boessenkool wrote: > > Hi, > > > On Wed, Apr 26, 2017 at 03:14:16PM +0100, Andre Przywara wrote: > >> It seems that many people (even outside the Linux kernel community) use > >> the cross compilers provided at kernel.org/pub/tools/crosstool. > >> The latest compiler I find there is 4.9.0, which celebrated its third > >> birthday at the weekend, also has been superseded by 4.9.4 meanwhile. > >> > >> So I took Segher's buildall scripts from [1] and threw binutils 2.28 and > >> GCC 6.3.0 at them. > > > > Happy to see people are still using these! > > > >> After removing --enable-sjlj-exceptions from build-gcc > > > > This was needed to build some targets. It does prevent aarch64 from > > building without patch. > > I saw you fixing that in your repo, thanks for that! > > >> and adding --disable-multilib (for building x86-64 on a x86-64 > >> box without 32-bit libs) > > > > Why is this needed? What error are you seeing. > > It was something along the lines of not finding 32-bit compat libraries > (which I don't have on that build machine): > > configure: error: I suspect your system does not have 32-bit development > libraries (libc and headers). If you have them, rerun configure with > --enable-multilib. If you do not have them, and want to build a > 64-bit-only compiler, rerun configure with --disable-multilib. > > > I don't think we need multilib for a kernel build, also we have an i386 > compiler for 32-bit kernels. So adding this flags allows x86_64 to be > build on pure 64-bit systems. > > >> I was able to build (bare-metal) toolchains for > >> all architectures except arc, m68k, tilegx and tilepro. > > > > arc needs a more recent GCC; the other probably as well. GCC 7 should > > be out very soon, you probably want to wait for that :-) > > Well, GCC 7 indeed builds better, but then again is a very new compiler. > For instance in the moment it spits a lot of warnings when compiling the > kernel (mostly due to some *printf analysis). It's not hard to fix, but > this will take a while to trickle in and it's questionable whether this > will be backported everywhere. > So in addition to GCC 7.1 I'd like to have at least GCC 6.3 around, > which builds kernels without warnings today. > > For GCC 6.3 (and probably before) arc was breaking because missing a > (default) CPU type. Adding "--with-cpu=arc700" to EXTRA_GCC_CONF fixed > that, but GCC 7 indeed builds fine, even without it. > I was wondering if we could have that flags added to the new > TARGET_GCC_CONF variable for arc-elf, to cover pre GCC 7 compilers. > I think arcv2 requires gcc 7. > >> $ ./buildall --toolchain > >> $ PATH=$PATH:/opt/cross/bin > >> $ ./buildall --kernel > > > > You should have the target dir in your PATH before doing anything else. > > Is this not documented? Hrm I guess not, let me fix that. > > > >> And what is a good build setup, so that the binaries run on as many > >> systems as possible? > > > > Run contrib/download_prerequisites in the gcc source dir: this will > > make GMP, MPFR, MPC statically linked, and use a version of each that > > is known to work (and work correctly). > > Ah, that's a good hint! Thanks, that solves a lot of problems. > > Also I removed documentation (share/ directory, which won't be in > MANPATH mostly anyway) from the tarball and stripped the (host) binaries > to get the tarball size down (to about 16 MB per arch) > > I built both toolchains and kernels for almost all 31 supported > architectures. Some kernel builds fails (sparc, sparc64, arc), but not Are those missing libgcc ? If so, I think it is a toolchain issue. I solved it by adding all-target-libgcc / install-target-libgcc to the build (and installing the kernel headers where needed, ie for m68k and tilegx/tilepro). > So while I have now 31 GCC 7.1.0 tarballs, pushing those binaries to > due to toolchain issues, as it seems. Only sh4 complains: > sh4-linux-gcc: error: command line option '-m4-nofpu' is not supported What configuration are you trying to build, and how does your make command line look like (wondering, because I don't see the problem) ? > by this configuration. > some public webspace raises some legal eyebrows here, so I was wondering > if someone (probably with less affiliation to a hardware (arch) vendor) > could build, package and upload them? I am happy to assist with that > process. > How do you create the per-target tarballs ? The script in the repository pushes eherything into a single target directory. Thanks, Guenter
Re: Updating kernel.org cross compilers?
On 30/04/17 06:29, Segher Boessenkool wrote: Hi, > On Wed, Apr 26, 2017 at 03:14:16PM +0100, Andre Przywara wrote: >> It seems that many people (even outside the Linux kernel community) use >> the cross compilers provided at kernel.org/pub/tools/crosstool. >> The latest compiler I find there is 4.9.0, which celebrated its third >> birthday at the weekend, also has been superseded by 4.9.4 meanwhile. >> >> So I took Segher's buildall scripts from [1] and threw binutils 2.28 and >> GCC 6.3.0 at them. > > Happy to see people are still using these! > >> After removing --enable-sjlj-exceptions from build-gcc > > This was needed to build some targets. It does prevent aarch64 from > building without patch. I saw you fixing that in your repo, thanks for that! >> and adding --disable-multilib (for building x86-64 on a x86-64 >> box without 32-bit libs) > > Why is this needed? What error are you seeing. It was something along the lines of not finding 32-bit compat libraries (which I don't have on that build machine): configure: error: I suspect your system does not have 32-bit development libraries (libc and headers). If you have them, rerun configure with --enable-multilib. If you do not have them, and want to build a 64-bit-only compiler, rerun configure with --disable-multilib. I don't think we need multilib for a kernel build, also we have an i386 compiler for 32-bit kernels. So adding this flags allows x86_64 to be build on pure 64-bit systems. >> I was able to build (bare-metal) toolchains for >> all architectures except arc, m68k, tilegx and tilepro. > > arc needs a more recent GCC; the other probably as well. GCC 7 should > be out very soon, you probably want to wait for that :-) Well, GCC 7 indeed builds better, but then again is a very new compiler. For instance in the moment it spits a lot of warnings when compiling the kernel (mostly due to some *printf analysis). It's not hard to fix, but this will take a while to trickle in and it's questionable whether this will be backported everywhere. So in addition to GCC 7.1 I'd like to have at least GCC 6.3 around, which builds kernels without warnings today. For GCC 6.3 (and probably before) arc was breaking because missing a (default) CPU type. Adding "--with-cpu=arc700" to EXTRA_GCC_CONF fixed that, but GCC 7 indeed builds fine, even without it. I was wondering if we could have that flags added to the new TARGET_GCC_CONF variable for arc-elf, to cover pre GCC 7 compilers. >> $ ./buildall --toolchain >> $ PATH=$PATH:/opt/cross/bin >> $ ./buildall --kernel > > You should have the target dir in your PATH before doing anything else. > Is this not documented? Hrm I guess not, let me fix that. > >> And what is a good build setup, so that the binaries run on as many >> systems as possible? > > Run contrib/download_prerequisites in the gcc source dir: this will > make GMP, MPFR, MPC statically linked, and use a version of each that > is known to work (and work correctly). Ah, that's a good hint! Thanks, that solves a lot of problems. Also I removed documentation (share/ directory, which won't be in MANPATH mostly anyway) from the tarball and stripped the (host) binaries to get the tarball size down (to about 16 MB per arch) I built both toolchains and kernels for almost all 31 supported architectures. Some kernel builds fails (sparc, sparc64, arc), but not due to toolchain issues, as it seems. Only sh4 complains: sh4-linux-gcc: error: command line option '-m4-nofpu' is not supported by this configuration. So while I have now 31 GCC 7.1.0 tarballs, pushing those binaries to some public webspace raises some legal eyebrows here, so I was wondering if someone (probably with less affiliation to a hardware (arch) vendor) could build, package and upload them? I am happy to assist with that process. Cheers, Andre. >> [1] http://git.infradead.org/users/segher/buildall.git/ > > I'll push some changes that make aarch64 work out-of-the-box in a > minute. It does some other things as well (e.g., make ia64 work again > after *that* change); more changes are still needed to make sh4 work > without patches. > > > Segher >
Re: Updating kernel.org cross compilers?
Hi, On Wed, Apr 26, 2017 at 03:14:16PM +0100, Andre Przywara wrote: > It seems that many people (even outside the Linux kernel community) use > the cross compilers provided at kernel.org/pub/tools/crosstool. > The latest compiler I find there is 4.9.0, which celebrated its third > birthday at the weekend, also has been superseded by 4.9.4 meanwhile. > > So I took Segher's buildall scripts from [1] and threw binutils 2.28 and > GCC 6.3.0 at them. Happy to see people are still using these! > After removing --enable-sjlj-exceptions from build-gcc This was needed to build some targets. It does prevent aarch64 from building without patch. > and adding --disable-multilib (for building x86-64 on a x86-64 > box without 32-bit libs) Why is this needed? What error are you seeing. > I was able to build (bare-metal) toolchains for > all architectures except arc, m68k, tilegx and tilepro. arc needs a more recent GCC; the other probably as well. GCC 7 should be out very soon, you probably want to wait for that :-) > $ ./buildall --toolchain > $ PATH=$PATH:/opt/cross/bin > $ ./buildall --kernel You should have the target dir in your PATH before doing anything else. Is this not documented? Hrm I guess not, let me fix that. > And what is a good build setup, so that the binaries run on as many > systems as possible? Run contrib/download_prerequisites in the gcc source dir: this will make GMP, MPFR, MPC statically linked, and use a version of each that is known to work (and work correctly). > [1] http://git.infradead.org/users/segher/buildall.git/ I'll push some changes that make aarch64 work out-of-the-box in a minute. It does some other things as well (e.g., make ia64 work again after *that* change); more changes are still needed to make sh4 work without patches. Segher
Re: Updating kernel.org cross compilers?
On 04/26/2017 07:14 AM, Andre Przywara wrote: Hi! (Tony: I've seen you redirecting to "Chris" in an older email, but the web archive doesn't have his email address) It seems that many people (even outside the Linux kernel community) use the cross compilers provided at kernel.org/pub/tools/crosstool. The latest compiler I find there is 4.9.0, which celebrated its third birthday at the weekend, also has been superseded by 4.9.4 meanwhile. So I took Segher's buildall scripts from [1] and threw binutils 2.28 and GCC 6.3.0 at them. After removing --enable-sjlj-exceptions from build-gcc and adding --disable-multilib (for building x86-64 on a x86-64 box without 32-bit libs) I was able to build (bare-metal) toolchains for all architectures except arc, m68k, tilegx and tilepro. Now my understanding is that a baremetal/stage 1 compiler should be build with "all-gcc" instead of the implicit "all" make target, and "install-gcc" instead of "install". And indeed adding this to build-gcc makes all toolchains build now - except arc, which needs an explicit "--with-cpu=arc700" on the GCC configure command line. So after those changes, the sequence: $ ./buildall --toolchain $ PATH=$PATH:/opt/cross/bin $ ./buildall --kernel worked reasonably well for me (binutils 2.28, GCC 6.3.0, Linux 4.11-rc8 on an Intel host with Ubuntu 14.04.5). Just tilegx/pro is still broken due to toolchain issues. Those need libgcc, and to build libgcc you'll have to install the kernel header files. Same is true for cris (and possibly others). Also, crisv32 builds as well. So I wonder what the process is to get those compilers updated? If you find out let me know. Guenter And what is a good build setup, so that the binaries run on as many systems as possible? Also the web page (https://www.kernel.org/pub/tools/crosstool) seems to only link outdated builds (4.6.3, mostly), is that on purpose? Cheers, Andre. [1] http://git.infradead.org/users/segher/buildall.git/
