re: ./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
why can't you just use "nbmake-$foo -k " instead of enabling an option that will lead to odd build issues and support requests. i'd really rather avoid this option in build.sh itself. .mrg.
Re: ./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
On Mon, Jun 25, 2018 at 10:44:36PM +0200, Kamil Rytarowski wrote: > I propose to ensure that MAKEFLAGS is possible to be set from a command > line, from the ./build.sh command line using -V. > > The build process has various stages and after a single failure there > can be false positives in next reports. I wouldn't use it personally. I agree, we should not support this - the build is complex enough. Build failures are rare and I don't see why this option would give any significant benefit. If you are developing something in a subsystem (so build failures would happen more often), you are better off running make manually anyway (and it is easy to do for most parts). Martin
Re: ./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
> The full release build is a number of substeps. It makes sense to > use -k for the main build of the system, after the tools build, to > get the "find all problems" property you want. But it's not clear to > me that it makes sense to build the main system if the tool build > fails. In make(1) terminology, this means that the entire main system build should depend on the tools build. If make(1) is not used for that ordering, whatever tool is used should implement -k semantics itself as aoppropriate, seems to me. > And it's not clear that sets should be produced. Similarly, sets should depend on the main build. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: ./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
On 25.06.2018 22:33, Reinoud Zandijk wrote: > On Mon, Jun 25, 2018 at 10:29:30PM +0200, Kamil Rytarowski wrote: >> I'm usually going the other direction. If I hit a problem and an issue >> is not visible from a multi-job build (multiple processes print onto the >> same screen concurrently), I go for -j1. If this is still no enough I >> change the verbosity level and check commands that caused a problem. >> >> The -k option wouldn't not help me. >> >> On the other hand, can you just specify -V MAKEFLAGS="-k" in your local >> build? >> > > AFAIK, the MAKEFLAGS are cleared on start; I remember me having tried > something like this before. > > Well, it can also help to just do everything with say -k -j10 and inspect all > failures again with the -k -j1 > I propose to ensure that MAKEFLAGS is possible to be set from a command line, from the ./build.sh command line using -V. The build process has various stages and after a single failure there can be false positives in next reports. I wouldn't use it personally. signature.asc Description: OpenPGP digital signature
Re: ./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
On Mon, Jun 25, 2018 at 10:29:30PM +0200, Kamil Rytarowski wrote: > I'm usually going the other direction. If I hit a problem and an issue > is not visible from a multi-job build (multiple processes print onto the > same screen concurrently), I go for -j1. If this is still no enough I > change the verbosity level and check commands that caused a problem. > > The -k option wouldn't not help me. > > On the other hand, can you just specify -V MAKEFLAGS="-k" in your local > build? > AFAIK, the MAKEFLAGS are cleared on start; I remember me having tried something like this before. Well, it can also help to just do everything with say -k -j10 and inspect all failures again with the -k -j1 Reinoud
Re: ./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
[grudgingly keeping tech-kern and current-users both :-]
Reinoud Zandijk writes:
> @@ -1084,6 +1084,9 @@ Usage: ${progname} [-EhnoPRrUuxy] [-a ar
> Should not be used without expert knowledge of the build
> system.
> -h Print this help message.
> -j njobRun up to njob jobs in parallel; see make(1) -j.
> +-k Continue processing after errors are encountered, but
> only on
> + those targets that do not depend on the target whose
> creation
> + caused the error.
> -M obj Set obj root directory to obj; sets MAKEOBJDIRPREFIX.
> Unsets MAKEOBJDIR.
> -m machSet MACHINE to mach. Some mach values are actually
a few unrelated comments:
I think it's pretty clear that if you don't pass -k, nothing will
change. So people that don't like -k or think it's unsafe can just not
pass it.
It would be nice if invoking build.sh with -k resulted in two
properties: if any subpart fails, getting a non-zero exit status from
build.sh, and having that be clear from the end of the log. Currently
'make -k' in pkgsrc results in -k being passed to the WRKSRC make
invocation, which results in a zero status, which results in a
.build_done cookie, which is wrong.
The full release build is a number of substeps. It makes sense to use
-k for the main build of the system, after the tools build, to get the
"find all problems" property you want. But it's not clear to me that it
makes sense to build the main system if the tool build fails. And it's
not clear that sets should be produced. Therefore, I wonder if passing
-k to make should be limited to the main build, and the main build
should be deemed to fail if there are any errors, so that there are no
sets produced in that case.
signature.asc
Description: PGP signature
Re: ./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
On 25.06.2018 20:23, Reinoud Zandijk wrote: > On Sun, Jun 24, 2018 at 10:01:42PM +0200, Reinoud Zandijk wrote: >> On Wed, May 30, 2018 at 07:11:19PM +0800, Harry Pantazis wrote: >>> Continuing.. >>> >>> This first errors are located in >>> src/sys/external/bsd/drm2/dist/drm/i915/intel_ddi.c and are specific to >>> the switch statement concerning that the case flags are not reducing >>> directly to integer constants. >> >> I'd like to request a -k flag to ./build.sh that as with a normal make(1) >> continues to build as much as possible. This will result in reporting all >> errors in one go without needing the 1st to be resolved before the 2nd is >> showing up! > > Attached patch will do, objections against me comitting it? It allows all that > is buildable to be build and the failing files to be compiled later when > patched with the -u option. > > With regards, > Reinoud > I'm usually going the other direction. If I hit a problem and an issue is not visible from a multi-job build (multiple processes print onto the same screen concurrently), I go for -j1. If this is still no enough I change the verbosity level and check commands that caused a problem. The -k option wouldn't not help me. On the other hand, can you just specify -V MAKEFLAGS="-k" in your local build? signature.asc Description: OpenPGP digital signature
Re: ./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
>> Attached patch will do, objections against me comitting it? It >> allows all that is buildable to be build and the failing files to be >> compiled later when patched with the -u option. > I don't really like this. I would not be surprised if this broke > things in interesting ways for full world builds if you run into a > problem, cvs up and try an update build again. I submit that, if that is the case, the build is already broken and you just haven't yet tripped over a case that makes it unmistakeable. /~\ The ASCII Mouse \ / Ribbon Campaign X Against HTMLmo...@rodents-montreal.org / \ Email! 7D C8 61 52 5D E7 2D 39 4E F1 31 3E E8 B3 27 4B
Re: ./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
On Mon, Jun 25, 2018 at 08:23:22PM +0200, Reinoud Zandijk wrote: > On Sun, Jun 24, 2018 at 10:01:42PM +0200, Reinoud Zandijk wrote: > > On Wed, May 30, 2018 at 07:11:19PM +0800, Harry Pantazis wrote: > > > Continuing.. > > > > > > This first errors are located in > > > src/sys/external/bsd/drm2/dist/drm/i915/intel_ddi.c and are specific to > > > the switch statement concerning that the case flags are not reducing > > > directly to integer constants. > > > > I'd like to request a -k flag to ./build.sh that as with a normal make(1) > > continues to build as much as possible. This will result in reporting all > > errors in one go without needing the 1st to be resolved before the 2nd is > > showing up! > > Attached patch will do, objections against me comitting it? It allows all that > is buildable to be build and the failing files to be compiled later when > patched with the -u option. I don't really like this. I would not be surprised if this broke things in interesting ways for full world builds if you run into a problem, cvs up and try an update build again. Joerg
Re: ./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
the patch:
Index: build.sh
===
RCS file: /cvsroot/src/build.sh,v
retrieving revision 1.327
diff -u -p -r1.327 build.sh
--- build.sh2 May 2018 07:34:44 - 1.327
+++ build.sh25 Jun 2018 18:26:34 -
@@ -1027,7 +1027,7 @@ usage()
cat <<_usage_
Usage: ${progname} [-EhnoPRrUuxy] [-a arch] [-B buildid] [-C cdextras]
-[-D dest] [-j njob] [-M obj] [-m mach] [-N noisy]
+[-D dest] [-j njob] [-k] [-M obj] [-m mach] [-N noisy]
[-O obj] [-R release] [-S seed] [-T tools]
[-V var=[value]] [-w wrapper] [-X x11src] [-Y extsrcsrc]
[-Z var]
@@ -1084,6 +1084,9 @@ Usage: ${progname} [-EhnoPRrUuxy] [-a ar
Should not be used without expert knowledge of the build
system.
-h Print this help message.
-j njobRun up to njob jobs in parallel; see make(1) -j.
+-k Continue processing after errors are encountered, but only
on
+ those targets that do not depend on the target whose
creation
+ caused the error.
-M obj Set obj root directory to obj; sets MAKEOBJDIRPREFIX.
Unsets MAKEOBJDIR.
-m machSet MACHINE to mach. Some mach values are actually
@@ -1128,7 +1131,7 @@ _usage_
parseoptions()
{
- opts='a:B:C:D:Ehj:M:m:N:nO:oPR:rS:T:UuV:w:X:xY:yZ:'
+ opts='a:B:C:D:Ehj:kM:m:N:nO:oPR:rS:T:UuV:w:X:xY:yZ:'
opt_a=false
opt_m=false
@@ -1188,6 +1191,10 @@ parseoptions()
parallel="-j ${OPTARG}"
;;
+ -k)
+ MAKEFLAGS="-k ${MAKEFLAGS}"
+ ;;
+
-M)
eval ${optargcmd}; resolvepath OPTARG
case "${OPTARG}" in
signature.asc
Description: PGP signature
Re: ./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
On Sun, Jun 24, 2018 at 10:01:42PM +0200, Reinoud Zandijk wrote: > On Wed, May 30, 2018 at 07:11:19PM +0800, Harry Pantazis wrote: > > Continuing.. > > > > This first errors are located in > > src/sys/external/bsd/drm2/dist/drm/i915/intel_ddi.c and are specific to > > the switch statement concerning that the case flags are not reducing > > directly to integer constants. > > I'd like to request a -k flag to ./build.sh that as with a normal make(1) > continues to build as much as possible. This will result in reporting all > errors in one go without needing the 1st to be resolved before the 2nd is > showing up! Attached patch will do, objections against me comitting it? It allows all that is buildable to be build and the failing files to be compiled later when patched with the -u option. With regards, Reinoud signature.asc Description: PGP signature
./build.sh -k feature request (was Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled)
Hi folks, On Wed, May 30, 2018 at 07:11:19PM +0800, Harry Pantazis wrote: > Continuing.. > > This first errors are located in > src/sys/external/bsd/drm2/dist/drm/i915/intel_ddi.c and are specific > to the switch statement concerning that the case flags are not > reducing directly to integer constants. I'd like to request a -k flag to ./build.sh that as with a normal make(1) continues to build as much as possible. This will result in reporting all errors in one go without needing the 1st to be resolved before the 2nd is showing up! With regards, Reinoud
Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled
In article <13557.1527729...@jinx.noi.kre.to>, Robert Elz wrote: >Date:Wed, 30 May 2018 11:39:50 + (UTC) >From:chris...@astron.com (Christos Zoulas) >Message-ID: > > | Looks like a bug to me; it overflows 32 bits. Try changing them from > | >https://nxr.netbsd.org/xref/src/sys/external/bsd/drm2/dist/drm/i915/i915_reg.h#5429 > | > | (3 << 29) -> (3ULL << 29) > >The LL would be wrong, it looks to be a 32 bit (4 byte) register. Just the >'U' is needed. > Indeed, only U is needed. christos
Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled
Date:Wed, 30 May 2018 11:39:50 + (UTC) From:chris...@astron.com (Christos Zoulas) Message-ID: | Looks like a bug to me; it overflows 32 bits. Try changing them from | https://nxr.netbsd.org/xref/src/sys/external/bsd/drm2/dist/drm/i915/i915_reg.h#5429 | | (3 << 29) -> (3ULL << 29) The LL would be wrong, it looks to be a 32 bit (4 byte) register. Just the 'U' is needed. kre
Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled
Date:Wed, 30 May 2018 19:11:19 +0800 From:Harry Pantazis Message-ID: | Not a big deal, I'm writing a patch to overcome this issue and | continuing the error-checking. | All are encouraged to submit patches for a fix. The constants are overflowing int (into the sign bit) - they just need to be made unsigned 4U << 29 (etc). kre
Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled
In article , Harry Pantazis wrote: >-=-=-=-=-=- > >Sorry, I faultily sent before completing the mail. > >Continuing.. > >This first errors are located in >src/sys/external/bsd/drm2/dist/drm/i915/intel_ddi.c and are specific >to the switch statement concerning that the case flags are not >reducing directly to integer constants. > >Not a big deal, I'm writing a patch to overcome this issue and >continuing the error-checking. >All are encouraged to submit patches for a fix. > >I've included just the necessary portion of the build output I >received cause the build file is too big. >Looking forward in finding bigger issues. Looks like a bug to me; it overflows 32 bits. Try changing them from https://nxr.netbsd.org/xref/src/sys/external/bsd/drm2/dist/drm/i915/i915_reg.h#5429 (3 << 29) -> (3ULL << 29) christos
Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled
Sorry, I faultily sent before completing the mail. Continuing.. This first errors are located in src/sys/external/bsd/drm2/dist/drm/i915/intel_ddi.c and are specific to the switch statement concerning that the case flags are not reducing directly to integer constants. Not a big deal, I'm writing a patch to overcome this issue and continuing the error-checking. All are encouraged to submit patches for a fix. I've included just the necessary portion of the build output I received cause the build file is too big. Looking forward in finding bigger issues. Regards, Harry /root/lx0_fork/obj/tooldir.NetBSD-8.99.14-amd64/bin/x86_64--netbsd-gcc -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -mno-avx -msoft-float -mindirect-branch=thunk-inline -mindirect-branch-register -ffreestanding -fno-zero-initialized-in-bss -fno-delete-null-pointer-checks -O2 -fno-omit-frame-pointer -fsanitize=undefined -fstack-protector -Wstack-protector --param ssp-buffer-size=1 -fno-strict-aliasing -fno-common -std=gnu99 -Werror -Wall -Wno-main -Wno-format-zero-length -Wpointer-arith -Wmissing-prototypes -Wstrict-prototypes -Wold-style-definition -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wno-unreachable-code -Wno-pointer-sign -Wno-attributes -Wextra -Wno-unused-parameter -Wold-style-definition -Wno-sign-compare --sysroot=/root/lx0_fork/obj/destdir.amd64 -Damd64 -Dx86_64 -I. -I/root/lx0_fork/sys/external/bsd/acpica/dist -I/root/lx0_fork/sys/../common/lib/libx86emu -I/root/lx0_fork/sys/../common/include -I/root/lx0_fork/sys/arch -I/root/lx0_fork/sys -nostdinc -DDIAGNOSTIC -DCOMPAT_44 -D_KERNEL -D_KERNEL_OPT -std=gnu99 -I/root/lx0_fork/sys/lib/libkern/../../../common/lib/libc/quad -I/root/lx0_fork/sys/lib/libkern/../../../common/lib/libc/string -I/root/lx0_fork/sys/lib/libkern/../../../common/lib/libc/arch/x86_64/string -D_FORTIFY_SOURCE=2 -I/root/lx0_fork/sys/external/bsd/ipf -I/root/lx0_fork/sys/external/isc/atheros_hal/dist -I/root/lx0_fork/sys/external/isc/atheros_hal/ic -I/root/lx0_fork/sys/external/bsd/common/include -I/root/lx0_fork/sys/external/bsd/drm2/include -I/root/lx0_fork/sys/external/bsd/common/include -I/root/lx0_fork/sys/external/bsd/drm2/include -I/root/lx0_fork/sys/external/bsd/drm2/include/drm -I/root/lx0_fork/sys/external/bsd/drm2/dist -I/root/lx0_fork/sys/external/bsd/drm2/dist/include -I/root/lx0_fork/sys/external/bsd/drm2/dist/include/drm -I/root/lx0_fork/sys/external/bsd/drm2/dist/uapi -I/root/lx0_fork/sys/external/bsd/common/include -D__KERNEL__ -DCONFIG_FB=0 -DCONFIG_BACKLIGHT_CLASS_DEVICE=0 -DCONFIG_BACKLIGHT_CLASS_DEVICE_MODULE=0 -I/root/lx0_fork/sys/../common/include -DCONFIG_AGP -I/root/lx0_fork/sys/external/bsd/drm2/dist/drm/i915 -I/root/lx0_fork/sys/external/bsd/drm2/i915drm -DCONFIG_DRM_I915_FBDEV=1 -DCONFIG_DRM_I915_PRELIMINARY_HW_SUPPORT=0 -I/root/lx0_fork/sys/external/bsd/drm2/dist/drm/radeon -I/root/lx0_fork/sys/external/bsd/drm2/include/radeon -I/root/lx0_fork/sys/external/bsd/drm2/radeon -I/root/lx0_fork/sys/external/bsd/drm2/dist/drm/nouveau -I/root/lx0_fork/sys/external/bsd/drm2/dist/drm/nouveau/core -I/root/lx0_fork/sys/external/bsd/drm2/dist/drm/nouveau/core/include -I/root/lx0_fork/sys/external/bsd/drm2/nouveau -DCONFIG_NOUVEAU_DEBUG=5 -DCONFIG_NOUVEAU_DEBUG_DEFAULT=3 -I/root/lx0_fork/sys/external/bsd/acpica/dist/include -c /root/lx0_fork/sys/netinet/in_proto.c -o in_proto.o --- in6_pcb.o --- /root/lx0_fork/obj/tooldir.NetBSD-8.99.14-amd64/bin/nbctfconvert -g -L VERSION in6_pcb.o --- in_proto.o --- /root/lx0_fork/obj/tooldir.NetBSD-8.99.14-amd64/bin/nbctfconvert -g -L VERSION in_proto.o --- init_main.o --- --- init_sysctl.o --- --- init_main.o --- # compile UBSAN_ENABLED/init_main.o --- init_sysctl.o --- # compile UBSAN_ENABLED/init_sysctl.o --- init_main.o --- /root/lx0_fork/obj/tooldir.NetBSD-8.99.14-amd64/bin/x86_64--netbsd-gcc -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -mno-avx -msoft-float -mindirect-branch=thunk-inline -mindirect-branch-register -ffreestanding -fno-zero-initialized-in-bss -fno-delete-null-pointer-checks -O2 -fno-omit-frame-pointer -fsanitize=undefined -fstack-protector -Wstack-protector --param ssp-buffer-size=1 -fno-strict-aliasing -fno-common -std=gnu99 -Werror -Wall -Wno-main -Wno-format-zero-length -Wpointer-arith -Wmissing-prototypes -Wstrict-prototypes -Wold-style-definition -Wswitch -Wshadow -Wcast-qual -Wwrite-strings -Wno-unreachable-code -Wno-pointer-sign -Wno-attributes -Wextra -Wno-unused-parameter -Wold-style-definition -Wno-sign-compare --sysroot=/root/lx0_fork/obj/destdir.amd64 -Damd64 -Dx86_64 -I. -I/root/lx0_fork/sys/external/bsd/acpica/dist -I/root/lx0_fork/sys/../common/lib/libx86emu -I/root/lx0_fork/sys/../common/include -I/root/lx0_fork/sys/arch -I/root/lx0_fork/sys -nostdinc -DDIAGNOSTIC -DCOMPAT_44 -D_KERNEL -D_KERN--- init_sysctl.o --- /root/lx0_fork/obj/tooldir.NetBSD-8.99.14-amd64/bin/x86_64--netbsd-gcc -mcmodel=kernel -mno-red-zone -mno-mmx -mno-sse -mno-avx -msoft-
Re: GENERIC Kernel Build errors with -fsanitize=undefined option enabled
On 30.05.2018 12:51, Harry Pantazis wrote: > Greetings, > > I'm working on the task of utilizing Undefined Behavior checks for the > NetBSD/amd64 kernel (KUBSan). I have built the GENERIC kernel with the > -fsanitize=undefined option enabled and I've caught a first series of > errors caused indirectly by Undefined Behavior. > > This first errors are located in > src/sys/external/bsd/drm2/dist/drm/i915/intel_ddi.c and are specific > to the switch statement concerning that the case FLA > Please show the the exact build failure log to make it clearer what's wrong. signature.asc Description: OpenPGP digital signature
