04.07.2025 02:25, Theo de Raadt пишет:
> I agree, let's do it.
>
> Klemens Nanni <[email protected]> wrote:
>
>> 22.05.2025 06:07, Klemens Nanni пишет:
>>> 15.05.2025 13:05, Mark Kettenis пишет:
>>>>> Date: Thu, 15 May 2025 11:22:17 +0200
>>>>> From: Claudio Jeker <[email protected]>
>>>>>
>>>>> On Thu, May 15, 2025 at 06:28:42PM +1000, Jonathan Matthew wrote:
>>>>>> On Tue, May 13, 2025 at 07:55:03AM +0200, Otto Moerbeek wrote:
>>>>>>> On Mon, May 12, 2025 at 08:07:11PM +0200, Sebastien Marie wrote:
>>>>>>>
>>>>>>>>> I suppose the same could occurs with lld (untested for now).
>>>>>>>>
>>>>>>>> I confirm it is the same problem with lld.
>>>>>>>>
>>>>>>>> $ cd /usr/share/relink/kernel/GENERIC.MP
>>>>>>>> $ ld -T ld.script -X --warn-common -nopie -o /tmp/newbsd *.o && echo ok
>>>>>>>>
>>>>>>>> /tmp: write failed, file system is full
>>>>>>>> ok
>>>>>>>> $ ls -l /tmp/newbsd
>>>>>>>> -rwxr-xr-x 1 semarie wheel 236434608 May 12 20:05 /tmp/newbsd*
>>>>>>>>
>>>>>>>> And my dmesg has the following:
>>>>>>>> uvn_flush: obj=0xfffffd86e3311608, offset=0x16b0000. error during
>>>>>>>> pageout.
>>>>>>>> uvn_flush: WARNING: changes to page may be lost!
>>>>>>>
>>>>>>> So this code is using mmapped files for writing, which makes proper
>>>>>>> error handling extremely difficult or even impossible. Best bet is
>>>>>>> making sure enough space is available before starting.
>>>>>>
>>>>>> lld has a --no-mmap-output-file option that causes it to use plain
>>>>>> write(2)
>>>>>> calls to generate the output file. Perhaps it'd be worth using that for
>>>>>> kernel linking and other stuff we relink at boot time?
>>>>>
>>>>> Maybe that should be the default. Having lld produce bad binaries but exit
>>>>> 0 is just very wrong. Not sure if this a problem that only manifests on
>>>>> OpenBSD since there is no unified buffer cache or if other systems would
>>>>> hit the same issue as well. As Otto mentioned detection IO errors when
>>>>> using mmap to write files is not trivial.
>>>>
>>>> I think the same problem exists on other OSes, even those with a
>>>> unified buffer cache. I suppose lld(1) could use msync(2) (with the
>>>> MS_SYNC) flag to make sure everything lands on disk correctly. But
>>>> that obviously would remove some of the benefits of using mmap(),
>>>> namely the async completion of the writes.
>>>>
>>>> It would be intersting to see what the impact on build times of
>>>> changing the defaults would be. But I'm somewhat hesitant to change
>>>> the default since the --no-mmap-output-file code path isn't tested
>>>> much on other OSes.
>>>
>>> Shall we first try it for kernels only?
>>>
>>> # make && make install && reboot
>>> ld -T ld.script -X --warn-common -nopie --no-mmap-output-file -o bsd
>>> ${SYSTEM_HEAD} vers.o ${OBJS}
>>> text data bss dec hex
>>> 26918218 492744 1351680 28762642 1b6e212
>>> mv bsd bsd.gdb
>>> ctfstrip -S -o bsd bsd.gdb
>>>
>>> $ cat /usr/share/relink/kernel/GENERIC.MP/relink.log
>>> (SHA256) /bsd: OK
>>> LD="ld" sh makegap.sh 0xcccccccc gapdummy.o
>>> ld -T ld.script -X --warn-common -nopie --no-mmap-output-file -o newbsd
>>> ${SYSTEM_HEAD} vers.o ${OBJS}
>>> text data bss dec hex
>>> 26908374 489800 1347584 28745758 1b6a01e
>>> mv newbsd newbsd.gdb
>>> [...]
>>>
>>> This also effects installs and upgrades.
>>
>> Any thoughts on doing this for all architectures using clang?
>> Looks like that'd be <bsd.own.mk> CLANG_ARCH modulo sparc64.
It should match 'LLD_ARCH=aarch64 amd64 arm i386 powerpc powerpc64 riscv64',
octeon is in CLANG_ARCH, but uses GNU ld 2.17 which doesn't know this flag:
ld -e start -T ld.script -Ttext=0xffffffff82000000 --warn-common -nopie
--no-mmap-output-file -o bsd ${SYSTEM_HEAD} vers.o ${OBJS}
ld: unrecognized option '--no-mmap-output-file'
ld: use the --help option for usage information
*** Error 1 in /usr/src/sys/arch/octeon/compile/BOOT (Makefile:305 'bsd':
@echo ld -e start -T ld.script -Ttext=0xffffffff82000000 --warn-c...)
New mechanical diff, I tested only amd64:
M sys/arch/amd64/conf/Makefile.amd64
M sys/arch/arm64/conf/Makefile.arm64
M sys/arch/armv7/conf/Makefile.armv7
M sys/arch/i386/conf/Makefile.i386
M sys/arch/macppc/conf/Makefile.macppc
M sys/arch/powerpc64/conf/Makefile.powerpc64
M sys/arch/riscv64/conf/Makefile.riscv64
Index: amd64/conf/Makefile.amd64
===================================================================
RCS file: /cvs/src/sys/arch/amd64/conf/Makefile.amd64,v
diff -u -p -r1.143 Makefile.amd64
--- amd64/conf/Makefile.amd64 3 Jun 2025 10:44:52 -0000 1.143
+++ amd64/conf/Makefile.amd64 4 Jun 2025 20:05:21 -0000
@@ -111,6 +111,8 @@ COPTIMIZE?= -O2
CFLAGS= ${DEBUG} ${CWARNFLAGS} ${CMACHFLAGS} ${COPTIMIZE}
${COPTS} ${PIPE}
AFLAGS= -D_LOCORE -x assembler-with-cpp ${CWARNFLAGS}
${CMACHFLAGS}
LINKFLAGS= -T ld.script -X --warn-common -nopie
+# propagate failure like ENOSPC on relink
+LINKFLAGS+= --no-mmap-output-file
HOSTCC?= ${CC}
HOSTED_CPPFLAGS=${CPPFLAGS:S/^-nostdinc$//}
Index: arm64/conf/Makefile.arm64
===================================================================
RCS file: /cvs/src/sys/arch/arm64/conf/Makefile.arm64,v
diff -u -p -r1.54 Makefile.arm64
--- arm64/conf/Makefile.arm64 5 May 2025 20:43:32 -0000 1.54
+++ arm64/conf/Makefile.arm64 3 Jul 2025 23:05:55 -0000
@@ -103,6 +103,8 @@ COPTIMIZE?= -O2
CFLAGS= ${DEBUG} ${CWARNFLAGS} ${CMACHFLAGS} ${COPTIMIZE}
${COPTS} ${PIPE}
AFLAGS= -D_LOCORE -x assembler-with-cpp ${CWARNFLAGS}
${CMACHFLAGS}
LINKFLAGS= -T ld.script -X --warn-common -nopie
+# propagate failure like ENOSPC on relink
+LINKFLAGS+= --no-mmap-output-file
HOSTCC?= ${CC}
HOSTED_CPPFLAGS=${CPPFLAGS:S/^-nostdinc$//}
Index: armv7/conf/Makefile.armv7
===================================================================
RCS file: /cvs/src/sys/arch/armv7/conf/Makefile.armv7,v
diff -u -p -r1.59 Makefile.armv7
--- armv7/conf/Makefile.armv7 5 May 2025 20:43:32 -0000 1.59
+++ armv7/conf/Makefile.armv7 3 Jul 2025 23:26:35 -0000
@@ -45,6 +45,8 @@ COPTIMIZE?= -O2
CFLAGS= ${DEBUG} ${CWARNFLAGS} ${CMACHFLAGS} ${COPTIMIZE}
${COPTS} ${PIPE}
AFLAGS= -D_LOCORE -x assembler-with-cpp ${CWARNFLAGS}
${CMACHFLAGS}
LINKFLAGS= -T ld.script --warn-common -nopie
+# propagate failure like ENOSPC on relink
+LINKFLAGS+= --no-mmap-output-file
HOSTCC?= ${CC}
HOSTED_CPPFLAGS=${CPPFLAGS:S/^-nostdinc$//}
Index: i386/conf/Makefile.i386
===================================================================
RCS file: /cvs/src/sys/arch/i386/conf/Makefile.i386,v
diff -u -p -r1.149 Makefile.i386
--- i386/conf/Makefile.i386 5 May 2025 20:43:32 -0000 1.149
+++ i386/conf/Makefile.i386 3 Jul 2025 23:10:46 -0000
@@ -53,6 +53,8 @@ COPTIMIZE?= -O2
CFLAGS= ${DEBUG} ${CWARNFLAGS} ${CMACHFLAGS} ${COPTIMIZE}
${COPTS} ${PIPE}
AFLAGS= -D_LOCORE -x assembler-with-cpp ${CWARNFLAGS}
${CMACHFLAGS}
LINKFLAGS= -T ld.script -X --warn-common -nopie
+# propagate failure like ENOSPC on relink
+LINKFLAGS+= --no-mmap-output-file
.if ${MACHINE} == "amd64"
CFLAGS+= -m32
Index: macppc/conf/Makefile.macppc
===================================================================
RCS file: /cvs/src/sys/arch/macppc/conf/Makefile.macppc,v
diff -u -p -r1.111 Makefile.macppc
--- macppc/conf/Makefile.macppc 5 May 2025 20:43:32 -0000 1.111
+++ macppc/conf/Makefile.macppc 3 Jul 2025 23:31:27 -0000
@@ -53,6 +53,8 @@ COPTIMIZE?= -O2
CFLAGS= ${DEBUG} ${CWARNFLAGS} ${CMACHFLAGS} ${COPTIMIZE}
${COPTS} ${PIPE}
AFLAGS= -D_LOCORE ${CMACHFLAGS}
LINKFLAGS= -N -T ld.script --warn-common -nopie
+# propagate failure like ENOSPC on relink
+LINKFLAGS+= --no-mmap-output-file
.if ${MACHINE} == "powerpc64"
CFLAGS+= -m32
Index: powerpc64/conf/Makefile.powerpc64
===================================================================
RCS file: /cvs/src/sys/arch/powerpc64/conf/Makefile.powerpc64,v
diff -u -p -r1.22 Makefile.powerpc64
--- powerpc64/conf/Makefile.powerpc64 5 May 2025 20:43:32 -0000 1.22
+++ powerpc64/conf/Makefile.powerpc64 3 Jul 2025 23:12:32 -0000
@@ -97,6 +97,8 @@ COPTIMIZE?= -O2
CFLAGS= ${DEBUG} ${CWARNFLAGS} ${CMACHFLAGS} ${COPTIMIZE}
${COPTS} ${PIPE}
AFLAGS= -D_LOCORE -x assembler-with-cpp ${CWARNFLAGS}
${CMACHFLAGS}
LINKFLAGS= -T ld.script -X --warn-common -pie
+# propagate failure like ENOSPC on relink
+LINKFLAGS+= --no-mmap-output-file
HOSTCC?= ${CC}
HOSTED_CPPFLAGS=${CPPFLAGS:S/^-nostdinc$//}
Index: riscv64/conf/Makefile.riscv64
===================================================================
RCS file: /cvs/src/sys/arch/riscv64/conf/Makefile.riscv64,v
diff -u -p -r1.23 Makefile.riscv64
--- riscv64/conf/Makefile.riscv64 5 May 2025 20:43:32 -0000 1.23
+++ riscv64/conf/Makefile.riscv64 3 Jul 2025 23:32:18 -0000
@@ -48,6 +48,8 @@ COPTIMIZE?= -O2
CFLAGS= ${DEBUG} ${CWARNFLAGS} ${CMACHFLAGS} ${COPTIMIZE}
${COPTS} ${PIPE}
AFLAGS= -D_LOCORE -x assembler-with-cpp ${CWARNFLAGS}
${CMACHFLAGS}
LINKFLAGS= -T ld.script -X --warn-common -nopie
+# propagate failure like ENOSPC on relink
+LINKFLAGS+= --no-mmap-output-file
HOSTCC?= ${CC}
HOSTED_CPPFLAGS=${CPPFLAGS:S/^-nostdinc$//}
