[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 Kubilay Kocak changed: What|Removed |Added Flags||mfc-stable12?, ||mfc-stable11- -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 --- Comment #15 from Bjoern A. Zeeb --- (In reply to Ed Maste from comment #14) In HEAD it is. I think it was never MFCed due to my personal events last year after returning from Canada. Sigh. Should I try to get this into 12.2-RC1? -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 --- Comment #14 from Ed Maste --- Is this resolved now? -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 --- Comment #13 from Bjoern A. Zeeb --- *** Bug 238012 has been marked as a duplicate of this bug. *** -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 --- Comment #12 from commit-h...@freebsd.org --- A commit references this bug: Author: bz Date: Sat Jun 8 17:44:43 UTC 2019 New revision: 348808 URL: https://svnweb.freebsd.org/changeset/base/348808 Log: Fix dpcpu and vnet panics with complex types at the end of the section. Apply a linker script when linking i386 kernel modules to apply padding to a set_pcpu or set_vnet section. The padding value is kind-of random and is used to catch modules not compiled with the linker-script, so possibly still having problems leading to kernel panics. This is needed as the code generated on certain architectures for non-simple-types, e.g., an array can generate an absolute relocation on the edge (just outside) the section and thus will not be properly relocated. Adding the padding to the end of the section will ensure that even absolute relocations of complex types will be inside the section, if they are the last object in there and hence relocation will work properly and avoid panics such as observed with carp.ko or ipsec.ko. There is a rather lengthy discussion of various options to apply in the mentioned PRs and their depends/blocks, and the review. There seems no best solution working across multiple toolchains and multiple version of them, so I took the liberty of taking one, as currently our users (and our CI system) are hitting this on just i386 and we need some solution. I wish we would have a proper fix rather than another "hack". Also backout r340009 which manually, temporarily fixed CARP before 12.0-R "by chance" after a lead-up of various other link-elf.c and related fixes. PR: 230857,238012 With suggestions from:arichardson (originally last year) Tested by:lwhsu Event:Waterloo Hackathon 2019 Reported by: lwhsu, olivier MFC after:6 weeks Differential Revision:https://reviews.freebsd.org/D17512 Changes: head/UPDATING head/sys/conf/kmod.mk head/sys/conf/ldscript.set_padding head/sys/kern/link_elf.c head/sys/netinet/ip_carp.c head/sys/sys/param.h -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 Li-Wen Hsu changed: What|Removed |Added URL||https://reviews.freebsd.org ||/D17512 -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 Kubilay Kocak changed: What|Removed |Added See Also||https://bugs.freebsd.org/bu ||gzilla/show_bug.cgi?id=2380 ||12 -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 Bug 230857 depends on bug 232289, which changed state. Bug 232289 Summary: kern/link_elf.c fails for small sections sizes (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232289 What|Removed |Added Status|In Progress |Closed Resolution|--- |FIXED -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 --- Comment #10 from Bjoern A. Zeeb --- As Alex R points out if we run into symbol reordering problems with the extra variable fix committed, we could add an extra .o file to the end of the list which (unless there is a linker-script doing magic) should always stay at the end of the list. We could add that to just problematic modules or to all modules (and then could still ignore the extra bytes). We'd use a static __used variable to not conflict with duplicate symbols or if that does not work, asm. I am just adding this to the PR as to write own more possible ways to fix this. He also mentions -fPIC would probably solve the initial problem (but that's a totally different can of worms on our i386). -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 --- Comment #9 from commit-h...@freebsd.org --- A commit references this bug: Author: bz Date: Thu Nov 1 17:26:18 UTC 2018 New revision: 340009 URL: https://svnweb.freebsd.org/changeset/base/340009 Log: carpstats are the last virtualised variable in the file and end up at the end of the vnet_set. The generated code uses an absolute relocation at one byte beyond the end of the carpstats array. This means the relocation for the vnet does not happen for carpstats initialisation and as a result the kernel panics on module load. This problem has only been observed with carp and only on i386. We considered various possible solutions including using linker scripts to add padding to all kernel modules for pcpu and vnet sections. While the symbols (by chance) stay in the order of appearance in the file adding an unused non-file-local variable at the end of the file will extend the size of set_vnet and hence make the absolute relocation for carpstats work (think of this as a single-module set_vnet padding). This is a (tmporary) hack. It is the least intrusive one as we need a timely solution for the upcoming release. We will revisit the problem in HEAD. For a lot more information and the possible alternate solutions please see the PR and the references therein. PR: 230857 MFC after:3 days Changes: head/sys/netinet/ip_carp.c -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 Bjoern A. Zeeb changed: What|Removed |Added Depends on||232291, 232289 --- Comment #8 from Bjoern A. Zeeb --- Track the dependency problems; need to solve at least the link_elf one before we can do the BYTE(1) linker script and follow-up link_elf checks. Referenced Bugs: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232289 [Bug 232289] kern/link_elf.c fails for small sections sizes (https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=232291 [Bug 232291] ld.bfd (newer) and ld.lld (6 and imho 7) create empty sections when they should not -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 --- Comment #7 from Bjoern A. Zeeb --- https://reviews.freebsd.org/D17512 -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 --- Comment #6 from Bjoern A. Zeeb --- Created attachment 198004 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=198004=edit kmod.mk adjustments along with two new linker scripts -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857
Bjoern A. Zeeb changed:
What|Removed |Added
CC||d...@freebsd.org
Keywords||toolchain
--- Comment #5 from Bjoern A. Zeeb ---
I'll start describing the problem from a reduced piece of code, which is not as
big as carp, replicating carpstats, assuming VIMAGE is on:
#include
#include
#include
#include
#include
#include
#include
struct xstats {
uint64_tfoo1;
uint64_tbar1;
uint64_tbaz1;
uint64_tmad1;
uint64_tfoo2;
uint64_tbar2;
uint64_tbaz2;
uint64_tmad2;
uint64_tfoo3;
uint64_tbar3;
uint64_tbaz3;
uint64_tmad3;
uint64_tfoo4;
uint64_tbar4;
uint64_tbaz4;
uint64_tmad4;
};
VNET_PCPUSTAT_DEFINE(struct xstats, xstats);
VNET_PCPUSTAT_SYSINIT(xstats);
This unrolls into:
1 struct _hack;
2 counter_u64_t vnet_entry_xstats[sizeof(struct xstats) / sizeof(uint64_t)]
__attribute__((__section__("set_vnet"))) __attribute__((__used__));
3
4
5 static void
6 vnet_xstats_init(const void *unused)
7 {
8 do {
9 for (int i = 0; i < (sizeof((*(__typeof(vnet_entry_xstats)
*) (__curthread())->td_vnet))->vnet_data_base) + (uintptr_t) &
vnet_entry_xstats))) / sizeof(counter_u64_t)); i++)
10 ((*(__typeof(vnet_entry_xstats) *)
(__curthread())->td_vnet))->vnet_data_base) + (uintptr_t) &
vnet_entry_xstats)))[i] = counter_u64_alloc((0x0002));
11 } while (0);
12 }
In essance what looks so complicated is (on a per vnet base):
void *array[16];
for (i=0 ; i<16; i++)
array[i] = alloc();
The above (with the vnet bits), on i386, is translated into:
0340 :
340: 55 push %ebp
341: 89 e5 mov%esp,%ebp
343: 56 push %esi
344: 50 push %eax
345: be c0 ff ff ff mov$0xffc0,%esi
34a: 90 nop
34b: 90 nop
34c: 90 nop
34d: 90 nop
34e: 90 nop
34f: 90 nop
350: c7 04 24 02 00 00 00movl $0x2,(%esp)
357: e8 fc ff ff ff call 358
358: R_386_PC32 counter_u64_alloc
35c: 64 8b 0d 00 00 00 00mov%fs:0x0,%ecx
363: 8b 89 1c 03 00 00 mov0x31c(%ecx),%ecx
369: 8b 49 1cmov0x1c(%ecx),%ecx
36c: 89 84 31 88 14 00 00mov%eax,0x1488(%ecx,%esi,1)
36f: R_386_RELATIVE *ABS*
373: 83 c6 04add$0x4,%esi
376: 75 d8 jne350
378: 83 c4 04add$0x4,%esp
37b: 5e pop%esi
37c: 5d pop%ebp
37d: c3 ret
Now here's the problem:
__start_set_vnet is 0x1448
__stop_set_vnet is 0x1488
The problem is that the code generated goes like this:
%esi = -64
repeat:
%eax = alloc()
We do all the curthread->td_vnet->vnet_data_base in %ecx and then do
mov%eax,0x1488(%ecx,%esi,1)
Which is:
move the alloc() result into curthread->td_vnet->vnet_data_base + 0x1488 +
(1 * -64)
Now 0x1488 - 64 gets us to the beginning of the array[] or array[0].
%esi += 4
So next iteration it'll be 0x1488 - 60 or array[1] ... and so on.
while %esi != 0 goto repeat;
It's an easy way to to the for(i=0; i<16; i++) loop.
The problem is that 0x1488 was not relocated.
When we are going over the relocations and calling into elf_relocaddr() the
check for VIMAGE is:
if (x >= ef->vnet_start && x < ef->vnet_stop) {
In our case we have an *ABS* R_386_RELATIVE of 0x1488, which is ==
ef->vnet_stop but not < ef->vnet_stop.
The real problem is that with non-simple-types the code generated with an
absolute relocation might be just outside the range. We cannot adjust the
check as there might be a simple-type following in the next section which would
then be relocated.
For CARP this showed up because the VNET_PCPUSTAT_DEFINE() went into the VNET
section the last and hence the problem showed up. If there was any other, say
int V_Foo after it, we'd never have noticed. We cannot fully control the
order in which symbols go into the section, or at least not to the extend we'd
like to, so we cannot make sure there's always a char at the end.
The only solution arichardson and I agreed to would be to add 1 byte of padding
to the end of the section.
Using BYTE(1) in a linker script however would always create a set_vnet section
in
[Bug 230857] loading carp module panic i386 kernel (VIMAGE related)
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=230857 Bjoern A. Zeeb changed: What|Removed |Added Status|Open|In Progress --- Comment #4 from Bjoern A. Zeeb --- Ok, so the short explanation is that having a non-simple-type at the end of the dpcpu or vnet linker sets and an intelligent compiler/linker combination can result in the last symbol not being relocated. In the case of i386/carp this was the PCPU stats glebius introduced which is an array of 16 pointers. I've spent a day to think of possible work around and the only one was to add padding to the end of the section; with the help of arichardson managed to work my way around linker scripts and with an extra 8 hours I have a dual-stage linker-script solution which will only adjust the kernel modules which actually do have a vnet_set or pcpu_set section and not create one in every module with the size of 1 byte. I'll write the entire details up including sample code and the hacked up prototype solution and post it all here and in phab sometime the next days (possibly after the weekend). TODO: investigate which other architectures but i386 are possibly affected by this as well. -- You are receiving this mail because: You are on the CC list for the bug. ___ freebsd-toolchain@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-toolchain To unsubscribe, send any mail to "freebsd-toolchain-unsubscr...@freebsd.org"
