[9fans] Plan9 on Pi 3B+
Has anyone tried Plan 9 on the new Pi 3B+? I've run into something that confuses me a bit. First, it seems you need the new version of start_cd.elf to bring up the 3B+. However, with that, the kernel throws a lock loop error. In tracking down the loop, it happens in startcpus() in archbcm2.c. The code here looks like: for(i = 0; i < ncpu; i++) lock(&startlock[i]); cachedwbse(startlock, sizeof startlock); for(i = 1; i < ncpu; i++) { if(startcpu(i) < 0) return i; lock(&startlock[i]); unlock(&startlock[i]); } So we grab a lock on all the CPUs, then drop into a second loop where we start CPUs 1 to n. But in that loop we grab the lock again and then immediately unlock it. This is where the lock loop happens. Everything seems to continue to be happy on both a 2 and a 3B+ if I comment out the lock in the second loop. But what is the rationale for the lock in the second loop? Was there a reason for putting it there, or was it an oversight that wasn't exposed until the new boot code? Thanks, BLS
Re: [9fans] Plan9 on Pi 3B+
I would guess it's a bug because the second bug is starting the first loop st 0- erik
Re: [9fans] Plan9 on Pi 3B+
> But what is the > rationale for the lock in the second loop? Sorry for the confusing code. The startlocks are locked by cpu0 before booting the secondary cores. Each core unlocks its own startlock when it starts. Cpu0 blocks on each startlock waiting for the corresponding core to unlock it, as a way of serialising the booting of the secondary cores (and detecting that they have started). If for some reason the secondary cpus don't start up, the result is a lock loop. I really should have used canlock() instead, to expose a more helpful error message. But it's a "should never happen" condition. It's happening with the new start_cd.elf because the Pi Foundation made a little change in the firmware which makes secondary cores wait with a WFE when booting, so the OS now has to do a SEV to wake them up. I'm just about to release an update to the bcm port for the 3B+, including drivers for the new "gigabit" ethernet (not really, because it's squeezed through a slow usb2 adapter) and the updated wifi chip. But in the meantime the following should let you boot any pi2 or pi3 with the latest start_cd.elf: --- /sys/src/9/bcm/archbcm2.c Fri Mar 16 17:08:21 2018 *** *** 167,172 --- 167,174 if(mb->clr[cpu].startcpu) return -1; mb->set[cpu].startcpu = PADDR(cpureset); + coherence(); + sev(); for(i = 0; i < 1000; i++) if(mb->clr[cpu].startcpu == 0) return 0; *** *** 196,202 int startcpus(uint ncpu) { ! int i; for(i = 0; i < ncpu; i++) lock(&startlock[i]); --- 198,204 int startcpus(uint ncpu) { ! int i, timeout; for(i = 0; i < ncpu; i++) lock(&startlock[i]); *** *** 204,210 for(i = 1; i < ncpu; i++){ if(startcpu(i) < 0) return i; ! lock(&startlock[i]); unlock(&startlock[i]); } return ncpu; --- 206,215 for(i = 1; i < ncpu; i++){ if(startcpu(i) < 0) return i; ! timeout = 1000; ! while(!canlock(&startlock[i])) ! if(--timeout == 0) ! return i; unlock(&startlock[i]); } return ncpu; --- /sys/src/9/bcm/armv7.s Fri Mar 16 16:47:18 2018 *** *** 17,22 --- 17,23 #define WFI WORD$0xe320f003 /* wait for interrupt */ #define WFI_EQWORD$0x0320f003 /* wait for interrupt if eq */ #define ERET WORD$0xe160006e /* exception return from HYP */ + #define SEV WORD$0xe320f004 /* send event */ /* tas/cas strex debugging limits; started at 1 */ #define MAXSC 100 *** *** 386,391 --- 387,396 TEXT coherence(SB), $-4 BARRIERS + RET + + TEXT sev(SB), $-4 + SEV RET /*
Re: [9fans] Plan9 on Pi 3B+
Thanks! I got one recently and have been meaning to ask about his. Running Linux, 3B+ seems perceptibly faster than 3B. On Tue, Apr 3, 2018 at 2:05 PM, Richard Miller <9f...@hamnavoe.com> wrote: > > But what is the > > rationale for the lock in the second loop? > > Sorry for the confusing code. The startlocks are locked by > cpu0 before booting the secondary cores. Each core unlocks > its own startlock when it starts. Cpu0 blocks on each startlock > waiting for the corresponding core to unlock it, as a way of > serialising the booting of the secondary cores (and detecting > that they have started). > > If for some reason the secondary cpus don't start up, the > result is a lock loop. I really should have used canlock() > instead, to expose a more helpful error message. But it's > a "should never happen" condition. > > It's happening with the new start_cd.elf because the Pi > Foundation made a little change in the firmware which makes > secondary cores wait with a WFE when booting, so the OS > now has to do a SEV to wake them up. > > I'm just about to release an update to the bcm port for > the 3B+, including drivers for the new "gigabit" ethernet > (not really, because it's squeezed through a slow usb2 adapter) > and the updated wifi chip. But in the meantime the following > should let you boot any pi2 or pi3 with the latest start_cd.elf: > > --- /sys/src/9/bcm/archbcm2.c Fri Mar 16 17:08:21 2018 > *** > *** 167,172 > --- 167,174 > if(mb->clr[cpu].startcpu) > return -1; > mb->set[cpu].startcpu = PADDR(cpureset); > + coherence(); > + sev(); > for(i = 0; i < 1000; i++) > if(mb->clr[cpu].startcpu == 0) > return 0; > *** > *** 196,202 > int > startcpus(uint ncpu) > { > ! int i; > > for(i = 0; i < ncpu; i++) > lock(&startlock[i]); > --- 198,204 > int > startcpus(uint ncpu) > { > ! int i, timeout; > > for(i = 0; i < ncpu; i++) > lock(&startlock[i]); > *** > *** 204,210 > for(i = 1; i < ncpu; i++){ > if(startcpu(i) < 0) > return i; > ! lock(&startlock[i]); > unlock(&startlock[i]); > } > return ncpu; > --- 206,215 > for(i = 1; i < ncpu; i++){ > if(startcpu(i) < 0) > return i; > ! timeout = 1000; > ! while(!canlock(&startlock[i])) > ! if(--timeout == 0) > ! return i; > unlock(&startlock[i]); > } > return ncpu; > > --- /sys/src/9/bcm/armv7.s Fri Mar 16 16:47:18 2018 > *** > *** 17,22 > --- 17,23 > #define WFI WORD$0xe320f003 /* wait for interrupt */ > #define WFI_EQWORD$0x0320f003 /* wait for interrupt if > eq */ > #define ERET WORD$0xe160006e /* exception return from HYP */ > + #define SEV WORD$0xe320f004 /* send event */ > > /* tas/cas strex debugging limits; started at 1 */ > #define MAXSC 100 > *** > *** 386,391 > --- 387,396 > > TEXT coherence(SB), $-4 > BARRIERS > + RET > + > + TEXT sev(SB), $-4 > + SEV > RET > > /* > > >