[9fans] Plan9 on Pi 3B+
Has anyone tried Plan 9 on the new Pi 3B+? I've
run into something that confuses me a bit. First,
it seems you need the new version of start_cd.elf
to bring up the 3B+. However, with that, the kernel
throws a lock loop error. In tracking down the loop,
it happens in startcpus() in archbcm2.c. The code
here looks like:
for(i = 0; i < ncpu; i++)
lock(&startlock[i]);
cachedwbse(startlock, sizeof startlock);
for(i = 1; i < ncpu; i++) {
if(startcpu(i) < 0)
return i;
lock(&startlock[i]);
unlock(&startlock[i]);
}
So we grab a lock on all the CPUs, then drop into
a second loop where we start CPUs 1 to n. But
in that loop we grab the lock again and then
immediately unlock it. This is where the lock
loop happens. Everything seems to continue
to be happy on both a 2 and a 3B+ if I comment
out the lock in the second loop. But what is the
rationale for the lock in the second loop? Was
there a reason for putting it there, or was it an
oversight that wasn't exposed until the new boot
code?
Thanks,
BLS
Re: [9fans] Plan9 on Pi 3B+
I would guess it's a bug because the second bug is starting the first loop st 0- erik
Re: [9fans] Plan9 on Pi 3B+
> But what is the
> rationale for the lock in the second loop?
Sorry for the confusing code. The startlocks are locked by
cpu0 before booting the secondary cores. Each core unlocks
its own startlock when it starts. Cpu0 blocks on each startlock
waiting for the corresponding core to unlock it, as a way of
serialising the booting of the secondary cores (and detecting
that they have started).
If for some reason the secondary cpus don't start up, the
result is a lock loop. I really should have used canlock()
instead, to expose a more helpful error message. But it's
a "should never happen" condition.
It's happening with the new start_cd.elf because the Pi
Foundation made a little change in the firmware which makes
secondary cores wait with a WFE when booting, so the OS
now has to do a SEV to wake them up.
I'm just about to release an update to the bcm port for
the 3B+, including drivers for the new "gigabit" ethernet
(not really, because it's squeezed through a slow usb2 adapter)
and the updated wifi chip. But in the meantime the following
should let you boot any pi2 or pi3 with the latest start_cd.elf:
--- /sys/src/9/bcm/archbcm2.c Fri Mar 16 17:08:21 2018
***
*** 167,172
--- 167,174
if(mb->clr[cpu].startcpu)
return -1;
mb->set[cpu].startcpu = PADDR(cpureset);
+ coherence();
+ sev();
for(i = 0; i < 1000; i++)
if(mb->clr[cpu].startcpu == 0)
return 0;
***
*** 196,202
int
startcpus(uint ncpu)
{
! int i;
for(i = 0; i < ncpu; i++)
lock(&startlock[i]);
--- 198,204
int
startcpus(uint ncpu)
{
! int i, timeout;
for(i = 0; i < ncpu; i++)
lock(&startlock[i]);
***
*** 204,210
for(i = 1; i < ncpu; i++){
if(startcpu(i) < 0)
return i;
! lock(&startlock[i]);
unlock(&startlock[i]);
}
return ncpu;
--- 206,215
for(i = 1; i < ncpu; i++){
if(startcpu(i) < 0)
return i;
! timeout = 1000;
! while(!canlock(&startlock[i]))
! if(--timeout == 0)
! return i;
unlock(&startlock[i]);
}
return ncpu;
--- /sys/src/9/bcm/armv7.s Fri Mar 16 16:47:18 2018
***
*** 17,22
--- 17,23
#define WFI WORD$0xe320f003 /* wait for interrupt */
#define WFI_EQWORD$0x0320f003 /* wait for interrupt if eq */
#define ERET WORD$0xe160006e /* exception return from HYP */
+ #define SEV WORD$0xe320f004 /* send event */
/* tas/cas strex debugging limits; started at 1 */
#define MAXSC 100
***
*** 386,391
--- 387,396
TEXT coherence(SB), $-4
BARRIERS
+ RET
+
+ TEXT sev(SB), $-4
+ SEV
RET
/*
Re: [9fans] Plan9 on Pi 3B+
Thanks! I got one recently and have been meaning to ask about his.
Running Linux, 3B+ seems perceptibly faster than 3B.
On Tue, Apr 3, 2018 at 2:05 PM, Richard Miller <9f...@hamnavoe.com> wrote:
> > But what is the
> > rationale for the lock in the second loop?
>
> Sorry for the confusing code. The startlocks are locked by
> cpu0 before booting the secondary cores. Each core unlocks
> its own startlock when it starts. Cpu0 blocks on each startlock
> waiting for the corresponding core to unlock it, as a way of
> serialising the booting of the secondary cores (and detecting
> that they have started).
>
> If for some reason the secondary cpus don't start up, the
> result is a lock loop. I really should have used canlock()
> instead, to expose a more helpful error message. But it's
> a "should never happen" condition.
>
> It's happening with the new start_cd.elf because the Pi
> Foundation made a little change in the firmware which makes
> secondary cores wait with a WFE when booting, so the OS
> now has to do a SEV to wake them up.
>
> I'm just about to release an update to the bcm port for
> the 3B+, including drivers for the new "gigabit" ethernet
> (not really, because it's squeezed through a slow usb2 adapter)
> and the updated wifi chip. But in the meantime the following
> should let you boot any pi2 or pi3 with the latest start_cd.elf:
>
> --- /sys/src/9/bcm/archbcm2.c Fri Mar 16 17:08:21 2018
> ***
> *** 167,172
> --- 167,174
> if(mb->clr[cpu].startcpu)
> return -1;
> mb->set[cpu].startcpu = PADDR(cpureset);
> + coherence();
> + sev();
> for(i = 0; i < 1000; i++)
> if(mb->clr[cpu].startcpu == 0)
> return 0;
> ***
> *** 196,202
> int
> startcpus(uint ncpu)
> {
> ! int i;
>
> for(i = 0; i < ncpu; i++)
> lock(&startlock[i]);
> --- 198,204
> int
> startcpus(uint ncpu)
> {
> ! int i, timeout;
>
> for(i = 0; i < ncpu; i++)
> lock(&startlock[i]);
> ***
> *** 204,210
> for(i = 1; i < ncpu; i++){
> if(startcpu(i) < 0)
> return i;
> ! lock(&startlock[i]);
> unlock(&startlock[i]);
> }
> return ncpu;
> --- 206,215
> for(i = 1; i < ncpu; i++){
> if(startcpu(i) < 0)
> return i;
> ! timeout = 1000;
> ! while(!canlock(&startlock[i]))
> ! if(--timeout == 0)
> ! return i;
> unlock(&startlock[i]);
> }
> return ncpu;
>
> --- /sys/src/9/bcm/armv7.s Fri Mar 16 16:47:18 2018
> ***
> *** 17,22
> --- 17,23
> #define WFI WORD$0xe320f003 /* wait for interrupt */
> #define WFI_EQWORD$0x0320f003 /* wait for interrupt if
> eq */
> #define ERET WORD$0xe160006e /* exception return from HYP */
> + #define SEV WORD$0xe320f004 /* send event */
>
> /* tas/cas strex debugging limits; started at 1 */
> #define MAXSC 100
> ***
> *** 386,391
> --- 387,396
>
> TEXT coherence(SB), $-4
> BARRIERS
> + RET
> +
> + TEXT sev(SB), $-4
> + SEV
> RET
>
> /*
>
>
>
