On Thu, Feb 2, 2023 at 3:13 AM Daniel Henrique Barboza <dbarb...@ventanamicro.com> wrote: > > As it is now, riscv_compute_fdt_addr() is receiving a dram_base, a > mem_size (which is defaulted to MachineState::ram_size in all boards) > and the FDT pointer. And it makes a very important assumption: the DRAM > interval dram_base + mem_size is contiguous. This is indeed the case for > most boards that use a FDT. > > The Icicle Kit board works with 2 distinct RAM banks that are separated > by a gap. We have a lower bank with 1GiB size, a gap follows, then at > 64GiB the high memory starts. MachineClass::default_ram_size for this > board is set to 1.5Gb, and machine_init() is enforcing it as minimal RAM > size, meaning that there we'll always have at least 512 MiB in the Hi > RAM area. > > Using riscv_compute_fdt_addr() in this board is weird because not only > the board has sparse RAM, and it's calling it using the base address of > the Lo RAM area, but it's also using a mem_size that we have guarantees > that it will go up to the Hi RAM. All the function assumptions doesn't > work for this board. > > In fact, what makes the function works at all in this case is a > coincidence. Commit 1a475d39ef54 introduced a 3GB boundary for the FDT, > down from 4Gb, that is enforced if dram_base is lower than 3072 MiB. For > the Icicle Kit board, memmap[MICROCHIP_PFSOC_DRAM_LO].base is 0x80000000 > (2 Gb) and it has a 1Gb size, so it will fall in the conditions to put > the FDT under a 3Gb address, which happens to be exactly at the end of > DRAM_LO. If the base address of the Lo area started later than 3Gb this > function would be unusable by the board. Changing any assumptions inside > riscv_compute_fdt_addr() can also break it by accident as well. > > Let's change riscv_compute_fdt_addr() semantics to be appropriate to the > Icicle Kit board and for future boards that might have sparse RAM > topologies to worry about: > > - relieve the condition that the dram_base + mem_size area is contiguous, > since this is already not the case today; > > - receive an extra 'dram_size' size attribute that refers to a contiguous > RAM block that the board wants the FDT to reside on. > > Together with 'mem_size' and 'fdt', which are now now being consumed by a > MachineState pointer, we're able to make clear assumptions based on the > DRAM block and total mem_size available to ensure that the FDT will be put > in a valid RAM address. > > Signed-off-by: Daniel Henrique Barboza <dbarb...@ventanamicro.com>
Reviewed-by: Alistair Francis <alistair.fran...@wdc.com> Alistair > --- > hw/riscv/boot.c | 35 +++++++++++++++++++++++------------ > hw/riscv/microchip_pfsoc.c | 3 ++- > hw/riscv/sifive_u.c | 3 ++- > hw/riscv/spike.c | 3 ++- > hw/riscv/virt.c | 3 ++- > include/hw/riscv/boot.h | 2 +- > 6 files changed, 32 insertions(+), 17 deletions(-) > > diff --git a/hw/riscv/boot.c b/hw/riscv/boot.c > index 2e53494b08..c7e0e50bd8 100644 > --- a/hw/riscv/boot.c > +++ b/hw/riscv/boot.c > @@ -250,33 +250,44 @@ void riscv_load_initrd(MachineState *machine, uint64_t > kernel_entry) > } > > /* > - * The FDT should be put at the farthest point possible to > - * avoid overwriting it with the kernel/initrd. > + * This function makes an assumption that the DRAM interval > + * 'dram_base' + 'dram_size' is contiguous. > * > - * This function makes an assumption that the DRAM is > - * contiguous. It also cares about 32-bit systems and > - * will limit fdt_addr to be addressable by them even for > - * 64-bit CPUs. > + * Considering that 'dram_end' is the lowest value between > + * the end of the DRAM block and MachineState->ram_size, the > + * FDT location will vary according to 'dram_base': > + * > + * - if 'dram_base' is less that 3072 MiB, the FDT will be > + * put at the lowest value between 3072 MiB and 'dram_end'; > + * > + * - if 'dram_base' is higher than 3072 MiB, the FDT will be > + * put at 'dram_end'. > * > * The FDT is fdt_packed() during the calculation. > */ > -uint64_t riscv_compute_fdt_addr(hwaddr dram_base, uint64_t mem_size, > - void *fdt) > +uint64_t riscv_compute_fdt_addr(hwaddr dram_base, hwaddr dram_size, > + MachineState *ms) > { > - uint64_t temp; > - hwaddr dram_end = dram_base + mem_size; > - int ret = fdt_pack(fdt); > + int ret = fdt_pack(ms->fdt); > + hwaddr dram_end, temp; > int fdtsize; > > /* Should only fail if we've built a corrupted tree */ > g_assert(ret == 0); > > - fdtsize = fdt_totalsize(fdt); > + fdtsize = fdt_totalsize(ms->fdt); > if (fdtsize <= 0) { > error_report("invalid device-tree"); > exit(1); > } > > + /* > + * A dram_size == 0, usually from a MemMapEntry[].size element, > + * means that the DRAM block goes all the way to ms->ram_size. > + */ > + dram_end = dram_base; > + dram_end += dram_size ? MIN(ms->ram_size, dram_size) : ms->ram_size; > + > /* > * We should put fdt as far as possible to avoid kernel/initrd > overwriting > * its content. But it should be addressable by 32 bit system as well. > diff --git a/hw/riscv/microchip_pfsoc.c b/hw/riscv/microchip_pfsoc.c > index 17499d4152..2b91e49561 100644 > --- a/hw/riscv/microchip_pfsoc.c > +++ b/hw/riscv/microchip_pfsoc.c > @@ -642,7 +642,8 @@ static void > microchip_icicle_kit_machine_init(MachineState *machine) > > /* Compute the fdt load address in dram */ > fdt_load_addr = > riscv_compute_fdt_addr(memmap[MICROCHIP_PFSOC_DRAM_LO].base, > - machine->ram_size, > machine->fdt); > + > memmap[MICROCHIP_PFSOC_DRAM_LO].size, > + machine); > riscv_load_fdt(fdt_load_addr, machine->fdt); > > /* Load the reset vector */ > diff --git a/hw/riscv/sifive_u.c b/hw/riscv/sifive_u.c > index 626d4dc2f3..d3ab7a9cda 100644 > --- a/hw/riscv/sifive_u.c > +++ b/hw/riscv/sifive_u.c > @@ -617,7 +617,8 @@ static void sifive_u_machine_init(MachineState *machine) > } > > fdt_load_addr = riscv_compute_fdt_addr(memmap[SIFIVE_U_DEV_DRAM].base, > - machine->ram_size, machine->fdt); > + memmap[SIFIVE_U_DEV_DRAM].size, > + machine); > riscv_load_fdt(fdt_load_addr, machine->fdt); > > if (!riscv_is_32bit(&s->soc.u_cpus)) { > diff --git a/hw/riscv/spike.c b/hw/riscv/spike.c > index f1114f2c71..cc3f6dac17 100644 > --- a/hw/riscv/spike.c > +++ b/hw/riscv/spike.c > @@ -325,7 +325,8 @@ static void spike_board_init(MachineState *machine) > } > > fdt_load_addr = riscv_compute_fdt_addr(memmap[SPIKE_DRAM].base, > - machine->ram_size, machine->fdt); > + memmap[SPIKE_DRAM].size, > + machine); > riscv_load_fdt(fdt_load_addr, machine->fdt); > > /* load the reset vector */ > diff --git a/hw/riscv/virt.c b/hw/riscv/virt.c > index 2e0a0cdb17..a061151a6f 100644 > --- a/hw/riscv/virt.c > +++ b/hw/riscv/virt.c > @@ -1304,7 +1304,8 @@ static void virt_machine_done(Notifier *notifier, void > *data) > } > > fdt_load_addr = riscv_compute_fdt_addr(memmap[VIRT_DRAM].base, > - machine->ram_size, machine->fdt); > + memmap[VIRT_DRAM].size, > + machine); > riscv_load_fdt(fdt_load_addr, machine->fdt); > > /* load the reset vector */ > diff --git a/include/hw/riscv/boot.h b/include/hw/riscv/boot.h > index 46de4ec46b..511390f60e 100644 > --- a/include/hw/riscv/boot.h > +++ b/include/hw/riscv/boot.h > @@ -48,7 +48,7 @@ target_ulong riscv_load_kernel(MachineState *machine, > symbol_fn_t sym_cb); > void riscv_load_initrd(MachineState *machine, uint64_t kernel_entry); > uint64_t riscv_compute_fdt_addr(hwaddr dram_start, uint64_t dram_size, > - void *fdt); > + MachineState *ms); > void riscv_load_fdt(hwaddr fdt_addr, void *fdt); > void riscv_setup_rom_reset_vec(MachineState *machine, RISCVHartArrayState > *harts, > hwaddr saddr, > -- > 2.39.1 > >