Hi Matthias, thanks for your time and response. > Just a random guess, but bcn2837-rpi-3-b.dtb does look right, the DTB should > be > called bcm2837-rpi-3-b.dtb
Sorry, that was an OCR error (wasn't sure how to screenshot or log to a file in this context, so pictures -> OCR seemed like my best option, pictures available here: https://discourse.nixos.org/t/btrfs-pi-wont-boot-from-compressed-subvolume/18462/2?u=n8henrie). It is indeed `bc*m*`... in the actual output. > Then you try to fixup that address which has no valid DTB and fails. > > Anyway to me that sounds like a nixOS problem rather then a U-Boot problem. > I'm > not even sure which version of U-Boot you are using and if it has any patches > on > top of the upstream version. Sorry, I included the link to the relevant code (https://github.com/NixOS/nixpkgs/blob/3d4e586313d292c9e764a8e88a1fdccb16ffb2d3/pkgs/misc/uboot/default.nix) but should have included those details: ``` defaultSrc = fetchurl { url = "ftp://ftp.denx.de/pub/u-boot/u-boot-${defaultVersion}.tar.bz2"; hash = "sha256-gbRUMifbIowD+KG/XdvIE7C7j2VVzkYGTvchpvxoBBM="; defaultVersion = "2022.01"; ``` By default it will have two patches: - https://github.com/NixOS/nixpkgs/blob/master/pkgs/misc/uboot/0001-configs-rpi-allow-for-bigger-kernels.patch - https://github.com/NixOS/nixpkgs/blob/master/pkgs/misc/uboot/0001-rpi-Copy-properties-from-firmware-dtb-to-the-loaded-.patch The latter looks like it might be relevant. I was suspicious about it being a u-boot issue because everything works fine by default, it's only when I enable zstd compression that I get the above errors. However I'm pretty new to NixOS and obviously know little about u-boot, so you may well be correct! Nate On Wed, May 25, 2022 at 2:58 AM Matthias Brugger <matthias....@gmail.com> wrote: > > > > On 22/05/2022 17:36, Nathan Henrie wrote: > > Hello u-boot team, > > > > I’ve been experimenting for about a year with a NixOS-based Raspberry Pi > > image, with the end goal of a zstd-compressed image that uses BTRFS > > subvolumes, having the root filesystem at @ and boot at @boot (with several > > other subvolumes). The NixOS Raspberry Pi images use u-boot by default. > > I’ve started a few discussions in the NixOS community as I learn and > > iterate: > > https://discourse.nixos.org/t/raspberry-pi-nixos-on-btrfs-root/14081 and > > I’ve made my code available at: https://github.com/n8henrie/nixos-btrfs-pi > > . I’m sure it’s obvious (or will be soon) that I’m a hobbyist, a novice > > with NixOS, and that I understand very little about u-boot and bootloaders > > in general, so I hope you can be patient as I humbly ask for someone to > > point me in the right direction. > > > > For reference, the official NixOS code for generating u-boot images is > > here: > > https://github.com/NixOS/nixpkgs/blob/3d4e586313d292c9e764a8e88a1fdccb16ffb2d3/pkgs/misc/uboot/default.nix > > > > I’ve made a lot of progress, including an image that reliably boots from > > @boot to the @ root subvolume with the u-boot configuration: > > > > CONFIG_CMD_BTRFS=y > > CONFIG_ZSTD=y > > CONFIG_BOOTCOMMAND="setenv boot_prefixes / /boot/ /@/ /@boot/; run > > distro_bootcmd;" > > > > with kernel params: "root=LABEL=NIXOS_SD" "rootfstype=btrfs" > > "rootflags=subvol=@" > > > > Unfortunately, after the initial boot and going through the initial OS > > installation steps, if I have compression enabled, it reboots into a boot > > loop with the following error messages: > > > > Scanning mmc 0:2... > > Found /@boot/extlinux/extlinux.conf > > Retrieving file: /@boot/extlinux/extlinux.conf > > 2938 bytes read in 27 ms (105.5 KiB/s) > > > > 1 NixOS Default > > 2 NixOS Configuration 4 (2022-02-24 21:34 22.05pre356135.7f9b6c2babf) > > 3 NixOS Configuration 3 (2022-02-25 03:59 22.05pre356435.7/9b6eZbabf) > > 4: NixOS Configuration 2 (2022-02-24 21:34 22.05pre356435.7/9b6c2babr) > > 5: NixOS > > Enter choice: 1: NixOS - Default > > Retrieving file: > > /@boot/extlinux/..nixos/zhbharzhzqga617uc37uk0g6q1bx891u-initrd-linux-5.10.101-initrd > > 20588420 bytes read in 1910 ms (10.3 MiB/s) > > Retrieving file: > > /@boot/extlinux/..nixos/6c1m6j0k1gmjrmikOyyizariiry?Zlqr-linux-5.10.101-Image > > 50629120 bytes read in 6863 ms (7 MiB/s) > > append: > > init-mix/store/pikl8ju4077iaix36n8mifzu6l6rcajz-nixos-system-nixpi-22.05p > > re356435-7/9b6eZbabf/init console=tty1 console=ttyAMAO console=ttyS0, > > 1115200 root= LABEL=-NIXOS_SD rootfstype=btrfs rootflags=subvol=@ > > loglevel=4 > > Retrieving file: > > /@boot/extlinux/../nixos/6c16j0k1gmjrmikOyyizari1ry72lqr-linux-5.10.101-dtbs/broadcon/bcn2837-rpi-3-b.dtb > > Just a random guess, but bcn2837-rpi-3-b.dtb does look right, the DTB should > be > called bcm2837-rpi-3-b.dtb > > > 14310 bytes read in 66 ms (210.9 KiB/s) > > Moving Image from 0x80000 to 0x200000, end=32f0000 > > ## Flattened Device Tree blob at 03700000 > > Booting using the fdt blob at 0x3700000 > > Using Device Tree in place at 0000000003700000 end 00000000037067e5 > > fdt_find_or_add_subnode: memory: FDT_ERR_BADSTRUCTURE > > ERROR: arch-specific fdt fixup failed > > - must RESET the board to recover. > > > > Then you try to fixup that address which has no valid DTB and fails. > > Anyway to me that sounds like a nixOS problem rather then a U-Boot problem. > I'm > not even sure which version of U-Boot you are using and if it has any patches > on > top of the upstream version. > > Regards, > Matthias > > > FDT creation failed! > > resetting ... > > > > This happens even though I have CONFIG_ZSTD=y. If I do *not* enable > > compression (in my BTRFS mount options), it reboots normally and everything > > works! > > > > I may have made some OCR / C&P errors above; I’ve posted some pictures of > > the error messages (and similar discussion to this email, with no > > responses) here: > > https://discourse.nixos.org/t/btrfs-pi-wont-boot-from-compressed-subvolume/18462/2 > > > > I’m not currently using BTRFS RAID, just a single device. One of my other > > (x86_64) machines uses a similar setup with @, @boot, and zstd-compression > > with grub and works well, so I’m hoping this is also possible with u-boot. > > > > Seeing that the errors mention the FDT and fixup, I tried > > CONFIG_ARCH_FIXUP_FDT_MEMORY with both =y and =n without interesting > > changes to the error messages. > > > > What I’m assuming, based on the above, is that re-writing the @boot > > subvolume with compression is moving the location of the device tree files > > and that u-boot is no longer able to find the necessary files (some of > > which I think may also be in the nix store in the @nix subvolume). > > > > Does that sound like a reasonable guess as to what is going on? If so, are > > there config options or settings that I might be able to use to help u-boot > > find the necessary files after things are re-written with compression? > > > > Many thanks in advance for your time and attention, and thanks for your > > hard work on u-boot!