Hi Tim, On Thu, Oct 14, 2021 at 10:03 PM Tim Harvey <thar...@gateworks.com> wrote: > > On Mon, Oct 11, 2021 at 3:15 PM Tim Harvey <thar...@gateworks.com> wrote: > > > > Greetings, > > > > Is anyone successfully booting U-Boot with OP-TEE support on the IMX8M? > > > > My understanding is that you need to add tee.bin to the images in the > > FIT image and include it in loadables following the ATF. > > > > While this was done with arch/arm/mach-imx/mkimage_fit_atf.sh before > > the switch to binman by simply having tee.bin in the U-Boot directory > > and passing in TEE_LOAD_ADDR (or accepting the default of 0xfe000000) > > once you switch to binman it needs to be added for your board. > > > > Additionally in order to use OP-TEE from U-Boot (ie for dek_blob > > command) you need to add a node with compatible=linaro,optee-tz as > > well. > > > > I've done the following to add OP-TEE for imx8mm_venice: > > diff --git a/configs/imx8mm_venice_defconfig > > b/configs/imx8mm_venice_defconfig > > index d85827b588..85d2b20810 100644 > > -u-boot.dtsi > > index e0fa9ff4bf..7a71b974e1 100644 > > --- a/arch/arm/dts/imx8mm-venice-u-boot.dtsi > > +++ b/arch/arm/dts/imx8mm-venice-u-boot.dtsi > > @@ -10,6 +10,13 @@ > > multiple-images; > > }; > > > > + firmware { > > + optee { > > + compatible = "linaro,optee-tz"; > > + method = "smc"; > > + }; > > + }; > > + > > wdt-reboot { > > compatible = "wdt-reboot"; > > wdt = <&wdog1>; > > @@ -152,6 +159,16 @@ > > }; > > }; > > > > + tee { > > + description = "TEE firmware"; > > + type = "firmware"; > > + arch = "arm64"; > > + compression = "none"; > > + data = "tee.bin"; > > + load = <0xbe000000>; > > + entry = <0xbe000000>; > > + }; > > + > > @fdt-SEQ { > > description = "NAME"; > > type = "flat_dt"; > > @@ -165,7 +182,7 @@ > > @config-SEQ { > > description = "NAME"; > > firmware = "uboot"; > > - loadables = "atf"; > > + loadables = "atf", "tee"; > > fdt = "fdt-SEQ"; > > }; > > }; > > > > > > However, when I attempt to boot I hang when the ATF is run. > > > > Where does the TEE_LOAD_ADDR come from specifically? I would think > > this needs to be defined when building tee and needs to match the load > > address used in the FIT image. It appears that perhaps this is supped > > to be DDR_BASE + DDR_SIZE - 32MIB but I'm not entirely sure. > > > > I'm currently using NXP's ATF (imx_5.4.3_2.0.0) and NXP's TEE > > (imx_5.4.70_2.3.0) and would also like to understand if NXP's branches > > are strictly required here and if so what the pros and cons of using > > them are. > > > > Anyone using IMX8MM OP-TEE that could point me in the right direction? > > > > Here's what I've learned so far via various responses, research and testing. > > There are multiple boot flows that can be used with OP-TEE: > 1. SPL -> ATF -> OP-TEE -> U-boot -> LInux This is the case for ARMv8: TF-A provides runtime service here as Secure Monitor (running on EL3), arbitrating context switches between Secure/Non-secure worlds. OP-TEE OS core is running on Secure EL1.
> 2. SPL -> OP-TEE -> U-boot -> Linux This is the case for ARMv7, where there is only one Secure PL1, so OP-TEE also fulfills the function of SM (arbitrating context switches between NW/SW). > > I'm not really clear what the pros and cons of each are - can anyone > shed some light on this? > > Method #1 is what I've found the most info on and have mostly gotten > working so I will go over that here. As IMX8MM runs on Cortex-A Arm > TrustZone is available and thus TF-A can be used. > > By the way, I'm a bit confused as to ATF vs TF-A. From what I can find: > - ATF (Arm Trusted Firmware) is a Trusted Firmware reference > implementation of secure world software for Armv7-A, Armv8-A, and > Armv8-M architectures. > - TF-A (Trusted Firmware-A) also appears to provide the same thing > Are these the same? Which one is it that NXP uses for imx-atf [1] It's the same product, which I guess was renamed to avoid confusion, as ARM introduced Trusted Firmware M for M-cores. > > When OP-TEE is used with TF-A the TF-A gets invoked with the TEE as > bl32 and main u-boot as bl33. Once it has done its startup TF-A jumps > into the bl32(OP-TEE) for the TEE startup, returns to TF-A and then > jumps to bl33(U-Boot). > > For this boot flow on IMX8MM you need to do the following: > - build TF-A (bl31.bin) with OP-TEE as a Secure Payload Dispatcher > (SPD) (add 'SPD=opteed' to env) > - build OP-TEE (tee.bin) > - add tee.bin to U-Boot FIT image along with TF-A binary in images node > * for boards still using arch/arm/mach-imx/mkimage_fit_atf.sh FIT > generation this is done automatically when tee.bin is U-Boot's build > directory but you also need to set TEE_LOAD_ADDR env to specify the > load addr of the TEE which depends on the board DRAM size > * for boards using binman the generation of the FIT image has moved > to using the 'binman' tool thus adding tee.bin requires you to modify > your board's u-boot.dtsi itb node to add tee.bin to itb/fit/images/ > and add a reference to it in the loadables property of your boot > configuration following atf. This is shown in my patch above except > there I neglected to add the required 'tee_blob: blob-ext { filename = > "tee.bin"; }. Here you must set the TEE_LOAD_ADDR as the load address > manually. > - To enable OP-TEE support in U-Boot a /firmware/optee node needs to > be added to your board u-boot.dtsi with 'compatible = > "linaro,optee-tz"' and 'method = "smc"' nodes > * This node is used by the U-Boot dek_blob command Well, to be more precise it's just for probing optee (drivers/tee/optee) driver. It can be also used in AVB 2.0 implementation (in this case OP-TEE is used for RPMB eMMC access for storing rollback indexes or other persistent values to RPMB) > * If present then U-Boot will also copy it to your Linux dtb via > image_setup_libfdt->optee_copy_fdt_nodes > - To enable dek_blob in U-Boot enable CONFIG_CMD_DEKBLOB=y which will > enable CONFIG_IMX_OPTEE_DEK_ENCAP=y > - To enable optee driver in Linux enable CONFIG_TEE and CONFIG_OPTEE > and optionally CONFIG_HW_RANDOM_OPTEE > - Also, I believe you need to add memory reservations in the dt via a > /reserved-memory/optee_core and /reserved-memory/optee_shm which will > also be copied to the linux dtb as well if present and also depend on > the OP-TEE memory location/configuration > > If you are using NXP's imx-optee-os [2] for IMX8MM then you can use > the 'mx8mmevk' platform which simply configures the following which > can be overridden via env: > CFG_UART_BASE ?= UART2_BASE > CFG_DDR_SIZE ?= 0x80000000 > > In core/arch/arm/plat-imx/conf.mk for imx8mm CFG_DDR_SIZE is used to > compute CFG_TZDRAM_START where optee gets loaded (TEE_LOAD_ADDR): > # put optee end of ddr for AARCH64 > CFG_TZDRAM_START ?= ($(CFG_DRAM_BASE) - 0x02000000 + $(CFG_DDR_SIZE)) > CFG_TZDRAM_SIZE ?= 0x01c00000 > CFG_SHMEM_START ?= ($(CFG_TZDRAM_START) + $(CFG_TZDRAM_SIZE)) > CFG_SHMEM_SIZE ?= 0x00400000 > > thus: > CFG_DDR_SIZE = 0x40000000 (1GiB) results in > CFG_TZDRAM_START=0x7e000000 (TEE_LOAD_ADDR) and > CFG_SHMEM_START=0x7fc00000 > CFG_DDR_SIZE = 0x80000000 (2GiB) results in > CFG_TZDRAM_START=0xbe000000 (TEE_LOAD_ADDR) and > CFG_SHMEM_START=0xbfc00000 > CFG_DDR_SIZE = 0x100000000 (4GiB) results in > CFG_TZDRAM_START=0x13e000000 (TEE_LOAD_ADDR) and > CFG_SHMEM_START=0x13fc00000 > > So for a 2GiB board your dt changes for a board already using binman > should look like this: > @@ -6,6 +6,29 @@ > #include "imx8mm-u-boot.dtsi" > > / { > + firmware { > + optee { > + compatible = "linaro,optee-tz"; > + method = "smc"; > + }; > + }; > + > + reserved-memory { > + #address-cells = <2>; > + #size-cells = <2>; > + ranges; > + > + /* 32MiB */ > + optee_core@be000000 { > + reg = <0x00 0xbe000000 0x00 0x2000000>; > + }; > + > + /* 4K */ > + optee_shm@bfc00000 { > + reg = <0x00 0xbfc00000 0x00 0x400000>; > + }; > + }; > + > binman: binman { > multiple-images; > }; > @@ -152,6 +175,19 @@ > }; > }; > > + tee { > + description = "TEE firmware"; > + type = "firmware"; > + arch = "arm64"; > + compression = "none"; > + load = <0xbe000000>; > + entry = <0xbe000000>; > + > + tee_blob: blob-ext { > + filename = "tee.bin"; > + }; > + }; > + > @fdt-SEQ { > description = "NAME"; > type = "flat_dt"; > @@ -165,7 +201,7 @@ > @config-SEQ { > description = "NAME"; > firmware = "uboot"; > - loadables = "atf"; > + loadables = "atf", "tee"; > fdt = "fdt-SEQ"; > }; > }; > > I have done all of the above using imx-atf [1] and imx-optee-os [2] on > a 2GiB IMX8MM board and successfully boot to U-Boot where I verify > dek_blob works, and then successfully booting to Linux where I verify > that the Linux optee driver probes. > > But, what confuses me relates to optee memory usage and location > (TEE_LOAD_ADDR) based on testing on 1GiB and 4GiB IMX8MM boards: > - CFG_DDR_SIZE = 0x40000000 (1GiB); TEE_LOAD_ADDR=0x7e000000 > - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a > 1GiB DRAM board (why? makes no sense) > - panic's at core/arch/arm/mm/core_mmu.c <check_pa_matches_va> on a > 2GiB DRAM board (this makes sense as 0x7e000000 should be out of > bounds!) > - hangs in opteed_init on a 4GiB DRAM board (why?) > - CFG_DDR_SIZE = 0x80000000 (2GiB); TEE_LOAD_ADDR=0xbe000000 > - boots on a 1GiB DRAM board... shouldn't that crash when accessing > 0xbe000000? > - boots on a 2GiB DRAM board > - boots on a 4GiB DRAM board > CFG_DDR_SIZE = 0x100000000 (4GiB); TEE_LOAD_ADDR=0x13e000000 > - fails building: 'Error: u-boot.its:63.13-24 Value out of range for > 32-bit array element' > - also this may be an issue as currently IMX8MM boards have DRAM > limited to 3GiB due to various IP drivers not being 64bit ready > > Also, I find that the memory reservations produce an error when I TFTP > a file due to (tftp_init_load_addr -> lmb_init_and_reserve) > u-boot=> tftpboot $fdt_addr_r venice/imx8mm-venice-gw73xx-0x.dtb > Using ethernet@30be0000 device > TFTP from server 192.168.1.146; our IP address is 192.168.1.1 > Filename 'venice/imx8mm-venice-gw73xx-0x.dtb'. > ERROR: reserving fdt memory region failed (addr=bfc00000 size=400000 flags=0) > > Can anyone explain the memory configurations needed for IMX8MM OP-TEE? Well, I'd suggest just to take existing buildroot based setup with vanilla OP-TEE for Verdin iMX8MM ([1], [2], [3]), which I created ~2 years ago and just take a look what build flags are used there. I'm not sure if it's still works, as I don't have Verdin SoM at hand, however this definitely will save you a lot of time trying to build/bring up everything manually. Also there is a makefile for iMX8MQ EVK [4], which is currently maintained by Jens Wiklander from Linaro (never tried to use it, but looks like it's regularly updated, so it should just work out of the box). [1] https://github.com/OP-TEE/manifest/blob/master/verdin.xml [2] https://github.com/OP-TEE/build/blob/master/verdin.mk [3] https://optee.readthedocs.io/en/latest/building/index.html [4] https://github.com/OP-TEE/build/blob/master/imx.mk > > Best regards, > > Tim > [1] https://source.codeaurora.org/external/imx/imx-atf/ > [2] https://source.codeaurora.org/external/imx/imx-optee-os/ Hope that helps! Regards, Igor -- Best regards - Freundliche GrĂ¼sse - Meilleures salutations Igor Opaniuk Embedded Software Engineer T: +380 938364067 E: igor.opan...@foundries.io W: www.foundries.io