On 2023-01-26 23:16, Jonas Karlman wrote: > Hi Jagan, > On 2023-01-26 20:17, Jagan Teki wrote: >> On Fri, 27 Jan 2023 at 00:33, Jonas Karlman <jo...@kwiboo.se> wrote: >>> >>> On 2023-01-26 19:26, Jagan Teki wrote: >>>> Hi Simon, >>>> >>>> On Thu, 26 Jan 2023 at 23:34, Simon Glass <s...@chromium.org> wrote: >>>>> >>>>> Hi Jagan, >>>>> >>>>> On Thu, 26 Jan 2023 at 10:42, Jagan Teki <ja...@edgeble.ai> wrote: >>>>>> >>>>>> On Thu, 26 Jan 2023 at 22:28, Jonas Karlman <jo...@kwiboo.se> wrote: >>>>>>> >>>>>>> Hi Jagan, >>>>>>> On 2023-01-26 17:51, Jagan Teki wrote: >>>>>>>> Hi Jonas, >>>>>>>> >>>>>>>> On Thu, 26 Jan 2023 at 04:17, Jonas Karlman <jo...@kwiboo.se> wrote: >>>>>>>>> >>>>>>>>> Hi Jagan, >>>>>>>>> >>>>>>>>> On 2023-01-25 23:27, Jagan Teki wrote: >>>>>>>>>> This series support Rockchip RK3588. All the device tree files are >>>>>>>>>> synced from linux-next with the proper SHA1 mentioned in the commit >>>>>>>>>> messages. >>>>>>>>>> >>>>>>>>>> Unfortunately, the BL31 from rkbin is not compatible with U-Boot so >>>>>>>>>> it is failing to load ATF entry from SPL and hang. >>>>>>>>>> >>>>>>>>>> Verified below BL31 versions, >>>>>>>>>> bl31-v1.15 >>>>>>>>>> bl31-v1.21 >>>>>>>>>> bl31-v1.22 >>>>>>>>>> bl31-v1.23 >>>>>>>>>> bl31-v1.24 >>>>>>>>>> bl31-v1.25 >>>>>>>>>> bl31-v1.26 >>>>>>>>>> >>>>>>>>>> Rever-engineered with respect to rockchip u-boot by using the same >>>>>>>>>> FIT_GENERATOR being used in Mainline, rockchip u-boot is booting but >>>>>>>>>> mainline showing the same issue. >>>>>>>>>> >>>>>>>>>> Log: >>>>>>>>>> >>>>>>>>>> LPDDR4X, 2112MHz01-00642-g6bdfd31756-dirty (Jan 26 2023 ���3:44:34 >>>>>>>>>> +0530) >>>>>>>>>> channel[0] BW=16 Col=10 Bk=8 CS0 Row=17 CS1 Row=17 CS=2 Die BW=8 >>>>>>>>>> Size=4096MB >>>>>>>>>> channel[1] BW=16 Col=10 Bk=8 CS0 Row=17 CS1 Row=17 CS=2 Die BW=8 >>>>>>>>>> Size=4096MB >>>>>>>>>> channel[2] BW=16 Col=10 Bk=8 CS0 Row=17 CS1 Row=17 CS=2 Die BW=8 >>>>>>>>>> Size=4096MB >>>>>>>>>> channel[3] BW=16 Col=10 Bk=8 CS0 Row=17 CS1 Row=17 CS=2 Die BW=8 >>>>>>>>>> Size=4096MB >>>>>>>>>> change to F1: 528MHz >>>>>>>>>> change to F2: 1068MHz >>>>>>>>>> change to F3: 1560MHz >>>>>>>>>> change to F0: 2112MHz >>>>>>>>>> out >>>>>>>>>> >>>>>>>>>> U-Boot SPL 2023.01-00642-g6bdfd31756-dirty (Jan 26 2023 - 03:44:34 >>>>>>>>>> +0530) >>>>>>>>>> Trying to boot from MMC1 >>>>>>>>>> bl31_entry: atf_entry start >>>>>>>>>> << hang >> >>>>>>>>>> >>>>>>>>>> Any information on BL31 for RK3588 please share. >>>>>>>>> >>>>>>>>> I had a similar strange booing issue with RK3568 and mainline U-Boot, >>>>>>>>> turned out to be related to all parts of ATF not being properly loaded >>>>>>>>> into PMU SRAM. >>>>>>>>> >>>>>>>>> Using my series at [1] I managed to get ATF to be fully loaded into >>>>>>>>> PMU SRAM. Using CONFIG_SPL_FIT_SIGNATURE=y helped me finding out that >>>>>>>>> the segment being loaded ended up corrupted. >>>>>>>>> >>>>>>>>> The use of 512 bytes alignment of the FIT helped mitigate that issue. >>>>>>>>> Vendor U-Boot use a bounce buffer for all parts that is written into >>>>>>>>> SRAM (anything loaded outside the gd->ram_base to gd->ram_top range). >>>>>>>>> >>>>>>>>> You can also find newer bl31 at [2], up to version v1.32. >>>>>>>>> >>>>>>>>> [1] https://patchwork.ozlabs.org/project/uboot/list/?series=337891 >>>>>>>>> [2] >>>>>>>>> https://gitlab.com/rk3588_linux/rk/rkbin/-/tree/linux-5.10-gen-rkr3.5/bin/rk35 >>>>>>>>> Thanks for the details. I did apply this set on the master. No change >>>>>>>> in the behavior, used BL31 and ddr from [2] as well as in >>>>>>>> rkbin/master. >>>>>>> >>>>>>> I did some tests on my Radxa ROCK 3 Model B with >>>>>>> CONFIG_SPL_FIT_SIGNATURE=y >>>>>>> and it looked like it failed to read data into memory, see below. >>>>>>> >>>>>>> It also looks like the sdhci compatible is not supported by the driver. >>>>>>> Something that may need to be added to driver to properly read data? >>>>>>> >>>>>>> >>>>>>> DDR V1.09 a930779e06 typ 22/11/21-17:50:56 >>>>>>> LPDDR4X, 2112MHz >>>>>>> channel[0] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 >>>>>>> Size=2048MB >>>>>>> channel[1] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 >>>>>>> Size=2048MB >>>>>>> channel[2] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 >>>>>>> Size=2048MB >>>>>>> channel[3] BW=16 Col=10 Bk=8 CS0 Row=16 CS1 Row=16 CS=2 Die BW=16 >>>>>>> Size=2048MB >>>>>>> Manufacturer ID:0x6 >>>>>>> CH0 RX Vref:31.7%, TX Vref:21.8%,20.8% >>>>>>> CH1 RX Vref:31.7%, TX Vref:21.8%,21.8% >>>>>>> CH2 RX Vref:32.7%, TX Vref:22.8%,21.8% >>>>>>> CH3 RX Vref:32.7%, TX Vref:21.8%,20.8% >>>>>>> change to F1: 528MHz >>>>>>> change to F2: 1068MHz >>>>>>> change to F3: 1560MHz >>>>>>> change to F0: 2112MHz >>>>>>> out >>>>>>> >>>>>>> U-Boot SPL 2023.01 (Jan 26 2023 - 00:24:53 +0000) >>>>>>> Trying to boot from MMC1 >>>>>>> ## Checking hash(es) for config config_1 ... OK >>>>>>> ## Checking hash(es) for Image atf_1 ... sha256 error! >>>>>>> Bad hash value for 'hash' hash node in 'atf_1' image node >>>>>>> mmc_load_image_raw_sector: mmc block read error >>>>>>> Trying to boot from MMC1 >>>>>>> ## Checking hash(es) for config config_1 ... OK >>>>>>> ## Checking hash(es) for Image atf_1 ... sha256 error! >>>>>>> Bad hash value for 'hash' hash node in 'atf_1' image node >>>>>> >>>>>> Look like this is something wrong with your patch series or master >>>>>> changes on binman, not with the driver. I have observed the same if I >>>>>> enable CONFIG_SPL_FIT_SIGNATURE. >>>>> >>>>> There are some more changes in dm/master that I'm about to pull in. >>>>> One of them from Jonas Karlman adds hash nodes so may be involved. >>>> >>>> I found the same issue on the dm/master >>>> >>>> U-Boot SPL 2023.01-00176-gb21fb7a9c0 (Jan 26 2023 - 23:55:11 +0530) >>>> Trying to boot from MMC1 >>>> ## Checking hash(es) for config config-1 ... OK >>>> ## Checking hash(es) for Image atf-1 ... sha256 error! >>>> Bad hash value for 'hash' hash node in 'atf-1' image node >>>> mmc_load_image_raw_sector: mmc block read error >>>> SPL: failed to boot from all boot devices >>>> ### ERROR ### Please RESET the board ### >>> >>> On my RK3568 ROCK 3A board this is working correctly, see below. >>> >>> This was using u-boot master 17e8e58fe62c019b2cc26af221b6defc3368229f >>> with a few patches on top, see [1]. >>> >>> ------------- >>> U-Boot SPL 2023.01 (Jan 26 2023 - 18:29:56 +0000) >>> Trying to boot from MMC1 >>> ## Checking hash(es) for config config-1 ... OK >>> ## Checking hash(es) for Image atf-1 ... sha256+ OK >>> ## Checking hash(es) for Image u-boot ... sha256+ OK >>> ## Checking hash(es) for Image fdt-1 ... sha256+ OK >>> ## Checking hash(es) for Image atf-2 ... sha256+ OK >>> ## Checking hash(es) for Image atf-3 ... sha256+ OK >>> ## Checking hash(es) for Image atf-4 ... sha256+ OK >>> ## Checking hash(es) for Image atf-5 ... sha256+ OK >>> ## Checking hash(es) for Image atf-6 ... sha256+ OK >>> INFO: Preloader serial: 2 >>> NOTICE: BL31: v2.3():v2.3-460-g2c8be93f9:huan.he >>> NOTICE: BL31: Built : 19:45:17, Nov 8 2022 >>> INFO: GICv3 without legacy support detected. >>> INFO: ARM GICv3 driver initialized in EL3 >>> INFO: pmu v1 is valid 220114 >>> INFO: dfs DDR fsp_param[0].freq_mhz= 1560MHz >>> INFO: dfs DDR fsp_param[1].freq_mhz= 324MHz >>> INFO: dfs DDR fsp_param[2].freq_mhz= 528MHz >>> INFO: dfs DDR fsp_param[3].freq_mhz= 780MHz >>> INFO: Using opteed sec cpu_context! >>> INFO: boot cpu mask: 0 >>> INFO: BL31: Initializing runtime services >>> WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE >>> initialization. SMC`s destined for OPTEE will return SMC_UNK >>> ERROR: Error initializing runtime service opteed_fast >>> INFO: BL31: Preparing for EL3 exit to normal world >>> INFO: Entry point address = 0xa00000 >>> INFO: SPSR = 0x3c9 >>> >>> >>> U-Boot 2023.01 (Jan 26 2023 - 18:29:56 +0000) >>> >>> Model: Rockchip RK3568 EVB1 DDR4 V10 Board >>> DRAM: 8 GiB (effective 7.7 GiB) >>> Core: 58 devices, 17 uclasses, devicetree: separate >>> MMC: mmc@fe2b0000: 1, mmc@fe310000: 0 >>> ... >>> ------------- >>> >>> However, on my RK3588 ROCK 5B device I get a similar error you get. >>> Seems to be the mmc reading that times out all of sudden. >>> >>> It can read the FIP and config, and then some mmc command/transfer times >>> out. >>> See below for details with LOG_DEBUG defined at top of include/log.h >>> >>> This was trying to boot from a SD-card, trying to use eMMC fails earlier. >>> Because there is no driver for emmc compatible, SPL only tries to boot from >>> SD-card. >>> >>> I believe this is related to your << hang >>, reading atf >>> from mmc fails in the background and there is no error message. >>> >>> That is why I suggested trying with CONFIG_SPL_FIT_SIGNATURE=y, >>> to see if there is any hidden issue trying to load ATF. >>> >>> ------------- >>> Trying to boot from MMC1 >>> ## Checking hash(es) for config config-1 ... OK >>> mmc_load_image_raw_sector: mmc block read error >>> Trying to boot from MMC1 >>> mmc_load_image_raw_sector: mmc block read error >>> SPL: failed to boot from all boot devices >>> ### ERROR ### Please RESET the board ### >>> ------------- >>> Trying to boot from MMC1 >>> 0 >>> - 0 'mmc@fe2c0000' >>> - found >>> blk_find_device: uclass_id=67, devnum=0: m...@fe2c0000.blk, 67, 0 >>> clock is disabled (0Hz) >>> Buswidth = 0, clock: 0 >>> Buswidth = 1, clock: 0 >>> clock is enabled (400000Hz) >>> Buswidth = 1, clock: 400000 >>> Sending CMD0 >>> Sending CMD8 >>> Sending CMD55 >>> Sending CMD41 >>> Sending CMD55 >>> Sending CMD41 >>> Sending CMD2 >>> Sending CMD3 >>> Sending CMD9 >>> Sending CMD7 >>> Sending CMD55 >>> Sending CMD51 >>> Sending CMD6 >>> Sending CMD55 >>> Sending CMD6 >>> Buswidth = 4, clock: 400000 >>> Sending CMD6 >>> clock is enabled (50000000Hz) >>> Buswidth = 4, clock: 50000000 >>> clk_set_rate(clk=500000, rate=50000000) >>> rockchip_dwmmc_get_mmc_clk: err=-2 >>> spl: mmc boot mode: raw >>> blk_find_device: uclass_id=67, devnum=0: m...@fe2c0000.blk, 67, 0 >>> Sending CMD16 >>> Sending CMD17 >>> hdr read sector 4000, count=1 >>> Found FIT >>> size=a00, ptr=ac0, limit=100000: aligned to 5000c0 >>> blk_find_device: uclass_id=67, devnum=0: m...@fe2c0000.blk, 67, 0 >>> Sending CMD16 >>> Sending CMD18 >>> Sending CMD12 >>> fit read sector 4000, sectors=5, dst=5000c0, count=5, size=0xa00 >>> Selecting default config 'rk3588-evb.dtb' >>> ## Checking hash(es) for config config-1 ... >>> fit_config_verify_required_keys: No signature node found: FDT_ERR_NOTFOUND >>> OK >>> firmware: 'atf-1' >>> blk_find_device: uclass_id=67, devnum=0: m...@fe2c0000.blk, 67, 0 >>> Sending CMD16 >>> Sending CMD18 >>> dwmci_data_transfer: Timeout waiting for data! >>> mmc_load_image_raw_sector: mmc block read error >>> spl: mmc boot mode: fs >>> Trying to boot from MMC1 >>> 0 >>> - 0 'mmc@fe2c0000' >>> - found >>> blk_find_device: uclass_id=67, devnum=0: m...@fe2c0000.blk, 67, 0 >>> spl: mmc boot mode: raw >>> blk_find_device: uclass_id=67, devnum=0: m...@fe2c0000.blk, 67, 0 >>> Sending CMD16 >>> dwmci_send_cmd: Response Timeout. >>> hdr read sector 4000, count=0 >>> mmc_load_image_raw_sector: mmc block read error >>> spl: mmc boot mode: fs >>> SPL: failed to boot from all boot devices >>> ### ERROR ### Please RESET the board ### >> >> I'm thinking this is fit/binman issue rather than mmc. >> >> U-Boot SPL 2023.01-00176-gb21fb7a9c0-dirty (Jan 27 2023 - 00:43:52 +0530) >> SPL malloc() before relocation used 0xbe0 bytes (2 KB) >>>> SPL: board_init_r() >> spl_init >> size=a0, ptr=a0, limit=100000: 500000 >> size=8, ptr=a8, limit=100000: 5000a0 >> fdtdec_get_addr_size_auto_parent: na=2, ns=2, >> fdtdec_get_addr_size_fixed: reg: addr=00000000x >> ofnode_read_u32_index: bus-width: x (4) >> ofnode_read_bool: non-removable: false >> ofnode_read_u32_index: fifo-depth: x (256) >> ofnode_read_bool: fifo-mode: false >> ofnode_read_bool: u-boot,spl-fifo-mode: false >> ofnode_read_u32_array: clock-freq-min-max: fdtdec_get_int_array: >> clock-freq-min-max >> get_prop_check_min_len: clock-freq-min-max >> ofnode_read_u32_index: max-frequency: x (200000000) >> clk_set_defaults(mmc@fe2c0000) >> clk_set_default_parents: could not read assigned-clock-parents for 3f60700 >> ofnode_read_prop: assigned-clock-rates: <not found> >> fdtdec_get_int: #clock-cells: x (1) >> fdtdec_get_int: #clock-cells: x (1) >> Looking for clock-controller@fd7c0000 >> Looking for clock-controller@fd7c0000 >> - checking clock-1 >> - checking clock-controller@fd7c0000 >> - result for clock-controller@fd7c0000: clock-controller@fd7c0000 (ret=0) >> - result for clock-controller@fd7c0000: clock-controller@fd7c0000 (ret=0) >> clk_of_xlate_default(clk=500000) >> clk_request(dev=3f60320, clk=500000) >> clk_set_rate(clk=500000, rate=400000) >> rockchip_dwmmc_get_mmc_clk: err=-2 >> Trying to boot from MMC1 >> 0 >> - 0 'mmc@fe2c0000' >> - found >> blk_find_device: uclass_id=67, devnum=0: m...@fe2c0000.blk, 67, 0 >> clock is disabled (0Hz) >> Buswidth = 0, clock: 0 >> Buswidth = 1, clock: 0 >> clock is enabled (400000Hz) >> Buswidth = 1, clock: 400000 >> Sending CMD0 >> Sending CMD8 >> Sending CMD55 >> Sending CMD41 >> Sending CMD55 >> Sending CMD41 >> Sending CMD55 >> Sending CMD41 >> Sending CMD2 >> Sending CMD3 >> Sending CMD9 >> Sending CMD7 >> Sending CMD55 >> Unaligned buffer length u >> size=40, ptr=100, limit=100000: aligned to 5000c0 >> Sending CMD51 >> Sending CMD6 >> Sending CMD55 >> Sending CMD6 >> Buswidth = 4, clock: 400000 >> Sending CMD6 >> clock is enabled (50000000Hz) >> Buswidth = 4, clock: 50000000 >> clk_set_rate(clk=500000, rate=50000000) >> rockchip_dwmmc_get_mmc_clk: err=-2 >> spl: mmc boot mode: raw >> blk_find_device: uclass_id=67, devnum=0: m...@fe2c0000.blk, 67, 0 >> Sending CMD16 >> Sending CMD17 >> hdr read sector 4000, count=1 >> Found FIT >> size=a00, ptr=b00, limit=100000: aligned to 500100 >> blk_find_device: uclass_id=67, devnum=0: m...@fe2c0000.blk, 67, 0 >> Sending CMD16 >> Sending CMD18 >> Sending CMD12 >> fit read sector 4000, sectors=5, dst=500100, count=5, size=0xa00 >> Selecting default config 'rk3588-edgeble-neu6a-io.dtb' >> ## Checking hash(es) for config config-1 ... >> fit_config_verify_required_keys: No signature node found: >> FDT_ERR_NOTFOUND >> OK >> firmware: 'atf-1' >> blk_find_device: uclass_id=67, devnum=0: m...@fe2c0000.blk, 67, 0 >> Sending CMD16 >> Sending CMD18 >> dwmci_data_transfer: Timeout waiting for data! >> Sending CMD12 >> External data: dst=40000, offset=ad200, size=2e940 >> ## Checking hash(es) for Image atf-1 ... >> fit_image_verify_required_sigs: No signature node found: >> FDT_ERR_NOTFOUND >> sha256 error! >> Bad hash value for 'hash' hash node in 'atf-1' image node >> mmc_load_image_raw_sector: mmc block read error >> spl: mmc boot mode: fs >> SPL: failed to boot from all boot devices >> ### ERROR ### Please RESET the board ### > > As you can see in the logs above there is timeout waiting for data. > > I managed to find the issue and have a workaround that gets me longer > in the boot process, there still seem to be other issue with the rk3588 > startup. > > -------- > U-Boot SPL 2023.01 (Jan 26 2023 - 22:03:00 +0000) > Trying to boot from MMC1 > ## Checking hash(es) for config config-1 ... OK > ## Checking hash(es) for Image atf-1 ... sha256+ OK > ## Checking hash(es) for Image u-boot ... sha256+ OK > ## Checking hash(es) for Image fdt-1 ... sha256+ OK > ## Checking hash(es) for Image atf-2 ... sha256+ OK > ## Checking hash(es) for Image atf-3 ... sha256+ OK > INFO: Preloader serial: 2 > NOTICE: BL31: v2.3():v2.3-468-ge529a2760:derrick.huang > NOTICE: BL31: Built : 09:59:49, Nov 21 2022 > INFO: spec: 0x1 > INFO: ext 32k is not valid > INFO: ddr: stride-en 4CH > INFO: GICv3 without legacy support detected. > INFO: ARM GICv3 driver initialized in EL3 > INFO: valid_cpu_msk=0xff bcore0_rst = 0x0, bcore1_rst = 0x0 > INFO: system boots from cpu-hwid-0 > INFO: idle_st=0x21fff, pd_st=0x11fff9, repair_st=0xfff70001 > INFO: dfs DDR fsp_params[0].freq_mhz= 2112MHz > INFO: dfs DDR fsp_params[1].freq_mhz= 528MHz > INFO: dfs DDR fsp_params[2].freq_mhz= 1068MHz > INFO: dfs DDR fsp_params[3].freq_mhz= 1560MHz > INFO: BL31: Initialising Exception Handling Framework > INFO: BL31: Initializing runtime services > WARNING: No OPTEE provided by BL2 boot loader, Booting device without OPTEE > initialization. SMC`s destined for OPTEE will return SMC_UNK > ERROR: Error initializing runtime service opteed_fast > INFO: BL31: Preparing for EL3 exit to normal world > INFO: Entry point address = 0xa00000 > INFO: SPSR = 0x3c9 > "Synchronous Abort" handler, esr 0x96000000 > elr: 0000000000a23650 lr : 0000000000a24d9c > x0 : 0000000000b7fbe8 x1 : 350003402a0003f3 > x2 : 0000000000000000 x3 : 0000000000b80ff0 > x4 : 0000000000b80ff0 x5 : 0000000000b80e88 > x6 : 0000000000000054 x7 : 0000000000000044 > x8 : 000000000000000a x9 : 0000000000000000 > x10: 0000000000000034 x11: 0000000000000002 > x12: 0000000000001988 x13: 0000000000b7fadc > x14: 0000000000a7e808 x15: 0000000000a7e808 > x16: 0000000000000000 x17: 0000000000000000 > x18: 0000000000b7fe50 x19: 0000000000b7fbe8 > x20: 000000003c14dc00 x21: 000000003c14dc00 > x22: 0000000000a7e808 x23: 0000000000000000 > x24: 0000000000000000 x25: 0000000000000000 > x26: 0000000000000000 x27: 0000000000000000 > x28: 0000000000000000 x29: 0000000000b7fb80 > > Code: f90013f5 f9400001 b4000201 f9400021 (f9403435) > Resetting CPU ... > -------- > > This was running on top of u-boot-dm/master > 060a65e899859dcbf42049a18be20ce7118e7c0e > with some rk3568 patches and this series, see [1]. > > The last 3 commits contains workaround to issue with sdmmc clock. > dwmmc driver set sclk to (uint)-2, my workaround just adds a > fallback to default 400khz clock. > > Next issue is the sync abort, looks it happens when u-boot > tries to set clock rates based on devicetree. this is the > last debug line before the crash. > > clk_set_rate(clk=0000000000b7fba8, rate=1008000000)
With the commit at [2] I can now successfully run U-Boot proper. Source of the two main issues to get this series to run have been the scmi clocks. Vendor u-boot first load its scmi driver before trying to set the cpu clocks. We can just remove it and leave setting a faster cpu rate to linux. I also noticed that my sdram size series only detect the first two channels of memory, will respin a v2 of that series to add detection of all 4 channels of memory. [2] https://github.com/Kwiboo/u-boot-rockchip/commit/7e4fee945e725c0ec9ef3905c41ce367e77833b3 Regards, Jonas > > [1] > https://github.com/Kwiboo/u-boot-rockchip/compare/060a65e899859dcbf42049a18be20ce7118e7c0e...be21c34067d6e3a2036ce70d571700170c59d270 > > Regards, > Jonas > >> >> Please check this repo, it is on top of the dm/master >> https://github.com/edgeble/u-boot/tree/neu6-v1 >> >> Jagan. >
