Hi Leo, On Thu, Apr 7, 2022 at 11:13 AM Leo Liang <ycli...@andestech.com> wrote: > > Hi Alex, > On Thu, Mar 10, 2022 at 09:03:08AM +0100, Alexandre Ghiti wrote: > > Hi Leo, > > > > On Wed, Mar 9, 2022 at 7:31 AM Leo Liang <ycli...@andestech.com> wrote: > > > > > > Hi Alex, > > > On Thu, Mar 03, 2022 at 11:06:18AM +0000, Leo Liang wrote: > > > > Hi Alex, > > > > On Tue, Mar 01, 2022 at 03:21:56AM +0000, Leo Liang wrote: > > > > > Hi Alex, > > > > > On Mon, Feb 21, 2022 at 05:42:41PM +0100, Alexandre Ghiti wrote: > > > > > > On Sat, Feb 19, 2022 at 9:52 AM Leo Liang <ycli...@andestech.com> > > > > > > wrote: > > > > > > > > > > > > > > Hi Alex, > > > > > > > On Thu, Feb 17, 2022 at 11:28:46AM +0100, Alexandre Ghiti wrote: > > > > > > > > Hi Leo, > > > > > > > > > > > > > > > > On Thu, Feb 17, 2022 at 10:25 AM Leo Liang > > > > > > > > <ycli...@andestech.com> wrote: > > > > > > > > > > > > > > > > > > Hi Alexandre, > > > > > > > > > On Fri, Jan 28, 2022 at 02:47:13PM +0100, Alexandre Ghiti > > > > > > > > > wrote: > > > > > > > > > > The following description is copied from the equivalent > > > > > > > > > > patch for the > > > > > > > > > > Linux Kernel proposed by Aurelien Jarno: > > > > > > > > > > > > > > > > > > > > From version 2.38, binutils default to ISA spec version > > > > > > > > > > 20191213. This > > > > > > > > > > means that the csr read/write (csrr*/csrw*) instructions > > > > > > > > > > and fence.i > > > > > > > > > > instruction has separated from the `I` extension, become > > > > > > > > > > two standalone > > > > > > > > > > extensions: Zicsr and Zifencei. As the kernel uses those > > > > > > > > > > instruction, > > > > > > > > > > this causes the following build failure: > > > > > > > > > > > > > > > > > > > > arch/riscv/cpu/mtrap.S: Assembler messages: > > > > > > > > > > arch/riscv/cpu/mtrap.S:65: Error: unrecognized opcode `csrr > > > > > > > > > > a0,scause' > > > > > > > > > > arch/riscv/cpu/mtrap.S:66: Error: unrecognized opcode `csrr > > > > > > > > > > a1,sepc' > > > > > > > > > > arch/riscv/cpu/mtrap.S:67: Error: unrecognized opcode `csrr > > > > > > > > > > a2,stval' > > > > > > > > > > arch/riscv/cpu/mtrap.S:70: Error: unrecognized opcode `csrw > > > > > > > > > > sepc,a0' > > > > > > > > > > > > > > > > > > > > Signed-off-by: Alexandre Ghiti > > > > > > > > > > <alexandre.gh...@canonical.com> > > > > > > > > > > --- > > > > > > > > > > arch/riscv/Makefile | 11 ++++++++++- > > > > > > > > > > 1 file changed, 10 insertions(+), 1 deletion(-) > > > > > > > > > > > > > > > > > > This patch seems to fail CI somehow. > > > > > > > > > (https://source.denx.de/u-boot/custodians/u-boot-riscv/-/pipelines/11004) > > > > > > > > > > > > > > > > > > Could you take a look at it ? > > > > > > > > > > > > > > > > I have just tried on master (commit ab8903a24db1) and it failed > > > > > > > > for > > > > > > > > the same reason, so this is not related to this patch. > > > > > > > > Nevertheless, > > > > > > > > I'll try to bisect the problem :) > > > > > > > > > > > > > > > > Thanks, > > > > > > > > > > > > > > > > Alex > > > > > > > > > > > > > > > > > > > > > > Thanks for putting the effort into it! > > > > > > > > > > > > > > AFAIK, this patch does nothing related to the error message > > > > > > > "undefined reference to `__ashldi3'" from the failed CI. > > > > > > > > > > > > > > Nonetheless, I have tried a few times myself, > > > > > > > and found that CI could pass with ab8903a24db1 but cannot pass > > > > > > > with this patch on my side. > > > > > > > https://source.denx.de/u-boot/custodians/u-boot-riscv/-/commits/staging > > > > > > > https://source.denx.de/u-boot/custodians/u-boot-riscv/-/pipelines > > > > > > > > > > > > > > > > > > > To me it is an issue with the toolchain: libgcc is missing those > > > > > > symbols. If I use an Ubuntu toolchain, it fails no matter which > > > > > > commit > > > > > > I am on (I tested as far as v2021.10). But if I use a toolchain from > > > > > > https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/, > > > > > > it works fine. > > > > > > > > > > > > What I don't understand is how you manage to have different build > > > > > > results with the same docker image: can you confirm that you use the > > > > > > same toolchains in the following builds? > > > > > > > > > > > > https://source.denx.de/u-boot/custodians/u-boot-riscv/-/jobs/393701 > > > > > > https://source.denx.de/u-boot/custodians/u-boot-riscv/-/jobs/393783 > > > > > > > > > > > > > > > > Sorry for the late reply. > > > > > I have checked the toolchain version of these two builds, > > > > > they are using the same toolchain[1] from Tom's docker image on > > > > > docker hub[2]. > > > > > > > > > > Also the fail is reproducible using this docker image with the > > > > > following commands: > > > > > > > > > > leo@host sudo docker run -it --name leo-test > > > > > trini/u-boot-gitlab-ci-runner:focal-20220113-03Feb2022 /bin/bash > > > > > uboot@356268d27bf0:~$ git clone > > > > > https://source.denx.de/u-boot/custodians/u-boot-riscv.git && cd > > > > > u-boot-riscv > > > > > uboot@356268d27bf0:~/u-boot-riscv$ git checkout staging > > > > > uboot@356268d27bf0:~/u-boot-riscv$ export > > > > > PATH=/opt/gcc-11.1.0-nolibc/riscv64-linux/bin:$PATH > > > > > uboot@356268d27bf0:~/u-boot-riscv$ export CROSS_COMPILE=riscv64-linux- > > > > > uboot@356268d27bf0:~/u-boot-riscv$ make qemu-riscv32_spl_defconfig > > > > > uboot@356268d27bf0:~/u-boot-riscv$ make -j`nproc` V=1 > > > > > > > > > > ... > > > > > > > > > > riscv64-linux-ld.bfd -m elf32lriscv --gc-sections -static -pie > > > > > -Bstatic --no-dynamic-linker -z notext --build-id=none -Ttext > > > > > 0x81200000 -o u-boot -T u-boot.lds arch/riscv/cpu/start.o > > > > > --whole-archive arch/riscv/cpu/built-in.o > > > > > arch/riscv/cpu/generic/built-in.o arch/riscv/lib/built-in.o > > > > > board/emulation/common/built-in.o > > > > > board/emulation/qemu-riscv/built-in.o boot/built-in.o > > > > > cmd/built-in.o common/built-in.o disk/built-in.o > > > > > drivers/built-in.o drivers/usb/cdns3/built-in.o > > > > > drivers/usb/common/built-in.o drivers/usb/dwc3/built-in.o > > > > > drivers/usb/emul/built-in.o drivers/usb/eth/built-in.o > > > > > drivers/usb/host/built-in.o drivers/usb/mtu3/built-in.o > > > > > drivers/usb/musb-new/built-in.o drivers/usb/musb/built-in.o > > > > > drivers/usb/phy/built-in.o drivers/usb/ulpi/built-in.o > > > > > env/built-in.o fs/built-in.o lib/built-in.o net/built-in.o > > > > > --no-whole-archive -L > > > > > /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0 > > > > > -lgcc -Map u-boot.map; true > > > > > riscv64-linux-ld.bfd: drivers/virtio/virtio-uclass.o: in function > > > > > `virtio_uclass_child_pre_probe': > > > > > /home/uboot/u-boot-riscv/drivers/virtio/virtio-uclass.c:339: > > > > > undefined reference to `__lshrdi3' > > > > > riscv64-linux-ld.bfd: > > > > > /home/uboot/u-boot-riscv/drivers/virtio/virtio-uclass.c:311: > > > > > undefined reference to `__ashldi3' > > > > > riscv64-linux-ld.bfd: > > > > > /home/uboot/u-boot-riscv/drivers/virtio/virtio-uclass.c:310: > > > > > undefined reference to `__ashldi3' > > > > > riscv64-linux-ld.bfd: drivers/nvme/nvme.o: in function `nvme_blk_rw': > > > > > /home/uboot/u-boot-riscv/drivers/nvme/nvme.c:776: undefined reference > > > > > to `__lshrdi3' > > > > > riscv64-linux-ld.bfd: fs/ext4/ext4_common.o: in function > > > > > `ext4fs_get_extent_block': > > > > > /home/uboot/u-boot-riscv/fs/ext4/ext4_common.c:1560: undefined > > > > > reference to `__ashldi3' > > > > > riscv64-linux-ld.bfd: fs/ext4/dev.o: in function `ext4fs_set_blk_dev': > > > > > /home/uboot/u-boot-riscv/fs/ext4/dev.c:46: undefined reference to > > > > > `__lshrdi3' > > > > > riscv64-linux-ld.bfd: lib/display_options.o: in function `print_size': > > > > > /home/uboot/u-boot-riscv/lib/display_options.c:103: undefined > > > > > reference to `__lshrdi3' > > > > > riscv64-linux-ld.bfd: > > > > > /home/uboot/u-boot-riscv/lib/display_options.c:102: undefined > > > > > reference to `__ashldi3' > > > > > riscv64-linux-ld.bfd: > > > > > /home/uboot/u-boot-riscv/lib/display_options.c:95: undefined > > > > > reference to `__ashldi3' > > > > > riscv64-linux-ld.bfd: > > > > > /home/uboot/u-boot-riscv/lib/display_options.c:124: undefined > > > > > reference to `__lshrdi3' > > > > > riscv64-linux-ld.bfd: lib/vsprintf.o: in function `number': > > > > > /home/uboot/u-boot-riscv/lib/vsprintf.c:213: undefined reference to > > > > > `__lshrdi3' > > > > > make: *** [Makefile:1800: u-boot] Error 1 > > > > > > > > > > Furthermore, Using the same container and perform the identical build > > > > > commands listed above on master branch produce no error. > > > > > > > > > > What seems a bit odd is that when testing qemu-riscv32_spl, > > > > > buildman still uses 64 bit toolchain for the build. > > > > > I am not sure what the effect of "riscv64-linux-ld.bfd -m > > > > > elf32lriscv" is. > > > > > > > > > > Do you have any thoughts ? > > > > > > > > > > [1] > > > > > https://mirrors.edge.kernel.org/pub/tools/crosstool/files/bin/x86_64/11.1.0/x86_64-gcc-11.1.0-nolibc-riscv64-linux.tar.xz > > > > > [2] > > > > > https://hub.docker.com/layers/trini/u-boot-gitlab-ci-runner/focal-20220113-03Feb2022/images/sha256-b1cddec69526015c550ac8e528114c976ccd1a0e75676328c4f81b834e06b2b3?context=explore > > > > > > > > > > Best regards, > > > > > Leo > > > > > > > > > > > > > I re-ran again with buildman's verbose option > > > > (./tools/buildman/buildman -o /tmp/qemu-riscv32_spl/ -w -E -W -e -V -v > > > > --board qemu-riscv32_spl) > > > > and found that the failed command is as below: > > > > > > > > riscv64-linux-ld.bfd -m elf32lriscv --gc-sections -static -pie > > > > -Bstatic --no-dynamic-linker -z notext --build-id=none -Ttext > > > > 0x81200000 -o u-boot -T u-boot.lds arch/riscv/cpu/start.o > > > > --whole-archive arch/riscv/cpu/built-in.o > > > > arch/riscv/cpu/generic/built-in.o arch/riscv/lib/built-in.o > > > > board/emulation/common/built-in.o > > > > board/emulation/qemu-riscv/built-in.o boot/built-in.o cmd/built-in.o > > > > common/built-in.o disk/built-in.o drivers/built-in.o > > > > drivers/usb/cdns3/built-in.o drivers/usb/common/built-in.o > > > > drivers/usb/dwc3/built-in.o drivers/usb/emul/built-in.o > > > > drivers/usb/eth/built-in.o drivers/usb/host/built-in.o > > > > drivers/usb/mtu3/built-in.o drivers/usb/musb-new/built-in.o > > > > drivers/usb/musb/built-in.o drivers/usb/phy/built-in.o > > > > drivers/usb/ulpi/built-in.o env/built-in.o fs/built-in.o > > > > lib/built-in.o net/built-in.o --no-whole-archive -L > > > > /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0 > > > > -lgcc -Map u-boot.map; true > > > > ^^^^^^^^^^^^^^^^^^^^^^ > > > > > > > > The reason for this to fail is due to the wrong search path for libgcc. > > > > The above command uses "-L > > > > /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0", > > > > where it should be "-L > > > > /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0/lib32/ilp32[d]". > > > > > > > > This search path is generated from Makefile:865 `PLATFORM_LIBGCC := -L > > > > $(shell dirname `$(CC) $(c_flags) -print-libgcc-file-name`) -lgcc`. > > > > > > > > I'll try to find out how buildman executes the make process. > > > > > > > > ``` > > > > $ ./tools/buildman/buildman -o /tmp/qemu-riscv32_spl/ -w -E -W -e -V -v > > > > --board qemu-riscv32_spl > > > > $ cat /tmp/qemu-riscv32_spl/log | grep u-boot.map > > > > riscv64-linux-ld.bfd -m elf32lriscv --gc-sections -static -pie > > > > -Bstatic --no-dynamic-linker -z notext --build-id=none -Ttext > > > > 0x81200000 -o u-boot -T u-boot.lds arch/riscv/cpu/start.o > > > > --whole-archive arch/riscv/cpu/built-in.o > > > > arch/riscv/cpu/generic/built-in.o arch/riscv/lib/built-in.o > > > > board/emulation/common/built-in.o > > > > board/emulation/qemu-riscv/built-in.o boot/built-in.o cmd/built-in.o > > > > common/built-in.o disk/built-in.o drivers/built-in.o > > > > drivers/usb/cdns3/built-in.o drivers/usb/common/built-in.o > > > > drivers/usb/dwc3/built-in.o drivers/usb/emul/built-in.o > > > > drivers/usb/eth/built-in.o drivers/usb/host/built-in.o > > > > drivers/usb/mtu3/built-in.o drivers/usb/musb-new/built-in.o > > > > drivers/usb/musb/built-in.o drivers/usb/phy/built-in.o > > > > drivers/usb/ulpi/built-in.o env/built-in.o fs/built-in.o > > > > lib/built-in.o net/built-in.o --no-whole-archive -L > > > > /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0 > > > > -lgcc -Map u-boot.map; true > > > > ``` > > > > > > > > > > This patch looks valid, it's the CI process which causes the error to > > > occur. > > > > > > The command "dirname $(riscv64-linux-gcc -march=rv32imac -mabi=ilp32 > > > -print-libgcc-filename)" gives > > > different result from "dirname $(riscv64-linux-gcc > > > -march=rv32imac_zicsr_zifencei -mabi=ilp32 -print-libgcc-filename)". > > > > > > The former gives > > > "/opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0/lib32/ilp32" > > > while the latter gives > > > "/opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0". > > > > > > The symbol __ashldi3 and __lshrdi3 only exist in libgcc.a under > > > /opt/gcc-11.1.0-nolibc/riscv64-linux/bin/../lib/gcc/riscv64-linux/11.1.0/lib32/ilp32, > > > so linker search path have to be set to it to link successfully. > > > > > > The multilib setting does not cover zifencei and zicsr, so we get a > > > default multilib path from -print-libgcc-file-name, and thus run into > > > error like this. > > > > > > We should be able to fix this by using 32 bit toolchain in the CI process > > > for 32 bit platform/configuration. > > > > > > Any thoughts or comments ? > > > > Let me ask internally to the guy who handles the toolchains, he may > > have an idea. > > > > Thanks for looking into that, > > > > A gentle ping. > > Any comments? > Or should we spin a patch for updating the toolchain used for 32 bit build?
Sorry for being very slow here, if the patch is too much of a burden, maybe you should drop it for now, I will have more time to work on this in 2 weeks, sorry again. Alex > > Best regards, > Leo > > > Alex > > > > > > > > Best regards, > > > Leo > > > > > > > Best regards, > > > > Leo > > > > > > > > > > > > > > > The only difference I see here is the gitlab-runner version (line > > > > > > 1). > > > > > > > > > > > > Thanks, > > > > > > > > > > > > Alex > > > > > > > > > > > > > Best regards, > > > > > > > Leo > > > > > > > > > > > > > > > > > > > > > > > > > Best regards, > > > > > > > > > Leo
