Hi Dave,

This discussion about cross-compilation didn't go anywhere. We're thinking of 
talking about the cross compilation in the monthly VPP call tomorrow, now that 
latest release is done.. I want to get some agreement on how to properly do the 
cross-compilation - how feasible the qemu emulation is, how much of the current 
ebuild system should (or shouldn't) we use or do we need to do something 
completely different (that would presumably be a lot simpler).

Thanks,
Juraj

From: Juraj Linkeš <juraj.lin...@pantheon.tech>
Sent: Tuesday, December 17, 2019 10:25 AM
To: Juraj Linkeš <juraj.lin...@pantheon.tech>; dmar...@me.com; Benoit Ganne 
(bganne) <bga...@cisco.com>
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Change in vpp[master]: ebuild: Cross compilation aarch64 
Ubuntu support

I tried a more basic thing in my previous patch: 
https://gerrit.fd.io/r/c/vpp/+/21035

The patch has one file with all cross-compile arguments/parameters: 
build-data/platforms/aarch64-generic.mk.

It also modifies the ebuild system to propagate those into both VPP and 
external (dpdk, rdma and such) builds. If we don't want to modify the the 
ebuild system in this way, I can create separate make targets like Damjan did 
here: https://gerrit.fd.io/r/c/vpp/+/23153

If anyone's interested in trying it out, there's a script that builds aarch64 
dpdk and other external libs along with VPP libraries: 
build-root/scripts/aarch64-crossbuild.sh. The binaries should be in the same 
repository.

What do you think about this approach? That is, splitting the cross compilation 
from the environment setup. It seems like a good first step if we're not going 
to do the emulation builds (see my previous e-mail for reasons why I think it's 
inadequate).

Thanks,
Juraj

From: Juraj Linkeš 
<juraj.lin...@pantheon.tech<mailto:juraj.lin...@pantheon.tech>>
Sent: Wednesday, December 4, 2019 3:36 PM
To: dmar...@me.com<mailto:dmar...@me.com>; Benoit Ganne (bganne) 
<bga...@cisco.com<mailto:bga...@cisco.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] Change in vpp[master]: ebuild: Cross compilation aarch64 
Ubuntu support

I looked into this and there are some problems.

The first problem is the inability to fine tune any parameters we might want to 
for target cpu/microarchitecture (for arm, that would be building packages with 
specifics for ThunderX, McBin, Raspberry PI etc.). I'm not sure how Qemu does 
the emulation, but I doubt there's extensive support for a myriad of cpus. And 
on top of that, it's slow.

The other thing is with using the x86 version of aarch64-linux-gnu-gcc. In that 
case it's just regular cross-compilation (x86 version of aarch64-linux-gnu-gcc 
is the cross-compiler) and what we gain from these target platform containers 
is having the proper libraries that VPP build depends on. For Ubuntu I've found 
that it's actually not worth it - it's easier to have an x86 container and 
install arm dependency packages that to install x86 cross compiler inside an 
aarch64 container. Another thing I've observed (in htop) is that even when 
using the x86 version of aarch64-linux-gnu-gcc inside the aarch64 container, it 
was still going through the emulator, but maybe I did something wrong.

Damjan, you mentioned that my current patch doesn't solve anything. It 
certainly isn't a comprehensive solution, but it does one thing and that is it 
allows users to specify platform specific config args (well, at least some of 
the supported ones in build-data/platforms/<platform>.mk) which then get 
propagated to all parts of the build and it's possible to do cross compilation 
given that the environemnt has been already set up. It modifies the current 
ebuild system, but that might not be the appropriate place to do that. However, 
I don't how else would we do this.

I'm not sure what all of this means, but the docker solution is certainly 
incomplete, if not outright unsuitable. Maybe we could use containers for just 
environment setup (e.g. for Ubuntu, installing both host and target packages) 
and then could run cross-compilation in them with a solution that would do 
something like my patch (i.e. cross-compile DPDK and VPP with config args 
defined in one file).

Thoughts?
Juraj

From: Damjan Marion via Lists.Fd.Io 
<dmarion=me....@lists.fd.io<mailto:dmarion=me....@lists.fd.io>>
Sent: Thursday, October 31, 2019 1:49 PM
To: Benoit Ganne (bganne) <bga...@cisco.com<mailto:bga...@cisco.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] Change in vpp[master]: ebuild: Cross compilation aarch64 
Ubuntu support



> On 31 Oct 2019, at 13:18, Benoit Ganne (bganne) 
> <bga...@cisco.com<mailto:bga...@cisco.com>> wrote:
>
>> I was going to remain silent, but since there's now multiple people saying
>> this sounds good -- I think this sounds horrible. :)
>> To wit, it seems too complex and too much setup/overhead. I'll try and
>> look closer at this soon to see if I can feed back our local changes that
>> seem to be working.
>
> It is not that bad in my opinion [1] :
> 1) add support for multiarch (must be done once after reboot)
> ~# docker run --rm --privileged multiarch/qemu-user-static --reset 
> --persistent yes --credential yes
> 2) create your chroot (must be done once - I am sharing my homedir with my 
> chroot and same UID/GID)
> ~# docker run --name aarch64_u1804 --privileged --net host -v $HOME:$HOME -v 
> /dev:/dev -v/lib/modules:/lib/modules/host:ro -td arm64v8/ubuntu:18.04 
> /bin/bash
> ~# docker container exec aarch64_u1804 sh -c "apt -qy update && apt 
> dist-upgrade -qy && apt install -qy vim sudo make git && groupadd -g $(id 
> -rg) $USER && useradd -u $(id -ru) -g $(id -rg) -M -d $HOME -s /bin/bash 
> $USER && echo '$USER ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers && echo 
> aarch64_u1804 > /etc/debian_chroot"
> 3) compile vpp (I already checked out VPP in $HOME/src/vpp but you can 
> checkout it there too if you prefer)
> ~# docker container exec aarch64_u1804 su "$USER" -l -c "UNATTENTED=y make -C 
> src/vpp install-dep install-ext-deps pkg-deb"
> [...]
> dpkg-deb: building package 'libvppinfra-dev' in 
> '../libvppinfra-dev_20.01-rc0~538-gbb41ee925_arm64.deb'.
> dpkg-deb: building package 'vpp-dbg' in 
> '../vpp-dbg_20.01-rc0~538-gbb41ee925_arm64.deb'.
> dpkg-deb: building package 'libvppinfra' in 
> '../libvppinfra_20.01-rc0~538-gbb41ee925_arm64.deb'.
> dpkg-deb: building package 'vpp-api-python' in 
> '../vpp-api-python_20.01-rc0~538-gbb41ee925_arm64.deb'.
> dpkg-deb: building package 'vpp' in 
> '../vpp_20.01-rc0~538-gbb41ee925_arm64.deb'.
> dpkg-deb: building package 'vpp-plugin-dpdk' in 
> '../vpp-plugin-dpdk_20.01-rc0~538-gbb41ee925_arm64.deb'.
> dpkg-deb: building package 'python3-vpp-api' in 
> '../python3-vpp-api_20.01-rc0~538-gbb41ee925_arm64.deb'.
> dpkg-deb: building package 'vpp-dev' in 
> '../vpp-dev_20.01-rc0~538-gbb41ee925_arm64.deb'.
> dpkg-deb: building package 'vpp-plugin-core' in 
> '../vpp-plugin-core_20.01-rc0~538-gbb41ee925_arm64.deb'.
> make[2]: Leaving directory 
> '/home/bganne/src/vpp/build-root/build-vpp-native/vpp'
> dpkg-genbuildinfo --build=binary
> dpkg-genchanges --build=binary >../vpp_20.01-rc0~538-gbb41ee925_arm64.changes
> dpkg-genchanges: info: binary-only upload (no source code included)
> dpkg-source --after-build vpp
> dpkg-buildpackage: info: binary-only upload (no source included)
> make[1]: Leaving directory '/home/bganne/src/vpp/build-root'
> make: Leaving directory '/home/bganne/src/vpp'
>
> Enjoy :)

This is so cool :) I wanted to invest time to investigate how to do exactly 
this, so thanks for saving my time

Only nice thing will be to find way to run x86 version of  
aarch64-linux-gnu-gcc instead of aarch64 version of gcc so it runs faster...


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15372): https://lists.fd.io/g/vpp-dev/message/15372
Mute This Topic: https://lists.fd.io/mt/39781289/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Reply via email to