Re: RFR (S) 8208665: Amend cross-compilation docs with qemu-debootstrap recipe

[Erik Joelsson]

Looks good to me.

/Erik


On 2018-08-23 03:11, Aleksey Shipilev wrote:

Hi,

Updated webrev, now with regenerated building.html (thanks Erik J.!):
   http://cr.openjdk.java.net/~shade/8208665/webrev.04/

There are also a few formatting changes to make it render correctly.


On 08/14/2018 10:24 PM, John Paul Adrian Glaubitz wrote:

Thanks for this. This is actually a clever idea and should also work with
Debian Ports, i.e. the non-release architectures in Debian such as sparc64,
m68k and sh4. [...]

Yes, it also does not require you to mess with any virtualization during the 
actual build. For all I
know, we can package that chroot into the tarball and seed build machines with 
it. Or even share it
over NFS/CIFS?


I should give this a shot and if it works, it might be a good idea to add
the instructions for Debian Ports as well. Since I have some more patches
for m68k and co in the pipeline, I can actually test it for that ;-).

Yup, please do. I left the placeholder mention that Debian Ports might work for 
other architectures.


With current versions of qemu (either from git or Debian unstable), you should
even be able to build OpenJDK natively inside the emulated chroot. In Debian
Ports, we're heavily making use of qemu-user and qemu-system (for riscv64) to
build packages which hashelped finding and fixing tons of bugs in qemu (I have
alone reported countless bugs ;-)).

Oh yes, QEMU with OpenJDK was broken for a long time. Anyhow, people around me 
who used to build/run
OpenJDK in QEMU report substantial build performance losses. The trick with 
qemu-deboostrap
basically gives you native compilation performance, because you still use the 
build compiler to do
most of the compilation work.

-Aleksey

Re: RFR (S) 8208665: Amend cross-compilation docs with qemu-debootstrap recipe

[Andrew Haley]

On 08/14/2018 09:24 PM, John Paul Adrian Glaubitz wrote:
> With current versions of qemu (either from git or Debian unstable), you should
> even be able to build OpenJDK natively inside the emulated chroot.

Gosh. Does that mean that, after all these years, they've fixed
the bug that delivered segfault signals to the wrong thread?
:-)

-- 
Andrew Haley
Java Platform Lead Engineer
Red Hat UK Ltd. 
EAC8 43EB D3EF DB98 CC77 2FAD A5CD 6035 332F A671

Re: RFR (S) 8208665: Amend cross-compilation docs with qemu-debootstrap recipe

[Aleksey Shipilev]

Hi,

Updated webrev, now with regenerated building.html (thanks Erik J.!):
  http://cr.openjdk.java.net/~shade/8208665/webrev.04/

There are also a few formatting changes to make it render correctly.


On 08/14/2018 10:24 PM, John Paul Adrian Glaubitz wrote:
> Thanks for this. This is actually a clever idea and should also work with
> Debian Ports, i.e. the non-release architectures in Debian such as sparc64,
> m68k and sh4. [...]
Yes, it also does not require you to mess with any virtualization during the 
actual build. For all I
know, we can package that chroot into the tarball and seed build machines with 
it. Or even share it
over NFS/CIFS?

> I should give this a shot and if it works, it might be a good idea to add
> the instructions for Debian Ports as well. Since I have some more patches
> for m68k and co in the pipeline, I can actually test it for that ;-).

Yup, please do. I left the placeholder mention that Debian Ports might work for 
other architectures.

> With current versions of qemu (either from git or Debian unstable), you should
> even be able to build OpenJDK natively inside the emulated chroot. In Debian
> Ports, we're heavily making use of qemu-user and qemu-system (for riscv64) to
> build packages which hashelped finding and fixing tons of bugs in qemu (I have
> alone reported countless bugs ;-)).

Oh yes, QEMU with OpenJDK was broken for a long time. Anyhow, people around me 
who used to build/run
OpenJDK in QEMU report substantial build performance losses. The trick with 
qemu-deboostrap
basically gives you native compilation performance, because you still use the 
build compiler to do
most of the compilation work.

-Aleksey

Re: RFR (S) 8208665: Amend cross-compilation docs with qemu-debootstrap recipe

[John Paul Adrian Glaubitz]

Hi Aleksey!

On 08/13/2018 12:26 PM, Aleksey Shipilev wrote:
> This is the recipe I have been using for creating artifacts on my personal CI 
> server [1], and it
> seems to work reliably starting from jdk11. It is partially applicable for 
> building jdk{8,9,10}, but
> freetype and friends still need to be pointed out explicitly there. 
> Nevertheless, this seems to be
> the most straightforward way to cross-compile to foreign architectures if 
> there are no devkit
> bundles available.

Thanks for this. This is actually a clever idea and should also work with
Debian Ports, i.e. the non-release architectures in Debian such as sparc64,
m68k and sh4.

For Debian Ports, there is only sid/unstable though and the FTP server is
a different one. Also, you need to include the package 
debian-ports-archive-keyring:

+  sudo qemu-debootstrap --arch=m68k --verbose \
+
--include=fakeroot,build-essential,libx11-dev,libxext-dev,libxrender-dev,libxtst-dev,libxt-dev,libcups2-dev,libfontconfig1-dev,libasound2-dev,libfreetype6-dev,libpng12-dev,debian-ports-archive-keyring
\
+   --resolve-deps sid /chroots/m68k 
http://ftp.ports.debian.org/debian-ports/

I should give this a shot and if it works, it might be a good idea to add
the instructions for Debian Ports as well. Since I have some more patches
for m68k and co in the pipeline, I can actually test it for that ;-).

With current versions of qemu (either from git or Debian unstable), you should
even be able to build OpenJDK natively inside the emulated chroot. In Debian
Ports, we're heavily making use of qemu-user and qemu-system (for riscv64) to
build packages which hashelped finding and fixing tons of bugs in qemu (I have
alone reported countless bugs ;-)).

Adrian

-- 
 .''`.  John Paul Adrian Glaubitz
: :' :  Debian Developer - glaub...@debian.org
`. `'   Freie Universitaet Berlin - glaub...@physik.fu-berlin.de
  `-GPG: 62FF 8A75 84E0 2956 9546  0006 7426 3B37 F5B5 F913

Re: RFR (S) 8208665: Amend cross-compilation docs with qemu-debootstrap recipe

[Aleksey Shipilev]

On 08/13/2018 05:16 PM, Martin Buchholz wrote:
> Aleksey, your use of "base" platform seems a bit unusual.  Elsewhere in the 
> same document, it's
> referred to as "build".  Otherwise looks good (thanks for documenting).

Right. Got mixed up with Debian's "base system". Fixed here:
 http://cr.openjdk.java.net/~shade/8208665/webrev.03/

-Aleksey

Re: RFR (S) 8208665: Amend cross-compilation docs with qemu-debootstrap recipe

[Martin Buchholz]

Aleksey, your use of "base" platform seems a bit unusual.  Elsewhere in the
same document, it's referred to as "build".  Otherwise looks good (thanks
for documenting).

On Mon, Aug 13, 2018 at 3:26 AM, Aleksey Shipilev  wrote:

> RFE:
>   https://bugs.openjdk.java.net/browse/JDK-8208665
>
> Webrev:
>   http://cr.openjdk.java.net/~shade/8208665/webrev.02/
>
> Not sure if building.html is supposed to be generated automatically?
>
> This is the recipe I have been using for creating artifacts on my personal
> CI server [1], and it
> seems to work reliably starting from jdk11. It is partially applicable for
> building jdk{8,9,10}, but
> freetype and friends still need to be pointed out explicitly there.
> Nevertheless, this seems to be
> the most straightforward way to cross-compile to foreign architectures if
> there are no devkit
> bundles available.
>
> Thanks,
> -Aleksey
>
> [1] https://builds.shipilev.net/
>
>

RFR (S) 8208665: Amend cross-compilation docs with qemu-debootstrap recipe

[Aleksey Shipilev]

RFE:
  https://bugs.openjdk.java.net/browse/JDK-8208665

Webrev:
  http://cr.openjdk.java.net/~shade/8208665/webrev.02/

Not sure if building.html is supposed to be generated automatically?

This is the recipe I have been using for creating artifacts on my personal CI 
server [1], and it
seems to work reliably starting from jdk11. It is partially applicable for 
building jdk{8,9,10}, but
freetype and friends still need to be pointed out explicitly there. 
Nevertheless, this seems to be
the most straightforward way to cross-compile to foreign architectures if there 
are no devkit
bundles available.

Thanks,
-Aleksey

[1] https://builds.shipilev.net/