Re: Restrictions of libnet (was: Re: VW ELF loader)

[David Gibson]

On Mon, Feb 10, 2020 at 10:39:52AM +0100, Michal Suchánek wrote:
> On Mon, Feb 10, 2020 at 06:55:16PM +1100, David Gibson wrote:
> > On Wed, Feb 05, 2020 at 07:24:04AM +0100, Thomas Huth wrote:
> > > On 05/02/2020 06.30, David Gibson wrote:
> > > > On Tue, Feb 04, 2020 at 10:20:14AM +0100, Thomas Huth wrote:
> > > >> On 04/02/2020 09.54, Cornelia Huck wrote:
> > > >>> On Tue, 4 Feb 2020 07:16:46 +0100
> > > >>> Thomas Huth  wrote:
> > > >>>
> > >  On 04/02/2020 00.26, Paolo Bonzini wrote:
> > > >
> > > >
> > > > Il mar 4 feb 2020, 00:20 Alexey Kardashevskiy  > > > > ha scritto:
> > > >
> > > > Speaking seriously, what would I put into the guest?
> > > >
> > > > Only things that would be considered drivers. Ignore the partitions
> > > > issue for now so that you can just pass the device tree services to 
> > > > QEMU
> > > > with hypercalls.
> > > >
> > > > Netboot's dhcp/tftp/ip/ipv6 client? It is going to be another 
> > > > SLOF,
> > > > smaller but adhoc with only a couple of people knowing it.
> > > >
> > > >
> > > > You can generalize and reuse the s390 code. All you have to write 
> > > > is the
> > > > PCI scan and virtio-pci setup.  
> > > 
> > >  Well, for netbooting, the s390-ccw bios uses the libnet code from 
> > >  SLOF,
> > >  so re-using this for a slim netboot client on ppc64 would certainly 
> > >  be
> > >  feasible (especially since there are also already virtio drivers in 
> > >  SLOF
> > >  that are written in C), but I think it is not very future proof. The
> > >  libnet from SLOF only supports UDP, and no TCP. So for advanced boot
> > >  scenarios like booting from HTTP or even HTTPS, you need something 
> > >  else
> > >  (i.e. maybe grub is the better option, indeed).
> > > >>>
> > > >>> That makes me wonder what that means for s390: We're inheriting
> > > >>> libnet's limitations, but we don't have grub -- do we need to come up
> > > >>> with something different? Or improve libnet?
> > > >>
> > > >> I don't think that it makes sense to re-invent the wheel yet another
> > > >> time and write yet another TCP implementation (which is likely quite a
> > > >> bit of work, too, especially if you also want to do secure HTTPS in the
> > > >> end). So yes, in the long run (as soon as somebody seriously asks for
> > > >> HTTP booting on s390x) we need something different here.
> > > >>
> > > >> Now looking at our standard s390x bootloader zipl - this has been 
> > > >> giving
> > > >> us a headache a couple of times in the past, too (from a distro point 
> > > >> of
> > > >> view since s390x is the only major platform left that does not use 
> > > >> grub,
> > > >> but also from a s390-ccw bios point of view, see e.g.
> > > >> https://lists.gnu.org/archive/html/qemu-devel/2019-12/msg03046.html and
> > > >> related discussions).
> > > >>
> > > >> So IMHO the s390x world should move towards grub2, too. We could e.g.
> > > >> link it initially into the s390-ccw bios bios ... and if that works out
> > > >> well, later also use it as normal bootloader instead of zipl (not sure
> > > >> if that works in all cases, though, IIRC there were some size
> > > >> constraints and stuff like that).
> > > > 
> > > > petitboot would be another reasonable thing to consider here.  Since
> > > > it's Linux based, you have all the drivers you have there.  It's not
> > > > quite grub, but it does at least parse the same configuration files.
> > > > 
> > > > You do need kexec() of course, I don't know if you have that already
> > > > for s390 or not.
> > > 
> > > AFAIK we have kexec on s390. So yes, petitboot would be another option
> > > for replacing the s390-ccw bios. But when it comes to LPARs and z/VMs, I
> > > don't think it's really feasible to replace the zipl bootloader there
> > > with petitboot, so in that case grub2 still sounds like the better
> > > option to me.
> > 
> > Actually, between that and Paolo's suggestions, I thought of another
> > idea that could be helpful for both s390 and power.  Could we load
> > non-kexec() things (legacy kernels, non-Linux OSes) from Petitboot by
> > having it kexec() into a shim mini-kernel that just sets up the boot
> > environment for the other thing.
> > 
> > What I'm imagining is that petitboot loads everything that will be
> > needed for the other OS into RAM - probably as (or part of) the
> > "initrd" image.  That means the shim doesn't need to have drivers or
> > a network stack to load that in.  It just needs to construct
> > environment and jump into the real kernel.
> 
> How does that differ from what kexec normally does?
> 
> It does support the kernel format that is usually booted on the
> architecture.

It's not a question of format, but of environment.

By the time a kexec() occurs there won't be any OF client interface
available, whether or not there ever was one.  So, kexec() won't 

Re: Restrictions of libnet (was: Re: VW ELF loader)

[Michal Suchánek]

On Mon, Feb 10, 2020 at 06:55:16PM +1100, David Gibson wrote:
> On Wed, Feb 05, 2020 at 07:24:04AM +0100, Thomas Huth wrote:
> > On 05/02/2020 06.30, David Gibson wrote:
> > > On Tue, Feb 04, 2020 at 10:20:14AM +0100, Thomas Huth wrote:
> > >> On 04/02/2020 09.54, Cornelia Huck wrote:
> > >>> On Tue, 4 Feb 2020 07:16:46 +0100
> > >>> Thomas Huth  wrote:
> > >>>
> >  On 04/02/2020 00.26, Paolo Bonzini wrote:
> > >
> > >
> > > Il mar 4 feb 2020, 00:20 Alexey Kardashevskiy  > > > ha scritto:
> > >
> > > Speaking seriously, what would I put into the guest?
> > >
> > > Only things that would be considered drivers. Ignore the partitions
> > > issue for now so that you can just pass the device tree services to 
> > > QEMU
> > > with hypercalls.
> > >
> > > Netboot's dhcp/tftp/ip/ipv6 client? It is going to be another 
> > > SLOF,
> > > smaller but adhoc with only a couple of people knowing it.
> > >
> > >
> > > You can generalize and reuse the s390 code. All you have to write is 
> > > the
> > > PCI scan and virtio-pci setup.  
> > 
> >  Well, for netbooting, the s390-ccw bios uses the libnet code from SLOF,
> >  so re-using this for a slim netboot client on ppc64 would certainly be
> >  feasible (especially since there are also already virtio drivers in 
> >  SLOF
> >  that are written in C), but I think it is not very future proof. The
> >  libnet from SLOF only supports UDP, and no TCP. So for advanced boot
> >  scenarios like booting from HTTP or even HTTPS, you need something else
> >  (i.e. maybe grub is the better option, indeed).
> > >>>
> > >>> That makes me wonder what that means for s390: We're inheriting
> > >>> libnet's limitations, but we don't have grub -- do we need to come up
> > >>> with something different? Or improve libnet?
> > >>
> > >> I don't think that it makes sense to re-invent the wheel yet another
> > >> time and write yet another TCP implementation (which is likely quite a
> > >> bit of work, too, especially if you also want to do secure HTTPS in the
> > >> end). So yes, in the long run (as soon as somebody seriously asks for
> > >> HTTP booting on s390x) we need something different here.
> > >>
> > >> Now looking at our standard s390x bootloader zipl - this has been giving
> > >> us a headache a couple of times in the past, too (from a distro point of
> > >> view since s390x is the only major platform left that does not use grub,
> > >> but also from a s390-ccw bios point of view, see e.g.
> > >> https://lists.gnu.org/archive/html/qemu-devel/2019-12/msg03046.html and
> > >> related discussions).
> > >>
> > >> So IMHO the s390x world should move towards grub2, too. We could e.g.
> > >> link it initially into the s390-ccw bios bios ... and if that works out
> > >> well, later also use it as normal bootloader instead of zipl (not sure
> > >> if that works in all cases, though, IIRC there were some size
> > >> constraints and stuff like that).
> > > 
> > > petitboot would be another reasonable thing to consider here.  Since
> > > it's Linux based, you have all the drivers you have there.  It's not
> > > quite grub, but it does at least parse the same configuration files.
> > > 
> > > You do need kexec() of course, I don't know if you have that already
> > > for s390 or not.
> > 
> > AFAIK we have kexec on s390. So yes, petitboot would be another option
> > for replacing the s390-ccw bios. But when it comes to LPARs and z/VMs, I
> > don't think it's really feasible to replace the zipl bootloader there
> > with petitboot, so in that case grub2 still sounds like the better
> > option to me.
> 
> Actually, between that and Paolo's suggestions, I thought of another
> idea that could be helpful for both s390 and power.  Could we load
> non-kexec() things (legacy kernels, non-Linux OSes) from Petitboot by
> having it kexec() into a shim mini-kernel that just sets up the boot
> environment for the other thing.
> 
> What I'm imagining is that petitboot loads everything that will be
> needed for the other OS into RAM - probably as (or part of) the
> "initrd" image.  That means the shim doesn't need to have drivers or
> a network stack to load that in.  It just needs to construct
> environment and jump into the real kernel.

How does that differ from what kexec normally does?

It does support the kernel format that is usually booted on the
architecture.

Thanks

Michal

Re: Restrictions of libnet (was: Re: VW ELF loader)

[David Gibson]

On Wed, Feb 05, 2020 at 07:24:04AM +0100, Thomas Huth wrote:
> On 05/02/2020 06.30, David Gibson wrote:
> > On Tue, Feb 04, 2020 at 10:20:14AM +0100, Thomas Huth wrote:
> >> On 04/02/2020 09.54, Cornelia Huck wrote:
> >>> On Tue, 4 Feb 2020 07:16:46 +0100
> >>> Thomas Huth  wrote:
> >>>
>  On 04/02/2020 00.26, Paolo Bonzini wrote:
> >
> >
> > Il mar 4 feb 2020, 00:20 Alexey Kardashevskiy  > > ha scritto:
> >
> > Speaking seriously, what would I put into the guest?
> >
> > Only things that would be considered drivers. Ignore the partitions
> > issue for now so that you can just pass the device tree services to QEMU
> > with hypercalls.
> >
> > Netboot's dhcp/tftp/ip/ipv6 client? It is going to be another SLOF,
> > smaller but adhoc with only a couple of people knowing it.
> >
> >
> > You can generalize and reuse the s390 code. All you have to write is the
> > PCI scan and virtio-pci setup.  
> 
>  Well, for netbooting, the s390-ccw bios uses the libnet code from SLOF,
>  so re-using this for a slim netboot client on ppc64 would certainly be
>  feasible (especially since there are also already virtio drivers in SLOF
>  that are written in C), but I think it is not very future proof. The
>  libnet from SLOF only supports UDP, and no TCP. So for advanced boot
>  scenarios like booting from HTTP or even HTTPS, you need something else
>  (i.e. maybe grub is the better option, indeed).
> >>>
> >>> That makes me wonder what that means for s390: We're inheriting
> >>> libnet's limitations, but we don't have grub -- do we need to come up
> >>> with something different? Or improve libnet?
> >>
> >> I don't think that it makes sense to re-invent the wheel yet another
> >> time and write yet another TCP implementation (which is likely quite a
> >> bit of work, too, especially if you also want to do secure HTTPS in the
> >> end). So yes, in the long run (as soon as somebody seriously asks for
> >> HTTP booting on s390x) we need something different here.
> >>
> >> Now looking at our standard s390x bootloader zipl - this has been giving
> >> us a headache a couple of times in the past, too (from a distro point of
> >> view since s390x is the only major platform left that does not use grub,
> >> but also from a s390-ccw bios point of view, see e.g.
> >> https://lists.gnu.org/archive/html/qemu-devel/2019-12/msg03046.html and
> >> related discussions).
> >>
> >> So IMHO the s390x world should move towards grub2, too. We could e.g.
> >> link it initially into the s390-ccw bios bios ... and if that works out
> >> well, later also use it as normal bootloader instead of zipl (not sure
> >> if that works in all cases, though, IIRC there were some size
> >> constraints and stuff like that).
> > 
> > petitboot would be another reasonable thing to consider here.  Since
> > it's Linux based, you have all the drivers you have there.  It's not
> > quite grub, but it does at least parse the same configuration files.
> > 
> > You do need kexec() of course, I don't know if you have that already
> > for s390 or not.
> 
> AFAIK we have kexec on s390. So yes, petitboot would be another option
> for replacing the s390-ccw bios. But when it comes to LPARs and z/VMs, I
> don't think it's really feasible to replace the zipl bootloader there
> with petitboot, so in that case grub2 still sounds like the better
> option to me.

Actually, between that and Paolo's suggestions, I thought of another
idea that could be helpful for both s390 and power.  Could we load
non-kexec() things (legacy kernels, non-Linux OSes) from Petitboot by
having it kexec() into a shim mini-kernel that just sets up the boot
environment for the other thing.

What I'm imagining is that petitboot loads everything that will be
needed for the other OS into RAM - probably as (or part of) the
"initrd" image.  That means the shim doesn't need to have drivers or
a network stack to load that in.  It just needs to construct
environment and jump into the real kernel.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: Restrictions of libnet (was: Re: VW ELF loader)

[Thomas Huth]

On 05/02/2020 06.30, David Gibson wrote:
> On Tue, Feb 04, 2020 at 10:20:14AM +0100, Thomas Huth wrote:
>> On 04/02/2020 09.54, Cornelia Huck wrote:
>>> On Tue, 4 Feb 2020 07:16:46 +0100
>>> Thomas Huth  wrote:
>>>
 On 04/02/2020 00.26, Paolo Bonzini wrote:
>
>
> Il mar 4 feb 2020, 00:20 Alexey Kardashevskiy  > ha scritto:
>
> Speaking seriously, what would I put into the guest?
>
> Only things that would be considered drivers. Ignore the partitions
> issue for now so that you can just pass the device tree services to QEMU
> with hypercalls.
>
> Netboot's dhcp/tftp/ip/ipv6 client? It is going to be another SLOF,
> smaller but adhoc with only a couple of people knowing it.
>
>
> You can generalize and reuse the s390 code. All you have to write is the
> PCI scan and virtio-pci setup.  

 Well, for netbooting, the s390-ccw bios uses the libnet code from SLOF,
 so re-using this for a slim netboot client on ppc64 would certainly be
 feasible (especially since there are also already virtio drivers in SLOF
 that are written in C), but I think it is not very future proof. The
 libnet from SLOF only supports UDP, and no TCP. So for advanced boot
 scenarios like booting from HTTP or even HTTPS, you need something else
 (i.e. maybe grub is the better option, indeed).
>>>
>>> That makes me wonder what that means for s390: We're inheriting
>>> libnet's limitations, but we don't have grub -- do we need to come up
>>> with something different? Or improve libnet?
>>
>> I don't think that it makes sense to re-invent the wheel yet another
>> time and write yet another TCP implementation (which is likely quite a
>> bit of work, too, especially if you also want to do secure HTTPS in the
>> end). So yes, in the long run (as soon as somebody seriously asks for
>> HTTP booting on s390x) we need something different here.
>>
>> Now looking at our standard s390x bootloader zipl - this has been giving
>> us a headache a couple of times in the past, too (from a distro point of
>> view since s390x is the only major platform left that does not use grub,
>> but also from a s390-ccw bios point of view, see e.g.
>> https://lists.gnu.org/archive/html/qemu-devel/2019-12/msg03046.html and
>> related discussions).
>>
>> So IMHO the s390x world should move towards grub2, too. We could e.g.
>> link it initially into the s390-ccw bios bios ... and if that works out
>> well, later also use it as normal bootloader instead of zipl (not sure
>> if that works in all cases, though, IIRC there were some size
>> constraints and stuff like that).
> 
> petitboot would be another reasonable thing to consider here.  Since
> it's Linux based, you have all the drivers you have there.  It's not
> quite grub, but it does at least parse the same configuration files.
> 
> You do need kexec() of course, I don't know if you have that already
> for s390 or not.

AFAIK we have kexec on s390. So yes, petitboot would be another option
for replacing the s390-ccw bios. But when it comes to LPARs and z/VMs, I
don't think it's really feasible to replace the zipl bootloader there
with petitboot, so in that case grub2 still sounds like the better
option to me.

 Thomas

Re: Restrictions of libnet (was: Re: VW ELF loader)

[David Gibson]

On Tue, Feb 04, 2020 at 10:20:14AM +0100, Thomas Huth wrote:
> On 04/02/2020 09.54, Cornelia Huck wrote:
> > On Tue, 4 Feb 2020 07:16:46 +0100
> > Thomas Huth  wrote:
> > 
> >> On 04/02/2020 00.26, Paolo Bonzini wrote:
> >>>
> >>>
> >>> Il mar 4 feb 2020, 00:20 Alexey Kardashevskiy  >>> > ha scritto:
> >>>
> >>> Speaking seriously, what would I put into the guest?
> >>>
> >>> Only things that would be considered drivers. Ignore the partitions
> >>> issue for now so that you can just pass the device tree services to QEMU
> >>> with hypercalls.
> >>>
> >>> Netboot's dhcp/tftp/ip/ipv6 client? It is going to be another SLOF,
> >>> smaller but adhoc with only a couple of people knowing it.
> >>>
> >>>
> >>> You can generalize and reuse the s390 code. All you have to write is the
> >>> PCI scan and virtio-pci setup.  
> >>
> >> Well, for netbooting, the s390-ccw bios uses the libnet code from SLOF,
> >> so re-using this for a slim netboot client on ppc64 would certainly be
> >> feasible (especially since there are also already virtio drivers in SLOF
> >> that are written in C), but I think it is not very future proof. The
> >> libnet from SLOF only supports UDP, and no TCP. So for advanced boot
> >> scenarios like booting from HTTP or even HTTPS, you need something else
> >> (i.e. maybe grub is the better option, indeed).
> > 
> > That makes me wonder what that means for s390: We're inheriting
> > libnet's limitations, but we don't have grub -- do we need to come up
> > with something different? Or improve libnet?
> 
> I don't think that it makes sense to re-invent the wheel yet another
> time and write yet another TCP implementation (which is likely quite a
> bit of work, too, especially if you also want to do secure HTTPS in the
> end). So yes, in the long run (as soon as somebody seriously asks for
> HTTP booting on s390x) we need something different here.
> 
> Now looking at our standard s390x bootloader zipl - this has been giving
> us a headache a couple of times in the past, too (from a distro point of
> view since s390x is the only major platform left that does not use grub,
> but also from a s390-ccw bios point of view, see e.g.
> https://lists.gnu.org/archive/html/qemu-devel/2019-12/msg03046.html and
> related discussions).
> 
> So IMHO the s390x world should move towards grub2, too. We could e.g.
> link it initially into the s390-ccw bios bios ... and if that works out
> well, later also use it as normal bootloader instead of zipl (not sure
> if that works in all cases, though, IIRC there were some size
> constraints and stuff like that).

petitboot would be another reasonable thing to consider here.  Since
it's Linux based, you have all the drivers you have there.  It's not
quite grub, but it does at least parse the same configuration files.

You do need kexec() of course, I don't know if you have that already
for s390 or not.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: Restrictions of libnet (was: Re: VW ELF loader)

[Michal Suchánek]

Hello,

On Tue, Feb 04, 2020 at 10:20:14AM +0100, Thomas Huth wrote:

> 
> So IMHO the s390x world should move towards grub2, too. We could e.g.
> link it initially into the s390-ccw bios bios ... and if that works out
> well, later also use it as normal bootloader instead of zipl (not sure
> if that works in all cases, though, IIRC there were some size
> constraints and stuff like that).

AFAIK the main reason why it does not work is that grub does not have
ccw drivers.

That aside it would be welcome to get it working.

Thanks

Michal

Re: Restrictions of libnet (was: Re: VW ELF loader)

[Thomas Huth]

On 04/02/2020 10.20, Thomas Huth wrote:
[...]
> So IMHO the s390x world should move towards grub2, too. We could e.g.
> link it initially into the s390-ccw bios bios ... and if that works out
> well, later also use it as normal bootloader instead of zipl

I meant to say "use it as normal bootloader instead of zipl on LPARs and
z/VM, too".

 Thomas

Restrictions of libnet (was: Re: VW ELF loader)

[Thomas Huth]

On 04/02/2020 09.54, Cornelia Huck wrote:
> On Tue, 4 Feb 2020 07:16:46 +0100
> Thomas Huth  wrote:
> 
>> On 04/02/2020 00.26, Paolo Bonzini wrote:
>>>
>>>
>>> Il mar 4 feb 2020, 00:20 Alexey Kardashevskiy >> > ha scritto:
>>>
>>> Speaking seriously, what would I put into the guest?
>>>
>>> Only things that would be considered drivers. Ignore the partitions
>>> issue for now so that you can just pass the device tree services to QEMU
>>> with hypercalls.
>>>
>>> Netboot's dhcp/tftp/ip/ipv6 client? It is going to be another SLOF,
>>> smaller but adhoc with only a couple of people knowing it.
>>>
>>>
>>> You can generalize and reuse the s390 code. All you have to write is the
>>> PCI scan and virtio-pci setup.  
>>
>> Well, for netbooting, the s390-ccw bios uses the libnet code from SLOF,
>> so re-using this for a slim netboot client on ppc64 would certainly be
>> feasible (especially since there are also already virtio drivers in SLOF
>> that are written in C), but I think it is not very future proof. The
>> libnet from SLOF only supports UDP, and no TCP. So for advanced boot
>> scenarios like booting from HTTP or even HTTPS, you need something else
>> (i.e. maybe grub is the better option, indeed).
> 
> That makes me wonder what that means for s390: We're inheriting
> libnet's limitations, but we don't have grub -- do we need to come up
> with something different? Or improve libnet?

I don't think that it makes sense to re-invent the wheel yet another
time and write yet another TCP implementation (which is likely quite a
bit of work, too, especially if you also want to do secure HTTPS in the
end). So yes, in the long run (as soon as somebody seriously asks for
HTTP booting on s390x) we need something different here.

Now looking at our standard s390x bootloader zipl - this has been giving
us a headache a couple of times in the past, too (from a distro point of
view since s390x is the only major platform left that does not use grub,
but also from a s390-ccw bios point of view, see e.g.
https://lists.gnu.org/archive/html/qemu-devel/2019-12/msg03046.html and
related discussions).

So IMHO the s390x world should move towards grub2, too. We could e.g.
link it initially into the s390-ccw bios bios ... and if that works out
well, later also use it as normal bootloader instead of zipl (not sure
if that works in all cases, though, IIRC there were some size
constraints and stuff like that).

Just my 0.02 € of course, though.

 Thomas