Re: [Qemu-devel] RFC: changing ROM bundling in tar dists for releases
Hi, > > i.e. something like this: > > > > https://www.kraxel.org/cgit/qemu-firmware/ > > > > (only seabios is there for now) > > So presumably we'd want a 'make install' target there too, to place > the > various ROMS in the location QEMU expects ? Yes. Or place them in $prefix/share/qemu-firmware, and add that directory to the qemu default search path. > So non-distro users would need to download & install the ROMs, > and then download, build & install QEMU ? Yes. > How would you view ROM install working for devs working against GIT ? > > eg when we update a ROM in the firmware directory, how do devs know > to > update their firmware. Would the qemu-firmware.git be a submodule of > qemu.git, so that a 'make install' would deploy roms for devs ? Not sure yet how to handle that best. I'd tend to not make it a submodule. Typically there are no lockstep updates for qemu and firmware, in most cases new firmware is only required if you want use a new feature which needs firmware support. So I think we don't need a tight coupling between qemu and firmware. Just updating the firmware checkout now and then (or even simply using the distro builds) should work in the vast majority of cases. cheers, Gerd
Re: [Qemu-devel] RFC: changing ROM bundling in tar dists for releases
On Fri, Sep 01, 2017 at 04:11:44PM +0200, Gerd Hoffmann wrote: > Hi, > > > I think it makes sense to create a separate project for the firmware > > blobs. Move over the firmware binaries and source submodules to the > > new project. This way updating both firmware sources and binaries > > can > > be done with a single commit, like we handle this today, just in the > > new firmware repo instead of the main qemu repo. > > i.e. something like this: > > https://www.kraxel.org/cgit/qemu-firmware/ > > (only seabios is there for now) So presumably we'd want a 'make install' target there too, to place the various ROMS in the location QEMU expects ? So non-distro users would need to download & install the ROMs, and then download, build & install QEMU ? How would you view ROM install working for devs working against GIT ? eg when we update a ROM in the firmware directory, how do devs know to update their firmware. Would the qemu-firmware.git be a submodule of qemu.git, so that a 'make install' would deploy roms for devs ? Regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [Qemu-devel] RFC: changing ROM bundling in tar dists for releases
Hi, > I think it makes sense to create a separate project for the firmware > blobs. Move over the firmware binaries and source submodules to the > new project. This way updating both firmware sources and binaries > can > be done with a single commit, like we handle this today, just in the > new firmware repo instead of the main qemu repo. i.e. something like this: https://www.kraxel.org/cgit/qemu-firmware/ (only seabios is there for now) cheers, Gerd
Re: [Qemu-devel] RFC: changing ROM bundling in tar dists for releases
On 1 September 2017 at 10:24, Daniel P. Berrange wrote: > I've always wondered why we need to specialcase those modules at all. > Personally I'd just remove them entirely and let people install the > -dev packages on their distro just like they do for any other build > prerequisite of QEMU. Two uses: (1) we've tended to want newer versions of dtc than distros actually had released (in fact right now we need a pretty new dtc that not many distros have) and (2) for hosts like OSX and Windows dependencies are not a simple matter of "just install the distro's dev package". thanks -- PMM
Re: [Qemu-devel] RFC: changing ROM bundling in tar dists for releases
Hi, > > Question is what to do with the non-firmware submodules (pixman, > > dtc, > > more?) then. I think they are not that big, so I doubt it is worth > > the > > hassle to create two tarball versions. And license-wise it isn't a > > issue too. > > I've always wondered why we need to specialcase those modules at all. > Personally I'd just remove them entirely and let people install the > -dev packages on their distro just like they do for any other build > prerequisite of QEMU. Yes, we can probably do that now with pixman. A few years back when the submodule was added there was the problem that some distros shipped too old versions of pixman. Known offender: RHEL-5. But RHEL-5 is EOL meanwhile and I think qemu raised the minimum glib version to something newer than what RHEL-5 ships, so it qemu doesn't build on RHEL-5 anyway. Dunno about dtc. cheers, Gerd
Re: [Qemu-devel] RFC: changing ROM bundling in tar dists for releases
On Fri, Sep 01, 2017 at 10:31:17AM +0200, Gerd Hoffmann wrote:
> Hi,
>
> > > - The qemu release dists get ever larger as we add more ROMS.
> > > Adding
> > > EFI ROM builds for i386, x86_64, and aarch64 will make the dists
> > > larger still.
> >
> > I think these make sense. Should we tie this into the
> > recent suggestion (by Gerd?) that we should put all the
> > rom blobs into git submodules, and otherwise generally
> > try to regularise our handling of blobs?
>
> It surely makes sense to consider both together, so we can work out a
> reasonable workflow for firmware updates and distribution.
>
> I think it makes sense to create a separate project for the firmware
> blobs. Move over the firmware binaries and source submodules to the
> new project. This way updating both firmware sources and binaries can
> be done with a single commit, like we handle this today, just in the
> new firmware repo instead of the main qemu repo.
>
> When moving only the binaries to a separate git submodule and
> continuing to have the sources as main qemu repo submodules firmware
> updates become more complicated.
>
> > > 3. Change existing dist, and add a new one with bundled bits
> > >
> > > qemu-X.Y.Z.tar.bz2 - qemu source only
> > > qemu-addons-X.Y.Z.tar.bz2 - bundled ROMS + libs
>
> With the scheme above it makes sense to have a qemu-firmware-
> ${version}.tar.gz addon tarball. Maybe even two (one with the prebuilt
> binaries and one with the sources).
>
> Question is what to do with the non-firmware submodules (pixman, dtc,
> more?) then. I think they are not that big, so I doubt it is worth the
> hassle to create two tarball versions. And license-wise it isn't a
> issue too.
I've always wondered why we need to specialcase those modules at all.
Personally I'd just remove them entirely and let people install the
-dev packages on their distro just like they do for any other build
prerequisite of QEMU.
Regards,
Daniel
--
|: https://berrange.com -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [Qemu-devel] RFC: changing ROM bundling in tar dists for releases
Hi,
> > - The qemu release dists get ever larger as we add more ROMS.
> > Adding
> > EFI ROM builds for i386, x86_64, and aarch64 will make the dists
> > larger still.
>
> I think these make sense. Should we tie this into the
> recent suggestion (by Gerd?) that we should put all the
> rom blobs into git submodules, and otherwise generally
> try to regularise our handling of blobs?
It surely makes sense to consider both together, so we can work out a
reasonable workflow for firmware updates and distribution.
I think it makes sense to create a separate project for the firmware
blobs. Move over the firmware binaries and source submodules to the
new project. This way updating both firmware sources and binaries can
be done with a single commit, like we handle this today, just in the
new firmware repo instead of the main qemu repo.
When moving only the binaries to a separate git submodule and
continuing to have the sources as main qemu repo submodules firmware
updates become more complicated.
> > 3. Change existing dist, and add a new one with bundled bits
> >
> > qemu-X.Y.Z.tar.bz2 - qemu source only
> > qemu-addons-X.Y.Z.tar.bz2 - bundled ROMS + libs
With the scheme above it makes sense to have a qemu-firmware-
${version}.tar.gz addon tarball. Maybe even two (one with the prebuilt
binaries and one with the sources).
Question is what to do with the non-firmware submodules (pixman, dtc,
more?) then. I think they are not that big, so I doubt it is worth the
hassle to create two tarball versions. And license-wise it isn't a
issue too.
cheers,
Gerd
Re: [Qemu-devel] RFC: changing ROM bundling in tar dists for releases
On 08/31/2017 09:29 AM, Daniel P. Berrange wrote: [...]> There are the following options I see 1. Keep existing dist, and add a new minimal one qemu-X.Y.Z.tar.bz2 - qemu source + bundled ROMS + libs qemu-minimal-X.Y.Z.tar.bz2 - qemu source only Least impact for current non-distro users, distros just switch. 2. Change existing dist, and add a new one with everything qemu-X.Y.Z.tar.bz2 - qemu source only qemu-full-X.Y.Z.tar.bz2 - qemu source + bundled ROMS + libs Non-distro users need to download a different dist from what they have known previously, but otherwise unchanged build process. 3. Change existing dist, and add a new one with bundled bits qemu-X.Y.Z.tar.bz2 - qemu source only qemu-addons-X.Y.Z.tar.bz2 - bundled ROMS + libs Non-distro users need to manually download & install an extra piece compared to now. Debian style: linux-kbuild-4.9 - Kbuild infrastructure for Linux 4.9 linux-kbuild-4.12 - Kbuild infrastructure for Linux 4.12 linux-source - Linux kernel source (meta-package) linux-source-4.12 - Linux kernel source for version 4.12 with Debian patches firmware-linux - Binary firmware for various drivers in the Linux kernel (meta-package) firmware-linux-nonfree - Binary firmware for various drivers in the Linux kernel (meta-package) firmware-misc-nonfree - Binary firmware for various drivers in the Linux kernel firmware-zd1211 - binary firmware for the zd1211rw wireless driver firmware-bnx2 - Binary firmware for Broadcom NetXtremeII ... qemu-efi - UEFI firmware for 64-bit ARM virtual machines grub-firmware-qemu - GRUB firmware image for QEMU ipxe-qemu - PXE boot firmware - ROM images for qemu ovmf - UEFI firmware for 64-bit x86 virtual machines vgabios - VGA BIOS software for the Bochs and Qemu emulated VGA card seabios - Legacy BIOS implementation sgabios - bios option rom to provide legacy serial console for x86 $ apt-cache show qemu-system-x86 Package: qemu-system-x86 Provides: qemu-system-i386, qemu-system-x86-64 Suggests: samba, vde2, qemu-block-extra, kmod, sgabios, ovmf $ apt-cache show qemu-system-arm Recommends: qemu-utils, ipxe-qemu, qemu-efi This seems to me the cleaner one, the distrib choose to provide dependencies in the package description, a package to build, and keep exclusive licenses in different packages. 4. Change existing dist, never distribute bundled ROMs + libs at all qemu-X.Y.Z.tar.bz2 - qemu source only Only here for completeness, not a serious suggestion Linux style :) 5. Change nothing Continue ignoring the problem I aim to solve. My patch did option 2, but I really open to any of 1/2/3 without a strong preference, as long as we can get some agreement on one. I guess option 1 might be easiest to get agreement on given its minimal impact. It would be easier to have people building from sources to answer but they usually don't read this list, you might want to ask on qemu-stable. Regards, Phil.
Re: [Qemu-devel] RFC: changing ROM bundling in tar dists for releases
On 31 August 2017 at 13:29, Daniel P. Berrange wrote: > A while back now I posted an RFC patch that changes qemu-X.Y.Z.tar.bz2 > archive to *not* have any ROMs or 3rd party libs bundled, and create a > new dist qemu-bundled-X.Y.Z.tar.bz2 for the fully bundled dist: > > http://lists.gnu.org/archive/html/qemu-devel/2017-04/msg03335.html > > With 2.10 out of the way, and KVM Forum approaching, I figure now is > a reasonable time to revive the idea to see if it has support > > The core issues I'm aiming to solve are: > > - Distro vendors don't want the bundled ROMs / libs. They want to >fully build everything from source to ensure they are distributing >clean soure & builds in compliance with the licenses. Currently they >strip bundled bits from the build tree, but would prefer if the source >dist did not have them either. > > - The qemu release dists get ever larger as we add more ROMS. Adding >EFI ROM builds for i386, x86_64, and aarch64 will make the dists >larger still. I think these make sense. Should we tie this into the recent suggestion (by Gerd?) that we should put all the rom blobs into git submodules, and otherwise generally try to regularise our handling of blobs? > There are the following options I see > > 1. Keep existing dist, and add a new minimal one > >qemu-X.Y.Z.tar.bz2 - qemu source + bundled ROMS + libs >qemu-minimal-X.Y.Z.tar.bz2 - qemu source only > > Least impact for current non-distro users, distros just switch. > > 2. Change existing dist, and add a new one with everything > >qemu-X.Y.Z.tar.bz2 - qemu source only >qemu-full-X.Y.Z.tar.bz2 - qemu source + bundled ROMS + libs > > Non-distro users need to download a different dist from what they > have known previously, but otherwise unchanged build process. > > 3. Change existing dist, and add a new one with bundled bits > >qemu-X.Y.Z.tar.bz2 - qemu source only >qemu-addons-X.Y.Z.tar.bz2 - bundled ROMS + libs > > Non-distro users need to manually download & install an > extra piece compared to now. I don't use the tarballs, so I don't really have a strong preference. thanks -- PMM
