On 03/11/19 11:35, Daniel P. Berrangé wrote: > On Sun, Mar 10, 2019 at 12:21:55PM +0100, Philippe Mathieu-Daudé wrote: >> On 3/10/19 4:56 AM, Michael S. Tsirkin wrote: >>> On Sat, Mar 09, 2019 at 01:48:16AM +0100, Laszlo Ersek wrote: >>>> Repo: https://github.com/lersek/qemu.git >>>> Branch: edk2_build >>>> >>>> This series advances the roms/edk2 submodule to the "edk2-stable201903" >>>> release, and builds and captures platform firmware binaries from that >>>> release. At this point they are meant to be used by both end-users and >>>> by Igor's ACPI unit tests in qtest ("make check"). >>>> >>>> Previous discussion: >>>> >>>> [Qemu-devel] bundling edk2 platform firmware images with QEMU >>>> >>>> 80f0bae3-e79a-bb68-04c4-1c9c684d95b8@redhat.com">http://mid.mail-archive.com/80f0bae3-e79a-bb68-04c4-1c9c684d95b8@redhat.com >>>> https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg02601.html >> >> There David raised a concern about "[adding] ~206 MB of binaries to the >> pc-bios directory". I'm also worried. >> >> GitHub kindly suggest to use git-lfs. It is an extra dependency I'd >> rather strongly avoid (because we support a wide range of host OS, each >> using a wide types of filesystems). >> >> What about storing those binaries on a file server (http/ftp) altogether >> with a file containing its hashed digest (SHA1/SHA256)? Then we already >> have all the required tools to fetch and verify those blob roms with the >> build system. >> Or we could store the hashes in the QEMU repository too. > > A simple approach would simply be to 'xz' compress all the edk*.fd > files but still store them in git. They're already opaque binary > files, so replacing one binary format with another binary format > is no big deal IMHO > > $ ls -alsh edk2-* > 1.1M -rw-rw-r--. 1 berrange berrange 1.1M Mar 11 10:29 edk2-aarch64-code.fd.xz > 2.1M -rw-rw-r--. 1 berrange berrange 64M Mar 11 10:29 edk2-arm-code.fd > 772K -rw-rw-r--. 1 berrange berrange 64M Mar 11 10:29 edk2-arm-vars.fd > 3.5M -rw-rw-r--. 1 berrange berrange 3.5M Mar 11 10:29 edk2-i386-code.fd > 3.5M -rw-rw-r--. 1 berrange berrange 3.5M Mar 11 10:29 > edk2-i386-secure-code.fd > 528K -rw-rw-r--. 1 berrange berrange 528K Mar 11 10:29 edk2-i386-vars.fd > 12K -rw-rw-r--. 1 berrange berrange 11K Mar 11 10:29 edk2-licenses.txt > 3.5M -rw-rw-r--. 1 berrange berrange 3.5M Mar 11 10:29 edk2-x86_64-code.fd > 3.5M -rw-rw-r--. 1 berrange berrange 3.5M Mar 11 10:29 > edk2-x86_64-secure-code.fd > > This gives a 50% disk space saving over the sparse size: > > $ ls -alsh edk2-* > 1.1M -rw-rw-r--. 1 berrange berrange 1.1M Mar 11 10:29 edk2-aarch64-code.fd.xz > 1.1M -rw-rw-r--. 1 berrange berrange 1.1M Mar 11 10:29 edk2-arm-code.fd.xz > 12K -rw-rw-r--. 1 berrange berrange 9.8K Mar 11 10:29 edk2-arm-vars.fd.xz > 1.6M -rw-rw-r--. 1 berrange berrange 1.6M Mar 11 10:29 edk2-i386-code.fd.xz > 1.8M -rw-rw-r--. 1 berrange berrange 1.8M Mar 11 10:29 > edk2-i386-secure-code.fd.xz > 4.0K -rw-rw-r--. 1 berrange berrange 320 Mar 11 10:29 edk2-i386-vars.fd.xz > 12K -rw-rw-r--. 1 berrange berrange 11K Mar 11 10:29 edk2-licenses.txt > 1.6M -rw-rw-r--. 1 berrange berrange 1.6M Mar 11 10:29 edk2-x86_64-code.fd.xz > 1.9M -rw-rw-r--. 1 berrange berrange 1.9M Mar 11 10:29 > edk2-x86_64-secure-code.fd.xz > > So about 9 MB compressed, instead of 20MB for the uncompressed sparse > files, which is on a par with the existing ROM sizes. > > We would then need a "make" rule that runs unxz to "build" the firmware > files. Wouldn't need to be more complicated that this: > > edk2-%.fd: edk2-%.fd.xz > unzx -c $< > $@ > > CLEANFILES += $(wildcard edk2*.fd)
The problem with this idea is that such *.xz files are not directly consumeable by Igor's ACPI regression test cases in qtest. The above approach is suitable for "make install" only, not for "make check", AIUI. I had thought of something similar earlier however. We could use compressed qcow2 disk images for all of the *fd files here (that is, under pc-bios/), because Igor's test code *could* consume those directly. What blocked me there was that I didn't know if / how we could decompress such qcow2 images as a part of "make install" (that is, from pc-bios to the final location in the filesystem). Invoking "qemu-img convert" (for conversion to "raw") *seems* simple enough, but what qemu-img binary should I use? Is it guaranteed that there will be a qemu-img binary at all, at "make install" time? Thanks Laszlo
