Script 'mail_helper' called by obssrc
Hello community,
here is the log from the commit of package virtme for openSUSE:Factory checked
in at 2024-10-20 10:05:08
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Comparing /work/SRC/openSUSE:Factory/virtme (Old)
and /work/SRC/openSUSE:Factory/.virtme.new.26871 (New)
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "virtme"
Sun Oct 20 10:05:08 2024 rev:21 rq:1208978 version:1.31
Changes:
--------
--- /work/SRC/openSUSE:Factory/virtme/virtme.changes 2024-09-24
17:35:19.374219907 +0200
+++ /work/SRC/openSUSE:Factory/.virtme.new.26871/virtme.changes 2024-10-20
10:05:19.198074429 +0200
@@ -1,0 +2,21 @@
+Sat Oct 19 07:57:42 UTC 2024 - Michael Vetter <mvet...@suse.com>
+
+- Update to 1.31:
+ * Fix a packaging issue, after an attempt to modernize the build
+ system we realized that we were not installing the bash
+ completion file anymore, so we have temporarily reverted the
+ change to cut this new release.
+ See gh/arighi/virtme-ng#181
+ * Initial support to run virtme-ng on macOS
+
+-------------------------------------------------------------------
+Tue Oct 15 09:06:54 UTC 2024 - Michael Vetter <mvet...@suse.com>
+
+- Update to 1.30:
+ * Initial support for NVIDIA GPUs passthrough
+ * Possibility to use pre-compiled -rc kernels from Ubuntu mainline builds
+ * Possibility to use virtiofs natively on arm64
+ * Some improvements to run virtme-ng cross-architecture and cross-distro
+ * Bug fixes
+
+-------------------------------------------------------------------
Old:
----
virtme-ng-1.29.tar.xz
New:
----
virtme-ng-1.31.tar.xz
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Other differences:
------------------
++++++ virtme.spec ++++++
--- /var/tmp/diff_new_pack.B6bwJA/_old 2024-10-20 10:05:20.834136212 +0200
+++ /var/tmp/diff_new_pack.B6bwJA/_new 2024-10-20 10:05:20.838136363 +0200
@@ -22,7 +22,7 @@
%global pythons python311
%endif
Name: virtme
-Version: 1.29
+Version: 1.31
Release: 0
Summary: Tools for virtualize the running distro or a rootfs
License: GPL-2.0-only
++++++ _service ++++++
--- /var/tmp/diff_new_pack.B6bwJA/_old 2024-10-20 10:05:21.130147390 +0200
+++ /var/tmp/diff_new_pack.B6bwJA/_new 2024-10-20 10:05:21.186149505 +0200
@@ -3,7 +3,7 @@
<param name="url">https://github.com/arighi/virtme-ng.git</param>
<param name="scm">git</param>
<param name="submodules">enable</param>
- <param name="revision">v1.29</param>
+ <param name="revision">v1.31</param>
<param name="versionformat">@PARENT_TAG@</param>
<param name="versionrewrite-pattern">v(.*)</param>
<param name="versionrewrite-replacement">\1</param>
++++++ virtme-ng-1.29.tar.xz -> virtme-ng-1.31.tar.xz ++++++
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/Makefile new/virtme-ng-1.31/Makefile
--- old/virtme-ng-1.29/Makefile 2024-09-11 08:38:45.000000000 +0200
+++ new/virtme-ng-1.31/Makefile 2024-10-18 15:30:58.000000000 +0200
@@ -3,10 +3,6 @@
# Get git version information for make install
GIT_DESCRIBE := $(shell git describe --always --long --dirty)
-.PHONY: init
-init:
- cd virtme_ng_init && cargo install --path . --root ../virtme/guest
-
# see README.md '* Install from source'
install: install_from_source
install_from_source:
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/README.md new/virtme-ng-1.31/README.md
--- old/virtme-ng-1.29/README.md 2024-09-11 08:38:45.000000000 +0200
+++ new/virtme-ng-1.31/README.md 2024-10-18 15:30:58.000000000 +0200
@@ -358,6 +358,26 @@
See also: `.github/workflows/run.yml` as a practical example on how to use
virtme-ng inside docker.
+ - Run virtme-ng with gpu passthrough:
+```
+ # Confirm host kernel has VFIO and IOMMU support
+ # Check if NVIDIA module is installed on the host
+ $ modinfo nvidia
+ # If the nvidia module is installed, blacklist the nvidia modules
+ $ sudo bash -c 'echo -e "blacklist nvidia\nblacklist nvidia-drm\nblacklist
nvidia-modeset\nblacklist nvidia-peermem\nblacklist nvidia-uvm" >
/etc/modprobe.d/blacklist-nvidia.conf'
+ # Host will need to be rebooted for blacklist to take effect.
+ # Get GPU device ID
+ $ lspci -nn | grep NVIDIA
+ 0000:01:00.0 VGA compatible controller [0300]: NVIDIA Corporation
AD104GLM [RTX 3500 Ada Generation Laptop GPU] [10de:27bb] (rev a1)
+ 0000:01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:22bc]
(rev a1))
+ # Configure VFIO for device passthrough
+ $ sudo bash -c 'options vfio-pci ids=10de:27bb,10de:22bc' >
/etc/modprobe.d/vfio.conf
+ # Load VFIO module
+ $ sudo modprobe vfio-pci
+ # Pass PCI address to virtme-ng
+ $ sudo vng --nvgpu "01:00.0" -r linux
+```
+
Implementation details
======================
@@ -499,11 +519,13 @@
- [LWN: Faster kernel testing with virtme-ng (November,
2023)](https://lwn.net/Articles/951313/)
- [LPC 2023: Speeding up Kernel Testing and Debugging with
virtme-ng](https://lpc.events/event/17/contributions/1506/attachments/1143/2441/virtme-ng.pdf)
+ - [Kernel Recipes 2024:
virtme-ng](https://kernel-recipes.org/en/2024/virtme-ng/)
+ - [Linux Foundation Mentorship Session: Speeding Up Kernel Development With
virtme-ng](https://www.youtube.com/watch?v=ZgMLGM2UazY)
Credits
=======
-virtme-ng is written by Andrea Righi <andrea.ri...@canonical.com>
+virtme-ng is written by Andrea Righi <ari...@nvidia.com>
virtme-ng is based on virtme, written by Andy Lutomirski <l...@kernel.org>
([web][korg-web] | [git][korg-git]).
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/requirements.txt
new/virtme-ng-1.31/requirements.txt
--- old/virtme-ng-1.29/requirements.txt 2024-09-11 08:38:45.000000000 +0200
+++ new/virtme-ng-1.31/requirements.txt 2024-10-18 15:30:58.000000000 +0200
@@ -1,3 +1,5 @@
argcomplete
argparse-manpage
requests
+setuptools
+
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/setup.py new/virtme-ng-1.31/setup.py
--- old/virtme-ng-1.29/setup.py 2024-09-11 08:38:45.000000000 +0200
+++ new/virtme-ng-1.31/setup.py 2024-10-18 15:30:58.000000000 +0200
@@ -77,7 +77,7 @@
argparse-manpage \
--pyfile ./virtme_ng/run.py --function make_parser \
--prog vng --version v{VERSION} \
- --author "virtme-ng is written by Andrea Righi <andrea.ri...@canonical.com>"
\
+ --author "virtme-ng is written by Andrea Righi <ari...@nvidia.com>" \
--author "Based on virtme by Andy Lutomirski <l...@kernel.org>" \
--project-name virtme-ng --manual-title virtme-ng \
--description "Quickly run kernels inside a virtualized snapshot of your
live system" \
@@ -90,10 +90,22 @@
print(f"BUILD_VIRTME_NG_INIT: {build_virtme_ng_init}")
# Build virtme-ng-init
if build_virtme_ng_init:
- check_call(["make", "init"])
+ cwd = "virtme_ng_init"
+ root = "../virtme/guest"
+ args = ["cargo", "install", "--path", ".", "--root", root]
+ if platform.system() == "Darwin":
+ machine = platform.machine()
+ if machine == "arm64":
+ machine = "aarch64"
+ target = f"{machine}-unknown-linux-musl"
+ args.extend([
+ "--target", target,
+ "--config", f"target.{target}.linker = \"rust-lld\"",
+ ])
+ check_call(args, cwd="virtme_ng_init")
check_call(
- ["strip", "-s", "../virtme/guest/bin/virtme-ng-init"],
- cwd="virtme_ng_init",
+ ["strip", os.path.join(root, "bin", "virtme-ng-init")],
+ cwd=cwd,
)
# Generate manpage
if which('argparse-manpage'):
@@ -165,7 +177,7 @@
name="virtme-ng",
version=VERSION,
author="Andrea Righi",
- author_email="andrea.ri...@canonical.com",
+ author_email="ari...@nvidia.com",
description="Build and run a kernel inside a virtualized snapshot of your
live system",
url="https://git.launchpad.net/~arighi/+git/virtme-ng";,
license="GPLv2",
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/virtme/architectures.py
new/virtme-ng-1.31/virtme/architectures.py
--- old/virtme-ng-1.29/virtme/architectures.py 2024-09-11 08:38:45.000000000
+0200
+++ new/virtme-ng-1.31/virtme/architectures.py 2024-10-18 15:30:58.000000000
+0200
@@ -23,9 +23,10 @@
return False
@staticmethod
- def qemuargs(is_native, use_kvm) -> List[str]:
+ def qemuargs(is_native, use_kvm, use_gpu) -> List[str]:
_ = is_native
_ = use_kvm
+ _ = use_gpu
return []
@staticmethod
@@ -51,6 +52,10 @@
return ["-vga", "none", "-display", "none"]
@staticmethod
+ def qemu_nodisplay_nvgpu_args() -> List[str]:
+ return ["-display", "none"]
+
+ @staticmethod
def qemu_display_args() -> List[str]:
return ["-device", "virtio-gpu-pci"]
@@ -71,6 +76,9 @@
def kimg_path(self) -> str:
return "arch/%s/boot/bzImage" % self.linuxname
+ def img_name(self) -> str:
+ return "vmlinuz"
+
@staticmethod
def dtb_path() -> Optional[str]:
return None
@@ -78,8 +86,8 @@
class Arch_unknown(Arch):
@staticmethod
- def qemuargs(is_native, use_kvm):
- return Arch.qemuargs(is_native, use_kvm)
+ def qemuargs(is_native, use_kvm, use_gpu):
+ return Arch.qemuargs(is_native, use_kvm, use_gpu)
class Arch_x86(Arch):
@@ -94,8 +102,8 @@
return True
@staticmethod
- def qemuargs(is_native, use_kvm):
- ret = Arch.qemuargs(is_native, use_kvm)
+ def qemuargs(is_native, use_kvm, use_gpu):
+ ret = Arch.qemuargs(is_native, use_kvm, use_gpu)
# Add a watchdog. This is useful for testing.
ret.extend(["-device", "i6300esb,id=watchdog0"])
@@ -103,7 +111,10 @@
if is_native and use_kvm:
# If we're likely to use KVM, request a full-featured CPU.
# (NB: if KVM fails, this will cause problems. We should probe.)
- ret.extend(["-cpu", "host"]) # We can't migrate regardless.
+ cpu_str = "host"
+ if use_gpu:
+ cpu_str += ",host-phys-bits-limit=0x28"
+ ret.extend(["-cpu", cpu_str])
else:
ret.extend(["-machine", "q35"])
@@ -179,8 +190,8 @@
]
@staticmethod
- def qemuargs(is_native, use_kvm):
- ret = Arch.qemuargs(is_native, use_kvm)
+ def qemuargs(is_native, use_kvm, use_gpu):
+ ret = Arch.qemuargs(is_native, use_kvm, use_gpu)
# Use microvm architecture for faster boot
ret.extend(["-M", "microvm,accel=kvm,pcie=on,rtc=on"])
@@ -200,8 +211,8 @@
self.defconfig_target = "vexpress_defconfig"
@staticmethod
- def qemuargs(is_native, use_kvm):
- ret = Arch.qemuargs(is_native, use_kvm)
+ def qemuargs(is_native, use_kvm, use_gpu):
+ ret = Arch.qemuargs(is_native, use_kvm, use_gpu)
# Emulate a vexpress-a15.
ret.extend(["-M", "vexpress-a15"])
@@ -242,18 +253,22 @@
class Arch_aarch64(Arch):
- def __init__(self):
- Arch.__init__(self, "aarch64")
+ def __init__(self, name):
+ Arch.__init__(self, name)
self.qemuname = "aarch64"
self.linuxname = "arm64"
self.gccname = "aarch64"
@staticmethod
- def qemuargs(is_native, use_kvm):
- ret = Arch.qemuargs(is_native, use_kvm)
+ def virtiofs_support() -> bool:
+ return True
+
+ @staticmethod
+ def qemuargs(is_native, use_kvm, use_gpu):
+ ret = Arch.qemuargs(is_native, use_kvm, use_gpu)
- if is_native:
+ if is_native and use_kvm:
ret.extend(["-M", "virt,gic-version=host"])
ret.extend(["-cpu", "host"])
else:
@@ -292,8 +307,8 @@
self.gccname = "powerpc64le"
@staticmethod
- def qemuargs(is_native, use_kvm):
- ret = Arch.qemuargs(is_native, use_kvm)
+ def qemuargs(is_native, use_kvm, use_gpu):
+ ret = Arch.qemuargs(is_native, use_kvm, use_gpu)
ret.extend(["-M", "pseries"])
return ret
@@ -312,23 +327,24 @@
# Apparently SLOF (QEMU's bundled firmware?) can't boot a zImage.
return "vmlinux"
+ def img_name(self) -> str:
+ return "vmlinux"
+
class Arch_riscv64(Arch):
def __init__(self):
Arch.__init__(self, "riscv64")
self.defconfig_target = "defconfig"
- self.qemuname = "riscv64"
self.linuxname = "riscv"
- self.gccname = "riscv64"
@staticmethod
def virtiofs_support() -> bool:
return True
@staticmethod
- def qemuargs(is_native, use_kvm):
- ret = Arch.qemuargs(is_native, use_kvm)
+ def qemuargs(is_native, use_kvm, use_gpu):
+ ret = Arch.qemuargs(is_native, use_kvm, use_gpu)
ret.extend(["-machine", "virt"])
ret.extend(["-bios", "default"])
@@ -347,13 +363,11 @@
Arch.__init__(self, "sparc64")
self.defconfig_target = "sparc64_defconfig"
- self.qemuname = "sparc64"
self.linuxname = "sparc"
- self.gccname = "sparc64"
@staticmethod
- def qemuargs(is_native, use_kvm):
- return Arch.qemuargs(is_native, use_kvm)
+ def qemuargs(is_native, use_kvm, use_gpu):
+ return Arch.qemuargs(is_native, use_kvm, use_gpu)
def kimg_path(self):
return "arch/sparc/boot/image"
@@ -368,17 +382,15 @@
def __init__(self):
Arch.__init__(self, "s390x")
- self.qemuname = "s390x"
self.linuxname = "s390"
- self.gccname = "s390x"
@staticmethod
def virtio_dev_type(virtiotype):
return "virtio-%s-ccw" % virtiotype
@staticmethod
- def qemuargs(is_native, use_kvm):
- ret = Arch.qemuargs(is_native, use_kvm)
+ def qemuargs(is_native, use_kvm, use_gpu):
+ ret = Arch.qemuargs(is_native, use_kvm, use_gpu)
# Ask for the latest version of s390-ccw
ret.extend(["-M", "s390-ccw-virtio"])
@@ -401,6 +413,9 @@
def qemu_serial_console_args():
return ["-device", "sclpconsole,chardev=console"]
+ def img_name(self) -> str:
+ return "image"
+
ARCHES = {
arch.virtmename: arch
@@ -409,7 +424,8 @@
Arch_x86("x86_64"),
Arch_x86("i386"),
Arch_arm(),
- Arch_aarch64(),
+ Arch_aarch64("aarch64"),
+ Arch_aarch64("arm64"),
Arch_ppc("ppc64"),
Arch_ppc("ppc64le"),
Arch_riscv64(),
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/virtme/commands/configkernel.py
new/virtme-ng-1.31/virtme/commands/configkernel.py
--- old/virtme-ng-1.29/virtme/commands/configkernel.py 2024-09-11
08:38:45.000000000 +0200
+++ new/virtme-ng-1.31/virtme/commands/configkernel.py 2024-10-18
15:30:58.000000000 +0200
@@ -10,12 +10,13 @@
import sys
import argparse
import os
+import platform
import shlex
import shutil
import subprocess
import multiprocessing
from .. import architectures
-from ..util import SilentError, uname
+from ..util import SilentError
def make_parser():
@@ -27,7 +28,7 @@
"--arch",
action="store",
metavar="ARCHITECTURE",
- default=uname.machine,
+ default=platform.machine(),
help="Target architecture",
)
@@ -285,6 +286,7 @@
args = _ARGPARSER.parse_args()
arch = architectures.get(args.arch)
+ is_native = args.arch == platform.machine()
custom_conf = []
if args.custom:
@@ -325,7 +327,7 @@
if args.cross_compile != "":
cross_compile_prefix = args.cross_compile
- if shutil.which(f"{cross_compile_prefix}-gcc") and arch.gccname !=
uname.machine:
+ if not is_native and shutil.which(f"{cross_compile_prefix}-gcc"):
gccname = shlex.quote(f"{cross_compile_prefix}-gcc")
archargs.append(f"CROSS_COMPILE={gccname}")
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/virtme/commands/run.py
new/virtme-ng-1.31/virtme/commands/run.py
--- old/virtme-ng-1.29/virtme/commands/run.py 2024-09-11 08:38:45.000000000
+0200
+++ new/virtme-ng-1.31/virtme/commands/run.py 2024-10-18 15:30:58.000000000
+0200
@@ -11,6 +11,7 @@
import argparse
import tempfile
import os
+import platform
import errno
import fcntl
import sys
@@ -29,7 +30,7 @@
from .. import qemu_helpers
from .. import architectures
from .. import resources
-from ..util import SilentError, uname, get_username, find_binary_or_raise
+from ..util import SilentError, get_username, find_binary_or_raise
def make_parser() -> argparse.ArgumentParser:
@@ -46,7 +47,7 @@
"--installed-kernel",
action="store",
nargs="?",
- const=uname.release,
+ const=platform.release(),
default=None,
metavar="VERSION",
help="[Deprecated] use --kimg instead.",
@@ -56,7 +57,7 @@
"--kimg",
action="store",
nargs="?",
- const=uname.release,
+ const=platform.release(),
default=None,
help="Use specified kernel image or an installed kernel version. "
+ "If no argument is specified the running kernel will be used.",
@@ -74,7 +75,7 @@
"--mods",
action="store",
choices=["none", "use", "auto"],
- default="use",
+ required=True,
help="Setup loadable kernel modules inside a compiled kernel source
directory "
+ "(used in conjunction with --kdir); "
+ "none: ignore kernel modules, use: asks user to refresh virtme's
kernel modules directory, "
@@ -201,7 +202,7 @@
"--arch",
action="store",
metavar="ARCHITECTURE",
- default=uname.machine,
+ default=platform.machine(),
help="Guest architecture",
)
g.add_argument(
@@ -312,6 +313,9 @@
help="Supply a directory that is r/w to the guest but read-only in the
host. Use --overlay-rwdir=path.",
)
+ g.add_argument(
+ "--nvgpu", action="store", default=None, help="Set guest NVIDIA GPU."
+ )
return parser
@@ -368,7 +372,7 @@
return os.path.abspath(path)
-def get_kernel_version(path):
+def get_kernel_version(img_name, path):
if not os.path.exists(path):
arg_fail(
"kernel file %s does not exist, try --build to build the kernel" %
path
@@ -380,7 +384,7 @@
["file", path], capture_output=True, text=True, check=False
)
for item in result.stdout.split(", "):
- match = re.search(r"^[vV]ersion (\S+)", item)
+ match = re.search(r"^[vV]ersion (\S{3,})", item)
if match:
kernel_version = match.group(1)
return kernel_version
@@ -393,10 +397,18 @@
result = subprocess.run(
["strings", path], capture_output=True, text=True, check=False
)
- match = re.search(r"Linux version (\S+)", result.stdout)
+ match = re.search(r"Linux version (\S{3,})", result.stdout)
if match:
kernel_version = match.group(1)
return kernel_version
+
+ # The version detection fails s390x using file or strings tools, so check
+ # if the file itself contains the version number.
+ if img_name:
+ match = re.search(fr"{img_name}-(\S{{3,}})", path)
+ if match:
+ return match.group(1)
+
return None
@@ -428,16 +440,18 @@
args.kimg = None
if args.kimg is not None:
+ img_name = arch.img_name()
+
# Try to resolve kimg as a kernel version first, then check if a file
# is provided.
- kimg = "/usr/lib/modules/%s/vmlinuz" % args.kimg
+ kimg = "/usr/lib/modules/%s/%s" % (args.kimg, img_name)
if not os.path.exists(kimg):
- kimg = "/boot/vmlinuz-%s" % args.kimg
+ kimg = "/boot/%s-%s" % (img_name, args.kimg)
if not os.path.exists(kimg):
kimg = args.kimg
if not os.path.exists(kimg):
arg_fail("%s does not exist" % args.kimg)
- kver = get_kernel_version(kimg)
+ kver = get_kernel_version(img_name, kimg)
if kver is None:
# Unable to detect kernel version, try to boot without
# automatically detecting modules.
@@ -484,7 +498,7 @@
elif args.kdir is not None:
kimg = os.path.join(args.kdir, arch.kimg_path())
# Run get_kernel_version to check at least if the kernel image exist.
- kernel.version = get_kernel_version(kimg)
+ kernel.version = get_kernel_version(None, kimg)
kernel.kimg = kimg
virtme_mods = os.path.join(args.kdir, ".virtme_mods")
mod_file = os.path.join(args.kdir, "modules.order")
@@ -573,7 +587,7 @@
def _get_virtiofsd_path(self):
# Define the possible virtiofsd paths.
#
- # NOTE: do not use the C implemention of qemu's virtiofsd, because it
+ # NOTE: do not use the C implementation of qemu's virtiofsd, because it
# doesn't support unprivileged-mode execution and it would be totally
# unsafe to export the whole rootfs of the host running as root.
#
@@ -652,48 +666,59 @@
return True
+class VirtioFSConfig:
+ def __init__(self, path: str, mount_tag: str, guest_tools_path=None,
memory=None):
+ self.path = path
+ self.mount_tag = mount_tag
+ self.guest_tools_path = guest_tools_path
+ self.memory = memory
+
+
def export_virtiofs(
arch: architectures.Arch,
qemuargs: List[str],
- path: str,
- mount_tag: str,
- guest_tools_path=None,
- memory=None,
+ config: VirtioFSConfig,
verbose=False,
-) -> None:
+) -> bool:
if not arch.virtiofs_support():
return False
- # Try to start virtiofsd deamon
- virtio_fs = VirtioFS(guest_tools_path)
- ret = virtio_fs.start(path, verbose)
+ # Try to start virtiofsd daemon
+ virtio_fs = VirtioFS(config.guest_tools_path)
+ ret = virtio_fs.start(config.path, verbose)
if not ret:
return False
# Adjust qemu options to use virtiofsd
fsid = "virtfs%d" % len(qemuargs)
-
vhost_dev_type = arch.vhost_dev_type()
+
qemuargs.extend(["-chardev",
f"socket,id=char{fsid},path={virtio_fs.sock}"])
- qemuargs.extend(["-device",
f"{vhost_dev_type},chardev=char{fsid},tag={mount_tag}"])
- if memory is None:
- memory = "128M"
- elif memory == 0:
+ qemuargs.extend(["-device",
f"{vhost_dev_type},chardev=char{fsid},tag={config.mount_tag}"])
+
+ memory = config.memory if config.memory is not None else "128M"
+ if memory == 0:
return True
+
qemuargs.extend(["-object",
f"memory-backend-memfd,id=mem,size={memory},share=on"])
qemuargs.extend(["-numa", "node,memdev=mem"])
return True
+class VirtFSConfig:
+ def __init__(self, path: str, mount_tag: str, security_model="none",
readonly=True):
+ self.path = path
+ self.mount_tag = mount_tag
+ self.security_model = security_model
+ self.readonly = readonly
+
+
def export_virtfs(
qemu: qemu_helpers.Qemu,
arch: architectures.Arch,
qemuargs: List[str],
- path: str,
- mount_tag: str,
- security_model="none",
- readonly=True,
+ config: VirtFSConfig,
) -> None:
# NB: We can't use -virtfs for this, because it can't handle a mount_tag
# that isn't a valid QEMU identifier.
@@ -704,9 +729,9 @@
"local,id=%s,path=%s,security_model=%s%s%s"
% (
fsid,
- qemu.quote_optarg(path),
- security_model,
- ",readonly=on" if readonly else "",
+ qemu.quote_optarg(config.path),
+ config.security_model,
+ ",readonly=on" if config.readonly else "",
",multidevs=remap" if qemu.has_multidevs else "",
),
]
@@ -715,7 +740,7 @@
[
"-device",
"%s,fsdev=%s,mount_tag=%s"
- % (arch.virtio_dev_type("9p"), fsid, qemu.quote_optarg(mount_tag)),
+ % (arch.virtio_dev_type("9p"), fsid,
qemu.quote_optarg(config.mount_tag)),
]
)
@@ -817,7 +842,7 @@
args = _ARGPARSER.parse_args()
arch = architectures.get(args.arch)
- is_native = args.arch == uname.machine
+ is_native = args.arch == platform.machine()
qemu = qemu_helpers.Qemu(args.qemu_bin, arch.qemuname)
qemu.probe()
@@ -864,9 +889,12 @@
# Propagate /proc/sys/fs/nr_open from the host to the guest, otherwise we
# may see some EPERM errors, because certain applications/settings may
# expect to be able to use a higher limit of the max number of open files.
- with open('/proc/sys/fs/nr_open', 'r', encoding="utf-8") as file:
- nr_open = file.readline().strip()
- kernelargs.append(f"nr_open={nr_open}")
+ try:
+ with open('/proc/sys/fs/nr_open', 'r', encoding="utf-8") as file:
+ nr_open = file.readline().strip()
+ kernelargs.append(f"nr_open={nr_open}")
+ except FileNotFoundError:
+ pass
# Parse NUMA settings.
if args.numa:
@@ -926,16 +954,19 @@
virt_arch = architectures.get("microvm")
else:
virt_arch = arch
- use_virtiofs = export_virtiofs(
- virt_arch,
- qemuargs,
- args.root,
- "ROOTFS",
+ virtiofs_config = VirtioFSConfig(
+ path=args.root,
+ mount_tag="ROOTFS",
guest_tools_path=guest_tools_path,
# virtiofsd requires a NUMA not, if --numa is specified simply use
# the user-defined NUMA node, otherwise create a NUMA node with all
# the memory.
memory=0 if args.numa else args.memory,
+ )
+ use_virtiofs = export_virtiofs(
+ virt_arch,
+ qemuargs,
+ virtiofs_config,
verbose=args.verbose,
)
if can_use_microvm(args) and use_virtiofs:
@@ -943,9 +974,12 @@
sys.stderr.write("virtme: use 'microvm' QEMU architecture\n")
arch = virt_arch
if not use_virtiofs:
- export_virtfs(
- qemu, arch, qemuargs, args.root, "/dev/root", readonly=(not
args.rw)
+ virtfs_config = VirtFSConfig(
+ path=args.root,
+ mount_tag="/dev/root",
+ readonly=(not args.rw),
)
+ export_virtfs(qemu, arch, qemuargs, virtfs_config)
# Use the faster virtme-ng-init if we are running on a native architecture.
if (
@@ -960,7 +994,11 @@
if args.root == "/":
initcmds = [f"init={guest_tools_path}/{virtme_init_cmd}"]
else:
- export_virtfs(qemu, arch, qemuargs, guest_tools_path,
"virtme.guesttools")
+ virtfs_config = VirtFSConfig(
+ path=guest_tools_path,
+ mount_tag="virtme.guesttools",
+ )
+ export_virtfs(qemu, arch, qemuargs, virtfs_config)
initcmds = [
"init=/bin/sh",
"--",
@@ -1027,9 +1065,12 @@
idx = mount_index
mount_index += 1
tag = "virtme.initmount%d" % idx
- export_virtfs(
- qemu, arch, qemuargs, hostpath, tag, readonly=(dirtype != "rwdir")
+ virtfs_config = VirtFSConfig(
+ path=hostpath,
+ mount_tag=tag,
+ readonly=(dirtype != "rwdir"),
)
+ export_virtfs(qemu, arch, qemuargs, virtfs_config)
kernelargs.append("virtme_initmount%d=%s" % (idx, guestpath))
for i, d in enumerate(args.overlay_rwdir):
@@ -1037,11 +1078,14 @@
# Turn on KVM if available
kvm_ok = can_use_kvm(args)
- if is_native and kvm_ok:
- qemuargs.extend(["-machine", "accel=kvm:tcg"])
+ if is_native:
+ if kvm_ok:
+ qemuargs.extend(["-machine", "accel=kvm:tcg"])
+ elif platform.system() == "Darwin":
+ qemuargs.extend(["-machine", "accel=hvf"])
# Add architecture-specific options
- qemuargs.extend(arch.qemuargs(is_native, kvm_ok))
+ qemuargs.extend(arch.qemuargs(is_native, kvm_ok, args.nvgpu is not None))
# Set up / override baseline devices
qemuargs.extend(["-parallel", "none"])
@@ -1077,7 +1121,10 @@
kernelargs.extend(["virtme_console=" + arg for arg in
arch.serial_console_args()])
- qemuargs.extend(arch.qemu_nodisplay_args())
+ if args.nvgpu is None:
+ qemuargs.extend(arch.qemu_nodisplay_args())
+ else:
+ qemuargs.extend(arch.qemu_nodisplay_nvgpu_args())
# PS/2 probing is slow; give the kernel a hint to speed it up.
kernelargs.extend(["psmouse.proto=exps"])
@@ -1162,7 +1209,10 @@
def do_script(shellcmd: str, ret_path=None, show_boot_console=False) ->
None:
if args.graphics is None:
# Turn off default I/O
- qemuargs.extend(arch.qemu_nodisplay_args())
+ if args.nvgpu is None:
+ qemuargs.extend(arch.qemu_nodisplay_args())
+ else:
+ qemuargs.extend(arch.qemu_nodisplay_nvgpu_args())
# Check if we can redirect stdin/stdout/stderr.
if not can_access_file("/proc/self/fd/0") or \
@@ -1275,7 +1325,7 @@
show_boot_console=args.show_boot_console,
)
- if args.graphics is not None:
+ if args.graphics is not None and args.nvgpu is None:
video_args = arch.qemu_display_args()
if video_args:
qemuargs.extend(video_args)
@@ -1329,6 +1379,9 @@
if args.user:
kernelargs.append("virtme_user=%s" % args.user)
+ if args.nvgpu:
+ qemuargs.extend(["-device", args.nvgpu])
+
# If we are running as root on the host pass this information to the guest
# (this can be useful to properly support running virtme-ng instances
# inside docker)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/virtme/cpiowriter.py
new/virtme-ng-1.31/virtme/cpiowriter.py
--- old/virtme-ng-1.29/virtme/cpiowriter.py 2024-09-11 08:38:45.000000000
+0200
+++ new/virtme-ng-1.31/virtme/cpiowriter.py 2024-10-18 15:30:58.000000000
+0200
@@ -5,6 +5,30 @@
# as a file called LICENSE with SHA-256 hash:
# 8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643
+class FileMetaData:
+ def __init__(self, **kwargs):
+ # Define default values for the metadata
+ defaults = {
+ 'ino': None,
+ 'nlink': None,
+ 'uid': 0,
+ 'gid': 0,
+ 'mtime': 0,
+ 'devmajor': 0,
+ 'devminor': 0,
+ 'rdevmajor': 0,
+ 'rdevminor': 0
+ }
+
+ # Update defaults with any provided keyword arguments
+ self.meta_data = {**defaults, **kwargs}
+
+ def get(self, key):
+ return self.meta_data.get(key)
+
+ def set(self, key, value):
+ self.meta_data[key] = value
+
class CpioWriter:
TYPE_DIR = 0o0040000
@@ -22,23 +46,13 @@
self.__f.write(data)
self.__totalsize += len(data)
- def write_object(
- self,
- name,
- body,
- mode,
- ino=None,
- nlink=None,
- uid=0,
- gid=0,
- mtime=0,
- devmajor=0,
- devminor=0,
- rdevmajor=0,
- rdevminor=0,
- ):
- if nlink is None:
- nlink = 2 if (mode & CpioWriter.TYPE_MASK) == CpioWriter.TYPE_DIR
else 1
+ def write_object(self, name, body, mode, meta_data=None):
+ # Set default metadata if not provided
+ meta_data = meta_data or FileMetaData()
+
+ # Ensure nlink is set correctly based on mode
+ if meta_data.get('nlink') is None:
+ meta_data.set('nlink', 2 if (mode & CpioWriter.TYPE_MASK) ==
CpioWriter.TYPE_DIR else 1)
if b"\0" in name:
raise ValueError("Filename cannot contain a NUL")
@@ -51,25 +65,28 @@
filesize = body.seek(0, 2)
body.seek(0)
- if ino is None:
- ino = self.__next_ino
+ # Set default ino if not provided
+ if meta_data.get('ino') is None:
+ meta_data.set('ino', self.__next_ino)
self.__next_ino += 1
+ # Prepare fields list using metadata
fields = [
- ino,
+ meta_data.get('ino'),
mode,
- uid,
- gid,
- nlink,
- mtime,
+ meta_data.get('uid'),
+ meta_data.get('gid'),
+ meta_data.get('nlink'),
+ meta_data.get('mtime'),
filesize,
- devmajor,
- devminor,
- rdevmajor,
- rdevminor,
+ meta_data.get('devmajor'),
+ meta_data.get('devminor'),
+ meta_data.get('rdevmajor'),
+ meta_data.get('rdevminor'),
namesize,
0,
]
+
hdr = ("070701" + "".join("%08X" % f for f in fields)).encode("ascii")
self.__write(hdr)
@@ -89,14 +106,14 @@
self.__write(((-filesize) % 4) * b"\0")
def write_trailer(self):
- self.write_object(name=b"TRAILER!!!", body=b"", mode=0, ino=0, nlink=1)
+ self.write_object(name=b"TRAILER!!!", body=b"", mode=0,
meta_data=FileMetaData(ino=0, nlink=1))
self.__write(((-self.__totalsize) % 512) * b"\0")
def mkdir(self, name, mode):
- self.write_object(name=name, mode=CpioWriter.TYPE_DIR | mode, body=b"")
+ self.write_object(name=name, body=b"", mode=CpioWriter.TYPE_DIR | mode)
def symlink(self, src, dst):
- self.write_object(name=dst, mode=CpioWriter.TYPE_SYMLINK | 0o777,
body=src)
+ self.write_object(name=dst, body=src, mode=CpioWriter.TYPE_SYMLINK |
0o777)
def write_file(self, name, body, mode):
self.write_object(name=name, body=body, mode=CpioWriter.TYPE_REG |
mode)
@@ -105,8 +122,10 @@
major, minor = dev
self.write_object(
name=name,
- mode=CpioWriter.TYPE_CHRDEV | mode,
- rdevmajor=major,
- rdevminor=minor,
body=b"",
- )
+ mode=CpioWriter.TYPE_CHRDEV | mode,
+ meta_data=FileMetaData(
+ rdevmajor=major,
+ rdevminor=minor,
+ )
+ )
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/virtme/modfinder.py
new/virtme-ng-1.31/virtme/modfinder.py
--- old/virtme-ng-1.29/virtme/modfinder.py 2024-09-11 08:38:45.000000000
+0200
+++ new/virtme-ng-1.31/virtme/modfinder.py 2024-10-18 15:30:58.000000000
+0200
@@ -15,7 +15,7 @@
import re
import subprocess
-import os
+import platform
import itertools
from . import util
@@ -29,7 +29,7 @@
args += ["-C", "/var/empty"]
if root is not None:
args += ["-d", root]
- if kver is not None and kver != os.uname().release:
+ if kver is not None and kver != platform.release():
# If booting the loaded kernel, skip -S. This helps certain
# buggy modprobe versions that don't support -S.
args += ["-S", kver]
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/virtme/qemu_helpers.py
new/virtme-ng-1.31/virtme/qemu_helpers.py
--- old/virtme-ng-1.29/virtme/qemu_helpers.py 2024-09-11 08:38:45.000000000
+0200
+++ new/virtme-ng-1.31/virtme/qemu_helpers.py 2024-10-18 15:30:58.000000000
+0200
@@ -6,6 +6,7 @@
# 8177f97513213526df2cf6184d8ff986c675afb514d4e68a404010521b880643
import os
+import platform
import re
import shutil
import subprocess
@@ -23,7 +24,7 @@
if not qemubin:
qemubin = shutil.which("qemu-system-%s" % arch)
- if qemubin is None and arch == os.uname().machine:
+ if qemubin is None and arch == platform.machine():
qemubin = shutil.which("qemu-kvm")
if qemubin is None:
raise ValueError("cannot find qemu for %s" % arch)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/virtme/util.py
new/virtme-ng-1.31/virtme/util.py
--- old/virtme-ng-1.29/virtme/util.py 2024-09-11 08:38:45.000000000 +0200
+++ new/virtme-ng-1.31/virtme/util.py 2024-10-18 15:30:58.000000000 +0200
@@ -12,8 +12,6 @@
import getpass
import itertools
-uname = os.uname()
-
class SilentError(Exception):
pass
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/virtme_ng/mainline.py
new/virtme-ng-1.31/virtme_ng/mainline.py
--- old/virtme-ng-1.29/virtme_ng/mainline.py 2024-09-11 08:38:45.000000000
+0200
+++ new/virtme-ng-1.31/virtme_ng/mainline.py 2024-10-18 15:30:58.000000000
+0200
@@ -25,7 +25,7 @@
self.version = version
self.arch = arch
self.verbose = verbose
- self.target = f"{self.kernel_dir}/boot/vmlinuz*generic*"
+ self.target = f"{self.kernel_dir}/boot/vmlinuz*generic"
if not glob(self.target):
self._fetch_kernel()
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/virtme_ng/run.py
new/virtme-ng-1.31/virtme_ng/run.py
--- old/virtme-ng-1.29/virtme_ng/run.py 2024-09-11 08:38:45.000000000 +0200
+++ new/virtme-ng-1.31/virtme_ng/run.py 2024-10-18 15:30:58.000000000 +0200
@@ -6,6 +6,7 @@
import argparse
import re
import os
+import platform
import sys
import socket
import shutil
@@ -29,7 +30,7 @@
# pylint: disable=unused-argument
pass
-from virtme.util import SilentError, uname, get_username
+from virtme.util import SilentError, get_username
from virtme_ng.utils import CONF_FILE, spinner_decorator
from virtme_ng.mainline import KernelDownloader
from virtme_ng.version import VERSION
@@ -114,7 +115,7 @@
"-r",
action="store",
nargs="?",
- const=uname.release,
+ const=platform.release(),
default=None,
help="Run a specified kernel; "
"--run can accept one of the following arguments: 1) nothing (in this "
@@ -465,6 +466,13 @@
nargs="*",
help="Additional Makefile variables",
)
+
+ parser.add_argument(
+ "--nvgpu",
+ action="store",
+ metavar="[GPU PCI Address]",
+ help="Add a passthrough NVIDIA GPU",
+ )
return parser
@@ -535,11 +543,17 @@
# Use Ubuntu's cloud images to create a rootfs, these images are fairly
# small and they provide a nice environment to test kernels.
if release is None:
- release = (
- check_output("lsb_release -s -c", shell=True)
- .decode(sys.stdout.encoding)
- .rstrip()
- )
+ try:
+ release = (
+ check_output("lsb_release -s -c", shell=True)
+ .decode(sys.stdout.encoding)
+ .rstrip()
+ )
+ if release == "n/a":
+ raise ValueError("unknown release")
+ except (CalledProcessError, ValueError):
+ print("Unknown release, try specifying an Ubuntu release with
--root-release")
+ sys.exit(1)
url = (
"https://cloud-images.ubuntu.com/";
+ f"{release}/current/{release}-server-cloudimg-{arch}-root.tar.xz"
@@ -551,6 +565,20 @@
os.chdir(prevdir)
+def get_host_arch():
+ """Translate host architecture to the corresponding virtme-ng arch name."""
+ arch = platform.machine()
+ arch_map = {
+ 'x86_64': 'amd64',
+ 'aarch64': 'arm64',
+ 'armv7l': 'armhf',
+ 'ppc64le': 'ppc64el',
+ 'riscv64': 'riscv64',
+ 's390x': 's390x',
+ }
+ return arch_map.get(arch, None)
+
+
class KernelSource:
"""Main class that implement actions to perform on a kernel source
directory."""
@@ -745,7 +773,7 @@
arg_fail(f"unsupported architecture: {arch}")
target = ARCH_MAPPING[arch]["kernel_target"]
cross_compile = ARCH_MAPPING[arch]["cross_compile"]
- if args.cross_compile != "":
+ if args.cross_compile:
cross_compile = args.cross_compile
cross_arch = ARCH_MAPPING[arch]["linux_name"]
@@ -816,7 +844,7 @@
def _get_virtme_root(self, args):
if args.root is not None:
- create_root(args.root, args.arch or "amd64", args.root_release)
+ create_root(args.root, args.arch or get_host_arch(),
args.root_release)
self.virtme_param["root"] = f"--root {args.root}"
else:
self.virtme_param["root"] = ""
@@ -840,12 +868,12 @@
def _get_virtme_rodir(self, args):
self.virtme_param["rodir"] = ""
for item in args.rodir:
- self.virtme_param["rodir"] += "--rodir " + item
+ self.virtme_param["rodir"] += f"--rodir {item} "
def _get_virtme_rwdir(self, args):
self.virtme_param["rwdir"] = ""
for item in args.rwdir:
- self.virtme_param["rwdir"] += "--rwdir " + item
+ self.virtme_param["rwdir"] += f"--rwdir {item} "
def _get_virtme_overlay_rwdir(self, args):
# Set default overlays if rootfs is mounted in read-only mode.
@@ -865,9 +893,9 @@
# If an upstream version is specified (using an upstream tag) fetch
# and run the corresponding kernel from the Ubuntu mainline
# repository.
- if re.match(r'^v\d+(\.\d+)*$', args.run):
+ if re.match(r'^v\d+(\.\d+)*(-rc\d+)?$', args.run):
if args.arch is None:
- arch = 'amd64'
+ arch = get_host_arch()
else:
arch = args.arch
try:
@@ -886,7 +914,7 @@
self.virtme_param["kdir"] = "--kdir ./"
def _get_virtme_mods(self, args):
- if args.skip_modules:
+ if args.skip_modules or platform.system() != "Linux":
self.virtme_param["mods"] = "--mods none"
else:
self.virtme_param["mods"] = "--mods auto"
@@ -1041,6 +1069,12 @@
else:
self.virtme_param["qemu_opts"] = ""
+ def _get_virtme_nvgpu(self, args):
+ if args.nvgpu is not None:
+ self.virtme_param["nvgpu"] = f"--nvgpu
'vfio-pci,host={args.nvgpu}'"
+ else:
+ self.virtme_param["nvgpu"] = ""
+
def run(self, args):
"""Execute a kernel inside virtme-ng."""
self._get_virtme_name(args)
@@ -1076,6 +1110,7 @@
self._get_virtme_busybox(args)
self._get_virtme_qemu(args)
self._get_virtme_qemu_opts(args)
+ self._get_virtme_nvgpu(args)
# Start VM using virtme-run
cmd = (
@@ -1113,6 +1148,7 @@
+ f'{self.virtme_param["busybox"]} '
+ f'{self.virtme_param["qemu"]} '
+ f'{self.virtme_param["qemu_opts"]} '
+ + f'{self.virtme_param["nvgpu"]} '
)
check_call(cmd, shell=True)
diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn'
'--exclude=.svnignore' old/virtme-ng-1.29/virtme_ng/version.py
new/virtme-ng-1.31/virtme_ng/version.py
--- old/virtme-ng-1.29/virtme_ng/version.py 2024-09-11 08:38:45.000000000
+0200
+++ new/virtme-ng-1.31/virtme_ng/version.py 2024-10-18 15:30:58.000000000
+0200
@@ -7,7 +7,7 @@
from subprocess import check_output, DEVNULL, CalledProcessError
import pkg_resources
-PKG_VERSION = "1.29"
+PKG_VERSION = "1.31"
def get_package_version():
@@ -34,7 +34,7 @@
# Otherwise fallback to the static version defined in PKG_VERSION.
version = (
check_output(
- "cd %s && [ -e ../.git ] && git describe --always --long
--dirty" % os.path.dirname(__file__),
+ "cd %s && [ -e ../.git ] && git describe --long --dirty" %
os.path.dirname(__file__),
shell=True,
stderr=DEVNULL,
)
