Re: [pve-devel] [PATCH storage v2 1/3] compact regex for backup file filter
On Wed, Feb 12, 2020 at 11:19:06AM +0100, Thomas Lamprecht wrote:
> On 1/31/20 5:00 PM, Alwin Antreich wrote:
> > --- a/PVE/Storage/Plugin.pm
> > +++ b/PVE/Storage/Plugin.pm
> > @@ -423,7 +423,7 @@ sub parse_volname {
> > return ('vztmpl', $1);
> > } elsif ($volname =~ m!^rootdir/(\d+)$!) {
> > return ('rootdir', $1, $1);
> > -} elsif ($volname =~
> > m!^backup/([^/]+(\.(tar|tar\.gz|tar\.lzo|tgz|vma|vma\.gz|vma\.lzo)))$!) {
> > +} elsif ($volname =~
> > m!^backup/([^/]+(\.(tgz|((tar|vma)(\.(gz|lzo))?$!) {
> > my $fn = $1;
> > if ($fn =~ m/^vzdump-(openvz|lxc|qemu)-(\d+)-.+/) {
> > return ('backup', $fn, $2);
> > @@ -910,7 +910,7 @@ my $get_subdir_files = sub {
> >
> > } elsif ($tt eq 'backup') {
> > next if defined($vmid) && $fn !~ m/\S+-$vmid-\S+/;
> > - next if $fn !~
> > m!/([^/]+\.(tar|tar\.gz|tar\.lzo|tgz|vma|vma\.gz|vma\.lzo))$!;
> > + next if $fn !~ m!/([^/]+\.(tgz|((tar|vma)(\.(gz|lzo))?)))$!;
> >
> > $info = { volid => "$sid:backup/$1", format => $2 };
> writing test for this up front as a first patch would be great.
> It should be pretty simple, i.e., have a array of maps with params and
> expected
> result as two key => value entries. Additionally "expected error" checks
> would be
> great too.
> While there's already a sub test in zfspoolplugin test I'd just do a new one.
>
> This way we would have a safety net for such and future changes.
There are no negative tests or maps, but I once posted some tests for
parse_volname
[0] to the list. Maybe they could be a starting point.
[0] https://pve.proxmox.com/pipermail/pve-devel/2019-December/040887.html
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH ifupdown2] add patch to ifup/ifdown allow=ovs on start/stop
This is fixing possible race condition with ovs 2.12 ans also cleanly remove ovs interfaces on networking service stop Signed-off-by: Alexandre Derumier --- ...ce-ifup-ifdown-allow-ovs-on-start-st.patch | 26 +++ debian/patches/series | 1 + 2 files changed, 27 insertions(+) create mode 100644 debian/patches/pve/0009-networking.service-ifup-ifdown-allow-ovs-on-start-st.patch diff --git a/debian/patches/pve/0009-networking.service-ifup-ifdown-allow-ovs-on-start-st.patch b/debian/patches/pve/0009-networking.service-ifup-ifdown-allow-ovs-on-start-st.patch new file mode 100644 index 000..0d7b6fa --- /dev/null +++ b/debian/patches/pve/0009-networking.service-ifup-ifdown-allow-ovs-on-start-st.patch @@ -0,0 +1,26 @@ +From aaf288fd97929140348d2de6751c4ca758997455 Mon Sep 17 00:00:00 2001 +From: Alexandre Derumier +Date: Thu, 13 Feb 2020 07:16:33 +0100 +Subject: [PATCH] networking.service: ifup/ifdown allow=ovs on start/stop + +Signed-off-by: Alexandre Derumier +--- + debian/ifupdown2.networking.service | 2 ++ + 1 file changed, 2 insertions(+) + +diff --git a/debian/ifupdown2.networking.service b/debian/ifupdown2.networking.service +index b2acd97..4347bd2 100644 +--- a/debian/ifupdown2.networking.service b/debian/ifupdown2.networking.service +@@ -11,6 +11,8 @@ RemainAfterExit=yes + SyslogIdentifier=networking + TimeoutStopSec=30s + ExecStart=/sbin/ifup -a ++ExecStart=/sbin/ifup --allow=ovs ++ExecStop=/sbin/ifdown --allow=ovs + ExecStop=/sbin/ifdown -a + ExecReload=/sbin/ifreload -a + +-- +2.20.1 + diff --git a/debian/patches/series b/debian/patches/series index 6f81e1f..0f3c1fa 100644 --- a/debian/patches/series +++ b/debian/patches/series @@ -7,3 +7,4 @@ pve/0005-ifreload-down-up-vxlan-interfaces-when-ifreload_down.patch pve/0006-config-tuning.patch pve/0007-networking.service-fix-dependencies-and-ordering.patch pve/0008-execute-addons-scripts-before-modules.patch +pve/0009-networking.service-ifup-ifdown-allow-ovs-on-start-st.patch -- 2.20.1 ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH openvswitch 4/7] update changelog to 2.12.0
--- debian/changelog | 1551 ++ 1 file changed, 52 insertions(+), 1499 deletions(-) diff --git a/debian/changelog b/debian/changelog index 9d0196e..e687156 100644 --- a/debian/changelog +++ b/debian/changelog @@ -1,1546 +1,99 @@ -openvswitch (2.11.0+2019.06.25+git.9ebe795035+ds1-8) unstable; urgency=medium +openvswitch (2.12.0-1) buster; urgency=low - * Removed python:any dependency (Closes: #937216). + * update to 2.12.0 - -- Thomas Goirand Mon, 20 Jan 2020 08:46:10 +0100 + -- Proxmox Support Team Fri, 12 Feb 2020 22:00:00 +0200 -openvswitch (2.11.0+2019.06.25+git.9ebe795035+ds1-7) unstable; urgency=medium +openvswitch (2.7.0-3) stretch; urgency=low - * Fix moving ovs-vswitchd (Closes: #943990). + * ifupdown2 compatibility fixes - -- Thomas Goirand Sat, 02 Nov 2019 20:01:23 +0100 + -- Proxmox Support Team Fri, 15 Jun 2018 10:06:52 +0200 -openvswitch (2.11.0+2019.06.25+git.9ebe795035+ds1-6) unstable; urgency=medium +openvswitch (2.7.0-2) stretch; urgency=low - * Fix again moving /usr/sbin/ovs-vswitchd away. + * added missing dependency: net-tools - -- Thomas Goirand Fri, 01 Nov 2019 11:57:30 +0100 + -- Proxmox Support Team Mon, 27 Mar 2017 14:23:35 +0200 -openvswitch (2.11.0+2019.06.25+git.9ebe795035+ds1-5) unstable; urgency=medium +openvswitch (2.7.0-1) unstable; urgency=medium - * Add || true when moving /usr/sbin/ovs-vswitchd so that it also works on -arch all. + * update to 2.7.0, recompile for Debian stretch - -- Thomas Goirand Fri, 01 Nov 2019 11:42:17 +0100 + -- Proxmox Support Team Fri, 17 Mar 2017 17:11:08 +0100 -openvswitch (2.11.0+2019.06.25+git.9ebe795035+ds1-4) unstable; urgency=medium +openvswitch (2.6.0-2) unstable; urgency=medium - * Fix installing vtep/vtep-ctl (Closes: #866319). - * Fix moving /usr/sbin/ovs-vswitchd around on non-dpdk arch, fixing FTBFS on -these. - * Removed SYSTEMCTL_SKIP_REDIRECT from: -- openvswitch-switch init scripts (Closes: #910474. -- ovn-central.init (Closes: #910472). -- ovn-host.init (Closes: #910475). -- ovn-controller-vtep.init (Closes: #910476). - * Added mkdir -p /var/run in ovs startup (Closes: #930843). + * drop System V init script - -- Thomas Goirand Thu, 31 Oct 2019 11:38:08 +0100 + -- Proxmox Support Team Mon, 21 Nov 2016 15:35:52 +0100 -openvswitch (2.11.0+2019.06.25+git.9ebe795035+ds1-3) unstable; urgency=medium +openvswitch (2.6.0-1) unstable; urgency=medium - [ Ondřej Nový ] - * Run wrap-and-sort -bastk. - * Use debhelper-compat instead of debian/compat. + * update to 2.6.0 - [ Thomas Goirand ] - * Rebuilt source-only. + -- Proxmox Support Team Mon, 21 Nov 2016 15:15:48 +0100 - -- Thomas Goirand Tue, 29 Oct 2019 15:17:29 +0100 +openvswitch (2.5.0-1) unstable; urgency=medium -openvswitch (2.11.0+2019.06.25+git.9ebe795035+ds1-2) unstable; urgency=medium + * update to 2.5.0 - [ Jakub Safarik ] - * Reintroduce the openvswitch-ipsec package. - * Fix lintian errors: -- openvswitch-switch: init.d-script-not-included-in-package. -- openvswitch source: python-depends-but-no-python-helper. -- openvswitch source: missing-notice-file-for-apache-license. + -- Proxmox Support Team Wed, 20 Apr 2016 07:00:45 +0200 - [ Thomas Goirand ] - * Fixed d/copyright (Closes: #942056). +openvswitch (2.3.2-3) unstable; urgency=medium - -- Thomas Goirand Thu, 10 Oct 2019 09:13:37 +0200 + * Fix CVE-2016-2074 -openvswitch (2.11.0+2019.06.25+git.9ebe795035+ds1-1) experimental; urgency=medium + -- Proxmox Support Team Fri, 01 Apr 2016 07:39:07 +0200 - * New upstream release. - * Rebased patches. - * Removed patch applied upstream: -- ovs-dev-ovs-macros-Make-tests-log-how-long-they-waited-when-they-s...diff - * Ran wrap-and-sort -bast. - * Add dpdk support. - * Converge with Ubuntu package. - * Add 0001-acinclude-Also-use-LIBS-from-dpkg-pkg-config.patch. - * Add disable-failing-ovn-tests.patch. - * Removed Python 2 support (Closes: #937216). - * Fixed debian/ifupdown.sh typo: ovs_vsctl -> ovs-vsctl. - * Remove --parallel when calling dh. - * Temporarily remove bugtools from openvswitch-common.install (it's -currently broken, and need a fix). +openvswitch (2.3.2-2) unstable; urgency=medium - -- Thomas Goirand Thu, 11 Jul 2019 22:31:36 +0200 + * fix systemd service dependencies -openvswitch (2.10.0+2018.08.28+git.8ca7c82b7d+ds1-13) unstable; urgency=medium + -- Proxmox Support Team Fri, 27 Nov 2015 10:58:40 +0100 - * Some fixups in debian/ifupdown.sh to allow setting-up the MTU. - * Document how to do Bond + Bridge + VLAN + MTU. - * Correct dependency on python3-six instead of python-six (Closes: #931104). +openvswitch (2.3.2-1) unstable; urgency=medium - -- Thomas Goirand Mon, 24 Jun 2019 08:53:33 +0200 + * update to 2.3.2 -openvswitch (2.10.0+2018.08.28+git.8ca7c82b7d+ds1-12) unstable; urgency=medium + -- Proxmox Support Team Sat, 27 Jun 2015 18:37:45
[pve-devel] [PATCH openvswitch 2/7] update makefile to 2.12
---
Makefile | 15 +--
1 file changed, 5 insertions(+), 10 deletions(-)
diff --git a/Makefile b/Makefile
index ff48b92..c76b5d8 100644
--- a/Makefile
+++ b/Makefile
@@ -1,6 +1,6 @@
# also add entry in changelog.Debian
-OVSVER=2.7.0
-PKGRELEASE=3
+OVSVER=2.12.0
+PKGRELEASE=1
OVSDIR=openvswitch-${OVSVER}
OVSSRC=openvswitch-${OVSVER}.tar.gz
@@ -21,14 +21,9 @@ $(DEB2): $(DEB1)
$(DEB1): ${OVSSRC}
rm -rf ${OVSDIR}
tar xf ${OVSSRC}
- cd ${OVSDIR}; ln -s ../pvepatches patches
- cd ${OVSDIR}; quilt push -a
- mv ${OVSDIR}/debian/changelog ${OVSDIR}/debian/changelog.org
- cat changelog.Debian ${OVSDIR}/debian/changelog.org>
${OVSDIR}/debian/changelog
- echo "git clone git://git.proxmox.com/git/openvswitch.git\\ngit
checkout ${GITVERSION}" > ${OVSDIR}/debian/SOURCE
- echo "debian/SOURCE" >> ${OVSDIR}/debian/openvswitch-common.docs
- echo "debian/SOURCE" >> ${OVSDIR}/debian/openvswitch-switch.docs
- cd ${OVSDIR}; dpkg-buildpackage -b -jauto -us -uc
+ rm -rf ${OVSDIR}/debian
+ cp -rf debian ${OVSDIR}
+ cd ${OVSDIR}; DEB_BUILD_OPTIONS=nocheck dpkg-buildpackage -b -jauto -us
-uc
.PHONY: download
${OVSSRC} download:
--
2.20.1
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH openvswitch 5/7] remove 2 patches not working on 2.12
--- ...e-Also-use-LIBS-from-dpkg-pkg-config.patch | 366 -- .../patches/disable-failing-ovn-tests.patch | 327 debian/patches/series | 2 - 3 files changed, 695 deletions(-) delete mode 100644 debian/patches/0001-acinclude-Also-use-LIBS-from-dpkg-pkg-config.patch delete mode 100644 debian/patches/disable-failing-ovn-tests.patch diff --git a/debian/patches/0001-acinclude-Also-use-LIBS-from-dpkg-pkg-config.patch b/debian/patches/0001-acinclude-Also-use-LIBS-from-dpkg-pkg-config.patch deleted file mode 100644 index af649da..000 --- a/debian/patches/0001-acinclude-Also-use-LIBS-from-dpkg-pkg-config.patch +++ /dev/null @@ -1,366 +0,0 @@ -From 5b5aa2d8a54c006b6c5239d04b7a751ca5ff5d44 Mon Sep 17 00:00:00 2001 -From: Christian Ehrhardt -Date: Tue, 12 Feb 2019 07:29:58 +0100 -Subject: [PATCH] acinclude: Also use LIBS from dpkg pkg-config - -DPDK 18.11 builds using the more modern meson build system no more -provide the -ldpdk linker script. Instead it is expected to use -pkgconfig for linker options as well. - -This change will set DPDK_LIB from pkg-config (if pkg-config was -available) and since that already carries the whole-archive flags -around the PMDs skips the further wrapping in more whole-archive -if that is already part of DPDK_LIB. - -To work reliable in all environments this needs pkg-config 0.29.1. -We want to be able to use PKG_CHECK_MODULES_STATIC which -is not yet available in 0.24. Therefore update pkg.m4 -to pkg-config 0.29.1. - -This should be backport-safe as these macro files are all versioned. -autoconf is smart enough to check the version if you have it locally, -and if the system's is higher, it will use that one instead. - -Acked-by: Luca Boccassi -Acked-by: Aaron Conole -Signed-off-by: Christian Ehrhardt -Signed-off-by: Ian Stokes - -Origin: backport, https://github.com/openvswitch/ovs/commit/5b5aa2d8a54c006b6c5239d04b7a751ca5ff5d44 -Last-Update: 2019-02-14 - - acinclude.m4 | 20 +++-- - m4/pkg.m4| 217 +-- - 2 files changed, 153 insertions(+), 84 deletions(-) - -Index: openvswitch/acinclude.m4 -=== openvswitch.orig/acinclude.m4 -+++ openvswitch/acinclude.m4 -@@ -223,9 +223,11 @@ AC_DEFUN([OVS_CHECK_DPDK], [ - case "$with_dpdk" in - yes) - DPDK_AUTO_DISCOVER="true" --PKG_CHECK_MODULES([DPDK], [libdpdk], -- [DPDK_INCLUDE="$DPDK_CFLAGS"], -- [DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk"]) -+PKG_CHECK_MODULES_STATIC([DPDK], [libdpdk], [ -+DPDK_INCLUDE="$DPDK_CFLAGS" -+DPDK_LIB="$DPDK_LIBS"], [ -+DPDK_INCLUDE="-I/usr/local/include/dpdk -I/usr/include/dpdk" -+DPDK_LIB="-ldpdk"]) - ;; - *) - DPDK_AUTO_DISCOVER="false" -@@ -238,11 +240,10 @@ AC_DEFUN([OVS_CHECK_DPDK], [ -DPDK_INCLUDE="-I$DPDK_INCLUDE_PATH/dpdk" - fi - DPDK_LIB_DIR="$with_dpdk/lib" -+DPDK_LIB="-ldpdk" - ;; - esac - --DPDK_LIB="-ldpdk" -- - ovs_save_CFLAGS="$CFLAGS" - ovs_save_LDFLAGS="$LDFLAGS" - CFLAGS="$CFLAGS $DPDK_INCLUDE" -@@ -377,7 +378,14 @@ AC_DEFUN([OVS_CHECK_DPDK], [ - # - # These options are specified inside a single -Wl directive to prevent - # autotools from reordering them. --DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole-archive -+# -+# OTOH newer versions of dpdk pkg-config (generated with Meson) -+# will already have flagged just the right set of libs with -+# --whole-archive - in those cases do not wrap it once more. -+case "$DPDK_LIB" in -+ *whole-archive*) DPDK_vswitchd_LDFLAGS=$DPDK_LIB;; -+ *) DPDK_vswitchd_LDFLAGS=-Wl,--whole-archive,$DPDK_LIB,--no-whole-archive -+esac - AC_SUBST([DPDK_vswitchd_LDFLAGS]) - AC_DEFINE([DPDK_NETDEV], [1], [System uses the DPDK module.]) - fi -Index: openvswitch/m4/pkg.m4 -=== openvswitch.orig/m4/pkg.m4 -+++ openvswitch/m4/pkg.m4 -@@ -1,29 +1,60 @@ --# pkg.m4 - Macros to locate and utilise pkg-config.-*- Autoconf -*- --# serial 1 (pkg-config-0.24) --# --# Copyright © 2004 Scott James Remnant . --# --# This program is free software; you can redistribute it and/or modify --# it under the terms of the GNU General Public License as published by --# the Free Software Foundation; either version 2 of the License, or --# (at your option) any later version. --# --# This program is distributed in the hope that it will be useful, but --# WITHOUT ANY WARRANTY; without even the implied warranty of --# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU --# General Public License for more details. --# --# You should have received a copy of the GNU General Public License --# along with this
[pve-devel] [PATCH openvswitch 0/7] openvswitch 2.12 update
This patches series is reworked from debian sid ovs 2.11 patches. All is building fine (including ovs-dpdk is users want it) last patch is a fix for ovs_mtu with bond. (I have contacted the upstream maintainer for this) I have tested all differents kind of ovs interfaces, all is working fine. Tested with ifupdown && ifupdown2. I'm seeing some race condition with ovs 2.12 and ifupdown2 2.0, where ifupdown2 can't setup ip on ovs interfaces. (Seem to be something new in ovs, I have also tried ovs 2.12, keeping all olds systemd units and script from 2.10, and it's happening too) I'll send a small fix for ifupdown2 to avoid that today. Alexandre Derumier (7): remove old proxmox patches update makefile to 2.12 add debian sid (ovs 2.11) "debian" directory update changelog to 2.12.0 remove 2 patches not working on 2.12 readd bugtool-plugin to common package fix ovs_mtu for bond slaves Makefile | 15 +- debian/README.Debian | 36 + debian/README.source | 5 + changelog.Debian => debian/changelog | 6 + debian/control| 323 +++ debian/copyright | 750 debian/dkms.conf.in | 11 + debian/ifupdown.sh| 128 +++ debian/openvswitch-common.dirs| 1 + debian/openvswitch-common.docs| 1 + debian/openvswitch-common.install | 10 + debian/openvswitch-common.manpages| 41 + debian/openvswitch-common.postinst| 11 + debian/openvswitch-common.postrm | 11 + debian/openvswitch-common.prerm | 11 + debian/openvswitch-dev.install| 4 + debian/openvswitch-ipsec.init | 181 debian/openvswitch-ipsec.install | 1 + debian/openvswitch-pki.dirs | 1 + debian/openvswitch-pki.postinst | 41 + debian/openvswitch-pki.postrm | 43 + debian/openvswitch-switch-dpdk.postinst | 11 + debian/openvswitch-switch-dpdk.postrm | 11 + debian/openvswitch-switch-dpdk.prerm | 11 + debian/openvswitch-switch.README.Debian | 246 ++ debian/openvswitch-switch.dirs| 2 + debian/openvswitch-switch.init| 154 debian/openvswitch-switch.install | 2 + debian/openvswitch-switch.links | 2 + debian/openvswitch-switch.logrotate | 16 + ...witch-switch.openvswitch-nonetwork.service | 15 + debian/openvswitch-switch.postinst| 75 ++ debian/openvswitch-switch.postrm | 48 ++ debian/openvswitch-switch.service | 14 + debian/openvswitch-switch.template| 8 + .../openvswitch-testcontroller.README.Debian | 12 + debian/openvswitch-testcontroller.default | 29 + debian/openvswitch-testcontroller.dirs| 1 + debian/openvswitch-testcontroller.init| 278 ++ debian/openvswitch-testcontroller.postinst| 52 ++ debian/openvswitch-testcontroller.postrm | 44 + debian/openvswitch-vtep.default | 4 + debian/openvswitch-vtep.init | 79 ++ debian/openvswitch-vtep.install | 1 + debian/ovn-central.dirs | 1 + debian/ovn-central.init | 59 ++ debian/ovn-central.install| 2 + debian/ovn-central.postinst | 49 ++ debian/ovn-central.postrm | 48 ++ debian/ovn-central.template | 5 + debian/ovn-controller-vtep.init | 53 ++ debian/ovn-host.dirs | 1 + debian/ovn-host.init | 53 ++ debian/ovn-host.postinst | 49 ++ debian/ovn-host.postrm| 44 + debian/ovn-host.template | 5 + ...async-msg-ctrl-of1.3-because-of-mips.patch | 135 +++ debian/patches/disable-even-more-tests.patch | 140 +++ debian/patches/disable-failed-tests.patch | 20 + debian/patches/fix-ovs-monitor-ipsec.patch| 117 +++ debian/patches/py3-compat.patch | 812 ++ debian/patches/remove-bfd-decay-tests.patch | 158 .../remove-include-debian-automake.mk.patch | 17 + .../remove-non-deterministic-tests.patch | 46 + ...-tests-broken-in-mips64el-and-mipsel.patch | 52 ++ ...remove-yet-another-mips-failing-test.patch | 218 + ...-which-are-failing-in-mips-and-armel.patch | 305 +++ debian/patches/series | 12 + .../use-python3-m-sphinx-to-build-doc.patch | 18 + debian/python3-openvswitch.install| 2 + debian/rules | 133 +++ debian/salsa-ci.yml | 3 + debian/source/format
[pve-devel] [PATCH openvswitch 7/7] fix ovs_mtu for bond slaves
ovs_mtu can't be applied on bond directly,
it need to be setup on physical interfaces, but only
after they are in the bond.
Easy way to to apply ovs_mtu from bond interfaces, to it's slave
directly
---
debian/ifupdown.sh | 3 +++
1 file changed, 3 insertions(+)
diff --git a/debian/ifupdown.sh b/debian/ifupdown.sh
index 8c39b4d..04ce3cc 100755
--- a/debian/ifupdown.sh
+++ b/debian/ifupdown.sh
@@ -83,6 +83,9 @@ if [ "${MODE}" = "start" ]; then
for slave in ${IF_OVS_BONDS}
do
if_up "${slave}"
+if [ -n "${IF_OVS_MTU}" ] ; then
+ovs-vsctl set Interface "${slave}"
mtu_request=${IF_OVS_MTU}
+fi
done
;;
OVSPatchPort)
--
2.20.1
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH openvswitch 6/7] readd bugtool-plugin to common package
--- debian/openvswitch-common.install | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/debian/openvswitch-common.install b/debian/openvswitch-common.install index 2770725..1aa9563 100644 --- a/debian/openvswitch-common.install +++ b/debian/openvswitch-common.install @@ -1,5 +1,5 @@ -#usr/share/openvswitch/bugtool-plugins -#usr/share/openvswitch/scripts/ovs-bugtool-* +/usr/share/openvswitch/bugtool-plugins +/usr/share/openvswitch/scripts/ovs-bugtool-* /usr/share/openvswitch/scripts etc/bash_completion.d/ovs-appctl-bashcomp.bash usr/share/bash-completion/completions etc/bash_completion.d/ovs-vsctl-bashcomp.bash usr/share/bash-completion/completions -- 2.20.1 ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH pve-manager 1/2] add sdn gui
Hi, >>did you work on these gui patches, or is it roughly the same as last >>time? (as in: should i start reviewing these?) It's almost the same. Only some minor changes in forms field to match the last updates of the plugins. So no review needed for now. (I'll try to polish it a little bit more in coming weeks) >>i'll look at them regardless the next days, to see if we could >>apply some version of them and hide them for most users >>(like thomas suggested) Could be great :) Thanks! - Mail original - De: "Dominik Csapak" À: "pve-devel" Cc: "aderumier" Envoyé: Mercredi 12 Février 2020 12:56:22 Objet: Re: [pve-devel] [PATCH pve-manager 1/2] add sdn gui Hi, did you work on these gui patches, or is it roughly the same as last time? (as in: should i start reviewing these?) i'll look at them regardless the next days, to see if we could apply some version of them and hide them for most users (like thomas suggested) kind regards Dominik ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH v2 docs] rewrite and extend pct documentation
On 1/14/20 5:47 PM, Oguz Bektas wrote:
> * rephrase some parts.
> * update old information
> * add info about pending changes and other "new" features
>
> Co-Authored-by: Aaron Lauterer
> Signed-off-by: Oguz Bektas
> ---
>
> v1->v2:
> changed some of the writing in terms of phrasing and style, with
> feedback from aaron. thanks!
>
> pct.adoc | 442 ---
> 1 file changed, 259 insertions(+), 183 deletions(-)
>
> diff --git a/pct.adoc b/pct.adoc
> index 2f1d329..f8804e6 100644
> --- a/pct.adoc
> +++ b/pct.adoc
> @@ -28,32 +28,27 @@ ifdef::wiki[]
> :title: Linux Container
> endif::wiki[]
>
> -Containers are a lightweight alternative to fully virtualized
> -VMs. Instead of emulating a complete Operating System (OS), containers
> -simply use the OS of the host they run on. This implies that all
> -containers use the same kernel, and that they can access resources
> -from the host directly.
> +Containers are a lightweight alternative to fully virtualized VMs. They use
extra space before "They"
> +the kernel of the host system that they run on, instead of emulating a full
> +operating system (OS). This means that containers can access resources on the
> +host system directly.
Containers are a lightweight alternative to fully virtualized machines (VMs).
They use the kernel of the host system that they run on, instead of emulating a
full operating system (OS). This means that containers can access resources on
the host system directly.
>
> -This is great because containers do not waste CPU power nor memory due
> -to kernel emulation. Container run-time costs are close to zero and
> -usually negligible. But there are also some drawbacks you need to
> -consider:
> +The runtime costs for containers is low, usually negligible, because of the
> low
> +overhead in terms of CPU and memory resources. However, there are some
> drawbacks
> +that need be considered:
The runtime costs for containers is low, usually negligible. However, there are
some drawbacks that need be considered:
(makes no sense to say "runtime cost is low because overhead is low", that's
almost
like saying it's faster because it's faster
>
> -* You can only run Linux based OS inside containers, i.e. it is not
> - possible to run FreeBSD or MS Windows inside.
> +* Only Linux distributions can be run in containers, i.e. It is not
> + possible to run FreeBSD or MS Windows inside a container.
Avoid the "i.e." here (and if it would need a comma after and the "it" lower
case), maybe:
* Only Linux distributions can be run in containers. It is not possible to run
FreeBSD, MS Windows or other Operating Systems inside a container.
>
> -* For security reasons, access to host resources needs to be
> - restricted. This is done with AppArmor, SecComp filters and other
> - kernel features. Be prepared that some syscalls are not allowed
> - inside containers.
> +* For security reasons, access to host resources needs to be restricted. Some
> + syscalls are not allowed within containers. This is done with AppArmor,
> SecComp
> + filters, and other kernel features.
I'd drop the apparmor & seccomp mentioning, IMO not relevant here for a
introduction,
this could be nicer:
* For security reasons, a container may not access all of the host resources.
It runs in its own separate namespace. Additionally some syscalls are not
allowed within containers.
(but it still misses the real drawback, e.g., that some programs may not expect
this and thus not work out of the box, but that's not really easy to write here
while underlining that this is rather the exception)
>
> {pve} uses https://linuxcontainers.org/[LXC] as underlying container
> -technology. We consider LXC as low-level library, which provides
> -countless options. It would be too difficult to use those tools
> -directly. Instead, we provide a small wrapper called `pct`, the
> -"Proxmox Container Toolkit".
> +technology. The "Proxmox Container Toolkit" (`pct`) simplifies the usage of
> LXC
> +containers.
use ``Proxmox Container Toolkit'' to get nicely rendered quotes?
>
> -The toolkit is tightly coupled with {pve}. That means that it is aware
> +The `pct` is tightly coupled with {pve}. This means that it is aware
The `pct` sounds meh IMO,
Containers are tightly integrated with {pve}. This means that they are aware of
the cluster setup, and it can use the same network and storage resources as
virtual machines. You can even use the {pve} firewall, or manage containers
using the HA framework.
> of the cluster setup, and it can use the same network and storage
> resources as fully virtualized VMs. You can even use the {pve}
> firewall, or manage containers using the HA framework.
> @@ -62,7 +57,7 @@ Our primary goal is to offer an environment as one would
> get from a
> VM, but without the additional overhead. We call this "System
> Containers".
>
> -NOTE: If you want to run micro-containers (with docker,
[pve-devel] [PATCH v8 qemu-server 6/8] Rework get_cpu_options and allow custom CPU models
If a cputype is custom (check via prefix), try to load options from the
custom CPU model config, and set values accordingly.
While at it, extract currently hardcoded values into seperate sub and add
reasonings.
Since the new flag resolving outputs flags in sorted order for
consistency, adapt the test cases to not break. Only the order is
changed, not which flags are present.
Signed-off-by: Stefan Reiter
---
PVE/QemuServer/CPUConfig.pm | 191 +-
test/cfg2cmd/i440fx-win10-hostpci.conf.cmd| 2 +-
test/cfg2cmd/minimal-defaults.conf.cmd| 2 +-
test/cfg2cmd/pinned-version.conf.cmd | 2 +-
.../q35-linux-hostpci-multifunction.conf.cmd | 2 +-
test/cfg2cmd/q35-linux-hostpci.conf.cmd | 2 +-
test/cfg2cmd/q35-win10-hostpci.conf.cmd | 2 +-
test/cfg2cmd/simple1.conf.cmd | 2 +-
test/cfg2cmd/spice-enhancments.conf.cmd | 2 +-
test/cfg2cmd/spice-linux-4.1.conf.cmd | 2 +-
test/cfg2cmd/spice-usb3.conf.cmd | 2 +-
test/cfg2cmd/spice-win.conf.cmd | 2 +-
12 files changed, 152 insertions(+), 61 deletions(-)
diff --git a/PVE/QemuServer/CPUConfig.pm b/PVE/QemuServer/CPUConfig.pm
index 14c1bde..850a76c 100644
--- a/PVE/QemuServer/CPUConfig.pm
+++ b/PVE/QemuServer/CPUConfig.pm
@@ -377,99 +377,190 @@ sub print_cpuflag_hash {
return $formatted;
}
+sub parse_cpuflag_list {
+my ($re, $reason, $flaglist) = @_;
+
+my $res = {};
+return $res if !$flaglist;
+
+foreach my $flag (split(";", $flaglist)) {
+ if ($flag =~ $re) {
+ $res->{$2} = { op => $1, reason => $reason };
+ }
+}
+
+return $res;
+}
+
# Calculate QEMU's '-cpu' argument from a given VM configuration
sub get_cpu_options {
my ($conf, $arch, $kvm, $kvm_off, $machine_version, $winversion,
$gpu_passthrough) = @_;
-my $cpuFlags = [];
-my $ostype = $conf->{ostype};
-
-my $cpu = $kvm ? "kvm64" : "qemu64";
+my $cputype = $kvm ? "kvm64" : "qemu64";
if ($arch eq 'aarch64') {
- $cpu = 'cortex-a57';
-}
-my $hv_vendor_id;
-if (my $cputype = $conf->{cpu}) {
- my $cpuconf = PVE::JSONSchema::parse_property_string($cpu_fmt, $cputype)
- or die "Cannot parse cpu description: $cputype\n";
- $cpu = $cpuconf->{cputype};
- $kvm_off = 1 if $cpuconf->{hidden};
- $hv_vendor_id = $cpuconf->{'hv-vendor-id'};
-
- if (defined(my $flags = $cpuconf->{flags})) {
- push @$cpuFlags, split(";", $flags);
- }
+ $cputype = 'cortex-a57';
}
-push @$cpuFlags , '+lahf_lm' if $cpu eq 'kvm64' && $arch eq 'x86_64';
+my $cpu = {};
+my $custom_cpu;
+my $hv_vendor_id;
+if (my $cpu_prop_str = $conf->{cpu}) {
+ $cpu = parse_vm_cpu_conf($cpu_prop_str)
+ or die "Cannot parse cpu description: $cpu_prop_str\n";
-push @$cpuFlags , '-x2apic' if $ostype && $ostype eq 'solaris';
+ $cputype = $cpu->{cputype};
-push @$cpuFlags, '+sep' if $cpu eq 'kvm64' || $cpu eq 'kvm32';
+ if (is_custom_model($cputype)) {
+ $custom_cpu = get_custom_model($cputype);
-push @$cpuFlags, '-rdtscp' if $cpu =~ m/^Opteron/;
+ $cputype = $custom_cpu->{'reported-model'} //
+ $cpu_fmt->{'reported-model'}->{default};
+ $kvm_off = $custom_cpu->{hidden}
+ if defined($custom_cpu->{hidden});
+ $hv_vendor_id = $custom_cpu->{'hv-vendor-id'};
+ }
-if (min_version($machine_version, 2, 3) && $arch eq 'x86_64') {
+ # VM-specific settings override custom CPU config
+ $kvm_off = $cpu->{hidden}
+ if defined($cpu->{hidden});
+ $hv_vendor_id = $cpu->{'hv-vendor-id'}
+ if defined($cpu->{'hv-vendor-id'});
+}
- push @$cpuFlags , '+kvm_pv_unhalt' if $kvm;
- push @$cpuFlags , '+kvm_pv_eoi' if $kvm;
+my $pve_flags = get_pve_cpu_flags($conf, $kvm, $cputype, $arch,
+ $machine_version);
+
+my $hv_flags = get_hyperv_enlightenments($winversion, $machine_version,
+ $conf->{bios}, $gpu_passthrough, $hv_vendor_id) if $kvm;
+
+my $custom_cputype_flags = parse_cpuflag_list($cpu_flag_any_re,
+ "set by custom CPU model", $custom_cpu->{flags});
+
+my $vm_flags = parse_cpuflag_list($cpu_flag_supported_re,
+ "manually set for VM", $cpu->{flags});
+
+my $pve_forced_flags = {};
+$pve_forced_flags->{'enforce'} = {
+ reason => "error if requested CPU settings not available",
+} if $cputype ne 'host' && $kvm && $arch eq 'x86_64';
+$pve_forced_flags->{'kvm'} = {
+ value => "off",
+ reason => "hide KVM virtualization from guest",
+} if $kvm_off;
+
+# $cputype is the "reported-model" for custom types, so we can just look up
+# the vendor in the default list
+my $cpu_vendor = $cpu_vendor_list->{$cputype};
+if ($cpu_vendor) {
+ $pve_forced_flags->{'vendor'} =
[pve-devel] [PATCH v8 0/8] Add basics for custom CPU models
Based on the RFC and following on- and off-list discussion about custom CPU
models [0].
In essence, this revised patch allows a user to specify custom CPU models in
/etc/pve/cpu-models.conf (section-config style [1]), where VMs using that CPU
model inherit details from the definition. This removes any fragile
"auto-magical" CPU flag detection, while still giving the user a way to create
VMs with the best possible subset of CPU features maintaining live-migration
compatibility.
Includes the infrastructure for broadcasting supported CPU flags for each
cluster-node via the key-value store - this is not necessary for the
custom-cpu feature in particular, but I think could prove useful for
implementing the GUI part (e.g. show the user which flags are supported on which
nodes).
v7 -> v8:
* rebase on master, fix tests now using +pve0
* fixed nits noted by Thomas
* moved live-migration/snapshot patches forward to avoid temporary breakage
* fix CPU hotplug with custom CPUs
* guard mkdir with check_cfs_is_mounted and also run before write
* fix rebase-error in cfg2cmd tests (getaddrinfo_all)
* dropped applied patches
I deliberately didn't add any version_guard checks, since I don't think a bump
is the right thing to do here (I say, after I waited for the pve-version series
to be applied for making this v8).
Doing so would break custom CPUs on older machine versions (since I'd have to
pin it to 4.1~pve2) for no reason. The idea would have been to show a neat error
message, but since up to this point we didn't even check the pve-version on
migration, it wouldn't help much. Besides, migration to older versions already
fails cleanly with a semi-readable message ('Unknown option: force-cpu').
< see [2] for older history >
[0]: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038268.html
[1]: e.g.:
cpu-model: custom-cpu-name
host-phys-bits 1
flags +aes;+avx;+avx2
reported-model kvm64
[2] https://pve.proxmox.com/pipermail/pve-devel/2020-January/041278.html
qemu-server: Stefan Reiter (8):
Adapt CPUConfig to handle custom models
Verify VM-specific CPU configs seperately
Include "-cpu" parameter with live-migration
Include "-cpu" parameter with snapshots/suspend
Add helpers to better structure CPU option handling
Rework get_cpu_options and allow custom CPU models
fix #2318: allow phys-bits and host-phys-bits CPU settings
cfg2cmd: add test cases for custom CPU models
PVE/API2/Qemu.pm | 10 +-
PVE/QemuConfig.pm | 36 +-
PVE/QemuMigrate.pm| 22 +
PVE/QemuServer.pm | 44 +-
PVE/QemuServer/CPUConfig.pm | 477 +++---
test/cfg2cmd/custom-cpu-model-defaults.conf | 8 +
.../custom-cpu-model-defaults.conf.cmd| 24 +
test/cfg2cmd/custom-cpu-model.conf| 8 +
test/cfg2cmd/custom-cpu-model.conf.cmd| 27 +
test/cfg2cmd/i440fx-win10-hostpci.conf.cmd| 2 +-
test/cfg2cmd/minimal-defaults.conf.cmd| 2 +-
test/cfg2cmd/pinned-version.conf.cmd | 2 +-
.../q35-linux-hostpci-multifunction.conf.cmd | 2 +-
test/cfg2cmd/q35-linux-hostpci.conf.cmd | 2 +-
test/cfg2cmd/q35-win10-hostpci.conf.cmd | 2 +-
test/cfg2cmd/simple1.conf.cmd | 2 +-
test/cfg2cmd/spice-enhancments.conf.cmd | 2 +-
test/cfg2cmd/spice-linux-4.1.conf.cmd | 2 +-
test/cfg2cmd/spice-usb3.conf.cmd | 2 +-
test/cfg2cmd/spice-win.conf.cmd | 2 +-
test/run_config2command_tests.pl | 23 +
21 files changed, 609 insertions(+), 92 deletions(-)
create mode 100644 test/cfg2cmd/custom-cpu-model-defaults.conf
create mode 100644 test/cfg2cmd/custom-cpu-model-defaults.conf.cmd
create mode 100644 test/cfg2cmd/custom-cpu-model.conf
create mode 100644 test/cfg2cmd/custom-cpu-model.conf.cmd
--
2.20.1
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH v8 qemu-server 8/8] cfg2cmd: add test cases for custom CPU models
Requires a mock CPU-model config, which is given as a raw string to also test parsing capabilities. Also tests defaulting behaviour. Signed-off-by: Stefan Reiter --- test/cfg2cmd/custom-cpu-model-defaults.conf | 8 ++ .../custom-cpu-model-defaults.conf.cmd| 24 + test/cfg2cmd/custom-cpu-model.conf| 8 ++ test/cfg2cmd/custom-cpu-model.conf.cmd| 27 +++ test/run_config2command_tests.pl | 23 5 files changed, 90 insertions(+) create mode 100644 test/cfg2cmd/custom-cpu-model-defaults.conf create mode 100644 test/cfg2cmd/custom-cpu-model-defaults.conf.cmd create mode 100644 test/cfg2cmd/custom-cpu-model.conf create mode 100644 test/cfg2cmd/custom-cpu-model.conf.cmd diff --git a/test/cfg2cmd/custom-cpu-model-defaults.conf b/test/cfg2cmd/custom-cpu-model-defaults.conf new file mode 100644 index 000..cdef285 --- /dev/null +++ b/test/cfg2cmd/custom-cpu-model-defaults.conf @@ -0,0 +1,8 @@ +# TEST: Check if custom CPU models are resolving defaults correctly +cores: 3 +cpu: custom-alldefault +name: customcpu-defaults +numa: 0 +ostype: l26 +smbios1: uuid=2ea3f676-dfa5-11e9-ae82-c721e12f3fce +sockets: 1 diff --git a/test/cfg2cmd/custom-cpu-model-defaults.conf.cmd b/test/cfg2cmd/custom-cpu-model-defaults.conf.cmd new file mode 100644 index 000..ca8fcb0 --- /dev/null +++ b/test/cfg2cmd/custom-cpu-model-defaults.conf.cmd @@ -0,0 +1,24 @@ +/usr/bin/kvm \ + -id 8006 \ + -name customcpu-defaults \ + -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server,nowait' \ + -mon 'chardev=qmp,mode=control' \ + -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \ + -mon 'chardev=qmp-event,mode=control' \ + -pidfile /var/run/qemu-server/8006.pid \ + -daemonize \ + -smbios 'type=1,uuid=2ea3f676-dfa5-11e9-ae82-c721e12f3fce' \ + -smp '3,sockets=1,cores=3,maxcpus=3' \ + -nodefaults \ + -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \ + -vnc unix:/var/run/qemu-server/8006.vnc,password \ + -cpu kvm64,enforce,+kvm_pv_eoi,+kvm_pv_unhalt,+lahf_lm,+sep \ + -m 512 \ + -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \ + -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \ + -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \ + -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \ + -device 'VGA,id=vga,bus=pci.0,addr=0x2' \ + -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \ + -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \ + -machine 'type=pc+pve0' diff --git a/test/cfg2cmd/custom-cpu-model.conf b/test/cfg2cmd/custom-cpu-model.conf new file mode 100644 index 000..cc7c60e --- /dev/null +++ b/test/cfg2cmd/custom-cpu-model.conf @@ -0,0 +1,8 @@ +# TEST: Check if custom CPU models are resolved correctly +cores: 3 +cpu: custom-qemu64,flags=+virt-ssbd,host-phys-bits=true +name: customcpu +numa: 0 +ostype: win10 +smbios1: uuid=2ea3f676-dfa5-11e9-ae82-c721e12f3fcf +sockets: 1 diff --git a/test/cfg2cmd/custom-cpu-model.conf.cmd b/test/cfg2cmd/custom-cpu-model.conf.cmd new file mode 100644 index 000..60b4584 --- /dev/null +++ b/test/cfg2cmd/custom-cpu-model.conf.cmd @@ -0,0 +1,27 @@ +/usr/bin/kvm \ + -id 8006 \ + -name customcpu \ + -chardev 'socket,id=qmp,path=/var/run/qemu-server/8006.qmp,server,nowait' \ + -mon 'chardev=qmp,mode=control' \ + -chardev 'socket,id=qmp-event,path=/var/run/qmeventd.sock,reconnect=5' \ + -mon 'chardev=qmp-event,mode=control' \ + -pidfile /var/run/qemu-server/8006.pid \ + -daemonize \ + -smbios 'type=1,uuid=2ea3f676-dfa5-11e9-ae82-c721e12f3fcf' \ + -smp '3,sockets=1,cores=3,maxcpus=3' \ + -nodefaults \ + -boot 'menu=on,strict=on,reboot-timeout=1000,splash=/usr/share/qemu-server/bootsplash.jpg' \ + -vnc unix:/var/run/qemu-server/8006.vnc,password \ + -no-hpet \ + -cpu 'athlon,+aes,+avx,enforce,hv_ipi,hv_relaxed,hv_reset,hv_runtime,hv_spinlocks=0x1fff,hv_stimer,hv_synic,hv_time,hv_vapic,hv_vendor_id=testvend,hv_vpindex,+kvm_pv_eoi,-kvm_pv_unhalt,vendor=AuthenticAMD,+virt-ssbd,host-phys-bits=true' \ + -m 512 \ + -device 'pci-bridge,id=pci.1,chassis_nr=1,bus=pci.0,addr=0x1e' \ + -device 'pci-bridge,id=pci.2,chassis_nr=2,bus=pci.0,addr=0x1f' \ + -device 'piix3-usb-uhci,id=uhci,bus=pci.0,addr=0x1.0x2' \ + -device 'usb-tablet,id=tablet,bus=uhci.0,port=1' \ + -device 'VGA,id=vga,bus=pci.0,addr=0x2' \ + -device 'virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x3' \ + -iscsi 'initiator-name=iqn.1993-08.org.debian:01:aabbccddeeff' \ + -rtc 'driftfix=slew,base=localtime' \ + -machine 'type=pc+pve0' \ + -global 'kvm-pit.lost_tick_policy=discard' diff --git a/test/run_config2command_tests.pl b/test/run_config2command_tests.pl index c3e5092..3168176 100755 --- a/test/run_config2command_tests.pl +++ b/test/run_config2command_tests.pl @@ -17,6 +17,7 @@ use PVE::QemuConfig; use PVE::QemuServer; use PVE::QemuServer::Monitor; use
[pve-devel] [PATCH v8 qemu-server 4/8] Include "-cpu" parameter with snapshots/suspend
Just like with live-migration, custom CPU models might change after a
snapshot has been taken (or a VM suspended), which would lead to a
different QEMU invocation on rollback/resume.
Save the "-cpu" argument as a new "runningcpu" option into the VM conf
akin to "runningmachine" and use as override during rollback/resume.
No functional change with non-custom CPU types intended.
Signed-off-by: Stefan Reiter
---
New in v7.
PVE/API2/Qemu.pm | 1 +
PVE/QemuConfig.pm | 36 +++-
PVE/QemuServer.pm | 28
3 files changed, 48 insertions(+), 17 deletions(-)
diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index 5ea6cf2..e96a0c0 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -1099,6 +1099,7 @@ my $update_vm_api = sub {
push @delete, 'lock'; # this is the real deal to write it out
}
push @delete, 'runningmachine' if $conf->{runningmachine};
+ push @delete, 'runningcpu' if $conf->{runningcpu};
}
PVE::QemuConfig->check_lock($conf) if !$skiplock;
diff --git a/PVE/QemuConfig.pm b/PVE/QemuConfig.pm
index 1ba728a..852c309 100644
--- a/PVE/QemuConfig.pm
+++ b/PVE/QemuConfig.pm
@@ -5,9 +5,10 @@ use warnings;
use PVE::AbstractConfig;
use PVE::INotify;
+use PVE::QemuServer;
+use PVE::QemuServer::CPUConfig;
use PVE::QemuServer::Helpers;
use PVE::QemuServer::Monitor qw(mon_cmd);
-use PVE::QemuServer;
use PVE::QemuServer::Machine;
use PVE::Storage;
use PVE::Tools;
@@ -156,15 +157,26 @@ sub __snapshot_save_vmstate {
my $statefile = PVE::Storage::vdisk_alloc($storecfg, $target, $vmid,
'raw', $name, $size*1024);
my $runningmachine =
PVE::QemuServer::Machine::get_current_qemu_machine($vmid);
-if ($suspend) {
- $conf->{vmstate} = $statefile;
- $conf->{runningmachine} = $runningmachine;
-} else {
- my $snap = $conf->{snapshots}->{$snapname};
- $snap->{vmstate} = $statefile;
- $snap->{runningmachine} = $runningmachine;
+# get current QEMU -cpu argument
+my $runningcpu;
+if (my $pid = PVE::QemuServer::check_running($vmid)) {
+ my $cmdline = PVE::QemuServer::Helpers::parse_cmdline($pid);
+ die "could not read commandline of running machine\n"
+ if !$cmdline->{cpu}->{value};
+
+ # sanitize and untaint value
+ $cmdline->{cpu}->{value} =~
$PVE::QemuServer::CPUConfig::qemu_cmdline_cpu_re;
+ $runningcpu = $1;
}
+if (!$suspend) {
+ $conf = $conf->{snapshots}->{$snapname};
+}
+
+$conf->{vmstate} = $statefile;
+$conf->{runningmachine} = $runningmachine;
+$conf->{runningcpu} = $runningcpu;
+
return $statefile;
}
@@ -310,6 +322,11 @@ sub __snapshot_rollback_hook {
if (defined($conf->{runningmachine})) {
$data->{forcemachine} = $conf->{runningmachine};
delete $conf->{runningmachine};
+
+ # runningcpu is newer than runningmachine, so assume it only exists
+ # here, if at all
+ $data->{forcecpu} = delete $conf->{runningcpu}
+ if defined($conf->{runningcpu});
} else {
# Note: old code did not store 'machine', so we try to be smart
# and guess the snapshot was generated with kvm 1.4 (pc-i440fx-1.4).
@@ -362,7 +379,8 @@ sub __snapshot_rollback_vm_start {
my ($class, $vmid, $vmstate, $data) = @_;
my $storecfg = PVE::Storage::config();
-PVE::QemuServer::vm_start($storecfg, $vmid, $vmstate, undef, undef, undef,
$data->{forcemachine});
+PVE::QemuServer::vm_start($storecfg, $vmid, $vmstate, undef, undef, undef,
+ $data->{forcemachine}, undef, undef, undef, undef, undef,
$data->{forcecpu});
}
sub __snapshot_rollback_get_unused {
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 4252b68..ab890cb 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -532,8 +532,15 @@ EODESCR
optional => 1,
}),
runningmachine => get_standard_option('pve-qemu-machine', {
- description => "Specifies the Qemu machine type of the running vm. This
is used internally for snapshots.",
+ description => "Specifies the QEMU machine type of the running vm. This
is used internally for snapshots.",
}),
+runningcpu => {
+ description => "Specifies the QEMU '-cpu' parameter of the running vm.
This is used internally for snapshots.",
+ optional => 1,
+ type => 'string',
+ pattern => $PVE::QemuServer::CPUConfig::qemu_cmdline_cpu_re,
+ format_description => 'QEMU -cpu parameter'
+},
machine => get_standard_option('pve-qemu-machine'),
arch => {
description => "Virtual processor architecture. Defaults to the host.",
@@ -2364,7 +2371,8 @@ sub json_config_properties {
my $prop = shift;
foreach my $opt (keys %$confdesc) {
- next if $opt eq 'parent' || $opt eq 'snaptime' || $opt eq 'vmstate' ||
$opt eq 'runningmachine';
+ next if $opt eq
[pve-devel] [PATCH v8 qemu-server 5/8] Add helpers to better structure CPU option handling
To avoid hardcoding even more CPU-flag related things for custom CPU
models, introduce a dynamic approach to resolving flags.
resolve_cpu_flags takes a list of hashes (as documented in the
comment) and resolves them to a valid "-cpu" argument without
duplicates. This also helps by providing a reason why specific CPU flags
have been added, and thus allows for useful warning messages should a
flag be overwritten by another.
Signed-off-by: Stefan Reiter
---
PVE/QemuServer/CPUConfig.pm | 64 +
1 file changed, 64 insertions(+)
diff --git a/PVE/QemuServer/CPUConfig.pm b/PVE/QemuServer/CPUConfig.pm
index 315ef75..14c1bde 100644
--- a/PVE/QemuServer/CPUConfig.pm
+++ b/PVE/QemuServer/CPUConfig.pm
@@ -313,6 +313,70 @@ sub print_cpu_device {
return
"$cpu-x86_64-cpu,id=cpu$id,socket-id=$current_socket,core-id=$current_core,thread-id=0";
}
+# Resolves multiple arrays of hashes representing CPU flags with metadata to a
+# single string in QEMU "-cpu" compatible format. Later arrays have higher
+# priority.
+#
+# Hashes take the following format:
+# {
+# aes => {
+# op => "+", # defaults to "" if undefined
+# reason => "to support AES acceleration", # for override warnings
+# value => "" # needed for kvm=off (value: off) etc...
+# },
+# ...
+# }
+sub resolve_cpu_flags {
+my $flags = {};
+
+for my $hash (@_) {
+ for my $flag_name (keys %$hash) {
+ my $flag = $hash->{$flag_name};
+ my $old_flag = $flags->{$flag_name};
+
+ $flag->{op} //= "";
+ $flag->{reason} //= "unknown origin";
+
+ if ($old_flag) {
+ my $value_changed = (defined($flag->{value}) !=
defined($old_flag->{value})) ||
+ (defined($flag->{value}) && $flag->{value}
ne $old_flag->{value});
+
+ if ($old_flag->{op} eq $flag->{op} && !$value_changed) {
+ $flags->{$flag_name}->{reason} .= " & $flag->{reason}";
+ next;
+ }
+
+ my $old = print_cpuflag_hash($flag_name, $flags->{$flag_name});
+ my $new = print_cpuflag_hash($flag_name, $flag);
+ warn "warning: CPU flag/setting $new overwrites $old\n";
+ }
+
+ $flags->{$flag_name} = $flag;
+ }
+}
+
+my $flag_str = '';
+# sort for command line stability
+for my $flag_name (sort keys %$flags) {
+ $flag_str .= ',';
+ $flag_str .= $flags->{$flag_name}->{op};
+ $flag_str .= $flag_name;
+ $flag_str .= "=$flags->{$flag_name}->{value}"
+ if $flags->{$flag_name}->{value};
+}
+
+return $flag_str;
+}
+
+sub print_cpuflag_hash {
+my ($flag_name, $flag) = @_;
+my $formatted = "'$flag->{op}$flag_name";
+$formatted .= "=$flag->{value}" if defined($flag->{value});
+$formatted .= "'";
+$formatted .= " ($flag->{reason})" if defined($flag->{reason});
+return $formatted;
+}
+
# Calculate QEMU's '-cpu' argument from a given VM configuration
sub get_cpu_options {
my ($conf, $arch, $kvm, $kvm_off, $machine_version, $winversion,
$gpu_passthrough) = @_;
--
2.20.1
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH v8 qemu-server 1/8] Adapt CPUConfig to handle custom models
Turn CPUConfig into a SectionConfig with parsing/writing support for
custom CPU models. IO is handled using cfs.
Namespacing will be provided using "custom-" prefix for custom model
names (in VM config only, cpu-models.conf will contain unprefixed
names).
Includes two overrides to avoid writing redundant information to the
config file, additionally get_custom_model is used to retrieve a custom
model configuration by name.
Resolve custom names in print_cpu_device when a custom cpu is passed.
Signed-off-by: Stefan Reiter
---
Depends on pve-cluster including the cpu-models.conf as watched file.
v8:
* fix CPU hotplugging with custom models
PVE/QemuServer/CPUConfig.pm | 123 +++-
1 file changed, 120 insertions(+), 3 deletions(-)
diff --git a/PVE/QemuServer/CPUConfig.pm b/PVE/QemuServer/CPUConfig.pm
index 86febe8..a2b5e8a 100644
--- a/PVE/QemuServer/CPUConfig.pm
+++ b/PVE/QemuServer/CPUConfig.pm
@@ -4,15 +4,31 @@ use strict;
use warnings;
use PVE::JSONSchema;
+use PVE::Cluster qw(cfs_register_file cfs_read_file);
use PVE::QemuServer::Helpers qw(min_version);
-use base qw(Exporter);
+use base qw(PVE::SectionConfig Exporter);
our @EXPORT_OK = qw(
print_cpu_device
get_cpu_options
);
+# under certain race-conditions, this module might be loaded before pve-cluster
+# has started completely, so ensure we don't prevent the FUSE mount with our
dir
+if (PVE::Cluster::check_cfs_is_mounted()) {
+mkdir "/etc/pve/virtual-guest";
+}
+
+my $default_filename = "virtual-guest/cpu-models.conf";
+cfs_register_file($default_filename,
+ sub { PVE::QemuServer::CPUConfig->parse_config(@_); },
+ sub { PVE::QemuServer::CPUConfig->write_config(@_); });
+
+sub load_custom_model_conf {
+return cfs_read_file($default_filename);
+}
+
my $cpu_vendor_list = {
# Intel CPUs
486 => 'GenuineIntel',
@@ -84,11 +100,20 @@ my $cpu_flag = qr/[+-](@{[join('|',
@supported_cpu_flags)]})/;
our $cpu_fmt = {
cputype => {
- description => "Emulated CPU type.",
+ description => "Emulated CPU type. Can be default or custom name
(custom model names must be prefixed with 'custom-').",
type => 'string',
- enum => [ sort { "\L$a" cmp "\L$b" } keys %$cpu_vendor_list ],
+ format_description => 'string',
default => 'kvm64',
default_key => 1,
+ optional => 1,
+},
+'reported-model' => {
+ description => "CPU model and vendor to report to the guest. Must be a
QEMU/KVM supported model."
+. " Only valid for custom CPU model definitions, default
models will always report themselves to the guest OS.",
+ type => 'string',
+ enum => [ sort { lc("$a") cmp lc("$b") } keys %$cpu_vendor_list ],
+ default => 'kvm64',
+ optional => 1,
},
hidden => {
description => "Do not identify as a KVM virtual machine.",
@@ -114,6 +139,88 @@ our $cpu_fmt = {
},
};
+# Section config settings
+my $defaultData = {
+# shallow copy, since SectionConfig modifies propertyList internally
+propertyList => { %$cpu_fmt },
+};
+
+sub private {
+return $defaultData;
+}
+
+sub options {
+return { %$cpu_fmt };
+}
+
+sub type {
+return 'cpu-model';
+}
+
+sub parse_section_header {
+my ($class, $line) = @_;
+
+my ($type, $sectionId, $errmsg, $config) =
+ $class->SUPER::parse_section_header($line);
+
+return undef if !$type;
+return ($type, $sectionId, $errmsg, {
+ # name is given by section header, and we can always prepend 'custom-'
+ # since we're reading the custom CPU file
+ cputype => "custom-$sectionId",
+});
+}
+
+sub write_config {
+my ($class, $filename, $cfg) = @_;
+
+mkdir "/etc/pve/virtual-guest";
+
+for my $model (keys %{$cfg->{ids}}) {
+ my $model_conf = $cfg->{ids}->{$model};
+
+ die "internal error: tried saving built-in CPU model (or missing
prefix): $model_conf->{cputype}\n"
+ if !is_custom_model($model_conf->{cputype});
+
+ die "internal error: tried saving custom cpumodel with cputype
(ignoring prefix: $model_conf->{cputype}) not equal to \$cfg->ids entry
($model)\n"
+ if "custom-$model" ne $model_conf->{cputype};
+
+ # saved in section header
+ delete $model_conf->{cputype};
+}
+
+$class->SUPER::write_config($filename, $cfg);
+}
+
+sub is_custom_model {
+my ($cputype) = @_;
+return $cputype =~ m/^custom-/;
+}
+
+# Use this to get a single model in the format described by $cpu_fmt.
+# Allows names with and without custom- prefix.
+sub get_custom_model {
+my ($name, $noerr) = @_;
+
+$name =~ s/^custom-//;
+my $conf = load_custom_model_conf();
+
+my $entry = $conf->{ids}->{$name};
+if (!defined($entry)) {
+ die "Custom cputype '$name' not found\n" if !$noerr;
+ return undef;
+}
+
+my $model = {};
+for my $property (keys %$cpu_fmt) {
+ if (my
[pve-devel] [PATCH v8 qemu-server 7/8] fix #2318: allow phys-bits and host-phys-bits CPU settings
Can be specified for a particular VM or via a custom CPU model (VM takes
precedence).
QEMU's default limit only allows up to 1TB of RAM per VM. Increasing the
physical address bits available to a VM can fix this.
Signed-off-by: Stefan Reiter
---
PVE/QemuServer/CPUConfig.pm | 24
1 file changed, 24 insertions(+)
diff --git a/PVE/QemuServer/CPUConfig.pm b/PVE/QemuServer/CPUConfig.pm
index 850a76c..e4f90d5 100644
--- a/PVE/QemuServer/CPUConfig.pm
+++ b/PVE/QemuServer/CPUConfig.pm
@@ -142,6 +142,19 @@ my $cpu_fmt = {
pattern => qr/$cpu_flag_any_re(;$cpu_flag_any_re)*/,
optional => 1,
},
+'phys-bits' => {
+ type => 'integer',
+ minimum => 8,
+ maximum => 64,
+ description => "The physical memory address bits that are reported to
the guest OS. Should be smaller or equal to the host's.",
+ optional => 1,
+},
+'host-phys-bits' => {
+ type => 'boolean',
+ default => 0,
+ description => "Whether to report the host's physical memory address
bits. Overrides 'phys-bits' when set.",
+ optional => 1,
+},
};
# $cpu_fmt describes both the CPU config passed as part of a VM config, as well
@@ -465,6 +478,17 @@ sub get_cpu_options {
$cpu_str .= resolve_cpu_flags($pve_flags, $hv_flags, $custom_cputype_flags,
$vm_flags, $pve_forced_flags);
+my $phys_bits = '';
+foreach my $conf ($custom_cpu, $cpu) {
+ next if !defined($conf);
+ if ($conf->{'host-phys-bits'}) {
+ $phys_bits = ",host-phys-bits=true";
+ } elsif ($conf->{'phys-bits'}) {
+ $phys_bits = ",phys-bits=$conf->{'phys-bits'}";
+ }
+}
+$cpu_str .= $phys_bits;
+
return ('-cpu', $cpu_str);
}
--
2.20.1
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH v8 qemu-server 2/8] Verify VM-specific CPU configs seperately
$cpu_fmt is being reused for custom CPUs as well as VM-specific CPU
settings. The "pve-vm-cpu-conf" format is introduced to verify a config
specifically for use as VM-specific settings.
"pve-cpu-conf" is registered for use in custom CPU API calls (where no
additional checks are required).
Signed-off-by: Stefan Reiter
---
PVE/QemuServer.pm | 2 +-
PVE/QemuServer/CPUConfig.pm | 73 ++---
2 files changed, 69 insertions(+), 6 deletions(-)
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 23176dd..f0c8abd 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -510,7 +510,7 @@ EODESCR
optional => 1,
description => "Emulated CPU type.",
type => 'string',
- format => $PVE::QemuServer::CPUConfig::cpu_fmt,
+ format => 'pve-vm-cpu-conf',
},
parent => get_standard_option('pve-snapshot-name', {
optional => 1,
diff --git a/PVE/QemuServer/CPUConfig.pm b/PVE/QemuServer/CPUConfig.pm
index a2b5e8a..65f0d76 100644
--- a/PVE/QemuServer/CPUConfig.pm
+++ b/PVE/QemuServer/CPUConfig.pm
@@ -96,9 +96,10 @@ my @supported_cpu_flags = (
'hv-evmcs',
'aes'
);
-my $cpu_flag = qr/[+-](@{[join('|', @supported_cpu_flags)]})/;
+my $cpu_flag_supported_re = qr/([+-])(@{[join('|', @supported_cpu_flags)]})/;
+my $cpu_flag_any_re = qr/([+-])([a-zA-Z0-9\-_\.]+)/;
-our $cpu_fmt = {
+my $cpu_fmt = {
cputype => {
description => "Emulated CPU type. Can be default or custom name
(custom model names must be prefixed with 'custom-').",
type => 'string',
@@ -131,14 +132,76 @@ our $cpu_fmt = {
flags => {
description => "List of additional CPU flags separated by ';'."
. " Use '+FLAG' to enable, '-FLAG' to disable a flag."
-. " Currently supported flags: @{[join(', ',
@supported_cpu_flags)]}.",
+. " Custom CPU models can specify any flag supported by"
+. " QEMU/KVM, VM-specific flags must be from the following"
+. " set for security reasons: @{[join(', ',
@supported_cpu_flags)]}.",
format_description => '+FLAG[;-FLAG...]',
type => 'string',
- pattern => qr/$cpu_flag(;$cpu_flag)*/,
+ pattern => qr/$cpu_flag_any_re(;$cpu_flag_any_re)*/,
optional => 1,
},
};
+# $cpu_fmt describes both the CPU config passed as part of a VM config, as well
+# as the definition of a custom CPU model. There are some slight differences
+# though, which we catch in the custom verification function below.
+PVE::JSONSchema::register_format('pve-cpu-conf', \_cpu_conf_basic);
+sub parse_cpu_conf_basic {
+my ($cpu_str, $noerr) = @_;
+
+my $cpu = eval { PVE::JSONSchema::parse_property_string($cpu_fmt,
$cpu_str) };
+if ($@) {
+die $@ if !$noerr;
+return undef;
+}
+
+# required, but can't be forced in schema since it's encoded in section
+# header for custom models
+if (!$cpu->{cputype}) {
+ die "CPU is missing cputype\n" if !$noerr;
+ return undef;
+}
+
+return $cpu;
+}
+
+PVE::JSONSchema::register_format('pve-vm-cpu-conf', \_vm_cpu_conf);
+sub parse_vm_cpu_conf {
+my ($cpu_str, $noerr) = @_;
+
+my $cpu = parse_cpu_conf_basic($cpu_str, $noerr);
+return undef if !$cpu;
+
+my $cputype = $cpu->{cputype};
+
+# a VM-specific config is only valid if the cputype exists
+if (is_custom_model($cputype)) {
+ eval { get_custom_model($cputype); };
+ if ($@) {
+ die $@ if !$noerr;
+ return undef;
+ }
+} else {
+ if (!defined($cpu_vendor_list->{$cputype})) {
+ die "Built-in cputype '$cputype' is not defined (missing 'custom-'
prefix?)\n" if !$noerr;
+ return undef;
+ }
+}
+
+# in a VM-specific config, certain properties are limited/forbidden
+
+if ($cpu->{flags} && $cpu->{flags} !~
m/$cpu_flag_supported_re(;$cpu_flag_supported_re)*/) {
+ die "VM-specific CPU flags must be a subset of: @{[join(', ',
@supported_cpu_flags)]}\n"
+ if !$noerr;
+ return undef;
+}
+
+die "Property 'reported-model' not allowed in VM-specific CPU config.\n"
+ if defined($cpu->{'reported-model'});
+
+return $cpu;
+}
+
# Section config settings
my $defaultData = {
# shallow copy, since SectionConfig modifies propertyList internally
@@ -228,7 +291,7 @@ sub print_cpu_device {
my $kvm = $conf->{kvm} // 1;
my $cpu = $kvm ? "kvm64" : "qemu64";
if (my $cputype = $conf->{cpu}) {
- my $cpuconf = PVE::JSONSchema::parse_property_string($cpu_fmt, $cputype)
+ my $cpuconf = parse_cpu_conf_basic($cputype)
or die "Cannot parse cpu description: $cputype\n";
$cpu = $cpuconf->{cputype};
--
2.20.1
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH v8 qemu-server 3/8] Include "-cpu" parameter with live-migration
This is required to support custom CPU models, since the
"cpu-models.conf" file is not versioned, and can be changed while a VM
using a custom model is running. Changing the file in such a state can
lead to a different "-cpu" argument on the receiving side.
This patch fixes this by passing the entire "-cpu" option (extracted
from /proc/.../cmdline) as a "qm start" parameter. Note that this is
only done if the VM to migrate is using a custom model (which we can
check just fine, since the .conf *is* versioned with pending
changes), thus not breaking any live-migration directionality.
Signed-off-by: Stefan Reiter
---
New in v6.
PVE/API2/Qemu.pm| 9 -
PVE/QemuMigrate.pm | 22 ++
PVE/QemuServer.pm | 14 ++
PVE/QemuServer/CPUConfig.pm | 2 ++
4 files changed, 42 insertions(+), 5 deletions(-)
diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index caca430..5ea6cf2 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -1986,6 +1986,11 @@ __PACKAGE__->register_method({
optional => 1,
},
machine => get_standard_option('pve-qemu-machine'),
+ 'force-cpu' => {
+ description => "Override QEMU's -cpu argument with the given
string.",
+ type => 'string',
+ optional => 1,
+ },
targetstorage => {
description => "Target storage for the migration. (Can be '1'
to use the same storage id as on the source node.)",
type => 'string',
@@ -2014,6 +2019,7 @@ __PACKAGE__->register_method({
my $timeout = extract_param($param, 'timeout');
my $machine = extract_param($param, 'machine');
+ my $force_cpu = extract_param($param, 'force-cpu');
my $get_root_param = sub {
my $value = extract_param($param, $_[0]);
@@ -2066,7 +2072,8 @@ __PACKAGE__->register_method({
syslog('info', "start VM $vmid: $upid\n");
PVE::QemuServer::vm_start($storecfg, $vmid, $stateuri,
$skiplock, $migratedfrom, undef, $machine,
- $spice_ticket, $migration_network,
$migration_type, $targetstorage, $timeout);
+ $spice_ticket, $migration_network,
$migration_type, $targetstorage, $timeout,
+ $force_cpu);
return;
};
diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index 6db5e62..de27328 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -11,6 +11,8 @@ use PVE::Tools;
use PVE::Cluster;
use PVE::Storage;
use PVE::QemuServer;
+use PVE::QemuServer::CPUConfig;
+use PVE::QemuServer::Helpers;
use PVE::QemuServer::Machine;
use PVE::QemuServer::Monitor qw(mon_cmd);
use Time::HiRes qw( usleep );
@@ -219,7 +221,23 @@ sub prepare {
$self->{forcemachine} =
PVE::QemuServer::Machine::qemu_machine_pxe($vmid, $conf);
+ # To support custom CPU types, we keep QEMU's "-cpu" parameter intact.
+ # Since the parameter itself contains no reference to a custom model,
+ # this makes migration independent of changes to "cpu-models.conf".
+ if ($conf->{cpu}) {
+ my $cpuconf =
PVE::QemuServer::CPUConfig::parse_cpu_conf_basic($conf->{cpu});
+ if ($cpuconf &&
PVE::QemuServer::CPUConfig::is_custom_model($cpuconf->{cputype})) {
+ my $cmdline = PVE::QemuServer::Helpers::parse_cmdline($pid);
+ die "could not read commandline of running machine\n"
+ if !$cmdline->{cpu}->{value};
+
+ # sanitize and untaint value
+ $cmdline->{cpu}->{value} =~
$PVE::QemuServer::CPUConfig::qemu_cmdline_cpu_re;
+ $self->{forcecpu} = $1;
+ }
+ }
}
+
my $loc_res = PVE::QemuServer::check_local_resources($conf, 1);
if (scalar @$loc_res) {
if ($self->{running} || !$self->{opts}->{force}) {
@@ -582,6 +600,10 @@ sub phase2 {
push @$cmd, '--machine', $self->{forcemachine};
}
+if ($self->{forcecpu}) {
+ push @$cmd, '--force-cpu', $self->{forcecpu};
+}
+
if ($self->{online_local_volumes}) {
push @$cmd, '--targetstorage', ($self->{opts}->{targetstorage} // '1');
}
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index f0c8abd..4252b68 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -3406,7 +3406,7 @@ sub query_understood_cpu_flags {
}
sub config_to_command {
-my ($storecfg, $vmid, $conf, $defaults, $forcemachine) = @_;
+my ($storecfg, $vmid, $conf, $defaults, $forcemachine, $force_cpu) = @_;
my $cmd = [];
my $globalFlags = [];
@@ -3804,7 +3804,12 @@ sub config_to_command {
push @$rtcFlags, 'base=localtime';
}
-push @$cmd, get_cpu_options($conf, $arch, $kvm, $kvm_off,
$machine_version, $winversion, $gpu_passthrough);
+if ($force_cpu) {
+ push @$cmd, '-cpu', $force_cpu;
+
[pve-devel] applied: [PATCH v2 docs] network: add note for possible fix/workaround in NAT setup
On 2/10/20 4:06 PM, Oguz Bektas wrote: > apparently sometimes users have problems reaching outside internet with > some network setups. this is the workaround a user suggested that > we should add in the wiki. > > Signed-off-by: Oguz Bektas > --- > > v1->v2: > * add more rationale as suggested by stoiko > * fix indent on one line in the example config > * add links stoiko posted in mailing list for reference > > pve-network.adoc | 20 +++- > 1 file changed, 19 insertions(+), 1 deletion(-) > > diff --git a/pve-network.adoc b/pve-network.adoc > index c61cd42..1913498 100644 > --- a/pve-network.adoc > +++ b/pve-network.adoc > @@ -243,11 +243,29 @@ iface vmbr0 inet static > bridge_stp off > bridge_fd 0 > > -post-up echo 1 > /proc/sys/net/ipv4/ip_forward > +post-up echo 1 > /proc/sys/net/ipv4/ip_forward > post-up iptables -t nat -A POSTROUTING -s '10.10.10.0/24' -o eno1 > -j MASQUERADE > post-down iptables -t nat -D POSTROUTING -s '10.10.10.0/24' -o eno1 > -j MASQUERADE > > > +NOTE: In some masquerade setups with firewall enabled, conntrack zones might > be > +needed for outgoing connections. Otherwise the firewall could block outgoing > +connections since they will prefer the `POSTROUTING` of the VM bridge (and > not > +`MASQUERADE`). > + > +Adding these lines in the `/etc/network/interfaces` can fix this problem: > + > + > +post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1 > +post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1 > + > + > +For more information about this, refer to the following links: > +https://commons.wikimedia.org/wiki/File:Netfilter-packet-flow.svg[Netfilter > Packet Flow] > +https://lwn.net/Articles/370152/[Patch on netdev-list introducing conntrack > zones] > +https://blog.lobraun.de/2019/05/19/prox/[Blog post with a good explanation > by using TRACE in the raw table] > + > + > > Linux Bond > ~~ > applied, thanks! ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] applied: [PATCH docs 2/2] include allowtoken info in api-viewer
On 1/30/20 1:54 PM, Fabian Grünbichler wrote:
> Signed-off-by: Fabian Grünbichler
> ---
> api-viewer/PVEAPI.js | 3 +++
> 1 file changed, 3 insertions(+)
>
> diff --git a/api-viewer/PVEAPI.js b/api-viewer/PVEAPI.js
> index 2416161..53bc36c 100644
> --- a/api-viewer/PVEAPI.js
> +++ b/api-viewer/PVEAPI.js
> @@ -354,6 +354,9 @@ Ext.onReady(function() {
> permhtml += "Unknown systax!";
> }
> }
> + if (!info.allowtoken) {
> + permhtml += "This API endpoint is not available for
> API tokens."
> + }
>
> sections.push({
> title: 'Required permissions',
>
applied this one already, thanks!
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH qemu-server] fix #2570: add 'keephugepages' config
We already keep hugepages if they are created with the kernel
commandline (hugepagesz=x hugepages=y), but some setups (specifically
hugepages across multiple NUMA nodes) cannot be configured that way.
Since we always clear these hugepages at VM shutdown, rebooting a VM
that uses them might not work, since the requested count might not be
available anymore by the time we want to use them (also, we would then
no longer allocate them correctly on the NUMA nodes).
Add a 'keephugepages' parameter to skip cleanup and simply leave them
untouched.
Signed-off-by: Stefan Reiter
---
I tried adding it as a 'keep' sub-parameter first (i.e.
'hugepages: 1024,keep=1' etc.), but it turns out that we hardcode
$config->{hugepages} to be a numeric string in a *lot* of different places, so I
opted for this variant instead. Open for suggestions ofc.
PVE/QemuServer.pm | 14 --
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 23176dd..4741707 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -384,6 +384,14 @@ EODESC
description => "Enable/disable hugepages memory.",
enum => [qw(any 2 1024)],
},
+keephugepages => {
+ optional => 1,
+ type => 'boolean',
+ default => 0,
+ description => "Use together with hugepages. If enabled, hugepages will"
+. " not be deleted after VM shutdown and can be used for"
+. " subsequent starts.",
+},
vcpus => {
optional => 1,
type => 'integer',
@@ -5441,11 +5449,13 @@ sub vm_start {
eval { $run_qemu->() };
if (my $err = $@) {
-
PVE::QemuServer::Memory::hugepages_reset($hugepages_host_topology);
+
PVE::QemuServer::Memory::hugepages_reset($hugepages_host_topology)
+ if !$conf->{keephugepages};
die $err;
}
-
PVE::QemuServer::Memory::hugepages_pre_deallocate($hugepages_topology);
+
PVE::QemuServer::Memory::hugepages_pre_deallocate($hugepages_topology)
+ if !$conf->{keephugepages};
};
eval { PVE::QemuServer::Memory::hugepages_update_locked($code); };
--
2.20.1
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH qemu-server v2] fix efidisks on storages with minimum sizes bigger than OVMF_VARS.fd
on storages where the minimum size of images is bigger than the real
OVMF_VARS.fd file, they get padded to their minimum size
when using such an image, qemu maps it fully to the vm, but the efi
does not find the vars region and creates a file on the first efi
partition it finds
this breaks some settings in the ovmf, such as resolution
to fix this, we have to specify the size for the pflash, so that
qemu only maps the first n bytes in the vm (this only works for
raw files, not for qcow2)
we also have to use the correct size when converting between storages
in 'clone_disk' (used for move disk and cloning vms) and when
live migrating to different storages
when we now expect that the source image is always correctly used/created
(e.g. raw with size=x in pflash argument) then we always create the
target correctly
when encountering users which have a non-valid image (e.g. a efidisk
moved from zfs to qcow2 before this patch), we have to tell them to
recreate the efidisk and the settings on it
we have to version_guard it to 4.1+pve2 (since we haven't bumped yet
since the change to pve2)
also add 2 tests, one for the old version and one for the new
Signed-off-by: Dominik Csapak
---
changes from v1:
* rebase on master (drop PVE version increase; drop test adaptions)
* use version_guard instead of min_version
* add tests for raw efidisk
PVE/API2/Qemu.pm | 4 +--
PVE/QemuMigrate.pm| 6 +
PVE/QemuServer.pm | 43 ---
test/cfg2cmd/efi-raw-old.conf | 5
test/cfg2cmd/efi-raw-old.conf.cmd | 26 +++
test/cfg2cmd/efi-raw.conf | 4 +++
test/cfg2cmd/efi-raw.conf.cmd | 26 +++
7 files changed, 109 insertions(+), 5 deletions(-)
create mode 100644 test/cfg2cmd/efi-raw-old.conf
create mode 100644 test/cfg2cmd/efi-raw-old.conf.cmd
create mode 100644 test/cfg2cmd/efi-raw.conf
create mode 100644 test/cfg2cmd/efi-raw.conf.cmd
diff --git a/PVE/API2/Qemu.pm b/PVE/API2/Qemu.pm
index caca430..86af4a1 100644
--- a/PVE/API2/Qemu.pm
+++ b/PVE/API2/Qemu.pm
@@ -2908,7 +2908,7 @@ __PACKAGE__->register_method({
my $newdrive = PVE::QemuServer::clone_disk($storecfg,
$vmid, $running, $opt, $drive, $snapname,
$newid,
$storage, $format, $fullclone->{$opt}, $newvollist,
- $jobs,
$skipcomplete, $oldconf->{agent}, $clonelimit);
+ $jobs,
$skipcomplete, $oldconf->{agent}, $clonelimit, $oldconf);
$newconf->{$opt} =
PVE::QemuServer::print_drive($newdrive);
@@ -3095,7 +3095,7 @@ __PACKAGE__->register_method({
my $movelimit = PVE::Storage::get_bandwidth_limit('move',
[$oldstoreid, $storeid], $bwlimit);
my $newdrive = PVE::QemuServer::clone_disk($storecfg,
$vmid, $running, $disk, $drive, undef,
- $vmid, $storeid,
$format, 1, $newvollist, undef, undef, undef, $movelimit);
+ $vmid, $storeid,
$format, 1, $newvollist, undef, undef, undef, $movelimit, $conf);
$conf->{$disk} = PVE::QemuServer::print_drive($newdrive);
diff --git a/PVE/QemuMigrate.pm b/PVE/QemuMigrate.pm
index 6db5e62..ca3feb2 100644
--- a/PVE/QemuMigrate.pm
+++ b/PVE/QemuMigrate.pm
@@ -454,6 +454,12 @@ sub sync_disks {
}
});
+ # we want to set the efidisk size in the config to the size of the
+ # real OVMF_VARS.fd image, else we can create a too big image, which
does not work
+ if (defined($conf->{efidisk0})) {
+ PVE::QemuServer::update_efidisk_size($conf);
+ }
+
$self->log('info', "copying local disk images") if
scalar(%$local_volumes);
foreach my $volid (keys %$local_volumes) {
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 23176dd..3700ced 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -3544,8 +3544,14 @@ sub config_to_command {
$format = 'raw';
}
+ my $size_str = "";
+
+ if ($format eq 'raw' && $version_guard->(4, 1, 2)) {
+ $size_str = ",size=" . (-s $ovmf_vars);
+ }
+
push @$cmd, '-drive',
"if=pflash,unit=0,format=raw,readonly,file=$ovmf_code";
- push @$cmd, '-drive',
"if=pflash,unit=1,format=$format,id=drive-efidisk0,file=$path";
+ push @$cmd, '-drive',
"if=pflash,unit=1,format=$format,id=drive-efidisk0$size_str,file=$path";
}
# load q35 config
@@ -7021,7 +7027,7 @@ sub qemu_blockjobs_cancel {
sub clone_disk {
my ($storecfg, $vmid, $running, $drivename, $drive, $snapname,
- $newvmid, $storage, $format, $full, $newvollist, $jobs, $skipcomplete,
$qga, $bwlimit) = @_;
+ $newvmid, $storage, $format, $full,
Re: [pve-devel] [PATCH pve-manager 1/2] add sdn gui
Hi, did you work on these gui patches, or is it roughly the same as last time? (as in: should i start reviewing these?) i'll look at them regardless the next days, to see if we could apply some version of them and hide them for most users (like thomas suggested) kind regards Dominik ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] applied: [PATCH manager] gui: form/ControllerSelector: fix autoselection
On 2/12/20 10:51 AM, Dominik Csapak wrote: > changing from Ext.Array.each to a for loop and changing the return > to a 'break', resulted in exiting the wrong loop (previously the outer, > now the inner) resulting in wrongly looping through all controllers > instead of only until we found a match > > using a label and labeled break, we can break the outer loop > > Signed-off-by: Dominik Csapak > --- > www/manager6/form/ControllerSelector.js | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > argh, thanks for finding and fixing this! applied ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] applied: [PATCH manager] gui: dc/Summary: improve dashboard storage calculation
On 2/12/20 11:16 AM, Dominik Csapak wrote:
> if no storage is selected in 'My Settings' our calculatin was done as follows:
> * for every node, count 'local',
> * every other storage gets only counted *once*, even if it is a local storage
>
> as we have the 'shared' flag available, we can count local storages
> for every node where they are defined which results in a storage total
> calculation:
> sum of all local storages on all nodes + all shared storages once
>
> should make a better overview of storage for a cluster with local storages
>
> Signed-off-by: Dominik Csapak
> ---
> www/manager6/dc/Summary.js | 3 +--
> 1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/www/manager6/dc/Summary.js b/www/manager6/dc/Summary.js
> index 5e613eb1..ff7edaee 100644
> --- a/www/manager6/dc/Summary.js
> +++ b/www/manager6/dc/Summary.js
> @@ -183,8 +183,7 @@ Ext.define('PVE.dc.Summary', {
> break;
> }
> if (!countedStorages[item.data.storage] ||
> - (item.data.storage === 'local' &&
> - !countedStorages[item.data.id])) {
> + (!item.data.shared &&
> !countedStorages[item.data.id])) {
> used += item.data.disk;
> total += item.data.maxdisk;
>
>
applied, thanks!
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH storage v2 2/3] storage: merge archive format/compressor
On 1/31/20 5:01 PM, Alwin Antreich wrote:
> detection into a separate function to reduce code duplication and allow
> for easier modification.
>
> Signed-off-by: Alwin Antreich
> ---
> PVE/Storage.pm | 78 --
> 1 file changed, 57 insertions(+), 21 deletions(-)
>
> diff --git a/PVE/Storage.pm b/PVE/Storage.pm
> index 1688077..390b343 100755
> --- a/PVE/Storage.pm
> +++ b/PVE/Storage.pm
> @@ -1265,6 +1265,52 @@ sub foreach_volid {
> }
> }
>
> +sub archive_info {
this mixes two things which shouldn't be mixed, IMO.
1. getting the archive info
2. getting a decompressor from $format and $compressor
I'd split that up in two.
> +my ($archive, $comp, $format) = @_;
With the split up you can then remove $com and $format here, they make for
a strange method signature IMO, that should be handled by the caller.
But as the single caller even passing this at all (qemu-server patch 1/2) is
interested for the decompressor only anyway the aforementioned split up
will solve that for you anyway and give a much nicer interface and two
method doing only one thing, not two mixed with edge cases.
A few other comments below, but I wrote them before I checked the overall
design. You may read them, but I suggest in doing the split up first before
all - it should solve most issues.
> +my $type;
> +
> +if (!defined($comp) || !defined($format)) {
> + my $volid = basename($archive);
> + if ($volid =~
> /vzdump-(lxc|openvz|qemu)-\d+-(\d{4})_(\d{2})_(\d{2})-(\d{2})_(\d{2})_(\d{2})\.(tgz|((tar|vma)(\.(gz|lzo))?))$/)
> {
please use non-capturing groups, it would be great if we could get the
$format and $compressor without the case separation below.
> + $type = $1;
> +
> + if ($8 eq 'tgz') {
> + $format = 'tar';
> + $comp = 'gz';
> + } else {
> + $format = $10;
> + $comp = $12 if defined($12);
> + }
> + } else {
> + die "ERROR: couldn't determine format and compression type\n";
> + }
> +}
> +
> +my $decompressor = {
> + gz => {
> + 'vma' => [ "zcat", $archive ],
> + 'tar' => [ "tar", "-z", $archive ],
> + },
> + lzo => {
> + 'vma' => [ "lzop", "-d", "-c", $archive ],
> + 'tar' => [ "tar", "--lzop", $archive ],
> + },
I'd rather inverse it and add tgz as "virtual format"
vma => {
...
},
tar => {
...
},
tgz => [],
die "..." if !defined($format); # always an error
my $decomp = $decompressor->{$format};
> +};
> +
> +my $info;
> +$info->{'format'} = $format;
> +$info->{'type'} = $type;
> +$info->{'compression'} = $comp;
please just set it as hash directly:
my $info = {
format => $format,
type => $type,
compressor => $comp,
};
> +
> +if (defined($comp) && defined($format)) {
> + my $dcomp = $decompressor->{$comp}->{$format};
> + pop(@$dcomp) if !defined($archive);
doesn't above just produces a empty list? seems like a confusing and
complicated way for a fall back value...
either just include $archive in the check above or just directly assign it?
a die like it existed befor below for the ($comp && !$decompressor) case
would be more sensible, IMO.
> + $info->{'decompressor'} = $dcomp;
> +}
> +
> +return $info;
> +}
> +
> sub extract_vzdump_config_tar {
> my ($archive, $conf_re) = @_;
>
> @@ -1310,16 +1356,12 @@ sub extract_vzdump_config_vma {
> };
>
>
> +my $info = archive_info($archive);
> +$comp //= $info->{compression};
> +my $decompressor = $info->{decompressor};
> +
> if ($comp) {
> - my $uncomp;
> - if ($comp eq 'gz') {
> - $uncomp = ["zcat", $archive];
> - } elsif ($comp eq 'lzo') {
> - $uncomp = ["lzop", "-d", "-c", $archive];
> - } else {
> - die "unknown compression method '$comp'\n";
> - }
> - $cmd = [$uncomp, ["vma", "config", "-"]];
> + $cmd = [ $decompressor, ["vma", "config", "-"] ];
>
> # in some cases, lzop/zcat exits with 1 when its stdout pipe is
> # closed early by vma, detect this and ignore the exit code later
> @@ -1360,20 +1402,14 @@ sub extract_vzdump_config {
> my ($cfg, $volid) = @_;
>
> my $archive = abs_filesystem_path($cfg, $volid);
> +my $info = archive_info($archive);
> +my $format = $info->{format};
> +my $comp = $info->{compression};
> +my $type = $info->{type};
>
> -if ($volid =~
> /vzdump-(lxc|openvz)-\d+-(\d{4})_(\d{2})_(\d{2})-(\d{2})_(\d{2})_(\d{2})\.(tgz|(tar(\.(gz|lzo))?))$/)
> {
> +if ($type eq 'lxc' || $type eq 'openvz') {
> return extract_vzdump_config_tar($archive,
> qr!^(\./etc/vzdump/(pct|vps)\.conf)$!);
> -} elsif ($volid =~
> /vzdump-qemu-\d+-(\d{4})_(\d{2})_(\d{2})-(\d{2})_(\d{2})_(\d{2})\.(tgz|((tar|vma)(\.(gz|lzo))?))$/)
> {
> - my $format;
> - my $comp;
> - if ($7 eq 'tgz') {
> - $format = 'tar';
> -
[pve-devel] applied: [PATCH v4 qemu-server] Simplify QEMU version check and require 3.0+
On 2/12/20 11:10 AM, Stefan Reiter wrote: > Some of the recent QMP changes require at least 2.8.0, but since the > oldest version we officially package for 6.x is 4.0.0 anyway, checking > for at least 3.0 should not break anyone's setup. > > Note that this does not affect machine version checks, only the > installed QEMU binary version. > > Signed-off-by: Stefan Reiter > --- > > Whoops wrong file, sorry for noise. > > v3 -> v4: > * use correct test description for new test > > v2 -> v3: > * include test case and fix existing simple1 test > > v1 -> v2: > * handle error in kvm_user_version > * corrected commit message (2.12 -> 2.8) > > PVE/QemuServer.pm | 14 +- > test/cfg2cmd/old-qemu.conf | 4 > test/cfg2cmd/simple1.conf | 2 +- > 3 files changed, 10 insertions(+), 10 deletions(-) > create mode 100644 test/cfg2cmd/old-qemu.conf > applied, thanks! ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH storage v2 1/3] compact regex for backup file filter
On 1/31/20 5:00 PM, Alwin Antreich wrote:
> this, more compact form of the regex should allow easier addition of new
> file extensions.
>
> Signed-off-by: Alwin Antreich
> ---
> PVE/Storage.pm| 2 +-
> PVE/Storage/Plugin.pm | 4 ++--
> 2 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/PVE/Storage.pm b/PVE/Storage.pm
> index 0bd103e..1688077 100755
> --- a/PVE/Storage.pm
> +++ b/PVE/Storage.pm
> @@ -514,7 +514,7 @@ sub path_to_volume_id {
> } elsif ($path =~ m!^$privatedir/(\d+)$!) {
> my $vmid = $1;
> return ('rootdir', "$sid:rootdir/$vmid");
> - } elsif ($path =~
> m!^$backupdir/([^/]+\.(tar|tar\.gz|tar\.lzo|tgz|vma|vma\.gz|vma\.lzo))$!) {
> + } elsif ($path =~
> m!^$backupdir/([^/]+\.(tgz|((tar|vma)(\.(gz|lzo))?)))$!) {
the "feature" of the old regex was that $2 included the extension,
you want to throw some non-capturing groups (?:) in there for that.
Maybe we could strike a balance by pulling out the compressors into a regex
variable?
(tar(?:\.$COMPRESSOR_RE)?|tgz|vma(?:\.$COMPRESSOR_RE))
Would allow for reuse below.
> my $name = $1;
> return ('iso', "$sid:backup/$name");
> }
> diff --git a/PVE/Storage/Plugin.pm b/PVE/Storage/Plugin.pm
> index 0c39cbd..58a801a 100644
> --- a/PVE/Storage/Plugin.pm
> +++ b/PVE/Storage/Plugin.pm
> @@ -423,7 +423,7 @@ sub parse_volname {
> return ('vztmpl', $1);
> } elsif ($volname =~ m!^rootdir/(\d+)$!) {
> return ('rootdir', $1, $1);
> -} elsif ($volname =~
> m!^backup/([^/]+(\.(tar|tar\.gz|tar\.lzo|tgz|vma|vma\.gz|vma\.lzo)))$!) {
> +} elsif ($volname =~
> m!^backup/([^/]+(\.(tgz|((tar|vma)(\.(gz|lzo))?$!) {
> my $fn = $1;
> if ($fn =~ m/^vzdump-(openvz|lxc|qemu)-(\d+)-.+/) {
> return ('backup', $fn, $2);
> @@ -910,7 +910,7 @@ my $get_subdir_files = sub {
>
> } elsif ($tt eq 'backup') {
> next if defined($vmid) && $fn !~ m/\S+-$vmid-\S+/;
> - next if $fn !~
> m!/([^/]+\.(tar|tar\.gz|tar\.lzo|tgz|vma|vma\.gz|vma\.lzo))$!;
> + next if $fn !~ m!/([^/]+\.(tgz|((tar|vma)(\.(gz|lzo))?)))$!;
>
> $info = { volid => "$sid:backup/$1", format => $2 };
>
>
writing test for this up front as a first patch would be great.
It should be pretty simple, i.e., have a array of maps with params and expected
result as two key => value entries. Additionally "expected error" checks would
be
great too.
While there's already a sub test in zfspoolplugin test I'd just do a new one.
This way we would have a safety net for such and future changes.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH manager] gui: dc/Summary: improve dashboard storage calculation
if no storage is selected in 'My Settings' our calculatin was done as follows:
* for every node, count 'local',
* every other storage gets only counted *once*, even if it is a local storage
as we have the 'shared' flag available, we can count local storages
for every node where they are defined which results in a storage total
calculation:
sum of all local storages on all nodes + all shared storages once
should make a better overview of storage for a cluster with local storages
Signed-off-by: Dominik Csapak
---
www/manager6/dc/Summary.js | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/www/manager6/dc/Summary.js b/www/manager6/dc/Summary.js
index 5e613eb1..ff7edaee 100644
--- a/www/manager6/dc/Summary.js
+++ b/www/manager6/dc/Summary.js
@@ -183,8 +183,7 @@ Ext.define('PVE.dc.Summary', {
break;
}
if (!countedStorages[item.data.storage] ||
- (item.data.storage === 'local' &&
- !countedStorages[item.data.id])) {
+ (!item.data.shared &&
!countedStorages[item.data.id])) {
used += item.data.disk;
total += item.data.maxdisk;
--
2.20.1
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH v4 qemu-server] Simplify QEMU version check and require 3.0+
Some of the recent QMP changes require at least 2.8.0, but since the
oldest version we officially package for 6.x is 4.0.0 anyway, checking
for at least 3.0 should not break anyone's setup.
Note that this does not affect machine version checks, only the
installed QEMU binary version.
Signed-off-by: Stefan Reiter
---
Whoops wrong file, sorry for noise.
v3 -> v4:
* use correct test description for new test
v2 -> v3:
* include test case and fix existing simple1 test
v1 -> v2:
* handle error in kvm_user_version
* corrected commit message (2.12 -> 2.8)
PVE/QemuServer.pm | 14 +-
test/cfg2cmd/old-qemu.conf | 4
test/cfg2cmd/simple1.conf | 2 +-
3 files changed, 10 insertions(+), 10 deletions(-)
create mode 100644 test/cfg2cmd/old-qemu.conf
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 27b9866..23176dd 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -3415,7 +3415,6 @@ sub config_to_command {
my $devices = [];
my $pciaddr = '';
my $bridges = {};
-my $vernum = 0; # unknown
my $ostype = $conf->{ostype};
my $winversion = windows_version($ostype);
my $kvm = $conf->{kvm};
@@ -3425,6 +3424,11 @@ sub config_to_command {
my $kvm_binary = get_command_for_arch($arch);
my $kvmver = kvm_user_version($kvm_binary);
+if (!$kvmver || $kvmver !~ m/^(\d+)\.(\d+)/ || $1 < 3) {
+ $kvmver //= "undefined";
+ die "Detected old QEMU binary ('$kvmver', at least 3.0 is required)\n";
+}
+
my $add_pve_version = min_version($kvmver, 4, 1);
my $machine_type = get_vm_machine($conf, $forcemachine, $arch,
$add_pve_version);
@@ -3458,14 +3462,6 @@ sub config_to_command {
if !defined kvm_version();
}
-if ($kvmver =~ m/^(\d+)\.(\d+)$/) {
- $vernum = $1*100+$2*1000;
-} elsif ($kvmver =~ m/^(\d+)\.(\d+)\.(\d+)$/) {
- $vernum = $1*100+$2*1000+$3;
-}
-
-die "detected old qemu-kvm binary ($kvmver)\n" if $vernum < 15000;
-
my $q35 = PVE::QemuServer::Machine::machine_type_is_q35($conf);
my $hotplug_features = parse_hotplug_features(defined($conf->{hotplug}) ?
$conf->{hotplug} : '1');
my $use_old_bios_files = undef;
diff --git a/test/cfg2cmd/old-qemu.conf b/test/cfg2cmd/old-qemu.conf
new file mode 100644
index 000..08e852c
--- /dev/null
+++ b/test/cfg2cmd/old-qemu.conf
@@ -0,0 +1,4 @@
+# TEST: Test QEMU version detection and expect fail on old version
+# QEMU_VERSION: 2.12.1
+# EXPECT_ERROR: Detected old QEMU binary ('2.12.1', at least 3.0 is required)
+smbios1: uuid=7b10d7af-b932-4c66-b2c3-3996152ec465
diff --git a/test/cfg2cmd/simple1.conf b/test/cfg2cmd/simple1.conf
index 10a4c31..16a2402 100644
--- a/test/cfg2cmd/simple1.conf
+++ b/test/cfg2cmd/simple1.conf
@@ -1,5 +1,5 @@
# TEST: Simple test for a basic configuration with no special things
-# QEMU_VERSION: 2.12.1
+# QEMU_VERSION: 3.0
bootdisk: scsi0
cores: 3
ide2: none,media=cdrom
--
2.20.1
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH v3 qemu-server] Simplify QEMU version check and require 3.0+
Some of the recent QMP changes require at least 2.8.0, but since the
oldest version we officially package for 6.x is 4.0.0 anyway, checking
for at least 3.0 should not break anyone's setup.
Note that this does not affect machine version checks, only the
installed QEMU binary version.
Signed-off-by: Stefan Reiter
---
v2 -> v3:
* include test case and fix existing simple1 test
v1 -> v2:
* handle error in kvm_user_version
* corrected commit message (2.12 -> 2.8)
PVE/QemuServer.pm | 14 +-
test/cfg2cmd/old-qemu.conf | 4
test/cfg2cmd/simple1.conf | 2 +-
3 files changed, 10 insertions(+), 10 deletions(-)
create mode 100644 test/cfg2cmd/old-qemu.conf
diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
index 27b9866..23176dd 100644
--- a/PVE/QemuServer.pm
+++ b/PVE/QemuServer.pm
@@ -3415,7 +3415,6 @@ sub config_to_command {
my $devices = [];
my $pciaddr = '';
my $bridges = {};
-my $vernum = 0; # unknown
my $ostype = $conf->{ostype};
my $winversion = windows_version($ostype);
my $kvm = $conf->{kvm};
@@ -3425,6 +3424,11 @@ sub config_to_command {
my $kvm_binary = get_command_for_arch($arch);
my $kvmver = kvm_user_version($kvm_binary);
+if (!$kvmver || $kvmver !~ m/^(\d+)\.(\d+)/ || $1 < 3) {
+ $kvmver //= "undefined";
+ die "Detected old QEMU binary ('$kvmver', at least 3.0 is required)\n";
+}
+
my $add_pve_version = min_version($kvmver, 4, 1);
my $machine_type = get_vm_machine($conf, $forcemachine, $arch,
$add_pve_version);
@@ -3458,14 +3462,6 @@ sub config_to_command {
if !defined kvm_version();
}
-if ($kvmver =~ m/^(\d+)\.(\d+)$/) {
- $vernum = $1*100+$2*1000;
-} elsif ($kvmver =~ m/^(\d+)\.(\d+)\.(\d+)$/) {
- $vernum = $1*100+$2*1000+$3;
-}
-
-die "detected old qemu-kvm binary ($kvmver)\n" if $vernum < 15000;
-
my $q35 = PVE::QemuServer::Machine::machine_type_is_q35($conf);
my $hotplug_features = parse_hotplug_features(defined($conf->{hotplug}) ?
$conf->{hotplug} : '1');
my $use_old_bios_files = undef;
diff --git a/test/cfg2cmd/old-qemu.conf b/test/cfg2cmd/old-qemu.conf
new file mode 100644
index 000..4594844
--- /dev/null
+++ b/test/cfg2cmd/old-qemu.conf
@@ -0,0 +1,4 @@
+# TEST: Simple test for a basic configuration with no special things
+# QEMU_VERSION: 2.12.1
+# EXPECT_ERROR: Detected old QEMU binary ('2.12.1', at least 3.0 is required)
+smbios1: uuid=7b10d7af-b932-4c66-b2c3-3996152ec465
diff --git a/test/cfg2cmd/simple1.conf b/test/cfg2cmd/simple1.conf
index 10a4c31..16a2402 100644
--- a/test/cfg2cmd/simple1.conf
+++ b/test/cfg2cmd/simple1.conf
@@ -1,5 +1,5 @@
# TEST: Simple test for a basic configuration with no special things
-# QEMU_VERSION: 2.12.1
+# QEMU_VERSION: 3.0
bootdisk: scsi0
cores: 3
ide2: none,media=cdrom
--
2.20.1
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] [PATCH manager] gui: form/ControllerSelector: fix autoselection
changing from Ext.Array.each to a for loop and changing the return
to a 'break', resulted in exiting the wrong loop (previously the outer,
now the inner) resulting in wrongly looping through all controllers
instead of only until we found a match
using a label and labeled break, we can break the outer loop
Signed-off-by: Dominik Csapak
---
www/manager6/form/ControllerSelector.js | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/www/manager6/form/ControllerSelector.js
b/www/manager6/form/ControllerSelector.js
index 8ba46f17..89ecdf4a 100644
--- a/www/manager6/form/ControllerSelector.js
+++ b/www/manager6/form/ControllerSelector.js
@@ -54,13 +54,14 @@ Ext.define('PVE.form.ControllerSelector', {
clist = me.sortByPreviousUsage(me.vmconfig, clist);
}
+clist_loop:
for (const controller of clist) {
bussel.setValue(controller);
for (let i = 0; i < PVE.Utils.diskControllerMaxIDs[controller];
i++) {
let confid = controller + i.toString();
if (!Ext.isDefined(me.vmconfig[confid])) {
deviceid.setValue(i);
- break;
+ break clist_loop; // we found the desired controller/id
combo
}
}
}
--
2.20.1
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] [PATCH v2 qemu-server] Simplify QEMU version check and require 3.0+
On 2/10/20 12:20 PM, Stefan Reiter wrote:
> Some of the recent QMP changes require at least 2.8.0, but since the
> oldest version we officially package for 6.x is 4.0.0 anyway, checking
> for at least 3.0 should not break anyone's setup.
>
> Note that this does not affect machine version checks, only the
> installed QEMU binary version.
>
> Signed-off-by: Stefan Reiter
> ---
>
> v1 -> v2:
> * handle error in kvm_user_version
> * corrected commit message (2.12 -> 2.8)
>
> @Thomas: If kvm_user_version() failed, we actually died in min_version before.
> You're suggested code also didn't quite work, because it still printed
> warnings
> in case $kvmver was undef.
>
> PVE/QemuServer.pm | 14 +-
> 1 file changed, 5 insertions(+), 9 deletions(-)
>
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> index 318ee54..d30cdb5 100644
> --- a/PVE/QemuServer.pm
> +++ b/PVE/QemuServer.pm
> @@ -3405,7 +3405,6 @@ sub config_to_command {
> my $devices = [];
> my $pciaddr = '';
> my $bridges = {};
> -my $vernum = 0; # unknown
> my $ostype = $conf->{ostype};
> my $winversion = windows_version($ostype);
> my $kvm = $conf->{kvm};
> @@ -3415,6 +3414,11 @@ sub config_to_command {
> my $kvm_binary = get_command_for_arch($arch);
> my $kvmver = kvm_user_version($kvm_binary);
>
> +if (!$kvmver || $kvmver !~ m/^(\d+)\.(\d+)/ || $1 < 3) {
> + $kvmver //= "undefined";
> + die "Detected old QEMU binary ('$kvmver', at least 3.0 is required)\n";
> +}
> +
> my $add_pve_version = min_version($kvmver, 4, 1);
>
> my $machine_type = get_vm_machine($conf, $forcemachine, $arch,
> $add_pve_version);
> @@ -3430,14 +3434,6 @@ sub config_to_command {
> if !defined kvm_version();
> }
>
> -if ($kvmver =~ m/^(\d+)\.(\d+)$/) {
> - $vernum = $1*100+$2*1000;
> -} elsif ($kvmver =~ m/^(\d+)\.(\d+)\.(\d+)$/) {
> - $vernum = $1*100+$2*1000+$3;
> -}
> -
> -die "detected old qemu-kvm binary ($kvmver)\n" if $vernum < 15000;
> -
> my $q35 = PVE::QemuServer::Machine::machine_type_is_q35($conf);
> my $hotplug_features = parse_hotplug_features(defined($conf->{hotplug})
> ? $conf->{hotplug} : '1');
> my $use_old_bios_files = undef;
>
test system fails with this:
not ok 9 - 'simple1.conf' - Simple test for a basic configuration with no
special things
got unexpected error: Detected old QEMU binary ('2.12.1', at least 3.0 is
required)
we probably want to move it up to 3.0, and copy it over for negative testing as
well?
Looks OK besides that.
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] applied: [PATCH qemu-server 2/2] version_guard scsi drive count
On 2/10/20 4:05 PM, Stefan Reiter wrote:
> Live-migrating a VM with more than 14 SCSI disks to a node that doesn't
> support it yet is broken. Use a bumped pve-version to represent that and
> give the user a nice error message instead.
>
> Signed-off-by: Stefan Reiter
> ---
>
> This is more for correctness, since older versions (i.e. ones that don't
> support
> this feature) do not include the code to die with a useful message. It does,
> however, give a nice example of how $version_guard is supposed to be used.
>
>
> PVE/QemuServer.pm | 3 +++
> PVE/QemuServer/Machine.pm | 2 +-
> 2 files changed, 4 insertions(+), 1 deletion(-)
>
> diff --git a/PVE/QemuServer.pm b/PVE/QemuServer.pm
> index 37ffc86..27b9866 100644
> --- a/PVE/QemuServer.pm
> +++ b/PVE/QemuServer.pm
> @@ -3931,6 +3931,9 @@ sub config_to_command {
>
> my ($maxdev, $controller, $controller_prefix) = scsihw_infos($conf,
> $drive);
>
> + die "scsi$drive->{index}: machine version 4.1~pve2 or higher is
> required to use more than 14 SCSI disks\n"
> + if $drive->{index} > 13 && !&$version_guard(4, 1, 2);
> +
> $pciaddr = print_pci_addr("$controller_prefix$controller",
> $bridges, $arch, $machine_type);
> my $scsihw_type = $scsihw =~ m/^virtio-scsi-single/ ?
> "virtio-scsi-pci" : $scsihw;
>
> diff --git a/PVE/QemuServer/Machine.pm b/PVE/QemuServer/Machine.pm
> index 9fbe9a8..0817d2a 100644
> --- a/PVE/QemuServer/Machine.pm
> +++ b/PVE/QemuServer/Machine.pm
> @@ -9,7 +9,7 @@ use PVE::QemuServer::Monitor;
> # Bump this for VM HW layout changes during a release (where the QEMU machine
> # version stays the same)
> our $PVE_MACHINE_VERSION = {
> -'4.1' => 1,
> +'4.1' => 2,
> };
>
> sub machine_type_is_q35 {
>
applied
___
pve-devel mailing list
pve-devel@pve.proxmox.com
https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
[pve-devel] applied: [PATCH qemu-server 1/2] Use 'QEMU version' -> '+pve-version' mapping for machine types
On 2/10/20 4:05 PM, Stefan Reiter wrote: > The previously introduced approach can fail for pinned versions when a > new QEMU release is introduced. The saner approach is to use a mapping > that gives one pve-version for each QEMU release. > > Fortunately, the old system has not been bumped yet, so we can still > change it without too much effort. > > QEMU versions without a mapping are assumed to be pve0, 4.1 is mapped to > pve1 since thats what we had as our default previously. > > Pinned machine versions (i.e. pc-i440fx-4.1) are always assumed to be > pve0, for specific pve-versions they'd have to be pinned as well (i.e. > pc-i440fx-4.1+pve1). > > The new logic also makes the pve-version dynamic, and starts VMs with > the lowest possible 'feature-level', i.e. if a feature is only available > with 4.1+pve2, but the VM isn't using it, we still start it with > 4.1+pve0. > > We die if we don't support a version that is requested from us. This > allows us to use the pve-version as live-migration blocks (i.e. bumping > the version and then live-migrating a VM which uses the new feature (so > is running with the bumped version) to an outdated node will present the > user with a helpful error message and fail instead of silently modifying > the config and only failing *after* the migration). > > $version_guard is introduced in config_to_command to use for features > that need to check pve-version, it automatically handles selecting the > newest necessary pve-version for the VM. > > Tests have to be adjusted, since all of them now resolve to pve0 instead > of pve1. EXPECT_ERROR matching is changed to use 'eq' instead of regex > to allow special characters in error messages. > > Signed-off-by: Stefan Reiter > --- > > For some reason, thinking this through is astonishingly confusing. My head is > spinning with QEMU versions, and I'll probably have nightmares of '+pve0' > tonight, but I *think* this should now cover all of our bases? > > Based on several off-list email and in-person discussions with Thomas and > Dominik. > > Speaking of @Dominik: You'd have to rebase your recent patch [0] to use the > new > format. > > Anyway, thourough review appreciated, let's avoid having to change this again. > > [0] https://pve.proxmox.com/pipermail/pve-devel/2020-February/041588.html > > > PVE/QemuServer.pm | 53 +++ > PVE/QemuServer/Machine.pm | 38 - > test/cfg2cmd/i440fx-win10-hostpci.conf.cmd| 2 +- > .../minimal-defaults-to-new-machine.conf | 2 +- > ...imal-defaults-unsupported-pve-version.conf | 5 ++ > test/cfg2cmd/minimal-defaults.conf.cmd| 2 +- > test/cfg2cmd/pinned-version.conf.cmd | 2 +- > .../q35-linux-hostpci-multifunction.conf.cmd | 2 +- > test/cfg2cmd/q35-linux-hostpci.conf.cmd | 2 +- > test/cfg2cmd/q35-win10-hostpci.conf.cmd | 2 +- > test/cfg2cmd/spice-linux-4.1.conf.cmd | 2 +- > test/run_config2command_tests.pl | 2 +- > 12 files changed, 92 insertions(+), 22 deletions(-) > create mode 100644 test/cfg2cmd/minimal-defaults-unsupported-pve-version.conf > applied, head now also spinning, and it's probably a good idea to let this update traverse a bit slower through our repository cascade than normal.. Thanks! ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
Re: [pve-devel] applied-series: [PATCH v4 docs 0/7] Documenation overhaul chapt. 1.12 to 3.2
On 2/11/20 5:42 PM, Thomas Lamprecht wrote: applied series, but not really happy how you just ignored Stefan's and my comment about why you change title capitalization in just a single patch (sys requirements one) to something nowhere else used in the docs.. I'm sorry I forgot to answer that. Our current technical writing style guide states that first and second level headlines should use title capitalization and starting with third level headlines sentence style should be used. We can of course think about changing that, but changing the style guide too often will be problematic. Fixed that up and made a few general followup patches, renaming the "USB flash drive" to "Prepare Installation Media" and re-ordering it before "Using the Proxmox VE Installer" which makes quite more sense. IMO, a series and months-of-work effort called "overhauling the documentation" should also care for such details, i.e., do a step backward and see if the general order, flow, .. is all right and somewhat sound. As else I'd rather just have "grammar + typo fixes" getting in in continuous small patches and see bigger effort on expanding the documentation with new content. This patch series started a long time ago when I was anxious to not break anything (e.g. deep links). Therefore restructuring was saved for later. ___ pve-devel mailing list pve-devel@pve.proxmox.com https://pve.proxmox.com/cgi-bin/mailman/listinfo/pve-devel
