Script 'mail_helper' called by obssrc Hello community, here is the log from the commit of package virt-tuner for openSUSE:Factory checked in at 2021-08-27 21:44:20 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/virt-tuner (Old) and /work/SRC/openSUSE:Factory/.virt-tuner.new.1899 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "virt-tuner" Fri Aug 27 21:44:20 2021 rev:2 rq:914649 version:0.0.2 Changes: -------- --- /work/SRC/openSUSE:Factory/virt-tuner/virt-tuner.changes 2021-07-27 14:32:21.683466304 +0200 +++ /work/SRC/openSUSE:Factory/.virt-tuner.new.1899/virt-tuner.changes 2021-08-27 21:45:30.954084074 +0200 @@ -1,0 +2,6 @@ +Fri Aug 27 15:10:28 UTC 2021 - C??dric Bosdonnat <cbosdon...@suse.com> + +- Release 0.0.2: + - Allocate entire 1GiB hugepages for each cell on the single template + +------------------------------------------------------------------- Old: ---- virt-tuner-0.0.1.tar.gz New: ---- virt-tuner-0.0.2.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ virt-tuner.spec ++++++ --- /var/tmp/diff_new_pack.8pFuew/_old 2021-08-27 21:45:31.446084661 +0200 +++ /var/tmp/diff_new_pack.8pFuew/_new 2021-08-27 21:45:31.450084665 +0200 @@ -1,6 +1,7 @@ # -# spec file for package specRPM_CREATION_NAME +# spec file for package virt-tuner # +# Copyright (c) 2021 SUSE LLC # Copyright (c) specCURRENT_YEAR SUSE LINUX GmbH, Nuernberg, Germany. # # All modifications and additions to the file contributed by third parties @@ -12,26 +13,27 @@ # license that conforms to the Open Source Definition (Version 1.9) # published by the Open Source Initiative. -# Please submit bugfixes or comments via http://bugs.opensuse.org/ +# Please submit bugfixes or comments via https://bugs.opensuse.org/ # + %{?!python_module:%define python_module() python3-%{**}} %define pythons python3 Name: virt-tuner -Version: 0.0.1 +Version: 0.0.2 Release: 0 Summary: Virtual Machine definition tuner License: GPL-3.0-or-later Group: Productivity/Other -Url: https://github.com/cbosdo/virt-tuner +URL: https://github.com/cbosdo/virt-tuner Source: %{name}-%{version}.tar.gz -BuildRequires: %{python_module setuptools} BuildRequires: %{python_module libvirt-python} +BuildRequires: %{python_module setuptools} BuildRequires: fdupes Requires: python-libvirt-python Requires(post): update-alternatives -Requires(postun): update-alternatives +Requires(postun):update-alternatives BuildArch: noarch %python_subpackages ++++++ virt-tuner-0.0.1.tar.gz -> virt-tuner-0.0.2.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/virt-tuner-0.0.1/ChangeLog new/virt-tuner-0.0.2/ChangeLog --- old/virt-tuner-0.0.1/ChangeLog 2021-07-06 10:38:13.000000000 +0200 +++ new/virt-tuner-0.0.2/ChangeLog 2021-08-27 17:28:12.000000000 +0200 @@ -1,14 +1,57 @@ -2021-07-06 C??dric Bosdonnat <cbosdon...@suse.com> +2021-08-27 C??dric Bosdonnat <cbosdon...@suse.com> + + Fix numa memory merging whathever the unit is in CLI + + +2021-08-27 C??dric Bosdonnat <cbosdon...@suse.com> + + Bump version to 0.0.2 + + +2021-08-27 C??dric Bosdonnat <cbosdon...@suse.com> + + Fix memory computation in single template + The memory calculation needs to consider two rules: + * each NUMA cell needs to have a multiple of 1GiB of memory + * each NUMA cell should have the same amount of memory + * keep the host 9% or more for the host + + The previous computation only took a multiple of 1GiB of memory for the + entire VM and didn't care about the amount on hugepages that could fit + on each cell. + + +2021-08-24 C??dric Bosdonnat <cbosdon...@suse.com> + + Remove unneeded run script + + +2021-08-24 C??dric Bosdonnat <cbosdon...@suse.com> + + Fix run instruction in README + + +2021-07-19 C??dric Bosdonnat <cbosdon...@suse.com> + + Flag virt_tuner descriptions as localizable + + +2021-07-13 C??dric Bosdonnat <cbosdon...@suse.com> + + Fix repostory URL in setup.py + + +2021-07-13 C??dric Bosdonnat <cbosdon...@suse.com> Fix entry point and --template handling -2021-06-28 C??dric Bosdonnat <cbosdon...@suse.com> +2021-07-13 C??dric Bosdonnat <cbosdon...@suse.com> Handle single vm per host tuning -2021-06-28 C??dric Bosdonnat <cbosdon...@suse.com> +2021-07-13 C??dric Bosdonnat <cbosdon...@suse.com> Initial commit diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/virt-tuner-0.0.1/PKG-INFO new/virt-tuner-0.0.2/PKG-INFO --- old/virt-tuner-0.0.1/PKG-INFO 2021-07-06 10:38:13.776485000 +0200 +++ new/virt-tuner-0.0.2/PKG-INFO 2021-08-27 17:28:12.306105900 +0200 @@ -1,8 +1,8 @@ Metadata-Version: 2.1 Name: virt-tuner -Version: 0.0.1 +Version: 0.0.2 Summary: VM definition tuner -Home-page: https://github.com/cbosdo/virt-tuner +Home-page: https://github.com/SUSE/virt-tuner Author: Cedric Bosdonnat Author-email: cbosdon...@suse.com License: GPLv3+ @@ -22,7 +22,7 @@ use the run script. For example to run virt-tuner, use a command like the following one: - ./run src/virt_tuner/main.py --help + PYTHONPATH=$PWD/src python3 -m virt_tuner --help The following commands will be useful for anyone writing patches: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/virt-tuner-0.0.1/README.md new/virt-tuner-0.0.2/README.md --- old/virt-tuner-0.0.1/README.md 2021-06-28 14:01:54.000000000 +0200 +++ new/virt-tuner-0.0.2/README.md 2021-08-24 09:13:13.000000000 +0200 @@ -14,7 +14,7 @@ use the run script. For example to run virt-tuner, use a command like the following one: - ./run src/virt_tuner/main.py --help + PYTHONPATH=$PWD/src python3 -m virt_tuner --help The following commands will be useful for anyone writing patches: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/virt-tuner-0.0.1/setup.py new/virt-tuner-0.0.2/setup.py --- old/virt-tuner-0.0.1/setup.py 2021-07-06 10:37:37.000000000 +0200 +++ new/virt-tuner-0.0.2/setup.py 2021-08-24 09:13:13.000000000 +0200 @@ -202,7 +202,7 @@ description="VM definition tuner", license="GPLv3+", long_description=read("README.md"), - url="https://github.com/cbosdo/virt-tuner";, + url="https://github.com/SUSE/virt-tuner";, keywords="virtualization", package_dir={"": "src"}, packages=setuptools.find_packages("src"), diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/virt-tuner-0.0.1/src/virt_tuner/__init__.py new/virt-tuner-0.0.2/src/virt_tuner/__init__.py --- old/virt-tuner-0.0.1/src/virt_tuner/__init__.py 2021-06-28 14:01:54.000000000 +0200 +++ new/virt-tuner-0.0.2/src/virt_tuner/__init__.py 2021-08-27 17:05:31.000000000 +0200 @@ -21,13 +21,23 @@ """ from collections import namedtuple +import gettext import logging import itertools import virt_tuner.virt +gettext.bindtextdomain("virt-tuner", "/usr/share/locale") +gettext.textdomain("virt-tuner") +try: + gettext.install("virt-tuner", localedir="/usr/share/locale") +except IOError: + import builtins + + builtins.__dict__["_"] = str + log = logging.getLogger(__name__) -__version__ = "0.0.1" +__version__ = "0.0.2" Template = namedtuple("Template", ["description", "function", "parameters"]) @@ -51,8 +61,18 @@ }.pop(), } - # Round to multiple of 1GiB due to enforced 1GiB hugepages - vm_memory = int(virt_tuner.virt.host_memory() * 0.9 / 1024) * 1024 + # Compute the amount of 1GiB pages per cell. + # We want an even number of pages on each cell + # and at least 9% of the total memory amount for the host + cells_mem = [cell.memory for cell in cells] + cells_pages = [int(mem / (1024 ** 2)) for mem in cells_mem] + cell_pages_max = min(cells_pages) + + pages_per_cell = int(sum(cells_mem) * 0.91 / (1024 ** 2) / len(cells_mem)) + if cell_pages_max * len(cells_mem) * 1024 ** 2 / sum(cells_mem) <= 0.91: + pages_per_cell = cell_pages_max + + vm_memory = pages_per_cell * len(cells) return { "cpu": { @@ -74,7 +94,7 @@ "numa": { c.id: { "cpus": ",".join([str(cpu["id"]) for cpu in c.cpus]), - "memory": str(int(vm_memory / len(cells))) + " MiB", + "memory": str(pages_per_cell) + " GiB", "distances": c.distances, } for c in cells @@ -90,8 +110,8 @@ }, }, "mem": { - "boot": str(vm_memory) + " MiB", - "current": str(vm_memory) + " MiB", + "boot": str(vm_memory) + " GiB", + "current": str(vm_memory) + " GiB", "nosharepages": True, "hugepages": [{"size": "1 G"}], }, @@ -108,7 +128,7 @@ templates = { "single": Template( - "Single virtual machine using almost all the host resources", + _("Single virtual machine using almost all the host resources"), single, [], ), diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/virt-tuner-0.0.1/src/virt_tuner/main.py new/virt-tuner-0.0.2/src/virt_tuner/main.py --- old/virt-tuner-0.0.1/src/virt_tuner/main.py 2021-07-06 10:33:10.000000000 +0200 +++ new/virt-tuner-0.0.2/src/virt_tuner/main.py 2021-08-24 09:13:13.000000000 +0200 @@ -21,7 +21,6 @@ """ import argparse -import gettext import logging import os.path import sys @@ -29,15 +28,6 @@ import virt_tuner import virt_tuner.xmlutil as xmlutil -gettext.bindtextdomain("virt-tuner", "/usr/share/locale") -gettext.textdomain("virt-tuner") -try: - gettext.install("virt-tuner", localedir="/usr/share/locale") -except IOError: - import builtins - - builtins.__dict__["_"] = str - logger = logging.getLogger("virt_tuner.main") diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/virt-tuner-0.0.1/src/virt_tuner/virt.py new/virt-tuner-0.0.2/src/virt_tuner/virt.py --- old/virt-tuner-0.0.1/src/virt_tuner/virt.py 2021-06-28 14:01:54.000000000 +0200 +++ new/virt-tuner-0.0.2/src/virt_tuner/virt.py 2021-08-27 16:35:29.000000000 +0200 @@ -43,9 +43,7 @@ Cell( int(node.get("id")), [cpu.attrib for cpu in node.findall("cpus/cpu")], - "{} {}".format( - node.find("memory").text, node.find("memory").get("unit") - ), + int(node.find("memory").text), # libvirt always outputs memory in KiB { int(dist.get("id")): int(dist.get("value")) for dist in node.findall("distances/sibling") @@ -66,19 +64,3 @@ cnx.close() return cells - - -def host_memory(): - """ - The amount of memory of the virtual host in MiB - """ - cnx = libvirt.open() - mem = 0 - try: - info = cnx.getInfo() - mem = info[1] - except libvirt.libvirtError as err: - log.error(err) - finally: - cnx.close() - return mem diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/virt-tuner-0.0.1/src/virt_tuner/xmlutil.py new/virt-tuner-0.0.2/src/virt_tuner/xmlutil.py --- old/virt-tuner-0.0.1/src/virt_tuner/xmlutil.py 2021-06-28 14:01:54.000000000 +0200 +++ new/virt-tuner-0.0.2/src/virt_tuner/xmlutil.py 2021-08-27 17:24:02.000000000 +0200 @@ -108,10 +108,10 @@ numa_path = ["cpu", "numa", f"cell[@id='{cell_id}']"] set_attribute(doc, numa_path, "cpus", numa.get("cpus")) memory = numa.get("memory") - if memory and memory.endswith(" MiB"): - memory = memory.split(" ")[0] + if memory: + (memory, unit) = memory.split(" ") set_attribute(doc, numa_path, "memory", memory) - set_attribute(doc, numa_path, "unit", "MiB") + set_attribute(doc, numa_path, "unit", unit) for sibling_id, distance in numa.get("distances", {}).items(): set_attribute( diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/virt-tuner-0.0.1/src/virt_tuner.egg-info/PKG-INFO new/virt-tuner-0.0.2/src/virt_tuner.egg-info/PKG-INFO --- old/virt-tuner-0.0.1/src/virt_tuner.egg-info/PKG-INFO 2021-07-06 10:38:13.000000000 +0200 +++ new/virt-tuner-0.0.2/src/virt_tuner.egg-info/PKG-INFO 2021-08-27 17:28:12.000000000 +0200 @@ -1,8 +1,8 @@ Metadata-Version: 2.1 Name: virt-tuner -Version: 0.0.1 +Version: 0.0.2 Summary: VM definition tuner -Home-page: https://github.com/cbosdo/virt-tuner +Home-page: https://github.com/SUSE/virt-tuner Author: Cedric Bosdonnat Author-email: cbosdon...@suse.com License: GPLv3+ @@ -22,7 +22,7 @@ use the run script. For example to run virt-tuner, use a command like the following one: - ./run src/virt_tuner/main.py --help + PYTHONPATH=$PWD/src python3 -m virt_tuner --help The following commands will be useful for anyone writing patches: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/virt-tuner-0.0.1/tests/test_virt.py new/virt-tuner-0.0.2/tests/test_virt.py --- old/virt-tuner-0.0.1/tests/test_virt.py 2021-06-28 14:01:54.000000000 +0200 +++ new/virt-tuner-0.0.2/tests/test_virt.py 2021-08-27 16:37:20.000000000 +0200 @@ -172,29 +172,10 @@ "core_id": "0", "siblings": "0,48", } - assert topology[1].memory == "32982940 KiB" + assert topology[1].memory == 32982940 assert topology[0].distances == {0: 10, 1: 21} assert topology[0].pages == [ {"size": "4 KiB", "count": 8161648}, {"size": "2048 KiB", "count": 0}, {"size": "1048576 KiB", "count": 0}, ] - - -def test_host_memory(): - """ - test the virt.host_memory() function - """ - libvirt_mock = MagicMock() - libvirt_mock.open.return_value.getInfo.return_value = [ - "x86_64", - 31944, - 8, - 2700, - 1, - 1, - 4, - 2, - ] - virt_tuner.virt.libvirt = libvirt_mock - assert virt_tuner.virt.host_memory() == 31944 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/virt-tuner-0.0.1/tests/test_virt_tuner.py new/virt-tuner-0.0.2/tests/test_virt_tuner.py --- old/virt-tuner-0.0.1/tests/test_virt_tuner.py 2021-06-28 14:01:54.000000000 +0200 +++ new/virt-tuner-0.0.2/tests/test_virt_tuner.py 2021-08-27 16:55:47.000000000 +0200 @@ -2,11 +2,20 @@ Tests for the virt_tuner module """ -from unittest.mock import MagicMock +from unittest.mock import patch +import pytest import virt_tuner -def test_single(): +@pytest.mark.parametrize( + "cell_mems, expected_cell_pages", + [ + ([13757440, 19270488, 15854592, 16808124], 13), + ([15854592, 16808124, 15854592, 17173336], 14), + ], + ids=["unbalanced cells", "balanced cells"], +) +def test_single(cell_mems, expected_cell_pages): """ Test the virt_tuner.single() function """ @@ -19,7 +28,7 @@ {"id": 2, "socket_id": 0, "core_id": 0, "siblings": "0,2"}, {"id": 3, "socket_id": 0, "core_id": 1, "siblings": "1,3"}, ], - "905532 KiB", + cell_mems[0], {"0": 10, "1": 20, "2": 20, "3": 20}, [ {"size": "4 KiB", "count": 226383}, @@ -35,7 +44,7 @@ {"id": 6, "socket_id": 1, "core_id": 0, "siblings": "4,6"}, {"id": 7, "socket_id": 1, "core_id": 1, "siblings": "5,7"}, ], - "909096 KiB", + cell_mems[1], {"0": 20, "1": 10, "2": 20, "3": 20}, [ {"size": "4 KiB", "count": 227274}, @@ -51,7 +60,7 @@ {"id": 10, "socket_id": 2, "core_id": 0, "siblings": "8,10"}, {"id": 11, "socket_id": 2, "core_id": 1, "siblings": "9,11"}, ], - "908072 KiB", + cell_mems[2], {"0": 20, "1": 20, "2": 10, "3": 20}, [ {"size": "4 KiB", "count": 227018}, @@ -67,7 +76,7 @@ {"id": 14, "socket_id": 3, "core_id": 0, "siblings": "12,14"}, {"id": 15, "socket_id": 3, "core_id": 1, "siblings": "13,15"}, ], - "808460 KiB", + cell_mems[3], {"0": 20, "1": 20, "2": 20, "3": 10}, [ {"size": "4 KiB", "count": 202115}, @@ -77,90 +86,88 @@ ), ] - virt_mock = MagicMock() - virt_mock.host_topology.return_value = cells - virt_mock.host_memory.return_value = 3448 - virt_tuner.virt = virt_mock + with patch("virt_tuner.virt") as virt_mock: + virt_mock.host_topology.return_value = cells - assert virt_tuner.single() == { - "cpu": { - "placement": "static", - "maximum": 16, - "topology": {"sockets": 4, "cores": 2, "threads": 2}, - "mode": "host-passthrough", - "check": "none", - "features": { - "rdtscp": "require", - "invtsc": "require", - "x2apic": "require", - }, - "tuning": { - "vcpupin": { - 0: "0,2", - 1: "1,3", - 2: "0,2", - 3: "1,3", - 4: "4,6", - 5: "5,7", - 6: "4,6", - 7: "5,7", - 8: "8,10", - 9: "9,11", - 10: "8,10", - 11: "9,11", - 12: "12,14", - 13: "13,15", - 14: "12,14", - 15: "13,15", + assert virt_tuner.single() == { + "cpu": { + "placement": "static", + "maximum": 16, + "topology": {"sockets": 4, "cores": 2, "threads": 2}, + "mode": "host-passthrough", + "check": "none", + "features": { + "rdtscp": "require", + "invtsc": "require", + "x2apic": "require", }, - }, - "numa": { - 0: { - "cpus": "0,1,2,3", - "memory": "768 MiB", - "distances": {"0": 10, "1": 20, "2": 20, "3": 20}, - }, - 1: { - "cpus": "4,5,6,7", - "memory": "768 MiB", - "distances": {"0": 20, "1": 10, "2": 20, "3": 20}, - }, - 2: { - "cpus": "8,9,10,11", - "memory": "768 MiB", - "distances": {"0": 20, "1": 20, "2": 10, "3": 20}, - }, - 3: { - "cpus": "12,13,14,15", - "memory": "768 MiB", - "distances": {"0": 20, "1": 20, "2": 20, "3": 10}, + "tuning": { + "vcpupin": { + 0: "0,2", + 1: "1,3", + 2: "0,2", + 3: "1,3", + 4: "4,6", + 5: "5,7", + 6: "4,6", + 7: "5,7", + 8: "8,10", + 9: "9,11", + 10: "8,10", + 11: "9,11", + 12: "12,14", + 13: "13,15", + 14: "12,14", + 15: "13,15", + }, + }, + "numa": { + 0: { + "cpus": "0,1,2,3", + "memory": str(expected_cell_pages) + " GiB", + "distances": {"0": 10, "1": 20, "2": 20, "3": 20}, + }, + 1: { + "cpus": "4,5,6,7", + "memory": str(expected_cell_pages) + " GiB", + "distances": {"0": 20, "1": 10, "2": 20, "3": 20}, + }, + 2: { + "cpus": "8,9,10,11", + "memory": str(expected_cell_pages) + " GiB", + "distances": {"0": 20, "1": 20, "2": 10, "3": 20}, + }, + 3: { + "cpus": "12,13,14,15", + "memory": str(expected_cell_pages) + " GiB", + "distances": {"0": 20, "1": 20, "2": 20, "3": 10}, + }, }, }, - }, - "numatune": { - "memory": { - "mode": "strict", - "nodeset": "0,1,2,3", - }, - "memnodes": { - 0: {"mode": "strict", "nodeset": 0}, - 1: {"mode": "strict", "nodeset": 1}, - 2: {"mode": "strict", "nodeset": 2}, - 3: {"mode": "strict", "nodeset": 3}, + "numatune": { + "memory": { + "mode": "strict", + "nodeset": "0,1,2,3", + }, + "memnodes": { + 0: {"mode": "strict", "nodeset": 0}, + 1: {"mode": "strict", "nodeset": 1}, + 2: {"mode": "strict", "nodeset": 2}, + 3: {"mode": "strict", "nodeset": 3}, + }, }, - }, - "hypervisor_features": {"kvm-hint-dedicated": True}, - "mem": { - "boot": "3072 MiB", - "current": "3072 MiB", - "nosharepages": True, - "hugepages": [{"size": "1 G"}], - }, - "clock": { - "timers": { - "rtc": {"tickpolicy": "catchup"}, - "pit": {"tickpolicy": "catchup"}, - "hpet": {"present": False}, + "hypervisor_features": {"kvm-hint-dedicated": True}, + "mem": { + "boot": str(expected_cell_pages * len(cell_mems)) + " GiB", + "current": str(expected_cell_pages * len(cell_mems)) + " GiB", + "nosharepages": True, + "hugepages": [{"size": "1 G"}], + }, + "clock": { + "timers": { + "rtc": {"tickpolicy": "catchup"}, + "pit": {"tickpolicy": "catchup"}, + "hpet": {"present": False}, + }, }, - }, - } + }
