Re: [PATCH] linux-user/signal: Map exit signals in SIGCHLD siginfo_t
On 23/10/2021 21:59, Matthias Schiffer wrote:
When converting a siginfo_t from waitid(), the interpretation of si_status
depends on the value of si_code: For CLD_EXITED, it is an exit code and
should be copied verbatim. For other codes, it is a signal number
(possibly with additional high bits from ptrace) that should be mapped.
This code was previously changed in commit 1c3dfb506ea3
("linux-user/signal: Decode waitid si_code"), but the fix was
incomplete.
ping
Tested with the following test program:
#include
#include
#include
#include
int main() {
pid_t pid = fork();
if (pid == 0) {
exit(12);
} else {
siginfo_t siginfo = {};
waitid(P_PID, pid, , WEXITED);
printf("Code: %d, status: %d\n", (int)siginfo.si_code,
(int)siginfo.si_status);
}
pid = fork();
if (pid == 0) {
raise(SIGUSR2);
} else {
siginfo_t siginfo = {};
waitid(P_PID, pid, , WEXITED);
printf("Code: %d, status: %d\n", (int)siginfo.si_code,
(int)siginfo.si_status);
}
}
Output with an x86_64 host and mips64el target before 1c3dfb506ea3
(incorrect: exit code 12 is translated like a signal):
Code: 1, status: 17
Code: 2, status: 17
After 1c3dfb506ea3 (incorrect: signal number is not translated):
Code: 1, status: 12
Code: 2, status: 12
With this patch:
Code: 1, status: 12
Code: 2, status: 17
Signed-off-by: Matthias Schiffer
---
linux-user/signal.c | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/linux-user/signal.c b/linux-user/signal.c
index 14d8fdfde152..8e3af98ec0a7 100644
--- a/linux-user/signal.c
+++ b/linux-user/signal.c
@@ -403,7 +403,12 @@ static inline void
host_to_target_siginfo_noswap(target_siginfo_t *tinfo,
case TARGET_SIGCHLD:
tinfo->_sifields._sigchld._pid = info->si_pid;
tinfo->_sifields._sigchld._uid = info->si_uid;
-tinfo->_sifields._sigchld._status = info->si_status;
+if (si_code == CLD_EXITED)
+tinfo->_sifields._sigchld._status = info->si_status;
+else
+tinfo->_sifields._sigchld._status
+= host_to_target_signal(info->si_status & 0x7f)
+| (info->si_status & ~0x7f);
tinfo->_sifields._sigchld._utime = info->si_utime;
tinfo->_sifields._sigchld._stime = info->si_stime;
si_type = QEMU_SI_CHLD;
OpenPGP_signature
Description: OpenPGP digital signature
[PATCH] linux-user/signal: Map exit signals in SIGCHLD siginfo_t
When converting a siginfo_t from waitid(), the interpretation of si_status
depends on the value of si_code: For CLD_EXITED, it is an exit code and
should be copied verbatim. For other codes, it is a signal number
(possibly with additional high bits from ptrace) that should be mapped.
This code was previously changed in commit 1c3dfb506ea3
("linux-user/signal: Decode waitid si_code"), but the fix was
incomplete.
Tested with the following test program:
#include
#include
#include
#include
int main() {
pid_t pid = fork();
if (pid == 0) {
exit(12);
} else {
siginfo_t siginfo = {};
waitid(P_PID, pid, , WEXITED);
printf("Code: %d, status: %d\n", (int)siginfo.si_code,
(int)siginfo.si_status);
}
pid = fork();
if (pid == 0) {
raise(SIGUSR2);
} else {
siginfo_t siginfo = {};
waitid(P_PID, pid, , WEXITED);
printf("Code: %d, status: %d\n", (int)siginfo.si_code,
(int)siginfo.si_status);
}
}
Output with an x86_64 host and mips64el target before 1c3dfb506ea3
(incorrect: exit code 12 is translated like a signal):
Code: 1, status: 17
Code: 2, status: 17
After 1c3dfb506ea3 (incorrect: signal number is not translated):
Code: 1, status: 12
Code: 2, status: 12
With this patch:
Code: 1, status: 12
Code: 2, status: 17
Signed-off-by: Matthias Schiffer
---
linux-user/signal.c | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/linux-user/signal.c b/linux-user/signal.c
index 14d8fdfde152..8e3af98ec0a7 100644
--- a/linux-user/signal.c
+++ b/linux-user/signal.c
@@ -403,7 +403,12 @@ static inline void
host_to_target_siginfo_noswap(target_siginfo_t *tinfo,
case TARGET_SIGCHLD:
tinfo->_sifields._sigchld._pid = info->si_pid;
tinfo->_sifields._sigchld._uid = info->si_uid;
-tinfo->_sifields._sigchld._status = info->si_status;
+if (si_code == CLD_EXITED)
+tinfo->_sifields._sigchld._status = info->si_status;
+else
+tinfo->_sifields._sigchld._status
+= host_to_target_signal(info->si_status & 0x7f)
+| (info->si_status & ~0x7f);
tinfo->_sifields._sigchld._utime = info->si_utime;
tinfo->_sifields._sigchld._stime = info->si_stime;
si_type = QEMU_SI_CHLD;
--
2.33.1
[PATCH] plugins: new syscalls plugin
This commit adds a new syscalls plugin that displays the syscalls
as they are executed and returned. This plugin outputs the number
of the syscall as well as the syscall return value.
Works in *-user only.
Essentially, this commit restores:
https://lists.gnu.org/archive/html/qemu-devel/2018-06/msg00846.html
by using the new QEMU plugin API.
Signed-off-by: Matthias Weckbecker
---
tests/plugin/Makefile | 1 +
tests/plugin/syscall.c | 49 ++
2 files changed, 50 insertions(+)
create mode 100644 tests/plugin/syscall.c
diff --git a/tests/plugin/Makefile b/tests/plugin/Makefile
index e9348fde4a..fc176909e9 100644
--- a/tests/plugin/Makefile
+++ b/tests/plugin/Makefile
@@ -21,6 +21,7 @@ NAMES += hotblocks
NAMES += howvec
NAMES += hotpages
NAMES += lockstep
+NAMES += syscall
SONAMES := $(addsuffix .so,$(addprefix lib,$(NAMES)))
diff --git a/tests/plugin/syscall.c b/tests/plugin/syscall.c
new file mode 100644
index 00..53ee2ab6c4
--- /dev/null
+++ b/tests/plugin/syscall.c
@@ -0,0 +1,49 @@
+/*
+ * Copyright (C) 2020, Matthias Weckbecker
+ *
+ * License: GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include
+
+QEMU_PLUGIN_EXPORT int qemu_plugin_version = QEMU_PLUGIN_VERSION;
+
+static void vcpu_syscall(qemu_plugin_id_t id, unsigned int vcpu_index,
+ int64_t num, uint64_t a1, uint64_t a2,
+ uint64_t a3, uint64_t a4, uint64_t a5,
+ uint64_t a6, uint64_t a7, uint64_t a8)
+{
+g_autofree gchar *out = g_strdup_printf("syscall #%" PRIi64 "\n", num);
+qemu_plugin_outs(out);
+}
+
+static void vcpu_syscall_ret(qemu_plugin_id_t id, unsigned int vcpu_idx,
+ int64_t num, int64_t ret)
+{
+g_autofree gchar *out;
+out = g_strdup_printf("syscall #%" PRIi64 " returned -> %" PRIi64 "\n",
+num, ret);
+qemu_plugin_outs(out);
+}
+
+/* * */
+
+static void plugin_exit(qemu_plugin_id_t id, void *p) {}
+
+QEMU_PLUGIN_EXPORT int qemu_plugin_install(qemu_plugin_id_t id,
+ const qemu_info_t *info,
+ int argc, char **argv)
+{
+qemu_plugin_register_vcpu_syscall_cb(id, vcpu_syscall);
+qemu_plugin_register_vcpu_syscall_ret_cb(id, vcpu_syscall_ret);
+qemu_plugin_register_atexit_cb(id, plugin_exit, NULL);
+return 0;
+}
--
2.23.0
Re: [PATCH v2] linux-user: implement TARGET_SO_PEERSEC
Hi I also tested the v2 patch - works fine! Best regards Matthias
Re: [Bug 1823790] Re: QEMU mishandling of SO_PEERSEC forces systemd into tight loop
> Could you test the attached patch? > Works great! This is my test setup: Host machine: Ubuntu 18.04, amd64 LXD container: Debian Buster, arm64, systemd 241 QEMU: qemu-aarch64(-static), compiled from source (4.2.0), patched with your patch. Many thanks! Matthias -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1823790 Title: QEMU mishandling of SO_PEERSEC forces systemd into tight loop Status in QEMU: Confirmed Bug description: While building Debian images for embedded ARM target systems I detected that QEMU seems to force newer systemd daemons into a tight loop. My setup is the following: Host machine: Ubuntu 18.04, amd64 LXD container: Debian Buster, arm64, systemd 241 QEMU: qemu-aarch64-static, 4.0.0-rc2 (custom build) and 3.1.0 (Debian 1:3.1+dfsg-7) To easily reproduce the issue I have created the following repository: https://github.com/lueschem/edi-qemu The call where systemd gets looping is the following: 2837 getsockopt(3,1,31,274891889456,274887218756,274888927920) = -1 errno=34 (Numerical result out of range) Furthermore I also verified that the issue is not related to LXD. The same behavior can be reproduced using systemd-nspawn. This issue reported against systemd seems to be related: https://github.com/systemd/systemd/issues/11557 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1823790/+subscriptions
[Qemu-devel] [Bug 1823790] Re: QEMU mishandling of SO_PEERSEC forces systemd into tight loop
I have just studied a bit the systemd code and this brought me to the following idea/temporary workaround: What about returning -1 (error) and setting errno when getsockopt(fd, SOL_SOCKET, SO_PEERSEC, ...) gets called? This would then let systemd know that SO_PEERSEC is not (yet) implemented. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1823790 Title: QEMU mishandling of SO_PEERSEC forces systemd into tight loop Status in QEMU: New Bug description: While building Debian images for embedded ARM target systems I detected that QEMU seems to force newer systemd daemons into a tight loop. My setup is the following: Host machine: Ubuntu 18.04, amd64 LXD container: Debian Buster, arm64, systemd 241 QEMU: qemu-aarch64-static, 4.0.0-rc2 (custom build) and 3.1.0 (Debian 1:3.1+dfsg-7) To easily reproduce the issue I have created the following repository: https://github.com/lueschem/edi-qemu The call where systemd gets looping is the following: 2837 getsockopt(3,1,31,274891889456,274887218756,274888927920) = -1 errno=34 (Numerical result out of range) Furthermore I also verified that the issue is not related to LXD. The same behavior can be reproduced using systemd-nspawn. This issue reported against systemd seems to be related: https://github.com/systemd/systemd/issues/11557 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1823790/+subscriptions
[Qemu-devel] [Bug 1823790] Re: QEMU mishandling of SO_PEERSEC forces systemd into tight loop
This is probably the tight loop that gets triggered: https://github.com/systemd/systemd/commit/217d89678269334f461e9abeeffed57077b21454 It looks like the previous implementation was just a bit more "tolerant". -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1823790 Title: QEMU mishandling of SO_PEERSEC forces systemd into tight loop Status in QEMU: New Bug description: While building Debian images for embedded ARM target systems I detected that QEMU seems to force newer systemd daemons into a tight loop. My setup is the following: Host machine: Ubuntu 18.04, amd64 LXD container: Debian Buster, arm64, systemd 241 QEMU: qemu-aarch64-static, 4.0.0-rc2 (custom build) and 3.1.0 (Debian 1:3.1+dfsg-7) To easily reproduce the issue I have created the following repository: https://github.com/lueschem/edi-qemu The call where systemd gets looping is the following: 2837 getsockopt(3,1,31,274891889456,274887218756,274888927920) = -1 errno=34 (Numerical result out of range) Furthermore I also verified that the issue is not related to LXD. The same behavior can be reproduced using systemd-nspawn. This issue reported against systemd seems to be related: https://github.com/systemd/systemd/issues/11557 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1823790/+subscriptions
[Qemu-devel] [Bug 1823790] [NEW] QEMU forces systemd into tight loop
Public bug reported: While building Debian images for embedded ARM target systems I detected that QEMU seems to force newer systemd daemons into a tight loop. My setup is the following: Host machine: Ubuntu 18.04, amd64 LXD container: Debian Buster, arm64, systemd 241 QEMU: qemu-aarch64-static, 4.0.0-rc2 (custom build) and 3.1.0 (Debian 1:3.1+dfsg-7) To easily reproduce the issue I have created the following repository: https://github.com/lueschem/edi-qemu The call where systemd gets looping is the following: 2837 getsockopt(3,1,31,274891889456,274887218756,274888927920) = -1 errno=34 (Numerical result out of range) Furthermore I also verified that the issue is not related to LXD. The same behavior can be reproduced using systemd-nspawn. This issue reported against systemd seems to be related: https://github.com/systemd/systemd/issues/11557 ** Affects: qemu Importance: Undecided Status: New -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1823790 Title: QEMU forces systemd into tight loop Status in QEMU: New Bug description: While building Debian images for embedded ARM target systems I detected that QEMU seems to force newer systemd daemons into a tight loop. My setup is the following: Host machine: Ubuntu 18.04, amd64 LXD container: Debian Buster, arm64, systemd 241 QEMU: qemu-aarch64-static, 4.0.0-rc2 (custom build) and 3.1.0 (Debian 1:3.1+dfsg-7) To easily reproduce the issue I have created the following repository: https://github.com/lueschem/edi-qemu The call where systemd gets looping is the following: 2837 getsockopt(3,1,31,274891889456,274887218756,274888927920) = -1 errno=34 (Numerical result out of range) Furthermore I also verified that the issue is not related to LXD. The same behavior can be reproduced using systemd-nspawn. This issue reported against systemd seems to be related: https://github.com/systemd/systemd/issues/11557 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1823790/+subscriptions
[Qemu-devel] [PATCH v2] hw/pci-bridge: Fix invalid free()
When loadvm'ing a *running* snapshot qemu crashes due to an invalid free. It's fortunately caught early by glibc heap memory corruption protection and qemu gets killed with SIGABRT. Steps to reproduce: 1) Create VM (e.g w/ virsh define) 2) Start the VM and take a snapshot while it's running and having a PCI bridge attached 3) Destroy the VM and revert the running snapshot. This commit fixes the issue. Signed-off-by: Matthias Weckbecker --- hw/pci/pci_bridge.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/hw/pci/pci_bridge.c b/hw/pci/pci_bridge.c index ee9dff2d3a..b9143ac88b 100644 --- a/hw/pci/pci_bridge.c +++ b/hw/pci/pci_bridge.c @@ -241,9 +241,9 @@ void pci_bridge_update_mappings(PCIBridge *br) * while another accesses an unaffected region. */ memory_region_transaction_begin(); pci_bridge_region_del(br, br->windows); +pci_bridge_region_cleanup(br, w); br->windows = pci_bridge_region_init(br); memory_region_transaction_commit(); -pci_bridge_region_cleanup(br, w); } /* default write_config function for PCI-to-PCI bridge */ -- 2.16.3
Re: [Qemu-devel] Fwd: [PATCH] QEMU patch for PCI handling bug (invalid free)
On Mon, Dec 10, 2018 at 03:56:53PM +0100, Kevin Wolf wrote: > Am 10.12.2018 um 14:38 hat Matthias Weckbecker geschrieben: > > Hi Kevin, > > > > I'm attaching a patch for qemu. Read below for details. > > > > There's a bug in qemu in the PCI bridge handling that can be triggered when > > following the steps below: > > > > 1) Create some VM (e.g. w/ virsh define) > > 2) Ensure there is a PCI bridge attached, e.g. w/ libvirt like this: > > > > > > > > > > > >> function='0x1'/> > > > > > > 3) Take a snapshot while it's *running*, like this: > > > > % virsh start mips64el-vm > > % virsh snapshot-create-as mips64el-vm mips64el-snap > > > > 4) Destroy the VM and revert from snapshot: > > > > % virsh destroy mips64el-vm > > % virsh snapshot-revert mips64el-vm mips64el-snap --running > >error: internal error: process exited while connecting to monitor: > > corrupted size vs. prev_size > > > > 5) qemu-system-mips64el crashes > > > > The attached patch resolves the issue. Can you maybe review+commit, please? > > Hi Matthias, > Hi Kevin, > thanks for sending the patch. The next step is that it needs to be > reviewed on the qemu-devel mailing list (CCed) and then the PCI > subsystem maintainers (Michael and Marcel, CCed as well) can commit it. > thank you for looking into it and forwarding it. > Maybe some of the explanation above should actually be moved to the > commit message of the patch itself? > yes, I agree. I have --amend'ed my commit message and re-attached it. > Kevin Thanks, Matthias > > - Forwarded message from Matthias Weckbecker > - > > (gdb) bt > #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 > #1 0x7fde12dfc801 in __GI_abort () at abort.c:79 > #2 0x7fde12e45897 in __libc_message (action=action@entry=do_abort, > fmt=fmt@entry=0x7fde12f72b9a "%s\n") at ../sysdeps/posix/libc_fatal.c:181 > #3 0x7fde12e4c90a in malloc_printerr (str=str@entry=0x7fde12f70c9d > "corrupted size vs. prev_size") at malloc.c:5350 > #4 0x7fde12e4cb0c in malloc_consolidate (av=av@entry=0x7fde131a7c40 > ) at malloc.c:4456 > #5 0x7fde12e5403b in _int_free (have_lock=0, p=, > av=0x7fde131a7c40 ) at malloc.c:4362 > #6 __GI___libc_free (mem=0x55f089173c20) at malloc.c:3124 > #7 0x55f086c85062 in phys_section_destroy (mr=0x55f089173c20) at > ./exec.c:1317 > #8 phys_sections_free (map=0x55f0890f1560) at ./exec.c:1325 > #9 address_space_dispatch_free (d=0x55f0890f1550) at ./exec.c:2777 > #10 0x55f086cc0509 in flatview_destroy (view=0x55f088a5caf0) at > ./memory.c:301 > #11 0x55f087031dc0 in call_rcu_thread (opaque=) at > ./util/rcu.c:272 > #12 0x7fde131b46db in start_thread (arg=0x7fde0aa39700) at > pthread_create.c:463 > #13 0x7fde12edd88f in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:95 > > ==13641== Thread 2: > > [0/5041] > ==13641== Invalid read of size 8 > ==13641==at 0x42755B: memory_region_unref (memory.c:1749) > ==13641==by 0x42755B: flatview_destroy (memory.c:307) > ==13641==by 0x798DBF: call_rcu_thread (rcu.c:272) > ==13641==by 0x97BF6DA: start_thread (pthread_create.c:463) > ==13641==by 0x9AF888E: clone (clone.S:95) > ==13641== Address 0x408e4670 is 64 bytes inside a block of size 1,440 free'd > ==13641==at 0x4C30D3B: free (in > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==13641==by 0x5E988B: get_pci_config_device (pci.c:491) > ==13641==by 0x660069: vmstate_load_state (vmstate.c:140) > ==13641==by 0x660236: vmstate_load_state (vmstate.c:137) > ==13641==by 0x659994: vmstate_load (savevm.c:748) > ==13641==by 0x65A7B1: qemu_loadvm_section_start_full.isra.11 > (savevm.c:1918) > ==13641==by 0x65AA67: qemu_loadvm_state_main.isra.13 (savevm.c:2013) > ==13641==by 0x65D7CE: qemu_loadvm_state (savevm.c:2092) > ==13641==by 0x65E40D: load_snapshot (savevm.c:2406) > ==13641==by 0x3E28C2: main (vl.c:4918) > ==13641== Block was alloc'd at > ==13641==at 0x4C2FB0F: malloc (in > /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so) > ==13641==by 0x8D35A28: g_malloc (in > /usr/lib/x86_64-linux-gnu/libglib-2.0.so.0.5600.3) > ==13641==by 0x5ECA8A: pci_bridge_region_init (pci_bridge.c:187) > ==13641==by 0x5ECEFC: pci_bridge_initfn (pci_bridge.c:3
Re: [Qemu-devel] [PATCH v3 1/2] qapi: open files in binary mode and use explicit decoding/encoding in common.py
On Mon, Jun 18, 2018, at 00:25 CDT, Markus Armbruster wrote: > Matthias Maier writes: > >> This is a different approach to fix the locale dependent encode/decode >> problem in common.py utilizing the binary read/write mode [1,2], and (if >> a python 3 interpreter is used) with explicit decode/encode arguments >> [3]. > > Why can't we simply pass encoding='utf-8' to open()? Because this breaks python-2 compatibility. Best, Matthias
[Qemu-devel] [PATCH v3 1/2] qapi: open files in binary mode and use explicit decoding/encoding in common.py
This is a different approach to fix the locale dependent encode/decode
problem in common.py utilizing the binary read/write mode [1,2], and (if
a python 3 interpreter is used) with explicit decode/encode arguments
[3].
This approach is preferred over the fix in commit d4e5ec877ca because it
is (a) locale independent, and (b) does not depend on the en_US.UTF_8
locale to be available.
[1] https://docs.python.org/3.6/library/stdtypes.html#bytes.decode
[2] https://docs.python.org/3.6/library/stdtypes.html#str.encode
[3] https://docs.python.org/3/howto/unicode.html
Signed-off-by: Arfrever Frehtes Taifersar Arahesis
Signed-off-by: Matthias Maier
---
scripts/qapi/common.py | 11 ---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 2462fc0291..a368e11a38 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -16,6 +16,7 @@ import errno
import os
import re
import string
+import sys
from collections import OrderedDict
builtin_types = {
@@ -259,6 +260,8 @@ class QAPISchemaParser(object):
previously_included.append(os.path.abspath(fp.name))
self.incl_info = incl_info
self.src = fp.read()
+if sys.version_info[0] >= 3:
+self.src = self.src.decode()
if self.src == '' or self.src[-1] != '\n':
self.src += '\n'
self.cursor = 0
@@ -340,7 +343,7 @@ class QAPISchemaParser(object):
return None
try:
-fobj = open(incl_fname, 'r')
+fobj = open(incl_fname, 'rb')
except IOError as e:
raise QAPISemError(info, '%s: %s' % (e.strerror, incl_fname))
return QAPISchemaParser(fobj, previously_included, info)
@@ -1492,7 +1495,7 @@ class QAPISchemaEvent(QAPISchemaEntity):
class QAPISchema(object):
def __init__(self, fname):
self._fname = fname
-parser = QAPISchemaParser(open(fname, 'r'))
+parser = QAPISchemaParser(open(fname, 'rb'))
exprs = check_exprs(parser.exprs)
self.docs = parser.docs
self._entity_list = []
@@ -2006,9 +2009,11 @@ class QAPIGen(object):
if e.errno != errno.EEXIST:
raise
fd = os.open(pathname, os.O_RDWR | os.O_CREAT, 0o666)
-f = os.fdopen(fd, 'r+')
+f = os.fdopen(fd, 'r+b')
text = (self._top(fname) + self._preamble + self._body
+ self._bottom(fname))
+if sys.version_info[0] >= 3:
+text = text.encode()
oldtext = f.read(len(text) + 1)
if text != oldtext:
f.seek(0)
--
2.16.4
[Qemu-devel] [PATCH v3 2/2] Revert commit d4e5ec877ca
This commit removes the PYTHON_UTF8 workaround. The problem with setting LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 is that the en_US.UTF-8 locale might not be available. In this case setting above locales results in build errors even though another UTF-8 locale was originally set [1]. The only stable way of fixing the encoding problem is by explicitly annotating encoding/decoding in the python script. [1] https://bugs.gentoo.org/657766 Signed-off-by: Arfrever Frehtes Taifersar Arahesis Signed-off-by: Matthias Maier --- Makefile | 6 ++ tests/Makefile.include | 6 +++--- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/Makefile b/Makefile index e46f2b625a..7ed9cc4a21 100644 --- a/Makefile +++ b/Makefile @@ -20,8 +20,6 @@ ifneq ($(wildcard config-host.mak),) all: include config-host.mak -PYTHON_UTF8 = LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 $(PYTHON) - git-submodule-update: .PHONY: git-submodule-update @@ -576,7 +574,7 @@ qga/qapi-generated/qga-qapi-commands.h qga/qapi-generated/qga-qapi-commands.c \ qga/qapi-generated/qga-qapi-doc.texi: \ qga/qapi-generated/qapi-gen-timestamp ; qga/qapi-generated/qapi-gen-timestamp: $(SRC_PATH)/qga/qapi-schema.json $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o qga/qapi-generated -p "qga-" $<, \ "GEN","$(@:%-timestamp=%)") @>$@ @@ -676,7 +674,7 @@ qapi/qapi-introspect.h qapi/qapi-introspect.c \ qapi/qapi-doc.texi: \ qapi-gen-timestamp ; qapi-gen-timestamp: $(qapi-modules) $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o "qapi" -b $<, \ "GEN","$(@:%-timestamp=%)") @>$@ diff --git a/tests/Makefile.include b/tests/Makefile.include index ca91da26cb..88f1bc1242 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -678,13 +678,13 @@ tests/test-qapi-events.c tests/test-qapi-events.h \ tests/test-qapi-introspect.c tests/test-qapi-introspect.h: \ tests/test-qapi-gen-timestamp ; tests/test-qapi-gen-timestamp: $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o tests -p "test-" $<, \ "GEN","$(@:%-timestamp=%)") @>$@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.json $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o tests/qapi-schema -p "doc-good-" $<, \ "GEN","$@") @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ @@ -942,7 +942,7 @@ check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y)) $(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: $(SRC_PATH)/%.json $(call quiet-command, PYTHONPATH=$(SRC_PATH)/scripts \ - $(PYTHON_UTF8) $(SRC_PATH)/tests/qapi-schema/test-qapi.py \ + $(PYTHON) $(SRC_PATH)/tests/qapi-schema/test-qapi.py \ $^ >$*.test.out 2>$*.test.err; \ echo $$? >$*.test.exit, \ "TEST","$*.out") -- 2.16.4
[Qemu-devel] [PATCH v3 0/2] Fix compilation with python-3 if en_US.UTF-8 is unavailable
Hi, this is hopefully the final iteration of the patches. v3: - reverse order of patches - rename second patch to "Revert commit ..." - drop "UTF-8" argument in decode()/encode(); bot functions default to utf-8 already. v2: - ensure compatibility with python2 by only calling encode()/decode() if script is run with a python3 interpreter Best, Matthias
Re: [Qemu-devel] [PATCH v2 2/2] qapi: open files in binary mode and use explicit decoding/encoding in common.py
On Fri, Jun 15, 2018, at 10:31 CDT, Markus Armbruster wrote: > If I understand 7.2.3. Standard Encodings[*] correctly, the canonical > name is "utf-8". Let's use that. Wait, it's the default, no need to > pass an argument. Roger. I will change this in v3.
Re: [Qemu-devel] [PATCH 1/2] Partially revert commit d4e5ec877ca
On Fri, Jun 15, 2018, at 15:34 CDT, Eduardo Habkost wrote: > Thanks for the patches. It's nice to see that hack go away. > > Patch 2/2 is necessary to make the removal of PYTHON_UTF8 safe, > correct? In that case, I plan to reverse the order of the > patches before applying. Yes, the order of the patches should be reversed.
Re: [Qemu-devel] [PATCH v2 1/2] Partially revert commit d4e5ec877ca
On Fri, Jun 15, 2018, at 10:17 CDT, Markus Armbruster wrote: > "Partially revert"? Which part isn't reverted? Yes, it ended up being a full revert of the commit in question. I am sorry for the sloppy wording. > [...] >>Recently, however, glibc introduced a new locale "C.utf-8" that just >>uses UTF-8 as its charset, but otherwise leaves the semantics alone. >>Just setting the right character set is enough for our use case, so we >>can just hardcode this one without having to be afraid of nasty side >>effects. > Looks like your frustration about upstream glibc is quite stale :) Unfortunately, this statement is not correct. The corresponding glibc bug report summarizes the current situation [1]. Fact is that a lot of distributions ship a custom C.UTF-8 locale, for example Debian [2] (for the currenct glibc-2.27 release). Unfortunately, not everyone applies such custom patches. :-/ Best, Matthias [1] https://sourceware.org/bugzilla/show_bug.cgi?id=17318 [2] https://sources.debian.org/patches/glibc/2.27-3/localedata/locale-C.diff/
Re: [Qemu-devel] [PATCH v2 1/2] Partially revert commit d4e5ec877ca
On Fri, Jun 15, 2018, at 04:42 CDT, Daniel P. Berrangé wrote: > On Thu, Jun 14, 2018 at 11:40:41PM -0500, Matthias Maier wrote: >> This commit removes the PYTHON_UTF8 workaround. The problem with setting >> >> LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 >> >> is that the en_US.UTF-8 locale might not be available. In this case > > What platform are you using where UTF8 locale is not available ? For example, neither Debian (and for that matter Ubuntu) nor Gentoo guarantee that the en_US.UTF-8 locale is available. We in particular encounter build problems on Gentoo when users have only set very specific, non en_US locales, for example de_DE.UTF-8 (or similar). > Indeed I would ideally like to make the entire of QEMU build with an > explicit en_US.UTF-8 or C.UTF-8 locale, to ensure that we get reliably > reproducible builds, as locale differences have been known to impact > output of many tools not just python. We face the same problem in Gentoo and usually advice users to set LC_ALL=C when submitting bug reports. (It is frustrating that glibc upstream doesn't get their act together fixing and merging the current C.UTF-8 proposal.) So what about making the build system more robust (by merging the patches, or a variant) and either setting C.UTF-8, or C globally (depending on availability)? Best, Matthias
[Qemu-devel] [PATCH v2 1/2] Partially revert commit d4e5ec877ca
This commit removes the PYTHON_UTF8 workaround. The problem with setting LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 is that the en_US.UTF-8 locale might not be available. In this case setting above locales results in build errors even though another UTF-8 locale was originally set [1]. The only stable way of fixing the encoding problem is by explicitly annotating encoding/decoding in the python script. [1] https://bugs.gentoo.org/657766 Signed-off-by: Arfrever Frehtes Taifersar Arahesis Signed-off-by: Matthias Maier --- Makefile | 6 ++ tests/Makefile.include | 6 +++--- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/Makefile b/Makefile index e46f2b625a..7ed9cc4a21 100644 --- a/Makefile +++ b/Makefile @@ -20,8 +20,6 @@ ifneq ($(wildcard config-host.mak),) all: include config-host.mak -PYTHON_UTF8 = LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 $(PYTHON) - git-submodule-update: .PHONY: git-submodule-update @@ -576,7 +574,7 @@ qga/qapi-generated/qga-qapi-commands.h qga/qapi-generated/qga-qapi-commands.c \ qga/qapi-generated/qga-qapi-doc.texi: \ qga/qapi-generated/qapi-gen-timestamp ; qga/qapi-generated/qapi-gen-timestamp: $(SRC_PATH)/qga/qapi-schema.json $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o qga/qapi-generated -p "qga-" $<, \ "GEN","$(@:%-timestamp=%)") @>$@ @@ -676,7 +674,7 @@ qapi/qapi-introspect.h qapi/qapi-introspect.c \ qapi/qapi-doc.texi: \ qapi-gen-timestamp ; qapi-gen-timestamp: $(qapi-modules) $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o "qapi" -b $<, \ "GEN","$(@:%-timestamp=%)") @>$@ diff --git a/tests/Makefile.include b/tests/Makefile.include index 607afe5bed..b5121f2851 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -674,13 +674,13 @@ tests/test-qapi-events.c tests/test-qapi-events.h \ tests/test-qapi-introspect.c tests/test-qapi-introspect.h: \ tests/test-qapi-gen-timestamp ; tests/test-qapi-gen-timestamp: $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o tests -p "test-" $<, \ "GEN","$(@:%-timestamp=%)") @>$@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.json $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o tests/qapi-schema -p "doc-good-" $<, \ "GEN","$@") @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ @@ -938,7 +938,7 @@ check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y)) $(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: $(SRC_PATH)/%.json $(call quiet-command, PYTHONPATH=$(SRC_PATH)/scripts \ - $(PYTHON_UTF8) $(SRC_PATH)/tests/qapi-schema/test-qapi.py \ + $(PYTHON) $(SRC_PATH)/tests/qapi-schema/test-qapi.py \ $^ >$*.test.out 2>$*.test.err; \ echo $$? >$*.test.exit, \ "TEST","$*.out") -- 2.16.4
[Qemu-devel] [PATCH v2 2/2] qapi: open files in binary mode and use explicit decoding/encoding in common.py
This is a different approach to fix the locale dependent encode/decode
problem in common.py utilizing the binary read/write mode [1,2] and
decode/encode with explicit UTF-8 encoding arguments [3].
This approach is preferred over the fix in commit d4e5ec877ca because it
is (a) locale independent, and (b) does not depend on the en_US.UTF_8
locale to be available.
[1] https://docs.python.org/3.6/library/stdtypes.html#bytes.decode
[2] https://docs.python.org/3.6/library/stdtypes.html#str.encode
[3] https://docs.python.org/3/howto/unicode.html
Signed-off-by: Arfrever Frehtes Taifersar Arahesis
Signed-off-by: Matthias Maier
---
scripts/qapi/common.py | 11 ---
1 file changed, 8 insertions(+), 3 deletions(-)
diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 2462fc0291..44270cd703 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -16,6 +16,7 @@ import errno
import os
import re
import string
+import sys
from collections import OrderedDict
builtin_types = {
@@ -259,6 +260,8 @@ class QAPISchemaParser(object):
previously_included.append(os.path.abspath(fp.name))
self.incl_info = incl_info
self.src = fp.read()
+if sys.version_info[0] >= 3:
+self.src = self.src.decode("UTF-8")
if self.src == '' or self.src[-1] != '\n':
self.src += '\n'
self.cursor = 0
@@ -340,7 +343,7 @@ class QAPISchemaParser(object):
return None
try:
-fobj = open(incl_fname, 'r')
+fobj = open(incl_fname, 'rb')
except IOError as e:
raise QAPISemError(info, '%s: %s' % (e.strerror, incl_fname))
return QAPISchemaParser(fobj, previously_included, info)
@@ -1492,7 +1495,7 @@ class QAPISchemaEvent(QAPISchemaEntity):
class QAPISchema(object):
def __init__(self, fname):
self._fname = fname
-parser = QAPISchemaParser(open(fname, 'r'))
+parser = QAPISchemaParser(open(fname, 'rb'))
exprs = check_exprs(parser.exprs)
self.docs = parser.docs
self._entity_list = []
@@ -2006,9 +2009,11 @@ class QAPIGen(object):
if e.errno != errno.EEXIST:
raise
fd = os.open(pathname, os.O_RDWR | os.O_CREAT, 0o666)
-f = os.fdopen(fd, 'r+')
+f = os.fdopen(fd, 'r+b')
text = (self._top(fname) + self._preamble + self._body
+ self._bottom(fname))
+if sys.version_info[0] >= 3:
+text = text.encode("UTF-8")
oldtext = f.read(len(text) + 1)
if text != oldtext:
f.seek(0)
--
2.16.4
[Qemu-devel] [PATCH v2 0/2] Fix compilation with python-3 if en_US.UTF-8 is unavailable
Hi, This new version of the patch is now also fully python2 compatible... Original message: This patch series, - removes the PYTHON_UTF8 workaround introduced in d4e5ec877ca - adds a different workaround that avoids the locale problem altogether by opening files in binary read/write mode and setting encoding/decoding (in utf-8) explicitly The problem with setting LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 is that the en_US.UTF-8 locale might not be available. In this case setting above locales results in build errors even though another UTF-8 locale was originally set [1]. We propose a different approach to fix the locale dependent encode/decode problem in common.py utilizing the binary read/write mode [2,3] and decode/encode with explicit UTF-8 encoding arguments [4]. This approach is preferred over the fix in commit d4e5ec877ca because it is (a) locale independent, and (b) does not depend on the en_US.UTF_8 locale to be available. Best, Matthias and Arfrever [1] https://bugs.gentoo.org/657766 [2] https://docs.python.org/3.6/library/stdtypes.html#bytes.decode [3] https://docs.python.org/3.6/library/stdtypes.html#str.encode [4] https://docs.python.org/3/howto/unicode.html
[Qemu-devel] [PATCH 1/2] Partially revert commit d4e5ec877ca
From: Arfrever Frehtes Taifersar Arahesis This commit removes the PYTHON_UTF8 workaround. The problem with setting LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 is that the en_US.UTF-8 locale might not be available. In this case setting above locales results in build errors even though another UTF-8 locale was originally set [1]. The only stable way of fixing the encoding problem is by explicitly annotating encoding/decoding in the python script. [1] https://bugs.gentoo.org/657766 Signed-off-by: Arfrever Frehtes Taifersar Arahesis Signed-off-by: Matthias Maier --- Makefile | 6 ++ tests/Makefile.include | 6 +++--- 2 files changed, 5 insertions(+), 7 deletions(-) diff --git a/Makefile b/Makefile index 6c6664d9a3..9f7fd9e5dc 100644 --- a/Makefile +++ b/Makefile @@ -20,8 +20,6 @@ ifneq ($(wildcard config-host.mak),) all: include config-host.mak -PYTHON_UTF8 = LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 $(PYTHON) - git-submodule-update: .PHONY: git-submodule-update @@ -575,7 +573,7 @@ qga/qapi-generated/qga-qapi-commands.h qga/qapi-generated/qga-qapi-commands.c \ qga/qapi-generated/qga-qapi-doc.texi: \ qga/qapi-generated/qapi-gen-timestamp ; qga/qapi-generated/qapi-gen-timestamp: $(SRC_PATH)/qga/qapi-schema.json $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o qga/qapi-generated -p "qga-" $<, \ "GEN","$(@:%-timestamp=%)") @>$@ @@ -675,7 +673,7 @@ qapi/qapi-introspect.h qapi/qapi-introspect.c \ qapi/qapi-doc.texi: \ qapi-gen-timestamp ; qapi-gen-timestamp: $(qapi-modules) $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o "qapi" -b $<, \ "GEN","$(@:%-timestamp=%)") @>$@ diff --git a/tests/Makefile.include b/tests/Makefile.include index bb08e37b9d..618ed36cbb 100644 --- a/tests/Makefile.include +++ b/tests/Makefile.include @@ -674,13 +674,13 @@ tests/test-qapi-events.c tests/test-qapi-events.h \ tests/test-qapi-introspect.c tests/test-qapi-introspect.h: \ tests/test-qapi-gen-timestamp ; tests/test-qapi-gen-timestamp: $(SRC_PATH)/tests/qapi-schema/qapi-schema-test.json $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o tests -p "test-" $<, \ "GEN","$(@:%-timestamp=%)") @>$@ tests/qapi-schema/doc-good.test.texi: $(SRC_PATH)/tests/qapi-schema/doc-good.json $(qapi-py) - $(call quiet-command,$(PYTHON_UTF8) $(SRC_PATH)/scripts/qapi-gen.py \ + $(call quiet-command,$(PYTHON) $(SRC_PATH)/scripts/qapi-gen.py \ -o tests/qapi-schema -p "doc-good-" $<, \ "GEN","$@") @mv tests/qapi-schema/doc-good-qapi-doc.texi $@ @@ -938,7 +938,7 @@ check-tests/qemu-iotests-quick.sh: tests/qemu-iotests-quick.sh qemu-img$(EXESUF) .PHONY: $(patsubst %, check-%, $(check-qapi-schema-y)) $(patsubst %, check-%, $(check-qapi-schema-y)): check-%.json: $(SRC_PATH)/%.json $(call quiet-command, PYTHONPATH=$(SRC_PATH)/scripts \ - $(PYTHON_UTF8) $(SRC_PATH)/tests/qapi-schema/test-qapi.py \ + $(PYTHON) $(SRC_PATH)/tests/qapi-schema/test-qapi.py \ $^ >$*.test.out 2>$*.test.err; \ echo $$? >$*.test.exit, \ "TEST","$*.out") -- 2.16.4
[Qemu-devel] [PATCH 2/2] qapi: open files in binary mode and use explicit decoding/encoding in common.py
From: Arfrever Frehtes Taifersar Arahesis
This is a different approach to fix the locale dependent encode/decode
problem in common.py utilizing the binary read/write mode [1,2] and
decode/encode with explicit UTF-8 encoding arguments [3].
This approach is preferred over the fix in commit d4e5ec877ca because it
is (a) locale independent, and (b) does not depend on the en_US.UTF_8
locale to be available.
[1] https://docs.python.org/3.6/library/stdtypes.html#bytes.decode
[2] https://docs.python.org/3.6/library/stdtypes.html#str.encode
[3] https://docs.python.org/3/howto/unicode.html
Signed-off-by: Arfrever Frehtes Taifersar Arahesis
Signed-off-by: Matthias Maier
---
scripts/qapi/common.py | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/scripts/qapi/common.py b/scripts/qapi/common.py
index 2462fc0291..8f8ccc65c3 100644
--- a/scripts/qapi/common.py
+++ b/scripts/qapi/common.py
@@ -258,7 +258,7 @@ class QAPISchemaParser(object):
self.fname = fp.name
previously_included.append(os.path.abspath(fp.name))
self.incl_info = incl_info
-self.src = fp.read()
+self.src = fp.read().decode("UTF-8")
if self.src == '' or self.src[-1] != '\n':
self.src += '\n'
self.cursor = 0
@@ -340,7 +340,7 @@ class QAPISchemaParser(object):
return None
try:
-fobj = open(incl_fname, 'r')
+fobj = open(incl_fname, 'rb')
except IOError as e:
raise QAPISemError(info, '%s: %s' % (e.strerror, incl_fname))
return QAPISchemaParser(fobj, previously_included, info)
@@ -1492,7 +1492,7 @@ class QAPISchemaEvent(QAPISchemaEntity):
class QAPISchema(object):
def __init__(self, fname):
self._fname = fname
-parser = QAPISchemaParser(open(fname, 'r'))
+parser = QAPISchemaParser(open(fname, 'rb'))
exprs = check_exprs(parser.exprs)
self.docs = parser.docs
self._entity_list = []
@@ -2006,14 +2006,14 @@ class QAPIGen(object):
if e.errno != errno.EEXIST:
raise
fd = os.open(pathname, os.O_RDWR | os.O_CREAT, 0o666)
-f = os.fdopen(fd, 'r+')
+f = os.fdopen(fd, 'r+b')
text = (self._top(fname) + self._preamble + self._body
+ self._bottom(fname))
-oldtext = f.read(len(text) + 1)
+oldtext = f.read(len(text) + 1).decode("UTF-8")
if text != oldtext:
f.seek(0)
f.truncate(0)
-f.write(text)
+f.write(text.encode("UTF-8"))
f.close()
--
2.16.4
[Qemu-devel] [PATCH 0/2] Fix compilation with python-3 if en_US.UTF-8 is unavailable
Hi, This patch series, - removes the PYTHON_UTF8 workaround introduced in d4e5ec877ca - adds a different workaround that avoids the locale problem altogether by opening files in binary read/write mode and setting encoding/decoding (in utf-8) explicitly The problem with setting LC_ALL= LANG=C LC_CTYPE=en_US.UTF-8 is that the en_US.UTF-8 locale might not be available. In this case setting above locales results in build errors even though another UTF-8 locale was originally set [1]. We propose a different approach to fix the locale dependent encode/decode problem in common.py utilizing the binary read/write mode [2,3] and decode/encode with explicit UTF-8 encoding arguments [4]. This approach is preferred over the fix in commit d4e5ec877ca because it is (a) locale independent, and (b) does not depend on the en_US.UTF_8 locale to be available. Best, Matthias and Arfrever [1] https://bugs.gentoo.org/657766 [2] https://docs.python.org/3.6/library/stdtypes.html#bytes.decode [3] https://docs.python.org/3.6/library/stdtypes.html#str.encode [4] https://docs.python.org/3/howto/unicode.html
[Qemu-devel] [Bug 1749393] Re: sbrk() not working under qemu-user with a PIE-compiled binary?
qemu patch proposed at http://lists.nongnu.org/archive/html/qemu- devel/2018-03/msg04700.html -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1749393 Title: sbrk() not working under qemu-user with a PIE-compiled binary? Status in QEMU: New Status in qemu package in Ubuntu: Confirmed Bug description: In Debian unstable, we recently switched bash to be a PIE-compiled binary (for hardening). Unfortunately this resulted in bash being broken when run under qemu-user (for all target architectures, host being amd64 for me). $ sudo chroot /srv/chroots/sid-i386/ qemu-i386-static /bin/bash bash: xmalloc: .././shell.c:1709: cannot allocate 10 bytes (0 bytes allocated) bash has its own malloc implementation based on sbrk(): https://git.savannah.gnu.org/cgit/bash.git/tree/lib/malloc/malloc.c When we disable this internal implementation and rely on glibc's malloc, then everything is fine. But it might be that glibc has a fallback when sbrk() is not working properly and it might hide the underlying problem in qemu-user. This issue has also been reported to the bash upstream author and he suggested that the issue might be in qemu-user so I'm opening a ticket here. Here's the discussion with the bash upstream author: https://lists.gnu.org/archive/html/bug-bash/2018-02/threads.html#00080 You can find the problematic bash binary in that .deb file: http://snapshot.debian.org/archive/debian/20180206T154716Z/pool/main/b/bash/bash_4.4.18-1_i386.deb The version of qemu I have been using is 2.11 (Debian package qemu- user-static version 1:2.11+dfsg-1) but I have had reports that the problem is reproducible with older versions (back to 2.8 at least). Here are the related Debian bug reports: https://bugs.debian.org/889869 https://bugs.debian.org/865599 It's worth noting that bash used to have this problem (when compiled as a PIE binary) even when run directly but then something got fixed in the kernel and now the problem only appears when run under qemu-user: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518483 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1749393/+subscriptions
[Qemu-devel] [Bug 1749393] Re: sbrk() not working under qemu-user with a PIE-compiled binary?
** Also affects: qemu (Ubuntu) Importance: Undecided Status: New -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1749393 Title: sbrk() not working under qemu-user with a PIE-compiled binary? Status in QEMU: New Status in qemu package in Ubuntu: New Bug description: In Debian unstable, we recently switched bash to be a PIE-compiled binary (for hardening). Unfortunately this resulted in bash being broken when run under qemu-user (for all target architectures, host being amd64 for me). $ sudo chroot /srv/chroots/sid-i386/ qemu-i386-static /bin/bash bash: xmalloc: .././shell.c:1709: cannot allocate 10 bytes (0 bytes allocated) bash has its own malloc implementation based on sbrk(): https://git.savannah.gnu.org/cgit/bash.git/tree/lib/malloc/malloc.c When we disable this internal implementation and rely on glibc's malloc, then everything is fine. But it might be that glibc has a fallback when sbrk() is not working properly and it might hide the underlying problem in qemu-user. This issue has also been reported to the bash upstream author and he suggested that the issue might be in qemu-user so I'm opening a ticket here. Here's the discussion with the bash upstream author: https://lists.gnu.org/archive/html/bug-bash/2018-02/threads.html#00080 You can find the problematic bash binary in that .deb file: http://snapshot.debian.org/archive/debian/20180206T154716Z/pool/main/b/bash/bash_4.4.18-1_i386.deb The version of qemu I have been using is 2.11 (Debian package qemu- user-static version 1:2.11+dfsg-1) but I have had reports that the problem is reproducible with older versions (back to 2.8 at least). Here are the related Debian bug reports: https://bugs.debian.org/889869 https://bugs.debian.org/865599 It's worth noting that bash used to have this problem (when compiled as a PIE binary) even when run directly but then something got fixed in the kernel and now the problem only appears when run under qemu-user: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1518483 To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1749393/+subscriptions
[Qemu-devel] [Bug 1581936] Re: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1)
** Also affects: qemu (Ubuntu) Importance: Undecided Status: New ** Tags added: regression-update -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1581936 Title: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1) Status in QEMU: Fix Released Status in qemu package in Ubuntu: New Bug description: Hi, As already posted on the QEMU devel list [1] I stumbled upon a problem with QEMU in version 2.5.1.1 and 2.6.0. the VM shows Windows loading files for the installation, then the "Starting Windows" screen appears here it hangs and never continues. Changing the "-vga" option to cirrus solves this, the installation can proceed and finish. When changing back to std (or also qxl, vmware) the installed VM also hangs on the "Starting Windows" screen while qemu showing a little but no excessive load. This phenomena appears also with QEMU 2.6.0 but not with 2.6.0-rc4, a git bisect shows fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 (vga: make sure vga register setup for vbe stays intact (CVE-2016-3712)) as the culprit for this regression, as its a fix for a DoS its not an option to just revert it, I guess. The bisect log is: git bisect start # bad: [bfc766d38e1fae5767d43845c15c79ac8fa6d6af] Update version for v2.6.0 release git bisect bad bfc766d38e1fae5767d43845c15c79ac8fa6d6af # good: [975eb6a547f809608ccb08c221552f11af25] Update version for v2.6.0-rc4 release git bisect good 975eb6a547f809608ccb08c221552f11af25 # good: [2068192dcccd8a80dddfcc8df6164cf9c26e0fc4] vga: update vga register setup on vbe changes git bisect good 2068192dcccd8a80dddfcc8df6164cf9c26e0fc4 # bad: [53db932604dfa7bb9241d132e0173894cf54261c] Merge remote-tracking branch 'remotes/kraxel/tags/pull-vga-20160509-1' into staging git bisect bad 53db932604dfa7bb9241d132e0173894cf54261c # bad: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). git bisect bad fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 # first bad commit: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). I could reproduce that with QEMU 2.5.1 and QEMU 2.6 on a Debian derivate (Promox VE) with 4.4 Kernel and also with QEMU 2.6 on an Arch Linux System with a 4.5 Kernel, so it should not be host distro depended. Both machines have Intel x86_64 processors. The problem should be reproducible with said Versions or a build from git including the above mentioned commit (fd3c136) by starting a VM with an Windows 7 ISO, e.g.: Freezing installation (as vga defaults to std I marked it as optional): ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 [-vga (std|qxl|vmware)] Working installation: ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 -vga cirrus If someone has already an installed Windows 7 VM this behaviour should be also observable when trying to start it with the new versions of QEMU. Noteworthy may be that Windows 10 is working, I do not had time to get other Windows versions and test them, I'll do that as soon as possible. Various Linux system also seems do work fine, at least I did not ran into an issue there yet. I also tried testing with SeaBIOS and OVMF as firmware, as initially I had no idea what broke, both lead to the same result - without the CVE-2016-3712 fix they both work, with not. Further, KVM enabled and disabled does not make any difference. [1] http://lists.nongnu.org/archive/html/qemu-devel/2016-05/msg02416.html To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1581936/+subscriptions
[Qemu-devel] [Bug 1581936] Re: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1)
** Patch added: "Proposed fix for xenial" https://bugs.launchpad.net/qemu/+bug/1581936/+attachment/4864875/+files/qemu-xenial.debdiff -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1581936 Title: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1) Status in QEMU: Fix Released Bug description: Hi, As already posted on the QEMU devel list [1] I stumbled upon a problem with QEMU in version 2.5.1.1 and 2.6.0. the VM shows Windows loading files for the installation, then the "Starting Windows" screen appears here it hangs and never continues. Changing the "-vga" option to cirrus solves this, the installation can proceed and finish. When changing back to std (or also qxl, vmware) the installed VM also hangs on the "Starting Windows" screen while qemu showing a little but no excessive load. This phenomena appears also with QEMU 2.6.0 but not with 2.6.0-rc4, a git bisect shows fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 (vga: make sure vga register setup for vbe stays intact (CVE-2016-3712)) as the culprit for this regression, as its a fix for a DoS its not an option to just revert it, I guess. The bisect log is: git bisect start # bad: [bfc766d38e1fae5767d43845c15c79ac8fa6d6af] Update version for v2.6.0 release git bisect bad bfc766d38e1fae5767d43845c15c79ac8fa6d6af # good: [975eb6a547f809608ccb08c221552f11af25] Update version for v2.6.0-rc4 release git bisect good 975eb6a547f809608ccb08c221552f11af25 # good: [2068192dcccd8a80dddfcc8df6164cf9c26e0fc4] vga: update vga register setup on vbe changes git bisect good 2068192dcccd8a80dddfcc8df6164cf9c26e0fc4 # bad: [53db932604dfa7bb9241d132e0173894cf54261c] Merge remote-tracking branch 'remotes/kraxel/tags/pull-vga-20160509-1' into staging git bisect bad 53db932604dfa7bb9241d132e0173894cf54261c # bad: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). git bisect bad fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 # first bad commit: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). I could reproduce that with QEMU 2.5.1 and QEMU 2.6 on a Debian derivate (Promox VE) with 4.4 Kernel and also with QEMU 2.6 on an Arch Linux System with a 4.5 Kernel, so it should not be host distro depended. Both machines have Intel x86_64 processors. The problem should be reproducible with said Versions or a build from git including the above mentioned commit (fd3c136) by starting a VM with an Windows 7 ISO, e.g.: Freezing installation (as vga defaults to std I marked it as optional): ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 [-vga (std|qxl|vmware)] Working installation: ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 -vga cirrus If someone has already an installed Windows 7 VM this behaviour should be also observable when trying to start it with the new versions of QEMU. Noteworthy may be that Windows 10 is working, I do not had time to get other Windows versions and test them, I'll do that as soon as possible. Various Linux system also seems do work fine, at least I did not ran into an issue there yet. I also tried testing with SeaBIOS and OVMF as firmware, as initially I had no idea what broke, both lead to the same result - without the CVE-2016-3712 fix they both work, with not. Further, KVM enabled and disabled does not make any difference. [1] http://lists.nongnu.org/archive/html/qemu-devel/2016-05/msg02416.html To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1581936/+subscriptions
[Qemu-devel] [Bug 1581936] Re: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1)
** Patch added: "Proposed fix for trusty" https://bugs.launchpad.net/qemu/+bug/1581936/+attachment/4864874/+files/qemu-trusty.debdiff -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1581936 Title: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1) Status in QEMU: Fix Released Bug description: Hi, As already posted on the QEMU devel list [1] I stumbled upon a problem with QEMU in version 2.5.1.1 and 2.6.0. the VM shows Windows loading files for the installation, then the "Starting Windows" screen appears here it hangs and never continues. Changing the "-vga" option to cirrus solves this, the installation can proceed and finish. When changing back to std (or also qxl, vmware) the installed VM also hangs on the "Starting Windows" screen while qemu showing a little but no excessive load. This phenomena appears also with QEMU 2.6.0 but not with 2.6.0-rc4, a git bisect shows fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 (vga: make sure vga register setup for vbe stays intact (CVE-2016-3712)) as the culprit for this regression, as its a fix for a DoS its not an option to just revert it, I guess. The bisect log is: git bisect start # bad: [bfc766d38e1fae5767d43845c15c79ac8fa6d6af] Update version for v2.6.0 release git bisect bad bfc766d38e1fae5767d43845c15c79ac8fa6d6af # good: [975eb6a547f809608ccb08c221552f11af25] Update version for v2.6.0-rc4 release git bisect good 975eb6a547f809608ccb08c221552f11af25 # good: [2068192dcccd8a80dddfcc8df6164cf9c26e0fc4] vga: update vga register setup on vbe changes git bisect good 2068192dcccd8a80dddfcc8df6164cf9c26e0fc4 # bad: [53db932604dfa7bb9241d132e0173894cf54261c] Merge remote-tracking branch 'remotes/kraxel/tags/pull-vga-20160509-1' into staging git bisect bad 53db932604dfa7bb9241d132e0173894cf54261c # bad: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). git bisect bad fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 # first bad commit: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). I could reproduce that with QEMU 2.5.1 and QEMU 2.6 on a Debian derivate (Promox VE) with 4.4 Kernel and also with QEMU 2.6 on an Arch Linux System with a 4.5 Kernel, so it should not be host distro depended. Both machines have Intel x86_64 processors. The problem should be reproducible with said Versions or a build from git including the above mentioned commit (fd3c136) by starting a VM with an Windows 7 ISO, e.g.: Freezing installation (as vga defaults to std I marked it as optional): ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 [-vga (std|qxl|vmware)] Working installation: ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 -vga cirrus If someone has already an installed Windows 7 VM this behaviour should be also observable when trying to start it with the new versions of QEMU. Noteworthy may be that Windows 10 is working, I do not had time to get other Windows versions and test them, I'll do that as soon as possible. Various Linux system also seems do work fine, at least I did not ran into an issue there yet. I also tried testing with SeaBIOS and OVMF as firmware, as initially I had no idea what broke, both lead to the same result - without the CVE-2016-3712 fix they both work, with not. Further, KVM enabled and disabled does not make any difference. [1] http://lists.nongnu.org/archive/html/qemu-devel/2016-05/msg02416.html To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1581936/+subscriptions
[Qemu-devel] [Bug 1581936] Re: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1)
Please let me know if there is anything I can do to help get these patches accepted for trusty/xenial. -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1581936 Title: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1) Status in QEMU: Fix Released Bug description: Hi, As already posted on the QEMU devel list [1] I stumbled upon a problem with QEMU in version 2.5.1.1 and 2.6.0. the VM shows Windows loading files for the installation, then the "Starting Windows" screen appears here it hangs and never continues. Changing the "-vga" option to cirrus solves this, the installation can proceed and finish. When changing back to std (or also qxl, vmware) the installed VM also hangs on the "Starting Windows" screen while qemu showing a little but no excessive load. This phenomena appears also with QEMU 2.6.0 but not with 2.6.0-rc4, a git bisect shows fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 (vga: make sure vga register setup for vbe stays intact (CVE-2016-3712)) as the culprit for this regression, as its a fix for a DoS its not an option to just revert it, I guess. The bisect log is: git bisect start # bad: [bfc766d38e1fae5767d43845c15c79ac8fa6d6af] Update version for v2.6.0 release git bisect bad bfc766d38e1fae5767d43845c15c79ac8fa6d6af # good: [975eb6a547f809608ccb08c221552f11af25] Update version for v2.6.0-rc4 release git bisect good 975eb6a547f809608ccb08c221552f11af25 # good: [2068192dcccd8a80dddfcc8df6164cf9c26e0fc4] vga: update vga register setup on vbe changes git bisect good 2068192dcccd8a80dddfcc8df6164cf9c26e0fc4 # bad: [53db932604dfa7bb9241d132e0173894cf54261c] Merge remote-tracking branch 'remotes/kraxel/tags/pull-vga-20160509-1' into staging git bisect bad 53db932604dfa7bb9241d132e0173894cf54261c # bad: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). git bisect bad fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 # first bad commit: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). I could reproduce that with QEMU 2.5.1 and QEMU 2.6 on a Debian derivate (Promox VE) with 4.4 Kernel and also with QEMU 2.6 on an Arch Linux System with a 4.5 Kernel, so it should not be host distro depended. Both machines have Intel x86_64 processors. The problem should be reproducible with said Versions or a build from git including the above mentioned commit (fd3c136) by starting a VM with an Windows 7 ISO, e.g.: Freezing installation (as vga defaults to std I marked it as optional): ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 [-vga (std|qxl|vmware)] Working installation: ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 -vga cirrus If someone has already an installed Windows 7 VM this behaviour should be also observable when trying to start it with the new versions of QEMU. Noteworthy may be that Windows 10 is working, I do not had time to get other Windows versions and test them, I'll do that as soon as possible. Various Linux system also seems do work fine, at least I did not ran into an issue there yet. I also tried testing with SeaBIOS and OVMF as firmware, as initially I had no idea what broke, both lead to the same result - without the CVE-2016-3712 fix they both work, with not. Further, KVM enabled and disabled does not make any difference. [1] http://lists.nongnu.org/archive/html/qemu-devel/2016-05/msg02416.html To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1581936/+subscriptions
[Qemu-devel] [Bug 1581936] Re: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1)
... and trusty is affected, too. Would it help if I provide patches for trusty/xenial? I'd probably also need to update the description for SRU? -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1581936 Title: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1) Status in QEMU: Fix Released Bug description: Hi, As already posted on the QEMU devel list [1] I stumbled upon a problem with QEMU in version 2.5.1.1 and 2.6.0. the VM shows Windows loading files for the installation, then the "Starting Windows" screen appears here it hangs and never continues. Changing the "-vga" option to cirrus solves this, the installation can proceed and finish. When changing back to std (or also qxl, vmware) the installed VM also hangs on the "Starting Windows" screen while qemu showing a little but no excessive load. This phenomena appears also with QEMU 2.6.0 but not with 2.6.0-rc4, a git bisect shows fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 (vga: make sure vga register setup for vbe stays intact (CVE-2016-3712)) as the culprit for this regression, as its a fix for a DoS its not an option to just revert it, I guess. The bisect log is: git bisect start # bad: [bfc766d38e1fae5767d43845c15c79ac8fa6d6af] Update version for v2.6.0 release git bisect bad bfc766d38e1fae5767d43845c15c79ac8fa6d6af # good: [975eb6a547f809608ccb08c221552f11af25] Update version for v2.6.0-rc4 release git bisect good 975eb6a547f809608ccb08c221552f11af25 # good: [2068192dcccd8a80dddfcc8df6164cf9c26e0fc4] vga: update vga register setup on vbe changes git bisect good 2068192dcccd8a80dddfcc8df6164cf9c26e0fc4 # bad: [53db932604dfa7bb9241d132e0173894cf54261c] Merge remote-tracking branch 'remotes/kraxel/tags/pull-vga-20160509-1' into staging git bisect bad 53db932604dfa7bb9241d132e0173894cf54261c # bad: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). git bisect bad fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 # first bad commit: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). I could reproduce that with QEMU 2.5.1 and QEMU 2.6 on a Debian derivate (Promox VE) with 4.4 Kernel and also with QEMU 2.6 on an Arch Linux System with a 4.5 Kernel, so it should not be host distro depended. Both machines have Intel x86_64 processors. The problem should be reproducible with said Versions or a build from git including the above mentioned commit (fd3c136) by starting a VM with an Windows 7 ISO, e.g.: Freezing installation (as vga defaults to std I marked it as optional): ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 [-vga (std|qxl|vmware)] Working installation: ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 -vga cirrus If someone has already an installed Windows 7 VM this behaviour should be also observable when trying to start it with the new versions of QEMU. Noteworthy may be that Windows 10 is working, I do not had time to get other Windows versions and test them, I'll do that as soon as possible. Various Linux system also seems do work fine, at least I did not ran into an issue there yet. I also tried testing with SeaBIOS and OVMF as firmware, as initially I had no idea what broke, both lead to the same result - without the CVE-2016-3712 fix they both work, with not. Further, KVM enabled and disabled does not make any difference. [1] http://lists.nongnu.org/archive/html/qemu-devel/2016-05/msg02416.html To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1581936/+subscriptions
[Qemu-devel] [Bug 1581936] Re: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1)
Will the fix be backported? Right now, this is a regression in xenial (caused by the security update in 1:2.5+dfsg-5ubuntu10.6). -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1581936 Title: Frozen Windows 7 VMs with VGA CVE-2016-3712 fix (2.6.0 and 2.5.1.1) Status in QEMU: Fix Released Bug description: Hi, As already posted on the QEMU devel list [1] I stumbled upon a problem with QEMU in version 2.5.1.1 and 2.6.0. the VM shows Windows loading files for the installation, then the "Starting Windows" screen appears here it hangs and never continues. Changing the "-vga" option to cirrus solves this, the installation can proceed and finish. When changing back to std (or also qxl, vmware) the installed VM also hangs on the "Starting Windows" screen while qemu showing a little but no excessive load. This phenomena appears also with QEMU 2.6.0 but not with 2.6.0-rc4, a git bisect shows fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 (vga: make sure vga register setup for vbe stays intact (CVE-2016-3712)) as the culprit for this regression, as its a fix for a DoS its not an option to just revert it, I guess. The bisect log is: git bisect start # bad: [bfc766d38e1fae5767d43845c15c79ac8fa6d6af] Update version for v2.6.0 release git bisect bad bfc766d38e1fae5767d43845c15c79ac8fa6d6af # good: [975eb6a547f809608ccb08c221552f11af25] Update version for v2.6.0-rc4 release git bisect good 975eb6a547f809608ccb08c221552f11af25 # good: [2068192dcccd8a80dddfcc8df6164cf9c26e0fc4] vga: update vga register setup on vbe changes git bisect good 2068192dcccd8a80dddfcc8df6164cf9c26e0fc4 # bad: [53db932604dfa7bb9241d132e0173894cf54261c] Merge remote-tracking branch 'remotes/kraxel/tags/pull-vga-20160509-1' into staging git bisect bad 53db932604dfa7bb9241d132e0173894cf54261c # bad: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). git bisect bad fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7 # first bad commit: [fd3c136b3e1482cd0ec7285d6bc2a3e6a62c38d7] vga: make sure vga register setup for vbe stays intact (CVE-2016-3712). I could reproduce that with QEMU 2.5.1 and QEMU 2.6 on a Debian derivate (Promox VE) with 4.4 Kernel and also with QEMU 2.6 on an Arch Linux System with a 4.5 Kernel, so it should not be host distro depended. Both machines have Intel x86_64 processors. The problem should be reproducible with said Versions or a build from git including the above mentioned commit (fd3c136) by starting a VM with an Windows 7 ISO, e.g.: Freezing installation (as vga defaults to std I marked it as optional): ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 [-vga (std|qxl|vmware)] Working installation: ./x86_64-softmmu/qemu-system-x86_64 -boot d -cdrom win7.iso -m 1024 -vga cirrus If someone has already an installed Windows 7 VM this behaviour should be also observable when trying to start it with the new versions of QEMU. Noteworthy may be that Windows 10 is working, I do not had time to get other Windows versions and test them, I'll do that as soon as possible. Various Linux system also seems do work fine, at least I did not ran into an issue there yet. I also tried testing with SeaBIOS and OVMF as firmware, as initially I had no idea what broke, both lead to the same result - without the CVE-2016-3712 fix they both work, with not. Further, KVM enabled and disabled does not make any difference. [1] http://lists.nongnu.org/archive/html/qemu-devel/2016-05/msg02416.html To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1581936/+subscriptions
[Qemu-devel] Error on OpenBSD/macppc
(qemu) unknown keycodes `macintosh_aliases(qwerty)', please report to qemu-devel@nongnu.org Which i am doing with this email :)
Re: [Qemu-devel] [PATCH v2] i386/acpi: add _HID to processor objects
On 10/23/2015 03:08 PM, Igor Mammedov wrote:
> On Thu, 22 Oct 2015 22:03:24 +0200
> Matthias Lange <matthias.la...@kernkonzept.com> wrote:
>
>> This patch appends "ACPI0007" as the HID to each processor object.
>>
>> Until commit 20843d processor objects used to have a _HID. According
>> to the ACPI spec this is not required but removing it breaks systems
> Pls answer Michael's question about motivation of this patch.
> i.e. what guests it exactly breaks?
It broke the L4Re OS (specifically the mechanism to match ACPI drivers
to ACPI devices).
However, you can ignore this patch. I decided to change L4Re instead.
Best,
Matthias.
>> which relied on the HID. As it does no harm it is safe to add _HID
>> to processor objects and restore the old behaviour.
>>
>> Signed-off-by: Matthias Lange <matthias.la...@kernkonzept.com>
>> ---
>> hw/i386/acpi-build.c | 3 +++
>> 1 file changed, 3 insertions(+)
>>
>> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
>> index 95e0c65..314cd0b 100644
>> --- a/hw/i386/acpi-build.c
>> +++ b/hw/i386/acpi-build.c
>> @@ -1153,6 +1153,9 @@ build_ssdt(GArray *table_data, GArray *linker,
>> for (i = 0; i < acpi_cpus; i++) {
>> dev = aml_processor(i, 0, 0, "CP%.02X", i);
>>
>> +/* for processor objects a _HID is not strictly required,
>> however it
>> + * does no harm and preserves compatibility with other BIOSes */
>> +aml_append(dev, aml_name_decl("_HID", aml_string("ACPI0007")));
> Spec doesn't tell anything about using ACPI0007 with Processor statement,
> it's only mentioned in context of Device statement.
>
>> method = aml_method("_MAT", 0);
>> aml_append(method, aml_return(aml_call1("CPMA", aml_int(i;
>> aml_append(dev, method);
[Qemu-devel] [PATCH v2] i386/acpi: add _HID to processor objects
This patch appends "ACPI0007" as the HID to each processor object.
Until commit 20843d processor objects used to have a _HID. According
to the ACPI spec this is not required but removing it breaks systems
which relied on the HID. As it does no harm it is safe to add _HID
to processor objects and restore the old behaviour.
Signed-off-by: Matthias Lange <matthias.la...@kernkonzept.com>
---
hw/i386/acpi-build.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 95e0c65..314cd0b 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1153,6 +1153,9 @@ build_ssdt(GArray *table_data, GArray *linker,
for (i = 0; i < acpi_cpus; i++) {
dev = aml_processor(i, 0, 0, "CP%.02X", i);
+/* for processor objects a _HID is not strictly required, however
it
+ * does no harm and preserves compatibility with other BIOSes */
+aml_append(dev, aml_name_decl("_HID", aml_string("ACPI0007")));
method = aml_method("_MAT", 0);
aml_append(method, aml_return(aml_call1("CPMA", aml_int(i;
aml_append(dev, method);
--
1.9.1
Re: [Qemu-devel] [PATCH] i386/acpi: add _HID to processor nodes
On 10/22/2015 03:02 PM, Michael S. Tsirkin wrote:
> On Thu, Oct 22, 2015 at 02:42:05PM +0200, Matthias Lange wrote:
>> On 10/22/2015 01:48 PM, Michael S. Tsirkin wrote:
>>> On Thu, Oct 22, 2015 at 12:38:11PM +0200, Matthias Lange wrote:
>>>> Processor nodes created via the acpi/aml framework currently don't
>>>> feature a _HID string. This patch appends "ACPI0007" as the _HID
>>>> string to each processor node.
>>>>
>>>> Signed-off-by: Matthias Lange <matthias.la...@kernkonzept.com>
>>>
>>> Indeed, it does that. But why?
>>>
>>> ACPI0007 Processor Device. This device provides an alternative to
>>> declaring processors using the
>>> Processor ASL statement.
>>>
>>> And “Declaring Processors” says
>>>
>>>
>>> Each processor in the system must be declared in the ACPI namespace in
>>> either the \_SB or \_PR
>>> scope but not both. Declaration of processors in the \_PR scope is
>>> required for platforms desiring
>>> compatibility with ACPI 1.0-based OSPM implementations. Processors are
>>> declared either via the
>>> ASL Processor statement or the ASL Device statement. A Processor
>>> definition declares a processor
>>> object that provides processor configuration information and points to
>>> the processor register block
>>> (P_BLK). A Device definition for a processor is declared using the
>>> ACPI0007 hardware identifier
>>> (HID). In this case, processor configuration information is provided
>>> exclusively by objects in the
>>> processor device’s object list.
>>
>> Thanks for pointing me to the right section.
>>
>>> In other words, Processor directive does not need a HID.
>>> In fact, ACPI 1.0 didn't include the ACPI0007 HID at all.
>>
>> Right. But, does Qemu strive for ACPI 1.0 compatibility?
>
> Where possible - it's handy for old guests.
I understand. But then, why are processors declared under the \_SB scope
and not under \_PR?
>> And e.g.
>> seabios also defines processors using the ALS processor statement
>> including the HID.
>
> I don't see it there:
> [mst@robin seabios]$ git grep ACI0007
> [mst@robin seabios]$
It's a typo in your command. Have a look into
seabios/src/fw/ssdt-proc.dsl
Matthias.
>>> Please include this info in both the commit log and a code comment.
>>
>> Will do.
>>
> Not the ACPI spec info. I have that. The answer to my question - the
> actual motivation for the patch.
>
>>>> ---
>>>> hw/i386/acpi-build.c | 1 +
>>>> 1 file changed, 1 insertion(+)
>>>>
>>>> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
>>>> index 95e0c65..95f7bf9 100644
>>>> --- a/hw/i386/acpi-build.c
>>>> +++ b/hw/i386/acpi-build.c
>>>> @@ -1153,6 +1153,7 @@ build_ssdt(GArray *table_data, GArray *linker,
>>>> for (i = 0; i < acpi_cpus; i++) {
>>>> dev = aml_processor(i, 0, 0, "CP%.02X", i);
>>>>
>>>> +aml_append(dev, aml_name_decl("_HID",
>>>> aml_string("ACPI0007")));
>>>> method = aml_method("_MAT", 0);
>>>> aml_append(method, aml_return(aml_call1("CPMA", aml_int(i;
>>>> aml_append(dev, method);
>>>> --
>>>> 1.9.1
>>
>>
>> --
>> Matthias Lange, matthias.la...@kernkonzept.com, +49 - 351 - 41 88 86 14
>>
>> Kernkonzept GmbH. Sitz: Dresden. Amtsgericht Dresden, HRB 31129.
>> Geschäftsführer: Dr.-Ing. Michael Hohmuth
--
Matthias Lange, matthias.la...@kernkonzept.com, +49 - 351 - 41 88 86 14
Kernkonzept GmbH. Sitz: Dresden. Amtsgericht Dresden, HRB 31129.
Geschäftsführer: Dr.-Ing. Michael Hohmuth
Re: [Qemu-devel] [PATCH] i386/acpi: add _HID to processor nodes
On 10/22/2015 01:48 PM, Michael S. Tsirkin wrote:
> On Thu, Oct 22, 2015 at 12:38:11PM +0200, Matthias Lange wrote:
>> Processor nodes created via the acpi/aml framework currently don't
>> feature a _HID string. This patch appends "ACPI0007" as the _HID
>> string to each processor node.
>>
>> Signed-off-by: Matthias Lange <matthias.la...@kernkonzept.com>
>
> Indeed, it does that. But why?
>
> ACPI0007 Processor Device. This device provides an alternative to
> declaring processors using the
> Processor ASL statement.
>
> And “Declaring Processors” says
>
>
> Each processor in the system must be declared in the ACPI namespace in
> either the \_SB or \_PR
> scope but not both. Declaration of processors in the \_PR scope is
> required for platforms desiring
> compatibility with ACPI 1.0-based OSPM implementations. Processors are
> declared either via the
> ASL Processor statement or the ASL Device statement. A Processor
> definition declares a processor
> object that provides processor configuration information and points to
> the processor register block
> (P_BLK). A Device definition for a processor is declared using the
> ACPI0007 hardware identifier
> (HID). In this case, processor configuration information is provided
> exclusively by objects in the
> processor device’s object list.
Thanks for pointing me to the right section.
> In other words, Processor directive does not need a HID.
> In fact, ACPI 1.0 didn't include the ACPI0007 HID at all.
Right. But, does Qemu strive for ACPI 1.0 compatibility? And e.g.
seabios also defines processors using the ALS processor statement
including the HID.
> Please include this info in both the commit log and a code comment.
Will do.
Matthias.
>> ---
>> hw/i386/acpi-build.c | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
>> index 95e0c65..95f7bf9 100644
>> --- a/hw/i386/acpi-build.c
>> +++ b/hw/i386/acpi-build.c
>> @@ -1153,6 +1153,7 @@ build_ssdt(GArray *table_data, GArray *linker,
>> for (i = 0; i < acpi_cpus; i++) {
>> dev = aml_processor(i, 0, 0, "CP%.02X", i);
>>
>> +aml_append(dev, aml_name_decl("_HID", aml_string("ACPI0007")));
>> method = aml_method("_MAT", 0);
>> aml_append(method, aml_return(aml_call1("CPMA", aml_int(i;
>> aml_append(dev, method);
>> --
>> 1.9.1
--
Matthias Lange, matthias.la...@kernkonzept.com, +49 - 351 - 41 88 86 14
Kernkonzept GmbH. Sitz: Dresden. Amtsgericht Dresden, HRB 31129.
Geschäftsführer: Dr.-Ing. Michael Hohmuth
[Qemu-devel] [PATCH] i386/acpi: add _HID to processor nodes
Processor nodes created via the acpi/aml framework currently don't
feature a _HID string. This patch appends "ACPI0007" as the _HID
string to each processor node.
Signed-off-by: Matthias Lange <matthias.la...@kernkonzept.com>
---
hw/i386/acpi-build.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/hw/i386/acpi-build.c b/hw/i386/acpi-build.c
index 95e0c65..95f7bf9 100644
--- a/hw/i386/acpi-build.c
+++ b/hw/i386/acpi-build.c
@@ -1153,6 +1153,7 @@ build_ssdt(GArray *table_data, GArray *linker,
for (i = 0; i < acpi_cpus; i++) {
dev = aml_processor(i, 0, 0, "CP%.02X", i);
+aml_append(dev, aml_name_decl("_HID", aml_string("ACPI0007")));
method = aml_method("_MAT", 0);
aml_append(method, aml_return(aml_call1("CPMA", aml_int(i;
aml_append(dev, method);
--
1.9.1
Re: [Qemu-devel] [PATCH v2 0/3] Make thread pool implementation modular
2013/11/11 Stefan Hajnoczi stefa...@redhat.com: On Mon, Nov 11, 2013 at 11:00:45AM +0100, Matthias Brugger wrote: 2013/11/5 Stefan Hajnoczi stefa...@gmail.com: I'd also like to see the thread pool implementation you wish to add before we add a layer of indirection which has no users yet. Fair enough, I will evaluate if it will make more sense to implement a new AIO infrastructure instead to try reuse the thread-pool. Actually my implementation will differ in the way, that we will have several workerthreads with everyone of them having its own queue. The requests will be distributed between them depending on an identifier. The request function which the worker_thread call will be the same as using aio=threads, so I'm not quite sure which will be the way to go. Any opinions and hints, like the one you gave are highly appreciated. If I understand the slides you linked to correctly, the guest will pass an identifier with each request. The host has worker threads allowing each stream of requests to be serviced independently. The idea is to associate guest processes with unique identifiers. The guest I/O scheduler is supposed to submit requests in a way that meets certain policies (e.g. fairness between processes, deadlines, etc). Why is it necessary to push this task down into the host? I don't understand the advantage of this approach except that maybe it works around certain misconfigurations, I/O scheduler quirks, or plain old bugs - all of which should be investigated and fixed at the source instead of adding another layer of code to mask them. It is about I/O scheduling. CFQ the state of the art I/O scheduler merges adjacent requests from the same PID before dispatching them to the disk. If we can distinguish between the different threads of a virtual machine that read/write a file, the I/O scheduler in the host can merge requests in an effective way for sequential access. Qemu fails in this, because of its architecture. Apart that at the moment there is no way to distinguish the guest threads from each other (I'm working on some kernel patches), Qemu has one big queue from which several workerthreads grab requests and dispatch them to the disk. Even if you have one large read from just one thread in the guest, the I/O scheduler in the host will get the requests from different PIDs (= workerthreads) and won't be able to merge them. In former versions, there was some work done to merge requests in Qemu, but I don't think they were very useful, because you don't know how the layout of the image file looks like on the physical disk. Anyway I think this code parts have been removed. The only layer where you really know how the blocks of the virtual disk image are distributed over the disk is the block layer of the host. So you have to do the block request merging there. With the new architecture this would come for free, as you can map every thread from a guest to one workerthread of Qemu. Matthias Stefan -- motzblog.wordpress.com
Re: [Qemu-devel] [PATCH v2 0/3] Make thread pool implementation modular
2013/11/5 Stefan Hajnoczi stefa...@gmail.com: On Mon, Nov 04, 2013 at 11:28:41AM +0100, Matthias Brugger wrote: v2: - fix issues found by checkpatch.pl - change the descritpion of patch 3 This patch series makes the thread pool implementation modular. This allows each drive to use a special implementation. The patch series prepares qemu to be able to include thread pools different to the one actually implemented. It will allow to implement approaches like paravirtualized block requests [1]. [1] http://www.linux-kvm.org/wiki/images/5/53/2012-forum-Brugger-lightningtalk.pdf -drive aio=threads|native already distinguishes between different AIO implementations. IMO -drive aio= is the logical interface if you want to add a new AIO mechanism. Good point. I'd also like to see the thread pool implementation you wish to add before we add a layer of indirection which has no users yet. Fair enough, I will evaluate if it will make more sense to implement a new AIO infrastructure instead to try reuse the thread-pool. Actually my implementation will differ in the way, that we will have several workerthreads with everyone of them having its own queue. The requests will be distributed between them depending on an identifier. The request function which the worker_thread call will be the same as using aio=threads, so I'm not quite sure which will be the way to go. Any opinions and hints, like the one you gave are highly appreciated. -- motzblog.wordpress.com
[Qemu-devel] [PATCH v2 0/3] Make thread pool implementation modular
v2: - fix issues found by checkpatch.pl - change the descritpion of patch 3 This patch series makes the thread pool implementation modular. This allows each drive to use a special implementation. The patch series prepares qemu to be able to include thread pools different to the one actually implemented. It will allow to implement approaches like paravirtualized block requests [1]. [1] http://www.linux-kvm.org/wiki/images/5/53/2012-forum-Brugger-lightningtalk.pdf Matthias Brugger (3): Make thread pool implementation modular Block layer uses modular thread pool Add workerthreads configuration option async.c | 4 ++-- block/raw-posix.c | 15 +++ block/raw-win32.c | 9 +++-- blockdev.c | 13 + include/block/aio.h | 2 +- include/block/thread-pool.h | 11 +++ include/qemu-common.h | 2 ++ qemu-options.hx | 2 +- thread-pool.c | 33 + 9 files changed, 81 insertions(+), 10 deletions(-) -- 1.8.1.2
[Qemu-devel] [PATCH v2 3/3] Add workerthreads configuration option
This patch allows to choose at the command line level which thread pool
implementation will be used by every block device.
Signed-off-by: Matthias Brugger matthias@gmail.com
---
blockdev.c | 13 +
qemu-options.hx | 2 +-
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/blockdev.c b/blockdev.c
index b260477..a16cc9a 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -387,6 +387,15 @@ static DriveInfo *blockdev_init(QDict *bs_opts,
}
}
#endif
+buf = qemu_opt_get(opts, workerthreads);
+if (buf != NULL) {
+if (!strcmp(buf, pool)) {
+/* this is the default */
+} else {
+error_report(invalid workerthreads option);
+return NULL;
+}
+}
if ((buf = qemu_opt_get(opts, format)) != NULL) {
if (is_help_option(buf)) {
@@ -2269,6 +2278,10 @@ QemuOptsList qemu_common_drive_opts = {
.type = QEMU_OPT_STRING,
.help = disk serial number,
},{
+.name = workerthreads,
+.type = QEMU_OPT_STRING,
+.help = type of worker threads (pool),
+},{
.name = rerror,
.type = QEMU_OPT_STRING,
.help = read error action,
diff --git a/qemu-options.hx b/qemu-options.hx
index 5dc8b75..6f22242 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -408,7 +408,7 @@ DEF(drive, HAS_ARG, QEMU_OPTION_drive,
[,cyls=c,heads=h,secs=s[,trans=t]][,snapshot=on|off]\n
[,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n
[,serial=s][,addr=A][,id=name][,aio=threads|native]\n
- [,readonly=on|off][,copy-on-read=on|off]\n
+ [,workerthreads=pool][,readonly=on|off][,copy-on-read=on|off]\n
[[,bps=b]|[[,bps_rd=r][,bps_wr=w]]]\n
[[,iops=i]|[[,iops_rd=r][,iops_wr=w]]]\n
[[,bps_max=bm]|[[,bps_rd_max=rm][,bps_wr_max=wm]]]\n
--
1.8.1.2
[Qemu-devel] [PATCH v2 1/3] Make thread pool implementation modular
This patch introduces function pointers for the thread pool, so that
it's implementation can be set at run-time.
Signed-off-by: Matthias Brugger matthias@gmail.com
---
include/block/thread-pool.h | 11 +++
thread-pool.c | 33 +
2 files changed, 44 insertions(+)
diff --git a/include/block/thread-pool.h b/include/block/thread-pool.h
index 32afcdd..1f73712 100644
--- a/include/block/thread-pool.h
+++ b/include/block/thread-pool.h
@@ -38,4 +38,15 @@ int coroutine_fn thread_pool_submit_co(ThreadPool *pool,
ThreadPoolFunc *func, void *arg);
void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg);
+ThreadPoolFuncArr *thread_pool_probe(void);
+void thread_pool_delete(ThreadPoolFuncArr *tpf);
+
+struct ThreadPoolFuncArr {
+BlockDriverAIOCB *(*thread_pool_submit_aio)(ThreadPool *pool,
+ThreadPoolFunc *func, void *arg, BlockDriverCompletionFunc *cb,
+void *opaque);
+ThreadPool *(*thread_pool_new)(AioContext *ctx);
+};
+
+
#endif
diff --git a/thread-pool.c b/thread-pool.c
index 3735fd3..53294a9 100644
--- a/thread-pool.c
+++ b/thread-pool.c
@@ -26,6 +26,7 @@
#include qemu/main-loop.h
static void do_spawn_thread(ThreadPool *pool);
+static void thread_pool_aio_free(ThreadPool *pool);
typedef struct ThreadPoolElement ThreadPoolElement;
@@ -77,6 +78,7 @@ struct ThreadPool {
int pending_threads; /* threads created but not running yet */
int pending_cancellations; /* whether we need a cond_broadcast */
bool stopping;
+void (*thread_pool_free)(ThreadPool *pool);
};
static void *worker_thread(void *opaque)
@@ -300,6 +302,7 @@ static void thread_pool_init_one(ThreadPool *pool,
AioContext *ctx)
qemu_sem_init(pool-sem, 0);
pool-max_threads = 64;
pool-new_thread_bh = aio_bh_new(ctx, spawn_thread_bh_fn, pool);
+pool-thread_pool_free = thread_pool_aio_free;
QLIST_INIT(pool-head);
QTAILQ_INIT(pool-request_list);
@@ -316,6 +319,11 @@ ThreadPool *thread_pool_new(AioContext *ctx)
void thread_pool_free(ThreadPool *pool)
{
+pool-thread_pool_free(pool);
+}
+
+void thread_pool_aio_free(ThreadPool *pool)
+{
if (!pool) {
return;
}
@@ -346,3 +354,28 @@ void thread_pool_free(ThreadPool *pool)
event_notifier_cleanup(pool-notifier);
g_free(pool);
}
+
+ThreadPoolFuncArr *thread_pool_probe(void)
+{
+ThreadPoolFuncArr *tpf_pool = NULL;
+
+if (tpf_pool) {
+return tpf_pool;
+}
+
+tpf_pool = g_new(ThreadPoolFuncArr, 1);
+if (!tpf_pool) {
+printf(error allocating thread pool\n);
+return NULL;
+}
+
+tpf_pool-thread_pool_submit_aio = thread_pool_submit_aio;
+tpf_pool-thread_pool_new = thread_pool_new;
+
+return tpf_pool;
+}
+
+void thread_pool_delete(ThreadPoolFuncArr *tpf)
+{
+g_free(tpf);
+}
--
1.8.1.2
[Qemu-devel] [PATCH v2 2/3] Block layer uses modular thread pool
With this patch, the calls to the thread pool functions pass through the
new modular thread pool implementation.
Signed-off-by: Matthias Brugger matthias@gmail.com
---
async.c | 4 ++--
block/raw-posix.c | 15 +++
block/raw-win32.c | 9 +++--
include/block/aio.h | 2 +-
include/qemu-common.h | 2 ++
5 files changed, 23 insertions(+), 9 deletions(-)
diff --git a/async.c b/async.c
index 5fb3fa6..e66f70f 100644
--- a/async.c
+++ b/async.c
@@ -232,10 +232,10 @@ GSource *aio_get_g_source(AioContext *ctx)
return ctx-source;
}
-ThreadPool *aio_get_thread_pool(AioContext *ctx)
+ThreadPool *aio_get_thread_pool(AioContext *ctx, ThreadPoolFuncArr *tpf)
{
if (!ctx-thread_pool) {
-ctx-thread_pool = thread_pool_new(ctx);
+ctx-thread_pool = tpf-thread_pool_new(ctx);
}
return ctx-thread_pool;
}
diff --git a/block/raw-posix.c b/block/raw-posix.c
index f6d48bb..f747301 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -142,6 +142,7 @@ typedef struct BDRVRawState {
bool is_xfs : 1;
#endif
bool has_discard : 1;
+ThreadPoolFuncArr *tpf;
} BDRVRawState;
typedef struct BDRVRawReopenState {
@@ -345,6 +346,9 @@ static int raw_open(BlockDriverState *bs, QDict *options,
int flags,
int ret;
s-type = FTYPE_FILE;
+
+s-tpf = thread_pool_probe();
+
ret = raw_open_common(bs, options, flags, 0, local_err);
if (error_is_set(local_err)) {
error_propagate(errp, local_err);
@@ -792,6 +796,7 @@ static BlockDriverAIOCB *paio_submit(BlockDriverState *bs,
int fd,
int64_t sector_num, QEMUIOVector *qiov, int nb_sectors,
BlockDriverCompletionFunc *cb, void *opaque, int type)
{
+BDRVRawState *s = bs-opaque;
RawPosixAIOData *acb = g_slice_new(RawPosixAIOData);
ThreadPool *pool;
@@ -807,8 +812,8 @@ static BlockDriverAIOCB *paio_submit(BlockDriverState *bs,
int fd,
acb-aio_offset = sector_num * 512;
trace_paio_submit(acb, opaque, sector_num, nb_sectors, type);
-pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+pool = aio_get_thread_pool(bdrv_get_aio_context(bs), s-tpf);
+return s-tpf-thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
}
static BlockDriverAIOCB *raw_aio_submit(BlockDriverState *bs,
@@ -874,6 +879,8 @@ static void raw_close(BlockDriverState *bs)
qemu_close(s-fd);
s-fd = -1;
}
+
+thread_pool_delete(s-tpf);
}
static int raw_truncate(BlockDriverState *bs, int64_t offset)
@@ -1490,8 +1497,8 @@ static BlockDriverAIOCB *hdev_aio_ioctl(BlockDriverState
*bs,
acb-aio_offset = 0;
acb-aio_ioctl_buf = buf;
acb-aio_ioctl_cmd = req;
-pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+pool = aio_get_thread_pool(bdrv_get_aio_context(bs), s-tpf);
+return s-tpf-thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
}
#elif defined(__FreeBSD__) || defined(__FreeBSD_kernel__)
diff --git a/block/raw-win32.c b/block/raw-win32.c
index 2741e4d..76df266 100644
--- a/block/raw-win32.c
+++ b/block/raw-win32.c
@@ -46,6 +46,7 @@ typedef struct RawWin32AIOData {
size_t aio_nbytes;
off64_t aio_offset;
int aio_type;
+ThreadPoolFuncArr *tpf;
} RawWin32AIOData;
typedef struct BDRVRawState {
@@ -159,8 +160,8 @@ static BlockDriverAIOCB *paio_submit(BlockDriverState *bs,
HANDLE hfile,
acb-aio_offset = sector_num * 512;
trace_paio_submit(acb, opaque, sector_num, nb_sectors, type);
-pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+pool = aio_get_thread_pool(bdrv_get_aio_context(bs), s-tpf);
+return s-tpf-thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
}
int qemu_ftruncate64(int fd, int64_t length)
@@ -248,6 +249,8 @@ static int raw_open(BlockDriverState *bs, QDict *options,
int flags,
s-type = FTYPE_FILE;
+s-tpf = thread_pool_probe();
+
opts = qemu_opts_create_nofail(raw_runtime_opts);
qemu_opts_absorb_qdict(opts, options, local_err);
if (error_is_set(local_err)) {
@@ -339,6 +342,8 @@ static void raw_close(BlockDriverState *bs)
{
BDRVRawState *s = bs-opaque;
CloseHandle(s-hfile);
+
+thread_pool_delete(s-tpf);
}
static int raw_truncate(BlockDriverState *bs, int64_t offset)
diff --git a/include/block/aio.h b/include/block/aio.h
index 2efdf41..22d6fa0 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -231,7 +231,7 @@ void aio_set_event_notifier(AioContext *ctx,
GSource *aio_get_g_source(AioContext *ctx);
/* Return the ThreadPool bound to this AioContext */
-struct ThreadPool *aio_get_thread_pool(AioContext *ctx);
+struct ThreadPool *aio_get_thread_pool(AioContext *ctx, ThreadPoolFuncArr
*tpf);
/* Functions to operate on the main QEMU
[Qemu-devel] Make thread pool implementation modular
This patch series makes the thread pool implementation modular. This allows each drive to use a special implementation. The patch series prepares qemu to be able to include thread pools different the one actually implemented. It will allow to implement approaches like paravirtualized block requests [1]. async.c | 4 ++-- block/raw-posix.c | 15 +++ block/raw-win32.c | 9 +++-- blockdev.c | 13 + include/block/aio.h | 2 +- include/block/thread-pool.h | 9 + include/qemu-common.h | 2 ++ qemu-options.hx | 2 +- thread-pool.c | 32 9 files changed, 78 insertions(+), 10 deletions(-) [1] http://www.linux-kvm.org/wiki/images/5/53/2012-forum-Brugger-lightningtalk.pdf
[Qemu-devel] [PATCH 3/3] Add workerthreads configuration option
This patch allows the definition which thread pool will be used by
every block device. The defintion of the workerthreads option
allows this at the command line level.
At the moment only the thread pool implementation pool can be chosen.
Signed-off-by: Matthias Brugger matthias@gmail.com
---
blockdev.c | 13 +
qemu-options.hx | 2 +-
2 files changed, 14 insertions(+), 1 deletion(-)
diff --git a/blockdev.c b/blockdev.c
index b260477..8b83611 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -388,6 +388,15 @@ static DriveInfo *blockdev_init(QDict *bs_opts,
}
#endif
+if ((buf = qemu_opt_get(opts, workerthreads)) != NULL) {
+if (!strcmp(buf, pool)) {
+/* this is the default */
+} else {
+error_report(invalid workerthreads option);
+return NULL;
+}
+}
+
if ((buf = qemu_opt_get(opts, format)) != NULL) {
if (is_help_option(buf)) {
error_printf(Supported formats:);
@@ -2269,6 +2278,10 @@ QemuOptsList qemu_common_drive_opts = {
.type = QEMU_OPT_STRING,
.help = disk serial number,
},{
+.name = workerthreads,
+.type = QEMU_OPT_STRING,
+.help = type of worker threads (pool),
+},{
.name = rerror,
.type = QEMU_OPT_STRING,
.help = read error action,
diff --git a/qemu-options.hx b/qemu-options.hx
index 5dc8b75..6f22242 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -408,7 +408,7 @@ DEF(drive, HAS_ARG, QEMU_OPTION_drive,
[,cyls=c,heads=h,secs=s[,trans=t]][,snapshot=on|off]\n
[,cache=writethrough|writeback|none|directsync|unsafe][,format=f]\n
[,serial=s][,addr=A][,id=name][,aio=threads|native]\n
- [,readonly=on|off][,copy-on-read=on|off]\n
+ [,workerthreads=pool][,readonly=on|off][,copy-on-read=on|off]\n
[[,bps=b]|[[,bps_rd=r][,bps_wr=w]]]\n
[[,iops=i]|[[,iops_rd=r][,iops_wr=w]]]\n
[[,bps_max=bm]|[[,bps_rd_max=rm][,bps_wr_max=wm]]]\n
--
1.8.1.2
[Qemu-devel] [PATCH 1/3] Make thread pool implementation modular
This patch introduces function pointers for the thread pool, so that
it's implementation can be set at run-time.
Signed-off-by: Matthias Brugger matthias@gmail.com
---
include/block/thread-pool.h | 9 +
thread-pool.c | 32
2 files changed, 41 insertions(+)
diff --git a/include/block/thread-pool.h b/include/block/thread-pool.h
index 32afcdd..53a779f 100644
--- a/include/block/thread-pool.h
+++ b/include/block/thread-pool.h
@@ -38,4 +38,13 @@ int coroutine_fn thread_pool_submit_co(ThreadPool *pool,
ThreadPoolFunc *func, void *arg);
void thread_pool_submit(ThreadPool *pool, ThreadPoolFunc *func, void *arg);
+ThreadPoolFuncArr *thread_pool_probe(void);
+void thread_pool_delete(ThreadPoolFuncArr *tpf);
+
+struct ThreadPoolFuncArr {
+BlockDriverAIOCB *(*thread_pool_submit_aio)(ThreadPool *pool,
ThreadPoolFunc *func, void *arg, BlockDriverCompletionFunc *cb, void *opaque);
+ThreadPool *(*thread_pool_new)(AioContext *ctx);
+};
+
+
#endif
diff --git a/thread-pool.c b/thread-pool.c
index 3735fd3..8c5d1a2 100644
--- a/thread-pool.c
+++ b/thread-pool.c
@@ -26,6 +26,7 @@
#include qemu/main-loop.h
static void do_spawn_thread(ThreadPool *pool);
+void thread_pool_aio_free(ThreadPool *pool);
typedef struct ThreadPoolElement ThreadPoolElement;
@@ -77,6 +78,7 @@ struct ThreadPool {
int pending_threads; /* threads created but not running yet */
int pending_cancellations; /* whether we need a cond_broadcast */
bool stopping;
+void (*thread_pool_free)(ThreadPool *pool);
};
static void *worker_thread(void *opaque)
@@ -300,6 +302,7 @@ static void thread_pool_init_one(ThreadPool *pool,
AioContext *ctx)
qemu_sem_init(pool-sem, 0);
pool-max_threads = 64;
pool-new_thread_bh = aio_bh_new(ctx, spawn_thread_bh_fn, pool);
+pool-thread_pool_free = thread_pool_aio_free;
QLIST_INIT(pool-head);
QTAILQ_INIT(pool-request_list);
@@ -316,6 +319,11 @@ ThreadPool *thread_pool_new(AioContext *ctx)
void thread_pool_free(ThreadPool *pool)
{
+pool-thread_pool_free(pool);
+}
+
+void thread_pool_aio_free(ThreadPool *pool)
+{
if (!pool) {
return;
}
@@ -346,3 +354,27 @@ void thread_pool_free(ThreadPool *pool)
event_notifier_cleanup(pool-notifier);
g_free(pool);
}
+
+ThreadPoolFuncArr *thread_pool_probe(void)
+{
+ThreadPoolFuncArr *tpf_pool = NULL;
+
+if (tpf_pool)
+return tpf_pool;
+
+tpf_pool = g_new(ThreadPoolFuncArr, 1); //TODO right now, this leaks!
+if (!tpf_pool) {
+printf(error allocating thread pool\n);
+return NULL;
+}
+
+tpf_pool-thread_pool_submit_aio = thread_pool_submit_aio;
+tpf_pool-thread_pool_new = thread_pool_new;
+
+return tpf_pool;
+}
+
+void thread_pool_delete(ThreadPoolFuncArr *tpf)
+{
+g_free(tpf);
+}
--
1.8.1.2
[Qemu-devel] [PATCH 2/3] Block layer uses modular thread pool
With this patch, the calls to the thread pool functions pass through the
new modular thread pool implementation.
Signed-off-by: Matthias Brugger matthias@gmail.com
---
async.c | 4 ++--
block/raw-posix.c | 15 +++
block/raw-win32.c | 9 +++--
include/block/aio.h | 2 +-
include/qemu-common.h | 2 ++
5 files changed, 23 insertions(+), 9 deletions(-)
diff --git a/async.c b/async.c
index 5fb3fa6..e66f70f 100644
--- a/async.c
+++ b/async.c
@@ -232,10 +232,10 @@ GSource *aio_get_g_source(AioContext *ctx)
return ctx-source;
}
-ThreadPool *aio_get_thread_pool(AioContext *ctx)
+ThreadPool *aio_get_thread_pool(AioContext *ctx, ThreadPoolFuncArr *tpf)
{
if (!ctx-thread_pool) {
-ctx-thread_pool = thread_pool_new(ctx);
+ctx-thread_pool = tpf-thread_pool_new(ctx);
}
return ctx-thread_pool;
}
diff --git a/block/raw-posix.c b/block/raw-posix.c
index 6f03fbf..22842ed 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -142,6 +142,7 @@ typedef struct BDRVRawState {
bool is_xfs : 1;
#endif
bool has_discard : 1;
+ThreadPoolFuncArr *tpf;
} BDRVRawState;
typedef struct BDRVRawReopenState {
@@ -345,6 +346,9 @@ static int raw_open(BlockDriverState *bs, QDict *options,
int flags,
int ret;
s-type = FTYPE_FILE;
+
+s-tpf = thread_pool_probe();
+
ret = raw_open_common(bs, options, flags, 0, local_err);
if (error_is_set(local_err)) {
error_propagate(errp, local_err);
@@ -792,6 +796,7 @@ static BlockDriverAIOCB *paio_submit(BlockDriverState *bs,
int fd,
int64_t sector_num, QEMUIOVector *qiov, int nb_sectors,
BlockDriverCompletionFunc *cb, void *opaque, int type)
{
+BDRVRawState *s = bs-opaque;
RawPosixAIOData *acb = g_slice_new(RawPosixAIOData);
ThreadPool *pool;
@@ -807,8 +812,8 @@ static BlockDriverAIOCB *paio_submit(BlockDriverState *bs,
int fd,
acb-aio_offset = sector_num * 512;
trace_paio_submit(acb, opaque, sector_num, nb_sectors, type);
-pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+pool = aio_get_thread_pool(bdrv_get_aio_context(bs), s-tpf);
+return s-tpf-thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
}
static BlockDriverAIOCB *raw_aio_submit(BlockDriverState *bs,
@@ -874,6 +879,8 @@ static void raw_close(BlockDriverState *bs)
qemu_close(s-fd);
s-fd = -1;
}
+
+thread_pool_delete(s-tpf);
}
static int raw_truncate(BlockDriverState *bs, int64_t offset)
@@ -1490,8 +1497,8 @@ static BlockDriverAIOCB *hdev_aio_ioctl(BlockDriverState
*bs,
acb-aio_offset = 0;
acb-aio_ioctl_buf = buf;
acb-aio_ioctl_cmd = req;
-pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+pool = aio_get_thread_pool(bdrv_get_aio_context(bs), s-tpf);
+return s-tpf-thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
}
#elif defined(__FreeBSD__) || defined(__FreeBSD_kernel__)
diff --git a/block/raw-win32.c b/block/raw-win32.c
index 676b570..dad1255 100644
--- a/block/raw-win32.c
+++ b/block/raw-win32.c
@@ -46,6 +46,7 @@ typedef struct RawWin32AIOData {
size_t aio_nbytes;
off64_t aio_offset;
int aio_type;
+ThreadPoolFuncArr *tpf;
} RawWin32AIOData;
typedef struct BDRVRawState {
@@ -159,8 +160,8 @@ static BlockDriverAIOCB *paio_submit(BlockDriverState *bs,
HANDLE hfile,
acb-aio_offset = sector_num * 512;
trace_paio_submit(acb, opaque, sector_num, nb_sectors, type);
-pool = aio_get_thread_pool(bdrv_get_aio_context(bs));
-return thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
+pool = aio_get_thread_pool(bdrv_get_aio_context(bs), s-tpf);
+return s-tpf-thread_pool_submit_aio(pool, aio_worker, acb, cb, opaque);
}
int qemu_ftruncate64(int fd, int64_t length)
@@ -248,6 +249,8 @@ static int raw_open(BlockDriverState *bs, QDict *options,
int flags,
s-type = FTYPE_FILE;
+s-tpf = thread_pool_probe();
+
opts = qemu_opts_create_nofail(raw_runtime_opts);
qemu_opts_absorb_qdict(opts, options, local_err);
if (error_is_set(local_err)) {
@@ -339,6 +342,8 @@ static void raw_close(BlockDriverState *bs)
{
BDRVRawState *s = bs-opaque;
CloseHandle(s-hfile);
+
+thread_pool_delete(s-tpf);
}
static int raw_truncate(BlockDriverState *bs, int64_t offset)
diff --git a/include/block/aio.h b/include/block/aio.h
index 2efdf41..22d6fa0 100644
--- a/include/block/aio.h
+++ b/include/block/aio.h
@@ -231,7 +231,7 @@ void aio_set_event_notifier(AioContext *ctx,
GSource *aio_get_g_source(AioContext *ctx);
/* Return the ThreadPool bound to this AioContext */
-struct ThreadPool *aio_get_thread_pool(AioContext *ctx);
+struct ThreadPool *aio_get_thread_pool(AioContext *ctx, ThreadPoolFuncArr
*tpf);
/* Functions to operate on the main QEMU
[Qemu-devel] [Bug 1129571] Re: libreoffice armhf FTBFS
The relevant snippet from the build log doesn't even show the complete command ... -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/1129571 Title: libreoffice armhf FTBFS Status in QEMU: New Status in “qemu” package in Ubuntu: Confirmed Bug description: We have been experiencing FTBFS of LibreOffice 3.5.7, 12.04, armhf in the launchpad buildds. We believe this is likely due to an error in qemu. While we do not have a small test case yet, we do have a build log (attaching here). The relevant snippet from the build log is: 3.5.7/solver/unxlngr.pro/bin/jaxp.jar:/build/buildd/libreoffice-3.5.7/solver/unxlngr.pro/bin/juh.jar:/build/buildd/libreoffice-3.5.7/solver/unxlngr.pro/bin/parser.jar:/build/buildd/libreoffice-3.5.7/solver/unxlngr.pro/bin/xt.jar:/build/buildd/libreoffice-3.5.7/solver/unxlngr.pro/bin/unoil.jar:/build/buildd/libreoffice-3.5.7/solver/unxlngr.pro/bin/ridl.jar:/build/buildd/libreoffice-3.5.7/solver/unxlngr.pro/bin/jurt.jar:/build/buildd/libreoffice-3.5.7/solver/unxlngr.pro/bin/xmlsearch.jar:/build/buildd/libreoffice-3.5.7/solver/unxlngr.pro/bin/LuceneHelpWrapper.jar:/build/buildd/libreoffice-3.5.7/solver/unxlngr.pro/bin/HelpIndexerTool.jar:/build/buildd/libreoffice-3.5.7/solver/unxlngr.pro/bin/lucene-core-2.3.jar:/build/buildd/libreoffice-3.5.7/solver/unxlngr.pro/bin/lucene-analyzers-2.3.jar com.sun.star.help.HelpIndexerTool -lang cs -mod swriter -zipdir ../../unxlngr.pro/misc/ziptmpswriter_cs -o ../../unxlngr.pro/bin/swriter_cs.zip.unxlngr.pro dmake: Error code 132, while making '../../unxlngr.pro/bin/swriter_cs.zip' We believe this is from bash error code 128 + 4, where 4 is illegal instruction, thus leading us to suspect qemu. Any help in tracking this down would be appreciated. To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/1129571/+subscriptions
[Qemu-devel] [PATCH] Fixing some spelling in docs/libcacard.txt
From: Matthias Brugger matthias@gmail.com Signed-off-by: Matthias Brugger matthias@gmail.com --- docs/libcacard.txt | 12 ++-- 1 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/libcacard.txt b/docs/libcacard.txt index 5dee6fa..576af57 100644 --- a/docs/libcacard.txt +++ b/docs/libcacard.txt @@ -170,7 +170,7 @@ public entry point: int cert_count); The parameters for this are: - card - the virtual card structure which will prepresent this card. + card - the virtual card structure which will represent this card. flags - option flags that may be specific to this card type. cert - array of binary certificates. cert_len - array of lengths of each of the certificates specified in cert. @@ -179,7 +179,7 @@ public entry point: cert_count - number of entries in cert, cert_len, and key arrays. Any cert, cert_len, or key with the same index are matching sets. That is - cert[0] is cert_len[0] long and has the corresponsing private key of key[0]. + cert[0] is cert_len[0] long and has the corresponding private key of key[0]. The card type emulator is expected to own the VCardKeys, but it should copy any raw cert data it wants to save. It can create new applets and add them to @@ -261,7 +261,7 @@ Prior to processing calling the card type emulator's VCardProcessAPDU function, apdu-a_Le - The expected length of any returned data. apdu-a_cla - The raw apdu class. apdu-a_channel - The channel (decoded from the class). - apdu-a_secure_messaging_type - The decoded secure messagin type + apdu-a_secure_messaging_type - The decoded secure messaging type (from class). apdu-a_type - The decode class type. apdu-a_gen_type - the generic class type (7816, PROPRIETARY, RFU, PTS). @@ -273,7 +273,7 @@ Creating a Response -- The expected result of any APDU call is a response. The card type emulator must set *response with an appropriate VCardResponse value if it returns VCARD_DONE. -Reponses could be as simple as returning a 2 byte status word response, to as +Responses could be as simple as returning a 2 byte status word response, to as complex as returning a block of data along with a 2 byte response. Which is returned will depend on the semantics of the APDU. The following functions will create card responses. @@ -282,12 +282,12 @@ create card responses. This is the most basic function to get a response. This function will return a response the consists soley one 2 byte status code. If that status -code is defined in card_7816t.h, then this function is guarrenteed to +code is defined in card_7816t.h, then this function is guaranteed to return a response with that status. If a cart type specific status code is passed and vcard_make_response fails to allocate the appropriate memory for that response, then vcard_make_response will return a VCardResponse of VCARD7816_STATUS_EXC_ERROR_MEMORY. In any case, this function is -guarrenteed to return a valid VCardResponse. +guaranteed to return a valid VCardResponse. VCardResponse *vcard_response_new(unsigned char *buf, int len, VCard7816Status status); -- 1.7.1
Re: [Qemu-devel] [PATCH] block.c typo in comment fixed
On 11/01/2011 08:39 AM, Stefan Hajnoczi wrote: On Tue, Nov 01, 2011 at 01:36:42AM +, matthias@googlemail.com wrote: From: Matthias Bruggermatthias@gmail.com Signed-off-by: Matthias Bruggermatthias@gmail.com --- block.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) Writable is also a common spelling. Try git grep writable and compare against git grep writeable. In qemu.git writable actually dominates by a large majority so we should leave this alone. Stefan Alright, I didn't know that. Sorry for the noise. Regards, Matthias -- --- http://motzblog.wordpress.com/
[Qemu-devel] [PATCH] block.c typo in comment fixed
From: Matthias Brugger matthias@gmail.com
Signed-off-by: Matthias Brugger matthias@gmail.com
---
block.c |2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/block.c b/block.c
index 9bb236c..480aae2 100644
--- a/block.c
+++ b/block.c
@@ -497,7 +497,7 @@ static int bdrv_open_common(BlockDriverState *bs, const
char *filename,
open_flags = flags ~(BDRV_O_SNAPSHOT | BDRV_O_NO_BACKING);
/*
- * Snapshots should be writable.
+ * Snapshots should be writeable.
*/
if (bs-is_temporary) {
open_flags |= BDRV_O_RDWR;
--
1.7.1
[Qemu-devel] [Bug 824716] Re: linux-user broken for targets with TARGET_ABI32 (i.e. qemu-sparc32plus)
Bugfix as sent to qemu-devel mailinglist on 2011/08/15 ** Patch added: 0001-linux-user-fix-abi_-u-long-target_ulong-mismatch.patch https://bugs.launchpad.net/qemu/+bug/824716/+attachment/2285481/+files/0001-linux-user-fix-abi_-u-long-target_ulong-mismatch.patch -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/824716 Title: linux-user broken for targets with TARGET_ABI32 (i.e. qemu- sparc32plus) Status in QEMU: New Bug description: I just debugged a problem I had with linux-user for qemu-sparc32plus. Turns out that sparc32plus is defined as a 64bit target with TARGET_ABI32 set. This correctly leads to abi_ulong (and others) being defined as uint32_t. However most of the code (in syscall.c) uses tswapl for these values, which swaps the endianess of a target long (which is 64bit). This doesn't match the uin32_t abi_ulongs and fails! So it appears to me like one would need to define something like an aswapl which swaps abi_ulongs and replace most of the tswapls there... To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/824716/+subscriptions
[Qemu-devel] [PATCH] linux-user: fix abi_(u)long, target_ulong mismatch
abi_(u)long might be different from target_ulong, so don't use tswapl
but introduce a new tswapal
see also https://bugs.launchpad.net/qemu/+bug/824716
Signed-off-by: Matthias Braun ma...@braunis.de
---
linux-user/qemu-types.h | 12 +++
linux-user/signal.c | 22 ++--
linux-user/strace.c |4 +-
linux-user/syscall.c | 240
++--
linux-user/syscall_defs.h |8 +-
linux-user/vm86.c |4 +-
6 files changed, 151 insertions(+), 139 deletions(-)
diff --git a/linux-user/qemu-types.h b/linux-user/qemu-types.h
index 1adda9f..fe7f662 100644
--- a/linux-user/qemu-types.h
+++ b/linux-user/qemu-types.h
@@ -9,6 +9,12 @@ typedef int32_t abi_long;
#define TARGET_ABI_FMT_ld %d
#define TARGET_ABI_FMT_lu %u
#define TARGET_ABI_BITS 32
+
+static inline abi_ulong tswapal(abi_ulong v)
+{
+return tswap32(v);
+}
+
#else
typedef target_ulong abi_ulong;
typedef target_long abi_long;
@@ -20,5 +26,11 @@ typedef target_long abi_long;
#if TARGET_ABI_BITS == 32
#define TARGET_ABI32 1
#endif
+
+static inline abi_ulong tswapal(abi_ulong v)
+{
+return tswapl(v);
+}
+
#endif
#endif
diff --git a/linux-user/signal.c b/linux-user/signal.c
index 07ad07a..c70f7b2 100644
--- a/linux-user/signal.c
+++ b/linux-user/signal.c
@@ -152,7 +152,7 @@ void host_to_target_sigset(target_sigset_t *d, const
sigset_t *s)
host_to_target_sigset_internal(d1, s);
for(i = 0;i TARGET_NSIG_WORDS; i++)
-d-sig[i] = tswapl(d1.sig[i]);
+d-sig[i] = tswapal(d1.sig[i]);
}
static void target_to_host_sigset_internal(sigset_t *d,
@@ -173,7 +173,7 @@ void target_to_host_sigset(sigset_t *d, const
target_sigset_t *s)
int i;
for(i = 0;i TARGET_NSIG_WORDS; i++)
-s1.sig[i] = tswapl(s-sig[i]);
+s1.sig[i] = tswapal(s-sig[i]);
target_to_host_sigset_internal(d, s1);
}
@@ -234,14 +234,14 @@ static void tswap_siginfo(target_siginfo_t *tinfo,
if (sig == SIGILL || sig == SIGFPE || sig == SIGSEGV ||
sig == SIGBUS || sig == SIGTRAP) {
tinfo-_sifields._sigfault._addr =
-tswapl(info-_sifields._sigfault._addr);
+tswapal(info-_sifields._sigfault._addr);
} else if (sig == SIGIO) {
tinfo-_sifields._sigpoll._fd = tswap32(info-_sifields._sigpoll._fd);
} else if (sig = TARGET_SIGRTMIN) {
tinfo-_sifields._rt._pid = tswap32(info-_sifields._rt._pid);
tinfo-_sifields._rt._uid = tswap32(info-_sifields._rt._uid);
tinfo-_sifields._rt._sigval.sival_ptr =
-tswapl(info-_sifields._rt._sigval.sival_ptr);
+tswapal(info-_sifields._rt._sigval.sival_ptr);
}
}
@@ -262,7 +262,7 @@ void target_to_host_siginfo(siginfo_t *info, const
target_siginfo_t *tinfo)
info-si_pid = tswap32(tinfo-_sifields._rt._pid);
info-si_uid = tswap32(tinfo-_sifields._rt._uid);
info-si_value.sival_ptr =
-(void
*)(long)tswapl(tinfo-_sifields._rt._sigval.sival_ptr);
+(void
*)(long)tswapal(tinfo-_sifields._rt._sigval.sival_ptr);
}
static int fatal_signal (int sig)
@@ -586,19 +586,19 @@ int do_sigaction(int sig, const struct
target_sigaction *act,
sig, act, oact);
#endif
if (oact) {
-oact-_sa_handler = tswapl(k-_sa_handler);
-oact-sa_flags = tswapl(k-sa_flags);
+oact-_sa_handler = tswapal(k-_sa_handler);
+oact-sa_flags = tswapal(k-sa_flags);
#if !defined(TARGET_MIPS)
-oact-sa_restorer = tswapl(k-sa_restorer);
+oact-sa_restorer = tswapal(k-sa_restorer);
#endif
oact-sa_mask = k-sa_mask;
}
if (act) {
/* FIXME: This is not threadsafe. */
-k-_sa_handler = tswapl(act-_sa_handler);
-k-sa_flags = tswapl(act-sa_flags);
+k-_sa_handler = tswapal(act-_sa_handler);
+k-sa_flags = tswapal(act-sa_flags);
#if !defined(TARGET_MIPS)
-k-sa_restorer = tswapl(act-sa_restorer);
+k-sa_restorer = tswapal(act-sa_restorer);
#endif
k-sa_mask = act-sa_mask;
diff --git a/linux-user/strace.c b/linux-user/strace.c
index fe9326a..90027a1 100644
--- a/linux-user/strace.c
+++ b/linux-user/strace.c
@@ -169,7 +169,7 @@ print_fdset(int n, abi_ulong target_fds_addr)
return;
for (i=n; i=0; i--) {
-if ((tswapl(target_fds[i / TARGET_ABI_BITS]) (i
(TARGET_ABI_BITS - 1))) 1)
+if ((tswapal(target_fds[i / TARGET_ABI_BITS]) (i
(TARGET_ABI_BITS - 1))) 1)
gemu_log(%d,, i );
}
unlock_user(target_fds, target_fds_addr, 0);
@@ -245,7 +245,7 @@ print_execve(const struct syscallname *name,
arg_ptr = lock_user(VERIFY_READ, arg_ptr_addr, sizeof(abi_ulong), 1);
if (!arg_ptr)
return;
- arg_addr = tswapl(*arg_ptr);
+arg_addr = tswapal(*arg_ptr);
unlock_user(arg_ptr, arg_ptr_addr, 0);
if (!arg_addr)
break;
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
[Qemu-devel] [Bug 824716] [NEW] linux-user broken for targets with TARGET_ABI32 (i.e. qemu-sparc32plus)
Public bug reported: I just debugged a problem I had with linux-user for qemu-sparc32plus. Turns out that sparc32plus is defined as a 64bit target with TARGET_ABI32 set. This correctly leads to abi_ulong (and others) being defined as uint32_t. However most of the code (in syscall.c) uses tswapl for these values, which swaps the endianess of a target long (which is 64bit). This doesn't match the uin32_t abi_ulongs and fails! So it appears to me like one would need to define something like an aswapl which swaps abi_ulongs and replace most of the tswapls there... ** Affects: qemu Importance: Undecided Status: New -- You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. https://bugs.launchpad.net/bugs/824716 Title: linux-user broken for targets with TARGET_ABI32 (i.e. qemu- sparc32plus) Status in QEMU: New Bug description: I just debugged a problem I had with linux-user for qemu-sparc32plus. Turns out that sparc32plus is defined as a 64bit target with TARGET_ABI32 set. This correctly leads to abi_ulong (and others) being defined as uint32_t. However most of the code (in syscall.c) uses tswapl for these values, which swaps the endianess of a target long (which is 64bit). This doesn't match the uin32_t abi_ulongs and fails! So it appears to me like one would need to define something like an aswapl which swaps abi_ulongs and replace most of the tswapls there... To manage notifications about this bug go to: https://bugs.launchpad.net/qemu/+bug/824716/+subscriptions
[Qemu-devel] Vista guest doesn t start after libvirt/qemu upgrade
Hi, I've updated my Debian lenny to Debian squeeze last week. So I have now: qemu 0.12.5+dfsg-3 qemu-keymaps 0.12.5+dfsg-3 qemu-kvm 0.12.5+dfsg-5 qemu-system 0.12.5+dfsg-3 qemu-user 0.12.5+dfsg-3 qemu-utils 0.12.5+dfsg-3 libvirt 0.8.3-5 virt-manager 0.8.4-8 Yesterday I try to start my Vista quest again. I am surprised because it won't starts. In /var/log/libvirt/qemu/devel-vista.log stands Read failed. The Console stays as a black screen after a short SeaBIOS message. I tried system recovery too but Vista system recovery identified the Vista installation without any errors. Nearly the same occurs with my Windows7 guest. The logfile seems fine but the console present me: Starting SeaBIOS (version...) Booting from Hard Disk... Boot failed: not a bootable disk I tried system recovery too and indeed, W7 doesn't found a Windows Installation. linux-dev:~# LANG==C ls -lh --block-size=1M /var/lib/libvirt/images/ total 90399 -rwxr-xr-x 1 root root 1 Feb 12 19:20 devel-CentOS.img -rwxr-xr-x 1 root root 1 Feb 15 20:17 devel-SUSE.img -rwxr-xr-x 1 root root 10001 Feb 16 22:55 devel-fedora.img -rwxr-xr-x 1 root root 1 Feb 16 19:23 devel-ubuntu.img -rwxr-xr-x 1 root root 12575 Feb 1 07:56 devel-vista.qcow2 -rwxr-xr-x 1 libvirt-qemu kvm 40961 Feb 21 23:50 devel-vista.qcow2.2del -rwxr-xr-x 1 root root 9330 Jan 23 18:03 devel-w7.qcow2 -rwxr-xr-x 1 root root 10136 Feb 13 20:58 devel-w7.qcow2.2del -rwxr-xr-x 1 root root 10241 Feb 22 00:34 devel-xp.qcow2 the *.2del files are the original images which doesn't start. The responding files without .2del are restored from a backup. WindowsXP as well as different Linux guests start and run fine. There are any experience with this obscurse behavior? Thanks in advance Matthias -- Don't Panic
[Qemu-devel] [Bug 612297] [NEW] qemu-kvm fails to detect mouse while Windows 95 setup
Public bug reported: CPU: AMD Phenom II X4 945 KVM-Version: qemu-kvm-0.12.4+dfsg (Debian package) Kernel: linux-2.6.26.8-rt16 arch: x86_64 Guest: Windows 95 B I'm trying to install Windows 95 B on qemu-kvm with this options: kvm /var/mmn/vm/Win95/Win95.img -name Windows 95 -M pc-0.12 -m 512M -rtc base=localtime -k de -soundhw sb16 -vga cirrus -net user,hostname=w95vm -net nic,model=ne2k_pci -boot a -fda /var/mmn/vm/floppy/win95B_Drive-D-boot.img -cdrom /var/mmn/vm/CD/win95-setup.iso -no-acpi -no-kvm -usb I've also tried some other option, but always the same: no ps/2 mouse detection. And usb mouse is not supported by Windows 95 B while setup process. This is only possible later by installing the extension usbsupp.exe after the system setup. ** Affects: qemu Importance: Undecided Status: New -- qemu-kvm fails to detect mouse while Windows 95 setup https://bugs.launchpad.net/bugs/612297 You received this bug notification because you are a member of qemu- devel-ml, which is subscribed to QEMU. Status in QEMU: New Bug description: CPU: AMD Phenom II X4 945 KVM-Version: qemu-kvm-0.12.4+dfsg (Debian package) Kernel: linux-2.6.26.8-rt16 arch: x86_64 Guest: Windows 95 B I'm trying to install Windows 95 B on qemu-kvm with this options: kvm /var/mmn/vm/Win95/Win95.img -name Windows 95 -M pc-0.12 -m 512M -rtc base=localtime -k de -soundhw sb16 -vga cirrus -net user,hostname=w95vm -net nic,model=ne2k_pci -boot a -fda /var/mmn/vm/floppy/win95B_Drive-D-boot.img -cdrom /var/mmn/vm/CD/win95-setup.iso -no-acpi -no-kvm -usb I've also tried some other option, but always the same: no ps/2 mouse detection. And usb mouse is not supported by Windows 95 B while setup process. This is only possible later by installing the extension usbsupp.exe after the system setup.
Re: [libvirt] [Qemu-devel] Re: Libvirt debug API
2010/4/26 Avi Kivity a...@redhat.com: [...] In theory, it does support this with the session urls but they are currently second-class citizens in libvirt. The remote dispatch also adds a fair bit of complexity and at least for the use-cases I'm interested in, it's not an important feature. If libvirt needs a local wrapper for interesting use cases, then it has failed. You can't have a local wrapper with the esx driver, for example. This is off-topic, but can you detail why you don't want remote dispatch (I assume we're talking about a multiple node deployment). Because there are dozens of remote management APIs and then all have a concept of agents that run on the end nodes. When fitting virtualization management into an existing management infrastructure, you are going to always use a local API. When you manage esx, do you deploy an agent? I thought it was all done via their remote APIs. No, you don't deploy any agents. If you manage an ESX server using VMware tools only, then you use VMware management tools on the client PC. This tools directly use the SOAP based remote API provided by the ESX server. The ESX driver in libvirt does the same and uses this remote API as the VMware management tools. No libvirt specific agents are installed on the ESX server and no VMware specific agents (except libvirt itself) are installed on the client PC. Matthias
[Qemu-devel] Different qemu mashines networking- mac adress
When I try to start several qemu mashines I get problems with the networking. Normal I start mashines with qemu -boot c -net nic,vlan=0 -net tap,vlan=0,ifname=tap0 image.img But starting severel mashines, they would get the same ip from the dns-dhcp server because of the mac adresses. So I start with qemu -boot c -net nic,vlan=0,macaddr=52:54:00:12:34:56 -net tap,vlan=0,ifname=tap0 image1.img and qemu -boot c -net nic,vlan=0,macaddr=52:54:00:12:34:57 -net tap,vlan=0,ifname=tap1 image2.img I am searching for a script with automatic sets a free mac adress and a free tapX number. mfg Matthias Taube ___ Qemu-devel mailing list Qemu-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/qemu-devel
