Re: [Qemu-devel] [PATCH 2/2] qemu-thread: let cur_mon be per-thread
On Wed, Apr 11, 2018 at 08:06:04AM -0500, Eric Blake wrote: > On 04/11/2018 04:48 AM, Peter Xu wrote: > > > Okay. :) Thanks for confirming. Then let me repost this patch without > > touching the qemu-threads. > > > > Btw, do you want me to repost the first patch separately too, or keep > > the code as is? I believe it depends on whether you treat that one as > > a cleanup or not. Thanks, > > The first patch is no longer necessary for your new approach, but as it > is a cleanup and you've already written it, it does not hurt to still > send it as a separate cleanup patch useful in its own right. Thank you, Eric. I will repost. -- Peter Xu
Re: [Qemu-devel] [PATCH 2/2] qemu-thread: let cur_mon be per-thread
On 04/11/2018 04:48 AM, Peter Xu wrote: > Okay. :) Thanks for confirming. Then let me repost this patch without > touching the qemu-threads. > > Btw, do you want me to repost the first patch separately too, or keep > the code as is? I believe it depends on whether you treat that one as > a cleanup or not. Thanks, The first patch is no longer necessary for your new approach, but as it is a cleanup and you've already written it, it does not hurt to still send it as a separate cleanup patch useful in its own right. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature
Re: [Qemu-devel] [PATCH 2/2] qemu-thread: let cur_mon be per-thread
On Wed, Apr 11, 2018 at 11:38:58AM +0200, Paolo Bonzini wrote: > On 11/04/2018 11:35, Peter Xu wrote: > > Yeah, the inheritance will only make sure cur_mon be initialized > > always with correct value just like when we are without Out-Of-Band. > > For example, it's still possible a thread is created within a QMP > > handler. If without current change, the cur_mon in the new thread > > would be NULL. > > > > AFAIU even if cur_mon==NULL we should mostly be fine (e.g., > > error_vprintf will handle that case well). If any of you can help me > > confirm this, then I agree that this patch is not really needed. > > Yes, though the solution is to not use error_vprintf from auxiliary > threads. :) Just make sure all the communication happens through a > struct that's passed in the thread entry point, and possibly bottom > halves from the auxiliary thread to the monitor iothread. Okay. :) Thanks for confirming. Then let me repost this patch without touching the qemu-threads. Btw, do you want me to repost the first patch separately too, or keep the code as is? I believe it depends on whether you treat that one as a cleanup or not. Thanks, -- Peter Xu
Re: [Qemu-devel] [PATCH 2/2] qemu-thread: let cur_mon be per-thread
On 11/04/2018 11:35, Peter Xu wrote: > Yeah, the inheritance will only make sure cur_mon be initialized > always with correct value just like when we are without Out-Of-Band. > For example, it's still possible a thread is created within a QMP > handler. If without current change, the cur_mon in the new thread > would be NULL. > > AFAIU even if cur_mon==NULL we should mostly be fine (e.g., > error_vprintf will handle that case well). If any of you can help me > confirm this, then I agree that this patch is not really needed. Yes, though the solution is to not use error_vprintf from auxiliary threads. :) Just make sure all the communication happens through a struct that's passed in the thread entry point, and possibly bottom halves from the auxiliary thread to the monitor iothread. Paolo > If so, maybe even we don't need to setup cur_mon at entry of monitor > iothread, since cur_mon is always used in the way like:
Re: [Qemu-devel] [PATCH 2/2] qemu-thread: let cur_mon be per-thread
On Wed, Apr 11, 2018 at 11:23:57AM +0200, Paolo Bonzini wrote: > On 11/04/2018 05:49, Peter Xu wrote: > > On Wed, Apr 11, 2018 at 09:45:32AM +0800, Stefan Hajnoczi wrote: > >> On Tue, Apr 10, 2018 at 08:49:13PM +0800, Peter Xu wrote: > >>> cur_mon was only used in main loop so we don't really need that to be > >>> per-thread variable. Now it's possible that we have more than one > >>> thread to operate on it. Let's start to let it be per-thread variable. > >> Trying to understand the reason for this patch: > >> > >> Are there any users of per-thread cur_mon? > > > > Currently no. But if considering future OOB-capable commands, they > > will modify cur_mon in monitor IOThread at least. > > That's fine, but it shouldn't need the inheritance part. The monitor > IOThread can set cur_mon when it starts. Yeah, the inheritance will only make sure cur_mon be initialized always with correct value just like when we are without Out-Of-Band. For example, it's still possible a thread is created within a QMP handler. If without current change, the cur_mon in the new thread would be NULL. AFAIU even if cur_mon==NULL we should mostly be fine (e.g., error_vprintf will handle that case well). If any of you can help me confirm this, then I agree that this patch is not really needed. If so, maybe even we don't need to setup cur_mon at entry of monitor iothread, since cur_mon is always used in the way like: old_mon = cur_mon; cur_mon = xxx; ... (do something) cur_mon = old_mon; And it'll be fine old_mon==NULL here. Then IMHO the only thing we need to do is to mark cur_mon as per-thread and we're done. Thanks, -- Peter Xu
Re: [Qemu-devel] [PATCH 2/2] qemu-thread: let cur_mon be per-thread
On 11/04/2018 05:49, Peter Xu wrote: > On Wed, Apr 11, 2018 at 09:45:32AM +0800, Stefan Hajnoczi wrote: >> On Tue, Apr 10, 2018 at 08:49:13PM +0800, Peter Xu wrote: >>> cur_mon was only used in main loop so we don't really need that to be >>> per-thread variable. Now it's possible that we have more than one >>> thread to operate on it. Let's start to let it be per-thread variable. >> Trying to understand the reason for this patch: >> >> Are there any users of per-thread cur_mon? > > Currently no. But if considering future OOB-capable commands, they > will modify cur_mon in monitor IOThread at least. That's fine, but it shouldn't need the inheritance part. The monitor IOThread can set cur_mon when it starts. In general, relying on cur_mon should be avoided. Paolo
Re: [Qemu-devel] [PATCH 2/2] qemu-thread: let cur_mon be per-thread
On Wed, Apr 11, 2018 at 09:45:32AM +0800, Stefan Hajnoczi wrote: > On Tue, Apr 10, 2018 at 08:49:13PM +0800, Peter Xu wrote: > > cur_mon was only used in main loop so we don't really need that to be > > per-thread variable. Now it's possible that we have more than one > > thread to operate on it. Let's start to let it be per-thread variable. > > Trying to understand the reason for this patch: > > Are there any users of per-thread cur_mon? Currently no. But if considering future OOB-capable commands, they will modify cur_mon in monitor IOThread at least. > > or > > Does this patch fix a bug? No; currently we have no bug. But we have encounter the bug when we start to add more OOB commands. Here is the problem (Marc-Andre reported this, and I'll try to summarize): after we have valid OOB commands, monitor_qmp_dispatch_one() can be run not only in main thread, but also in monitor iothread. When that happens, both of them (main thread, and monitor iothread) can be modifying the cur_mon variable at the same time. [1] Considering that cur_mon is only used "just like" a stack variable, it should be perfectly fine we just make it as a per thread variable, hence this patch. > > > In case we'll create threads within a valid cur_mon setup, we'd better > > let the child threads to inherit the cur_mon from parent thread too. Do > > that for both posix and win32 threads. > > Without actual users I don't like this. It sounds like "let's make it > global just in case something needs it some day". > > It's ugly for QEMU's thread API to know about the monitor - that's a > layering violation. Yes, I'm sorry about it. Actually I don't like it too. But that seems to be an efficient and simple solution to me now. The ideal solution should be totally removing cur_mon, which is non-trivial. And for sure we can try to avoid layer violation. For example, we can have something like qemu_thread_register_variable(pthread_key_t), then monitor code can register the cur_mon pthread_key to the qemu thread module. That'll somehow achieve isolation between modules but I'm not sure whether that would be necessary for now, hence I chose the simple. > > If there's a legitimate need I think this patch might be necessary, but > I don't see enough justification to do this yet. The problem was described at [1]. Thanks, -- Peter Xu
Re: [Qemu-devel] [PATCH 2/2] qemu-thread: let cur_mon be per-thread
On Tue, Apr 10, 2018 at 08:54:31AM -0500, Eric Blake wrote: > On 04/10/2018 07:49 AM, Peter Xu wrote: > > cur_mon was only used in main loop so we don't really need that to be > > per-thread variable. Now it's possible that we have more than one > > thread to operate on it. Let's start to let it be per-thread variable. > > > > In case we'll create threads within a valid cur_mon setup, we'd better > > let the child threads to inherit the cur_mon from parent thread too. Do > > that for both posix and win32 threads. > > > > Signed-off-by: Peter Xu > > --- > > include/monitor/monitor.h | 2 +- > > include/qemu/thread-win32.h | 1 + > > monitor.c | 2 +- > > stubs/monitor.c | 2 +- > > tests/test-util-sockets.c | 2 +- > > util/qemu-thread-posix.c| 6 ++ > > util/qemu-thread-win32.c| 6 ++ > > 7 files changed, 17 insertions(+), 4 deletions(-) > > > > > @@ -494,6 +496,9 @@ static void *qemu_thread_start(void *args) > > void *(*start_routine)(void *) = qemu_thread_args->start_routine; > > void *arg = qemu_thread_args->arg; > > > > +/* Inherit the cur_mon pointer from father thread */ > > More typical as s/father/parent/ Fixed. > > > +++ b/util/qemu-thread-win32.c > > > @@ -339,6 +341,9 @@ static unsigned __stdcall win32_start_routine(void *arg) > > void *(*start_routine)(void *) = data->start_routine; > > void *thread_arg = data->arg; > > > > +/* Inherit the cur_mon pointer from father thread */ > > +cur_mon = data->current_monitor; > > Otherwise makes sense to me. > > I agree with your analysis that the set of existing OOB commands (just > 'x-oob-test') has no direct use of cur_mon. I'm a little fuzzier on > whether the OOB changes can cause cur_mon to be modified by two threads > in parallel (monitor_qmp_dispatch_one() is futzing around with 'cur_mon' > around the call to qmp_dispatch(), and at least > qmp_human_monitor_command() is also futzing around with it; is there a > case where handling qmp_human_monitor_command() in the dispatch thread > in parallel with more input on the main thread could break?) Thus I'm > not sure whether this is needed for 2.12 to avoid a regression. Could I ask what's the "more input on the main thread"? AFAIU, if we don't take x-oob-test into account, then still only the main thread will touch (not only modify, even read) the cur_mon variable. And as long as that's true, we are keeping the old behavior as when we are without Out-Of-Band, then IMHO we'll be fine. qmp_human_monitor_command() is only one QMP command handler, now it can only be run within main thread. So even we can have the stack like monitor_qmp_dispatch_one (which modifies cur_mon) calls qmp_human_monitor_command (which modifies cur_mon again), still we'll be fine too as long as they are in the same thread, just like before. That's why I think it's not a 2.12 regression. We just need to be prepared for what might come. Thanks, -- Peter Xu
Re: [Qemu-devel] [PATCH 2/2] qemu-thread: let cur_mon be per-thread
On Tue, Apr 10, 2018 at 08:49:13PM +0800, Peter Xu wrote: > cur_mon was only used in main loop so we don't really need that to be > per-thread variable. Now it's possible that we have more than one > thread to operate on it. Let's start to let it be per-thread variable. Trying to understand the reason for this patch: Are there any users of per-thread cur_mon? or Does this patch fix a bug? > In case we'll create threads within a valid cur_mon setup, we'd better > let the child threads to inherit the cur_mon from parent thread too. Do > that for both posix and win32 threads. Without actual users I don't like this. It sounds like "let's make it global just in case something needs it some day". It's ugly for QEMU's thread API to know about the monitor - that's a layering violation. If there's a legitimate need I think this patch might be necessary, but I don't see enough justification to do this yet. Stefan signature.asc Description: PGP signature
Re: [Qemu-devel] [PATCH 2/2] qemu-thread: let cur_mon be per-thread
On 04/10/2018 07:49 AM, Peter Xu wrote: > cur_mon was only used in main loop so we don't really need that to be > per-thread variable. Now it's possible that we have more than one > thread to operate on it. Let's start to let it be per-thread variable. > > In case we'll create threads within a valid cur_mon setup, we'd better > let the child threads to inherit the cur_mon from parent thread too. Do > that for both posix and win32 threads. > > Signed-off-by: Peter Xu > --- > include/monitor/monitor.h | 2 +- > include/qemu/thread-win32.h | 1 + > monitor.c | 2 +- > stubs/monitor.c | 2 +- > tests/test-util-sockets.c | 2 +- > util/qemu-thread-posix.c| 6 ++ > util/qemu-thread-win32.c| 6 ++ > 7 files changed, 17 insertions(+), 4 deletions(-) > > @@ -494,6 +496,9 @@ static void *qemu_thread_start(void *args) > void *(*start_routine)(void *) = qemu_thread_args->start_routine; > void *arg = qemu_thread_args->arg; > > +/* Inherit the cur_mon pointer from father thread */ More typical as s/father/parent/ > +++ b/util/qemu-thread-win32.c > @@ -339,6 +341,9 @@ static unsigned __stdcall win32_start_routine(void *arg) > void *(*start_routine)(void *) = data->start_routine; > void *thread_arg = data->arg; > > +/* Inherit the cur_mon pointer from father thread */ > +cur_mon = data->current_monitor; Otherwise makes sense to me. I agree with your analysis that the set of existing OOB commands (just 'x-oob-test') has no direct use of cur_mon. I'm a little fuzzier on whether the OOB changes can cause cur_mon to be modified by two threads in parallel (monitor_qmp_dispatch_one() is futzing around with 'cur_mon' around the call to qmp_dispatch(), and at least qmp_human_monitor_command() is also futzing around with it; is there a case where handling qmp_human_monitor_command() in the dispatch thread in parallel with more input on the main thread could break?) Thus I'm not sure whether this is needed for 2.12 to avoid a regression. -- Eric Blake, Principal Software Engineer Red Hat, Inc. +1-919-301-3266 Virtualization: qemu.org | libvirt.org signature.asc Description: OpenPGP digital signature
[Qemu-devel] [PATCH 2/2] qemu-thread: let cur_mon be per-thread
cur_mon was only used in main loop so we don't really need that to be
per-thread variable. Now it's possible that we have more than one
thread to operate on it. Let's start to let it be per-thread variable.
In case we'll create threads within a valid cur_mon setup, we'd better
let the child threads to inherit the cur_mon from parent thread too. Do
that for both posix and win32 threads.
Signed-off-by: Peter Xu
---
include/monitor/monitor.h | 2 +-
include/qemu/thread-win32.h | 1 +
monitor.c | 2 +-
stubs/monitor.c | 2 +-
tests/test-util-sockets.c | 2 +-
util/qemu-thread-posix.c| 6 ++
util/qemu-thread-win32.c| 6 ++
7 files changed, 17 insertions(+), 4 deletions(-)
diff --git a/include/monitor/monitor.h b/include/monitor/monitor.h
index d6ab70cae2..2ef5e04b37 100644
--- a/include/monitor/monitor.h
+++ b/include/monitor/monitor.h
@@ -6,7 +6,7 @@
#include "qapi/qapi-types-misc.h"
#include "qemu/readline.h"
-extern Monitor *cur_mon;
+extern __thread Monitor *cur_mon;
/* flags for monitor_init */
/* 0x01 unused */
diff --git a/include/qemu/thread-win32.h b/include/qemu/thread-win32.h
index 3a05e3b3aa..f4d4cd96a1 100644
--- a/include/qemu/thread-win32.h
+++ b/include/qemu/thread-win32.h
@@ -39,6 +39,7 @@ typedef struct QemuThreadData QemuThreadData;
struct QemuThread {
QemuThreadData *data;
unsigned tid;
+Monitor *current_monitor;
};
/* Only valid for joinable threads. */
diff --git a/monitor.c b/monitor.c
index 51f4cf480f..5035e42364 100644
--- a/monitor.c
+++ b/monitor.c
@@ -266,7 +266,7 @@ static mon_cmd_t info_cmds[];
QmpCommandList qmp_commands, qmp_cap_negotiation_commands;
-Monitor *cur_mon;
+__thread Monitor *cur_mon;
static QEMUClockType event_clock_type = QEMU_CLOCK_REALTIME;
diff --git a/stubs/monitor.c b/stubs/monitor.c
index e018c8f594..3890771bb5 100644
--- a/stubs/monitor.c
+++ b/stubs/monitor.c
@@ -3,7 +3,7 @@
#include "qemu-common.h"
#include "monitor/monitor.h"
-Monitor *cur_mon = NULL;
+__thread Monitor *cur_mon;
int monitor_get_fd(Monitor *mon, const char *name, Error **errp)
{
diff --git a/tests/test-util-sockets.c b/tests/test-util-sockets.c
index acadd85e8f..6195a3ac36 100644
--- a/tests/test-util-sockets.c
+++ b/tests/test-util-sockets.c
@@ -69,7 +69,7 @@ int monitor_get_fd(Monitor *mon, const char *fdname, Error
**errp)
* stubs/monitor.c is defined, to make sure monitor.o is discarded
* otherwise we get duplicate syms at link time.
*/
-Monitor *cur_mon;
+__thread Monitor *cur_mon;
void monitor_init(Chardev *chr, int flags) {}
diff --git a/util/qemu-thread-posix.c b/util/qemu-thread-posix.c
index 3ae96210d6..8d13da1b09 100644
--- a/util/qemu-thread-posix.c
+++ b/util/qemu-thread-posix.c
@@ -14,6 +14,7 @@
#include "qemu/thread.h"
#include "qemu/atomic.h"
#include "qemu/notify.h"
+#include "monitor/monitor.h"
#include "trace.h"
static bool name_threads;
@@ -486,6 +487,7 @@ typedef struct {
void *(*start_routine)(void *);
void *arg;
char *name;
+Monitor *current_monitor;
} QemuThreadArgs;
static void *qemu_thread_start(void *args)
@@ -494,6 +496,9 @@ static void *qemu_thread_start(void *args)
void *(*start_routine)(void *) = qemu_thread_args->start_routine;
void *arg = qemu_thread_args->arg;
+/* Inherit the cur_mon pointer from father thread */
+cur_mon = qemu_thread_args->current_monitor;
+
/* Attempt to set the threads name; note that this is for debug, so
* we're not going to fail if we can't set it.
*/
@@ -533,6 +538,7 @@ void qemu_thread_create(QemuThread *thread, const char
*name,
qemu_thread_args->name = g_strdup(name);
qemu_thread_args->start_routine = start_routine;
qemu_thread_args->arg = arg;
+qemu_thread_args->current_monitor = cur_mon;
err = pthread_create(&thread->thread, &attr,
qemu_thread_start, qemu_thread_args);
diff --git a/util/qemu-thread-win32.c b/util/qemu-thread-win32.c
index ab60c0d557..b5197dbc78 100644
--- a/util/qemu-thread-win32.c
+++ b/util/qemu-thread-win32.c
@@ -19,6 +19,7 @@
#include "qemu-common.h"
#include "qemu/thread.h"
#include "qemu/notify.h"
+#include "monitor/monitor.h"
#include "trace.h"
#include
@@ -298,6 +299,7 @@ struct QemuThreadData {
void *arg;
short mode;
NotifierList exit;
+Monitor *current_monitor;
/* Only used for joinable threads. */
bool exited;
@@ -339,6 +341,9 @@ static unsigned __stdcall win32_start_routine(void *arg)
void *(*start_routine)(void *) = data->start_routine;
void *thread_arg = data->arg;
+/* Inherit the cur_mon pointer from father thread */
+cur_mon = data->current_monitor;
+
qemu_thread_data = data;
qemu_thread_exit(start_routine(thread_arg));
abort();
@@ -401,6 +406,7 @@ void qemu_thread_create(QemuThread *thread, const char
*name,
data->arg = arg;
data
