Re: [Qemu-devel] [PATCH v6 0/5] Coroutines for better asynchronous programming
On Sat, Jul 16, 2011 at 11:08 AM, Blue Swirl blauwir...@gmail.com wrote: On Mon, Jun 27, 2011 at 12:21 PM, Stefan Hajnoczi stefa...@gmail.com wrote: Here is the latest version of the coroutine series with Mac OS X fixes from Andreas Färber andreas.faer...@web.de. QEMU is event-driven and suffers when blocking operations are performed because VM execution may be stopped until the operation completes. Therefore many operations that could block are performed asynchronously and a callback is invoked when the operation has completed. This allows QEMU to continue executing while the operation is pending. The downside to callbacks is that they split up code into many smaller functions, each of which is a single step in a state machine that quickly becomes complex and hard to understand. Callback functions also result in lots of noise as variables are packed and unpacked into temporary structs that pass state to the callback function. This patch series introduces coroutines as a solution for writing asynchronous code while still having a nice sequential control flow. The semantics are explained in the second patch. The fourth patch adds automated tests. A nice feature of coroutines is that it is relatively easy to take synchronous code and lift it into a coroutine to make it asynchronous. Work has been done to move qcow2 request processing into coroutines and thereby make it asynchronous (today qcow2 will perform synchronous metadata accesses). This qcow2 work is still ongoing and not quite ready for mainline yet. Coroutines are also being used for virtfs (virtio-9p) so I have submitted this patch now because virtfs patches that depend on coroutines are being published. Coroutines seem to work fine on OpenBSD (Sparc64, also tests are passed) and mingw32, with a few minor issues. With simpletrace enabled there is a link failure: $ make test-coroutine LINK test-coroutine simpletrace.o: In function `get_clock': /src/qemu/qemu-timer.h:117: undefined reference to `use_rt_clock' This can be fixed for example by adding qemu-timer-common.o to the list of linked objects. Do you want me to send a patch to add qemu-timer-common.o or is this easy to do while merging? I can't compile coroutine tests for Mingw32 because of this: $ make test-coroutine CC test-coroutine.o In file included from /usr/local/i686-mingw32msvc/include/glib-2.0/glib/galloca.h:30:0, from /usr/local/i686-mingw32msvc/include/glib-2.0/glib.h:30, from /src/qemu/test-coroutine.c:14: /usr/local/i686-mingw32msvc/include/glib-2.0/glib/gtypes.h:30:24: fatal error: glibconfig.h: No such file or directory I used glib win32 zips from http://ftp.gnome.org/pub/gnome/binaries/win32/glib/2.12/, is there a better site? Otherwise QEMU builds fine and runs under Wine nicely. Paolo, Stefan Weil: Any ideas regarding w32 glib headers above? Stefan
Re: [Qemu-devel] [PATCH v6 0/5] Coroutines for better asynchronous programming
On Mon, Jun 27, 2011 at 12:21 PM, Stefan Hajnoczi stefa...@gmail.com wrote: Here is the latest version of the coroutine series with Mac OS X fixes from Andreas Färber andreas.faer...@web.de. QEMU is event-driven and suffers when blocking operations are performed because VM execution may be stopped until the operation completes. Therefore many operations that could block are performed asynchronously and a callback is invoked when the operation has completed. This allows QEMU to continue executing while the operation is pending. The downside to callbacks is that they split up code into many smaller functions, each of which is a single step in a state machine that quickly becomes complex and hard to understand. Callback functions also result in lots of noise as variables are packed and unpacked into temporary structs that pass state to the callback function. This patch series introduces coroutines as a solution for writing asynchronous code while still having a nice sequential control flow. The semantics are explained in the second patch. The fourth patch adds automated tests. A nice feature of coroutines is that it is relatively easy to take synchronous code and lift it into a coroutine to make it asynchronous. Work has been done to move qcow2 request processing into coroutines and thereby make it asynchronous (today qcow2 will perform synchronous metadata accesses). This qcow2 work is still ongoing and not quite ready for mainline yet. Coroutines are also being used for virtfs (virtio-9p) so I have submitted this patch now because virtfs patches that depend on coroutines are being published. Coroutines seem to work fine on OpenBSD (Sparc64, also tests are passed) and mingw32, with a few minor issues. With simpletrace enabled there is a link failure: $ make test-coroutine LINK test-coroutine simpletrace.o: In function `get_clock': /src/qemu/qemu-timer.h:117: undefined reference to `use_rt_clock' This can be fixed for example by adding qemu-timer-common.o to the list of linked objects. I can't compile coroutine tests for Mingw32 because of this: $ make test-coroutine CCtest-coroutine.o In file included from /usr/local/i686-mingw32msvc/include/glib-2.0/glib/galloca.h:30:0, from /usr/local/i686-mingw32msvc/include/glib-2.0/glib.h:30, from /src/qemu/test-coroutine.c:14: /usr/local/i686-mingw32msvc/include/glib-2.0/glib/gtypes.h:30:24: fatal error: glibconfig.h: No such file or directory I used glib win32 zips from http://ftp.gnome.org/pub/gnome/binaries/win32/glib/2.12/, is there a better site? Otherwise QEMU builds fine and runs under Wine nicely. v6: * Use GThread on Mac OS X, fix from Andreas Färber andreas.faer...@web.de * abort(3) if coroutine-ucontext.c fails to create a coroutine v5: * GThread-based implementation for platforms without makecontext(3) (Aneesh Kumar K.V aneesh.ku...@linux.vnet.ibm.com) * Switch to gtester test framework v4: * Windows Fibers support (Paolo Bonzini pbonz...@redhat.com) * Return-after-setjmp() fix (Aneesh Kumar K.V aneesh.ku...@linux.vnet.ibm.com) * Re-entrancy for multi-threaded coroutines support * qemu-coroutine.h cleanup and documentation v3: * Updated LGPL v2 license header to use web link * Removed atexit(3) pool freeing * Removed thread-local current/leader * Documented thread-safety limitation * Disabled trace events v2: * Added ./check-coroutine --lifecycle-benchmark for performance measurement * Split pooling into a separate patch with performance justification * Set maximum pool size to prevent holding onto too many free coroutines * Added atexit(3) handler to free pool * Coding style cleanups Aneesh Kumar K.V (1): coroutine: implement coroutines using gthread Anthony Liguori (1): Add hard build dependency on glib Kevin Wolf (1): coroutine: introduce coroutines Stefan Hajnoczi (2): coroutine: add test-coroutine automated tests coroutine: add test-coroutine --benchmark-lifecycle .gitignore | 1 + Makefile | 5 +- Makefile.objs | 13 +++ Makefile.target | 1 + configure | 31 +++ coroutine-gthread.c | 131 coroutine-ucontext.c | 230 ++ coroutine-win32.c | 92 qemu-coroutine-int.h | 48 +++ qemu-coroutine.c | 75 qemu-coroutine.h | 95 + test-coroutine.c | 192 + trace-events | 5 + 13 files changed, 918 insertions(+), 1 deletions(-) create mode 100644 coroutine-gthread.c create mode 100644 coroutine-ucontext.c create mode 100644 coroutine-win32.c create mode 100644 qemu-coroutine-int.h create mode 100644 qemu-coroutine.c create mode 100644 qemu-coroutine.h create mode 100644 test-coroutine.c --
[Qemu-devel] [PATCH v6 0/5] Coroutines for better asynchronous programming
Here is the latest version of the coroutine series with Mac OS X fixes from Andreas Färber andreas.faer...@web.de. QEMU is event-driven and suffers when blocking operations are performed because VM execution may be stopped until the operation completes. Therefore many operations that could block are performed asynchronously and a callback is invoked when the operation has completed. This allows QEMU to continue executing while the operation is pending. The downside to callbacks is that they split up code into many smaller functions, each of which is a single step in a state machine that quickly becomes complex and hard to understand. Callback functions also result in lots of noise as variables are packed and unpacked into temporary structs that pass state to the callback function. This patch series introduces coroutines as a solution for writing asynchronous code while still having a nice sequential control flow. The semantics are explained in the second patch. The fourth patch adds automated tests. A nice feature of coroutines is that it is relatively easy to take synchronous code and lift it into a coroutine to make it asynchronous. Work has been done to move qcow2 request processing into coroutines and thereby make it asynchronous (today qcow2 will perform synchronous metadata accesses). This qcow2 work is still ongoing and not quite ready for mainline yet. Coroutines are also being used for virtfs (virtio-9p) so I have submitted this patch now because virtfs patches that depend on coroutines are being published. v6: * Use GThread on Mac OS X, fix from Andreas Färber andreas.faer...@web.de * abort(3) if coroutine-ucontext.c fails to create a coroutine v5: * GThread-based implementation for platforms without makecontext(3) (Aneesh Kumar K.V aneesh.ku...@linux.vnet.ibm.com) * Switch to gtester test framework v4: * Windows Fibers support (Paolo Bonzini pbonz...@redhat.com) * Return-after-setjmp() fix (Aneesh Kumar K.V aneesh.ku...@linux.vnet.ibm.com) * Re-entrancy for multi-threaded coroutines support * qemu-coroutine.h cleanup and documentation v3: * Updated LGPL v2 license header to use web link * Removed atexit(3) pool freeing * Removed thread-local current/leader * Documented thread-safety limitation * Disabled trace events v2: * Added ./check-coroutine --lifecycle-benchmark for performance measurement * Split pooling into a separate patch with performance justification * Set maximum pool size to prevent holding onto too many free coroutines * Added atexit(3) handler to free pool * Coding style cleanups Aneesh Kumar K.V (1): coroutine: implement coroutines using gthread Anthony Liguori (1): Add hard build dependency on glib Kevin Wolf (1): coroutine: introduce coroutines Stefan Hajnoczi (2): coroutine: add test-coroutine automated tests coroutine: add test-coroutine --benchmark-lifecycle .gitignore |1 + Makefile |5 +- Makefile.objs| 13 +++ Makefile.target |1 + configure| 31 +++ coroutine-gthread.c | 131 coroutine-ucontext.c | 230 ++ coroutine-win32.c| 92 qemu-coroutine-int.h | 48 +++ qemu-coroutine.c | 75 qemu-coroutine.h | 95 + test-coroutine.c | 192 + trace-events |5 + 13 files changed, 918 insertions(+), 1 deletions(-) create mode 100644 coroutine-gthread.c create mode 100644 coroutine-ucontext.c create mode 100644 coroutine-win32.c create mode 100644 qemu-coroutine-int.h create mode 100644 qemu-coroutine.c create mode 100644 qemu-coroutine.h create mode 100644 test-coroutine.c -- 1.7.5.4
