Re: [Qemu-devel] ResettRe: [Xen-devel] [v5][PATCH 0/5] xen: add Intel IGD passthrough support

2014-07-04 Thread Paolo Bonzini
Il 03/07/2014 21:09, Jesse Barnes ha scritto: Practically speaking, we could probably assume specific CPU/PCH combos, as I don't think they're generally mixed across generations, though SNB and IVB did have compatible sockets, so there is the possibility of mixing CPT and PPT PCHs, but those are

Re: [Qemu-devel] [PATCH 0/2] coroutine: dynamically scale pool size

2014-07-04 Thread Paolo Bonzini
Il 04/07/2014 07:10, Ming Lei ha scritto: With both the two coroutine patches and the block plugunplug patches, performance of qemu.git/master virtio-blk dataplane can recover to level of QEMU 2.0. So the excessive writes (to eventfd and irqfd) are not a problem? That'd be a relief. :)

Re: [Qemu-devel] [PATCH v1 1/1] char: cadence_uart: Convert to realize()

2014-07-04 Thread Alexander Graf
On 03.07.14 18:59, Andreas Färber wrote: Am 03.07.2014 18:46, schrieb Peter Maydell: On 24 June 2014 07:06, Alistair Francis alistair.fran...@xilinx.com wrote: SysBusDevice::init is deprecated. Convert to Object::init and Note that there is no Object::init, only TypeInfo::instance_init.

Re: [Qemu-devel] [PATCH for-2.1?!?] AioContext: speed up aio_notify

2014-07-04 Thread Paolo Bonzini
Il 03/07/2014 20:17, Stefan Hajnoczi ha scritto: On Thu, Jul 3, 2014 at 6:59 PM, Paolo Bonzini pbonz...@redhat.com wrote: Looks like a very interesting optimization for the block layer. If we can get some performance results - especially how it improves the virtio-blk dataplane regression -

Re: [Qemu-devel] [PATCH 0/2] coroutine: dynamically scale pool size

2014-07-04 Thread Ming Lei
On Fri, Jul 4, 2014 at 2:31 PM, Paolo Bonzini pbonz...@redhat.com wrote: Il 04/07/2014 07:10, Ming Lei ha scritto: With both the two coroutine patches and the block plugunplug patches, performance of qemu.git/master virtio-blk dataplane can recover to level of QEMU 2.0. So the excessive

Re: [Qemu-devel] [RFC PATCH 00/11] Adding FreeBSD's Capsicum security framework (part 1)

2014-07-04 Thread Paolo Bonzini
Il 03/07/2014 20:39, David Drysdale ha scritto: On Thu, Jul 03, 2014 at 11:12:33AM +0200, Paolo Bonzini wrote: Given Linux's previous experience with BPF filters, what do you think about attaching specific BPF programs to file descriptors? Then whenever a syscall is run that affects a file

Re: [Qemu-devel] [PATCH 0/2] coroutine: dynamically scale pool size

2014-07-04 Thread Paolo Bonzini
Il 04/07/2014 09:02, Ming Lei ha scritto: So the excessive writes (to eventfd and irqfd) are not a problem? That'd be a relief. :) I mean it is in a level, but your aio_notify() patch still can improve virtioblk dataplane performance some, in my test, with 5~10K IOPS improvement, which should

Re: [Qemu-devel] [PATCH for-2.1?!?] AioContext: speed up aio_notify

2014-07-04 Thread Ming Lei
On Fri, Jul 4, 2014 at 2:51 PM, Paolo Bonzini pbonz...@redhat.com wrote: Il 03/07/2014 20:17, Stefan Hajnoczi ha scritto: On Thu, Jul 3, 2014 at 6:59 PM, Paolo Bonzini pbonz...@redhat.com wrote: Looks like a very interesting optimization for the block layer. If we can get some performance

Re: [Qemu-devel] [PATCH for-2.1?!?] AioContext: speed up aio_notify

2014-07-04 Thread Paolo Bonzini
Il 04/07/2014 09:23, Ming Lei ha scritto: I think it is good and better to go to 2.1, and it should save lots of write syscall. Also should regression be caused, the per thread trick may be resorted to, which should be simple. If we have the right solution (which we do, unlike the plug/unplug

Re: [Qemu-devel] [RFC] alpha qemu arithmetic exceptions

2014-07-04 Thread Al Viro
On Thu, Jul 03, 2014 at 09:30:05PM -0700, Richard Henderson wrote: Another one is probably not worth bothering - PERR, CTPOP, CTLZ, UNPKBx and PKxB don't accept literal argument. For one thing, as(1) won't let you generate those, so it would have to be explicit .long 0x70001620

Re: [Qemu-devel] [RFC] icount: warp in the main_loop.

2014-07-04 Thread Frederic Konrad
On 01/07/2014 18:13, fred.kon...@greensocs.com wrote: From: KONRAD Frederic fred.kon...@greensocs.com This fixes a bug where QEMU stall in icount mode. It happens when a simple timer callback is created on VIRTUAL CLOCK modding itself regularly. The actual warping mechanism is called once and

Re: [Qemu-devel] [PATCH for 2.1 2/2] memory-backend-file: improve error handling

2014-07-04 Thread Hu Tao
On Thu, Jul 03, 2014 at 10:05:56PM -0600, Eric Blake wrote: On 07/03/2014 08:56 PM, Hu Tao wrote: On Thu, Jul 03, 2014 at 06:33:28AM -0600, Eric Blake wrote: On 07/03/2014 12:10 AM, Hu Tao wrote: This patch fixes two problems of memory-backend-file: 1. If user adds a memory-backend-file

Re: [Qemu-devel] [PATCH for 2.1 2/2] memory-backend-file: improve error handling

2014-07-04 Thread Paolo Bonzini
Il 04/07/2014 09:43, Hu Tao ha scritto: The memory size is rounded up to hpagesize when allocating memory, we can waste at most hpagesize-1 bytes memory. I don't think it's a problem. In the other side, do you think we should give an error in the case like -object

Re: [Qemu-devel] [PATCH 0/2] coroutine: dynamically scale pool size

2014-07-04 Thread Stefan Hajnoczi
On Fri, Jul 04, 2014 at 01:10:39PM +0800, Ming Lei wrote: On Thu, Jul 3, 2014 at 9:51 PM, Stefan Hajnoczi stefa...@redhat.com wrote: The coroutine pool reuses exited coroutines to make qemu_coroutine_create() cheap. The size of the pool is capped to prevent it from hogging memory after

Re: [Qemu-devel] [RFC] icount: warp in the main_loop.

2014-07-04 Thread Paolo Bonzini
Il 04/07/2014 09:30, Frederic Konrad ha scritto: +/* + * In icount mode, sometimes the VCPU is blocked and an event is needed to + * continue. + * Just warp to make the time grows and have a chance to run the CPU. + */ +qemu_clock_warp(QEMU_CLOCK_VIRTUAL); return

Re: [Qemu-devel] from which version qemu support clone on rbd

2014-07-04 Thread Stefan Hajnoczi
On Wed, Jul 02, 2014 at 11:17:40PM +0800, yue wrote: i now look at qemu 2.0, i do not find rbd-api related to clone. if qemu support this function? and from which version? clone api of rbd is very simple(one api), why qemu does not implement?what is the reason? rbd_snap_create()? QEMU

Re: [Qemu-devel] [RFC] COLO HA Project proposal

2014-07-04 Thread Dr. David Alan Gilbert
* Dong, Eddie (eddie.d...@intel.com) wrote: I didn't quite understand a couple of things though, perhaps you can explain: 1) If we ignore the TCP sequence number problem, in an SMP machine don't we get other randomnesses - e.g. which core completes something first, or who wins

Re: [Qemu-devel] [PATCH v5 2/3] linux-aio: implement io plug, unplug and flush io queue

2014-07-04 Thread Stefan Hajnoczi
On Fri, Jul 04, 2014 at 11:06:41AM +0800, Ming Lei wrote: +static int ioq_submit(struct qemu_laio_state *s) +{ +int ret, i = 0; +int len = s-io_q.idx; + +do { +ret = io_submit(s-ctx, len, s-io_q.iocbs); +} while (i++ 3 ret == -EAGAIN); + +/* empty io queue

[Qemu-devel] [PATCH v2 2/2] block: bump coroutine pool size for drives

2014-07-04 Thread Stefan Hajnoczi
When a BlockDriverState is associated with a storage controller DeviceState we expect guest I/O. Use this opportunity to bump the coroutine pool size by 64. This patch ensures that the coroutine pool size scales with the number of drives attached to the guest. It should increase coroutine pool

[Qemu-devel] [PATCH v2 1/2] coroutine: make pool size dynamic

2014-07-04 Thread Stefan Hajnoczi
Allow coroutine users to adjust the pool size. For example, if the guest has multiple emulated disk drives we should keep around more coroutines. Signed-off-by: Stefan Hajnoczi stefa...@redhat.com --- include/block/coroutine.h | 11 +++ qemu-coroutine.c | 26

[Qemu-devel] [PATCH v2 0/2] coroutine: dynamically scale pool size

2014-07-04 Thread Stefan Hajnoczi
v2: * Assert that callers never reduce pool below default size [eblake] The coroutine pool reuses exited coroutines to make qemu_coroutine_create() cheap. The size of the pool is capped to prevent it from hogging memory after a period of high coroutine activity. Previously the max size was

Re: [Qemu-devel] [PATCH v4 2/3] linux-aio: implement io plug, unplug and flush io queue

2014-07-04 Thread Ming Lei
Hi Stefan, Sorry for missing your comments. On Thu, Jul 3, 2014 at 8:24 PM, Stefan Hajnoczi stefa...@gmail.com wrote: On Wed, Jul 02, 2014 at 08:18:47PM +0800, Ming Lei wrote: @@ -36,9 +38,19 @@ struct qemu_laiocb { QLIST_ENTRY(qemu_laiocb) node; }; +struct laio_queue { This file

Re: [Qemu-devel] [PATCH v4 2/3] linux-aio: implement io plug, unplug and flush io queue

2014-07-04 Thread Ming Lei
On Fri, Jul 4, 2014 at 5:18 PM, Ming Lei ming@canonical.com wrote: This leaks requests when ret len. I think the loop below should be used in that case to fail unsubmitted requests with -EIO. I thought about the problem before, but looks it may not return 'ret' which is less than len,

Re: [Qemu-devel] Help debugging audio problem

2014-07-04 Thread Anshul Makkar
Use of glue is heavily uses in audio code. I completely redesigned it for Virtualbox and removed all the hard to understand glue code :) . Not sure if this glue magic is such heavily used anywhere else also. Moreover audio code uses one big monolythic big audio file audio.c . So bringing

[Qemu-devel] [PATCH] virtio-pci: fix MSI memory region use after tree

2014-07-04 Thread Paolo Bonzini
After memory region QOMification QEMU is stricter in detecting wrong usage of the memory region API. Here it detected a memory_region_destroy done before the corresponding memory_region_del_subregion; the memory_region_destroy is done by msix_uninit_exclusive_bar, the memory_region_del_subregion

Re: [Qemu-devel] [PATCH v2 2/2] block: bump coroutine pool size for drives

2014-07-04 Thread Markus Armbruster
Stefan Hajnoczi stefa...@redhat.com writes: When a BlockDriverState is associated with a storage controller DeviceState we expect guest I/O. Use this opportunity to bump the coroutine pool size by 64. This patch ensures that the coroutine pool size scales with the number of drives attached

[Qemu-devel] [PATCH v6 0/3] linux-aio: introduce submit I/O as a batch

2014-07-04 Thread Ming Lei
Hi, The commit 580b6b2aa2(dataplane: use the QEMU block layer for I/O) introduces ~40% throughput regression on virtio-blk dataplane, and one of causes is that submitting I/O as a batch is removed. This patchset trys to introduce this mechanism on block, at least, linux-aio can benefit from

[Qemu-devel] [PATCH v6 1/3] block: block: introduce APIs for submitting IO as a batch

2014-07-04 Thread Ming Lei
This patch introduces three APIs so that following patches can support queuing I/O requests and submitting them as a batch for improving I/O performance. Reviewed-by: Paolo Bonzini pbonz...@redhat.com Signed-off-by: Ming Lei ming@canonical.com --- block.c | 31

[Qemu-devel] [PATCH v6 3/3] dataplane: submit I/O as a batch

2014-07-04 Thread Ming Lei
Before commit 580b6b2aa2(dataplane: use the QEMU block layer for I/O), dataplane for virtio-blk submits block I/O as a batch. This commit 580b6b2aa2 replaces the custom linux AIO implementation(including submit I/O as a batch) with QEMU block layer, but this commit causes ~40% throughput

[Qemu-devel] [PATCH v6 2/3] linux-aio: implement io plug, unplug and flush io queue

2014-07-04 Thread Ming Lei
This patch implements .bdrv_io_plug, .bdrv_io_unplug and .bdrv_flush_io_queue callbacks for linux-aio Block Drivers, so that submitting I/O as a batch can be supported on linux-aio. Signed-off-by: Ming Lei ming@canonical.com --- block/linux-aio.c | 96

Re: [Qemu-devel] [PATCH for-2.1?!?] AioContext: speed up aio_notify

2014-07-04 Thread Kevin Wolf
Am 04.07.2014 um 09:27 hat Paolo Bonzini geschrieben: Il 04/07/2014 09:23, Ming Lei ha scritto: I think it is good and better to go to 2.1, and it should save lots of write syscall. Also should regression be caused, the per thread trick may be resorted to, which should be simple. If we

Re: [Qemu-devel] [RFC] icount: warp in the main_loop.

2014-07-04 Thread Frederic Konrad
On 04/07/2014 09:57, Paolo Bonzini wrote: Il 04/07/2014 09:30, Frederic Konrad ha scritto: +/* + * In icount mode, sometimes the VCPU is blocked and an event is needed to + * continue. + * Just warp to make the time grows and have a chance to run the CPU. + */ +

Re: [Qemu-devel] [PATCH v2 2/2] block: bump coroutine pool size for drives

2014-07-04 Thread Lluís Vilanova
Stefan Hajnoczi writes: When a BlockDriverState is associated with a storage controller DeviceState we expect guest I/O. Use this opportunity to bump the coroutine pool size by 64. This patch ensures that the coroutine pool size scales with the number of drives attached to the guest. It

Re: [Qemu-devel] [RFC] icount: warp in the main_loop.

2014-07-04 Thread Paolo Bonzini
Right, but when I put a timer eg on QEMU_VIRTUAL_CLOCK the guest is stuck. icount_warp_rt is not called neither qemu_clock_warp(..).. It should be. timer_mod_ns - timerlist_rearm - qemu_clock_warp. So yes as you said seems a qemu_clock_warp is missing somewhere. Shouldn't icount_warp_rt

Re: [Qemu-devel] [PATCH for-2.1?!?] AioContext: speed up aio_notify

2014-07-04 Thread Paolo Bonzini
We have a rather long freeze phase this time (which I think is a good thing). This patch fixes a regression, even if it may not be the most important one because it is in experimental code. But I still think that this time in the hard freeze is the right time to commit patches like this. I

Re: [Qemu-devel] [RFC] COLO HA Project proposal

2014-07-04 Thread Andreas Färber
Am 01.07.2014 14:12, schrieb Dr. David Alan Gilbert: Are there any standard APIs for the heartbeet thing we can already tie into? Maybe the http://www.linux-ha.org/wiki/Heartbeat daemon? Andreas -- SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany GF: Jeff Hawn, Jennifer

Re: [Qemu-devel] [PATCH 06/10] mm: sys_remap_anon_pages

2014-07-04 Thread Michael Kerrisk
[CC+=linux-api@] Hi Andrea, On Wed, Jul 2, 2014 at 6:50 PM, Andrea Arcangeli aarca...@redhat.com wrote: This new syscall will move anon pages across vmas, atomically and without touching the vmas. Please CC linux-api on patches that change the API/ABI. (See

[Qemu-devel] [PATCH V2 2/6] runner: Tool for fuzz tests execution

2014-07-04 Thread Maria Kustova
The purpose of the test runner is to prepare test environment (e.g. create a work directory, a test image, etc), execute the program under test with parameters, indicate a test failure if the program was killed during test execution and collect core dumps, logs and other test artifacts. The test

[Qemu-devel] [PATCH V2 1/6] docs: Specification for the image fuzzer

2014-07-04 Thread Maria Kustova
'Overall fuzzer requirements' chapter contains the current product vision and features done and to be done. This chapter is still in progress. v1 - v2: * added description for fuzzer configuration files * added the example fuzzer.conf file * various fixes based on the Eric's review

[Qemu-devel] [PATCH V2 3/6] fuzz: Fuzzing functions for qcow2 images

2014-07-04 Thread Maria Kustova
Fuzz submodule of qcow2 image generator contains fuzzing functions for image fields. Each fuzzing function contains list of constraints and call of a helper function that randomly selects a fuzzed value satisfied to one of constraints. For now constraints are only known as invalid or potentially

[Qemu-devel] [PATCH V2 5/6] package: Public API for image-fuzzer/runner/runner.py

2014-07-04 Thread Maria Kustova
__init__.py provides the public API required by the test runner Signed-off-by: Maria Kustova mari...@catit.be --- tests/image-fuzzer/qcow2/__init__.py | 1 + 1 file changed, 1 insertion(+) create mode 100644 tests/image-fuzzer/qcow2/__init__.py diff --git a/tests/image-fuzzer/qcow2/__init__.py

[Qemu-devel] [PATCH V2 0/6] tests: Add the image fuzzer with qcow2 support

2014-07-04 Thread Maria Kustova
This patch series introduces the image fuzzer, a tool for stability and reliability testing. Its approach is to run large amount of tests in background. During every test a program (e.g. qemu-img) is called to read or modify an invalid test image. A test image has valid inner structure defined by

[Qemu-devel] [PATCH V2 6/6] image-fuzzer: GPLv2 license file

2014-07-04 Thread Maria Kustova
Signed-off-by: Maria Kustova mari...@catit.be --- tests/image-fuzzer/LICENSE | 339 + 1 file changed, 339 insertions(+) create mode 100644 tests/image-fuzzer/LICENSE diff --git a/tests/image-fuzzer/LICENSE b/tests/image-fuzzer/LICENSE new file mode

[Qemu-devel] [PATCH V2 4/6] layout: Generator of fuzzed qcow2 images

2014-07-04 Thread Maria Kustova
Layout submodule of qcow2 package creates a random valid image, randomly selects some amount of its fields, fuzzes them and write the fuzzed image to the file. Now only header and header extensions are generated, a remaining file is filled by zeroes. v1 - v2: * Added support of fuzzer

Re: [Qemu-devel] Super nested KVM

2014-07-04 Thread Vasiliy Tolstov
2014-07-04 1:00 GMT+04:00 Richard W.M. Jones rjo...@redhat.com: Well I got bored with the TCG test after about 4 hours. It appears to hang launching the L3 guest, reproducible on two different hosts both running qemu 2.0.0. In my tests qemu tcg sometimes fail when run first vm. (kernel 3.12)

Re: [Qemu-devel] [RFC] COLO HA Project proposal

2014-07-04 Thread Dr. David Alan Gilbert
* Dong, Eddie (eddie.d...@intel.com) wrote: Let me clarify on this issue. COLO didn't ignore the TCP sequence number, but uses a new implementation to make the sequence number to be best effort identical between the primary VM (PVM) and secondary VM (SVM). Likely, VMM has to

[Qemu-devel] [PATCH 2/2] virtio-blk: dataplane: notify guest as a batch

2014-07-04 Thread Ming Lei
Now requests are submitted as a batch, so it is natural to notify guest as a batch too. This may supress interrupt notification to VM: - in my test, decreased by ~13K/sec Signed-off-by: Ming Lei ming@canonical.com --- hw/block/dataplane/virtio-blk.c | 13 -

[Qemu-devel] [PATCH 0/2] virtio-blk: dataplane: one fix plus one optimization

2014-07-04 Thread Ming Lei
Hi, The first one fixes one problem introduced recently. The second one supresses notifications to guest. Thanks, -- Ming Lei

[Qemu-devel] [PATCH 1/2] virtio-blk: data-plane: fix save/set .complete_request in start

2014-07-04 Thread Ming Lei
The callback has to be saved and reset in virtio_blk_data_plane_start(), otherwise dataplane's requests will be completed in qemu aio context. Signed-off-by: Ming Lei ming@canonical.com --- hw/block/dataplane/virtio-blk.c |7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff

Re: [Qemu-devel] Super nested KVM

2014-07-04 Thread Richard W.M. Jones
On Fri, Jul 04, 2014 at 04:17:11PM +0400, Vasiliy Tolstov wrote: 2014-07-04 1:00 GMT+04:00 Richard W.M. Jones rjo...@redhat.com: Well I got bored with the TCG test after about 4 hours. It appears to hang launching the L3 guest, reproducible on two different hosts both running qemu 2.0.0.

[Qemu-devel] [PATCH] qemu-char: add chr_add_watch support in mux chardev

2014-07-04 Thread Kirill Batuzov
Forward chr_add_watch call from mux chardev to underlying implementation. This should fix bug #1335444 Signed-off-by: Kirill Batuzov batuz...@ispras.ru --- qemu-char.c |9 + 1 file changed, 9 insertions(+) diff --git a/qemu-char.c b/qemu-char.c index 51917de..e1bd6f5 100644 ---

Re: [Qemu-devel] [PATCH 2/2] virtio-blk: dataplane: notify guest as a batch

2014-07-04 Thread Paolo Bonzini
Il 04/07/2014 14:27, Ming Lei ha scritto: Now requests are submitted as a batch, so it is natural to notify guest as a batch too. This may supress interrupt notification to VM: - in my test, decreased by ~13K/sec I don't think this can work. Requests are not completed FIFO. What

Re: [Qemu-devel] [PATCH v5 00/10] Clean up around bdrv_getlength()

2014-07-04 Thread Markus Armbruster
Series contains a few bug fixes; you might want to consider it for 2.1. Markus Armbruster arm...@redhat.com writes: Issues addressed in this series: * BlockDriver method bdrv_getlength() generally returns -errno, but some implementations return -1 instead. Fix them [PATCH 1]. * Frequent

Re: [Qemu-devel] [PATCH] qemu-char: add chr_add_watch support in mux chardev

2014-07-04 Thread Paolo Bonzini
Il 04/07/2014 14:43, Kirill Batuzov ha scritto: Forward chr_add_watch call from mux chardev to underlying implementation. This should fix bug #1335444 Signed-off-by: Kirill Batuzov batuz...@ispras.ru --- qemu-char.c |9 + 1 file changed, 9 insertions(+) diff --git a/qemu-char.c

Re: [Qemu-devel] [RFC] COLO HA Project proposal

2014-07-04 Thread Dong, Eddie
I didn't quite understand a couple of things though, perhaps you can explain: 1) If we ignore the TCP sequence number problem, in an SMP machine don't we get other randomnesses - e.g. which core completes something first, or who wins a lock contention, so the output stream might not

Re: [Qemu-devel] [RFC] COLO HA Project proposal

2014-07-04 Thread Dong, Eddie
Let me clarify on this issue. COLO didn't ignore the TCP sequence number, but uses a new implementation to make the sequence number to be best effort identical between the primary VM (PVM) and secondary VM (SVM). Likely, VMM has to synchronize the emulation of randomization number

[Qemu-devel] [PATCH v2 2.1 1/4] virtio-blk: Factor common checks out of virtio_blk_handle_read/write()

2014-07-04 Thread Markus Armbruster
Signed-off-by: Markus Armbruster arm...@redhat.com Reviewed-by: Fam Zheng f...@redhat.com --- hw/block/virtio-blk.c | 24 ++-- 1 file changed, 14 insertions(+), 10 deletions(-) diff --git a/hw/block/virtio-blk.c b/hw/block/virtio-blk.c index aec3146..d946fa9 100644 ---

[Qemu-devel] [PATCH v2 2.1 2/4] virtio-blk: Bypass error action and I/O accounting on invalid r/w

2014-07-04 Thread Markus Armbruster
When a device model's I/O operation fails, we execute the error action. This lets layers above QEMU implement thin provisioning, or attempt to correct errors before they reach the guest. But when the I/O operation fails because it's invalid, reporting the error to the guest is the only sensible

[Qemu-devel] [PATCH v2 2.1 0/4] Suppress error action on r/w beyond end

2014-07-04 Thread Markus Armbruster
When a device model's I/O operation fails, we execute the error action. This lets layers above QEMU implement thin provisioning, or attempt to correct errors before they reach the guest. But when the I/O operation fails because its invalid, reporting the error to the guest is the only sensible

[Qemu-devel] [PATCH v2 2.1 4/4] ide: Treat read/write beyond end as invalid

2014-07-04 Thread Markus Armbruster
The block layer fails such reads and writes just fine. However, they then get treated like valid operations that fail: the error action gets executed. Unwanted; reporting the error to the guest is the only sensible action. Reject them before passing them to the block layer. This bypasses the

[Qemu-devel] [PATCH v2 2.1 3/4] virtio-blk: Treat read/write beyond end as invalid

2014-07-04 Thread Markus Armbruster
The block layer fails such reads and writes just fine. However, they then get treated like valid operations that fail: the error action gets executed. Unwanted; reporting the error to the guest is the only sensible action. Reject them before passing them to the block layer. This bypasses the

[Qemu-devel] [PATCH] watchdog: fix deadlock with -watchdog-action pause

2014-07-04 Thread Paolo Bonzini
qemu_clock_enable says: /* Disabling the clock will wait for related timerlists to stop * executing qemu_run_timers. Thus, this functions should not * be used from the callback of a timer that is based on @clock. * Doing so would cause a deadlock. */ and it indeed does: vm_stop uses

Re: [Qemu-devel] [PATCH] watchdog: fix deadlock with -watchdog-action pause

2014-07-04 Thread Luiz Capitulino
On Fri, 4 Jul 2014 15:48:50 +0200 Paolo Bonzini pbonz...@redhat.com wrote: qemu_clock_enable says: /* Disabling the clock will wait for related timerlists to stop * executing qemu_run_timers. Thus, this functions should not * be used from the callback of a timer that is based on @clock.

Re: [Qemu-devel] [PATCH] Makefile: Don't build generated headers before Makefile is reread

2014-07-04 Thread Peter Maydell
On 30 June 2014 13:09, Paolo Bonzini pbonz...@redhat.com wrote: Il 28/06/2014 18:59, Peter Maydell ha scritto: Having a direct dependency Makefile: $(GENERATED_HEADERS) can result in not-from-clean builds failing sometimes, because it means that when Make does its is any makefile or

Re: [Qemu-devel] [PATCH 2/2] virtio-blk: dataplane: notify guest as a batch

2014-07-04 Thread Ming Lei
On Fri, Jul 4, 2014 at 8:55 PM, Paolo Bonzini pbonz...@redhat.com wrote: Il 04/07/2014 14:27, Ming Lei ha scritto: Now requests are submitted as a batch, so it is natural to notify guest as a batch too. This may supress interrupt notification to VM: - in my test, decreased by

Re: [Qemu-devel] [PATCH RFC V2 6/6] xen: introduce xenpv-softmmu.mak

2014-07-04 Thread Stefano Stabellini
I realize now that patch #5 and #6 of this series feel through the cracks. Could you please rebase and resend? On Mon, 27 Jan 2014, Wei Liu wrote: Signed-off-by: Wei Liu wei.l...@citrix.com --- configure |7 +-- default-configs/xenpv-softmmu.mak |2 ++ 2

[Qemu-devel] Atomic instruction.

2014-07-04 Thread Frederic Konrad
Hi everybody, We are experimenting with multi-core QEMU. We have Multiple QEMU cores running on multiple host threads and we are now looking at the issue of ‘atomic’ instructions. Our initial thought was to push some sort of set of flags out to the memory chain (eg: ATOMIC or NORMAL much as

Re: [Qemu-devel] [RFC PATCH V3 0/6] icount: Implement delay algorithm between guest and host clocks

2014-07-04 Thread Sebastian Tanase
- Mail original - De: Paolo Bonzini pbonz...@redhat.com À: Sebastian Tanase sebastian.tan...@openwide.fr, qemu-devel@nongnu.org Cc: aligu...@amazon.com, afaer...@suse.de, r...@twiddle.net, peter maydell peter.mayd...@linaro.org, mich...@walle.cc, a...@alex.org.uk,

Re: [Qemu-devel] [PATCH] Makefile: Don't build generated headers before Makefile is reread

2014-07-04 Thread Paolo Bonzini
Il 04/07/2014 16:33, Peter Maydell ha scritto: On 30 June 2014 13:09, Paolo Bonzini pbonz...@redhat.com wrote: Il 28/06/2014 18:59, Peter Maydell ha scritto: Having a direct dependency Makefile: $(GENERATED_HEADERS) can result in not-from-clean builds failing sometimes, because it means

Re: [Qemu-devel] [PATCH 2/2] virtio-blk: dataplane: notify guest as a batch

2014-07-04 Thread Paolo Bonzini
Il 04/07/2014 16:52, Ming Lei ha scritto: What you can do is change notify_guest to something like qemu_bh_schedule(req-dev-dataplane-notify_guest_bh); and do the actual notification in the bottom half. This should ensure that multiple notifications are coalesced, but it may also

Re: [Qemu-devel] [RFC PATCH V3 0/6] icount: Implement delay algorithm between guest and host clocks

2014-07-04 Thread Paolo Bonzini
Il 04/07/2014 17:36, Sebastian Tanase ha scritto: I've also made some changes to patch 4 (added Sleep() function call when compiling for Windows and other small modifications); is it ok if I merge them with the existing patch 4 on your icount branch and then rebase the other patches? Sure.

Re: [Qemu-devel] [RFC] COLO HA Project proposal

2014-07-04 Thread Dong, Eddie
Thanks Dave: Whether the randomness value/branch/code path the PVM and SVM may have, It is only a performance issue. COLO never assumes the PVM and SVM has same internal Machine state. From correctness p.o.v, as if the PVM and SVM generate Identical response, we can view the SVM is

[Qemu-devel] [PATCH 0/4] block: Fix qiov sizes

2014-07-04 Thread Kevin Wolf
Some callers use larger qiovs than nb_sectors for their read requests. This is bad because raw-posix uses nb_sectors to allocate a bounce buffer and then copies the whole iov over. This series fixes some more cases (mirroring was already fixed earlier this week) and adds assertions to catch any

[Qemu-devel] [PATCH 4/4] block: Assert qiov length matches request length

2014-07-04 Thread Kevin Wolf
At least raw-posix relies on this because it can allocate bounce buffers based on the request length, but access it using all of the qiov entries later. Signed-off-by: Kevin Wolf kw...@redhat.com --- block.c | 2 ++ block/raw-posix.c | 15 +++ 2 files changed, 13

[Qemu-devel] [PATCH 2/4] qcow2: Make qiov match request size until backing file EOF

2014-07-04 Thread Kevin Wolf
If a qcow2 image has a shorter backing file and a read request to unallocated clusters goes across EOF of the backing file, the backing file sees a shortened request and the rest is filled with zeros. However, the original too long qiov was used with the shortened request. This patch makes the

[Qemu-devel] [PATCH 3/4] qed: Make qiov match request size until backing file EOF

2014-07-04 Thread Kevin Wolf
If a QED image has a shorter backing file and a read request to unallocated clusters goes across EOF of the backing file, the backing file sees a shortened request and the rest is filled with zeros. However, the original too long qiov was used with the shortened request. This patch makes the qiov

[Qemu-devel] [PATCH 1/4] block: Make qiov match the request size until EOF

2014-07-04 Thread Kevin Wolf
If a read request goes across EOF, the block driver sees a shortened request that stops at EOF (the rest is memsetted in block.c), however the original qiov was used for this request. This patch makes the qiov size match the request size, avoiding a potential buffer overflow in raw-posix.

Re: [Qemu-devel] [PATCH 2/2] virtio-blk: dataplane: notify guest as a batch

2014-07-04 Thread Ming Lei
On Fri, Jul 4, 2014 at 11:48 PM, Paolo Bonzini pbonz...@redhat.com wrote: Il 04/07/2014 16:52, Ming Lei ha scritto: What you can do is change notify_guest to something like qemu_bh_schedule(req-dev-dataplane-notify_guest_bh); and do the actual notification in the bottom half. This

Re: [Qemu-devel] Atomic instruction.

2014-07-04 Thread Peter Maydell
On 4 July 2014 16:35, Frederic Konrad fred.kon...@greensocs.com wrote: Hi everybody, We are experimenting with multi-core QEMU. We have Multiple QEMU cores running on multiple host threads and we are now looking at the issue of ‘atomic’ instructions. Our initial thought was to push some

Re: [Qemu-devel] [RFC PATCH V3 0/6] icount: Implement delay algorithm between guest and host clocks

2014-07-04 Thread Michael Tokarev
Can we pretty please take me off the list of recepients. I have nothing to do with this, I receive all list emails already, and where I am now it is quite dificult to sort mail, and it costs me quite some time to do so too. When I'm back I'll concider dropping mails sent to list and Cc'd to me.

Re: [Qemu-devel] [PATCH 2/2] virtio-blk: dataplane: notify guest as a batch

2014-07-04 Thread Paolo Bonzini
Il 04/07/2014 17:57, Ming Lei ha scritto: But we have two cases to consider: - one submitted IO includes requests from multi vq(virtio-blk or virtio-scsi maybe), and each vq has to notify guest - one submitted IO includes requests from multi bs for scsi device The 2nd case should be easy to

[Qemu-devel] [PATCH 00/46] Postcopy implementation

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Hi, This is the 1st cut of my version of postcopy; it is designed for use with the Linux kernel additions recently posted by Andrea Arcangeli here: http://lists.gnu.org/archive/html/qemu-devel/2014-07/msg00525.html The current status is:

[Qemu-devel] [PATCH 26/46] Postcopy page-map-incoming (PMI) structure

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com The PMI holds the state of each page on the incoming side, so that we can tell if the page is missing, already received or there is a request outstanding for it. Signed-off-by: Dr. David Alan Gilbert dgilb...@redhat.com ---

[Qemu-devel] [PATCH 03/46] QEMUSizedBuffer/QEMUFile

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Stefan Berger's patch to create a QEMUFile that goes to a memory buffer; from: http://lists.gnu.org/archive/html/qemu-devel/2013-03/msg05036.html Using the QEMUFile interface, this patch adds support functions for operating on in-memory sized

[Qemu-devel] [PATCH 04/46] improve DPRINTF macros, add to savevm

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Improve the existing DPRINTF macros in migration.c and arch_init by: 1) Making them go to stderr rather than stdout (so you can run with -nographic and redirect your debug to a file) 2) Making them print the ms time with each debug - useful for

[Qemu-devel] [PATCH 07/46] Return path: Open a return path on QEMUFile for sockets

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Postcopy needs a method to send messages from the destination back to the source, this is the 'return path'. Wire it up for 'socket' QEMUFile's using a dup'd fd. Signed-off-by: Dr. David Alan Gilbert dgilb...@redhat.com ---

[Qemu-devel] [PATCH 37/46] Add assertion to check migration_dirty_pages doesn't go -ve; have seen it happen once but not sure why

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Signed-off-by: Dr. David Alan Gilbert dgilb...@redhat.com --- arch_init.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch_init.c b/arch_init.c index c006d21..58eccc1 100644 --- a/arch_init.c +++ b/arch_init.c @@ -439,6 +439,7 @@

[Qemu-devel] [PATCH 08/46] Return path: socket_writev_buffer: Block even on non-blocking fd's

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com The return path uses a non-blocking fd so as not to block waiting for the (possibly broken) destination to finish returning a message, however we still want outbound data to behave in the same way and block. Signed-off-by: Dr. David Alan Gilbert

[Qemu-devel] [PATCH 09/46] Migration commands

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Create QEMU_VM_COMMAND section type for sending commands from source to destination. These commands are not intended to convey guest state but to control the migration process. For use in postcopy. Signed-off-by: Dr. David Alan Gilbert

[Qemu-devel] [PATCH 10/46] Return path: Control commands

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Add two src-dest commands: * OPENRP - To request that the destination open the return path * REQACK - Request an acknowledge from the destination Signed-off-by: Dr. David Alan Gilbert dgilb...@redhat.com --- include/migration/migration.h |

[Qemu-devel] [PATCH 46/46] Start documenting how postcopy works.

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Signed-off-by: Dr. David Alan Gilbert dgilb...@redhat.com --- docs/migration.txt | 148 + 1 file changed, 148 insertions(+) diff --git a/docs/migration.txt b/docs/migration.txt index

[Qemu-devel] [PATCH 44/46] postcopy: Use userfaultfd

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com userfaultfd is a Linux syscall that gives an fd that receives a stream of notifications of accesses to pages marked as MADV_USERFAULT, and allows the program to acknowledge those stalls and tell the accessing thread to carry on. Signed-off-by: Dr.

[Qemu-devel] [PATCH 23/46] MIG_STATE_POSTCOPY_ACTIVE: Add new migration state

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com 'MIG_STATE_POSTCOPY_ACTIVE' is entered after the precopy timelimit has expired and migration switches to postcopy. 'migration_postcopy_phase' is provided for other sections to know if they're in postcopy. Signed-off-by: Dr. David Alan Gilbert

[Qemu-devel] [PATCH 45/46] End of migration for postcopy

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Tweak the end of migration cleanup; we don't want to close stuff down at the end of the main stream, since the postcopy is still sending pages on the other thread. Signed-off-by: Dr. David Alan Gilbert dgilb...@redhat.com --- migration.c | 20

[Qemu-devel] [PATCH 25/46] Postcopy: Maintain sentmap during postcopy pre phase

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Where postcopy is preceeded by a period of precopy, the destination will have received pages that may have been dirtied on the source after the page was sent. The destination must throw these pages away before starting it's CPUs. Maintain a

[Qemu-devel] [PATCH 24/46] qemu_savevm_state_complete: Postcopy changes

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com When postcopy calls qemu_savevm_state_complete it's not really the end of migration, so skip: a) Finishing postcopiable iterative devices - they'll carry on b) The termination byte on the end of the stream. We then also add:

[Qemu-devel] [PATCH 27/46] postcopy: Add incoming_init/cleanup functions

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Provide functions to be called before the start of a postcopy enabled migration (even if it's not eventually used) and at the end. During the init we must disable huge pages in the RAM that we will receive postcopy data into, since if they start

[Qemu-devel] [PATCH 28/46] postcopy: Incoming initialisation

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Signed-off-by: Dr. David Alan Gilbert dgilb...@redhat.com --- arch_init.c | 11 +++ include/migration/migration.h | 1 + migration.c | 2 ++ 3 files changed, 14 insertions(+) diff --git a/arch_init.c

[Qemu-devel] [PATCH 29/46] postcopy: ram_enable_notify to switch on userfault

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com Signed-off-by: Dr. David Alan Gilbert dgilb...@redhat.com --- include/migration/postcopy-ram.h | 5 + postcopy-ram.c | 36 +++- 2 files changed, 40 insertions(+), 1 deletion(-) diff --git

[Qemu-devel] [PATCH 36/46] Page request: Consume pages off the post-copy queue

2014-07-04 Thread Dr. David Alan Gilbert (git)
From: Dr. David Alan Gilbert dgilb...@redhat.com When transmitting RAM pages, consume pages that have been queued by MIG_RPCOMM_REQPAGE commands and send them ahead of normal page scanning. Note: a) After a queued page the linear walk carries on from after the unqueued page; there is a

  1   2   >