Re: [KVM][Kemari]: Build error fix

On 2012/06/28 5:34, Sterling Windmill wrote: Is Kemari still in active development? No, it's not. Currently we have no intention to add new features into Kemari. Thanks, Kei Best regards, Sterling Windmill On Sun, Dec 4, 2011 at 9:45 PM, OHMURA Kei mailto:ohmura@lab.ntt.co.jp>

Re: [KVM][Kemari]: Build error fix

On 2011/12/02 21:51, Pradeep Kumar wrote: > It fixes build failure. > > I hit this error, after succsfull migration and sync. > > (qemu) qemu-system-x86_64: fill buffer failed, Interrupted system call > > qemu-system-x86_64: recv header failed > > qemu-system-x86_64: recv ack failed > > qemu_t

[PATCH 17/18] migration-tcp: modify tcp_accept_incoming_migration() to handle ft_mode, and add a hack not to close fd when ft_mode is enabled.

the fd for it. vm_change_state_handler is added to turn off ft_mode when cont is pressed. Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei --- migration-tcp.c | 68 ++- 1 files changed, 67 insertions(+), 1 deletions(-) diff --git a

[PATCH 07/18] Introduce fault tolerant VM transaction QEMUFile and ft_mode.

This code implements VM transaction protocol. Like buffered_file, it sits between savevm and migration layer. With this architecture, VM transaction protocol is implemented mostly independent from other existing code. Signed-off-by: OHMURA Kei Signed-off-by: Yoshiaki Tamura --- Makefile.objs

[PATCH 00/18] Kemari for KVM v0.2.14

posting yet. To remove the dirty bitmap optimization, please look at HEAD~5 of the tree. git://kemari.git.sourceforge.net/gitroot/kemari/kemari next Thanks, Kei OHMURA Kei (2): Introduce fault tolerant VM transaction QEMUFile and ft_mode. Introduce event-tap. Yoshiaki Tamura (16): Make QEMUFil

[PATCH 12/18] Insert event_tap_mmio() to cpu_physical_memory_rw() in exec.c.

From: Yoshiaki Tamura Record mmio write event to replay it upon failover. Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei --- exec.c |4 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/exec.c b/exec.c index c3dc68a..3c3cece 100644 --- a/exec.c +++ b/exec.c

[PATCH 09/18] Introduce event-tap.

event-tap controls when to start FT transaction, and provides proxy functions to called from net/block devices. While FT transaction, it queues up net/block requests, and flush them when the transaction gets completed. Signed-off-by: OHMURA Kei Signed-off-by: Yoshiaki Tamura

[PATCH 14/18] block: insert event-tap to bdrv_aio_writev(), bdrv_aio_flush() and bdrv_flush().

From: Yoshiaki Tamura event-tap function is called only when it is on, and requests were sent from device emulators. Signed-off-by: Yoshiaki Tamura Acked-by: Kevin Wolf Signed-off-by: OHMURA Kei --- block.c | 15 +++ 1 files changed, 15 insertions(+), 0 deletions(-) diff

[PATCH 11/18] ioport: insert event_tap_ioport() to ioport_write().

From: Yoshiaki Tamura Record ioport event to replay it upon failover. Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei --- ioport.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/ioport.c b/ioport.c index 2e971fa..f485bab 100644 --- a/ioport.c +++ b

[PATCH 10/18] Call init handler of event-tap at main() in vl.c.

From: Yoshiaki Tamura Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei --- vl.c |3 +++ 1 files changed, 3 insertions(+), 0 deletions(-) diff --git a/vl.c b/vl.c index a18792d..2dbda4d 100644 --- a/vl.c +++ b/vl.c @@ -160,6 +160,7 @@ int main(int argc, char **argv) #include

[PATCH 15/18] savevm: introduce qemu_savevm_trans_{begin,commit}.

: OHMURA Kei --- savevm.c | 157 +++--- sysemu.h |2 + 2 files changed, 101 insertions(+), 58 deletions(-) diff --git a/savevm.c b/savevm.c index 5b57e94..dfbdc6c 100644 --- a/savevm.c +++ b/savevm.c @@ -1630,29 +1630,68 @@ bool

[PATCH 05/18] vl.c: add deleted flag for deleting the handler.

From: Yoshiaki Tamura Make deleting handlers robust against deletion of any elements in a handler by using a deleted flag like in file descriptors. Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei --- vl.c | 15 ++- 1 files changed, 10 insertions(+), 5 deletions

[PATCH 03/18] Introduce qemu_loadvm_state_no_header() and make qemu_loadvm_state() a wrapper.

From: Yoshiaki Tamura Introduce qemu_loadvm_state_no_header() so that it can be called iteratively without reading the header, and qemu_loadvm_state() becomes a wrapper of it. Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei --- savevm.c | 45

[PATCH 06/18] virtio: decrement last_avail_idx with inuse before saving.

From: Yoshiaki Tamura For regular migration inuse == 0 always as requests are flushed before save. However, event-tap log when enabled introduces an extra queue for requests which is not being flushed, thus the last inuse requests are left in the event-tap queue. Move the last_avail_idx value se

[PATCH 18/18] Introduce "kemari:" to enable FT migration mode (Kemari).

From: Yoshiaki Tamura When "kemari:" is set in front of URI of migrate command, it will turn on ft_mode to start FT migration mode (Kemari). On the receiver side, the option looks like, -incoming kemari::: Signed-off-by: Yoshiaki Tamura Acked-by: Paolo Bonzini Signed-off-by:

[PATCH 16/18] migration: introduce migrate_ft_trans_{put,get}_ready(), and modify migrate_fd_put_ready() when ft_mode is on.

called to receive ack from the receiver. Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei --- migration.c | 266 ++- 1 files changed, 265 insertions(+), 1 deletions(-) diff --git a/migration.c b/migration.c index 1c2d956..d536df0

[PATCH 01/18] Make QEMUFile buf expandable, and introduce qemu_realloc_buffer() and qemu_clear_buffer().

From: Yoshiaki Tamura Currently buf size is fixed at 32KB. It would be useful if it could be flexible. Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei --- hw/hw.h |2 ++ savevm.c | 20 +++- 2 files changed, 21 insertions(+), 1 deletions(-) diff --git a/hw

[PATCH 08/18] savevm: introduce util functions to control ft_trans_file from savevm layer.

From: Yoshiaki Tamura To utilize ft_trans_file function, savevm needs interfaces to be exported. Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei --- hw/hw.h |5 ++ savevm.c | 150 ++ 2 files changed, 155 insertions

[PATCH 02/18] Introduce read() to FdMigrationState.

From: Yoshiaki Tamura Currently FdMigrationState doesn't support read(), and this patch introduces it to get response from the other side. Note that this won't change the existing migration protocol to be bi-directional. Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei ---

[PATCH 04/18] qemu-char: export socket_set_nodelay().

From: Yoshiaki Tamura Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei --- qemu-char.c |2 +- qemu_socket.h |1 + 2 files changed, 2 insertions(+), 1 deletions(-) diff --git a/qemu-char.c b/qemu-char.c index 03858d4..fef33b0 100644 --- a/qemu-char.c +++ b/qemu-char.c

[PATCH 13/18] net: insert event-tap to qemu_send_packet() and qemu_sendv_packet_async().

From: Yoshiaki Tamura event-tap function is called only when it is on. Signed-off-by: Yoshiaki Tamura Signed-off-by: OHMURA Kei --- net.c |9 + 1 files changed, 9 insertions(+), 0 deletions(-) diff --git a/net.c b/net.c index 4f777c3..8bcc504 100644 --- a/net.c +++ b/net.c

Re: virtio_blk_load() question

On 2010/03/18 21:07, Juan Quintela wrote: Really, ordering doesn't matter (in this case): see virtio-blk.c:virtio_blk_dma_restart_bh() QLIST_FOREACH_SAFE(req, &rq_copy, next, next_req) { QLIST_REMOVE(req, next); virtio_blk_handle_request(req, &mrb); } This mean that we

Re: virtio_blk_load() question

Thanks for your reply. When I ported virtio to vmstate, I was unable to get that list not empty for more than I tried. It should be not empty in the case of one error or similar, but I was not able to reproduce it. Actually, I wasn't able to get that condition either. We're having problem in

virtio_blk_load() question

Hi, I have a question regarding virtio_blk_load(). (qemu-kvm.git d1fa468c1cc03ea362d8fe3ed9269bab4d197510) VirtIOBlockReq structure is linked list of requests, but it doesn't seem to be properly linked in virtio_blk_load(). ... req->next = s->rq; s->rq = req->next; ... In this case, we're losing

Re: [Qemu-devel] Re: [PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

"We think"? I mean - yes, I think so too. But have you actually measured it? How much improvement are we talking here? Is it still faster when a bswap is involved? Thanks for pointing out. I will post the data for x86 later. However, I don't have a test environment to check the impact of bswap. W

Re: [Qemu-devel] Re: [PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

"We think"? I mean - yes, I think so too. But have you actually measured it? How much improvement are we talking here? Is it still faster when a bswap is involved? Thanks for pointing out. I will post the data for x86 later. However, I don't have a test environment to check the impact of bswap. W

Re: [Qemu-devel] Re: [PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

"We think"? I mean - yes, I think so too. But have you actually measured it? How much improvement are we talking here? Is it still faster when a bswap is involved? Thanks for pointing out. I will post the data for x86 later. However, I don't have a test environment to check the impact of bswap.

Re: [PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

dirty-bitmap-traveling is carried out by byte size in qemu-kvm.c. But We think that dirty-bitmap-traveling by long size is faster than by byte size especially when most of memory is not dirty. Signed-off-by: OHMURA Kei --- bswap.h|2 ++ qemu-kvm.c | 31

Re: [PATCH 1/3] qemu-kvm: Wrap phys_ram_dirty with additional inline functions.

> Why do you need a counter? It may be sufficient to set a single bit. > This reduces the memory overhead and perhaps cache thrashing. Thanks for looking into this. I agree with your opinion. Our motivation here is to skip traveling when the dirty bitmap is really sparse or dense, so either sett

Re: [PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

On 02/11/2010 Anthony Liguori wrote: > Oh, I see what's happening here. Yes, I think a leul_to_cpu() makes more > sense. Maybe I'm missing something here. I couldn't find leul_to_cpu(), so have defined it in bswap.h. Correct? --- a/bswap.h +++ b/bswap.h @@ -205,8 +205,10 @@ static inline void cp

[PATCH v2] qemu-kvm: Speed up of the dirty-bitmap-traveling

dirty-bitmap-traveling is carried out by byte size in qemu-kvm.c. But We think that dirty-bitmap-traveling by long size is faster than by byte size especially when most of memory is not dirty. Signed-off-by: OHMURA Kei --- bswap.h|1 - qemu-kvm.c | 30 -- 2

Re: [PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

> Please reuse the changelog when reposing a patch, this makes it easier > for me to apply it. Thanks. Will follow it from next time. > Should be a host long size, not guest. This will fail when running a > 32-bit qemu-system-x86_64 binary. Sorry. That was our mistake. > Instead of using a n

Re: [PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

Thank you for your comments. We have implemented the code which applied your comments. This is patch for upstream. Signed-off-by: OHMURA Kei --- kvm-all.c | 54 +++--- 1 files changed, 39 insertions(+), 15 deletions(-) diff --git a/kvm-all.c

Re: [PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

Thank you for your comments. We have implemented the code which applied your comments. This is patch for qemu-kvm.c. Signed-off-by: OHMURA Kei --- qemu-kvm.c | 31 +++ 1 files changed, 19 insertions(+), 12 deletions(-) diff --git a/qemu-kvm.c b/qemu-kvm.c index

Re: [PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

>> Would be great if you could provide a version for upstream as well >> because it will likely replace this qemu-kvm code on day. > O.K. We'll prepare it. We have implemented the version for upstream. Some source code are borrowed from qemu-kvm.c. It is not fully tested yet, though. We also

[PATCH 1/3] qemu-kvm: Wrap phys_ram_dirty with additional inline functions.

-dirty using phys_ram_dirty_by_word. Signed-off-by: OHMURA Kei --- cpu-all.h | 74 cpu-defs.h |1 + 2 files changed, 75 insertions(+), 0 deletions(-) diff --git a/cpu-all.h b/cpu-all.h index 8ed76c7..2251f14 100644 --- a/cpu

[PATCH 0/3] qemu-kvm: Check the dirty and non-dirty pages by multiple size

The dirty and non-dirty pages are checked one by one in vl.c. Since we believe that most of the memory is not dirty, checking the dirty and non-dirty pages by multiple page size should be much faster than checking them one by one. We think there is mostly two kind of situation. One is almost all t

[PATCH 2/3] Check continuous dirty and non-dirty pages.

We will check the dirty and non-dirty pages as follows: 1. Checked by 64 pages block. Check pages using phys_ram_dirty_by_word. Count up/down it when phys_ram_dirty when is 0xff or not. 2. Checked by TARGET_LONG_SIZE pages block. 3. Checked by one page. Signed-off-by: OHMURA Kei --- exec.c

[PATCH 3/3] qemu-kvm: Change the methods of get dirty pages.

Get number of the dirty and non-dirty pages using cpu_physical_memory_get_{dirty|non_dirty}_range(). Signed-off-by: OHMURA Kei --- vl.c | 57 ++--- 1 files changed, 38 insertions(+), 19 deletions(-) diff --git a/vl.c b/vl.c index 4ef6a78

Re: [PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

> Sounds logical - do you have numbers on the improvement? Sure. The patch showed approximately 3-7 times speed up when measured with rdtsc. The test environment and detailed results are described below. --- tmp = rdtsc(); /* function of original code*/ t1 += rdtsc() - tmp; tmp = rdtsc();

[PATCH] qemu-kvm: Speed up of the dirty-bitmap-traveling

dirty-bitmap-traveling is carried out by byte size in qemu-kvm.c. But We think that dirty-bitmap-traveling by long size is faster than by byte size especially when most of memory is not dirty. Signed-off-by: OHMURA Kei --- qemu-kvm.c | 49 ++--- 1