Re: [Qemu-devel] kvm segfaulting

[Stefan Priebe - Profihost AG]

Another hint: I've never seens this using qemu 1.3.1

Stefan

Am 13.02.2013 08:49, schrieb Stefan Priebe - Profihost AG:
> Hi Paolo,
> 
> sadly no luck. A VM crashed again.
> 
> [ ~]# addr2line -e /usr/lib/debug/usr/bin/kvm -f 24040c
> virtio_scsi_command_complete
> hw/virtio-scsi.c:429
> 
> Same point like last time:
> static void virtio_scsi_command_complete(SCSIRequest *r, uint32_t status,
>  size_t resid)
> {
> VirtIOSCSIReq *req = r->hba_private;
> uint32_t sense_len;
> 
> 429   =>req->resp.cmd->response = VIRTIO_SCSI_S_OK;
> req->resp.cmd->status = status;
> if (req->resp.cmd->status == GOOD) {
> req->resp.cmd->resid = tswap32(resid);
> } else {
> req->resp.cmd->resid = 0;
> sense_len = scsi_req_get_sense(r, req->resp.cmd->sense,
>VIRTIO_SCSI_SENSE_SIZE);
> req->resp.cmd->sense_len = tswap32(sense_len);
> }
> virtio_scsi_complete_req(req);
> 
> Greets
> Stefan
> 
> Am 12.02.2013 15:34, schrieb Paolo Bonzini:
>> Il 12/02/2013 14:46, Stefan Priebe - Profihost AG ha scritto:
>>> Hi,
>>>
>>> thanks - i applied the patch to the latest master. I hope that this will
>>> solve my issue. Will this one get integrated in 1.4 final?

Re: [Qemu-devel] [PATCH v10 0/4] Add subcommand compare for qemu-img

[Miroslav Rezanina]

Please ignore this serie. It's resended v10 instead of v11.
Sorry for spam.

- Original Message -
> From: "Miroslav Rezanina" 
> To: qemu-devel@nongnu.org, qemu-devel@nongnu.org
> Cc: kw...@redhat.com, pbonz...@redhat.com, stefa...@redhat.com, 
> ebl...@redhat.com, "Miroslav Rezanina"
> 
> Sent: Wednesday, February 13, 2013 9:05:53 AM
> Subject: [PATCH v10 0/4] Add subcommand compare for qemu-img
> 
> This is 10th version of patch adding compare subcommand that
> compares two images. Compare has following criteria:
>  - only data part is compared
>  - unallocated sectors are not read
>  to be zeroed/unallocated to compare rest
>  - qemu-img returns:
> - 0 if images are identical
> - 1 if images differ
> - >2 error on execution
> 
> v10:
>  - rebased to kwolf's block-next branch due to conflict patch (patch
>  2)
>  - exit status updates (patch 3)
>  - minor non-functional fixes
> 
> v9:
>  - Merged patch 3 and 4 (subcommmand implementation and
>  documentation) of v8
>  - Added basic test for qemu-img compare
>  - Fixed incorrect indentation
>  - add bdrv_is_allocated_above() return value check
>  - Add description of exit codes into the documentation
>  - minor non-functional changes
> 
> v8:
>  - minor grammar fixes (no behavior change)
>  
> v7:
>  - split patch into pieces
>  - Quiet mode added for all relevant subcommands
>  - check non-shared part of disk after shared one
>  - minor docummentation and naming fixes
> 
> v6:
>  - added handling -?, -h options for compare subcommand
> 
> v5 (only minor changes):
>  - removed redundant comment
>  - removed dead code (goto after help())
>  - set final total_sectors on first assignment
> 
> v4:
>  - Fixed various typos
>  - Added functions for empty sector check and sector-to-bytes offset
>  conversion
>  - Fixed command-line parameters processing
> 
> v3:
>  - options -f/-F are orthogonal
>  - documentation updated to new syntax and behavior
>  - used byte offset instead of sector number for output
>  
> v2:
>  - changed option for second image format to -F
>  - changed handling of -f and -F [1]
>  - added strict mode (-s)
>  - added quiet mode (-q)
>  - improved output messages [2]
>  - rename variables for larger image handling
>  - added man page content
> 
> Signed-off-by: Miroslav Rezanina 
> 
> Miroslav Rezanina (4):
>   block: Add synchronous wrapper for bdrv_co_is_allocated_above
>   qemu-img: Add "Quiet mode" option
>   This patch adds new qemu-img subcommand that compares content of
>   two
> disk images.
>   qemu-iotests: Add qemu-img compare test
> 
>  block.c|   51 +-
>  blockdev.c |6 +-
>  include/block/block.h  |5 +-
>  qemu-img-cmds.hx   |   34 ++--
>  qemu-img.c |  441
>  +++-
>  qemu-img.texi  |   56 ++
>  tests/qemu-iotests/048 |   77 
>  tests/qemu-iotests/048.out |   25 +++
>  tests/qemu-iotests/group   |1 +
>  9 files changed, 626 insertions(+), 70 deletions(-)
>  create mode 100755 tests/qemu-iotests/048
>  create mode 100644 tests/qemu-iotests/048.out
> 
> 

-- 
Miroslav Rezanina
Software Engineer - Virtualization Team

[Qemu-devel] [PATCH v11 0/4] Add subcommand compare for qemu-img

[Miroslav Rezanina]

This is 11th version of patch adding compare subcommand that
compares two images. Compare has following criteria:
 - only data part is compared
 - unallocated sectors are not read 
 - qemu-img returns:
- 0 if images are identical
- 1 if images differ
- >2 error on execution

v11:
 - add minor enhancements to qemu-img compare test (patch 4)

v10:
 - rebased to kwolf's block-next branch due to conflict patch (patch 2)
 - exit status updates (patch 3)
 - minor non-functional fixes

v9:
 - Merged patch 3 and 4 (subcommmand implementation and documentation) of v8
 - Added basic test for qemu-img compare
 - Fixed incorrect indentation
 - add bdrv_is_allocated_above() return value check
 - Add description of exit codes into the documentation
 - minor non-functional changes

v8:
 - minor grammar fixes (no behavior change)
 
v7:
 - split patch into pieces
 - Quiet mode added for all relevant subcommands
 - check non-shared part of disk after shared one
 - minor docummentation and naming fixes

v6:
 - added handling -?, -h options for compare subcommand

v5 (only minor changes):
 - removed redundant comment
 - removed dead code (goto after help())
 - set final total_sectors on first assignment

v4:
 - Fixed various typos
 - Added functions for empty sector check and sector-to-bytes offset
 conversion
 - Fixed command-line parameters processing

v3:
 - options -f/-F are orthogonal
 - documentation updated to new syntax and behavior
 - used byte offset instead of sector number for output
 
v2:
 - changed option for second image format to -F
 - changed handling of -f and -F [1]
 - added strict mode (-s)
 - added quiet mode (-q)
 - improved output messages [2]
 - rename variables for larger image handling
 - added man page content

Signed-off-by: Miroslav Rezanina 

Miroslav Rezanina (4):
  block: Add synchronous wrapper for bdrv_co_is_allocated_above
  qemu-img: Add "Quiet mode" option
  qemu-img: Add compare subcommand
  qemu-iotests: Add qemu-img compare test

 block.c|   51 +-
 blockdev.c |6 +-
 include/block/block.h  |5 +-
 qemu-img-cmds.hx   |   34 ++--
 qemu-img.c |  441 +++-
 qemu-img.texi  |   56 ++
 tests/qemu-iotests/048 |   78 
 tests/qemu-iotests/048.out |   31 +++
 tests/qemu-iotests/group   |1 +
 9 files changed, 633 insertions(+), 70 deletions(-)
 create mode 100755 tests/qemu-iotests/048
 create mode 100644 tests/qemu-iotests/048.out

Re: [Qemu-devel] kvm segfaulting

[Stefan Priebe - Profihost AG]

Hi,

could this be this one?

commit 47a150a4bbb06e45ef439a8222e9f46a7c4cca3f
Author: Paolo Bonzini 
Date:   Thu Jan 10 15:49:08 2013 +0100

virtio-scsi: abort in-flight I/O when the device is reset

When the device is reset, the SCSI bus should also be reset so
that in-flight I/O is cancelled.

Signed-off-by: Paolo Bonzini 
Signed-off-by: Anthony Liguori 

Greets,
Stefan
Am 13.02.2013 09:01, schrieb Stefan Priebe - Profihost AG:
> Another hint: I've never seens this using qemu 1.3.1
> 
> Stefan
> 
> Am 13.02.2013 08:49, schrieb Stefan Priebe - Profihost AG:
>> Hi Paolo,
>>
>> sadly no luck. A VM crashed again.
>>
>> [ ~]# addr2line -e /usr/lib/debug/usr/bin/kvm -f 24040c
>> virtio_scsi_command_complete
>> hw/virtio-scsi.c:429
>>
>> Same point like last time:
>> static void virtio_scsi_command_complete(SCSIRequest *r, uint32_t status,
>>  size_t resid)
>> {
>> VirtIOSCSIReq *req = r->hba_private;
>> uint32_t sense_len;
>>
>> 429   =>req->resp.cmd->response = VIRTIO_SCSI_S_OK;
>> req->resp.cmd->status = status;
>> if (req->resp.cmd->status == GOOD) {
>> req->resp.cmd->resid = tswap32(resid);
>> } else {
>> req->resp.cmd->resid = 0;
>> sense_len = scsi_req_get_sense(r, req->resp.cmd->sense,
>>VIRTIO_SCSI_SENSE_SIZE);
>> req->resp.cmd->sense_len = tswap32(sense_len);
>> }
>> virtio_scsi_complete_req(req);
>>
>> Greets
>> Stefan
>>
>> Am 12.02.2013 15:34, schrieb Paolo Bonzini:
>>> Il 12/02/2013 14:46, Stefan Priebe - Profihost AG ha scritto:
 Hi,

 thanks - i applied the patch to the latest master. I hope that this will
 solve my issue. Will this one get integrated in 1.4 final?

[Qemu-devel] [PATCH v11 2/4] qemu-img: Add "Quiet mode" option

[Miroslav Rezanina]

There can be a need to turn output to stdout off. This patch adds a -q option
that enable "Quiet mode". In Quiet mode, only errors are printed out.

Signed-off-by: Miroslav Rezanina 
---
 block.c   |   12 ++--
 blockdev.c|6 +-
 include/block/block.h |3 +-
 qemu-img-cmds.hx  |   28 +-
 qemu-img.c|  151 ++---
 qemu-img.texi |3 +
 6 files changed, 134 insertions(+), 69 deletions(-)

diff --git a/block.c b/block.c
index 08039d2..a4d7125 100644
--- a/block.c
+++ b/block.c
@@ -4470,7 +4470,8 @@ bdrv_acct_done(BlockDriverState *bs, BlockAcctCookie 
*cookie)
 
 void bdrv_img_create(const char *filename, const char *fmt,
  const char *base_filename, const char *base_fmt,
- char *options, uint64_t img_size, int flags, Error **errp)
+ char *options, uint64_t img_size, int flags,
+ Error **errp, bool quiet)
 {
 QEMUOptionParameter *param = NULL, *create_options = NULL;
 QEMUOptionParameter *backing_fmt, *backing_file, *size;
@@ -4579,10 +4580,11 @@ void bdrv_img_create(const char *filename, const char 
*fmt,
 }
 }
 
-printf("Formatting '%s', fmt=%s ", filename, fmt);
-print_option_parameters(param);
-puts("");
-
+if (!quiet) {
+printf("Formatting '%s', fmt=%s ", filename, fmt);
+print_option_parameters(param);
+puts("");
+}
 ret = bdrv_create(drv, filename, param);
 if (ret < 0) {
 if (ret == -ENOTSUP) {
diff --git a/blockdev.c b/blockdev.c
index 63e6f1e..3859ac2 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -791,7 +791,7 @@ void qmp_transaction(BlockdevActionList *dev_list, Error 
**errp)
 bdrv_img_create(new_image_file, format,
 states->old_bs->filename,
 states->old_bs->drv->format_name,
-NULL, -1, flags, &local_err);
+NULL, -1, flags, &local_err, false);
 if (error_is_set(&local_err)) {
 error_propagate(errp, local_err);
 goto delete_and_fail;
@@ -1284,7 +1284,7 @@ void qmp_drive_mirror(const char *device, const char 
*target,
 /* create new image w/o backing file */
 assert(format && drv);
 bdrv_img_create(target, format,
-NULL, NULL, NULL, size, flags, &local_err);
+NULL, NULL, NULL, size, flags, &local_err, false);
 } else {
 switch (mode) {
 case NEW_IMAGE_MODE_EXISTING:
@@ -1295,7 +1295,7 @@ void qmp_drive_mirror(const char *device, const char 
*target,
 bdrv_img_create(target, format,
 source->filename,
 source->drv->format_name,
-NULL, size, flags, &local_err);
+NULL, size, flags, &local_err, false);
 break;
 default:
 abort();
diff --git a/include/block/block.h b/include/block/block.h
index 8309fc1..371b42f 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -352,7 +352,8 @@ int bdrv_load_vmstate(BlockDriverState *bs, uint8_t *buf,
 
 void bdrv_img_create(const char *filename, const char *fmt,
  const char *base_filename, const char *base_fmt,
- char *options, uint64_t img_size, int flags, Error 
**errp);
+ char *options, uint64_t img_size, int flags,
+ Error **errp, bool quiet);
 
 void bdrv_set_buffer_alignment(BlockDriverState *bs, int align);
 void *qemu_blockalign(BlockDriverState *bs, size_t size);
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 259fc14..9283776 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -10,27 +10,27 @@ STEXI
 ETEXI
 
 DEF("check", img_check,
-"check [-f fmt] [--output=ofmt] [-r [leaks | all]] filename")
+"check [-q] [-f fmt] [--output=ofmt]  [-r [leaks | all]] filename")
 STEXI
-@item check [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] 
@var{filename}
+@item check [-q] [-f @var{fmt}] [--output=@var{ofmt}] [-r [leaks | all]] 
@var{filename}
 ETEXI
 
 DEF("create", img_create,
-"create [-f fmt] [-o options] filename [size]")
+"create [-q] [-f fmt] [-o options] filename [size]")
 STEXI
-@item create [-f @var{fmt}] [-o @var{options}] @var{filename} [@var{size}]
+@item create [-q] [-f @var{fmt}] [-o @var{options}] @var{filename} [@var{size}]
 ETEXI
 
 DEF("commit", img_commit,
-"commit [-f fmt] [-t cache] filename")
+"commit [-q] [-f fmt] [-t cache] filename")
 STEXI
-@item commit [-f @var{fmt}] [-t @var{cache}] @var{filename}
+@item commit [-q] [-f @var{fmt}] [-t @var{cache}] @var{filename}
 ETEXI
 
 DEF("convert", img_convert,
-"convert [-c] [-p] [-f fmt] [-t cache] [-O output_fmt] [-o options] [-s 
snapshot_name] [-S sparse_size] filename [filename2 [...]] outpu

Re: [Qemu-devel] [RFC PATCH RDMA support v2: 2/6] create migration-rdma.c for core RDMA migration code

[Orit Wasserman]

Hi Michael,
I would prefer that only migration specific code will be included in 
migration-rdma.c.
The general RDMA code should be in rdma.c file (like the rdma.h header file).

Thanks,
Orit

On 02/12/2013 12:49 AM, Michael R. Hines wrote:
> From: "Michael R. Hines" 
> 
> 
> Signed-off-by: Michael R. Hines 
> ---
>  include/qemu/rdma.h |  281 ++
>  migration-rdma.c| 1444 
> +++
>  2 files changed, 1725 insertions(+)
>  create mode 100644 include/qemu/rdma.h
>  create mode 100644 migration-rdma.c
> 
> diff --git a/include/qemu/rdma.h b/include/qemu/rdma.h
> new file mode 100644
> index 000..2dc2519
> --- /dev/null
> +++ b/include/qemu/rdma.h
> @@ -0,0 +1,281 @@
> +/*
> + *  Copyright (C) 2013 Michael R. Hines 
> + *  Copyright (C) 2013 Jiuxing Liu 
> + *
> + *  RDMA data structures and helper functions (for migration)
> + *
> + *  This program is free software; you can redistribute it and/or modify
> + *  it under the terms of the GNU General Public License as published by
> + *  the Free Software Foundation; under version 2 of the License.
> + *
> + *  This program is distributed in the hope that it will be useful,
> + *  but WITHOUT ANY WARRANTY; without even the implied warranty of
> + *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> + *  GNU General Public License for more details.
> + *
> + *  You should have received a copy of the GNU General Public License
> + *  along with this program; if not, see .
> + */
> +
> +#ifndef _RDMA_H
> +#define _RDMA_H
> +
> +#include "config-host.h"
> +#ifdef CONFIG_RDMA
> +#include 
> +#endif
> +#include "monitor/monitor.h"
> +
> +extern int rdmaport;
> +extern char rdmahost[64];
> +
> +struct rdma_context {
> +/* cm_id also has ibv_conext, rdma_event_channel, and ibv_qp in
> +   cm_id->verbs, cm_id->channel, and cm_id->qp. */
> +struct rdma_cm_id *cm_id;
> +struct rdma_cm_id *listen_id;
> +
> +struct ibv_context *verbs;
> +struct rdma_event_channel *channel;
> +struct ibv_qp *qp;
> +
> +struct ibv_comp_channel *comp_channel;
> +struct ibv_pd *pd;
> +struct ibv_cq *cq;
> +};
> +
> +static inline void rdma_init_context(struct rdma_context *rdma_ctx)
> +{
> +rdma_ctx->cm_id = NULL;
> +rdma_ctx->listen_id = NULL;
> +rdma_ctx->verbs = NULL;
> +rdma_ctx->channel = NULL;
> +rdma_ctx->qp = NULL;
> +rdma_ctx->comp_channel = NULL;
> +rdma_ctx->pd = NULL;
> +rdma_ctx->cq = NULL;
> +}
> +
> +void cpu_physical_memory_reset_dirty_all(void);
> +
> +int rdma_resolve_host(struct rdma_context *rdma_ctx,
> +const char *host, int port);
> +int rdma_alloc_pd_cq(struct rdma_context *rdma_ctx);
> +int rdma_alloc_qp(struct rdma_context *rdma_ctx);
> +int rdma_migrate_connect(struct rdma_context *rdma_ctx,
> +void *in_data, int *in_len, void *out_data, int out_len);
> +int rdma_migrate_accept(struct rdma_context *rdma_ctx,
> +void *in_data, int *in_len, void *out_data, int out_len);
> +void rdma_migrate_disconnect(struct rdma_context *rdma_ctx);
> +
> +/* Instead of registering whole ram blocks, we can register them in smaller
> + * chunks. This may be benefial if the ram blocks have holes in them */
> +#define RDMA_CHUNK_REGISTRATION
> +
> +#define RDMA_LAZY_REGISTRATION
> +
> +#define RDMA_REG_CHUNK_SHIFT 20
> +#define RDMA_REG_CHUNK_SIZE (1UL << (RDMA_REG_CHUNK_SHIFT))
> +#define RDMA_REG_CHUNK_INDEX(start_addr, host_addr) \
> +(((unsigned long)(host_addr) >> RDMA_REG_CHUNK_SHIFT) - \
> +((unsigned long)(start_addr) >> RDMA_REG_CHUNK_SHIFT))
> +#define RDMA_REG_NUM_CHUNKS(rdma_ram_block) \
> +(RDMA_REG_CHUNK_INDEX((rdma_ram_block)->local_host_addr,\
> +(rdma_ram_block)->local_host_addr +\
> +(rdma_ram_block)->length) + 1)
> +#define RDMA_REG_CHUNK_START(rdma_ram_block, i) ((uint8_t *)\
> +unsigned long)((rdma_ram_block)->local_host_addr) >> \
> +RDMA_REG_CHUNK_SHIFT) + (i)) << \
> +RDMA_REG_CHUNK_SHIFT))
> +#define RDMA_REG_CHUNK_END(rdma_ram_block, i) \
> +(RDMA_REG_CHUNK_START(rdma_ram_block, i) + \
> + RDMA_REG_CHUNK_SIZE)
> +
> +struct rdma_ram_block {
> +uint8_t *local_host_addr;
> +uint64_t remote_host_addr;
> +uint64_t offset;
> +uint64_t length;
> +struct ibv_mr **pmr;
> +struct ibv_mr *mr;
> +uint32_t remote_rkey;
> +};
> +
> +struct rdma_remote_ram_block {
> +uint64_t remote_host_addr;
> +uint64_t offset;
> +uint64_t length;
> +uint32_t remote_rkey;
> +};
> +
> +#define RDMA_MAX_RAM_BLOCKS 64
> +
> +struct rdma_ram_blocks {
> +int num_blocks;
> +struct rdma_ram_block block[RDMA_MAX_RAM_BLOCKS];
> +};
> +
> +struct rdma_remote_ram_blocks {
> +int num_blocks;
> +struct rdma_remote_ram_block block[RDMA_MAX_RAM_BLOCKS];
> +};
> +
> +int rdma_init_ram_blocks(struct rdma_ram_blocks *

Re: [Qemu-devel] [PATCH for-1.4 stable] block/curl: disable extra protocols to prevent CVE-2013-0249

[Stefan Hajnoczi]

On Tue, Feb 12, 2013 at 08:31:38PM +0100, Andreas Färber wrote:
> Am 08.02.2013 08:49, schrieb Stefan Hajnoczi:
> > There is a buffer overflow in libcurl POP3/SMTP/IMAP.  The workaround is
> > simple: disable extra protocols so that they cannot be exploited.  Full
> > details here:
> > 
> >   http://curl.haxx.se/docs/adv_20130206.html
> > 
> > QEMU only cares about HTTP, HTTPS, FTP, FTPS, and TFTP.  I have tested
> > that this fix prevents the exploit on my host with
> > libcurl-7.27.0-5.fc18.
> > 
> > Signed-off-by: Stefan Hajnoczi 
> > ---
> > The vulnerability public and is in libcurl, not QEMU.  We can work around
> > it in order to protect users whose machines have libcurl <7.29.
> > 
> > Please add to QEMU 1.4-rc2.
> 
> Stefan, this seems to have broken my setup on Mac OS X. You seem to
> require a newer version of cURL than configure checks...

Sending a fix.

Stefan

[Qemu-devel] [PATCH for-1.4] block/curl: only restrict protocols with libcurl>=7.19.4

[Stefan Hajnoczi]

The curl_easy_setopt(state->curl, CURLOPT_PROTOCOLS, ...) interface was
introduced in libcurl 7.19.4.  Therefore we cannot protect against
CVE-2013-0249 when linking against an older libcurl.

This fixes the build failure introduced by
fb6d1bbd246c7a57ef53d3847ef225cd1349d602.

Reported-by: Andreas Färber 
Signed-off-by: Stefan Hajnoczi 
---
 block/curl.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/block/curl.c b/block/curl.c
index f6226b3..98947da 100644
--- a/block/curl.c
+++ b/block/curl.c
@@ -309,9 +309,13 @@ static CURLState *curl_init_state(BDRVCURLState *s)
 /* Restrict supported protocols to avoid security issues in the more
  * obscure protocols.  For example, do not allow POP3/SMTP/IMAP see
  * CVE-2013-0249.
+ *
+ * Restricting protocols is only supported from 7.19.4 upwards.
  */
+#if LIBCURL_VERSION_NUM >= 0x071304
 curl_easy_setopt(state->curl, CURLOPT_PROTOCOLS, PROTOCOLS);
 curl_easy_setopt(state->curl, CURLOPT_REDIR_PROTOCOLS, PROTOCOLS);
+#endif
 
 #ifdef DEBUG_VERBOSE
 curl_easy_setopt(state->curl, CURLOPT_VERBOSE, 1);
-- 
1.8.1.2

Re: [Qemu-devel] [PATCH for-1.4] net: Avoid NULL function pointer dereference on cleanup

[Stefan Hajnoczi]

On Tue, Feb 12, 2013 at 11:16:06PM +0100, Andreas Färber wrote:
> The pSeries machine and some other devices don't supply a cleanup
> callback. Revert part of 1ceef9f27359cbe92ef124bf74de6f792e71f6fb that
> started calling it unconditionally.
> 
> Cc: Jason Wang 
> Signed-off-by: Andreas Färber 
> ---
>  net/net.c |4 +++-
>  1 Datei geändert, 3 Zeilen hinzugefügt(+), 1 Zeile entfernt(-)

Acked-by: Stefan Hajnoczi 

Re: [Qemu-devel] [Qemu-ppc] [0/8] RFC: target-ppc: Start disentangling different MMU types

[David Gibson]

On Tue, Feb 12, 2013 at 11:40:37AM +0100, Andreas Färber wrote:
> Am 12.02.2013 03:00, schrieb David Gibson:
> > The target-ppc code supports CPUs with a number of different MMU
> > types: there's both the 32-bit and 64-bit versions of the "classic"
> > hash page table based powerpc mmu and there's also the BookE and 40x
> > MMUs.
> > 
> > Currently handling of all these has a roughly shared path in
> > mmu_helper.c.  Although code sharing is usually a good idea, in this
> > case the MMUs really aren't similar enough for this to be useful.
> > Instead it results in checking and behaving differently at many, many
> > different points in the path leading to an unreadable tangle of code.
> > 
> > This patch series is a first step to cleaning this up, moving to a
> > model where we have a single switch on the MMU family at the top-level
> > entry points, then have a simpler, clearer separate code path for each
> > MMU type.  More specifically, it disentangles the path for the 64-bit
> > classic hash MMU into its own new file.  The other MMU types keep the
> > existing code (minus 64-bit hash specific conditionals) for now.
> > Disentangling those as well would be a good idea, but would be best
> > done by someone with more resources to test those other platforms than
> > I have.
> > 
> > For now, the resulting 64-bit hash path retains the same structure as
> > the original shared code, just the obvious conditionals on other mmu
> > types are removed.  This path is fairly ugly in itself, but cleaning
> > that up is a later step, now much simpler without the other MMU types
> > to deal with at the same time.
> 
> Some general comments: The idea of the ongoing QOM work just sent out is
> to change the hierarchy from:
> 
> Object
> - DeviceState
>   - CPUState
> - PowerPCCPU
>   - 970 vX.Y
>   - POWER7 vX.Y
>   ...
> 
> to:
> 
> Object
> - DeviceState
>   - CPUState
> - PowerPCCPU
>   - 970 family
> - 970 vX.Y
>   - POWER7 family
> - POWER7 vX.Y
>   ...
> 
> PowerPCCPUClass is expected to grow methods overridden per family or
> model. I.e., where sensible the class should serve as indirection for
> which MMU/... implementation to choose rather than ifs or #ifdefs or
> _family glue sprinkled througout code.

So currently MMU selection is actually basde on the mmu_model field in
the env structure, rather than the cpu type directly.  In any cases my
cleanups greatly reduce the number of places we check the mmu type, so
it won't be very difficult to adjust for the new scheme.

> As reminded repeatedly, please do not introduce new static or global
> helper functions using CPUPPCState, use PowerPCCPU instead.

Ok, I'll keep that in mind.  At the moment I've been using CPUPPCState
because that's what's available at the top-level and passed down.
I'll look at this if I add global functions, but in any case it should
be pretty easy to update either way around.

> If you introduce global functions, please make them unique by using ppc_
> prefix for arbitrary functions and ppc_cpu_ for functions taking
> PowerPCCPU as first argument.

Ok, noted.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: Digital signature

Re: [Qemu-devel] [RFC PATCH RDMA support v2: 5/6] connection-setup code between client/server

[Orit Wasserman]

On 02/12/2013 12:49 AM, Michael R. Hines wrote:
> From: "Michael R. Hines" 
> 
> 
> Signed-off-by: Michael R. Hines 
> ---
>  migration-tcp.c |   19 +++
>  migration.c |   53 -
>  2 files changed, 71 insertions(+), 1 deletion(-)
> 
> diff --git a/migration-tcp.c b/migration-tcp.c
> index e78a296..113576e 100644
> --- a/migration-tcp.c
> +++ b/migration-tcp.c
> @@ -14,6 +14,7 @@
>   */
>  
>  #include "qemu-common.h"
> +#include "qemu/rdma.h"
>  #include "qemu/sockets.h"
>  #include "migration/migration.h"
>  #include "migration/qemu-file.h"
> @@ -55,6 +56,9 @@ static void tcp_wait_for_connect(int fd, void *opaque)
>  
>  if (fd < 0) {
>  DPRINTF("migrate connect error\n");
> +if (migration_use_rdma()) {
> +rdma_cleanup(&rdma_mdata);
> +}

Those should be moved to migration-rdma.c
Are you still using the tcp for transferring device state?
If so you can call the tcp functions from the migration rdma code as a first 
step
but I would prefer it to use RDMA too.

>  s->fd = -1;
>  migrate_fd_error(s);
>  } else {
> @@ -62,6 +66,10 @@ static void tcp_wait_for_connect(int fd, void *opaque)
>  s->fd = fd;
>  socket_set_block(s->fd);
>  migrate_fd_connect(s);
> +
> +if (migration_use_rdma() && rdma_wait_for_connect(fd, opaque)) {
> +migrate_fd_error(s);
> +}
See above
>  }
>  }
>  
> @@ -101,6 +109,12 @@ static void tcp_accept_incoming_migration(void *opaque)
>  goto out;
>  }
>  
> +if (migration_use_rdma() && 
> +rdma_accept_incoming_migration(&rdma_mdata)) {
> +close(s);
> +return;
> +}
> +
See above
>  process_incoming_migration(f);
>  return;
>  
> @@ -117,6 +131,11 @@ void tcp_start_incoming_migration(const char *host_port, 
> Error **errp)
>  return;
>  }
>  
> +if (migration_use_rdma() && rdma_start_incoming_migration(s)) {
> +close(s);
> +return;
> +}
> +
See above
>  qemu_set_fd_handler2(s, NULL, tcp_accept_incoming_migration, NULL,
>   (void *)(intptr_t)s);
>  }
> diff --git a/migration.c b/migration.c
> index 77c1971..5ef923d 100644
> --- a/migration.c
> +++ b/migration.c
> @@ -22,6 +22,7 @@
>  #include "qemu/sockets.h"
>  #include "migration/block.h"
>  #include "qemu/thread.h"
> +#include "qemu/rdma.h"
>  #include "qmp-commands.h"
>  
>  //#define DEBUG_MIGRATION
> @@ -262,6 +263,16 @@ void 
> qmp_migrate_set_capabilities(MigrationCapabilityStatusList *params,
>  }
>  
>  for (cap = params; cap; cap = cap->next) {
> +if (cap->value->capability == MIGRATION_CAPABILITY_RDMA) {
> +#ifndef CONFIG_RDMA
> +error_set(errp, QERR_MIGRATION_RDMA_NOT_ENABLED);
> +continue;
> +#endif
> +if (!migration_use_rdma()) {
> +error_set(errp, QERR_MIGRATION_RDMA_NOT_CONFIGURED, 
> rdma_mdata.host, rdma_mdata.port);
> +continue;
> +}
> +}

I would let the user to enable the RDMA capability first than set the RDMA 
host/port.

>  s->enabled_capabilities[cap->value->capability] = cap->value->state;
>  }
>  }
> @@ -279,6 +290,11 @@ static int migrate_fd_cleanup(MigrationState *s)
>  }
>  
>  assert(s->fd == -1);
> +
> +if (migrate_rdma_enabled()) {
> +rdma_cleanup(&rdma_mdata);
> +}
> +
>  return ret;
>  }
>  
> @@ -386,6 +402,9 @@ static MigrationState *migrate_init(const MigrationParams 
> *params)
>  s->params = *params;
>  memcpy(s->enabled_capabilities, enabled_capabilities,
> sizeof(enabled_capabilities));
> +
> +rdma_update_capability(s);
Why? the user/management needs to enable the capability manually.
The fact that you can set the capability means this QEMU version supports it.
> +
>  s->xbzrle_cache_size = xbzrle_cache_size;
>  
>  s->bandwidth_limit = bandwidth_limit;
> @@ -481,6 +500,29 @@ int64_t qmp_query_migrate_cache_size(Error **errp)
>  return migrate_xbzrle_cache_size();
>  }
>  
> +void qmp_migrate_set_rdma_port(int64_t port, Error **errp)
> +{
> +MigrationState *s = migrate_get_current();
> +if (s && (s->state == MIG_STATE_ACTIVE)) {
> +return;
> +}
> +DPRINTF("rdma migration port: %" PRId64 "\n", port);
> +rdma_mdata.port = port;
> +rdma_update_capability(s);
I would allow setting the port only if the capability is set.
> +}
> +
> +void qmp_migrate_set_rdma_host(const char *host, Error **errp)
> +{
> +MigrationState *s = migrate_get_current();
> +if (s && (s->state == MIG_STATE_ACTIVE)) {
> +return;
> +}
> +DPRINTF("rdma migration host name: %s\n", host);
> +strncpy(rdma_mdata.host, host, 64);
> +rdma_mdata.host[63] = '\0';
> +rdma_update_capability(s);
see above.
> +}
> +
>  void qmp_migrate_set_speed(int64_t value, Error **errp)
>  {
>  MigrationState *s;
>

Re: [Qemu-devel] [PATCH] qemu-log: default to stderr for logging output

[Stefan Hajnoczi]

On Tue, Feb 12, 2013 at 05:33:30PM +, Peter Maydell wrote:
> Switch the default for qemu_log logging output from "/tmp/qemu.log"
> to stderr. This is an incompatible change in some sense, but logging
> is mostly used for debugging purposes so it shouldn't affect production
> use. The previous behaviour can be obtained by adding "-D /tmp/qemu.log"
> to the command line.
> 
> This change requires us to:
>  * update all the documentation/help text
>  * make linux-user and bsd-user defer to qemu-log for the default
>logging destination rather than overriding it themselves
>  * ensure that all logfile closing is done via qemu_log_close()
>and that that function doesn't close stderr
> as well as the obvious change to the behaviour of do_qemu_set_log()
> when no logfile name has been specified.
> 
> Signed-off-by: Peter Maydell 
> ---
> Stefan suggested that qemu_log should default to stderr, and I
> agree that it makes more sense than a random file in /tmp/.
> As noted in the commit message, this is technically an incompatible
> change.
> 
> This patchset sits on top of my recent 6 patch qemu_log
> cleanup series.
> 
>  bsd-user/main.c|   15 +++
>  hmp-commands.hx|4 ++--
>  include/qemu/log.h |8 ++--
>  linux-user/main.c  |   14 --
>  qemu-doc.texi  |8 
>  qemu-log.c |   29 +++--
>  qemu-options.hx|   10 +-
>  tcg/tci/README |2 +-
>  8 files changed, 40 insertions(+), 50 deletions(-)

Reviewed-by: Stefan Hajnoczi 

Re: [Qemu-devel] [Qemu-trivial] [PATCH] xilinx_axienet.c: Assert no error when making link

[Stefan Hajnoczi]

On Mon, Feb 11, 2013 at 05:16:05PM +1000, Peter Crosthwaite wrote:
> This gives an awful silent failure when it doesn't work. Assert against link
> creation failure.
> 
> Signed-off-by: Peter Crosthwaite 
> ---
>  hw/xilinx_axienet.c |4 +++-
>  1 files changed, 3 insertions(+), 1 deletions(-)

Thanks, applied to the trivial patches -next tree:
https://github.com/stefanha/qemu/commits/trivial-patches-next

Stefan

Re: [Qemu-devel] kvm segfaulting

[Paolo Bonzini]

Il 13/02/2013 09:19, Stefan Priebe - Profihost AG ha scritto:
> Hi,
> 
> could this be this one?
> 
> commit 47a150a4bbb06e45ef439a8222e9f46a7c4cca3f
> Author: Paolo Bonzini 
> Date:   Thu Jan 10 15:49:08 2013 +0100
> 
> virtio-scsi: abort in-flight I/O when the device is reset
> 
> When the device is reset, the SCSI bus should also be reset so
> that in-flight I/O is cancelled.
> 
> Signed-off-by: Paolo Bonzini 
> Signed-off-by: Anthony Liguori 

You can certainly try reverting it, but this patch is fixing a real bug.

Paolo

> Greets,
> Stefan
> Am 13.02.2013 09:01, schrieb Stefan Priebe - Profihost AG:
>> Another hint: I've never seens this using qemu 1.3.1
>>
>> Stefan
>>
>> Am 13.02.2013 08:49, schrieb Stefan Priebe - Profihost AG:
>>> Hi Paolo,
>>>
>>> sadly no luck. A VM crashed again.
>>>
>>> [ ~]# addr2line -e /usr/lib/debug/usr/bin/kvm -f 24040c
>>> virtio_scsi_command_complete
>>> hw/virtio-scsi.c:429
>>>
>>> Same point like last time:
>>> static void virtio_scsi_command_complete(SCSIRequest *r, uint32_t status,
>>>  size_t resid)
>>> {
>>> VirtIOSCSIReq *req = r->hba_private;
>>> uint32_t sense_len;
>>>
>>> 429   =>req->resp.cmd->response = VIRTIO_SCSI_S_OK;
>>> req->resp.cmd->status = status;
>>> if (req->resp.cmd->status == GOOD) {
>>> req->resp.cmd->resid = tswap32(resid);
>>> } else {
>>> req->resp.cmd->resid = 0;
>>> sense_len = scsi_req_get_sense(r, req->resp.cmd->sense,
>>>VIRTIO_SCSI_SENSE_SIZE);
>>> req->resp.cmd->sense_len = tswap32(sense_len);
>>> }
>>> virtio_scsi_complete_req(req);
>>>
>>> Greets
>>> Stefan
>>>
>>> Am 12.02.2013 15:34, schrieb Paolo Bonzini:
 Il 12/02/2013 14:46, Stefan Priebe - Profihost AG ha scritto:
> Hi,
>
> thanks - i applied the patch to the latest master. I hope that this will
> solve my issue. Will this one get integrated in 1.4 final?
> 
> 

Re: [Qemu-devel] kvm segfaulting

[Stefan Priebe - Profihost AG]

Hi,
Am 13.02.2013 09:57, schrieb Paolo Bonzini:
> Il 13/02/2013 09:19, Stefan Priebe - Profihost AG ha scritto:
>> Hi,
>>
>> could this be this one?
>>
>> commit 47a150a4bbb06e45ef439a8222e9f46a7c4cca3f
...
> You can certainly try reverting it, but this patch is fixing a real bug.

Will try that. Yes but even if it fixes a bug and raises another one
(kvm segfault) which is the worst one. It should be fixed.

Greets,
Stefan

[Qemu-devel] [PATCH v11 3/4] qemu-img: Add compare subcommand

[Miroslav Rezanina]

This patch adds new qemu-img subcommand that compares content of two disk
images.

Signed-off-by: Miroslav Rezanina 
---
 qemu-img-cmds.hx |6 +
 qemu-img.c   |  290 +-
 qemu-img.texi|   53 ++
 3 files changed, 348 insertions(+), 1 deletions(-)

diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index 9283776..4ca7e95 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -27,6 +27,12 @@ STEXI
 @item commit [-q] [-f @var{fmt}] [-t @var{cache}] @var{filename}
 ETEXI
 
+DEF("compare", img_compare,
+"compare [-f fmt] [-F fmt] [-p] [-q] [-s] filename1 filename2")
+STEXI
+@item compare [-f @var{fmt}] [-F @var{fmt}] [-p] [-q] [-s] @var{filename1} 
@var{filename2}
+ETEXI
+
 DEF("convert", img_convert,
 "convert [-c] [-p] [-q] [-f fmt] [-t cache] [-O output_fmt] [-o options] 
[-s snapshot_name] [-S sparse_size] filename [filename2 [...]] output_filename")
 STEXI
diff --git a/qemu-img.c b/qemu-img.c
index a2a2044..5f0c0c1 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -113,7 +113,12 @@ static void help(void)
"  '-a' applies a snapshot (revert disk to saved state)\n"
"  '-c' creates a snapshot\n"
"  '-d' deletes a snapshot\n"
-   "  '-l' lists all snapshots in the given image\n";
+   "  '-l' lists all snapshots in the given image\n"
+   "\n"
+   "Parameters to compare subcommand:\n"
+   "  '-f' first image format\n"
+   "  '-F' second image format\n"
+   "  '-s' run in Strict mode - fail on different image size or sector 
allocation\n";
 
 printf("%s\nSupported formats:", help_msg);
 bdrv_iterate_format(format_print, NULL);
@@ -817,6 +822,289 @@ static int compare_sectors(const uint8_t *buf1, const 
uint8_t *buf2, int n,
 
 #define IO_BUF_SIZE (2 * 1024 * 1024)
 
+static int64_t sectors_to_bytes(int64_t sectors)
+{
+return sectors << BDRV_SECTOR_BITS;
+}
+
+static int64_t sectors_to_process(int64_t total, int64_t from)
+{
+return MIN(total - from, IO_BUF_SIZE >> BDRV_SECTOR_BITS);
+}
+
+/*
+ * Check if passed sectors are empty (not allocated or contain only 0 bytes)
+ *
+ * Returns 0 in case sectors are filled with 0, 1 if sectors contain non-zero
+ * data and negative value on error.
+ *
+ * @param bs:  Driver used for accessing file
+ * @param sect_num: Number of first sector to check
+ * @param sect_count: Number of sectors to check
+ * @param filename: Name of disk file we are checking (logging purpose)
+ * @param buffer: Allocated buffer for storing read data
+ * @param quiet: Flag for quiet mode
+ */
+static int check_empty_sectors(BlockDriverState *bs, int64_t sect_num,
+   int sect_count, const char *filename,
+   uint8_t *buffer, bool quiet)
+{
+int pnum, ret = 0;
+ret = bdrv_read(bs, sect_num, buffer, sect_count);
+if (ret < 0) {
+error_report("Error while reading offset %" PRId64 " of %s: %s",
+ sectors_to_bytes(sect_num), filename, strerror(-ret));
+return ret;
+}
+ret = is_allocated_sectors(buffer, sect_count, &pnum);
+if (ret || pnum != sect_count) {
+qprintf(quiet, "Content mismatch at offset %" PRId64 "!\n",
+sectors_to_bytes(ret ? sect_num : sect_num + pnum));
+return 1;
+}
+
+return 0;
+}
+
+/*
+ * Compares two images. Exit codes:
+ *
+ * 0 - Images are identical
+ * 1 - Images differ
+ * >1 - Error occurred
+ */
+static int img_compare(int argc, char **argv)
+{
+const char *fmt1 = NULL, *fmt2 = NULL, *filename1, *filename2;
+BlockDriverState *bs1, *bs2;
+int64_t total_sectors1, total_sectors2;
+uint8_t *buf1 = NULL, *buf2 = NULL;
+int pnum1, pnum2;
+int allocated1, allocated2;
+int ret = 0; /* return value - 0 Ident, 1 Different, >1 Error */
+bool progress = false, quiet = false, strict = false;
+int64_t total_sectors;
+int64_t sector_num = 0;
+int64_t nb_sectors;
+int c, pnum;
+uint64_t bs_sectors;
+uint64_t progress_base;
+
+for (;;) {
+c = getopt(argc, argv, "hpf:F:sq");
+if (c == -1) {
+break;
+}
+switch (c) {
+case '?':
+case 'h':
+help();
+break;
+case 'f':
+fmt1 = optarg;
+break;
+case 'F':
+fmt2 = optarg;
+break;
+case 'p':
+progress = true;
+break;
+case 'q':
+quiet = true;
+break;
+case 's':
+strict = true;
+break;
+}
+}
+
+/* Progress is not shown in Quiet mode */
+if (quiet) {
+progress = false;
+}
+
+
+if (optind > argc - 2) {
+help();
+}
+filename1 = argv[optind++];
+filename2 = argv[optind++];
+
+/* Initialize before goto out */
+qemu_progress_init(progress, 2.0);
+
+bs1 = bdrv_new_ope

Re: [Qemu-devel] [PATCH v11 4/4] qemu-iotests: Add qemu-img compare test

[Kevin Wolf]

Am 13.02.2013 09:09, schrieb Miroslav Rezanina:
> Simple test for qemu-img compare to check it's working correctly.
> 
> Signed-off-by: Miroslav Rezanina 
> ---
>  tests/qemu-iotests/048 |   78 
> 
>  tests/qemu-iotests/048.out |   31 +
>  tests/qemu-iotests/group   |1 +
>  3 files changed, 110 insertions(+), 0 deletions(-)
>  create mode 100755 tests/qemu-iotests/048
>  create mode 100644 tests/qemu-iotests/048.out
> 
> diff --git a/tests/qemu-iotests/048 b/tests/qemu-iotests/048
> new file mode 100755
> index 000..597e4ba
> --- /dev/null
> +++ b/tests/qemu-iotests/048
> @@ -0,0 +1,78 @@
> +#!/bin/bash
> +##
> +## qemu-img compare test 

New trailing whitespace here. Can probably be fixed when applying the patch.

Kevin

Re: [Qemu-devel] [PATCH v11 0/4] Add subcommand compare for qemu-img

[Kevin Wolf]

Am 13.02.2013 09:09, schrieb Miroslav Rezanina:
> This is 11th version of patch adding compare subcommand that
> compares two images. Compare has following criteria:
>  - only data part is compared
>  - unallocated sectors are not read 
>  - qemu-img returns:
> - 0 if images are identical
> - 1 if images differ
> - >2 error on execution

> Miroslav Rezanina (4):
>   block: Add synchronous wrapper for bdrv_co_is_allocated_above
>   qemu-img: Add "Quiet mode" option
>   qemu-img: Add compare subcommand
>   qemu-iotests: Add qemu-img compare test
> 
>  block.c|   51 +-
>  blockdev.c |6 +-
>  include/block/block.h  |5 +-
>  qemu-img-cmds.hx   |   34 ++--
>  qemu-img.c |  441 
> +++-
>  qemu-img.texi  |   56 ++
>  tests/qemu-iotests/048 |   78 
>  tests/qemu-iotests/048.out |   31 +++
>  tests/qemu-iotests/group   |1 +
>  9 files changed, 633 insertions(+), 70 deletions(-)
>  create mode 100755 tests/qemu-iotests/048
>  create mode 100644 tests/qemu-iotests/048.out

Reviewed-by: Kevin Wolf 

[Qemu-devel] [Bug 921208] Re: win7/x64 installer hangs on startup with 0x0000005d.

[Muelli]

FWIW: there is a downstream report here
https://bugzilla.redhat.com/show_bug.cgi?id=785293

** Bug watch added: Red Hat Bugzilla #785293
   https://bugzilla.redhat.com/show_bug.cgi?id=785293

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/921208

Title:
  win7/x64 installer hangs on startup with 0x005d.

Status in QEMU:
  Confirmed

Bug description:
  hi,

  during booting win7/x64 installer i'm observing a bsod with 0x005d
  ( msdn: unsupported_processor ).

  used command line: qemu-system-x86_64 -m 2048 -hda w7-system.img
  -cdrom win7_x64.iso -boot d

  adding '-machine accel=kvm' instead of default tcg accel helps to
  boot.

  
  installed software:

  qemu-1.0
  linux-3.2.1
  glibc-2.14.1
  gcc-4.6.2

  hw cpu:

  processor   : 0..7
  vendor_id   : GenuineIntel
  cpu family  : 6
  model   : 42
  model name  : Intel(R) Core(TM) i7-2630QM CPU @ 2.00GHz
  stepping: 7
  microcode   : 0x14
  cpu MHz : 1995.739
  cache size  : 6144 KB
  physical id : 0
  siblings: 8
  core id : 3
  cpu cores   : 4
  apicid  : 7
  initial apicid  : 7
  fpu : yes
  fpu_exception   : yes
  cpuid level : 13
  wp  : yes
  flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca 
cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx 
rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology 
nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 
cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer xsave avx 
lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid
  bogomips: 3992.23
  clflush size: 64
  cache_alignment : 64
  address sizes   : 36 bits physical, 48 bits virtual

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/921208/+subscriptions

[Qemu-devel] [PATCH v11 1/4] block: Add synchronous wrapper for bdrv_co_is_allocated_above

[Miroslav Rezanina]

There's no synchronous wrapper for bdrv_co_is_allocated_above function
so it's not possible to check for sector allocation in an image with
a backing file.

Signed-off-by: Miroslav Rezanina 
---
 block.c   |   39 +++
 include/block/block.h |2 ++
 2 files changed, 41 insertions(+), 0 deletions(-)

diff --git a/block.c b/block.c
index 50dab8e..08039d2 100644
--- a/block.c
+++ b/block.c
@@ -2681,6 +2681,7 @@ int bdrv_has_zero_init(BlockDriverState *bs)
 
 typedef struct BdrvCoIsAllocatedData {
 BlockDriverState *bs;
+BlockDriverState *base;
 int64_t sector_num;
 int nb_sectors;
 int *pnum;
@@ -2813,6 +2814,44 @@ int coroutine_fn 
bdrv_co_is_allocated_above(BlockDriverState *top,
 return 0;
 }
 
+/* Coroutine wrapper for bdrv_is_allocated_above() */
+static void coroutine_fn bdrv_is_allocated_above_co_entry(void *opaque)
+{
+BdrvCoIsAllocatedData *data = opaque;
+BlockDriverState *top = data->bs;
+BlockDriverState *base = data->base;
+
+data->ret = bdrv_co_is_allocated_above(top, base, data->sector_num,
+   data->nb_sectors, data->pnum);
+data->done = true;
+}
+
+/*
+ * Synchronous wrapper around bdrv_co_is_allocated_above().
+ *
+ * See bdrv_co_is_allocated_above() for details.
+ */
+int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base,
+int64_t sector_num, int nb_sectors, int *pnum)
+{
+Coroutine *co;
+BdrvCoIsAllocatedData data = {
+.bs = top,
+.base = base,
+.sector_num = sector_num,
+.nb_sectors = nb_sectors,
+.pnum = pnum,
+.done = false,
+};
+
+co = qemu_coroutine_create(bdrv_is_allocated_above_co_entry);
+qemu_coroutine_enter(co, &data);
+while (!data.done) {
+qemu_aio_wait();
+}
+return data.ret;
+}
+
 BlockInfo *bdrv_query_info(BlockDriverState *bs)
 {
 BlockInfo *info = g_malloc0(sizeof(*info));
diff --git a/include/block/block.h b/include/block/block.h
index ce61883..8309fc1 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -279,6 +279,8 @@ int bdrv_co_discard(BlockDriverState *bs, int64_t 
sector_num, int nb_sectors);
 int bdrv_has_zero_init(BlockDriverState *bs);
 int bdrv_is_allocated(BlockDriverState *bs, int64_t sector_num, int nb_sectors,
   int *pnum);
+int bdrv_is_allocated_above(BlockDriverState *top, BlockDriverState *base,
+int64_t sector_num, int nb_sectors, int *pnum);
 
 void bdrv_set_on_error(BlockDriverState *bs, BlockdevOnError on_read_error,
BlockdevOnError on_write_error);
-- 
1.7.1

[Qemu-devel] [PATCH v11 4/4] qemu-iotests: Add qemu-img compare test

[Miroslav Rezanina]

Simple test for qemu-img compare to check it's working correctly.

Signed-off-by: Miroslav Rezanina 
---
 tests/qemu-iotests/048 |   78 
 tests/qemu-iotests/048.out |   31 +
 tests/qemu-iotests/group   |1 +
 3 files changed, 110 insertions(+), 0 deletions(-)
 create mode 100755 tests/qemu-iotests/048
 create mode 100644 tests/qemu-iotests/048.out

diff --git a/tests/qemu-iotests/048 b/tests/qemu-iotests/048
new file mode 100755
index 000..597e4ba
--- /dev/null
+++ b/tests/qemu-iotests/048
@@ -0,0 +1,78 @@
+#!/bin/bash
+##
+## qemu-img compare test 
+##
+##
+## Copyright (C) 2013 Red Hat, Inc.
+##
+## This program is free software; you can redistribute it and/or modify
+## it under the terms of the GNU General Public License as published by
+## the Free Software Foundation; either version 2 of the License, or
+## (at your option) any later version.
+##
+## This program is distributed in the hope that it will be useful,
+## but WITHOUT ANY WARRANTY; without even the implied warranty of
+## MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+## GNU General Public License for more details.
+##
+## You should have received a copy of the GNU General Public License
+## along with this program.  If not, see .
+##
+#
+# creator
+owner=mreza...@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+status=1# failure is the default!
+
+_cleanup()
+{
+echo "Cleanup"
+_cleanup_test_img
+rm ${TEST_IMG2}
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_compare()
+{
+$QEMU_IMG compare "$@" $TEST_IMG ${TEST_IMG2}
+echo $?
+}
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+. ./common.pattern
+
+_supported_fmt raw qcow qcow2 qed
+_supported_proto file
+_supported_os Linux
+
+# Setup test basic parameters
+TEST_IMG2=$TEST_IMG.2
+CLUSTER_SIZE=4096
+size=1024M
+
+_make_test_img $size
+io_pattern write 524288 $CLUSTER_SIZE $CLUSTER_SIZE 4 45
+
+# Compare identical images
+cp $TEST_IMG ${TEST_IMG2}
+_compare
+_compare -q
+
+# Compare images with different size
+$QEMU_IMG resize $TEST_IMG +512M
+_compare
+_compare -s
+
+# Compare images with different content
+io_pattern write 1228800 $CLUSTER_SIZE 0 1 67
+_compare
+io_pattern write 0 $CLUSTER_SIZE 0 1 123
+_compare
+
+# Cleanup
+status=0
diff --git a/tests/qemu-iotests/048.out b/tests/qemu-iotests/048.out
new file mode 100644
index 000..68f65d5
--- /dev/null
+++ b/tests/qemu-iotests/048.out
@@ -0,0 +1,31 @@
+QA output created by 048
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824
+=== IO: pattern 45
+qemu-io> wrote 4096/4096 bytes at offset 524288
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io> wrote 4096/4096 bytes at offset 528384
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io> wrote 4096/4096 bytes at offset 532480
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io> wrote 4096/4096 bytes at offset 536576
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io> Images are identical.
+0
+0
+Image resized.
+Warning: Image size mismatch!
+Images are identical.
+0
+Strict mode: Image size mismatch!
+1
+=== IO: pattern 67
+qemu-io> wrote 4096/4096 bytes at offset 1228800
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io> Content mismatch at offset 1228800!
+1
+=== IO: pattern 123
+qemu-io> wrote 4096/4096 bytes at offset 0
+4 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io> Content mismatch at offset 0!
+1
+Cleanup
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 1bbd2bf..1c0dfef 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -54,3 +54,4 @@
 045 rw auto
 046 rw auto aio
 047 rw auto
+048 img auto quick
-- 
1.7.1

Re: [Qemu-devel] [RFC PATCH RDMA support v2: 6/6] send memory over RDMA as blocks are iterated

[Orit Wasserman]

On 02/12/2013 12:49 AM, Michael R. Hines wrote:
> From: "Michael R. Hines" 
> 
> 
> Signed-off-by: Michael R. Hines 
> ---
>  arch_init.c |   84 
> ---
>  savevm.c|   59 -
>  2 files changed, 139 insertions(+), 4 deletions(-)
> 
> diff --git a/arch_init.c b/arch_init.c
> index dada6de..76092cc 100644
> --- a/arch_init.c
> +++ b/arch_init.c
> @@ -42,6 +42,7 @@
>  #include "migration/migration.h"
>  #include "exec/gdbstub.h"
>  #include "hw/smbios.h"
> +#include "qemu/rdma.h"
>  #include "exec/address-spaces.h"
>  #include "hw/pcspk.h"
>  #include "migration/page_cache.h"
> @@ -170,6 +171,15 @@ static int is_dup_page(uint8_t *page)
>  VECTYPE val = SPLAT(page);
>  int i;
>  
> +/*
> + * RFC RDMA: The empirical cost of searching for zero pages here
> + *   plus the cost of communicating with the other side
> + *   seems to take significantly more time than simply
> + *   dumping the page into remote memory.
> + */
> +if (migrate_rdma_enabled())
> +return 0;
> +
I would change ram_save_block not this function.
So I would move the if to before calling is_dup.

>  for (i = 0; i < TARGET_PAGE_SIZE / sizeof(VECTYPE); i++) {
>  if (!ALL_EQ(val, p[i])) {
>  return 0;
> @@ -282,6 +292,44 @@ static size_t save_block_hdr(QEMUFile *f, RAMBlock 
> *block, ram_addr_t offset,
>  return size;
>  }
>  
> +static size_t save_rdma_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset,
> + int cont)
> +{
> +size_t bytes_sent = 0;
> +ram_addr_t current_addr;
> +
> +acct_info.norm_pages++;
> +
> +/*
> + * use RDMA to send page
> + */
> +current_addr = block->offset + offset;
> +if (rdma_write(&rdma_mdata, current_addr,
> +TARGET_PAGE_SIZE)) {
> +fprintf(stderr, "rdma migration: write error!\n");
> +qemu_file_set_error(f, -EIO);
we lose the error here,
why not let rdma_write to set the error, let it get the QemuFile as a 
parameter. 
> +return 0;
> +}
> +
> +/*
> + * do some polling
> + */
> +while (1) {
> +int ret = rdma_poll(&rdma_mdata);
> +if (ret == RDMA_WRID_NONE) {
> +break;
> +}
> +if (ret < 0) {
> +fprintf(stderr, "rdma migration: polling error!\n");
> +qemu_file_set_error(f, -EIO);
see above
> +return 0;
> +}
> +}
> +
> +bytes_sent += TARGET_PAGE_SIZE;
> +return bytes_sent;
> +}
> +
>  #define ENCODING_FLAG_XBZRLE 0x1
>  
>  static int save_xbzrle_page(QEMUFile *f, uint8_t *current_data,
> @@ -474,6 +522,8 @@ static int ram_save_block(QEMUFile *f, bool last_stage)
>  if (!last_stage) {
>  p = get_cached_data(XBZRLE.cache, current_addr);
>  }
> +} else if (migrate_rdma_enabled()) {
> +bytes_sent = save_rdma_page(f, block, offset, cont);
>  }
You can move it to the top and than you won't need to change is_dup.
>  
>  /* XBZRLE overflow or normal page */
> @@ -601,12 +651,14 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
>  return 0;
>  }
>  
> +static int tprate = 1000;
> +
>  static int ram_save_iterate(QEMUFile *f, void *opaque)
>  {
>  int ret;
>  int i;
> -int64_t t0;
> -int total_sent = 0;
> +int64_t t0, tp0;
> +int total_sent = 0, last_total_sent = 0;
>  
>  qemu_mutex_lock_ramlist();
>  
> @@ -625,23 +677,49 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)
>  break;
>  }
>  total_sent += bytes_sent;
> +last_total_sent += bytes_sent;
>  acct_info.iterations++;
>  /* we want to check in the 1st loop, just in case it was the 1st time
> and we had to sync the dirty bitmap.
> qemu_get_clock_ns() is a bit expensive, so we only check each some
> iterations
>  */
> +
> +/*
> + * RFC RDMA: Can we have something like this to periodically print
> + *   out throughput? This is just a rough-sketch that 
> + *   partially worked for me. I assume there a better way 
> + *   that everyone would prefer. Perhaps we could set a QMP 
> + *   command that toggled a "periodic printing" option that 
> + *   allowed more details to be printed on stdout.?
> + */
>  if ((i & 63) == 0) {
> -uint64_t t1 = (qemu_get_clock_ns(rt_clock) - t0) / 100;
> +uint64_t curr = qemu_get_clock_ns(rt_clock);
> +uint64_t t1 = (curr - t0) / 100;
> +double tp;
>  if (t1 > MAX_WAIT) {
>  DPRINTF("big wait: %" PRIu64 " milliseconds, %d 
> iterations\n",
>  t1, i);
>  break;
>

Re: [Qemu-devel] [BUG] Guest OS hangs on boot when 64bit BAR present (kvm-apic -msi resource conflict)

[Michael S. Tsirkin]

On Wed, Feb 13, 2013 at 06:06:37PM +1300, Alexey Korolev wrote:
> Sometime ago I reported an issue about guest OS hang when 64bit BAR present.
> http://lists.gnu.org/archive/html/qemu-devel/2012-01/msg03189.html
> http://lists.gnu.org/archive/html/qemu-devel/2012-12/msg00413.html
> 
> Some more investigation has been done, so in this post I'll try to explain 
> why it happens and offer possible solutions:
> 
> *When the issue happens*
> The issue occurs on Linux guest OS if kernel version <2.6.36
> A Guest OS hangs on boot when a 64bit PCI BAR is present in a system (if we 
> use ivshmem driver for example) and occupies range within first
> 4 GB.
> 
> *How to reproduce*
> I used the following qemu command to reproduce the case:
> /usr/local/bin/qemu-system-x86_64 -M pc-1.3 -enable-kvm -m 2000 -smp 
> 1,sockets=1,cores=1,threads=1 -name Rh5332 -chardev
> socket,id=charmonitor,path=/var/lib/libvirt/qemu/Rh5332.monitor,server,nowait 
> -mon chardev=charmonitor,id=monitor,mode=readline -rtc
> base=utc -boot cd -drive 
> file=/home/akorolev/rh5332.img,if=none,id=drive-ide0-0-0,format=raw -device
> ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -chardev 
> file,id=charserial0,path=/home/akorolev/serial.log -device
> isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -k en-us -vga 
> cirrus -device ivshmem,shm,size=32M-device
> virtio-balloon-pci,id=balloon0
> 
> Tried different guests: Centos 5.8 64bit, RHEL 5.3 32bit, FC 12 64bit on all 
> machines hang occurs in 100% cases
> 
> *Why it happens*
> The issue basically comes from Linux PCI enumeration code.
> 
> The OS enumerates 64BIT bars when device is enabled using the following 
> procedure.
> 1. Write all FF's to lower half of 64bit BAR
> 2. Write address back to lower half of 64bit BAR
> 3. Write all FF's to higher half of 64bit BAR
> 4. Write address back to higher half of 64bit BAR
> 
> For qemu it means that  qemu pci_default_write_config() recevies all FFs for 
> lower part of the 64bit BAR.
> Then it applies the mask and converts the value to "All FF's - size + 1" 
> (FE00 if size is 32MB).
> 
> So for short period of time the range [0xFE00 - 0x] will be 
> occupied by ivshmem resource.
> For some reason it is lethal for further boot process.
> 
> We have found that boot process screws up completely if kvm-apic-msi range is 
> overlapped even for short period of time.  (We still don't
> know why it happens, hope that the qemu maintainers can answer?)
> 
> If we look at kvm-apic-msi memory region it is a non-overlapable memory 
> region with hardcoded address range [0xFEE0 - 0xFEF0].

Thanks for looking into this!

> Here is a log we collected from render_memory_regions:
> 
>  system overlap 0 pri 0 [0x0 - 0x7fff]
>  kvmvapic-rom overlap 1 pri 1000 [0xca000 - 0xcd000]
>  pc.ram overlap 0 pri 0 [0xca000 - 0xcd000]
>  ++ pc.ram [0xca000 - 0xcd000] is added to view
>  
>  smram-region overlap 1 pri 1 [0xa - 0xc]
>  pci overlap 0 pri 0 [0xa - 0xc]
>  cirrus-lowmem-container overlap 1 pri 1 [0xa - 0xc]
>  cirrus-low-memory overlap 0 pri 0 [0xa - 0xc]
> ++cirrus-low-memory [0xa - 0xc] is added to view
>  kvm-ioapic overlap 0 pri 0 [0xfec0 - 0xfec01000]
> ++kvm-ioapic [0xfec0 - 0xfec01000] is added to view
>  pci-hole64 overlap 0 pri 0 [0x1 - 0x4001]
>  pci overlap 0 pri 0 [0x1 - 0x4001]
>  pci-hole overlap 0 pri 0 [0x7d00 - 0x1]

So we have ioapic and pci-hole which should be non-overlap,
actually overlap each other.
Isn't this a problem?

>  pci overlap 0 pri 0 [0x7d00 - 0x1]
>  ivshmem-bar2-container overlap 1 pri 1 [0xfe00 - 0x1]
>  ivshmem.bar2 overlap 0 pri 0 [0xfe00 - 0x1]
> ++ivshmem.bar2 [0xfe00 - 0xfec0] is added to view
> ++ivshmem.bar2  [0xfec01000 - 0x1] is added to view
>  ivshmem-mmio overlap 1 pri 1 [0xfebf1000 - 0xfebf1100]
>  e1000-mmio overlap 1 pri 1 [0xfeba - 0xfebc]
>  cirrus-mmio overlap 1 pri 1 [0xfebf - 0xfebf1000]
>  cirrus-pci-bar0 overlap 1 pri 1 [0xfa00 - 0xfc00]
>  vga.vram overlap 1 pri 1 [0xfa00 - 0xfa80]
> ++vga.vram [0xfa00 - 0xfa80] is added to view
>  cirrus-bitblt-mmio overlap 0 pri 0 [0xfb00 - 0xfb40]
> ++cirrus-bitblt-mmio [0xfb00 - 0xfb40] is added to 
> view
>  cirrus-linear-io overlap 0 pri 0 [0xfa00 - 0xfa80]
>  pc.bios overlap 0 pri 0 [0xfffe - 0x1]
>  ram-below-4g overlap 0 pri 0 [0x0 - 0x7d00]
>  pc.ram overlap 0 pri 0 [0x0 - 0x7d00]
> ++pc.ram [0x0 - 0xa] is added to view
>  

Re: [Qemu-devel] [PATCH] Fix guest OS hangs on boot when 64bit PCI BAR present

[Michael S. Tsirkin]

On Wed, Feb 13, 2013 at 06:14:33PM +1300, Alexey Korolev wrote:
> At the moment may_overlap flag of MemoryRegion structure
> is ignored by the address range assignment process.
> This may lead to guest OS hangs if critical qemu
> resources are overlapped by PCI BARs. For example
> ivshmem 64bit PCI BAR may overlap kvm-apic-msi under
> certain conditions. This patch adds a rule that the
> regions which should not be overlapped are added to the
> view first (i.e. having highest priority). The patch
> also corrects ivshmem bar resource to be overlapable
> which is the default for PCI BARs
> 
> Signed-off-by: Alexey Korolev 

Since overlap is currently used inconsistently, it's hard to
know what this will do. Maybe we should just drop the overlap
flag and use priorities instead?

> ---
>  hw/ivshmem.c |2 +-
>  memory.c |   15 ++-
>  2 files changed, 11 insertions(+), 6 deletions(-)
> 
> diff --git a/hw/ivshmem.c b/hw/ivshmem.c
> index afaf9b3..1770fa3 100644
> --- a/hw/ivshmem.c
> +++ b/hw/ivshmem.c
> @@ -341,7 +341,7 @@ static void create_shared_memory_BAR(IVShmemState *s, int 
> fd) {
>  memory_region_init_ram_ptr(&s->ivshmem, "ivshmem.bar2",
> s->ivshmem_size, ptr);
>  vmstate_register_ram(&s->ivshmem, &s->dev.qdev);
> -memory_region_add_subregion(&s->bar, 0, &s->ivshmem);
> +memory_region_add_subregion_overlap(&s->bar, 0, &s->ivshmem, 1);
>  
>  /* region for shared memory */
>  pci_register_bar(&s->dev, 2, s->ivshmem_attr, &s->bar);

So why this change, exactly?

> diff --git a/memory.c b/memory.c
> index cd7d5e0..f1119e7 100644
> --- a/memory.c
> +++ b/memory.c
> @@ -475,7 +475,8 @@ static void render_memory_region(FlatView *view,
>   MemoryRegion *mr,
>   Int128 base,
>   AddrRange clip,
> - bool readonly)
> + bool readonly,
> + bool overlap)
>  {
>  MemoryRegion *subregion;
>  unsigned i;
> @@ -503,16 +504,16 @@ static void render_memory_region(FlatView *view,
>  if (mr->alias) {
>  int128_subfrom(&base, int128_make64(mr->alias->addr));
>  int128_subfrom(&base, int128_make64(mr->alias_offset));
> -render_memory_region(view, mr->alias, base, clip, readonly);
> +render_memory_region(view, mr->alias, base, clip, readonly, overlap);
>  return;
>  }
>  
>  /* Render subregions in priority order. */
>  QTAILQ_FOREACH(subregion, &mr->subregions, subregions_link) {
> -render_memory_region(view, subregion, base, clip, readonly);
> +render_memory_region(view, subregion, base, clip, readonly, overlap);
>  }
>  
> -if (!mr->terminates) {
> +if (mr->may_overlap != overlap || !mr->terminates) {
>  return;
>  }
>  
> @@ -567,7 +568,11 @@ static FlatView generate_memory_topology(MemoryRegion 
> *mr)
>  
>  if (mr) {
>  render_memory_region(&view, mr, int128_zero(),
> - addrrange_make(int128_zero(), int128_2_64()), 
> false);
> + addrrange_make(int128_zero(), int128_2_64()),
> + false, false);
> +render_memory_region(&view, mr, int128_zero(),
> + addrrange_make(int128_zero(), int128_2_64()),
> + false, true);
>  }
>  flatview_simplify(&view);
>  
> -- 
> 1.7.9.5

Re: [Qemu-devel] [PATCH 2/5] blockdev: add discard suboption to -drive

[Stefan Hajnoczi]

On Fri, Feb 08, 2013 at 02:06:12PM +0100, Paolo Bonzini wrote:
> @@ -1489,6 +1496,10 @@ QemuOptsList qemu_drive_opts = {
>  .type = QEMU_OPT_STRING,
>  .help = "disk image",
>  },{
> +.name = "discard",
> +.type = QEMU_OPT_STRING,
> +.help = "discard operation (ignore/off, unmap/on)",

off/on aliases will be confusing when we plan to add "anchor" in the
future, making this non-boolean.  Please support ignore/unmap only.

[Qemu-devel] [PATCH v2] ui/vnc: VA API based H.264 encoding for VNC

[David Verbeiren]

This patch implements H.264 encoding of the VNC framebuffer updates
using hardware acceleration through the VA API.

This is experimental support to let the community explore the possibilities
offered by the potential bandwidth and latency reductions that H.264
encoding allows. This may be particularly useful for use cases such as
online gaming, hosted desktops, hosted set top boxes...
This patch provides the VNC server side support. Corresponding VNC
client side support is required. To this end, we are also contributing
patches to the gtk-vnc and libvncserver/libvncclient projects which can be
used to test this experimental feature.
See instructions below for how to build a test VNC client.

In case multiple regions are updated, only the first framebuffer
update message of the batch carries the H.264 frame data.
Subsequent update framebuffer messages will contain only the
coordinates and size of the other updated regions.

This is backwards compatible with standard VNC clients thanks to
the encoding scheme negotiation included in VNC. If the client doesn't
support H.264 encoding, the server will fall back to one of the usual
VNC encodings.

Instructions/Requirements:
* Currently only works with libva 1.0: use branch "v1.0-branch" for libva
and intel-driver. Those can be built as follows:
   cd libva
   git checkout v1.0-branch
   ./autogen.sh
   make
   sudo make install
   cd ..
   git clone git://anongit.freedesktop.org/vaapi/intel-driver
   cd intel-driver
   git checkout v1.0-branch
   ./autogen.sh
   make
   sudo make install
* A graphical environment must be running as the v1.0-branch of VA API
does not support headless operation.
* When using Intel integrated graphics, hardware encoding support requires
a 2nd generation (or later) i3, i5 or i7 processor ("Sandy Bridge" or
later), or similar, with enabled Intel(R) HD graphics.
See http://intellinuxgraphics.org/h264.html for details.

Instructions for building and using a gtk-vnc test client:
* Get gtk-vnc project
   git clone git://git.gnome.org/gtk-vnc
   cd gtk-vnc
   git checkout a4f1d1912090d5
* Download and apply (git apply ) patch from:
   https://mail.gnome.org/archives/gtk-vnc-list/2013-February/msg0.html
* Build
   ./autogen.sh --with-libva
   make -j4
* Run the client, for example:
   ./examples/gvncviewer :1

Instructions for building and using a libvncclient test client:
* Get LibVNCServer project
   git clone 
git://libvncserver.git.sourceforge.net/gitroot/libvncserver/libvncserver
   cd libvncserver
   git checkout 55bdab02574e3ac
* Download and apply (git apply ) following two patches
  in sequence:
   http://sourceforge.net/mailarchive/message.php?msg_id=30323804
   http://sourceforge.net/mailarchive/message.php?msg_id=30327573
* Build:
   ./autogen.sh --with-libva
   make -j4
* Run the client, for example:
   ./client_examples/gtkvncviewer :5901

Signed-off-by: David Verbeiren 
---
 Changes for v1->v2:
  * No more statics; most moved into the VncDisplayH264 struct.
  * All variable declarations at top of funcs
  * VA encoder init now performed as part of set_encodings() so we can
fallback to another encoding in case VA init fails.
  * configure script now defaults to libva=""
  * (no code change) "VA H.264" RFB encoding type number is now registered
with IANA at http://www.iana.org/assignments/rfb/rfb.xml

 Also note that you can now use clients based on either libvncclient
 or gtk-vnc as a patch for the latter project was also submitted.


 configure |   39 
 ui/Makefile.objs  |1 +
 ui/vnc-enc-h264.c |  616 +
 ui/vnc-enc-h264.h |   74 +++
 ui/vnc-jobs.c |6 +
 ui/vnc.c  |   26 +++
 ui/vnc.h  |   18 ++
 7 files changed, 780 insertions(+)
 create mode 100644 ui/vnc-enc-h264.c
 create mode 100644 ui/vnc-enc-h264.h

diff --git a/configure b/configure
index 8789324..d742a6c 100755
--- a/configure
+++ b/configure
@@ -213,6 +213,7 @@ pie=""
 zero_malloc=""
 trace_backend="nop"
 trace_file="trace"
+libva=""
 spice=""
 rbd=""
 smartcard_nss=""
@@ -771,6 +772,10 @@ for opt do
   ;;
   --enable-spice) spice="yes"
   ;;
+  --disable-libva) libva="no"
+  ;;
+  --enable-libva) libva="yes"
+  ;;
   --disable-libiscsi) libiscsi="no"
   ;;
   --enable-libiscsi) libiscsi="yes"
@@ -1129,6 +1134,8 @@ echo "  --with-trace-file=NAME   Full PATH,NAME of file 
to store traces"
 echo "   Default:trace-"
 echo "  --disable-spice  disable spice"
 echo "  --enable-spice   enable spice"
+echo "  --disable-libva  disable H.264 encoding with libva"
+echo "  --enable-libva   enable H.264 encoding with libva"
 echo "  --enable-rbd enable building the rados block device (rbd)"
 echo "  --disable-libiscsi   disable iscsi support"
 echo "  --enable-libiscsienable iscsi support"
@@ -2842,6 +2849,33 @@ EOF
   fi
 fi
 
+##
+# libva probe
+if test 

Re: [Qemu-devel] [BUG] Guest OS hangs on boot when 64bit BAR present (kvm-apic -msi resource conflict)

[Jan Kiszka]

On 2013-02-13 11:24, Michael S. Tsirkin wrote:
> On Wed, Feb 13, 2013 at 06:06:37PM +1300, Alexey Korolev wrote:
>> Sometime ago I reported an issue about guest OS hang when 64bit BAR present.
>> http://lists.gnu.org/archive/html/qemu-devel/2012-01/msg03189.html
>> http://lists.gnu.org/archive/html/qemu-devel/2012-12/msg00413.html
>>
>> Some more investigation has been done, so in this post I'll try to explain 
>> why it happens and offer possible solutions:
>>
>> *When the issue happens*
>> The issue occurs on Linux guest OS if kernel version <2.6.36
>> A Guest OS hangs on boot when a 64bit PCI BAR is present in a system (if we 
>> use ivshmem driver for example) and occupies range within first
>> 4 GB.
>>
>> *How to reproduce*
>> I used the following qemu command to reproduce the case:
>> /usr/local/bin/qemu-system-x86_64 -M pc-1.3 -enable-kvm -m 2000 -smp 
>> 1,sockets=1,cores=1,threads=1 -name Rh5332 -chardev
>> socket,id=charmonitor,path=/var/lib/libvirt/qemu/Rh5332.monitor,server,nowait
>>  -mon chardev=charmonitor,id=monitor,mode=readline -rtc
>> base=utc -boot cd -drive 
>> file=/home/akorolev/rh5332.img,if=none,id=drive-ide0-0-0,format=raw -device
>> ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -chardev 
>> file,id=charserial0,path=/home/akorolev/serial.log -device
>> isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -k en-us 
>> -vga cirrus -device ivshmem,shm,size=32M-device
>> virtio-balloon-pci,id=balloon0
>>
>> Tried different guests: Centos 5.8 64bit, RHEL 5.3 32bit, FC 12 64bit on all 
>> machines hang occurs in 100% cases
>>
>> *Why it happens*
>> The issue basically comes from Linux PCI enumeration code.
>>
>> The OS enumerates 64BIT bars when device is enabled using the following 
>> procedure.
>> 1. Write all FF's to lower half of 64bit BAR
>> 2. Write address back to lower half of 64bit BAR
>> 3. Write all FF's to higher half of 64bit BAR
>> 4. Write address back to higher half of 64bit BAR
>>
>> For qemu it means that  qemu pci_default_write_config() recevies all FFs for 
>> lower part of the 64bit BAR.
>> Then it applies the mask and converts the value to "All FF's - size + 1" 
>> (FE00 if size is 32MB).
>>
>> So for short period of time the range [0xFE00 - 0x] will be 
>> occupied by ivshmem resource.
>> For some reason it is lethal for further boot process.
>>
>> We have found that boot process screws up completely if kvm-apic-msi range 
>> is overlapped even for short period of time.  (We still don't
>> know why it happens, hope that the qemu maintainers can answer?)
>>
>> If we look at kvm-apic-msi memory region it is a non-overlapable memory 
>> region with hardcoded address range [0xFEE0 - 0xFEF0].
> 
> Thanks for looking into this!
> 
>> Here is a log we collected from render_memory_regions:
>>
>>  system overlap 0 pri 0 [0x0 - 0x7fff]
>>  kvmvapic-rom overlap 1 pri 1000 [0xca000 - 0xcd000]
>>  pc.ram overlap 0 pri 0 [0xca000 - 0xcd000]
>>  ++ pc.ram [0xca000 - 0xcd000] is added to view
>>  
>>  smram-region overlap 1 pri 1 [0xa - 0xc]
>>  pci overlap 0 pri 0 [0xa - 0xc]
>>  cirrus-lowmem-container overlap 1 pri 1 [0xa - 0xc]
>>  cirrus-low-memory overlap 0 pri 0 [0xa - 0xc]
>> ++cirrus-low-memory [0xa - 0xc] is added to view
>>  kvm-ioapic overlap 0 pri 0 [0xfec0 - 0xfec01000]
>> ++kvm-ioapic [0xfec0 - 0xfec01000] is added to view
>>  pci-hole64 overlap 0 pri 0 [0x1 - 0x4001]
>>  pci overlap 0 pri 0 [0x1 - 0x4001]
>>  pci-hole overlap 0 pri 0 [0x7d00 - 0x1]
> 
> So we have ioapic and pci-hole which should be non-overlap,
> actually overlap each other.
> Isn't this a problem?
> 
>>  pci overlap 0 pri 0 [0x7d00 - 0x1]
>>  ivshmem-bar2-container overlap 1 pri 1 [0xfe00 - 
>> 0x1]
>>  ivshmem.bar2 overlap 0 pri 0 [0xfe00 - 0x1]
>> ++ivshmem.bar2 [0xfe00 - 0xfec0] is added to view
>> ++ivshmem.bar2  [0xfec01000 - 0x1] is added to view
>>  ivshmem-mmio overlap 1 pri 1 [0xfebf1000 - 0xfebf1100]
>>  e1000-mmio overlap 1 pri 1 [0xfeba - 0xfebc]
>>  cirrus-mmio overlap 1 pri 1 [0xfebf - 0xfebf1000]
>>  cirrus-pci-bar0 overlap 1 pri 1 [0xfa00 - 0xfc00]
>>  vga.vram overlap 1 pri 1 [0xfa00 - 0xfa80]
>> ++vga.vram [0xfa00 - 0xfa80] is added to view
>>  cirrus-bitblt-mmio overlap 0 pri 0 [0xfb00 - 0xfb40]
>> ++cirrus-bitblt-mmio [0xfb00 - 0xfb40] is added to 
>> view
>>  cirrus-linear-io overlap 0 pri 0 [0xfa00 - 0xfa80]
>>  pc.bios overlap 0 pri 0 [0xfffe - 0x1]
>>  ram-below

Re: [Qemu-devel] [BUG] Guest OS hangs on boot when 64bit BAR present (kvm-apic -msi resource conflict)

[Michael S. Tsirkin]

On Wed, Feb 13, 2013 at 11:34:52AM +0100, Jan Kiszka wrote:
> On 2013-02-13 11:24, Michael S. Tsirkin wrote:
> > On Wed, Feb 13, 2013 at 06:06:37PM +1300, Alexey Korolev wrote:
> >> Sometime ago I reported an issue about guest OS hang when 64bit BAR 
> >> present.
> >> http://lists.gnu.org/archive/html/qemu-devel/2012-01/msg03189.html
> >> http://lists.gnu.org/archive/html/qemu-devel/2012-12/msg00413.html
> >>
> >> Some more investigation has been done, so in this post I'll try to explain 
> >> why it happens and offer possible solutions:
> >>
> >> *When the issue happens*
> >> The issue occurs on Linux guest OS if kernel version <2.6.36
> >> A Guest OS hangs on boot when a 64bit PCI BAR is present in a system (if 
> >> we use ivshmem driver for example) and occupies range within first
> >> 4 GB.
> >>
> >> *How to reproduce*
> >> I used the following qemu command to reproduce the case:
> >> /usr/local/bin/qemu-system-x86_64 -M pc-1.3 -enable-kvm -m 2000 -smp 
> >> 1,sockets=1,cores=1,threads=1 -name Rh5332 -chardev
> >> socket,id=charmonitor,path=/var/lib/libvirt/qemu/Rh5332.monitor,server,nowait
> >>  -mon chardev=charmonitor,id=monitor,mode=readline -rtc
> >> base=utc -boot cd -drive 
> >> file=/home/akorolev/rh5332.img,if=none,id=drive-ide0-0-0,format=raw -device
> >> ide-drive,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0 -chardev 
> >> file,id=charserial0,path=/home/akorolev/serial.log -device
> >> isa-serial,chardev=charserial0,id=serial0 -usb -vnc 127.0.0.1:0 -k en-us 
> >> -vga cirrus -device ivshmem,shm,size=32M-device
> >> virtio-balloon-pci,id=balloon0
> >>
> >> Tried different guests: Centos 5.8 64bit, RHEL 5.3 32bit, FC 12 64bit on 
> >> all machines hang occurs in 100% cases
> >>
> >> *Why it happens*
> >> The issue basically comes from Linux PCI enumeration code.
> >>
> >> The OS enumerates 64BIT bars when device is enabled using the following 
> >> procedure.
> >> 1. Write all FF's to lower half of 64bit BAR
> >> 2. Write address back to lower half of 64bit BAR
> >> 3. Write all FF's to higher half of 64bit BAR
> >> 4. Write address back to higher half of 64bit BAR
> >>
> >> For qemu it means that  qemu pci_default_write_config() recevies all FFs 
> >> for lower part of the 64bit BAR.
> >> Then it applies the mask and converts the value to "All FF's - size + 1" 
> >> (FE00 if size is 32MB).
> >>
> >> So for short period of time the range [0xFE00 - 0x] will be 
> >> occupied by ivshmem resource.
> >> For some reason it is lethal for further boot process.
> >>
> >> We have found that boot process screws up completely if kvm-apic-msi range 
> >> is overlapped even for short period of time.  (We still don't
> >> know why it happens, hope that the qemu maintainers can answer?)
> >>
> >> If we look at kvm-apic-msi memory region it is a non-overlapable memory 
> >> region with hardcoded address range [0xFEE0 - 0xFEF0].
> > 
> > Thanks for looking into this!
> > 
> >> Here is a log we collected from render_memory_regions:
> >>
> >>  system overlap 0 pri 0 [0x0 - 0x7fff]
> >>  kvmvapic-rom overlap 1 pri 1000 [0xca000 - 0xcd000]
> >>  pc.ram overlap 0 pri 0 [0xca000 - 0xcd000]
> >>  ++ pc.ram [0xca000 - 0xcd000] is added to view
> >>  
> >>  smram-region overlap 1 pri 1 [0xa - 0xc]
> >>  pci overlap 0 pri 0 [0xa - 0xc]
> >>  cirrus-lowmem-container overlap 1 pri 1 [0xa - 0xc]
> >>  cirrus-low-memory overlap 0 pri 0 [0xa - 0xc]
> >> ++cirrus-low-memory [0xa - 0xc] is added to view
> >>  kvm-ioapic overlap 0 pri 0 [0xfec0 - 0xfec01000]
> >> ++kvm-ioapic [0xfec0 - 0xfec01000] is added to view
> >>  pci-hole64 overlap 0 pri 0 [0x1 - 0x4001]
> >>  pci overlap 0 pri 0 [0x1 - 0x4001]
> >>  pci-hole overlap 0 pri 0 [0x7d00 - 0x1]
> > 
> > So we have ioapic and pci-hole which should be non-overlap,
> > actually overlap each other.
> > Isn't this a problem?
> > 
> >>  pci overlap 0 pri 0 [0x7d00 - 0x1]
> >>  ivshmem-bar2-container overlap 1 pri 1 [0xfe00 - 
> >> 0x1]
> >>  ivshmem.bar2 overlap 0 pri 0 [0xfe00 - 0x1]
> >> ++ivshmem.bar2 [0xfe00 - 0xfec0] is added to view
> >> ++ivshmem.bar2  [0xfec01000 - 0x1] is added to view
> >>  ivshmem-mmio overlap 1 pri 1 [0xfebf1000 - 0xfebf1100]
> >>  e1000-mmio overlap 1 pri 1 [0xfeba - 0xfebc]
> >>  cirrus-mmio overlap 1 pri 1 [0xfebf - 0xfebf1000]
> >>  cirrus-pci-bar0 overlap 1 pri 1 [0xfa00 - 0xfc00]
> >>  vga.vram overlap 1 pri 1 [0xfa00 - 0xfa80]
> >> ++vga.vram [0xfa00 - 0xfa80] is added to view
> >>  cirrus-bitblt-mmio overlap 0 pri 0 [0xfb00 - 
> >> 

Re: [Qemu-devel] [Qemu-ppc] [PATCH ppc-next 37/39] target-ppc: Extract POWER7 alias

[David Gibson]

On Tue, Feb 12, 2013 at 11:13:30AM +0100, Andreas Färber wrote:
> Signed-off-by: Andreas Färber 
> ---
>  target-ppc/translate_init.c |3 +--
>  1 Datei geändert, 1 Zeile hinzugefügt(+), 2 Zeilen entfernt(-)
> 
> diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
> index 5e8756c..fa6eba8 100644
> --- a/target-ppc/translate_init.c
> +++ b/target-ppc/translate_init.c
> @@ -7317,7 +7317,6 @@ enum {
>  CPU_POWERPC_POWER6 = 0x003E,
>  CPU_POWERPC_POWER6_5   = 0x0F01, /* POWER6 in POWER5 mode */
>  CPU_POWERPC_POWER6A= 0x0F02,
> -#define CPU_POWERPC_POWER7   CPU_POWERPC_POWER7_v20
>  CPU_POWERPC_POWER7_v20 = 0x003F0200,
>  CPU_POWERPC_POWER7_v21 = 0x003F0201,
>  CPU_POWERPC_POWER7_v23 = 0x003F0203,
> @@ -8705,7 +8704,6 @@ static const ppc_def_t ppc_defs[] = {
>  POWERPC_DEF("POWER6A",   CPU_POWERPC_POWER6A,POWER6)
>  #endif
>  /* POWER7
> */
> -POWERPC_DEF("POWER7",CPU_POWERPC_POWER7, POWER7)
>  POWERPC_DEF("POWER7_v2.0",   CPU_POWERPC_POWER7_v20, POWER7)
>  POWERPC_DEF("POWER7_v2.1",   CPU_POWERPC_POWER7_v21, POWER7)
>  POWERPC_DEF("POWER7_v2.3",   CPU_POWERPC_POWER7_v23, POWER7)
> @@ -9069,6 +9067,7 @@ static const PowerPCCPUAlias ppc_cpu_aliases[] = {
>  { "Boxer", "POWER3" },
>  { "Dino",  "POWER3" },
>  { "POWER3+", "631" },
> +{ "POWER7", "POWER7_v2.0" },

Hrm, we should probably alias plain "POWER7" to the more recent v2.3
rather than v2.0.  They're all identical for now, but it's always
possible that could change in future.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: Digital signature

Re: [Qemu-devel] [RFC PATCH 0/5] Make discard visible to the guests

[Stefan Hajnoczi]

On Fri, Feb 08, 2013 at 02:06:10PM +0100, Paolo Bonzini wrote:
> The major missing step in order to implement thin provisioning is
> exposing discard commands to the guest.  This series implements
> minimal control of this capability.
> 
> Whether to enable discard or not is made a host parameter, so that
> choices in storage technology do not affect the hardware that the
> guest sees.  While it is still possible to fine-tune the parameters
> (at least for SCSI disks), the default granularity is set to the
> logical block size or 4k, whichever is largest, because cluster
> sizes below 4k are rarely used and 4K is a typical block size
> for files.
> 
> Nevertheless, using DISCARD with filesystems can cause very severe
> fragmentation, so it is left default-off for now.  This can change later
> when we implement the "anchor" operation for efficient management of
> preallocated files.
> 
> Paolo
> 
> 
> Paolo Bonzini (5):
>   block: implement BDRV_O_UNMAP
>   blockdev: add discard suboption to -drive
>   qemu-nbd: add --discard option
>   blockdev: enable discard by default
>   pc: add compatibility machine types for 1.4
> 
>  block.c   | 25 +
>  blockdev.c| 11 +++
>  hw/block-common.h |  2 +-
>  hw/ide/qdev.c |  5 -
>  hw/pc.h   | 32 
>  hw/pc_piix.c  | 18 --
>  hw/pc_q35.c   | 19 ---
>  hw/scsi-disk.c| 13 ++---
>  include/block/block.h |  2 ++
>  qemu-io.c | 11 +--
>  qemu-nbd.c| 18 +++---
>  qemu-nbd.texi |  4 
>  qemu-options.hx   |  2 ++
>  13 files changed, 147 insertions(+), 15 deletions(-)
> 
> -- 
> 1.8.1.2
> 
> 

I posted one comment, otherwise looks good.

Re: [Qemu-devel] [Qemu-ppc] [PATCH ppc-next 37/39] target-ppc: Extract POWER7 alias

[Alexander Graf]

On 13.02.2013, at 11:57, David Gibson wrote:

> On Tue, Feb 12, 2013 at 11:13:30AM +0100, Andreas Färber wrote:
>> Signed-off-by: Andreas Färber 
>> ---
>> target-ppc/translate_init.c |3 +--
>> 1 Datei geändert, 1 Zeile hinzugefügt(+), 2 Zeilen entfernt(-)
>> 
>> diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
>> index 5e8756c..fa6eba8 100644
>> --- a/target-ppc/translate_init.c
>> +++ b/target-ppc/translate_init.c
>> @@ -7317,7 +7317,6 @@ enum {
>> CPU_POWERPC_POWER6 = 0x003E,
>> CPU_POWERPC_POWER6_5   = 0x0F01, /* POWER6 in POWER5 mode */
>> CPU_POWERPC_POWER6A= 0x0F02,
>> -#define CPU_POWERPC_POWER7   CPU_POWERPC_POWER7_v20
>> CPU_POWERPC_POWER7_v20 = 0x003F0200,
>> CPU_POWERPC_POWER7_v21 = 0x003F0201,
>> CPU_POWERPC_POWER7_v23 = 0x003F0203,
>> @@ -8705,7 +8704,6 @@ static const ppc_def_t ppc_defs[] = {
>> POWERPC_DEF("POWER6A",   CPU_POWERPC_POWER6A,POWER6)
>> #endif
>> /* POWER7
>> */
>> -POWERPC_DEF("POWER7",CPU_POWERPC_POWER7, POWER7)
>> POWERPC_DEF("POWER7_v2.0",   CPU_POWERPC_POWER7_v20, POWER7)
>> POWERPC_DEF("POWER7_v2.1",   CPU_POWERPC_POWER7_v21, POWER7)
>> POWERPC_DEF("POWER7_v2.3",   CPU_POWERPC_POWER7_v23, POWER7)
>> @@ -9069,6 +9067,7 @@ static const PowerPCCPUAlias ppc_cpu_aliases[] = {
>> { "Boxer", "POWER3" },
>> { "Dino",  "POWER3" },
>> { "POWER3+", "631" },
>> +{ "POWER7", "POWER7_v2.0" },
> 
> Hrm, we should probably alias plain "POWER7" to the more recent v2.3
> rather than v2.0.  They're all identical for now, but it's always
> possible that could change in future.

Agreed, but that's for a future patch. This one is only refactoring :)


Alex

Re: [Qemu-devel] [PATCH ppc-next 00/39] PowerPCCPU subclasses, remixed

[Alexander Graf]

On 12.02.2013, at 11:12, Andreas Färber wrote:

> Hi Alex,
> 
> Here's a polished and squashed series of POWERPC_DEF() QOM'ifications.
> In light of David's patch I am sending this out in the hope that it can be
> applied to ppc-next before David's significantly smaller patch gets applied
> (or that you take care of rebasing yourself).
> 
> It is based on (and depends on) the two for-1.4 fixes I sent out yesterday.
> Further QOM'ifying the glue(..., _type) stuff is intended as follow-up.

Have you verified that this compiles well on a ppc kvm host?


Alex

> 
> Conflicts:
> * Comma on all macro definitions gets removed in patch 1.
> * "Trident" definition gets moved to aliases list in patch 2.
> 
> Available from:
> git://github.com/afaerber/qemu-cpu.git qom-cpu-ppc-types.v1
> https://github.com/afaerber/qemu-cpu/commits/qom-cpu-ppc-types.v1
> 
> Thanks,
> Andreas
> 
> Cc: Alexander Graf 
> Cc: qemu-ppc 
> Cc: David Gibson 
> 
> Andreas Färber (39):
>  target-ppc: Inline comma into POWERPC_DEF_SVR() macro
>  target-ppc: Extract aliases from definitions list
>  target-ppc: Make -cpu "ppc" an alias to "ppc32"
>  target-ppc: Extract MPC5xx aliases
>  target-ppc: Extract MGT823/MPC8xx as aliases
>  target-ppc: Extract 40x aliases
>  target-ppc: Extract 440 aliases
>  target-ppc: Turn "ppc32" and "ppc64" CPUs into aliases
>  target-ppc: Extract 74x7[A] aliases
>  target-ppc: Extract 74x5 as aliases
>  target-ppc: Extract 74x1 aliases
>  target-ppc: Extract 7450 alias
>  target-ppc: Extract 7448 alias
>  target-ppc: Extract 7410 alias
>  target-ppc: Extract 7400 alias
>  target-ppc: Extract 7x5 aliases
>  target-ppc: Extract 750 aliases
>  target-ppc: Extract 740/750 aliases
>  target-ppc: Extract 604e alias
>  target-ppc: Extract 603e alias
>  target-ppc: Extract 603r alias
>  target-ppc: Extract 601/601v aliases
>  target-ppc: Extract MPC85xx aliases
>  target-ppc: Extract e500v1/e500v2 aliases
>  target-ppc: Extract MPC83xx aliases
>  target-ppc: Extract e300 alias
>  target-ppc: Extract e200 alias
>  target-ppc: Extract MPC82xx alias
>  target-ppc: Extract MPC8247/MPC8248/MPC8270-80 aliases
>  target-ppc: Extract MPC82xx aliases to *_HiP4
>  target-ppc: Extract MPC82xx_HiP{3,4} aliases
>  target-ppc: Extract MPC52xx alias
>  target-ppc: Extract MPC5200/MPC5200B aliases
>  target-ppc: Extract MPC8240 alias
>  target-ppc: Extract 405GPe alias
>  target-ppc: Extract 970 aliases
>  target-ppc: Extract POWER7 alias
>  target-ppc: Get model name from type name
>  target-ppc: Convert CPU definitions
> 
> target-ppc/cpu-qom.h|   17 +-
> target-ppc/cpu.h|   20 -
> target-ppc/translate_init.c | 1960 ++-
> 3 Dateien geändert, 850 Zeilen hinzugefügt(+), 1147 Zeilen entfernt(-)
> 
> -- 
> 1.7.10.4
> 

Re: [Qemu-devel] kvm segfaulting

[Paolo Bonzini]

Il 13/02/2013 10:07, Stefan Priebe - Profihost AG ha scritto:
>>> >>
>>> >> commit 47a150a4bbb06e45ef439a8222e9f46a7c4cca3f
> ...
>> > You can certainly try reverting it, but this patch is fixing a real bug.
> Will try that. Yes but even if it fixes a bug and raises another one
> (kvm segfault) which is the worst one. It should be fixed.

The KVM segfault is exposing a potential consistency problem.  What is
worse is not obvious.  Also, it is happening at reset time if this is
the culprit.  Reset usually happens at places where no data loss is caused.

Can you find out what the VM was doing when it segfaulted?  (Or even,
can you place the corefile and kvm executable somewhere where I can
download it?)

I'll prepare a test program that resets the adapter while doing I/O and
try to reproduce it myself, in the meanwhile: can you grep the VM's
/var/log/messages with kernel messages regarding the storage ("aborting
cmd" and other things after it)?  If not, do your VMs reset themselves
often for example?  Can you reproduce it on non-rbd storage?

Paolo

Re: [Qemu-devel] [PATCH 4/7 v2] KVM regsync: Add register bitmap parameter to do_kvm_cpu_synchronize_state

[Jan Kiszka]

On 2013-02-11 23:49, Marcelo Tosatti wrote:
> On Fri, Feb 01, 2013 at 10:47:37AM -0500, Jason J. Herne wrote:
>> On 01/24/2013 07:40 AM, Alexander Graf wrote:
>>> I think for now the best choice for get_regs() would be to ignore the 
>>> FULL/RESET bits and always keep the syncing as it happens today under the 
>>> RUNTIME umbrella only. So all of get_regs() only checks for RUNTIME.
>>>
>>> Whenever get_xxx() happens, a bit gets set for set_xxx(). Up to this point, 
>>> only the RUNTIME bit is ever set, because that's what 
>>> cpu_synchronize_registers() sets.
>>>
>>> Then s390 can add special separate bits for "sync GPRs" and "sync CRs". 
>>> SYNC_RUNTIME would include those bits. The kvm hypercall exit calls a new 
>>> synchronize_registers() function with a parameter telling it to only sync 
>>> GPRs. This marks GPRs dirty, but not RUNTIME. The set_registers() function 
>>> in s390 specific code could handle this particular case specially.
>>>
>>> That way everything's solved and scalable, no?
>>>
>>> Alex
>>>
>>
>> Ok, based on the discussions we've had I think I have a plan of
>> attack based on Alex's above suggestion.  I believe it also
>> satisfies the concerns Marcelo pointed out.  Please correct me if
>> I'm wrong.
>>
>> kvm_arch_get_registers() stays exactly as is for all architectures
>> (reads RUNTIME state only). No new parameters.
> 
> kvm_arch_get_registers reads the entire state, ie. FULL state. 
> 
>> Each architecture defines arch specific bits for runtime/reset/full states:
>> #define KVM_REGSYNC_I386_RUNTIME_BIT  (1 << 1)
>> #define KVM_REGSYNC_I386_RESET_BIT(1 << 2)
>> #define KVM_REGSYNC_I386_FULL_BIT (1 << 3)


These states remain levels, i.e. are cumulative: RESET implies RUNTIME,
FULL implies RESET+RUNTIME. The encoding shall express this.

>>
>> Each architecture defines generic bits (for use in platform agnostic
>> code: kvm-all.c) for runtime/reset/full states:
>> #define KVM_REGSYNC_RUNTIME_STATEKVM_REGSYNC_I386_RUNTIME_BIT
>> #define KVM_REGSYNC_RESET_STATE  KVM_REGSYNC_I386_RESET_BIT
>> #define KVM_REGSYNC_FULL_STATE   KVM_REGSYNC_I386_FULL_BIT
>>
>> S390: replace KVM_REGSYNC_S390_RUNTIME_BIT with two new bits so the
>> S390 arch specific bits look like this:
>> #define KVM_REGSYNC_S390_RUNTIME_SOME_BIT  (1 << 1)
>> #define KVM_REGSYNC_S390_RUNTIME_REST_BIT  (1 << 2)
>> #define KVM_REGSYNC_S390_RESET_BIT (1 << 3)
>> #define KVM_REGSYNC_S390_FULL_BIT  (1 << 4)
>> The idea being that SOME represents the set of RUNTIME registers we
>> always want to read when we exit from KVM.

...and only do s390-specific stuff with the registers. At the point
generic code starts to access the state, you must read the FULL state as
that is what generic bits can assume today.

>>
>> And REST represents the
>> set of RUNTIME registers we want to read for migration/dump and
>> potentially other special cases.  My understanding is that SOME and
>> REST should be mutually exclusive.  I think they need better names
>> as well :).
>>
>> S390 defines it's generic bits like this:
>> #define KVM_REGSYNC_RUNTIME_STATE
>> (KVM_REGSYNC_S390_RUNTIME_SOME_BIT |
>> KVM_REGSYNC_S390_RUNTIME_REST_BIT)
>> #define KVM_REGSYNC_RESET_STATE  KVM_REGSYNC_S390_RESET_BIT
>> #define KVM_REGSYNC_FULL_STATEKVM_REGSYNC_S390_FULL_BIT
>>
>> S390: A new function is created:
>> s390_sync_partial_runtime_registers(int bitmap).  The bitmap
>> argument indicates which of the SOME/REST register sets to read.
>> Either this new function or perhaps the caller will update the
>> cpu->kvm_vcpu_dirty bitmap to indicate which regs are now dirty.
>>
>> S390: On the hot paths we call 
>> s390_sync_partial_runtime_registers(KVM_REGSYNC_S390_RUNTIME_SOME_BIT)
>> instead of cpu_synchronize_state() to read only the set of runtime
>> registers we need on the hot path.  If at some later point
>> cpu_synchronize_state() happens to be called then the S390 version
>> of kvm_arch_get_registers() needs to be smart enough to avoid data
>> loss. So we make it write back all dirty registers
>> (env->kvm_vcpu_dirty) before getting anything.
>>
>> I think this works.  Comments please and thank you!! :)
> 
> The idea behind s390_sync_partial_runtime_registers was that no generic
> modifications have to be done. This (containment of the modifications 
> to S/390) is possible if: 
> 
> 1) The hot paths in question are vcpu local. That is, only executed in
> VCPU context. Which seems to be the case because these hot paths are hot 
> because they are VM-exits handled in userspace.
> 
> Can you confirm this?
> 
> Because frankly, i dislike splitting the register sets without adding 
> accessors
> (so thinking is, either go all the way and have an interface which is
> difficult to make mistakes with or contain the register splitting 
> changes to S/390).
> 
> But, the original author of this interface is Jan Kiskza, so he
> should be consulte

[Qemu-devel] [PATCH][QEMU] vmxcap: Open MSR file in unbuffered mode

[Jan Kiszka]

Python may otherwise decide to to read larger chunks, applying the seek
only on the software buffer. This will return results from the wrong
MSRs.

Signed-off-by: Jan Kiszka 
---
 scripts/kvm/vmxcap |4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/scripts/kvm/vmxcap b/scripts/kvm/vmxcap
index 0b23f77..6363e73 100755
--- a/scripts/kvm/vmxcap
+++ b/scripts/kvm/vmxcap
@@ -27,9 +27,9 @@ MSR_IA32_VMX_VMFUNC = 0x491
 class msr(object):
 def __init__(self):
 try:
-self.f = file('/dev/cpu/0/msr')
+self.f = open('/dev/cpu/0/msr', 'r', 0)
 except:
-self.f = file('/dev/msr0')
+self.f = open('/dev/msr0', 'r', 0)
 def read(self, index, default = None):
 import struct
 self.f.seek(index)
-- 
1.7.3.4

[Qemu-devel] [PATCH][QEMU] vmxcap: Augment reported information

[Jan Kiszka]

Parse the Basic VMX Information MSR and add the bit for the new posted
interrupts.

Signed-off-by: Jan Kiszka 
---
 scripts/kvm/vmxcap |   14 ++
 1 files changed, 14 insertions(+), 0 deletions(-)

diff --git a/scripts/kvm/vmxcap b/scripts/kvm/vmxcap
index 6363e73..a1a44a0 100755
--- a/scripts/kvm/vmxcap
+++ b/scripts/kvm/vmxcap
@@ -96,6 +96,19 @@ class Misc(object):
 print '  %-40s %s' % (self.bits[bits], fmt(v))
 
 controls = [
+Misc(
+name = 'Basic VMX Information',
+bits = {
+(0, 31): 'Revision',
+(32,44): 'VMCS size',
+48: 'VMCS restricted to 32 bit addresses',
+49: 'Dual-monitor support',
+(50, 53): 'VMCS memory type',
+54: 'INS/OUTS instruction information',
+55: 'IA32_VMX_TRUE_*_CTLS support',
+},
+msr = MSR_IA32_VMX_BASIC,
+),
 Control(
 name = 'pin-based controls',
 bits = {
@@ -103,6 +116,7 @@ controls = [
 3: 'NMI exiting',
 5: 'Virtual NMIs',
 6: 'Activate VMX-preemption timer',
+7: 'Process posted interrupts',
 },
 cap_msr = MSR_IA32_VMX_PINBASED_CTLS,
 true_cap_msr = MSR_IA32_VMX_TRUE_PINBASED_CTLS,
-- 
1.7.3.4

Re: [Qemu-devel] [Qemu-ppc] [PATCH ppc-next 37/39] target-ppc: Extract POWER7 alias

[David Gibson]

On Wed, Feb 13, 2013 at 12:15:41PM +0100, Alexander Graf wrote:
> 
> On 13.02.2013, at 11:57, David Gibson wrote:
> 
> > On Tue, Feb 12, 2013 at 11:13:30AM +0100, Andreas Färber wrote:
> >> Signed-off-by: Andreas Färber 
> >> ---
> >> target-ppc/translate_init.c |3 +--
> >> 1 Datei geändert, 1 Zeile hinzugefügt(+), 2 Zeilen entfernt(-)
> >> 
> >> diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
> >> index 5e8756c..fa6eba8 100644
> >> --- a/target-ppc/translate_init.c
> >> +++ b/target-ppc/translate_init.c
> >> @@ -7317,7 +7317,6 @@ enum {
> >> CPU_POWERPC_POWER6 = 0x003E,
> >> CPU_POWERPC_POWER6_5   = 0x0F01, /* POWER6 in POWER5 mode 
> >> */
> >> CPU_POWERPC_POWER6A= 0x0F02,
> >> -#define CPU_POWERPC_POWER7   CPU_POWERPC_POWER7_v20
> >> CPU_POWERPC_POWER7_v20 = 0x003F0200,
> >> CPU_POWERPC_POWER7_v21 = 0x003F0201,
> >> CPU_POWERPC_POWER7_v23 = 0x003F0203,
> >> @@ -8705,7 +8704,6 @@ static const ppc_def_t ppc_defs[] = {
> >> POWERPC_DEF("POWER6A",   CPU_POWERPC_POWER6A,
> >> POWER6)
> >> #endif
> >> /* POWER7  
> >>   */
> >> -POWERPC_DEF("POWER7",CPU_POWERPC_POWER7, 
> >> POWER7)
> >> POWERPC_DEF("POWER7_v2.0",   CPU_POWERPC_POWER7_v20, 
> >> POWER7)
> >> POWERPC_DEF("POWER7_v2.1",   CPU_POWERPC_POWER7_v21, 
> >> POWER7)
> >> POWERPC_DEF("POWER7_v2.3",   CPU_POWERPC_POWER7_v23, 
> >> POWER7)
> >> @@ -9069,6 +9067,7 @@ static const PowerPCCPUAlias ppc_cpu_aliases[] = {
> >> { "Boxer", "POWER3" },
> >> { "Dino",  "POWER3" },
> >> { "POWER3+", "631" },
> >> +{ "POWER7", "POWER7_v2.0" },
> > 
> > Hrm, we should probably alias plain "POWER7" to the more recent v2.3
> > rather than v2.0.  They're all identical for now, but it's always
> > possible that could change in future.
> 
> Agreed, but that's for a future patch. This one is only refactoring
> :)

Yeah, fair enough.  I missed that the PVR it referenced was itself
already set to the v2.0 one.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: Digital signature

Re: [Qemu-devel] [RFC v1 3/3] make address_space_map safe

[Vasilis Liaskovitis]

Hi,

I am looking at this old ref/unref patchset for safely removing hot-plugged
dimms/MemoryRegions. I am not sure if the set is still actively worked on or
relevant for qemu-master, but I had a small comment below:

On Fri, Nov 09, 2012 at 11:14:30AM +0800, Liu Ping Fan wrote:
> From: Liu Ping Fan 
> 
> Signed-off-by: Liu Ping Fan 
> ---
>  cpu-common.h  |8 ++--
>  cputlb.c  |4 ++--
>  dma-helpers.c |4 +++-
>  dma.h |5 -
>  exec.c|   45 +
>  memory.h  |4 +++-
>  target-i386/kvm.c |4 ++--
>  7 files changed, 57 insertions(+), 17 deletions(-)
> 
[snip]
> diff --git a/exec.c b/exec.c
> index e5f1c0f..e9bd695 100644
> --- a/exec.c
> +++ b/exec.c
[snip]
> @@ -3822,7 +3837,8 @@ void address_space_unmap(AddressSpace *as, void 
> *buffer, target_phys_addr_t len,
>  {
>  if (buffer != bounce.buffer) {
>  if (is_write) {
> -ram_addr_t addr1 = qemu_ram_addr_from_host_nofail(buffer);
> +/* Will release RAM refcnt */
> +ram_addr_t addr1 = qemu_ram_addr_from_host_nofail(buffer, true);
>  while (access_len) {
>  unsigned l;
>  l = TARGET_PAGE_SIZE;

Since qemu_ram_addr_from_host_nofail(buffer, true) will decrease the reference
counter for this memoryregion, I think is should be called regardless of
read/write i.e.  outside of the "if (is_write)" clause. Otherwise references for
reads are not decreased properly.

thanks,

- Vasilis

Re: [Qemu-devel] [PATCH for-1.4 1/2] trace: use glib atomic int types

[Juan Quintela]

Markus Armbruster  wrote:
> Stefan Hajnoczi  writes:
>
>> Juan reported that RHEL 6.4 hosts give compiler warnings because we use
>> unsigned int while glib prototypes use volatile gint in trace/simple.c.
>>
>>   trace/simple.c:223: error: pointer targets in passing argument 1
>> of 'g_atomic_int_compare_and_exchange' differ in signedness
>
> Meh.  Contrary to documentation and how current GLib versions behave, in
> other words a bug in need of a workaround.
>
>> These variables are only accessed with glib atomic int functions so
>> let's play it by the book and use volatile gint.
>
> gint is a silly alias for int, and used completely interchangeably, even
> within GLib APIs.  Any pretentions of treating it as something more
> abstract break down at the first printf(), if not earlier.  But if you
> think it helps...

>> -static unsigned int trace_idx;
>> +static volatile gint trace_idx;

Problem was this bit.  int vs unsigned.

Later, Juan.

Re: [Qemu-devel] [PATCH for-1.4 1/2] trace: use glib atomic int types

[Stefan Hajnoczi]

On Wed, Feb 13, 2013 at 01:25:02PM +0100, Juan Quintela wrote:
> Markus Armbruster  wrote:
> > Stefan Hajnoczi  writes:
> >
> >> Juan reported that RHEL 6.4 hosts give compiler warnings because we use
> >> unsigned int while glib prototypes use volatile gint in trace/simple.c.
> >>
> >>   trace/simple.c:223: error: pointer targets in passing argument 1
> >> of 'g_atomic_int_compare_and_exchange' differ in signedness
> >
> > Meh.  Contrary to documentation and how current GLib versions behave, in
> > other words a bug in need of a workaround.
> >
> >> These variables are only accessed with glib atomic int functions so
> >> let's play it by the book and use volatile gint.
> >
> > gint is a silly alias for int, and used completely interchangeably, even
> > within GLib APIs.  Any pretentions of treating it as something more
> > abstract break down at the first printf(), if not earlier.  But if you
> > think it helps...
> 
> >> -static unsigned int trace_idx;
> >> +static volatile gint trace_idx;
> 
> Problem was this bit.  int vs unsigned.

We only access the variable through the glib API, never directly.  So in
this case I don't see much benefit in using nicer types, better to
conform precisely to the function declaration.

Stefan

Re: [Qemu-devel] kvm segfaulting

[Stefan Priebe - Profihost AG]

Hi,
Am 13.02.2013 12:36, schrieb Paolo Bonzini:
> Il 13/02/2013 10:07, Stefan Priebe - Profihost AG ha scritto:
>>
>> commit 47a150a4bbb06e45ef439a8222e9f46a7c4cca3f
>> ...
 You can certainly try reverting it, but this patch is fixing a real bug.
>> Will try that. Yes but even if it fixes a bug and raises another one
>> (kvm segfault) which is the worst one. It should be fixed.
> 
> The KVM segfault is exposing a potential consistency problem.  What is
> worse is not obvious.  Also, it is happening at reset time if this is
> the culprit.  Reset usually happens at places where no data loss is caused.
> 
> Can you find out what the VM was doing when it segfaulted?  (Or even,
> can you place the corefile and kvm executable somewhere where I can
> download it?)

Yes it was doing an fstrim -v / which resulted in:

[45648.453698] end_request: I/O error, dev sda, sector 9066952

> I'll prepare a test program that resets the adapter while doing I/O and
> try to reproduce it myself, in the meanwhile: can you grep the VM's
> /var/log/messages with kernel messages regarding the storage ("aborting
> cmd" and other things after it)? 
Sadly not as i don't have acore dump. The kvm processes are started
through variuos Daemons and there seems no way to activate core dumps
for an already running process and i don't know which VM will crash next.

> If not, do your VMs reset themselves
> often for example?
No

> Can you reproduce it on non-rbd storage?
I don't have another storage type. ;-(

Stefan

Re: [Qemu-devel] [PATCH] qemu-log: default to stderr for logging output

[Markus Armbruster]

Peter Maydell  writes:

> Switch the default for qemu_log logging output from "/tmp/qemu.log"
> to stderr. This is an incompatible change in some sense, but logging
> is mostly used for debugging purposes so it shouldn't affect production
> use. The previous behaviour can be obtained by adding "-D /tmp/qemu.log"
> to the command line.
>
> This change requires us to:
>  * update all the documentation/help text
>  * make linux-user and bsd-user defer to qemu-log for the default
>logging destination rather than overriding it themselves
>  * ensure that all logfile closing is done via qemu_log_close()
>and that that function doesn't close stderr
> as well as the obvious change to the behaviour of do_qemu_set_log()
> when no logfile name has been specified.
>
> Signed-off-by: Peter Maydell 
> ---
> Stefan suggested that qemu_log should default to stderr, and I
> agree that it makes more sense than a random file in /tmp/.
> As noted in the commit message, this is technically an incompatible
> change.
>
> This patchset sits on top of my recent 6 patch qemu_log
> cleanup series.

Related: [PATCH] qemu-log: Remove qemu_log_try_set_file() and its users

>  bsd-user/main.c|   15 +++
>  hmp-commands.hx|4 ++--
>  include/qemu/log.h |8 ++--
>  linux-user/main.c  |   14 --
>  qemu-doc.texi  |8 
>  qemu-log.c |   29 +++--
>  qemu-options.hx|   10 +-
>  tcg/tci/README |2 +-
>  8 files changed, 40 insertions(+), 50 deletions(-)
>
> diff --git a/bsd-user/main.c b/bsd-user/main.c
> index 097fbfe..1ec8636 100644
> --- a/bsd-user/main.c
> +++ b/bsd-user/main.c
> @@ -34,8 +34,6 @@
>  #include "qemu/timer.h"
>  #include "qemu/envlist.h"
>  
> -#define DEBUG_LOGFILE "/tmp/qemu.log"
> -
>  int singlestep;
>  #if defined(CONFIG_USE_GUEST_BASE)
>  unsigned long mmap_min_addr;
> @@ -691,8 +689,8 @@ static void usage(void)
> "-bsd type select emulated BSD type 
> FreeBSD/NetBSD/OpenBSD (default)\n"
> "\n"
> "Debug options:\n"
> -   "-d options   activate log (default logfile=%s)\n"
> -   "-D logfile   override default logfile location\n"
> +   "-d options   activate log (default is to log to stderr)\n"
> +   "-D logfile   write logs to 'logfile' rather than stderr\n"
> "-p pagesize  set the host page size to 'pagesize'\n"
> "-singlestep  always run in singlestep mode\n"
> "-strace  log system calls\n"

No need to mention the default twice.

Pointing to -d help would be nice.

> @@ -709,8 +707,7 @@ static void usage(void)
> ,
> TARGET_ARCH,
> interp_prefix,
> -   x86_stack_size,
> -   DEBUG_LOGFILE);
> +   x86_stack_size);
>  exit(1);
>  }
>  
> @@ -733,7 +730,7 @@ int main(int argc, char **argv)
>  {
>  const char *filename;
>  const char *cpu_model;
> -const char *log_file = DEBUG_LOGFILE;
> +const char *log_file = NULL;
>  const char *log_mask = NULL;
>  struct target_pt_regs regs1, *regs = ®s1;
>  struct image_info info1, *info = &info1;
> @@ -861,7 +858,9 @@ int main(int argc, char **argv)
>  }
>  
>  /* init debug */
> -qemu_set_log_filename(log_file);
> +if (log_file) {
> +qemu_set_log_filename(log_file);
> +}
>  if (log_mask) {
>  int mask;
>  

Doesn't qemu_set_log_filename(NULL) do the right thing?

> diff --git a/hmp-commands.hx b/hmp-commands.hx
> index 64008a9..cef7708 100644
> --- a/hmp-commands.hx
> +++ b/hmp-commands.hx
> @@ -295,14 +295,14 @@ ETEXI
>  .name   = "log",
>  .args_type  = "items:s",
>  .params = "item1[,...]",
> -.help   = "activate logging of the specified items to 
> '/tmp/qemu.log'",
> +.help   = "activate logging of the specified items",
>  .mhandler.cmd = do_log,
>  },
>  
>  STEXI
>  @item log @var{item1}[,...]
>  @findex log
> -Activate logging of the specified items to @file{/tmp/qemu.log}.
> +Activate logging of the specified items.
>  ETEXI
>  
>  {
> diff --git a/include/qemu/log.h b/include/qemu/log.h
> index 4527003..6b0db02 100644
> --- a/include/qemu/log.h
> +++ b/include/qemu/log.h
> @@ -116,8 +116,12 @@ static inline void qemu_log_flush(void)
>  /* Close the log file */
>  static inline void qemu_log_close(void)
>  {
> -fclose(qemu_logfile);
> -qemu_logfile = NULL;
> +if (qemu_logfile) {
> +if (qemu_logfile != stderr) {
> +fclose(qemu_logfile);
> +}
> +qemu_logfile = NULL;
> +}
>  }
>  
>  /* Set up a new log file */
> diff --git a/linux-user/main.c b/linux-user/main.c
> index 8a61ea4..55e8326 100644
> --- a/linux-user/main.c
> +++ b/linux-user/main.c
> @@ -35,8 +35,6 @@
>  #include "qemu/envlist.h"
>  #include "elf.h"
>  
> -#define DEBUG_LOGFILE "/tmp/qemu.log"
> -
>  char *exec_path;
>  
>  int singlestep

[Qemu-devel] [RFC PATCH v2 01/23] qcow2: Handle dependencies earlier

[Kevin Wolf]

Handling overlapping allocations isn't just a detail of cluster
allocation. It is rather one of three ways to get the host cluster
offset for a write request:

1. If a request overlaps an in-flight allocations, the cluster offset
   can be taken from there (this is what handle_dependencies will evolve
   into) or the request must just wait until the allocation has
   completed. Accessing the L2 is not valid in this case, it has
   outdated information.

2. Outside overlapping areas, check the clusters that can be written to
   as they are, with no COW involved.

3. If a COW is required, allocate new clusters

Changing the code to reflect this doesn't change the behaviour because
overlaps cannot exist for clusters that are kept in step 2. It does
however make it easier for later patches to work on clusters that belong
to an allocation that is still in flight.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   59 +++-
 block/qcow2.h |5 
 2 files changed, 48 insertions(+), 16 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 56fccf9..c3d57e0 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -821,16 +821,10 @@ static int do_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t guest_offset,
 uint64_t *host_offset, unsigned int *nb_clusters)
 {
 BDRVQcowState *s = bs->opaque;
-int ret;
 
 trace_qcow2_do_alloc_clusters_offset(qemu_coroutine_self(), guest_offset,
  *host_offset, *nb_clusters);
 
-ret = handle_dependencies(bs, guest_offset, nb_clusters);
-if (ret < 0) {
-return ret;
-}
-
 /* Allocate new clusters */
 trace_qcow2_cluster_alloc_phys(qemu_coroutine_self());
 if (*host_offset == 0) {
@@ -842,7 +836,7 @@ static int do_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t guest_offset,
 *host_offset = cluster_offset;
 return 0;
 } else {
-ret = qcow2_alloc_clusters_at(bs, *host_offset, *nb_clusters);
+int ret = qcow2_alloc_clusters_at(bs, *host_offset, *nb_clusters);
 if (ret < 0) {
 return ret;
 }
@@ -882,20 +876,55 @@ int qcow2_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t offset,
 trace_qcow2_alloc_clusters_offset(qemu_coroutine_self(), offset,
   n_start, n_end);
 
-/* Find L2 entry for the first involved cluster */
 again:
-ret = get_cluster_table(bs, offset, &l2_table, &l2_index);
-if (ret < 0) {
-return ret;
-}
-
 /*
  * Calculate the number of clusters to look for. We stop at L2 table
  * boundaries to keep things simple.
  */
+l2_index = offset_to_l2_index(s, offset);
 nb_clusters = MIN(size_to_clusters(s, n_end << BDRV_SECTOR_BITS),
   s->l2_size - l2_index);
 
+/*
+ * Now start gathering as many contiguous clusters as possible:
+ *
+ * 1. Check for overlaps with in-flight allocations
+ *
+ *  a) Overlap not in the first cluster -> shorten this request and let
+ * the caller handle the rest in its next loop iteration.
+ *
+ *  b) Real overlaps of two requests. Yield and restart the search for
+ * contiguous clusters (the situation could have changed while we
+ * were sleeping)
+ *
+ *  c) TODO: Request starts in the same cluster as the in-flight
+ * allocation ends. Shorten the COW of the in-fight allocation, set
+ * cluster_offset to write to the same cluster and set up the right
+ * synchronisation between the in-flight request and the new one.
+ *
+ * 2. Count contiguous COPIED clusters.
+ *TODO: Consider cluster_offset if set in step 1c.
+ *
+ * 3. If the request still hasn't completed, allocate new clusters,
+ *considering any cluster_offset of steps 1c or 2.
+ */
+ret = handle_dependencies(bs, offset, &nb_clusters);
+if (ret == -EAGAIN) {
+goto again;
+} else if (ret < 0) {
+return ret;
+} else {
+/* handle_dependencies() may have decreased cur_bytes (shortened
+ * the allocations below) so that the next dependency is processed
+ * correctly during the next loop iteration. */
+}
+
+/* Find L2 entry for the first involved cluster */
+ret = get_cluster_table(bs, offset, &l2_table, &l2_index);
+if (ret < 0) {
+return ret;
+}
+
 cluster_offset = be64_to_cpu(l2_table[l2_index]);
 
 /*
@@ -960,9 +989,7 @@ again:
 /* Allocate, if necessary at a given offset in the image file */
 ret = do_alloc_cluster_offset(bs, alloc_offset, &alloc_cluster_offset,
   &nb_clusters);
-if (ret == -EAGAIN) {
-goto again;
-} else if (ret < 0) {
+if (ret < 0) {
 goto fail;
 }
 
diff --git a/b

[Qemu-devel] [RFC PATCH v2 02/23] qcow2: Improve check for overlapping allocations

[Kevin Wolf]

The old code detected an overlapping allocation even when the
allocations didn't actually overlap, but were only adjacent.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c  |2 +-
 tests/qemu-iotests/038.out |   10 +-
 2 files changed, 6 insertions(+), 6 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index c3d57e0..0e804ba 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -770,7 +770,7 @@ static int handle_dependencies(BlockDriverState *bs, 
uint64_t guest_offset,
 uint64_t old_start = old_alloc->offset >> s->cluster_bits;
 uint64_t old_end = old_start + old_alloc->nb_clusters;
 
-if (end < old_start || start > old_end) {
+if (end <= old_start || start >= old_end) {
 /* No intersection */
 } else {
 if (start < old_start) {
diff --git a/tests/qemu-iotests/038.out b/tests/qemu-iotests/038.out
index acc7629..9cd0cd8 100644
--- a/tests/qemu-iotests/038.out
+++ b/tests/qemu-iotests/038.out
@@ -517,9 +517,7 @@ qemu-io> wrote 65536/65536 bytes at offset 16711680
 qemu-io> Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=6442450944 
backing_file='TEST_DIR/t.IMGFMT.base' 
 
 == Some concurrent requests touching the same cluster ==
-qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> wrote 
81920/81920 bytes at offset XXX
-80 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset XXX
+qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> 
qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> qemu-io> wrote 
65536/65536 bytes at offset XXX
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 65536/65536 bytes at offset XXX
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
@@ -579,6 +577,8 @@ wrote 65536/65536 bytes at offset XXX
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 65536/65536 bytes at offset XXX
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 81920/81920 bytes at offset XXX
+80 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 65536/65536 bytes at offset XXX
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 65536/65536 bytes at offset XXX
@@ -645,6 +645,8 @@ wrote 65536/65536 bytes at offset XXX
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 65536/65536 bytes at offset XXX
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 81920/81920 bytes at offset XXX
+80 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 65536/65536 bytes at offset XXX
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 65536/65536 bytes at offset XXX
@@ -703,8 +705,6 @@ wrote 65536/65536 bytes at offset XXX
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 65536/65536 bytes at offset XXX
 64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 81920/81920 bytes at offset XXX
-80 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 
 == Verify image content ==
 qemu-io> read 4096/4096 bytes at offset 2064384
-- 
1.7.6.5

[Qemu-devel] [RFC PATCH v2 14/23] qcow2: Use byte granularity in qcow2_alloc_cluster_offset()

[Kevin Wolf]

This gets rid of the nb_clusters and keep_clusters and the associated
complicated calculations. Just advance the number of bytes that have
been processed and everything is fine.

This patch advances the variables even after the last operation even
though they aren't used any more afterwards to make things look more
uniform. A later patch will turn the whole thing into a loop and then
it actually starts making sense.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   82 +
 1 files changed, 28 insertions(+), 54 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 90fe36c..9e6b746 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1116,28 +1116,24 @@ int qcow2_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t offset,
 int n_start, int n_end, int *num, uint64_t *host_offset, QCowL2Meta **m)
 {
 BDRVQcowState *s = bs->opaque;
-int l2_index, ret, sectors;
-unsigned int nb_clusters, keep_clusters;
+uint64_t start, remaining;
 uint64_t cluster_offset;
 uint64_t cur_bytes;
+int ret;
 
 trace_qcow2_alloc_clusters_offset(qemu_coroutine_self(), offset,
   n_start, n_end);
 
+assert(n_start * BDRV_SECTOR_SIZE == offset_into_cluster(s, offset));
+offset = start_of_cluster(s, offset);
+
 again:
+start = offset + (n_start << BDRV_SECTOR_BITS);
+remaining = (n_end - n_start) << BDRV_SECTOR_BITS;
 cluster_offset = 0;
 *host_offset = 0;
 
 /*
- * Calculate the number of clusters to look for. We stop at L2 table
- * boundaries to keep things simple.
- */
-l2_index = offset_to_l2_index(s, offset);
-nb_clusters = MIN(size_to_clusters(s, n_end << BDRV_SECTOR_BITS),
-  s->l2_size - l2_index);
-n_end = MIN(n_end, nb_clusters * s->cluster_sectors);
-
-/*
  * Now start gathering as many contiguous clusters as possible:
  *
  * 1. Check for overlaps with in-flight allocations
@@ -1154,8 +1150,8 @@ again:
  * cluster_offset to write to the same cluster and set up the right
  * synchronisation between the in-flight request and the new one.
  */
-cur_bytes = (n_end - n_start) * BDRV_SECTOR_SIZE;
-ret = handle_dependencies(bs, offset, &cur_bytes);
+cur_bytes = remaining;
+ret = handle_dependencies(bs, start, &cur_bytes);
 if (ret == -EAGAIN) {
 goto again;
 } else if (ret < 0) {
@@ -1166,31 +1162,28 @@ again:
  * correctly during the next loop iteration. */
 }
 
-nb_clusters = size_to_clusters(s, offset + cur_bytes)
-- (offset >> s->cluster_bits);
-
 /*
  * 2. Count contiguous COPIED clusters.
  */
-ret = handle_copied(bs, offset, &cluster_offset, &cur_bytes, m);
+ret = handle_copied(bs, start, &cluster_offset, &cur_bytes, m);
 if (ret < 0) {
 return ret;
 } else if (ret) {
-keep_clusters =
-size_to_clusters(s, cur_bytes + offset_into_cluster(s, offset));
-nb_clusters -= keep_clusters;
-
 if (!*host_offset) {
 *host_offset = cluster_offset;
 }
+
+start   += cur_bytes;
+remaining   -= cur_bytes;
+cluster_offset  += cur_bytes;
+
+cur_bytes = remaining;
 } else if (cur_bytes == 0) {
 goto done;
-} else {
-keep_clusters = 0;
 }
 
 /* If there is something left to allocate, do that now */
-if (nb_clusters == 0) {
+if (remaining == 0) {
 goto done;
 }
 
@@ -1198,43 +1191,24 @@ again:
  * 3. If the request still hasn't completed, allocate new clusters,
  *considering any cluster_offset of steps 1c or 2.
  */
-int alloc_n_start;
-int alloc_n_end;
-
-if (keep_clusters != 0) {
-offset = start_of_cluster(s, offset
- + keep_clusters * 
s->cluster_size);
-cluster_offset = start_of_cluster(s, cluster_offset
- + keep_clusters * 
s->cluster_size);
-
-alloc_n_start = 0;
-alloc_n_end = n_end - keep_clusters * s->cluster_sectors;
-} else {
-alloc_n_start = n_start;
-alloc_n_end = n_end;
-}
-
-cur_bytes = ((alloc_n_end - alloc_n_start) << BDRV_SECTOR_BITS);
-
-ret = handle_alloc(bs, offset, &cluster_offset, &cur_bytes, m);
+ret = handle_alloc(bs, start, &cluster_offset, &cur_bytes, m);
 if (ret < 0) {
 return ret;
-}
+} else if (ret) {
+if (!*host_offset) {
+*host_offset = cluster_offset;
+}
 
-if (!*host_offset) {
-*host_offset = cluster_offset;
+start   += cur_bytes;
+remaining   -= cur_bytes;
+cluster_offset  += cur_bytes;
 }
-nb_clusters = size_to_clusters(s, cur_bytes + offset_into_cluster(s, 
offset));
 
 /* Some cleanup work 

[Qemu-devel] [RFC PATCH v2 19/23] qcow2: Add error handling to the l2meta coroutine

[Kevin Wolf]

Not exactly bisectable, but one large patch isn't much better either :-(

m->error is used to allow bdrv_drain() to stop with l2meta in error
state rather than go into an endless loop.

Signed-off-by: Kevin Wolf 
---
 block/qcow2.c |   44 
 block/qcow2.h |3 +++
 2 files changed, 43 insertions(+), 4 deletions(-)

diff --git a/block/qcow2.c b/block/qcow2.c
index 57552aa..2819336 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -774,11 +774,33 @@ static void coroutine_fn process_l2meta(void *opaque)
 m->sleeping = false;
 }
 
+again:
 qemu_co_mutex_lock(&s->lock);
 
 ret = qcow2_alloc_cluster_link_l2(bs, m);
 if (ret < 0) {
-/* FIXME */
+/*
+ * This is a nasty situation: We have already completed the allocation
+ * write request and returned success, so just failing it isn't
+ * possible. We need to make sure to return an error during the next
+ * flush.
+ *
+ * However, we still can't drop the l2meta because we want I/O errors
+ * to be recoverable e.g. after the block device has been grown or the
+ * network connection restored. Sleep until the next flush comes and
+ * then retry.
+ */
+s->flush_error = ret;
+
+qemu_co_mutex_unlock(&s->lock);
+qemu_co_rwlock_unlock(&s->l2meta_flush);
+m->sleeping = true;
+m->error = true;
+qemu_coroutine_yield();
+m->error = false;
+m->sleeping = false;
+qemu_co_rwlock_rdlock(&s->l2meta_flush);
+goto again;
 }
 
 qemu_co_mutex_unlock(&s->lock);
@@ -801,11 +823,12 @@ static bool qcow2_drain(BlockDriverState *bs)
 {
 BDRVQcowState *s = bs->opaque;
 QCowL2Meta *m;
+bool busy = false;
 
 s->in_l2meta_flush = true;
 again:
 QLIST_FOREACH(m, &s->cluster_allocs, next_in_flight) {
-if (m->sleeping) {
+if (m->sleeping && !m->error) {
 qemu_coroutine_enter(m->co, NULL);
 /* next_in_flight link could have become invalid */
 goto again;
@@ -813,7 +836,19 @@ again:
 }
 s->in_l2meta_flush = false;
 
-return !QLIST_EMPTY(&s->cluster_allocs);
+/*
+ * If there's still a sleeping l2meta, then an error must have occured.
+ * Don't consider l2metas in this state as busy, they only get active on
+ * flushes.
+ */
+QLIST_FOREACH(m, &s->cluster_allocs, next_in_flight) {
+if (!m->sleeping) {
+busy = true;
+break;
+}
+}
+
+return busy;
 }
 
 static inline coroutine_fn void stop_l2meta(BlockDriverState *bs)
@@ -1683,7 +1718,8 @@ static coroutine_fn int 
qcow2_co_flush_to_os(BlockDriverState *bs)
 }
 }
 
-ret = 0;
+ret = s->flush_error;
+s->flush_error = 0;
 fail:
 qemu_co_mutex_unlock(&s->lock);
 resume_l2meta(bs);
diff --git a/block/qcow2.h b/block/qcow2.h
index 1d7cdab..504f10f 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -171,6 +171,8 @@ typedef struct BDRVQcowState {
 CoRwlock l2meta_flush;
 bool in_l2meta_flush;
 
+int flush_error;
+
 uint32_t crypt_method; /* current crypt method, 0 if no key yet */
 uint32_t crypt_method_header;
 AES_KEY aes_encrypt_key;
@@ -250,6 +252,7 @@ typedef struct QCowL2Meta
  * be reentered in order to cancel the timer.
  */
 bool sleeping;
+bool error;
 
 /** Coroutine that handles delayed COW and updates L2 entry */
 Coroutine *co;
-- 
1.7.6.5

[Qemu-devel] [RFC PATCH v2 17/23] qcow2: Move COW and L2 update into own coroutine

[Kevin Wolf]

This creates a separate coroutine for processing the COW and the L2
table update of allocating requests. The request itself can then
complete while the second part is still being processed.

We need a qemu_aio_flush() hook in order to ensure that these
coroutines for the second part aren't still running after bdrv_drain_all
(e.g. when the VM is stopped).

Signed-off-by: Kevin Wolf 
---
 block.c   |5 ++
 block/qcow2.c |  131 ++---
 block/qcow2.h |8 +++
 include/block/block_int.h |3 +
 4 files changed, 127 insertions(+), 20 deletions(-)

diff --git a/block.c b/block.c
index 50dab8e..5ae80a0 100644
--- a/block.c
+++ b/block.c
@@ -1225,7 +1225,12 @@ void bdrv_drain_all(void)
 qemu_co_queue_restart_all(&bs->throttled_reqs);
 busy = true;
 }
+
+if (bs->drv && bs->drv->bdrv_drain) {
+busy |= bs->drv->bdrv_drain(bs);
+}
 }
+
 } while (busy);
 
 /* If requests are still pending there is a bug somewhere */
diff --git a/block/qcow2.c b/block/qcow2.c
index 07f7493..3f169b8 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -483,6 +483,7 @@ static int qcow2_open(BlockDriverState *bs, int flags)
 
 /* Initialise locks */
 qemu_co_mutex_init(&s->lock);
+qemu_co_rwlock_init(&s->l2meta_flush);
 
 /* Repair image if dirty */
 if (!(flags & BDRV_O_CHECK) && !bs->read_only &&
@@ -745,6 +746,70 @@ fail:
 return ret;
 }
 
+typedef struct ProcessL2Meta {
+BlockDriverState *bs;
+QCowL2Meta *m;
+} ProcessL2Meta;
+
+/**
+ * Processes the second part of a request that wrote to newly allocated
+ * clusters (most importantly, doing COW and updating the L2).
+ *
+ * Make sure that s->l2meta_flush is held as a reader when when entering the
+ * coroutine.
+ */
+static void coroutine_fn process_l2meta(void *opaque)
+{
+ProcessL2Meta *p = opaque;
+QCowL2Meta *m = p->m;
+BlockDriverState *bs = p->bs;
+BDRVQcowState *s = bs->opaque;
+int ret;
+
+assert(s->l2meta_flush.reader > 0);
+qemu_co_mutex_lock(&s->lock);
+
+ret = qcow2_alloc_cluster_link_l2(bs, m);
+if (ret < 0) {
+/* FIXME */
+}
+
+qemu_co_mutex_unlock(&s->lock);
+
+/* Take the request off the list of running requests */
+if (m->nb_clusters != 0) {
+QLIST_REMOVE(m, next_in_flight);
+}
+
+/* Meanwhile some new dependencies could have accumulated */
+qemu_co_queue_restart_all(&m->dependent_requests);
+
+g_free(m);
+
+qemu_co_rwlock_unlock(&s->l2meta_flush);
+}
+
+static bool qcow2_drain(BlockDriverState *bs)
+{
+BDRVQcowState *s = bs->opaque;
+
+return !QLIST_EMPTY(&s->cluster_allocs);
+}
+
+static inline coroutine_fn void stop_l2meta(BlockDriverState *bs)
+{
+BDRVQcowState *s = bs->opaque;
+
+qemu_co_rwlock_wrlock(&s->l2meta_flush);
+}
+
+static inline coroutine_fn void resume_l2meta(BlockDriverState *bs)
+{
+BDRVQcowState *s = bs->opaque;
+
+qemu_co_rwlock_unlock(&s->l2meta_flush);
+}
+
 static coroutine_fn int qcow2_co_writev(BlockDriverState *bs,
int64_t sector_num,
int remaining_sectors,
@@ -824,26 +889,37 @@ static coroutine_fn int qcow2_co_writev(BlockDriverState 
*bs,
 goto fail;
 }
 
-while (l2meta != NULL) {
-QCowL2Meta *next;
+if (l2meta != NULL) {
+qemu_co_mutex_unlock(&s->lock);
 
-l2meta->is_written = true;
+while (l2meta != NULL) {
+Coroutine *co;
+QCowL2Meta *next;
 
-ret = qcow2_alloc_cluster_link_l2(bs, l2meta);
-if (ret < 0) {
-goto fail;
-}
+ProcessL2Meta p = {
+.bs = bs,
+.m  = l2meta,
+};
 
-/* Take the request off the list of running requests */
-if (l2meta->nb_clusters != 0) {
-QLIST_REMOVE(l2meta, next_in_flight);
-}
+/*
+ * Must take l2meta_flush already here instead of in the
+ * coroutine; otherwise it would be possible that a concurrent
+ * flush would claim that this request is written to the disk
+ * when the metadata isn't written yet in fact.
+ */
+qemu_co_rwlock_rdlock(&s->l2meta_flush);
+l2meta->is_written = true;
+
+/* l2meta might already be freed after the coroutine has run */
+next = l2meta->next;
 
-qemu_co_queue_restart_all(&l2meta->dependent_requests);
+co = qemu_coroutine_create(process_l2meta);
+qemu_coroutine_enter(co, &p);
 
-next = l2meta->next;
-g_free(l2meta);
-l2meta = next;
+l2meta = next;
+}
+
+   

[Qemu-devel] [RFC PATCH v2 22/23] qcow2: Move cluster gathering to a non-looping loop

[Kevin Wolf]

This patch is mainly to separate the indentation change from the
semantic changes. All that really changes here is that everything moves
into a while loop, all 'goto done' become 'break' and at the end of the
loop a new 'break is inserted.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |  150 +---
 1 files changed, 78 insertions(+), 72 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 3ef1cff..5a97d87 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1336,89 +1336,95 @@ again:
 *host_offset = 0;
 *m = NULL;
 
-/*
- * Now start gathering as many contiguous clusters as possible:
- *
- * 1. Check for overlaps with in-flight allocations
- *
- *  a) Overlap not in the first cluster -> shorten this request and let
- * the caller handle the rest in its next loop iteration.
- *
- *  b) Real overlaps of two requests. Yield and restart the search for
- * contiguous clusters (the situation could have changed while we
- * were sleeping)
- *
- *  c) Request starts in the same cluster as the in-flight
- * allocation ends. Shorten the COW of the in-fight allocation, set
- * cluster_offset to write to the same cluster and set up the right
- * synchronisation between the in-flight request and the new one.
- */
-cur_bytes = remaining;
-ret = handle_dependencies(bs, start, &cluster_offset, &cur_bytes, m);
-if (ret == -EAGAIN) {
-goto again;
-} else if (ret < 0) {
-return ret;
-} else if (ret) {
-*host_offset = start_of_cluster(s, cluster_offset);
+while (true) {
+/*
+ * Now start gathering as many contiguous clusters as possible:
+ *
+ * 1. Check for overlaps with in-flight allocations
+ *
+ *  a) Overlap not in the first cluster -> shorten this request and
+ * let the caller handle the rest in its next loop iteration.
+ *
+ *  b) Real overlaps of two requests. Yield and restart the search
+ * for contiguous clusters (the situation could have changed
+ * while we were sleeping)
+ *
+ *  c) Request starts in the same cluster as the in-flight
+ * allocation ends. Shorten the COW of the in-fight allocation,
+ * set cluster_offset to write to the same cluster and set up
+ * the right synchronisation between the in-flight request and
+ * the new one.
+ */
+cur_bytes = remaining;
+ret = handle_dependencies(bs, start, &cluster_offset, &cur_bytes, m);
+if (ret == -EAGAIN) {
+goto again;
+} else if (ret < 0) {
+return ret;
+} else if (ret) {
+*host_offset = start_of_cluster(s, cluster_offset);
 
-start   += cur_bytes;
-remaining   -= cur_bytes;
-cluster_offset  += cur_bytes;
+start   += cur_bytes;
+remaining   -= cur_bytes;
+cluster_offset  += cur_bytes;
 
-goto done;
-} else if (cur_bytes == 0) {
-goto done;
-} else {
-/* handle_dependencies() may have decreased cur_bytes (shortened
- * the allocations below) so that the next dependency is processed
- * correctly during the next loop iteration. */
-}
-
-/*
- * 2. Count contiguous COPIED clusters.
- */
-ret = handle_copied(bs, start, &cluster_offset, &cur_bytes, m);
-if (ret < 0) {
-return ret;
-} else if (ret) {
-if (!*host_offset) {
-*host_offset = cluster_offset;
+break;
+} else if (cur_bytes == 0) {
+break;
+} else {
+/* handle_dependencies() may have decreased cur_bytes (shortened
+ * the allocations below) so that the next dependency is processed
+ * correctly during the next loop iteration. */
 }
 
-start   += cur_bytes;
-remaining   -= cur_bytes;
-cluster_offset  += cur_bytes;
+/*
+ * 2. Count contiguous COPIED clusters.
+ */
+ret = handle_copied(bs, start, &cluster_offset, &cur_bytes, m);
+if (ret < 0) {
+return ret;
+} else if (ret) {
+if (!*host_offset) {
+*host_offset = cluster_offset;
+}
 
-cur_bytes = remaining;
-} else if (cur_bytes == 0) {
-goto done;
-}
+start   += cur_bytes;
+remaining   -= cur_bytes;
+cluster_offset  += cur_bytes;
 
-/* If there is something left to allocate, do that now */
-if (remaining == 0) {
-goto done;
-}
+cur_bytes = remaining;
+} else if (cur_bytes == 0) {
+br

Re: [Qemu-devel] [PATCH for-1.4] qapi: Improve chardev-add documentation

[Markus Armbruster]

Anthony Liguori  writes:

> mdroth  writes:
>
>> On Tue, Feb 12, 2013 at 05:56:00PM +0100, Markus Armbruster wrote:
>>> Gerd Hoffmann  writes:
>>> 
>>> >   Hi,
>>> >
>>> >> But why nested discriminators?
>>> >> 
>>> >> regular files: type=file
>>> >> serial   : type=port, data.type=serial
>>> >> parallel : type=port, data.type=parallel
>>> >> 
>>> >> Simpler, and closer to existing -chardev:
>>> >> 
>>> >> regular files: type=file
>>> >> serial   : type=serial
>>> >> parallel : type=parallel
>>> >
>>> > Matter of taste IMHO.
>>> > I can live with that too.
>>> > Feel free to send patches.
>>> >
>>> >> I also dislike the pointless '"data" : {}' required by type pty and
>>> >> null, but I can't figure out how to express 'void' in the schema.
>>> >
>>> > Someone mentioned it is possible to leave out empty data altogether.
>>> > Didn't try whenever our marshaller actually accepts that though.
>>> 
>>> Looks like it doesn't :(
>>> 
>>> Empty objects work fine here:
>>> 
>>> { 'type': 'ChardevDummy', 'data': { } }
>>> 
>>> Generates the obvious
>>> 
>>> struct ChardevDummy
>>> {
>>> };
>>> 
>>> They don't work here:
>>> 
>>> { 'union': 'ChardevBackend', 'data': { 'file'   : 'ChardevFile',
>>>'hostdev': 'ChardevHostdev',
>>>'socket' : 'ChardevSocket',
>>>'pty': 'ChardevDummy',
>>>'null'   : {} } }
>>> 
>>> Generates
>>> 
>>> struct ChardevBackend
>>> {
>>> ChardevBackendKind kind;
>>> union {
>>> void *data;
>>> ChardevFile * file;
>>> ChardevHostdev * hostdev;
>>> ChardevSocket * socket;
>>> ChardevDummy * pty;
>>> void null;
>>> };
>>> };
>>> 
>>> which is kind of cute, but the compiler doesn't like it.
>>> 
>>> Anthony, Mike, is this a bug?
>>
>> Not exactly, but it's a relic that doesn't seem to be needed anymore.
>> The code that does this is in scripts/qapi.py:
>>
>> def c_type(name):
>> ...
>> elif name == None or len(name) == 0:
>> return 'void'
>> ...
>> else:
>> return '%s *' % name
>>
>> The 'name' param being the value/type of a particular param/key in a
>> QAPI dictionary that defines a schema-defined type.
>>
>> The reason '{}' maps to 'void' is so that in qapi-commands.py, where we 
>> generate
>> stuff like the function signatures for qmp commands, we'd map something like:
>>
>> { 'command': 'my_cmd',
>>   'data': { 'param1': 'Foo' },
>>   'returns': {} }
>>
>> to:
>>
>> void my_cmd(Foo *param1);
>>
>> At some point in development we rightly decided that:
>>
>> { 'command': 'my_cmd',
>>   'data': { 'param1': 'Foo' } }
>>
>> was more efficient, which triggers the 'name == None' case and has the same
>> effect.
>>
>> So, as long as we stay consistent about defining commands in this fashion, we
>> can map 'param_name': {} to something like 'struct {}', and use it in place 
>> of
>> schema-defined dummy types.
>>
>> Though, as I mentioned on IRC, I think just defining a:
>>
>> { 'type': 'Dummy', 'data' {} }
>>
>> Somewhere in the schema and re-using that might actually be cleaner and have
>> the same affect.
>
> Yes.  I don't think we really ought to support inline structures.  It
> keeps the declarations easier to read to make all structs top level
> types.

This argument smells too much of Pascal for my taste.  But it's not
important enough for me to argue.

More important: how things look on the wire.  Here's an instance of
union ChardevBackend:

{ "type" : "null", "data" : {} }

Note that member "data" is mandatory, and must be empty.  Design bug or
feature?

By the way, omitting it results in a mildly bogus error message:

{ "execute" : "chardev-add", "arguments" : { "id" : "foo", "backend" : { 
"type" : "null" } } }
{"error": {"class": "GenericError", "desc": "Invalid parameter type for 
'data', expected: QDict"}}

'data' doesn't have an "invalid parameter type", it's not there.

Re: [Qemu-devel] kvm segfaulting

[Paolo Bonzini]

Il 13/02/2013 13:55, Stefan Priebe - Profihost AG ha scritto:
> Hi,
> Am 13.02.2013 12:36, schrieb Paolo Bonzini:
>> Il 13/02/2013 10:07, Stefan Priebe - Profihost AG ha scritto:
>>>
>>> commit 47a150a4bbb06e45ef439a8222e9f46a7c4cca3f
>>> ...
> You can certainly try reverting it, but this patch is fixing a real bug.
>>> Will try that. Yes but even if it fixes a bug and raises another one
>>> (kvm segfault) which is the worst one. It should be fixed.
>>
>> The KVM segfault is exposing a potential consistency problem.  What is
>> worse is not obvious.  Also, it is happening at reset time if this is
>> the culprit.  Reset usually happens at places where no data loss is caused.
>>
>> Can you find out what the VM was doing when it segfaulted?  (Or even,
>> can you place the corefile and kvm executable somewhere where I can
>> download it?)
> 
> Yes it was doing an fstrim -v / which resulted in:
> 
> [45648.453698] end_request: I/O error, dev sda, sector 9066952

Ok, very helpful.  One thing is to find why this failed.  This can
come later though.

First of all, please run "cat /sys/block/*/device/scsi_disk/*/provisioning_mode"
in a VM with a similar configuration as the one that crashed last.

Second, I attach another patch.

Third, if possible please compile QEMU with --enable-trace-backend=simple,
and run it with

  -trace events='bdrv_aio_discard
scsi_req_cancel
',file=qemu.$$.trace

This can give some clues.  The files should remain quite small,
so you can enable it on all VMs safely.

> Sadly not as i don't have acore dump. The kvm processes are started
> through variuos Daemons and there seems no way to activate core dumps
> for an already running process and i don't know which VM will crash next.

Probably the next that invokes fstrim. :)

>> If not, do your VMs reset themselves
>> often for example?
> No

Ok, good to know.

>> Can you reproduce it on non-rbd storage?
> I don't have another storage type. ;-(

No problem.

Paolo
diff --git a/hw/scsi-bus.c b/hw/scsi-bus.c
index a97f1cd..01e1dec 100644
--- a/hw/scsi-bus.c
+++ b/hw/scsi-bus.c
@@ -1508,6 +1508,10 @@ void scsi_req_unref(SCSIRequest *req)
will start the next chunk or complete the command.  */
 void scsi_req_continue(SCSIRequest *req)
 {
+if (req->io_canceled) {
+trace_scsi_req_continue_canceled(req->dev->id, req->lun, req->tag);
+return;
+}
 trace_scsi_req_continue(req->dev->id, req->lun, req->tag);
 if (req->cmd.mode == SCSI_XFER_TO_DEV) {
 req->ops->write_data(req);
diff --git a/hw/scsi-disk.c b/hw/scsi-disk.c
index d411586..4a0673c 100644
--- a/hw/scsi-disk.c
+++ b/hw/scsi-disk.c
@@ -178,6 +178,9 @@ static void scsi_aio_complete(void *opaque, int ret)
 assert(r->req.aiocb != NULL);
 r->req.aiocb = NULL;
 bdrv_acct_done(s->qdev.conf.bs, &r->acct);
+if (r->req.io_canceled) {
+goto done;
+}
 
 if (ret < 0) {
 if (scsi_handle_rw_error(r, -ret)) {
@@ -223,6 +226,10 @@ static void scsi_write_do_fua(SCSIDiskReq *r)
 {
 SCSIDiskState *s = DO_UPCAST(SCSIDiskState, qdev, r->req.dev);
 
+if (r->req.io_canceled) {
+goto done;
+}
+
 if (scsi_is_cmd_fua(&r->req.cmd)) {
 bdrv_acct_start(s->qdev.conf.bs, &r->acct, 0, BDRV_ACCT_FLUSH);
 r->req.aiocb = bdrv_aio_flush(s->qdev.conf.bs, scsi_aio_complete, r);
@@ -230,6 +237,8 @@ static void scsi_write_do_fua(SCSIDiskReq *r)
 }
 
 scsi_req_complete(&r->req, GOOD);
+
+done:
 if (!r->req.io_canceled) {
 scsi_req_unref(&r->req);
 }
@@ -243,6 +252,9 @@ static void scsi_dma_complete(void *opaque, int ret)
 assert(r->req.aiocb != NULL);
 r->req.aiocb = NULL;
 bdrv_acct_done(s->qdev.conf.bs, &r->acct);
+if (r->req.io_canceled) {
+goto done;
+}
 
 if (ret < 0) {
 if (scsi_handle_rw_error(r, -ret)) {
@@ -274,6 +286,9 @@ static void scsi_read_complete(void * opaque, int ret)
 assert(r->req.aiocb != NULL);
 r->req.aiocb = NULL;
 bdrv_acct_done(s->qdev.conf.bs, &r->acct);
+if (r->req.io_canceled) {
+goto done;
+}
 
 if (ret < 0) {
 if (scsi_handle_rw_error(r, -ret)) {
@@ -305,6 +320,9 @@ static void scsi_do_read(void *opaque, int ret)
 r->req.aiocb = NULL;
 bdrv_acct_done(s->qdev.conf.bs, &r->acct);
 }
+if (r->req.io_canceled) {
+goto done;
+}
 
 if (ret < 0) {
 if (scsi_handle_rw_error(r, -ret)) {
@@ -312,10 +330,6 @@ static void scsi_do_read(void *opaque, int ret)
 }
 }
 
-if (r->req.io_canceled) {
-return;
-}
-
 /* The request is used as the AIO opaque value, so add a ref.  */
 scsi_req_ref(&r->req);
 
@@ -423,6 +437,9 @@ static void scsi_write_complete(void * opaque, int ret)
 r->req.aiocb = NULL;
 bdrv_acct_done(s->qdev.conf.bs, &r->acct);
 }
+if (r->req.io_canceled) {
+goto done;
+}
 
 if (ret < 0) {
 if (scsi_handle_rw_error(r, -ret)) {
@@ -1478,13 +149

Re: [Qemu-devel] [PATCH v11 3/4] qemu-img: Add compare subcommand

[Eric Blake]

On 02/13/2013 01:09 AM, Miroslav Rezanina wrote:
> This patch adds new qemu-img subcommand that compares content of two disk
> images.
> 
> Signed-off-by: Miroslav Rezanina 
> ---

> +Compare exits with @code{0} in case the images are equal and with @code{1}
> +in case the images differ. Negative exit code means an error occurred during

There's no such thing as a negative exit code.

> +execution and standard error output should contain an error message.
> +Following table sumarize all exit codes of compare subcommand:

s/sumarize/summarizes/

> +
> +@table @option
> +
> +@item 0
> +Images are identical
> +@item 1
> +Images differ
> +@item 2
> +Error on opening an image
> +@item 3
> +Error on checking a sector allocation
> +@item 4
> +Error on reading data
> +
> +@end table

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH for-1.4] block/curl: only restrict protocols with libcurl>=7.19.4

[Andreas Färber]

Am 13.02.2013 09:25, schrieb Stefan Hajnoczi:
> The curl_easy_setopt(state->curl, CURLOPT_PROTOCOLS, ...) interface was
> introduced in libcurl 7.19.4.  Therefore we cannot protect against
> CVE-2013-0249 when linking against an older libcurl.
> 
> This fixes the build failure introduced by
> fb6d1bbd246c7a57ef53d3847ef225cd1349d602.
> 
> Reported-by: Andreas Färber 
> Signed-off-by: Stefan Hajnoczi 

Tested-by: Andreas Färber 

That fixes the build without --disable-curl on Mac OS X v10.5.8 for me
(7.16.4).

Thanks,
Andreas

[Qemu-devel] [RFC PATCH v2 16/23] qcow2: Reading from areas not in L2 tables yet

[Kevin Wolf]

In preparation of delayed COW (i.e. completing the guest write request
before the associated COWs have completed) we must make sure that after
the guest data has written the new data is read back, even if the COW
hasn't completed and the new cluster isn't linked in the L2 table yet.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   39 +++
 block/qcow2.c |2 ++
 block/qcow2.h |   19 +++
 3 files changed, 60 insertions(+), 0 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 645ea25..be75c17 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -372,6 +372,40 @@ out:
 return ret;
 }
 
+static bool overlaps_allocation(BlockDriverState *bs, uint64_t start,
+int *num, uint64_t *cluster_offset)
+{
+BDRVQcowState *s = bs->opaque;
+QCowL2Meta *m;
+uint64_t end = start + (*num << BDRV_SECTOR_BITS);
+
+QLIST_FOREACH(m, &s->cluster_allocs, next_in_flight) {
+
+uint64_t old_start = l2meta_req_start(m);
+uint64_t old_end = l2meta_req_end(m);
+
+/* If the write hasn't completed yet and the allocating request can't
+ * have completed yet therefore, we're free to read the old data. */
+if (!m->is_written) {
+continue;
+}
+
+if (start >= old_start && start < old_end) {
+/* Start of the new request overlaps: Read from the newly allocated
+ * cluster even if it isn't in the L2 table yet. */
+*num = MIN(*num, (old_end - start) >> BDRV_SECTOR_BITS);
+*cluster_offset = m->alloc_offset
++ ((start - old_start) & ~(s->cluster_size - 1));
+return true;
+} else if (start < old_start && end > old_start) {
+/* Overlap somewhere after the start. Shorten this request so that
+ * no overlap occurs. */
+*num = MIN(*num, (old_start - start) >> BDRV_SECTOR_BITS);
+}
+}
+
+return false;
+}
 
 /*
  * get_cluster_offset
@@ -398,6 +432,11 @@ int qcow2_get_cluster_offset(BlockDriverState *bs, 
uint64_t offset,
 uint64_t nb_available, nb_needed;
 int ret;
 
+/* Check overlap with not yet completed allocations */
+if (overlaps_allocation(bs, offset, num, cluster_offset)) {
+return QCOW2_CLUSTER_NORMAL;
+}
+
 index_in_cluster = (offset >> 9) & (s->cluster_sectors - 1);
 nb_needed = *num + index_in_cluster;
 
diff --git a/block/qcow2.c b/block/qcow2.c
index 345487e..07f7493 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -827,6 +827,8 @@ static coroutine_fn int qcow2_co_writev(BlockDriverState 
*bs,
 while (l2meta != NULL) {
 QCowL2Meta *next;
 
+l2meta->is_written = true;
+
 ret = qcow2_alloc_cluster_link_l2(bs, l2meta);
 if (ret < 0) {
 goto fail;
diff --git a/block/qcow2.h b/block/qcow2.h
index 9b16538..4c139d0 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -229,6 +229,14 @@ typedef struct QCowL2Meta
 int nb_clusters;
 
 /**
+ * true if the guest data (but not necessarily the related COW) has been
+ * written to disk, so that read requests can (and after having completed
+ * this request actually _must_) read the new data instead of reading the
+ * old data that the L2 table still refers to.
+ */
+bool is_written;
+
+/**
  * Requests that overlap with this allocation and wait to be restarted
  * when the allocating request has completed.
  */
@@ -316,6 +324,17 @@ static inline bool 
qcow2_need_accurate_refcounts(BDRVQcowState *s)
 return !(s->incompatible_features & QCOW2_INCOMPAT_DIRTY);
 }
 
+static inline uint64_t l2meta_req_start(QCowL2Meta *m)
+{
+return (m->offset + m->cow_start.offset)
++ (m->cow_start.nb_sectors << BDRV_SECTOR_BITS);
+}
+
+static inline uint64_t l2meta_req_end(QCowL2Meta *m)
+{
+return m->offset + (m->nb_available << BDRV_SECTOR_BITS);
+}
+
 static inline uint64_t l2meta_cow_start(QCowL2Meta *m)
 {
 return m->offset + m->cow_start.offset;
-- 
1.7.6.5

Re: [Qemu-devel] kvm segfaulting

[Stefan Priebe - Profihost AG]

Hi paolo,

thanks for your work. Should i still apply your "old" patch to scsi-disk
or should i remove it?

Stefan
Am 13.02.2013 14:39, schrieb Paolo Bonzini:
> Il 13/02/2013 13:55, Stefan Priebe - Profihost AG ha scritto:
>> Hi,
>> Am 13.02.2013 12:36, schrieb Paolo Bonzini:
>>> Il 13/02/2013 10:07, Stefan Priebe - Profihost AG ha scritto:

 commit 47a150a4bbb06e45ef439a8222e9f46a7c4cca3f
 ...
>> You can certainly try reverting it, but this patch is fixing a real bug.
 Will try that. Yes but even if it fixes a bug and raises another one
 (kvm segfault) which is the worst one. It should be fixed.
>>>
>>> The KVM segfault is exposing a potential consistency problem.  What is
>>> worse is not obvious.  Also, it is happening at reset time if this is
>>> the culprit.  Reset usually happens at places where no data loss is caused.
>>>
>>> Can you find out what the VM was doing when it segfaulted?  (Or even,
>>> can you place the corefile and kvm executable somewhere where I can
>>> download it?)
>>
>> Yes it was doing an fstrim -v / which resulted in:
>>
>> [45648.453698] end_request: I/O error, dev sda, sector 9066952
> 
> Ok, very helpful.  One thing is to find why this failed.  This can
> come later though.
> 
> First of all, please run "cat 
> /sys/block/*/device/scsi_disk/*/provisioning_mode"
> in a VM with a similar configuration as the one that crashed last.
> 
> Second, I attach another patch.
> 
> Third, if possible please compile QEMU with --enable-trace-backend=simple,
> and run it with
> 
>   -trace events='bdrv_aio_discard
> scsi_req_cancel
> ',file=qemu.$$.trace
> 
> This can give some clues.  The files should remain quite small,
> so you can enable it on all VMs safely.
> 
>> Sadly not as i don't have acore dump. The kvm processes are started
>> through variuos Daemons and there seems no way to activate core dumps
>> for an already running process and i don't know which VM will crash next.
> 
> Probably the next that invokes fstrim. :)
> 
>>> If not, do your VMs reset themselves
>>> often for example?
>> No
> 
> Ok, good to know.
> 
>>> Can you reproduce it on non-rbd storage?
>> I don't have another storage type. ;-(
> 
> No problem.
> 
> Paolo
> 

[Qemu-devel] [RFC PATCH v2 11/23] qcow2: handle_copied(): Get rid of nb_clusters parameter

[Kevin Wolf]

handle_copied() uses its bytes parameter now to determine how many
clusters it should try to find.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   24 ++--
 1 files changed, 18 insertions(+), 6 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index cdbce20..cbe98bc 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -828,24 +828,35 @@ static int handle_dependencies(BlockDriverState *bs, 
uint64_t guest_offset,
  *
  *  -errno: in error cases
  *
- * TODO Get rid of keep_clusters, nb_clusters parameters
+ * TODO Get rid of keep_clusters parameter
  * TODO Make bytes behave like described above
  * TODO Make non-zero host_offset behave like describe above
  */
 static int handle_copied(BlockDriverState *bs, uint64_t guest_offset,
 uint64_t *host_offset, uint64_t *bytes, QCowL2Meta **m,
-unsigned int *keep_clusters, unsigned int *nb_clusters)
+unsigned int *keep_clusters)
 {
 BDRVQcowState *s = bs->opaque;
 int l2_index;
 uint64_t cluster_offset;
 uint64_t *l2_table;
+unsigned int nb_clusters;
 int ret, pret;
 
 trace_qcow2_handle_copied(qemu_coroutine_self(), guest_offset, 
*host_offset,
   *bytes);
 assert(*host_offset == 0);
 
+/*
+ * Calculate the number of clusters to look for. We stop at L2 table
+ * boundaries to keep things simple.
+ */
+nb_clusters =
+size_to_clusters(s, offset_into_cluster(s, guest_offset) + *bytes);
+
+l2_index = offset_to_l2_index(s, guest_offset);
+nb_clusters = MIN(nb_clusters, s->l2_size - l2_index);
+
 /* Find L2 entry for the first involved cluster */
 ret = get_cluster_table(bs, guest_offset, &l2_table, &l2_index);
 if (ret < 0) {
@@ -860,11 +871,10 @@ static int handle_copied(BlockDriverState *bs, uint64_t 
guest_offset,
 {
 /* We keep all QCOW_OFLAG_COPIED clusters */
 *keep_clusters =
-count_contiguous_clusters(*nb_clusters, s->cluster_size,
+count_contiguous_clusters(nb_clusters, s->cluster_size,
   &l2_table[l2_index], 0,
   QCOW_OFLAG_COPIED | QCOW_OFLAG_ZERO);
-assert(*keep_clusters <= *nb_clusters);
-*nb_clusters -= *keep_clusters;
+assert(*keep_clusters <= nb_clusters);
 
 ret = 1;
 } else {
@@ -1155,10 +1165,12 @@ again:
  *TODO: Consider cluster_offset if set in step 1c.
  */
 ret = handle_copied(bs, offset, &cluster_offset, &cur_bytes, m,
-&keep_clusters, &nb_clusters);
+&keep_clusters);
 if (ret < 0) {
 return ret;
 } else if (ret) {
+nb_clusters -= keep_clusters;
+
 if (!*host_offset) {
 *host_offset = cluster_offset;
 }
-- 
1.7.6.5

Re: [Qemu-devel] [RFC PATCH RDMA support v2: 6/6] send memory over RDMA as blocks are iterated

[Michael R. Hines]

Very helpful, thank you. I'll implement these and all the other ones soon.

- Michael

On 02/13/2013 04:50 AM, Orit Wasserman wrote:

On 02/12/2013 12:49 AM, Michael R. Hines wrote:

From: "Michael R. Hines" 


Signed-off-by: Michael R. Hines 
---
  arch_init.c |   84 ---
  savevm.c|   59 -
  2 files changed, 139 insertions(+), 4 deletions(-)

diff --git a/arch_init.c b/arch_init.c
index dada6de..76092cc 100644
--- a/arch_init.c
+++ b/arch_init.c
@@ -42,6 +42,7 @@
  #include "migration/migration.h"
  #include "exec/gdbstub.h"
  #include "hw/smbios.h"
+#include "qemu/rdma.h"
  #include "exec/address-spaces.h"
  #include "hw/pcspk.h"
  #include "migration/page_cache.h"
@@ -170,6 +171,15 @@ static int is_dup_page(uint8_t *page)
  VECTYPE val = SPLAT(page);
  int i;
  
+/*

+ * RFC RDMA: The empirical cost of searching for zero pages here
+ *   plus the cost of communicating with the other side
+ *   seems to take significantly more time than simply
+ *   dumping the page into remote memory.
+ */
+if (migrate_rdma_enabled())
+return 0;
+

I would change ram_save_block not this function.
So I would move the if to before calling is_dup.


  for (i = 0; i < TARGET_PAGE_SIZE / sizeof(VECTYPE); i++) {
  if (!ALL_EQ(val, p[i])) {
  return 0;
@@ -282,6 +292,44 @@ static size_t save_block_hdr(QEMUFile *f, RAMBlock *block, 
ram_addr_t offset,
  return size;
  }
  
+static size_t save_rdma_page(QEMUFile *f, RAMBlock *block, ram_addr_t offset,

+ int cont)
+{
+size_t bytes_sent = 0;
+ram_addr_t current_addr;
+
+acct_info.norm_pages++;
+
+/*
+ * use RDMA to send page
+ */
+current_addr = block->offset + offset;
+if (rdma_write(&rdma_mdata, current_addr,
+TARGET_PAGE_SIZE)) {
+fprintf(stderr, "rdma migration: write error!\n");
+qemu_file_set_error(f, -EIO);

we lose the error here,
why not let rdma_write to set the error, let it get the QemuFile as a parameter.

+return 0;
+}
+
+/*
+ * do some polling
+ */
+while (1) {
+int ret = rdma_poll(&rdma_mdata);
+if (ret == RDMA_WRID_NONE) {
+break;
+}
+if (ret < 0) {
+fprintf(stderr, "rdma migration: polling error!\n");
+qemu_file_set_error(f, -EIO);

see above

+return 0;
+}
+}
+
+bytes_sent += TARGET_PAGE_SIZE;
+return bytes_sent;
+}
+
  #define ENCODING_FLAG_XBZRLE 0x1
  
  static int save_xbzrle_page(QEMUFile *f, uint8_t *current_data,

@@ -474,6 +522,8 @@ static int ram_save_block(QEMUFile *f, bool last_stage)
  if (!last_stage) {
  p = get_cached_data(XBZRLE.cache, current_addr);
  }
+} else if (migrate_rdma_enabled()) {
+bytes_sent = save_rdma_page(f, block, offset, cont);
  }

You can move it to the top and than you won't need to change is_dup.
  
  /* XBZRLE overflow or normal page */

@@ -601,12 +651,14 @@ static int ram_save_setup(QEMUFile *f, void *opaque)
  return 0;
  }
  
+static int tprate = 1000;

+
  static int ram_save_iterate(QEMUFile *f, void *opaque)
  {
  int ret;
  int i;
-int64_t t0;
-int total_sent = 0;
+int64_t t0, tp0;
+int total_sent = 0, last_total_sent = 0;
  
  qemu_mutex_lock_ramlist();
  
@@ -625,23 +677,49 @@ static int ram_save_iterate(QEMUFile *f, void *opaque)

  break;
  }
  total_sent += bytes_sent;
+last_total_sent += bytes_sent;
  acct_info.iterations++;
  /* we want to check in the 1st loop, just in case it was the 1st time
 and we had to sync the dirty bitmap.
 qemu_get_clock_ns() is a bit expensive, so we only check each some
 iterations
  */
+
+/*
+ * RFC RDMA: Can we have something like this to periodically print
+ *   out throughput? This is just a rough-sketch that
+ *   partially worked for me. I assume there a better way
+ *   that everyone would prefer. Perhaps we could set a QMP
+ *   command that toggled a "periodic printing" option that
+ *   allowed more details to be printed on stdout.?
+ */
  if ((i & 63) == 0) {
-uint64_t t1 = (qemu_get_clock_ns(rt_clock) - t0) / 100;
+uint64_t curr = qemu_get_clock_ns(rt_clock);
+uint64_t t1 = (curr - t0) / 100;
+double tp;
  if (t1 > MAX_WAIT) {
  DPRINTF("big wait: %" PRIu64 " milliseconds, %d iterations\n",
  t1, i);
  break;
  }
+
+if ((i % tprate) == 0) {
+uint64_t tp1 = (curr

[Qemu-devel] [PATCH v2] qemu-iotests: Test qcow2 image creation options

[Kevin Wolf]

Just create lots of images and try out each of the creation options that
qcow2 provides (except backing_file/fmt for now)

I'm not totally happy with the behaviour of qemu-img in each of the
cases, but let's be explicit and update the test when we do change
things later.

Signed-off-by: Kevin Wolf 
---

v2:
- Filter out the path for images, /home/kwolf/... shouldn't be in there
- Commit 7216ae3 removed helpful error messages, updated golden output

 tests/qemu-iotests/049 |  123 +
 tests/qemu-iotests/049.out |  212 
 tests/qemu-iotests/group   |1 +
 3 files changed, 336 insertions(+), 0 deletions(-)
 create mode 100755 tests/qemu-iotests/049
 create mode 100644 tests/qemu-iotests/049.out

diff --git a/tests/qemu-iotests/049 b/tests/qemu-iotests/049
new file mode 100755
index 000..6c6017e
--- /dev/null
+++ b/tests/qemu-iotests/049
@@ -0,0 +1,123 @@
+#!/bin/bash
+#
+# Check qemu-img option parsing
+#
+# Copyright (C) 2013 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+# creator
+owner=kw...@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+
+_cleanup()
+{
+   _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+function filter_test_dir()
+{
+sed -e "s#$IMGPROTO:$TEST_DIR#TEST_DIR#g" \
+-e "s#$TEST_DIR#TEST_DIR#g"
+}
+
+function test_qemu_img()
+{
+echo qemu-img "$@" | filter_test_dir
+$QEMU_IMG "$@" 2>&1 | filter_test_dir
+echo
+}
+
+echo "=== Check correct interpretation of suffixes for image size ==="
+echo
+sizes="1024 1024b 1k 1K 1M 1G 1T "
+sizes+="1024.0 1024.0b 1.5k 1.5K 1.5M 1.5G 1.5T"
+
+echo "== 1. Traditional size parameter =="
+echo
+for s in $sizes; do
+test_qemu_img create -f $IMGFMT $TEST_IMG $s
+done
+
+echo "== 2. Specifying size via -o =="
+echo
+for s in $sizes; do
+test_qemu_img create -f $IMGFMT -o size=$s $TEST_IMG
+done
+
+echo "== 3. Invalid sizes =="
+echo
+sizes="-1024 -1k 1kilobyte foobar"
+
+for s in $sizes; do
+test_qemu_img create -f $IMGFMT $TEST_IMG -- $s
+test_qemu_img create -f $IMGFMT -o size=$s $TEST_IMG
+done
+
+echo "== Check correct interpretation of suffixes for cluster size =="
+echo
+sizes="1024 1024b 1k 1K 1M "
+sizes+="1024.0 1024.0b 0.5k 0.5K 0.5M"
+
+for s in $sizes; do
+test_qemu_img create -f $IMGFMT -o cluster_size=$s $TEST_IMG 64M
+done
+
+echo "== Check compat level option =="
+echo
+test_qemu_img create -f $IMGFMT -o compat=0.10 $TEST_IMG 64M
+test_qemu_img create -f $IMGFMT -o compat=1.1 $TEST_IMG 64M
+
+test_qemu_img create -f $IMGFMT -o compat=0.42 $TEST_IMG 64M
+test_qemu_img create -f $IMGFMT -o compat=foobar $TEST_IMG 64M
+
+echo "== Check preallocation option =="
+echo
+test_qemu_img create -f $IMGFMT -o preallocation=off $TEST_IMG 64M
+test_qemu_img create -f $IMGFMT -o preallocation=metadata $TEST_IMG 64M
+test_qemu_img create -f $IMGFMT -o preallocation=1234 $TEST_IMG 64M
+
+echo "== Check encryption option =="
+echo
+test_qemu_img create -f $IMGFMT -o encryption=off $TEST_IMG 64M
+test_qemu_img create -f $IMGFMT -o encryption=on $TEST_IMG 64M
+
+echo "== Check lazy_refcounts option (only with v3) =="
+echo
+test_qemu_img create -f $IMGFMT -o compat=1.1,lazy_refcounts=off $TEST_IMG 64M
+test_qemu_img create -f $IMGFMT -o compat=1.1,lazy_refcounts=on $TEST_IMG 64M
+
+test_qemu_img create -f $IMGFMT -o compat=0.10,lazy_refcounts=off $TEST_IMG 64M
+test_qemu_img create -f $IMGFMT -o compat=0.10,lazy_refcounts=on $TEST_IMG 64M
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/049.out b/tests/qemu-iotests/049.out
new file mode 100644
index 000..72db13f
--- /dev/null
+++ b/tests/qemu-iotests/049.out
@@ -0,0 +1,212 @@
+QA output created by 049
+=== Check correct interpretation of suffixes for image size ===
+
+== 1. Traditional size parameter ==
+
+qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off 
cluster_size=65536 lazy_refcounts=off 
+
+qemu-img create -f qcow2 TEST_DIR/t.qcow2 1024b
+Formatting 'TEST_DIR/t.qcow2', fmt=qcow2 size=1024 encryption=off 
clu

Re: [Qemu-devel] kvm segfaulting

[Stefan Priebe - Profihost AG]

Output of cat:
[: ~]# cat /sys/block/*/device/scsi_disk/*/provisioning_mode
writesame_16

Stefan

Am 13.02.2013 14:39, schrieb Paolo Bonzini:
> Il 13/02/2013 13:55, Stefan Priebe - Profihost AG ha scritto:
>> Hi,
>> Am 13.02.2013 12:36, schrieb Paolo Bonzini:
>>> Il 13/02/2013 10:07, Stefan Priebe - Profihost AG ha scritto:

 commit 47a150a4bbb06e45ef439a8222e9f46a7c4cca3f
 ...
>> You can certainly try reverting it, but this patch is fixing a real bug.
 Will try that. Yes but even if it fixes a bug and raises another one
 (kvm segfault) which is the worst one. It should be fixed.
>>>
>>> The KVM segfault is exposing a potential consistency problem.  What is
>>> worse is not obvious.  Also, it is happening at reset time if this is
>>> the culprit.  Reset usually happens at places where no data loss is caused.
>>>
>>> Can you find out what the VM was doing when it segfaulted?  (Or even,
>>> can you place the corefile and kvm executable somewhere where I can
>>> download it?)
>>
>> Yes it was doing an fstrim -v / which resulted in:
>>
>> [45648.453698] end_request: I/O error, dev sda, sector 9066952
> 
> Ok, very helpful.  One thing is to find why this failed.  This can
> come later though.
> 
> First of all, please run "cat 
> /sys/block/*/device/scsi_disk/*/provisioning_mode"
> in a VM with a similar configuration as the one that crashed last.
> 
> Second, I attach another patch.
> 
> Third, if possible please compile QEMU with --enable-trace-backend=simple,
> and run it with
> 
>   -trace events='bdrv_aio_discard
> scsi_req_cancel
> ',file=qemu.$$.trace
> 
> This can give some clues.  The files should remain quite small,
> so you can enable it on all VMs safely.
> 
>> Sadly not as i don't have acore dump. The kvm processes are started
>> through variuos Daemons and there seems no way to activate core dumps
>> for an already running process and i don't know which VM will crash next.
> 
> Probably the next that invokes fstrim. :)
> 
>>> If not, do your VMs reset themselves
>>> often for example?
>> No
> 
> Ok, good to know.
> 
>>> Can you reproduce it on non-rbd storage?
>> I don't have another storage type. ;-(
> 
> No problem.
> 
> Paolo
> 

[Qemu-devel] [RFC PATCH v2 00/23] qcow2: Delayed COW

[Kevin Wolf]

During the last few releases we have got rid of most of the overhead of
metadata writes during cluster allocation. What's left is the COW for
unaligned allocating write requests, and it's quite expensive.

In the general case, this cost cannot be avoided. However, if we're
lucky enough that before the next flush the data copied during COW
would be overwritten, we can do without the COW. Sequential writes
always overwrite the COW area at the end of the cluster immediately,
so delaying the COW a bit and cancelling it if it's overwritten is a
worthwhile optimisation.


iozone results with and without this series show significant difference
for allocating writes:

 random  random
   KB  reclen   write rewritereadrereadread   write

base65536   817271945125461253924491836
patch   65536   819341949122631252124631796

base  1048576 256   22344   38437   105823   106135   37743   32167
patch 1048576 256   35989   38542   105231   105994   38301   33036


Changes in v2:
- Many of the v1 patches that make sense independently have already gone
  into qemu.git
- Half of the series that basically rewrites qcow2_alloc_cluster_offset()
  is new. This was required in order to make the really interesting patch
  in this series ("qcow2: Cancel COW when overwritten") correct.
- Addressed a few comments on v1 by Paolo and Blue

Still to do:
- Figure out what to do with failing qemu-iotests cases that rely on a
  specific order in which requests complete. This series can very
  obviously change that behaviour. Maybe we should redirect the output
  of all qemu-io instances to /dev/null if they use AIO and the output
  is non-deterministic therefore.

Kevin Wolf (23):
  qcow2: Handle dependencies earlier
  qcow2: Improve check for overlapping allocations
  qcow2: Change handle_dependency to byte granularity
  qcow2: Decouple cluster allocation from cluster reuse code
  qcow2: Factor out handle_alloc()
  qcow2: handle_alloc(): Get rid of nb_clusters parameter
  qcow2: handle_alloc(): Get rid of keep_clusters parameter
  qcow2: Finalise interface of handle_alloc()
  qcow2: Clean up handle_alloc()
  qcow2: Factor out handle_copied()
  qcow2: handle_copied(): Get rid of nb_clusters parameter
  qcow2: handle_copied(): Get rid of keep_clusters parameter
  qcow2: handle_copied(): Implement non-zero host_offset
  qcow2: Use byte granularity in qcow2_alloc_cluster_offset()
  qcow2: Allow requests with multiple l2metas
  qcow2: Reading from areas not in L2 tables yet
  qcow2: Move COW and L2 update into own coroutine
  qcow2: Delay the COW
  qcow2: Add error handling to the l2meta coroutine
  qcow2: Cancel COW when overwritten
  qemu-iotests: Another concurrent multicluster allocation case
  qcow2: Move cluster gathering to a non-looping loop
  qcow2: Gather clusters in a looping loop

 block.c|5 +
 block/qcow2-cluster.c  |  707 ++--
 block/qcow2.c  |  209 -
 block/qcow2.h  |  110 +++
 include/block/block_int.h  |3 +
 tests/qemu-iotests/038.out |   10 +-
 tests/qemu-iotests/046 |   19 ++
 tests/qemu-iotests/046.out |   16 +
 trace-events   |4 +
 9 files changed, 905 insertions(+), 178 deletions(-)

-- 
1.7.6.5

[Qemu-devel] [RFC PATCH v2 08/23] qcow2: Finalise interface of handle_alloc()

[Kevin Wolf]

The interface works completely on a byte granularity now and duplicated
parameters are removed.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   29 -
 block/qcow2.h |5 +
 2 files changed, 21 insertions(+), 13 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 2f64bf7..e32bfe3 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -872,13 +872,9 @@ static int do_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t guest_offset,
  *  newly allocated cluster.
  *
  *  -errno: in error cases
- *
- * TODO Get rid of n_start, n_end
- * TODO Make *bytes actually behave as specified above
  */
 static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
-uint64_t *host_offset, uint64_t *bytes, QCowL2Meta **m,
-int n_start, int n_end)
+uint64_t *host_offset, uint64_t *bytes, QCowL2Meta **m)
 {
 BDRVQcowState *s = bs->opaque;
 int l2_index;
@@ -898,8 +894,11 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
  * Calculate the number of clusters to look for. We stop at L2 table
  * boundaries to keep things simple.
  */
+nb_clusters =
+size_to_clusters(s, offset_into_cluster(s, guest_offset) + *bytes);
+
 l2_index = offset_to_l2_index(s, guest_offset);
-nb_clusters = MIN(size_to_clusters(s, *bytes), s->l2_size - l2_index);
+nb_clusters = MIN(nb_clusters, s->l2_size - l2_index);
 
 /* Find L2 entry for the first involved cluster */
 ret = get_cluster_table(bs, guest_offset, &l2_table, &l2_index);
@@ -951,10 +950,13 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
  * newly allocated cluster to the end of the aread that the write
  * request actually writes to (excluding COW at the end)
  */
-int requested_sectors = n_end;
+int requested_sectors =
+(*bytes + offset_into_cluster(s, guest_offset))
+>> BDRV_SECTOR_BITS;
 int avail_sectors = nb_clusters
 << (s->cluster_bits - BDRV_SECTOR_BITS);
-int alloc_n_start = *host_offset == 0 ? n_start : 0;
+int alloc_n_start = offset_into_cluster(s, guest_offset)
+>> BDRV_SECTOR_BITS;
 int nb_sectors = MIN(requested_sectors, avail_sectors);
 
 if (*host_offset == 0) {
@@ -981,7 +983,9 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
 qemu_co_queue_init(&(*m)->dependent_requests);
 QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight);
 
-*bytes = nb_clusters * s->cluster_size;
+*bytes = MIN(*bytes, (nb_sectors * BDRV_SECTOR_SIZE)
+ - offset_into_cluster(s, guest_offset));
+assert(*bytes != 0);
 } else {
 *bytes = 0;
 return 0;
@@ -1136,10 +1140,9 @@ again:
 alloc_n_end = n_end;
 }
 
-cur_bytes = nb_clusters * s->cluster_size;
+cur_bytes = ((alloc_n_end - alloc_n_start) << BDRV_SECTOR_BITS);
 
-ret = handle_alloc(bs, offset, &cluster_offset, &cur_bytes, m,
-   alloc_n_start, alloc_n_end);
+ret = handle_alloc(bs, offset, &cluster_offset, &cur_bytes, m);
 if (ret < 0) {
 return ret;
 }
@@ -1147,7 +1150,7 @@ again:
 if (!*host_offset) {
 *host_offset = cluster_offset;
 }
-nb_clusters = size_to_clusters(s, cur_bytes);
+nb_clusters = size_to_clusters(s, cur_bytes + offset_into_cluster(s, 
offset));
 
 /* Some cleanup work */
 done:
diff --git a/block/qcow2.h b/block/qcow2.h
index f90bfb1..bfdf71d 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -267,6 +267,11 @@ static inline int64_t start_of_cluster(BDRVQcowState *s, 
int64_t offset)
 return offset & ~(s->cluster_size - 1);
 }
 
+static inline int64_t offset_into_cluster(BDRVQcowState *s, int64_t offset)
+{
+return offset & (s->cluster_size - 1);
+}
+
 static inline int size_to_clusters(BDRVQcowState *s, int64_t size)
 {
 return (size + (s->cluster_size - 1)) >> s->cluster_bits;
-- 
1.7.6.5

Re: [Qemu-devel] [PATCH v11 3/4] qemu-img: Add compare subcommand

[Miroslav Rezanina]

- Original Message -
> From: "Eric Blake" 
> To: "Miroslav Rezanina" 
> Cc: qemu-devel@nongnu.org, kw...@redhat.com, pbonz...@redhat.com, 
> stefa...@redhat.com
> Sent: Wednesday, February 13, 2013 2:39:59 PM
> Subject: Re: [PATCH v11 3/4] qemu-img: Add compare subcommand
> 
> On 02/13/2013 01:09 AM, Miroslav Rezanina wrote:
> > This patch adds new qemu-img subcommand that compares content of
> > two disk
> > images.
> > 
> > Signed-off-by: Miroslav Rezanina 
> > ---
> 
> > +Compare exits with @code{0} in case the images are equal and with
> > @code{1}
> > +in case the images differ. Negative exit code means an error
> > occurred during
> 
> There's no such thing as a negative exit code.

Aaa, I forget to fix this. 

@stefanha: Can you please replace the work Negative with Others when applying.

> 
> > +execution and standard error output should contain an error
> > message.
> > +Following table sumarize all exit codes of compare subcommand:
> 
> s/sumarize/summarizes/
> 

@stefanha: Fix this typo too, thanks.

> > +
> > +@table @option
> > +
> > +@item 0
> > +Images are identical
> > +@item 1
> > +Images differ
> > +@item 2
> > +Error on opening an image
> > +@item 3
> > +Error on checking a sector allocation
> > +@item 4
> > +Error on reading data
> > +
> > +@end table
> 
> --
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
> 
> 

Thanks a lot, Eric.

-- 
Miroslav Rezanina
Software Engineer - Virtualization Team

[Qemu-devel] [RFC PATCH v2 05/23] qcow2: Factor out handle_alloc()

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |  240 +++--
 trace-events  |1 +
 2 files changed, 152 insertions(+), 89 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 5c4b7cc..919cb39 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -856,6 +856,146 @@ static int do_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t guest_offset,
 }
 
 /*
+ * Allocates new clusters for an area that either is yet unallocated or needs a
+ * copy on write. If *host_offset is non-zero, clusters are only allocated if
+ * the new allocation can match the specified host offset.
+ *
+ * Note that guest_offset may not be cluster aligned.
+ *
+ * Returns:
+ *   0: if no clusters could be allocated. *bytes is set to 0,
+ *  *host_offset is left unchanged.
+ *
+ *   1: if new clusters were allocated. *bytes may be decreased if the
+ *  new allocation doesn't cover all of the requested area.
+ *  *host_offset is updated to contain the host offset of the first
+ *  newly allocated cluster.
+ *
+ *  -errno: in error cases
+ *
+ * TODO Get rid of nb_clusters, keep_clusters, n_start, n_end
+ * TODO Make *bytes actually behave as specified above
+ */
+static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
+uint64_t *host_offset, uint64_t *bytes, QCowL2Meta **m,
+unsigned int nb_clusters, int keep_clusters, int n_start, int n_end)
+{
+BDRVQcowState *s = bs->opaque;
+int l2_index;
+uint64_t *l2_table;
+uint64_t entry;
+int ret;
+
+uint64_t alloc_offset;
+uint64_t alloc_cluster_offset;
+uint64_t keep_bytes = keep_clusters * s->cluster_size;
+
+trace_qcow2_handle_alloc(qemu_coroutine_self(), guest_offset, *host_offset,
+ *bytes);
+assert(*bytes > 0);
+
+/* Find L2 entry for the first involved cluster */
+ret = get_cluster_table(bs, guest_offset, &l2_table, &l2_index);
+if (ret < 0) {
+return ret;
+}
+
+entry = be64_to_cpu(l2_table[l2_index + keep_clusters]);
+
+/* For the moment, overwrite compressed clusters one by one */
+if (entry & QCOW_OFLAG_COMPRESSED) {
+nb_clusters = 1;
+} else {
+nb_clusters = count_cow_clusters(s, nb_clusters, l2_table,
+ l2_index + keep_clusters);
+}
+
+ret = qcow2_cache_put(bs, s->l2_table_cache, (void**) &l2_table);
+if (ret < 0) {
+return ret;
+}
+
+if (nb_clusters == 0) {
+*bytes = 0;
+return 0;
+}
+
+/* Calculate start and size of allocation */
+alloc_offset = guest_offset + keep_bytes;
+
+if (keep_clusters == 0) {
+alloc_cluster_offset = 0;
+} else {
+alloc_cluster_offset = *host_offset + keep_bytes;
+}
+
+/* Allocate, if necessary at a given offset in the image file */
+ret = do_alloc_cluster_offset(bs, alloc_offset, &alloc_cluster_offset,
+  &nb_clusters);
+if (ret < 0) {
+goto fail;
+}
+
+/* save info needed for meta data update */
+if (nb_clusters > 0) {
+/*
+ * requested_sectors: Number of sectors from the start of the first
+ * newly allocated cluster to the end of the (possibly shortened
+ * before) write request.
+ *
+ * avail_sectors: Number of sectors from the start of the first
+ * newly allocated to the end of the last newly allocated cluster.
+ *
+ * nb_sectors: The number of sectors from the start of the first
+ * newly allocated cluster to the end of the aread that the write
+ * request actually writes to (excluding COW at the end)
+ */
+int requested_sectors = n_end - keep_clusters * s->cluster_sectors;
+int avail_sectors = nb_clusters
+<< (s->cluster_bits - BDRV_SECTOR_BITS);
+int alloc_n_start = keep_clusters == 0 ? n_start : 0;
+int nb_sectors = MIN(requested_sectors, avail_sectors);
+
+if (keep_clusters == 0) {
+*host_offset = alloc_cluster_offset;
+}
+
+*m = g_malloc0(sizeof(**m));
+
+**m = (QCowL2Meta) {
+.alloc_offset   = alloc_cluster_offset,
+.offset = alloc_offset & ~(s->cluster_size - 1),
+.nb_clusters= nb_clusters,
+.nb_available   = nb_sectors,
+
+.cow_start = {
+.offset = 0,
+.nb_sectors = alloc_n_start,
+},
+.cow_end = {
+.offset = nb_sectors * BDRV_SECTOR_SIZE,
+.nb_sectors = avail_sectors - nb_sectors,
+},
+};
+qemu_co_queue_init(&(*m)->dependent_requests);
+QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight);
+
+*bytes = nb_clusters * s->cluster_size;
+} else {
+*bytes = 0;
+   

[Qemu-devel] [RFC PATCH v2 20/23] qcow2: Cancel COW when overwritten

[Kevin Wolf]

This is the first part of an optimisation to improve the performance of
sequential cluster allocations.

Typically, write requests aren't aligned to cluster boundaries, so
sequential allocation means that every other request has to wait for the
COW of the previous request to complete. We can do better: Just cancel
the COW instead of waiting for it and then overwriting the same area
with the second write request.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |  150 
 block/qcow2.c |   16 +
 block/qcow2.h |   37 
 trace-events  |2 +
 4 files changed, 192 insertions(+), 13 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 33e595a..3ef1cff 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -659,6 +659,8 @@ static int perform_cow(BlockDriverState *bs, QCowL2Meta *m, 
Qcow2COWRegion *r)
 BDRVQcowState *s = bs->opaque;
 int ret;
 
+r->final = true;
+
 if (r->nb_sectors == 0) {
 return 0;
 }
@@ -689,6 +691,7 @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, 
QCowL2Meta *m)
 int i, j = 0, l2_index, ret;
 uint64_t *old_cluster, *l2_table;
 uint64_t cluster_offset = m->alloc_offset;
+bool has_wr_lock = false;
 
 trace_qcow2_cluster_link_l2(qemu_coroutine_self(), m->nb_clusters);
 assert(m->nb_clusters > 0);
@@ -707,6 +710,16 @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, 
QCowL2Meta *m)
 }
 
 /* Update L2 table. */
+qemu_co_mutex_unlock(&s->lock);
+qemu_co_rwlock_wrlock(&m->l2_writeback_lock);
+has_wr_lock = true;
+qemu_co_mutex_lock(&s->lock);
+
+if (m->no_l2_update) {
+ret = 0;
+goto err;
+}
+
 if (s->compatible_features & QCOW2_COMPAT_LAZY_REFCOUNTS) {
 qcow2_mark_dirty(bs);
 }
@@ -753,6 +766,9 @@ int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, 
QCowL2Meta *m)
 
 ret = 0;
 err:
+if (has_wr_lock) {
+qemu_co_rwlock_unlock(&m->l2_writeback_lock);
+}
 g_free(old_cluster);
 return ret;
  }
@@ -847,17 +863,24 @@ void qcow2_delete_kick_l2meta_bh(void *opaque)
  *   bytes from guest_offset that can be read before the next
  *   dependency must be processed (or the request is complete)
  *
+ *   1   if we're reusing an in-flight cluster allocation. *cur_bytes
+ *   indicates the number of bytes from guest_offset that can be
+ *   used for writing to this cluster.
+ *
  *   -EAGAIN if we had to wait for another request, previously gathered
  *   information on cluster allocation may be invalid now. The caller
  *   must start over anyway, so consider *cur_bytes undefined.
  */
 static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset,
-uint64_t *cur_bytes)
+uint64_t *host_offset, uint64_t *cur_bytes, QCowL2Meta **m)
 {
 BDRVQcowState *s = bs->opaque;
 QCowL2Meta *old_alloc;
 uint64_t bytes = *cur_bytes;
 
+trace_qcow2_handle_dep(qemu_coroutine_self(), guest_offset, *host_offset,
+   *cur_bytes);
+
 QLIST_FOREACH(old_alloc, &s->cluster_allocs, next_in_flight) {
 
 uint64_t start = guest_offset;
@@ -868,22 +891,111 @@ static int handle_dependencies(BlockDriverState *bs, 
uint64_t guest_offset,
 if (end <= old_start || start >= old_end) {
 /* No intersection */
 } else {
+uint64_t new_bytes;
+uint64_t old_cow_end;
+
+   /*
+* Shorten the request to stop at the start of a running
+* allocation.
+*/
 if (start < old_start) {
-/* Stop at the start of a running allocation */
-bytes = old_start - start;
+new_bytes = old_start - start;
 } else {
-bytes = 0;
+new_bytes = 0;
+}
+
+if (new_bytes > 0) {
+bytes = new_bytes;
+continue;
 }
 
-if (bytes == 0) {
-/* Wait for the dependency to complete. We need to recheck
- * the free/allocated clusters when we continue. */
-qemu_co_mutex_unlock(&s->lock);
+/* Stop if already an l2meta exists. We would have to consider
+ * locks held by it (l2_writeback_lock) and set m->next etc. */
+if (*m) {
+*cur_bytes = 0;
+return 0;
+}
+
+
+/*
+ * Check if we're just overwriting some COW of the old allocation
+ * that is safe to be replaced by the data of this request.
+ */
+old_cow_end = old_alloc->offset + old_alloc->cow_end.offset;
+
+if ((old_end & (s->cluster_size - 1)) == 0
+&& start >= old_cow_end
+&& !old_alloc->cow_end.final
+&& !*host_offset)

[Qemu-devel] [RFC PATCH v2 06/23] qcow2: handle_alloc(): Get rid of nb_clusters parameter

[Kevin Wolf]

We already communicate the same information in *bytes.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   19 +++
 1 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 919cb39..2492235 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -873,17 +873,18 @@ static int do_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t guest_offset,
  *
  *  -errno: in error cases
  *
- * TODO Get rid of nb_clusters, keep_clusters, n_start, n_end
+ * TODO Get rid of keep_clusters, n_start, n_end
  * TODO Make *bytes actually behave as specified above
  */
 static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
 uint64_t *host_offset, uint64_t *bytes, QCowL2Meta **m,
-unsigned int nb_clusters, int keep_clusters, int n_start, int n_end)
+int keep_clusters, int n_start, int n_end)
 {
 BDRVQcowState *s = bs->opaque;
 int l2_index;
 uint64_t *l2_table;
 uint64_t entry;
+unsigned int nb_clusters;
 int ret;
 
 uint64_t alloc_offset;
@@ -894,6 +895,13 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
  *bytes);
 assert(*bytes > 0);
 
+/*
+ * Calculate the number of clusters to look for. We stop at L2 table
+ * boundaries to keep things simple.
+ */
+l2_index = offset_to_l2_index(s, guest_offset);
+nb_clusters = MIN(size_to_clusters(s, *bytes), s->l2_size - l2_index);
+
 /* Find L2 entry for the first involved cluster */
 ret = get_cluster_table(bs, guest_offset, &l2_table, &l2_index);
 if (ret < 0) {
@@ -1100,6 +1108,7 @@ again:
 }
 
 cluster_offset &= L2E_OFFSET_MASK;
+*host_offset = cluster_offset;
 
 /*
  * The L2 table isn't used any more after this. As long as the cache works
@@ -1120,11 +1129,14 @@ again:
 
 cur_bytes = nb_clusters * s->cluster_size;
 ret = handle_alloc(bs, offset, &cluster_offset, &cur_bytes, m,
-   nb_clusters, keep_clusters, n_start, n_end);
+   keep_clusters, n_start, n_end);
 if (ret < 0) {
 return ret;
 }
 
+if (!*host_offset) {
+*host_offset = cluster_offset;
+}
 nb_clusters = size_to_clusters(s, cur_bytes);
 
 /* Some cleanup work */
@@ -1136,7 +1148,6 @@ done:
 
 assert(sectors > n_start);
 *num = sectors - n_start;
-*host_offset = cluster_offset;
 
 return 0;
 }
-- 
1.7.6.5

Re: [Qemu-devel] [RFC PATCH RDMA support v2: 2/6] create migration-rdma.c for core RDMA migration code

[Michael R. Hines]

Acknowledged.

On 02/13/2013 03:23 AM, Orit Wasserman wrote:

Hi Michael,
I would prefer that only migration specific code will be included in 
migration-rdma.c.
The general RDMA code should be in rdma.c file (like the rdma.h header file).

Thanks,
Orit

On 02/12/2013 12:49 AM, Michael R. Hines wrote:

From: "Michael R. Hines" 


Signed-off-by: Michael R. Hines 
---
  include/qemu/rdma.h |  281 ++
  migration-rdma.c| 1444 +++
  2 files changed, 1725 insertions(+)
  create mode 100644 include/qemu/rdma.h
  create mode 100644 migration-rdma.c

diff --git a/include/qemu/rdma.h b/include/qemu/rdma.h
new file mode 100644
index 000..2dc2519
--- /dev/null
+++ b/include/qemu/rdma.h
@@ -0,0 +1,281 @@
+/*
+ *  Copyright (C) 2013 Michael R. Hines 
+ *  Copyright (C) 2013 Jiuxing Liu 
+ *
+ *  RDMA data structures and helper functions (for migration)
+ *
+ *  This program is free software; you can redistribute it and/or modify
+ *  it under the terms of the GNU General Public License as published by
+ *  the Free Software Foundation; under version 2 of the License.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU General Public License for more details.
+ *
+ *  You should have received a copy of the GNU General Public License
+ *  along with this program; if not, see .
+ */
+
+#ifndef _RDMA_H
+#define _RDMA_H
+
+#include "config-host.h"
+#ifdef CONFIG_RDMA
+#include 
+#endif
+#include "monitor/monitor.h"
+
+extern int rdmaport;
+extern char rdmahost[64];
+
+struct rdma_context {
+/* cm_id also has ibv_conext, rdma_event_channel, and ibv_qp in
+   cm_id->verbs, cm_id->channel, and cm_id->qp. */
+struct rdma_cm_id *cm_id;
+struct rdma_cm_id *listen_id;
+
+struct ibv_context *verbs;
+struct rdma_event_channel *channel;
+struct ibv_qp *qp;
+
+struct ibv_comp_channel *comp_channel;
+struct ibv_pd *pd;
+struct ibv_cq *cq;
+};
+
+static inline void rdma_init_context(struct rdma_context *rdma_ctx)
+{
+rdma_ctx->cm_id = NULL;
+rdma_ctx->listen_id = NULL;
+rdma_ctx->verbs = NULL;
+rdma_ctx->channel = NULL;
+rdma_ctx->qp = NULL;
+rdma_ctx->comp_channel = NULL;
+rdma_ctx->pd = NULL;
+rdma_ctx->cq = NULL;
+}
+
+void cpu_physical_memory_reset_dirty_all(void);
+
+int rdma_resolve_host(struct rdma_context *rdma_ctx,
+const char *host, int port);
+int rdma_alloc_pd_cq(struct rdma_context *rdma_ctx);
+int rdma_alloc_qp(struct rdma_context *rdma_ctx);
+int rdma_migrate_connect(struct rdma_context *rdma_ctx,
+void *in_data, int *in_len, void *out_data, int out_len);
+int rdma_migrate_accept(struct rdma_context *rdma_ctx,
+void *in_data, int *in_len, void *out_data, int out_len);
+void rdma_migrate_disconnect(struct rdma_context *rdma_ctx);
+
+/* Instead of registering whole ram blocks, we can register them in smaller
+ * chunks. This may be benefial if the ram blocks have holes in them */
+#define RDMA_CHUNK_REGISTRATION
+
+#define RDMA_LAZY_REGISTRATION
+
+#define RDMA_REG_CHUNK_SHIFT 20
+#define RDMA_REG_CHUNK_SIZE (1UL << (RDMA_REG_CHUNK_SHIFT))
+#define RDMA_REG_CHUNK_INDEX(start_addr, host_addr) \
+(((unsigned long)(host_addr) >> RDMA_REG_CHUNK_SHIFT) - \
+((unsigned long)(start_addr) >> RDMA_REG_CHUNK_SHIFT))
+#define RDMA_REG_NUM_CHUNKS(rdma_ram_block) \
+(RDMA_REG_CHUNK_INDEX((rdma_ram_block)->local_host_addr,\
+(rdma_ram_block)->local_host_addr +\
+(rdma_ram_block)->length) + 1)
+#define RDMA_REG_CHUNK_START(rdma_ram_block, i) ((uint8_t *)\
+unsigned long)((rdma_ram_block)->local_host_addr) >> \
+RDMA_REG_CHUNK_SHIFT) + (i)) << \
+RDMA_REG_CHUNK_SHIFT))
+#define RDMA_REG_CHUNK_END(rdma_ram_block, i) \
+(RDMA_REG_CHUNK_START(rdma_ram_block, i) + \
+ RDMA_REG_CHUNK_SIZE)
+
+struct rdma_ram_block {
+uint8_t *local_host_addr;
+uint64_t remote_host_addr;
+uint64_t offset;
+uint64_t length;
+struct ibv_mr **pmr;
+struct ibv_mr *mr;
+uint32_t remote_rkey;
+};
+
+struct rdma_remote_ram_block {
+uint64_t remote_host_addr;
+uint64_t offset;
+uint64_t length;
+uint32_t remote_rkey;
+};
+
+#define RDMA_MAX_RAM_BLOCKS 64
+
+struct rdma_ram_blocks {
+int num_blocks;
+struct rdma_ram_block block[RDMA_MAX_RAM_BLOCKS];
+};
+
+struct rdma_remote_ram_blocks {
+int num_blocks;
+struct rdma_remote_ram_block block[RDMA_MAX_RAM_BLOCKS];
+};
+
+int rdma_init_ram_blocks(struct rdma_ram_blocks *rdma_ram_blocks);
+int rdma_reg_chunk_ram_blocks(struct rdma_context *rdma_ctx,
+struct rdma_ram_blocks *rdma_ram_blocks);
+int rdma_reg_whole_ram_blocks(struct rdma_context *rdma_ctx,
+struct rdma_

[Qemu-devel] [RFC PATCH v2 10/23] qcow2: Factor out handle_copied()

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |  133 ++---
 trace-events  |1 +
 2 files changed, 94 insertions(+), 40 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 0642ffa..cdbce20 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -809,6 +809,84 @@ static int handle_dependencies(BlockDriverState *bs, 
uint64_t guest_offset,
 }
 
 /*
+ * Checks how many already allocated clusters that don't require a copy on
+ * write there are at the given guest_offset (up to *bytes). If
+ * *host_offset is not zero, only physically contiguous clusters beginning at
+ * this host offset are counted.
+ *
+ * Note that guest_offset may not be cluster aligned.
+ *
+ * Returns:
+ *   0: if no allocated clusters are available at the given offset.
+ *  *bytes is normally unchanged. It is set to 0 if the cluster
+ *  is allocated and doesn't need COW, but doesn't have the right
+ *  physical offset.
+ *
+ *   1: if allocated clusters that don't require a COW are available at
+ *  the requested offset. *bytes may have decreased and describes
+ *  the length of the area that can be written to.
+ *
+ *  -errno: in error cases
+ *
+ * TODO Get rid of keep_clusters, nb_clusters parameters
+ * TODO Make bytes behave like described above
+ * TODO Make non-zero host_offset behave like describe above
+ */
+static int handle_copied(BlockDriverState *bs, uint64_t guest_offset,
+uint64_t *host_offset, uint64_t *bytes, QCowL2Meta **m,
+unsigned int *keep_clusters, unsigned int *nb_clusters)
+{
+BDRVQcowState *s = bs->opaque;
+int l2_index;
+uint64_t cluster_offset;
+uint64_t *l2_table;
+int ret, pret;
+
+trace_qcow2_handle_copied(qemu_coroutine_self(), guest_offset, 
*host_offset,
+  *bytes);
+assert(*host_offset == 0);
+
+/* Find L2 entry for the first involved cluster */
+ret = get_cluster_table(bs, guest_offset, &l2_table, &l2_index);
+if (ret < 0) {
+return ret;
+}
+
+cluster_offset = be64_to_cpu(l2_table[l2_index]);
+
+/* Check how many clusters are already allocated and don't need COW */
+if (qcow2_get_cluster_type(cluster_offset) == QCOW2_CLUSTER_NORMAL
+&& (cluster_offset & QCOW_OFLAG_COPIED))
+{
+/* We keep all QCOW_OFLAG_COPIED clusters */
+*keep_clusters =
+count_contiguous_clusters(*nb_clusters, s->cluster_size,
+  &l2_table[l2_index], 0,
+  QCOW_OFLAG_COPIED | QCOW_OFLAG_ZERO);
+assert(*keep_clusters <= *nb_clusters);
+*nb_clusters -= *keep_clusters;
+
+ret = 1;
+} else {
+*keep_clusters = 0;
+cluster_offset = 0;
+
+ret = 0;
+}
+
+cluster_offset &= L2E_OFFSET_MASK;
+*host_offset = cluster_offset;
+
+/* Cleanup */
+pret = qcow2_cache_put(bs, s->l2_table_cache, (void**) &l2_table);
+if (pret < 0) {
+return pret;
+}
+
+return ret;
+}
+
+/*
  * Allocates new clusters for the given guest_offset.
  *
  * At most *nb_clusters are allocated, and on return *nb_clusters is updated to
@@ -1020,7 +1098,6 @@ int qcow2_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t offset,
 {
 BDRVQcowState *s = bs->opaque;
 int l2_index, ret, sectors;
-uint64_t *l2_table;
 unsigned int nb_clusters, keep_clusters;
 uint64_t cluster_offset;
 uint64_t cur_bytes;
@@ -1029,6 +1106,9 @@ int qcow2_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t offset,
   n_start, n_end);
 
 again:
+cluster_offset = 0;
+*host_offset = 0;
+
 /*
  * Calculate the number of clusters to look for. We stop at L2 table
  * boundaries to keep things simple.
@@ -1054,12 +1134,6 @@ again:
  * allocation ends. Shorten the COW of the in-fight allocation, set
  * cluster_offset to write to the same cluster and set up the right
  * synchronisation between the in-flight request and the new one.
- *
- * 2. Count contiguous COPIED clusters.
- *TODO: Consider cluster_offset if set in step 1c.
- *
- * 3. If the request still hasn't completed, allocate new clusters,
- *considering any cluster_offset of steps 1c or 2.
  */
 cur_bytes = (n_end - n_start) * BDRV_SECTOR_SIZE;
 ret = handle_dependencies(bs, offset, &cur_bytes);
@@ -1076,43 +1150,18 @@ again:
 nb_clusters = size_to_clusters(s, offset + cur_bytes)
 - (offset >> s->cluster_bits);
 
-/* Find L2 entry for the first involved cluster */
-ret = get_cluster_table(bs, offset, &l2_table, &l2_index);
-if (ret < 0) {
-return ret;
-}
-
-cluster_offset = be64_to_cpu(l2_table[l2_index]);
-
-/* Check how many clusters are already allocated and don't need COW */
-if (qcow2_get_clu

Re: [Qemu-devel] kvm segfaulting

[Stefan Priebe - Profihost AG]

Hi,

I added this:
-trace events=/tmp/events,file=/root/qemu.123.trace

and put the events in the events file as i couldn't handle \n in my app
starting the kvm process. But even when doing an fstrim the trace file
stays at 24 bytes - is this correct?

Stefan
Am 13.02.2013 14:39, schrieb Paolo Bonzini:
> Il 13/02/2013 13:55, Stefan Priebe - Profihost AG ha scritto:
>> Hi,
>> Am 13.02.2013 12:36, schrieb Paolo Bonzini:
>>> Il 13/02/2013 10:07, Stefan Priebe - Profihost AG ha scritto:

 commit 47a150a4bbb06e45ef439a8222e9f46a7c4cca3f
 ...
>> You can certainly try reverting it, but this patch is fixing a real bug.
 Will try that. Yes but even if it fixes a bug and raises another one
 (kvm segfault) which is the worst one. It should be fixed.
>>>
>>> The KVM segfault is exposing a potential consistency problem.  What is
>>> worse is not obvious.  Also, it is happening at reset time if this is
>>> the culprit.  Reset usually happens at places where no data loss is caused.
>>>
>>> Can you find out what the VM was doing when it segfaulted?  (Or even,
>>> can you place the corefile and kvm executable somewhere where I can
>>> download it?)
>>
>> Yes it was doing an fstrim -v / which resulted in:
>>
>> [45648.453698] end_request: I/O error, dev sda, sector 9066952
> 
> Ok, very helpful.  One thing is to find why this failed.  This can
> come later though.
> 
> First of all, please run "cat 
> /sys/block/*/device/scsi_disk/*/provisioning_mode"
> in a VM with a similar configuration as the one that crashed last.
> 
> Second, I attach another patch.
> 
> Third, if possible please compile QEMU with --enable-trace-backend=simple,
> and run it with
> 
>   -trace events='bdrv_aio_discard
> scsi_req_cancel
> ',file=qemu.$$.trace
> 
> This can give some clues.  The files should remain quite small,
> so you can enable it on all VMs safely.
> 
>> Sadly not as i don't have acore dump. The kvm processes are started
>> through variuos Daemons and there seems no way to activate core dumps
>> for an already running process and i don't know which VM will crash next.
> 
> Probably the next that invokes fstrim. :)
> 
>>> If not, do your VMs reset themselves
>>> often for example?
>> No
> 
> Ok, good to know.
> 
>>> Can you reproduce it on non-rbd storage?
>> I don't have another storage type. ;-(
> 
> No problem.
> 
> Paolo
> 

[Qemu-devel] [RFC PATCH v2 04/23] qcow2: Decouple cluster allocation from cluster reuse code

[Kevin Wolf]

This moves some code that prepares the allocation of new clusters to
where the actual allocation happens. This is the minimum required to be
able to move it to a separate function in the next patch.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   35 ---
 1 files changed, 20 insertions(+), 15 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index a3b2447..5c4b7cc 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -943,10 +943,7 @@ again:
 
 cluster_offset = be64_to_cpu(l2_table[l2_index]);
 
-/*
- * Check how many clusters are already allocated and don't need COW, and 
how
- * many need a new allocation.
- */
+/* Check how many clusters are already allocated and don't need COW */
 if (qcow2_get_cluster_type(cluster_offset) == QCOW2_CLUSTER_NORMAL
 && (cluster_offset & QCOW_OFLAG_COPIED))
 {
@@ -962,17 +959,6 @@ again:
 cluster_offset = 0;
 }
 
-if (nb_clusters > 0) {
-/* For the moment, overwrite compressed clusters one by one */
-uint64_t entry = be64_to_cpu(l2_table[l2_index + keep_clusters]);
-if (entry & QCOW_OFLAG_COMPRESSED) {
-nb_clusters = 1;
-} else {
-nb_clusters = count_cow_clusters(s, nb_clusters, l2_table,
- l2_index + keep_clusters);
-}
-}
-
 cluster_offset &= L2E_OFFSET_MASK;
 
 /*
@@ -993,6 +979,25 @@ again:
 uint64_t alloc_cluster_offset;
 uint64_t keep_bytes = keep_clusters * s->cluster_size;
 
+ret = get_cluster_table(bs, offset, &l2_table, &l2_index);
+if (ret < 0) {
+return ret;
+}
+
+/* For the moment, overwrite compressed clusters one by one */
+uint64_t entry = be64_to_cpu(l2_table[l2_index + keep_clusters]);
+if (entry & QCOW_OFLAG_COMPRESSED) {
+nb_clusters = 1;
+} else {
+nb_clusters = count_cow_clusters(s, nb_clusters, l2_table,
+ l2_index + keep_clusters);
+}
+
+ret = qcow2_cache_put(bs, s->l2_table_cache, (void**) &l2_table);
+if (ret < 0) {
+return ret;
+}
+
 /* Calculate start and size of allocation */
 alloc_offset = offset + keep_bytes;
 
-- 
1.7.6.5

[Qemu-devel] [RFC PATCH v2 18/23] qcow2: Delay the COW

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   47 +++
 block/qcow2.c |   28 +---
 block/qcow2.h |   14 ++
 3 files changed, 86 insertions(+), 3 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index be75c17..33e595a 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -791,6 +791,52 @@ out:
 return i;
 }
 
+struct KickL2Meta {
+QEMUBH *bh;
+QCowL2Meta *m;
+};
+
+static void kick_l2meta_bh(void *opaque)
+{
+struct KickL2Meta *k = opaque;
+QCowL2Meta *m = k->m;
+
+m->kick_l2meta = NULL;
+qemu_bh_delete(k->bh);
+g_free(k);
+
+if (m->sleeping) {
+qemu_coroutine_enter(m->co, NULL);
+}
+}
+
+static void kick_l2meta(QCowL2Meta *m)
+{
+struct KickL2Meta *k;
+
+if (m->kick_l2meta) {
+return;
+}
+
+k = g_malloc(sizeof(*k));
+k->bh = qemu_bh_new(kick_l2meta_bh, k);
+k->m = m;
+
+m->kick_l2meta = k;
+
+qemu_bh_schedule(k->bh);
+}
+
+void qcow2_delete_kick_l2meta_bh(void *opaque)
+{
+if (opaque) {
+struct KickL2Meta *k = opaque;
+
+qemu_bh_delete(k->bh);
+g_free(k);
+}
+}
+
 /*
  * Check if there already is an AIO write request in flight which allocates
  * the same cluster. In this case we need to wait until the previous
@@ -833,6 +879,7 @@ static int handle_dependencies(BlockDriverState *bs, 
uint64_t guest_offset,
 /* Wait for the dependency to complete. We need to recheck
  * the free/allocated clusters when we continue. */
 qemu_co_mutex_unlock(&s->lock);
+kick_l2meta(old_alloc);
 qemu_co_queue_wait(&old_alloc->dependent_requests);
 qemu_co_mutex_lock(&s->lock);
 return -EAGAIN;
diff --git a/block/qcow2.c b/block/qcow2.c
index 3f169b8..57552aa 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -767,6 +767,13 @@ static void coroutine_fn process_l2meta(void *opaque)
 int ret;
 
 assert(s->l2meta_flush.reader > 0);
+
+if (!s->in_l2meta_flush) {
+m->sleeping = true;
+co_sleep_ns(rt_clock, 100);
+m->sleeping = false;
+}
+
 qemu_co_mutex_lock(&s->lock);
 
 ret = qcow2_alloc_cluster_link_l2(bs, m);
@@ -784,6 +791,7 @@ static void coroutine_fn process_l2meta(void *opaque)
 /* Meanwhile some new dependencies could have accumulated */
 qemu_co_queue_restart_all(&m->dependent_requests);
 
+qcow2_delete_kick_l2meta_bh(m->kick_l2meta);
 g_free(m);
 
 qemu_co_rwlock_unlock(&s->l2meta_flush);
@@ -792,6 +800,18 @@ static void coroutine_fn process_l2meta(void *opaque)
 static bool qcow2_drain(BlockDriverState *bs)
 {
 BDRVQcowState *s = bs->opaque;
+QCowL2Meta *m;
+
+s->in_l2meta_flush = true;
+again:
+QLIST_FOREACH(m, &s->cluster_allocs, next_in_flight) {
+if (m->sleeping) {
+qemu_coroutine_enter(m->co, NULL);
+/* next_in_flight link could have become invalid */
+goto again;
+}
+}
+s->in_l2meta_flush = false;
 
 return !QLIST_EMPTY(&s->cluster_allocs);
 }
@@ -800,6 +820,9 @@ static inline coroutine_fn void 
stop_l2meta(BlockDriverState *bs)
 {
 BDRVQcowState *s = bs->opaque;
 
+/* Kick the requests once if they are sleepeing and then just wait until
+ * they complete and we get the lock */
+qcow2_drain(bs);
 qemu_co_rwlock_wrlock(&s->l2meta_flush);
 }
 
@@ -893,7 +916,6 @@ static coroutine_fn int qcow2_co_writev(BlockDriverState 
*bs,
 qemu_co_mutex_unlock(&s->lock);
 
 while (l2meta != NULL) {
-Coroutine *co;
 QCowL2Meta *next;
 
 ProcessL2Meta p = {
@@ -913,8 +935,8 @@ static coroutine_fn int qcow2_co_writev(BlockDriverState 
*bs,
 /* l2meta might already be freed after the coroutine has run */
 next = l2meta->next;
 
-co = qemu_coroutine_create(process_l2meta);
-qemu_coroutine_enter(co, &p);
+l2meta->co = qemu_coroutine_create(process_l2meta);
+qemu_coroutine_enter(l2meta->co, &p);
 
 l2meta = next;
 }
diff --git a/block/qcow2.h b/block/qcow2.h
index 46ed112..1d7cdab 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -169,6 +169,7 @@ typedef struct BDRVQcowState {
  * Writers: Anyone who requires l2meta to be flushed
  */
 CoRwlock l2meta_flush;
+bool in_l2meta_flush;
 
 uint32_t crypt_method; /* current crypt method, 0 if no key yet */
 uint32_t crypt_method_header;
@@ -245,6 +246,15 @@ typedef struct QCowL2Meta
 bool is_written;
 
 /**
+ * true if the request is sleeping in the COW delay and the coroutine may
+ * be reentered in order to cancel the timer.
+ */
+bool sleeping;
+
+/** Coroutine that handles delayed COW and updates L2 entry */
+

[Qemu-devel] using -net dump with tap networking

[Alexey Kardashevskiy]

Hi!

I am running qemu as:

qemu/ppc64-softmmu/qemu-system-ppc64 -m 1024 -M pseries -trace 
events=trace_events -netdev user,id=virtnet,hostfwd=tcp::5000-:22 -device 
virtio-net-pci,netdev=virtnet -nographic -vga none -enable-kvm -kernel 
vml36_64k -initrd 1.cpio


Now I want to enable network dump. With the old "-net" syntax I could do 
that with "-net dump" but I cannot with the new syntax, tried many 
variants, none works. What would the correct syntax be for the case above?


Thanks!


--
Alexey

[Qemu-devel] [PATCH for-1.4 1/2] chardev: Fix manual page and qemu-doc for -chardev tty

[Markus Armbruster]

Broken in commit d59044ef.

Signed-off-by: Markus Armbruster 
---
 qemu-options.hx | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/qemu-options.hx b/qemu-options.hx
index 046bdc0..9d7131a 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -1956,7 +1956,7 @@ Connect to a local BrlAPI server. @option{braille} does 
not take any options.
 @item -chardev tty ,id=@var{id} ,path=@var{path}
 
 @option{tty} is only available on Linux, Sun, FreeBSD, NetBSD, OpenBSD and
-DragonFlyBSD hosts.  It is an alias for -serial.
+DragonFlyBSD hosts.  It is an alias for @option{serial}.
 
 @option{path} specifies the path to the tty. @option{path} is required.
 
-- 
1.7.11.7

[Qemu-devel] [PATCH for-1.4 2/2] qapi: Flatten away ChardevPort

[Markus Armbruster]

Simplifies the schema and the code.

QMP command

{ "execute" : "chardev-add",
  "arguments" : { "id" : "ser0",
  "backend" : { "type" : "port",
"data" : { "type": "serial",
   "device":"/dev/ttyS0"} } } }

becomes

{ "execute" : "chardev-add",
  "arguments" : { "id" : "ser0",
  "backend" : { "type" : "serial",
"data" : { "device":"/dev/ttyS0"} } } }

Bonus: nicer error messages.  "unknown chardev port (1)" becomes
"character device backend type 'parallel' not supported".

Signed-off-by: Markus Armbruster 
---
 qapi-schema.json | 11 +++-
 qemu-char.c  | 76 ++--
 2 files changed, 45 insertions(+), 42 deletions(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index bd289ae..7275b5d 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -3095,7 +3095,7 @@
'out' : 'str' } }
 
 ##
-# @ChardevPort:
+# @ChardevHostdev:
 #
 # Configuration info for device chardevs.
 #
@@ -3105,11 +3105,7 @@
 #
 # Since: 1.4
 ##
-{ 'enum': 'ChardevPortKind', 'data': [ 'serial',
-   'parallel' ] }
-
-{ 'type': 'ChardevPort', 'data': { 'device' : 'str',
-   'type'   : 'ChardevPortKind'} }
+{ 'type': 'ChardevHostdev', 'data': { 'device' : 'str' } }
 
 ##
 # @ChardevSocket:
@@ -3142,7 +3138,8 @@
 { 'type': 'ChardevDummy', 'data': { } }
 
 { 'union': 'ChardevBackend', 'data': { 'file'   : 'ChardevFile',
-   'port'   : 'ChardevPort',
+   'serial' : 'ChardevHostdev',
+   'parallel': 'ChardevHostdev',
'socket' : 'ChardevSocket',
'pty': 'ChardevDummy',
'null'   : 'ChardevDummy' } }
diff --git a/qemu-char.c b/qemu-char.c
index 574d3d2..e4b0f53 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -3269,15 +3269,17 @@ static CharDriverState 
*qmp_chardev_open_file(ChardevFile *file, Error **errp)
 return qemu_chr_open_win_file(out);
 }
 
-static CharDriverState *qmp_chardev_open_port(ChardevPort *port, Error **errp)
+static CharDriverState *qmp_chardev_open_serial(ChardevHostdev *serial,
+Error **errp)
 {
-switch (port->type) {
-case CHARDEV_PORT_KIND_SERIAL:
-return qemu_chr_open_win_path(port->device);
-default:
-error_setg(errp, "unknown chardev port (%d)", port->type);
-return NULL;
-}
+return qemu_chr_open_win_path(serial->device);
+}
+
+static CharDriverState *qmp_chardev_open_parallel(ChardevHostdev *parallel,
+  Error **errp)
+{
+error_setg(errp, "character device backend type 'parallel' not supported");
+return NULL;
 }
 
 #else /* WIN32 */
@@ -3316,38 +3318,39 @@ static CharDriverState 
*qmp_chardev_open_file(ChardevFile *file, Error **errp)
 return qemu_chr_open_fd(in, out);
 }
 
-static CharDriverState *qmp_chardev_open_port(ChardevPort *port, Error **errp)
+static CharDriverState *qmp_chardev_open_serial(ChardevHostdev *serial,
+Error **errp)
 {
-switch (port->type) {
 #ifdef HAVE_CHARDEV_TTY
-case CHARDEV_PORT_KIND_SERIAL:
-{
-int flags, fd;
-flags = O_RDWR;
-fd = qmp_chardev_open_file_source(port->device, flags, errp);
-if (error_is_set(errp)) {
-return NULL;
-}
-socket_set_nonblock(fd);
-return qemu_chr_open_tty_fd(fd);
+int fd;
+
+fd = qmp_chardev_open_file_source(serial->device, O_RDWR, errp);
+if (error_is_set(errp)) {
+return NULL;
 }
+socket_set_nonblock(fd);
+return qemu_chr_open_tty_fd(fd);
+#else
+error_setg(errp, "character device backend type 'serial' not supported");
+return NULL;
 #endif
+}
+
+static CharDriverState *qmp_chardev_open_parallel(ChardevHostdev *parallel,
+  Error **errp)
+{
 #ifdef HAVE_CHARDEV_PARPORT
-case CHARDEV_PORT_KIND_PARALLEL:
-{
-int flags, fd;
-flags = O_RDWR;
-fd = qmp_chardev_open_file_source(port->device, flags, errp);
-if (error_is_set(errp)) {
-return NULL;
-}
-return qemu_chr_open_pp_fd(fd);
-}
-#endif
-default:
-error_setg(errp, "unknown chardev port (%d)", port->type);
+int fd;
+
+fd = qmp_chardev_open_file_source(parallel->device, O_RDWR, errp);
+if (error_is_set(errp)) {
 return NULL;
 }
+return qemu_chr_open_pp_fd(fd);
+#else
+error_setg(errp, "character device backend type 'parallel' not supported");
+return NULL;
+#endif
 }
 
 #endif /* WIN3

[Qemu-devel] [PATCH for-1.4 0/2] chardev-add

[Markus Armbruster]

Why am I proposing these patches for 1.4?

PATCH 1/2 is a straightforward doc fix.

PATCH 2/2 cleans up a part of QAPI that has never been released,
before it's too late.  If we're already beyond the point where we can
do that, too bad, we'll live.

Please consider seriously.

Markus Armbruster (2):
  chardev: Fix manual page and qemu-doc for -chardev tty
  qapi: Flatten away ChardevPort

 qapi-schema.json | 11 +++-
 qemu-char.c  | 76 ++--
 qemu-options.hx  |  2 +-
 3 files changed, 46 insertions(+), 43 deletions(-)

-- 
1.7.11.7

[Qemu-devel] [RFC PATCH v2 21/23] qemu-iotests: Another concurrent multicluster allocation case

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
---
 tests/qemu-iotests/046 |   19 +++
 tests/qemu-iotests/046.out |   16 
 2 files changed, 35 insertions(+), 0 deletions(-)

diff --git a/tests/qemu-iotests/046 b/tests/qemu-iotests/046
index e0176f4..6c73675 100755
--- a/tests/qemu-iotests/046
+++ b/tests/qemu-iotests/046
@@ -153,6 +153,20 @@ aio_write -P 101 0xaa000 0xe000
 resume A
 aio_flush
 EOF
+
+# First overlap to the next cluster, and then add a sequential write
+cat < wrote 8192/8192 bytes at offset XXX
 8 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 wrote 57344/57344 bytes at offset XXX
 56 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io> qemu-io> blkdebug: Suspended request 'A'
+qemu-io> qemu-io> qemu-io> blkdebug: Suspended request 'B'
+qemu-io> qemu-io> blkdebug: Resuming request 'A'
+qemu-io> blkdebug: Resuming request 'B'
+qemu-io> wrote 65536/65536 bytes at offset XXX
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 8192/8192 bytes at offset XXX
+8 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 qemu-io> 
 == Verify image content ==
 qemu-io> read 65536/65536 bytes at offset 0
@@ -159,5 +167,13 @@ qemu-io> read 57344/57344 bytes at offset 696320
 56 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 qemu-io> read 32768/32768 bytes at offset 753664
 32 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io> read 49152/49152 bytes at offset 786432
+48 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io> read 65536/65536 bytes at offset 835584
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io> read 8192/8192 bytes at offset 901120
+8 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-io> read 8192/8192 bytes at offset 909312
+8 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 qemu-io> No errors were found on the image.
 *** done
-- 
1.7.6.5

[Qemu-devel] [RFC PATCH v2 15/23] qcow2: Allow requests with multiple l2metas

[Kevin Wolf]

Instead of expecting a single l2meta, have a list of them. This allows
to still have a single I/O request for the guest data, even though
multiple l2meta may be needed in order to describe both a COW overwrite
and a new cluster allocation (typical sequential write case).

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |3 +++
 block/qcow2.c |   14 +++---
 block/qcow2.h |3 +++
 3 files changed, 17 insertions(+), 3 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 9e6b746..645ea25 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1057,12 +1057,15 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
 int alloc_n_start = offset_into_cluster(s, guest_offset)
 >> BDRV_SECTOR_BITS;
 int nb_sectors = MIN(requested_sectors, avail_sectors);
+QCowL2Meta *old_m = *m;
 
 *host_offset = alloc_cluster_offset;
 
 *m = g_malloc0(sizeof(**m));
 
 **m = (QCowL2Meta) {
+.next   = old_m,
+
 .alloc_offset   = *host_offset,
 .offset = start_of_cluster(s, guest_offset),
 .nb_clusters= nb_clusters,
diff --git a/block/qcow2.c b/block/qcow2.c
index 971dd42..345487e 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -824,7 +824,9 @@ static coroutine_fn int qcow2_co_writev(BlockDriverState 
*bs,
 goto fail;
 }
 
-if (l2meta != NULL) {
+while (l2meta != NULL) {
+QCowL2Meta *next;
+
 ret = qcow2_alloc_cluster_link_l2(bs, l2meta);
 if (ret < 0) {
 goto fail;
@@ -837,8 +839,9 @@ static coroutine_fn int qcow2_co_writev(BlockDriverState 
*bs,
 
 qemu_co_queue_restart_all(&l2meta->dependent_requests);
 
+next = l2meta->next;
 g_free(l2meta);
-l2meta = NULL;
+l2meta = next;
 }
 
 remaining_sectors -= cur_nr_sectors;
@@ -851,12 +854,17 @@ static coroutine_fn int qcow2_co_writev(BlockDriverState 
*bs,
 fail:
 qemu_co_mutex_unlock(&s->lock);
 
-if (l2meta != NULL) {
+while (l2meta != NULL) {
+QCowL2Meta *next;
+
 if (l2meta->nb_clusters != 0) {
 QLIST_REMOVE(l2meta, next_in_flight);
 }
 qemu_co_queue_restart_all(&l2meta->dependent_requests);
+
+next = l2meta->next;
 g_free(l2meta);
+l2meta = next;
 }
 
 qemu_iovec_destroy(&hd_qiov);
diff --git a/block/qcow2.h b/block/qcow2.h
index bfdf71d..9b16538 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -246,6 +246,9 @@ typedef struct QCowL2Meta
  */
 Qcow2COWRegion cow_end;
 
+/** Pointer to next L2Meta of the same write request */
+struct QCowL2Meta *next;
+
 QLIST_ENTRY(QCowL2Meta) next_in_flight;
 } QCowL2Meta;
 
-- 
1.7.6.5

[Qemu-devel] [RFC PATCH v2 23/23] qcow2: Gather clusters in a looping loop

[Kevin Wolf]

Instead of just checking once in exactly this order if there are
dependendies, non-COW clusters and new allocation, this starts looping
around these. This way we can, for example, gather non-COW clusters after
new allocations as long as the host cluster offsets stay contiguous.

More importantly, after overwriting a COW in handle_dependencies() we
can now continue with gathering other clusters (we couldn't do that
before because we would miss a possible second dependency in one of the
next clusters).

This means that in the typical sequential write case, we can combine the
COW overwrite of one cluster with the allocation of the next cluster.
Only by avoiding splitting requests this way Delayed COW actually starts
improving performance noticably.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   66 ++--
 1 files changed, 30 insertions(+), 36 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 5a97d87..34f7299 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1209,16 +1209,16 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
 nb_clusters = count_cow_clusters(s, nb_clusters, l2_table, l2_index);
 }
 
+/* This function is only called when there were no non-COW clusters, so if
+ * we can't find any unallocated or COW clusters either, something is
+ * wrong with our code. */
+assert(nb_clusters > 0);
+
 ret = qcow2_cache_put(bs, s->l2_table_cache, (void**) &l2_table);
 if (ret < 0) {
 return ret;
 }
 
-if (nb_clusters == 0) {
-*bytes = 0;
-return 0;
-}
-
 /* Allocate, if necessary at a given offset in the image file */
 alloc_cluster_offset = *host_offset;
 ret = do_alloc_cluster_offset(bs, guest_offset, &alloc_cluster_offset,
@@ -1334,9 +1334,27 @@ again:
 remaining = (n_end - n_start) << BDRV_SECTOR_BITS;
 cluster_offset = 0;
 *host_offset = 0;
+cur_bytes = 0;
 *m = NULL;
 
 while (true) {
+
+if (!*host_offset) {
+*host_offset = start_of_cluster(s, cluster_offset);
+}
+
+assert(remaining >= cur_bytes);
+
+start   += cur_bytes;
+remaining   -= cur_bytes;
+cluster_offset  += cur_bytes;
+
+if (remaining == 0) {
+break;
+}
+
+cur_bytes = remaining;
+
 /*
  * Now start gathering as many contiguous clusters as possible:
  *
@@ -1355,20 +1373,17 @@ again:
  * the right synchronisation between the in-flight request and
  * the new one.
  */
-cur_bytes = remaining;
 ret = handle_dependencies(bs, start, &cluster_offset, &cur_bytes, m);
 if (ret == -EAGAIN) {
+/* Currently handle_dependencies() doesn't yield if we already had
+ * an allocation. If it did, we would have to clean up the L2Meta
+ * structs before starting over. */
+assert(*m == NULL);
 goto again;
 } else if (ret < 0) {
 return ret;
 } else if (ret) {
-*host_offset = start_of_cluster(s, cluster_offset);
-
-start   += cur_bytes;
-remaining   -= cur_bytes;
-cluster_offset  += cur_bytes;
-
-break;
+continue;
 } else if (cur_bytes == 0) {
 break;
 } else {
@@ -1384,24 +1399,11 @@ again:
 if (ret < 0) {
 return ret;
 } else if (ret) {
-if (!*host_offset) {
-*host_offset = cluster_offset;
-}
-
-start   += cur_bytes;
-remaining   -= cur_bytes;
-cluster_offset  += cur_bytes;
-
-cur_bytes = remaining;
+continue;
 } else if (cur_bytes == 0) {
 break;
 }
 
-/* If there is something left to allocate, do that now */
-if (remaining == 0) {
-break;
-}
-
 /*
  * 3. If the request still hasn't completed, allocate new clusters,
  *considering any cluster_offset of steps 1c or 2.
@@ -1410,15 +1412,7 @@ again:
 if (ret < 0) {
 return ret;
 } else if (ret) {
-if (!*host_offset) {
-*host_offset = cluster_offset;
-}
-
-start   += cur_bytes;
-remaining   -= cur_bytes;
-cluster_offset  += cur_bytes;
-
-break;
+continue;
 } else {
 assert(cur_bytes == 0);
 break;
-- 
1.7.6.5

[Qemu-devel] [RFC PATCH v2 13/23] qcow2: handle_copied(): Implement non-zero host_offset

[Kevin Wolf]

Look only for clusters that start at a given physical offset.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   26 ++
 1 files changed, 18 insertions(+), 8 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 5ce2c88..90fe36c 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -827,8 +827,6 @@ static int handle_dependencies(BlockDriverState *bs, 
uint64_t guest_offset,
  *  the length of the area that can be written to.
  *
  *  -errno: in error cases
- *
- * TODO Make non-zero host_offset behave like describe above
  */
 static int handle_copied(BlockDriverState *bs, uint64_t guest_offset,
 uint64_t *host_offset, uint64_t *bytes, QCowL2Meta **m)
@@ -843,7 +841,6 @@ static int handle_copied(BlockDriverState *bs, uint64_t 
guest_offset,
 
 trace_qcow2_handle_copied(qemu_coroutine_self(), guest_offset, 
*host_offset,
   *bytes);
-assert(*host_offset == 0);
 
 /*
  * Calculate the number of clusters to look for. We stop at L2 table
@@ -867,6 +864,15 @@ static int handle_copied(BlockDriverState *bs, uint64_t 
guest_offset,
 if (qcow2_get_cluster_type(cluster_offset) == QCOW2_CLUSTER_NORMAL
 && (cluster_offset & QCOW_OFLAG_COPIED))
 {
+/* If a specific host_offset is required, check it */
+if (*host_offset != 0
+&& (cluster_offset & L2E_OFFSET_MASK) != *host_offset)
+{
+*bytes = 0;
+ret = 0;
+goto out;
+}
+
 /* We keep all QCOW_OFLAG_COPIED clusters */
 keep_clusters =
 count_contiguous_clusters(nb_clusters, s->cluster_size,
@@ -880,19 +886,22 @@ static int handle_copied(BlockDriverState *bs, uint64_t 
guest_offset,
 
 ret = 1;
 } else {
-cluster_offset = 0;
 ret = 0;
 }
 
-cluster_offset &= L2E_OFFSET_MASK;
-*host_offset = cluster_offset;
-
 /* Cleanup */
+out:
 pret = qcow2_cache_put(bs, s->l2_table_cache, (void**) &l2_table);
 if (pret < 0) {
 return pret;
 }
 
+/* Only return a host offset if we actually made progress. Otherwise we
+ * would make requirements for handle_alloc() that it can't fulfill */
+if (ret) {
+*host_offset = cluster_offset & L2E_OFFSET_MASK;
+}
+
 return ret;
 }
 
@@ -1162,7 +1171,6 @@ again:
 
 /*
  * 2. Count contiguous COPIED clusters.
- *TODO: Consider cluster_offset if set in step 1c.
  */
 ret = handle_copied(bs, offset, &cluster_offset, &cur_bytes, m);
 if (ret < 0) {
@@ -1175,6 +1183,8 @@ again:
 if (!*host_offset) {
 *host_offset = cluster_offset;
 }
+} else if (cur_bytes == 0) {
+goto done;
 } else {
 keep_clusters = 0;
 }
-- 
1.7.6.5

Re: [Qemu-devel] [PATCH 3/5] target-i386: Slim conversion to X86CPU subclasses

[Igor Mammedov]

On Tue, 12 Feb 2013 12:48:47 -0200
Eduardo Habkost  wrote:

> On Mon, Feb 11, 2013 at 02:52:49AM +0100, Igor Mammedov wrote:
> > On Fri, 8 Feb 2013 16:13:02 -0200
> > Eduardo Habkost  wrote:
> > 
> > > On Fri, Feb 08, 2013 at 05:54:50PM +0100, Andreas Färber wrote:
> > > > Am 08.02.2013 15:52, schrieb Eduardo Habkost:
> > > > > On Fri, Feb 08, 2013 at 01:58:42PM +0100, Igor Mammedov wrote:
> > > > >> On Fri, 08 Feb 2013 12:16:17 +0100
> > > > >> Andreas Färber  wrote:
> > > > >>> Am 08.02.2013 10:03, schrieb Igor Mammedov:
> > > >  On Thu, 7 Feb 2013 13:08:19 -0200
> > > >  Eduardo Habkost  wrote:
> > > > 
> > > > > On Tue, Feb 05, 2013 at 05:39:22PM +0100, Igor Mammedov wrote:
> > > > >> @@ -2236,6 +2083,44 @@ static void x86_cpu_initfn(Object *obj)
> > > > >>  }
> > > > >>  }
> > > > >>  
> > > > >> +static void x86_cpu_def_class_init(ObjectClass *oc, void
> > > > >> *data) +{
> > > > >> +X86CPUClass *xcc = X86_CPU_CLASS(oc);
> > > > >> +ObjectClass *hoc =
> > > > >> object_class_by_name(TYPE_HOST_X86_CPU);
> > > > >> +X86CPUClass *hostcc;
> > > > >> +x86_def_t *def = data;
> > > > >> +int i;
> > > > >> +static const char *versioned_models[] = { "qemu32",
> > > > >> "qemu64", "athlon" }; +
> > > > >> +memcpy(&xcc->info, def, sizeof(x86_def_t));
> > > > >> +
> > > > >> +/* host cpu class is available if KVM is enabled,
> > > > >> + * get kvm overrides from it */
> > > > >> +if (hoc) {
> > > > >> +hostcc = X86_CPU_CLASS(hoc);
> > > > >> +/* sysenter isn't supported in compatibility mode on
> > > > >> AMD,
> > > > >> + * syscall isn't supported in compatibility mode on
> > > > >> Intel.
> > > > >> + * Normally we advertise the actual CPU vendor, but
> > > > >> you can
> > > > >> + * override this using the 'vendor' property if you
> > > > >> want to use
> > > > >> + * KVM's sysenter/syscall emulation in compatibility
> > > > >> mode and
> > > > >> + * when doing cross vendor migration
> > > > >> + */
> > > > >> +memcpy(xcc->info.vendor, hostcc->info.vendor,
> > > > >> +   sizeof(xcc->info.vendor));
> > > > >> +}
> > > > >
> > > > > Again, we have the same problem we had before, but now in the
> > > > > non-host classes. What if class_init is called before KVM is
> > > > > initialized? I believe we will be forced to move this hack to
> > > > > the instance init function.
> > > >  I believe, the in the case where non-host CPU classes might be
> > > >  initialized before KVM "-cpu ?" we do not care what their
> > > >  defaults are, since we only would use class names there and then
> > > >  exit.
> > > > 
> > > >  For case where classes could be inspected over QMP, OQM, KVM
> > > >  would be already initialized if enabled and we would get proper
> > > >  initialization order without hack.
> > > > > 
> > > > > Who guarantees that KVM will be already initialized when we get a
> > > > > QMP monitor? We can't do that today because of limitations in the
> > > > > QEMU main code, but I believe we want to get rid of this limitation
> > > > > eventually, instead of making it harder to get rid of.
> > > > > 
> > > > > If we could initialize KVM before QMP is initialized, we could
> > > > > simply initialize KVM before class_init is called, instead. It
> > > > > would be easier to reason about, and it would make the limitations
> > > > > of our code very clear to anybody reading the code in main().
> > > > That wouldn't work (currently) due to -device and -object being
> > > > command line options just like -enable-kvm, -disable-kvm and -machine
> > > > accel=.
> > > 
> > > Well, we could loop over the command-line options twice.
> > > 
> > > It is just an alternative that would be better than making class_init
> > > unreliable. I don't think it would be a great solution anyway.
> > > 
> > > > 
> > > > >>>
> > > > >>> I think you're missing Eduardo's and my point:
> > > > >>>
> > > > >>> diff --git a/vl.c b/vl.c
> > > > >>> index a8dc73d..6b9378e 100644
> > > > >>> --- a/vl.c
> > > > >>> +++ b/vl.c
> > > > >>> @@ -2844,6 +2844,7 @@ int main(int argc, char **argv, char **envp)
> > > > >>>  }
> > > > >>>
> > > > >>>  module_call_init(MODULE_INIT_QOM);
> > > > >>> +object_class_foreach(walkerfn, TYPE_OBJECT, false, NULL);
> > > > >>>
> > > > >>>  qemu_add_opts(&qemu_drive_opts);
> > > > >>>  qemu_add_opts(&qemu_chardev_opts);
> > > > >>>
> > > > >>> Anyone may iterate over QOM classes at any time after their type
> > > > >>> registration, which is before the first round of option parsing.
> > > > >>> Sometime later, after option parsing, there's the -cpu ? handling
> > > > >>> in vl.c:3854, then vl.c:4018:configure_accelerator().
> > > > >>>
> > > > >>> Like I said, mos

Re: [Qemu-devel] kvm segfaulting

[Paolo Bonzini]

Il 13/02/2013 15:30, Stefan Priebe - Profihost AG ha scritto:
> I added this:
> -trace events=/tmp/events,file=/root/qemu.123.trace
> 
> and put the events in the events file as i couldn't handle \n in my app
> starting the kvm process. But even when doing an fstrim the trace file
> stays at 24 bytes - is this correct?

Right... it would eventually flush, but not if qemu-kvm crash.

Answering your other question, the patch subsumes the other.  But if the
provisioning mode is writesame_16, this hunk alone will most likely fix
the crash:

diff --git a/hw/scsi-disk.c b/hw/scsi-disk.c
index d411586..4a0673c 100644
--- a/hw/scsi-disk.c
+++ b/hw/scsi-disk.c
@@ -178,6 +178,9 @@ static void scsi_aio_complete(void *opaque, int ret)
 assert(r->req.aiocb != NULL);
 r->req.aiocb = NULL;
 bdrv_acct_done(s->qdev.conf.bs, &r->acct);
+if (r->req.io_canceled) {
+goto done;
+}

 if (ret < 0) {
 if (scsi_handle_rw_error(r, -ret)) {

Paolo

[Qemu-devel] [RFC PATCH v2 03/23] qcow2: Change handle_dependency to byte granularity

[Kevin Wolf]

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   40 
 block/qcow2.h |   11 +++
 2 files changed, 39 insertions(+), 12 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 0e804ba..a3b2447 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -756,31 +756,41 @@ out:
  * Check if there already is an AIO write request in flight which allocates
  * the same cluster. In this case we need to wait until the previous
  * request has completed and updated the L2 table accordingly.
+ *
+ * Returns:
+ *   0   if there was no dependency. *cur_bytes indicates the number of
+ *   bytes from guest_offset that can be read before the next
+ *   dependency must be processed (or the request is complete)
+ *
+ *   -EAGAIN if we had to wait for another request, previously gathered
+ *   information on cluster allocation may be invalid now. The caller
+ *   must start over anyway, so consider *cur_bytes undefined.
  */
 static int handle_dependencies(BlockDriverState *bs, uint64_t guest_offset,
-unsigned int *nb_clusters)
+uint64_t *cur_bytes)
 {
 BDRVQcowState *s = bs->opaque;
 QCowL2Meta *old_alloc;
+uint64_t bytes = *cur_bytes;
 
 QLIST_FOREACH(old_alloc, &s->cluster_allocs, next_in_flight) {
 
-uint64_t start = guest_offset >> s->cluster_bits;
-uint64_t end = start + *nb_clusters;
-uint64_t old_start = old_alloc->offset >> s->cluster_bits;
-uint64_t old_end = old_start + old_alloc->nb_clusters;
+uint64_t start = guest_offset;
+uint64_t end = start + bytes;
+uint64_t old_start = l2meta_cow_start(old_alloc);
+uint64_t old_end = l2meta_cow_end(old_alloc);
 
 if (end <= old_start || start >= old_end) {
 /* No intersection */
 } else {
 if (start < old_start) {
 /* Stop at the start of a running allocation */
-*nb_clusters = old_start - start;
+bytes = old_start - start;
 } else {
-*nb_clusters = 0;
+bytes = 0;
 }
 
-if (*nb_clusters == 0) {
+if (bytes == 0) {
 /* Wait for the dependency to complete. We need to recheck
  * the free/allocated clusters when we continue. */
 qemu_co_mutex_unlock(&s->lock);
@@ -791,9 +801,9 @@ static int handle_dependencies(BlockDriverState *bs, 
uint64_t guest_offset,
 }
 }
 
-if (!*nb_clusters) {
-abort();
-}
+/* Make sure that existing clusters and new allocations are only used up to
+ * the next dependency if we shortened the request above */
+*cur_bytes = bytes;
 
 return 0;
 }
@@ -872,6 +882,7 @@ int qcow2_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t offset,
 uint64_t *l2_table;
 unsigned int nb_clusters, keep_clusters;
 uint64_t cluster_offset;
+uint64_t cur_bytes;
 
 trace_qcow2_alloc_clusters_offset(qemu_coroutine_self(), offset,
   n_start, n_end);
@@ -884,6 +895,7 @@ again:
 l2_index = offset_to_l2_index(s, offset);
 nb_clusters = MIN(size_to_clusters(s, n_end << BDRV_SECTOR_BITS),
   s->l2_size - l2_index);
+n_end = MIN(n_end, nb_clusters * s->cluster_sectors);
 
 /*
  * Now start gathering as many contiguous clusters as possible:
@@ -908,7 +920,8 @@ again:
  * 3. If the request still hasn't completed, allocate new clusters,
  *considering any cluster_offset of steps 1c or 2.
  */
-ret = handle_dependencies(bs, offset, &nb_clusters);
+cur_bytes = (n_end - n_start) * BDRV_SECTOR_SIZE;
+ret = handle_dependencies(bs, offset, &cur_bytes);
 if (ret == -EAGAIN) {
 goto again;
 } else if (ret < 0) {
@@ -919,6 +932,9 @@ again:
  * correctly during the next loop iteration. */
 }
 
+nb_clusters = size_to_clusters(s, offset + cur_bytes)
+- (offset >> s->cluster_bits);
+
 /* Find L2 entry for the first involved cluster */
 ret = get_cluster_table(bs, offset, &l2_table, &l2_index);
 if (ret < 0) {
diff --git a/block/qcow2.h b/block/qcow2.h
index 1322012..72e6d12 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -303,6 +303,17 @@ static inline bool 
qcow2_need_accurate_refcounts(BDRVQcowState *s)
 return !(s->incompatible_features & QCOW2_INCOMPAT_DIRTY);
 }
 
+static inline uint64_t l2meta_cow_start(QCowL2Meta *m)
+{
+return m->offset + m->cow_start.offset;
+}
+
+static inline uint64_t l2meta_cow_end(QCowL2Meta *m)
+{
+return m->offset + m->cow_end.offset
++ (m->cow_end.nb_sectors << BDRV_SECTOR_BITS);
+}
+
 // FIXME Need qcow2_ prefix to global functions
 
 /* qcow2.c functions */
-- 
1.7.6.5

[Qemu-devel] [RFC PATCH v2 12/23] qcow2: handle_copied(): Get rid of keep_clusters parameter

[Kevin Wolf]

Now *bytes is used to return the length of the area that can be written
to without performing an allocation or COW.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   23 +--
 1 files changed, 13 insertions(+), 10 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index cbe98bc..5ce2c88 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -828,19 +828,17 @@ static int handle_dependencies(BlockDriverState *bs, 
uint64_t guest_offset,
  *
  *  -errno: in error cases
  *
- * TODO Get rid of keep_clusters parameter
- * TODO Make bytes behave like described above
  * TODO Make non-zero host_offset behave like describe above
  */
 static int handle_copied(BlockDriverState *bs, uint64_t guest_offset,
-uint64_t *host_offset, uint64_t *bytes, QCowL2Meta **m,
-unsigned int *keep_clusters)
+uint64_t *host_offset, uint64_t *bytes, QCowL2Meta **m)
 {
 BDRVQcowState *s = bs->opaque;
 int l2_index;
 uint64_t cluster_offset;
 uint64_t *l2_table;
 unsigned int nb_clusters;
+unsigned int keep_clusters;
 int ret, pret;
 
 trace_qcow2_handle_copied(qemu_coroutine_self(), guest_offset, 
*host_offset,
@@ -870,17 +868,19 @@ static int handle_copied(BlockDriverState *bs, uint64_t 
guest_offset,
 && (cluster_offset & QCOW_OFLAG_COPIED))
 {
 /* We keep all QCOW_OFLAG_COPIED clusters */
-*keep_clusters =
+keep_clusters =
 count_contiguous_clusters(nb_clusters, s->cluster_size,
   &l2_table[l2_index], 0,
   QCOW_OFLAG_COPIED | QCOW_OFLAG_ZERO);
-assert(*keep_clusters <= nb_clusters);
+assert(keep_clusters <= nb_clusters);
+
+*bytes = MIN(*bytes,
+ keep_clusters * s->cluster_size
+ - offset_into_cluster(s, guest_offset));
 
 ret = 1;
 } else {
-*keep_clusters = 0;
 cluster_offset = 0;
-
 ret = 0;
 }
 
@@ -1164,16 +1164,19 @@ again:
  * 2. Count contiguous COPIED clusters.
  *TODO: Consider cluster_offset if set in step 1c.
  */
-ret = handle_copied(bs, offset, &cluster_offset, &cur_bytes, m,
-&keep_clusters);
+ret = handle_copied(bs, offset, &cluster_offset, &cur_bytes, m);
 if (ret < 0) {
 return ret;
 } else if (ret) {
+keep_clusters =
+size_to_clusters(s, cur_bytes + offset_into_cluster(s, offset));
 nb_clusters -= keep_clusters;
 
 if (!*host_offset) {
 *host_offset = cluster_offset;
 }
+} else {
+keep_clusters = 0;
 }
 
 /* If there is something left to allocate, do that now */
-- 
1.7.6.5

[Qemu-devel] [RFC v5] target-i386: Slim conversion to X86CPU subclasses + KVM subclasses

[Igor Mammedov]

From: Andreas Färber 

Depends on http://lists.gnu.org/archive/html/qemu-devel/2013-02/msg00677.html

Move x86_def_t definition to header and embed into X86CPUClass.
Register types per built-in model definition.

Move version initialization from x86_cpudef_setup() to class_init().

Move default setting of CPUID_EXT_HYPERVISOR to class_init().

Move KVM specific built-in CPU defaults overrides in a kvm specific
x86_cpu_kvm_def_class_init(). And select TCG vs KVM class of CPU
to create at runtime in x86_cpu_class_by_name() when kvm_enable()
is available.

Inline cpu_x86_register() into the X86CPU initfn.
Since instance_init cannot reports errors, die there if some
of default values are incorrect, instead of ignoring errors.

Replace cpu_x86_find_by_name() with x86_cpu_class_by_name().
Move handling of KVM host vendor override from cpu_x86_find_by_name()
to the kvm_arch_init() and class_init(). Use TYPE_X86_CPU class to
communicate kvm specific defaults to other sub-classes.

Register host-kvm-{i386,x86_64}-cpu type from KVM code to avoid #ifdefs
and only when KVM is enabled to avoid workarounds in name to class
lookup code in x86_cpu_class_by_name().
Make kvm_cpu_fill_host() into a host specific class_init and inline
cpu_x86_fill_model_id().

Let kvm_check_features_against_host() obtain host-kvm-{i386,86_64}-cpu
for comparison.

Signed-off-by: Andreas Färber 
Signed-off-by: Igor Mammedov 
---
v5:
  * remove special case for 'host' CPU check in x86_cpu_class_by_name(),
due to 'host' CPU will not find anything if not in KVM mode or
return 'host' CPU class in KVM mode, i.e. treat it as regular CPUs.
  * register KVM specific subclasses for built-in CPU models.
  * abort() in instance_init() if property setter fails to set default
value.
v4:
  * set error if cpu model is not found and goto out;
  * copy vendor override from 'host' CPU class in sub-class'es
class_init() if 'host' CPU class is available.
  * register type TYPE_HOST_X86_CPU in kvm_arch_init(), this type
should be available only in KVM mode and we haven't printed it in
-cpu ? output so far, so we can continue doing so. It's not
really confusing to show 'host' cpu (even if we do it) when KVM
is not enabled.
---
 target-i386/cpu-qom.h |   24 
 target-i386/cpu.c |  348 +++--
 target-i386/cpu.h |5 +-
 target-i386/kvm.c |   72 ++
 4 files changed, 232 insertions(+), 217 deletions(-)

diff --git a/target-i386/cpu-qom.h b/target-i386/cpu-qom.h
index 48e6b54..c8f320d 100644
--- a/target-i386/cpu-qom.h
+++ b/target-i386/cpu-qom.h
@@ -30,6 +30,27 @@
 #define TYPE_X86_CPU "i386-cpu"
 #endif
 
+#define TYPE_HOST_X86_CPU "host-kvm-" TYPE_X86_CPU
+
+typedef struct x86_def_t {
+const char *name;
+uint32_t level;
+/* vendor is zero-terminated, 12 character ASCII string */
+char vendor[CPUID_VENDOR_SZ + 1];
+int family;
+int model;
+int stepping;
+uint32_t features, ext_features, ext2_features, ext3_features;
+uint32_t kvm_features, svm_features;
+uint32_t xlevel;
+char model_id[48];
+/* Store the results of Centaur's CPUID instructions */
+uint32_t ext4_features;
+uint32_t xlevel2;
+/* The feature bits on CPUID[EAX=7,ECX=0].EBX */
+uint32_t cpuid_7_0_ebx_features;
+} x86_def_t;
+
 #define X86_CPU_CLASS(klass) \
 OBJECT_CLASS_CHECK(X86CPUClass, (klass), TYPE_X86_CPU)
 #define X86_CPU(obj) \
@@ -41,6 +62,7 @@
  * X86CPUClass:
  * @parent_realize: The parent class' realize handler.
  * @parent_reset: The parent class' reset handler.
+ * @info: Model-specific data.
  *
  * An x86 CPU model or family.
  */
@@ -51,6 +73,8 @@ typedef struct X86CPUClass {
 
 DeviceRealize parent_realize;
 void (*parent_reset)(CPUState *cpu);
+
+x86_def_t info;
 } X86CPUClass;
 
 /**
diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 1aee097..b786a57 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -47,8 +47,8 @@
 #include "hw/apic_internal.h"
 #endif
 
-static void x86_cpu_vendor_words2str(char *dst, uint32_t vendor1,
- uint32_t vendor2, uint32_t vendor3)
+void x86_cpu_vendor_words2str(char *dst, uint32_t vendor1,
+  uint32_t vendor2, uint32_t vendor3)
 {
 int i;
 for (i = 0; i < 4; i++) {
@@ -346,25 +346,6 @@ static void add_flagname_to_bitmaps(const char *flagname,
 }
 }
 
-typedef struct x86_def_t {
-const char *name;
-uint32_t level;
-/* vendor is zero-terminated, 12 character ASCII string */
-char vendor[CPUID_VENDOR_SZ + 1];
-int family;
-int model;
-int stepping;
-uint32_t features, ext_features, ext2_features, ext3_features;
-uint32_t kvm_features, svm_features;
-uint32_t xlevel;
-char model_id[48];
-/* Store the results of Centaur's CPUID instructions */
-uint32_t ext4_features;
-uint32_t xlevel2;
-/* The feature bits on CPUID[EAX=7,ECX=0].EBX */
-

[Qemu-devel] [RFC PATCH v2 09/23] qcow2: Clean up handle_alloc()

[Kevin Wolf]

Things can be simplified a bit now. No semantic changes.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |  110 +++-
 1 files changed, 53 insertions(+), 57 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index e32bfe3..0642ffa 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -883,7 +883,6 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
 unsigned int nb_clusters;
 int ret;
 
-uint64_t alloc_offset;
 uint64_t alloc_cluster_offset;
 
 trace_qcow2_handle_alloc(qemu_coroutine_self(), guest_offset, *host_offset,
@@ -925,72 +924,69 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
 return 0;
 }
 
-/* Calculate start and size of allocation */
-alloc_offset = guest_offset;
-alloc_cluster_offset = *host_offset;
-
 /* Allocate, if necessary at a given offset in the image file */
-ret = do_alloc_cluster_offset(bs, alloc_offset, &alloc_cluster_offset,
+alloc_cluster_offset = *host_offset;
+ret = do_alloc_cluster_offset(bs, guest_offset, &alloc_cluster_offset,
   &nb_clusters);
 if (ret < 0) {
 goto fail;
 }
 
-/* save info needed for meta data update */
-if (nb_clusters > 0) {
-/*
- * requested_sectors: Number of sectors from the start of the first
- * newly allocated cluster to the end of the (possibly shortened
- * before) write request.
- *
- * avail_sectors: Number of sectors from the start of the first
- * newly allocated to the end of the last newly allocated cluster.
- *
- * nb_sectors: The number of sectors from the start of the first
- * newly allocated cluster to the end of the aread that the write
- * request actually writes to (excluding COW at the end)
- */
-int requested_sectors =
-(*bytes + offset_into_cluster(s, guest_offset))
->> BDRV_SECTOR_BITS;
-int avail_sectors = nb_clusters
-<< (s->cluster_bits - BDRV_SECTOR_BITS);
-int alloc_n_start = offset_into_cluster(s, guest_offset)
->> BDRV_SECTOR_BITS;
-int nb_sectors = MIN(requested_sectors, avail_sectors);
-
-if (*host_offset == 0) {
-*host_offset = alloc_cluster_offset;
-}
-
-*m = g_malloc0(sizeof(**m));
-
-**m = (QCowL2Meta) {
-.alloc_offset   = alloc_cluster_offset,
-.offset = alloc_offset & ~(s->cluster_size - 1),
-.nb_clusters= nb_clusters,
-.nb_available   = nb_sectors,
-
-.cow_start = {
-.offset = 0,
-.nb_sectors = alloc_n_start,
-},
-.cow_end = {
-.offset = nb_sectors * BDRV_SECTOR_SIZE,
-.nb_sectors = avail_sectors - nb_sectors,
-},
-};
-qemu_co_queue_init(&(*m)->dependent_requests);
-QLIST_INSERT_HEAD(&s->cluster_allocs, *m, next_in_flight);
-
-*bytes = MIN(*bytes, (nb_sectors * BDRV_SECTOR_SIZE)
- - offset_into_cluster(s, guest_offset));
-assert(*bytes != 0);
-} else {
+/* Can't extend contiguous allocation */
+if (nb_clusters == 0) {
 *bytes = 0;
 return 0;
 }
 
+/*
+ * Save info needed for meta data update.
+ *
+ * requested_sectors: Number of sectors from the start of the first
+ * newly allocated cluster to the end of the (possibly shortened
+ * before) write request.
+ *
+ * avail_sectors: Number of sectors from the start of the first
+ * newly allocated to the end of the last newly allocated cluster.
+ *
+ * nb_sectors: The number of sectors from the start of the first
+ * newly allocated cluster to the end of the aread that the write
+ * request actually writes to (excluding COW at the end)
+ */
+int requested_sectors =
+(*bytes + offset_into_cluster(s, guest_offset))
+>> BDRV_SECTOR_BITS;
+int avail_sectors = nb_clusters
+<< (s->cluster_bits - BDRV_SECTOR_BITS);
+int alloc_n_start = offset_into_cluster(s, guest_offset)
+>> BDRV_SECTOR_BITS;
+int nb_sectors = MIN(requested_sectors, avail_sectors);
+
+*host_offset = alloc_cluster_offset;
+
+*m = g_malloc0(sizeof(**m));
+
+**m = (QCowL2Meta) {
+.alloc_offset   = *host_offset,
+.offset = start_of_cluster(s, guest_offset),
+.nb_clusters= nb_clusters,
+.nb_available   = nb_sectors,
+
+.cow_start = {
+.offset = 0,
+.nb_sectors = alloc_n_start,
+},
+.cow_end = {
+.offset = nb_sectors * BDRV_SECTOR_SIZE,
+.nb_sectors = avail_sectors - nb_sectors,
+ 

[Qemu-devel] [RFC PATCH v2 07/23] qcow2: handle_alloc(): Get rid of keep_clusters parameter

[Kevin Wolf]

handle_alloc() is now called with the offset at which the actual new
allocation starts instead of the offset at which the whole write request
starts, part of which may already be processed.

Signed-off-by: Kevin Wolf 
---
 block/qcow2-cluster.c |   44 +++-
 block/qcow2.h |5 +
 2 files changed, 32 insertions(+), 17 deletions(-)

diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 2492235..2f64bf7 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -873,12 +873,12 @@ static int do_alloc_cluster_offset(BlockDriverState *bs, 
uint64_t guest_offset,
  *
  *  -errno: in error cases
  *
- * TODO Get rid of keep_clusters, n_start, n_end
+ * TODO Get rid of n_start, n_end
  * TODO Make *bytes actually behave as specified above
  */
 static int handle_alloc(BlockDriverState *bs, uint64_t guest_offset,
 uint64_t *host_offset, uint64_t *bytes, QCowL2Meta **m,
-int keep_clusters, int n_start, int n_end)
+int n_start, int n_end)
 {
 BDRVQcowState *s = bs->opaque;
 int l2_index;
@@ -889,7 +889,6 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
 
 uint64_t alloc_offset;
 uint64_t alloc_cluster_offset;
-uint64_t keep_bytes = keep_clusters * s->cluster_size;
 
 trace_qcow2_handle_alloc(qemu_coroutine_self(), guest_offset, *host_offset,
  *bytes);
@@ -908,14 +907,13 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
 return ret;
 }
 
-entry = be64_to_cpu(l2_table[l2_index + keep_clusters]);
+entry = be64_to_cpu(l2_table[l2_index]);
 
 /* For the moment, overwrite compressed clusters one by one */
 if (entry & QCOW_OFLAG_COMPRESSED) {
 nb_clusters = 1;
 } else {
-nb_clusters = count_cow_clusters(s, nb_clusters, l2_table,
- l2_index + keep_clusters);
+nb_clusters = count_cow_clusters(s, nb_clusters, l2_table, l2_index);
 }
 
 ret = qcow2_cache_put(bs, s->l2_table_cache, (void**) &l2_table);
@@ -929,13 +927,8 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
 }
 
 /* Calculate start and size of allocation */
-alloc_offset = guest_offset + keep_bytes;
-
-if (keep_clusters == 0) {
-alloc_cluster_offset = 0;
-} else {
-alloc_cluster_offset = *host_offset + keep_bytes;
-}
+alloc_offset = guest_offset;
+alloc_cluster_offset = *host_offset;
 
 /* Allocate, if necessary at a given offset in the image file */
 ret = do_alloc_cluster_offset(bs, alloc_offset, &alloc_cluster_offset,
@@ -958,13 +951,13 @@ static int handle_alloc(BlockDriverState *bs, uint64_t 
guest_offset,
  * newly allocated cluster to the end of the aread that the write
  * request actually writes to (excluding COW at the end)
  */
-int requested_sectors = n_end - keep_clusters * s->cluster_sectors;
+int requested_sectors = n_end;
 int avail_sectors = nb_clusters
 << (s->cluster_bits - BDRV_SECTOR_BITS);
-int alloc_n_start = keep_clusters == 0 ? n_start : 0;
+int alloc_n_start = *host_offset == 0 ? n_start : 0;
 int nb_sectors = MIN(requested_sectors, avail_sectors);
 
-if (keep_clusters == 0) {
+if (*host_offset == 0) {
 *host_offset = alloc_cluster_offset;
 }
 
@@ -1127,9 +1120,26 @@ again:
 goto done;
 }
 
+int alloc_n_start;
+int alloc_n_end;
+
+if (keep_clusters != 0) {
+offset = start_of_cluster(s, offset
+ + keep_clusters * 
s->cluster_size);
+cluster_offset = start_of_cluster(s, cluster_offset
+ + keep_clusters * 
s->cluster_size);
+
+alloc_n_start = 0;
+alloc_n_end = n_end - keep_clusters * s->cluster_sectors;
+} else {
+alloc_n_start = n_start;
+alloc_n_end = n_end;
+}
+
 cur_bytes = nb_clusters * s->cluster_size;
+
 ret = handle_alloc(bs, offset, &cluster_offset, &cur_bytes, m,
-   keep_clusters, n_start, n_end);
+   alloc_n_start, alloc_n_end);
 if (ret < 0) {
 return ret;
 }
diff --git a/block/qcow2.h b/block/qcow2.h
index 72e6d12..f90bfb1 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -262,6 +262,11 @@ enum {
 
 #define REFT_OFFSET_MASK 0xff00ULL
 
+static inline int64_t start_of_cluster(BDRVQcowState *s, int64_t offset)
+{
+return offset & ~(s->cluster_size - 1);
+}
+
 static inline int size_to_clusters(BDRVQcowState *s, int64_t size)
 {
 return (size + (s->cluster_size - 1)) >> s->cluster_bits;
-- 
1.7.6.5

[Qemu-devel] [PATCH 1/2] block: use Error in do_check_io_limits()

[Stefan Hajnoczi]

The do_check_io_limits() function returns false when I/O limits are
invalid but it doesn't set an Error to indicate why.  The two
do_check_io_limits() callers duplicate error reporting.  Solve this by
passing an Error pointer into do_check_io_limits().

Note that the two callers report slightly different errors: drive_init()
prints a custom error message while qmp_block_set_io_throttle() does
error_set(errp, QERR_INVALID_PARAMETER_COMBINATION).

QERR_INVALID_PARAMETER_COMBINATION is a generic error, see
include/qapi/qmp/qerror.h:

  #define QERR_INVALID_PARAMETER_COMBINATION \
ERROR_CLASS_GENERIC_ERROR, "Invalid parameter combination"

Since it is generic we are not obliged to keep this error.  Switch to
the custom error message which contains more information.

This patch prepares for adding additional checks with their own error
messages to do_check_io_limits().  The next patch adds a new check.

Signed-off-by: Stefan Hajnoczi 
---
 blockdev.c | 14 --
 1 file changed, 8 insertions(+), 6 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index 63e6f1e..9b03513 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -255,7 +255,7 @@ static int parse_block_error_action(const char *buf, bool 
is_read)
 }
 }
 
-static bool do_check_io_limits(BlockIOLimit *io_limits)
+static bool do_check_io_limits(BlockIOLimit *io_limits, Error **errp)
 {
 bool bps_flag;
 bool iops_flag;
@@ -269,6 +269,8 @@ static bool do_check_io_limits(BlockIOLimit *io_limits)
  && ((io_limits->iops[BLOCK_IO_LIMIT_READ] != 0)
  || (io_limits->iops[BLOCK_IO_LIMIT_WRITE] != 0));
 if (bps_flag || iops_flag) {
+error_setg(errp, "bps(iops) and bps_rd/bps_wr(iops_rd/iops_wr) "
+ "cannot be used at the same time");
 return false;
 }
 
@@ -297,6 +299,7 @@ DriveInfo *drive_init(QemuOpts *opts, BlockInterfaceType 
block_default_type)
 int snapshot = 0;
 bool copy_on_read;
 int ret;
+Error *error = NULL;
 
 translation = BIOS_ATA_TRANSLATION_AUTO;
 media = MEDIA_DISK;
@@ -427,9 +430,9 @@ DriveInfo *drive_init(QemuOpts *opts, BlockInterfaceType 
block_default_type)
 io_limits.iops[BLOCK_IO_LIMIT_WRITE] =
qemu_opt_get_number(opts, "iops_wr", 0);
 
-if (!do_check_io_limits(&io_limits)) {
-error_report("bps(iops) and bps_rd/bps_wr(iops_rd/iops_wr) "
- "cannot be used at the same time");
+if (!do_check_io_limits(&io_limits, &error)) {
+error_report("%s", error_get_pretty(error));
+error_free(error);
 return NULL;
 }
 
@@ -975,8 +978,7 @@ void qmp_block_set_io_throttle(const char *device, int64_t 
bps, int64_t bps_rd,
 io_limits.iops[BLOCK_IO_LIMIT_READ] = iops_rd;
 io_limits.iops[BLOCK_IO_LIMIT_WRITE]= iops_wr;
 
-if (!do_check_io_limits(&io_limits)) {
-error_set(errp, QERR_INVALID_PARAMETER_COMBINATION);
+if (!do_check_io_limits(&io_limits, errp)) {
 return;
 }
 
-- 
1.8.1.2

[Qemu-devel] [PATCH 2/2] block: refuse negative iops and bps values

[Stefan Hajnoczi]

Negative I/O throttling iops and bps values do not make sense so reject
them with an error message.

Signed-off-by: Stefan Hajnoczi 
---
 blockdev.c | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/blockdev.c b/blockdev.c
index 9b03513..ba3759c 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -274,6 +274,16 @@ static bool do_check_io_limits(BlockIOLimit *io_limits, 
Error **errp)
 return false;
 }
 
+if (io_limits->bps[BLOCK_IO_LIMIT_TOTAL] < 0 ||
+io_limits->bps[BLOCK_IO_LIMIT_WRITE] < 0 ||
+io_limits->bps[BLOCK_IO_LIMIT_READ] < 0 ||
+io_limits->iops[BLOCK_IO_LIMIT_TOTAL] < 0 ||
+io_limits->iops[BLOCK_IO_LIMIT_WRITE] < 0 ||
+io_limits->iops[BLOCK_IO_LIMIT_READ] < 0) {
+error_setg(errp, "bps and iops values must be 0 or greater");
+return false;
+}
+
 return true;
 }
 
-- 
1.8.1.2

[Qemu-devel] [PATCH 0/2] block: refuse negative iops and bps values

[Stefan Hajnoczi]

These patches report an error if negative values are given for I/O throttling
iops or bps.

Patch 1 gets do_check_io_limits() into shape so that we can add checks.

Patch 2 adds the negative check.

Stefan Hajnoczi (2):
  block: use Error in do_check_io_limits()
  block: refuse negative iops and bps values

 blockdev.c | 24 ++--
 1 file changed, 18 insertions(+), 6 deletions(-)

-- 
1.8.1.2

Re: [Qemu-devel] [RFC ppc-next 39/39] target-ppc: Convert CPU definitions

[Andreas Färber]

Am 12.02.2013 17:48, schrieb Andreas Färber:
> Am 12.02.2013 11:13, schrieb Andreas Färber:
>> Turn the array of model definitions into a set of self-registering QOM
>> types with their own class_init. Unique identifiers are obtained from
>> the combination of PVR, SVR and family identifiers; this requires all
>> alias #defines to be removed from the list. Possibly there are some more
>> left after this commit that are not currently being compiled.
>>
>> Prepares for introducing abstract intermediate CPU types for families.
>>
>> Keep the right-aligned macro line breaks within 78 chars to aid
>> three-way merges.
>>
>> Signed-off-by: Andreas Färber 
>> ---
>>  target-ppc/cpu-qom.h|   17 -
>>  target-ppc/cpu.h|   20 --
>>  target-ppc/translate_init.c |  152 
>> ---
>>  3 Dateien geändert, 85 Zeilen hinzugefügt(+), 104 Zeilen entfernt(-)
> [...]
>> diff --git a/target-ppc/translate_init.c b/target-ppc/translate_init.c
>> index fd8bf00..93e38ba 100644
>> --- a/target-ppc/translate_init.c
>> +++ b/target-ppc/translate_init.c
> [...]
>> @@ -10316,20 +10317,7 @@ static const TypeInfo ppc_cpu_type_info = {
>>  
>>  static void ppc_cpu_register_types(void)
>>  {
>> -int i;
>> -
>>  type_register_static(&ppc_cpu_type_info);
>> -
>> -for (i = 0; i < ARRAY_SIZE(ppc_defs); i++) {
>> -const ppc_def_t *def = &ppc_defs[i];
>> -#if defined(TARGET_PPCEMB)
>> -/* When using the ppcemb target, we only support 440 style cores */
>> -if (def->mmu_model != POWERPC_MMU_BOOKE) {
>> -continue;
>> -}
>> -#endif
>> -ppc_cpu_register_model(def);
>> -}
>>  }
>>  
>>  type_init(ppc_cpu_register_types)
> 
> Sorry, I forgot to re-add the TARGET_PPCEMB check above.

Here's the fix (sorry for linewraps):

@@ -7507,6 +7507,14 @@ enum {
 /* PowerPC CPU definitions
  */
 #define POWERPC_DEF_PREFIX(pvr, svr, type)
 \
 glue(glue(glue(glue(pvr, _), svr), _), type)
+#if defined(TARGET_PPCEMB)
+#define POWERPC_DEF_CONDITION(type)
 \
+if (glue(POWERPC_MMU_, type) != POWERPC_MMU_BOOKE) {
 \
+return;
 \
+}
+#else
+#define POWERPC_DEF_CONDITION(type)
+#endif
 #define POWERPC_DEF_SVR(_name, _pvr, _svr, _type)
   \
 static void
 \
 glue(POWERPC_DEF_PREFIX(_pvr, _svr, _type), _cpu_class_init)
 \
@@ -7539,6 +7547,7 @@ enum {
 static void
 \
 glue(POWERPC_DEF_PREFIX(_pvr, _svr, _type),
_cpu_register_types)(void)  \
 {
 \
+POWERPC_DEF_CONDITION(_type)
 \
 type_register_static(
 \
 &glue(POWERPC_DEF_PREFIX(_pvr, _svr, _type),
_cpu_type_info));  \
 }
 \

I'll repost when the family definitions are cleaned up.

Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [PATCH 0/2] block: refuse negative iops and bps values

[Kevin Wolf]

Am 13.02.2013 16:53, schrieb Stefan Hajnoczi:
> These patches report an error if negative values are given for I/O throttling
> iops or bps.
> 
> Patch 1 gets do_check_io_limits() into shape so that we can add checks.
> 
> Patch 2 adds the negative check.
> 
> Stefan Hajnoczi (2):
>   block: use Error in do_check_io_limits()
>   block: refuse negative iops and bps values
> 
>  blockdev.c | 24 ++--
>  1 file changed, 18 insertions(+), 6 deletions(-)

Reviewed-by: Kevin Wolf 

Re: [Qemu-devel] [PATCH for-1.4 0/2] chardev-add

[Laszlo Ersek]

On 02/13/13 15:54, Markus Armbruster wrote:
> Why am I proposing these patches for 1.4?
> 
> PATCH 1/2 is a straightforward doc fix.
> 
> PATCH 2/2 cleans up a part of QAPI that has never been released,
> before it's too late.  If we're already beyond the point where we can
> do that, too bad, we'll live.
> 
> Please consider seriously.
> 
> Markus Armbruster (2):
>   chardev: Fix manual page and qemu-doc for -chardev tty
>   qapi: Flatten away ChardevPort
> 
>  qapi-schema.json | 11 +++-
>  qemu-char.c  | 76 
> ++--
>  qemu-options.hx  |  2 +-
>  3 files changed, 46 insertions(+), 43 deletions(-)
> 

Looks good to me.

Reviewed-by: Laszlo Ersek 

Re: [Qemu-devel] [PATCH for-1.4 0/2] chardev-add

[Eric Blake]

On 02/13/2013 07:54 AM, Markus Armbruster wrote:
> Why am I proposing these patches for 1.4?
> 
> PATCH 1/2 is a straightforward doc fix.
> 
> PATCH 2/2 cleans up a part of QAPI that has never been released,
> before it's too late.  If we're already beyond the point where we can
> do that, too bad, we'll live.
> 
> Please consider seriously.

Seems reasonable to me to get this into 1.4.

Series: Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH] pseries: Add cleanup hook for PAPR virtual LAN device

[Anthony Liguori]

Andreas Färber  writes:

> Am 11.02.2013 05:59, schrieb David Gibson:
>> Currently the spapr-vlan device does not supply a cleanup call for its
>> NetClientInfo structure.  With current qemu versions, that leads to a SEGV
>> on exit, when net_cleanup() attempts to call the cleanup handlers on all
>> net clients.
>> 
>> Signed-off-by: David Gibson 
>
> Tested-by: Andreas Färber 
>
> On Anthony's request I posted a patch that adds back the if surrounding
> the cleanup callback, fixing also the other affected nics.

Since this cleanup hook doesn't do anything useful and we need to apply
Andreas' patch anyway, I don't think there's any benefit to apply this one.

Regards,

Anthony Liguori

>
> Regards,
> Andreas
>
> -- 
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
> GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] [Qemu-ppc] [PATCH] Fix circular dependency for HOST_LONG_BITS qemu-common.h <-> bswap.h

[Anthony Liguori]

David Gibson  writes:

> On Tue, Feb 05, 2013 at 11:42:30AM +0100, Andreas Färber wrote:
>> Am 05.02.2013 01:07, schrieb Peter Maydell:
>> > On 4 February 2013 23:52, Richard Henderson  wrote:
>> >> On 2013-02-04 15:30, David Gibson wrote:
>> >>> Anthony, Richard, anyone?
>> >>>
>> >>> Please apply - qemu has now been build-broken on all big endian
>> >>> platforms for a month.
>> >>
>> >>
>> >> I know.  See also my bswap.h patch which also fixes the width
>> >> of long vs uintptr_t.  No one seems willing to pick these up...
>> > 
>> > In both cases, the patch:
>> >  * was sent out after the soft freeze
>> >  * doesn't have a "for-1.4" tag
>> >  * doesn't have a summary line that clearly says "fixes build
>> >failure" either
>> >  * hasn't got a Reviewed-by: tag from anybody
>> 
>> I ack'ed it, which in my terminology usually means that I reviewed and
>> tested it.
>> 
>> > 
>> > so I'm not terribly surprised they haven't got picked up.
>> > You could start by reviewing each others' patches :-)
>> 
>> Personally I see no reason to keep around misnamed HOST_LONG_BITS at all
>> when we can easily calculate its value using sizeof(uintptr_t) * 8 or
>> replace it by different conditions as suggested by rth. I thus prefer
>
> Ok, I missed rth's patch to do this differently.  Note that sizeof()
> will not work in this case, because we need the correct value at cpp
> time.
>
>> his patch and have been waiting for Blue to pick it up for 1.4.
>
> Sure, whatever.  Can we please just get whichever damn fix *in*.

For the sake of completeness, "rth's patch" means:

commit 91107fdf4443d2171e06840e87277bb7a047343b
Author: Richard Henderson 
Date:   Mon Feb 4 16:21:06 2013 -0800

bswap: Fix width of swap in leul_to_cpu

Correct?

If so, this was committed before you sent this note.  Can someone
confirm if we still have a problem on big endian hosts?

Regards,

Anthony Liguori

>
> -- 
> David Gibson  | I'll have my music baroque, and my code
> david AT gibson.dropbear.id.au| minimalist, thank you.  NOT _the_ 
> _other_
>   | _way_ _around_!
> http://www.ozlabs.org/~dgibson

Re: [Qemu-devel] [Qemu-ppc] [PATCH] Fix circular dependency for HOST_LONG_BITS qemu-common.h <-> bswap.h

[Andreas Färber]

Am 13.02.2013 18:41, schrieb Anthony Liguori:
> David Gibson  writes:
> 
>> On Tue, Feb 05, 2013 at 11:42:30AM +0100, Andreas Färber wrote:
>>> Am 05.02.2013 01:07, schrieb Peter Maydell:
 On 4 February 2013 23:52, Richard Henderson  wrote:
> On 2013-02-04 15:30, David Gibson wrote:
>> Anthony, Richard, anyone?
>>
>> Please apply - qemu has now been build-broken on all big endian
>> platforms for a month.
>
>
> I know.  See also my bswap.h patch which also fixes the width
> of long vs uintptr_t.  No one seems willing to pick these up...

 In both cases, the patch:
  * was sent out after the soft freeze
  * doesn't have a "for-1.4" tag
  * doesn't have a summary line that clearly says "fixes build
failure" either
  * hasn't got a Reviewed-by: tag from anybody
>>>
>>> I ack'ed it, which in my terminology usually means that I reviewed and
>>> tested it.
>>>

 so I'm not terribly surprised they haven't got picked up.
 You could start by reviewing each others' patches :-)
>>>
>>> Personally I see no reason to keep around misnamed HOST_LONG_BITS at all
>>> when we can easily calculate its value using sizeof(uintptr_t) * 8 or
>>> replace it by different conditions as suggested by rth. I thus prefer
>>
>> Ok, I missed rth's patch to do this differently.  Note that sizeof()
>> will not work in this case, because we need the correct value at cpp
>> time.
>>
>>> his patch and have been waiting for Blue to pick it up for 1.4.
>>
>> Sure, whatever.  Can we please just get whichever damn fix *in*.
> 
> For the sake of completeness, "rth's patch" means:
> 
> commit 91107fdf4443d2171e06840e87277bb7a047343b
> Author: Richard Henderson 
> Date:   Mon Feb 4 16:21:06 2013 -0800
> 
> bswap: Fix width of swap in leul_to_cpu
> 
> Correct?

Yes.

> If so, this was committed before you sent this note.  Can someone
> confirm if we still have a problem on big endian hosts?

Around Central European lunch time today things seemed to compile fine,
and `make check` worked, too. (Thanks!)

Regards,
Andreas

-- 
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg, Germany
GF: Jeff Hawn, Jennifer Guild, Felix Imendörffer; HRB 16746 AG Nürnberg

Re: [Qemu-devel] using -net dump with tap networking

[Laszlo Ersek]

On 02/13/13 15:48, Alexey Kardashevskiy wrote:
> Hi!
>
> I am running qemu as:
>
> qemu/ppc64-softmmu/qemu-system-ppc64 -m 1024 -M pseries -trace
> events=trace_events -netdev user,id=virtnet,hostfwd=tcp::5000-:22
> -device virtio-net-pci,netdev=virtnet -nographic -vga none -enable-kvm
> -kernel vml36_64k -initrd 1.cpio
>
> Now I want to enable network dump. With the old "-net" syntax I could do
> that with "-net dump" but I cannot with the new syntax, tried many
> variants, none works. What would the correct syntax be for the case above?

Ugh, I'm a bit confused, but if I say something stupid that should still
help "ignite" the discussion.

So, in general there are two ways to specify this:

(1) -net dump,id=dump0,vlan=VLAN_ID,len=SIZE_LIMIT,file=PATHNAME

(2) -netdev dump,id=dump0,len=SIZE_LIMIT,file=PATHNAME

I believe the first option (legacy) should work.

The second one will not work; actually I think it will trigger an
assert. The generic init code in net_client_init1() [net/net.c] says:

NetClientState *peer = NULL;

/* Do not add to a vlan if it's a -netdev or a nic with a netdev=
 * parameter. */
if (!is_netdev &&
(opts->kind != NET_CLIENT_OPTIONS_KIND_NIC ||
 !opts->nic->has_netdev)) {
peer = net_hub_add_port(u.net->has_vlan ? u.net->vlan : 0, NULL);
}

if (net_client_init_fun[opts->kind](opts, name, peer) < 0) {

So in (2) we don't add the dump netdev to any hub/vlan; however the
specific code (net_init_dump(), [net/dump.c]) asserts (peer != NULL).

Otherwise I think the idea would be to add the dump netdev *afterwards*
to a vlan/hub, by changing its vlan property. See set_vlan() in
[hw/qdev-properties-system.c]; it calls net_hub_port_find() [net/hub.c]
whose task is to "Find a available port on a hub; otherwise create one
new port".

See
.

Hence I think you're back to (1), the legacy format. Assuming qemu
doesn't barf on that option immediately, I believe you *also* have to
add your "-netdev user" to the same hub as the dumper is on.

In total you have to create both netdevs (a, b) and assign both to a
common hub/vlan (c, d). Again, unfortunately the dump netdev only works
with the legacy format, but that already includes the assignment to the
hub (a, c). So you have to take care of creating the other netdev
(-netdev user, b), and assign it through its vlan qdev property to the
same hub (d), so that data can flow from it to the dump netdev.

Hm... Looks like you can't do that directly on "-netdev user" (it seems
to have no such property). "virtio-net-pci" does have it however. At
least in a quick "info qtree" check:

bus: main-system-bus
  type System
  dev: i440FX-pcihost, id ""
bus: pci.0
  type PCI
  dev: virtio-net-pci, id "net0"
dev-prop: vlan = 

Also confirmed by "qemu-system-x86_64 -device virtio-net-pci,help".

So

-netdev user,id=virtnet,hostfwd=tcp::5000-:22 \
-device virtio-net-pci,netdev=virtnet,vlan=2 \
-net dump,vlan=2,len=SIZE_LIMIT,file=PATHNAME

Or some such...

Laszlo

Re: [Qemu-devel] [PATCH for-1.4 0/2] chardev-add

[Anthony Liguori]

Eric Blake  writes:

> On 02/13/2013 07:54 AM, Markus Armbruster wrote:
>> Why am I proposing these patches for 1.4?
>> 
>> PATCH 1/2 is a straightforward doc fix.
>> 
>> PATCH 2/2 cleans up a part of QAPI that has never been released,
>> before it's too late.  If we're already beyond the point where we can
>> do that, too bad, we'll live.
>> 
>> Please consider seriously.
>
> Seems reasonable to me to get this into 1.4.
>
> Series: Reviewed-by: Eric Blake 

Agreed. Assuming it doesn't break anything, I'll include it in -rc2.

Regards,

Anthony Liguori

>
> -- 
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org

[Qemu-devel] [PATCH for-1.4 1/7] help: Drop bogus help on -qtest and -qtest-log

[Markus Armbruster]

Signed-off-by: Markus Armbruster 
---
 qemu-options.hx | 10 +++---
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/qemu-options.hx b/qemu-options.hx
index 046bdc0..3800c9c 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -2928,13 +2928,9 @@ the @var{simple} tracing backend.
 @end table
 ETEXI
 
-DEF("qtest", HAS_ARG, QEMU_OPTION_qtest,
-"-qtest CHR  specify tracing options\n",
-QEMU_ARCH_ALL)
-
-DEF("qtest-log", HAS_ARG, QEMU_OPTION_qtest_log,
-"-qtest-log LOG  specify tracing options\n",
-QEMU_ARCH_ALL)
+HXCOMM Internal use
+DEF("qtest", HAS_ARG, QEMU_OPTION_qtest, "", QEMU_ARCH_ALL)
+DEF("qtest-log", HAS_ARG, QEMU_OPTION_qtest_log, "", QEMU_ARCH_ALL)
 
 #ifdef __linux__
 DEF("enable-fips", 0, QEMU_OPTION_enablefips,
-- 
1.7.11.7

[Qemu-devel] [PATCH for-1.4 5/7] help: Fix markup of heading "USB options" so it appears in -help

[Markus Armbruster]

Signed-off-by: Markus Armbruster 
---
 qemu-options.hx | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/qemu-options.hx b/qemu-options.hx
index 8d288f5..071b1b3 100644
--- a/qemu-options.hx
+++ b/qemu-options.hx
@@ -538,9 +538,10 @@ ETEXI
 STEXI
 @end table
 ETEXI
+DEFHEADING()
 
+DEFHEADING(USB options:)
 STEXI
-USB options:
 @table @option
 ETEXI
 
-- 
1.7.11.7

[Qemu-devel] qemu / KVM modification to yield to external component (how to make it ignore the time the external component is running)

[Lloret, Luis]

Hi, qemu experts,

We are using qemu with KVM for some experiments that involve qemu yielding to 
an external process periodically. By yielding, what I mean is that qemu will 
stop executing for some time while the external process does its job and 
reports back to qemu. The external process is a remote HW model. The handshake 
happens with normal TCP sockets, and the platform that this is running on has 
the constraint that qemu and the external model cannot execute concurrently. So 
they need to work in a kind of ping-pong mode. That is why we need this 
synchronization mechanism. 

The yield is done inside a normal qemu timer callback. And this is the relevant 
code for what I am talking about.
// Install the timer
sim_sync_timer = qemu_new_timer_ns (vm_clock, sim_sync_callback, NULL);

-

// This is the callback definition. It just calls the sync function and 
reprograms the periodic timer
static void sim_sync_callback (void *p){
  qemutbx_send_sync (qemu_get_clock_ns (vm_clock));
  qemu_mod_timer (sim_sync_timer, qemu_get_clock_ns (vm_clock) + 
syncTimerExpiredTime );
}

--

// This is the function where we yield to the external program 
int qemu_send_sync (unsigned long long vmtime)
{
struct qemu_message msg;
memset(&msg,0,sizeof(struct qemutbx_message));
//vm_stop_sync();
msg.mtype   = QEMU_MESSAGE_SYNC;
msg.mAck= QEMU_MESSAGE_SYNC_DONE;
msg.vm_time = vmtime;

// send the sync message to the external program. We will wait here until the 
external program replies back
qemu_send_msg_and_wait_ack(&msg); // This is just blocking TCP send and 
receive
//vm_start();
return 0;
}


What we want is that this time where the external process is executing and qemu 
is waiting, was completely transparent to the software, drivers, etc, running 
inside qemu. And we do not care if clocks and time inside qemu deviate from 
real wall clock time. What we want is that this external execution time is not 
seen by qemu and the guest in general. So, for example if qemu runs for 2 
seconds of virtual time, then yields to the external process that takes 3 
seconds to process, what we expect is that when qemu resumes it does at virtual 
time 2, without trying to synchronize at all with the real time on the host. 
Like if the 3 seconds of real time hadn't existed.

So far, I have been trying with linux clocksources (to change from the default 
kvm-clock), vm_stop and vm_start (so that the yield to external happens inside 
a vm_stop / vm_start block), but nothing seems to work as I expect.

For example, some driver code seems to time out, because it is probably seeing 
the time that this external process is running (and qemu isn't) as time that 
has elapsed from its point of view. 

Perhaps this would be easier without KVM (I don't know), but KVM is a must 
too... 

Can someone give some pointers as to what we should do to achieve this? Any 
idea would be greatly appreciated.

Thanks in advance,
Luis

[Qemu-devel] [Bug 1123975] [NEW] QEmu 1.3.90 cannot restore a 1.1.2 live snapshot

[Francois Gouget]

Public bug reported:

I have upgraded to QEmu 1.3.90 (Debian 1.4.0~rc0+dfsg-1exp) but now when
I try to restore a live snapshot made in QEmu 1.1.2 (Debian
1.1.2+dfsg-5) I get the following message:

virsh # snapshot-revert fgtbbuild wtb
error: operation failed: Error -22 while loading VM state

I have test VMs with live snapshots coreresponding to different testing
configurations. So I typically revert the VMs in one of the live
snapshots and run the tests. It would be pretty annoying to have to
recreate all these live snapshots any time I upgrade QEmu.


ipxe-qemu  1.0.0+git-20120202.f6840ba-3
qemu   1.4.0~rc0+dfsg-1exp
qemu-keymaps   1.4.0~rc0+dfsg-1exp
qemu-kvm   1.4.0~rc0+dfsg-1exp
qemu-system1.4.0~rc0+dfsg-1exp
qemu-system-arm1.4.0~rc0+dfsg-1exp
qemu-system-common 1.4.0~rc0+dfsg-1exp
qemu-system-mips   1.4.0~rc0+dfsg-1exp
qemu-system-misc   1.4.0~rc0+dfsg-1exp
qemu-system-ppc1.4.0~rc0+dfsg-1exp
qemu-system-sparc  1.4.0~rc0+dfsg-1exp
qemu-system-x861.4.0~rc0+dfsg-1exp
qemu-user  1.4.0~rc0+dfsg-1exp
qemu-utils 1.4.0~rc0+dfsg-1exp
libvirt-bin1.0.2-1
libvirt-dev1.0.2-1
libvirt-doc1.0.2-1
libvirt-glib-1.0-0 0.1.2-1
libvirt0   1.0.2-1
libvirtodbc0   6.1.4+dfsg1-5

** Affects: qemu
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1123975

Title:
  QEmu 1.3.90 cannot restore a 1.1.2 live snapshot

Status in QEMU:
  New

Bug description:
  I have upgraded to QEmu 1.3.90 (Debian 1.4.0~rc0+dfsg-1exp) but now
  when I try to restore a live snapshot made in QEmu 1.1.2 (Debian
  1.1.2+dfsg-5) I get the following message:

  virsh # snapshot-revert fgtbbuild wtb
  error: operation failed: Error -22 while loading VM state

  I have test VMs with live snapshots coreresponding to different
  testing configurations. So I typically revert the VMs in one of the
  live snapshots and run the tests. It would be pretty annoying to have
  to recreate all these live snapshots any time I upgrade QEmu.

  
  ipxe-qemu  1.0.0+git-20120202.f6840ba-3
  qemu   1.4.0~rc0+dfsg-1exp
  qemu-keymaps   1.4.0~rc0+dfsg-1exp
  qemu-kvm   1.4.0~rc0+dfsg-1exp
  qemu-system1.4.0~rc0+dfsg-1exp
  qemu-system-arm1.4.0~rc0+dfsg-1exp
  qemu-system-common 1.4.0~rc0+dfsg-1exp
  qemu-system-mips   1.4.0~rc0+dfsg-1exp
  qemu-system-misc   1.4.0~rc0+dfsg-1exp
  qemu-system-ppc1.4.0~rc0+dfsg-1exp
  qemu-system-sparc  1.4.0~rc0+dfsg-1exp
  qemu-system-x861.4.0~rc0+dfsg-1exp
  qemu-user  1.4.0~rc0+dfsg-1exp
  qemu-utils 1.4.0~rc0+dfsg-1exp
  libvirt-bin1.0.2-1
  libvirt-dev1.0.2-1
  libvirt-doc1.0.2-1
  libvirt-glib-1.0-0 0.1.2-1
  libvirt0   1.0.2-1
  libvirtodbc0   6.1.4+dfsg1-5

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1123975/+subscriptions