[Qemu-devel] [PATCH] Include setjmp.h in qemu/osdep.h (bug fix for w64)

[Stefan Weil]

setjmp must be declared before sysemu/os-win32.h
because it is redefined there for 64 bit Windows.

Signed-off-by: Stefan Weil 
---

This patch superseeds my previous patch
"Use special code for sigsetjmp only in cpu-exec.c".

It is needed for QEMU on 64 bit Windows to get the
correct definition of setjmp (without stack unwinding).

Regards,
Stefan


 include/qemu/osdep.h | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index 4538fdc..5bb374c 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -76,6 +76,9 @@ extern int daemon(int, int);
 #include 
 #include 
 #include 
+/* setjmp must be declared before sysemu/os-win32.h
+ * because it is redefined there. */
+#include 
 #include 
 
 #ifdef __OpenBSD__
-- 
2.1.4

Re: [Qemu-devel] [PATCH v4 4/4] hw/input/adb.c: implement QKeyCode support

[Programmingkid]

On Mar 11, 2016, at 10:30 PM, Peter Maydell wrote:

> 
>> +}
>> +keycode = s->data[s->rptr];
>> +if (++s->rptr == sizeof(s->data)) {
>> +s->rptr = 0;
>> }
>> +s->count--;
>> +
>> +obuf[0] = keycode;
> 
> You are still trying to put a two byte keycode (ADB_KEY_POWER)
> into this one-byte array slot. I don't know what the right way to
> send a two-byte keycode is but this is obviously not it, as
> I said before.
> 
>> +/* NOTE: could put a second keycode if needed */
>> +obuf[1] = 0xff;
>> +olen = 2;
>> +
>> return olen;
>> }

Is this ok?

/* The power key is the only two byte value key, so it is a special case. */
if (keycode == (ADB_KEY_POWER & 0x00ff)) {
obuf[0] = ADB_KEY_POWER & 0x00ff;
obuf[1] = ADB_KEY_POWER & 0xff00 >> 8;
olen = 2;
} else {
obuf[0] = keycode;
/* NOTE: could put a second keycode if needed */
obuf[1] = 0xff;
olen = 2;
}

The keycode value comes from an 8 bit array so holding the full value of the 
power key is not possible. That is the reason for the "if (keycode == 
(ADB_KEY_POWER & 0x00ff))". 

The code might be a little more efficient if we did this:

/* The power key is the only two byte value key, so it is a special case. */
if (keycode == 0x7f) {
obuf[0] = 0x7f;
obuf[1] = 0x7f;
olen = 2;
} else {
obuf[0] = keycode;
/* NOTE: could put a second keycode if needed */
obuf[1] = 0xff;
olen = 2;
}

The speed difference isn't noticeable so either way works well.

Re: [Qemu-devel] [PATCH v4 4/4] hw/input/adb.c: implement QKeyCode support

[Programmingkid]

On Mar 11, 2016, at 10:30 PM, Peter Maydell wrote:

> On 11 March 2016 at 09:32, Programmingkid  wrote:
>> Remove the old pc_to_adb_keycode array and replace it with QKeyCode support.
>> 
>> Signed-off-by: John Arbuckle 
>> ---
>> Some of the keys do not translate as logically as we would think they would. 
>> For
>> example the Q_KEY_CODE_CTRL_R does not work with ADB_KEY_RIGHT_CONTROL. The
>> wrong key would show up in the guest. These problem keys are commmented out 
>> and
>> replaced with the number that does work correctly. This patch can be easily
>> tested with the Linux command xev or Mac OS's Key Caps.
> 
> I'm not sure what you mean here. If you press right-control on the host
> then shouldn't this correspond to right-control on the guest ?

It should. It makes logical sense. But when I tried it using a Mac OS X and 
Linux guest, the wrong key would be pressed. The theories I have are incorrect 
keyboard detection to CUDA translation problems. 


>> /* debug ADB */
>> //#define DEBUG_ADB
>> @@ -59,6 +62,9 @@ do { printf("ADB: " fmt , ## __VA_ARGS__); } while (0)
>> /* error codes */
>> #define ADB_RET_NOTPRESENT (-2)
>> 
>> +/* The adb keyboard doesn't have every key imaginable */
>> +#define NO_KEY 0xff
>> +
>> static void adb_device_reset(ADBDevice *d)
>> {
>> qdev_reset_all(DEVICE(d));
>> @@ -187,23 +193,138 @@ typedef struct ADBKeyboardClass {
>> DeviceRealize parent_realize;
>> } ADBKeyboardClass;
>> 
>> -static const uint8_t pc_to_adb_keycode[256] = {
>> -  0, 53, 18, 19, 20, 21, 23, 22, 26, 28, 25, 29, 27, 24, 51, 48,
>> - 12, 13, 14, 15, 17, 16, 32, 34, 31, 35, 33, 30, 36, 54,  0,  1,
>> -  2,  3,  5,  4, 38, 40, 37, 41, 39, 50, 56, 42,  6,  7,  8,  9,
>> - 11, 45, 46, 43, 47, 44,123, 67, 58, 49, 57,122,120, 99,118, 96,
>> - 97, 98,100,101,109, 71,107, 89, 91, 92, 78, 86, 87, 88, 69, 83,
>> - 84, 85, 82, 65,  0,  0, 10,103,111,  0,  0,110, 81,  0,  0,  0,
>> -  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
>> -  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
>> -  0,  0,  0, 94,  0, 93,  0,  0,  0,  0,  0,  0,104,102,  0,  0,
>> -  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0, 76,125,  0,  0,
>> -  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,105,  0,  0,  0,  0,  0,
>> -  0,  0,  0,  0,  0, 75,  0,  0,124,  0,  0,  0,  0,  0,  0,  0,
>> -  0,  0,  0,  0,  0,  0,  0,115, 62,116,  0, 59,  0, 60,  0,119,
>> - 61,121,114,117,  0,  0,  0,  0,  0,  0,  0, 55,126,  0,127,  0,
>> -  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
>> -  0,  0,  0,  0,  0, 95,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
>> +int qcode_to_adb_keycode[] = {
>> +[Q_KEY_CODE_SHIFT] = ADB_KEY_LEFT_SHIFT,
>> +[Q_KEY_CODE_SHIFT_R]   = 123, /* ADB_KEY_RIGHT_SHIFT, */
> 
> These should definitely be using some ADB_KEY_* constant on
> the RHS, not a decimal constant.

Ok. It would look something like this:
[Q_KEY_CODE_SHIFT_R]   = ADB_KEY_LEFT,

It looks wrong, but it works.

> 
>> +[Q_KEY_CODE_ALT]   = ADB_KEY_LEFT_OPTION,
>> +[Q_KEY_CODE_ALT_R] = 124, /* ADB_KEY_RIGHT_OPTION,*/
>> +[Q_KEY_CODE_ALTGR] = ADB_KEY_RIGHT_OPTION,
>> +[Q_KEY_CODE_CTRL]  = 54, /* ADB_KEY_LEFT_CONTROL, */
>> +[Q_KEY_CODE_CTRL_R]= 125, /* ADB_KEY_RIGHT_CONTROL, */
>> +[Q_KEY_CODE_META_L]= ADB_KEY_LEFT_COMMAND,
>> +
>> + /* 126 works as right super in Linux */
>> + /* Use ADB_KEY_LEFT_COMMAND for Mac OS compatibility */
>> +[Q_KEY_CODE_META_R]= ADB_KEY_LEFT_COMMAND,
>> +[Q_KEY_CODE_SPC]   = ADB_KEY_SPACEBAR,
>> +
>> +[Q_KEY_CODE_ESC]   = ADB_KEY_ESC,
>> +[Q_KEY_CODE_1] = ADB_KEY_1,
>> +[Q_KEY_CODE_2] = ADB_KEY_2,
>> +[Q_KEY_CODE_3] = ADB_KEY_3,
>> +[Q_KEY_CODE_4] = ADB_KEY_4,
>> +[Q_KEY_CODE_5] = ADB_KEY_5,
>> +[Q_KEY_CODE_6] = ADB_KEY_6,
>> +[Q_KEY_CODE_7] = ADB_KEY_7,
>> +[Q_KEY_CODE_8] = ADB_KEY_8,
>> +[Q_KEY_CODE_9] = ADB_KEY_9,
>> +[Q_KEY_CODE_0] = ADB_KEY_0,
>> +[Q_KEY_CODE_MINUS] = ADB_KEY_MINUS,
>> +[Q_KEY_CODE_EQUAL] = ADB_KEY_EQUAL,
>> +[Q_KEY_CODE_BACKSPACE] = ADB_KEY_DELETE,
>> +[Q_KEY_CODE_TAB]   = ADB_KEY_TAB,
>> +[Q_KEY_CODE_Q] = ADB_KEY_Q,
>> +[Q_KEY_CODE_W] = ADB_KEY_W,
>> +[Q_KEY_CODE_E] = ADB_KEY_E,
>> +[Q_KEY_CODE_R] = ADB_KEY_R,
>> +[Q_KEY_CODE_T] = ADB_KEY_T,
>> +[Q_KEY_CODE_Y] = ADB_KEY_Y,
>> +[Q_KEY_CODE_U] = ADB_KEY_U,
>> +[Q_KEY_CODE_I] = ADB_KEY_I,
>> +[Q_KEY_CODE_O] = ADB_KEY_O,
>> +[Q_KEY_CODE_P] = ADB_KEY_P,
>> +[Q_KEY_CODE_BRACKET_LEFT]  = ADB_KEY_LEFT_BRACKET,
>> +[Q_KEY_CODE_BRACKET_RIGHT] = ADB_KEY_RIGHT_BRACKET,

[Qemu-devel] [Patch 1/1] net/socket: Allocating Large sized arrays to heap

[Pooja Dhannawat]

Signed-off-by: Pooja Dhannawat 
---
 net/socket.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/net/socket.c b/net/socket.c
index e32e3cb..483dcac 100644
--- a/net/socket.c
+++ b/net/socket.c
@@ -147,10 +147,10 @@ static void net_socket_send(void *opaque)
 NetSocketState *s = opaque;
 int size, err;
 unsigned l;
-uint8_t buf1[NET_BUFSIZE];
+uint8_t *buf1 = g_new(uint8_t, 1);
 const uint8_t *buf;
 
-size = qemu_recv(s->fd, buf1, sizeof(buf1), 0);
+size = qemu_recv(s->fd, (uint8_t *)buf1, sizeof(uint8_t), 0);
 if (size < 0) {
 err = socket_error();
 if (err != EWOULDBLOCK)
@@ -170,7 +170,6 @@ static void net_socket_send(void *opaque)
 s->index = 0;
 s->packet_len = 0;
 s->nc.link_down = true;
-memset(s->buf, 0, sizeof(s->buf));
 memset(s->nc.info_str, 0, sizeof(s->nc.info_str));
 
 return;
@@ -222,6 +221,7 @@ static void net_socket_send(void *opaque)
 break;
 }
 }
+g_free(buf1);
 }
 
 static void net_socket_send_dgram(void *opaque)
-- 
2.5.0

Re: [Qemu-devel] [PATCH v4 4/4] hw/input/adb.c: implement QKeyCode support

[Peter Maydell]

On 11 March 2016 at 09:32, Programmingkid  wrote:
> Remove the old pc_to_adb_keycode array and replace it with QKeyCode support.
>
> Signed-off-by: John Arbuckle 
> ---
> Some of the keys do not translate as logically as we would think they would. 
> For
> example the Q_KEY_CODE_CTRL_R does not work with ADB_KEY_RIGHT_CONTROL. The
> wrong key would show up in the guest. These problem keys are commmented out 
> and
> replaced with the number that does work correctly. This patch can be easily
> tested with the Linux command xev or Mac OS's Key Caps.

I'm not sure what you mean here. If you press right-control on the host
then shouldn't this correspond to right-control on the guest ?

>  /* debug ADB */
>  //#define DEBUG_ADB
> @@ -59,6 +62,9 @@ do { printf("ADB: " fmt , ## __VA_ARGS__); } while (0)
>  /* error codes */
>  #define ADB_RET_NOTPRESENT (-2)
>
> +/* The adb keyboard doesn't have every key imaginable */
> +#define NO_KEY 0xff
> +
>  static void adb_device_reset(ADBDevice *d)
>  {
>  qdev_reset_all(DEVICE(d));
> @@ -187,23 +193,138 @@ typedef struct ADBKeyboardClass {
>  DeviceRealize parent_realize;
>  } ADBKeyboardClass;
>
> -static const uint8_t pc_to_adb_keycode[256] = {
> -  0, 53, 18, 19, 20, 21, 23, 22, 26, 28, 25, 29, 27, 24, 51, 48,
> - 12, 13, 14, 15, 17, 16, 32, 34, 31, 35, 33, 30, 36, 54,  0,  1,
> -  2,  3,  5,  4, 38, 40, 37, 41, 39, 50, 56, 42,  6,  7,  8,  9,
> - 11, 45, 46, 43, 47, 44,123, 67, 58, 49, 57,122,120, 99,118, 96,
> - 97, 98,100,101,109, 71,107, 89, 91, 92, 78, 86, 87, 88, 69, 83,
> - 84, 85, 82, 65,  0,  0, 10,103,111,  0,  0,110, 81,  0,  0,  0,
> -  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
> -  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
> -  0,  0,  0, 94,  0, 93,  0,  0,  0,  0,  0,  0,104,102,  0,  0,
> -  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0, 76,125,  0,  0,
> -  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,105,  0,  0,  0,  0,  0,
> -  0,  0,  0,  0,  0, 75,  0,  0,124,  0,  0,  0,  0,  0,  0,  0,
> -  0,  0,  0,  0,  0,  0,  0,115, 62,116,  0, 59,  0, 60,  0,119,
> - 61,121,114,117,  0,  0,  0,  0,  0,  0,  0, 55,126,  0,127,  0,
> -  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
> -  0,  0,  0,  0,  0, 95,  0,  0,  0,  0,  0,  0,  0,  0,  0,  0,
> +int qcode_to_adb_keycode[] = {
> +[Q_KEY_CODE_SHIFT] = ADB_KEY_LEFT_SHIFT,
> +[Q_KEY_CODE_SHIFT_R]   = 123, /* ADB_KEY_RIGHT_SHIFT, */

These should definitely be using some ADB_KEY_* constant on
the RHS, not a decimal constant.

> +[Q_KEY_CODE_ALT]   = ADB_KEY_LEFT_OPTION,
> +[Q_KEY_CODE_ALT_R] = 124, /* ADB_KEY_RIGHT_OPTION,*/
> +[Q_KEY_CODE_ALTGR] = ADB_KEY_RIGHT_OPTION,
> +[Q_KEY_CODE_CTRL]  = 54, /* ADB_KEY_LEFT_CONTROL, */
> +[Q_KEY_CODE_CTRL_R]= 125, /* ADB_KEY_RIGHT_CONTROL, */
> +[Q_KEY_CODE_META_L]= ADB_KEY_LEFT_COMMAND,
> +
> + /* 126 works as right super in Linux */
> + /* Use ADB_KEY_LEFT_COMMAND for Mac OS compatibility */
> +[Q_KEY_CODE_META_R]= ADB_KEY_LEFT_COMMAND,
> +[Q_KEY_CODE_SPC]   = ADB_KEY_SPACEBAR,
> +
> +[Q_KEY_CODE_ESC]   = ADB_KEY_ESC,
> +[Q_KEY_CODE_1] = ADB_KEY_1,
> +[Q_KEY_CODE_2] = ADB_KEY_2,
> +[Q_KEY_CODE_3] = ADB_KEY_3,
> +[Q_KEY_CODE_4] = ADB_KEY_4,
> +[Q_KEY_CODE_5] = ADB_KEY_5,
> +[Q_KEY_CODE_6] = ADB_KEY_6,
> +[Q_KEY_CODE_7] = ADB_KEY_7,
> +[Q_KEY_CODE_8] = ADB_KEY_8,
> +[Q_KEY_CODE_9] = ADB_KEY_9,
> +[Q_KEY_CODE_0] = ADB_KEY_0,
> +[Q_KEY_CODE_MINUS] = ADB_KEY_MINUS,
> +[Q_KEY_CODE_EQUAL] = ADB_KEY_EQUAL,
> +[Q_KEY_CODE_BACKSPACE] = ADB_KEY_DELETE,
> +[Q_KEY_CODE_TAB]   = ADB_KEY_TAB,
> +[Q_KEY_CODE_Q] = ADB_KEY_Q,
> +[Q_KEY_CODE_W] = ADB_KEY_W,
> +[Q_KEY_CODE_E] = ADB_KEY_E,
> +[Q_KEY_CODE_R] = ADB_KEY_R,
> +[Q_KEY_CODE_T] = ADB_KEY_T,
> +[Q_KEY_CODE_Y] = ADB_KEY_Y,
> +[Q_KEY_CODE_U] = ADB_KEY_U,
> +[Q_KEY_CODE_I] = ADB_KEY_I,
> +[Q_KEY_CODE_O] = ADB_KEY_O,
> +[Q_KEY_CODE_P] = ADB_KEY_P,
> +[Q_KEY_CODE_BRACKET_LEFT]  = ADB_KEY_LEFT_BRACKET,
> +[Q_KEY_CODE_BRACKET_RIGHT] = ADB_KEY_RIGHT_BRACKET,
> +[Q_KEY_CODE_RET]   = ADB_KEY_RETURN,
> +[Q_KEY_CODE_A] = ADB_KEY_A,
> +[Q_KEY_CODE_S] = ADB_KEY_S,
> +[Q_KEY_CODE_D] = ADB_KEY_D,
> +[Q_KEY_CODE_F] = ADB_KEY_F,
> +[Q_KEY_CODE_G] = ADB_KEY_G,
> +[Q_KEY_CODE_H] = ADB_KEY_H,
> +[Q_KEY_CODE_J] = ADB_KEY_J,
> +[Q_KEY_CODE_K] = ADB_KEY_K,
> +[Q_KEY_CODE_L] = 

Re: [Qemu-devel] [PATCH v3 2/3] hw/intc: Add (new) ASPEED VIC device model

[Andrew Jeffery]

On Fri, 2016-03-11 at 16:03 +0700, Peter Maydell wrote:
> On 5 March 2016 at 11:29, Andrew Jeffery  wrote:
> > Implement a basic ASPEED VIC device model, enough to boot a Linux kernel
> > configured with aspeed_defconfig. The model implements the 'new'
> > (revised) register set and while the hardware exposes both the new and
> > legacy register sets, accesses to the legacy register set will not
> > be serviced (though the access will be logged).
> > 
> > Signed-off-by: Andrew Jeffery 
> 
> > +static void aspeed_vic_write(void *opaque, hwaddr offset, uint64_t data,
> > + unsigned size)
> > +{
> > +const bool high = !!(offset & 0x4);
> > +hwaddr n_offset = (offset & ~0x4);
> > +AspeedVICState *s = (AspeedVICState *)opaque;
> > +
> > +if (offset < AVIC_NEW_BASE_OFFSET) {
> > +qemu_log_mask(LOG_UNIMP,
> > +  "%s: Ignoring write to legacy registers at 0x%"
> > +  HWADDR_PRIx "[%u] <- 0x%" PRIx64 "\n", __func__, 
> > offset,
> > +  size, data);
> > +return;
> > +}
> > +
> > +n_offset -= AVIC_NEW_BASE_OFFSET;
> > +trace_aspeed_vic_write(offset, size, data);
> > +
> > +/* Given we have members using separate enable/clear registers, 
> > deposit64()
> > + * isn't quite the tool for the job. Instead, relocate the incoming 
> > bits to
> > + * the required bit offset based on the provided access address
> > + */
> > +if (high) {
> > +data &= AVIC_H_MASK;
> > +data <<= 32;
> > +} else {
> > +data &= AVIC_L_MASK;
> > +}
> > +
> > +switch (n_offset) {
> > +case 0x18: /* Interrupt Selection */
> > +/* Register has deposit64() semantics - overwrite requested 32 
> > bits */
> > +if (high) {
> > +s->select &= AVIC_L_MASK;
> > +} else {
> > +s->select &= ((uint64_t) AVIC_H_MASK) << 32;
> > +}
> > +s->select |= data;
> > +break;
> > +case 0x20: /* Interrupt Enable */
> > +s->enable |= data;
> > +break;
> > +case 0x28: /* Interrupt Enable Clear */
> > +s->enable &= ~data;
> > +break;
> > +case 0x30: /* Software Interrupt */
> > +qemu_log_mask(LOG_UNIMP, "%s: Software interrupts unavailable. "
> > +"IRQs requested: 0x%016" PRIx64 "\n", __func__, data);
> > +break;
> > +case 0x38: /* Software Interrupt Clear */
> > +qemu_log_mask(LOG_UNIMP, "%s: Software interrupts unavailable. "
> > +"IRQs to be cleared: 0x%016" PRIx64 "\n", __func__, data);
> > +break;
> > +case 0x50: /* Interrupt Event */
> > +/* Register has deposit64() semantics - overwrite the top four 
> > valid
> > + * IRQ bits, as only the top four IRQs (GPIOs) can change their 
> > event
> > + * type */
> > +g_assert(high);
> 
> Don't assert on conditions that can be triggered by a guest.

Good point, I'll change it to qemu_log_mask(LOG_GUEST_ERROR, ...)

> 
> > +s->event &= ~AVIC_EVENT_W_MASK;
> > +s->event |= (data & AVIC_EVENT_W_MASK);
> > +break;
> > +case 0x58: /* Edge Triggered Interrupt Clear */
> > +s->raw &= ~(data & ~s->sense);
> > +break;
> > +case 0x00: /* IRQ Status */
> > +case 0x08: /* FIQ Status */
> > +case 0x10: /* Raw Interrupt Status */
> > +case 0x40: /* Interrupt Sensitivity */
> > +case 0x48: /* Interrupt Both Edge Trigger Control */
> > +case 0x60: /* Edge Triggered Interrupt Status */
> > +qemu_log_mask(LOG_GUEST_ERROR,
> > +  "%s: Write of read-only register with offset 0x%"
> > +  HWADDR_PRIx "\n", __func__, offset);
> > +break;
> > +
> > +default:
> > +qemu_log_mask(LOG_GUEST_ERROR,
> > +  "%s: Bad register at offset 0x%" HWADDR_PRIx "\n",
> > +  __func__, offset);
> > +break;
> > +}
> > +aspeed_vic_update(s);
> > +}
> 
> Otherwise
> Reviewed-by: Peter Maydell 

Thanks,

Andrew
> 
> thanks
> -- PMM

signature.asc
Description: This is a digitally signed message part

[Qemu-devel] [PATCH] target-xtensa: use global registers for the register window

[Max Filippov]

Signed-off-by: Max Filippov 
---
 target-xtensa/cpu.c   |  1 +
 target-xtensa/cpu.h   |  5 +++--
 target-xtensa/op_helper.c | 48 ---
 target-xtensa/translate.c |  7 +--
 4 files changed, 21 insertions(+), 40 deletions(-)

diff --git a/target-xtensa/cpu.c b/target-xtensa/cpu.c
index d572d56..2b9575f 100644
--- a/target-xtensa/cpu.c
+++ b/target-xtensa/cpu.c
@@ -69,6 +69,7 @@ static void xtensa_cpu_reset(CPUState *s)
 XTENSA_OPTION_ATOMCTL) ? 0x28 : 0x15;
 env->sregs[CONFIGID0] = env->config->configid[0];
 env->sregs[CONFIGID1] = env->config->configid[1];
+rotate_window_abs(env, env->sregs[WINDOW_BASE]);
 
 env->pending_irq_level = 0;
 reset_mmu(env);
diff --git a/target-xtensa/cpu.h b/target-xtensa/cpu.h
index d0bd9da..5bace52 100644
--- a/target-xtensa/cpu.h
+++ b/target-xtensa/cpu.h
@@ -350,11 +350,11 @@ enum {
 
 typedef struct CPUXtensaState {
 const XtensaConfig *config;
-uint32_t regs[16];
+uint32_t *regs;
 uint32_t pc;
 uint32_t sregs[256];
 uint32_t uregs[256];
-uint32_t phys_regs[MAX_NAREG];
+uint32_t phys_regs[MAX_NAREG + 12];
 union {
 float32 f32[2];
 float64 f64;
@@ -408,6 +408,7 @@ void xtensa_timer_irq(CPUXtensaState *env, uint32_t id, 
uint32_t active);
 void xtensa_rearm_ccompare_timer(CPUXtensaState *env);
 int cpu_xtensa_signal_handler(int host_signum, void *pinfo, void *puc);
 void xtensa_cpu_list(FILE *f, fprintf_function cpu_fprintf);
+void rotate_window_abs(CPUXtensaState *env, uint32_t position);
 void xtensa_sync_window_from_phys(CPUXtensaState *env);
 void xtensa_sync_phys_from_window(CPUXtensaState *env);
 uint32_t xtensa_tlb_get_addr_mask(const CPUXtensaState *env, bool dtlb, 
uint32_t way);
diff --git a/target-xtensa/op_helper.c b/target-xtensa/op_helper.c
index 62fa33d..7f69f4b 100644
--- a/target-xtensa/op_helper.c
+++ b/target-xtensa/op_helper.c
@@ -172,39 +172,6 @@ uint32_t HELPER(nsau)(uint32_t v)
 return v ? clz32(v) : 32;
 }
 
-static void copy_window_from_phys(CPUXtensaState *env,
-uint32_t window, uint32_t phys, uint32_t n)
-{
-assert(phys < env->config->nareg);
-if (phys + n <= env->config->nareg) {
-memcpy(env->regs + window, env->phys_regs + phys,
-n * sizeof(uint32_t));
-} else {
-uint32_t n1 = env->config->nareg - phys;
-memcpy(env->regs + window, env->phys_regs + phys,
-n1 * sizeof(uint32_t));
-memcpy(env->regs + window + n1, env->phys_regs,
-(n - n1) * sizeof(uint32_t));
-}
-}
-
-static void copy_phys_from_window(CPUXtensaState *env,
-uint32_t phys, uint32_t window, uint32_t n)
-{
-assert(phys < env->config->nareg);
-if (phys + n <= env->config->nareg) {
-memcpy(env->phys_regs + phys, env->regs + window,
-n * sizeof(uint32_t));
-} else {
-uint32_t n1 = env->config->nareg - phys;
-memcpy(env->phys_regs + phys, env->regs + window,
-n1 * sizeof(uint32_t));
-memcpy(env->phys_regs, env->regs + window + n1,
-(n - n1) * sizeof(uint32_t));
-}
-}
-
-
 static inline unsigned windowbase_bound(unsigned a, const CPUXtensaState *env)
 {
 return a & (env->config->nareg / 4 - 1);
@@ -217,18 +184,27 @@ static inline unsigned windowstart_bit(unsigned a, const 
CPUXtensaState *env)
 
 void xtensa_sync_window_from_phys(CPUXtensaState *env)
 {
-copy_window_from_phys(env, 0, env->sregs[WINDOW_BASE] * 4, 16);
+if (env->sregs[WINDOW_BASE] * 4 + 16 > env->config->nareg) {
+memcpy(env->phys_regs + env->config->nareg, env->phys_regs,
+(env->sregs[WINDOW_BASE] * 4 + 16 - env->config->nareg) *
+sizeof(uint32_t));
+}
 }
 
 void xtensa_sync_phys_from_window(CPUXtensaState *env)
 {
-copy_phys_from_window(env, env->sregs[WINDOW_BASE] * 4, 0, 16);
+if (env->sregs[WINDOW_BASE] * 4 + 16 > env->config->nareg) {
+memcpy(env->phys_regs, env->phys_regs + env->config->nareg,
+(env->sregs[WINDOW_BASE] * 4 + 16 - env->config->nareg) *
+sizeof(uint32_t));
+}
 }
 
-static void rotate_window_abs(CPUXtensaState *env, uint32_t position)
+void rotate_window_abs(CPUXtensaState *env, uint32_t position)
 {
 xtensa_sync_phys_from_window(env);
 env->sregs[WINDOW_BASE] = windowbase_bound(position, env);
+env->regs = env->phys_regs + env->sregs[WINDOW_BASE] * 4;
 xtensa_sync_window_from_phys(env);
 }
 
diff --git a/target-xtensa/translate.c b/target-xtensa/translate.c
index 9894488..c988511 100644
--- a/target-xtensa/translate.c
+++ b/target-xtensa/translate.c
@@ -75,6 +75,7 @@ typedef struct DisasContext {
 } DisasContext;
 
 static TCGv_env cpu_env;
+static TCGv_ptr cpu_regs;
 static TCGv_i32 cpu_pc;
 static TCGv_i32 cpu_R[16];
 static TCGv_i32 cpu_FR[16];
@@ -218,12 +219,14 @@ void xtensa_translate_init(void)
 int i;
 
 

Re: [Qemu-devel] [PATCH v3 1/3] hw/timer: Add ASPEED timer device model

[Andrew Jeffery]

On Fri, 2016-03-11 at 15:56 +0700, Peter Maydell wrote:
> On 5 March 2016 at 11:29, Andrew Jeffery  wrote:
> > Implement basic AST2400 timer functionality: Up to 8 timers can
> > independently be configured, enabled, reset and disabled. A couple of
> > hardware features are not implemented, namely clock value matching and
> > pulse generation, but the implementation is enough to boot the Linux
> > kernel configured with aspeed_defconfig.
> > 
> > Signed-off-by: Andrew Jeffery 
> > ---
> > +/**
> > + * Avoid mutual references between AspeedTimerCtrlState and AspeedTimer
> > + * structs, as it's a waste of memory and it makes implementing
> > + * VMStateDescription a little clunky.
> 
> Not sure what you have in mind with the reference to VMStateDescription
> here. The vmstate struct only has to list the fields which contain
> actual volatile state -- things like backreference pointers to other
> structs aren't volatile state so don't appear.

Good point, looks like I was over-thinking things. I'll remove the part
referencing VMStateDescription.

> 
> > The ptimer BH callback needs to know
> > + * whether a specific AspeedTimer is enabled, but this information is held 
> > in
> > + * AspeedTimerCtrlState. So, provide a helper to hoist ourselves from an
> > + * arbitrary AspeedTimer to AspeedTimerCtrlState.
> > + */
> > +static inline struct AspeedTimerCtrlState *timer_to_ctrl(AspeedTimer *t)
> > +{
> > +AspeedTimer (*timers)[] = (void *)t - (t->id * sizeof(*t));
> > +return container_of(timers, AspeedTimerCtrlState, timers);
> > +}
> 
> > +static void aspeed_timer_expire(void *opaque)
> > +{
> > +AspeedTimer *t = opaque;
> > +
> > +/* Only support interrupts on match values of zero for the moment - 
> > this is
> > + * sufficient to boot an aspeed_defconfig Linux kernel. Non-zero match
> > + * values need some further consideration given the current ptimer API.
> > + * Maybe run multiple ptimers?
> > + */
> 
> See hw/timer/a9gtimer.c for an example of a timer with a comparator
> that can fire when the timer hits an arbitrary comparator value
> (it doesn't use ptimers but the principle is the same -- you set
> the timer to fire at the next interesting event, and then in the
> timer-fired handler you reset the timer to fire whenever the next
> event after that is, if any.) In any case this is probably ok for now.

Thanks for the pointer. I'll leave that change to a future patch given
it looks like this is converging on being acceptable, though I'll
expand the comment to cover a9gtimer.

> 
> > +bool match = !(t->match[0] && t->match[1]);
> > +bool interrupt = timer_overflow_interrupt(t) || match;
> > +if (timer_enabled(t) && interrupt) {
> > +t->level = !t->level;
> > +qemu_set_irq(t->irq, t->level);
> > +}
> > +}
> > +
> 
> Otherwise
> Reviewed-by: Peter Maydell 

Thanks!

Andrew

> 
> thanks
> -- PMM

signature.asc
Description: This is a digitally signed message part

Re: [Qemu-devel] [PATCH 0/5] Add ACPI and SMBIOS table entries for IPMI

[Corey Minyard]

I haven't seen any comments on this. Is it ok?  I'd like to
get this in before the next release so it works automatically.

On 02/25/2016 01:59 AM, miny...@acm.org wrote:

Now that Igor has reworked things to remove the SSDT, I've updated
IPMI code to work with the new format.  This is functionally the
same as before, just for ACPI it adds the data to the DSDT now.

This will avoid a lot of "Why doesn't IPMI work when I add it?"
questions, which I have already gotten.

-corey

Re: [Qemu-devel] [PATCH v4 3/4] adb-keys.h: initial commit

[Peter Maydell]

On 11 March 2016 at 09:29, Programmingkid  wrote:
> This commit implements the adb-keys.h file. It holds information on adb 
> keycode
> values.
>
> Signed-off-by: John Arbuckle 
> ---

Reviewed-by: Peter Maydell 

thanks
-- PMM

Re: [Qemu-devel] [PATCH v4 1/4] ui/cocoa.m: switch to QKeyCode

[Peter Maydell]

On 11 March 2016 at 09:25, Programmingkid  wrote:
> This patch removes the pc/xt keycode map and replaces it with the QKeyCode
> keymap.
>
> Signed-off-by: John Arbuckle 
> ---

Reviewed-by: Peter Maydell 

thanks
-- PMM

Re: [Qemu-devel] [PATCH v4 0/4] Implement some QKeyCode support

[Peter Maydell]

On 12 March 2016 at 00:19, Programmingkid  wrote:
> My git send-email is broken. I spend hours googling fixes that
> didn't work.

Unfortunately this makes it awkward for me to actually test
your patches, because the tooling for handling patch mails assumes
they're formatted correctly:

$ patches apply id:0ca9c4f8-7294-4a90-bc5c-328f7c73a...@gmail.com
Cannot apply series: series is either incomplete or improperly threaded.

> I will remember next time to add the titles to the cover letter.

You can just use the git format-patch --cover-letter option, which
will automatically generate a cover mail template with the format
people expect (which you can then edit to add the description/series
title/etc).

thanks
-- PMM

Re: [Qemu-devel] [PATCH] target-arm: Fix descriptor address masking in ARM address translation

[Peter Maydell]

On 12 March 2016 at 06:44, Sergey Sorokin  wrote:
> 11.03.2016, 11:41, "Peter Maydell" :
>>On 4 March 2016 at 23:04, Sergey Sorokin  wrote:
>>> There is a bug in ARM address translation regime with a long-descriptor
>>> format. On the descriptor reading its address is formed from an index
>>> which is a part of the input address. And on the first iteration this index
>>> is incorrectly masked with 'grainsize' mask. But it can be wider according
>>> to pseudo-code.
>>> On the other hand on the iterations other than first the descriptor address
>>> is formed from the previous level descriptor by masking with 'descaddrmask'
>>> value. It always clears just 12 lower bits, but it must clear 'grainsize'
>>> lower bits instead according to pseudo-code.
>>> The patch fixes both cases.
>>
>>This is pretty confusing to understand -- it might help if you
>>could give an example.
>
> According to documentation (ARMv8 ARM DDI 0487A.i J1.1.5:
> aarch64/translation/walk/AArch64.TranslationTableWalk):
>
> bits(48) index = ZeroExtend(inputaddr:'000');
> descaddr.paddress.physicaladdress = baseaddress OR index;
>
> For a first iteration of the descriptor reading:
>
> addrselecttop = inputsize - 1;
> addrselectbottom = (3-level)*stride + grainsize;
>
> Let's assume grainsize == 12 (so stride == 9), level == 1, inputsize == 43.
> Then index is
> inputaddr<42:30>:'000';

...which is more than 9 bits, so when does this happen?
I think this can only happen for the Stage-2 only
concatenated translation-tables case...

(I agree we have a bug here, I'm just trying to work out when it
can trigger; if it's only possible for S2 page tables then it's
not a visible bug yet because no CPUs have EL2 support enabled.)

>>> -/* The address field in the descriptor goes up to bit 39 for ARMv7
>>> - * but up to bit 47 for ARMv8.
>>> +/* The address field in the descriptor goes up to bit 39 for AArch32
>>> + * but up to bit 47 for AArch64.
>>>   */
>>
>>This is not correct -- the descriptor field widths are as the comment
>>states before your patch:
>> * up to bit 39 for ARMv7
>> * up to bit 47 for ARMv8 (whether AArch32 or AArch64)
>>
>>See the v8 ARM ARM AArch32.TranslationTableWalkLD pseudocode and in
>>particular note the width which it uses for AddressSizeFault checks.
>
> I see in ARMv8 ARM DDI 0487A.i J1.2.4
> aarch32/translation/walk/AArch32.TranslationTableWalkLD:
>
> Before 'repeat' cycle:
> baseaddress = baseregister<39:baselowerbound>:Zeros(baselowerbound);
>
> Inside the cycle:
> baseaddress = desc<39:grainsize>:Zeros(grainsize);

Yes, but this happens only after we have done the check:

  if !IsZero(desc<47:40>) then
 [take the AddressSize fault]

which tells us that the descriptor field really is up to bit 48.
We just haven't yet implemented the check in QEMU which will
generate the AddressSize fault if the top bits are nonzero.
(In contrast, in ARMv7 there really are only 40 bits there.)

If you want to implement the AddressSize checks that's fine,
but otherwise please leave this bit of the code alone.

thanks
-- PMM

Re: [Qemu-devel] [PATCH v2 00/18] Multiple fixes & improvements to QIOChannel & Win32

[Andrew Baumann]

Hi folks,

> From: Paolo Bonzini [mailto:pbonz...@redhat.com]
> Sent: Thursday, 10 March 2016 9:37 AM
> 
> On 10/03/2016 18:26, Daniel P. Berrange wrote:
> > This series started out as an attempt to fix the Win32 problems
> > identified by Andrew Baumann
> >
> >https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg01343.html
> >
> > It turned into a significantly larger cleanup of some chardev
> > and osdep win32 portability code.
[...]

Sorry for chiming in a bit late here. I've tested these patches (the complete 
set, not individually), and they do appear to fix my immediate problem: socket 
char devices now work again. So thank you!

However, I'm now seeing a problem I don't believe we had before: very slow 
responses to GDB commands. From looking at a packet capture (using a localhost 
tcp socket between qemu and my gdb client), it seems that a couple of 
operations will go through just fine, and then there is a 1 second delay 
between my client's request and qemu's response. After fiddling with poll 
timeouts, it became clear that we were noticing the socket events when waking 
up from the poll, but the events themselves were still not waking us. It turns 
out that we were not calling WSAEventSelect on the accept path. At least, the 
following patch fixed the problem for me:

diff --git a/qemu-char.c b/qemu-char.c
index 3bf30b5..c1be622 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -3047,6 +3047,7 @@ static gboolean tcp_chr_accept(QIOChannel *channel,
 return TRUE;
 }

+qio_channel_set_blocking(QIO_CHANNEL(sioc), false, NULL);
 tcp_chr_new_client(chr, sioc);

 object_unref(OBJECT(sioc));

However, I'd note that both callers of tcp_chr_new_client() make the same call 
to set blocking to false immediately before calling tcp_chr_new_client(). 
Furthermore, the doc comment for qio_channel_set_blocking() appears to suggest 
that non-blocking mode is the default. If that's true, maybe you don't even 
want to rely on the caller explicitly setting blocking to false?

Cheers,
Andrew

Re: [Qemu-devel] [PATCH] target-arm: Fix descriptor address masking in ARM address translation

[Sergey Sorokin]

11.03.2016, 11:41, "Peter Maydell" :
>On 4 March 2016 at 23:04, Sergey Sorokin  wrote:
>> There is a bug in ARM address translation regime with a long-descriptor
>> format. On the descriptor reading its address is formed from an index
>> which is a part of the input address. And on the first iteration this index
>> is incorrectly masked with 'grainsize' mask. But it can be wider according
>> to pseudo-code.
>> On the other hand on the iterations other than first the descriptor address
>> is formed from the previous level descriptor by masking with 'descaddrmask'
>> value. It always clears just 12 lower bits, but it must clear 'grainsize'
>> lower bits instead according to pseudo-code.
>> The patch fixes both cases.
>
>This is pretty confusing to understand -- it might help if you
>could give an example.

According to documentation (ARMv8 ARM DDI 0487A.i J1.1.5:
aarch64/translation/walk/AArch64.TranslationTableWalk):

bits(48) index = ZeroExtend(inputaddr:'000');
descaddr.paddress.physicaladdress = baseaddress OR index;

For a first iteration of the descriptor reading:

addrselecttop = inputsize - 1;
addrselectbottom = (3-level)*stride + grainsize;

Let's assume grainsize == 12 (so stride == 9), level == 1, inputsize == 43.
Then index is
inputaddr<42:30>:'000';

But currently QEMU incorrecly masks an input address with descmask value:

descmask = (1ULL << (stride + 3)) - 1;
...
descaddr |= (address >> (stride * (4 - level))) & descmask;
descaddr &= ~7ULL;

Then the index in my example in QEMU is:
address<38:30>:'000';

And my patch fixes this bug.

>
>Is this something that only causes problems for the "concatenated
>translation tables at the initial level" case (which is only
>possible for S2 tables) ?
>
>> Signed-off-by: Sergey Sorokin 
>> ---
>>  target-arm/helper.c | 29 ++---
>>  1 file changed, 10 insertions(+), 19 deletions(-)
>>
>> diff --git a/target-arm/helper.c b/target-arm/helper.c
>> index dec8e8b..b5f289c 100644
>> --- a/target-arm/helper.c
>> +++ b/target-arm/helper.c
>> @@ -7243,7 +7243,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, 
>> target_ulong address,
>>  uint32_t tg;
>>  uint64_t ttbr;
>>  int ttbr_select;
>> -hwaddr descaddr, descmask;
>> +hwaddr descaddr, indexmask, indexmask_grainsize;
>>  uint32_t tableattrs;
>>  target_ulong page_size;
>>  uint32_t attrs;
>> @@ -7431,28 +7431,18 @@ static bool get_phys_addr_lpae(CPUARMState *env, 
>> target_ulong address,
>>  }
>>  }
>>
>> -/* Clear the vaddr bits which aren't part of the within-region address,
>> - * so that we don't have to special case things when calculating the
>> - * first descriptor address.
>> - */
>> -if (va_size != inputsize) {
>> -address &= (1ULL << inputsize) - 1;
>> -}
>> -
>> -descmask = (1ULL << (stride + 3)) - 1;
>> +indexmask_grainsize = (1ULL << (stride + 3)) - 1;
>> +indexmask = (1ULL << (inputsize - (stride * (4 - level - 1;
>>
>>  /* Now we can extract the actual base address from the TTBR */
>>  descaddr = extract64(ttbr, 0, 48);
>> -descaddr &= ~((1ULL << (inputsize - (stride * (4 - level - 1);
>> +descaddr &= ~indexmask;
>>
>> -/* The address field in the descriptor goes up to bit 39 for ARMv7
>> - * but up to bit 47 for ARMv8.
>> +/* The address field in the descriptor goes up to bit 39 for AArch32
>> + * but up to bit 47 for AArch64.
>>   */
>
>This is not correct -- the descriptor field widths are as the comment
>states before your patch:
> * up to bit 39 for ARMv7
> * up to bit 47 for ARMv8 (whether AArch32 or AArch64)
>
>See the v8 ARM ARM AArch32.TranslationTableWalkLD pseudocode and in
>particular note the width which it uses for AddressSizeFault checks.

I see in ARMv8 ARM DDI 0487A.i J1.2.4
aarch32/translation/walk/AArch32.TranslationTableWalkLD:

Before 'repeat' cycle:
baseaddress = baseregister<39:baselowerbound>:Zeros(baselowerbound);

Inside the cycle:
baseaddress = desc<39:grainsize>:Zeros(grainsize);

We use 'descaddrmask' in QEMU to get this 'baseaddress' from a descriptor.
So my patch is correct and fixes the bug.

>
>> -if (arm_feature(env, ARM_FEATURE_V8)) {
>> -descaddrmask = 0xf000ULL;
>> -} else {
>> -descaddrmask = 0xfff000ULL;
>> -}
>> +descaddrmask = ((1ull << (va_size == 64 ? 48 : 40)) - 1) &
>> +   ~indexmask_grainsize;
>
>...so this part of the patch is wrong.
>
>>
>>  /* Secure accesses start with the page table in secure memory and
>>   * can be downgraded to non-secure at any step. Non-secure accesses
>> @@ -7464,7 +7454,7 @@ static bool get_phys_addr_lpae(CPUARMState *env, 
>> target_ulong address,
>>  uint64_t descriptor;
>>  bool nstable;
>>
>> -descaddr |= (address >> (stride * (4 - level))) & descmask;
>> +descaddr |= (address >> (stride * (4 - 

[Qemu-devel] [Bug 1556306] Re: vhost-user: qemu stops processing packets under high load of traffic

[Vincent JARDIN]

for tracking,
  
http://git.qemu.org/?p=qemu.git;a=patch;h=5669655aafdb88a8797c74a989dd0c0ebb1349fa

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1556306

Title:
   vhost-user: qemu stops processing packets under high load of traffic

Status in QEMU:
  New

Bug description:
  Description of problem:
  - qemu socket becomes full, causing qemu to send incomplete
  SET_VRING_CALL messages to vhost-user backend (without proper fd set in
  ancillary data).
  - after some time, some interrupts are lost, causing the VM to stop
  transmitting packets.

  How reproducible:
  Run a stress tests of a vhost-user interface using an UDP
  traffic generator. Traffic generator (IXIA) was connected to 2 physical ports 
that are in turn connected to 2 virtio ports through a linux bridge, VM
  (running linux) doing routing to forward packets between the 2 virtio ports.
  When traffic reaches high pps rates of small packets,

  Actual results:
  - VM stop transmitting packets

  Expected results:
  - VM should never stop transmitting packets

  Additional info:
  We do propose a fix at:
http://lists.nongnu.org/archive/html/qemu-devel/2015-12/msg00652.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1556306/+subscriptions

[Qemu-devel] [RFC PATCH v4 2/3] vfio: Enable sparse mmap capability

[Alex Williamson]

The sparse mmap capability in a vfio region info allows vfio to tell
us which sub-areas of a region may be mmap'd.  Thus rather than
assuming a single mmap covers the entire region and later frobbing it
ourselves for things like the PCI MSI-X vector table, we can read that
directly from vfio.

Signed-off-by: Alex Williamson 
---
 hw/vfio/common.c |   67 +++---
 trace-events |2 ++
 2 files changed, 65 insertions(+), 4 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 96ccb79..879a657 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -493,6 +493,54 @@ static void vfio_listener_release(VFIOContainer *container)
 memory_listener_unregister(>listener);
 }
 
+static struct vfio_info_cap_header *
+vfio_get_region_info_cap(struct vfio_region_info *info, uint16_t id)
+{
+struct vfio_info_cap_header *hdr;
+void *ptr = info;
+
+if (!(info->flags & VFIO_REGION_INFO_FLAG_CAPS)) {
+return NULL;
+}
+
+for (hdr = ptr + info->cap_offset; hdr != ptr; hdr = ptr + hdr->next) {
+if (hdr->id == id) {
+return hdr;
+}
+}
+
+return NULL;
+}
+
+static void vfio_setup_region_sparse_mmaps(VFIORegion *region,
+   struct vfio_region_info *info)
+{
+struct vfio_info_cap_header *hdr;
+struct vfio_region_info_cap_sparse_mmap *sparse;
+int i;
+
+hdr = vfio_get_region_info_cap(info, VFIO_REGION_INFO_CAP_SPARSE_MMAP);
+if (!hdr) {
+return;
+}
+
+sparse = container_of(hdr, struct vfio_region_info_cap_sparse_mmap, 
header);
+
+trace_vfio_region_sparse_mmap_header(region->vbasedev->name,
+ region->nr, sparse->nr_areas);
+
+region->nr_mmaps = sparse->nr_areas;
+region->mmaps = g_new0(VFIOMmap, region->nr_mmaps);
+
+for (i = 0; i < region->nr_mmaps; i++) {
+region->mmaps[i].offset = sparse->areas[i].offset;
+region->mmaps[i].size = sparse->areas[i].size;
+trace_vfio_region_sparse_mmap_entry(i, region->mmaps[i].offset,
+region->mmaps[i].offset +
+region->mmaps[i].size);
+}
+}
+
 int vfio_region_setup(Object *obj, VFIODevice *vbasedev, VFIORegion *region,
   int index, const char *name)
 {
@@ -519,11 +567,14 @@ int vfio_region_setup(Object *obj, VFIODevice *vbasedev, 
VFIORegion *region,
 region->flags & VFIO_REGION_INFO_FLAG_MMAP &&
 !(region->size & ~qemu_real_host_page_mask)) {
 
-region->nr_mmaps = 1;
-region->mmaps = g_new0(VFIOMmap, region->nr_mmaps);
+vfio_setup_region_sparse_mmaps(region, info);
 
-region->mmaps[0].offset = 0;
-region->mmaps[0].size = region->size;
+if (!region->nr_mmaps) {
+region->nr_mmaps = 1;
+region->mmaps = g_new0(VFIOMmap, region->nr_mmaps);
+region->mmaps[0].offset = 0;
+region->mmaps[0].size = region->size;
+}
 }
 }
 
@@ -1083,6 +1134,7 @@ int vfio_get_region_info(VFIODevice *vbasedev, int index,
 *info = g_malloc0(argsz);
 
 (*info)->index = index;
+retry:
 (*info)->argsz = argsz;
 
 if (ioctl(vbasedev->fd, VFIO_DEVICE_GET_REGION_INFO, *info)) {
@@ -1090,6 +1142,13 @@ int vfio_get_region_info(VFIODevice *vbasedev, int index,
 return -errno;
 }
 
+if ((*info)->argsz > argsz) {
+argsz = (*info)->argsz;
+*info = g_realloc(*info, argsz);
+
+goto retry;
+}
+
 return 0;
 }
 
diff --git a/trace-events b/trace-events
index bf07a8f..6f679ab 100644
--- a/trace-events
+++ b/trace-events
@@ -1732,6 +1732,8 @@ vfio_region_mmap(const char *name, unsigned long offset, 
unsigned long end) "Reg
 vfio_region_exit(const char *name, int index) "Device %s, region %d"
 vfio_region_finalize(const char *name, int index) "Device %s, region %d"
 vfio_region_mmaps_set_enabled(const char *name, bool enabled) "Region %s mmaps 
enabled: %d"
+vfio_region_sparse_mmap_header(const char *name, int index, int nr_areas) 
"Device %s region %d: %d sparse mmap entries"
+vfio_region_sparse_mmap_entry(int i, off_t start, off_t end) "sparse entry %d 
[0x%lx - 0x%lx]"
 
 # hw/vfio/platform.c
 vfio_platform_base_device_init(char *name, int groupid) "%s belongs to group 
#%d"

[Qemu-devel] [RFC PATCH v4 3/3] vfio/pci: Intel IGD support

[Alex Williamson]

Two modes are available for IGD assignment, Universal Passthrough
(UPT) and legacy.  UPT mode attempts to handle the IGD device as if it
were just a PCI device, requiring no collateral changes to the VM
chipset.  For the most part this works without these changes.  The one
feature found here for UPT mode is the addition of the OpRegion, which
enables local display support (at least for external monitors).  UPT
mode requires guest driver support and a Broadwell or newer GPU.  The
legacy mode provided here supports back through SandyBridge
processors and attempts to populate key properties of the VM chipset
to match the host device, along with certain quirks to enable the
device to work in the VM address space.

The code here attempts to guess the mode to apply based on the
configuration of the VM.  Some aspects are out of our control, for
example execution of the VGA BIOS requires the device to be at PCI
address 00:02.0.  The VGA BIOS also requires an ISA/LPC bridge with
IDs matching the host device at address 00:1f.0.  Execution of the VGA
ROM implies VGA support.  Therefore, legacy mode is automatically
enabled when the IGD device is found at PCI address 00:02.0, it
supports a ROM BAR and has a non-zero ROM size provided through VFIO,
the address 00:1f.0 is available for the LPC bridge, and the VGA
access is supported through vfio.  Anything outside of that
configuration assumes UPT mode.

Notable in that configuration requirement is the slot at 00:1f.0,
which is occupied in Q35 machine configurations.  We cannot simply
overwrite the device IDs of this component on Q35, therefore only UPT
mode is available currently on Q35.

UPT is intended to work with the IGD as a secondary graphics device
with an emulated graphics device as the primary.  Depending on the
guest operating system, the IGD and emulated devices may mirror the
same display, extend a shared desktop, or either device may be
disabled.  The primary graphics device should be at a lower PCI slot
address than any secondary devices.

In legacy mode, the IGD should be the primary graphics device and more
probably the exclusive graphics device based on my experience.

The changes here depend on vfio kernel changes which are currently in
the linux-next tree and slated for inclusion in kernel v4.6.  Also
required is a modified SeaBIOS with support for the fw_cfg features
added here: http://patchwork.ozlabs.org/patch/583731/  OVMF support
has not yet been investigated, but I have yet to see an IGD device
with UEFI ROM support.  This makes only the OpRegion support relevant
to a pure UEFI OVMF image.

Signed-off-by: Alex Williamson 
---
 hw/vfio/common.c  |2 
 hw/vfio/pci-quirks.c  |  548 +
 hw/vfio/pci.c |   68 +
 hw/vfio/pci.h |   10 +
 include/hw/vfio/vfio-common.h |2 
 trace-events  |7 -
 6 files changed, 635 insertions(+), 2 deletions(-)

diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 879a657..c201bee 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -493,7 +493,7 @@ static void vfio_listener_release(VFIOContainer *container)
 memory_listener_unregister(>listener);
 }
 
-static struct vfio_info_cap_header *
+struct vfio_info_cap_header *
 vfio_get_region_info_cap(struct vfio_region_info *info, uint16_t id)
 {
 struct vfio_info_cap_header *hdr;
diff --git a/hw/vfio/pci-quirks.c b/hw/vfio/pci-quirks.c
index 49ecf11..f4e1ece 100644
--- a/hw/vfio/pci-quirks.c
+++ b/hw/vfio/pci-quirks.c
@@ -11,9 +11,11 @@
  */
 
 #include "qemu/osdep.h"
+#include "hw/nvram/fw_cfg.h"
 #include "pci.h"
 #include "trace.h"
 #include "qemu/range.h"
+#include "qemu/error-report.h"
 
 /* Use uin32_t for vendor & device so PCI_ANY_ID expands and cannot match hw */
 static bool vfio_pci_is(VFIOPCIDevice *vdev, uint32_t vendor, uint32_t device)
@@ -962,6 +964,551 @@ static void vfio_probe_rtl8168_bar2_quirk(VFIOPCIDevice 
*vdev, int nr)
 }
 
 /*
+ * Intel IGD support
+ *
+ * We need to do a few things to support Intel Integrated Graphics Devices:
+ *  1) Define a stolen memory region and trap I/O port writes programming it
+ *  2) Expose the OpRegion if one is provided to us
+ *  3) Copy key PCI config space register values from the host bridge
+ *  4) Create an LPC/ISA bridge and do the same for it.
+ *
+ * Each of these is described below in more detail.
+ *
+ * There are two modes of operation, Universal Passthrough (UPT) mode, which
+ * technically requires none of this, but does benefit with local display
+ * output via the OpRegion support, and "legacy" mode, which makes use of all
+ * of these.  UPT should work on Broadwell and newer devices while legacy mode
+ * should work on SandyBridge and newer.  We try to guess which mode to use
+ * based on the configuration of the system.  To trigger legacy mode, you must:
+ *  - Configure the IGD device at address 00:02.0
+ *  - Not have a device at 00:1f.0 to 

[Qemu-devel] [RFC PATCH v4 1/3] linux-headers/vfio: Update matching current linux-next

[Alex Williamson]

These changes should appear in kernel v4.6.

Signed-off-by: Alex Williamson 
---
 linux-headers/linux/vfio.h |   92 +++-
 1 file changed, 90 insertions(+), 2 deletions(-)

diff --git a/linux-headers/linux/vfio.h b/linux-headers/linux/vfio.h
index 15e096c..759b850 100644
--- a/linux-headers/linux/vfio.h
+++ b/linux-headers/linux/vfio.h
@@ -59,6 +59,33 @@
 #define VFIO_TYPE  (';')
 #define VFIO_BASE  100
 
+/*
+ * For extension of INFO ioctls, VFIO makes use of a capability chain
+ * designed after PCI/e capabilities.  A flag bit indicates whether
+ * this capability chain is supported and a field defined in the fixed
+ * structure defines the offset of the first capability in the chain.
+ * This field is only valid when the corresponding bit in the flags
+ * bitmap is set.  This offset field is relative to the start of the
+ * INFO buffer, as is the next field within each capability header.
+ * The id within the header is a shared address space per INFO ioctl,
+ * while the version field is specific to the capability id.  The
+ * contents following the header are specific to the capability id.
+ */
+struct vfio_info_cap_header {
+   __u16   id; /* Identifies capability */
+   __u16   version;/* Version specific to the capability ID */
+   __u32   next;   /* Offset of next capability */
+};
+
+/*
+ * Callers of INFO ioctls passing insufficiently sized buffers will see
+ * the capability chain flag bit set, a zero value for the first capability
+ * offset (if available within the provided argsz), and argsz will be
+ * updated to report the necessary buffer size.  For compatibility, the
+ * INFO ioctl will not report error in this case, but the capability chain
+ * will not be available.
+ */
+
 /*  IOCTLs for VFIO file descriptor (/dev/vfio/vfio)  */
 
 /**
@@ -194,13 +221,73 @@ struct vfio_region_info {
 #define VFIO_REGION_INFO_FLAG_READ (1 << 0) /* Region supports read */
 #define VFIO_REGION_INFO_FLAG_WRITE(1 << 1) /* Region supports write */
 #define VFIO_REGION_INFO_FLAG_MMAP (1 << 2) /* Region supports mmap */
+#define VFIO_REGION_INFO_FLAG_CAPS (1 << 3) /* Info supports caps */
__u32   index;  /* Region index */
-   __u32   resv;   /* Reserved for alignment */
+   __u32   cap_offset; /* Offset within info struct of first cap */
__u64   size;   /* Region size (bytes) */
__u64   offset; /* Region offset from start of device fd */
 };
 #define VFIO_DEVICE_GET_REGION_INFO_IO(VFIO_TYPE, VFIO_BASE + 8)
 
+/*
+ * The sparse mmap capability allows finer granularity of specifying areas
+ * within a region with mmap support.  When specified, the user should only
+ * mmap the offset ranges specified by the areas array.  mmaps outside of the
+ * areas specified may fail (such as the range covering a PCI MSI-X table) or
+ * may result in improper device behavior.
+ *
+ * The structures below define version 1 of this capability.
+ */
+#define VFIO_REGION_INFO_CAP_SPARSE_MMAP   1
+
+struct vfio_region_sparse_mmap_area {
+   __u64   offset; /* Offset of mmap'able area within region */
+   __u64   size;   /* Size of mmap'able area */
+};
+
+struct vfio_region_info_cap_sparse_mmap {
+   struct vfio_info_cap_header header;
+   __u32   nr_areas;
+   __u32   reserved;
+   struct vfio_region_sparse_mmap_area areas[];
+};
+
+/*
+ * The device specific type capability allows regions unique to a specific
+ * device or class of devices to be exposed.  This helps solve the problem for
+ * vfio bus drivers of defining which region indexes correspond to which region
+ * on the device, without needing to resort to static indexes, as done by
+ * vfio-pci.  For instance, if we were to go back in time, we might remove
+ * VFIO_PCI_VGA_REGION_INDEX and let vfio-pci simply define that all indexes
+ * greater than or equal to VFIO_PCI_NUM_REGIONS are device specific and we'd
+ * make a "VGA" device specific type to describe the VGA access space.  This
+ * means that non-VGA devices wouldn't need to waste this index, and thus the
+ * address space associated with it due to implementation of device file
+ * descriptor offsets in vfio-pci.
+ *
+ * The current implementation is now part of the user ABI, so we can't use this
+ * for VGA, but there are other upcoming use cases, such as opregions for Intel
+ * IGD devices and framebuffers for vGPU devices.  We missed VGA, but we'll
+ * use this for future additions.
+ *
+ * The structure below defines version 1 of this capability.
+ */
+#define VFIO_REGION_INFO_CAP_TYPE  2
+
+struct vfio_region_info_cap_type {
+   struct vfio_info_cap_header header;
+   __u32 type; /* global per bus driver */
+   __u32 subtype;  /* type specific */
+};
+
+#define VFIO_REGION_TYPE_PCI_VENDOR_TYPE   (1 << 31)
+#define 

[Qemu-devel] [RFC PATCH v4 0/3] vfio IGD assignment

[Alex Williamson]

This series applies on top of my latest vfio pull request[1] or
directly to the tag mentioned there[2].  It also requires kernel
support that's currently in linux-next for v4.6, thus the RFC here.
IGD assignment also requires SeaBIOS support as noted in the commit
log of the 3rd patch here.  Patch 2 is not directly IGD related, but
it does make use of some of the same features added to the kernel and
is thus queued here as well for review and testing.  See patch 3 for a
full description of the level of IGD assignment support offered here.
Please review, test, and comment.  Thanks,

Alex

[1] https://lists.nongnu.org/archive/html/qemu-devel/2016-03/msg02743.html
[2] https://github.com/awilliam/qemu-vfio/tree/vfio-update-20160311.0

---

Alex Williamson (3):
  linux-headers/vfio: Update matching current linux-next
  vfio: Enable sparse mmap capability
  vfio/pci: Intel IGD support


 hw/vfio/common.c  |   67 +
 hw/vfio/pci-quirks.c  |  548 +
 hw/vfio/pci.c |   68 +
 hw/vfio/pci.h |   10 +
 include/hw/vfio/vfio-common.h |2 
 linux-headers/linux/vfio.h|   92 +++
 trace-events  |9 +
 7 files changed, 789 insertions(+), 7 deletions(-)

Re: [Qemu-devel] [PATCH v4 17/26] tests: refactor python I/O tests helper main method

[Eric Blake]

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote:
> The iotests.py helper provides a main() method for running
> tests via the python unit test framework. Not all tests
> will want to use this, so refactor it to split the testing
> of compatible formats and platforms into separate helper
> methods
> 
> Signed-off-by: Daniel P. Berrange 
> ---
>  tests/qemu-iotests/iotests.py | 18 --
>  1 file changed, 12 insertions(+), 6 deletions(-)
> 

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 16/26] tests: redirect stderr to stdout for iotests

[Eric Blake]

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote:
> The python I/O tests helper for running qemu-img/qemu-io
> setup stdout to be captured to a pipe, but left stderr
> untouched. As a result, if something failed in qemu-img/
> qemu-io, data written to stderr would get output directly
> and not line up with data on the test stdout due to
> buffering.  If we explicitly redirect stderr to the same
> pipe as stdout, things are much clearer when they go
> wrong.
> 
> Signed-off-by: Daniel P. Berrange 
> ---
>  tests/qemu-iotests/iotests.py | 7 +--
>  1 file changed, 5 insertions(+), 2 deletions(-)
> 

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [Bug 1556306] [NEW] vhost-user: qemu stops processing packets under high load of traffic

[Vincent JARDIN]

Public bug reported:

Description of problem:
- qemu socket becomes full, causing qemu to send incomplete
SET_VRING_CALL messages to vhost-user backend (without proper fd set in
ancillary data).
- after some time, some interrupts are lost, causing the VM to stop
transmitting packets.

How reproducible:
Run a stress tests of a vhost-user interface using an UDP
traffic generator. Traffic generator (IXIA) was connected to 2 physical ports 
that are in turn connected to 2 virtio ports through a linux bridge, VM
(running linux) doing routing to forward packets between the 2 virtio ports.
When traffic reaches high pps rates of small packets,

Actual results:
- VM stop transmitting packets

Expected results:
- VM should never stop transmitting packets

Additional info:
We do propose a fix at:
  http://lists.nongnu.org/archive/html/qemu-devel/2015-12/msg00652.html

** Affects: qemu
 Importance: Undecided
 Status: New


** Tags: 6wind

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1556306

Title:
   vhost-user: qemu stops processing packets under high load of traffic

Status in QEMU:
  New

Bug description:
  Description of problem:
  - qemu socket becomes full, causing qemu to send incomplete
  SET_VRING_CALL messages to vhost-user backend (without proper fd set in
  ancillary data).
  - after some time, some interrupts are lost, causing the VM to stop
  transmitting packets.

  How reproducible:
  Run a stress tests of a vhost-user interface using an UDP
  traffic generator. Traffic generator (IXIA) was connected to 2 physical ports 
that are in turn connected to 2 virtio ports through a linux bridge, VM
  (running linux) doing routing to forward packets between the 2 virtio ports.
  When traffic reaches high pps rates of small packets,

  Actual results:
  - VM stop transmitting packets

  Expected results:
  - VM should never stop transmitting packets

  Additional info:
  We do propose a fix at:
http://lists.nongnu.org/archive/html/qemu-devel/2015-12/msg00652.html

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1556306/+subscriptions

Re: [Qemu-devel] Memory on stellaris board

[Peter Maydell]

On 11 March 2016 at 21:12, Aurelio Remonda
 wrote:
> I don't quite understand what you mean with non-round-number, are you 
> suggesting
> we only accept for example:
> 64K-128K-256K-512K-1024k(or 1M)-2048K(or 2M)
> 4096K(or 4M)-8192K(or 8M)-16384K(or 16M)
> If that's the case we will never have, for example, the exact default
> dc0 value (0x00ff007f)

What I am suggesting is that we should accept the memory
sizes which correspond to what can be represented in dc0
(which presumably correspond to possible real hardware
configurations) and not any memory sizes which can't be
represented by a dc0 value.

> because you need to divide the given size by 1024 reverting what the
> prefix (K or M) have
> done and then multiply this value by 1000.

I don't know what you mean here, because ram_size is given
to the board code as a value in bytes. The M and K prefix
stuff is just user convenience when they specify values on
the command line.

thanks
-- PMM

Re: [Qemu-devel] [PATCH v4 13/26] crypto: implement the LUKS block encryption format

[Eric Blake]

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote:
> Provide a block encryption implementation that follows the
> LUKS/dm-crypt specification.
> 
> This supports all combinations of hash, cipher algorithm,
> cipher mode and iv generator that are implemented by the
> current crypto layer.
> 
> The notable missing feature is support for the 'xts'
> cipher mode, which is commonly used for disk encryption
> instead of 'cbc'. This is because it is not provided by
> either nettle or libgcrypt. A suitable implementation
> will be identified & integrated later.

Stale paragraph, you implemented it earlier in the series.

> 
> There is support for opening existing volumes formatted
> by dm-crypt, and for formatting new volumes. In the latter
> case it will only use key slot 0.
> 
> Signed-off-by: Daniel P. Berrange 
> ---


> +static int
> +qcrypto_block_luks_open(QCryptoBlock *block,
> +QCryptoBlockOpenOptions *options,
> +QCryptoBlockReadFunc readfunc,
> +void *opaque,
> +unsigned int flags,
> +Error **errp)
> +{

> +/* Read the entire LUKS header, minus the key material from
> + * the underling device */

s/underling/underlying/ (although the typo does read rather humorously -
I now have a mental image of a LUKS overlord :)


> +++ b/qapi/crypto.json
> @@ -117,12 +117,13 @@

>  ##
>  # QCryptoBlockOptionsBase:
> @@ -143,7 +144,8 @@
>  # The options that apply to QCow/QCow2 AES-CBC encryption format
>  #
>  # @key-secret: #optional the ID of a QCryptoSecret object providing the
> -#  decryption key
> +#  decryption key. Mandatory except when probing image for
> +#  metadata only.

Aha - I think this hunk may belong earlier in the series...

>  #
>  # Since: 2.6
>  ##
> @@ -151,6 +153,45 @@
>'data': { '*key-secret': 'str' }}
>  
>  ##
> +# QCryptoBlockOptionsLUKS:
> +#
> +# The options that apply to LUKS encryption format
> +#
> +# @key-secret: #optional the ID of a QCryptoSecret object providing the
> +#  decryption key

...Although you may want to duplicate it here.

Looks like my review on the earlier version helped, and you addressed
most of my comments.  What I pointed out above is minor enough that I'm
okay if you fix it on the pull request without needing another round of
review, so:

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH 1/1] Allocating Large sized arrays to heap

[Jaya Tiwari]

As per the list of functions in 
http://wiki.qemu.org/BiteSizedTasks#Large_frames,
qemu_get_virtqueue_element  and qemu_put_virtqueue_element have large arrays on 
stack
Hence, moving them to heap. This reduced their stack size from something 49248 
to fit into less than 200

Signed-off-by: Jaya Tiwari 
---
 hw/virtio/virtio.c | 39 +++
 1 file changed, 19 insertions(+), 20 deletions(-)

diff --git a/hw/virtio/virtio.c b/hw/virtio/virtio.c
index 08275a9..7a7afae 100644
--- a/hw/virtio/virtio.c
+++ b/hw/virtio/virtio.c
@@ -636,67 +636,66 @@ typedef struct VirtQueueElementOld {
 void *qemu_get_virtqueue_element(QEMUFile *f, size_t sz)
 {
 VirtQueueElement *elem;
-VirtQueueElementOld data;
+VirtQueueElementOld *data = g_new(VirtQueueElementOld, 1);
 int i;
 
-qemu_get_buffer(f, (uint8_t *), sizeof(VirtQueueElementOld));
+qemu_get_buffer(f, (uint8_t *)data, sizeof(VirtQueueElementOld));
 
-elem = virtqueue_alloc_element(sz, data.out_num, data.in_num);
-elem->index = data.index;
+elem = virtqueue_alloc_element(sz, data->out_num, data->in_num);
+elem->index = data->index;
 
 for (i = 0; i < elem->in_num; i++) {
-elem->in_addr[i] = data.in_addr[i];
+elem->in_addr[i] = data->in_addr[i];
 }
 
 for (i = 0; i < elem->out_num; i++) {
-elem->out_addr[i] = data.out_addr[i];
+elem->out_addr[i] = data->out_addr[i];
 }
 
 for (i = 0; i < elem->in_num; i++) {
 /* Base is overwritten by virtqueue_map.  */
 elem->in_sg[i].iov_base = 0;
-elem->in_sg[i].iov_len = data.in_sg[i].iov_len;
+elem->in_sg[i].iov_len = data->in_sg[i].iov_len;
 }
 
 for (i = 0; i < elem->out_num; i++) {
 /* Base is overwritten by virtqueue_map.  */
 elem->out_sg[i].iov_base = 0;
-elem->out_sg[i].iov_len = data.out_sg[i].iov_len;
+elem->out_sg[i].iov_len = data->out_sg[i].iov_len;
 }
-
+g_free(data);
 virtqueue_map(elem);
 return elem;
 }
 
 void qemu_put_virtqueue_element(QEMUFile *f, VirtQueueElement *elem)
 {
-VirtQueueElementOld data;
+VirtQueueElementOld *data = g_new0(VirtQueueElementOld, 1);
 int i;
-
-memset(, 0, sizeof(data));
-data.index = elem->index;
-data.in_num = elem->in_num;
-data.out_num = elem->out_num;
+data->index = elem->index;
+data->in_num = elem->in_num;
+data->out_num = elem->out_num;
 
 for (i = 0; i < elem->in_num; i++) {
-data.in_addr[i] = elem->in_addr[i];
+data->in_addr[i] = elem->in_addr[i];
 }
 
 for (i = 0; i < elem->out_num; i++) {
-data.out_addr[i] = elem->out_addr[i];
+data->out_addr[i] = elem->out_addr[i];
 }
 
 for (i = 0; i < elem->in_num; i++) {
 /* Base is overwritten by virtqueue_map when loading.  Do not
  * save it, as it would leak the QEMU address space layout.  */
-data.in_sg[i].iov_len = elem->in_sg[i].iov_len;
+data->in_sg[i].iov_len = elem->in_sg[i].iov_len;
 }
 
 for (i = 0; i < elem->out_num; i++) {
 /* Do not save iov_base as above.  */
-data.out_sg[i].iov_len = elem->out_sg[i].iov_len;
+data->out_sg[i].iov_len = elem->out_sg[i].iov_len;
 }
-qemu_put_buffer(f, (uint8_t *), sizeof(VirtQueueElementOld));
+qemu_put_buffer(f, (uint8_t *)data, sizeof(VirtQueueElementOld));
+free(data);
 }
 
 /* virtio device */
-- 
1.9.1

[Qemu-devel] [PATCH] BiteSizedTasks-LargeFrames

[Siddharth Gupta]

From 032be62f56a207833ae12cc9474e3e8be5ed8eb4 Mon Sep 17 00:00:00 2001
From: Siddharth Gupta 
Date: Fri, 11 Mar 2016 20:10:41 +0530
Subject: [PATCH] bitesizedtasks-large_frames-hw_dma_xilinx-hw_net_virtio

---
 hw/dma/xilinx_axidma.c |  5 -
 hw/net/virtio-net.c| 11 ++-
 2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/hw/dma/xilinx_axidma.c b/hw/dma/xilinx_axidma.c
index ce5c1e6..9c6bda2 100644
--- a/hw/dma/xilinx_axidma.c
+++ b/hw/dma/xilinx_axidma.c
@@ -255,13 +255,15 @@ static void stream_process_mem2s(struct Stream *s, StreamSlave *tx_data_dev,
  StreamSlave *tx_control_dev)
 {
 uint32_t prev_d;
-unsigned char txbuf[16 * 1024];
+unsigned char *txbuf;
 unsigned int txlen;
 
 if (!stream_running(s) || stream_idle(s)) {
 return;
 }
 
+txbuf = (unsigned char *) malloc(16 * 1024 * sizeof(unsigned char));
+
 while (1) {
 stream_desc_load(s, s->regs[R_CURDESC]);
 
@@ -303,6 +305,7 @@ static void stream_process_mem2s(struct Stream *s, StreamSlave *tx_data_dev,
 break;
 }
 }
+free(txbuf);
 }
 
 static size_t stream_process_s2mem(struct Stream *s, unsigned char *buf,
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 5798f87..ba6ebac 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -1094,7 +1094,7 @@ static ssize_t virtio_net_receive(NetClientState *nc, const uint8_t *buf, size_t
 VirtIONet *n = qemu_get_nic_opaque(nc);
 VirtIONetQueue *q = virtio_net_get_subqueue(nc);
 VirtIODevice *vdev = VIRTIO_DEVICE(n);
-struct iovec mhdr_sg[VIRTQUEUE_MAX_SIZE];
+struct iovec *mhdr_sg;
 struct virtio_net_hdr_mrg_rxbuf mhdr;
 unsigned mhdr_cnt = 0;
 size_t offset, i, guest_offset;
@@ -1113,6 +1113,8 @@ static ssize_t virtio_net_receive(NetClientState *nc, const uint8_t *buf, size_t
 
 offset = i = 0;
 
+mhdr_sg = (struct iovec *) malloc(VIRTQUEUE_MAX_SIZE * sizeof(struct iovec));
+
 while (offset < size) {
 VirtQueueElement *elem;
 int len, total;
@@ -1122,6 +1124,7 @@ static ssize_t virtio_net_receive(NetClientState *nc, const uint8_t *buf, size_t
 
 elem = virtqueue_pop(q->rx_vq, sizeof(VirtQueueElement));
 if (!elem) {
+free(mhdr_sg);
 if (i == 0)
 return -1;
 error_report("virtio-net unexpected empty queue: "
@@ -1136,11 +1139,15 @@ static ssize_t virtio_net_receive(NetClientState *nc, const uint8_t *buf, size_t
 
 if (elem->in_num < 1) {
 error_report("virtio-net receive queue contains no in buffers");
+free(mhdr_sg);
 exit(1);
 }
 
 sg = elem->in_sg;
 if (i == 0) {
+if (offset != 0) {
+free(mhdr_sg);
+}
 assert(offset == 0);
 if (n->mergeable_rx_bufs) {
 mhdr_cnt = iov_copy(mhdr_sg, ARRAY_SIZE(mhdr_sg),
@@ -1168,6 +1175,7 @@ static ssize_t virtio_net_receive(NetClientState *nc, const uint8_t *buf, size_t
 if (!n->mergeable_rx_bufs && offset < size) {
 virtqueue_discard(q->rx_vq, elem, total);
 g_free(elem);
+free(mhdr_sg);
 return size;
 }
 
@@ -1186,6 +1194,7 @@ static ssize_t virtio_net_receive(NetClientState *nc, const uint8_t *buf, size_t
 virtqueue_flush(q->rx_vq, i);
 virtio_notify(vdev, q->rx_vq);
 
+free(mhdr_sg);
 return size;
 }
 
-- 
2.5.0

Signed-off-by: Siddharth Gupta 

[Qemu-devel] [Patch] BiteSizedTasks LargeFrames

[Siddharth Gupta]

>From 032be62f56a207833ae12cc9474e3e8be5ed8eb4 Mon Sep 17 00:00:00 2001
From: Siddharth Gupta 
Date: Fri, 11 Mar 2016 20:10:41 +0530
Subject: [PATCH] bitesizedtasks-large_frames-hw_dma_xilinx-hw_net_virtio

---
 hw/dma/xilinx_axidma.c |  5 -
 hw/net/virtio-net.c| 11 ++-
 2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/hw/dma/xilinx_axidma.c b/hw/dma/xilinx_axidma.c
index ce5c1e6..9c6bda2 100644
--- a/hw/dma/xilinx_axidma.c
+++ b/hw/dma/xilinx_axidma.c
@@ -255,13 +255,15 @@ static void stream_process_mem2s(struct Stream *s,
StreamSlave *tx_data_dev,
  StreamSlave *tx_control_dev)
 {
 uint32_t prev_d;
-unsigned char txbuf[16 * 1024];
+unsigned char *txbuf;
 unsigned int txlen;

 if (!stream_running(s) || stream_idle(s)) {
 return;
 }

+txbuf = (unsigned char *) malloc(16 * 1024 * sizeof(unsigned char));
+
 while (1) {
 stream_desc_load(s, s->regs[R_CURDESC]);

@@ -303,6 +305,7 @@ static void stream_process_mem2s(struct Stream *s,
StreamSlave *tx_data_dev,
 break;
 }
 }
+free(txbuf);
 }

 static size_t stream_process_s2mem(struct Stream *s, unsigned char *buf,
diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
index 5798f87..ba6ebac 100644
--- a/hw/net/virtio-net.c
+++ b/hw/net/virtio-net.c
@@ -1094,7 +1094,7 @@ static ssize_t virtio_net_receive(NetClientState *nc,
const uint8_t *buf, size_t
 VirtIONet *n = qemu_get_nic_opaque(nc);
 VirtIONetQueue *q = virtio_net_get_subqueue(nc);
 VirtIODevice *vdev = VIRTIO_DEVICE(n);
-struct iovec mhdr_sg[VIRTQUEUE_MAX_SIZE];
+struct iovec *mhdr_sg;
 struct virtio_net_hdr_mrg_rxbuf mhdr;
 unsigned mhdr_cnt = 0;
 size_t offset, i, guest_offset;
@@ -1113,6 +1113,8 @@ static ssize_t virtio_net_receive(NetClientState *nc,
const uint8_t *buf, size_t

 offset = i = 0;

+mhdr_sg = (struct iovec *) malloc(VIRTQUEUE_MAX_SIZE * sizeof(struct
iovec));
+
 while (offset < size) {
 VirtQueueElement *elem;
 int len, total;
@@ -1122,6 +1124,7 @@ static ssize_t virtio_net_receive(NetClientState *nc,
const uint8_t *buf, size_t

 elem = virtqueue_pop(q->rx_vq, sizeof(VirtQueueElement));
 if (!elem) {
+free(mhdr_sg);
 if (i == 0)
 return -1;
 error_report("virtio-net unexpected empty queue: "
@@ -1136,11 +1139,15 @@ static ssize_t virtio_net_receive(NetClientState
*nc, const uint8_t *buf, size_t

 if (elem->in_num < 1) {
 error_report("virtio-net receive queue contains no in
buffers");
+free(mhdr_sg);
 exit(1);
 }

 sg = elem->in_sg;
 if (i == 0) {
+if (offset != 0) {
+free(mhdr_sg);
+}
 assert(offset == 0);
 if (n->mergeable_rx_bufs) {
 mhdr_cnt = iov_copy(mhdr_sg, ARRAY_SIZE(mhdr_sg),
@@ -1168,6 +1175,7 @@ static ssize_t virtio_net_receive(NetClientState *nc,
const uint8_t *buf, size_t
 if (!n->mergeable_rx_bufs && offset < size) {
 virtqueue_discard(q->rx_vq, elem, total);
 g_free(elem);
+free(mhdr_sg);
 return size;
 }

@@ -1186,6 +1194,7 @@ static ssize_t virtio_net_receive(NetClientState *nc,
const uint8_t *buf, size_t
 virtqueue_flush(q->rx_vq, i);
 virtio_notify(vdev, q->rx_vq);

+free(mhdr_sg);
 return size;
 }

-- 
2.5.0

Signed-off-by: Siddharth Gupta 

Re: [Qemu-devel] [PATCH v4 12/26] crypto: add block encryption framework

[Eric Blake]

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote:
> Add a generic framework for support different block encryption

s/support/supporting/

> formats. Upon instantiating a QCryptoBlock object, it will read
> the encryption header and extract the encryption keys. It is
> then possible to call methods to encrypt/decrypt data buffers.
> 
> There is also a mode whereby it will create/initialize a new
> encryption header on a previously unformatted volume.
> 
> The initial framework comes with support for the legacy QCow
> AES based encryption. This enables code in the QCow driver to
> be consolidated later.
> 
> Signed-off-by: Daniel P. Berrange 
> ---

> +/**
> + * qcrypto_block_open:
> + * @options: the encryption options
> + * @readfunc: callback for reading data from the volume
> + * @opaque: data to pass to @readfunc
> + * @flags: bitmask of QCryptoBlockOpenFlags values
> + * @errp: pointer to a NULL-initialized error object

> + *
> + * If any part of initializing the encryption context
> + * fails an error will be returned. This could be due
> + * to the volume being in the wrong format, an cipher

s/an/a/

> + * or IV generator algorithm that is not supported,
> + * or incorrect passphrases.
> + *
> + * Returns: a block encryption format, or NULL on error
> + */
> +QCryptoBlock *qcrypto_block_open(QCryptoBlockOpenOptions *options,
> + QCryptoBlockReadFunc readfunc,
> + void *opaque,
> + unsigned int flags,
> + Error **errp);
> +
> +/**
> + * qcrypto_block_create:

> + * If any part of initializing the encryption context
> + * fails an error will be returned. This could be due
> + * to the volume being in the wrong format, an cipher

and again

> +++ b/qapi/crypto.json
> @@ -109,3 +109,70 @@
>  { 'enum': 'QCryptoIVGenAlgorithm',

> +
> +##
> +# QCryptoBlockOptionsBase:
> +#
> +# The common options that apply to all full disk
> +# encryption formats
> +#
> +# @format: the encryption format
> +#
> +# Since: 2.6
> +##
> +{ 'struct': 'QCryptoBlockOptionsBase',
> +  'data': { 'format': 'QCryptoBlockFormat' }}

We are so close to having my patches in for anonymous base; depending on
whose lands first, we can clean this up to get rid of the one-shot base
(followup is fine).

https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg02296.html

> +
> +##
> +# QCryptoBlockOptionsQCow:
> +#
> +# The options that apply to QCow/QCow2 AES-CBC encryption format
> +#
> +# @key-secret: #optional the ID of a QCryptoSecret object providing the
> +#  decryption key

I mentioned in a previous round that you may want to mention that it is
optional only when probing the image metadata, and mandatory for
reading/writing guest-visible data.  Don't know if that is worth adding
words here.

With the typo fixes,
Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 11/26] crypto: wire up XTS mode for cipher APIs

[Eric Blake]

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote:
> Introduce 'XTS' as a permitted mode for the cipher APIs.
> With XTS the key provided must be twice the size of the
> key normally required for any given algorithm. This is
> because the key will be split into two pieces for use
> in XTS mode.
> 
> Signed-off-by: Daniel P. Berrange 
> ---

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 10/26] crypto: refactor code for dealing with AES cipher

[Eric Blake]

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote:
> The built-in and nettle cipher backends for AES maintain
> two separate AES contexts, one for encryption and one for
> decryption. This is going to be inconvenient for the future
> code dealing with XTS, so wrap them up in a single struct
> so there is just one pointer to pass around for both
> encryptin and decryption.

s/encryptin/encryption/

> 
> Signed-off-by: Daniel P. Berrange 
> ---
>  crypto/cipher-builtin.c | 126 
> ++--
>  crypto/cipher-nettle.c  |  57 +++---
>  2 files changed, 109 insertions(+), 74 deletions(-)

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 09/26] crypto: import an implementation of the XTS cipher mode

[Eric Blake]

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote:
> The XTS (XEX with tweaked-codebook and ciphertext stealing)
> cipher mode is commonly used in full disk encryption. There
> is unfortunately no implementation of it in either libgcrypt
> or nettle, so we need to provide our own.
> 
> The libtomcrypt project provides a repository of crypto
> algorithms under a choice of either "public domain" or
> the "what the fuck public license".
> 
> So this impl is taken from the libtomcrypt GIT repo and
> adapted to be compatible with the way we need to call
> ciphers provided by nettle/gcrypt.
> 
> Signed-off-by: Daniel P. Berrange 
> ---

> +++ b/crypto/xts.c
> @@ -0,0 +1,256 @@
> +/*
> + * QEMU Crypto XTS cipher mode
> + *
> + * Copyright (c) 2015 Red Hat, Inc.

Want to add 2016?

> +
> +#include "qemu/osdep.h"
> +#include "crypto/xts.h"
> +
> +static void xts_mult_x(uint8_t *I)
> +{
> +int x;
> +uint8_t t, tt;
> +
> +for (x = t = 0; x < 16; x++) {
> +tt = I[x] >> 7;
> +I[x] = ((I[x] << 1) | t) & 0xFF;

Why '& 0xf'f? I[x] is already an 8-bit field.  But since it is a direct
copy from
https://github.com/libtom/libtomcrypt/blob/develop/src/modes/xts/xts_mult_x.c,
I won't reject it.  (I could understand the mask if the original code
were using uint_fast8_t for speed at the expense of worrying about
potential padding bits, but no one does that in crypto...)


> +/**
> + * xts_tweak_uncrypt:
> + * @param ctxt: the cipher context
> + * @param func: the cipher function
> + * @src: buffer providing the cipher text of XTS_BLOCK_SIZE bytes
> + * @dst: buffer to output the plain text of XTS_BLOCK_SIZE bytes
> + * @iv: the initialization vector tweak of XTS_BLOCK_SIZE bytes
> + *
> + * Decrypt data with a tweak
> + */
> +static void xts_tweak_decrypt(const void *ctx,
> +  xts_cipher_func *func,
> +  const uint8_t *src,
> +  uint8_t *dst,
> +  uint8_t *iv)
> +{
> +unsigned long x;
> +
> +/* tweak encrypt block i */
> +#ifdef LTC_FAST
> +for (x = 0; x < XTS_BLOCK_SIZE; x += sizeof(LTC_FAST_TYPE)) {
> +*((LTC_FAST_TYPE *)[x]) =
> +*((LTC_FAST_TYPE *)[x]) ^ *((LTC_FAST_TYPE *)[x]);
> +}

Nothing in our configure sets LTC_FAST and friends; should we just nuke
these expressions as dead code?  I see the point of what it is trying to
do: if the data is aligned (or if the processor doesn't care about
alignment), then vectorize it...

> +#else
> +for (x = 0; x < XTS_BLOCK_SIZE; x++) {
> +dst[x] = src[x] ^ iv[x];
> +}

...but we've already argued that the compiler should be able to
auto-vectorize, or at least that hot-path tweaking can be done later.


> +void xts_decrypt(const void *datactx,
> + const void *tweakctx,

> +
> +/* if length not divide XTS_BLOCK_SIZE then */
> +if (mo > 0) {

If length is not a multiple of XTS_BLOCK_SIZE, then


> +void xts_encrypt(const void *datactx,
> + const void *tweakctx,

> +
> +/* if length not divide XTS_BLOCK_SIZE then */

and again

> +++ b/include/crypto/xts.h
> @@ -0,0 +1,86 @@
> +/*
> + * QEMU Crypto XTS cipher mode
> + *
> + * Copyright (c) 2015 Red Hat, Inc.

2016


> +++ b/tests/test-crypto-xts.c
> @@ -0,0 +1,423 @@
> +/*
> + * QEMU Crypto XTS cipher mode
> + *
> + * Copyright (c) 2015 Red Hat, Inc.

and again

Modulo comment tweaks and a decision about whether to nuke LTC_FAST,
Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 00/15] Dirty bitmap changes for migration/persistence work

[John Snow]

On 03/11/2016 08:57 AM, Max Reitz wrote:
> On 08.03.2016 05:44, Fam Zheng wrote:
>> v4: Rebase.
>> Add rev-by from John in patches 1-5, 7, 8.
>> Remove BdrvDirtyBitmap typedef from dirty-bitmap.h in patch 4. [Max]
>> Add assertion on bm->meta in patch 9. [John]
>>
>> Two major features are added to block dirty bitmap (and underlying HBitmap) 
>> in
>> this series: meta bitmap and serialization, together with all other 
>> supportive
>> patches.
>>
>> Both operations are common in dirty bitmap migration and persistence: they 
>> need
>> to find whether and which part of the dirty bitmap in question has changed 
>> with
>> meta dirty bitmap, and they need to write it to the target with 
>> serialization.
>>
>>
>> Fam Zheng (13):
>>   backup: Use Bitmap to replace "s->bitmap"
>>   block: Include hbitmap.h in block.h
>>   typedefs: Add BdrvDirtyBitmap
>>   block: Move block dirty bitmap code to separate files
>>   block: Remove unused typedef of BlockDriverDirtyHandler
>>   block: Hide HBitmap in block dirty bitmap interface
>>   HBitmap: Introduce "meta" bitmap to track bit changes
>>   tests: Add test code for meta bitmap
>>   block: Support meta dirty bitmap
>>   block: Add two dirty bitmap getters
>>   block: Assert that bdrv_release_dirty_bitmap succeeded
>>   tests: Add test code for hbitmap serialization
>>   block: More operations for meta dirty bitmap
>>
>> Vladimir Sementsov-Ogievskiy (2):
>>   hbitmap: serialization
>>   block: BdrvDirtyBitmap serialization interface
>>
>>  block.c  | 360 -
>>  block/Makefile.objs  |   2 +-
>>  block/backup.c   |  25 +-
>>  block/dirty-bitmap.c | 535 
>> +++
>>  block/mirror.c   |  15 +-
>>  include/block/block.h|  40 +---
>>  include/block/dirty-bitmap.h |  75 ++
>>  include/qemu/hbitmap.h   |  96 
>>  include/qemu/typedefs.h  |   2 +
>>  tests/test-hbitmap.c | 255 +
>>  util/hbitmap.c   | 203 ++--
>>  11 files changed, 1177 insertions(+), 431 deletions(-)
>>  create mode 100644 block/dirty-bitmap.c
>>  create mode 100644 include/block/dirty-bitmap.h
> 
> Thanks, applied patches 1 through 5 to my block tree (because of the
> large code movement in patch 4):
> 
> https://github.com/XanClic/qemu/commits/block
> 
> Max
> 

You're a saint, thank you :)

Re: [Qemu-devel] [PATCH v4 08/26] crypto: add support for the twofish cipher algorithm

[Eric Blake]

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote:
> New cipher algorithms 'twofish-128', 'twofish-192' and
> 'twofish-256' are defined for the Twofish algorithm.
> The gcrypt backend does not support 'twofish-192'.
> 
> The nettle and gcrypt cipher backends are updated to
> support the new cipher and a test vector added to the
> cipher test suite. The new algorithm is enabled in the
> LUKS block encryption driver.
> 
> Signed-off-by: Daniel P. Berrange 
> ---

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 07/26] crypto: add support for the serpent cipher algorithm

[Eric Blake]

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote:
> New cipher algorithms 'serpent-128', 'serpent-192' and
> 'serpent-256' are defined for the Serpent algorithm.
> 
> The nettle and gcrypt cipher backends are updated to
> support the new cipher and a test vector added to the
> cipher test suite. The new algorithm is enabled in the
> LUKS block encryption driver.
> 
> Signed-off-by: Daniel P. Berrange 
> ---

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 06/26] crypto: add support for the cast5-128 cipher algorithm

[Eric Blake]

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote:
> A new cipher algorithm 'cast-5-128' is defined for the
> Cast-5 algorithm with 128 bit key size. Smaller key sizes
> are supported by Cast-5, but nothing in QEMU should use
> them, so only 128 bit keys are permitted.
> 
> The nettle and gcrypt cipher backends are updated to
> support the new cipher and a test vector added to the
> cipher test suite. The new algorithm is enabled in the
> LUKS block encryption driver.
> 
> Signed-off-by: Daniel P. Berrange 
> ---

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 05/26] crypto: skip testing of unsupported cipher algorithms

[Eric Blake]

On 02/29/2016 05:00 AM, Daniel P. Berrange wrote:
> We don't guarantee that all crypto backends will support
> all cipher algorithms, so we should skip tests unless
> the crypto backend indicates support.
> 
> Signed-off-by: Daniel P. Berrange 
> ---
>  tests/test-crypto-cipher.c | 4 +++-
>  1 file changed, 3 insertions(+), 1 deletion(-)

Reviewed-by: Eric Blake 

> 
> diff --git a/tests/test-crypto-cipher.c b/tests/test-crypto-cipher.c
> index 9f912ec..7a073e9 100644
> --- a/tests/test-crypto-cipher.c
> +++ b/tests/test-crypto-cipher.c
> @@ -380,7 +380,9 @@ int main(int argc, char **argv)
>  g_assert(qcrypto_init(NULL) == 0);
>  
>  for (i = 0; i < G_N_ELEMENTS(test_data); i++) {
> -g_test_add_data_func(test_data[i].path, _data[i], test_cipher);
> +if (qcrypto_cipher_supports(test_data[i].alg)) {
> +g_test_add_data_func(test_data[i].path, _data[i], 
> test_cipher);
> +}
>  }
>  
>  g_test_add_func("/crypto/cipher/null-iv",
> 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v2 00/24] target-sparc improvements

[Artyom Tarasenko]

On Fri, Feb 26, 2016 at 10:44 AM, Mark Cave-Ayland
 wrote:
> On 23/02/16 21:11, Richard Henderson wrote:
>
>> The primary focus of this patch set is to reduce the number of
>> helpers that modify TCG globals, and thus increase the lifetime
>> of those globals within each TB, and thus decrease the number
>> of times that tcg must spill and fill them from backing store.
>>
>> As a byproduct, I also implement the bulk of the interesting v9 ASIs
>> inline, thus exposing e.g. the little-endian loads and stores as
>> simple tcg operations.
>>
>> The patch set is relative to my outstanding tcg pull request.
>> For reference, the complete tree can be found at
>>
>>   git://github.com/rth7680/qemu.git tgt-sparc
>>
>> Changes from v1 to v2:
>>   * Commit message refers to UA2005 instead of UA2011 when
>> introducing new asi.h defines. (Artyom)
>>   * Drop the MMU_REAL_IDX, and inline handling of ASI_REAL_*.
>> This appears to be the source of the regression that Artyom
>> identified wrt ss5 emulation.
>>
>>
>> r~
>>
>>
>> Richard Henderson (24):
>>   target-sparc: Mark more flags for helpers
>>   target-sparc: Remove softint as a TCG global
>>   target-sparc: Store mmu index in TB flags
>>   target-sparc: Create gen_exception
>>   target-sparc: Unify asi handling between 32 and 64-bit
>>   target-sparc: Store %asi in TB flags
>>   target-sparc: Introduce get_asi
>>   target-sparc: Pass TCGMemOp to gen_ld/st_asi
>>   target-sparc: Import linux/arch/sparc/include/uapi/asm/asi.h
>>   target-sparc: Add UA2005 defines to asi.h
>>   target-sparc: Use defines from asi.h
>>   target-sparc: Directly implement easy ld/st asis
>>   target-sparc: Use QT0 to return results from ldda
>>   target-sparc: Introduce gen_check_align
>>   target-sparc: Directly implement easy ldd/std asis
>>   target-sparc: Fix obvious error in ASI_M_BFILL
>>   target-sparc: Pass TCGMemOp constants to helper_ld/st_asi
>>   target-sparc: Directly implement easy ldf/stf asis
>>   target-sparc: Directly implement block and short ldf/stf asis
>>   target-sparc: Remove helper_ldf_asi, helper_stf_asi
>>   target-sparc: Use explicit writes to cpu_fsr
>>   target-sparc: Use cpu_fsr in stfsr
>>   target-sparc: Use cpu_loop_exit_restore from
>> helper_check_ieee_exceptions
>>   target-sparc: Elide duplicate updates to fprs
>>
>>  target-sparc/asi.h |  311 +++
>>  target-sparc/cpu.h |   28 +-
>>  target-sparc/fop_helper.c  |  229 +++-
>>  target-sparc/helper.h  |  168 +++---
>>  target-sparc/ldst_helper.c |  696 +++-
>>  target-sparc/translate.c   | 1250 
>> +++-
>>  6 files changed, 1580 insertions(+), 1102 deletions(-)
>>  create mode 100644 target-sparc/asi.h
>
> Hi Richard,
>
> I've just applied this to git master and run through my local tests and
> while I see no problems with qemu-system-sparc,

Can confirm this. 32 bit code passes my tests too (including running a
Java-written installer for a proprietary database).
Concerning the regression reported by Mark: it looks like asi 0xe0 is broken.

Artyom

> I do see a regression
> with qemu-system-sparc64 when trying to boot my Debian test image:
>
>
> $ ./qemu-system-sparc64 -cdrom debian-7.8.0-sparc-netinst.iso -boot d
> -nographic
>
> OpenBIOS for Sparc64
> Configuration device id QEMU version 1 machine id 0
> kernel cmdline
> CPUs: 1 x SUNW,UltraSPARC-IIi
> UUID: ----
> Welcome to OpenBIOS v1.1 built on Feb 4 2016 10:29
>   Type 'help' for detailed information
> Trying cdrom:f...
> Not a bootable ELF image
> Loading a.out image...
> Loaded 7680 bytes
> entry point is 0x4000
>
> Jumping to entry point 4000 for type 0005...
> switching to new context: entry point 0x4000 stack 0xffe84a09
> SILO Version 1.4.14
> EXT2 superblock magic is wrong
> EXT2 superblock magic is wrong
> \
>
>
>   Welcome to Debian GNU/Linux wheezy!
>
> This is a Debian installation CDROM, built on 20150110-20:41.
> Keep it once you have installed your system, as you can boot from it
> to repair the system on your hard disk if that ever becomes necessary.
>
> WARNING: You should completely back up all of your hard disks before
>   proceeding. The installation procedure can completely and irreversibly
>   erase them! If you haven't made backups yet, remove the rescue CD from
>   the drive and press L1-A to get back to the OpenBoot prompt.
>
> Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted
> by applicable law.
>
> [ ENTER - Boot install ]   [ Type "expert" - Boot into expert mode ]
>[ Type "rescue" - Boot into rescue mode ]
> boot:
> Allocated 64 Megs of memory at 0x4000 for kernel
> EXT2 superblock magic is wrong
> Loaded kernel version 3.2.65
> EXT2 superblock magic is wrong
> Loading initial ramdisk (5047556 bytes at 0x440 phys, 0x40C0
> virt)...
> -
> 

[Qemu-devel] [PATCH] spapr_rng: fix race with main loop

[Greg Kurz]

Since commit "60253ed1e6ec rng: add request queue support to rng-random",
the use of a spapr_rng device may hang vCPU threads.

The following path is taken without holding the lock to the main loop mutex:

h_random()
  rng_backend_request_entropy()
rng_random_request_entropy()
  qemu_set_fd_handler()

The consequence is that entropy_available() may be called before the vCPU
thread could even queue the request: depending on the scheduling, it may
happen that entropy_available() does not call random_recv()->qemu_sem_post().
The vCPU thread will then sleep forever in h_random()->qemu_sem_wait().

This could not happen before 60253ed1e6ec because entropy_available() used
to call random_recv() unconditionally.

This patch ensures the lock is held to avoid the race.

Signed-off-by: Greg Kurz 
---

Thomas,

This is the problem mentioned by Cedric in:

https://lists.nongnu.org/archive/html/qemu-devel/2016-03/msg02526.html

Cheers.

--
Greg

 hw/ppc/spapr_rng.c |4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/hw/ppc/spapr_rng.c b/hw/ppc/spapr_rng.c
index a39d472b66fd..02d6be49f58e 100644
--- a/hw/ppc/spapr_rng.c
+++ b/hw/ppc/spapr_rng.c
@@ -77,13 +77,13 @@ static target_ulong h_random(PowerPCCPU *cpu, 
sPAPRMachineState *spapr,
 hrdata.val.v64 = 0;
 hrdata.received = 0;
 
-qemu_mutex_unlock_iothread();
 while (hrdata.received < 8) {
 rng_backend_request_entropy(rngstate->backend, 8 - hrdata.received,
 random_recv, );
+qemu_mutex_unlock_iothread();
 qemu_sem_wait();
+qemu_mutex_lock_iothread();
 }
-qemu_mutex_lock_iothread();
 
 qemu_sem_destroy();
 args[0] = hrdata.val.v64;

[Qemu-devel] [PULL 2/2] kvm: Remove x2apic feature from CPU model when kernel_irqchip is off

[Eduardo Habkost]

From: Lan Tianyu 

x2apic feature is in the kvm_default_props and automatically added to all
CPU models when KVM is enabled. But userspace devices don't support x2apic
which can't be enabled without the in-kernel irqchip. It will trigger
warning of "host doesn't support requested feature: CPUID.01H:ECX.x2apic
[bit 21]" when kernel_irqchip is off. This patch is to fix it via removing
x2apic feature when kernel_irqchip is off.

Signed-off-by: Lan Tianyu 
Acked-by: Paolo Bonzini 
Reviewed-by: Eduardo Habkost 
Signed-off-by: Eduardo Habkost 
---
 target-i386/cpu.c | 4 
 1 file changed, 4 insertions(+)

diff --git a/target-i386/cpu.c b/target-i386/cpu.c
index 0f38d1e..3ea6b29 100644
--- a/target-i386/cpu.c
+++ b/target-i386/cpu.c
@@ -2132,6 +2132,10 @@ static void x86_cpu_load_def(X86CPU *cpu, 
X86CPUDefinition *def, Error **errp)
 
 /* Special cases not set in the X86CPUDefinition structs: */
 if (kvm_enabled()) {
+if (!kvm_irqchip_in_kernel()) {
+x86_cpu_change_kvm_default("x2apic", "off");
+}
+
 x86_cpu_apply_props(cpu, kvm_default_props);
 }
 
-- 
2.1.0

[Qemu-devel] [PULL 1/2] hyperv: cpu hotplug fix with HyperV enabled

[Eduardo Habkost]

From: "Denis V. Lunev" 

With Hyper-V enabled CPU hotplug stops working. The CPU appears
in device manager on Windows but does not appear in peformance
monitor and control panel.

The root of the problem is the following. Windows checks
HV_X64_CPU_DYNAMIC_PARTITIONING_AVAILABLE bit in CPUID. The
presence of this bit is enough to cure the situation.

The bit should be set when CPU hotplug is allowed for HyperV VM.
The check that hot_add_cpu callback is defined is enough from the
protocol point of view. Though this callback is defined almost
always thus there is no need to export that knowledge in the
other way.

Signed-off-by: Denis V. Lunev 
Reviewed-by: Roman Kagan 
CC: Paolo Bonzini 
CC: Richard Henderson 
CC: Eduardo Habkost 
CC: "Andreas Färber" 
Reviewed-by: Eduardo Habkost 
Signed-off-by: Eduardo Habkost 
---
 roms/SLOF | 2 +-
 roms/openbios | 2 +-
 target-i386/kvm.c | 1 +
 3 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/roms/SLOF b/roms/SLOF
index e3d0572..b4c9380 16
--- a/roms/SLOF
+++ b/roms/SLOF
@@ -1 +1 @@
-Subproject commit e3d05727a074619fc12d0a67f05cf2c42c875cce
+Subproject commit b4c93802a5b2c72f096649c497ec9ff5708e4456
diff --git a/roms/openbios b/roms/openbios
index 0dbda5d..bd95e4c 16
--- a/roms/openbios
+++ b/roms/openbios
@@ -1 +1 @@
-Subproject commit 0dbda5d935f95391d16431cd3c079fbf53d668df
+Subproject commit bd95e4c193905d5ed867e96f1a720ce4cb53b59f
diff --git a/target-i386/kvm.c b/target-i386/kvm.c
index 7974acb..08d6444 100644
--- a/target-i386/kvm.c
+++ b/target-i386/kvm.c
@@ -639,6 +639,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
 if (cpu->hyperv_crash && has_msr_hv_crash) {
 c->edx |= HV_X64_GUEST_CRASH_MSR_AVAILABLE;
 }
+c->edx |= HV_X64_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
 if (cpu->hyperv_reset && has_msr_hv_reset) {
 c->eax |= HV_X64_MSR_RESET_AVAILABLE;
 }
-- 
2.1.0

[Qemu-devel] [PULL 0/2] X86 fixes

[Eduardo Habkost]

The following changes since commit a648c137383d84bc4f95696e5293978d9541a26e:

  Merge remote-tracking branch 'remotes/kraxel/tags/pull-ui-20160309-1' into 
staging (2016-03-10 02:51:14 +)

are available in the git repository at:

  git://github.com/ehabkost/qemu.git tags/x86-pull-request

for you to fetch changes up to b04c3b6ad8b3e802fe7ad7a3ff5a9ab980d22578:

  kvm: Remove x2apic feature from CPU model when kernel_irqchip is off 
(2016-03-11 15:43:48 -0300)


X86 fixes



Denis V. Lunev (1):
  hyperv: cpu hotplug fix with HyperV enabled

Lan Tianyu (1):
  kvm: Remove x2apic feature from CPU model when kernel_irqchip is off

 roms/SLOF | 2 +-
 roms/openbios | 2 +-
 target-i386/cpu.c | 4 
 target-i386/kvm.c | 1 +
 4 files changed, 7 insertions(+), 2 deletions(-)

-- 
2.1.0

Re: [Qemu-devel] [PATCH] char: translate from QIOChannel error to errno

[Daniel P. Berrange]

On Fri, Mar 11, 2016 at 06:55:24PM +0100, marcandre.lur...@redhat.com wrote:
> From: Marc-André Lureau 
> 
> Caller of CharDriverState.chr* callback assume errno error conventions.
> Translate QIOChannel error to errno (this fixes potential EAGAIN
> regression, for ex if a vhost-user backend block, qemu_chr_fe_read_all()
> could get error -2 and not wait)
> 
> Signed-off-by: Marc-André Lureau 

Reviewed-by: Daniel P. Berrange 

> ---
>  qemu-char.c | 7 +++
>  1 file changed, 7 insertions(+)
> 
> diff --git a/qemu-char.c b/qemu-char.c
> index ad11b75..4317388 100644
> --- a/qemu-char.c
> +++ b/qemu-char.c
> @@ -2727,6 +2727,13 @@ static ssize_t tcp_chr_recv(CharDriverState *chr, char 
> *buf, size_t len)
>   NULL);
>  }
>  
> +if (ret == QIO_CHANNEL_ERR_BLOCK) {
> +errno = EAGAIN;
> +ret = -1;
> +} else if (ret == -1) {
> +errno = EIO;
> +}
> +
>  if (msgfds_num) {
>  /* close and clean read_msgfds */
>  for (i = 0; i < s->read_msgfds_num; i++) {

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

Re: [Qemu-devel] [PATCH v2] hw/i386: fix unbounded stack for load_multiboot

[Eduardo Habkost]

On Wed, Mar 09, 2016 at 04:30:10PM +0800, Peter Xu wrote:
> Use heap rather than stack for kcmdline.
> 
> Signed-off-by: Peter Xu 

Reviewed-by: Eduardo Habkost 

I believe it can go through the PC tree.

-- 
Eduardo

Re: [Qemu-devel] [RFC PATCH v2 3/3] VFIO: Type1 IOMMU mapping support for vGPU

[Neo Jia]

On Fri, Mar 11, 2016 at 10:56:24AM -0700, Alex Williamson wrote:
> On Fri, 11 Mar 2016 08:55:44 -0800
> Neo Jia  wrote:
> 
> > > > Alex, what's your opinion on this?  
> > > 
> > > The sticky point is how vfio, which is only handling the vGPU, has a
> > > reference to the physical GPU on which to call DMA API operations.  If
> > > that reference is provided by the vendor vGPU driver, for example
> > > vgpu_dma_do_translate_for_pci(gpa, pci_dev), I don't see any reason to
> > > be opposed to such an API.  I would not condone vfio deriving or owning
> > > a reference to the physical device on its own though, that's in the
> > > realm of the vendor vGPU driver.  It does seem a bit cleaner and should
> > > reduce duplicate code if the vfio vGPU iommu interface could handle the
> > > iommu mapping for the vendor vgpu driver when necessary.  Thanks,  
> > 
> > Hi Alex,
> > 
> > Since we don't want to allow vfio iommu to derive or own a reference to the
> > physical device, I think it is still better not providing such pci_dev to 
> > the 
> > vfio iommu type1 driver.
> > 
> > Also, I need to point out that if the vfio iommu is going to set up iommu 
> > page
> > table for the real underlying physical device, giving the fact of single 
> > RID we
> > are all having here, the iommu mapping code has to return the new "IOVA" 
> > that is
> > mapped to the HPA, which the GPU vendro driver will have to put on its DMA
> > engine. This is very different than the current VFIO IOMMU mapping logic.
> > 
> > And we still have to provide another interface to translate the GPA to
> > HPA for CPU mapping.
> > 
> > In the current RFC, we only need to have a single interface to provide the 
> > most
> > basic information to the GPU vendor driver and without taking the risk of
> > leaking a ref to VFIO IOMMU.
> 
> I don't see this as some fundamental difference of opinion, it's really
> just whether vfio provides a "pin this GFN and return the HPA" function
> or whether that function could be extended to include "... and also map
> it through the DMA API for the provided device and return the host
> IOVA".  It might even still be a single function to vfio for CPU vs
> device mapping where the device and IOVA return pointer are NULL when
> only pinning is required for CPU access (though maybe there are better
> ways to provide CPU access than pinning).  A wrapper could even give the
> appearance that those are two separate functions.
> 
> So long as vfio isn't owning or deriving the device for the DMA API
> calls and we don't introduce some complication in page accounting, this
> really just seems like a question of whether moving the DMA API
> handling into vfio is common between the vendor vGPU drivers and are we
> reducing the overall amount and complexity of code by giving the vendor
> drivers the opportunity to do both operations with one interface.

Hi Alex,

OK, I will look into of adding such facilitation and probably include it in a
bit later rev of VGPU IOMMU if we don't run any surprise or the issues you
mentioned above.

Thanks,
Neo

> If as Kevin suggest it also provides some additional abstractions
> for Xen vs KVM, even better.  Thanks,
> 
> Alex

Re: [Qemu-devel] [RFC PATCH v2 3/3] VFIO: Type1 IOMMU mapping support for vGPU

[Alex Williamson]

On Fri, 11 Mar 2016 08:55:44 -0800
Neo Jia  wrote:

> On Fri, Mar 11, 2016 at 09:13:15AM -0700, Alex Williamson wrote:
> > On Fri, 11 Mar 2016 04:46:23 +
> > "Tian, Kevin"  wrote:
> >   
> > > > From: Neo Jia [mailto:c...@nvidia.com]
> > > > Sent: Friday, March 11, 2016 12:20 PM
> > > > 
> > > > On Thu, Mar 10, 2016 at 11:10:10AM +0800, Jike Song wrote:
> > > > >
> > > > > >> Is it supposed to be the caller who should set
> > > > > >> up IOMMU by DMA api such as dma_map_page(), after calling
> > > > > >> vgpu_dma_do_translate()?
> > > > > >>
> > > > > >
> > > > > > Don't think you need to call dma_map_page here. Once you have the 
> > > > > > pfn available
> > > > > > to your GPU kernel driver, you can just go ahead to setup the 
> > > > > > mapping as you
> > > > > > normally do such as calling pci_map_sg and its friends.
> > > > > >
> > > > >
> > > > > Technically it's definitely OK to call DMA API from the caller rather 
> > > > > than here,
> > > > > however personally I think it is a bit counter-intuitive: IOMMU page 
> > > > > tables
> > > > > should be constructed within the VFIO IOMMU driver.
> > > > >
> > > > 
> > > > Hi Jike,
> > > > 
> > > > For vGPU, what we have is just a virtual device and a fake IOMMU group, 
> > > > therefore
> > > > the actual interaction with the real GPU should be managed by the GPU 
> > > > vendor driver.
> > > > 
> > > 
> > > Hi, Neo,
> > > 
> > > Seems we have a different thought on this. Regardless of whether it's a 
> > > virtual/physical 
> > > device, imo, VFIO should manage IOMMU configuration. The only difference 
> > > is:
> > > 
> > > - for physical device, VFIO directly invokes IOMMU API to set IOMMU entry 
> > > (GPA->HPA);
> > > - for virtual device, VFIO invokes kernel DMA APIs which indirectly lead 
> > > to IOMMU entry 
> > > set if CONFIG_IOMMU is enabled in kernel (GPA->IOVA);
> > > 
> > > This would provide an unified way to manage the translation in VFIO, and 
> > > then vendor
> > > specific driver only needs to query and use returned IOVA corresponding 
> > > to a GPA. 
> > > 
> > > Doing so has another benefit, to make underlying vGPU driver VMM 
> > > agnostic. For KVM,
> > > yes we can use pci_map_sg. However for Xen it's different (today Dom0 
> > > doesn't see
> > > IOMMU. In the future there'll be a PVIOMMU implementation) so different 
> > > code path is 
> > > required. It's better to abstract such specific knowledge out of vGPU 
> > > driver, which just
> > > uses whatever dma_addr returned by other agent (VFIO here, or another Xen 
> > > specific
> > > agent) in a centralized way.
> > > 
> > > Alex, what's your opinion on this?  
> > 
> > The sticky point is how vfio, which is only handling the vGPU, has a
> > reference to the physical GPU on which to call DMA API operations.  If
> > that reference is provided by the vendor vGPU driver, for example
> > vgpu_dma_do_translate_for_pci(gpa, pci_dev), I don't see any reason to
> > be opposed to such an API.  I would not condone vfio deriving or owning
> > a reference to the physical device on its own though, that's in the
> > realm of the vendor vGPU driver.  It does seem a bit cleaner and should
> > reduce duplicate code if the vfio vGPU iommu interface could handle the
> > iommu mapping for the vendor vgpu driver when necessary.  Thanks,  
> 
> Hi Alex,
> 
> Since we don't want to allow vfio iommu to derive or own a reference to the
> physical device, I think it is still better not providing such pci_dev to the 
> vfio iommu type1 driver.
> 
> Also, I need to point out that if the vfio iommu is going to set up iommu page
> table for the real underlying physical device, giving the fact of single RID 
> we
> are all having here, the iommu mapping code has to return the new "IOVA" that 
> is
> mapped to the HPA, which the GPU vendro driver will have to put on its DMA
> engine. This is very different than the current VFIO IOMMU mapping logic.
> 
> And we still have to provide another interface to translate the GPA to
> HPA for CPU mapping.
> 
> In the current RFC, we only need to have a single interface to provide the 
> most
> basic information to the GPU vendor driver and without taking the risk of
> leaking a ref to VFIO IOMMU.

I don't see this as some fundamental difference of opinion, it's really
just whether vfio provides a "pin this GFN and return the HPA" function
or whether that function could be extended to include "... and also map
it through the DMA API for the provided device and return the host
IOVA".  It might even still be a single function to vfio for CPU vs
device mapping where the device and IOVA return pointer are NULL when
only pinning is required for CPU access (though maybe there are better
ways to provide CPU access than pinning).  A wrapper could even give the
appearance that those are two separate functions.

So long as vfio isn't owning or deriving the device for the DMA API
calls and we don't 

[Qemu-devel] [PATCH] char: translate from QIOChannel error to errno

[marcandre . lureau]

From: Marc-André Lureau 

Caller of CharDriverState.chr* callback assume errno error conventions.
Translate QIOChannel error to errno (this fixes potential EAGAIN
regression, for ex if a vhost-user backend block, qemu_chr_fe_read_all()
could get error -2 and not wait)

Signed-off-by: Marc-André Lureau 
---
 qemu-char.c | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/qemu-char.c b/qemu-char.c
index ad11b75..4317388 100644
--- a/qemu-char.c
+++ b/qemu-char.c
@@ -2727,6 +2727,13 @@ static ssize_t tcp_chr_recv(CharDriverState *chr, char 
*buf, size_t len)
  NULL);
 }
 
+if (ret == QIO_CHANNEL_ERR_BLOCK) {
+errno = EAGAIN;
+ret = -1;
+} else if (ret == -1) {
+errno = EIO;
+}
+
 if (msgfds_num) {
 /* close and clean read_msgfds */
 for (i = 0; i < s->read_msgfds_num; i++) {
-- 
2.5.0

[Qemu-devel] [PATCH v4 09/28] migration: add helpers for creating QEMUFile from a QIOChannel

[Daniel P. Berrange]

Currently creating a QEMUFile instance from a QIOChannel is
quite simple only requiring a single call to
qemu_fopen_channel_input or  qemu_fopen_channel_output
depending on the end of migration connection.

When QEMU gains TLS support, however, there will need to be
a TLS negotiation done inbetween creation of the QIOChannel
and creation of the final QEMUFile. Introduce some helper
methods that will encapsulate this logic, isolating the
migration protocol drivers from knowledge about TLS.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/migration.h |  6 ++
 migration/migration.c | 21 +
 2 files changed, 27 insertions(+)

diff --git a/include/migration/migration.h b/include/migration/migration.h
index ac2c12c..e335380 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -179,6 +179,12 @@ void process_incoming_migration(QEMUFile *f);
 
 void qemu_start_incoming_migration(const char *uri, Error **errp);
 
+void migration_set_incoming_channel(MigrationState *s,
+QIOChannel *ioc);
+
+void migration_set_outgoing_channel(MigrationState *s,
+QIOChannel *ioc);
+
 uint64_t migrate_max_downtime(void);
 
 void exec_start_incoming_migration(const char *host_port, Error **errp);
diff --git a/migration/migration.c b/migration/migration.c
index 8fdd630..a4edbe5 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -428,6 +428,27 @@ void process_incoming_migration(QEMUFile *f)
 qemu_coroutine_enter(co, f);
 }
 
+
+void migration_set_incoming_channel(MigrationState *s,
+QIOChannel *ioc)
+{
+QEMUFile *f = qemu_fopen_channel_input(ioc);
+
+process_incoming_migration(f);
+}
+
+
+void migration_set_outgoing_channel(MigrationState *s,
+QIOChannel *ioc)
+{
+QEMUFile *f = qemu_fopen_channel_output(ioc);
+
+s->to_dst_file = f;
+
+migrate_fd_connect(s);
+}
+
+
 /*
  * Send a message on the return channel back to the source
  * of the migration.
-- 
2.5.0

Re: [Qemu-devel] [PATCH] BitsSizedTasks-LargeFrames

[Eric Blake]

On 03/11/2016 08:50 AM, Bastian Koppelmann wrote:
> Hi Siddharth,
> 
> thanks for the patch. Can you resend it inlined into the e-mail instead
> of an attachment? I usually use git send-email for that.
> My workflow (once you have configured git send-email) would be:
> 
> - git format-patch -1
> - git send-email --to="qemu-devel@nongnu.org"
> --cc="qemu-triv...@nongnu.org" *.patch
> 
> Also there is a page on the wiki on how to submit a patch [1]. If you
> want to avoid a lot of re-sending you might want to look into it :).
> 
> Cheers,
> Bastian
> 
> [1] http://wiki.qemu.org/Contribute/SubmitAPatch

Also, there's a typo in your subject line (should be 'bitesized', not
'bitssized'). And don't forget to add your Signed-off-by.


-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH v4 06/28] migration: introduce set_blocking function in QEMUFileOps

[Daniel P. Berrange]

Remove the assumption that every QEMUFile implementation has
a file descriptor available by introducing a new function
in QEMUFileOps to change the blocking state of a QEMUFile.

If not set, it will fallback to the original code using
the get_fd method.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/qemu-file.h |  5 +
 migration/migration.c |  4 +---
 migration/qemu-file.c | 10 +++---
 3 files changed, 13 insertions(+), 6 deletions(-)

diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index 1934a64..2dea81f 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -54,6 +54,10 @@ typedef int (QEMUFileCloseFunc)(void *opaque);
  */
 typedef int (QEMUFileGetFD)(void *opaque);
 
+/* Called to change the blocking mode of the file
+ */
+typedef int (QEMUFileSetBlocking)(void *opaque, bool enabled);
+
 /*
  * This function writes an iovec to file. The handler must write all
  * of the data or return a negative errno value.
@@ -107,6 +111,7 @@ typedef struct QEMUFileOps {
 QEMUFileGetBufferFunc *get_buffer;
 QEMUFileCloseFunc *close;
 QEMUFileGetFD *get_fd;
+QEMUFileSetBlocking *set_blocking;
 QEMUFileWritevBufferFunc *writev_buffer;
 QEMURetPathFunc *get_return_path;
 QEMUFileShutdownFunc *shut_down;
diff --git a/migration/migration.c b/migration/migration.c
index 7d13377..942c22d 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -422,11 +422,9 @@ static void process_incoming_migration_co(void *opaque)
 void process_incoming_migration(QEMUFile *f)
 {
 Coroutine *co = qemu_coroutine_create(process_incoming_migration_co);
-int fd = qemu_get_fd(f);
 
-assert(fd != -1);
 migrate_decompress_threads_create();
-qemu_set_nonblock(fd);
+qemu_file_set_blocking(f, false);
 qemu_coroutine_enter(co, f);
 }
 
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index b480b72..2b25dec 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -684,9 +684,13 @@ size_t qemu_get_counted_string(QEMUFile *f, char buf[256])
  */
 void qemu_file_set_blocking(QEMUFile *f, bool block)
 {
-if (block) {
-qemu_set_block(qemu_get_fd(f));
+if (f->ops->set_blocking) {
+f->ops->set_blocking(f->opaque, block);
 } else {
-qemu_set_nonblock(qemu_get_fd(f));
+if (block) {
+qemu_set_block(qemu_get_fd(f));
+} else {
+qemu_set_nonblock(qemu_get_fd(f));
+}
 }
 }
-- 
2.5.0

[Qemu-devel] [PATCH v4 14/28] migration: convert tcp socket protocol to use QIOChannel

[Daniel P. Berrange]

Drop the current TCP socket migration driver and extend
the new generic socket driver to cope with the TCP address
format

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 migration/Makefile.objs |   2 +-
 migration/socket.c  |  31 +++
 migration/tcp.c | 103 
 3 files changed, 32 insertions(+), 104 deletions(-)
 delete mode 100644 migration/tcp.c

diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index 9d83997..59e8c09 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -1,4 +1,4 @@
-common-obj-y += migration.o tcp.o socket.o
+common-obj-y += migration.o socket.o
 common-obj-y += vmstate.o
 common-obj-y += qemu-file.o qemu-file-buf.o qemu-file-unix.o qemu-file-stdio.o
 common-obj-y += qemu-file-channel.o
diff --git a/migration/socket.c b/migration/socket.c
index ef0a673..7e0d9ee 100644
--- a/migration/socket.c
+++ b/migration/socket.c
@@ -24,6 +24,23 @@
 #include "trace.h"
 
 
+static SocketAddress *tcp_build_address(const char *host_port, Error **errp)
+{
+InetSocketAddress *iaddr = inet_parse(host_port, errp);
+SocketAddress *saddr;
+
+if (!iaddr) {
+return NULL;
+}
+
+saddr = g_new0(SocketAddress, 1);
+saddr->type = SOCKET_ADDRESS_KIND_INET;
+saddr->u.inet = iaddr;
+
+return saddr;
+}
+
+
 static SocketAddress *unix_build_address(const char *path)
 {
 SocketAddress *saddr;
@@ -68,6 +85,14 @@ static void socket_start_outgoing_migration(MigrationState 
*s,
 qapi_free_SocketAddress(saddr);
 }
 
+void tcp_start_outgoing_migration(MigrationState *s,
+  const char *host_port,
+  Error **errp)
+{
+SocketAddress *saddr = tcp_build_address(host_port, errp);
+socket_start_outgoing_migration(s, saddr, errp);
+}
+
 void unix_start_outgoing_migration(MigrationState *s,
const char *path,
Error **errp)
@@ -124,6 +149,12 @@ static void socket_start_incoming_migration(SocketAddress 
*saddr,
 qapi_free_SocketAddress(saddr);
 }
 
+void tcp_start_incoming_migration(const char *host_port, Error **errp)
+{
+SocketAddress *saddr = tcp_build_address(host_port, errp);
+socket_start_incoming_migration(saddr, errp);
+}
+
 void unix_start_incoming_migration(const char *path, Error **errp)
 {
 SocketAddress *saddr = unix_build_address(path);
diff --git a/migration/tcp.c b/migration/tcp.c
deleted file mode 100644
index 48904e0..000
--- a/migration/tcp.c
+++ /dev/null
@@ -1,103 +0,0 @@
-/*
- * QEMU live migration
- *
- * Copyright IBM, Corp. 2008
- *
- * Authors:
- *  Anthony Liguori   
- *
- * This work is licensed under the terms of the GNU GPL, version 2.  See
- * the COPYING file in the top-level directory.
- *
- * Contributions after 2012-01-13 are licensed under the terms of the
- * GNU GPL, version 2 or (at your option) any later version.
- */
-
-#include "qemu/osdep.h"
-
-#include "qemu-common.h"
-#include "qemu/error-report.h"
-#include "qemu/sockets.h"
-#include "migration/migration.h"
-#include "migration/qemu-file.h"
-#include "block/block.h"
-#include "qemu/main-loop.h"
-
-//#define DEBUG_MIGRATION_TCP
-
-#ifdef DEBUG_MIGRATION_TCP
-#define DPRINTF(fmt, ...) \
-do { printf("migration-tcp: " fmt, ## __VA_ARGS__); } while (0)
-#else
-#define DPRINTF(fmt, ...) \
-do { } while (0)
-#endif
-
-static void tcp_wait_for_connect(int fd, Error *err, void *opaque)
-{
-MigrationState *s = opaque;
-
-if (fd < 0) {
-DPRINTF("migrate connect error: %s\n", error_get_pretty(err));
-s->to_dst_file = NULL;
-migrate_fd_error(s, err);
-} else {
-DPRINTF("migrate connect success\n");
-s->to_dst_file = qemu_fopen_socket(fd, "wb");
-migrate_fd_connect(s);
-}
-}
-
-void tcp_start_outgoing_migration(MigrationState *s, const char *host_port, 
Error **errp)
-{
-inet_nonblocking_connect(host_port, tcp_wait_for_connect, s, errp);
-}
-
-static void tcp_accept_incoming_migration(void *opaque)
-{
-struct sockaddr_in addr;
-socklen_t addrlen = sizeof(addr);
-int s = (intptr_t)opaque;
-QEMUFile *f;
-int c, err;
-
-do {
-c = qemu_accept(s, (struct sockaddr *), );
-err = socket_error();
-} while (c < 0 && err == EINTR);
-qemu_set_fd_handler(s, NULL, NULL, NULL);
-closesocket(s);
-
-DPRINTF("accepted migration\n");
-
-if (c < 0) {
-error_report("could not accept migration connection (%s)",
- strerror(err));
-return;
-}
-
-f = qemu_fopen_socket(c, "rb");
-if (f == NULL) {
-error_report("could not qemu_fopen socket");
-goto out;
-}
-
-process_incoming_migration(f);
-return;
-
-out:
-closesocket(c);
-}
-
-void tcp_start_incoming_migration(const char 

[Qemu-devel] [PATCH v4 19/28] migration: delete QEMUFile buffer implementation

[Daniel P. Berrange]

The qemu_bufopen() method is no longer used, so the memory
buffer based QEMUFile backend can be deleted entirely.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/qemu-file.h |  6 ---
 migration/qemu-file-buf.c | 96 ---
 2 files changed, 102 deletions(-)

diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index 0329ccc..6618d19 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -140,7 +140,6 @@ QEMUFile *qemu_fopen_socket(int fd, const char *mode);
 QEMUFile *qemu_fopen_channel_input(QIOChannel *ioc);
 QEMUFile *qemu_fopen_channel_output(QIOChannel *ioc);
 QEMUFile *qemu_popen_cmd(const char *command, const char *mode);
-QEMUFile *qemu_bufopen(const char *mode, QEMUSizedBuffer *input);
 void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks);
 int qemu_get_fd(QEMUFile *f);
 int qemu_fclose(QEMUFile *f);
@@ -166,11 +165,6 @@ ssize_t qsb_write_at(QEMUSizedBuffer *qsb, const uint8_t 
*buf,
  off_t pos, size_t count);
 
 
-/*
- * For use on files opened with qemu_bufopen
- */
-const QEMUSizedBuffer *qemu_buf_get(QEMUFile *f);
-
 static inline void qemu_put_ubyte(QEMUFile *f, unsigned int v)
 {
 qemu_put_byte(f, (int)v);
diff --git a/migration/qemu-file-buf.c b/migration/qemu-file-buf.c
index 7b8e78e..668ab35 100644
--- a/migration/qemu-file-buf.c
+++ b/migration/qemu-file-buf.c
@@ -366,99 +366,3 @@ ssize_t qsb_write_at(QEMUSizedBuffer *qsb, const uint8_t 
*source,
 
 return count;
 }
-
-typedef struct QEMUBuffer {
-QEMUSizedBuffer *qsb;
-QEMUFile *file;
-bool qsb_allocated;
-} QEMUBuffer;
-
-static ssize_t buf_get_buffer(void *opaque, uint8_t *buf, int64_t pos,
-  size_t size)
-{
-QEMUBuffer *s = opaque;
-ssize_t len = qsb_get_length(s->qsb) - pos;
-
-if (len <= 0) {
-return 0;
-}
-
-if (len > size) {
-len = size;
-}
-return qsb_get_buffer(s->qsb, pos, len, buf);
-}
-
-static ssize_t buf_put_buffer(void *opaque, const uint8_t *buf,
-  int64_t pos, size_t size)
-{
-QEMUBuffer *s = opaque;
-
-return qsb_write_at(s->qsb, buf, pos, size);
-}
-
-static int buf_close(void *opaque)
-{
-QEMUBuffer *s = opaque;
-
-if (s->qsb_allocated) {
-qsb_free(s->qsb);
-}
-
-g_free(s);
-
-return 0;
-}
-
-const QEMUSizedBuffer *qemu_buf_get(QEMUFile *f)
-{
-QEMUBuffer *p;
-
-qemu_fflush(f);
-
-p = f->opaque;
-
-return p->qsb;
-}
-
-static const QEMUFileOps buf_read_ops = {
-.get_buffer = buf_get_buffer,
-.close =  buf_close,
-};
-
-static const QEMUFileOps buf_write_ops = {
-.put_buffer = buf_put_buffer,
-.close =  buf_close,
-};
-
-QEMUFile *qemu_bufopen(const char *mode, QEMUSizedBuffer *input)
-{
-QEMUBuffer *s;
-
-if (mode == NULL || (mode[0] != 'r' && mode[0] != 'w') ||
-mode[1] != '\0') {
-error_report("qemu_bufopen: Argument validity check failed");
-return NULL;
-}
-
-s = g_new0(QEMUBuffer, 1);
-s->qsb = input;
-
-if (s->qsb == NULL) {
-s->qsb = qsb_create(NULL, 0);
-s->qsb_allocated = true;
-}
-if (!s->qsb) {
-g_free(s);
-error_report("qemu_bufopen: qsb_create failed");
-return NULL;
-}
-
-
-if (mode[0] == 'r') {
-s->file = qemu_fopen_ops(s, _read_ops);
-} else {
-s->file = qemu_fopen_ops(s, _write_ops);
-}
-return s->file;
-}
-- 
2.5.0

[Qemu-devel] [PATCH V1 1/2] arm: virt: Add an abstract ARM virt machine type

[Wei Huang]

In preparation for future ARM virt machine types, this patch creates
an abstract type for all ARM machines. The current machine type in
QEMU (i.e. "virt") is renamed to "virt-2.6", whose naming scheme is
similar to other architectures. For the purpose of backward compatibility,
"virt" is converted to an alias, pointing to "virt-2.6". With this patch,
"qemu -M ?" lists the following virtual machine types along with others:

virt QEMU 2.6 ARM Virtual Machine (alias of virt-2.6)
virt-2.6 QEMU 2.6 ARM Virtual Machine

Signed-off-by: Wei Huang 
---
 hw/arm/virt.c | 23 ++-
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 8c6c996..be9bbfb 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1345,6 +1345,19 @@ static void virt_set_gic_version(Object *obj, const char 
*value, Error **errp)
 }
 }
 
+static void virt_machine_class_init(ObjectClass *oc, void *data)
+{
+}
+
+static const TypeInfo virt_machine_info = {
+.name  = TYPE_VIRT_MACHINE,
+.parent= TYPE_MACHINE,
+.abstract  = true,
+.instance_size = sizeof(VirtMachineState),
+.class_size= sizeof(VirtMachineClass),
+.class_init= virt_machine_class_init,
+};
+
 static void virt_instance_init(Object *obj)
 {
 VirtMachineState *vms = VIRT_MACHINE(obj);
@@ -1382,7 +1395,8 @@ static void virt_class_init(ObjectClass *oc, void *data)
 {
 MachineClass *mc = MACHINE_CLASS(oc);
 
-mc->desc = "ARM Virtual Machine",
+mc->desc = "QEMU 2.6 ARM Virtual Machine";
+mc->alias = "virt";
 mc->init = machvirt_init;
 /* Start max_cpus at the maximum QEMU supports. We'll further restrict
  * it later in machvirt_init, where we have more information about the
@@ -1396,16 +1410,15 @@ static void virt_class_init(ObjectClass *oc, void *data)
 }
 
 static const TypeInfo machvirt_info = {
-.name = TYPE_VIRT_MACHINE,
-.parent = TYPE_MACHINE,
-.instance_size = sizeof(VirtMachineState),
+.name = MACHINE_TYPE_NAME("virt-2.6"),
+.parent = TYPE_VIRT_MACHINE,
 .instance_init = virt_instance_init,
-.class_size = sizeof(VirtMachineClass),
 .class_init = virt_class_init,
 };
 
 static void machvirt_machine_init(void)
 {
+type_register_static(_machine_info);
 type_register_static(_info);
 }
 
-- 
1.8.3.1

[Qemu-devel] [PATCH V1 0/2] Versioning ARM virt machine types

[Wei Huang]

We start to see more features been added to ARM virtual machine models.
For the purpose of backward compatibility (e.g. migration), it is time
to consider versioning machine types for ARM VMs. As a beginning step, this
patchset defines an abstract machine type for ARM VMs. The current
"virt" machine is re-written based on this new abstract type accordingly.
These patches have been verified by booting existing VMs.

RFC->V1:
 * Rename the machine type to "virt-2.6", matching the imminent QEMU version
 * Remove mc->is_default (Peter's comment)

Thanks,
-Wei

Wei Huang (2):
  arm: virt: Add an abstract ARM virt machine type
  arm: virt: Move machine class init code to the abstract machine type

 hw/arm/virt.c | 57 ++---
 1 file changed, 38 insertions(+), 19 deletions(-)

-- 
1.8.3.1

[Qemu-devel] [PATCH V1 2/2] arm: virt: Move machine class init code to the abstract machine type

[Wei Huang]

This patch moves the common class initialization code from
"virt-2.6" to the new abstract class. An empty property is added to
"virt-2.6" machine. In the meanwhile, related funtions are renamed
to "virt_2_6_*" for consistency.

Signed-off-by: Wei Huang 
---
 hw/arm/virt.c | 34 --
 1 file changed, 20 insertions(+), 14 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index be9bbfb..8c3ac0d 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -1347,6 +1347,18 @@ static void virt_set_gic_version(Object *obj, const char 
*value, Error **errp)
 
 static void virt_machine_class_init(ObjectClass *oc, void *data)
 {
+MachineClass *mc = MACHINE_CLASS(oc);
+
+mc->init = machvirt_init;
+/* Start max_cpus at the maximum QEMU supports. We'll further restrict
+ * it later in machvirt_init, where we have more information about the
+ * configuration of the particular instance.
+ */
+mc->max_cpus = MAX_CPUMASK_BITS;
+mc->has_dynamic_sysbus = true;
+mc->block_default_type = IF_VIRTIO;
+mc->no_cdrom = 1;
+mc->pci_allow_0_address = true;
 }
 
 static const TypeInfo virt_machine_info = {
@@ -1358,7 +1370,7 @@ static const TypeInfo virt_machine_info = {
 .class_init= virt_machine_class_init,
 };
 
-static void virt_instance_init(Object *obj)
+static void virt_2_6_instance_init(Object *obj)
 {
 VirtMachineState *vms = VIRT_MACHINE(obj);
 
@@ -1391,29 +1403,23 @@ static void virt_instance_init(Object *obj)
 "Valid values are 2, 3 and host", NULL);
 }
 
-static void virt_class_init(ObjectClass *oc, void *data)
+static void virt_2_6_class_init(ObjectClass *oc, void *data)
 {
 MachineClass *mc = MACHINE_CLASS(oc);
+static GlobalProperty compat_props[] = {
+{ /* end of list */ }
+};
 
 mc->desc = "QEMU 2.6 ARM Virtual Machine";
 mc->alias = "virt";
-mc->init = machvirt_init;
-/* Start max_cpus at the maximum QEMU supports. We'll further restrict
- * it later in machvirt_init, where we have more information about the
- * configuration of the particular instance.
- */
-mc->max_cpus = MAX_CPUMASK_BITS;
-mc->has_dynamic_sysbus = true;
-mc->block_default_type = IF_VIRTIO;
-mc->no_cdrom = 1;
-mc->pci_allow_0_address = true;
+mc->compat_props = compat_props;
 }
 
 static const TypeInfo machvirt_info = {
 .name = MACHINE_TYPE_NAME("virt-2.6"),
 .parent = TYPE_VIRT_MACHINE,
-.instance_init = virt_instance_init,
-.class_init = virt_class_init,
+.instance_init = virt_2_6_instance_init,
+.class_init = virt_2_6_class_init,
 };
 
 static void machvirt_machine_init(void)
-- 
1.8.3.1

[Qemu-devel] [PATCH v4 15/28] migration: convert fd socket protocol to use QIOChannel

[Daniel P. Berrange]

Convert the fd socket migration protocol driver to use
QIOChannel and QEMUFileChannel, instead of plain sockets
APIs. It can be unconditionally built because the
QIOChannel APIs it uses will take care to report suitable
error messages if needed.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 migration/Makefile.objs |  4 +--
 migration/fd.c  | 76 +++--
 migration/migration.c   |  4 ---
 trace-events|  4 +++
 4 files changed, 35 insertions(+), 53 deletions(-)

diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index 59e8c09..0987bb6 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -1,11 +1,11 @@
-common-obj-y += migration.o socket.o
+common-obj-y += migration.o socket.o fd.o
 common-obj-y += vmstate.o
 common-obj-y += qemu-file.o qemu-file-buf.o qemu-file-unix.o qemu-file-stdio.o
 common-obj-y += qemu-file-channel.o
 common-obj-y += xbzrle.o postcopy-ram.o
 
 common-obj-$(CONFIG_RDMA) += rdma.o
-common-obj-$(CONFIG_POSIX) += exec.o fd.o
+common-obj-$(CONFIG_POSIX) += exec.o
 
 common-obj-y += block.o
 
diff --git a/migration/fd.c b/migration/fd.c
index 085dd7c..1a7fd43 100644
--- a/migration/fd.c
+++ b/migration/fd.c
@@ -1,10 +1,11 @@
 /*
  * QEMU live migration via generic fd
  *
- * Copyright Red Hat, Inc. 2009
+ * Copyright Red Hat, Inc. 2009-2016
  *
  * Authors:
  *  Chris Lalancette 
+ *  Daniel P. Berrange 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
@@ -14,76 +15,57 @@
  */
 
 #include "qemu/osdep.h"
-#include "qemu-common.h"
-#include "qemu/main-loop.h"
-#include "qemu/sockets.h"
 #include "migration/migration.h"
 #include "monitor/monitor.h"
-#include "migration/qemu-file.h"
-#include "block/block.h"
+#include "io/channel-util.h"
+#include "trace.h"
 
-//#define DEBUG_MIGRATION_FD
-
-#ifdef DEBUG_MIGRATION_FD
-#define DPRINTF(fmt, ...) \
-do { printf("migration-fd: " fmt, ## __VA_ARGS__); } while (0)
-#else
-#define DPRINTF(fmt, ...) \
-do { } while (0)
-#endif
-
-static bool fd_is_socket(int fd)
-{
-struct stat stat;
-int ret = fstat(fd, );
-if (ret == -1) {
-/* When in doubt say no */
-return false;
-}
-return S_ISSOCK(stat.st_mode);
-}
 
 void fd_start_outgoing_migration(MigrationState *s, const char *fdname, Error 
**errp)
 {
+QIOChannel *ioc;
 int fd = monitor_get_fd(cur_mon, fdname, errp);
 if (fd == -1) {
 return;
 }
 
-if (fd_is_socket(fd)) {
-s->to_dst_file = qemu_fopen_socket(fd, "wb");
-} else {
-s->to_dst_file = qemu_fdopen(fd, "wb");
+trace_migration_fd_outgoing(fd);
+ioc = qio_channel_new_fd(fd, errp);
+if (!ioc) {
+close(fd);
+return;
 }
 
-migrate_fd_connect(s);
+migration_set_outgoing_channel(s, ioc);
+object_unref(OBJECT(ioc));
 }
 
-static void fd_accept_incoming_migration(void *opaque)
+static gboolean fd_accept_incoming_migration(QIOChannel *ioc,
+ GIOCondition condition,
+ gpointer opaque)
 {
-QEMUFile *f = opaque;
-
-qemu_set_fd_handler(qemu_get_fd(f), NULL, NULL, NULL);
-process_incoming_migration(f);
+migration_set_incoming_channel(migrate_get_current(), ioc);
+object_unref(OBJECT(ioc));
+return FALSE; /* unregister */
 }
 
 void fd_start_incoming_migration(const char *infd, Error **errp)
 {
+QIOChannel *ioc;
 int fd;
-QEMUFile *f;
-
-DPRINTF("Attempting to start an incoming migration via fd\n");
 
 fd = strtol(infd, NULL, 0);
-if (fd_is_socket(fd)) {
-f = qemu_fopen_socket(fd, "rb");
-} else {
-f = qemu_fdopen(fd, "rb");
-}
-if(f == NULL) {
-error_setg_errno(errp, errno, "failed to open the source descriptor");
+trace_migration_fd_incoming(fd);
+
+ioc = qio_channel_new_fd(fd, errp);
+if (!ioc) {
+close(fd);
 return;
 }
 
-qemu_set_fd_handler(fd, fd_accept_incoming_migration, NULL, f);
+qio_channel_add_watch(ioc,
+  G_IO_IN,
+  fd_accept_incoming_migration,
+  NULL,
+  NULL);
 }
diff --git a/migration/migration.c b/migration/migration.c
index ac373c3..bf7a976 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -317,10 +317,8 @@ void qemu_start_incoming_migration(const char *uri, Error 
**errp)
 #endif
 } else if (strstart(uri, "unix:", )) {
 unix_start_incoming_migration(p, errp);
-#if !defined(WIN32)
 } else if (strstart(uri, "fd:", )) {
 fd_start_incoming_migration(p, errp);
-#endif
 } else {
 error_setg(errp, "unknown migration protocol: %s", uri);
 }
@@ -1068,10 +1066,8 @@ void 

Re: [Qemu-devel] [PATCH v5 04/14] qapi: Adjust names of implicit types

[Eric Blake]

On 03/11/2016 12:48 AM, Markus Armbruster wrote:

 time to change our naming convention; we can instead use the 'q_'
 prefix that we reserved for ourselves back in commit 9fb081e0.  As
 long as we don't declare 'empty' or 'obj' ticklish, it shouldn't
 clash with c_name() prepending 'q_' to the user's ticklish names.
>>>
>>> Do we really want to rename :empty?  We're not going to generate C for
>>> it, are we?
>>
>> No, but it was easier to implement .is_implicit() as
>> "name.startswith('q_')" than as "name == ':empty' or
>> name.startswith('q_obj')".  I can stick with :empty if you want a
>> respin, though.
> 
> You avoid complicating .is_implicit() slightly, and you pay for that
> with a bit of patch churn elsewhere.  Sounds justified.
> 
> Is ':empty' the last use of the ':' prefix?

Yes.  And renaming it to 'q_empty' meant I didn't have to add ':' to the
set of characters to be transliterated in c_name().

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 0/4] Implement some QKeyCode support

[Programmingkid]

On Mar 11, 2016, at 12:14 PM, Eric Blake wrote:

> On 03/10/2016 07:24 PM, Programmingkid wrote:
>> This patchset adds QKeyCode support the adb and cocoa code. 
>> 
>> Note: you do not need to be on a Mac to test out the adb.c, qapi-schema.json,
>> and adb-keys.h files. Only the cocoa.m file changes are Mac specific.
>> 
>> If you are using Linux as a guest, then the xev command is what you could 
>> use to
>> test out these patches. For a Mac OS guest the Key Caps application would 
>> help
>> with testing out these patches.
>> 
>> John Arbuckle (4):
>>  cocoa.m
>>  qapi-schema.json
>>  adb-keys.h
>>  adb.c
> 
> This isn't the typical git cover letter - git lists the titles of the
> four patches, not the basenames of the files modified by the patches.
> Also, you still didn't manage to get threading quite right; the series
> ended up as 5 top-level threads, which makes it harder to find things in
> mailers that sort by threads with most recent activity.

My git send-email is broken. I spend hours googling fixes that didn't work. The 
5 top-level issue is because I sent the patches using my emailer. I will 
remember next time to add the titles to the cover letter. 

Re: [Qemu-devel] [PATCH v4 2/4] qapi-schema.json: Add power and keypad equal keys

[Programmingkid]

On Mar 11, 2016, at 12:12 PM, Eric Blake wrote:

> On 03/10/2016 07:28 PM, Programmingkid wrote:
>> Add the power and keypad equal keys. These keys are found on a real Macintosh
>> keyboard.
>> 
>> Signed-off-by: John Arbuckle 
> 
> This looks unchanged from v3:
> 
> https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg01244.html
> 
> which means you could have added my Reviewed-by from that mail series.

I must have forgotten. Sorry. 

[Qemu-devel] [PATCH v3] hw: fix error reporting for missing option ROMs

[Daniel P. Berrange]

If QEMU fails to load any of the VGA ROMs, it prints a message
to stderr and then carries on as if everything was fine, despite
the VGA interface not being functional. This extends the the
various rom_add_*() methods in loader.h to accept a 'Error **errp'
parameter. The VGA device realizefn() impls can now pass in the
errp they already have and get errors reported as fatal problems.

Addition of 'Error **errp' to the load_*() methods in loader.h is
left as an exercise for future interested developers, since it will
require fixing up a great many callers to propagate errors correctly.

Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrange 
---
 hw/core/loader.c| 38 +++---
 hw/display/cirrus_vga.c |  4 +++-
 hw/display/vga-isa.c|  4 +++-
 hw/i386/pc.c|  6 --
 hw/i386/pc_sysfw.c  |  6 --
 hw/misc/sga.c   |  4 +++-
 hw/pci/pci.c|  8 ++--
 include/hw/loader.h | 16 +---
 8 files changed, 55 insertions(+), 31 deletions(-)

diff --git a/hw/core/loader.c b/hw/core/loader.c
index 8e8031c..2c9be4e 100644
--- a/hw/core/loader.c
+++ b/hw/core/loader.c
@@ -142,7 +142,7 @@ int load_image_targphys(const char *filename,
 return -1;
 }
 if (size > 0) {
-rom_add_file_fixed(filename, addr, -1);
+rom_add_file_fixed(filename, addr, -1, NULL);
 }
 return size;
 }
@@ -162,7 +162,7 @@ int load_image_mr(const char *filename, MemoryRegion *mr)
 return -1;
 }
 if (size > 0) {
-if (rom_add_file_mr(filename, mr, -1) < 0) {
+if (rom_add_file_mr(filename, mr, -1, NULL) < 0) {
 return -1;
 }
 }
@@ -831,11 +831,13 @@ static void *rom_set_mr(Rom *rom, Object *owner, const 
char *name)
 
 int rom_add_file(const char *file, const char *fw_dir,
  hwaddr addr, int32_t bootindex,
- bool option_rom, MemoryRegion *mr)
+ bool option_rom, MemoryRegion *mr,
+ Error **errp)
 {
 MachineClass *mc = MACHINE_GET_CLASS(qdev_get_machine());
 Rom *rom;
-int rc, fd = -1;
+int fd = -1;
+ssize_t rc;
 char devpath[100];
 
 rom = g_malloc0(sizeof(*rom));
@@ -847,8 +849,7 @@ int rom_add_file(const char *file, const char *fw_dir,
 
 fd = open(rom->path, O_RDONLY | O_BINARY);
 if (fd == -1) {
-fprintf(stderr, "Could not open option rom '%s': %s\n",
-rom->path, strerror(errno));
+error_setg_file_open(errp, errno, rom->path);
 goto err;
 }
 
@@ -859,8 +860,9 @@ int rom_add_file(const char *file, const char *fw_dir,
 rom->addr = addr;
 rom->romsize  = lseek(fd, 0, SEEK_END);
 if (rom->romsize == -1) {
-fprintf(stderr, "rom: file %-20s: get size error: %s\n",
-rom->name, strerror(errno));
+error_setg_errno(errp, errno,
+ "Could not get size of option rom '%s'",
+ rom->path);
 goto err;
 }
 
@@ -868,9 +870,15 @@ int rom_add_file(const char *file, const char *fw_dir,
 rom->data = g_malloc0(rom->datasize);
 lseek(fd, 0, SEEK_SET);
 rc = read(fd, rom->data, rom->datasize);
-if (rc != rom->datasize) {
-fprintf(stderr, "rom: file %-20s: read error: rc=%d (expected %zd)\n",
-rom->name, rc, rom->datasize);
+if (rc < 0) {
+error_setg_errno(errp, errno,
+ "Could not read option rom '%s'",
+ rom->path);
+goto err;
+} else if (rc != rom->datasize) {
+error_setg_errno(errp, errno,
+ "Short read on option rom '%s' %zd vs %zd",
+ rom->path, rc, rom->datasize);
 goto err;
 }
 close(fd);
@@ -975,14 +983,14 @@ int rom_add_elf_program(const char *name, void *data, 
size_t datasize,
 return 0;
 }
 
-int rom_add_vga(const char *file)
+int rom_add_vga(const char *file, Error **errp)
 {
-return rom_add_file(file, "vgaroms", 0, -1, true, NULL);
+return rom_add_file(file, "vgaroms", 0, -1, true, NULL, errp);
 }
 
-int rom_add_option(const char *file, int32_t bootindex)
+int rom_add_option(const char *file, int32_t bootindex, Error **errp)
 {
-return rom_add_file(file, "genroms", 0, bootindex, true, NULL);
+return rom_add_file(file, "genroms", 0, bootindex, true, NULL, errp);
 }
 
 static void rom_reset(void *unused)
diff --git a/hw/display/cirrus_vga.c b/hw/display/cirrus_vga.c
index 57b91a7..7fbb2b0 100644
--- a/hw/display/cirrus_vga.c
+++ b/hw/display/cirrus_vga.c
@@ -2977,7 +2977,9 @@ static void isa_cirrus_vga_realizefn(DeviceState *dev, 
Error **errp)
isa_address_space(isadev),
isa_address_space_io(isadev));
 s->con = graphic_console_init(dev, 0, s->hw_ops, s);
-rom_add_vga(VGABIOS_CIRRUS_FILENAME);
+if 

Re: [Qemu-devel] [PATCH v4 0/4] Implement some QKeyCode support

[Eric Blake]

On 03/10/2016 07:24 PM, Programmingkid wrote:
> This patchset adds QKeyCode support the adb and cocoa code. 
> 
> Note: you do not need to be on a Mac to test out the adb.c, qapi-schema.json,
> and adb-keys.h files. Only the cocoa.m file changes are Mac specific.
> 
> If you are using Linux as a guest, then the xev command is what you could use 
> to
> test out these patches. For a Mac OS guest the Key Caps application would help
> with testing out these patches.
> 
> John Arbuckle (4):
>   cocoa.m
>   qapi-schema.json
>   adb-keys.h
>   adb.c

This isn't the typical git cover letter - git lists the titles of the
four patches, not the basenames of the files modified by the patches.
Also, you still didn't manage to get threading quite right; the series
ended up as 5 top-level threads, which makes it harder to find things in
mailers that sort by threads with most recent activity.

> 
>  hw/input/adb.c  | 223 ---
>  include/hw/input/adb-keys.h | 146 
>  qapi-schema.json|   3 +-
>  ui/cocoa.m  | 317 
> 
>  4 files changed, 466 insertions(+), 223 deletions(-)
>  create mode 100644 include/hw/input/adb-keys.h
> 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 2/4] qapi-schema.json: Add power and keypad equal keys

[Eric Blake]

On 03/10/2016 07:28 PM, Programmingkid wrote:
> Add the power and keypad equal keys. These keys are found on a real Macintosh
> keyboard.
> 
> Signed-off-by: John Arbuckle 

This looks unchanged from v3:

https://lists.gnu.org/archive/html/qemu-devel/2016-03/msg01244.html

which means you could have added my Reviewed-by from that mail series.


-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH v4 21/28] migration: delete QEMUFile sockets implementation

[Daniel P. Berrange]

Now that the tcp, unix and fd migration backends have converted
to use the QIOChannel based QEMUFile, there is no user remaining
for the sockets based QEMUFile impl and it can be deleted.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/qemu-file.h |   2 -
 migration/Makefile.objs   |   2 +-
 migration/qemu-file-unix.c| 325 --
 3 files changed, 1 insertion(+), 328 deletions(-)
 delete mode 100644 migration/qemu-file-unix.c

diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index edaf598..ba5fe08 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -128,8 +128,6 @@ typedef struct QEMUFileHooks {
 
 QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops);
 QEMUFile *qemu_fopen(const char *filename, const char *mode);
-QEMUFile *qemu_fdopen(int fd, const char *mode);
-QEMUFile *qemu_fopen_socket(int fd, const char *mode);
 QEMUFile *qemu_fopen_channel_input(QIOChannel *ioc);
 QEMUFile *qemu_fopen_channel_output(QIOChannel *ioc);
 QEMUFile *qemu_popen_cmd(const char *command, const char *mode);
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index 3e5a57c..2f9bf12 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -1,6 +1,6 @@
 common-obj-y += migration.o socket.o fd.o exec.o
 common-obj-y += vmstate.o
-common-obj-y += qemu-file.o qemu-file-unix.o qemu-file-stdio.o
+common-obj-y += qemu-file.o qemu-file-stdio.o
 common-obj-y += qemu-file-channel.o
 common-obj-y += xbzrle.o postcopy-ram.o
 
diff --git a/migration/qemu-file-unix.c b/migration/qemu-file-unix.c
deleted file mode 100644
index 61b059b..000
--- a/migration/qemu-file-unix.c
+++ /dev/null
@@ -1,325 +0,0 @@
-/*
- * QEMU System Emulator
- *
- * Copyright (c) 2003-2008 Fabrice Bellard
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to 
deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
- * THE SOFTWARE.
- */
-#include "qemu/osdep.h"
-#include "qemu-common.h"
-#include "qemu/error-report.h"
-#include "qemu/iov.h"
-#include "qemu/sockets.h"
-#include "qemu/coroutine.h"
-#include "migration/qemu-file.h"
-#include "migration/qemu-file-internal.h"
-
-typedef struct QEMUFileSocket {
-int fd;
-QEMUFile *file;
-} QEMUFileSocket;
-
-static ssize_t socket_writev_buffer(void *opaque, struct iovec *iov, int 
iovcnt,
-int64_t pos)
-{
-QEMUFileSocket *s = opaque;
-ssize_t len;
-ssize_t size = iov_size(iov, iovcnt);
-ssize_t offset = 0;
-int err;
-
-while (size > 0) {
-len = iov_send(s->fd, iov, iovcnt, offset, size);
-
-if (len > 0) {
-size -= len;
-offset += len;
-}
-
-if (size > 0) {
-err = socket_error();
-
-if (err != EAGAIN && err != EWOULDBLOCK) {
-error_report("socket_writev_buffer: Got err=%d for (%zu/%zu)",
- err, (size_t)size, (size_t)len);
-/*
- * If I've already sent some but only just got the error, I
- * could return the amount validly sent so far and wait for the
- * next call to report the error, but I'd rather flag the error
- * immediately.
- */
-return -err;
-}
-
-/* Emulate blocking */
-GPollFD pfd;
-
-pfd.fd = s->fd;
-pfd.events = G_IO_OUT | G_IO_ERR;
-pfd.revents = 0;
-TFR(err = g_poll(, 1, -1 /* no timeout */));
-/* Errors other than EINTR intentionally ignored */
-}
- }
-
-return offset;
-}
-
-static int socket_get_fd(void *opaque)
-{
-QEMUFileSocket *s = opaque;
-
-return s->fd;
-}
-
-static ssize_t socket_get_buffer(void *opaque, uint8_t *buf, int64_t pos,
- size_t size)
-{
-QEMUFileSocket *s = 

[Qemu-devel] [PATCH v4 18/28] migration: convert savevm to use QIOChannel for writing to files

[Daniel P. Berrange]

Convert the exec savevm code to use QIOChannel and QEMUFileChannel,
instead of the stdio APIs.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 migration/savevm.c   |  8 +---
 tests/Makefile   |  4 ++--
 tests/test-vmstate.c | 11 ++-
 3 files changed, 17 insertions(+), 6 deletions(-)

diff --git a/migration/savevm.c b/migration/savevm.c
index 93b7454..ab9a4fa 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -51,6 +51,7 @@
 #include "block/snapshot.h"
 #include "block/qapi.h"
 #include "io/channel-buffer.h"
+#include "io/channel-file.h"
 
 
 #ifndef ETH_P_RARP
@@ -2038,6 +2039,7 @@ void hmp_savevm(Monitor *mon, const QDict *qdict)
 void qmp_xen_save_devices_state(const char *filename, Error **errp)
 {
 QEMUFile *f;
+QIOChannelFile *ioc;
 int saved_vm_running;
 int ret;
 
@@ -2045,11 +2047,11 @@ void qmp_xen_save_devices_state(const char *filename, 
Error **errp)
 vm_stop(RUN_STATE_SAVE_VM);
 global_state_store_running();
 
-f = qemu_fopen(filename, "wb");
-if (!f) {
-error_setg_file_open(errp, errno, filename);
+ioc = qio_channel_file_new_path(filename, O_WRONLY | O_CREAT, 0660, errp);
+if (!ioc) {
 goto the_end;
 }
+f = qemu_fopen_channel_output(QIO_CHANNEL(ioc));
 ret = qemu_save_device_state(f);
 qemu_fclose(f);
 if (ret < 0) {
diff --git a/tests/Makefile b/tests/Makefile
index 9ed0754..6804101 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -424,8 +424,8 @@ tests/test-qdev-global-props$(EXESUF): 
tests/test-qdev-global-props.o \
$(test-qapi-obj-y)
 tests/test-vmstate$(EXESUF): tests/test-vmstate.o \
migration/vmstate.o migration/qemu-file.o \
-migration/qemu-file-unix.o qjson.o \
-   $(test-qom-obj-y)
+migration/qemu-file-channel.o qjson.o \
+   $(test-io-obj-y)
 tests/test-timed-average$(EXESUF): tests/test-timed-average.o qemu-timer.o \
$(test-util-obj-y)
 tests/test-base64$(EXESUF): tests/test-base64.o \
diff --git a/tests/test-vmstate.c b/tests/test-vmstate.c
index f337cf6..d19b16a 100644
--- a/tests/test-vmstate.c
+++ b/tests/test-vmstate.c
@@ -29,6 +29,7 @@
 #include "migration/migration.h"
 #include "migration/vmstate.h"
 #include "qemu/coroutine.h"
+#include "io/channel-file.h"
 
 static char temp_file[] = "/tmp/vmst.test.XX";
 static int temp_fd;
@@ -49,11 +50,17 @@ void yield_until_fd_readable(int fd)
 static QEMUFile *open_test_file(bool write)
 {
 int fd = dup(temp_fd);
+QIOChannel *ioc;
 lseek(fd, 0, SEEK_SET);
 if (write) {
 g_assert_cmpint(ftruncate(fd, 0), ==, 0);
 }
-return qemu_fdopen(fd, write ? "wb" : "rb");
+ioc = QIO_CHANNEL(qio_channel_file_new_fd(fd));
+if (write) {
+return qemu_fopen_channel_output(ioc);
+} else {
+return qemu_fopen_channel_input(ioc);
+}
 }
 
 #define SUCCESS(val) \
@@ -469,6 +476,8 @@ int main(int argc, char **argv)
 {
 temp_fd = mkstemp(temp_file);
 
+module_call_init(MODULE_INIT_QOM);
+
 g_test_init(, , NULL);
 g_test_add_func("/vmstate/simple/primitive", test_simple_primitive);
 g_test_add_func("/vmstate/versioned/load/v1", test_load_v1);
-- 
2.5.0

[Qemu-devel] [PATCH v4 20/28] migration: delete QEMUSizedBuffer struct

[Daniel P. Berrange]

Now that we don't have have a buffer based QemuFile
implementation, the QEMUSizedBuffer code is also
unused and can be deleted. A simpler buffer class
also exists in util/buffer.c which other code can
used as needed.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/qemu-file.h |  16 --
 include/qemu/typedefs.h   |   1 -
 migration/Makefile.objs   |   2 +-
 migration/qemu-file-buf.c | 368 --
 4 files changed, 1 insertion(+), 386 deletions(-)
 delete mode 100644 migration/qemu-file-buf.c

diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index 6618d19..edaf598 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -126,13 +126,6 @@ typedef struct QEMUFileHooks {
 QEMURamSaveFunc *save_page;
 } QEMUFileHooks;
 
-struct QEMUSizedBuffer {
-struct iovec *iov;
-size_t n_iov;
-size_t size; /* total allocated size in all iov's */
-size_t used; /* number of used bytes */
-};
-
 QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops);
 QEMUFile *qemu_fopen(const char *filename, const char *mode);
 QEMUFile *qemu_fdopen(int fd, const char *mode);
@@ -155,15 +148,6 @@ void qemu_put_buffer_async(QEMUFile *f, const uint8_t 
*buf, size_t size);
 bool qemu_file_mode_is_not_valid(const char *mode);
 bool qemu_file_is_writable(QEMUFile *f);
 
-QEMUSizedBuffer *qsb_create(const uint8_t *buffer, size_t len);
-void qsb_free(QEMUSizedBuffer *);
-size_t qsb_set_length(QEMUSizedBuffer *qsb, size_t length);
-size_t qsb_get_length(const QEMUSizedBuffer *qsb);
-ssize_t qsb_get_buffer(const QEMUSizedBuffer *, off_t start, size_t count,
-   uint8_t *buf);
-ssize_t qsb_write_at(QEMUSizedBuffer *qsb, const uint8_t *buf,
- off_t pos, size_t count);
-
 
 static inline void qemu_put_ubyte(QEMUFile *f, unsigned int v)
 {
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index 9a5ead6..6b2a9f6 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -79,7 +79,6 @@ typedef struct QemuOpt QemuOpt;
 typedef struct QemuOpts QemuOpts;
 typedef struct QemuOptsList QemuOptsList;
 typedef struct QEMUSGList QEMUSGList;
-typedef struct QEMUSizedBuffer QEMUSizedBuffer;
 typedef struct QEMUTimer QEMUTimer;
 typedef struct QEMUTimerListGroup QEMUTimerListGroup;
 typedef struct QObject QObject;
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index 86dd050..3e5a57c 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -1,6 +1,6 @@
 common-obj-y += migration.o socket.o fd.o exec.o
 common-obj-y += vmstate.o
-common-obj-y += qemu-file.o qemu-file-buf.o qemu-file-unix.o qemu-file-stdio.o
+common-obj-y += qemu-file.o qemu-file-unix.o qemu-file-stdio.o
 common-obj-y += qemu-file-channel.o
 common-obj-y += xbzrle.o postcopy-ram.o
 
diff --git a/migration/qemu-file-buf.c b/migration/qemu-file-buf.c
deleted file mode 100644
index 668ab35..000
--- a/migration/qemu-file-buf.c
+++ /dev/null
@@ -1,368 +0,0 @@
-/*
- * QEMU System Emulator
- *
- * Copyright (c) 2003-2008 Fabrice Bellard
- * Copyright (c) 2014 IBM Corp.
- *
- * Authors:
- *  Stefan Berger 
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to 
deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
- * THE SOFTWARE.
- */
-#include "qemu/osdep.h"
-#include "qemu-common.h"
-#include "qemu/error-report.h"
-#include "qemu/iov.h"
-#include "qemu/sockets.h"
-#include "qemu/coroutine.h"
-#include "migration/migration.h"
-#include "migration/qemu-file.h"
-#include "migration/qemu-file-internal.h"
-#include "trace.h"
-
-#define QSB_CHUNK_SIZE  (1 << 10)
-#define QSB_MAX_CHUNK_SIZE  (16 * QSB_CHUNK_SIZE)
-
-/**
- * Create a QEMUSizedBuffer
- * This type of buffer uses scatter-gather lists internally and
- * can grow to any size. Any data array in the scatter-gather list
- * can hold different amount of bytes.
- *
- * 

[Qemu-devel] [PATCH v4 28/28] migration: remove qemu_get_fd method from QEMUFile

[Daniel P. Berrange]

Now that there is a set_blocking callback in QEMUFileOps,
and all users needing non-blocking support have been
converted to QIOChannel, there is no longer any codepath
requiring the qemu_get_fd() method for QEMUFile. Remove it
to avoid further code being introduced with an expectation
of direct file handle access.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/qemu-file.h |  1 -
 migration/qemu-file.c | 14 --
 2 files changed, 15 deletions(-)

diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index 36af5f4..2409a98 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -103,7 +103,6 @@ typedef int (QEMUFileShutdownFunc)(void *opaque, bool rd, 
bool wr);
 typedef struct QEMUFileOps {
 QEMUFileGetBufferFunc *get_buffer;
 QEMUFileCloseFunc *close;
-QEMUFileGetFD *get_fd;
 QEMUFileSetBlocking *set_blocking;
 QEMUFileWritevBufferFunc *writev_buffer;
 QEMURetPathFunc *get_return_path;
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 6790040..8aea1c7 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -268,14 +268,6 @@ static ssize_t qemu_fill_buffer(QEMUFile *f)
 return len;
 }
 
-int qemu_get_fd(QEMUFile *f)
-{
-if (f->ops->get_fd) {
-return f->ops->get_fd(f->opaque);
-}
-return -1;
-}
-
 void qemu_update_position(QEMUFile *f, size_t size)
 {
 f->pos += size;
@@ -688,11 +680,5 @@ void qemu_file_set_blocking(QEMUFile *f, bool block)
 {
 if (f->ops->set_blocking) {
 f->ops->set_blocking(f->opaque, block);
-} else {
-if (block) {
-qemu_set_block(qemu_get_fd(f));
-} else {
-qemu_set_nonblock(qemu_get_fd(f));
-}
 }
 }
-- 
2.5.0

Re: [Qemu-devel] [PATCH v2] hw: fix error reporting for missing option ROMs

[Eric Blake]

On 03/11/2016 04:18 AM, Daniel P. Berrange wrote:
> If QEMU fails to load any of the VGA ROMs, it prints a message
> to stderr and then carries on as if everything was fine, despite
> the VGA interface not being functional. This extends the the
> various rom_add_*() methods in loader.h to accept a 'Error **errp'
> parameter. The VGA device realizefn() impls can now pass in the
> errp they already have and get errors reported as fatal problems.
> 
> Addition of 'Error **errp' to the load_*() methods in loader.h is
> left as an exercise for future interested developers, since it will
> require fixing up a great many callers to propagate errors correctly.
> 
> Signed-off-by: Daniel P. Berrange 
> ---
> 
> Changed in v2:
> 
>  - Use error_fatal instead of NULL in places lacking an
>Error **errp to propagate to
>  - Use error_setg_file_open instead of error_setg_errno
>  - Mention that load_*() methods are intentionally not converted
> 


> +++ b/hw/i386/pc_sysfw.c
> @@ -178,6 +178,7 @@ static void old_pc_system_rom_init(MemoryRegion 
> *rom_memory, bool isapc_ram_fw)
>  MemoryRegion *bios, *isa_bios;
>  int bios_size, isa_bios_size;
>  int ret;
> +Error *err = NULL;
>  
>  /* BIOS load */
>  if (bios_name == NULL) {
> @@ -199,10 +200,10 @@ static void old_pc_system_rom_init(MemoryRegion 
> *rom_memory, bool isapc_ram_fw)
>  if (!isapc_ram_fw) {
>  memory_region_set_readonly(bios, true);
>  }
> -ret = rom_add_file_fixed(bios_name, (uint32_t)(-bios_size), -1);
> +ret = rom_add_file_fixed(bios_name, (uint32_t)(-bios_size), -1, );
>  if (ret != 0) {
>  bios_error:
> -fprintf(stderr, "qemu: could not load PC BIOS '%s'\n", bios_name);
> +error_report_err(err);

You can get here through the 'goto bios_error' flow, at which point err
is not set.  You'll want to do an error_setg() just before that goto.

With that fixed,
Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [PATCH v4 24/28] migration: don't use an array for storing migrate parameters

[Daniel P. Berrange]

The MigrateState struct uses an array for storing migration
parameters. This presumes that all future parameters will
be integers too, which is not going to be the case. There
is no functional reason why an array is used, if anything
it makes the code less clear. The QAPI schema already
defines a struct - MigrationParameters - capable of storing
all the individual parameters, so just use that instead of
an array.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/migration.h |  5 +++-
 migration/migration.c | 56 +++
 migration/ram.c   |  6 ++---
 3 files changed, 30 insertions(+), 37 deletions(-)

diff --git a/include/migration/migration.h b/include/migration/migration.h
index 46c1bbe..9c5a002 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -135,9 +135,12 @@ struct MigrationState
 QemuThread thread;
 QEMUBH *cleanup_bh;
 QEMUFile *to_dst_file;
-int parameters[MIGRATION_PARAMETER__MAX];
+
+/* New style params from 'migrate-set-parameters' */
+MigrationParameters parameters;
 
 int state;
+/* Old style params from 'migrate' command */
 MigrationParams params;
 
 /* State related to return path */
diff --git a/migration/migration.c b/migration/migration.c
index 8219ea6..a056bcd 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -82,16 +82,14 @@ MigrationState *migrate_get_current(void)
 .bandwidth_limit = MAX_THROTTLE,
 .xbzrle_cache_size = DEFAULT_MIGRATE_CACHE_SIZE,
 .mbps = -1,
-.parameters[MIGRATION_PARAMETER_COMPRESS_LEVEL] =
-DEFAULT_MIGRATE_COMPRESS_LEVEL,
-.parameters[MIGRATION_PARAMETER_COMPRESS_THREADS] =
-DEFAULT_MIGRATE_COMPRESS_THREAD_COUNT,
-.parameters[MIGRATION_PARAMETER_DECOMPRESS_THREADS] =
-DEFAULT_MIGRATE_DECOMPRESS_THREAD_COUNT,
-.parameters[MIGRATION_PARAMETER_X_CPU_THROTTLE_INITIAL] =
-DEFAULT_MIGRATE_X_CPU_THROTTLE_INITIAL,
-.parameters[MIGRATION_PARAMETER_X_CPU_THROTTLE_INCREMENT] =
-DEFAULT_MIGRATE_X_CPU_THROTTLE_INCREMENT,
+.parameters = {
+.compress_level = DEFAULT_MIGRATE_COMPRESS_LEVEL,
+.compress_threads = DEFAULT_MIGRATE_COMPRESS_THREAD_COUNT,
+.decompress_threads = DEFAULT_MIGRATE_DECOMPRESS_THREAD_COUNT,
+.x_cpu_throttle_initial = DEFAULT_MIGRATE_X_CPU_THROTTLE_INITIAL,
+.x_cpu_throttle_increment =
+  DEFAULT_MIGRATE_X_CPU_THROTTLE_INCREMENT,
+},
 };
 
 if (!once) {
@@ -534,15 +532,11 @@ MigrationParameters *qmp_query_migrate_parameters(Error 
**errp)
 MigrationState *s = migrate_get_current();
 
 params = g_malloc0(sizeof(*params));
-params->compress_level = s->parameters[MIGRATION_PARAMETER_COMPRESS_LEVEL];
-params->compress_threads =
-s->parameters[MIGRATION_PARAMETER_COMPRESS_THREADS];
-params->decompress_threads =
-s->parameters[MIGRATION_PARAMETER_DECOMPRESS_THREADS];
-params->x_cpu_throttle_initial =
-s->parameters[MIGRATION_PARAMETER_X_CPU_THROTTLE_INITIAL];
-params->x_cpu_throttle_increment =
-s->parameters[MIGRATION_PARAMETER_X_CPU_THROTTLE_INCREMENT];
+params->compress_level = s->parameters.compress_level;
+params->compress_threads = s->parameters.compress_threads;
+params->decompress_threads = s->parameters.decompress_threads;
+params->x_cpu_throttle_initial = s->parameters.x_cpu_throttle_initial;
+params->x_cpu_throttle_increment = s->parameters.x_cpu_throttle_increment;
 
 return params;
 }
@@ -743,7 +737,8 @@ void qmp_migrate_set_parameters(bool has_compress_level,
 bool has_x_cpu_throttle_initial,
 int64_t x_cpu_throttle_initial,
 bool has_x_cpu_throttle_increment,
-int64_t x_cpu_throttle_increment, Error **errp)
+int64_t x_cpu_throttle_increment,
+Error **errp)
 {
 MigrationState *s = migrate_get_current();
 
@@ -780,26 +775,23 @@ void qmp_migrate_set_parameters(bool has_compress_level,
 }
 
 if (has_compress_level) {
-s->parameters[MIGRATION_PARAMETER_COMPRESS_LEVEL] = compress_level;
+s->parameters.compress_level = compress_level;
 }
 if (has_compress_threads) {
-s->parameters[MIGRATION_PARAMETER_COMPRESS_THREADS] = compress_threads;
+s->parameters.compress_threads = compress_threads;
 }
 if (has_decompress_threads) {
-s->parameters[MIGRATION_PARAMETER_DECOMPRESS_THREADS] =
-decompress_threads;
+s->parameters.decompress_threads = decompress_threads;
 }
 if 

Re: [Qemu-devel] [RFC PATCH v2 3/3] VFIO: Type1 IOMMU mapping support for vGPU

[Neo Jia]

On Fri, Mar 11, 2016 at 09:13:15AM -0700, Alex Williamson wrote:
> On Fri, 11 Mar 2016 04:46:23 +
> "Tian, Kevin"  wrote:
> 
> > > From: Neo Jia [mailto:c...@nvidia.com]
> > > Sent: Friday, March 11, 2016 12:20 PM
> > > 
> > > On Thu, Mar 10, 2016 at 11:10:10AM +0800, Jike Song wrote:  
> > > >  
> > > > >> Is it supposed to be the caller who should set
> > > > >> up IOMMU by DMA api such as dma_map_page(), after calling
> > > > >> vgpu_dma_do_translate()?
> > > > >>  
> > > > >
> > > > > Don't think you need to call dma_map_page here. Once you have the pfn 
> > > > > available
> > > > > to your GPU kernel driver, you can just go ahead to setup the mapping 
> > > > > as you
> > > > > normally do such as calling pci_map_sg and its friends.
> > > > >  
> > > >
> > > > Technically it's definitely OK to call DMA API from the caller rather 
> > > > than here,
> > > > however personally I think it is a bit counter-intuitive: IOMMU page 
> > > > tables
> > > > should be constructed within the VFIO IOMMU driver.
> > > >  
> > > 
> > > Hi Jike,
> > > 
> > > For vGPU, what we have is just a virtual device and a fake IOMMU group, 
> > > therefore
> > > the actual interaction with the real GPU should be managed by the GPU 
> > > vendor driver.
> > >   
> > 
> > Hi, Neo,
> > 
> > Seems we have a different thought on this. Regardless of whether it's a 
> > virtual/physical 
> > device, imo, VFIO should manage IOMMU configuration. The only difference is:
> > 
> > - for physical device, VFIO directly invokes IOMMU API to set IOMMU entry 
> > (GPA->HPA);
> > - for virtual device, VFIO invokes kernel DMA APIs which indirectly lead to 
> > IOMMU entry 
> > set if CONFIG_IOMMU is enabled in kernel (GPA->IOVA);
> > 
> > This would provide an unified way to manage the translation in VFIO, and 
> > then vendor
> > specific driver only needs to query and use returned IOVA corresponding to 
> > a GPA. 
> > 
> > Doing so has another benefit, to make underlying vGPU driver VMM agnostic. 
> > For KVM,
> > yes we can use pci_map_sg. However for Xen it's different (today Dom0 
> > doesn't see
> > IOMMU. In the future there'll be a PVIOMMU implementation) so different 
> > code path is 
> > required. It's better to abstract such specific knowledge out of vGPU 
> > driver, which just
> > uses whatever dma_addr returned by other agent (VFIO here, or another Xen 
> > specific
> > agent) in a centralized way.
> > 
> > Alex, what's your opinion on this?
> 
> The sticky point is how vfio, which is only handling the vGPU, has a
> reference to the physical GPU on which to call DMA API operations.  If
> that reference is provided by the vendor vGPU driver, for example
> vgpu_dma_do_translate_for_pci(gpa, pci_dev), I don't see any reason to
> be opposed to such an API.  I would not condone vfio deriving or owning
> a reference to the physical device on its own though, that's in the
> realm of the vendor vGPU driver.  It does seem a bit cleaner and should
> reduce duplicate code if the vfio vGPU iommu interface could handle the
> iommu mapping for the vendor vgpu driver when necessary.  Thanks,

Hi Alex,

Since we don't want to allow vfio iommu to derive or own a reference to the
physical device, I think it is still better not providing such pci_dev to the 
vfio iommu type1 driver.

Also, I need to point out that if the vfio iommu is going to set up iommu page
table for the real underlying physical device, giving the fact of single RID we
are all having here, the iommu mapping code has to return the new "IOVA" that is
mapped to the HPA, which the GPU vendro driver will have to put on its DMA
engine. This is very different than the current VFIO IOMMU mapping logic.

And we still have to provide another interface to translate the GPA to
HPA for CPU mapping.

In the current RFC, we only need to have a single interface to provide the most
basic information to the GPU vendor driver and without taking the risk of
leaking a ref to VFIO IOMMU.

Thanks,
Neo

> 
> Alex

Re: [Qemu-devel] [PATCH v4 04/26] crypto: add support for anti-forensic split algorithm

[Daniel P. Berrange]

On Mon, Mar 07, 2016 at 01:51:40PM +0800, Fam Zheng wrote:
> On Mon, 02/29 12:00, Daniel P. Berrange wrote:
> > The LUKS format specifies an anti-forensic split algorithm which
> > is used to artificially expand the size of the key material on
> > disk. This is an implementation of that algorithm.
> > 
> > Signed-off-by: Daniel P. Berrange 
> > ---
> >  crypto/Makefile.objs|   1 +
> >  crypto/afsplit.c| 158 
> >  include/crypto/afsplit.h| 135 +++
> >  tests/.gitignore|   1 +
> >  tests/Makefile  |   2 +
> >  tests/test-crypto-afsplit.c | 190 
> > 
> >  6 files changed, 487 insertions(+)
> >  create mode 100644 crypto/afsplit.c
> >  create mode 100644 include/crypto/afsplit.h
> >  create mode 100644 tests/test-crypto-afsplit.c
> > 

> > +static int qcrypto_afsplit_hash(QCryptoHashAlgorithm hash,
> > +size_t blocklen,
> > +uint8_t *block,
> > +Error **errp)
> > +{
> > +size_t digestlen = qcrypto_hash_digest_len(hash);
> > +
> > +size_t hashcount = blocklen / digestlen;
> 
> Do you want to use DIV_ROUND_UP? Because if blocklen < digestlen, hashcount is
> 0, and your for loop below will be skipped.

It is not needed actually - look a couple of lines
further where we do  'if (finallen) { hashcount ++ }'.
This achieves the same end result.

> 
> Fam
> 
> > +size_t finallen = blocklen % digestlen;
> > +uint32_t i;
> > +
> > +if (finallen) {
> > +hashcount++;
> > +} else {
> > +finallen = digestlen;
> > +}
> > +
> > +for (i = 0; i < hashcount; i++) {
> > +uint8_t *out = NULL;
> > +size_t outlen = 0;
> > +uint32_t iv = cpu_to_be32(i);
> > +struct iovec in[] = {
> > +{ .iov_base = ,
> > +  .iov_len = sizeof(iv) },
> > +{ .iov_base = block + (i * digestlen),
> > +  .iov_len = (i == (hashcount - 1)) ? finallen : digestlen },
> > +};
> > +
> > +if (qcrypto_hash_bytesv(hash,
> > +in,
> > +G_N_ELEMENTS(in),
> > +, ,
> > +errp) < 0) {
> > +return -1;
> > +}
> > +
> > +assert(outlen == digestlen);
> > +memcpy(block + (i * digestlen), out,
> > +   (i == (hashcount - 1)) ? finallen : digestlen);
> > +g_free(out);
> > +}
> > +
> > +return 0;
> > +}

Regards,
Daniel
-- 
|: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org  -o- http://virt-manager.org :|
|: http://autobuild.org   -o- http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc :|

[Qemu-devel] [PATCH v4 26/28] migration: add support for encrypting data with TLS

[Daniel P. Berrange]

This extends the migration_set_incoming_channel and
migration_set_outgoing_channel methods so that they
will automatically wrap the QIOChannel in a
QIOChannelTLS instance if TLS credentials are configured
in the migration parameters.

This allows TLS to work for tcp, unix, fd and exec
migration protocols. It does not (currently) work for
RDMA since it does not use these APIs, but it is
unlikely that TLS would be desired with RDMA anyway
since it would degrade the performance to that seen
with TCP defeating the purpose of using RDMA.

On the target host, QEMU would be launched with a set
of TLS credentials for a server endpoint

 $ qemu-system-x86_64 -monitor stdio -incoming defer \
-object 
tls-creds-x509,dir=/home/berrange/security/qemutls,endpoint=server,id=tls0 \
...other args...

To enable incoming TLS migration 2 monitor commands are
then used

  (qemu) migrate_set_str_parameter tls-creds tls0
  (qemu) migrate_incoming tcp:myhostname:9000

On the source host, QEMU is launched in a similar
manner but using client endpoint credentials

 $ qemu-system-x86_64 -monitor stdio \
-object 
tls-creds-x509,dir=/home/berrange/security/qemutls,endpoint=client,id=tls0 \
...other args...

To enable outgoing TLS migration 2 monitor commands are
then used

  (qemu) migrate_set_str_parameter tls-creds tls0
  (qemu) migrate tcp:otherhostname:9000

Thanks to earlier improvements to error reporting,
TLS errors can be seen 'info migrate' when doing a
detached migration. For example:

  (qemu) info migrate
  capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: 
off compress: off events: off x-postcopy-ram: off
  Migration status: failed
  total time: 0 milliseconds
  error description: TLS handshake failed: The TLS connection was non-properly 
terminated.

Or

  (qemu) info migrate
  capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: 
off compress: off events: off x-postcopy-ram: off
  Migration status: failed
  total time: 0 milliseconds
  error description: Certificate does not match the hostname localhost

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/migration.h |  12 +++-
 migration/Makefile.objs   |   1 +
 migration/exec.c  |   2 +-
 migration/fd.c|   2 +-
 migration/migration.c |  40 +--
 migration/socket.c|  34 +++--
 migration/tls.c   | 160 ++
 trace-events  |  12 +++-
 8 files changed, 246 insertions(+), 17 deletions(-)
 create mode 100644 migration/tls.c

diff --git a/include/migration/migration.h b/include/migration/migration.h
index 9c5a002..332b198 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -188,8 +188,18 @@ void qemu_start_incoming_migration(const char *uri, Error 
**errp);
 void migration_set_incoming_channel(MigrationState *s,
 QIOChannel *ioc);
 
+void migration_tls_set_incoming_channel(MigrationState *s,
+QIOChannel *ioc,
+Error **errp);
+
 void migration_set_outgoing_channel(MigrationState *s,
-QIOChannel *ioc);
+QIOChannel *ioc,
+const char *hostname);
+
+void migration_tls_set_outgoing_channel(MigrationState *s,
+QIOChannel *ioc,
+const char *hostname,
+Error **errp);
 
 uint64_t migrate_max_downtime(void);
 
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index 7b9051c..e68b54d 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -1,4 +1,5 @@
 common-obj-y += migration.o socket.o fd.o exec.o
+common-obj-y += tls.o
 common-obj-y += vmstate.o
 common-obj-y += qemu-file.o
 common-obj-y += qemu-file-channel.o
diff --git a/migration/exec.c b/migration/exec.c
index 4f439b4..a5debc6 100644
--- a/migration/exec.c
+++ b/migration/exec.c
@@ -36,7 +36,7 @@ void exec_start_outgoing_migration(MigrationState *s, const 
char *command, Error
 return;
 }
 
-migration_set_outgoing_channel(s, ioc);
+migration_set_outgoing_channel(s, ioc, NULL);
 object_unref(OBJECT(ioc));
 }
 
diff --git a/migration/fd.c b/migration/fd.c
index 1a7fd43..e089bf4 100644
--- a/migration/fd.c
+++ b/migration/fd.c
@@ -36,7 +36,7 @@ void fd_start_outgoing_migration(MigrationState *s, const 
char *fdname, Error **
 return;
 }
 
-migration_set_outgoing_channel(s, ioc);
+migration_set_outgoing_channel(s, ioc, NULL);
 object_unref(OBJECT(ioc));
 }
 
diff --git a/migration/migration.c b/migration/migration.c
index 378041e..188fb2d 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -35,6 +35,7 @@
 

[Qemu-devel] [PATCH v4 23/28] migration: move definition of struct QEMUFile back into qemu-file.c

[Daniel P. Berrange]

Now that the memory buffer based QEMUFile impl is gone, there
is no need for any backend to be accessing internals of the
QEMUFile struct, so it can be moved back into qemu-file.c

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 migration/qemu-file-internal.h | 54 --
 migration/qemu-file.c  | 24 ++-
 2 files changed, 23 insertions(+), 55 deletions(-)
 delete mode 100644 migration/qemu-file-internal.h

diff --git a/migration/qemu-file-internal.h b/migration/qemu-file-internal.h
deleted file mode 100644
index 8fdfa95..000
--- a/migration/qemu-file-internal.h
+++ /dev/null
@@ -1,54 +0,0 @@
-/*
- * QEMU System Emulator
- *
- * Copyright (c) 2003-2008 Fabrice Bellard
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to 
deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
- * THE SOFTWARE.
- */
-
-#ifndef QEMU_FILE_INTERNAL_H
-#define QEMU_FILE_INTERNAL_H 1
-
-#include "qemu-common.h"
-#include "qemu/iov.h"
-
-#define IO_BUF_SIZE 32768
-#define MAX_IOV_SIZE MIN(IOV_MAX, 64)
-
-struct QEMUFile {
-const QEMUFileOps *ops;
-const QEMUFileHooks *hooks;
-void *opaque;
-
-int64_t bytes_xfer;
-int64_t xfer_limit;
-
-int64_t pos; /* start of buffer when writing, end of buffer
-when reading */
-int buf_index;
-int buf_size; /* 0 when writing */
-uint8_t buf[IO_BUF_SIZE];
-
-struct iovec iov[MAX_IOV_SIZE];
-unsigned int iovcnt;
-
-int last_error;
-};
-
-#endif
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 2b25dec..cf743d1 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -30,9 +30,31 @@
 #include "qemu/coroutine.h"
 #include "migration/migration.h"
 #include "migration/qemu-file.h"
-#include "migration/qemu-file-internal.h"
 #include "trace.h"
 
+#define IO_BUF_SIZE 32768
+#define MAX_IOV_SIZE MIN(IOV_MAX, 64)
+
+struct QEMUFile {
+const QEMUFileOps *ops;
+const QEMUFileHooks *hooks;
+void *opaque;
+
+int64_t bytes_xfer;
+int64_t xfer_limit;
+
+int64_t pos; /* start of buffer when writing, end of buffer
+when reading */
+int buf_index;
+int buf_size; /* 0 when writing */
+uint8_t buf[IO_BUF_SIZE];
+
+struct iovec iov[MAX_IOV_SIZE];
+unsigned int iovcnt;
+
+int last_error;
+};
+
 /*
  * Stop a file from being read/written - not all backing files can do this
  * typically only sockets can.
-- 
2.5.0

[Qemu-devel] [PATCH v4 25/28] migration: define 'tls-creds' and 'tls-hostname' migration parameters

[Daniel P. Berrange]

Define two new migration parameters to be used with TLS encryption.
The 'tls-creds' parameter provides the ID of an instance of the
'tls-creds' object type, or rather a subclass such as 'tls-creds-x509'.
Providing these credentials will enable use of TLS on the migration
data stream.

If using x509 certificates, together with a migration URI that does
not include a hostname, the 'tls-hostname' parameter provides the
hostname to use when verifying the server's x509 certificate. This
allows TLS to be used in combination with fd: and exec: protocols
where a TCP connection is established by a 3rd party outside of
QEMU.

NB, this requires changing the migrate_set_parameter method in the
HMP to accept a 's' (string) value instead of 'i' (integer). This
is backwards compatible, because the parsing of strings allows the
quotes to be optional, thus any integer is also a valid string.

Signed-off-by: Daniel P. Berrange 
---
 hmp-commands.hx   |  2 +-
 hmp.c | 36 ++--
 migration/migration.c | 14 +
 qapi-schema.json  | 58 ---
 4 files changed, 100 insertions(+), 10 deletions(-)

diff --git a/hmp-commands.hx b/hmp-commands.hx
index 664d794..47939b5 100644
--- a/hmp-commands.hx
+++ b/hmp-commands.hx
@@ -1008,7 +1008,7 @@ ETEXI
 
 {
 .name   = "migrate_set_parameter",
-.args_type  = "parameter:s,value:i",
+.args_type  = "parameter:s,value:s",
 .params = "parameter value",
 .help   = "Set the parameter for migration",
 .mhandler.cmd = hmp_migrate_set_parameter,
diff --git a/hmp.c b/hmp.c
index 7126f17..885b514 100644
--- a/hmp.c
+++ b/hmp.c
@@ -293,6 +293,12 @@ void hmp_info_migrate_parameters(Monitor *mon, const QDict 
*qdict)
 monitor_printf(mon, " %s: %" PRId64,
 
MigrationParameter_lookup[MIGRATION_PARAMETER_X_CPU_THROTTLE_INCREMENT],
 params->x_cpu_throttle_increment);
+monitor_printf(mon, " %s: '%s'",
+MigrationParameter_lookup[MIGRATION_PARAMETER_TLS_CREDS],
+params->tls_creds ? : "");
+monitor_printf(mon, " %s: '%s'",
+MigrationParameter_lookup[MIGRATION_PARAMETER_TLS_HOSTNAME],
+params->tls_hostname ? : "");
 monitor_printf(mon, "\n");
 }
 
@@ -1242,13 +1248,16 @@ void hmp_migrate_set_capability(Monitor *mon, const 
QDict *qdict)
 void hmp_migrate_set_parameter(Monitor *mon, const QDict *qdict)
 {
 const char *param = qdict_get_str(qdict, "parameter");
-int value = qdict_get_int(qdict, "value");
+int valueint = 0;
+const char *valuestr = NULL;
 Error *err = NULL;
 bool has_compress_level = false;
 bool has_compress_threads = false;
 bool has_decompress_threads = false;
 bool has_x_cpu_throttle_initial = false;
 bool has_x_cpu_throttle_increment = false;
+bool has_tls_creds = false;
+bool has_tls_hostname = false;
 int i;
 
 for (i = 0; i < MIGRATION_PARAMETER__MAX; i++) {
@@ -1256,25 +1265,40 @@ void hmp_migrate_set_parameter(Monitor *mon, const 
QDict *qdict)
 switch (i) {
 case MIGRATION_PARAMETER_COMPRESS_LEVEL:
 has_compress_level = true;
+valueint = qdict_get_int(qdict, "value");
 break;
 case MIGRATION_PARAMETER_COMPRESS_THREADS:
 has_compress_threads = true;
+valueint = qdict_get_int(qdict, "value");
 break;
 case MIGRATION_PARAMETER_DECOMPRESS_THREADS:
 has_decompress_threads = true;
+valueint = qdict_get_int(qdict, "value");
 break;
 case MIGRATION_PARAMETER_X_CPU_THROTTLE_INITIAL:
 has_x_cpu_throttle_initial = true;
+valueint = qdict_get_int(qdict, "value");
 break;
 case MIGRATION_PARAMETER_X_CPU_THROTTLE_INCREMENT:
 has_x_cpu_throttle_increment = true;
+valueint = qdict_get_int(qdict, "value");
+break;
+case MIGRATION_PARAMETER_TLS_CREDS:
+has_tls_creds = true;
+valuestr = qdict_get_str(qdict, "value");
+break;
+case MIGRATION_PARAMETER_TLS_HOSTNAME:
+has_tls_hostname = true;
+valuestr = qdict_get_str(qdict, "value");
 break;
 }
-qmp_migrate_set_parameters(has_compress_level, value,
-   has_compress_threads, value,
-   has_decompress_threads, value,
-   has_x_cpu_throttle_initial, value,
-   has_x_cpu_throttle_increment, value,
+qmp_migrate_set_parameters(has_compress_level, valueint,
+   has_compress_threads, 

[Qemu-devel] [PATCH v4 27/28] migration: remove support for non-iovec based write handlers

[Daniel P. Berrange]

All the remaining QEMUFile implementations provide an iovec
based write handler, so the put_buffer callback can be removed
to simplify the code.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/qemu-file.h |  9 -
 migration/qemu-file.c | 36 
 migration/savevm.c|  8 
 3 files changed, 8 insertions(+), 45 deletions(-)

diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index 43eba9b..36af5f4 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -28,14 +28,6 @@
 #include "io/channel.h"
 
 
-/* This function writes a chunk of data to a file at the given position.
- * The pos argument can be ignored if the file is only being used for
- * streaming.  The handler must write all of the data or return a negative
- * errno value.
- */
-typedef ssize_t (QEMUFilePutBufferFunc)(void *opaque, const uint8_t *buf,
-int64_t pos, size_t size);
-
 /* Read a chunk of data from a file at the given position.  The pos argument
  * can be ignored if the file is only be used for streaming.  The number of
  * bytes actually read should be returned.
@@ -109,7 +101,6 @@ typedef QEMUFile *(QEMURetPathFunc)(void *opaque);
 typedef int (QEMUFileShutdownFunc)(void *opaque, bool rd, bool wr);
 
 typedef struct QEMUFileOps {
-QEMUFilePutBufferFunc *put_buffer;
 QEMUFileGetBufferFunc *get_buffer;
 QEMUFileCloseFunc *close;
 QEMUFileGetFD *get_fd;
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index cf743d1..6790040 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -129,7 +129,7 @@ void qemu_file_set_error(QEMUFile *f, int ret)
 
 bool qemu_file_is_writable(QEMUFile *f)
 {
-return f->ops->writev_buffer || f->ops->put_buffer;
+return f->ops->writev_buffer;
 }
 
 /**
@@ -148,16 +148,9 @@ void qemu_fflush(QEMUFile *f)
 return;
 }
 
-if (f->ops->writev_buffer) {
-if (f->iovcnt > 0) {
-expect = iov_size(f->iov, f->iovcnt);
-ret = f->ops->writev_buffer(f->opaque, f->iov, f->iovcnt, f->pos);
-}
-} else {
-if (f->buf_index > 0) {
-expect = f->buf_index;
-ret = f->ops->put_buffer(f->opaque, f->buf, f->pos, f->buf_index);
-}
+if (f->iovcnt > 0) {
+expect = iov_size(f->iov, f->iovcnt);
+ret = f->ops->writev_buffer(f->opaque, f->iov, f->iovcnt, f->pos);
 }
 
 if (ret >= 0) {
@@ -337,11 +330,6 @@ static void add_to_iovec(QEMUFile *f, const uint8_t *buf, 
size_t size)
 
 void qemu_put_buffer_async(QEMUFile *f, const uint8_t *buf, size_t size)
 {
-if (!f->ops->writev_buffer) {
-qemu_put_buffer(f, buf, size);
-return;
-}
-
 if (f->last_error) {
 return;
 }
@@ -365,9 +353,7 @@ void qemu_put_buffer(QEMUFile *f, const uint8_t *buf, 
size_t size)
 }
 memcpy(f->buf + f->buf_index, buf, l);
 f->bytes_xfer += l;
-if (f->ops->writev_buffer) {
-add_to_iovec(f, f->buf + f->buf_index, l);
-}
+add_to_iovec(f, f->buf + f->buf_index, l);
 f->buf_index += l;
 if (f->buf_index == IO_BUF_SIZE) {
 qemu_fflush(f);
@@ -388,9 +374,7 @@ void qemu_put_byte(QEMUFile *f, int v)
 
 f->buf[f->buf_index] = v;
 f->bytes_xfer++;
-if (f->ops->writev_buffer) {
-add_to_iovec(f, f->buf + f->buf_index, 1);
-}
+add_to_iovec(f, f->buf + f->buf_index, 1);
 f->buf_index++;
 if (f->buf_index == IO_BUF_SIZE) {
 qemu_fflush(f);
@@ -554,12 +538,8 @@ int64_t qemu_ftell_fast(QEMUFile *f)
 int64_t ret = f->pos;
 int i;
 
-if (f->ops->writev_buffer) {
-for (i = 0; i < f->iovcnt; i++) {
-ret += f->iov[i].iov_len;
-}
-} else {
-ret += f->buf_index;
+for (i = 0; i < f->iovcnt; i++) {
+ret += f->iov[i].iov_len;
 }
 
 return ret;
diff --git a/migration/savevm.c b/migration/savevm.c
index ab9a4fa..fc2820b 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -160,13 +160,6 @@ static ssize_t block_writev_buffer(void *opaque, struct 
iovec *iov, int iovcnt,
 return qiov.size;
 }
 
-static ssize_t block_put_buffer(void *opaque, const uint8_t *buf,
-int64_t pos, size_t size)
-{
-bdrv_save_vmstate(opaque, buf, pos, size);
-return size;
-}
-
 static ssize_t block_get_buffer(void *opaque, uint8_t *buf, int64_t pos,
 size_t size)
 {
@@ -184,7 +177,6 @@ static const QEMUFileOps bdrv_read_ops = {
 };
 
 static const QEMUFileOps bdrv_write_ops = {
-.put_buffer = block_put_buffer,
 .writev_buffer  = block_writev_buffer,
 .close  = bdrv_fclose
 };
-- 
2.5.0

[Qemu-devel] [PATCH v4 22/28] migration: delete QEMUFile stdio implementation

[Daniel P. Berrange]

Now that the exec migration backend and savevm have converted
to use the QIOChannel based QEMUFile, there is no user remaining
for the stdio based QEMUFile impl and it can be deleted.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/qemu-file.h |   2 -
 migration/Makefile.objs   |   2 +-
 migration/qemu-file-stdio.c   | 196 --
 3 files changed, 1 insertion(+), 199 deletions(-)
 delete mode 100644 migration/qemu-file-stdio.c

diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index ba5fe08..43eba9b 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -127,10 +127,8 @@ typedef struct QEMUFileHooks {
 } QEMUFileHooks;
 
 QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps *ops);
-QEMUFile *qemu_fopen(const char *filename, const char *mode);
 QEMUFile *qemu_fopen_channel_input(QIOChannel *ioc);
 QEMUFile *qemu_fopen_channel_output(QIOChannel *ioc);
-QEMUFile *qemu_popen_cmd(const char *command, const char *mode);
 void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks);
 int qemu_get_fd(QEMUFile *f);
 int qemu_fclose(QEMUFile *f);
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index 2f9bf12..7b9051c 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -1,6 +1,6 @@
 common-obj-y += migration.o socket.o fd.o exec.o
 common-obj-y += vmstate.o
-common-obj-y += qemu-file.o qemu-file-stdio.o
+common-obj-y += qemu-file.o
 common-obj-y += qemu-file-channel.o
 common-obj-y += xbzrle.o postcopy-ram.o
 
diff --git a/migration/qemu-file-stdio.c b/migration/qemu-file-stdio.c
deleted file mode 100644
index f402e8f..000
--- a/migration/qemu-file-stdio.c
+++ /dev/null
@@ -1,196 +0,0 @@
-/*
- * QEMU System Emulator
- *
- * Copyright (c) 2003-2008 Fabrice Bellard
- *
- * Permission is hereby granted, free of charge, to any person obtaining a copy
- * of this software and associated documentation files (the "Software"), to 
deal
- * in the Software without restriction, including without limitation the rights
- * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
- * copies of the Software, and to permit persons to whom the Software is
- * furnished to do so, subject to the following conditions:
- *
- * The above copyright notice and this permission notice shall be included in
- * all copies or substantial portions of the Software.
- *
- * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
- * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
- * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
- * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
- * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
- * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
- * THE SOFTWARE.
- */
-#include "qemu/osdep.h"
-#include "qemu-common.h"
-#include "qemu/coroutine.h"
-#include "migration/qemu-file.h"
-
-typedef struct QEMUFileStdio {
-FILE *stdio_file;
-QEMUFile *file;
-} QEMUFileStdio;
-
-static int stdio_get_fd(void *opaque)
-{
-QEMUFileStdio *s = opaque;
-
-return fileno(s->stdio_file);
-}
-
-static ssize_t stdio_put_buffer(void *opaque, const uint8_t *buf, int64_t pos,
-size_t size)
-{
-QEMUFileStdio *s = opaque;
-size_t res;
-
-res = fwrite(buf, 1, size, s->stdio_file);
-
-if (res != size) {
-return -errno;
-}
-return res;
-}
-
-static ssize_t stdio_get_buffer(void *opaque, uint8_t *buf, int64_t pos,
-size_t size)
-{
-QEMUFileStdio *s = opaque;
-FILE *fp = s->stdio_file;
-ssize_t bytes;
-
-for (;;) {
-clearerr(fp);
-bytes = fread(buf, 1, size, fp);
-if (bytes != 0 || !ferror(fp)) {
-break;
-}
-if (errno == EAGAIN) {
-yield_until_fd_readable(fileno(fp));
-} else if (errno != EINTR) {
-break;
-}
-}
-return bytes;
-}
-
-static int stdio_pclose(void *opaque)
-{
-QEMUFileStdio *s = opaque;
-int ret;
-ret = pclose(s->stdio_file);
-if (ret == -1) {
-ret = -errno;
-} else if (!WIFEXITED(ret) || WEXITSTATUS(ret) != 0) {
-/* close succeeded, but non-zero exit code: */
-ret = -EIO; /* fake errno value */
-}
-g_free(s);
-return ret;
-}
-
-static int stdio_fclose(void *opaque)
-{
-QEMUFileStdio *s = opaque;
-int ret = 0;
-
-if (qemu_file_is_writable(s->file)) {
-int fd = fileno(s->stdio_file);
-struct stat st;
-
-ret = fstat(fd, );
-if (ret == 0 && S_ISREG(st.st_mode)) {
-/*
- * If the file handle is a regular file make sure the
- * data is flushed to disk before signaling success.
- 

[Qemu-devel] [PATCH v4 17/28] migration: convert RDMA to use QIOChannel interface

[Daniel P. Berrange]

This converts the RDMA code to provide a subclass of QIOChannel
that uses RDMA for the data transport.

This implementation of RDMA does not correctly handle non-blocking
mode. Reads might block if there was not already some pending data
and writes will block until all data is sent. This flawed behaviour
was already present in the existing impl, so appears to not be a
critical problem at this time. It should be on the list of things
to fix in the future though.

The RDMA code would be much better off it it could be split up in
a generic RDMA layer, a QIOChannel impl based on RMDA, and then
the RMDA migration glue. This is left as a future exercise for
the brave.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 migration/rdma.c | 374 ---
 1 file changed, 275 insertions(+), 99 deletions(-)

diff --git a/migration/rdma.c b/migration/rdma.c
index cd33d90..4756149 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -2,10 +2,12 @@
  * RDMA protocol and interfaces
  *
  * Copyright IBM, Corp. 2010-2013
+ * Copyright Red Hat, Inc. 2015-2016
  *
  * Authors:
  *  Michael R. Hines 
  *  Jiuxing Liu 
+ *  Daniel P. Berrange 
  *
  * This work is licensed under the terms of the GNU GPL, version 2 or
  * later.  See the COPYING file in the top-level directory.
@@ -372,14 +374,20 @@ typedef struct RDMAContext {
 GHashTable *blockmap;
 } RDMAContext;
 
-/*
- * Interface to the rest of the migration call stack.
- */
-typedef struct QEMUFileRDMA {
+#define TYPE_QIO_CHANNEL_RDMA "qio-channel-rdma"
+#define QIO_CHANNEL_RDMA(obj) \
+OBJECT_CHECK(QIOChannelRDMA, (obj), TYPE_QIO_CHANNEL_RDMA)
+
+typedef struct QIOChannelRDMA QIOChannelRDMA;
+
+
+struct QIOChannelRDMA {
+QIOChannel parent;
 RDMAContext *rdma;
+QEMUFile *file;
 size_t len;
-void *file;
-} QEMUFileRDMA;
+bool blocking; /* XXX we don't actually honour this yet */
+};
 
 /*
  * Main structure for IB Send/Recv control messages.
@@ -2516,15 +2524,19 @@ static void *qemu_rdma_data_init(const char *host_port, 
Error **errp)
  * SEND messages for control only.
  * VM's ram is handled with regular RDMA messages.
  */
-static ssize_t qemu_rdma_put_buffer(void *opaque, const uint8_t *buf,
-int64_t pos, size_t size)
-{
-QEMUFileRDMA *r = opaque;
-QEMUFile *f = r->file;
-RDMAContext *rdma = r->rdma;
-size_t remaining = size;
-uint8_t * data = (void *) buf;
+static ssize_t qio_channel_rdma_writev(QIOChannel *ioc,
+   const struct iovec *iov,
+   size_t niov,
+   int *fds,
+   size_t nfds,
+   Error **errp)
+{
+QIOChannelRDMA *rioc = QIO_CHANNEL_RDMA(ioc);
+QEMUFile *f = rioc->file;
+RDMAContext *rdma = rioc->rdma;
 int ret;
+ssize_t done = 0;
+size_t i;
 
 CHECK_ERROR_STATE();
 
@@ -2538,27 +2550,31 @@ static ssize_t qemu_rdma_put_buffer(void *opaque, const 
uint8_t *buf,
 return ret;
 }
 
-while (remaining) {
-RDMAControlHeader head;
+for (i = 0; i < niov; i++) {
+size_t remaining = iov[i].iov_len;
+uint8_t * data = (void *)iov[i].iov_base;
+while (remaining) {
+RDMAControlHeader head;
 
-r->len = MIN(remaining, RDMA_SEND_INCREMENT);
-remaining -= r->len;
+rioc->len = MIN(remaining, RDMA_SEND_INCREMENT);
+remaining -= rioc->len;
 
-/* Guaranteed to fit due to RDMA_SEND_INCREMENT MIN above */
-head.len = (uint32_t)r->len;
-head.type = RDMA_CONTROL_QEMU_FILE;
+head.len = rioc->len;
+head.type = RDMA_CONTROL_QEMU_FILE;
 
-ret = qemu_rdma_exchange_send(rdma, , data, NULL, NULL, NULL);
+ret = qemu_rdma_exchange_send(rdma, , data, NULL, NULL, NULL);
 
-if (ret < 0) {
-rdma->error_state = ret;
-return ret;
-}
+if (ret < 0) {
+rdma->error_state = ret;
+return ret;
+}
 
-data += r->len;
+data += rioc->len;
+done += rioc->len;
+}
 }
 
-return size;
+return done;
 }
 
 static size_t qemu_rdma_fill(RDMAContext *rdma, uint8_t *buf,
@@ -2583,41 +2599,74 @@ static size_t qemu_rdma_fill(RDMAContext *rdma, uint8_t 
*buf,
  * RDMA links don't use bytestreams, so we have to
  * return bytes to QEMUFile opportunistically.
  */
-static ssize_t qemu_rdma_get_buffer(void *opaque, uint8_t *buf,
-int64_t pos, size_t size)
-{
-QEMUFileRDMA *r = opaque;
-RDMAContext *rdma = r->rdma;
+static ssize_t qio_channel_rdma_readv(QIOChannel *ioc,

[Qemu-devel] [PATCH v4 16/28] migration: convert exec socket protocol to use QIOChannel

[Daniel P. Berrange]

Convert the exec socket migration protocol driver to use
QIOChannel and QEMUFileChannel, instead of the stdio
popen APIs. It can be unconditionally built because the
QIOChannelCommand class can report suitable error messages
on platforms which can't fork processes.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 migration/Makefile.objs |  3 +--
 migration/exec.c| 63 +
 migration/migration.c   |  4 
 trace-events|  4 
 4 files changed, 37 insertions(+), 37 deletions(-)

diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index 0987bb6..86dd050 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -1,11 +1,10 @@
-common-obj-y += migration.o socket.o fd.o
+common-obj-y += migration.o socket.o fd.o exec.o
 common-obj-y += vmstate.o
 common-obj-y += qemu-file.o qemu-file-buf.o qemu-file-unix.o qemu-file-stdio.o
 common-obj-y += qemu-file-channel.o
 common-obj-y += xbzrle.o postcopy-ram.o
 
 common-obj-$(CONFIG_RDMA) += rdma.o
-common-obj-$(CONFIG_POSIX) += exec.o
 
 common-obj-y += block.o
 
diff --git a/migration/exec.c b/migration/exec.c
index 62f892d..4f439b4 100644
--- a/migration/exec.c
+++ b/migration/exec.c
@@ -3,10 +3,12 @@
  *
  * Copyright IBM, Corp. 2008
  * Copyright Dell MessageOne 2008
+ * Copyright Red Hat, Inc. 2015-2016
  *
  * Authors:
  *  Anthony Liguori   
  *  Charles Duffy 
+ *  Daniel P. Berrange 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
@@ -16,54 +18,53 @@
  */
 
 #include "qemu/osdep.h"
-#include "qemu-common.h"
-#include "qemu/sockets.h"
-#include "qemu/main-loop.h"
 #include "migration/migration.h"
-#include "migration/qemu-file.h"
-#include "block/block.h"
-#include 
+#include "io/channel-command.h"
+#include "trace.h"
 
-//#define DEBUG_MIGRATION_EXEC
-
-#ifdef DEBUG_MIGRATION_EXEC
-#define DPRINTF(fmt, ...) \
-do { printf("migration-exec: " fmt, ## __VA_ARGS__); } while (0)
-#else
-#define DPRINTF(fmt, ...) \
-do { } while (0)
-#endif
 
 void exec_start_outgoing_migration(MigrationState *s, const char *command, 
Error **errp)
 {
-s->to_dst_file = qemu_popen_cmd(command, "w");
-if (s->to_dst_file == NULL) {
-error_setg_errno(errp, errno, "failed to popen the migration target");
+QIOChannel *ioc;
+const char *argv[] = { "/bin/sh", "-c", command, NULL };
+
+trace_migration_exec_outgoing(command);
+ioc = QIO_CHANNEL(qio_channel_command_new_spawn(argv,
+O_WRONLY,
+errp));
+if (!ioc) {
 return;
 }
 
-migrate_fd_connect(s);
+migration_set_outgoing_channel(s, ioc);
+object_unref(OBJECT(ioc));
 }
 
-static void exec_accept_incoming_migration(void *opaque)
+static gboolean exec_accept_incoming_migration(QIOChannel *ioc,
+   GIOCondition condition,
+   gpointer opaque)
 {
-QEMUFile *f = opaque;
-
-qemu_set_fd_handler(qemu_get_fd(f), NULL, NULL, NULL);
-process_incoming_migration(f);
+migration_set_incoming_channel(migrate_get_current(), ioc);
+object_unref(OBJECT(ioc));
+return FALSE; /* unregister */
 }
 
 void exec_start_incoming_migration(const char *command, Error **errp)
 {
-QEMUFile *f;
+QIOChannel *ioc;
+const char *argv[] = { "/bin/sh", "-c", command, NULL };
 
-DPRINTF("Attempting to start an incoming migration\n");
-f = qemu_popen_cmd(command, "r");
-if(f == NULL) {
-error_setg_errno(errp, errno, "failed to popen the migration source");
+trace_migration_exec_incoming(command);
+ioc = QIO_CHANNEL(qio_channel_command_new_spawn(argv,
+O_RDONLY,
+errp));
+if (!ioc) {
 return;
 }
 
-qemu_set_fd_handler(qemu_get_fd(f), exec_accept_incoming_migration, NULL,
-f);
+qio_channel_add_watch(ioc,
+  G_IO_IN,
+  exec_accept_incoming_migration,
+  NULL,
+  NULL);
 }
diff --git a/migration/migration.c b/migration/migration.c
index bf7a976..8219ea6 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -311,10 +311,8 @@ void qemu_start_incoming_migration(const char *uri, Error 
**errp)
 } else if (strstart(uri, "rdma:", )) {
 rdma_start_incoming_migration(p, errp);
 #endif
-#if !defined(WIN32)
 } else if (strstart(uri, "exec:", )) {
 exec_start_incoming_migration(p, errp);
-#endif
 } else if (strstart(uri, "unix:", )) {
 unix_start_incoming_migration(p, errp);
 

[Qemu-devel] [PATCH v4 10/28] migration: add reporting of errors for outgoing migration

[Daniel P. Berrange]

Currently if an application initiates an outgoing migration,
it may or may not, get an error reported back on failure. If
the error occurs synchronously to the 'migrate' command
execution, the client app will see the error message. This
is the case for DNS lookup failures. If the error occurs
asynchronously to the monitor command though, the error
will be thrown away and the client left guessing about
what went wrong. This is the case for failure to connect
to the TCP server (eg due to wrong port, or firewall
rules, or other similar errors).

In the future we'll be adding more scope for errors to
happen asynchronously with the TLS protocol handshake.
TLS errors are hard to diagnose even when they are well
reported, so discarding errors entirely will make it
impossible to debug TLS connection problems.

Management apps which do migration are already using
'query-migrate' / 'info migrate' to check up on progress
of background migration operations and to see their end
status. This is a fine place to also include the error
message when things go wrong.

This patch thus adds an 'error-desc' field to the
MigrationInfo struct, which will be populated when
the 'status' is set to 'failed':

(qemu) migrate -d tcp:localhost:9001
(qemu) info migrate
capabilities: xbzrle: off rdma-pin-all: off auto-converge: off zero-blocks: off 
compress: off events: off x-postcopy-ram: off
Migration status: failed (Error connecting to socket: Connection refused)
total time: 0 milliseconds

In the HMP, when doing non-detached migration, it is
also possible to display this error message directly
to the app.

(qemu) migrate tcp:localhost:9001
Error connecting to socket: Connection refused

Or with QMP

  {
"execute": "query-migrate",
"arguments": {}
  }
  {
"return": {
  "status": "failed",
  "error-desc": "address resolution failed for myhost:9000: No address 
associated with hostname"
}
  }

Signed-off-by: Daniel P. Berrange 
---
 hmp.c | 13 -
 include/migration/migration.h |  5 -
 include/qapi/error.h  |  2 +-
 migration/migration.c | 15 ---
 migration/rdma.c  | 10 +++---
 migration/tcp.c   |  2 +-
 migration/unix.c  |  2 +-
 qapi-schema.json  |  7 ++-
 trace-events  |  2 +-
 util/error.c  |  2 +-
 10 files changed, 42 insertions(+), 18 deletions(-)

diff --git a/hmp.c b/hmp.c
index 5b6084a..7126f17 100644
--- a/hmp.c
+++ b/hmp.c
@@ -34,6 +34,7 @@
 #include "ui/console.h"
 #include "block/qapi.h"
 #include "qemu-io.h"
+#include "qemu/error-report.h"
 
 #ifdef CONFIG_SPICE
 #include 
@@ -167,8 +168,15 @@ void hmp_info_migrate(Monitor *mon, const QDict *qdict)
 }
 
 if (info->has_status) {
-monitor_printf(mon, "Migration status: %s\n",
+monitor_printf(mon, "Migration status: %s",
MigrationStatus_lookup[info->status]);
+if (info->status == MIGRATION_STATUS_FAILED &&
+info->has_error_desc) {
+monitor_printf(mon, " (%s)\n", info->error_desc);
+} else {
+monitor_printf(mon, "\n");
+}
+
 monitor_printf(mon, "total time: %" PRIu64 " milliseconds\n",
info->total_time);
 if (info->has_expected_downtime) {
@@ -1532,6 +1540,9 @@ static void hmp_migrate_status_cb(void *opaque)
 if (status->is_block_migration) {
 monitor_printf(status->mon, "\n");
 }
+if (info->has_error_desc) {
+error_report("%s", info->error_desc);
+}
 monitor_resume(status->mon);
 timer_del(status->timer);
 g_free(status);
diff --git a/include/migration/migration.h b/include/migration/migration.h
index e335380..46c1bbe 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -171,6 +171,9 @@ struct MigrationState
 QSIMPLEQ_HEAD(src_page_requests, MigrationSrcPageRequest) 
src_page_requests;
 /* The RAMBlock used in the last src_page_request */
 RAMBlock *last_req_rb;
+
+/* The last error that occurred */
+Error *error;
 };
 
 void migrate_set_state(int *state, int old_state, int new_state);
@@ -207,7 +210,7 @@ void rdma_start_outgoing_migration(void *opaque, const char 
*host_port, Error **
 
 void rdma_start_incoming_migration(const char *host_port, Error **errp);
 
-void migrate_fd_error(MigrationState *s);
+void migrate_fd_error(MigrationState *s, const Error *error);
 
 void migrate_fd_connect(MigrationState *s);
 
diff --git a/include/qapi/error.h b/include/qapi/error.h
index 02e9dd2..c7e2869 100644
--- a/include/qapi/error.h
+++ b/include/qapi/error.h
@@ -139,7 +139,7 @@ typedef enum ErrorClass {
 /*
  * Get @err's human-readable error message.
  */
-const char *error_get_pretty(Error *err);
+const char *error_get_pretty(const Error *err);
 
 /*
  * Get @err's error class.
diff --git 

[Qemu-devel] [PATCH v4 02/28] io: avoid double-free when closing QIOChannelBuffer

[Daniel P. Berrange]

The QIOChannelBuffer's close implementation will free
the internal data buffer. It failed to reset the pointer
to NULL though, so when the object is later finalized
it will free it a second time with predictable crash.

Signed-off-by: Daniel P. Berrange 
---
 io/channel-buffer.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/io/channel-buffer.c b/io/channel-buffer.c
index 3e5117b..43d7959 100644
--- a/io/channel-buffer.c
+++ b/io/channel-buffer.c
@@ -140,6 +140,7 @@ static int qio_channel_buffer_close(QIOChannel *ioc,
 QIOChannelBuffer *bioc = QIO_CHANNEL_BUFFER(ioc);
 
 g_free(bioc->data);
+bioc->data = NULL;
 bioc->capacity = bioc->usage = bioc->offset = 0;
 
 return 0;
-- 
2.5.0

[Qemu-devel] [PATCH v4 13/28] migration: rename unix.c to socket.c

[Daniel P. Berrange]

The unix.c file will be nearly the same as the tcp.c file,
only differing in the initial SocketAddress creation code.
Rename unix.c to socket.c and refactor it a little to
prepare for merging the TCP code.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 migration/Makefile.objs|  2 +-
 migration/{unix.c => socket.c} | 53 +-
 trace-events   |  8 +++
 3 files changed, 37 insertions(+), 26 deletions(-)
 rename migration/{unix.c => socket.c} (65%)

diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index a5f8a03..9d83997 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -1,4 +1,4 @@
-common-obj-y += migration.o tcp.o unix.o
+common-obj-y += migration.o tcp.o socket.o
 common-obj-y += vmstate.o
 common-obj-y += qemu-file.o qemu-file-buf.o qemu-file-unix.o qemu-file-stdio.o
 common-obj-y += qemu-file-channel.o
diff --git a/migration/unix.c b/migration/socket.c
similarity index 65%
rename from migration/unix.c
rename to migration/socket.c
index f24a058..ef0a673 100644
--- a/migration/unix.c
+++ b/migration/socket.c
@@ -37,42 +37,49 @@ static SocketAddress *unix_build_address(const char *path)
 }
 
 
-static void unix_outgoing_migration(Object *src,
-Error *err,
-gpointer opaque)
+static void socket_outgoing_migration(Object *src,
+  Error *err,
+  gpointer opaque)
 {
 MigrationState *s = opaque;
 QIOChannel *sioc = QIO_CHANNEL(src);
 
 if (err) {
-trace_migration_unix_outgoing_error(error_get_pretty(err));
+trace_migration_socket_outgoing_error(error_get_pretty(err));
 s->to_dst_file = NULL;
 migrate_fd_error(s, err);
 } else {
-trace_migration_unix_outgoing_connected();
+trace_migration_socket_outgoing_connected();
 migration_set_outgoing_channel(s, sioc);
 }
 object_unref(src);
 }
 
-
-void unix_start_outgoing_migration(MigrationState *s, const char *path, Error 
**errp)
+static void socket_start_outgoing_migration(MigrationState *s,
+SocketAddress *saddr,
+Error **errp)
 {
-SocketAddress *saddr = unix_build_address(path);
-QIOChannelSocket *sioc;
-sioc = qio_channel_socket_new();
+QIOChannelSocket *sioc = qio_channel_socket_new();
 qio_channel_socket_connect_async(sioc,
  saddr,
- unix_outgoing_migration,
+ socket_outgoing_migration,
  s,
  NULL);
 qapi_free_SocketAddress(saddr);
 }
 
+void unix_start_outgoing_migration(MigrationState *s,
+   const char *path,
+   Error **errp)
+{
+SocketAddress *saddr = unix_build_address(path);
+socket_start_outgoing_migration(s, saddr, errp);
+}
+
 
-static gboolean unix_accept_incoming_migration(QIOChannel *ioc,
-   GIOCondition condition,
-   gpointer opaque)
+static gboolean socket_accept_incoming_migration(QIOChannel *ioc,
+ GIOCondition condition,
+ gpointer opaque)
 {
 QIOChannelSocket *sioc;
 Error *err = NULL;
@@ -85,7 +92,7 @@ static gboolean unix_accept_incoming_migration(QIOChannel 
*ioc,
 goto out;
 }
 
-trace_migration_unix_incoming_accepted();
+trace_migration_socket_incoming_accepted();
 
 migration_set_incoming_channel(migrate_get_current(),
QIO_CHANNEL(sioc));
@@ -98,12 +105,11 @@ out:
 }
 
 
-void unix_start_incoming_migration(const char *path, Error **errp)
+static void socket_start_incoming_migration(SocketAddress *saddr,
+Error **errp)
 {
-SocketAddress *saddr = unix_build_address(path);
-QIOChannelSocket *listen_ioc;
+QIOChannelSocket *listen_ioc = qio_channel_socket_new();
 
-listen_ioc = qio_channel_socket_new();
 if (qio_channel_socket_listen_sync(listen_ioc, saddr, errp) < 0) {
 object_unref(OBJECT(listen_ioc));
 qapi_free_SocketAddress(saddr);
@@ -112,9 +118,14 @@ void unix_start_incoming_migration(const char *path, Error 
**errp)
 
 qio_channel_add_watch(QIO_CHANNEL(listen_ioc),
   G_IO_IN,
-  unix_accept_incoming_migration,
+  socket_accept_incoming_migration,
   listen_ioc,
   (GDestroyNotify)object_unref);
-
 qapi_free_SocketAddress(saddr);
 }
+

[Qemu-devel] [PATCH v4 11/28] migration: convert post-copy to use QIOChannelBuffer

[Daniel P. Berrange]

The post-copy code does some I/O to/from an intermediate
in-memory buffer rather than direct to the underlying
I/O channel. Switch this code to use QIOChannelBuffer
instead of QEMUSizedBuffer.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 docs/migration.txt  |  4 ++--
 include/sysemu/sysemu.h |  2 +-
 migration/migration.c   | 15 +++
 migration/savevm.c  | 47 ---
 4 files changed, 26 insertions(+), 42 deletions(-)

diff --git a/docs/migration.txt b/docs/migration.txt
index fda8d61..11703de 100644
--- a/docs/migration.txt
+++ b/docs/migration.txt
@@ -403,8 +403,8 @@ listen thread: --- page -- page -- page 
-- page -- page --
 
 On receipt of CMD_PACKAGED (1)
All the data associated with the package - the ( ... ) section in the
-diagram - is read into memory (into a QEMUSizedBuffer), and the main thread
-recurses into qemu_loadvm_state_main to process the contents of the package (2)
+diagram - is read into memory, and the main thread recurses into
+qemu_loadvm_state_main to process the contents of the package (2)
 which contains commands (3,6) and devices (4...)
 
 On receipt of 'postcopy listen' - 3 -(i.e. the 1st command in the package)
diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 3bb8897..ea488de 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -120,7 +120,7 @@ void qemu_savevm_command_send(QEMUFile *f, enum qemu_vm_cmd 
command,
   uint16_t len, uint8_t *data);
 void qemu_savevm_send_ping(QEMUFile *f, uint32_t value);
 void qemu_savevm_send_open_return_path(QEMUFile *f);
-int qemu_savevm_send_packaged(QEMUFile *f, const QEMUSizedBuffer *qsb);
+int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t *buf, size_t len);
 void qemu_savevm_send_postcopy_advise(QEMUFile *f);
 void qemu_savevm_send_postcopy_listen(QEMUFile *f);
 void qemu_savevm_send_postcopy_run(QEMUFile *f);
diff --git a/migration/migration.c b/migration/migration.c
index 6b2e128..732ef84 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -34,6 +34,7 @@
 #include "qom/cpu.h"
 #include "exec/memory.h"
 #include "exec/address-spaces.h"
+#include "io/channel-buffer.h"
 
 #define MAX_THROTTLE  (32 << 20)  /* Migration transfer speed throttling */
 
@@ -1449,7 +1450,8 @@ static int 
await_return_path_close_on_source(MigrationState *ms)
 static int postcopy_start(MigrationState *ms, bool *old_vm_running)
 {
 int ret;
-const QEMUSizedBuffer *qsb;
+QIOChannelBuffer *bioc;
+QEMUFile *fb;
 int64_t time_at_stop = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
 migrate_set_state(>state, MIGRATION_STATUS_ACTIVE,
   MIGRATION_STATUS_POSTCOPY_ACTIVE);
@@ -1508,11 +1510,9 @@ static int postcopy_start(MigrationState *ms, bool 
*old_vm_running)
  * So we wrap the device state up in a package with a length at the start;
  * to do this we use a qemu_buf to hold the whole of the device state.
  */
-QEMUFile *fb = qemu_bufopen("w", NULL);
-if (!fb) {
-error_report("Failed to create buffered file");
-goto fail;
-}
+bioc = qio_channel_buffer_new(4096);
+fb = qemu_fopen_channel_output(QIO_CHANNEL(bioc));
+object_unref(OBJECT(bioc));
 
 /*
  * Make sure the receiver can get incoming pages before we send the rest
@@ -1526,10 +1526,9 @@ static int postcopy_start(MigrationState *ms, bool 
*old_vm_running)
 qemu_savevm_send_postcopy_run(fb);
 
 /* <><> end of stuff going into the package */
-qsb = qemu_buf_get(fb);
 
 /* Now send that blob */
-if (qemu_savevm_send_packaged(ms->to_dst_file, qsb)) {
+if (qemu_savevm_send_packaged(ms->to_dst_file, bioc->data, bioc->usage)) {
 goto fail_closefb;
 }
 qemu_fclose(fb);
diff --git a/migration/savevm.c b/migration/savevm.c
index 96e7db5..93b7454 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -50,6 +50,7 @@
 #include "qemu/iov.h"
 #include "block/snapshot.h"
 #include "block/qapi.h"
+#include "io/channel-buffer.h"
 
 
 #ifndef ETH_P_RARP
@@ -760,10 +761,8 @@ void qemu_savevm_send_open_return_path(QEMUFile *f)
  *0 on success
  *-ve on error
  */
-int qemu_savevm_send_packaged(QEMUFile *f, const QEMUSizedBuffer *qsb)
+int qemu_savevm_send_packaged(QEMUFile *f, const uint8_t *buf, size_t len)
 {
-size_t cur_iov;
-size_t len = qsb_get_length(qsb);
 uint32_t tmp;
 
 if (len > MAX_VM_CMD_PACKAGED_SIZE) {
@@ -777,18 +776,7 @@ int qemu_savevm_send_packaged(QEMUFile *f, const 
QEMUSizedBuffer *qsb)
 trace_qemu_savevm_send_packaged();
 qemu_savevm_command_send(f, MIG_CMD_PACKAGED, 4, (uint8_t *));
 
-/* all the data follows (concatinating the iov's) */
-for (cur_iov = 0; cur_iov < qsb->n_iov; cur_iov++) {
-/* The iov entries are partially filled */
-size_t towrite = 

[Qemu-devel] [PATCH v4 12/28] migration: convert unix socket protocol to use QIOChannel

[Daniel P. Berrange]

Convert the unix socket migration protocol driver to use
QIOChannel and QEMUFileChannel, instead of plain sockets
APIs. It can be unconditionally built, since the socket
impl of QIOChannel will report a suitable error on platforms
where UNIX sockets are unavailable.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 migration/Makefile.objs |   4 +-
 migration/migration.c   |   4 ++
 migration/unix.c| 119 +++-
 trace-events|   5 ++
 4 files changed, 79 insertions(+), 53 deletions(-)

diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index b357e2f..a5f8a03 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -1,11 +1,11 @@
-common-obj-y += migration.o tcp.o
+common-obj-y += migration.o tcp.o unix.o
 common-obj-y += vmstate.o
 common-obj-y += qemu-file.o qemu-file-buf.o qemu-file-unix.o qemu-file-stdio.o
 common-obj-y += qemu-file-channel.o
 common-obj-y += xbzrle.o postcopy-ram.o
 
 common-obj-$(CONFIG_RDMA) += rdma.o
-common-obj-$(CONFIG_POSIX) += exec.o unix.o fd.o
+common-obj-$(CONFIG_POSIX) += exec.o fd.o
 
 common-obj-y += block.o
 
diff --git a/migration/migration.c b/migration/migration.c
index 732ef84..ac373c3 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -314,8 +314,10 @@ void qemu_start_incoming_migration(const char *uri, Error 
**errp)
 #if !defined(WIN32)
 } else if (strstart(uri, "exec:", )) {
 exec_start_incoming_migration(p, errp);
+#endif
 } else if (strstart(uri, "unix:", )) {
 unix_start_incoming_migration(p, errp);
+#if !defined(WIN32)
 } else if (strstart(uri, "fd:", )) {
 fd_start_incoming_migration(p, errp);
 #endif
@@ -1063,8 +1065,10 @@ void qmp_migrate(const char *uri, bool has_blk, bool blk,
 #if !defined(WIN32)
 } else if (strstart(uri, "exec:", )) {
 exec_start_outgoing_migration(s, p, _err);
+#endif
 } else if (strstart(uri, "unix:", )) {
 unix_start_outgoing_migration(s, p, _err);
+#if !defined(WIN32)
 } else if (strstart(uri, "fd:", )) {
 fd_start_outgoing_migration(s, p, _err);
 #endif
diff --git a/migration/unix.c b/migration/unix.c
index b3537fd..f24a058 100644
--- a/migration/unix.c
+++ b/migration/unix.c
@@ -1,10 +1,11 @@
 /*
  * QEMU live migration via Unix Domain Sockets
  *
- * Copyright Red Hat, Inc. 2009
+ * Copyright Red Hat, Inc. 2009-2016
  *
  * Authors:
  *  Chris Lalancette 
+ *  Daniel P. Berrange 
  *
  * This work is licensed under the terms of the GNU GPL, version 2.  See
  * the COPYING file in the top-level directory.
@@ -17,87 +18,103 @@
 
 #include "qemu-common.h"
 #include "qemu/error-report.h"
-#include "qemu/sockets.h"
-#include "qemu/main-loop.h"
 #include "migration/migration.h"
 #include "migration/qemu-file.h"
-#include "block/block.h"
+#include "io/channel-socket.h"
+#include "trace.h"
 
-//#define DEBUG_MIGRATION_UNIX
 
-#ifdef DEBUG_MIGRATION_UNIX
-#define DPRINTF(fmt, ...) \
-do { printf("migration-unix: " fmt, ## __VA_ARGS__); } while (0)
-#else
-#define DPRINTF(fmt, ...) \
-do { } while (0)
-#endif
+static SocketAddress *unix_build_address(const char *path)
+{
+SocketAddress *saddr;
+
+saddr = g_new0(SocketAddress, 1);
+saddr->type = SOCKET_ADDRESS_KIND_UNIX;
+saddr->u.q_unix = g_new0(UnixSocketAddress, 1);
+saddr->u.q_unix->path = g_strdup(path);
+
+return saddr;
+}
 
-static void unix_wait_for_connect(int fd, Error *err, void *opaque)
+
+static void unix_outgoing_migration(Object *src,
+Error *err,
+gpointer opaque)
 {
 MigrationState *s = opaque;
+QIOChannel *sioc = QIO_CHANNEL(src);
 
-if (fd < 0) {
-DPRINTF("migrate connect error: %s\n", error_get_pretty(err));
+if (err) {
+trace_migration_unix_outgoing_error(error_get_pretty(err));
 s->to_dst_file = NULL;
 migrate_fd_error(s, err);
 } else {
-DPRINTF("migrate connect success\n");
-s->to_dst_file = qemu_fopen_socket(fd, "wb");
-migrate_fd_connect(s);
+trace_migration_unix_outgoing_connected();
+migration_set_outgoing_channel(s, sioc);
 }
+object_unref(src);
 }
 
+
 void unix_start_outgoing_migration(MigrationState *s, const char *path, Error 
**errp)
 {
-unix_nonblocking_connect(path, unix_wait_for_connect, s, errp);
+SocketAddress *saddr = unix_build_address(path);
+QIOChannelSocket *sioc;
+sioc = qio_channel_socket_new();
+qio_channel_socket_connect_async(sioc,
+ saddr,
+ unix_outgoing_migration,
+ s,
+ NULL);
+qapi_free_SocketAddress(saddr);
 }
 
-static void unix_accept_incoming_migration(void *opaque)
+
+static gboolean 

[Qemu-devel] [PATCH v4 05/28] migration: split migration hooks out of QEMUFileOps

[Daniel P. Berrange]

The QEMUFileOps struct contains the I/O subsystem callbacks
and the migration stage hooks. Split the hooks out into a
separate QEMUFileHooks struct to make it easier to refactor
the I/O side of QEMUFile without affecting the hooks.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/qemu-file.h  | 10 +++---
 migration/qemu-file-internal.h |  1 +
 migration/qemu-file.c  | 24 +++-
 migration/rdma.c   |  8 
 4 files changed, 31 insertions(+), 12 deletions(-)

diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index 5909ff0..1934a64 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -108,13 +108,16 @@ typedef struct QEMUFileOps {
 QEMUFileCloseFunc *close;
 QEMUFileGetFD *get_fd;
 QEMUFileWritevBufferFunc *writev_buffer;
+QEMURetPathFunc *get_return_path;
+QEMUFileShutdownFunc *shut_down;
+} QEMUFileOps;
+
+typedef struct QEMUFileHooks {
 QEMURamHookFunc *before_ram_iterate;
 QEMURamHookFunc *after_ram_iterate;
 QEMURamHookFunc *hook_ram_load;
 QEMURamSaveFunc *save_page;
-QEMURetPathFunc *get_return_path;
-QEMUFileShutdownFunc *shut_down;
-} QEMUFileOps;
+} QEMUFileHooks;
 
 struct QEMUSizedBuffer {
 struct iovec *iov;
@@ -129,6 +132,7 @@ QEMUFile *qemu_fdopen(int fd, const char *mode);
 QEMUFile *qemu_fopen_socket(int fd, const char *mode);
 QEMUFile *qemu_popen_cmd(const char *command, const char *mode);
 QEMUFile *qemu_bufopen(const char *mode, QEMUSizedBuffer *input);
+void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks);
 int qemu_get_fd(QEMUFile *f);
 int qemu_fclose(QEMUFile *f);
 int64_t qemu_ftell(QEMUFile *f);
diff --git a/migration/qemu-file-internal.h b/migration/qemu-file-internal.h
index d95e853..8fdfa95 100644
--- a/migration/qemu-file-internal.h
+++ b/migration/qemu-file-internal.h
@@ -33,6 +33,7 @@
 
 struct QEMUFile {
 const QEMUFileOps *ops;
+const QEMUFileHooks *hooks;
 void *opaque;
 
 int64_t bytes_xfer;
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 656db4a..b480b72 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -80,6 +80,12 @@ QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps 
*ops)
 return f;
 }
 
+
+void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks)
+{
+f->hooks = hooks;
+}
+
 /*
  * Get last error for stream f
  *
@@ -149,8 +155,8 @@ void ram_control_before_iterate(QEMUFile *f, uint64_t flags)
 {
 int ret = 0;
 
-if (f->ops->before_ram_iterate) {
-ret = f->ops->before_ram_iterate(f, f->opaque, flags, NULL);
+if (f->hooks && f->hooks->before_ram_iterate) {
+ret = f->hooks->before_ram_iterate(f, f->opaque, flags, NULL);
 if (ret < 0) {
 qemu_file_set_error(f, ret);
 }
@@ -161,8 +167,8 @@ void ram_control_after_iterate(QEMUFile *f, uint64_t flags)
 {
 int ret = 0;
 
-if (f->ops->after_ram_iterate) {
-ret = f->ops->after_ram_iterate(f, f->opaque, flags, NULL);
+if (f->hooks && f->hooks->after_ram_iterate) {
+ret = f->hooks->after_ram_iterate(f, f->opaque, flags, NULL);
 if (ret < 0) {
 qemu_file_set_error(f, ret);
 }
@@ -173,8 +179,8 @@ void ram_control_load_hook(QEMUFile *f, uint64_t flags, 
void *data)
 {
 int ret = -EINVAL;
 
-if (f->ops->hook_ram_load) {
-ret = f->ops->hook_ram_load(f, f->opaque, flags, data);
+if (f->hooks && f->hooks->hook_ram_load) {
+ret = f->hooks->hook_ram_load(f, f->opaque, flags, data);
 if (ret < 0) {
 qemu_file_set_error(f, ret);
 }
@@ -193,9 +199,9 @@ size_t ram_control_save_page(QEMUFile *f, ram_addr_t 
block_offset,
  ram_addr_t offset, size_t size,
  uint64_t *bytes_sent)
 {
-if (f->ops->save_page) {
-int ret = f->ops->save_page(f, f->opaque, block_offset,
-offset, size, bytes_sent);
+if (f->hooks && f->hooks->save_page) {
+int ret = f->hooks->save_page(f, f->opaque, block_offset,
+  offset, size, bytes_sent);
 
 if (ret != RAM_SAVE_CONTROL_DELAYED) {
 if (bytes_sent && *bytes_sent > 0) {
diff --git a/migration/rdma.c b/migration/rdma.c
index bcae1e8..187fc1c 100644
--- a/migration/rdma.c
+++ b/migration/rdma.c
@@ -3378,12 +3378,18 @@ static const QEMUFileOps rdma_read_ops = {
 .get_buffer= qemu_rdma_get_buffer,
 .get_fd= qemu_rdma_get_fd,
 .close = qemu_rdma_close,
+};
+
+static const QEMUFileHooks rdma_read_hooks = {
 .hook_ram_load = rdma_load_hook,
 };
 
 static const QEMUFileOps rdma_write_ops = {
 .put_buffer = qemu_rdma_put_buffer,
 .close  = qemu_rdma_close,
+};
+
+static const QEMUFileHooks rdma_write_hooks = {
 

[Qemu-devel] [PATCH v4 07/28] migration: force QEMUFile to blocking mode for outgoing migration

[Daniel P. Berrange]

Instead of relying on the default QEMUFile I/O blocking flag
state, explicitly turn on blocking I/O for outgoing migration
since it takes place in a background thread.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 migration/migration.c | 1 +
 1 file changed, 1 insertion(+)

diff --git a/migration/migration.c b/migration/migration.c
index 942c22d..8fdd630 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1770,6 +1770,7 @@ void migrate_fd_connect(MigrationState *s)
 s->expected_downtime = max_downtime/100;
 s->cleanup_bh = qemu_bh_new(migrate_fd_cleanup, s);
 
+qemu_file_set_blocking(s->to_dst_file, true);
 qemu_file_set_rate_limit(s->to_dst_file,
  s->bandwidth_limit / XFER_LIMIT_RATIO);
 
-- 
2.5.0

[Qemu-devel] [PATCH v4 00/28] Convert migration to QIOChannel & support TLS

[Daniel P. Berrange]

This is an update of patches that were previously posted

  FYI: https://lists.gnu.org/archive/html/qemu-devel/2015-09/msg00829.html
   v1: https://lists.gnu.org/archive/html/qemu-devel/2016-01/msg01914.html
   v2: https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg03509.html
   v3: https://lists.gnu.org/archive/html/qemu-devel/2016-02/msg06279.html

The primary goal of this series of patches is to support TLS on the
migration data channel. The bulk of the work in this series though,
is converting the various QEMUFile implementations to be based on the
new QIOChannel framework.

At the end of this current series there is just one remaining impl
of QEMUFileOps that is not based on QIOChannel - the one in savevm.c
that is using BlockDriverState. It would be possible to create a
QIOChannel wrapper around BlockDriverState too, at which point all
QEMUFile impls would be QIOChannel based. This would then let us
cut out the QEMUFileOps driver callbacks entirely and thus simply
code even more. This patch series is already too large, so I left
that for now.

The first 7 patches are some basic clean ups/fixes mostly to
the QEMUFile code

The 8th patch introduces the QIOChannel based QEMUFile impl
and the 9th adds helpers for using it to start migrations.

Patch 10 adds very long overdue support for reporting errors
during migration back to the management app, which is critical
for TLS otherwise it is impossible to debug any failures.

Patches 11-18 convert the various migration protocols to use
the QIOChannel based QEMUFile impl. In this refactoring the
TCP and UNIX implementations were able to be merged into a
generic sockets impl.

Patches 19-22 remove the now unused QEMUFile impls that do
not use QIOChanel

Patches 23 & 24 do some more cleanup

Patch 25 defines some new migration parameters which are used
to enable use of TLS

Patch 26 actually implements support for TLS with migration,
working with tcp, unix, fd and exec migration backend protocols.
Only RDMA is unsupported with TLS. The commit message shows the
example usage via the HMP

Patches 27 & 28 do some final cleanup.

Overall we have a net win of deleting ~350 lines of code,
despite adding more features, which is always nice.

I have been testing the various migration protocols, including
RDMA and appear to be still functional.

In terms of performance, I have tested TCP with TLS migration
enabled over a 10 Gig-E network interface.

With plain TCP we were able to reach 8500mbs (according to
'info migrate' stats).

With TCP and TLS enabled, we are only able to reach 1800 mbs.
IOW, we can max out 1 Gig-E NICs with TLS, but not 10 Gig-E
where we only reach 21% of potential plain text throughput.

The source host migration thread is only hitting 60% CPU
utilization, but the target host incoming migration thread
is hitting 100% CPU.

The source migration thread is dominated solely by GNUTLS
AES encryption functions as would be expected.

The target migration thread is dominated by the same GNUTLS
AES encryption functions, but also memcpy(). IIUC, the memcpy
is QEMU generic migration code copying RAM pages into place.

In talking with Dave Gilbert we thought it might be possible
to use two threads for incoming migration on the target host.
The first would be responsible for doing network I/O into
local buffers, including the TLS decryption. The second
would be responsible for processing the data. That way the
memcpy() of RAM would move into another thread, allowing the
first thread to spend 100% of its time doing TLS decryption.

If we assume the decryption + encryption take equal amounts
of time, then it ought to let us raise TLS throughput from
1800 mbs, to approx 3000 mbs. Still a good way off 8500mbs
from non-TLS migration, but a worth while improvement none
the less.

NB, these TLS migration results were on a CPU with native AES
instructionset support. CPUs with AES instructions would be
even worse performance.

Changed in v4:

 (Only patches 2, 8, 10 & 25 have changes since v3)

 - Expanded docs for new 'error_desc' field in query-migrate
 - Drop new HMP migrate_set_str_parameter command and just
   change migrate_set_parameter to accept a string instead
   of only int
 - Add 'get_return_path' impl for QIOChannel based QEMUFile
   to make post-copy work
 - Replace logic which tried to modify struct iovec elements
   in-replace, with iov_copy + iov_discard_front to avoid
   issue with niov == 0
 - Fix double-free in QIOChannelBuffer triggered by post-copy
 - Reset error_desc field in migrate_init so old errors don't
   persist when restarting a failed migrate
 - Keep the first reported migration error message instead of
   the last reported on.

Changed in v3:

 - Rebase to resolve conflicts with recent merged
   patches
 - Fix up include qemu/osdep.h in various new files

Changed in v2:

 - Switch to setting migration parameters for TLS instead
   of adding to the URI syntax
 - Support TLS over tcp, unix, fd, and socket protocols, not
   just 

[Qemu-devel] [PATCH v4 01/28] s390: use FILE instead of QEMUFile for creating text file

[Daniel P. Berrange]

The s390 skeys monitor command needs to write out a plain text
file. Currently it is using the QEMUFile class for this, but
work is ongoing to refactor QEMUFile and eliminate much code
related to it. The only feature qemu_fopen() gives over fopen()
is support for QEMU FD passing, but this can be achieved with
qemu_open() + fdopen() too. Switching to regular stdio FILE
APIs avoids the need to sprintf via an intermedia buffer which
slightly simplifies the code.

Reviewed-by: Eric Blake 
Signed-off-by: Daniel P. Berrange 
---
 hw/s390x/s390-skeys.c | 26 ++
 1 file changed, 14 insertions(+), 12 deletions(-)

diff --git a/hw/s390x/s390-skeys.c b/hw/s390x/s390-skeys.c
index 6528ffe..1d4e537 100644
--- a/hw/s390x/s390-skeys.c
+++ b/hw/s390x/s390-skeys.c
@@ -46,15 +46,11 @@ void s390_skeys_init(void)
 qdev_init_nofail(DEVICE(obj));
 }
 
-static void write_keys(QEMUFile *f, uint8_t *keys, uint64_t startgfn,
+static void write_keys(FILE *f, uint8_t *keys, uint64_t startgfn,
uint64_t count, Error **errp)
 {
 uint64_t curpage = startgfn;
 uint64_t maxpage = curpage + count - 1;
-const char *fmt = "page=%03" PRIx64 ": key(%d) => ACC=%X, FP=%d, REF=%d,"
-  " ch=%d, reserved=%d\n";
-char buf[128];
-int len;
 
 for (; curpage <= maxpage; curpage++) {
 uint8_t acc = (*keys & 0xF0) >> 4;
@@ -63,10 +59,9 @@ static void write_keys(QEMUFile *f, uint8_t *keys, uint64_t 
startgfn,
 int ch = (*keys & 0x02);
 int res = (*keys & 0x01);
 
-len = snprintf(buf, sizeof(buf), fmt, curpage,
-   *keys, acc, fp, ref, ch, res);
-assert(len < sizeof(buf));
-qemu_put_buffer(f, (uint8_t *)buf, len);
+fprintf(f, "page=%03" PRIx64 ": key(%d) => ACC=%X, FP=%d, REF=%d,"
+" ch=%d, reserved=%d\n",
+curpage, *keys, acc, fp, ref, ch, res);
 keys++;
 }
 }
@@ -115,7 +110,8 @@ void qmp_dump_skeys(const char *filename, Error **errp)
 vaddr cur_gfn = 0;
 uint8_t *buf;
 int ret;
-QEMUFile *f;
+int fd;
+FILE *f;
 
 /* Quick check to see if guest is using storage keys*/
 if (!skeyclass->skeys_enabled(ss)) {
@@ -124,8 +120,14 @@ void qmp_dump_skeys(const char *filename, Error **errp)
 return;
 }
 
-f = qemu_fopen(filename, "wb");
+fd = qemu_open(filename, O_WRONLY | O_CREAT | O_TRUNC, 0600);
+if (fd < 0) {
+error_setg_file_open(errp, errno, filename);
+return;
+}
+f = fdopen(fd, "wb");
 if (!f) {
+close(fd);
 error_setg_file_open(errp, errno, filename);
 return;
 }
@@ -161,7 +163,7 @@ out_free:
 error_propagate(errp, lerr);
 g_free(buf);
 out:
-qemu_fclose(f);
+fclose(f);
 }
 
 static void qemu_s390_skeys_init(Object *obj)
-- 
2.5.0

[Qemu-devel] [PATCH v4 08/28] migration: introduce a new QEMUFile impl based on QIOChannel

[Daniel P. Berrange]

Introduce a new QEMUFile implementation that is based on
the QIOChannel objects. This impl is different from existing
impls in that there is no file descriptor that can be made
available, as some channels may be based on higher level
protocols such as TLS.

Although the QIOChannel based implementation can trivially
provide a bi-directional stream, initially we have separate
functions for opening input & output directions to fit with
the expectation of the current QEMUFile interface.

Signed-off-by: Daniel P. Berrange 
---
 include/migration/qemu-file.h |   4 +
 migration/Makefile.objs   |   1 +
 migration/qemu-file-channel.c | 180 ++
 3 files changed, 185 insertions(+)
 create mode 100644 migration/qemu-file-channel.c

diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index 2dea81f..0329ccc 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -23,7 +23,9 @@
  */
 #ifndef QEMU_FILE_H
 #define QEMU_FILE_H 1
+#include "qemu-common.h"
 #include "exec/cpu-common.h"
+#include "io/channel.h"
 
 
 /* This function writes a chunk of data to a file at the given position.
@@ -135,6 +137,8 @@ QEMUFile *qemu_fopen_ops(void *opaque, const QEMUFileOps 
*ops);
 QEMUFile *qemu_fopen(const char *filename, const char *mode);
 QEMUFile *qemu_fdopen(int fd, const char *mode);
 QEMUFile *qemu_fopen_socket(int fd, const char *mode);
+QEMUFile *qemu_fopen_channel_input(QIOChannel *ioc);
+QEMUFile *qemu_fopen_channel_output(QIOChannel *ioc);
 QEMUFile *qemu_popen_cmd(const char *command, const char *mode);
 QEMUFile *qemu_bufopen(const char *mode, QEMUSizedBuffer *input);
 void qemu_file_set_hooks(QEMUFile *f, const QEMUFileHooks *hooks);
diff --git a/migration/Makefile.objs b/migration/Makefile.objs
index 0cac6d7..b357e2f 100644
--- a/migration/Makefile.objs
+++ b/migration/Makefile.objs
@@ -1,6 +1,7 @@
 common-obj-y += migration.o tcp.o
 common-obj-y += vmstate.o
 common-obj-y += qemu-file.o qemu-file-buf.o qemu-file-unix.o qemu-file-stdio.o
+common-obj-y += qemu-file-channel.o
 common-obj-y += xbzrle.o postcopy-ram.o
 
 common-obj-$(CONFIG_RDMA) += rdma.o
diff --git a/migration/qemu-file-channel.c b/migration/qemu-file-channel.c
new file mode 100644
index 000..45c13f1
--- /dev/null
+++ b/migration/qemu-file-channel.c
@@ -0,0 +1,180 @@
+/*
+ * QEMUFile backend for QIOChannel objects
+ *
+ * Copyright (c) 2015-2016 Red Hat, Inc
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to 
deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice shall be included in
+ * all copies or substantial portions of the Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING 
FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+ * THE SOFTWARE.
+ */
+
+#include "qemu/osdep.h"
+#include "migration/qemu-file.h"
+#include "io/channel-socket.h"
+#include "qemu/iov.h"
+
+
+static ssize_t channel_writev_buffer(void *opaque,
+ struct iovec *iov,
+ int iovcnt,
+ int64_t pos)
+{
+QIOChannel *ioc = QIO_CHANNEL(opaque);
+ssize_t done = 0;
+struct iovec *local_iov = g_new(struct iovec, iovcnt);
+struct iovec *local_iov_head = local_iov;
+unsigned int nlocal_iov = iovcnt;
+
+nlocal_iov = iov_copy(local_iov, nlocal_iov,
+  iov, iovcnt,
+  0, iov_size(iov, iovcnt));
+
+while (nlocal_iov > 0) {
+ssize_t len;
+len = qio_channel_writev(ioc, local_iov, nlocal_iov, NULL);
+if (len == QIO_CHANNEL_ERR_BLOCK) {
+qio_channel_wait(ioc, G_IO_OUT);
+continue;
+}
+if (len < 0) {
+/* XXX handle Error objects */
+done = -EIO;
+goto cleanup;
+}
+
+iov_discard_front(_iov, _iov, len);
+done += len;
+}
+
+ cleanup:
+g_free(local_iov_head);
+return done;
+}
+
+
+static ssize_t channel_get_buffer(void *opaque,
+  uint8_t *buf,
+  int64_t pos,
+  size_t size)
+{
+QIOChannel 

[Qemu-devel] [PATCH v4 04/28] migration: ensure qemu_fflush() always writes full data amount

[Daniel P. Berrange]

The QEMUFile writev_buffer / put_buffer functions are expected
to write out the full set of requested data, blocking until
complete. The qemu_fflush() caller does not expect to deal with
partial writes. Clarify the function comments and add a sanity
check to the code to catch mistaken implementations.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 include/migration/qemu-file.h |  6 --
 migration/qemu-file.c | 16 
 2 files changed, 16 insertions(+), 6 deletions(-)

diff --git a/include/migration/qemu-file.h b/include/migration/qemu-file.h
index 3f6b4ed..5909ff0 100644
--- a/include/migration/qemu-file.h
+++ b/include/migration/qemu-file.h
@@ -28,7 +28,8 @@
 
 /* This function writes a chunk of data to a file at the given position.
  * The pos argument can be ignored if the file is only being used for
- * streaming.  The handler should try to write all of the data it can.
+ * streaming.  The handler must write all of the data or return a negative
+ * errno value.
  */
 typedef ssize_t (QEMUFilePutBufferFunc)(void *opaque, const uint8_t *buf,
 int64_t pos, size_t size);
@@ -54,7 +55,8 @@ typedef int (QEMUFileCloseFunc)(void *opaque);
 typedef int (QEMUFileGetFD)(void *opaque);
 
 /*
- * This function writes an iovec to file.
+ * This function writes an iovec to file. The handler must write all
+ * of the data or return a negative errno value.
  */
 typedef ssize_t (QEMUFileWritevBufferFunc)(void *opaque, struct iovec *iov,
int iovcnt, int64_t pos);
diff --git a/migration/qemu-file.c b/migration/qemu-file.c
index 6f4a129..656db4a 100644
--- a/migration/qemu-file.c
+++ b/migration/qemu-file.c
@@ -108,11 +108,13 @@ bool qemu_file_is_writable(QEMUFile *f)
  * Flushes QEMUFile buffer
  *
  * If there is writev_buffer QEMUFileOps it uses it otherwise uses
- * put_buffer ops.
+ * put_buffer ops. This will flush all pending data. If data was
+ * only partially flushed, it will set an error state.
  */
 void qemu_fflush(QEMUFile *f)
 {
 ssize_t ret = 0;
+ssize_t expect = 0;
 
 if (!qemu_file_is_writable(f)) {
 return;
@@ -120,21 +122,27 @@ void qemu_fflush(QEMUFile *f)
 
 if (f->ops->writev_buffer) {
 if (f->iovcnt > 0) {
+expect = iov_size(f->iov, f->iovcnt);
 ret = f->ops->writev_buffer(f->opaque, f->iov, f->iovcnt, f->pos);
 }
 } else {
 if (f->buf_index > 0) {
+expect = f->buf_index;
 ret = f->ops->put_buffer(f->opaque, f->buf, f->pos, f->buf_index);
 }
 }
+
 if (ret >= 0) {
 f->pos += ret;
 }
+/* We expect the QEMUFile write impl to send the full
+ * data set we requested, so sanity check that.
+ */
+if (ret != expect) {
+qemu_file_set_error(f, ret < 0 ? ret : -EIO);
+}
 f->buf_index = 0;
 f->iovcnt = 0;
-if (ret < 0) {
-qemu_file_set_error(f, ret);
-}
 }
 
 void ram_control_before_iterate(QEMUFile *f, uint64_t flags)
-- 
2.5.0

[Qemu-devel] [PATCH v4 03/28] migration: remove use of qemu_bufopen from vmstate tests

[Daniel P. Berrange]

Some of the test-vmstate.c test cases use a temporary file
while others use a memory buffer. To facilitate the future
removal of the qemu_bufopen() function, convert all the tests
to use a temporary file.

Reviewed-by: Dr. David Alan Gilbert 
Signed-off-by: Daniel P. Berrange 
---
 tests/Makefile   |  2 +-
 tests/test-vmstate.c | 44 +---
 2 files changed, 14 insertions(+), 32 deletions(-)

diff --git a/tests/Makefile b/tests/Makefile
index cd4bbd4..9ed0754 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -423,7 +423,7 @@ tests/test-qdev-global-props$(EXESUF): 
tests/test-qdev-global-props.o \
hw/core/fw-path-provider.o \
$(test-qapi-obj-y)
 tests/test-vmstate$(EXESUF): tests/test-vmstate.o \
-   migration/vmstate.o migration/qemu-file.o migration/qemu-file-buf.o \
+   migration/vmstate.o migration/qemu-file.o \
 migration/qemu-file-unix.o qjson.o \
$(test-qom-obj-y)
 tests/test-timed-average$(EXESUF): tests/test-timed-average.o qemu-timer.o \
diff --git a/tests/test-vmstate.c b/tests/test-vmstate.c
index 713d444..f337cf6 100644
--- a/tests/test-vmstate.c
+++ b/tests/test-vmstate.c
@@ -44,11 +44,6 @@ void yield_until_fd_readable(int fd)
 select(fd + 1, , NULL, NULL, NULL);
 }
 
-/*
- * Some tests use 'open_test_file' to work on a real fd, some use
- * an in memory file (QEMUSizedBuffer+qemu_bufopen); we could pick one
- * but this way we test both.
- */
 
 /* Duplicate temp_fd and seek to the beginning of the file */
 static QEMUFile *open_test_file(bool write)
@@ -61,20 +56,6 @@ static QEMUFile *open_test_file(bool write)
 return qemu_fdopen(fd, write ? "wb" : "rb");
 }
 
-/*
- * Check that the contents of the memory-buffered file f match
- * the given size/data.
- */
-static void check_mem_file(QEMUFile *f, void *data, size_t size)
-{
-uint8_t *result = g_malloc(size);
-const QEMUSizedBuffer *qsb = qemu_buf_get(f);
-g_assert_cmpint(qsb_get_length(qsb), ==, size);
-g_assert_cmpint(qsb_get_buffer(qsb, 0, size, result), ==, size);
-g_assert_cmpint(memcmp(result, data, size), ==, 0);
-g_free(result);
-}
-
 #define SUCCESS(val) \
 g_assert_cmpint((val), ==, 0)
 
@@ -392,7 +373,7 @@ static const VMStateDescription vmstate_skipping = {
 
 static void test_save_noskip(void)
 {
-QEMUFile *fsave = qemu_bufopen("w", NULL);
+QEMUFile *fsave = open_test_file(true);
 TestStruct obj = { .a = 1, .b = 2, .c = 3, .d = 4, .e = 5, .f = 6,
.skip_c_e = false };
 vmstate_save_state(fsave, _skipping, , NULL);
@@ -406,13 +387,14 @@ static void test_save_noskip(void)
 0, 0, 0, 5, /* e */
 0, 0, 0, 0, 0, 0, 0, 6, /* f */
 };
-check_mem_file(fsave, expected, sizeof(expected));
+
 qemu_fclose(fsave);
+compare_vmstate(expected, sizeof(expected));
 }
 
 static void test_save_skip(void)
 {
-QEMUFile *fsave = qemu_bufopen("w", NULL);
+QEMUFile *fsave = open_test_file(true);
 TestStruct obj = { .a = 1, .b = 2, .c = 3, .d = 4, .e = 5, .f = 6,
.skip_c_e = true };
 vmstate_save_state(fsave, _skipping, , NULL);
@@ -424,13 +406,14 @@ static void test_save_skip(void)
 0, 0, 0, 0, 0, 0, 0, 4, /* d */
 0, 0, 0, 0, 0, 0, 0, 6, /* f */
 };
-check_mem_file(fsave, expected, sizeof(expected));
 
 qemu_fclose(fsave);
+compare_vmstate(expected, sizeof(expected));
 }
 
 static void test_load_noskip(void)
 {
+QEMUFile *fsave = open_test_file(true);
 uint8_t buf[] = {
 0, 0, 0, 10, /* a */
 0, 0, 0, 20, /* b */
@@ -440,10 +423,10 @@ static void test_load_noskip(void)
 0, 0, 0, 0, 0, 0, 0, 60, /* f */
 QEMU_VM_EOF, /* just to ensure we won't get EOF reported prematurely */
 };
+qemu_put_buffer(fsave, buf, sizeof(buf));
+qemu_fclose(fsave);
 
-QEMUSizedBuffer *qsb = qsb_create(buf, sizeof(buf));
-g_assert(qsb);
-QEMUFile *loading = qemu_bufopen("r", qsb);
+QEMUFile *loading = open_test_file(false);
 TestStruct obj = { .skip_c_e = false };
 vmstate_load_state(loading, _skipping, , 2);
 g_assert(!qemu_file_get_error(loading));
@@ -454,11 +437,11 @@ static void test_load_noskip(void)
 g_assert_cmpint(obj.e, ==, 50);
 g_assert_cmpint(obj.f, ==, 60);
 qemu_fclose(loading);
-qsb_free(qsb);
 }
 
 static void test_load_skip(void)
 {
+QEMUFile *fsave = open_test_file(true);
 uint8_t buf[] = {
 0, 0, 0, 10, /* a */
 0, 0, 0, 20, /* b */
@@ -466,10 +449,10 @@ static void test_load_skip(void)
 0, 0, 0, 0, 0, 0, 0, 60, /* f */
 QEMU_VM_EOF, /* just to ensure we won't get EOF reported prematurely */
 };
+qemu_put_buffer(fsave, buf, sizeof(buf));
+qemu_fclose(fsave);
 
-QEMUSizedBuffer *qsb = qsb_create(buf, sizeof(buf));
-g_assert(qsb);
-QEMUFile 

Re: [Qemu-devel] BiteSizedPatch-LargeFrames

[Alex Bennée]

Siddharth Gupta  writes:

> From 032be62f56a207833ae12cc9474e3e8be5ed8eb4 Mon Sep 17 00:00:00 2001
> From: Siddharth Gupta 
> Date: Fri, 11 Mar 2016 20:10:41 +0530
> Subject: [PATCH]
> bitesizedtasks-large_frames-hw_dma_xilinx-hw_net_virtio

I think the subject should be what was done rather than where the work
came from.

Paolo has already mentioned that this may not be a good idea,
especially for virtio so the following comments are general QEMU style
comments for future reference.

>
> ---
>  hw/dma/xilinx_axidma.c |  5 -
>  hw/net/virtio-net.c| 11 ++-
>  2 files changed, 14 insertions(+), 2 deletions(-)
>
> diff --git a/hw/dma/xilinx_axidma.c b/hw/dma/xilinx_axidma.c
> index ce5c1e6..9c6bda2 100644
> --- a/hw/dma/xilinx_axidma.c
> +++ b/hw/dma/xilinx_axidma.c
> @@ -255,13 +255,15 @@ static void stream_process_mem2s(struct Stream *s,
> StreamSlave *tx_data_dev,
>   StreamSlave *tx_control_dev)
>  {
>  uint32_t prev_d;
> -unsigned char txbuf[16 * 1024];
> +unsigned char *txbuf;
>  unsigned int txlen;
>
>  if (!stream_running(s) || stream_idle(s)) {
>  return;
>  }
>
> +txbuf = (unsigned char *) malloc(16 * 1024 * sizeof(unsigned
> char));

QEMU uses g_malloc/g_free wrappers for memory allocation. For one thing
g_malloc can never fail where as here you may end up with a NULL ptr.

> +
>  while (1) {
>  stream_desc_load(s, s->regs[R_CURDESC]);
>
> @@ -303,6 +305,7 @@ static void stream_process_mem2s(struct Stream *s,
> StreamSlave *tx_data_dev,
>  break;
>  }
>  }
> +free(txbuf);
>  }
>
>  static size_t stream_process_s2mem(struct Stream *s, unsigned char *buf,
> diff --git a/hw/net/virtio-net.c b/hw/net/virtio-net.c
> index 5798f87..ba6ebac 100644
> --- a/hw/net/virtio-net.c
> +++ b/hw/net/virtio-net.c
> @@ -1094,7 +1094,7 @@ static ssize_t virtio_net_receive(NetClientState *nc,
> const uint8_t *buf, size_t
>  VirtIONet *n = qemu_get_nic_opaque(nc);
>  VirtIONetQueue *q = virtio_net_get_subqueue(nc);
>  VirtIODevice *vdev = VIRTIO_DEVICE(n);
> -struct iovec mhdr_sg[VIRTQUEUE_MAX_SIZE];
> +struct iovec *mhdr_sg;
>  struct virtio_net_hdr_mrg_rxbuf mhdr;
>  unsigned mhdr_cnt = 0;
>  size_t offset, i, guest_offset;
> @@ -1113,6 +1113,8 @@ static ssize_t virtio_net_receive(NetClientState *nc,
> const uint8_t *buf, size_t
>
>  offset = i = 0;
>
> +mhdr_sg = (struct iovec *) malloc(VIRTQUEUE_MAX_SIZE * sizeof(struct
> iovec));
> +
>  while (offset < size) {
>  VirtQueueElement *elem;
>  int len, total;
> @@ -1122,6 +1124,7 @@ static ssize_t virtio_net_receive(NetClientState *nc,
> const uint8_t *buf, size_t
>
>  elem = virtqueue_pop(q->rx_vq, sizeof(VirtQueueElement));
>  if (!elem) {
> +free(mhdr_sg);
>  if (i == 0)
>  return -1;
>  error_report("virtio-net unexpected empty queue: "
> @@ -1136,11 +1139,15 @@ static ssize_t virtio_net_receive(NetClientState
> *nc, const uint8_t *buf, size_t
>
>  if (elem->in_num < 1) {
>  error_report("virtio-net receive queue contains no in
> buffers");
> +free(mhdr_sg);
>  exit(1);
>  }
>
>  sg = elem->in_sg;
>  if (i == 0) {
> +if (offset != 0) {
> +free(mhdr_sg);
> +}
>  assert(offset == 0);
>  if (n->mergeable_rx_bufs) {
>  mhdr_cnt = iov_copy(mhdr_sg, ARRAY_SIZE(mhdr_sg),
> @@ -1168,6 +1175,7 @@ static ssize_t virtio_net_receive(NetClientState *nc,
> const uint8_t *buf, size_t
>  if (!n->mergeable_rx_bufs && offset < size) {
>  virtqueue_discard(q->rx_vq, elem, total);
>  g_free(elem);
> +free(mhdr_sg);
>  return size;
>  }
>
> @@ -1186,6 +1194,7 @@ static ssize_t virtio_net_receive(NetClientState *nc,
> const uint8_t *buf, size_t
>  virtqueue_flush(q->rx_vq, i);
>  virtio_notify(vdev, q->rx_vq);
>
> +free(mhdr_sg);
>  return size;
>  }


--
Alex Bennée

Re: [Qemu-devel] [PATCH] iotests: Correct 081's reference output

[Max Reitz]

On 11.03.2016 15:14, Max Reitz wrote:
> The newly added type parameter for the QUORUM_REPORT_BAD event changed
> the output of iotest 081, so the reference should be amended
> accordingly.
> 
> Signed-off-by: Max Reitz 
> ---
>  tests/qemu-iotests/081.out | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Applied to my block tree.

Max



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 15/15] block: More operations for meta dirty bitmap

[Max Reitz]

On 08.03.2016 05:45, Fam Zheng wrote:
> Callers can create an iterator of meta bitmap with
> bdrv_dirty_meta_iter_new(), then use the bdrv_dirty_iter_* operations on
> it. Meta iterators are also counted by bitmap->active_iterators.
> 
> Also add a couple of functions to retrieve granularity and count.
> 
> Signed-off-by: Fam Zheng 
> ---
>  block/dirty-bitmap.c | 19 +++
>  include/block/dirty-bitmap.h |  3 +++
>  2 files changed, 22 insertions(+)

Reviewed-by: Max Reitz 



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v4 12/15] hbitmap: serialization

[Max Reitz]

On 08.03.2016 05:45, Fam Zheng wrote:
> From: Vladimir Sementsov-Ogievskiy 
> 
> Functions to serialize / deserialize(restore) HBitmap. HBitmap should be
> saved to linear sequence of bits independently of endianness and bitmap
> array element (unsigned long) size. Therefore Little Endian is chosen.
> 
> These functions are appropriate for dirty bitmap migration, restoring
> the bitmap in several steps is available. To save performance, every
> step writes only the last level of the bitmap. All other levels are
> restored by hbitmap_deserialize_finish() as a last step of restoring.
> So, HBitmap is inconsistent while restoring.
> 
> Signed-off-by: Vladimir Sementsov-Ogievskiy 
> [Fix left shift operand to 1UL; add "finish" parameter. - Fam]
> Signed-off-by: Fam Zheng 
> ---
>  include/qemu/hbitmap.h |  79 
>  util/hbitmap.c | 137 
> +
>  2 files changed, 216 insertions(+)

[...]

> diff --git a/util/hbitmap.c b/util/hbitmap.c
> index 2d3d04c..5f02c17 100644
> --- a/util/hbitmap.c
> +++ b/util/hbitmap.c
> @@ -395,6 +395,143 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item)
>  return (hb->levels[HBITMAP_LEVELS - 1][pos >> BITS_PER_LEVEL] & bit) != 
> 0;
>  }
>  
> +uint64_t hbitmap_serialization_granularity(const HBitmap *hb)
> +{
> +/* Require at least 64 bit granularity to be safe on both 64 bit and 32 
> bit
> + * hosts. */
> +return 64 << hb->granularity;
> +}
> +
> +/* Start should be aligned to serialization granularity, chunk size should be
> + * aligned to serialization granularity too, except for last chunk.
> + */
> +static void serialization_chunk(const HBitmap *hb,
> +uint64_t start, uint64_t count,
> +unsigned long **first_el, size_t *el_count)
> +{
> +uint64_t last = start + count - 1;
> +uint64_t gran = hbitmap_serialization_granularity(hb);
> +
> +assert((start & (gran - 1)) == 0);
> +assert((last >> hb->granularity) < hb->size);
> +if ((last >> hb->granularity) != hb->size - 1) {
> +assert((count & (gran - 1)) == 0);
> +}
> +
> +start = (start >> hb->granularity) >> BITS_PER_LEVEL;
> +last = (last >> hb->granularity) >> BITS_PER_LEVEL;
> +
> +*first_el = >levels[HBITMAP_LEVELS - 1][start];
> +*el_count = last - start + 1;
> +}
> +
> +uint64_t hbitmap_serialization_size(const HBitmap *hb,
> +uint64_t start, uint64_t count)
> +{
> +uint64_t el_count;
> +unsigned long *cur;
> +
> +if (!count) {
> +return 0;
> +}
> +serialization_chunk(hb, start, count, , _count);
> +
> +return el_count * sizeof(unsigned long);
> +}
> +
> +void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
> +uint64_t start, uint64_t count)
> +{
> +uint64_t el_count;
> +unsigned long *cur, *end;
> +
> +if (!count) {
> +return;
> +}
> +serialization_chunk(hb, start, count, , _count);
> +end = cur + el_count;
> +
> +while (cur != end) {
> +unsigned long el =
> +(BITS_PER_LONG == 32 ? cpu_to_le32(*cur) : cpu_to_le64(*cur));

Looks a bit fishy, but I can't come up with anything better.

(Other than adding cpu_to_leul(); we already do have leul_to_cpu(), so
that wouldn't be too far off.)

> +
> +memcpy(buf, , sizeof(el));

One could have used cpu_to_le32/64w((uint32/64_t *)buf, *cur); instead.

Maybe I'd like the following better:

#if BITS_PER_LONG == 32
cpu_to_le32w((uint32_t *)buf, *cur);
#elif BITS_PER_LONG == 64
cpu_to_le64w((uint64_t *)buf, *cur);
#else
#error Unknown long size
#endif

Or just

#else /* BITS_PER_LONG == 64 */

instead of the #elif. I think that's safe to assume.

> +buf += sizeof(el);
> +cur++;
> +}
> +}
> +
> +void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
> +  uint64_t start, uint64_t count,
> +  bool finish)
> +{
> +uint64_t el_count;
> +unsigned long *cur, *end;
> +
> +if (!count) {
> +return;
> +}
> +serialization_chunk(hb, start, count, , _count);
> +end = cur + el_count;
> +
> +while (cur != end) {
> +memcpy(cur, buf, sizeof(*cur));
> +
> +if (BITS_PER_LONG == 32) {
> +le32_to_cpus((uint32_t *)cur);
> +} else {
> +le64_to_cpus((uint64_t *)cur);
> +}

Here, I'd definitely like that variant better, i.e.

#if BITS_PER_LONG == 32
le32_to_cpuw(cur, *(uint32_t *)buf);
#else /* BITS_PER_LONG == 64 */
le64_to_cpuw(cur, *(uint64_t *)buf);
#endif

Unless a language lawyer NACKs this because the pointer cast violates
strict aliasing.

If so, I still strongly recommend replacing the if by an #if, not least
because this saves us the pointer cast on cur.

(Or does it? Maybe one still needs to 

Re: [Qemu-devel] [PATCH v3 00/13] tests: Introducing docker tests

[Alex Bennée]

Fam Zheng  writes:

> v3 changes:

I think we are almost there. There a just a few tweaks to be made to
help text and prompts. Can you ensure that all examples in commit
messages and help text actually do run as expected?

Is it proposed this goes through Daniel's treee?

>   - Merge all docker_* tools into docker.py as subcommands, and add simple 
> help
> texts; clean up docker.py a bit.
>   - For ease of management of the series, squashed Alex's COPY_SRC patch into
> patch 2.
>   - Pick up Alex's tweaks on Makefile changes.
>   - Change the way we copy source. Now we send source tarballs to the
> container, and untar it there. Beside qemu.tgz, also copy dtc.tgz and
> pixman.tgz if submodules are initialized in the tree.  The first is
> required by mingw test.
>   - Update test runner to adapt to above change.
>   - Tweak "make docker" help text. Dropped "docker-@IMAGE" because it is
> equivalent to "make docker-test IMAGES=XXX". Too many targets will be hard
> to memorize, and we can always add more shortcuts if desired.
>   - Drop "PAUSE=1" env and add "DEBUG=1", which will also enable networking,
> and drop to shell when test fails.
>   - Add "NOCACHE=1" env var to add "--no-cache" to "docker build" command,
> which is useful in certain cases to workaround image build failure. For
> example in ubuntu when "apt-get update" is cached, "apt-get install xxx"
> can get 404.
>   - Add libfdt-devel to images.
>   - Add epel, and ccache in centos6.
>   - Add "TARGET_LIST=" env var.
>   - Add "sparse" from multiverse for ubuntu, which is used by travis tool.
>
> This series adds a new "docker" make target family to run tests in created
> docker containers.
>
> To begin with, this can be a place to store standard env/command combinations 
> to
> build and test QEMU.
>
> Secondly, CI usually provides "docker" capability, where we specify
> standard/repeatable test environments, and run tests in them.  However, what
> tests to cover is better maintained in-tree, in order to keep in sync with the
> code development.
>
> Lastly, this makes it very simple for developers to replicate such tests
> themselves.
>
>
> Fam Zheng (13):
>   tests: Add utilities for docker testing
>   Makefile: Rules for docker testing
>   docker: Add images
>   docker: Add test runner
>   docker: Add common.rc
>   docker: Add quick test
>   docker: Add full test
>   docker: Add clang test
>   docker: Add mingw test
>   docker: Add travis tool
>   docs: Add text for tests/docker in build-system.txt
>   .gitignore: Ignore temporary dockerfile
>   MAINTAINERS: Add tests/docker
>
>  .gitignore  |   1 +
>  MAINTAINERS |   7 ++
>  Makefile|   4 +-
>  docs/build-system.txt   |   5 +
>  tests/docker/Makefile.include   | 121 +
>  tests/docker/common.rc  |  31 ++
>  tests/docker/docker.py  | 180 
> 
>  tests/docker/dockerfiles/centos6.docker |   6 ++
>  tests/docker/dockerfiles/fedora.docker  |   7 ++
>  tests/docker/dockerfiles/ubuntu.docker  |  11 ++
>  tests/docker/run|  58 ++
>  tests/docker/test-clang |  25 +
>  tests/docker/test-full  |  17 +++
>  tests/docker/test-mingw |  34 ++
>  tests/docker/test-quick |  19 
>  tests/docker/travis |  21 
>  tests/docker/travis.py  |  48 +
>  17 files changed, 594 insertions(+), 1 deletion(-)
>  create mode 100644 tests/docker/Makefile.include
>  create mode 100755 tests/docker/common.rc
>  create mode 100755 tests/docker/docker.py
>  create mode 100644 tests/docker/dockerfiles/centos6.docker
>  create mode 100644 tests/docker/dockerfiles/fedora.docker
>  create mode 100644 tests/docker/dockerfiles/ubuntu.docker
>  create mode 100755 tests/docker/run
>  create mode 100755 tests/docker/test-clang
>  create mode 100755 tests/docker/test-full
>  create mode 100755 tests/docker/test-mingw
>  create mode 100755 tests/docker/test-quick
>  create mode 100755 tests/docker/travis
>  create mode 100755 tests/docker/travis.py


--
Alex Bennée

Re: [Qemu-devel] [PATCH v3 12/13] .gitignore: Ignore temporary dockerfile

[Alex Bennée]

Fam Zheng  writes:

> Docker build requires a "context" directory and we use the
> $QEMU_SRC/tests/docker/ directory, and the temoprary dockerfile has to be in
> the context.
>
> docker_build normally cleans up this file but let's add an entry here just in
> case it fails to.

I think the need for this will go away if you use python's tempfile.

>
> Signed-off-by: Fam Zheng 
> ---
>  .gitignore | 1 +
>  1 file changed, 1 insertion(+)
>
> diff --git a/.gitignore b/.gitignore
> index 88a80ff..a335b7b 100644
> --- a/.gitignore
> +++ b/.gitignore
> @@ -109,3 +109,4 @@ cscope.*
>  tags
>  TAGS
>  *~
> +/tests/docker/*.docker.tmp


--
Alex Bennée

Re: [Qemu-devel] [PATCH v3 11/13] docs: Add text for tests/docker in build-system.txt

[Alex Bennée]

Fam Zheng  writes:

> Signed-off-by: Fam Zheng 

Reviewed-by: Alex Bennée 

> ---
>  docs/build-system.txt | 5 +
>  1 file changed, 5 insertions(+)
>
> diff --git a/docs/build-system.txt b/docs/build-system.txt
> index 5ea..2af1e66 100644
> --- a/docs/build-system.txt
> +++ b/docs/build-system.txt
> @@ -438,6 +438,11 @@ top level Makefile, so anything defined in this file 
> will influence the
>  entire build system. Care needs to be taken when writing rules for tests
>  to ensure they only apply to the unit test execution / build.
>
> +- tests/docker/Makefile.include
> +
> +Rules for Docker tests. Like tests/Makefile, this file is included
> +directly by the top level Makefile, anything defined in this file will
> +influence the entire build system.
>
>  - po/Makefile


--
Alex Bennée

Re: [Qemu-devel] [PATCH v3 10/13] docker: Add travis tool

[Alex Bennée]

Fam Zheng  writes:

> The script is not named test-travis.sh so it won't run with "make
> docker-run", because it can take too long.
>
> Run it with "make docker-run-travis.sh@ubuntu".

16:08 alex@zen/x86_64  [qemu.git/review/docker-v3] >make 
docker-run-travis.sh@ubuntu
ARCHIVE qemu.tgz
COPY RUNNER
RUN travis.sh in ubuntu
./run: line 49: /tmp/qemu-test/src/tests/docker/travis.sh: No such file or 
directory

>
> Signed-off-by: Fam Zheng 
> ---
>  tests/docker/travis| 21 +
>  tests/docker/travis.py | 48 
>  2 files changed, 69 insertions(+)
>  create mode 100755 tests/docker/travis
>  create mode 100755 tests/docker/travis.py
>
> diff --git a/tests/docker/travis b/tests/docker/travis
> new file mode 100755
> index 000..d345393
> --- /dev/null
> +++ b/tests/docker/travis
> @@ -0,0 +1,21 @@
> +#!/bin/bash -e
> +#
> +# Mimic a travis testing matrix
> +#
> +# Copyright (c) 2016 Red Hat Inc.
> +#
> +# Authors:
> +#  Fam Zheng 
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +. common.rc
> +
> +requires pyyaml
> +cmdfile=/tmp/travis_cmd_list.sh
> +$QEMU_SRC/tests/docker/travis.py $QEMU_SRC/.travis.yml > $cmdfile
> +chmod +x $cmdfile
> +cd "$QEMU_SRC"
> +$cmdfile
> diff --git a/tests/docker/travis.py b/tests/docker/travis.py
> new file mode 100755
> index 000..8dcc964
> --- /dev/null
> +++ b/tests/docker/travis.py
> @@ -0,0 +1,48 @@
> +#!/usr/bin/env python
> +#
> +# Travis YAML config parser
> +#
> +# Copyright (c) 2016 Red Hat Inc.
> +#
> +# Authors:
> +#  Fam Zheng 
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +import sys
> +import yaml
> +import itertools
> +
> +def load_yaml(fname):
> +return yaml.load(open(fname, "r").read())
> +
> +def conf_iter(conf):
> +def env_to_list(env):
> +return env if isinstance(env, list) else [env]
> +global_env = conf["env"]["global"]
> +for entry in conf["matrix"]["include"]:
> +yield {"env": global_env + env_to_list(entry["env"]),
> +   "compiler": entry["compiler"]}
> +for entry in itertools.product(conf["compiler"],
> +   conf["env"]["matrix"]):
> +yield {"env": global_env + env_to_list(entry[1]),
> +   "compiler": entry[0]}
> +
> +def main():
> +if len(sys.argv) < 2:
> +sys.stderr.write("Usage: %s \n" % sys.argv[0])
> +return 1
> +conf = load_yaml(sys.argv[1])
> +for config in conf_iter(conf):
> +print "("
> +print "\n".join(config["env"])
> +print "alias cc=" + config["compiler"]
> +print "\n".join(conf["before_script"])
> +print "\n".join(conf["script"])
> +print ")"
> +return 0
> +
> +if __name__ == "__main__":
> +sys.exit(main())


--
Alex Bennée

Re: [Qemu-devel] [PATCH v3 09/13] docker: Add mingw test

[Alex Bennée]

Fam Zheng  writes:

> Signed-off-by: Fam Zheng 

Reviewed-by: Alex Bennée 

> ---
>  tests/docker/test-mingw | 34 ++
>  1 file changed, 34 insertions(+)
>  create mode 100755 tests/docker/test-mingw
>
> diff --git a/tests/docker/test-mingw b/tests/docker/test-mingw
> new file mode 100755
> index 000..c03757a
> --- /dev/null
> +++ b/tests/docker/test-mingw
> @@ -0,0 +1,34 @@
> +#!/bin/bash -e
> +#
> +# Cross compile QEMU with mingw toolchain on Linux.
> +#
> +# Copyright (c) 2016 Red Hat Inc.
> +#
> +# Authors:
> +#  Fam Zheng 
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +. common.rc
> +
> +requires mingw dtc
> +
> +for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
> +TARGET_LIST=x86_64-softmmu,aarch64-softmmu \
> +build_qemu --cross-prefix=$prefix \
> +--enable-trace-backends=simple \
> +--enable-debug \
> +--enable-gnutls \
> +--enable-nettle \
> +--enable-curl \
> +--enable-vnc \
> +--enable-bzip2 \
> +--enable-guest-agent \
> +--with-sdlabi=1.2 \
> +--with-gtkabi=2.0
> +make clean
> +
> +done
> +


--
Alex Bennée

Re: [Qemu-devel] [RFC PATCH v2 3/3] VFIO: Type1 IOMMU mapping support for vGPU

[Alex Williamson]

On Fri, 11 Mar 2016 04:46:23 +
"Tian, Kevin"  wrote:

> > From: Neo Jia [mailto:c...@nvidia.com]
> > Sent: Friday, March 11, 2016 12:20 PM
> > 
> > On Thu, Mar 10, 2016 at 11:10:10AM +0800, Jike Song wrote:  
> > >  
> > > >> Is it supposed to be the caller who should set
> > > >> up IOMMU by DMA api such as dma_map_page(), after calling
> > > >> vgpu_dma_do_translate()?
> > > >>  
> > > >
> > > > Don't think you need to call dma_map_page here. Once you have the pfn 
> > > > available
> > > > to your GPU kernel driver, you can just go ahead to setup the mapping 
> > > > as you
> > > > normally do such as calling pci_map_sg and its friends.
> > > >  
> > >
> > > Technically it's definitely OK to call DMA API from the caller rather 
> > > than here,
> > > however personally I think it is a bit counter-intuitive: IOMMU page 
> > > tables
> > > should be constructed within the VFIO IOMMU driver.
> > >  
> > 
> > Hi Jike,
> > 
> > For vGPU, what we have is just a virtual device and a fake IOMMU group, 
> > therefore
> > the actual interaction with the real GPU should be managed by the GPU 
> > vendor driver.
> >   
> 
> Hi, Neo,
> 
> Seems we have a different thought on this. Regardless of whether it's a 
> virtual/physical 
> device, imo, VFIO should manage IOMMU configuration. The only difference is:
> 
> - for physical device, VFIO directly invokes IOMMU API to set IOMMU entry 
> (GPA->HPA);
> - for virtual device, VFIO invokes kernel DMA APIs which indirectly lead to 
> IOMMU entry 
> set if CONFIG_IOMMU is enabled in kernel (GPA->IOVA);
> 
> This would provide an unified way to manage the translation in VFIO, and then 
> vendor
> specific driver only needs to query and use returned IOVA corresponding to a 
> GPA. 
> 
> Doing so has another benefit, to make underlying vGPU driver VMM agnostic. 
> For KVM,
> yes we can use pci_map_sg. However for Xen it's different (today Dom0 doesn't 
> see
> IOMMU. In the future there'll be a PVIOMMU implementation) so different code 
> path is 
> required. It's better to abstract such specific knowledge out of vGPU driver, 
> which just
> uses whatever dma_addr returned by other agent (VFIO here, or another Xen 
> specific
> agent) in a centralized way.
> 
> Alex, what's your opinion on this?

The sticky point is how vfio, which is only handling the vGPU, has a
reference to the physical GPU on which to call DMA API operations.  If
that reference is provided by the vendor vGPU driver, for example
vgpu_dma_do_translate_for_pci(gpa, pci_dev), I don't see any reason to
be opposed to such an API.  I would not condone vfio deriving or owning
a reference to the physical device on its own though, that's in the
realm of the vendor vGPU driver.  It does seem a bit cleaner and should
reduce duplicate code if the vfio vGPU iommu interface could handle the
iommu mapping for the vendor vgpu driver when necessary.  Thanks,

Alex

Re: [Qemu-devel] [PATCH v3 08/13] docker: Add clang test

[Alex Bennée]

Fam Zheng  writes:

> The (currently partially commented out) configure options are suggested
> by John Snow .
>
> Signed-off-by: Fam Zheng 

Reviewed-by: Alex Bennée 

> ---
>  tests/docker/test-clang | 25 +
>  1 file changed, 25 insertions(+)
>  create mode 100755 tests/docker/test-clang
>
> diff --git a/tests/docker/test-clang b/tests/docker/test-clang
> new file mode 100755
> index 000..7b5e65e
> --- /dev/null
> +++ b/tests/docker/test-clang
> @@ -0,0 +1,25 @@
> +#!/bin/bash -e
> +#
> +# Compile and check with clang.
> +#
> +# Copyright (c) 2016 Red Hat Inc.
> +#
> +# Authors:
> +#  Fam Zheng 
> +#
> +# This work is licensed under the terms of the GNU GPL, version 2
> +# or (at your option) any later version. See the COPYING file in
> +# the top-level directory.
> +
> +. common.rc
> +
> +requires clang
> +
> +OPTS="--enable-debug --cxx=clang++ --cc=clang --host-cc=clang"
> +# -fsanitize=undefined is broken on Fedora 23, skip it for now
> +# See also: https://bugzilla.redhat.com/show_bug.cgi?id=1263834
> +#OPTS="$OPTS --extra-cflags=-fsanitize=undefined \
> +#--extra-cflags=-fno-sanitize=float-divide-by-zero"
> +TARGET_LIST=x86_64-softmmu,aarch64-softmmu
> +build_qemu $OPTS
> +make $MAKEFLAGS check


--
Alex Bennée