Re: [Qemu-devel] [PATCH] cpu: cpu_save/cpu_load is no more

[Michael Tokarev]

19.01.2016 15:08, Paolo Bonzini wrote:
> Everything has been converted to vmstate.

Applied to -trivial, thanks!

/mjt\

Re: [Qemu-devel] [PATCH v1 1/1] qom: Correct object_property_get_int() description

[Michael Tokarev]

18.01.2016 21:42, Alistair Francis wrote:
> The description of object_property_get_int() stated that on an error
> it returns NULL. This is not the case and the function will return -1
> if an error occurs. Update the commented documentation accordingly.

Applied to -trivial, thanks!

/mjt

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Roy Shterman]

You were right, I worked with old libvirt, Thanks!

I have another question about QEMU memory allocation,

There is a possibility to put in libvirt :




On Thu, Jan 28, 2016 at 10:53 AM, Daniel P. Berrange 
wrote:

> On Thu, Jan 28, 2016 at 09:46:26AM +0100, Paolo Bonzini wrote:
> >
> >
> > On 28/01/2016 08:28, Roy Shterman wrote:
> > >
> > > Important to understand that after modifying and saving configuration
> of
> > > xml with virsh edit $name_of_guest
> > >
> > > when i reenter the xml i can't see the iothread configuration in there.
> > > don't understand why.
> > >
> >
> > I suspect that your libvirt is too old.
>
> iothread support requires libvirt 1.2.8 or newer
>
> Regards,
> Daniel
> --
> |: http://berrange.com  -o-http://www.flickr.com/photos/dberrange/
> :|
> |: http://libvirt.org  -o- http://virt-manager.org
> :|
> |: http://autobuild.org   -o- http://search.cpan.org/~danberr/
> :|
> |: http://entangle-photo.org   -o-   http://live.gnome.org/gtk-vnc
> :|
>

Re: [Qemu-devel] [PATCH 1/1] arm: virt: change GPIO trigger interrupt to pulse

[Shannon Zhao]

On 2016/1/29 22:35, Wei Huang wrote:
> 
> On 01/29/2016 04:10 AM, Shannon Zhao wrote:
>> > Hi，
>> > 
>> > This makes ACPI work well but makes DT not work. The reason is systemd or
>> > acpid open /dev/input/event0 failed.
To correct, systemd or acpid open /dev/input/event0 successfully but it
waits for events and when we input "system_powerdown", it doesn't get
the event.

> So the interrupt could be injected and
>> > could see under /proc/interrupts but guest doesn't have any action. I'll
>> > investigate why it opens failed later.
> That is interesting. Could you try it with the following? This reverses
> the order to down-up and worked on ACPI case.
> 
> qemu_set_irq(qdev_get_gpio_in(pl061_dev, 3), 0);
> qemu_set_irq(qdev_get_gpio_in(pl061_dev, 3), 1);
If we don't input "system_powerdown" during guest booting(before device
driver load), current QEMU with above codes could work for DT. But if we
input "system_powerdown" during guest booting, guest has no action when
we input "system_powerdown" again after guest booting completely.
I add dc->reset for pl061 but get the same result.

Below is the output of acpid:

./acpid -l -n -d
input layer /dev/input/event0 (gpio-keys) opened successfully, fd 4
inotify fd: 5
inotify wd: 1
RTNETLINK1 answers: No such file or directory
acpid: error talking to the kernel via netlink
netlink opened successfully
acpid: starting up with netlink and the input layer
parsing conf file /etc/acpi/events/powerbtn
acpid: 1 rule loaded
acpid: waiting for events: event logging is on

It will stuck here even if we input "system_powerdown".

Below is the right output when we input "system_powerdown".

./acpid -l -n -d
input layer /dev/input/event0 (gpio-keys) opened successfully, fd 4
inotify fd: 5
inotify wd: 1
RTNETLINK1 answers: No such file or directory
acpid: error talking to the kernel via netlink
netlink opened successfully
acpid: starting up with netlink and the input layer
parsing conf file /etc/acpi/events/powerbtn
acpid: 1 rule loaded
acpid: waiting for events: event logging is on
acpid: received input layer event "button/power PBTN 0080 "
acpid: rule from /etc/acpi/events/powerbtn matched
acpid: executing action "/etc/acpi/powerbtn.sh"

Thanks,
-- 
Shannon

Re: [Qemu-devel] virtio-scsi/blk dataplane and guest memory allocation

[Roy Shterman]

Sorry about the spam...

You were right, I worked with old libvirt, Thanks!

I have another question about QEMU memory allocation,

There is a possibility to put in libvirt :


  


This is suppose to lock all the memory that has been allocated for the
guest into host's ram,

Is there any way to just upraise the maximum available pinnable-memory of
guest? without locking all 2G (in my guest)

because default maximum pinnable memory in QEMU is 60k.

Thank you all for your help,
Roy

On Sat, Jan 30, 2016 at 10:29 AM, Roy Shterman 
wrote:

> You were right, I worked with old libvirt, Thanks!
>
> I have another question about QEMU memory allocation,
>
> There is a possibility to put in libvirt :
>
> 
>
>
> On Thu, Jan 28, 2016 at 10:53 AM, Daniel P. Berrange 
> wrote:
>
>> On Thu, Jan 28, 2016 at 09:46:26AM +0100, Paolo Bonzini wrote:
>> >
>> >
>> > On 28/01/2016 08:28, Roy Shterman wrote:
>> > >
>> > > Important to understand that after modifying and saving configuration
>> of
>> > > xml with virsh edit $name_of_guest
>> > >
>> > > when i reenter the xml i can't see the iothread configuration in
>> there.
>> > > don't understand why.
>> > >
>> >
>> > I suspect that your libvirt is too old.
>>
>> iothread support requires libvirt 1.2.8 or newer
>>
>> Regards,
>> Daniel
>> --
>> |: http://berrange.com  -o-
>> http://www.flickr.com/photos/dberrange/ :|
>> |: http://libvirt.org  -o-
>> http://virt-manager.org :|
>> |: http://autobuild.org   -o-
>> http://search.cpan.org/~danberr/ :|
>> |: http://entangle-photo.org   -o-
>> http://live.gnome.org/gtk-vnc :|
>>
>
>

Re: [Qemu-devel] [PATCH COLO-Frame v13 10/39] COLO: Implement colo checkpoint protocol

[Hailiang Zhang]

On 2016/1/29 21:08, Dr. David Alan Gilbert wrote:

* zhanghailiang (zhang.zhanghaili...@huawei.com) wrote:

We need communications protocol of user-defined to control the checkpoint
process.

The new checkpoint request is started by Primary VM, and the interactive process
like below:
Checkpoint synchronizing points:

Primary   Secondary
 initial work
'checkpoint-ready'< @

'checkpoint-request'  @ >
 Suspend (Only in hybrid mode)
'checkpoint-reply'< @
   Suspend state
'vmstate-send'@ >
   Send stateReceive state
'vmstate-received'< @
   Release packets   Load state
'vmstate-load'< @
   ResumeResume (Only in hybrid mode)

   Start Comparing (Only in hybrid mode)
NOTE:
  1) '@' who sends the message
  2) Every sync-point is synchronized by two sides with only
 one handshake(single direction) for low-latency.
 If more strict synchronization is required, a opposite direction
 sync-point should be added.
  3) Since sync-points are single direction, the remote side may
 go forward a lot when this side just receives the sync-point.
  4) For now, we only support 'periodic' checkpoint, for which
the Secondary VM is not running, later we will support 'hybrid' mode.

Signed-off-by: zhanghailiang 
Signed-off-by: Li Zhijian 
Signed-off-by: Gonglei 
Cc: Eric Blake 
Cc: Markus Armbruster 
Cc: Dr. David Alan Gilbert 
---
v13:
- Refactor colo command related helper functions, use 'Error **errp' parameter
   instead of return value to indicate success or failure.
- Fix some other comments from Markus.

v12:
- Rename colo_ctl_put() to colo_put_cmd()
- Rename colo_ctl_get() to colo_get_check_cmd() and drop
   the third parameter
- Rename colo_ctl_get_cmd() to colo_get_cmd()
- Remove useless 'invalid' member for COLOcommand enum.
v11:
- Add missing 'checkpoint-ready' communication in comment.
- Use parameter to return 'value' for colo_ctl_get() (Dave's suggestion)
- Fix trace for colo_ctl_get() to trace command and value both
v10:
- Rename enum COLOCmd to COLOCommand (Eric's suggestion).
- Remove unused 'ram-steal'
---
  migration/colo.c | 201 ++-
  qapi-schema.json |  25 +++
  trace-events |   2 +
  3 files changed, 226 insertions(+), 2 deletions(-)

diff --git a/migration/colo.c b/migration/colo.c
index 65ac0c9..c76e1fa 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -10,6 +10,7 @@
   * later.  See the COPYING file in the top-level directory.
   */

+#include 
  #include "sysemu/sysemu.h"
  #include "migration/colo.h"
  #include "trace.h"
@@ -34,22 +35,147 @@ bool migration_incoming_in_colo_state(void)
  return mis && (mis->state == MIGRATION_STATUS_COLO);
  }

+static void colo_put_cmd(QEMUFile *f, COLOCommand cmd,
+ Error **errp)
+{
+int ret;
+
+if (cmd >= COLO_COMMAND__MAX) {
+error_setg(errp, "%s: Invalid cmd", __func__);
+return;
+}
+qemu_put_be32(f, cmd);
+qemu_fflush(f);
+
+ret = qemu_file_get_error(f);
+if (ret < 0) {
+error_setg_errno(errp, -ret, "Can't put COLO command");
+}
+trace_colo_put_cmd(COLOCommand_lookup[cmd]);
+}
+
+static COLOCommand colo_get_cmd(QEMUFile *f, Error **errp)
+{
+COLOCommand cmd;
+int ret;
+
+cmd = qemu_get_be32(f);
+ret = qemu_file_get_error(f);
+if (ret < 0) {
+error_setg_errno(errp, -ret, "Can't get COLO command");
+return cmd;
+}
+if (cmd >= COLO_COMMAND__MAX) {


I'm not sure this is guaranteed to do the right thing if the value
read was negative and converted to the enum;  it seems to depend
a bit on the compiler as to whether an enum is signed or not;
but it seems the normal behaviour is for the enum to be unsigned


Interesting question, we can define a negative enumeration-constant,
enum test {
TEST_A = -2,
TEST_B = 0,
} value;
So i suppose an enum is signed, and ISO C standard (c99 6.4.4.3) states that
the enumeration constants are of type int.
But it seems that gcc does not follow this standard strictly,
i got the exact answer from its specification:
https://gcc.gnu.org/onlinedocs/gcc/Structures-unions-enumerations-and-bit-fields-implementation.html#Structures-unions-enumerations-and-bit-fields-implementation
"Normally, the type is unsigned int if there are no negative values in the 
enumeration, otherwise int."


as long as there are no negative values when it's declared; so it looks
safe in practice.

Reviewed-by: Dr. 

Re: [Qemu-devel] [PATCH v2] man: virtfs-proxy-helper: Rework awkward sentence

[Michael Tokarev]

18.01.2016 12:51, Christophe Fergeau wrote:
> There was a 'capbilities' typo in this man page. This commit
> reformulates the sentence the typo was in to make it easier to grasp.
> This is based on a suggestion from Eric Blake.

Applied to -trivial, thank you!

/mjt

Re: [Qemu-devel] [PATCH] qemu-sockets: simplify error handling

[Michael Tokarev]

22.01.2016 14:28, Paolo Bonzini wrote:
> Just go always through the err label.  (Noticed because Coverity
> complains that peer is always non-NULL in the error cleanup code,
> but removing the "if" is arguably more prone to introducing the
> opposite bug in the future).

Applied to -trivial, thanks!

/mjt

[Qemu-devel] [PATCH v12 7/9] hw/ptimer: Fix counter - 1 returned by ptimer_get_count for the active timer

[Dmitry Osipenko]

Due to rounding down performed by ptimer_get_count, it returns counter - 1 for
the active timer. That's incorrect because counter should decrement only after
period been expired, not before. I.e. if running timer has been loaded with
value X, then timer counter should stay with X until period expired and
decrement after. Fix this by adding 1 to the counter value for the active and
unexpired timer.

Signed-off-by: Dmitry Osipenko 
---
 hw/core/ptimer.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/core/ptimer.c b/hw/core/ptimer.c
index 62f8cb1..b2044fb 100644
--- a/hw/core/ptimer.c
+++ b/hw/core/ptimer.c
@@ -136,7 +136,7 @@ uint64_t ptimer_get_count(ptimer_state *s)
 if ((uint32_t)(period_frac << shift))
 div += 1;
 }
-counter = rem / div;
+counter = rem / div + (expired ? 0 : 1);
 
 if (expired && counter != 0) {
 /* Wrap around periodic counter.  */
-- 
2.7.0

[Qemu-devel] [PATCH v12 8/9] hw/ptimer: Perform delayed tick instead of immediate if delta = 0

[Dmitry Osipenko]

It might be necessary by some emulated HW to perform the tick after one
period if delta = 0. Given that it is much less churny to implement immediate
tick by the ptimer user itself, let's make ptimer do the delayed tick.

Signed-off-by: Dmitry Osipenko 
---
 hw/core/ptimer.c | 34 +++---
 1 file changed, 15 insertions(+), 19 deletions(-)

diff --git a/hw/core/ptimer.c b/hw/core/ptimer.c
index b2044fb..bcd090c 100644
--- a/hw/core/ptimer.c
+++ b/hw/core/ptimer.c
@@ -36,19 +36,7 @@ static void ptimer_reload(ptimer_state *s)
 {
 uint32_t period_frac = s->period_frac;
 uint64_t period = s->period;
-
-if (s->delta == 0) {
-ptimer_trigger(s);
-}
-
-if (s->delta == 0 && s->enabled == 1) {
-s->delta = s->limit;
-}
-
-if (s->delta == 0) {
-ptimer_stop(s);
-return;
-}
+uint64_t delta = MAX(1, s->delta);
 
 /*
  * Artificially limit timeout rate to something
@@ -59,15 +47,15 @@ static void ptimer_reload(ptimer_state *s)
  * on the current generation of host machines.
  */
 
-if (s->enabled == 1 && (s->delta * period < 1) && !use_icount) {
-period = 1 / s->delta;
+if (s->enabled == 1 && (delta * period < 1) && !use_icount) {
+period = 1 / delta;
 period_frac = 0;
 }
 
 s->last_event = s->next_event;
-s->next_event = s->last_event + s->delta * period;
+s->next_event = s->last_event + delta * period;
 if (period_frac) {
-s->next_event += ((int64_t)period_frac * s->delta) >> 32;
+s->next_event += ((int64_t)period_frac * delta) >> 32;
 }
 timer_mod(s->timer, s->next_event);
 }
@@ -75,8 +63,16 @@ static void ptimer_reload(ptimer_state *s)
 static void ptimer_tick(void *opaque)
 {
 ptimer_state *s = (ptimer_state *)opaque;
-s->delta = 0;
-ptimer_reload(s);
+
+s->delta = (s->enabled == 1) ? s->limit : 0;
+
+if (s->delta == 0) {
+s->enabled = 0;
+} else {
+ptimer_reload(s);
+}
+
+ptimer_trigger(s);
 }
 
 uint64_t ptimer_get_count(ptimer_state *s)
-- 
2.7.0

[Qemu-devel] [PATCH v12 6/9] hw/ptimer: Legalize running with delta = load = 0 and abort on period = 0

[Dmitry Osipenko]

Currently ptimer would print error message and clear enable flag for an
arming timer that has delta = load = 0. That actually could be a valid case
for some hardware, like instant IRQ trigger for oneshot timer or continuous
in periodic mode. Support those cases by removing the error message and
stopping the timer when delta = 0. Abort execution when period = 0 instead
of printing the error message, otherwise there is a chance to miss that error.

In addition, don't re-load oneshot timer when delta = 0 and remove duplicated
code from ptimer_tick(), since ptimer_reload would invoke trigger and stop
the timer.

Signed-off-by: Dmitry Osipenko 
Reviewed-by: Peter Crosthwaite 
---
 hw/core/ptimer.c | 28 +---
 1 file changed, 13 insertions(+), 15 deletions(-)

diff --git a/hw/core/ptimer.c b/hw/core/ptimer.c
index bf62fdd..62f8cb1 100644
--- a/hw/core/ptimer.c
+++ b/hw/core/ptimer.c
@@ -39,11 +39,14 @@ static void ptimer_reload(ptimer_state *s)
 
 if (s->delta == 0) {
 ptimer_trigger(s);
+}
+
+if (s->delta == 0 && s->enabled == 1) {
 s->delta = s->limit;
 }
-if (s->delta == 0 || s->period == 0) {
-fprintf(stderr, "Timer with period zero, disabling\n");
-s->enabled = 0;
+
+if (s->delta == 0) {
+ptimer_stop(s);
 return;
 }
 
@@ -72,27 +75,22 @@ static void ptimer_reload(ptimer_state *s)
 static void ptimer_tick(void *opaque)
 {
 ptimer_state *s = (ptimer_state *)opaque;
-ptimer_trigger(s);
 s->delta = 0;
-if (s->enabled == 2) {
-s->enabled = 0;
-} else {
-ptimer_reload(s);
-}
+ptimer_reload(s);
 }
 
 uint64_t ptimer_get_count(ptimer_state *s)
 {
 uint64_t counter;
 
-if (s->enabled) {
+if (s->enabled && s->delta != 0) {
 int64_t now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 int64_t next = s->next_event;
 bool expired = (now - next >= 0);
 bool oneshot = (s->enabled == 2);
 
 /* Figure out the current counter value.  */
-if (s->period == 0 || (expired && (oneshot || use_icount))) {
+if (expired && (oneshot || use_icount)) {
 /* Prevent timer underflowing if it should already have
triggered.  */
 counter = 0;
@@ -164,12 +162,10 @@ void ptimer_run(ptimer_state *s, int oneshot)
 {
 bool was_disabled = !s->enabled;
 
-if (was_disabled && s->period == 0) {
-fprintf(stderr, "Timer with period zero, disabling\n");
-return;
-}
 s->enabled = oneshot ? 2 : 1;
+
 if (was_disabled) {
+g_assert(s->period != 0);
 s->next_event = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
 ptimer_reload(s);
 }
@@ -190,6 +186,7 @@ void ptimer_stop(ptimer_state *s)
 /* Set counter increment interval in nanoseconds.  */
 void ptimer_set_period(ptimer_state *s, int64_t period)
 {
+g_assert(period != 0);
 s->delta = ptimer_get_count(s);
 s->period = period;
 s->period_frac = 0;
@@ -202,6 +199,7 @@ void ptimer_set_period(ptimer_state *s, int64_t period)
 /* Set counter frequency in Hz.  */
 void ptimer_set_freq(ptimer_state *s, uint32_t freq)
 {
+g_assert(freq != 0 && freq <= 10);
 s->delta = ptimer_get_count(s);
 s->period = 10ll / freq;
 s->period_frac = (10ll << 32) / freq;
-- 
2.7.0

[Qemu-devel] [PATCH v12 3/9] hw/ptimer: Update .delta on period/freq change

[Dmitry Osipenko]

Delta value must be updated on period/freq change, otherwise running timer
would be restarted (counter reloaded with old delta). Only m68k/mcf520x
and arm/arm_timer devices are currently doing freq change correctly, i.e.
stopping the timer. Perform delta update to fix affected devices and
eliminate potential further mistakes.

Signed-off-by: Dmitry Osipenko 
Reviewed-by: Peter Crosthwaite 
---
 hw/core/ptimer.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/hw/core/ptimer.c b/hw/core/ptimer.c
index d4452d3..f58790a 100644
--- a/hw/core/ptimer.c
+++ b/hw/core/ptimer.c
@@ -189,6 +189,7 @@ void ptimer_stop(ptimer_state *s)
 /* Set counter increment interval in nanoseconds.  */
 void ptimer_set_period(ptimer_state *s, int64_t period)
 {
+s->delta = ptimer_get_count(s);
 s->period = period;
 s->period_frac = 0;
 if (s->enabled) {
@@ -200,6 +201,7 @@ void ptimer_set_period(ptimer_state *s, int64_t period)
 /* Set counter frequency in Hz.  */
 void ptimer_set_freq(ptimer_state *s, uint32_t freq)
 {
+s->delta = ptimer_get_count(s);
 s->period = 10ll / freq;
 s->period_frac = (10ll << 32) / freq;
 if (s->enabled) {
-- 
2.7.0

[Qemu-devel] [PATCH v12 1/9] hw/ptimer: Fix issues caused by the adjusted timer limit value

[Dmitry Osipenko]

Multiple issues here related to the timer with a adjusted .limit value:

1) ptimer_get_count() returns incorrect counter value for the disabled
timer after loading the counter with a small value, because adjusted limit
value is used instead of the original.

For instance:
1) ptimer_stop(t)
2) ptimer_set_period(t, 1)
3) ptimer_set_limit(t, 0, 1)
4) ptimer_get_count(t) <-- would return 1 instead of 0

2) ptimer_get_count() might return incorrect value for the timer running
with a adjusted limit value.

For instance:
1) ptimer_stop(t)
2) ptimer_set_period(t, 1)
3) ptimer_set_limit(t, 10, 1)
4) ptimer_run(t)
5) ptimer_get_count(t) <-- might return value > 10

3) Neither ptimer_set_period() nor ptimer_set_freq() are adjusting the
limit value, so it is still possible to make timer timeout value
arbitrary small.

For instance:
1) ptimer_set_period(t, 1)
2) ptimer_set_limit(t, 1, 0)
3) ptimer_set_period(t, 1) <-- bypass limit correction

Fix all of the above issues by adjusting timer period instead of the limit.
Perform the adjustment for periodic timer only. Use the delta value instead
of the limit to make decision whether adjustment is required, as limit could
be altered while timer is running, resulting in incorrect value returned by
ptimer_get_count.

Signed-off-by: Dmitry Osipenko 
Reviewed-by: Peter Crosthwaite 
---
 hw/core/ptimer.c | 51 +++
 1 file changed, 31 insertions(+), 20 deletions(-)

diff --git a/hw/core/ptimer.c b/hw/core/ptimer.c
index edf077c..5edcd71 100644
--- a/hw/core/ptimer.c
+++ b/hw/core/ptimer.c
@@ -34,6 +34,9 @@ static void ptimer_trigger(ptimer_state *s)
 
 static void ptimer_reload(ptimer_state *s)
 {
+uint32_t period_frac = s->period_frac;
+uint64_t period = s->period;
+
 if (s->delta == 0) {
 ptimer_trigger(s);
 s->delta = s->limit;
@@ -44,10 +47,24 @@ static void ptimer_reload(ptimer_state *s)
 return;
 }
 
+/*
+ * Artificially limit timeout rate to something
+ * achievable under QEMU.  Otherwise, QEMU spends all
+ * its time generating timer interrupts, and there
+ * is no forward progress.
+ * About ten microseconds is the fastest that really works
+ * on the current generation of host machines.
+ */
+
+if (s->enabled == 1 && (s->delta * period < 1) && !use_icount) {
+period = 1 / s->delta;
+period_frac = 0;
+}
+
 s->last_event = s->next_event;
-s->next_event = s->last_event + s->delta * s->period;
-if (s->period_frac) {
-s->next_event += ((int64_t)s->period_frac * s->delta) >> 32;
+s->next_event = s->last_event + s->delta * period;
+if (period_frac) {
+s->next_event += ((int64_t)period_frac * s->delta) >> 32;
 }
 timer_mod(s->timer, s->next_event);
 }
@@ -82,6 +99,13 @@ uint64_t ptimer_get_count(ptimer_state *s)
 uint64_t div;
 int clz1, clz2;
 int shift;
+uint32_t period_frac = s->period_frac;
+uint64_t period = s->period;
+
+if ((s->enabled == 1) && !use_icount && (s->delta * period < 
1)) {
+period = 1 / s->delta;
+period_frac = 0;
+}
 
 /* We need to divide time by period, where time is stored in
rem (64-bit integer) and period is stored in period/period_frac
@@ -94,7 +118,7 @@ uint64_t ptimer_get_count(ptimer_state *s)
 */
 
 rem = s->next_event - now;
-div = s->period;
+div = period;
 
 clz1 = clz64(rem);
 clz2 = clz64(div);
@@ -103,13 +127,13 @@ uint64_t ptimer_get_count(ptimer_state *s)
 rem <<= shift;
 div <<= shift;
 if (shift >= 32) {
-div |= ((uint64_t)s->period_frac << (shift - 32));
+div |= ((uint64_t)period_frac << (shift - 32));
 } else {
 if (shift != 0)
-div |= (s->period_frac >> (32 - shift));
+div |= (period_frac >> (32 - shift));
 /* Look at remaining bits of period_frac and round div up if 
necessary.  */
-if ((uint32_t)(s->period_frac << shift))
+if ((uint32_t)(period_frac << shift))
 div += 1;
 }
 counter = rem / div;
@@ -181,19 +205,6 @@ void ptimer_set_freq(ptimer_state *s, uint32_t freq)
count = limit.  */
 void ptimer_set_limit(ptimer_state *s, uint64_t limit, int reload)
 {
-/*
- * Artificially limit timeout rate to something
- * achievable under QEMU.  Otherwise, QEMU spends all
- * its time generating timer interrupts, and there
- * is no forward progress.
- * About ten microseconds is the fastest that really works
- * on the current generation of host 

[Qemu-devel] [PATCH v12 0/9] PTimer fixes/features and ARM MPTimer conversion

[Dmitry Osipenko]

Changelog for ARM MPTimer QEMUTimer to ptimer conversion:

V2: Fixed changing periodic timer counter value "on the fly". I added a
test to the gist to cover that issue.

V3: Fixed starting the timer with load = 0 and counter != 0, added tests
to the gist for this issue. Changed vmstate version for all VMSD's,
since loadvm doesn't check version of nested VMSD.

V4: Fixed spurious IT bit set for the timer starting in the periodic mode
with counter = 0. Test added.

V5: Code cleanup, now depends on ptimer_set_limit() fix.

V6: No code change, added test to check ptimer_get_count() with corrected
.limit value.

V7: No change.

V8: No change.

V9: No change.

V10: Correctly handle cases when counter = load = 0 and prescaler != 0,
 i.e. triggering interrupt in that case. Call ptimer_* only when
 certain MPTimer state was changed, like prescaler change. Factor out
 timerblock_set_count from timerblock_run and inline both.
 Tests updated.

V11: Fixed missed periodic timer stopping on setting counter => 0 with
 load = 0 and prescaler = 0.

v12: Timer isn't doing uninterruptible IRQ, but ticks continuously.
 On setting counter/load to 0 with prescaler != 0, timer would trigger
 IRQ after one period. Verified on real HW, tests updated.

ARM MPTimer tests: https://gist.github.com/digetx/dbd46109503b1a91941a


Patches for ptimer are introduced since V5 of "ARM MPTimer conversion".

Changelog for the ptimer patches:

V5: Only fixed ptimer_set_limit() for the disabled timer.

V6: As was pointed by Peter Maydell, there are other issues beyond
ptimer_set_limit(), so V6 supposed to cover all those issues.

V7: Added accidentally removed !use_icount check.
Added missed "else" statement.

V8: Adjust period instead of the limit and do it for periodic timer only
(.limit adjusting bug). Added patch/fix for freq/period change and
ptimer_get_count() improvement.

V9: Don't do wrap around if counter == 0, otherwise polled periodic
timer won't ever return counter = 0.

V10: Addressed V8/9 review comments.
 Adjust timer period based on delta instead of limit.
 Don't wrap around when in icount mode.
 New patches: "on the fly" mode switch, silence error msg when
  delta = load = 0, introduce ptimer_get_limit.

V11: Dropped timer tick from "Perform tick and counter wrap around if
 timer already expired" patch since it would cause bogus tick after
 QEMU been reset if ptimer was stopped and it's QEMUtimer expired
 during reset.
 Patch "Legalize running with delta = load = 0" now explicitly
 forbids period = 0.

v12: Fixed missed abort on setting freq > 10.
 New patches:
"Fix counter - 1 returned by ptimer_get_count for the active timer"
"Perform delayed tick instead of immediate if delta = 0"

Dmitry Osipenko (9):
  hw/ptimer: Fix issues caused by the adjusted timer limit value
  hw/ptimer: Perform counter wrap around if timer already expired
  hw/ptimer: Update .delta on period/freq change
  hw/ptimer: Support "on the fly" timer mode switch
  hw/ptimer: Introduce ptimer_get_limit
  hw/ptimer: Legalize running with delta = load = 0 and abort on period
= 0
  hw/ptimer: Fix counter - 1 returned by ptimer_get_count for the active
timer
  hw/ptimer: Perform delayed tick instead of immediate if delta = 0
  arm_mptimer: Convert to use ptimer

 hw/core/ptimer.c   | 114 ---
 hw/timer/arm_mptimer.c | 133 +
 include/hw/ptimer.h|   1 +
 include/hw/timer/arm_mptimer.h |   5 +-
 4 files changed, 138 insertions(+), 115 deletions(-)

-- 
2.7.0

[Qemu-devel] [PATCH v12 4/9] hw/ptimer: Support "on the fly" timer mode switch

[Dmitry Osipenko]

Allow switching between periodic <-> oneshot modes while timer is running.

Signed-off-by: Dmitry Osipenko 
Reviewed-by: Peter Crosthwaite 
---
 hw/core/ptimer.c | 13 +++--
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/hw/core/ptimer.c b/hw/core/ptimer.c
index f58790a..78ae443 100644
--- a/hw/core/ptimer.c
+++ b/hw/core/ptimer.c
@@ -162,16 +162,17 @@ void ptimer_set_count(ptimer_state *s, uint64_t count)
 
 void ptimer_run(ptimer_state *s, int oneshot)
 {
-if (s->enabled) {
-return;
-}
-if (s->period == 0) {
+bool was_disabled = !s->enabled;
+
+if (was_disabled && s->period == 0) {
 fprintf(stderr, "Timer with period zero, disabling\n");
 return;
 }
 s->enabled = oneshot ? 2 : 1;
-s->next_event = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
-ptimer_reload(s);
+if (was_disabled) {
+s->next_event = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+ptimer_reload(s);
+}
 }
 
 /* Pause a timer.  Note that this may cause it to "lose" time, even if it
-- 
2.7.0

Re: [Qemu-devel] how to setup a watchdog?

[Richard W.M. Jones]

On Mon, Jan 18, 2016 at 11:43:02AM +, lejeczek wrote:
> apologies I bother devel, but..
> I tried to get help on libvirt mailing list but not luck, then qemu,
> still nothing
> I hope maybe somebody here?
> 
> I'm trying Qemu's watchdog.
> My understanding was that hardware (here qemu's watchdog) would take
> action, eg. cold reboot the system if there is no ping from the OS
> watchdog, so I
> thought stopping watchdog service in VM should be a quick test,
> right?
> 
> I have this in the guest:
> 
> 
>function='0x0'/>
> 
> 
> and I see /dev/watchdog in my guest. Yet nothing happens,
> guest(linux) runs uninterrupted.
> I must be missing something, an expert said it's config problem, is
> it really is?

Sorry for the late response.

If your guest is Linux, you must also run the watchdog daemon in the
guest.  The watchdog device's timer isn't primed until it is opened
and set up by the watchdog daemon.

There is basically no watchdog framework or support in Windows, so
you'll have to write the software and driver yourself.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

[Qemu-devel] [PATCH v12 9/9] arm_mptimer: Convert to use ptimer

[Dmitry Osipenko]

Current ARM MPTimer implementation uses QEMUTimer for the actual timer,
this implementation isn't complete and mostly tries to duplicate of what
generic ptimer is already doing fine.

Conversion to ptimer brings the following benefits and fixes:
- Simple timer pausing implementation
- Fixes counter value preservation after stopping the timer
- Correctly handles prescaler != 0 / counter = 0 / load = 0 cases
- Code simplification and reduction

Bump VMSD to version 3, since VMState is changed and is not compatible
with the previous implementation.

Signed-off-by: Dmitry Osipenko 
---
 hw/timer/arm_mptimer.c | 133 +
 include/hw/timer/arm_mptimer.h |   5 +-
 2 files changed, 70 insertions(+), 68 deletions(-)

diff --git a/hw/timer/arm_mptimer.c b/hw/timer/arm_mptimer.c
index 5dfab66..f002458 100644
--- a/hw/timer/arm_mptimer.c
+++ b/hw/timer/arm_mptimer.c
@@ -19,8 +19,9 @@
  * with this program; if not, see .
  */
 
+#include "hw/ptimer.h"
 #include "hw/timer/arm_mptimer.h"
-#include "qemu/timer.h"
+#include "qemu/main-loop.h"
 #include "qom/cpu.h"
 
 /* This device implements the per-cpu private timer and watchdog block
@@ -42,55 +43,54 @@ static inline void timerblock_update_irq(TimerBlock *tb)
 }
 
 /* Return conversion factor from mpcore timer ticks to qemu timer ticks.  */
-static inline uint32_t timerblock_scale(TimerBlock *tb)
+static inline uint32_t timerblock_scale(uint32_t control)
 {
-return (((tb->control >> 8) & 0xff) + 1) * 10;
+return (((control >> 8) & 0xff) + 1) * 10;
 }
 
-static void timerblock_reload(TimerBlock *tb, int restart)
+static inline void timerblock_set_count(struct ptimer_state *timer,
+uint32_t control, uint64_t *count)
 {
-if (tb->count == 0) {
-return;
+/* PTimer would immediately trigger interrupt for periodic timer
+ * when counter set to 0, MPtimer under certain condition only.
+ */
+if ((control & 3) == 3 && (control & 0xff00) == 0 && *count == 0) {
+*count = ptimer_get_limit(timer);
 }
-if (restart) {
-tb->tick = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+ptimer_set_count(timer, *count);
+}
+
+static inline void timerblock_run(struct ptimer_state *timer,
+  uint32_t control, uint32_t load)
+{
+if ((control & 1) && ((control & 0xff00) || load != 0)) {
+ptimer_run(timer, !(control & 2));
 }
-tb->tick += (int64_t)tb->count * timerblock_scale(tb);
-timer_mod(tb->timer, tb->tick);
 }
 
 static void timerblock_tick(void *opaque)
 {
 TimerBlock *tb = (TimerBlock *)opaque;
 tb->status = 1;
-if (tb->control & 2) {
-tb->count = tb->load;
-timerblock_reload(tb, 0);
-} else {
-tb->count = 0;
-}
 timerblock_update_irq(tb);
+/* Periodic timer with load = 0 and prescaler != 0 would re-trigger
+ * IRQ after one period, otherwise it either stops or wraps around.
+ */
+if ((tb->control & 2) && (tb->control & 0xff00) &&
+ptimer_get_limit(tb->timer) == 0) {
+ptimer_run(tb->timer, 0);
+}
 }
 
 static uint64_t timerblock_read(void *opaque, hwaddr addr,
 unsigned size)
 {
 TimerBlock *tb = (TimerBlock *)opaque;
-int64_t val;
 switch (addr) {
 case 0: /* Load */
-return tb->load;
+return ptimer_get_limit(tb->timer);
 case 4: /* Counter.  */
-if (((tb->control & 1) == 0) || (tb->count == 0)) {
-return 0;
-}
-/* Slow and ugly, but hopefully won't happen too often.  */
-val = tb->tick - qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
-val /= timerblock_scale(tb);
-if (val < 0) {
-val = 0;
-}
-return val;
+return ptimer_get_count(tb->timer);
 case 8: /* Control.  */
 return tb->control;
 case 12: /* Interrupt status.  */
@@ -104,37 +104,45 @@ static void timerblock_write(void *opaque, hwaddr addr,
  uint64_t value, unsigned size)
 {
 TimerBlock *tb = (TimerBlock *)opaque;
-int64_t old;
+uint32_t control = tb->control;
 switch (addr) {
 case 0: /* Load */
-tb->load = value;
-/* Fall through.  */
-case 4: /* Counter.  */
-if ((tb->control & 1) && tb->count) {
-/* Cancel the previous timer.  */
-timer_del(tb->timer);
+/* Setting load to 0 stops the timer without doing the tick if
+ * prescaler = 0.
+ */
+if ((control & 1) && (control & 0xff00) == 0 && value == 0) {
+ptimer_stop(tb->timer);
 }
-tb->count = value;
-if (tb->control & 1) {
-timerblock_reload(tb, 1);
+ptimer_set_limit(tb->timer, value, 1);
+timerblock_run(tb->timer, control, value);
+break;
+case 4: /* 

[Qemu-devel] [PATCH v12 2/9] hw/ptimer: Perform counter wrap around if timer already expired

[Dmitry Osipenko]

ptimer_get_count() might be called while QEMU timer already been expired.
In that case ptimer would return counter = 0, which might be undesirable
in case of polled timer. Do counter wrap around for periodic timer to keep
it distributed. In order to achieve more accurate emulation behaviour of
certain hardware, don't perform wrap around when in icount mode and return
counter = 0 in that case (that doesn't affect polled counter distribution).

Signed-off-by: Dmitry Osipenko 
Reviewed-by: Peter Crosthwaite 
---
 hw/core/ptimer.c | 19 +--
 1 file changed, 13 insertions(+), 6 deletions(-)

diff --git a/hw/core/ptimer.c b/hw/core/ptimer.c
index 5edcd71..d4452d3 100644
--- a/hw/core/ptimer.c
+++ b/hw/core/ptimer.c
@@ -83,14 +83,16 @@ static void ptimer_tick(void *opaque)
 
 uint64_t ptimer_get_count(ptimer_state *s)
 {
-int64_t now;
 uint64_t counter;
 
 if (s->enabled) {
-now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+int64_t now = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL);
+int64_t next = s->next_event;
+bool expired = (now - next >= 0);
+bool oneshot = (s->enabled == 2);
+
 /* Figure out the current counter value.  */
-if (now - s->next_event > 0
-|| s->period == 0) {
+if (s->period == 0 || (expired && (oneshot || use_icount))) {
 /* Prevent timer underflowing if it should already have
triggered.  */
 counter = 0;
@@ -102,7 +104,7 @@ uint64_t ptimer_get_count(ptimer_state *s)
 uint32_t period_frac = s->period_frac;
 uint64_t period = s->period;
 
-if ((s->enabled == 1) && !use_icount && (s->delta * period < 
1)) {
+if (!oneshot && (s->delta * period < 1) && !use_icount) {
 period = 1 / s->delta;
 period_frac = 0;
 }
@@ -117,7 +119,7 @@ uint64_t ptimer_get_count(ptimer_state *s)
backwards.
 */
 
-rem = s->next_event - now;
+rem = expired ? now - next : next - now;
 div = period;
 
 clz1 = clz64(rem);
@@ -137,6 +139,11 @@ uint64_t ptimer_get_count(ptimer_state *s)
 div += 1;
 }
 counter = rem / div;
+
+if (expired && counter != 0) {
+/* Wrap around periodic counter.  */
+counter = s->limit - (counter - 1) % s->limit;
+}
 }
 } else {
 counter = s->delta;
-- 
2.7.0

[Qemu-devel] [PATCH v12 5/9] hw/ptimer: Introduce ptimer_get_limit

[Dmitry Osipenko]

Currently ptimer users are used to store copy of the limit value, because
ptimer doesn't provide facility to retrieve the limit. Let's provide it.

Signed-off-by: Dmitry Osipenko 
Reviewed-by: Peter Crosthwaite 
---
 hw/core/ptimer.c| 5 +
 include/hw/ptimer.h | 1 +
 2 files changed, 6 insertions(+)

diff --git a/hw/core/ptimer.c b/hw/core/ptimer.c
index 78ae443..bf62fdd 100644
--- a/hw/core/ptimer.c
+++ b/hw/core/ptimer.c
@@ -224,6 +224,11 @@ void ptimer_set_limit(ptimer_state *s, uint64_t limit, int 
reload)
 }
 }
 
+uint64_t ptimer_get_limit(ptimer_state *s)
+{
+return s->limit;
+}
+
 const VMStateDescription vmstate_ptimer = {
 .name = "ptimer",
 .version_id = 1,
diff --git a/include/hw/ptimer.h b/include/hw/ptimer.h
index 8ebacbb..e397db5 100644
--- a/include/hw/ptimer.h
+++ b/include/hw/ptimer.h
@@ -19,6 +19,7 @@ typedef void (*ptimer_cb)(void *opaque);
 ptimer_state *ptimer_init(QEMUBH *bh);
 void ptimer_set_period(ptimer_state *s, int64_t period);
 void ptimer_set_freq(ptimer_state *s, uint32_t freq);
+uint64_t ptimer_get_limit(ptimer_state *s);
 void ptimer_set_limit(ptimer_state *s, uint64_t limit, int reload);
 uint64_t ptimer_get_count(ptimer_state *s);
 void ptimer_set_count(ptimer_state *s, uint64_t count);
-- 
2.7.0

Re: [Qemu-devel] [PATCH v3 0/3] Travis updates

[Michael Tokarev]

28.01.2016 17:23, Alex Bennée wrote:
> Hi,
> 
> The first patch has been reviewed and signed off. Long term I think
> it is worth applying but it look like the performance increase it
> negligible compared to the old style VM builds at the moment. I
> suspect this may be because the new infrastructure is under more load
> as more projects have migrated.

I'm applying all this to -trivial, but without trying to understand
what it is all about, as I don't know travis.

Thanks,

/mjt

Re: [Qemu-devel] [PATCH] char: fix parameter name / type in BSD codepath

[Michael Tokarev]

22.01.2016 20:35, Daniel P. Berrange wrote:
> The BSD impl of qemu_chr_open_pp_fd had mis-declared
> its parameter type as ChardevBackend instead of
> ChardevCommon. It had also mistakenly used the variable
> name 'common' instead of 'backend'.

Applied to -trivial, fixed qemu-devel@ addres.

Thanks!

/mjt

> Tested-by: Sean Bruno 
> Signed-off-by: Daniel P. Berrange 
> ---
>  qemu-char.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/qemu-char.c b/qemu-char.c
> index e133f4f..aa2e660 100644
> --- a/qemu-char.c
> +++ b/qemu-char.c
> @@ -1835,12 +1835,12 @@ static int pp_ioctl(CharDriverState *chr, int cmd, 
> void *arg)
>  }
>  
>  static CharDriverState *qemu_chr_open_pp_fd(int fd,
> -ChardevBackend *backend,
> +ChardevCommon *backend,
>  Error **errp)
>  {
>  CharDriverState *chr;
>  
> -chr = qemu_chr_alloc(common, errp);
> +chr = qemu_chr_alloc(backend, errp);
>  if (!chr) {
>  return NULL;
>  }
> 

Re: [Qemu-devel] [PATCH COLO-Frame v13 32/39] COLO: Separate the process of saving/loading ram and device state

[Hailiang Zhang]

On 2016/1/27 22:14, Dr. David Alan Gilbert wrote:

* zhanghailiang (zhang.zhanghaili...@huawei.com) wrote:

We separate the process of saving/loading ram and device state when do 
checkpoint,
we add new helpers for save/load ram/device. With this change, we can directly
transfer ram from master to slave without using QEMUSizeBuffer as assistant,
which also reduce the size of extra memory been used during checkpoint.

Besides, we move the colo_flush_ram_cache to the proper position after the
above change.

Signed-off-by: zhanghailiang 
Signed-off-by: Li Zhijian 
---
v13:
- Re-use some existed helper functions to realize saving/loading ram and device.
v11:
- Remove load configuration section in qemu_loadvm_state_begin()
---
  include/sysemu/sysemu.h |  6 +
  migration/colo.c| 47 ++-
  migration/ram.c |  5 
  migration/savevm.c  | 66 ++---
  4 files changed, 104 insertions(+), 20 deletions(-)

diff --git a/include/sysemu/sysemu.h b/include/sysemu/sysemu.h
index 91eeda3..f553f8a 100644
--- a/include/sysemu/sysemu.h
+++ b/include/sysemu/sysemu.h
@@ -133,7 +133,13 @@ void qemu_savevm_send_postcopy_ram_discard(QEMUFile *f, 
const char *name,
 uint64_t *start_list,
 uint64_t *length_list);

+void qemu_savevm_live_state(QEMUFile *f);
+int qemu_save_device_state(QEMUFile *f);
+
  int qemu_loadvm_state(QEMUFile *f);
+int qemu_loadvm_state_begin(QEMUFile *f);
+int qemu_load_ram_state(QEMUFile *f);
+int qemu_load_device_state(QEMUFile *f);

  typedef enum DisplayType
  {
diff --git a/migration/colo.c b/migration/colo.c
index fd87f34..4a99819 100644
--- a/migration/colo.c
+++ b/migration/colo.c
@@ -290,21 +290,34 @@ static int colo_do_checkpoint_transaction(MigrationState 
*s,
  goto out;
  }

+colo_put_cmd(s->to_dst_file, COLO_COMMAND_VMSTATE_SEND, _err);
+if (local_err) {
+goto out;
+}
+
  /* Disable block migration */
  s->params.blk = 0;
  s->params.shared = 0;
-qemu_savevm_state_header(trans);
-qemu_savevm_state_begin(trans, >params);
-qemu_mutex_lock_iothread();
-qemu_savevm_state_complete_precopy(trans, false);
-qemu_mutex_unlock_iothread();
-
-qemu_fflush(trans);
+qemu_savevm_state_begin(s->to_dst_file, >params);
+ret = qemu_file_get_error(s->to_dst_file);
+if (ret < 0) {
+error_report("save vm state begin error");
+goto out;
+}

-colo_put_cmd(s->to_dst_file, COLO_COMMAND_VMSTATE_SEND, _err);
-if (local_err) {




+qemu_mutex_lock_iothread();
+/* Only save VM's live state, which not including device state */
+qemu_savevm_live_state(s->to_dst_file);


I'm a little worried that this is done with the lock held;
if the destination hung during this point and there was
enough data to fill the network buffers, couldn't we hang
and then we wouldn't be able to issue the colo_lost_heartbeat
command?



Yes, here we may encounter this problem, i'm wondering if it is
safe to save the remaining pages without holding this lock,
do you know why we must hold this lock while in the completion stage
of migration while VM is stopped?


+/* Note: device state is saved into buffer */
+ret = qemu_save_device_state(trans);
+if (ret < 0) {
+error_report("save device state error\n");
+qemu_mutex_unlock_iothread();
  goto out;
  }
+qemu_fflush(trans);
+qemu_mutex_unlock_iothread();


You can save an unlock by doing:
ret = ...
qemu_mutex_unlock_iothread();
if (ret < 0) {



I will fix it in next version.




  /* we send the total size of the vmstate first */
  size = qsb_get_length(buffer);
  colo_put_cmd_value(s->to_dst_file, COLO_COMMAND_VMSTATE_SIZE,
@@ -573,6 +586,16 @@ void *colo_process_incoming_thread(void *opaque)
  goto out;
  }

+ret = qemu_loadvm_state_begin(mis->from_src_file);
+if (ret < 0) {
+error_report("load vm state begin error, ret=%d", ret);
+goto out;
+}
+ret = qemu_load_ram_state(mis->from_src_file);
+if (ret < 0) {
+error_report("load ram state error");
+goto out;
+}
  /* read the VM state total size first */
  value = colo_get_cmd_value(mis->from_src_file,
   COLO_COMMAND_VMSTATE_SIZE, _err);
@@ -605,8 +628,10 @@ void *colo_process_incoming_thread(void *opaque)
  qemu_mutex_lock_iothread();
  qemu_system_reset(VMRESET_SILENT);
  vmstate_loading = true;
-if (qemu_loadvm_state(fb) < 0) {
-error_report("COLO: loadvm failed");
+colo_flush_ram_cache();
+ret = qemu_load_device_state(fb);
+if (ret < 0) {
+error_report("COLO: load device state 

[Qemu-devel] [PATCH 6/6] iotests: test external backup api

[Vladimir Sementsov-Ogievskiy]

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/151 | 77 ++
 tests/qemu-iotests/151.out |  7 +
 tests/qemu-iotests/group   |  1 +
 3 files changed, 85 insertions(+)
 create mode 100755 tests/qemu-iotests/151
 create mode 100644 tests/qemu-iotests/151.out

diff --git a/tests/qemu-iotests/151 b/tests/qemu-iotests/151
new file mode 100755
index 000..d2fc2dc
--- /dev/null
+++ b/tests/qemu-iotests/151
@@ -0,0 +1,77 @@
+#!/usr/bin/env python
+#
+# Test external backup api
+#
+# (C) Vladimir Sementsov-Ogievskiy 2016
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+import os
+import iotests
+import time
+from iotests import qemu_img
+
+disk = os.path.join(iotests.test_dir, 'disk')
+
+size   = 0x4000 # 1G
+sector_size = 512
+granularity = 0x1
+regions = [
+{ 'start': 0,  'count': 0x10 },
+{ 'start': 0x1000, 'count': 0x2  },
+{ 'start': 0x3999, 'count': 0x1  }
+]
+
+class TestExternalBackupApi(iotests.QMPTestCase):
+
+def setUp(self):
+qemu_img('create', '-f', iotests.imgfmt, disk, str(size))
+self.vm = iotests.VM().add_drive(disk)
+self.vm.launch()
+
+def tearDown(self):
+self.vm.shutdown()
+os.remove(disk)
+
+def printBitmap(self, block_io):
+print self.vm.qmp('query-block-dirty-bitmap-ranges', node='drive0',
+  name='bitmap', block_io=(block_io if 'on' else 
'off'),
+  max_ranges=100);
+
+def test_api(self):
+result = self.vm.qmp('block-dirty-bitmap-add', node='drive0',
+ name='bitmap', granularity=granularity)
+self.assert_qmp(result, 'return', {});
+
+for r in regions:
+self.vm.hmp_qemu_io('drive0',
+'write %d %d' % (r['start'], r['count']))
+
+result = self.vm.qmp('block-dirty-bitmap-create-successor',
+node='drive0', name='bitmap')
+self.assert_qmp(result, 'return', {});
+
+self.printBitmap(False)
+
+result = self.vm.qmp('block-dirty-bitmap-abdicate',
+  node='drive0', name='bitmap')
+self.assert_qmp(result, 'return', {});
+
+self.printBitmap(True)
+
+
+
+if __name__ == '__main__':
+iotests.main()
diff --git a/tests/qemu-iotests/151.out b/tests/qemu-iotests/151.out
new file mode 100644
index 000..baed190
--- /dev/null
+++ b/tests/qemu-iotests/151.out
@@ -0,0 +1,7 @@
+.
+--
+Ran 1 tests
+
+OK
+{u'return': [{u'count': 1048576, u'start': 0}, {u'count': 131072, u'start': 
268435456}, {u'count': 65536, u'start': 966328320}]}
+{u'return': []}
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 84db670..9425a23 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -143,3 +143,4 @@
 139 rw auto quick
 142 auto
 150 auto
+151 auto
-- 
1.8.3.1

[Qemu-devel] [PATCH 4/6] qapi: add qmp commands for some dirty bitmap functions

[Vladimir Sementsov-Ogievskiy]

Add access to bdrv_dirty_bitmap_create_successor,
bdrv_dirty_bitmap_abdicate, bdrv_reclaim_dirty_bitmap.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 blockdev.c   | 51 +++
 qapi/block-core.json | 36 ++
 qmp-commands.hx  | 85 
 3 files changed, 172 insertions(+)

diff --git a/blockdev.c b/blockdev.c
index fa34cf6..f03a415 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2733,6 +2733,57 @@ void qmp_block_dirty_bitmap_clear(const char *node, 
const char *name,
 aio_context_release(aio_context);
 }
 
+void qmp_block_dirty_bitmap_create_successor(const char *node,
+ const char *name, Error **errp)
+{
+AioContext *aio_context;
+BdrvDirtyBitmap *bitmap;
+BlockDriverState *bs;
+
+bitmap = block_dirty_bitmap_lookup(node, name, , _context, errp);
+if (!bitmap || !bs) {
+return;
+}
+
+bdrv_dirty_bitmap_create_successor(bs, bitmap, errp);
+
+aio_context_release(aio_context);
+}
+
+void qmp_block_dirty_bitmap_abdicate(const char *node, const char *name,
+ Error **errp)
+{
+AioContext *aio_context;
+BdrvDirtyBitmap *bitmap;
+BlockDriverState *bs;
+
+bitmap = block_dirty_bitmap_lookup(node, name, , _context, errp);
+if (!bitmap || !bs) {
+return;
+}
+
+bdrv_dirty_bitmap_abdicate(bs, bitmap, errp);
+
+aio_context_release(aio_context);
+}
+
+void qmp_block_dirty_bitmap_reclaim(const char *node, const char *name,
+ Error **errp)
+{
+AioContext *aio_context;
+BdrvDirtyBitmap *bitmap;
+BlockDriverState *bs;
+
+bitmap = block_dirty_bitmap_lookup(node, name, , _context, errp);
+if (!bitmap || !bs) {
+return;
+}
+
+bdrv_reclaim_dirty_bitmap(bs, bitmap, errp);
+
+aio_context_release(aio_context);
+}
+
 BlockDirtyRangeList *qmp_query_block_dirty_bitmap_ranges(const char *node,
 const char *name, bool has_start, uint64_t start, bool has_count,
 uint64_t count, uint32_t max_ranges, bool has_block_io, bool block_io,
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 89bdeaf..a83c286 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1238,6 +1238,42 @@
   'data': 'BlockDirtyBitmap' }
 
 ##
+# @block-dirty-bitmap-create-successor
+#
+# Interface to bdrv_dirty_bitmap_create_successor
+#
+# Returns: nothing on success
+#
+# Since 2.6
+##
+{ 'command': 'block-dirty-bitmap-create-successor',
+  'data': 'BlockDirtyBitmap' }
+
+##
+# @block-dirty-bitmap-abdicate
+#
+# Interface to bdrv_dirty_bitmap_abdicate
+#
+# Returns: nothing on success
+#
+# Since 2.6
+##
+{ 'command': 'block-dirty-bitmap-abdicate',
+  'data': 'BlockDirtyBitmap' }
+
+##
+# @block-dirty-bitmap-reclaim
+#
+# Interface to bdrv_reclaim_dirty_bitmap
+#
+# Returns: nothing on success
+#
+# Since 2.6
+##
+{ 'command': 'block-dirty-bitmap-reclaim',
+  'data': 'BlockDirtyBitmap' }
+
+##
 # @blockdev-mirror
 #
 # Start mirroring a block device's writes to a new destination.
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 02de44c..6098371 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -1457,6 +1457,91 @@ Example:
 EQMP
 
 {
+.name   = "block-dirty-bitmap-create-successor",
+.args_type  = "node:B,name:s",
+.mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_create_successor,
+},
+
+SQMP
+
+block-dirty-bitmap-create-successor
+
+Since 2.6
+
+Create a successor bitmap destined to replace this bitmap after an operation.
+Requires that the bitmap is not frozen and has no successor.
+
+Arguments:
+
+- "node": device/node on which to remove dirty bitmap (json-string)
+- "name": name of the dirty bitmap to remove (json-string)
+
+Example:
+
+-> { "execute": "block-dirty-bitmap-create-successor",
+ "arguments": { "node": "drive0", "name": "bitmap0" } }
+<- { "return": {} }
+
+EQMP
+
+{
+.name   = "block-dirty-bitmap-abdicate",
+.args_type  = "node:B,name:s",
+.mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_abdicate,
+},
+
+SQMP
+
+block-dirty-bitmap-abdicate
+
+Since 2.6
+
+For a bitmap with a successor, yield our name to the successor, delete the old
+bitmap, and return a handle to the new bitmap.
+
+Arguments:
+
+- "node": device/node on which to remove dirty bitmap (json-string)
+- "name": name of the dirty bitmap to remove (json-string)
+
+Example:
+
+-> { "execute": "block-dirty-bitmap-abdicate",
+ "arguments": { "node": "drive0", "name": "bitmap0" } }
+<- { "return": {} }
+
+EQMP
+
+{
+.name   = "block-dirty-bitmap-reclaim",
+.args_type  = "node:B,name:s",
+.mhandler.cmd_new = qmp_marshal_block_dirty_bitmap_reclaim,
+},
+
+SQMP
+
+block-dirty-bitmap-reclaim
+
+Since 2.6

[Qemu-devel] [PATCH v2 0/6] external backup api

[Vladimir Sementsov-Ogievskiy]

Hi all.

These series which aims to add external backup api. This is needed to allow
backup software use our dirty bitmaps.

Vmware and Parallels Cloud Server have this feature.

There are three things are done:
- add query-block-dirty-bitmap-ranges qmp command
- add qmp commands for dirty-bitmap functions: create_successor, abdicate,
  reclaim.
- make create-successor command transaction-able

Then, external backup should be done like this:

1.  qmp transaction {
external-snapshot
bitmap-create-successor
}

2.  qmp query frozen bitmap, not acquiring aio context.

3.  do external backup, using snapshot and bitmap


4.  if (success backup)
qmp bitmap-abdicate
else
qmp bitmap-reclaime

5.  qmp merge snapshot


v2: a lot of additions and changes, no sense to compare with v1


Vladimir Sementsov-Ogievskiy (6):
  block dirty bitmap: add next_zero function
  qmp: add query-block-dirty-bitmap-ranges
  iotests: test query-block-dirty-bitmap-ranges
  qapi: add qmp commands for some dirty bitmap functions
  qapi: make block-dirty-bitmap-create-successor transaction-able
  iotests: test external backup api

 block/dirty-bitmap.c |  60 ++
 blockdev.c   | 113 +
 include/block/dirty-bitmap.h |   9 
 include/qemu/hbitmap.h   |   8 +++
 qapi-schema.json |   4 +-
 qapi/block-core.json |  90 +
 qmp-commands.hx  | 118 +++
 tests/qemu-iotests/150   |  88 
 tests/qemu-iotests/150.out   |  21 
 tests/qemu-iotests/151   |  77 
 tests/qemu-iotests/151.out   |   7 +++
 tests/qemu-iotests/group |   2 +
 util/hbitmap.c   |  26 ++
 13 files changed, 622 insertions(+), 1 deletion(-)
 create mode 100755 tests/qemu-iotests/150
 create mode 100644 tests/qemu-iotests/150.out
 create mode 100755 tests/qemu-iotests/151
 create mode 100644 tests/qemu-iotests/151.out

-- 
1.8.3.1

[Qemu-devel] [PATCH 1/6] block dirty bitmap: add next_zero function

[Vladimir Sementsov-Ogievskiy]

The function searches for next zero bit (corresponding to next not-dirty
bit).

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block/dirty-bitmap.c |  5 +
 include/block/dirty-bitmap.h |  2 ++
 include/qemu/hbitmap.h   |  8 
 util/hbitmap.c   | 26 ++
 4 files changed, 41 insertions(+)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 919ce10..6da27d9 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -490,3 +490,8 @@ int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap)
 {
 return hbitmap_count(bitmap->bitmap);
 }
+
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t start)
+{
+return hbitmap_next_zero(bitmap->bitmap, start);
+}
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 6ba5bec..0033942 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -68,4 +68,6 @@ void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap 
*bitmap,
   bool finish);
 void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
 
+int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t start);
+
 #endif
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 00dbb60..852e423 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -259,6 +259,14 @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap 
*hb, uint64_t first);
  */
 unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi);
 
+/* hbitmap_next_zero:
+ * @hb: The HBitmap to operate on
+ * @start: The bit to start from.
+ *
+ * Find next not dirty bit.
+ */
+int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start);
+
 /* hbitmap_create_meta:
  * Create a "meta" hbitmap to track dirtiness of the bits in this HBitmap.
  * The caller owns the created bitmap and must call hbitmap_free_meta(hb) to
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 1e49ab7..bcddca9 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -169,6 +169,32 @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap 
*hb, uint64_t first)
 }
 }
 
+int64_t hbitmap_next_zero(const HBitmap *hb, uint64_t start)
+{
+size_t pos = (start >> hb->granularity) >> BITS_PER_LEVEL;
+unsigned long *last_lev = hb->levels[HBITMAP_LEVELS - 1];
+uint64_t sz = hb->sizes[HBITMAP_LEVELS - 1];
+unsigned long cur = last_lev[pos];
+unsigned start_bit_offset =
+(start >> hb->granularity) & (BITS_PER_LONG - 1);
+cur |= (1UL << start_bit_offset) - 1;
+
+if (cur == (unsigned long)-1) {
+do {
+pos++;
+} while (pos < sz && last_lev[pos] == (unsigned long)-1);
+
+if (pos >= sz) {
+return -1;
+}
+
+cur = last_lev[pos];
+}
+
+return MIN((pos << BITS_PER_LEVEL) + ctol(cur), hb->size - 1)
+<< hb->granularity;
+}
+
 bool hbitmap_empty(const HBitmap *hb)
 {
 return hb->count == 0;
-- 
1.8.3.1

[Qemu-devel] [PATCH 5/6] qapi: make block-dirty-bitmap-create-successor transaction-able

[Vladimir Sementsov-Ogievskiy]

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 qapi-schema.json | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/qapi-schema.json b/qapi-schema.json
index b3038b2..a7e97c7 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1650,6 +1650,7 @@
 # blockdev-snapshot since 2.5
 # block-dirty-bitmap-add since 2.5
 # block-dirty-bitmap-clear since 2.5
+# block-dirty-bitmap-create-successor since 2.6
 ##
 { 'union': 'TransactionAction',
   'data': {
@@ -1660,7 +1661,8 @@
'abort': 'Abort',
'blockdev-snapshot-internal-sync': 'BlockdevSnapshotInternal',
'block-dirty-bitmap-add': 'BlockDirtyBitmapAdd',
-   'block-dirty-bitmap-clear': 'BlockDirtyBitmap'
+   'block-dirty-bitmap-clear': 'BlockDirtyBitmap',
+   'block-dirty-bitmap-create-successor': 'BlockDirtyBitmap'
} }
 
 ##
-- 
1.8.3.1

[Qemu-devel] [PATCH 2/6] qmp: add query-block-dirty-bitmap-ranges

[Vladimir Sementsov-Ogievskiy]

Add qmp command to query dirty bitmap contents. This is needed for
external backup.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 block/dirty-bitmap.c | 55 +++
 blockdev.c   | 62 
 include/block/dirty-bitmap.h |  7 +
 qapi/block-core.json | 54 ++
 qmp-commands.hx  | 33 +++
 5 files changed, 211 insertions(+)

diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 6da27d9..4c108e4 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -495,3 +495,58 @@ int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap 
*bitmap, uint64_t start)
 {
 return hbitmap_next_zero(bitmap->bitmap, start);
 }
+
+/* bdrv_query_dirty_bitmap_ranges
+ * start, count and resulting ranges are all in bytes
+ */
+BlockDirtyRangeList *bdrv_query_dirty_bitmap_ranges(BlockDriverState *bs,
+BdrvDirtyBitmap *bitmap, uint64_t start, uint64_t count,
+uint32_t max_ranges)
+{
+BlockDirtyRangeList *list = NULL, *last_entry = NULL;
+BlockDirtyRangeList **plist = 
+uint64_t begin, end;
+uint64_t start_sector = start >> BDRV_SECTOR_BITS;
+uint64_t last_sector = (start + count - 1) >> BDRV_SECTOR_BITS;
+uint32_t nb_ranges = 0;
+
+BdrvDirtyBitmapIter *it = bdrv_dirty_iter_new(bitmap, start_sector);
+
+while ((begin = bdrv_dirty_iter_next(it)) != -1 && nb_ranges < max_ranges) 
{
+BlockDirtyRange *region;
+if (begin > last_sector) {
+break;
+}
+
+end = bdrv_dirty_bitmap_next_zero(bitmap, begin + 1);
+
+region = g_new0(BlockDirtyRange, 1);
+region->start = begin << BDRV_SECTOR_BITS;
+region->count = (end - begin) << BDRV_SECTOR_BITS;
+
+last_entry = g_new0(BlockDirtyRangeList, 1);
+last_entry->value = region;
+
+*plist = last_entry;
+plist = _entry->next;
+nb_ranges++;
+
+bdrv_set_dirty_iter(it, end + 1);
+}
+
+bdrv_dirty_iter_free(it);
+
+if (list != NULL) {
+if (list->value->start < start) {
+list->value->count -= start - list->value->start;
+list->value->start = start;
+}
+
+if (last_entry->value->start +
+last_entry->value->count > start + count) {
+last_entry->value->count = start + count - 
last_entry->value->start;
+}
+}
+
+return list;
+}
diff --git a/blockdev.c b/blockdev.c
index 1392fff..fa34cf6 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2733,6 +2733,68 @@ void qmp_block_dirty_bitmap_clear(const char *node, 
const char *name,
 aio_context_release(aio_context);
 }
 
+BlockDirtyRangeList *qmp_query_block_dirty_bitmap_ranges(const char *node,
+const char *name, bool has_start, uint64_t start, bool has_count,
+uint64_t count, uint32_t max_ranges, bool has_block_io, bool block_io,
+Error **errp)
+{
+AioContext *aio_context;
+BdrvDirtyBitmap *bitmap;
+BlockDriverState *bs;
+BlockDirtyRangeList *ret = NULL;
+int64_t disk_size;
+
+if (!has_block_io) {
+block_io = true;
+}
+
+bitmap = block_dirty_bitmap_lookup(node, name, ,
+   block_io ? _context : NULL,
+   errp);
+if (!bitmap) {
+return NULL;
+}
+
+if (!block_io && bdrv_dirty_bitmap_enabled(bitmap)) {
+error_setg(errp, "The bitmap is enabled, block-io must be on");
+return NULL;
+}
+
+disk_size = bdrv_getlength(bs);
+if (disk_size < 0) {
+error_setg_errno(errp, -disk_size, "bdrv_getlength failed");
+goto out;
+}
+
+if (has_start) {
+if (start >= disk_size) {
+error_setg(errp, "Start must be less than disk size");
+goto out;
+}
+} else {
+start = 0;
+}
+
+if (has_count) {
+if (start + count > disk_size) {
+error_setg(errp,
+   "(start + count) must not be greater than disk size");
+goto out;
+}
+} else {
+count = disk_size - start;
+}
+
+ret = bdrv_query_dirty_bitmap_ranges(bs, bitmap, start, count, max_ranges);
+
+out:
+if (block_io) {
+aio_context_release(aio_context);
+}
+
+return ret;
+}
+
 void hmp_drive_del(Monitor *mon, const QDict *qdict)
 {
 const char *id = qdict_get_str(qdict, "id");
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 0033942..7918310 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -70,4 +70,11 @@ void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap 
*bitmap);
 
 int64_t bdrv_dirty_bitmap_next_zero(BdrvDirtyBitmap *bitmap, uint64_t start);
 
+/* bdrv_query_dirty_bitmap_ranges
+ * start, count and resulting ranges are all in bytes
+ */

[Qemu-devel] [PATCH 3/6] iotests: test query-block-dirty-bitmap-ranges

[Vladimir Sementsov-Ogievskiy]

Add test for QMP command query-block-dirty-bitmap-ranges.

Signed-off-by: Vladimir Sementsov-Ogievskiy 
---
 tests/qemu-iotests/150 | 88 ++
 tests/qemu-iotests/150.out | 21 +++
 tests/qemu-iotests/group   |  1 +
 3 files changed, 110 insertions(+)
 create mode 100755 tests/qemu-iotests/150
 create mode 100644 tests/qemu-iotests/150.out

diff --git a/tests/qemu-iotests/150 b/tests/qemu-iotests/150
new file mode 100755
index 000..f208050
--- /dev/null
+++ b/tests/qemu-iotests/150
@@ -0,0 +1,88 @@
+#!/usr/bin/env python
+#
+# Test qmp command query-block-dirty-bitmap-ranges
+#
+# (C) Vladimir Sementsov-Ogievskiy 2015
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see .
+#
+
+import os
+import iotests
+import time
+from iotests import qemu_img
+
+disk = os.path.join(iotests.test_dir, 'disk')
+
+size   = 0x4000 # 1G
+sector_size = 512
+granularity = 0x1
+regions = [
+{ 'start': 0,  'count': 0x10 },
+{ 'start': 0x1000, 'count': 0x2  },
+{ 'start': 0x3999, 'count': 0x1  }
+]
+
+class TestQueryDirtyBitmap(iotests.QMPTestCase):
+
+def setUp(self):
+qemu_img('create', '-f', iotests.imgfmt, disk, str(size))
+self.vm = iotests.VM().add_drive(disk)
+self.vm.launch()
+
+def tearDown(self):
+self.vm.shutdown()
+os.remove(disk)
+
+def printRange(self, begin, end):
+print self.vm.qmp('query-block-dirty-bitmap-ranges', node='drive0',
+name='bitmap', start=begin, count=(end - begin),
+max_ranges=100);
+
+def printWhole(self, max_ranges=100):
+print self.vm.qmp('query-block-dirty-bitmap-ranges', node='drive0',
+name='bitmap', max_ranges=max_ranges);
+
+def test_query_bitmap(self):
+result = self.vm.qmp('block-dirty-bitmap-add', node='drive0',
+   name='bitmap', granularity=granularity)
+self.assert_qmp(result, 'return', {});
+
+for r in regions:
+  self.vm.hmp_qemu_io('drive0',
+'write %d %d' % (r['start'], r['count']))
+
+self.printWhole()
+self.printWhole(2)
+self.printRange(100, 200)
+self.printRange(0x10 - 10, 0x1000 + 10)
+print
+
+self.printRange(0x10 - 10, 0x1000 + 1)
+self.printRange(0x10 - 10, 0x1000)
+self.printRange(0x10 - 10, 0x1000 - 1)
+print
+
+self.printRange(0x10 + 1, 0x1000 - 1)
+self.printRange(0x10 + 1, 0x1000)
+self.printRange(0x10 + 1, 0x1000 + 1)
+print
+
+self.printRange(0x10, 0x1000 - 1)
+self.printRange(0x10, 0x1000)
+self.printRange(0x10, 0x1000 + 1)
+
+
+if __name__ == '__main__':
+iotests.main()
diff --git a/tests/qemu-iotests/150.out b/tests/qemu-iotests/150.out
new file mode 100644
index 000..c137925
--- /dev/null
+++ b/tests/qemu-iotests/150.out
@@ -0,0 +1,21 @@
+.
+--
+Ran 1 tests
+
+OK
+{u'return': [{u'count': 1048576, u'start': 0}, {u'count': 131072, u'start': 
268435456}, {u'count': 65536, u'start': 966328320}]}
+{u'return': [{u'count': 1048576, u'start': 0}, {u'count': 131072, u'start': 
268435456}]}
+{u'return': [{u'count': 100, u'start': 100}]}
+{u'return': [{u'count': 10, u'start': 1048566}, {u'count': 10, u'start': 
268435456}]}
+
+{u'return': [{u'count': 10, u'start': 1048566}, {u'count': 1, u'start': 
268435456}]}
+{u'return': [{u'count': 10, u'start': 1048566}]}
+{u'return': [{u'count': 10, u'start': 1048566}]}
+
+{u'return': []}
+{u'return': []}
+{u'return': [{u'count': 1, u'start': 268435456}]}
+
+{u'return': []}
+{u'return': []}
+{u'return': [{u'count': 1, u'start': 268435456}]}
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index d6e9219..84db670 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -142,3 +142,4 @@
 138 rw auto quick
 139 rw auto quick
 142 auto
+150 auto
-- 
1.8.3.1

Re: [Qemu-devel] [PATCH 2/2] qmp-spec: fix index in doc

[Michael Tokarev]

24.01.2016 17:09, Wei Yang wrote:
> The index is duplicated. Just change it.

It is indeed, with previous being 2.5 as well
and the next being 3. Applying to -trivial.

Please the next time send to qemu-devel@ and Cc to -trivial.

Thanks!

/mjt

> Signed-off-by: Wei Yang 
> ---
>  docs/qmp-spec.txt | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/docs/qmp-spec.txt b/docs/qmp-spec.txt
> index 4fb10a5..8e4bc3d 100644
> --- a/docs/qmp-spec.txt
> +++ b/docs/qmp-spec.txt
> @@ -180,7 +180,7 @@ Some events are rate-limited to at most one per second.  
> If additional
>  dropped, and the last one is delayed.  "Similar" normally means same
>  event type.  See qmp-events.txt for details.
>  
> -2.5 QGA Synchronization
> +2.6 QGA Synchronization
>  ---
>  
>  When using QGA, an additional synchronization feature is built into
> 

Re: [Qemu-devel] [PATCH 0/6] Some improvements and small fixes for migration

[Hailiang Zhang]

Hi Juan & Amit,

This series is prerequisite of COLO, and all of them have been reviewed by Dave,
Could you please review and merge them ?


Thanks,
Hailiang

On 2016/1/15 11:37, zhanghailiang wrote:

Patch 1 ~ patch 4 are picked from COLO and live memory snapshot series,
They are just small improvements for migration codes and have been reviewed
by Dave.

Patch 5, 6 are small fixes for migration releated documention.

Please review.

zhanghailiang (6):
   ram: Split host_from_stream_offset() into two helper functions
   migration: Rename the'file' member of MigrationState
   savevm: Split load vm state function qemu_loadvm_state
   migration/ram: Fix some helper functions' parameter to use
 PageSearchStatus
   qmp-commands.hx: Fix the missing options for migration parameters
 commands
   qmp-commands.hx: Document the missing options for migration capability
 commands

  include/exec/ram_addr.h   |   8 ++-
  include/migration/migration.h |   2 +-
  migration/exec.c  |   4 +-
  migration/fd.c|   4 +-
  migration/migration.c |  72 +--
  migration/postcopy-ram.c  |   6 +-
  migration/ram.c   |  73 +++
  migration/rdma.c  |   2 +-
  migration/savevm.c| 158 +-
  migration/tcp.c   |   4 +-
  migration/unix.c  |   4 +-
  qmp-commands.hx   |  33 +++--
  12 files changed, 222 insertions(+), 148 deletions(-)

Re: [Qemu-devel] [RFC 0/3] Draft implementation of HPT resizing (qemu side)

[David Gibson]

On Fri, Jan 29, 2016 at 08:18:39AM +0200, Alexander Graf wrote:
> 
> 
> > Am 29.01.2016 um 04:47 schrieb David Gibson :
> > 
> >> On Thu, Jan 28, 2016 at 10:04:58PM +0100, Alexander Graf wrote:
> >> 
> >> 
> >>> On 01/19/2016 12:02 PM, David Gibson wrote:
>  On Tue, Jan 19, 2016 at 01:18:17PM +0530, Bharata B Rao wrote:
> > On Mon, Jan 18, 2016 at 04:44:38PM +1100, David Gibson wrote:
> > Here is a draft qemu implementation of my proposed PAPR extension for
> > allowing runtime resizing of a KVM/ppc64 guest's hash page table.
> > That in turn will allow for more flexible memory hotplug.
> > 
> > This should work with the guest kernel side patches I also posted
> > recently [1].
> > 
> > Still required to make this into a full implementation:
> >  * Guest needs to auto-resize HPT on memory hotplug events
> > 
> >  * qemu needs to allocate HPT size based on current rather than
> >maximum memory if the guest is HPT resize aware
> > 
> >  * KVM host side implementation
> > 
> >  * PAPR standardization
>  So with the current patchset (QEMU and guest kernel changes), I should
>  be able to change the HTAB size of a PR guest right ? I see the below
>  failure though:
> >>> Uh.. to be honest I haven't really considered the KVM case at all.
> >>> I'm kind of surprised it didn't just refuse to do anything.
> >>> 
>  [root@localhost ~]# cat /sys/kernel/debug/powerpc/pft-size
>  24
>  [root@localhost ~]# echo 26 > /sys/kernel/debug/powerpc/pft-size
>  [   65.996845] lpar: Attempting to resize HPT to shift 26
>  [   65.996845] lpar: Attempting to resize HPT to shift 26
>  [   66.113596] lpar: HPT resize to shift 26 complete (109 ms / 6 ms)
>  [   66.113596] lpar: HPT resize to shift 26 complete (109 ms / 6 ms)
>  
>  PR guest just hangs here while I see tons of below messages in
>  the 1st level guest:
>  
>  KVM can't copy data from 0x3fff99e91400!
>  ...
>  Couldn't emulate instruction 0x (op 0 xop 0)
>  kvmppc_handle_exit_pr: emulation at 700 failed ()
> >>> Hm, not sure why that's happening.  At first I thought it was because
> >>> we weren't updating SDR1 with the address of the new htab, but that's
> >>> actually in there.  Maybe the KVM PR code isn't rereading it after
> >>> initial VM startup.
> >> 
> >> The KVM PR code doesn't care - it just rereads SDR1 on every pteg lookup 
> >> ;).
> >> There's no caching at all.
> > 
> > Ok, no idea why it's not working then.  I'll investigate when I get a 
> > chance.
> > 
> >> Of course, the guest needs to invalidate all pending tlb entries if they're
> >> now invalid.
> >> 
> >> Does this work on real hardware? Say, a G5?
> > 
> > As Paulus says it would be possible to do HPT resizing on real
> > hardware, but the implementation I've done is specific to PAPR.  And
> > obviously qemu wouldn't be relevant to that case.
> 
> So why make it specific to papr? Wouldn't it make sense to have it
> as a (ppc) generic interface in Linux?

Well, I sort of did, in that I added a ppc_md call for it.  I just
haven't implemented it for anything other than PAPR yet - the PAPR
implementation is quite different from what the native one would be,
since the hypervisor needs to handle the rehashing.

> For the PR PAPR case, QEMU allocates the HTAB, so it needs to make
> sure it pushes the changed address as new fake SDR1 value into kvm
> when it changes.

Yes, I'm doing that - have a look at the qemu series.  Not 100% sure
it's correct, since I haven't debugged with PR KVM yet.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v3 07/10] s390x/cpu: Move some CPU initialization into realize

[David Hildenbrand]

> In preparation for hotplug, defer some CPU initialization
> until the device is actually being realized.
> 
> Signed-off-by: Matthew Rosato 
> ---

Reviewed-by: David Hildenbrand 

David

Re: [Qemu-devel] [PATCH 1/2] rdma: remove check on time_spent when calculating mbs

[Michael Tokarev]

24.01.2016 17:09, Wei Yang wrote:
> Within the if statement, time_spent is assured to be non-zero.
> 
> This patch just removes the check on time_spent when calculating mbs.

The if statement is this one:

if (current_time >= initial_time + BUFFER_DELAY) {

Note that we use time_spent as a divisor to calculate
bandwidth too.

This is indeed a trivial patch, I'm applying it to -trivial,
but please the next time post it to qemu-devel (the main
mailinglist), and Cc qemu-trivial if it is trivial.

Thanks!

/mjt

> Signed-off-by: Wei Yang 
> ---
>  migration/migration.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/migration/migration.c b/migration/migration.c
> index c842499..40b87f2 100644
> --- a/migration/migration.c
> +++ b/migration/migration.c
> @@ -1677,8 +1677,8 @@ static void *migration_thread(void *opaque)
>  double bandwidth = (double)transferred_bytes / time_spent;
>  max_size = bandwidth * migrate_max_downtime() / 100;
>  
> -s->mbps = time_spent ? (((double) transferred_bytes * 8.0) /
> -((double) time_spent / 1000.0)) / 1000.0 / 1000.0 : -1;
> +s->mbps = (((double) transferred_bytes * 8.0) /
> +((double) time_spent / 1000.0)) / 1000.0 / 1000.0;
>  
>  trace_migrate_transferred(transferred_bytes, time_spent,
>bandwidth, max_size);
> 

Re: [Qemu-devel] [PATCH] cpu: cpu_save/cpu_load is no more

[Peter Maydell]

On 30 January 2016 at 09:36, Michael Tokarev  wrote:
> 19.01.2016 15:08, Paolo Bonzini wrote:
>> Everything has been converted to vmstate.
>
> Applied to -trivial, thanks!

Ah, I missed that Paolo had sent out this patch before me
(my version is http://patchwork.ozlabs.org/patch/572694/ but
a week later...)

thanks
-- PMM

[Qemu-devel] [Bug 1481272] Re: main-loop: WARNING: I/O thread spun for 1000 iterations

[Martin von Gagern]

I see the same with a Windows 10 guest on a Gentoo host. I tried setting
nonblocking = 0; in vl.c as
https://rafalcieslak.wordpress.com/2015/11/20/qemu-main-loop-warning-io-
thread-spun-for-1000-iterations/ suggests, but that didn't solve the
problem for me: the message is still there.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1481272

Title:
  main-loop: WARNING: I/O thread spun for 1000 iterations

Status in QEMU:
  New

Bug description:
  I compile the latest qemu to launch a VM but the monitor output the
  "main-loop: WARNING: I/O thread spun for 1000 iterations".

  # /usr/local/bin/qemu-system-x86_64 -name rhel6 -S -no-kvm -m 1024M -realtime 
mlock=off -smp 1,sockets=1,cores=1,threads=1 -uuid 
c9dd2a5c-40f2-fd3d-3c54-9cd84f8b9174 -rtc base=utc  -drive 
file=/home/samba-share/ubuntu.img,if=none,id=drive-virtio-disk0,format=qcow2,cache=none
 -device 
virtio-blk-pci,bus=pci.0,addr=0x4,drive=drive-virtio-disk0,id=disk,serial=425618d4-871f-4021-bc5d-bcd7f1b5ca9c,bootindex=0
 -vnc :1 -boot menu=on -monitor stdio
  QEMU 2.3.93 monitor - type 'help' for more information
  (qemu) c
  (qemu) main-loop: WARNING: I/O thread spun for 1000 iterations   
<---

  qemu]# git branch -v
  * master   e95edef Merge remote-tracking branch 
'remotes/sstabellini/tags/xen-migration-2.4-tag' into staging

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1481272/+subscriptions

[Qemu-devel] [Bug 1539940] [NEW] Qemu 2.5 Solaris 8 and 9 sparc hang after terminal type menu

[Zhen Ning Lim]

Public bug reported:

Qemu command:
qemu-system-sparc -nographic -monitor null -serial 
mon:telnet:localhost:3000,server -bios ../../Downloads/ss20_v2.25_rom -M SS-20 
-hda ./solsparc -m 512 -cdrom ./sol-9-905hw-ga-sparc-dvd.iso -boot d -cpu "TI 
SuperSparc 60" -net nic,vlan=1,macaddr=52:54:0:12:34:56


when i do disk2:d, the system loads until the terminal type menu.

What type of terminal are you using?
1) ANSI Standard CRT
2) DEC VT52
3) DEC VT100
4) Heathkit 19
5) Lear Siegler ADM31
6) PC Console
7) Sun Command Tool
8) Sun Workstation
9) Televideo 910
10) Televideo 925
11) Wyse Model 50
12) X Terminal Emulator (xterms)
13) CDE Terminal Emulator (dtterm)
14) Other
Type the number of your choice and press Return: 3
syslog service starting.
savecore: no dump device configured
Running in command line mode

And nothing happens after that. Anyone encountered this issue?

** Affects: qemu
 Importance: Undecided
 Status: New

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1539940

Title:
  Qemu 2.5 Solaris 8 and 9 sparc hang after terminal type menu

Status in QEMU:
  New

Bug description:
  Qemu command:
  qemu-system-sparc -nographic -monitor null -serial 
mon:telnet:localhost:3000,server -bios ../../Downloads/ss20_v2.25_rom -M SS-20 
-hda ./solsparc -m 512 -cdrom ./sol-9-905hw-ga-sparc-dvd.iso -boot d -cpu "TI 
SuperSparc 60" -net nic,vlan=1,macaddr=52:54:0:12:34:56

  
  when i do disk2:d, the system loads until the terminal type menu.

  What type of terminal are you using?
  1) ANSI Standard CRT
  2) DEC VT52
  3) DEC VT100
  4) Heathkit 19
  5) Lear Siegler ADM31
  6) PC Console
  7) Sun Command Tool
  8) Sun Workstation
  9) Televideo 910
  10) Televideo 925
  11) Wyse Model 50
  12) X Terminal Emulator (xterms)
  13) CDE Terminal Emulator (dtterm)
  14) Other
  Type the number of your choice and press Return: 3
  syslog service starting.
  savecore: no dump device configured
  Running in command line mode

  And nothing happens after that. Anyone encountered this issue?

To manage notifications about this bug go to:
https://bugs.launchpad.net/qemu/+bug/1539940/+subscriptions

[Qemu-devel] [PATCH RFC 1/3] linux-user: add rtnetlink(7) support

[Laurent Vivier]

rtnetlink is needed to use iproute package (ip addr, ip route)
and dhcp client.

Examples:

Without this patch:
# ip link
Cannot open netlink socket: Address family not supported by protocol
# ip addr
Cannot open netlink socket: Address family not supported by protocol
# ip route
Cannot open netlink socket: Address family not supported by protocol
# dhclient eth0
Cannot open netlink socket: Address family not supported by protocol
Cannot open netlink socket: Address family not supported by protocol

With this patch:
# ip link
1: lo:  mtu 65536 qdisc noqueue state UNKNOWN mode 
DEFAULT
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
51: eth0:  mtu 1500 qdisc fq_codel state 
UP mode DEFAULT qlen 1000
link/ether 00:16:3e:89:6b:d7 brd ff:ff:ff:ff:ff:ff
# ip addr show eth0
51: eth0:  mtu 1500 qdisc fq_codel state 
UP qlen 1000
link/ether 00:16:3e:89:6b:d7 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.197/24 brd 192.168.122.255 scope global eth0
   valid_lft forever preferred_lft forever
inet6 fe80::216:3eff:fe89:6bd7/64 scope link
   valid_lft forever preferred_lft forever
# ip route
default via 192.168.122.1 dev eth0
192.168.122.0/24 dev eth0  proto kernel  scope link  src 192.168.122.197
# ip addr flush eth0
# ip addr add 192.168.122.10 dev eth0
# ip addr show eth0
51: eth0:  mtu 1500 qdisc fq_codel state 
UP qlen 1000
link/ether 00:16:3e:89:6b:d7 brd ff:ff:ff:ff:ff:ff
inet 192.168.122.10/32 scope global eth0
   valid_lft forever preferred_lft forever
# ip route add 192.168.122.0/24 via 192.168.122.10
# ip route
192.168.122.0/24 via 192.168.122.10 dev eth0

Signed-off-by: Laurent Vivier 
---
 linux-user/syscall.c | 477 ++-
 1 file changed, 471 insertions(+), 6 deletions(-)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index dac5518..a1ed2f5 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -100,6 +100,8 @@ int __clone2(int (*fn)(void *), void *child_stack_base,
 #include 
 #include 
 #include "linux_loop.h"
+#include 
+#include 
 #include "uname.h"
 
 #include "qemu.h"
@@ -295,6 +297,14 @@ static TargetFdTrans **target_fd_trans;
 
 static unsigned int target_fd_max;
 
+static TargetFdDataFunc fd_trans_target_to_host_data(int fd)
+{
+if (fd >= 0 && fd < target_fd_max && target_fd_trans[fd]) {
+return target_fd_trans[fd]->target_to_host_data;
+}
+return NULL;
+}
+
 static TargetFdDataFunc fd_trans_host_to_target_data(int fd)
 {
 if (fd >= 0 && fd < target_fd_max && target_fd_trans[fd]) {
@@ -1222,6 +1232,11 @@ static inline abi_long host_to_target_sockaddr(abi_ulong 
target_addr,
 return -TARGET_EFAULT;
 memcpy(target_saddr, addr, len);
 target_saddr->sa_family = tswap16(addr->sa_family);
+if (addr->sa_family == AF_NETLINK) {
+struct sockaddr_nl *target_nl = (struct sockaddr_nl *)target_saddr;
+target_nl->nl_pid = tswap32(target_nl->nl_pid);
+target_nl->nl_groups = tswap32(target_nl->nl_groups);
+}
 unlock_user(target_saddr, target_addr, len);
 
 return 0;
@@ -1453,6 +1468,416 @@ static inline abi_long host_to_target_cmsg(struct 
target_msghdr *target_msgh,
 return 0;
 }
 
+static void tswap_nlmsghdr(struct nlmsghdr *nlh)
+{
+nlh->nlmsg_len = tswap32(nlh->nlmsg_len);
+nlh->nlmsg_type = tswap16(nlh->nlmsg_type);
+nlh->nlmsg_flags = tswap16(nlh->nlmsg_flags);
+nlh->nlmsg_seq = tswap32(nlh->nlmsg_seq);
+nlh->nlmsg_pid = tswap32(nlh->nlmsg_pid);
+}
+
+static abi_long host_to_target_for_each_nlmsg(struct nlmsghdr *nlh,
+  size_t len,
+  abi_long (*host_to_target_nlmsg)
+   (struct nlmsghdr *))
+{
+uint32_t nlmsg_len;
+abi_long ret;
+
+while (len > (int)sizeof(struct nlmsghdr)) {
+
+nlmsg_len = nlh->nlmsg_len;
+if (nlmsg_len < sizeof(struct nlmsghdr) ||
+nlmsg_len > len) {
+break;
+}
+
+if (nlh->nlmsg_type == NLMSG_DONE) {
+tswap_nlmsghdr(nlh);
+break;
+}
+switch (nlh->nlmsg_type) {
+case NLMSG_NOOP:
+break;
+case NLMSG_ERROR: {
+struct nlmsgerr *e = NLMSG_DATA(nlh);
+e->error = tswap32(e->error);
+tswap_nlmsghdr(>msg);
+tswap_nlmsghdr(nlh);
+}
+return 0;
+default:
+ret = host_to_target_nlmsg(nlh);
+if (ret < 0) {
+tswap_nlmsghdr(nlh);
+return ret;
+}
+break;
+}
+

[Qemu-devel] [PATCH RFC 0/3] linux-user: netlink support

[Laurent Vivier]

Since commit:
e36800c linux-user: add signalfd/signalfd4 syscalls

It is now possible to register handlers to a file descriptor
to translate a data stream transiting by this file descriptor.

We can now decode netlink information coming from the guest
and inject a translated one into the host, and vice-versa.

This series is an "RFC" because it works (we can boot a
container using systemd and use iproute tools) but some
problems remain.

Some results (x86_64 host) with some guests:

* ppc: it can boot a debian 8.2/8.3 (Jessie) LXC container
  and networking works fine (dhcp and "apt-get upgrade").

  "ip link" generates some traces in the kernel log:
  "netlink: 8 bytes leftover after parsing attributes in process `ip'."

* ppc64: it can boot a fedora 21 LXC container.

  Some issues with dhclient and "dnf update"
  (-> Invalid instruction, can be caused by a memory corruption done
   by netlink calls).

  "ip link" generates some traces in the kernel log:
  "netlink: 8 bytes leftover after parsing attributes in process `ip'."

* ppc64le: Debian 8.3 (Jessie) works fine.

* sh4: container doesn't work but 'ip' in a chroot works well.

* arm: Raspbian 8.3 (Jessie) works fine.

* s390x: container Debian 8.1 boots well, but "apt-get" hangs on
  networking (name resolution?).

  "ip link" generates some traces in the kernel log:
  "netlink: 8 bytes leftover after parsing attributes in process `ip'."

Laurent Vivier (3):
  linux-user: add rtnetlink(7) support
  linux-user: support netlink protocol NETLINK_KOBJECT_UEVENT
  linux-user: add netlink audit

 linux-user/syscall.c | 537 ++-
 1 file changed, 531 insertions(+), 6 deletions(-)

-- 
2.5.0

[Qemu-devel] [PATCH RFC 2/3] linux-user: support netlink protocol NETLINK_KOBJECT_UEVENT

[Laurent Vivier]

This is the protocol used by udevd to manage kernel events.

Signed-off-by: Laurent Vivier 
---
 linux-user/syscall.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index a1ed2f5..790ae49 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -2572,6 +2572,9 @@ static abi_long do_socket(int domain, int type, int 
protocol)
 case NETLINK_ROUTE:
 fd_trans_register(ret, _netlink_route_trans);
 break;
+case NETLINK_KOBJECT_UEVENT:
+/* nothing to do: messages are strings */
+break;
 default:
 close(ret);
 ret = -EPFNOSUPPORT;
-- 
2.5.0

[Qemu-devel] [PATCH RFC 3/3] linux-user: add netlink audit

[Laurent Vivier]

This is, for instance, needed to log in a container.

Without this, the user cannot be identified and the console login
fails with "Login incorrect".

Signed-off-by: Laurent Vivier 
---
 linux-user/syscall.c | 57 
 1 file changed, 57 insertions(+)

diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 790ae49..fa50299 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -102,6 +102,7 @@ int __clone2(int (*fn)(void *), void *child_stack_base,
 #include "linux_loop.h"
 #include 
 #include 
+#include 
 #include "uname.h"
 
 #include "qemu.h"
@@ -1878,6 +1879,44 @@ static abi_long target_to_host_nlmsg_route(struct 
nlmsghdr *nlh, size_t len)
 return target_to_host_for_each_nlmsg(nlh, len, target_to_host_data_route);
 }
 
+static abi_long host_to_target_data_audit(struct nlmsghdr *nlh)
+{
+switch (nlh->nlmsg_type) {
+default:
+fprintf(stderr, "Unknown host audit message type %d\n",
+nlh->nlmsg_type);
+return -TARGET_EINVAL;
+}
+return 0;
+}
+
+static inline abi_long host_to_target_nlmsg_audit(struct nlmsghdr *nlh,
+  size_t len)
+{
+return host_to_target_for_each_nlmsg(nlh, len, host_to_target_data_audit);
+}
+
+static abi_long target_to_host_data_audit(struct nlmsghdr *nlh)
+{
+switch (nlh->nlmsg_type) {
+case AUDIT_USER:
+case AUDIT_FIRST_USER_MSG ... AUDIT_LAST_USER_MSG:
+case AUDIT_FIRST_USER_MSG2 ... AUDIT_LAST_USER_MSG2:
+break;
+default:
+fprintf(stderr, "Unknown target audit message type %d\n",
+nlh->nlmsg_type);
+return -TARGET_EINVAL;
+}
+
+return 0;
+}
+
+static abi_long target_to_host_nlmsg_audit(struct nlmsghdr *nlh, size_t len)
+{
+return target_to_host_for_each_nlmsg(nlh, len, target_to_host_data_audit);
+}
+
 /* do_setsockopt() Must return target values and target errnos. */
 static abi_long do_setsockopt(int sockfd, int level, int optname,
   abi_ulong optval_addr, socklen_t optlen)
@@ -2543,6 +2582,21 @@ static TargetFdTrans target_netlink_route_trans = {
 .host_to_target_data = netlink_route_host_to_target,
 };
 
+static abi_long netlink_audit_target_to_host(void *buf, size_t len)
+{
+return target_to_host_nlmsg_audit(buf, len);
+}
+
+static abi_long netlink_audit_host_to_target(void *buf, size_t len)
+{
+return host_to_target_nlmsg_audit(buf, len);
+}
+
+static TargetFdTrans target_netlink_audit_trans = {
+.target_to_host_data = netlink_audit_target_to_host,
+.host_to_target_data = netlink_audit_host_to_target,
+};
+
 /* do_socket() Must return target values and target errnos. */
 static abi_long do_socket(int domain, int type, int protocol)
 {
@@ -2575,6 +2629,9 @@ static abi_long do_socket(int domain, int type, int 
protocol)
 case NETLINK_KOBJECT_UEVENT:
 /* nothing to do: messages are strings */
 break;
+case NETLINK_AUDIT:
+fd_trans_register(ret, _netlink_audit_trans);
+break;
 default:
 close(ret);
 ret = -EPFNOSUPPORT;
-- 
2.5.0

Re: [Qemu-devel] [PATCH v5 2/5] util: Use new error_report_fatal/abort instead of error_setg(_fatal/abort)

[David Gibson]

On Fri, Jan 29, 2016 at 02:33:08PM +0100, Lluís Vilanova wrote:
> David Gibson writes:
> 
> > On Thu, Jan 28, 2016 at 10:53:43PM +0100, Lluís Vilanova wrote:
> >> Replaces all direct uses of 'error_setg(_fatal/abort)' with
> >> 'error_report_fatal/abort'. Also reimplements the former on top of the
> >> latter.
> >> 
> >> Signed-off-by: Lluís Vilanova 
> 
> > I think the spapr parts of this will be obsoleted by the cleanups to
> > error handling included in the pull request I sent today.
> 
> Ok, then I'll rebase once merged. Is there an ETA for the merge?

Well, I'd been hoping a couple of days ago, but there were some
mistakes in the last pull request, so.. sometime this week, I hope?

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [Qemu-ppc] [PULL 03/39] macio: use the existing IDEDMA aiocb to hold the active DMA aiocb

[David Gibson]

On Fri, Jan 29, 2016 at 09:02:47AM +0100, Aurelien Jarno wrote:
> On 2016-01-29 16:06, David Gibson wrote:
> > From: Mark Cave-Ayland 
> > 
> > Currently the aiocb is held within MACIOIDEState, however the IDE core code
> > assumes that the current actvie DMA aiocb is held in aiocb in a few places,
> > e.g. ide_bus_reset() and ide_reset().
> > 
> > Switch over to using IDEDMA aiocb to store the aiocb for the current active
> > DMA request so that bus resets and restarts are handled correctly. As a
> > consequence we can now use ide_set_inactive() rather than handling its
> > functionality ourselves.
> > 
> > Signed-off-by: Mark Cave-Ayland 
> > Reviewed-by: John Snow 
> > Signed-off-by: David Gibson 
> > ---
> >  hw/ide/macio.c  |  20 +-
> >  hw/ide/macio.c.orig | 634 
> > 
> 
> I don't think you want to add this file to the git.

Sigh.  Indeed.  Not doing well with these pulls lately :(.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v3 2/3] .travis.yml: run make check for all matrix targets

[David Gibson]

On Thu, Jan 28, 2016 at 02:23:28PM +, Alex Bennée wrote:
> We only ran make check once before it used to be an unreliable target.
> It was only a stop gap measure and we should be able to revert it now.
> This also stops us needing a large all-MMU build.
> 
> We disable "make check" for a couple of the extra config targets which
> are currently broken.
> 
> Signed-off-by: Alex Bennée 

So, in general I like the idea of running make check more widely.

However.. I was wondering - what's the rationale for having separate
matrix builds for each target (or small group) rather than just doing
one build with all the targets?

I can't see any obvious benefit to splitting the build that way, but
it does increase the total build time significantly - and will do so
rather more so with make check added.

> ---
>  .travis.yml | 15 ---
>  1 file changed, 8 insertions(+), 7 deletions(-)
> 
> diff --git a/.travis.yml b/.travis.yml
> index 4a0c23a..16be23f 100644
> --- a/.travis.yml
> +++ b/.travis.yml
> @@ -40,7 +40,7 @@ notifications:
>  on_failure: always
>  env:
>global:
> -- TEST_CMD=""
> +- TEST_CMD="make check"
>  - EXTRA_CONFIG=""
>matrix:
>  # Group major targets together with their linux-user counterparts
> @@ -73,17 +73,14 @@ script:
>  matrix:
># We manually include a number of additional build for non-standard bits
>include:
> -# Make check target (we only do this once)
> -- env:
> -- 
> TARGETS=alpha-softmmu,arm-softmmu,aarch64-softmmu,cris-softmmu,i386-softmmu,x86_64-softmmu,m68k-softmmu,microblaze-softmmu,microblazeel-softmmu,mips-softmmu,mips64-softmmu,mips64el-softmmu,mipsel-softmmu,or32-softmmu,ppc-softmmu,ppc64-softmmu,ppcemb-softmmu,s390x-softmmu,sh4-softmmu,sh4eb-softmmu,sparc-softmmu,sparc64-softmmu,unicore32-softmmu,lm32-softmmu,moxie-softmmu,tricore-softmmu,xtensa-softmmu,xtensaeb-softmmu
> -  TEST_CMD="make check"
> -  compiler: gcc
>  # Debug related options
>  - env: TARGETS=i386-softmmu,x86_64-softmmu
> EXTRA_CONFIG="--enable-debug"
>compiler: gcc
> +# We currently disable "make check"
>  - env: TARGETS=i386-softmmu,x86_64-softmmu
> EXTRA_CONFIG="--enable-debug --enable-tcg-interpreter"
> +   TEST_CMD=""
>compiler: gcc
>  # Disable a few of the optional features
>  - env: TARGETS=i386-softmmu,x86_64-softmmu
> @@ -104,11 +101,15 @@ matrix:
>  - env: TARGETS=i386-softmmu,x86_64-softmmu
> EXTRA_CONFIG="--enable-trace-backends=simple"
>compiler: gcc
> +# We currently disable "make check"
>  - env: TARGETS=i386-softmmu,x86_64-softmmu
> EXTRA_CONFIG="--enable-trace-backends=ftrace"
> +   TEST_CMD=""
>compiler: gcc
> +# We currently disable "make check"
>  - env: TARGETS=i386-softmmu,x86_64-softmmu
> -  EXTRA_CONFIG="--enable-trace-backends=ust"
> +   EXTRA_CONFIG="--enable-trace-backends=ust"
> +   TEST_CMD=""
>compiler: gcc
>  - env: TARGETS=i386-softmmu,x86_64-softmmu
> EXTRA_CONFIG="--enable-modules"

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PULL 00/39] ppc-for-2.6 queue 20160129

[David Gibson]

On Fri, Jan 29, 2016 at 02:48:23PM +, Peter Maydell wrote:
> On 29 January 2016 at 05:06, David Gibson  wrote:
> > The following changes since commit 357e81c7e880f868833edf9f53cce1f3b09ea8ec:
> >
> >   Merge remote-tracking branch 'remotes/cohuck/tags/s390x-20160128' into 
> > staging (2016-01-28 11:46:34 +)
> >
> > are available in the git repository at:
> >
> >   git://github.com/dgibson/qemu.git tags/ppc-for-2.6-20160129
> >
> > for you to fetch changes up to 1699679e699276c0538008f6ca74cd04e6c68b42:
> >
> >   target-ppc: Make every FPSCR_ macro have a corresponding FP_ macro 
> > (2016-01-29 14:01:52 +1100)
> >
> > 
> > ppc patch queue for 2016-01-29
> >
> > Currently accumulated patches for target-ppc, pseries machine type and
> > related devices.
> >   * Cleanup of error handling code in spapr
> >   * A number of fixes for Macintosh devices for the benefit of MacOS 9 and X
> >   * Remove some abuses of the RTAS memory access functions in spapr
> >   * Fixes for the gdbstub (and monitor debug) for VMX and VSX extensions.
> >   * Fix pseries machine hotplug memory under TCG
> >   * Clean up and extend handling of multiple page sizes with 64-bit hash 
> > MMUs
> >
> 
> Hi. Unfortunately this generates errors when built with clang:
> 
> /home/petmay01/linaro/qemu-for-merges/target-ppc/mmu_helper.c:660:20:
> error: unused function 'ppc4xx_tlb_invalidate_virt'
> [-Werror,-Wunused-function]
> static inline void ppc4xx_tlb_invalidate_virt(CPUPPCState *env,
>^
> 1 error generated.
> 
> The function does appear from a quick grep to be entirely unused...
> 
> (GCC doesn't complain about this because it doesn't warn about unused
> static inline functions in a .c file, but clang does.)

Dammit.  Sorry.

Now.. why didn't travis pick that up :/.

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature