Re: [Qemu-devel] [PATCH v5 14/27] qemu-img: Add "-L" option to sub commands

[Fam Zheng]

On Tue, 05/24 20:06, Max Reitz wrote:
> On 17.05.2016 09:35, Fam Zheng wrote:
> > If specified, BDRV_O_NO_LOCK flag will be set when opening the image.
> > 
> > Signed-off-by: Fam Zheng 
> > ---
> >  qemu-img.c | 89 
> > ++
> >  1 file changed, 72 insertions(+), 17 deletions(-)
> > 
> > diff --git a/qemu-img.c b/qemu-img.c
> > index 4792366..b13755b 100644
> > --- a/qemu-img.c
> > +++ b/qemu-img.c
> 
> [...]
> 
> > @@ -1206,6 +1220,7 @@ static int img_compare(int argc, char **argv)
> >  qemu_progress_init(progress, 2.0);
> >  
> >  flags = 0;
> > +flags |= nolock ? BDRV_O_NO_LOCK : 0;
> 
> This reads weird. I'd either put this line below bdrv_parse_cache_mode()
> or drop the line initializing src_flags to 0 (and make this a plain
> assignment).
> 
> >  ret = bdrv_parse_cache_mode(cache, , );
> >  if (ret < 0) {
> >  error_report("Invalid source cache option: %s", cache);
> 
> [...]
> 
> > @@ -1907,6 +1926,7 @@ static int img_convert(int argc, char **argv)
> >  }
> >  
> >  src_flags = 0;
> > +src_flags |= nolock ? BDRV_O_NO_LOCK : 0;
> 
> Same here.

OK, will drop the dead assignment above.

> 
> Also: Should we have distinct flags for source and target for convert?
> For instance, I can imagine someone wanting not to lock the source but
> leave the target in default exclusive mode.

"-L" is a shorthand flag, for finer control (shared locking mode and separate
modes for src and dest), I would recommend using --image-opts then.

Fam

[Qemu-devel] [RFC PATCH] docker: Support ".pre" script when building image

[Fam Zheng]

When building "foo.docker", if a "foo.pre" script exists, it will be
executed in the building context before "docker build" is invoked.

Signed-off-by: Fam Zheng 
---
 tests/docker/docker.py | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
index 0151362..7b8f022 100755
--- a/tests/docker/docker.py
+++ b/tests/docker/docker.py
@@ -99,6 +99,12 @@ class Docker(object):
 tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
  _text_checksum(dockerfile))
 tmp_df.flush()
+pre_path = os.path.abspath(df_path[:-len(".docker")] + ".pre")
+if os.path.isfile(pre_path):
+if quiet:
+subprocess.check_output([pre_path], cwd=tmp_dir)
+else:
+subprocess.check_call([pre_path], cwd=tmp_dir)
 self._do(["build", "-t", tag, "-f", tmp_df.name] + argv + \
  [tmp_dir],
  quiet=quiet)
-- 
2.8.2

[Qemu-devel] [PATCH v7 15/15] MAINTAINERS: Add tests/docker

[Fam Zheng]

Reviewed-by: Alex Bennée 
Signed-off-by: Fam Zheng 
---
 MAINTAINERS | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/MAINTAINERS b/MAINTAINERS
index 3c949d5..091272e 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1615,3 +1615,10 @@ Build system architecture
 M: Daniel P. Berrange 
 S: Odd Fixes
 F: docs/build-system.txt
+
+Docker testing
+--
+Docker based testing framework and cases
+M: Fam Zheng 
+S: Maintained
+F: tests/docker/
-- 
2.8.2

[Qemu-devel] [PATCH v2 1/2] Makefile: Add a "FORCE" target

[Fam Zheng]

Signed-off-by: Fam Zheng 
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 3a3c5dc..1e2d6f9 100644
--- a/Makefile
+++ b/Makefile
@@ -81,7 +81,7 @@ Makefile: ;
 configure: ;
 
 .PHONY: all clean cscope distclean dvi html info install install-doc \
-   pdf recurse-all speed test dist msi
+   pdf recurse-all speed test dist msi FORCE
 
 $(call set-vpath, $(SRC_PATH))
 
-- 
2.8.2

[Qemu-devel] [PATCH v7 09/15] docker: Add full test

[Fam Zheng]

This builds all available targets.

Signed-off-by: Fam Zheng 
Reviewed-by: Alex Bennée 
---
 tests/docker/test-full | 17 +
 1 file changed, 17 insertions(+)
 create mode 100755 tests/docker/test-full

diff --git a/tests/docker/test-full b/tests/docker/test-full
new file mode 100755
index 000..fd9b798
--- /dev/null
+++ b/tests/docker/test-full
@@ -0,0 +1,17 @@
+#!/bin/bash -e
+#
+# Compile all the targets.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng 
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+build_qemu
+make check $MAKEFLAGS
-- 
2.8.2

[Qemu-devel] [PATCH v2 2/2] Makefile: Derive "PKGVERSION" from "git describe" by default

[Fam Zheng]

Currently, if not specified in "./configure", QEMU_PKGVERSION will be
empty. Write a rule in Makefile to generate a value from "git describe"
combined with a possible git tree cleanness suffix, and write into a new
header.

$ cat qemu-version.h
#define QEMU_PKGVERSION "-v2.6.0-557-gd6550e9-dirty"

Include the header in .c files where the macro is referenced. It's not
necessary to include it in all files, otherwise each time the content of
the file changes, all sources have to be recompiled.

Signed-off-by: Fam Zheng 
---
 Makefile  | 22 +-
 linux-user/main.c |  1 +
 qemu-img.c|  1 +
 qmp.c |  1 +
 scripts/create_config |  4 
 vl.c  |  1 +
 6 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index 1e2d6f9..6abb0cd 100644
--- a/Makefile
+++ b/Makefile
@@ -49,7 +49,7 @@ ifneq ($(filter-out $(UNCHECKED_GOALS),$(MAKECMDGOALS)),$(if 
$(MAKECMDGOALS),,fa
 endif
 endif
 
-GENERATED_HEADERS = config-host.h qemu-options.def
+GENERATED_HEADERS = qemu-version.h config-host.h qemu-options.def
 GENERATED_HEADERS += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h
 GENERATED_SOURCES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c
 GENERATED_HEADERS += qmp-introspect.h
@@ -166,6 +166,26 @@ endif
 
 all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all modules
 
+qemu-version.h: FORCE
+   $(call quiet-command, \
+   (cd $(SRC_PATH); \
+   printf '#define QEMU_PKGVERSION '; \
+   if test -n "$(PKGVERSION)"; then \
+   printf '"$(PKGVERSION)"\n'; \
+   else \
+   printf '"-'; \
+   if ! git status &>/dev/null; then \
+   printf "no-git"; \
+   else \
+   git describe 2>/dev/null | tr -d '\n'; \
+   if ! git diff-index --quiet HEAD &>/dev/null; 
then \
+   printf '-dirty'; \
+   fi \
+   fi; \
+   printf '"\n'; \
+   fi) > $@.tmp)
+   $(call quiet-command, cmp --quiet $@ $@.tmp || mv $@.tmp $@)
+
 config-host.h: config-host.h-timestamp
 config-host.h-timestamp: config-host.mak
 qemu-options.def: $(SRC_PATH)/qemu-options.hx
diff --git a/linux-user/main.c b/linux-user/main.c
index b2bc6ab..8a11d02 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -17,6 +17,7 @@
  *  along with this program; if not, see .
  */
 #include "qemu/osdep.h"
+#include "qemu-version.h"
 #include 
 #include 
 #include 
diff --git a/qemu-img.c b/qemu-img.c
index 4b56ad3..32e307c 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -22,6 +22,7 @@
  * THE SOFTWARE.
  */
 #include "qemu/osdep.h"
+#include "qemu-version.h"
 #include "qapi/error.h"
 #include "qapi-visit.h"
 #include "qapi/qmp-output-visitor.h"
diff --git a/qmp.c b/qmp.c
index 3165f87..7df6543 100644
--- a/qmp.c
+++ b/qmp.c
@@ -14,6 +14,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu-version.h"
 #include "qemu/cutils.h"
 #include "monitor/monitor.h"
 #include "sysemu/sysemu.h"
diff --git a/scripts/create_config b/scripts/create_config
index b2d2ebb..d47057b 100755
--- a/scripts/create_config
+++ b/scripts/create_config
@@ -9,10 +9,6 @@ case $line in
 version=${line#*=}
 echo "#define QEMU_VERSION \"$version\""
 ;;
- PKGVERSION=*) # configuration
-pkgversion=${line#*=}
-echo "#define QEMU_PKGVERSION \"$pkgversion\""
-;;
  qemu_*dir=*) # qemu-specific directory configuration
 name=${line%=*}
 value=${line#*=}
diff --git a/vl.c b/vl.c
index 18d1423..97cad71 100644
--- a/vl.c
+++ b/vl.c
@@ -22,6 +22,7 @@
  * THE SOFTWARE.
  */
 #include "qemu/osdep.h"
+#include "qemu-version.h"
 #include "qemu/cutils.h"
 #include "qemu/help_option.h"
 
-- 
2.8.2

[Qemu-devel] [PATCH v2 0/2] Let PKGVERSION include the "git describe" output

[Fam Zheng]

v2: Address Eric's comments:
"echo -n" -> "printf".
"-unclean" -> "-dirty".

Makefile happened to be in my working set because of the docker test work, so I
went ahead to try this nice feature last evening before going to bed. My
apologies if we have duplicated work.

Please review, especially please comment on the redundant version string:

$ qemu-img --version
qemu-img version 2.6.50-v2.6.0-603-g684a494, Copyright (c) 2004-2008 Fabrice 
Bellard


Fam Zheng (2):
  Makefile: Add a "FORCE" target
  Makefile: Derive "PKGVERSION" from "git describe" by default

 Makefile  | 24 ++--
 linux-user/main.c |  1 +
 qemu-img.c|  1 +
 qmp.c |  1 +
 scripts/create_config |  4 
 vl.c  |  1 +
 6 files changed, 26 insertions(+), 6 deletions(-)

-- 
2.8.2

[Qemu-devel] [PATCH v7 05/15] docker: Add images

[Fam Zheng]

Reviewed-by: Alex Bennée 
Signed-off-by: Fam Zheng 
---
 tests/docker/dockerfiles/centos6.docker |  6 ++
 tests/docker/dockerfiles/fedora.docker  |  7 +++
 tests/docker/dockerfiles/ubuntu.docker  | 11 +++
 3 files changed, 24 insertions(+)
 create mode 100644 tests/docker/dockerfiles/centos6.docker
 create mode 100644 tests/docker/dockerfiles/fedora.docker
 create mode 100644 tests/docker/dockerfiles/ubuntu.docker

diff --git a/tests/docker/dockerfiles/centos6.docker 
b/tests/docker/dockerfiles/centos6.docker
new file mode 100644
index 000..8f4fe46
--- /dev/null
+++ b/tests/docker/dockerfiles/centos6.docker
@@ -0,0 +1,6 @@
+FROM centos:6
+RUN yum install -y \
+tar git make gcc g++ \
+zlib-devel glib2-devel SDL-devel pixman-devel \
+epel-release
+RUN yum install -y libfdt-devel ccache
diff --git a/tests/docker/dockerfiles/fedora.docker 
b/tests/docker/dockerfiles/fedora.docker
new file mode 100644
index 000..6251e45
--- /dev/null
+++ b/tests/docker/dockerfiles/fedora.docker
@@ -0,0 +1,7 @@
+FROM fedora:23
+RUN dnf install -y \
+ccache git tar \
+glib2-devel pixman-devel zlib-devel SDL-devel libfdt-devel \
+gcc gcc-c++ clang make perl which bc findutils \
+
mingw{32,64}-{pixman,glib2,gmp,SDL,pkg-config,gtk2,gtk3,gnutls,nettle,libtasn1,libjpeg-turbo,libpng,curl,libssh2,bzip2}
+ENV FEATURES mingw clang
diff --git a/tests/docker/dockerfiles/ubuntu.docker 
b/tests/docker/dockerfiles/ubuntu.docker
new file mode 100644
index 000..725a7ca
--- /dev/null
+++ b/tests/docker/dockerfiles/ubuntu.docker
@@ -0,0 +1,11 @@
+FROM ubuntu:14.04
+RUN echo "deb http://archive.ubuntu.com/ubuntu/ trusty universe multiverse" >> 
\
+/etc/apt/sources.list
+RUN apt-get update
+RUN apt-get -y install \
+libusb-1.0-0-dev libiscsi-dev librados-dev libncurses5-dev \
+libseccomp-dev libgnutls-dev libssh2-1-dev  libspice-server-dev \
+libspice-protocol-dev libnss3-dev libfdt-dev \
+libgtk-3-dev libvte-2.90-dev libsdl1.2-dev libpng12-dev libpixman-1-dev \
+git make ccache python-yaml gcc clang sparse
+ENV FEATURES clang ccache pyyaml
-- 
2.8.2

[Qemu-devel] [PATCH v7 03/15] Makefile: Always include rules.mak

[Fam Zheng]

When config-host.mak is not found it is safe to assume SRC_PATH is ".".
So, it is okay to move inclusion of ruls.mak out of the ifeq condition.

Signed-off-by: Fam Zheng 
---
 Makefile | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index 3a3c5dc..1c2a7bd 100644
--- a/Makefile
+++ b/Makefile
@@ -30,7 +30,6 @@ CONFIG_ALL=y
 -include config-all-devices.mak
 -include config-all-disas.mak
 
-include $(SRC_PATH)/rules.mak
 config-host.mak: $(SRC_PATH)/configure
@echo $@ is out-of-date, running configure
@# TODO: The next lines include code which supports a smooth
@@ -49,6 +48,8 @@ ifneq ($(filter-out $(UNCHECKED_GOALS),$(MAKECMDGOALS)),$(if 
$(MAKECMDGOALS),,fa
 endif
 endif
 
+include $(SRC_PATH)/rules.mak
+
 GENERATED_HEADERS = config-host.h qemu-options.def
 GENERATED_HEADERS += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h
 GENERATED_SOURCES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c
-- 
2.8.2

[Qemu-devel] [PATCH v7 14/15] docker: Add EXTRA_CONFIGURE_OPTS

[Fam Zheng]

Whatever passed in this variable will be appended to all
configure commands.

Signed-off-by: Fam Zheng 
Reviewed-by: Alex Bennée 
---
 tests/docker/Makefile.include | 3 +++
 tests/docker/common.rc| 1 +
 2 files changed, 4 insertions(+)

diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
index 39ed7f8..a76be22 100644
--- a/tests/docker/Makefile.include
+++ b/tests/docker/Makefile.include
@@ -87,6 +87,8 @@ docker:
@echo
@echo 'Special variables:'
@echo 'TARGET_LIST=a,b,cOverride target list in builds.'
+   @echo 'EXTRA_CONFIGURE_OPTS="..."'
+   @echo ' Extra configure options.'
@echo 'IMAGES="a b c ..":   Filters which images to build or run.'
@echo 'TESTS="x y z .." Filters which tests to run (for 
docker-test).'
@echo 'J=[0..9]*Overrides the -jN parameter for make 
commands'
@@ -107,6 +109,7 @@ docker-run-%: docker-qemu-src
-t \
$(if $(DEBUG),-i,--net=none) \
-e TARGET_LIST=$(TARGET_LIST) \
+   -e EXTRA_CONFIGURE_OPTS=$(EXTRA_CONFIGURE_OPTS) 
\
-e V=$V -e J=$J -e DEBUG=$(DEBUG)\
-e CCACHE_DIR=/var/tmp/ccache \
-v $$(realpath 
$(DOCKER_SRC_COPY)):/var/tmp/qemu:z$(COMMA)ro \
diff --git a/tests/docker/common.rc b/tests/docker/common.rc
index 74b89d6..c493eeb 100755
--- a/tests/docker/common.rc
+++ b/tests/docker/common.rc
@@ -26,6 +26,7 @@ build_qemu()
 $QEMU_SRC/configure \
 --target-list="${TARGET_LIST}" \
 --prefix="$PWD/install" \
+$EXTRA_CONFIGURE_OPTS \
 "$@"
 make $MAKEFLAGS
 }
-- 
2.8.2

[Qemu-devel] [PATCH v7 12/15] docker: Add travis tool

[Fam Zheng]

The script is not prefixed with test- so it won't run with "make docker-test",
because it can take too long.

Run it with "make docker-travis@ubuntu".

Reviewed-by: Alex Bennée 
Signed-off-by: Fam Zheng 
---
 tests/docker/travis| 21 +
 tests/docker/travis.py | 48 
 2 files changed, 69 insertions(+)
 create mode 100755 tests/docker/travis
 create mode 100755 tests/docker/travis.py

diff --git a/tests/docker/travis b/tests/docker/travis
new file mode 100755
index 000..d345393
--- /dev/null
+++ b/tests/docker/travis
@@ -0,0 +1,21 @@
+#!/bin/bash -e
+#
+# Mimic a travis testing matrix
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng 
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+requires pyyaml
+cmdfile=/tmp/travis_cmd_list.sh
+$QEMU_SRC/tests/docker/travis.py $QEMU_SRC/.travis.yml > $cmdfile
+chmod +x $cmdfile
+cd "$QEMU_SRC"
+$cmdfile
diff --git a/tests/docker/travis.py b/tests/docker/travis.py
new file mode 100755
index 000..8dcc964
--- /dev/null
+++ b/tests/docker/travis.py
@@ -0,0 +1,48 @@
+#!/usr/bin/env python
+#
+# Travis YAML config parser
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng 
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+import sys
+import yaml
+import itertools
+
+def load_yaml(fname):
+return yaml.load(open(fname, "r").read())
+
+def conf_iter(conf):
+def env_to_list(env):
+return env if isinstance(env, list) else [env]
+global_env = conf["env"]["global"]
+for entry in conf["matrix"]["include"]:
+yield {"env": global_env + env_to_list(entry["env"]),
+   "compiler": entry["compiler"]}
+for entry in itertools.product(conf["compiler"],
+   conf["env"]["matrix"]):
+yield {"env": global_env + env_to_list(entry[1]),
+   "compiler": entry[0]}
+
+def main():
+if len(sys.argv) < 2:
+sys.stderr.write("Usage: %s \n" % sys.argv[0])
+return 1
+conf = load_yaml(sys.argv[1])
+for config in conf_iter(conf):
+print "("
+print "\n".join(config["env"])
+print "alias cc=" + config["compiler"]
+print "\n".join(conf["before_script"])
+print "\n".join(conf["script"])
+print ")"
+return 0
+
+if __name__ == "__main__":
+sys.exit(main())
-- 
2.8.2

[Qemu-devel] [PATCH v7 10/15] docker: Add clang test

[Fam Zheng]

The (currently partially commented out) configure options are suggested
by John Snow .

Signed-off-by: Fam Zheng 
Reviewed-by: Alex Bennée 
---
 tests/docker/test-clang | 26 ++
 1 file changed, 26 insertions(+)
 create mode 100755 tests/docker/test-clang

diff --git a/tests/docker/test-clang b/tests/docker/test-clang
new file mode 100755
index 000..6745dbe
--- /dev/null
+++ b/tests/docker/test-clang
@@ -0,0 +1,26 @@
+#!/bin/bash -e
+#
+# Compile and check with clang.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng 
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+requires clang
+
+OPTS="--enable-debug --cxx=clang++ --cc=clang --host-cc=clang"
+# -fsanitize=undefined is broken on Fedora 23, skip it for now
+# See also: https://bugzilla.redhat.com/show_bug.cgi?id=1263834
+#OPTS="$OPTS --extra-cflags=-fsanitize=undefined \
+#--extra-cflags=-fno-sanitize=float-divide-by-zero"
+DEF_TARGET_LIST="$(echo {x86_64,aarch64}-softmmu)"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu $OPTS
+make $MAKEFLAGS check
-- 
2.8.2

[Qemu-devel] [PATCH v7 13/15] docs: Add text for tests/docker in build-system.txt

[Fam Zheng]

Reviewed-by: Alex Bennée 
Signed-off-by: Fam Zheng 
---
 docs/build-system.txt | 5 +
 1 file changed, 5 insertions(+)

diff --git a/docs/build-system.txt b/docs/build-system.txt
index 5ea..2af1e66 100644
--- a/docs/build-system.txt
+++ b/docs/build-system.txt
@@ -438,6 +438,11 @@ top level Makefile, so anything defined in this file will 
influence the
 entire build system. Care needs to be taken when writing rules for tests
 to ensure they only apply to the unit test execution / build.
 
+- tests/docker/Makefile.include
+
+Rules for Docker tests. Like tests/Makefile, this file is included
+directly by the top level Makefile, anything defined in this file will
+influence the entire build system.
 
 - po/Makefile
 
-- 
2.8.2

[Qemu-devel] [PATCH v7 02/15] rules.mak: Add "COMMA" constant

[Fam Zheng]

Using "," literal in $(call quiet-command, ...) arguments is awkward.
Add this constant to make it at least doable.

Signed-off-by: Fam Zheng 
---
 rules.mak | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/rules.mak b/rules.mak
index d1ff311..4a8f464 100644
--- a/rules.mak
+++ b/rules.mak
@@ -1,4 +1,6 @@
 
+COMMA := ,
+
 # Don't use implicit rules or variables
 # we have explicit rules for everything
 MAKEFLAGS += -rR
-- 
2.8.2

[Qemu-devel] [PATCH v7 07/15] docker: Add common.rc

[Fam Zheng]

"requires" checks the "FEATURE" environment for specified prerequisits,
and skip the execution of test if not found.

"build_qemu" is the central routine to compile QEMU for tests to call.

Reviewed-by: Alex Bennée 
Signed-off-by: Fam Zheng 
---
 tests/docker/common.rc | 31 +++
 1 file changed, 31 insertions(+)
 create mode 100755 tests/docker/common.rc

diff --git a/tests/docker/common.rc b/tests/docker/common.rc
new file mode 100755
index 000..74b89d6
--- /dev/null
+++ b/tests/docker/common.rc
@@ -0,0 +1,31 @@
+#!/bin/sh
+#
+# Common routines for docker test scripts.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng 
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+requires()
+{
+for c in $@; do
+if ! echo "$FEATURES" | grep -wq -e "$c"; then
+echo "Prerequisite '$c' not present, skip"
+exit 0
+fi
+done
+}
+
+build_qemu()
+{
+$QEMU_SRC/configure \
+--target-list="${TARGET_LIST}" \
+--prefix="$PWD/install" \
+"$@"
+make $MAKEFLAGS
+}
-- 
2.8.2

[Qemu-devel] [PATCH v7 11/15] docker: Add mingw test

[Fam Zheng]

Reviewed-by: Alex Bennée 
Signed-off-by: Fam Zheng 
---
 tests/docker/test-mingw | 34 ++
 1 file changed, 34 insertions(+)
 create mode 100755 tests/docker/test-mingw

diff --git a/tests/docker/test-mingw b/tests/docker/test-mingw
new file mode 100755
index 000..c03757a
--- /dev/null
+++ b/tests/docker/test-mingw
@@ -0,0 +1,34 @@
+#!/bin/bash -e
+#
+# Cross compile QEMU with mingw toolchain on Linux.
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng 
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+requires mingw dtc
+
+for prefix in x86_64-w64-mingw32- i686-w64-mingw32-; do
+TARGET_LIST=x86_64-softmmu,aarch64-softmmu \
+build_qemu --cross-prefix=$prefix \
+--enable-trace-backends=simple \
+--enable-debug \
+--enable-gnutls \
+--enable-nettle \
+--enable-curl \
+--enable-vnc \
+--enable-bzip2 \
+--enable-guest-agent \
+--with-sdlabi=1.2 \
+--with-gtkabi=2.0
+make clean
+
+done
+
-- 
2.8.2

[Qemu-devel] [PATCH v7 01/15] tests: Add utilities for docker testing

[Fam Zheng]

docker.py is added with a number of useful subcommands to manager docker
images and instances for QEMU docker testing. Subcommands are:

run: A wrapper of "docker run" (or "sudo -n docker run" if necessary),
which takes care of killing and removing the running container at
SIGINT.

clean: Tear down all the containers including inactive ones that are
started by docker_run.

build: Compare an image from given dockerfile and rebuild it if they're
different.

Signed-off-by: Fam Zheng 
---
 tests/docker/docker.py | 194 +
 1 file changed, 194 insertions(+)
 create mode 100755 tests/docker/docker.py

diff --git a/tests/docker/docker.py b/tests/docker/docker.py
new file mode 100755
index 000..0151362
--- /dev/null
+++ b/tests/docker/docker.py
@@ -0,0 +1,194 @@
+#!/usr/bin/env python2
+#
+# Docker controlling module
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng 
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+import os
+import sys
+import subprocess
+import json
+import hashlib
+import atexit
+import uuid
+import argparse
+import tempfile
+from shutil import copy
+
+def _text_checksum(text):
+"""Calculate a digest string unique to the text content"""
+return hashlib.sha1(text).hexdigest()
+
+def _guess_docker_command():
+""" Guess a working docker command or raise exception if not found"""
+commands = [["docker"], ["sudo", "-n", "docker"]]
+for cmd in commands:
+if subprocess.call(cmd + ["images"],
+   stdout=subprocess.PIPE,
+   stderr=subprocess.PIPE) == 0:
+return cmd
+commands_txt = "\n".join(["  " + " ".join(x) for x in commands])
+raise Exception("Cannot find working docker command. Tried:\n%s" % \
+commands_txt)
+
+class Docker(object):
+""" Running Docker commands """
+def __init__(self):
+self._command = _guess_docker_command()
+self._instances = []
+atexit.register(self._kill_instances)
+
+def _do(self, cmd, quiet=True, **kwargs):
+if quiet:
+kwargs["stdout"] = subprocess.PIPE
+return subprocess.call(self._command + cmd, **kwargs)
+
+def _do_kill_instances(self, only_known, only_active=True):
+cmd = ["ps", "-q"]
+if not only_active:
+cmd.append("-a")
+for i in self._output(cmd).split():
+resp = self._output(["inspect", i])
+labels = json.loads(resp)[0]["Config"]["Labels"]
+active = json.loads(resp)[0]["State"]["Running"]
+if not labels:
+continue
+instance_uuid = labels.get("com.qemu.instance.uuid", None)
+if not instance_uuid:
+continue
+if only_known and instance_uuid not in self._instances:
+continue
+print "Terminating", i
+if active:
+self._do(["kill", i])
+self._do(["rm", i])
+
+def clean(self):
+self._do_kill_instances(False, False)
+return 0
+
+def _kill_instances(self):
+return self._do_kill_instances(True)
+
+def _output(self, cmd, **kwargs):
+return subprocess.check_output(self._command + cmd,
+   stderr=subprocess.STDOUT,
+   **kwargs)
+
+def get_image_dockerfile_checksum(self, tag):
+resp = self._output(["inspect", tag])
+labels = json.loads(resp)[0]["Config"].get("Labels", {})
+return labels.get("com.qemu.dockerfile-checksum", "")
+
+def build_image(self, tag, dockerfile, df_path, quiet=True, argv=None):
+if argv == None:
+argv = []
+tmp_dir = tempfile.mkdtemp(prefix="docker_build")
+
+tmp_df = tempfile.NamedTemporaryFile(dir=tmp_dir, suffix=".docker")
+tmp_df.write(dockerfile)
+
+tmp_df.write("\n")
+tmp_df.write("LABEL com.qemu.dockerfile-checksum=%s" %
+ _text_checksum(dockerfile))
+tmp_df.flush()
+self._do(["build", "-t", tag, "-f", tmp_df.name] + argv + \
+ [tmp_dir],
+ quiet=quiet)
+
+def image_matches_dockerfile(self, tag, dockerfile):
+try:
+checksum = self.get_image_dockerfile_checksum(tag)
+except Exception:
+return False
+return checksum == _text_checksum(dockerfile)
+
+def run(self, cmd, keep, quiet):
+label = uuid.uuid1().hex
+if not keep:
+self._instances.append(label)
+ret = self._do(["run", "--label",
+"com.qemu.instance.uuid=" + label] + cmd,
+   quiet=quiet)
+if not keep:
+self._instances.remove(label)
+return ret
+
+class 

[Qemu-devel] [PATCH v7 04/15] Makefile: Rules for docker testing

[Fam Zheng]

This adds a group of make targets to run docker tests, all are available
in source tree without running ./configure.

The usage is shown with "make docker".

Besides the fixed ones, dynamic targets for building each image and
running each test in each image are generated automatically by make,
scanning $(SRC_PATH)/tests/docker/ files with specific patterns.

Alternative to manually list particular targets (docker-TEST@IMAGE)
set, you can control which tests/images to run by filtering variables,
TESTS= and IMAGES=, which are expressed in Makefile pattern syntax,
"foo% %bar ...". For example:

$ make docker-test IMAGES="ubuntu fedora"

Unfortunately, it's impossible to propagate "-j $JOBS" into make in
containers, however since each combination is made a first class target
in the top Makefile, "make -j$N docker-test" still parallels the tests
coarsely.

Still, $J is made a magic variable to let all make invocations in
containers to use -j$J.

Instead of providing a live version of the source tree to the docker
container we snapshot it with git-archive. This ensures the tree is in a
pristine state for whatever operations the container is going to run on
them.

Uncommitted changes known to files known by the git index will be
included in the snapshot if there are any.

Signed-off-by: Fam Zheng 
Signed-off-by: Alex Bennée 
---
 Makefile  |   4 +-
 tests/docker/Makefile.include | 122 ++
 2 files changed, 125 insertions(+), 1 deletion(-)
 create mode 100644 tests/docker/Makefile.include

diff --git a/Makefile b/Makefile
index 1c2a7bd..b8563db 100644
--- a/Makefile
+++ b/Makefile
@@ -6,7 +6,7 @@ BUILD_DIR=$(CURDIR)
 # Before including a proper config-host.mak, assume we are in the source tree
 SRC_PATH=.
 
-UNCHECKED_GOALS := %clean TAGS cscope ctags
+UNCHECKED_GOALS := %clean TAGS cscope ctags docker docker-%
 
 # All following code might depend on configuration variables
 ifneq ($(wildcard config-host.mak),)
@@ -644,3 +644,5 @@ endif
 # Include automatically generated dependency files
 # Dependencies in Makefile.objs files come from our recursive subdir rules
 -include $(wildcard *.d tests/*.d)
+
+include $(SRC_PATH)/tests/docker/Makefile.include
diff --git a/tests/docker/Makefile.include b/tests/docker/Makefile.include
new file mode 100644
index 000..39ed7f8
--- /dev/null
+++ b/tests/docker/Makefile.include
@@ -0,0 +1,122 @@
+# Makefile for Docker tests
+
+.PHONY: docker docker-test docker-clean docker-image docker-qemu-src
+
+DOCKER_SUFFIX := .docker
+DOCKER_FILES_DIR := $(SRC_PATH)/tests/docker/dockerfiles
+DOCKER_IMAGES := $(notdir $(basename $(wildcard $(DOCKER_FILES_DIR)/*.docker)))
+DOCKER_TARGETS := $(patsubst %,docker-image-%,$(DOCKER_IMAGES))
+# Use a global constant ccache directory to speed up repetitive builds
+DOCKER_CCACHE_DIR := /var/tmp/qemu-docker-ccache
+
+DOCKER_TESTS := $(notdir $(shell \
+   find $(SRC_PATH)/tests/docker/ -name 'test-*' -type f))
+
+DOCKER_TOOLS := travis
+
+TESTS ?= %
+IMAGES ?= %
+
+# Make archive from git repo $1 to tar.gz $2
+make-archive-maybe = $(if $(wildcard $1/*), \
+   $(call quiet-command, \
+   (cd $1; if git diff-index --quiet HEAD -- &>/dev/null; then \
+   git archive -1 HEAD --format=tar.gz -o $2; \
+   else \
+   git archive -1 $$(git stash create) --format=tar.gz -o 
$2; \
+   fi), \
+   "  ARCHIVE $(notdir $2)"))
+
+CUR_TIME := $(shell date +%Y-%m-%d-%H.%M.%S.)
+# Makes the definition constant after the first expansion
+DOCKER_SRC_COPY = $(eval DOCKER_SRC_COPY := 
/tmp/docker-src.$(CUR_TIME))$(DOCKER_SRC_COPY)
+
+$(DOCKER_SRC_COPY):
+   @mkdir $@
+   $(call make-archive-maybe, $(SRC_PATH), $@/qemu.tgz)
+   $(call make-archive-maybe, $(SRC_PATH)/dtc, $@/dtc.tgz)
+   $(call make-archive-maybe, $(SRC_PATH)/pixman, $@/pixman.tgz)
+   $(call quiet-command, cp $(SRC_PATH)/tests/docker/run $@/run, \
+   "  COPY RUNNER")
+
+docker-qemu-src: $(DOCKER_SRC_COPY)
+
+docker-image: ${DOCKER_TARGETS}
+
+# General rule for building docker images
+docker-image-%: $(DOCKER_FILES_DIR)/%.docker
+   $(call quiet-command,\
+   $(SRC_PATH)/tests/docker/docker.py build qemu:$* $< \
+   $(if $V,,--quiet) $(if $(NOCACHE),--no-cache),\
+   "  BUILD $*")
+
+# Expand all the pre-requistes for each docker image and test combination
+$(foreach i,$(DOCKER_IMAGES), \
+   $(foreach t,$(DOCKER_TESTS) $(DOCKER_TOOLS), \
+   $(eval .PHONY: docker-$t@$i) \
+   $(eval docker-$t@$i: docker-image-$i docker-run-$t@$i) \
+   ) \
+   $(foreach t,$(DOCKER_TESTS), \
+   $(eval docker-test: docker-$t@$i) \
+   ) \
+)
+
+docker:
+   @echo 'Build QEMU and run tests inside Docker containers'
+   @echo
+   @echo 'Available targets:'
+   @echo
+   @echo '

[Qemu-devel] [PATCH v7 06/15] docker: Add test runner

[Fam Zheng]

It's better to have a launcher for all tests, to make it easier to
initialize and manage the environment.

If "DEBUG=1"  a shell prompt will show up before the test runs.

Reviewed-by: Alex Bennée 
Signed-off-by: Fam Zheng 
---
 tests/docker/run | 58 
 1 file changed, 58 insertions(+)
 create mode 100755 tests/docker/run

diff --git a/tests/docker/run b/tests/docker/run
new file mode 100755
index 000..ec3d119
--- /dev/null
+++ b/tests/docker/run
@@ -0,0 +1,58 @@
+#!/bin/bash -e
+#
+# Docker test runner
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng 
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+# Prepare the environment
+. /etc/profile || true
+export PATH=/usr/lib/ccache:$PATH
+
+if test -n "$J"; then
+export MAKEFLAGS="$MAKEFLAGS -j$J"
+fi
+
+# We are in the container so the whole file system belong to us
+export TEST_DIR=/tmp/qemu-test
+mkdir -p $TEST_DIR/{src,build,install}
+
+# Extract the source tarballs
+tar -C $TEST_DIR/src -xzf qemu.tgz
+for p in dtc pixman; do
+if test -f $p.tgz; then
+tar -C $TEST_DIR/src/$p -xzf $p.tgz
+export FEATURES="$FEATURES $p"
+fi
+done
+
+export QEMU_SRC="$TEST_DIR/src"
+
+cd "$QEMU_SRC/tests/docker"
+
+CMD="$QEMU_SRC/tests/docker/$@"
+
+if test -n "$DEBUG"; then
+echo "* Prepared to run command:"
+echo "  $CMD"
+echo "* Hit Ctrl-D to continue, or type 'exit 1' to abort"
+echo
+$SHELL
+fi
+
+if "$CMD"; then
+exit 0
+elif test -n "$DEBUG"; then
+echo "* Command failed:"
+echo "  $CMD"
+echo "* Hit Ctrl-D to exit"
+echo
+# Force error after shell exits
+$SHELL && exit 1
+fi
-- 
2.8.2

[Qemu-devel] [PATCH v7 08/15] docker: Add quick test

[Fam Zheng]

Reviewed-by: Alex Bennée 
Signed-off-by: Fam Zheng 
---
 tests/docker/test-quick | 19 +++
 1 file changed, 19 insertions(+)
 create mode 100755 tests/docker/test-quick

diff --git a/tests/docker/test-quick b/tests/docker/test-quick
new file mode 100755
index 000..07cdc59
--- /dev/null
+++ b/tests/docker/test-quick
@@ -0,0 +1,19 @@
+#!/bin/bash -e
+#
+# Quick compiling test that everyone already does. But why not automate it?
+#
+# Copyright (c) 2016 Red Hat Inc.
+#
+# Authors:
+#  Fam Zheng 
+#
+# This work is licensed under the terms of the GNU GPL, version 2
+# or (at your option) any later version. See the COPYING file in
+# the top-level directory.
+
+. common.rc
+
+DEF_TARGET_LIST="$(echo {x86_64,aarch64}-softmmu)"
+TARGET_LIST=${TARGET_LIST:-$DEF_TARGET_LIST} \
+build_qemu
+make check $MAKEFLAGS
-- 
2.8.2

[Qemu-devel] [PATCH v7 00/15] tests: Introducing docker tests

[Fam Zheng]

v7: Patch 2: add temporary directory for docker build. [Alex]
Patch 3: always include rules.mak. [Paolo]
Patch 4: adopt Paolo's snippets for src copy;
 rename SRC_COPY -> DOCKER_SRC_COPY. [Paolo]


Fam Zheng (15):
  tests: Add utilities for docker testing
  rules.mak: Add "COMMA" constant
  Makefile: Always include rules.mak
  Makefile: Rules for docker testing
  docker: Add images
  docker: Add test runner
  docker: Add common.rc
  docker: Add quick test
  docker: Add full test
  docker: Add clang test
  docker: Add mingw test
  docker: Add travis tool
  docs: Add text for tests/docker in build-system.txt
  docker: Add EXTRA_CONFIGURE_OPTS
  MAINTAINERS: Add tests/docker

 MAINTAINERS |   7 ++
 Makefile|   7 +-
 docs/build-system.txt   |   5 +
 rules.mak   |   2 +
 tests/docker/Makefile.include   | 125 
 tests/docker/common.rc  |  32 ++
 tests/docker/docker.py  | 194 
 tests/docker/dockerfiles/centos6.docker |   6 +
 tests/docker/dockerfiles/fedora.docker  |   7 ++
 tests/docker/dockerfiles/ubuntu.docker  |  11 ++
 tests/docker/run|  58 ++
 tests/docker/test-clang |  26 +
 tests/docker/test-full  |  17 +++
 tests/docker/test-mingw |  34 ++
 tests/docker/test-quick |  19 
 tests/docker/travis |  21 
 tests/docker/travis.py  |  48 
 17 files changed, 617 insertions(+), 2 deletions(-)
 create mode 100644 tests/docker/Makefile.include
 create mode 100755 tests/docker/common.rc
 create mode 100755 tests/docker/docker.py
 create mode 100644 tests/docker/dockerfiles/centos6.docker
 create mode 100644 tests/docker/dockerfiles/fedora.docker
 create mode 100644 tests/docker/dockerfiles/ubuntu.docker
 create mode 100755 tests/docker/run
 create mode 100755 tests/docker/test-clang
 create mode 100755 tests/docker/test-full
 create mode 100755 tests/docker/test-mingw
 create mode 100755 tests/docker/test-quick
 create mode 100755 tests/docker/travis
 create mode 100755 tests/docker/travis.py

-- 
2.8.2

Re: [Qemu-devel] [PATCH v7 12/17] net_pkt: Extend packet abstraction as required by e1000e functionality

[Jason Wang]

On 2016年05月31日 15:20, Dmitry Fleytman wrote:

From: Dmitry Fleytman 

This patch extends the TX/RX packet abstractions with features that will
be used by the e1000e device implementation.

Changes are:

   1. Support iovec lists for RX buffers
   2. Deeper RX packets parsing
   3. Loopback option for TX packets
   4. Extended VLAN headers handling
   5. RSS processing for RX packets

Signed-off-by: Dmitry Fleytman 
Signed-off-by: Leonid Bloch 
---
  hw/net/net_rx_pkt.c| 473 +
  hw/net/net_rx_pkt.h| 193 +++-
  hw/net/net_tx_pkt.c| 204 +
  hw/net/net_tx_pkt.h|  60 ++-
  include/net/checksum.h |   4 +-
  include/net/eth.h  | 153 +++-
  net/checksum.c |   7 +-
  net/eth.c  | 410 +-
  trace-events   |  40 +
  9 files changed, 1336 insertions(+), 208 deletions(-)


[...]


  struct udp_hdr {
uint16_t uh_sport;   /* source port */
uint16_t uh_dport;   /* destination port */
@@ -169,19 +194,22 @@ struct tcp_hdr {
  #define PKT_GET_IP_HDR(p) \
  ((struct ip_header *)(((uint8_t *)(p)) + eth_get_l2_hdr_length(p)))
  #define IP_HDR_GET_LEN(p) \
-struct ip_header *)p)->ip_ver_len & 0x0F) << 2)
+struct ip_header *)(p))->ip_ver_len & 0x0F) << 2)
  #define PKT_GET_IP_HDR_LEN(p) \
  (IP_HDR_GET_LEN(PKT_GET_IP_HDR(p)))
  #define PKT_GET_IP6_HDR(p)\
  ((struct ip6_header *) (((uint8_t *)(p)) + eth_get_l2_hdr_length(p)))
  #define IP_HEADER_VERSION(ip) \
-((ip->ip_ver_len >> 4)&0xf)
+(((ip)->ip_ver_len >> 4) & 0xf)
+#define IP4_IS_FRAGMENT(ip) \
+((be16_to_cpu((ip)->ip_off) & (IP_OFFMASK | IP_MF)) != 0)
  
  #define ETH_P_IP  (0x0800)  /* Internet Protocol packet  */

  #define ETH_P_ARP (0x0806)  /* Address Resolution packet 
*/
  #define ETH_P_IPV6(0x86dd)
  #define ETH_P_VLAN(0x8100)
  #define ETH_P_DVLAN   (0x88a8)
+#define ETH_P_UNKNOWN (0x)
  #define VLAN_VID_MASK 0x0fff
  #define IP_HEADER_VERSION_4   (4)
  #define IP_HEADER_VERSION_6   (6)
@@ -258,15 +286,25 @@ get_eth_packet_type(const struct eth_header *ehdr)
  }
  
  static inline uint32_t

-eth_get_l2_hdr_length(const void *p)
+eth_get_l2_hdr_length(const struct iovec *iov, int iovcnt)
  {


Looks like this changes breaks the above PKT_GET_IP_HDR and 
PKT_GET_IP6_HDR. This will be a problem e.g ENET series depends on this.


A solution is keeping current eth_get_l2_hdr_length() and call it in a 
new helper e.g eth_get_l2_hdr_length_iov().



-uint16_t proto = be16_to_cpu(PKT_GET_ETH_HDR(p)->h_proto);
-struct vlan_header *hvlan = PKT_GET_VLAN_HDR(p);
+uint8_t p[sizeof(struct eth_header) + sizeof(struct vlan_header)];
+size_t copied = iov_to_buf(iov, iovcnt, 0, p, ARRAY_SIZE(p));
+uint16_t proto;
+struct vlan_header *hvlan;
+
+if (copied < ARRAY_SIZE(p)) {
+return copied;
+}
+
+proto = be16_to_cpu(PKT_GET_ETH_HDR(p)->h_proto);
+hvlan = PKT_GET_VLAN_HDR(p);
+
  switch (proto) {
  case ETH_P_VLAN:
  return sizeof(struct eth_header) + sizeof(struct vlan_header);
  case ETH_P_DVLAN:
-if (hvlan->h_proto == ETH_P_VLAN) {
+if (be16_to_cpu(hvlan->h_proto) == ETH_P_VLAN) {
  return sizeof(struct eth_header) + 2 * sizeof(struct vlan_header);
  } else {
  return sizeof(struct eth_header) + sizeof(struct vlan_header);
@@ -290,51 +328,67 @@ eth_get_pkt_tci(const void *p)
  }
  }
  


[...]

Re: [Qemu-devel] [PATCH] qcow2 resize with snapshot

[zhangzhiming]

hi, thanks for the review and the format of code changed.
have a good day!

zhangzhiming
zhangzhimin...@meituan.com

Signed-off-by: zhangzhiming 
---
 block.c|4 ++--
 block/qcow2-cluster.c  |4 ++--
 block/qcow2-snapshot.c |5 ++---
 block/qcow2.c  |4 ++--
 4 files changed, 8 insertions(+), 9 deletions(-)

diff --git a/block.c b/block.c
index 729f820..b6f2004 100644
--- a/block.c
+++ b/block.c
@@ -2643,9 +2643,9 @@ int bdrv_apply_snapshot(BlockDriverState *bs, const char 
*snapshot_id,
 if (ret < 0) {
 return ret;
 }
-bdrv_dirty_bitmap_truncate(bs); /* void return */
+bdrv_dirty_bitmap_truncate(bs);
 if (bs->blk) {
-blk_dev_resize_cb(bs->blk); /* void return too */
+blk_dev_resize_cb(bs->blk);
 }
 return ret;
 }
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index e4c5c05..f921fd8 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -37,14 +37,14 @@ int shrink_l1_table(BlockDriverState *bs, int64_t 
new_l1_size)
 int64_t old_l1_size = s->l1_size;
 s->l1_size = new_l1_size;
 int ret = qcow2_update_snapshot_refcount(bs, s->l1_table_offset,
- s->l1_size, 1);
+ s->l1_size, 1);
 if (ret < 0) {
 return ret;
 }
 
 s->l1_size = old_l1_size;
 ret = qcow2_update_snapshot_refcount(bs, s->l1_table_offset,
- s->l1_size, -1);
+ s->l1_size, -1);
 if (ret < 0) {
 return ret;
 }
diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c
index 9c77096..1ed0e18 100644
--- a/block/qcow2-snapshot.c
+++ b/block/qcow2-snapshot.c
@@ -562,12 +562,11 @@ int qcow2_snapshot_goto(BlockDriverState *bs, const char 
*snapshot_id)
  * Now update the in-memory L1 table to be in sync with the on-disk one. We
  * need to do this even if updating refcounts failed.
  */
-memset(s->l1_table, 0, s->l1_size*sizeof(uint64_t));
-for(i = 0;i < sn->l1_size; i++) {
+memset(s->l1_table, 0, s->l1_size * sizeof(uint64_t));
+for (i = 0; i < sn->l1_size; i++) {
 s->l1_table[i] = be64_to_cpu(sn_l1_table[i]);
 }
 
-
 if (ret < 0) {
 goto fail;
 }
diff --git a/block/qcow2.c b/block/qcow2.c
index 70ec890..58b53e1 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2503,14 +2503,14 @@ static int qcow2_truncate(BlockDriverState *bs, int64_t 
offset)
 
 bool v3_truncate = (s->qcow_version == 3);
 
-/* cannot proceed if image has snapshots and qcow_version is not 3*/
+/* cannot proceed if image has snapshots and qcow_version is not 3 */
 if (!v3_truncate && s->nb_snapshots) {
 error_report("Can't resize an image which has snapshots and "
  "qcow_version is not 3");
 return -ENOTSUP;
 }
 
-/* shrinking is supported from version 3*/
+/* shrinking is supported from version 3 */
 if (!v3_truncate && offset < bs->total_sectors * 512) {
 error_report("qcow2 doesn't support shrinking images yet while"
  " qcow_version is not 3");
-- 
1.7.1


> On Jun 1, 2016, at 12:50 AM, Eric Blake  wrote:
> 
> On 05/27/2016 02:14 AM, zhangzhiming wrote:
>> Hi, i modified my code for qcow2 resize, and delete some code related to 
>> qemu monitor.
>> 
>> and thanks for the review.
>> 
>> zhangzhiming
>> zhangzhimin...@meituan.com 
> 
> Still missing a Signed-off-by designation, so it can't be applied as-is.
> 
>> 
>> ---
>> block.c  |   19 +++
>> block/qcow2-cluster.c|   29 +
>> block/qcow2-snapshot.c   |   34 --
>> block/qcow2.c|   29 -
>> block/qcow2.h|1 +
>> include/block/snapshot.h |1 +
>> 6 files changed, 94 insertions(+), 19 deletions(-)
>> 
>> diff --git a/block.c b/block.c
>> index 18a497f..729f820 100644
>> --- a/block.c
>> +++ b/block.c
>> @@ -2631,6 +2631,25 @@ int bdrv_truncate(BlockDriverState *bs, int64_t 
>> offset)
>> return ret;
>> }
>> 
>> +int bdrv_apply_snapshot(BlockDriverState *bs, const char *snapshot_id,
>> +uint64_t snapshot_size)
>> +{
>> +int ret = bdrv_snapshot_goto(bs, snapshot_id);
>> +if (ret < 0) {
>> +return ret;
>> +}
>> +
>> +ret = refresh_total_sectors(bs, snapshot_size >> BDRV_SECTOR_BITS);
>> +if (ret < 0) {
>> +return ret;
>> +}
>> +bdrv_dirty_bitmap_truncate(bs); /* void return */
>> +if (bs->blk) {
>> +blk_dev_resize_cb(bs->blk); /* void return too */
> 
> The comments don't add anything here.
> 
>> +}
>> +return ret;
>> +}
>> +
>> /**
>>  * Length of a allocated file in bytes. Sparse files are counted by actual
>>  * 

Re: [Qemu-devel] [PATCH v4 2/6] Migration: Split out ram part of qmp_query_migrate

[Amit Shah]

On (Tue) 31 May 2016 [17:10:48], Dr. David Alan Gilbert wrote:
> * Amit Shah (amit.s...@redhat.com) wrote:
> > On (Tue) 17 May 2016 [13:55:51], Dr. David Alan Gilbert (git) wrote:
> > > From: "Dr. David Alan Gilbert" 
> > > 
> > > The RAM section of qmp_query_migrate is reasonably complex
> > > and repeated 3 times.  Split it out into a helper.
> > > 
> > > Signed-off-by: Dr. David Alan Gilbert 
> > > Reviewed-by: Eric Blake 
> > > Reviwed-by: Denis V. Lunev 
> > > ---
> > >  migration/migration.c | 57 
> > > ---
> > >  1 file changed, 22 insertions(+), 35 deletions(-)
> > > 
> > > diff --git a/migration/migration.c b/migration/migration.c
> > > index 991313a..bfb326d 100644
> > > --- a/migration/migration.c
> > > +++ b/migration/migration.c
> > > @@ -561,6 +561,25 @@ static void get_xbzrle_cache_stats(MigrationInfo 
> > > *info)
> > >  }
> > >  }
> > >  
> > > +static void populate_ram_info(MigrationInfo *info, MigrationState *s)
> > > +{
> > > +info->has_ram = true;
> > > +info->ram = g_malloc0(sizeof(*info->ram));
> > > +info->ram->transferred = ram_bytes_transferred();
> > > +info->ram->total = ram_bytes_total();
> > > +info->ram->duplicate = dup_mig_pages_transferred();
> > > +info->ram->skipped = skipped_mig_pages_transferred();
> > > +info->ram->normal = norm_mig_pages_transferred();
> > > +info->ram->normal_bytes = norm_mig_bytes_transferred();
> > > +info->ram->mbps = s->mbps;
> > > +info->ram->dirty_sync_count = s->dirty_sync_count;
> > > +
> > > +if (s->state != MIGRATION_STATUS_COMPLETED) {
> > > +info->ram->remaining = ram_bytes_remaining();
> > > +info->ram->dirty_pages_rate = s->dirty_pages_rate;
> > 
> > In the 'else' case, shouldn't these be set to 0?
> 
> There's no need to, see that the info->ram is allocated a few
> lines above using a g_malloc0 and so it's already all zero.

Ah, yeah :(

One of the side-effects of using gmalloc0 is valgrind (or compiler
warnings) won't be helpful when using stuff without writing to them
first.  That's why I prefer explicitly setting fields.

Anyway, I'll pick this series up, thanks!

Amit

Re: [Qemu-devel] [PATCH qemu v16 19/19] spapr_pci/spapr_pci_vfio: Support Dynamic DMA Windows (DDW)

[Bharata B Rao]

On Fri, May 27, 2016 at 11:19 AM, Bharata B Rao  wrote:
> On Fri, May 27, 2016 at 10:14 AM, David Gibson
>  wrote:
>> On Tue, May 17, 2016 at 11:02:48AM +0530, Bharata B Rao wrote:
>>> On Mon, May 16, 2016 at 11:55 AM, Alexey Kardashevskiy  
>>> wrote:
>>> > On 05/13/2016 06:41 PM, Bharata B Rao wrote:
>>> >>
>>> >> On Wed, May 4, 2016 at 12:22 PM, Alexey Kardashevskiy 
>>> >> wrote:
>>> >
>>> >
>>> >>
>>> >>> +
>>> >>> +avail = SPAPR_PCI_DMA_MAX_WINDOWS -
>>> >>> spapr_phb_get_active_win_num(sphb);
>>> >>> +
>>> >>> +rtas_st(rets, 0, RTAS_OUT_SUCCESS);
>>> >>> +rtas_st(rets, 1, avail);
>>> >>> +rtas_st(rets, 2, max_window_size);
>>> >>> +rtas_st(rets, 3, pgmask);
>>> >>> +rtas_st(rets, 4, 0); /* DMA migration mask, not supported */
>>> >>> +
>>> >>> +trace_spapr_iommu_ddw_query(buid, addr, avail, max_window_size,
>>> >>> pgmask);
>>> >>> +return;
>>> >>> +
>>> >>> +param_error_exit:
>>> >>> +rtas_st(rets, 0, RTAS_OUT_PARAM_ERROR);
>>> >>> +}
>>> >>> +
>>> >>> +static void rtas_ibm_create_pe_dma_window(PowerPCCPU *cpu,
>>> >>> +  sPAPRMachineState *spapr,
>>> >>> +  uint32_t token, uint32_t
>>> >>> nargs,
>>> >>> +  target_ulong args,
>>> >>> +  uint32_t nret, target_ulong
>>> >>> rets)
>>> >>> +{
>>> >>> +sPAPRPHBState *sphb;
>>> >>> +sPAPRTCETable *tcet = NULL;
>>> >>> +uint32_t addr, page_shift, window_shift, liobn;
>>> >>> +uint64_t buid;
>>> >>> +
>>> >>> +if ((nargs != 5) || (nret != 4)) {
>>> >>> +goto param_error_exit;
>>> >>> +}
>>> >>> +
>>> >>> +buid = ((uint64_t)rtas_ld(args, 1) << 32) | rtas_ld(args, 2);
>>> >>> +addr = rtas_ld(args, 0);
>>> >>> +sphb = spapr_pci_find_phb(spapr, buid);
>>> >>> +if (!sphb || !sphb->ddw_enabled) {
>>> >>> +goto param_error_exit;
>>> >>> +}
>>> >>> +
>>> >>> +page_shift = rtas_ld(args, 3);
>>> >>> +window_shift = rtas_ld(args, 4);
>>> >>
>>> >>
>>> >> Kernel has a bug due to which wrong window_shift gets returned here. I
>>> >> have posted possible fix here:
>>> >> https://patchwork.ozlabs.org/patch/621497/
>>> >>
>>> >> I have tried to work around this issue in QEMU too
>>> >> https://lists.nongnu.org/archive/html/qemu-ppc/2016-04/msg00226.html
>>> >>
>>> >> But the above work around involves changing the memory representation
>>> >> in DT.
>>> >
>>> >
>>> > What is wrong with this workaround?
>>>
>>> The above workaround will result in different representations for
>>> memory in DT before and after the workaround.
>>>
>>> Currently for -m 2G, -numa node,nodeid=0,mem=1G -numa
>>> node,nodeid=1,mem=0.5G, we will have the following nodes in DT:
>>>
>>> memory@0
>>> memory@4000
>>> ibm,dynamic-reconfiguration-memory
>>>
>>> ibm,dynamic-memory will have only DR LMBs:
>>>
>>> [root@localhost ibm,dynamic-reconfiguration-memory]# hexdump 
>>> ibm,dynamic-memory
>>> 000  000a   8000  8000 0008
>>> 010        
>>> 020 9000  8000 0009    
>>> 030     a000  8000 000a
>>> 040        
>>> 050 b000  8000 000b    
>>> 060     c000  8000 000c
>>> 070        
>>> 080 d000  8000 000d    
>>> 090     e000  8000 000e
>>> 0a0        
>>> 0b0 f000  8000 000f    
>>> 0c0    0001   8000 0010
>>> 0d0        0001
>>> 0e0 1000  8000 0011    
>>> 0f0  
>>>
>>> The memory region looks like this:
>>>
>>> memory-region: system
>>>   - (prio 0, RW): system
>>> -5fff (prio 0, RW): ppc_spapr.ram
>>> 8000-00011fff (prio 0, RW): hotplug-memory
>>>
>>> After this workaround, all this will change like below:
>>>
>>> memory@0
>>> ibm,dynamic-reconfiguration-memory
>>>
>>> All LMBs in ibm,dynamic-memory:
>>>
>>> [root@localhost ibm,dynamic-reconfiguration-memory]# hexdump 
>>> ibm,dynamic-memory
>>>
>>> 000  0010     8000 
>>> 010      0080  
>>> 020 1000  8000 0001    
>>> 030  0080   2000  8000 0002
>>> 040      0080  
>>> 050 3000  8000 0003    
>>> 060  0080   4000  8000 0004
>>> 070    0001  0008  
>>> 080 5000  8000 0005    0001
>>> 090  0008   6000  8000 0006
>>> 0a0       

Re: [Qemu-devel] [PATCH 2/2] Makefile: Derive "PKGVERSION" from "git describe" by default

[Fam Zheng]

On Tue, 05/31 21:05, Eric Blake wrote:
> On 05/31/2016 07:31 PM, Fam Zheng wrote:
> > Currently, if not specified in "./configure", QEMU_PKGVERSION will be
> > empty. Write a rule in Makefile to generate a value from "git describe"
> > combined with a possible git tree cleanness suffix, and write into a new
> > header.
> > 
> > $ cat qemu-version.h
> > #define QEMU_PKGVERSION "-v2.6.0-557-gd6550e9-unclean"
> > 
> > Include the header in .c files where the macro is referenced. It's not
> > necessary to include it in all files, otherwise each time the content of
> > the file changes, all sources have to be recompiled.
> > 
> > Signed-off-by: Fam Zheng 
> > ---
> 
> > +qemu-version.h: FORCE
> > +   $(call quiet-command, \
> > +   (cd $(SRC_PATH); \
> 
> Can CDPATH interfere with this one?

I think SRC_PATH is either an absolute path, or is ".". Both cases are fine.

> 
> > +   echo -n '#define QEMU_PKGVERSION '; \
> 
> 'echo -n' is non-portable, use printf instead.

Will fix.

> 
> > +   if test -n "$(PKGVERSION)"; then \
> > +   echo '"$(PKGVERSION)"'; \
> > +   else \
> > +   echo -n '"-'; \
> 
> And again
> 
> > +   if ! git status &>/dev/null; then \
> > +   echo -n "no-git"; \
> 
> and again
> 
> > +   else \
> > +   git describe 2>/dev/null | tr -d '\n'; \
> > +   if ! git diff-index --quiet HEAD &>/dev/null; 
> > then \
> > +   echo -n '-unclean'; \
> 
> and again
> 
> Why -unclean instead of -dirty?

The non-native speaker can fix that.

> 
> > +   fi \
> > +   fi; \
> > +   echo '"'; \
> > +   fi) > $@.tmp)
> > +   $(call quiet-command, cmp --quiet $@ $@.tmp || mv $@.tmp $@)
> > +
> 
> 
> -- 
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org
> 

Thanks,

Fam

Re: [Qemu-devel] [PULL 03/31] hw/char: QOM'ify escc.c

[xiaoqiang zhao]

> 在 2016年6月1日，06:13，Mark Cave-Ayland  写道：
> 
>> On 27/05/16 11:06, Paolo Bonzini wrote:
>> 
>> From: xiaoqiang zhao 
>> 
>> * Drop the old SysBus init function and use instance_init
>> * Call qemu_chr_add_handlers in the realize callback
>> 
>> Signed-off-by: xiaoqiang zhao 
>> Message-Id: <1464158344-12266-2-git-send-email-zxq_yx_...@163.com>
>> Signed-off-by: Paolo Bonzini 
>> ---
>> hw/char/escc.c | 30 +++---
>> 1 file changed, 19 insertions(+), 11 deletions(-)
>> 
>> diff --git a/hw/char/escc.c b/hw/char/escc.c
>> index 7bf09a0..8e6a7df 100644
>> --- a/hw/char/escc.c
>> +++ b/hw/char/escc.c
>> @@ -983,9 +983,10 @@ void slavio_serial_ms_kbd_init(hwaddr base, qemu_irq 
>> irq,
>> sysbus_mmio_map(s, 0, base);
>> }
>> 
>> -static int escc_init1(SysBusDevice *dev)
>> +static void escc_init1(Object *obj)
>> {
>> -ESCCState *s = ESCC(dev);
>> +ESCCState *s = ESCC(obj);
>> +SysBusDevice *dev = SYS_BUS_DEVICE(obj);
>> unsigned int i;
>> 
>> s->chn[0].disabled = s->disabled;
>> @@ -994,17 +995,26 @@ static int escc_init1(SysBusDevice *dev)
>> sysbus_init_irq(dev, >chn[i].irq);
>> s->chn[i].chn = 1 - i;
>> s->chn[i].clock = s->frequency / 2;
>> -if (s->chn[i].chr) {
>> -qemu_chr_add_handlers(s->chn[i].chr, serial_can_receive,
>> -  serial_receive1, serial_event, 
>> >chn[i]);
>> -}
>> }
>> s->chn[0].otherchn = >chn[1];
>> s->chn[1].otherchn = >chn[0];
>> 
>> -memory_region_init_io(>mmio, OBJECT(s), _mem_ops, s, "escc",
>> +memory_region_init_io(>mmio, obj, _mem_ops, s, "escc",
>>   ESCC_SIZE << s->it_shift);
>> sysbus_init_mmio(dev, >mmio);
>> +}
>> +
>> +static void escc_realize(DeviceState *dev, Error **errp)
>> +{
>> +ESCCState *s = ESCC(dev);
>> +unsigned int i;
>> +
>> +for (i = 0; i < 2; i++) {
>> +if (s->chn[i].chr) {
>> +qemu_chr_add_handlers(s->chn[i].chr, serial_can_receive,
>> +  serial_receive1, serial_event, 
>> >chn[i]);
>> +}
>> +}
>> 
>> if (s->chn[0].type == mouse) {
>> qemu_add_mouse_event_handler(sunmouse_event, >chn[0], 0,
>> @@ -1014,8 +1024,6 @@ static int escc_init1(SysBusDevice *dev)
>> s->chn[1].hs = qemu_input_handler_register((DeviceState 
>> *)(>chn[1]),
>>_handler);
>> }
>> -
>> -return 0;
>> }
>> 
>> static Property escc_properties[] = {
>> @@ -1032,10 +1040,9 @@ static Property escc_properties[] = {
>> static void escc_class_init(ObjectClass *klass, void *data)
>> {
>> DeviceClass *dc = DEVICE_CLASS(klass);
>> -SysBusDeviceClass *k = SYS_BUS_DEVICE_CLASS(klass);
>> 
>> -k->init = escc_init1;
>> dc->reset = escc_reset;
>> +dc->realize = escc_realize;
>> dc->vmsd = _escc;
>> dc->props = escc_properties;
>> set_bit(DEVICE_CATEGORY_INPUT, dc->categories);
>> @@ -1045,6 +1052,7 @@ static const TypeInfo escc_info = {
>> .name  = TYPE_ESCC,
>> .parent= TYPE_SYS_BUS_DEVICE,
>> .instance_size = sizeof(ESCCState),
>> +.instance_init = escc_init1,
>> .class_init= escc_class_init,
>> };
> 
> Unfortunately this patch causes OpenBIOS to freeze on startup under
> qemu-system-ppc (presumably as there is a problem accessing the serial
> port). You can reproduce this by starting qemu-system-ppc with no
> parameters against the commits below:
> 
> Bad : e7c9136977cb99c6eb52c9139f7b8d8b5fa87db9
> Good: b138e654a0525f009e7e7c96fc67d74baf3e011b
> 
> Note that you'll currently need to use the above two hashes to reproduce
> the issue against git master as another regression has just crept in.
> 
> 
> ATB,
> 
> Mark.
> 
> 

Mark：
   Sorry for the inconvenience. This problem is due to the incorrect property 
value in the realize stage. I have fix this and the test is passed.

Paolo:  Do i need to send a new version or just this one?

Re: [Qemu-devel] [PATCH 2/2] Makefile: Derive "PKGVERSION" from "git describe" by default

[Eric Blake]

On 05/31/2016 07:31 PM, Fam Zheng wrote:
> Currently, if not specified in "./configure", QEMU_PKGVERSION will be
> empty. Write a rule in Makefile to generate a value from "git describe"
> combined with a possible git tree cleanness suffix, and write into a new
> header.
> 
> $ cat qemu-version.h
> #define QEMU_PKGVERSION "-v2.6.0-557-gd6550e9-unclean"
> 
> Include the header in .c files where the macro is referenced. It's not
> necessary to include it in all files, otherwise each time the content of
> the file changes, all sources have to be recompiled.
> 
> Signed-off-by: Fam Zheng 
> ---

> +qemu-version.h: FORCE
> + $(call quiet-command, \
> + (cd $(SRC_PATH); \

Can CDPATH interfere with this one?

> + echo -n '#define QEMU_PKGVERSION '; \

'echo -n' is non-portable, use printf instead.

> + if test -n "$(PKGVERSION)"; then \
> + echo '"$(PKGVERSION)"'; \
> + else \
> + echo -n '"-'; \

And again

> + if ! git status &>/dev/null; then \
> + echo -n "no-git"; \

and again

> + else \
> + git describe 2>/dev/null | tr -d '\n'; \
> + if ! git diff-index --quiet HEAD &>/dev/null; 
> then \
> + echo -n '-unclean'; \

and again

Why -unclean instead of -dirty?

> + fi \
> + fi; \
> + echo '"'; \
> + fi) > $@.tmp)
> + $(call quiet-command, cmp --quiet $@ $@.tmp || mv $@.tmp $@)
> +


-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v6 00/11] Add param Error ** for msi_init()

[Cao jin]

ping?

On 05/24/2016 12:04 PM, Cao jin wrote:

v6 changelog:
1. make "assert ENOSPC" the 1st one in the series, and remove ENOSPC line
of comments of msi_init(). also fix to other minor comments.
2. Because semantics changes, add hint message for devices who have msi/msix
property, to help old machine user to know what should do
3. update pci_bridge_dev hehaviour, because it has non-msi variant, but it can
never fall back to INTx before the patch. make it behaviour like others
4. mptsas: forget to assign s->msi_in_use before, now make it.

About test: Only compiled every patch.

cc: Gerd Hoffmann 
cc: John Snow 
cc: Dmitry Fleytman 
cc: Jason Wang 
cc: Michael S. Tsirkin 
cc: Hannes Reinecke 
cc: Paolo Bonzini 
cc: Alex Williamson 
cc: Markus Armbruster 
cc: Marcel Apfelbaum 

Cao jin (11):
   pci core: assert ENOSPC when add capability
   fix some coding style problems
   change pvscsi_init_msi() type to void
   megasas: Fix
   mptsas: change .realize function name
   usb xhci: change msi/msix property type
   intel-hda: change msi property type
   mptsas: change msi property type
   megasas: change msi/msix property type
   pci bridge dev: change msi property type
   pci: Convert msi_init() to Error and fix callers to check it

  hw/audio/intel-hda.c   | 23 ++
  hw/ide/ich.c   | 17 +-
  hw/net/vmxnet3.c   | 44 ++
  hw/pci-bridge/ioh3420.c| 12 --
  hw/pci-bridge/pci_bridge_dev.c | 31 +---
  hw/pci-bridge/xio3130_downstream.c | 11 +++--
  hw/pci-bridge/xio3130_upstream.c   |  8 ++-
  hw/pci/msi.c   | 25 ++--
  hw/pci/pci.c   |  6 ++---
  hw/scsi/megasas.c  | 48 +-
  hw/scsi/mptsas.c   | 32 ++---
  hw/scsi/mptsas.h   |  3 ++-
  hw/scsi/vmw_pvscsi.c   | 10 
  hw/usb/hcd-xhci.c  | 33 +++---
  hw/vfio/pci.c  |  7 --
  include/hw/pci/msi.h   |  3 ++-
  16 files changed, 209 insertions(+), 104 deletions(-)



--
Yours Sincerely,

Cao jin

Re: [Qemu-devel] [RFC PATCH 1/2] tests/docker/docker.py: support --qemu option

[Fam Zheng]

On Tue, 05/31 16:23, Alex Bennée wrote:
> >> +def _find_user_binary(binary_name):
> >> +""" Find a binary in the QEMU source tree. Used for finding 
> >> qemu-$arch."""
> >> +top = os.path.abspath("%s/../../.." % sys.argv[0])
> >
> > What if this is an out of tree build?
> 
> Yes I kinda avoided the complexity here. Do we have a programatic way of
> finding this out or should we just assume we get based a resolvable path?

As said below, let's assume the user provides an absolute path or a relative
path against the working directory, so we don't need to worry about path
guessing. The script caller should have more information.

Fam

Re: [Qemu-devel] [PATCH RFC v2 kernel] balloon: speed up inflating/deflating process

[Li, Liang Z]

Hi MST,

About the size of page bitmap, I have test the performance of filling the 
balloon to 15GB with a 16GB RAM VM.

===
32K Byte (cover 1GB of RAM)

Time spends on inflating: 2031ms
-
64K Byte (cover 2GB of RAM)

Time spends on inflating: 1507ms

512K Byte (cover 16GB of RAM)

Time spends on inflating: 1237ms


It shows the performance is better if using a larger page bitmap, should we use 
a 64K/ 128KB page bitmap
for a better  balance between the performance and the memory consumption?

BTW, about the VIRTIO_BALLOON_F_DEFLATE_ON_OOM feature, I found the 
configurable  'oom_pages' 
will be limited by the ARRAY_SIZE(vb->pfns), which means only 1MB of RAM will 
be recycled each time even
the users want more, is that too conservative? 

Liang


> -Original Message-
> From: Li, Liang Z
> Sent: Friday, May 27, 2016 6:34 PM
> To: linux-ker...@vger.kernel.org
> Cc: k...@vger.kernel.org; qemu-devel@nongnu.org; Li, Liang Z; Michael S.
> Tsirkin; Paolo Bonzini; Cornelia Huck; Amit Shah
> Subject: [PATCH RFC v2 kernel] balloon: speed up inflating/deflating process
> 
> The implementation of the current virtio-balloon is not very efficient, Bellow
> is test result of time spends on inflating the balloon to 3GB of a 4GB idle
> guest:
> 
> a. allocating pages (6.5%, 103ms)
> b. sending PFNs to host (68.3%, 787ms)
> c. address translation (6.1%, 96ms)
> d. madvise (19%, 300ms)
> 
> It takes about 1577ms for the whole inflating process to complete. The test
> shows that the bottle neck is the stage b and stage d.
> 
> If using a bitmap to send the page info instead of the PFNs, we can reduce
> the overhead spends on stage b quite a lot. Furthermore, it's possible to do
> the address translation and do the madvise with a bulk of pages, instead of
> the current page per page way, so the overhead of stage c and stage d can
> also be reduced a lot.
> 
> This patch is the kernel side implementation which is intended to speed up
> the inflating & deflating process by adding a new feature to the 
> virtio-balloon
> device. And now, inflating the balloon to 3GB of a 4GB idle guest only takes
> 200ms, it's about 8 times as fast as before.
> 
> TODO: optimize stage a by allocating/freeing a chunk of pages instead of a
> single page at a time.
> 
> v2 change:
> 1. Use a small page bitmap instead of a large one.
> 2. Address some of comments of v1.
> 
> Signed-off-by: Liang Li 
> Suggested-by: Michael S. Tsirkin 
> Cc: Michael S. Tsirkin 
> Cc: Paolo Bonzini 
> Cc: Cornelia Huck 
> Cc: Amit Shah 
> ---
>  drivers/virtio/virtio_balloon.c | 207
> ++--
>  include/uapi/linux/virtio_balloon.h |   1 +
>  2 files changed, 200 insertions(+), 8 deletions(-)
> 
> diff --git a/drivers/virtio/virtio_balloon.c b/drivers/virtio/virtio_balloon.c
> index 476c0e3..823b4e4 100644
> --- a/drivers/virtio/virtio_balloon.c
> +++ b/drivers/virtio/virtio_balloon.c
> @@ -40,11 +40,19 @@
>  #define VIRTIO_BALLOON_ARRAY_PFNS_MAX 256  #define
> OOM_VBALLOON_DEFAULT_PAGES 256  #define
> VIRTBALLOON_OOM_NOTIFY_PRIORITY 80
> +#define VIRTIO_BALLOON_PFNS_LIMIT ((2 * (1ULL << 30)) >> PAGE_SHIFT)
> /*
> +2GB */
> 
>  static int oom_pages = OOM_VBALLOON_DEFAULT_PAGES;
> module_param(oom_pages, int, S_IRUSR | S_IWUSR);
> MODULE_PARM_DESC(oom_pages, "pages to free on OOM");
> 
> +struct balloon_bmap_hdr {
> + __virtio32 type;
> + __virtio32 page_shift;
> + __virtio64 start_pfn;
> + __virtio64 bmap_len;
> +};
> +
>  struct virtio_balloon {
>   struct virtio_device *vdev;
>   struct virtqueue *inflate_vq, *deflate_vq, *stats_vq; @@ -62,6
> +70,13 @@ struct virtio_balloon {
> 
>   /* Number of balloon pages we've told the Host we're not using. */
>   unsigned int num_pages;
> + /* Bitmap and length used to tell the host the pages */
> + unsigned long *page_bitmap;
> + unsigned long bmap_len;
> + /* Used to record the processed pfn range */
> + unsigned long min_pfn, max_pfn, start_pfn, end_pfn;
> + /* Used for sending page bitmap and header */
> + struct scatterlist sg[2];
>   /*
>* The pages we've told the Host we're not using are enqueued
>* at vb_dev_info->pages list.
> @@ -111,15 +126,39 @@ static void balloon_ack(struct virtqueue *vq)
>   wake_up(>acked);
>  }
> 
> +static inline void init_pfn_range(struct virtio_balloon *vb) {
> + vb->min_pfn = (1UL << 48);
> + vb->max_pfn = 0;
> +}
> +
>  static void tell_host(struct virtio_balloon *vb, struct virtqueue *vq)  {
> - struct scatterlist sg;
>   unsigned int len;
> 
> - sg_init_one(, vb->pfns, sizeof(vb->pfns[0]) * vb->num_pfns);
> + if (virtio_has_feature(vb->vdev,
> 

Re: [Qemu-devel] [PATCH] virtio: move bi-endian target support to a single location

[David Gibson]

On Tue, May 31, 2016 at 03:15:21PM +0200, Paolo Bonzini wrote:
> 
> 
> On 31/05/2016 15:10, Greg Kurz wrote:
> >>> > > +#if defined(TARGET_PPC64) || defined(TARGET_ARM)
> >>> > > +#define LEGACY_VIRTIO_IS_BIENDIAN 1
> >>> > > +#endif  
> >> > 
> >> > These will only be correct if something else includes cpu.h.  Instead of
> > Unless I missed something, the TARGET_* macros come from the generated
> > config-target.h header, which is in turn included by qemu/osdep.h and
> > thus included by most of the code.
> 
> You're right.  Problems _could_ happen if virtio-access.h is included in
> a file compiled without -DNEED_CPU_H (i.e. with common-obj-y instead of
> obj-y) but include/exec/poison.h should take care of that.
> 
> >> > defining this, you should add
> >> > 
> >> > #include "cpu.h"
> >> > 
> >> > at the top of include/hw/virtio-access.h and leave the definitions in
> >> > target-*/cpu.h.
> >> > 
> > All this bi-endian stuff is really an old-virtio-only thing... it is
> > only to be used by virtio_access_is_big_endian(). The fact that it
> > broke silently with your cleanup series is yet another proof that
> > this workaround is fragile.
> 
> It is not fragile actually.  cpu.h doesn't exist in common-obj-y, so the
> TARGET_IS_BIENDIAN define can be safely taken from cpu.h.
> 
> Anyway because of poison.h your solution isn't fragile either, so
> 
> Reviewed-by: Paolo Bonzini 

Should I take this through my tree?

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH qemu v16 04/19] vmstate: Define VARRAY with VMS_ALLOC

[Alexey Kardashevskiy]

On 27/05/16 17:54, Alexey Kardashevskiy wrote:
> On 04/05/16 16:52, Alexey Kardashevskiy wrote:
>> This allows dynamic allocation for migrating arrays.
>>
>> Already existing VMSTATE_VARRAY_UINT32 requires an array to be
>> pre-allocated, however there are cases when the size is not known in
>> advance and there is no real need to enforce it.
>>
>> This defines another variant of VMSTATE_VARRAY_UINT32 with WMS_ALLOC
>> flag which tells the receiving side to allocate memory for the array
>> before receiving the data.
>>
>> The first user of it is a dynamic DMA window which existence and size
>> are totally dynamic.
>>
>> Signed-off-by: Alexey Kardashevskiy 
>> Reviewed-by: David Gibson 
>> Reviewed-by: Thomas Huth 
> 
> 
> In what tree is this going to go? pseries? Or migration?


Anyone?


> 
> 
> 
>> ---
>>  include/migration/vmstate.h | 10 ++
>>  1 file changed, 10 insertions(+)
>>
>> diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
>> index 84ee355..1622638 100644
>> --- a/include/migration/vmstate.h
>> +++ b/include/migration/vmstate.h
>> @@ -386,6 +386,16 @@ extern const VMStateInfo vmstate_info_bitmap;
>>  .offset = vmstate_offset_pointer(_state, _field, _type), \
>>  }
>>  
>> +#define VMSTATE_VARRAY_UINT32_ALLOC(_field, _state, _field_num, _version, 
>> _info, _type) {\
>> +.name   = (stringify(_field)),   \
>> +.version_id = (_version),\
>> +.num_offset = vmstate_offset_value(_state, _field_num, uint32_t),\
>> +.info   = &(_info),  \
>> +.size   = sizeof(_type), \
>> +.flags  = VMS_VARRAY_UINT32|VMS_POINTER|VMS_ALLOC,   \
>> +.offset = vmstate_offset_pointer(_state, _field, _type), \
>> +}
>> +
>>  #define VMSTATE_VARRAY_UINT16_UNSAFE(_field, _state, _field_num, _version, 
>> _info, _type) {\
>>  .name   = (stringify(_field)),   \
>>  .version_id = (_version),\
>>
> 
> 


-- 
Alexey

Re: [Qemu-devel] [PATCH v6 04/15] Makefile: Rules for docker testing

[Fam Zheng]

On Tue, 05/31 14:48, Paolo Bonzini wrote:
> 
> 
> On 31/05/2016 14:40, Fam Zheng wrote:
> > > I'm still a bit undecided about the pollution introduced by
> > > tests/docker/Makefile.include, but I guess that's okay.
> > 
> > I think it's also okay to switch to "make -C tests/docker" for docker 
> > targets
> > (so "make docker" becomes "make -C tests/docker help"), this way the top
> > Makefile is not touched.
> 
> That would be a bit harder to discover.  Go ahead with your current
> solution.  Perhaps rename SRC_COPY to DOCKER_SRC_COPY?

OK, good idea.

Fam

[Qemu-devel] [PATCH] tests: Rename tests/Makefile to tests/Makefile.include

[Fam Zheng]

The file is only included from the top Makefile. Rename it to reflect
this more obviously.

Signed-off-by: Fam Zheng 
---
 Makefile | 2 +-
 tests/{Makefile => Makefile.include} | 0
 2 files changed, 1 insertion(+), 1 deletion(-)
 rename tests/{Makefile => Makefile.include} (100%)

diff --git a/Makefile b/Makefile
index 3a3c5dc..0cd111b 100644
--- a/Makefile
+++ b/Makefile
@@ -161,7 +161,7 @@ dummy := $(call unnest-vars,, \
 common-obj-m)
 
 ifneq ($(wildcard config-host.mak),)
-include $(SRC_PATH)/tests/Makefile
+include $(SRC_PATH)/tests/Makefile.include
 endif
 
 all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all modules
diff --git a/tests/Makefile b/tests/Makefile.include
similarity index 100%
rename from tests/Makefile
rename to tests/Makefile.include
-- 
2.8.2

Re: [Qemu-devel] [PATCH 0/6] *** Vhost-pci RFC ***

[Wang, Wei W]

On Wed 6/1/2016 2:21 AM, Eric Blake wrote:
> On 05/28/2016 05:36 PM, Wei Wang wrote:
> > This RFC proposes a design of vhost-pci, which is a new virtio device type.
> > The vhost-pci device is used for inter-VM communication. Please read
> > the RFC patches for details.
> >
> >
> > Wei Wang (6):
> >   Vhost-pci RFC: Introduction
> >   Vhost-pci RFC: Modification Scope
> >   Vhost-pci RFC: Benefits to KVM
> >   Vhost-pci RFC: Detailed Description in the Virtio Specification Format
> >   Vhost-pci RFC: Future Security Enhancement
> >   Vhost-pci RFC: Experimental Results
> >
> >  Benefits  |   8 ++
> >  Details   | 324
> ++
> >  FutureWorks   |  21 
> >  Introduction  |  31 ++
> >  ModificationScope |   3 +
> >  Results   |  18 +++
> 
> Umm, are you really creating 6 new files?  Shouldn't this just be a single 
> patch,
> as a single file, under the docs/ subdirectory?

Yeah, I actually split it into 6 files. I think it's more convenient to review 
and discuss them.

Best,
Wei 


> 
> >  6 files changed, 405 insertions(+)
> >  create mode 100644 Benefits
> >  create mode 100644 Details
> >  create mode 100644 FutureWorks
> >  create mode 100644 Introduction
> >  create mode 100644 ModificationScope
> >  create mode 100644 Results
> >
> 
> --
> Eric Blake   eblake redhat com+1-919-301-3266
> Libvirt virtualization library http://libvirt.org

Re: [Qemu-devel] [Qemu-ppc] [PULL 04/12] ppc: tlbie, tlbia and tlbisync are HV only

[David Gibson]

On Tue, May 31, 2016 at 11:28:49PM +0100, Mark Cave-Ayland wrote:
> On 31/05/16 01:41, David Gibson wrote:
> 
> > From: Benjamin Herrenschmidt 
> > 
> > Not that anything remotely recent supports tlbia but ...
> > 
> > Signed-off-by: Benjamin Herrenschmidt 
> > Signed-off-by: David Gibson 
> > ---
> >  target-ppc/translate.c | 6 +++---
> >  1 file changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/target-ppc/translate.c b/target-ppc/translate.c
> > index dfd3010..690ffd2 100644
> > --- a/target-ppc/translate.c
> > +++ b/target-ppc/translate.c
> > @@ -4858,7 +4858,7 @@ static void gen_tlbie(DisasContext *ctx)
> >  #if defined(CONFIG_USER_ONLY)
> >  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
> >  #else
> > -if (unlikely(ctx->pr)) {
> > +if (unlikely(ctx->pr || !ctx->hv)) {
> >  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
> >  return;
> >  }
> > @@ -4879,7 +4879,7 @@ static void gen_tlbsync(DisasContext *ctx)
> >  #if defined(CONFIG_USER_ONLY)
> >  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
> >  #else
> > -if (unlikely(ctx->pr)) {
> > +if (unlikely(ctx->pr || !ctx->hv)) {
> >  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
> >  return;
> >  }
> > @@ -4898,7 +4898,7 @@ static void gen_slbia(DisasContext *ctx)
> >  #if defined(CONFIG_USER_ONLY)
> >  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
> >  #else
> > -if (unlikely(ctx->pr)) {
> > +if (unlikely(ctx->pr || !ctx->hv)) {
> >  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
> >  return;
> >  }
> 
> Unfortunately this patch breaks qemu-system-ppc for both g3beige and
> mac99 under TCG causing a freeze in OpenBIOS when starting
> qemu-system-ppc with no parameters.

Bother, sorry.

I think this is because I applied this without the patch that treats
machines with no hypervisor mode (e.g. Apples) as always being in
hypervisor mode.

> Note that there is also another regression that has recently landed in
> git master so you'll also need to revert
> e7c9136977cb99c6eb52c9139f7b8d8b5fa87db9 in order to get back to a
> functioning OpenBIOS.

I'd preter to see it fixed rather than just reverted..

-- 
David Gibson| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au  | minimalist, thank you.  NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v5 1/4] Provide support for the CUSE TPM

[Xu, Quan]

On Wednesday, June 01, 2016 2:59 AM, BICKFORD, JEFFREY E  wrote:
> > * Daniel P. Berrange (berra...@redhat.com) wrote:
> > > On Wed, Jan 20, 2016 at 10:54:47AM -0500, Stefan Berger wrote:
> > > > On 01/20/2016 10:46 AM, Daniel P. Berrange wrote:
> > > > >On Wed, Jan 20, 2016 at 10:31:56AM -0500, Stefan Berger wrote:
> > > > >>"Daniel P. Berrange"  wrote on 01/20/2016
> > > > >>10:00:41
> > > > >>AM:
> > > > >>
> > > > >>
> > > > >>>process at all - it would make sense if there was a single
> > > > >>>swtpm_cuse shared across all QEMU's, but if there's one per
> > > > >>>QEMU device, it feels like it'd be much simpler to just have
> > > > >>>the functionality linked in QEMU.  That avoids the problem
> > > > >>I tried having it linked in QEMU before. It was basically rejected.
> > > > >I remember an impl you did many years(?) ago now, but don't
> > > > >recall the results of the discussion. Can you elaborate on why it
> > > > >was rejected as an approach ? It just doesn't make much sense to
> > > > >me to have to create an external daemon, a CUSE device and comms
> > > > >protocol, simply to be able to read/write a plain file containing
> > > > >the TPM state. Its massive over engineering IMHO and adding way
> > > > >more complexity and thus scope for failure
> > > >
> > > > The TPM 1.2 implementation adds 10s of thousands of lines of code.
> > > > The TPM 2 implementation is in the same range. The concern was
> > > > having this code right in the QEMU address space. It's big, it can
> > > > have bugs, so we don't want it to harm QEMU. So we now put this
> > > > into an external process implemented by the swtpm project that
> > > > builds on libtpms which provides TPM 1.2 functionality (to be
> > > > extended with TPM 2). We cannot call APIs of libtpms directly
> > > > anymore, so we need a control channel, which is implemented through
> ioctls on the CUSE device.
> > >
> > > Ok, the security separation concern does make some sense. The use of
> > > CUSE still seems fairly questionable to me. CUSE makes sense if you
> > > want to provide a drop-in replacement for the kernel TPM device
> > > driver, which would avoid ned for a new QEMU backend. If you're not
> > > emulating an existing kernel driver ABI though, CUSE + ioctl is
> > > feels like a really awful RPC transport between 2 userspace processes.
> 
> > While I don't really like CUSE; I can see some of the reasoning here.
> > By providing the existing TPM ioctl interface I think it means you can
> > use existing host-side TPM tools to initialise/query the soft-tpm, and
> > those should be independent of the soft-tpm implementation.
> > As for the extra interfaces you need because it's a soft-tpm to set it
> > up, once you've already got that ioctl interface as above, then it
> > seems to make sense to extend that to add the extra interfaces needed.
> > The only thing you have to watch for there are that the extra
> > interfaces don't clash with any future kernel ioctl extensions, and
> > that the interface defined is generic enough for different soft-tpm
> implementations.
> 
> > Dave
> > Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
> 
> 
> Over the past several months, AT Security Research has been testing the
> Virtual TPM software from IBM on the Power (ppc64) platform.

What about x86 platform?

> Based on our
> testing results, the vTPM software works well and as expected. Support for
> libvirt and the CUSE TPM allows us to create VMs with the vTPM functionality
> and was tested in a full-fledged OpenStack environment.
>

Cool..

> We believe the vTPM functionality will improve various aspects of VM security
> in our enterprise-grade cloud environment. AT would like to see these
> patches accepted into the QEMU community as the default-standard build so
> this technology can be easily adopted in various open source cloud
> deployments.

Stefan: could you update status about this patch set? I'd really appreciate 
your patch..

-Quan

[Qemu-devel] [PATCH] block: Drop bdrv_ioctl_bh_cb

[Fam Zheng]

Similar to the "!drv || !drv->bdrv_aio_ioctl" case above, here it is
okay to set co.ret and return. As pointed out by Paolo, a BH will be
created as necessary by the caller (bdrv_co_maybe_schedule_bh).
Besides, as pointed out by Kevin, "data" was leaked before.

Reported-by: Kevin Wolf 
Reported-by: Paolo Bonzini 
Signed-off-by: Fam Zheng 
---
 block/io.c | 20 ++--
 1 file changed, 2 insertions(+), 18 deletions(-)

diff --git a/block/io.c b/block/io.c
index 2d832aa..c32f5b7 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2309,19 +2309,6 @@ int bdrv_discard(BlockDriverState *bs, int64_t 
sector_num, int nb_sectors)
 return rwco.ret;
 }
 
-typedef struct {
-CoroutineIOCompletion *co;
-QEMUBH *bh;
-} BdrvIoctlCompletionData;
-
-static void bdrv_ioctl_bh_cb(void *opaque)
-{
-BdrvIoctlCompletionData *data = opaque;
-
-bdrv_co_io_em_complete(data->co, -ENOTSUP);
-qemu_bh_delete(data->bh);
-}
-
 static int bdrv_co_do_ioctl(BlockDriverState *bs, int req, void *buf)
 {
 BlockDriver *drv = bs->drv;
@@ -2339,11 +2326,8 @@ static int bdrv_co_do_ioctl(BlockDriverState *bs, int 
req, void *buf)
 
 acb = drv->bdrv_aio_ioctl(bs, req, buf, bdrv_co_io_em_complete, );
 if (!acb) {
-BdrvIoctlCompletionData *data = g_new(BdrvIoctlCompletionData, 1);
-data->bh = aio_bh_new(bdrv_get_aio_context(bs),
-bdrv_ioctl_bh_cb, data);
-data->co = 
-qemu_bh_schedule(data->bh);
+co.ret = -ENOTSUP;
+goto out;
 }
 qemu_coroutine_yield();
 out:
-- 
2.8.3

Re: [Qemu-devel] [RFC PATCH 2/2] add debian-bootstrap.docker target

[Fam Zheng]

On Tue, 05/31 16:27, Alex Bennée wrote:
> 
> Fam Zheng  writes:
> 
> > On Thu, 05/26 15:27, Alex Bennée wrote:
> >> Together with some changes to the docker script you can now build an
> >> arbitrary architecture of Debian using debootstrap. To achieve this I
> >> introduce the concept of a HOST_CMD in the docker config file. While
> >> copying the file into workspace the HOST_CMD is run in the docker build
> >> context. This allows debootstrap to set up its first stage before the
> >> container is built.
> >
> > Could you instead introduce the concept of $IMAGE.pre file (in this case
> > debian-bootstrap.pre, aside debian-bootstrap.docker), and exec it in
> > docker.py?  It would be much more flexible, and we we wouldn't need to 
> > inject a
> > custom directive to filter it out.
> 
> I'm ambivalent about that. To be honest this is a bit of a gap in
> docker's image creation (or if there is a better more docker-y way of
> doing things I couldn't find it). It does have the benefit of keeping
> everything in one place.

I don't like that it pollutes the dockerfile, rendering it invalid for raw
"docker build", which is not very good.

> 
> We are copying the dockerfile to the build environment anyway so it
> seemed natural to do the operation while copying across. How would
> envison the .pre setup? A #! script we just exec inside the temp
> directory we create?

Yes, since I'm going to send a v7, I'll merge the tmp dir part in to my series,
and add a new patch for the "pre" idea to see if we like it.

Fam

[Qemu-devel] [PATCH 1/2] Makefile: Add a "FORCE" target

[Fam Zheng]

Signed-off-by: Fam Zheng 
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile b/Makefile
index a5d7e62..1548b3f 100644
--- a/Makefile
+++ b/Makefile
@@ -81,7 +81,7 @@ Makefile: ;
 configure: ;
 
 .PHONY: all clean cscope distclean dvi html info install install-doc \
-   pdf recurse-all speed test dist msi
+   pdf recurse-all speed test dist msi FORCE
 
 $(call set-vpath, $(SRC_PATH))
 
-- 
2.8.3

[Qemu-devel] [PATCH 0/2] Let PKGVERSION include the "git describe" output

[Fam Zheng]

Makefile happened to be in my working set because of the docker test work, so I
went ahead to try this nice feature last evening before going to bed. My
apologies if we have duplicated work.

Please review.

Fam Zheng (2):
  Makefile: Add a "FORCE" target
  Makefile: Derive "PKGVERSION" from "git describe" by default

 Makefile  | 24 ++--
 linux-user/main.c |  1 +
 qemu-img.c|  1 +
 qmp.c |  1 +
 scripts/create_config |  4 
 vl.c  |  1 +
 6 files changed, 26 insertions(+), 6 deletions(-)

-- 
2.8.3

[Qemu-devel] [PATCH 2/2] Makefile: Derive "PKGVERSION" from "git describe" by default

[Fam Zheng]

Currently, if not specified in "./configure", QEMU_PKGVERSION will be
empty. Write a rule in Makefile to generate a value from "git describe"
combined with a possible git tree cleanness suffix, and write into a new
header.

$ cat qemu-version.h
#define QEMU_PKGVERSION "-v2.6.0-557-gd6550e9-unclean"

Include the header in .c files where the macro is referenced. It's not
necessary to include it in all files, otherwise each time the content of
the file changes, all sources have to be recompiled.

Signed-off-by: Fam Zheng 
---
 Makefile  | 22 +-
 linux-user/main.c |  1 +
 qemu-img.c|  1 +
 qmp.c |  1 +
 scripts/create_config |  4 
 vl.c  |  1 +
 6 files changed, 25 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index 1548b3f..7b4c019 100644
--- a/Makefile
+++ b/Makefile
@@ -49,7 +49,7 @@ ifneq ($(filter-out $(UNCHECKED_GOALS),$(MAKECMDGOALS)),$(if 
$(MAKECMDGOALS),,fa
 endif
 endif
 
-GENERATED_HEADERS = config-host.h qemu-options.def
+GENERATED_HEADERS = qemu-version.h config-host.h qemu-options.def
 GENERATED_HEADERS += qmp-commands.h qapi-types.h qapi-visit.h qapi-event.h
 GENERATED_SOURCES += qmp-marshal.c qapi-types.c qapi-visit.c qapi-event.c
 GENERATED_HEADERS += qmp-introspect.h
@@ -169,6 +169,26 @@ endif
 
 all: $(DOCS) $(TOOLS) $(HELPERS-y) recurse-all modules
 
+qemu-version.h: FORCE
+   $(call quiet-command, \
+   (cd $(SRC_PATH); \
+   echo -n '#define QEMU_PKGVERSION '; \
+   if test -n "$(PKGVERSION)"; then \
+   echo '"$(PKGVERSION)"'; \
+   else \
+   echo -n '"-'; \
+   if ! git status &>/dev/null; then \
+   echo -n "no-git"; \
+   else \
+   git describe 2>/dev/null | tr -d '\n'; \
+   if ! git diff-index --quiet HEAD &>/dev/null; 
then \
+   echo -n '-unclean'; \
+   fi \
+   fi; \
+   echo '"'; \
+   fi) > $@.tmp)
+   $(call quiet-command, cmp --quiet $@ $@.tmp || mv $@.tmp $@)
+
 config-host.h: config-host.h-timestamp
 config-host.h-timestamp: config-host.mak
 qemu-options.def: $(SRC_PATH)/qemu-options.hx
diff --git a/linux-user/main.c b/linux-user/main.c
index b2bc6ab..8a11d02 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -17,6 +17,7 @@
  *  along with this program; if not, see .
  */
 #include "qemu/osdep.h"
+#include "qemu-version.h"
 #include 
 #include 
 #include 
diff --git a/qemu-img.c b/qemu-img.c
index 4b56ad3..32e307c 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -22,6 +22,7 @@
  * THE SOFTWARE.
  */
 #include "qemu/osdep.h"
+#include "qemu-version.h"
 #include "qapi/error.h"
 #include "qapi-visit.h"
 #include "qapi/qmp-output-visitor.h"
diff --git a/qmp.c b/qmp.c
index 3165f87..7df6543 100644
--- a/qmp.c
+++ b/qmp.c
@@ -14,6 +14,7 @@
  */
 
 #include "qemu/osdep.h"
+#include "qemu-version.h"
 #include "qemu/cutils.h"
 #include "monitor/monitor.h"
 #include "sysemu/sysemu.h"
diff --git a/scripts/create_config b/scripts/create_config
index b2d2ebb..d47057b 100755
--- a/scripts/create_config
+++ b/scripts/create_config
@@ -9,10 +9,6 @@ case $line in
 version=${line#*=}
 echo "#define QEMU_VERSION \"$version\""
 ;;
- PKGVERSION=*) # configuration
-pkgversion=${line#*=}
-echo "#define QEMU_PKGVERSION \"$pkgversion\""
-;;
  qemu_*dir=*) # qemu-specific directory configuration
 name=${line%=*}
 value=${line#*=}
diff --git a/vl.c b/vl.c
index 18d1423..97cad71 100644
--- a/vl.c
+++ b/vl.c
@@ -22,6 +22,7 @@
  * THE SOFTWARE.
  */
 #include "qemu/osdep.h"
+#include "qemu-version.h"
 #include "qemu/cutils.h"
 #include "qemu/help_option.h"
 
-- 
2.8.3

[Qemu-devel] [Bug 1580459] Re: Windows (10?) guest freezes entire host on shutdown if using PCI passthrough

[Jimi]

I know it didn't with the GTX 660. It worked perfectly fine. But, I went
fully into Steam streaming everything before I got the 960, so the 960
could have that issue for all I know.

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1580459

Title:
  Windows (10?) guest freezes entire host on shutdown if using PCI
  passthrough

Status in libvirt:
  New
Status in QEMU:
  New
Status in Arch Linux:
  New
Status in Debian:
  New
Status in Fedora:
  New

Bug description:
  Problem: after leaving a Windows VM that uses PCI passthrough (as we
  do for gaming graphics cards, sound cards, and in my case, a USB card)
  running for some amount of time between 1 and 2 hours (it's not
  consistent with exactly how long), and for any amount of time longer
  than that, shutting down that guest will, right as it finishes
  shutting down, freeze the host computer, making it require a hard
  reboot. Unbinding (or in the other user's case, unbinding and THEN
  binding) any PCI device in sysfs, even one that has nothing to do with
  the VM, also has the same effect as shutting down the VM (if the VM
  has been running long enough). So, it's probably an issue related to
  unbinding and binding PCI devices.

  There's a lot of info on this problem over at 
https://bbs.archlinux.org/viewtopic.php?id=206050
  Here's a better-organized list of main details:
  -at least 2 confirmed victims of this bug; 2 (including me) have provided 
lots of info in the link
  -I'm on Arch Linux and the other one is on Gentoo (distro-nonspecific)
  -issue affects my Windows 10 guest and others' Windows guests, but not my 
Arch Linux guest (the others don't have non-Windows guests to test)
  -I'm using libvirt but the other user is not, so it's not an issue with 
libvirt
  -It seems to be version non-specific, too. I first noticed it at, or when 
testing versions still had the issue at (whichever version is lower), Linux 4.1 
and qemu 2.4.0. It still persists in all releases of both since, including the 
newest ones.
  -I can't track down exactly what package downgrade can fix it, as downgrading 
further than Linux 4.1 and qemu 2.4.0 requires Herculean and system-destroying 
changes such as downgrading ncurses, meaning I don't know whether it's a bug in 
QEMU, the Linux kernel, or some weird seemingly unrelated thing.
  -According to the other user, "graphics intensive gameplay (GTA V) can cause 
the crash to happen sooner," as soon as "15 minutes"
  -Also, "bringing up a second passthrough VM with separate hardware will cause 
the same crash," and "bringing up another VM before the two-hour mark will not 
result in a crash," further cementing that it's triggered by the un/binding of 
PCI devices.
  -This is NOT related to the very similar bug that can be worked around by not 
passing through the HDMI device or sound card. Even when we removed all traces 
of any sort of sound card from the VM, it still had the same behavior.

To manage notifications about this bug go to:
https://bugs.launchpad.net/libvirt/+bug/1580459/+subscriptions

Re: [Qemu-devel] [PATCH V3] block/io: optimize bdrv_co_pwritev for small requests

[Stefan Hajnoczi]

On Mon, May 30, 2016 at 01:31:13PM +0200, Peter Lieven wrote:
> in a read-modify-write cycle a small request might cause
> head and tail to fall into the same aligned block. Currently
> QEMU reads the same block twice in this case which is
> not necessary.
> 
> Signed-off-by: Peter Lieven 
> ---
> v1->v2: following Paolos suggestions to simplify the if condition and
> adjusting the comment
> v2->v3: fix iotest 077 for requests that are within the same aligned block 
> [Fam, Kevin]
> 
>  block/io.c |  8 
>  tests/qemu-iotests/077 | 12 +---
>  tests/qemu-iotests/077.out | 26 --
>  3 files changed, 9 insertions(+), 37 deletions(-)

Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan


signature.asc
Description: PGP signature

[Qemu-devel] [Bug 1580459] Re: Windows (10?) guest freezes entire host on shutdown if using PCI passthrough

[Chris McCarron]

Jimi, does your HDMI sound lag?  I am using a usb sound card and tries
switching to the GTX970 sound and I got horrible lag, sounds like sound
is in slow motion.  Was completely unusable.


Chris

-- 
You received this bug notification because you are a member of qemu-
devel-ml, which is subscribed to QEMU.
https://bugs.launchpad.net/bugs/1580459

Title:
  Windows (10?) guest freezes entire host on shutdown if using PCI
  passthrough

Status in libvirt:
  New
Status in QEMU:
  New
Status in Arch Linux:
  New
Status in Debian:
  New
Status in Fedora:
  New

Bug description:
  Problem: after leaving a Windows VM that uses PCI passthrough (as we
  do for gaming graphics cards, sound cards, and in my case, a USB card)
  running for some amount of time between 1 and 2 hours (it's not
  consistent with exactly how long), and for any amount of time longer
  than that, shutting down that guest will, right as it finishes
  shutting down, freeze the host computer, making it require a hard
  reboot. Unbinding (or in the other user's case, unbinding and THEN
  binding) any PCI device in sysfs, even one that has nothing to do with
  the VM, also has the same effect as shutting down the VM (if the VM
  has been running long enough). So, it's probably an issue related to
  unbinding and binding PCI devices.

  There's a lot of info on this problem over at 
https://bbs.archlinux.org/viewtopic.php?id=206050
  Here's a better-organized list of main details:
  -at least 2 confirmed victims of this bug; 2 (including me) have provided 
lots of info in the link
  -I'm on Arch Linux and the other one is on Gentoo (distro-nonspecific)
  -issue affects my Windows 10 guest and others' Windows guests, but not my 
Arch Linux guest (the others don't have non-Windows guests to test)
  -I'm using libvirt but the other user is not, so it's not an issue with 
libvirt
  -It seems to be version non-specific, too. I first noticed it at, or when 
testing versions still had the issue at (whichever version is lower), Linux 4.1 
and qemu 2.4.0. It still persists in all releases of both since, including the 
newest ones.
  -I can't track down exactly what package downgrade can fix it, as downgrading 
further than Linux 4.1 and qemu 2.4.0 requires Herculean and system-destroying 
changes such as downgrading ncurses, meaning I don't know whether it's a bug in 
QEMU, the Linux kernel, or some weird seemingly unrelated thing.
  -According to the other user, "graphics intensive gameplay (GTA V) can cause 
the crash to happen sooner," as soon as "15 minutes"
  -Also, "bringing up a second passthrough VM with separate hardware will cause 
the same crash," and "bringing up another VM before the two-hour mark will not 
result in a crash," further cementing that it's triggered by the un/binding of 
PCI devices.
  -This is NOT related to the very similar bug that can be worked around by not 
passing through the HDMI device or sound card. Even when we removed all traces 
of any sort of sound card from the VM, it still had the same behavior.

To manage notifications about this bug go to:
https://bugs.launchpad.net/libvirt/+bug/1580459/+subscriptions

Re: [Qemu-devel] [PATCH] iostatus: fix comments for block_job_iostatus_reset

[Stefan Hajnoczi]

On Mon, May 30, 2016 at 05:28:11PM +0800, Changlong Xie wrote:
> Signed-off-by: Changlong Xie 
> ---
>  include/block/blockjob.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH 0/2] Block migration: Convert to BlockBackend

[Stefan Hajnoczi]

On Mon, May 30, 2016 at 11:24:12AM +0200, Kevin Wolf wrote:
> Users outside of the block layer shouldn't directly use BlockDriverState for
> issuing their I/O requests, but go through a BlockBackend to do so. Block
> migration ('migrate -b') is (one of?) the last remaining users that need to be
> converted.
> 
> Kevin Wolf (2):
>   migration/block: Convert load to BlockBackend
>   migration/block: Convert saving to BlockBackend
> 
>  migration/block.c | 147 
> --
>  1 file changed, 88 insertions(+), 59 deletions(-)
> 
> -- 
> 1.8.3.1
> 

Reviewed-by: Stefan Hajnoczi 


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH] block/io: Remove unused bdrv_aio_write_zeroes()

[Stefan Hajnoczi]

On Mon, May 30, 2016 at 11:17:32AM +0200, Kevin Wolf wrote:
> Signed-off-by: Kevin Wolf 
> ---
>  block/io.c| 11 ---
>  include/block/block.h |  3 ---
>  trace-events  |  1 -
>  3 files changed, 15 deletions(-)

Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH v4 0/2] trace: Add event for vCPU memory accesses

[Stefan Hajnoczi]

On Fri, May 27, 2016 at 06:00:53PM +0200, Lluís Vilanova wrote:
> This series adds an event to track information related to memory accesses
> performed by the guest CPUs ("guest_mem_before").
> 
> A future series might extend this to contain the physical address and memory
> value (e.g., "guest_mem_after").
> 
> Signed-off-by: Lluís Vilanova 
> ---
> 
> Changes in v4
> -
> 
> * Clarify alignment info is not on the trace.
> * Add event information on commit log. [Richard Henderson]

I'm happy to merge once TCG folks have reviewed it.


signature.asc
Description: PGP signature

Re: [Qemu-devel] [Qemu-ppc] [PULL 04/12] ppc: tlbie, tlbia and tlbisync are HV only

[Mark Cave-Ayland]

On 31/05/16 01:41, David Gibson wrote:

> From: Benjamin Herrenschmidt 
> 
> Not that anything remotely recent supports tlbia but ...
> 
> Signed-off-by: Benjamin Herrenschmidt 
> Signed-off-by: David Gibson 
> ---
>  target-ppc/translate.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/target-ppc/translate.c b/target-ppc/translate.c
> index dfd3010..690ffd2 100644
> --- a/target-ppc/translate.c
> +++ b/target-ppc/translate.c
> @@ -4858,7 +4858,7 @@ static void gen_tlbie(DisasContext *ctx)
>  #if defined(CONFIG_USER_ONLY)
>  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
>  #else
> -if (unlikely(ctx->pr)) {
> +if (unlikely(ctx->pr || !ctx->hv)) {
>  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
>  return;
>  }
> @@ -4879,7 +4879,7 @@ static void gen_tlbsync(DisasContext *ctx)
>  #if defined(CONFIG_USER_ONLY)
>  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
>  #else
> -if (unlikely(ctx->pr)) {
> +if (unlikely(ctx->pr || !ctx->hv)) {
>  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
>  return;
>  }
> @@ -4898,7 +4898,7 @@ static void gen_slbia(DisasContext *ctx)
>  #if defined(CONFIG_USER_ONLY)
>  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
>  #else
> -if (unlikely(ctx->pr)) {
> +if (unlikely(ctx->pr || !ctx->hv)) {
>  gen_inval_exception(ctx, POWERPC_EXCP_PRIV_OPC);
>  return;
>  }

Unfortunately this patch breaks qemu-system-ppc for both g3beige and
mac99 under TCG causing a freeze in OpenBIOS when starting
qemu-system-ppc with no parameters.

Note that there is also another regression that has recently landed in
git master so you'll also need to revert
e7c9136977cb99c6eb52c9139f7b8d8b5fa87db9 in order to get back to a
functioning OpenBIOS.


ATB,

Mark.

Re: [Qemu-devel] [PULL 03/31] hw/char: QOM'ify escc.c

[Mark Cave-Ayland]

On 27/05/16 11:06, Paolo Bonzini wrote:

> From: xiaoqiang zhao 
> 
> * Drop the old SysBus init function and use instance_init
> * Call qemu_chr_add_handlers in the realize callback
> 
> Signed-off-by: xiaoqiang zhao 
> Message-Id: <1464158344-12266-2-git-send-email-zxq_yx_...@163.com>
> Signed-off-by: Paolo Bonzini 
> ---
>  hw/char/escc.c | 30 +++---
>  1 file changed, 19 insertions(+), 11 deletions(-)
> 
> diff --git a/hw/char/escc.c b/hw/char/escc.c
> index 7bf09a0..8e6a7df 100644
> --- a/hw/char/escc.c
> +++ b/hw/char/escc.c
> @@ -983,9 +983,10 @@ void slavio_serial_ms_kbd_init(hwaddr base, qemu_irq irq,
>  sysbus_mmio_map(s, 0, base);
>  }
>  
> -static int escc_init1(SysBusDevice *dev)
> +static void escc_init1(Object *obj)
>  {
> -ESCCState *s = ESCC(dev);
> +ESCCState *s = ESCC(obj);
> +SysBusDevice *dev = SYS_BUS_DEVICE(obj);
>  unsigned int i;
>  
>  s->chn[0].disabled = s->disabled;
> @@ -994,17 +995,26 @@ static int escc_init1(SysBusDevice *dev)
>  sysbus_init_irq(dev, >chn[i].irq);
>  s->chn[i].chn = 1 - i;
>  s->chn[i].clock = s->frequency / 2;
> -if (s->chn[i].chr) {
> -qemu_chr_add_handlers(s->chn[i].chr, serial_can_receive,
> -  serial_receive1, serial_event, >chn[i]);
> -}
>  }
>  s->chn[0].otherchn = >chn[1];
>  s->chn[1].otherchn = >chn[0];
>  
> -memory_region_init_io(>mmio, OBJECT(s), _mem_ops, s, "escc",
> +memory_region_init_io(>mmio, obj, _mem_ops, s, "escc",
>ESCC_SIZE << s->it_shift);
>  sysbus_init_mmio(dev, >mmio);
> +}
> +
> +static void escc_realize(DeviceState *dev, Error **errp)
> +{
> +ESCCState *s = ESCC(dev);
> +unsigned int i;
> +
> +for (i = 0; i < 2; i++) {
> +if (s->chn[i].chr) {
> +qemu_chr_add_handlers(s->chn[i].chr, serial_can_receive,
> +  serial_receive1, serial_event, >chn[i]);
> +}
> +}
>  
>  if (s->chn[0].type == mouse) {
>  qemu_add_mouse_event_handler(sunmouse_event, >chn[0], 0,
> @@ -1014,8 +1024,6 @@ static int escc_init1(SysBusDevice *dev)
>  s->chn[1].hs = qemu_input_handler_register((DeviceState 
> *)(>chn[1]),
> _handler);
>  }
> -
> -return 0;
>  }
>  
>  static Property escc_properties[] = {
> @@ -1032,10 +1040,9 @@ static Property escc_properties[] = {
>  static void escc_class_init(ObjectClass *klass, void *data)
>  {
>  DeviceClass *dc = DEVICE_CLASS(klass);
> -SysBusDeviceClass *k = SYS_BUS_DEVICE_CLASS(klass);
>  
> -k->init = escc_init1;
>  dc->reset = escc_reset;
> +dc->realize = escc_realize;
>  dc->vmsd = _escc;
>  dc->props = escc_properties;
>  set_bit(DEVICE_CATEGORY_INPUT, dc->categories);
> @@ -1045,6 +1052,7 @@ static const TypeInfo escc_info = {
>  .name  = TYPE_ESCC,
>  .parent= TYPE_SYS_BUS_DEVICE,
>  .instance_size = sizeof(ESCCState),
> +.instance_init = escc_init1,
>  .class_init= escc_class_init,
>  };

Unfortunately this patch causes OpenBIOS to freeze on startup under
qemu-system-ppc (presumably as there is a problem accessing the serial
port). You can reproduce this by starting qemu-system-ppc with no
parameters against the commits below:

Bad : e7c9136977cb99c6eb52c9139f7b8d8b5fa87db9
Good: b138e654a0525f009e7e7c96fc67d74baf3e011b

Note that you'll currently need to use the above two hashes to reproduce
the issue against git master as another regression has just crept in.


ATB,

Mark.

Re: [Qemu-devel] [PATCH] virtio: drop duplicate virtio_queue_get_id() function

[Stefan Hajnoczi]

On Fri, May 20, 2016 at 11:04:21AM -0700, Stefan Hajnoczi wrote:
> The virtio_queue_get_id() function is the lesser used duplicate of
> virtio_get_queue_index().  Use the latter instead.
> 
> Signed-off-by: Stefan Hajnoczi 
> ---
>  hw/scsi/virtio-scsi.c  | 2 +-
>  hw/virtio/virtio.c | 7 ---
>  include/hw/virtio/virtio.h | 1 -
>  3 files changed, 1 insertion(+), 9 deletions(-)

Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block

Stefan


signature.asc
Description: PGP signature

Re: [Qemu-devel] [PATCH] ui: sdl2: Fix crash with -nodefaults -sdl

[Peter Maydell]

On 31 May 2016 at 21:56, Cole Robinson  wrote:
> $ ./x86_64-softmmu/qemu-system-x86_64 -nodefaults -sdl
> Segmentation fault (core dumped)
>
> 0  0x559631af in sdl_display_init (ds=, full_screen=0, 
> no_frame=) at ui/sdl2.c:822
> 1  0x556c8a9a in main (argc=, argv=, 
> envp=) at vl.c:4527
>
> Setting the window icon assumes there's always an SDL output window
> available, which isn't the case with when there's no video device,
> like via -nodefaults. So don't try to set a window icon when we don't
> have any outputs.

Presumably we also crash for boards like the arm 'virt'
which just don't have a display device at all...

thanks
-- PMM

Re: [Qemu-devel] [QEMU RFC PATCH v3 4/6] Migration: migrate QTAILQ

[Jianjun Duan]

On 05/31/2016 12:54 PM, Paolo Bonzini wrote:
> 
> 
> - Original Message -
>> From: "Jianjun Duan" 
>> To: qemu-devel@nongnu.org
>> Cc: qemu-...@nongnu.org, du...@linux.vnet.ibm.com, dmi...@daynix.com, "peter 
>> maydell" ,
>> kra...@redhat.com, m...@redhat.com, da...@gibson.dropbear.id.au, 
>> pbonz...@redhat.com, veroniaba...@gmail.com,
>> quint...@redhat.com, "amit shah" , mre...@redhat.com, 
>> kw...@redhat.com, r...@twiddle.net,
>> aurel...@aurel32.net, "leon alrae" , 
>> blauwir...@gmail.com, "mark cave-ayland"
>> , mdr...@linux.vnet.ibm.com
>> Sent: Tuesday, May 31, 2016 8:02:42 PM
>> Subject: [QEMU RFC PATCH v3 4/6] Migration: migrate QTAILQ
>>
>> Currently we cannot directly transfer a QTAILQ instance because of the
>> limitation in the migration code. Here we introduce an approach to
>> transfer such structures. In our approach such a structure is tagged
>> with VMS_CSTM. We then modified vmstate_save_state and vmstate_load_state
>> so that when VMS_CSTM is encountered, put and get from VMStateInfo are
>> called respectively. This approach will be used to transfer pending_events
>> and ccs_list in spapr state.
>>
>> We also create some macros in qemu/queue.h to access a QTAILQ using pointer
>> arithmetic. This ensures that we do not depend on the implementation
>> details about QTAILQ in the migration code.
>>
>> Signed-off-by: Jianjun Duan 
>> ---
>>  include/migration/vmstate.h | 22 +
>>  include/qemu/queue.h| 32 ++
>>  migration/vmstate.c | 79
>>  +
>>  3 files changed, 133 insertions(+)
>>
>> diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
>> index 56a4171..da4ef7f 100644
>> --- a/include/migration/vmstate.h
>> +++ b/include/migration/vmstate.h
>> @@ -185,6 +185,8 @@ enum VMStateFlags {
>>   * to determine the number of entries in the array. Only valid in
>>   * combination with one of VMS_VARRAY*. */
>>  VMS_MULTIPLY_ELEMENTS = 0x4000,
>> +/* For fields which need customized handling, such as QTAILQ in
>> queue.h*/
>> +VMS_CSTM= 0x8000,
> 
> Please call this VMS_LINKED.  It can be adapted to other data
> structures in qemu/queue.h if there is a need later on.
> 
>>  };
>>  
>>  struct VMStateField {
>> @@ -245,6 +247,7 @@ extern const VMStateInfo vmstate_info_timer;
>>  extern const VMStateInfo vmstate_info_buffer;
>>  extern const VMStateInfo vmstate_info_unused_buffer;
>>  extern const VMStateInfo vmstate_info_bitmap;
>> +extern const VMStateInfo vmstate_info_qtailq;
>>  
>>  #define type_check_2darray(t1,t2,n,m) ((t1(*)[n][m])0 - (t2*)0)
>>  #define type_check_array(t1,t2,n) ((t1(*)[n])0 - (t2*)0)
>> @@ -656,6 +659,25 @@ extern const VMStateInfo vmstate_info_bitmap;
>>  .offset   = offsetof(_state, _field),\
>>  }
>>  
>> +/* For QTAILQ that need customized handling
>> + * _type: type of QTAILQ element
>> + * _next: name of QTAILQ entry field in QTAILQ element
>> + * _vmsd: VMSD for QTAILQ element
>> + * size: size of QTAILQ element
>> + * start: offset of QTAILQ entry in QTAILQ element
>> + */
>> +#define VMSTATE_QTAILQ_V(_field, _state, _version, _vmsd, _type, _next)  \
>> +{\
>> +.name = (stringify(_field)), \
>> +.version_id   = (_version),  \
>> +.vmsd = &(_vmsd),\
>> +.size = sizeof(_type),   \
>> +.info = _info_qtailq,\
>> +.flags= VMS_CSTM,\
>> +.offset   = offsetof(_state, _field),\
>> +.start= offsetof(_type, _next),  \
>> +}
>> +
>>  /* _f : field name
>> _f_n : num of elements field_name
>> _n : num of elements
>> diff --git a/include/qemu/queue.h b/include/qemu/queue.h
>> index f781aa2..003e368 100644
>> --- a/include/qemu/queue.h
>> +++ b/include/qemu/queue.h
>> @@ -437,3 +437,35 @@ struct {
>> \
>>  (*(((struct headname *)((elm)->field.tqe_prev))->tqh_last))
>>  
>>  #endif  /* !QEMU_SYS_QUEUE_H_ */
>> +
>> +/*
>> + * Offsets of layout of a tail queue head.
>> + */
>> +#define QTAILQ_FIRST_OFFSET 0
>> +#define QTAILQ_LAST_OFFSET (sizeof(void *))
>> +
>> +/*
>> + * Offsets of layout of a tail queue element.
>> + */
>> +#define QTAILQ_NEXT_OFFSET 0
>> +#define QTAILQ_PREV_OFFSET (sizeof(void *))
>> +
>> +/*
>> + * Tail queue tranversal using pointer arithmetic.
>> + */
>> +#define QTAILQ_RAW_FOREACH(elm, head, entry)
>> \
>> +for ((elm) = *((void **) ((char *) (head) + 

Re: [Qemu-devel] [PATCH] configure: save git working tree information in "pkgversion"

[Laszlo Ersek]

On 05/31/16 19:45, Eric Blake wrote:
> On 05/31/2016 11:01 AM, Laszlo Ersek wrote:
> 
 Grepping git's Documentation/RelNotes/ directory, I find:
 - in "1.6.6.txt": the introduction of --dirty
 - in "1.7.6.4.txt": an apparently important bugfix for --dirty

(*)


 Version 1.7.6.4 of git was tagged on Sep 23 2011.

 Does this information help in deciding if we can use --dirty?
>>>
>>> 5 years old sounds new enough for my liking :-)
>>>
>>> I guess we could use --dirty and catch the non-zero exit code and just
>>> re-try without --dirty.
>>
>> But, if we can't use --dirty, I should probably use the plus-sign
>> fallback (we need *something* to mark a dirty state).
>>
>> In which case however, shouldn't we just go with the current patch,
>> which doesn't care about --dirty at all? Otherwise, some build hosts
>> will append "-dirty", and others will append "+".
>>
>> IMO we should either require --dirty, or go with the current patch.
> 
> Gnulib's build-aux/git-version-gen script doesn't yet use --dirty, but
> may be an inspiration for how to generate the same suffix:
> 
> # Test whether to append the "-dirty" suffix only if the version
> # string we're using came from git.  I.e., skip the test if it's "UNKNOWN"
> # or if it came from .tarball-version.
> if test "x$v_from_git" != x; then
>   # Don't declare a version "dirty" merely because a time stamp has changed.
>   git update-index --refresh > /dev/null 2>&1

(
This is exactly the fix (*) that went into git v1.7.6.4 (and v1.7.7):

$ git log --oneline --reverse v1.7.6.3..v1.7.6.4

0f64bfa9567f ls-files: fix pathspec display on error
e9d4f7405b6a branch.c: use the parsed branch name
13d6ec913330 read_gitfile_gently(): rename misnamed function to read_gitfile()
9b0ebc722cfc clone: allow to clone from .git file
dbc92b072dd7 clone: allow more than one --reference
e6baf4a1ae1b clone: clone from a repository with relative alternates
2f633f41d695 check-ref-format --print: Normalize refnames that start with 
slashes
f3738c1ce919 Forbid DEL characters in reference names
385ceec1cb46 t3005: do not assume a particular order of stdout and stderr of 
git-ls-files
dff4b0ef30cd am: format is in $patch_format, not parse_patch
e622f41dcd97 git-mergetool: check return value from read
40ffc4987661 Merge branch 'gb/maint-am-patch-format-error-message' into maint
503359f13abc Merge branch 'mg/branch-set-upstream-previous' into maint
be5acb3b63af Merge branch 'mh/check-ref-format-print-normalize' into maint
406c1c4dd4a8 Merge branch 'nd/maint-clone-gitdir' into maint
84b051462fca Merge branch 'jc/maint-clone-alternates' into maint
85b3c75f4fd3 describe: Refresh the index when run with --dirty
a0b1cb60ab29 Merge branch 'cb/maint-ls-files-error-report' into maint
632052641517 Git 1.7.6.4

--> https://github.com/git/git/commit/85b3c75f4fd3
"describe: Refresh the index when run with --dirty"
)

> 
>   dirty=`exec 2>/dev/null;git diff-index --name-only HEAD` || dirty=
>   case "$dirty" in
>   '') ;;
>   *) # Append the suffix only if there isn't one already.
>   case $v in
> *-dirty) ;;
> *) v="$v-dirty" ;;
>   esac ;;
>   esac
> fi

This seems to do the right thing, yes.

I'll submit a new version later.

Thanks!
Laszlo

Re: [Qemu-devel] [PATCH 02/10] qcow2: add qcow2_co_write_compressed

[Eric Blake]

On 05/31/2016 03:00 PM, Denis V. Lunev wrote:
> On 05/31/2016 09:42 PM, Eric Blake wrote:
>> On 05/30/2016 06:58 AM, Pavel Butsykin wrote:
>>
>>> Sorry, but it seems this will never happen, because the second write
>>> will not pass this check:
>>>
>>> uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs,
>>> uint64_t offset,
>>> int compressed_size)
>>> {
>>>  ...
>>>  /* Compression can't overwrite anything. Fail if the cluster was
>>> already
>>>   * allocated. */
>>>  cluster_offset = be64_to_cpu(l2_table[l2_index]);
>>>  if (cluster_offset & L2E_OFFSET_MASK) {
>>>  qcow2_cache_put(bs, s->l2_table_cache, (void**) _table);
>>>  return 0;
>>>  }
>>> ...
>>>
>>> As you can see we can't do the compressed write in the already allocated
>>> cluster.
>> Umm, doesn't that defeat the point of compression, if every compressed
>> cluster becomes the head of a new cluster?  The whole goal of
>> compression is to be able to fit multiple clusters within one.
>>
> AFAIK the file will be sparse in that unused areas

IIRC, on the NTFS file system, the minimum hole size is 64k. If you also
have 64k clusters, then you don't have a sparse file - every tail of
zero sectors will be explicit in the filesystem, if you are using 1:1
clusters for compression.  Other file systems may have finer granularity
for holes, but it's still rather awkward to be relying on sparseness
when a better solution is to pack compressed sectors consecutively.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 15/33] docs: update ACPI CPU hotplug spec with new protocol

[Michael S. Tsirkin]

On Tue, May 31, 2016 at 05:07:41PM +0200, Igor Mammedov wrote:
> On Tue, 31 May 2016 07:49:16 +0300
> "Michael S. Tsirkin"  wrote:
> 
> > On Tue, May 17, 2016 at 04:43:07PM +0200, Igor Mammedov wrote:
> > > Signed-off-by: Igor Mammedov 
> > > ---
> > >  docs/specs/acpi_cpu_hotplug.txt | 88 
> > > +++--
> > >  1 file changed, 76 insertions(+), 12 deletions(-)
> > > 
> > > diff --git a/docs/specs/acpi_cpu_hotplug.txt 
> > > b/docs/specs/acpi_cpu_hotplug.txt
> > > index 340b751..c5bce6a 100644
> > > --- a/docs/specs/acpi_cpu_hotplug.txt
> > > +++ b/docs/specs/acpi_cpu_hotplug.txt
> > > @@ -4,21 +4,85 @@ QEMU<->ACPI BIOS CPU hotplug interface
> > >  QEMU supports CPU hotplug via ACPI. This document
> > >  describes the interface between QEMU and the ACPI BIOS.
> > >  
> > > -ACPI GPE block (IO ports 0xafe0-0xafe3, byte access):
> > > --
> > > -
> > > -Generic ACPI GPE block. Bit 2 (GPE.2) used to notify CPU
> > > -hot-add/remove event to ACPI BIOS, via SCI interrupt.
> > > +ACPI BIOS GPE.2 handler is dedicated for notifying OS about CPU hot-add
> > > +and hot-remove events.
> > >  
> > > +
> > > +Legacy ACPI CPU hotplug interface registers:
> > > +
> > >  CPU present bitmap for:
> > > +  One bit per CPU. Bit position reflects corresponding CPU APIC ID. 
> > > Read-only.
> > >ICH9-LPC (IO port 0x0cd8-0xcf7, 1-byte access)
> > >PIIX-PM  (IO port 0xaf00-0xaf1f, 1-byte access)
> > >  ---
> > > -One bit per CPU. Bit position reflects corresponding CPU APIC ID.
> > > -Read-only.
> > > +QEMU sets corresponding CPU bit on hot-add event and issues SCI
> > > +with GPE.2 event set. CPU present map read by ACPI BIOS GPE.2 handler
> > > +to notify OS about CPU hot-add events. CPU hot-remove isn't supported.
> > > +
> > > +=
> > > +ACPI CPU hotplug interface registers:
> > > +-
> > > +Register block base address:
> > > +ICH9-LPC IO port 0x0cd8
> > > +PIIX-PM  IO port 0xaf00  
> > 
> > OK but this means we either use legacy or new one,
> > bot both, which is problematic for people using old seabios
> > without acpi loading support and with -M pc.
> > 
> > I don't say we must support them with >255 CPUs,
> > but I do say we should make an effort for simple
> > setups with <255 CPUs.
> 
> 
> it works with QEMU provided(shipped) BIOS,
> it works for migration case as legacy stays enables because of -M 
> src_legacy_machine
> 
> more that 255 cpus will break old BIOS in different
> ways /corrupt/hang/ depending on BIOS version.
> 
> I'm not sure we should care about using old BIOS
> with new QEMU+new machine type though and allow
> expectations to be beyond what hw vendors usually set.
> It's the same as real hw does i.e. new hardware
> shipped with new BIOS. 
> If user insist on old BIOS+new machine type he still has
> property knob to force legacy mode.
> 
> on the fist glance, it's probably not that very hard
> to switch IO ports handling from legacy to new interface
> by sending from new cpu-hotplug AML a command to do so,
> my concerns here is:
>  * +1 more state to migrate
>  * probably issues with migration as target started with
>different IO layout
>  * IO window freed after switching from legacy to new,
>will not be available to guest as it started with
>legacy window consumed by CPUS.CRS.
>  * that legacy switching business is only PC specific
> means having a knob to turn it on so it won't pollute ARM
> 
> all in all it's probably too much headache to make sure
> that improbable usecase would work, so after considering
> this idea I've dropped it and did it the way it's now.

Well I think it's worth the effort. I agree it's tricky
to implement but we do maintain compatibility for years.
Being orthogonal with bios version is very helpful for
a variety of reasons such as debugging.


> > > +Register block size:
> > > +ACPI_CPU_HOTPLUG_REG_LEN = 12
> > > +
> > > +read access:  
> > 
> > So this implies acpi must scan all cpus on each event, and
> > this seems too aggressive.
> > I think we need something hierarchical where
> > you read one level and know which cpus to probe.
> That's what we do for mem-hotplug as it's not
> performance critical path.
> 
> In addition to that depending on guest OS/version
> it will anyway do enumeration of all CPUs after
> our hotplug AML method scanned all CPUs and
> sent notifies.
> 
> not that I'm in favor of complicating this protocol,
> but I wouldn't do it hierarchical,
> that's what Notify(BUS_CHECK) is supposed to do
> but it's broken on some guests.

Interesting. Which ones? Would they be easy to detect?

> So if I'd do a more complicated protocol,
> I'd do polling from AML side telling QEMU
>   if (cpu = 

Re: [Qemu-devel] [PATCH 02/10] qcow2: add qcow2_co_write_compressed

[Denis V. Lunev]

On 05/31/2016 09:42 PM, Eric Blake wrote:

On 05/30/2016 06:58 AM, Pavel Butsykin wrote:


Sorry, but it seems this will never happen, because the second write
will not pass this check:

uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs,
uint64_t offset,
int compressed_size)
{
 ...
 /* Compression can't overwrite anything. Fail if the cluster was
already
  * allocated. */
 cluster_offset = be64_to_cpu(l2_table[l2_index]);
 if (cluster_offset & L2E_OFFSET_MASK) {
 qcow2_cache_put(bs, s->l2_table_cache, (void**) _table);
 return 0;
 }
...

As you can see we can't do the compressed write in the already allocated
cluster.

Umm, doesn't that defeat the point of compression, if every compressed
cluster becomes the head of a new cluster?  The whole goal of
compression is to be able to fit multiple clusters within one.


AFAIK the file will be sparse in that unused areas

[Qemu-devel] [PATCH] ui: sdl2: Fix crash with -nodefaults -sdl

[Cole Robinson]

$ ./x86_64-softmmu/qemu-system-x86_64 -nodefaults -sdl
Segmentation fault (core dumped)

0  0x559631af in sdl_display_init (ds=, full_screen=0, 
no_frame=) at ui/sdl2.c:822
1  0x556c8a9a in main (argc=, argv=, 
envp=) at vl.c:4527

Setting the window icon assumes there's always an SDL output window
available, which isn't the case with when there's no video device,
like via -nodefaults. So don't try to set a window icon when we don't
have any outputs.

https://bugzilla.redhat.com/show_bug.cgi?id=1340931
---
 ui/sdl2.c | 20 +++-
 1 file changed, 11 insertions(+), 9 deletions(-)

diff --git a/ui/sdl2.c b/ui/sdl2.c
index 909038f..d0e0a41 100644
--- a/ui/sdl2.c
+++ b/ui/sdl2.c
@@ -812,16 +812,18 @@ void sdl_display_init(DisplayState *ds, int full_screen, 
int no_frame)
 register_displaychangelistener(_console[i].dcl);
 }
 
-/* Load a 32x32x4 image. White pixels are transparent. */
-filename = qemu_find_file(QEMU_FILE_TYPE_BIOS, "qemu-icon.bmp");
-if (filename) {
-SDL_Surface *image = SDL_LoadBMP(filename);
-if (image) {
-uint32_t colorkey = SDL_MapRGB(image->format, 255, 255, 255);
-SDL_SetColorKey(image, SDL_TRUE, colorkey);
-SDL_SetWindowIcon(sdl2_console[0].real_window, image);
+if (sdl2_num_outputs) {
+/* Load a 32x32x4 image. White pixels are transparent. */
+filename = qemu_find_file(QEMU_FILE_TYPE_BIOS, "qemu-icon.bmp");
+if (filename) {
+SDL_Surface *image = SDL_LoadBMP(filename);
+if (image) {
+uint32_t colorkey = SDL_MapRGB(image->format, 255, 255, 255);
+SDL_SetColorKey(image, SDL_TRUE, colorkey);
+SDL_SetWindowIcon(sdl2_console[0].real_window, image);
+}
+g_free(filename);
 }
-g_free(filename);
 }
 
 if (full_screen) {
-- 
2.5.5

[Qemu-devel] Film scanner and large transfer sizes with pvscsi

[Jarkko Lavinen]

I am trying to get a Minolta Multi Pro film scanner to work in a Qemu VM with 
pvsci interface and linux as both guest and host (kernel 4.4).

The scanning stalls at VM with no error in the scanning program but I can spot 
out of memory error when initiating a transfer at host side log. The host has 
many gigabytes of free memory, though. And the same scanning program works 
alright at host side.

When using 022077 SCSI logging level at host (max level or error and timeouts). 
I can see on the host log:

  ...
  [sg6] tag#0 CDB: Test Unit Ready 00 00 00 00 00 00
  [sg6] sg_cmd_done: pack_id=0, res=0x0
  [sg6] sg_finish_rem_req: res_used=0
  [sg6] sg_remove_scat: k_use_sg=0

  [sg6] sg_ioctl: cmd=0x2285 (SG_IO)
  [sg6] sg_common_write:  scsi opcode=0x28, cmd_size=10 (0x28 is READ cmd)
  [sg6] sg_start_req: dxfer_len=30408704
  [sg6] sg_build_indirect: buff_size=30408704, blk_size=30408704
  [sg6] sg_build_indirect: k=0, num=32768, ret_sz=32768
  ...
  [sg6] sg_build_indirect: k=127, num=32768, ret_sz=32768
  [sg6] sg_build_indirect: k_use_sg=128, rem_sz=26214400
  [sg6] sg_common_write: start_req err=-12
  [sg6] sg_finish_rem_req: res_used=0
  [sg6] sg_remove_scat: k_use_sg=128
  [sg6] sg_remove_scat: k=0, pg=0xea802800
  ...
  [sg6] sg_remove_scat: k=127, pg=0xea00052e8600
  [sg6] sg_ioctl: cmd=0x2285

Two problems here. There is an error in host kernel side and the error does not
propagate to the guest side scanning program.

Jarkko Lavinen

Re: [Qemu-devel] [RFC v2 PATCH 09/13] tcg/sparc: Add support for fence

[Richard Henderson]

On 05/31/2016 11:39 AM, Pranith Kumar wrote:

+case INDEX_op_mb:
+/* membar #LoadLoad|#LoadStore|#StoreStore|#StoreLoad */
+tcg_out32(s, MEMBAR | 15);
+break;


With the argument, this needs to be

if (a0 == TCG_MB_WRITE) {
/* #StoreStore | #StoreLoad */
a0 = 0xa;
} else if (a0 == TCG_MB_READ) {
/* #LoadStore | #LoadLoad */
a0 = 0x5;
} else {
/* #StoreStore | #LoadStore | #StoreLoad | #LoadLoad */
a0 = 0xf;
}
tcg_out32(s, MEMBAR | a0);


r~

Re: [Qemu-devel] [RFC v2 PATCH 07/13] tcg/ppc: Add support for fence

[Richard Henderson]

On 05/31/2016 11:39 AM, Pranith Kumar wrote:

+#define HWSYNC XO31(598)
+#define LWSYNC (HWSYNC | (1u << 21))

...

+case INDEX_op_mb:
+/* ??? Do we want SEQ_CST or ACQ_REL memory model.  */
+tcg_out32(s, HWSYNC);
+break;


With the flags argument, this needs to be

#define EIEIO  XO31(854)

a0 = args[0];
if (a0 == TCG_MB_WRITE) {
tcg_out32(s, EIEIO);
} else if (a1 == TCG_MB_READ) {
tcg_out32(s, LWSYNC);
} else {
tcg_out32(s, HWSYNC);
}


r~

Re: [Qemu-devel] [RFC v2 PATCH 02/13] tcg/i386: Add support for fence

[Richard Henderson]

On 05/31/2016 11:39 AM, Pranith Kumar wrote:

+case INDEX_op_mb:
+tcg_out_mb(s);


You need to look at the barrier type and DTRT.  In particular, the Linux 
smp_rmb and smp_wmb types need not emit any code.



+{ INDEX_op_mb, { "r" } },


You certainly do *not* need the constant argument loaded into a register.  This 
should remain { }.



r~

Re: [Qemu-devel] [RFC v2 PATCH 03/13] tcg/aarch64: Add support for fence

[Richard Henderson]

On 05/31/2016 11:39 AM, Pranith Kumar wrote:

+/* System instructions.  */
+DMB_ISH = 0xd5033bbf,

...

+case INDEX_op_mb:
+tcg_out32(s, DMB_ISH);
+break;


With the flags argument, this needs to be split.

DMB_ISH = 0xd5033b8f
DMB_RD  = 0x0010
DMB_WR  = 0x0020

if (a0 == TCG_MB_READ) {
a0 = DMB_RD;
} else if (a0 == TCG_MB_WRITE) {
a0 = DMB_WR;
} else {
a0 = DMB_RD | DMB_WR;
}
tcg_out32(s, DMB_ISH | a0);



r~

Re: [Qemu-devel] [RFC v2 PATCH 01/13] Introduce TCGOpcode for memory barrier

[Richard Henderson]

On 05/31/2016 11:39 AM, Pranith Kumar wrote:

+* Memory Barrier support
+
+* mb <$arg>


Document what $arg should be.


+Generate a target memory barrier instruction to ensure memory ordering as being
+enforced by a corresponding guest memory barrier instruction. The ordering
+enforced by the backend may be stricter than the ordering required by the 
guest.
+It cannot be weaker. This opcode takes an optional constant argument if 
required
+to generate the appropriate barrier instruction. The backend should take care 
to


The argument is *not* optional.


+void tcg_gen_mb(TCGArg a)
+{
+/* ??? Enable only when MTTCG is enabled.  */
+tcg_gen_op1(_ctx, INDEX_op_mb, 0);


Pass A to tcg_gen_op1, not 0.


+/* TCGOpmb args */
+#define TCG_MB_FULL ((TCGArg)(0))
+#define TCG_MB_READ ((TCGArg)(1))
+#define TCG_MB_WRITE((TCGArg)(2))
+#define TCG_MB_ACQUIRE  ((TCGArg)(3))
+#define TCG_MB_RELEASE  ((TCGArg)(4))


This is, IMO, confused.  Either we should use the C++11 barrier types, or the 
Linux barrier types, but not both.



r~

Re: [Qemu-devel] [QEMU RFC PATCH v3 4/6] Migration: migrate QTAILQ

[Paolo Bonzini]

- Original Message -
> From: "Jianjun Duan" 
> To: qemu-devel@nongnu.org
> Cc: qemu-...@nongnu.org, du...@linux.vnet.ibm.com, dmi...@daynix.com, "peter 
> maydell" ,
> kra...@redhat.com, m...@redhat.com, da...@gibson.dropbear.id.au, 
> pbonz...@redhat.com, veroniaba...@gmail.com,
> quint...@redhat.com, "amit shah" , mre...@redhat.com, 
> kw...@redhat.com, r...@twiddle.net,
> aurel...@aurel32.net, "leon alrae" , 
> blauwir...@gmail.com, "mark cave-ayland"
> , mdr...@linux.vnet.ibm.com
> Sent: Tuesday, May 31, 2016 8:02:42 PM
> Subject: [QEMU RFC PATCH v3 4/6] Migration: migrate QTAILQ
> 
> Currently we cannot directly transfer a QTAILQ instance because of the
> limitation in the migration code. Here we introduce an approach to
> transfer such structures. In our approach such a structure is tagged
> with VMS_CSTM. We then modified vmstate_save_state and vmstate_load_state
> so that when VMS_CSTM is encountered, put and get from VMStateInfo are
> called respectively. This approach will be used to transfer pending_events
> and ccs_list in spapr state.
> 
> We also create some macros in qemu/queue.h to access a QTAILQ using pointer
> arithmetic. This ensures that we do not depend on the implementation
> details about QTAILQ in the migration code.
> 
> Signed-off-by: Jianjun Duan 
> ---
>  include/migration/vmstate.h | 22 +
>  include/qemu/queue.h| 32 ++
>  migration/vmstate.c | 79
>  +
>  3 files changed, 133 insertions(+)
> 
> diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
> index 56a4171..da4ef7f 100644
> --- a/include/migration/vmstate.h
> +++ b/include/migration/vmstate.h
> @@ -185,6 +185,8 @@ enum VMStateFlags {
>   * to determine the number of entries in the array. Only valid in
>   * combination with one of VMS_VARRAY*. */
>  VMS_MULTIPLY_ELEMENTS = 0x4000,
> +/* For fields which need customized handling, such as QTAILQ in
> queue.h*/
> +VMS_CSTM= 0x8000,

Please call this VMS_LINKED.  It can be adapted to other data
structures in qemu/queue.h if there is a need later on.

>  };
>  
>  struct VMStateField {
> @@ -245,6 +247,7 @@ extern const VMStateInfo vmstate_info_timer;
>  extern const VMStateInfo vmstate_info_buffer;
>  extern const VMStateInfo vmstate_info_unused_buffer;
>  extern const VMStateInfo vmstate_info_bitmap;
> +extern const VMStateInfo vmstate_info_qtailq;
>  
>  #define type_check_2darray(t1,t2,n,m) ((t1(*)[n][m])0 - (t2*)0)
>  #define type_check_array(t1,t2,n) ((t1(*)[n])0 - (t2*)0)
> @@ -656,6 +659,25 @@ extern const VMStateInfo vmstate_info_bitmap;
>  .offset   = offsetof(_state, _field),\
>  }
>  
> +/* For QTAILQ that need customized handling
> + * _type: type of QTAILQ element
> + * _next: name of QTAILQ entry field in QTAILQ element
> + * _vmsd: VMSD for QTAILQ element
> + * size: size of QTAILQ element
> + * start: offset of QTAILQ entry in QTAILQ element
> + */
> +#define VMSTATE_QTAILQ_V(_field, _state, _version, _vmsd, _type, _next)  \
> +{\
> +.name = (stringify(_field)), \
> +.version_id   = (_version),  \
> +.vmsd = &(_vmsd),\
> +.size = sizeof(_type),   \
> +.info = _info_qtailq,\
> +.flags= VMS_CSTM,\
> +.offset   = offsetof(_state, _field),\
> +.start= offsetof(_type, _next),  \
> +}
> +
>  /* _f : field name
> _f_n : num of elements field_name
> _n : num of elements
> diff --git a/include/qemu/queue.h b/include/qemu/queue.h
> index f781aa2..003e368 100644
> --- a/include/qemu/queue.h
> +++ b/include/qemu/queue.h
> @@ -437,3 +437,35 @@ struct {
> \
>  (*(((struct headname *)((elm)->field.tqe_prev))->tqh_last))
>  
>  #endif  /* !QEMU_SYS_QUEUE_H_ */
> +
> +/*
> + * Offsets of layout of a tail queue head.
> + */
> +#define QTAILQ_FIRST_OFFSET 0
> +#define QTAILQ_LAST_OFFSET (sizeof(void *))
> +
> +/*
> + * Offsets of layout of a tail queue element.
> + */
> +#define QTAILQ_NEXT_OFFSET 0
> +#define QTAILQ_PREV_OFFSET (sizeof(void *))
> +
> +/*
> + * Tail queue tranversal using pointer arithmetic.
> + */
> +#define QTAILQ_RAW_FOREACH(elm, head, entry)
> \
> +for ((elm) = *((void **) ((char *) (head) + QTAILQ_FIRST_OFFSET));
> \
> + (elm);
> \
> + (elm) =
> \
> + *((void **) ((char *) (elm) + (entry) +
> QTAILQ_NEXT_OFFSET)))
> +/*
> + * Tail 

[Qemu-devel] [Qemu-devel [RFC] [WIP] v1] Keeping the Destination side alive incase of network failure (Migration recover from network failure)

[Md Haris Iqbal]

---
 include/migration/migration.h |  3 +++
 migration/migration.c | 12 
 migration/savevm.c| 25 +
 3 files changed, 32 insertions(+), 8 deletions(-)

diff --git a/include/migration/migration.h b/include/migration/migration.h
index ac2c12c..73832ac 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -326,6 +326,9 @@ void flush_page_queue(MigrationState *ms);
 int ram_save_queue_pages(MigrationState *ms, const char *rbname,
  ram_addr_t start, ram_addr_t len);
 
+// New added function for postcopy migration recovery in case of failure
+void qemu_postcopy_migration_recovery(QEMUFile **f,MigrationIncomingState* 
mis);
+
 PostcopyState postcopy_state_get(void);
 /* Set the state and return the old state */
 PostcopyState postcopy_state_set(PostcopyState new_state);
diff --git a/migration/migration.c b/migration/migration.c
index 991313a..4301972 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -1797,6 +1797,18 @@ void migrate_fd_connect(MigrationState *s)
 s->migration_thread_running = true;
 }
 
+void qemu_postcopy_migration_recovery(QEMUFile **f,
+  MigrationIncomingState* mis)
+{
+migrate_set_state(>state, MIGRATION_STATUS_ACTIVE,
+   MIGRATION_STATUS_POSTCOPY_RECOVERING);
+/* Code for network recovery to be added here */
+while(1) {
+fprintf(stderr, "Not letting it fail\n");
+sleep(2);
+}
+}
+
 PostcopyState  postcopy_state_get(void)
 {
 return atomic_mb_read(_postcopy_state);
diff --git a/migration/savevm.c b/migration/savevm.c
index 16ba443..03467da 100644
--- a/migration/savevm.c
+++ b/migration/savevm.c
@@ -1832,7 +1832,8 @@ qemu_loadvm_section_part_end(QEMUFile *f, 
MigrationIncomingState *mis)
 static int qemu_loadvm_state_main(QEMUFile *f, MigrationIncomingState *mis)
 {
 uint8_t section_type;
-int ret;
+PostcopyState ps;
+int32_t ret;
 
 while ((section_type = qemu_get_byte(f)) != QEMU_VM_EOF) {
 
@@ -1841,21 +1842,16 @@ static int qemu_loadvm_state_main(QEMUFile *f, 
MigrationIncomingState *mis)
 case QEMU_VM_SECTION_START:
 case QEMU_VM_SECTION_FULL:
 ret = qemu_loadvm_section_start_full(f, mis);
-if (ret < 0) {
-return ret;
-}
 break;
 case QEMU_VM_SECTION_PART:
 case QEMU_VM_SECTION_END:
 ret = qemu_loadvm_section_part_end(f, mis);
-if (ret < 0) {
-return ret;
-}
 break;
 case QEMU_VM_COMMAND:
 ret = loadvm_process_command(f);
 trace_qemu_loadvm_state_section_command(ret);
-if ((ret < 0) || (ret & LOADVM_QUIT)) {
+if (ret & LOADVM_QUIT) {
+fprintf(stderr, "LOADVM_QUIT\n");
 return ret;
 }
 break;
@@ -1863,6 +1859,19 @@ static int qemu_loadvm_state_main(QEMUFile *f, 
MigrationIncomingState *mis)
 error_report("Unknown savevm section type %d", section_type);
 return -EINVAL;
 }
+
+if (ret < 0) {
+fprintf(stderr,  "Failure\n");
+
+ps = postcopy_state_get();
+ret = qemu_file_get_error(f);
+fprintf(stderr, "Error %s %d\n", strerror(-ret), -ret);
+
+if(ret != -EIO && ps == POSTCOPY_INCOMING_RUNNING)
+qemu_postcopy_migration_recovery(, mis);
+else
+return ret;
+}
 }
 
 return 0;
-- 
2.7.4

[Qemu-devel] [Qemu-devel [RFC] [WIP] v1] Keeping the Source side alive incase of network failure (Migration recover from network failure)

[Md Haris Iqbal]

---
 include/migration/migration.h |  1 +
 migration/migration.c | 41 -
 vl.c  |  4 
 3 files changed, 41 insertions(+), 5 deletions(-)

diff --git a/include/migration/migration.h b/include/migration/migration.h
index ac2c12c..33da695 100644
--- a/include/migration/migration.h
+++ b/include/migration/migration.h
@@ -325,6 +325,7 @@ void global_state_store_running(void);
 void flush_page_queue(MigrationState *ms);
 int ram_save_queue_pages(MigrationState *ms, const char *rbname,
  ram_addr_t start, ram_addr_t len);
+int qemu_migrate_postcopy_outgoing_recovery(MigrationState *ms);
 
 PostcopyState postcopy_state_get(void);
 /* Set the state and return the old state */
diff --git a/migration/migration.c b/migration/migration.c
index 991313a..ee0c2a8 100644
--- a/migration/migration.c
+++ b/migration/migration.c
@@ -539,6 +539,7 @@ static bool migration_is_setup_or_active(int state)
 case MIGRATION_STATUS_ACTIVE:
 case MIGRATION_STATUS_POSTCOPY_ACTIVE:
 case MIGRATION_STATUS_SETUP:
+case MIGRATION_STATUS_POSTCOPY_RECOVERY:
 return true;
 
 default:
@@ -1634,6 +1635,8 @@ static void *migration_thread(void *opaque)
 /* The active state we expect to be in; ACTIVE or POSTCOPY_ACTIVE */
 enum MigrationStatus current_active_state = MIGRATION_STATUS_ACTIVE;
 
+int32_t ret;
+
 rcu_register_thread();
 
 qemu_savevm_state_header(s->to_dst_file);
@@ -1700,11 +1703,26 @@ static void *migration_thread(void *opaque)
 }
 }
 
-if (qemu_file_get_error(s->to_dst_file)) {
-migrate_set_state(>state, current_active_state,
-  MIGRATION_STATUS_FAILED);
-trace_migration_thread_file_err();
-break;
+if ((ret = qemu_file_get_error(s->to_dst_file))) {
+fprintf(stderr, "1 : Error %s %d\n", strerror(-ret), -ret);
+if(ret != -EIO && s->state == MIGRATION_STATUS_POSTCOPY_ACTIVE) {
+/* Network Failure during postcopy */
+
+current_active_state = MIGRATION_STATUS_POSTCOPY_RECOVERY;
+runstate_set(RUN_STATE_POSTMIGRATE_RECOVERY);
+fprintf(stderr, "1.1 : Error %s %d\n", strerror(-ret), -ret);
+ret = qemu_migrate_postcopy_outgoing_recovery(s);
+if(ret < 0) {
+break;
+}
+
+} else {
+migrate_set_state(>state, current_active_state,
+ MIGRATION_STATUS_FAILED);
+fprintf(stderr, "1.2 : Error %s %d\n", strerror(-ret), -ret);
+trace_migration_thread_file_err();
+break;
+}
 }
 current_time = qemu_clock_get_ms(QEMU_CLOCK_REALTIME);
 if (current_time >= initial_time + BUFFER_DELAY) {
@@ -1797,6 +1815,19 @@ void migrate_fd_connect(MigrationState *s)
 s->migration_thread_running = true;
 }
 
+int qemu_migrate_postcopy_outgoing_recovery(MigrationState* ms)
+{
+migrate_set_state(>state, MIGRATION_STATUS_POSTCOPY_ACTIVE,
+  MIGRATION_STATUS_POSTCOPY_RECOVERY);
+
+/* Code for network recovery to be added here */
+while(1) {
+fprintf(stderr, "Not letting it fail\n");
+sleep(2);
+}
+
+}
+
 PostcopyState  postcopy_state_get(void)
 {
 return atomic_mb_read(_postcopy_state);
diff --git a/vl.c b/vl.c
index 5fd22cb..c237140 100644
--- a/vl.c
+++ b/vl.c
@@ -618,6 +618,10 @@ static const RunStateTransition runstate_transitions_def[] 
= {
 { RUN_STATE_FINISH_MIGRATE, RUN_STATE_RUNNING },
 { RUN_STATE_FINISH_MIGRATE, RUN_STATE_POSTMIGRATE },
 { RUN_STATE_FINISH_MIGRATE, RUN_STATE_PRELAUNCH },
+{ RUN_STATE_FINISH_MIGRATE, RUN_STATE_POSTMIGRATE_RECOVERY },
+
+{ RUN_STATE_POSTMIGRATE_RECOVERY, RUN_STATE_FINISH_MIGRATE },
+{ RUN_STATE_POSTMIGRATE_RECOVERY, RUN_STATE_SHUTDOWN },
 
 { RUN_STATE_RESTORE_VM, RUN_STATE_RUNNING },
 { RUN_STATE_RESTORE_VM, RUN_STATE_PRELAUNCH },
-- 
2.7.4

Re: [Qemu-devel] [PATCH v5 1/4] Provide support for the CUSE TPM

[Dr. David Alan Gilbert]

* BICKFORD, JEFFREY E (jb6...@att.com) wrote:
> > * Daniel P. Berrange (berra...@redhat.com) wrote:
> > > On Wed, Jan 20, 2016 at 10:54:47AM -0500, Stefan Berger wrote:
> > > > On 01/20/2016 10:46 AM, Daniel P. Berrange wrote:
> > > > >On Wed, Jan 20, 2016 at 10:31:56AM -0500, Stefan Berger wrote:
> > > > >>"Daniel P. Berrange"  wrote on 01/20/2016 
> > > > >>10:00:41
> > > > >>AM:
> > > > >>
> > > > >>
> > > > >>>process at all - it would make sense if there was a single
> > > > >>>swtpm_cuse shared across all QEMU's, but if there's one per
> > > > >>>QEMU device, it feels like it'd be much simpler to just have
> > > > >>>the functionality linked in QEMU.  That avoids the problem
> > > > >>I tried having it linked in QEMU before. It was basically rejected.
> > > > >I remember an impl you did many years(?) ago now, but don't recall
> > > > >the results of the discussion. Can you elaborate on why it was
> > > > >rejected as an approach ? It just doesn't make much sense to me
> > > > >to have to create an external daemon, a CUSE device and comms
> > > > >protocol, simply to be able to read/write a plain file containing
> > > > >the TPM state. Its massive over engineering IMHO and adding way
> > > > >more complexity and thus scope for failure
> > > > 
> > > > The TPM 1.2 implementation adds 10s of thousands of lines of code. The 
> > > > TPM 2
> > > > implementation is in the same range. The concern was having this code 
> > > > right
> > > > in the QEMU address space. It's big, it can have bugs, so we don't want 
> > > > it
> > > > to harm QEMU. So we now put this into an external process implemented 
> > > > by the
> > > > swtpm project that builds on libtpms which provides TPM 1.2 
> > > > functionality
> > > > (to be extended with TPM 2). We cannot call APIs of libtpms directly
> > > > anymore, so we need a control channel, which is implemented through 
> > > > ioctls
> > > > on the CUSE device.
> > > 
> > > Ok, the security separation concern does make some sense. The use of CUSE
> > > still seems fairly questionable to me. CUSE makes sense if you want to
> > > provide a drop-in replacement for the kernel TPM device driver, which
> > > would avoid ned for a new QEMU backend. If you're not emulating an 
> > > existing
> > > kernel driver ABI though, CUSE + ioctl is feels like a really awful RPC
> > > transport between 2 userspace processes.
> 
> > While I don't really like CUSE; I can see some of the reasoning here.
> > By providing the existing TPM ioctl interface I think it means you can use
> > existing host-side TPM tools to initialise/query the soft-tpm, and those
> > should be independent of the soft-tpm implementation.
> > As for the extra interfaces you need because it's a soft-tpm to set it up,
> > once you've already got that ioctl interface as above, then it seems to make
> > sense to extend that to add the extra interfaces needed.  The only thing
> > you have to watch for there are that the extra interfaces don't clash
> > with any future kernel ioctl extensions, and that the interface defined
> > is generic enough for different soft-tpm implementations.
> 
> > Dave
> > Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
> 
> 
> Over the past several months, AT Security Research has been testing the 
> Virtual TPM software from IBM on the Power (ppc64) platform. Based on our 
> testing results, the vTPM software works well and as expected. Support for 
> libvirt and the CUSE TPM allows us to create VMs with the vTPM functionality 
> and was tested in a full-fledged OpenStack environment. 
>  
> We believe the vTPM functionality will improve various aspects of VM security 
> in our enterprise-grade cloud environment. AT would like to see these 
> patches accepted into the QEMU community as the default-standard build so 
> this technology can be easily adopted in various open source cloud 
> deployments.

Interesting; however, I see Stefan has been contributing other kernel
patches that create a different vTPM setup without the use of CUSE;
if that's the case then I guess that's the preferable solution.

Jeffrey: Can you detail a bit more about your setup, and how
you're maanging the life cycle of the vTPM data?

Dave

> 
> Regards,
> Jeffrey Bickford
> AT Security Research Center
> jbickf...@att.com
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK

[Qemu-devel] [RFC] A clock framework in QEMU.

[KONRAD Frederic]

Hi,

We would like to have a way to have a clock tree inside QEMU:
  * models can have clock outputs and/or clock inputs.
  * changing the clock rate of propagates in the clock tree through
callbacks which will be implemented in the model (eg: like qemu_irq)
  * would be nice to be able to visualize the rate of a clock in the
monitor.

There is already an implementation in QEMU (in omap*) but:
  * it's not generic/usable in the whole QEMU tree.
  * it's not using QOM.

So the proposition are either to construct one new framework or to 
extract and reuse the old one:

  * new types must be created eg: qemu_clk_in, qemu_clk_out.
  * I think it shouldn't use qemu_irq (because this is confusing) but
maybe use a simple qom link to bound them.
  * The model which have the clock input will need to implement the
clock update/enable/disable callback.

So for example PLL or some clock gate units will just have one input and 
some outputs.

Then the outputs can be controlled by the output callbacks (for example
the input rate change or the clock is gated).

Does that makes sense?
Do you have any opinion about that?

Thanks,
Fred

Re: [Qemu-devel] [RFC v2 PATCH 03/13] tcg/aarch64: Add support for fence

[Claudio Fontana]

Acked-by: Claudio Fontana 

On Tuesday, 31 May 2016, Pranith Kumar  wrote:

> Cc: Claudio Fontana >
> Signed-off-by: Richard Henderson >
> Signed-off-by: Pranith Kumar >
> ---
>  tcg/aarch64/tcg-target.inc.c | 7 +++
>  1 file changed, 7 insertions(+)
>
> diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c
> index 08efdf4..c361a5c 100644
> --- a/tcg/aarch64/tcg-target.inc.c
> +++ b/tcg/aarch64/tcg-target.inc.c
> @@ -360,6 +360,9 @@ typedef enum {
>  I3510_EOR   = 0x4a00,
>  I3510_EON   = 0x4a20,
>  I3510_ANDS  = 0x6a00,
> +
> +/* System instructions.  */
> +DMB_ISH = 0xd5033bbf,
>  } AArch64Insn;
>
>  static inline uint32_t tcg_in32(TCGContext *s)
> @@ -1625,6 +1628,9 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc,
>  tcg_out_insn(s, 3508, SMULH, TCG_TYPE_I64, a0, a1, a2);
>  break;
>
> +case INDEX_op_mb:
> +tcg_out32(s, DMB_ISH);
> +break;
>  case INDEX_op_mov_i32:  /* Always emitted via tcg_out_mov.  */
>  case INDEX_op_mov_i64:
>  case INDEX_op_movi_i32: /* Always emitted via tcg_out_movi.  */
> @@ -1749,6 +1755,7 @@ static const TCGTargetOpDef aarch64_op_defs[] = {
>  { INDEX_op_muluh_i64, { "r", "r", "r" } },
>  { INDEX_op_mulsh_i64, { "r", "r", "r" } },
>
> +{ INDEX_op_mb, { "r" } },
>  { -1 },
>  };
>
> --
> 2.8.3
>
>

--

Re: [Qemu-devel] [PATCH v4 1/1] Introduce "xen-load-devices-state"

[Eric Blake]

On 05/30/2016 09:17 AM, Stefano Stabellini wrote:
> On Fri, 27 May 2016, Anthony PERARD wrote:
>> On Mon, Apr 11, 2016 at 11:56:02AM +0800, Changlong Xie wrote:
>>> From: Wen Congyang 
>>>
>>> Introduce a "xen-load-devices-state" QAPI command that can be used to
>>> load the state of all devices, but not the RAM or the block devices of
>>> the VM.
>>>
>>> We only have hmp commands savevm/loadvm, and qmp commands
>>> xen-save-devices-state.
>>>
>>> We use this new command for COLO:
>>> 1. suspend both primary vm and secondary vm
>>> 2. sync the state
>>> 3. resume both primary vm and secondary vm
>>>
>>> In such case, we need to update all devices' state in any time.
>>>
>>> Signed-off-by: Wen Congyang 
>>> Signed-off-by: Changlong Xie 
>>
>> This patch looks good to me.
>>
>> Reviewed-by: Anthony PERARD 
> 
> It would be nicer (and less problematic) to load the state from a file
> descriptor, but given that we still saving the state to file, it would
> be unfair to ask to use file descriptors here.

If you use qemu_open(), then you can pass the magic "/dev/fdset/1"
notation to use a file descriptor previously added to an fdset.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH v5 1/4] Provide support for the CUSE TPM

[BICKFORD, JEFFREY E]

> * Daniel P. Berrange (berra...@redhat.com) wrote:
> > On Wed, Jan 20, 2016 at 10:54:47AM -0500, Stefan Berger wrote:
> > > On 01/20/2016 10:46 AM, Daniel P. Berrange wrote:
> > > >On Wed, Jan 20, 2016 at 10:31:56AM -0500, Stefan Berger wrote:
> > > >>"Daniel P. Berrange"  wrote on 01/20/2016 10:00:41
> > > >>AM:
> > > >>
> > > >>
> > > >>>process at all - it would make sense if there was a single
> > > >>>swtpm_cuse shared across all QEMU's, but if there's one per
> > > >>>QEMU device, it feels like it'd be much simpler to just have
> > > >>>the functionality linked in QEMU.  That avoids the problem
> > > >>I tried having it linked in QEMU before. It was basically rejected.
> > > >I remember an impl you did many years(?) ago now, but don't recall
> > > >the results of the discussion. Can you elaborate on why it was
> > > >rejected as an approach ? It just doesn't make much sense to me
> > > >to have to create an external daemon, a CUSE device and comms
> > > >protocol, simply to be able to read/write a plain file containing
> > > >the TPM state. Its massive over engineering IMHO and adding way
> > > >more complexity and thus scope for failure
> > > 
> > > The TPM 1.2 implementation adds 10s of thousands of lines of code. The 
> > > TPM 2
> > > implementation is in the same range. The concern was having this code 
> > > right
> > > in the QEMU address space. It's big, it can have bugs, so we don't want it
> > > to harm QEMU. So we now put this into an external process implemented by 
> > > the
> > > swtpm project that builds on libtpms which provides TPM 1.2 functionality
> > > (to be extended with TPM 2). We cannot call APIs of libtpms directly
> > > anymore, so we need a control channel, which is implemented through ioctls
> > > on the CUSE device.
> > 
> > Ok, the security separation concern does make some sense. The use of CUSE
> > still seems fairly questionable to me. CUSE makes sense if you want to
> > provide a drop-in replacement for the kernel TPM device driver, which
> > would avoid ned for a new QEMU backend. If you're not emulating an existing
> > kernel driver ABI though, CUSE + ioctl is feels like a really awful RPC
> > transport between 2 userspace processes.

> While I don't really like CUSE; I can see some of the reasoning here.
> By providing the existing TPM ioctl interface I think it means you can use
> existing host-side TPM tools to initialise/query the soft-tpm, and those
> should be independent of the soft-tpm implementation.
> As for the extra interfaces you need because it's a soft-tpm to set it up,
> once you've already got that ioctl interface as above, then it seems to make
> sense to extend that to add the extra interfaces needed.  The only thing
> you have to watch for there are that the extra interfaces don't clash
> with any future kernel ioctl extensions, and that the interface defined
> is generic enough for different soft-tpm implementations.

> Dave
> Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK


Over the past several months, AT Security Research has been testing the 
Virtual TPM software from IBM on the Power (ppc64) platform. Based on our 
testing results, the vTPM software works well and as expected. Support for 
libvirt and the CUSE TPM allows us to create VMs with the vTPM functionality 
and was tested in a full-fledged OpenStack environment. 
 
We believe the vTPM functionality will improve various aspects of VM security 
in our enterprise-grade cloud environment. AT would like to see these patches 
accepted into the QEMU community as the default-standard build so this 
technology can be easily adopted in various open source cloud deployments.

Regards,
Jeffrey Bickford
AT Security Research Center
jbickf...@att.com

[Qemu-devel] [RFC v2 PATCH 03/13] tcg/aarch64: Add support for fence

[Pranith Kumar]

Cc: Claudio Fontana 
Signed-off-by: Richard Henderson 
Signed-off-by: Pranith Kumar 
---
 tcg/aarch64/tcg-target.inc.c | 7 +++
 1 file changed, 7 insertions(+)

diff --git a/tcg/aarch64/tcg-target.inc.c b/tcg/aarch64/tcg-target.inc.c
index 08efdf4..c361a5c 100644
--- a/tcg/aarch64/tcg-target.inc.c
+++ b/tcg/aarch64/tcg-target.inc.c
@@ -360,6 +360,9 @@ typedef enum {
 I3510_EOR   = 0x4a00,
 I3510_EON   = 0x4a20,
 I3510_ANDS  = 0x6a00,
+
+/* System instructions.  */
+DMB_ISH = 0xd5033bbf,
 } AArch64Insn;
 
 static inline uint32_t tcg_in32(TCGContext *s)
@@ -1625,6 +1628,9 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc,
 tcg_out_insn(s, 3508, SMULH, TCG_TYPE_I64, a0, a1, a2);
 break;
 
+case INDEX_op_mb:
+tcg_out32(s, DMB_ISH);
+break;
 case INDEX_op_mov_i32:  /* Always emitted via tcg_out_mov.  */
 case INDEX_op_mov_i64:
 case INDEX_op_movi_i32: /* Always emitted via tcg_out_movi.  */
@@ -1749,6 +1755,7 @@ static const TCGTargetOpDef aarch64_op_defs[] = {
 { INDEX_op_muluh_i64, { "r", "r", "r" } },
 { INDEX_op_mulsh_i64, { "r", "r", "r" } },
 
+{ INDEX_op_mb, { "r" } },
 { -1 },
 };
 
-- 
2.8.3

[Qemu-devel] [RFC v2 PATCH 10/13] tcg/tci: Add support for fence

[Pranith Kumar]

Cc: Stefan Weil 
Signed-off-by: Richard Henderson 
Signed-off-by: Pranith Kumar 
---
 tcg/tci/tcg-target.inc.c | 3 +++
 tci.c| 3 +++
 2 files changed, 6 insertions(+)

diff --git a/tcg/tci/tcg-target.inc.c b/tcg/tci/tcg-target.inc.c
index 4e91687..a507ceb 100644
--- a/tcg/tci/tcg-target.inc.c
+++ b/tcg/tci/tcg-target.inc.c
@@ -255,6 +255,7 @@ static const TCGTargetOpDef tcg_target_op_defs[] = {
 { INDEX_op_bswap32_i32, { R, R } },
 #endif
 
+{ INDEX_op_mb, { "r" } },
 { -1 },
 };
 
@@ -798,6 +799,8 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, const 
TCGArg *args,
 }
 tcg_out_i(s, *args++);
 break;
+case INDEX_op_mb:
+break;
 case INDEX_op_mov_i32:  /* Always emitted via tcg_out_mov.  */
 case INDEX_op_mov_i64:
 case INDEX_op_movi_i32: /* Always emitted via tcg_out_movi.  */
diff --git a/tci.c b/tci.c
index 7cbb39e..d35b24f 100644
--- a/tci.c
+++ b/tci.c
@@ -1230,6 +1230,9 @@ uintptr_t tcg_qemu_tb_exec(CPUArchState *env, uint8_t 
*tb_ptr)
 tcg_abort();
 }
 break;
+case INDEX_op_mb:
+smp_mb();
+break;
 default:
 TODO();
 break;
-- 
2.8.3

Re: [Qemu-devel] [PATCH 02/10] qcow2: add qcow2_co_write_compressed

[Eric Blake]

On 05/30/2016 06:58 AM, Pavel Butsykin wrote:

> 
> Sorry, but it seems this will never happen, because the second write
> will not pass this check:
> 
> uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs,
>uint64_t offset,
>int compressed_size)
> {
> ...
> /* Compression can't overwrite anything. Fail if the cluster was
> already
>  * allocated. */
> cluster_offset = be64_to_cpu(l2_table[l2_index]);
> if (cluster_offset & L2E_OFFSET_MASK) {
> qcow2_cache_put(bs, s->l2_table_cache, (void**) _table);
> return 0;
> }
>...
> 
> As you can see we can't do the compressed write in the already allocated
> cluster.

Umm, doesn't that defeat the point of compression, if every compressed
cluster becomes the head of a new cluster?  The whole goal of
compression is to be able to fit multiple clusters within one.

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [RFC v2 PATCH 08/13] tcg/s390: Add support for fence

[Pranith Kumar]

Cc: Alexander Graf 
Signed-off-by: Richard Henderson 
Signed-off-by: Pranith Kumar 
---
 tcg/s390/tcg-target.inc.c | 9 +
 1 file changed, 9 insertions(+)

diff --git a/tcg/s390/tcg-target.inc.c b/tcg/s390/tcg-target.inc.c
index e95b04b..b4f14bc 100644
--- a/tcg/s390/tcg-target.inc.c
+++ b/tcg/s390/tcg-target.inc.c
@@ -341,6 +341,7 @@ static tcg_insn_unit *tb_ret_addr;
 #define FACILITY_EXT_IMM   (1ULL << (63 - 21))
 #define FACILITY_GEN_INST_EXT  (1ULL << (63 - 34))
 #define FACILITY_LOAD_ON_COND   (1ULL << (63 - 45))
+#define FACILITY_FAST_BCR_SER   FACILITY_LOAD_ON_COND
 
 static uint64_t facilities;
 
@@ -2157,6 +2158,13 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode 
opc,
 tgen_deposit(s, args[0], args[2], args[3], args[4]);
 break;
 
+case INDEX_op_mb:
+/* The host memory model is quite strong, we simply need to
+   serialize the instruction stream.  */
+tcg_out_insn(s, RR, BCR,
+facilities & FACILITY_FAST_BCR_SER ? 14 : 15, 0);
+break;
+
 case INDEX_op_mov_i32:  /* Always emitted via tcg_out_mov.  */
 case INDEX_op_mov_i64:
 case INDEX_op_movi_i32: /* Always emitted via tcg_out_movi.  */
@@ -2278,6 +2286,7 @@ static const TCGTargetOpDef s390_op_defs[] = {
 { INDEX_op_movcond_i64, { "r", "r", "rC", "r", "0" } },
 { INDEX_op_deposit_i64, { "r", "0", "r" } },
 
+{ INDEX_op_mb, { "r" } },
 { -1 },
 };
 
-- 
2.8.3

[Qemu-devel] [RFC v2 PATCH 09/13] tcg/sparc: Add support for fence

[Pranith Kumar]

Cc: Blue Swirl 
Signed-off-by: Richard Henderson 
Signed-off-by: Pranith Kumar 
---
 tcg/sparc/tcg-target.inc.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/tcg/sparc/tcg-target.inc.c b/tcg/sparc/tcg-target.inc.c
index a611885..81f263f 100644
--- a/tcg/sparc/tcg-target.inc.c
+++ b/tcg/sparc/tcg-target.inc.c
@@ -249,6 +249,8 @@ static const int tcg_target_call_oarg_regs[] = {
 #define STWA   (INSN_OP(3) | INSN_OP3(0x14))
 #define STXA   (INSN_OP(3) | INSN_OP3(0x1e))
 
+#define MEMBAR (INSN_OP(2) | INSN_OP3(0x28) | INSN_RS1(15) | (1 << 13))
+
 #ifndef ASI_PRIMARY_LITTLE
 #define ASI_PRIMARY_LITTLE 0x88
 #endif
@@ -1450,6 +1452,11 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc,
tcg_out_arithc(s, a0, TCG_REG_G0, a1, const_args[1], c);
break;
 
+case INDEX_op_mb:
+/* membar #LoadLoad|#LoadStore|#StoreStore|#StoreLoad */
+tcg_out32(s, MEMBAR | 15);
+break;
+
 case INDEX_op_mov_i32:  /* Always emitted via tcg_out_mov.  */
 case INDEX_op_mov_i64:
 case INDEX_op_movi_i32: /* Always emitted via tcg_out_movi.  */
@@ -1551,6 +1558,7 @@ static const TCGTargetOpDef sparc_op_defs[] = {
 { INDEX_op_qemu_st_i32, { "sZ", "A" } },
 { INDEX_op_qemu_st_i64, { "SZ", "A" } },
 
+{ INDEX_op_mb, { "r" } },
 { -1 },
 };
 
-- 
2.8.3

Re: [Qemu-devel] [RFC v2 PATCH 00/13] tcg: Add fence gen support

[Pranith Kumar]

Added correct email for Sergey in CC.

I apologize for getting Sergey's email wrong. Please drop/correct his
email when replying to the patches in this series otherwise you will
see an email bounce.

On Tue, May 31, 2016 at 2:39 PM, Pranith Kumar  wrote:
> Hello,
>
> The following series adds fence instruction generation support to
> TCG. The current work has been rebased on-top of Richard's patch
> series.
>
> This has been tested and confirmed to fix ordering issues on a x86
> host with MTTCG enabled ARMv7 guest using KVM unit tests.
>
> Pranith Kumar (13):
>   Introduce TCGOpcode for memory barrier
>   tcg/i386: Add support for fence
>   tcg/aarch64: Add support for fence
>   tcg/arm: Add support for fence
>   tcg/ia64: Add support for fence
>   tcg/mips: Add support for fence
>   tcg/ppc: Add support for fence
>   tcg/s390: Add support for fence
>   tcg/sparc: Add support for fence
>   tcg/tci: Add support for fence
>   target-arm: Generate fences in ARMv7 frontend
>   target-alpha: Generate fence op
>   tcg: Generate fences only for SMP MTTCG guests
>
>  target-alpha/translate.c |  5 +++--
>  target-arm/translate.c   |  7 +--
>  tcg/README   | 17 +
>  tcg/aarch64/tcg-target.inc.c |  7 +++
>  tcg/arm/tcg-target.inc.c | 12 
>  tcg/i386/tcg-target.inc.c| 35 +++
>  tcg/ia64/tcg-target.inc.c|  5 +
>  tcg/mips/tcg-target.inc.c|  6 ++
>  tcg/ppc/tcg-target.inc.c |  8 
>  tcg/s390/tcg-target.inc.c|  9 +
>  tcg/sparc/tcg-target.inc.c   |  8 
>  tcg/tcg-op.c |  9 +
>  tcg/tcg-op.h |  2 ++
>  tcg/tcg-opc.h|  2 ++
>  tcg/tcg.h|  8 
>  tcg/tci/tcg-target.inc.c |  3 +++
>  tci.c|  3 +++
>  17 files changed, 142 insertions(+), 4 deletions(-)
>
> --
> 2.8.3
>



-- 
Pranith

[Qemu-devel] [RFC v2 PATCH 12/13] target-alpha: Generate fence op

[Pranith Kumar]

Signed-off-by: Richard Henderson 
Signed-off-by: Pranith Kumar 
---
 target-alpha/translate.c | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/target-alpha/translate.c b/target-alpha/translate.c
index 5b86992..17b68f5 100644
--- a/target-alpha/translate.c
+++ b/target-alpha/translate.c
@@ -2329,11 +2329,12 @@ static ExitStatus translate_one(DisasContext *ctx, 
uint32_t insn)
 break;
 case 0x4000:
 /* MB */
-/* No-op */
+tcg_gen_mb(TCG_MB_FULL);
 break;
 case 0x4400:
 /* WMB */
-/* No-op */
+/* TODO: Change this to write barrier */
+tcg_gen_mb(TCG_MB_FULL);
 break;
 case 0x8000:
 /* FETCH */
-- 
2.8.3

[Qemu-devel] [RFC v2 PATCH 02/13] tcg/i386: Add support for fence

[Pranith Kumar]

Generate mfence instruction on SSE2 enabled processors. For older
processors, generate a 'lock orl $0,0(%esp)' instruction which has
similar ordering semantics.

Signed-off-by: Pranith Kumar 
[rth: Check for sse2, fallback to locked memory op otherwise.]
Signed-off-by: Richard Henderson 

Signed-off-by: Pranith Kumar 
---
 tcg/i386/tcg-target.inc.c | 35 +++
 1 file changed, 35 insertions(+)

diff --git a/tcg/i386/tcg-target.inc.c b/tcg/i386/tcg-target.inc.c
index 8fd37f4..1fd5a99 100644
--- a/tcg/i386/tcg-target.inc.c
+++ b/tcg/i386/tcg-target.inc.c
@@ -121,6 +121,16 @@ static bool have_cmov;
 # define have_cmov 0
 #endif
 
+/* For 32-bit, we are going to attempt to determine at runtime whether
+   sse2 support is available.  */
+#if TCG_TARGET_REG_BITS == 64 || defined(__SSE2__)
+# define have_sse2 1
+#elif defined(CONFIG_CPUID_H) && defined(bit_SSE2)
+static bool have_sse2;
+#else
+# define have_sse2 0
+#endif
+
 /* If bit_MOVBE is defined in cpuid.h (added in GCC version 4.6), we are
going to attempt to determine at runtime whether movbe is available.  */
 #if defined(CONFIG_CPUID_H) && defined(bit_MOVBE)
@@ -686,6 +696,21 @@ static inline void tcg_out_pushi(TCGContext *s, 
tcg_target_long val)
 }
 }
 
+static inline void tcg_out_mb(TCGContext *s)
+{
+if (have_sse2) {
+/* mfence */
+tcg_out8(s, 0x0f);
+tcg_out8(s, 0xae);
+tcg_out8(s, 0xf0);
+} else {
+/* lock orl $0,0(%esp) */
+tcg_out8(s, 0xf0);
+tcg_out_modrm_offset(s, OPC_ARITH_EvIb, ARITH_OR, TCG_REG_ESP, 0);
+tcg_out8(s, 0);
+}
+}
+
 static inline void tcg_out_push(TCGContext *s, int reg)
 {
 tcg_out_opc(s, OPC_PUSH_r32 + LOWREGMASK(reg), 0, reg, 0);
@@ -2114,6 +2139,9 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode 
opc,
 }
 break;
 
+case INDEX_op_mb:
+tcg_out_mb(s);
+break;
 case INDEX_op_mov_i32:  /* Always emitted via tcg_out_mov.  */
 case INDEX_op_mov_i64:
 case INDEX_op_movi_i32: /* Always emitted via tcg_out_movi.  */
@@ -2179,6 +2207,8 @@ static const TCGTargetOpDef x86_op_defs[] = {
 { INDEX_op_add2_i32, { "r", "r", "0", "1", "ri", "ri" } },
 { INDEX_op_sub2_i32, { "r", "r", "0", "1", "ri", "ri" } },
 
+{ INDEX_op_mb, { "r" } },
+
 #if TCG_TARGET_REG_BITS == 32
 { INDEX_op_brcond2_i32, { "r", "r", "ri", "ri" } },
 { INDEX_op_setcond2_i32, { "r", "r", "r", "ri", "ri" } },
@@ -2356,6 +2386,11 @@ static void tcg_target_init(TCGContext *s)
available, we'll use a small forward branch.  */
 have_cmov = (d & bit_CMOV) != 0;
 #endif
+#ifndef have_sse2
+/* Likewise, almost all hardware supports SSE2, but we do
+   have a locked memory operation to use as a substitute.  */
+have_sse2 = (d & bit_SSE2) != 0;
+#endif
 #ifndef have_movbe
 /* MOVBE is only available on Intel Atom and Haswell CPUs, so we
need to probe for it.  */
-- 
2.8.3

[Qemu-devel] [RFC v2 PATCH 04/13] tcg/arm: Add support for fence

[Pranith Kumar]

Cc: Andrzej Zaborowski 
Cc: Peter Maydell 
Signed-off-by: Pranith Kumar 
Signed-off-by: Richard Henderson 
---
 tcg/arm/tcg-target.inc.c | 12 
 1 file changed, 12 insertions(+)

diff --git a/tcg/arm/tcg-target.inc.c b/tcg/arm/tcg-target.inc.c
index a914762..e88d8ce 100644
--- a/tcg/arm/tcg-target.inc.c
+++ b/tcg/arm/tcg-target.inc.c
@@ -305,6 +305,10 @@ typedef enum {
 INSN_LDRD_REG  = 0x00d0,
 INSN_STRD_IMM  = 0x004000f0,
 INSN_STRD_REG  = 0x00f0,
+
+INSN_DMB_ISH   = 0x5bf07ff5,
+INSN_DMB_MCR   = 0xba0f07ee,
+
 } ARMInsn;
 
 #define SHIFT_IMM_LSL(im)  (((im) << 7) | 0x00)
@@ -1905,6 +1909,13 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode 
opc,
 tcg_out_udiv(s, COND_AL, args[0], args[1], args[2]);
 break;
 
+case INDEX_op_mb:
+if (use_armv7_instructions) {
+tcg_out32(s, INSN_DMB_ISH);
+} else if (use_armv6_instructions) {
+tcg_out32(s, INSN_DMB_MCR);
+}
+break;
 case INDEX_op_mov_i32:  /* Always emitted via tcg_out_mov.  */
 case INDEX_op_movi_i32: /* Always emitted via tcg_out_movi.  */
 case INDEX_op_call: /* Always emitted via tcg_out_call.  */
@@ -1979,6 +1990,7 @@ static const TCGTargetOpDef arm_op_defs[] = {
 { INDEX_op_div_i32, { "r", "r", "r" } },
 { INDEX_op_divu_i32, { "r", "r", "r" } },
 
+{ INDEX_op_mb, { "r" } },
 { -1 },
 };
 
-- 
2.8.3

[Qemu-devel] [RFC v2 PATCH 01/13] Introduce TCGOpcode for memory barrier

[Pranith Kumar]

This commit introduces the TCGOpcode for memory barrier instruction.

This opcode takes an argument which is the type of memory barrier
which should be generated.

Signed-off-by: Pranith Kumar 
Signed-off-by: Richard Henderson 
---
 tcg/README| 17 +
 tcg/tcg-op.c  |  6 ++
 tcg/tcg-op.h  |  2 ++
 tcg/tcg-opc.h |  2 ++
 tcg/tcg.h |  8 
 5 files changed, 35 insertions(+)

diff --git a/tcg/README b/tcg/README
index f4a8ac1..cfe79d7 100644
--- a/tcg/README
+++ b/tcg/README
@@ -402,6 +402,23 @@ double-word product T0.  The later is returned in two 
single-word outputs.
 
 Similar to mulu2, except the two inputs T1 and T2 are signed.
 
+* Memory Barrier support
+
+* mb <$arg>
+
+Generate a target memory barrier instruction to ensure memory ordering as being
+enforced by a corresponding guest memory barrier instruction. The ordering
+enforced by the backend may be stricter than the ordering required by the 
guest.
+It cannot be weaker. This opcode takes an optional constant argument if 
required
+to generate the appropriate barrier instruction. The backend should take care 
to
+emit the target barrier instruction only when necessary i.e., for SMP guests 
and
+when MTTCG is enabled.
+
+The guest translators should generate this opcode for all guest instructions
+which have ordering side effects.
+
+Please see docs/atomics.txt for more information on memory barriers.
+
 * 64-bit guest on 32-bit host support
 
 The following opcodes are internal to TCG.  Thus they are to be implemented by
diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c
index f554b86..a6f01a7 100644
--- a/tcg/tcg-op.c
+++ b/tcg/tcg-op.c
@@ -143,6 +143,12 @@ void tcg_gen_op6(TCGContext *ctx, TCGOpcode opc, TCGArg 
a1, TCGArg a2,
 tcg_emit_op(ctx, opc, pi);
 }
 
+void tcg_gen_mb(TCGArg a)
+{
+/* ??? Enable only when MTTCG is enabled.  */
+tcg_gen_op1(_ctx, INDEX_op_mb, 0);
+}
+
 /* 32 bit ops */
 
 void tcg_gen_addi_i32(TCGv_i32 ret, TCGv_i32 arg1, int32_t arg2)
diff --git a/tcg/tcg-op.h b/tcg/tcg-op.h
index c446d3d..40920fb 100644
--- a/tcg/tcg-op.h
+++ b/tcg/tcg-op.h
@@ -261,6 +261,8 @@ static inline void tcg_gen_br(TCGLabel *l)
 tcg_gen_op1(_ctx, INDEX_op_br, label_arg(l));
 }
 
+void tcg_gen_mb(TCGArg a);
+
 /* Helper calls. */
 
 /* 32 bit ops */
diff --git a/tcg/tcg-opc.h b/tcg/tcg-opc.h
index 6d0410c..c0f3e83 100644
--- a/tcg/tcg-opc.h
+++ b/tcg/tcg-opc.h
@@ -42,6 +42,8 @@ DEF(br, 0, 0, 1, TCG_OPF_BB_END)
 # define IMPL64  TCG_OPF_64BIT
 #endif
 
+DEF(mb, 0, 1, 0, 0)
+
 DEF(mov_i32, 1, 1, 0, TCG_OPF_NOT_PRESENT)
 DEF(movi_i32, 1, 0, 1, TCG_OPF_NOT_PRESENT)
 DEF(setcond_i32, 1, 2, 1, 0)
diff --git a/tcg/tcg.h b/tcg/tcg.h
index a46d17c..a1d59f7 100644
--- a/tcg/tcg.h
+++ b/tcg/tcg.h
@@ -385,6 +385,14 @@ static inline intptr_t QEMU_ARTIFICIAL 
GET_TCGV_PTR(TCGv_ptr t)
 #define TCG_CALL_DUMMY_TCGV MAKE_TCGV_I32(-1)
 #define TCG_CALL_DUMMY_ARG  ((TCGArg)(-1))
 
+/* TCGOpmb args */
+#define TCG_MB_FULL ((TCGArg)(0))
+#define TCG_MB_READ ((TCGArg)(1))
+#define TCG_MB_WRITE((TCGArg)(2))
+#define TCG_MB_ACQUIRE  ((TCGArg)(3))
+#define TCG_MB_RELEASE  ((TCGArg)(4))
+
+
 /* Conditions.  Note that these are laid out for easy manipulation by
the functions below:
  bit 0 is used for inverting;
-- 
2.8.3

[Qemu-devel] [RFC v2 PATCH 11/13] target-arm: Generate fences in ARMv7 frontend

[Pranith Kumar]

Signed-off-by: Pranith Kumar 
Signed-off-by: Richard Henderson 
---
 target-arm/translate.c | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/target-arm/translate.c b/target-arm/translate.c
index c946c0e..e1b16c0 100644
--- a/target-arm/translate.c
+++ b/target-arm/translate.c
@@ -7980,9 +7980,11 @@ static void disas_arm_insn(DisasContext *s, unsigned int 
insn)
 gen_clrex(s);
 return;
 case 4: /* dsb */
+ARCH(7);
+return;
 case 5: /* dmb */
 ARCH(7);
-/* We don't emulate caches so these are a no-op.  */
+tcg_gen_mb(TCG_MB_FULL);
 return;
 case 6: /* isb */
 /* We need to break the TB after this insn to execute
@@ -10330,8 +10332,9 @@ static int disas_thumb2_insn(CPUARMState *env, 
DisasContext *s, uint16_t insn_hw
 gen_clrex(s);
 break;
 case 4: /* dsb */
+break;
 case 5: /* dmb */
-/* These execute as NOPs.  */
+tcg_gen_mb(TCG_MB_FULL);
 break;
 case 6: /* isb */
 /* We need to break the TB after this insn
-- 
2.8.3

[Qemu-devel] [RFC v2 PATCH 05/13] tcg/ia64: Add support for fence

[Pranith Kumar]

Cc: Aurelien Jarno 
Signed-off-by: Richard Henderson 
Signed-off-by: Pranith Kumar 
---
 tcg/ia64/tcg-target.inc.c | 5 +
 1 file changed, 5 insertions(+)

diff --git a/tcg/ia64/tcg-target.inc.c b/tcg/ia64/tcg-target.inc.c
index 261861f..88cc560 100644
--- a/tcg/ia64/tcg-target.inc.c
+++ b/tcg/ia64/tcg-target.inc.c
@@ -247,6 +247,7 @@ enum {
 OPC_LD4_M3= 0x0a08000ull,
 OPC_LD8_M1= 0x080c000ull,
 OPC_LD8_M3= 0x0a0c000ull,
+OPC_MF_M24= 0x0011000ull,
 OPC_MUX1_I3   = 0x0eca000ull,
 OPC_NOP_B9= 0x0400800ull,
 OPC_NOP_F16   = 0x800ull,
@@ -2213,6 +2214,9 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode 
opc,
 tcg_out_qemu_st(s, args);
 break;
 
+case INDEX_op_mb:
+tcg_out_bundle(s, mmI, OPC_MF_M24, INSN_NOP_M, INSN_NOP_I);
+break;
 case INDEX_op_mov_i32:  /* Always emitted via tcg_out_mov.  */
 case INDEX_op_mov_i64:
 case INDEX_op_movi_i32: /* Always emitted via tcg_out_movi.  */
@@ -2326,6 +2330,7 @@ static const TCGTargetOpDef ia64_op_defs[] = {
 { INDEX_op_qemu_st_i32, { "SZ", "r" } },
 { INDEX_op_qemu_st_i64, { "SZ", "r" } },
 
+{ INDEX_op_mb, { "r" } },
 { -1 },
 };
 
-- 
2.8.3

[Qemu-devel] [RFC v2 PATCH 13/13] tcg: Generate fences only for SMP MTTCG guests

[Pranith Kumar]

We need to generate fence instructions only for SMP MTTCG guests. This
patch enforces that.

Signed-off-by: Pranith Kumar 
---
 tcg/tcg-op.c | 7 +--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/tcg/tcg-op.c b/tcg/tcg-op.c
index a6f01a7..eeb0d0c 100644
--- a/tcg/tcg-op.c
+++ b/tcg/tcg-op.c
@@ -36,6 +36,8 @@ extern TCGv_i32 TCGV_HIGH_link_error(TCGv_i64);
 #define TCGV_HIGH TCGV_HIGH_link_error
 #endif
 
+extern int smp_cpus;
+
 /* Note that this is optimized for sequential allocation during translate.
Up to and including filling in the forward link immediately.  We'll do
proper termination of the end of the list after we finish translation.  */
@@ -145,8 +147,9 @@ void tcg_gen_op6(TCGContext *ctx, TCGOpcode opc, TCGArg a1, 
TCGArg a2,
 
 void tcg_gen_mb(TCGArg a)
 {
-/* ??? Enable only when MTTCG is enabled.  */
-tcg_gen_op1(_ctx, INDEX_op_mb, 0);
+if (qemu_tcg_mttcg_enabled() && smp_cpus > 1) {
+tcg_gen_op1(_ctx, INDEX_op_mb, 0);
+}
 }
 
 /* 32 bit ops */
-- 
2.8.3

[Qemu-devel] [RFC v2 PATCH 00/13] tcg: Add fence gen support

[Pranith Kumar]

Hello,

The following series adds fence instruction generation support to
TCG. The current work has been rebased on-top of Richard's patch
series.

This has been tested and confirmed to fix ordering issues on a x86
host with MTTCG enabled ARMv7 guest using KVM unit tests.

Pranith Kumar (13):
  Introduce TCGOpcode for memory barrier
  tcg/i386: Add support for fence
  tcg/aarch64: Add support for fence
  tcg/arm: Add support for fence
  tcg/ia64: Add support for fence
  tcg/mips: Add support for fence
  tcg/ppc: Add support for fence
  tcg/s390: Add support for fence
  tcg/sparc: Add support for fence
  tcg/tci: Add support for fence
  target-arm: Generate fences in ARMv7 frontend
  target-alpha: Generate fence op
  tcg: Generate fences only for SMP MTTCG guests

 target-alpha/translate.c |  5 +++--
 target-arm/translate.c   |  7 +--
 tcg/README   | 17 +
 tcg/aarch64/tcg-target.inc.c |  7 +++
 tcg/arm/tcg-target.inc.c | 12 
 tcg/i386/tcg-target.inc.c| 35 +++
 tcg/ia64/tcg-target.inc.c|  5 +
 tcg/mips/tcg-target.inc.c|  6 ++
 tcg/ppc/tcg-target.inc.c |  8 
 tcg/s390/tcg-target.inc.c|  9 +
 tcg/sparc/tcg-target.inc.c   |  8 
 tcg/tcg-op.c |  9 +
 tcg/tcg-op.h |  2 ++
 tcg/tcg-opc.h|  2 ++
 tcg/tcg.h|  8 
 tcg/tci/tcg-target.inc.c |  3 +++
 tci.c|  3 +++
 17 files changed, 142 insertions(+), 4 deletions(-)

-- 
2.8.3

[Qemu-devel] [RFC v2 PATCH 06/13] tcg/mips: Add support for fence

[Pranith Kumar]

Signed-off-by: Richard Henderson 
Signed-off-by: Pranith Kumar 
---
 tcg/mips/tcg-target.inc.c | 6 ++
 1 file changed, 6 insertions(+)

diff --git a/tcg/mips/tcg-target.inc.c b/tcg/mips/tcg-target.inc.c
index b2a839a..fc9c7fb 100644
--- a/tcg/mips/tcg-target.inc.c
+++ b/tcg/mips/tcg-target.inc.c
@@ -292,6 +292,7 @@ typedef enum {
 OPC_JALR = OPC_SPECIAL | 0x09,
 OPC_MOVZ = OPC_SPECIAL | 0x0A,
 OPC_MOVN = OPC_SPECIAL | 0x0B,
+OPC_SYNC = OPC_SPECIAL | 0x0F,
 OPC_MFHI = OPC_SPECIAL | 0x10,
 OPC_MFLO = OPC_SPECIAL | 0x12,
 OPC_MULT = OPC_SPECIAL | 0x18,
@@ -1636,6 +1637,9 @@ static inline void tcg_out_op(TCGContext *s, TCGOpcode 
opc,
 const_args[4], const_args[5], true);
 break;
 
+case INDEX_op_mb:
+tcg_out32(s, OPC_SYNC);
+break;
 case INDEX_op_mov_i32:  /* Always emitted via tcg_out_mov.  */
 case INDEX_op_movi_i32: /* Always emitted via tcg_out_movi.  */
 case INDEX_op_call: /* Always emitted via tcg_out_call.  */
@@ -1716,6 +1720,8 @@ static const TCGTargetOpDef mips_op_defs[] = {
 { INDEX_op_qemu_ld_i64, { "L", "L", "lZ", "lZ" } },
 { INDEX_op_qemu_st_i64, { "SZ", "SZ", "SZ", "SZ" } },
 #endif
+
+{ INDEX_op_mb, { "r" } },
 { -1 },
 };
 
-- 
2.8.3

[Qemu-devel] [RFC v2 PATCH 07/13] tcg/ppc: Add support for fence

[Pranith Kumar]

Signed-off-by: Richard Henderson 
Signed-off-by: Pranith Kumar 
---
 tcg/ppc/tcg-target.inc.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/tcg/ppc/tcg-target.inc.c b/tcg/ppc/tcg-target.inc.c
index 1039407..45a667f 100644
--- a/tcg/ppc/tcg-target.inc.c
+++ b/tcg/ppc/tcg-target.inc.c
@@ -469,6 +469,9 @@ static int tcg_target_const_match(tcg_target_long val, 
TCGType type,
 #define STHX   XO31(407)
 #define STWX   XO31(151)
 
+#define HWSYNC XO31(598)
+#define LWSYNC (HWSYNC | (1u << 21))
+
 #define SPR(a, b) a)<<5)|(b))<<11)
 #define LR SPR(8, 0)
 #define CTRSPR(9, 0)
@@ -2425,6 +2428,10 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc, 
const TCGArg *args,
 tcg_out32(s, MULHD | TAB(args[0], args[1], args[2]));
 break;
 
+case INDEX_op_mb:
+/* ??? Do we want SEQ_CST or ACQ_REL memory model.  */
+tcg_out32(s, HWSYNC);
+break;
 case INDEX_op_mov_i32:   /* Always emitted via tcg_out_mov.  */
 case INDEX_op_mov_i64:
 case INDEX_op_movi_i32:  /* Always emitted via tcg_out_movi.  */
@@ -2572,6 +2579,7 @@ static const TCGTargetOpDef ppc_op_defs[] = {
 { INDEX_op_qemu_st_i64, { "S", "S", "S", "S" } },
 #endif
 
+{ INDEX_op_mb, { "r" } },
 { -1 },
 };
 
-- 
2.8.3

[Qemu-devel] [PATCH] host-utils: Prefer 'false' for bool type

[Eric Blake]

Mixing '0' and 'bool' looks stupid.

Signed-off-by: Eric Blake 
---
 include/qemu/host-utils.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/include/qemu/host-utils.h b/include/qemu/host-utils.h
index 1cdae0d..3de7d4e 100644
--- a/include/qemu/host-utils.h
+++ b/include/qemu/host-utils.h
@@ -486,7 +486,7 @@ static inline uint64_t revbit64(uint64_t x)
 static inline bool is_power_of_2(uint64_t value)
 {
 if (!value) {
-return 0;
+return false;
 }

 return !(value & (value - 1));
-- 
2.5.5

Re: [Qemu-devel] [PATCH] block: assert that bs->request_alignment is a power of 2

[Eric Blake]

On 05/30/2016 05:59 AM, Peter Lieven wrote:
> at least bdrv_co_preadv/pwritev expect this.
> 
> Signed-off-by: Peter Lieven 
> ---
>  block.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Reviewed-by: Eric Blake 

And my work on converting write_zeroes to a byte interface will also
exploit it.


-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

Re: [Qemu-devel] [PATCH 0/6] *** Vhost-pci RFC ***

[Eric Blake]

On 05/28/2016 05:36 PM, Wei Wang wrote:
> This RFC proposes a design of vhost-pci, which is a new virtio device type.
> The vhost-pci device is used for inter-VM communication. Please read the RFC
> patches for details.
> 
> 
> Wei Wang (6):
>   Vhost-pci RFC: Introduction
>   Vhost-pci RFC: Modification Scope
>   Vhost-pci RFC: Benefits to KVM
>   Vhost-pci RFC: Detailed Description in the Virtio Specification Format
>   Vhost-pci RFC: Future Security Enhancement
>   Vhost-pci RFC: Experimental Results
> 
>  Benefits  |   8 ++
>  Details   | 324 
> ++
>  FutureWorks   |  21 
>  Introduction  |  31 ++
>  ModificationScope |   3 +
>  Results   |  18 +++

Umm, are you really creating 6 new files?  Shouldn't this just be a
single patch, as a single file, under the docs/ subdirectory?

>  6 files changed, 405 insertions(+)
>  create mode 100644 Benefits
>  create mode 100644 Details
>  create mode 100644 FutureWorks
>  create mode 100644 Introduction
>  create mode 100644 ModificationScope
>  create mode 100644 Results
> 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [QEMU RFC PATCH v3 5/6] Migration: migrate ccs_list in spapr state

[Jianjun Duan]

ccs_list in spapr state maintains the device tree related
information on the rtas side for hotplugged devices. In racing
situations between hotplug events and migration operation, a rtas
hotplug event could be migrated from the source guest to target
guest, or the source guest could have not yet finished fetching
the device tree when migration is started, the target will try
to finish fetching the device tree. By migrating ccs_list, the
target can fetch the device tree properly.

ccs_list is put in a subsection in the spapr state VMSD to make
sure migration across different versions is not broken.

Signed-off-by: Jianjun Duan 
---
 hw/ppc/spapr.c | 34 ++
 1 file changed, 34 insertions(+)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index 44e401a..f13584c 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1266,6 +1266,36 @@ static bool version_before_3(void *opaque, int 
version_id)
 return version_id < 3;
 }
 
+static bool spapr_ccs_list_needed(void *opaque)
+{
+sPAPRMachineState *spapr = (sPAPRMachineState *)opaque;
+return !QTAILQ_EMPTY(>ccs_list);
+}
+
+static const VMStateDescription vmstate_spapr_ccs = {
+.name = "spaprconfigureconnectorstate",
+.version_id = 1,
+.minimum_version_id = 1,
+.fields = (VMStateField[]) {
+VMSTATE_UINT32(drc_index, sPAPRConfigureConnectorState),
+VMSTATE_INT32(fdt_offset, sPAPRConfigureConnectorState),
+VMSTATE_INT32(fdt_depth, sPAPRConfigureConnectorState),
+VMSTATE_END_OF_LIST()
+},
+};
+
+static const VMStateDescription vmstate_spapr_ccs_list = {
+.name = "spaprccslist",
+.version_id = 1,
+.minimum_version_id = 1,
+.needed = spapr_ccs_list_needed,
+.fields = (VMStateField[]) {
+VMSTATE_QTAILQ_V(ccs_list, sPAPRMachineState, 1,
+ vmstate_spapr_ccs, sPAPRConfigureConnectorState, 
next),
+VMSTATE_END_OF_LIST()
+},
+};
+
 static const VMStateDescription vmstate_spapr = {
 .name = "spapr",
 .version_id = 3,
@@ -1281,6 +1311,10 @@ static const VMStateDescription vmstate_spapr = {
 VMSTATE_PPC_TIMEBASE_V(tb, sPAPRMachineState, 2),
 VMSTATE_END_OF_LIST()
 },
+.subsections = (const VMStateDescription*[]) {
+_spapr_ccs_list,
+NULL
+}
 };
 
 static int htab_save_setup(QEMUFile *f, void *opaque)
-- 
1.9.1

Re: [Qemu-devel] [PATCH] block/io: Remove unused bdrv_aio_write_zeroes()

[Eric Blake]

On 05/30/2016 03:17 AM, Kevin Wolf wrote:
> Signed-off-by: Kevin Wolf 
> ---
>  block/io.c| 11 ---
>  include/block/block.h |  3 ---
>  trace-events  |  1 -
>  3 files changed, 15 deletions(-)

Reviewed-by: Eric Blake 

-- 
Eric Blake   eblake redhat com+1-919-301-3266
Libvirt virtualization library http://libvirt.org



signature.asc
Description: OpenPGP digital signature

[Qemu-devel] [QEMU RFC PATCH v3 3/6] Migration: extend VMStateInfo

[Jianjun Duan]

Current migration code cannot handle some data structures such as
QTAILQ in qemu/queue.h. Here we extend the signatures of put/get
in VMStateInfo so that customized handling is supported.

Signed-off-by: Jianjun Duan 
---
 hw/net/vmxnet3.c| 18 ++---
 hw/nvram/eeprom93xx.c   |  6 ++-
 hw/nvram/fw_cfg.c   |  6 ++-
 hw/pci/msix.c   |  6 ++-
 hw/pci/pci.c| 12 --
 hw/pci/shpc.c   |  5 ++-
 hw/scsi/scsi-bus.c  |  6 ++-
 hw/timer/twl92230.c |  6 ++-
 hw/usb/redirect.c   | 18 ++---
 hw/virtio/virtio-pci.c  |  6 ++-
 hw/virtio/virtio.c  |  6 ++-
 include/migration/vmstate.h | 10 +++--
 migration/savevm.c  |  5 ++-
 migration/vmstate.c | 95 -
 target-alpha/machine.c  |  5 ++-
 target-arm/machine.c| 12 --
 target-i386/machine.c   | 21 ++
 target-mips/machine.c   | 10 +++--
 target-ppc/machine.c| 10 +++--
 target-sparc/machine.c  |  5 ++-
 20 files changed, 171 insertions(+), 97 deletions(-)

diff --git a/hw/net/vmxnet3.c b/hw/net/vmxnet3.c
index 20f26b7..7ba77a4 100644
--- a/hw/net/vmxnet3.c
+++ b/hw/net/vmxnet3.c
@@ -2439,7 +2439,8 @@ static void vmxnet3_put_tx_stats_to_file(QEMUFile *f,
 qemu_put_be64(f, tx_stat->pktsTxDiscard);
 }
 
-static int vmxnet3_get_txq_descr(QEMUFile *f, void *pv, size_t size)
+static int vmxnet3_get_txq_descr(QEMUFile *f, void *pv, size_t size,
+VMStateField *field)
 {
 Vmxnet3TxqDescr *r = pv;
 
@@ -2453,7 +2454,8 @@ static int vmxnet3_get_txq_descr(QEMUFile *f, void *pv, 
size_t size)
 return 0;
 }
 
-static void vmxnet3_put_txq_descr(QEMUFile *f, void *pv, size_t size)
+static void vmxnet3_put_txq_descr(QEMUFile *f, void *pv, size_t size,
+VMStateField *field, QJSON *vmdesc)
 {
 Vmxnet3TxqDescr *r = pv;
 
@@ -2500,7 +2502,8 @@ static void vmxnet3_put_rx_stats_to_file(QEMUFile *f,
 qemu_put_be64(f, rx_stat->pktsRxError);
 }
 
-static int vmxnet3_get_rxq_descr(QEMUFile *f, void *pv, size_t size)
+static int vmxnet3_get_rxq_descr(QEMUFile *f, void *pv, size_t size,
+VMStateField *field)
 {
 Vmxnet3RxqDescr *r = pv;
 int i;
@@ -2518,7 +2521,8 @@ static int vmxnet3_get_rxq_descr(QEMUFile *f, void *pv, 
size_t size)
 return 0;
 }
 
-static void vmxnet3_put_rxq_descr(QEMUFile *f, void *pv, size_t size)
+static void vmxnet3_put_rxq_descr(QEMUFile *f, void *pv, size_t size,
+VMStateField *field, QJSON *vmdesc)
 {
 Vmxnet3RxqDescr *r = pv;
 int i;
@@ -2562,7 +2566,8 @@ static const VMStateInfo rxq_descr_info = {
 .put = vmxnet3_put_rxq_descr
 };
 
-static int vmxnet3_get_int_state(QEMUFile *f, void *pv, size_t size)
+static int vmxnet3_get_int_state(QEMUFile *f, void *pv, size_t size,
+VMStateField *field)
 {
 Vmxnet3IntState *r = pv;
 
@@ -2573,7 +2578,8 @@ static int vmxnet3_get_int_state(QEMUFile *f, void *pv, 
size_t size)
 return 0;
 }
 
-static void vmxnet3_put_int_state(QEMUFile *f, void *pv, size_t size)
+static void vmxnet3_put_int_state(QEMUFile *f, void *pv, size_t size,
+VMStateField *field, QJSON *vmdesc)
 {
 Vmxnet3IntState *r = pv;
 
diff --git a/hw/nvram/eeprom93xx.c b/hw/nvram/eeprom93xx.c
index 2c16fc2..76d5f41 100644
--- a/hw/nvram/eeprom93xx.c
+++ b/hw/nvram/eeprom93xx.c
@@ -94,14 +94,16 @@ struct _eeprom_t {
This is a Big hack, but it is how the old state did it.
  */
 
-static int get_uint16_from_uint8(QEMUFile *f, void *pv, size_t size)
+static int get_uint16_from_uint8(QEMUFile *f, void *pv, size_t size,
+ VMStateField *field)
 {
 uint16_t *v = pv;
 *v = qemu_get_ubyte(f);
 return 0;
 }
 
-static void put_unused(QEMUFile *f, void *pv, size_t size)
+static void put_unused(QEMUFile *f, void *pv, size_t size, VMStateField *field,
+   QJSON *vmdesc)
 {
 fprintf(stderr, "uint16_from_uint8 is used only for backwards 
compatibility.\n");
 fprintf(stderr, "Never should be used to write a new state.\n");
diff --git a/hw/nvram/fw_cfg.c b/hw/nvram/fw_cfg.c
index cdbdfb5..c294f9a 100644
--- a/hw/nvram/fw_cfg.c
+++ b/hw/nvram/fw_cfg.c
@@ -524,14 +524,16 @@ static void fw_cfg_reset(DeviceState *d)
Or we broke compatibility in the state, or we can't use struct tm
  */
 
-static int get_uint32_as_uint16(QEMUFile *f, void *pv, size_t size)
+static int get_uint32_as_uint16(QEMUFile *f, void *pv, size_t size,
+VMStateField *field)
 {
 uint32_t *v = pv;
 *v = qemu_get_be16(f);
 return 0;
 }
 
-static void put_unused(QEMUFile *f, void *pv, size_t size)
+static void put_unused(QEMUFile *f, void *pv, size_t size, VMStateField *field,
+   QJSON *vmdesc)
 {
 fprintf(stderr, "uint32_as_uint16 is only used for backward 
compatibility.\n");
 fprintf(stderr, "This functions shouldn't be called.\n");
diff --git a/hw/pci/msix.c 

[Qemu-devel] [QEMU RFC PATCH v3 6/6] Migration: migrate pending_events of spapr state

[Jianjun Duan]

In racing situations between hotplug events and migration operation,
a rtas hotplug event could have not yet be delivered to the source
guest when migration is started. In this case the pending_events of
spapr state need be transmitted to the target so that the hotplug
event can be finished on the target.

All the different fields of the events are encoded as defined by
PAPR. We can migrate them as uint8_t binary stream without any
concerns about data padding or endianess.

pending_events is put in a subsection in the spapr state VMSD to make
sure migration across different versions is not broken.

Signed-off-by: Jianjun Duan 
---
 hw/ppc/spapr.c | 33 +
 hw/ppc/spapr_events.c  | 22 +-
 include/hw/ppc/spapr.h |  3 ++-
 3 files changed, 48 insertions(+), 10 deletions(-)

diff --git a/hw/ppc/spapr.c b/hw/ppc/spapr.c
index f13584c..b38cf04 100644
--- a/hw/ppc/spapr.c
+++ b/hw/ppc/spapr.c
@@ -1266,12 +1266,32 @@ static bool version_before_3(void *opaque, int 
version_id)
 return version_id < 3;
 }
 
+static bool spapr_pending_events_needed(void *opaque)
+{
+sPAPRMachineState *spapr = (sPAPRMachineState *)opaque;
+return !QTAILQ_EMPTY(>pending_events);
+}
+
 static bool spapr_ccs_list_needed(void *opaque)
 {
 sPAPRMachineState *spapr = (sPAPRMachineState *)opaque;
 return !QTAILQ_EMPTY(>ccs_list);
 }
 
+static const VMStateDescription vmstate_spapr_event_entry = {
+.name = "spapreventlogentry",
+.version_id = 1,
+.minimum_version_id = 1,
+.fields = (VMStateField[]) {
+VMSTATE_INT32(log_type, sPAPREventLogEntry),
+VMSTATE_BOOL(exception, sPAPREventLogEntry),
+VMSTATE_UINT32(data_size, sPAPREventLogEntry),
+VMSTATE_VARRAY_UINT32_ALLOC(data, sPAPREventLogEntry, data_size,
+0, vmstate_info_uint8, uint8_t),
+VMSTATE_END_OF_LIST()
+  },
+};
+
 static const VMStateDescription vmstate_spapr_ccs = {
 .name = "spaprconfigureconnectorstate",
 .version_id = 1,
@@ -1284,6 +1304,18 @@ static const VMStateDescription vmstate_spapr_ccs = {
 },
 };
 
+static const VMStateDescription vmstate_spapr_pending_events = {
+.name = "spaprpendingevents",
+.version_id = 1,
+.minimum_version_id = 1,
+.needed = spapr_pending_events_needed,
+.fields = (VMStateField[]) {
+VMSTATE_QTAILQ_V(pending_events, sPAPRMachineState, 1,
+ vmstate_spapr_event_entry, sPAPREventLogEntry, next),
+VMSTATE_END_OF_LIST()
+},
+};
+
 static const VMStateDescription vmstate_spapr_ccs_list = {
 .name = "spaprccslist",
 .version_id = 1,
@@ -1312,6 +1344,7 @@ static const VMStateDescription vmstate_spapr = {
 VMSTATE_END_OF_LIST()
 },
 .subsections = (const VMStateDescription*[]) {
+_spapr_pending_events,
 _spapr_ccs_list,
 NULL
 }
diff --git a/hw/ppc/spapr_events.c b/hw/ppc/spapr_events.c
index 049fb1b..1680c08 100644
--- a/hw/ppc/spapr_events.c
+++ b/hw/ppc/spapr_events.c
@@ -239,7 +239,8 @@ void spapr_events_fdt_skel(void *fdt, uint32_t 
check_exception_irq)
 _FDT((fdt_end_node(fdt)));
 }
 
-static void rtas_event_log_queue(int log_type, void *data, bool exception)
+static void rtas_event_log_queue(int log_type, void *data, bool exception,
+int data_size)
 {
 sPAPRMachineState *spapr = SPAPR_MACHINE(qdev_get_machine());
 sPAPREventLogEntry *entry = g_new(sPAPREventLogEntry, 1);
@@ -248,6 +249,7 @@ static void rtas_event_log_queue(int log_type, void *data, 
bool exception)
 entry->log_type = log_type;
 entry->exception = exception;
 entry->data = data;
+entry->data_size = data_size;
 QTAILQ_INSERT_TAIL(>pending_events, entry, next);
 }
 
@@ -350,6 +352,7 @@ static void spapr_powerdown_req(Notifier *n, void *opaque)
 struct rtas_event_log_v6_mainb *mainb;
 struct rtas_event_log_v6_epow *epow;
 struct epow_log_full *new_epow;
+uint32_t data_size;
 
 new_epow = g_malloc0(sizeof(*new_epow));
 hdr = _epow->hdr;
@@ -358,13 +361,13 @@ static void spapr_powerdown_req(Notifier *n, void *opaque)
 mainb = _epow->mainb;
 epow = _epow->epow;
 
+data_size = sizeof(*new_epow);
 hdr->summary = cpu_to_be32(RTAS_LOG_VERSION_6
| RTAS_LOG_SEVERITY_EVENT
| RTAS_LOG_DISPOSITION_NOT_RECOVERED
| RTAS_LOG_OPTIONAL_PART_PRESENT
| RTAS_LOG_TYPE_EPOW);
-hdr->extended_length = cpu_to_be32(sizeof(*new_epow)
-   - sizeof(new_epow->hdr));
+hdr->extended_length = cpu_to_be32(data_size - sizeof(new_epow->hdr));
 
 spapr_init_v6hdr(v6hdr);
 spapr_init_maina(maina, 3 /* Main-A, Main-B and EPOW */);
@@ -384,7 +387,7 @@ static void spapr_powerdown_req(Notifier *n, void *opaque)
 

[Qemu-devel] [QEMU RFC PATCH v3 4/6] Migration: migrate QTAILQ

[Jianjun Duan]

Currently we cannot directly transfer a QTAILQ instance because of the
limitation in the migration code. Here we introduce an approach to
transfer such structures. In our approach such a structure is tagged
with VMS_CSTM. We then modified vmstate_save_state and vmstate_load_state
so that when VMS_CSTM is encountered, put and get from VMStateInfo are
called respectively. This approach will be used to transfer pending_events
and ccs_list in spapr state.

We also create some macros in qemu/queue.h to access a QTAILQ using pointer
arithmetic. This ensures that we do not depend on the implementation
details about QTAILQ in the migration code.

Signed-off-by: Jianjun Duan 
---
 include/migration/vmstate.h | 22 +
 include/qemu/queue.h| 32 ++
 migration/vmstate.c | 79 +
 3 files changed, 133 insertions(+)

diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index 56a4171..da4ef7f 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -185,6 +185,8 @@ enum VMStateFlags {
  * to determine the number of entries in the array. Only valid in
  * combination with one of VMS_VARRAY*. */
 VMS_MULTIPLY_ELEMENTS = 0x4000,
+/* For fields which need customized handling, such as QTAILQ in queue.h*/
+VMS_CSTM= 0x8000,
 };
 
 struct VMStateField {
@@ -245,6 +247,7 @@ extern const VMStateInfo vmstate_info_timer;
 extern const VMStateInfo vmstate_info_buffer;
 extern const VMStateInfo vmstate_info_unused_buffer;
 extern const VMStateInfo vmstate_info_bitmap;
+extern const VMStateInfo vmstate_info_qtailq;
 
 #define type_check_2darray(t1,t2,n,m) ((t1(*)[n][m])0 - (t2*)0)
 #define type_check_array(t1,t2,n) ((t1(*)[n])0 - (t2*)0)
@@ -656,6 +659,25 @@ extern const VMStateInfo vmstate_info_bitmap;
 .offset   = offsetof(_state, _field),\
 }
 
+/* For QTAILQ that need customized handling
+ * _type: type of QTAILQ element
+ * _next: name of QTAILQ entry field in QTAILQ element
+ * _vmsd: VMSD for QTAILQ element
+ * size: size of QTAILQ element
+ * start: offset of QTAILQ entry in QTAILQ element
+ */
+#define VMSTATE_QTAILQ_V(_field, _state, _version, _vmsd, _type, _next)  \
+{\
+.name = (stringify(_field)), \
+.version_id   = (_version),  \
+.vmsd = &(_vmsd),\
+.size = sizeof(_type),   \
+.info = _info_qtailq,\
+.flags= VMS_CSTM,\
+.offset   = offsetof(_state, _field),\
+.start= offsetof(_type, _next),  \
+}
+
 /* _f : field name
_f_n : num of elements field_name
_n : num of elements
diff --git a/include/qemu/queue.h b/include/qemu/queue.h
index f781aa2..003e368 100644
--- a/include/qemu/queue.h
+++ b/include/qemu/queue.h
@@ -437,3 +437,35 @@ struct {   
 \
 (*(((struct headname *)((elm)->field.tqe_prev))->tqh_last))
 
 #endif  /* !QEMU_SYS_QUEUE_H_ */
+
+/*
+ * Offsets of layout of a tail queue head.
+ */
+#define QTAILQ_FIRST_OFFSET 0
+#define QTAILQ_LAST_OFFSET (sizeof(void *))
+
+/*
+ * Offsets of layout of a tail queue element.
+ */
+#define QTAILQ_NEXT_OFFSET 0
+#define QTAILQ_PREV_OFFSET (sizeof(void *))
+
+/*
+ * Tail queue tranversal using pointer arithmetic.
+ */
+#define QTAILQ_RAW_FOREACH(elm, head, entry)   
\
+for ((elm) = *((void **) ((char *) (head) + QTAILQ_FIRST_OFFSET)); 
\
+ (elm);
\
+ (elm) =   
\
+ *((void **) ((char *) (elm) + (entry) + QTAILQ_NEXT_OFFSET)))
+/*
+ * Tail queue insertion using pointer arithmetic.
+ */
+#define QTAILQ_RAW_INSERT_TAIL(head, elm, entry) do {  
\
+*((void **) ((char *) (elm) + (entry) + QTAILQ_NEXT_OFFSET)) = NULL;   
\
+*((void **) ((char *) (elm) + (entry) + QTAILQ_PREV_OFFSET)) = 
\
+*((void **) ((char *) (head) +QTAILQ_LAST_OFFSET));
\
+**((void ***)((char *) (head) +QTAILQ_LAST_OFFSET)) = (elm);   
\
+*((void **) ((char *) (head) + QTAILQ_LAST_OFFSET)) =  
\
+(void *) ((char *) (elm) + (entry) + QTAILQ_NEXT_OFFSET);  
\
+} while (/*CONSTCOND*/0)
diff --git a/migration/vmstate.c b/migration/vmstate.c
index 644ba1f..ff56650 100644
--- a/migration/vmstate.c
+++ b/migration/vmstate.c
@@ -5,7 +5,9 @@
 #include "migration/vmstate.h"
 

[Qemu-devel] [QEMU RFC PATCH v3 1/6] Migration: Defined VMStateDescription struct for spapr_drc

[Jianjun Duan]

To manage hotplug/unplug of dynamic resources such as PCI cards,
memory, and CPU on sPAPR guests, a firmware abstraction known as
a Dynamic Resource Connector (DRC) is used to assign a particular
dynamic resource to the guest, and provide an interface for the
guest to manage configuration/removal of the resource associated
with it.

To migrate the hotplugged resources in migration, the
associated DRC state need be migrated. To migrate the DRC state,
we defined the VMStateDescription struct for spapr_drc to enable
the transmission of spapr_drc state in migration.

Not all the elements in the DRC state are migrated. Only those
ones modifiable or needed by guest actions or device add/remove
operation are migrated. From the perspective of device
hotplugging, if we hotplug a device on the source, we need to
"coldplug" it on the target. The states across two hosts for the
same device are not the same. Ideally we want the states be same
after migration so that the device would function as hotplugged
on the target. For example we can unplug it. The minimum DRC
state we need to transfer should cover all the pieces changed by
hotplugging. Out of the elements of the DRC state, isolation_state,
allocation_sate, and configured are involved in the DR state
transition diagram from PAPR+ 2.7, 13.4. configured and signalled
are needed in attaching and detaching devices. indicator_state
provides users with hardware state information. These 6 elements
are migrated.

detach_cb in the DRC state is a function pointer that cannot be
migrated. We set it right after DRC state is migrated so that
a migrated hot-unplug event could finish its work.

Signed-off-by: Jianjun Duan 
---
 hw/ppc/spapr_drc.c | 61 ++
 hw/ppc/spapr_pci.c | 22 +
 include/hw/ppc/spapr_drc.h |  9 +++
 3 files changed, 92 insertions(+)

diff --git a/hw/ppc/spapr_drc.c b/hw/ppc/spapr_drc.c
index 94c875d..1fb5e23 100644
--- a/hw/ppc/spapr_drc.c
+++ b/hw/ppc/spapr_drc.c
@@ -617,6 +617,65 @@ static void spapr_dr_connector_instance_init(Object *obj)
 NULL, NULL, NULL, NULL);
 }
 
+static bool spapr_drc_needed(void *opaque)
+{
+sPAPRDRConnector *drc = (sPAPRDRConnector *)opaque;
+sPAPRDRConnectorClass *drck = SPAPR_DR_CONNECTOR_GET_CLASS(drc);
+bool rc = false;
+sPAPRDREntitySense value;
+
+drck->entity_sense(drc, );
+/* If no dev is plugged in there is no need to migrate the DRC state */
+if (value != SPAPR_DR_ENTITY_SENSE_PRESENT) {
+return false;
+}
+/*
+ * If there is dev plugged in, we need to migrate the DRC state when
+ * it is different from cold-plugged state
+ */
+switch(drc->type) {
+/* for PCI type */
+case SPAPR_DR_CONNECTOR_TYPE_PCI:
+rc = !((drc->isolation_state == SPAPR_DR_ISOLATION_STATE_UNISOLATED) &&
+   (drc->allocation_state == SPAPR_DR_ALLOCATION_STATE_USABLE) &&
+   drc->configured && drc->signalled && !drc->awaiting_release);
+break;
+/* for LMB type */
+case SPAPR_DR_CONNECTOR_TYPE_LMB:
+rc = !((drc->isolation_state == SPAPR_DR_ISOLATION_STATE_ISOLATED) &&
+   (drc->allocation_state == SPAPR_DR_ALLOCATION_STATE_UNUSABLE) &&
+   drc->configured && drc->signalled && !drc->awaiting_release);
+break;
+default:
+;
+}
+
+return rc;
+}
+
+/* detach_cb needs be set since it is not migrated */
+static void postmigrate_set_detach_cb(sPAPRDRConnector *drc,
+  spapr_drc_detach_cb *detach_cb)
+{
+drc->detach_cb = detach_cb;
+}
+
+static const VMStateDescription vmstate_spapr_drc = {
+.name = "spapr_drc",
+.version_id = 1,
+.minimum_version_id = 1,
+.needed = spapr_drc_needed,
+.fields  = (VMStateField []) {
+VMSTATE_UINT32(isolation_state, sPAPRDRConnector),
+VMSTATE_UINT32(allocation_state, sPAPRDRConnector),
+VMSTATE_UINT32(indicator_state, sPAPRDRConnector),
+VMSTATE_BOOL(configured, sPAPRDRConnector),
+VMSTATE_BOOL(awaiting_release, sPAPRDRConnector),
+VMSTATE_BOOL(signalled, sPAPRDRConnector),
+VMSTATE_END_OF_LIST()
+}
+};
+
 static void spapr_dr_connector_class_init(ObjectClass *k, void *data)
 {
 DeviceClass *dk = DEVICE_CLASS(k);
@@ -625,6 +684,7 @@ static void spapr_dr_connector_class_init(ObjectClass *k, 
void *data)
 dk->reset = reset;
 dk->realize = realize;
 dk->unrealize = unrealize;
+dk->vmsd = _spapr_drc;
 drck->set_isolation_state = set_isolation_state;
 drck->set_indicator_state = set_indicator_state;
 drck->set_allocation_state = set_allocation_state;
@@ -638,6 +698,7 @@ static void spapr_dr_connector_class_init(ObjectClass *k, 
void *data)
 drck->detach = detach;
 drck->release_pending = release_pending;
 drck->set_signalled = set_signalled;
+

[Qemu-devel] [QEMU RFC PATCH v3 2/6] vmstate: Define VARRAY with VMS_ALLOC

[Jianjun Duan]

From: Alexey Kardashevskiy 

This allows dynamic allocation for migrating arrays.

Already existing VMSTATE_VARRAY_UINT32 requires an array to be
pre-allocated, however there are cases when the size is not known in
advance and there is no real need to enforce it.

This defines another variant of VMSTATE_VARRAY_UINT32 with WMS_ALLOC
flag which tells the receiving side to allocate memory for the array
before receiving the data.

The first user of it is a dynamic DMA window which existence and size
are totally dynamic.

Signed-off-by: Alexey Kardashevskiy 
Reviewed-by: David Gibson 
Reviewed-by: Thomas Huth 
Signed-off-by: Jianjun Duan 
---
 include/migration/vmstate.h | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/include/migration/vmstate.h b/include/migration/vmstate.h
index 30ecc44..6c65811 100644
--- a/include/migration/vmstate.h
+++ b/include/migration/vmstate.h
@@ -386,6 +386,16 @@ extern const VMStateInfo vmstate_info_bitmap;
 .offset = vmstate_offset_pointer(_state, _field, _type), \
 }
 
+#define VMSTATE_VARRAY_UINT32_ALLOC(_field, _state, _field_num, _version, 
_info, _type) {\
+.name   = (stringify(_field)),   \
+.version_id = (_version),\
+.num_offset = vmstate_offset_value(_state, _field_num, uint32_t),\
+.info   = &(_info),  \
+.size   = sizeof(_type), \
+.flags  = VMS_VARRAY_UINT32|VMS_POINTER|VMS_ALLOC,   \
+.offset = vmstate_offset_pointer(_state, _field, _type), \
+}
+
 #define VMSTATE_VARRAY_UINT16_UNSAFE(_field, _state, _field_num, _version, 
_info, _type) {\
 .name   = (stringify(_field)),   \
 .version_id = (_version),\
-- 
1.9.1