[Xen-devel] [PATCH V2] x86/hvm: Allow guest_request vm_events coming from userspace

[Razvan Cojocaru]

Allow guest userspace code to request that a vm_event be sent out
via VMCALL. This functionality seems to be handy for a number of
Xen developers, as stated on the mailing list (thread "[Xen-devel]
HVMOP_guest_request_vm_event only works from guest in ring0").

Signed-off-by: Razvan Cojocaru 

---
Changes since V1:
 - No longer repeating the check when mode == 8.
 - Added /* Fallthrough */ tags for Coverity.
---
 xen/arch/x86/hvm/hvm.c | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/xen/arch/x86/hvm/hvm.c b/xen/arch/x86/hvm/hvm.c
index 893eff6..cb546e4 100644
--- a/xen/arch/x86/hvm/hvm.c
+++ b/xen/arch/x86/hvm/hvm.c
@@ -4146,15 +4146,25 @@ int hvm_do_hypercall(struct cpu_user_regs *regs)
 switch ( mode )
 {
 case 8:
+if ( eax == __HYPERVISOR_hvm_op &&
+ regs->rdi == HVMOP_guest_request_vm_event )
+break;
+/* Fallthrough */
 case 4:
+/* Fallthrough */
 case 2:
+if ( mode != 8 && eax == __HYPERVISOR_hvm_op &&
+ regs->_ebx == HVMOP_guest_request_vm_event )
+break;
 hvm_get_segment_register(curr, x86_seg_ss, &sreg);
 if ( unlikely(sreg.attr.fields.dpl) )
 {
+/* Fallthrough */
 default:
 regs->eax = -EPERM;
 return HVM_HCALL_completed;
 }
+/* Fallthrough */
 case 0:
 break;
 }
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-4.5-testing test] 100322: regressions - trouble: broken/fail/pass

[osstest service owner]

flight 100322 xen-4.5-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100322/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-xl-qemut-debianhvm-amd64 3 host-install(3) broken REGR. vs. 
99752
 test-amd64-i386-qemuu-rhel6hvm-amd  3 host-install(3)   broken REGR. vs. 99752
 test-amd64-amd64-xl-qemut-winxpsp3  3 host-install(3)   broken REGR. vs. 99752
 test-armhf-armhf-xl-credit2  15 guest-start/debian.repeat fail REGR. vs. 99752

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-xl-qemuu-win7-amd64 16 guest-stopfail blocked in 99752
 test-amd64-amd64-libvirt 14 guest-saverestorefail   like 99712
 test-armhf-armhf-xl-rtds 15 guest-start/debian.repeatfail   like 99712
 test-amd64-amd64-xl-rtds  6 xen-boot fail   like 99752
 test-amd64-amd64-xl-qemuu-win7-amd64 16 guest-stop fail like 99752
 test-amd64-i386-xl-qemut-win7-amd64 16 guest-stop  fail like 99752
 test-amd64-amd64-xl-qemut-win7-amd64 16 guest-stop fail like 99752

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-pvh-amd  11 guest-start  fail   never pass
 test-amd64-amd64-xl-pvh-intel 11 guest-start  fail  never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 13 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 12 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-arndale  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  13 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  10 guest-start  fail   never pass
 test-amd64-amd64-qemuu-nested-amd 16 debian-hvm-install/l1/l2  fail never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 12 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 13 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-credit2  13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 10 guest-start  fail never pass
 test-armhf-armhf-libvirt-raw 10 guest-start  fail   never pass
 test-armhf-armhf-xl-rtds 13 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 12 migrate-support-checkfail   never pass

version targeted for testing:
 xen  08313b45bfc75fa4cbadb9d25a0561e5f5b2fee6
baseline version:
 xen  c4c0312efaf8bd252ff06d55d6bf5b542a0a9421

Last test of basis99752  2016-07-28 12:30:52 Z   10 days
Failing since 99963  2016-08-05 12:45:04 Z2 days4 attempts
Testing same since99979  2016-08-06 02:33:37 Z2 days3 attempts


People who touched revisions under test:
  Andrew Cooper 
  Euan Harris 
  George Dunlap 
  Jan Beulich 
  Jiandi An 
  Kevin Tian 
  Stefano Stabellini 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-prev pass
 build-i386-prev  pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 build-amd64-rumpuserxen  pass
 build-i386-rumpuserxen   pass
 test-amd64-amd64-xl  pass
 test-armhf-armhf-xl  pass
 test-amd64-i386-xl   pass
 test-amd64-amd64-qemuu-nested-amdfail
 test-amd64-amd64-xl-pvh-amd  fail
 test-amd64-i386-qemut-rhel6hvm-amd   pass

[Xen-devel] [PATCH 3/3] tools: add config parameter for maximum memory of xenstore domain

[Juergen Gross]

Add a parameter to xencommons configuration file for specifying the
maximum memory size of the xenstore domain.

Signed-off-by: Juergen Gross 
---
 tools/hotplug/Linux/init.d/sysconfig.xencommons.in | 11 +++
 tools/hotplug/Linux/launch-xenstore.in |  1 +
 2 files changed, 12 insertions(+)

diff --git a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in 
b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
index cc8185c..92569cd 100644
--- a/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
+++ b/tools/hotplug/Linux/init.d/sysconfig.xencommons.in
@@ -70,6 +70,17 @@ XENSTORED_ARGS=
 #XENSTORE_DOMAIN_SIZE=8
 
 ## Type: string
+## Default: not set, no autoballooning of xenstore domain
+#
+# Maximum xenstore domain memory size. Can be specified as:
+# - plain integer value for max size in MiB
+# - fraction of host memory, e.g. 1/100
+# - combination of both in form of : (e.g. 8:1/100), resulting
+#   value will be the higher of both specifications
+# Only evaluated if XENSTORETYPE is "domain".
+#XENSTORE_MAX_DOMAIN_SIZE=
+
+## Type: string
 ## Default: ""
 #
 # Additional arguments for starting the xenstore domain.
diff --git a/tools/hotplug/Linux/launch-xenstore.in 
b/tools/hotplug/Linux/launch-xenstore.in
index 46defd6..01193be 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -75,6 +75,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . 
@CONFIG_DIR@/@CONFIG_LEAF
XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --kernel 
$XENSTORE_DOMAIN_KERNEL"
[ -z "$XENSTORE_DOMAIN_SIZE" ] && XENSTORE_DOMAIN_SIZE=8
XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --memory 
$XENSTORE_DOMAIN_SIZE"
+   [ -z "$XENSTORE_MAX_DOMAIN_SIZE" ] || 
XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --maxmem $XENSTORE_MAX_DOMAIN_SIZE"
 
echo -n Starting $XENSTORE_DOMAIN_KERNEL...
${LIBEXEC_BIN}/init-xenstore-domain $XENSTORE_DOMAIN_ARGS || exit 1
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 1/3] tools: add --maxmem parameter to init-xenstore-domain

[Juergen Gross]

Add a parameter to specify the maximum memory size of the xenstore
domain. In case the xenstore domain supports ballooning it will be
capable to adjust its own size according to its memory needs.

The maximum memory size can be specified as an absolute value in
MiB, as a fraction of the host's memory, or as a combination of
both (the maximum of the absolute and the fraction value):

--maxmem  maxmem is  MiB
--maxmem / maxmem is hostmem * a / b
--maxmem :/ maxmem is max( MiB, hostmem * a / b)

Signed-off-by: Juergen Gross 
---
 tools/helpers/init-xenstore-domain.c | 83 +++-
 1 file changed, 81 insertions(+), 2 deletions(-)

diff --git a/tools/helpers/init-xenstore-domain.c 
b/tools/helpers/init-xenstore-domain.c
index 53b4b01..a7c97d7 100644
--- a/tools/helpers/init-xenstore-domain.c
+++ b/tools/helpers/init-xenstore-domain.c
@@ -23,6 +23,7 @@ static char *flask;
 static char *param;
 static char *name = "Xenstore";
 static int memory;
+static int maxmem;
 
 static struct option options[] = {
 { "kernel", 1, NULL, 'k' },
@@ -31,6 +32,7 @@ static struct option options[] = {
 { "ramdisk", 1, NULL, 'r' },
 { "param", 1, NULL, 'p' },
 { "name", 1, NULL, 'n' },
+{ "maxmem", 1, NULL, 'M' },
 { NULL, 0, NULL, 0 }
 };
 
@@ -48,7 +50,11 @@ static void usage(void)
 "  --flask   optional flask label of the domain\n"
 "  --ramdiskoptional ramdisk file for the domain\n"
 "  --param   optional additional parameters for the domain\n"
-"  --name   name of the domain (default: Xenstore)\n");
+"  --name   name of the domain (default: Xenstore)\n"
+"  --maxmem maximum memory size in the format:\n"
+" |/|:/\n"
+" (an absolute value in MB, a fraction a/b of\n"
+" the host memory, or the maximum of both)\n");
 }
 
 static int build(xc_interface *xch)
@@ -58,7 +64,7 @@ static int build(xc_interface *xch)
 xen_domain_handle_t handle = { 0 };
 int rv, xs_fd;
 struct xc_dom_image *dom = NULL;
-int limit_kb = (memory + 1) * 1024;
+int limit_kb = (maxmem ? : (memory + 1)) * 1024;
 
 xs_fd = open("/dev/xen/xenbus_backend", O_RDWR);
 if ( xs_fd == -1 )
@@ -223,6 +229,63 @@ static int check_domain(xc_interface *xch)
 return 0;
 }
 
+static int parse_maxmem(xc_interface *xch, char *str)
+{
+xc_physinfo_t info;
+int rv;
+unsigned long mb = 0, a = 0, b = 0;
+unsigned long val;
+unsigned long *res;
+char buf[16];
+char *p;
+char *s = str;
+
+rv = xc_physinfo(xch, &info);
+if ( rv )
+{
+fprintf(stderr, "xc_physinfo failed\n");
+return -1;
+}
+
+res = &mb;
+for (p = s; *p; s = p + 1)
+{
+val = strtoul(s, &p, 10);
+if ( val == 0 || val >= INT_MAX / 1024 )
+goto err;
+if ( *p == '/' )
+{
+if ( res != &mb || a != 0 )
+goto err;
+a = val;
+res = &b;
+continue;
+}
+if ( *res != 0 )
+goto err;
+*res = val;
+if ( *p != 0 && *p != ':' )
+goto err;
+res = &mb;
+}
+if ( a && !b )
+goto err;
+
+val = a ? info.total_pages * a / (b * 1024 * 1024 / XC_PAGE_SIZE) : 0;
+if ( val >= INT_MAX / 1024 )
+goto err;
+
+maxmem = mb < val ? val : mb;
+if ( maxmem < memory )
+maxmem = 0;
+
+return maxmem;
+
+err:
+fprintf(stderr, "illegal value for maxmem: %s\n", str);
+return -1;
+}
+
 static void do_xs_write(struct xs_handle *xsh, char *path, char *val)
 {
 if ( !xs_write(xsh, XBT_NULL, path, val, strlen(val)) )
@@ -244,6 +307,7 @@ int main(int argc, char** argv)
 struct xs_handle *xsh;
 char buf[16];
 int rv, fd;
+char *maxmem_str = NULL;
 
 while ( (opt = getopt_long(argc, argv, "", options, NULL)) != -1 )
 {
@@ -267,6 +331,9 @@ int main(int argc, char** argv)
 case 'n':
 name = optarg;
 break;
+case 'M':
+maxmem_str = optarg;
+break;
 default:
 usage();
 return 2;
@@ -286,6 +353,16 @@ int main(int argc, char** argv)
 return 1;
 }
 
+if ( maxmem_str )
+{
+maxmem = parse_maxmem(xch, maxmem_str);
+if ( maxmem < 0 )
+{
+xc_interface_close(xch);
+return 1;
+}
+}
+
 rv = check_domain(xch);
 
 if ( !rv )
@@ -314,6 +391,8 @@ int main(int argc, char** argv)
 do_xs_write_dom(xsh, "name", name);
 snprintf(buf, 16, "%d", memory * 1024);
 do_xs_write_dom(xsh, "memory/target", buf);
+if (maxmem)
+snprintf(buf, 16, "%d", maxmem * 1024);
 do_xs_write_dom(xsh, "memory/static-max", buf);
 xs_close(xsh);
 
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@list

[Xen-devel] [PATCH 0/3] tools: support autoballooning of xenstore domain

[Juergen Gross]

Support xenstore domain autoballooning by:
- adding --maxmem parameter to init-xenstore-domain
- build xenstore stubdom with Mini-OS CONFIG_BALLOON set
- add XENSTORE_MAX_DOMAIN_SIZE parameter to sysconfig.xencommons

This series requires Mini-OS ballooning support, of course. I'm posting
it now because this will make it easier to test my Mini-OS series.

Juergen Gross (3):
  tools: add --maxmem parameter to init-xenstore-domain
  stubdom: add CONFIG_BALLOON to xenstore config
  tools: add config parameter for maximum memory of xenstore domain

 stubdom/xenstore-minios.cfg|  1 +
 tools/helpers/init-xenstore-domain.c   | 83 +-
 tools/hotplug/Linux/init.d/sysconfig.xencommons.in | 11 +++
 tools/hotplug/Linux/launch-xenstore.in |  1 +
 4 files changed, 94 insertions(+), 2 deletions(-)

-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 2/3] stubdom: add CONFIG_BALLOON to xenstore config

[Juergen Gross]

Compile xenstore stubdom with ballooning support.

Signed-off-by: Juergen Gross 
---
 stubdom/xenstore-minios.cfg | 1 +
 1 file changed, 1 insertion(+)

diff --git a/stubdom/xenstore-minios.cfg b/stubdom/xenstore-minios.cfg
index 6a09cce..c89e9f0 100644
--- a/stubdom/xenstore-minios.cfg
+++ b/stubdom/xenstore-minios.cfg
@@ -5,3 +5,4 @@ CONFIG_KBDFRONT=n
 CONFIG_CONSFRONT=n
 CONFIG_XENBUS=n
 CONFIG_LWIP=n
+CONFIG_BALLOON=y
-- 
2.6.6


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 02/14] DO NOT APPLY ts-leak-check: sleep 5 seconds before collecting stuff

[Wei Liu]

The system could be in the process of freeing up resources. Give it some
time to finish.

Signed-off-by: Wei Liu 
---
This won't be necessary once we fix all synchronisation issues in
xtf-runner.
---
 ts-leak-check | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/ts-leak-check b/ts-leak-check
index 8a97971..1f6d5b0 100755
--- a/ts-leak-check
+++ b/ts-leak-check
@@ -170,6 +170,9 @@ sub inventory () {
 }
 
 if (!eval {
+# Sleep 5 seconds before colleting stuff in case some resources are
+# being freed.
+sleep 5;
 &{ "start_$mode" }();
 inventory();
 &{ "finish_$mode" }();
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 03/14] ap-common: add xtf tree

[Wei Liu]

Signed-off-by: Wei Liu 
---
 ap-common | 4 
 1 file changed, 4 insertions(+)

diff --git a/ap-common b/ap-common
index 19c7580..46523f3 100644
--- a/ap-common
+++ b/ap-common
@@ -33,6 +33,10 @@
 
 : ${TREEVCS_LINUX:=git}
 
+: ${TREE_XTF:=git://xenbits.xen.org/xtf.git}
+: ${PUSH_TREE_XTF:=$XENBITS:/home/xen/git/osstest/xtf.git}
+: ${BASE_TREE_XTF:=git://xenbits.xen.org/osstest/xtf.git}
+
 : ${TREE_LIBVIRT:=git://libvirt.org/libvirt.git}
 : ${PUSH_TREE_LIBVIRT:=$XENBITS:/home/xen/git/libvirt.git}
 : ${BASE_TREE_LIBVIRT:=git://xenbits.xen.org/libvirt.git}
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 04/14] DO NOT APPLY point xtf to my personal tree

[Wei Liu]

---
 ap-common | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/ap-common b/ap-common
index 46523f3..e544605 100644
--- a/ap-common
+++ b/ap-common
@@ -33,9 +33,9 @@
 
 : ${TREEVCS_LINUX:=git}
 
-: ${TREE_XTF:=git://xenbits.xen.org/xtf.git}
-: ${PUSH_TREE_XTF:=$XENBITS:/home/xen/git/osstest/xtf.git}
-: ${BASE_TREE_XTF:=git://xenbits.xen.org/osstest/xtf.git}
+: ${TREE_XTF:=git://xenbits.xen.org/people/liuw/xtf.git}
+: ${PUSH_TREE_XTF:=/export/home/weil/xtf.git/.git}
+: ${BASE_TREE_XTF:=git://xenbits.xen.org/people/liuw/xtf.git}
 
 : ${TREE_LIBVIRT:=git://libvirt.org/libvirt.git}
 : ${PUSH_TREE_LIBVIRT:=$XENBITS:/home/xen/git/libvirt.git}
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 07/14] sg-run-job: create xtf build recipe

[Wei Liu]

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 
---
 sg-run-job | 5 +
 1 file changed, 5 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index 259fc3b..240b265 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -480,6 +480,7 @@ proc need-hosts/build {} { return BUILD }
 proc need-hosts/build-kern {} { return BUILD }
 proc need-hosts/build-libvirt {} { return BUILD }
 proc need-hosts/build-rumpuserxen {} { return BUILD }
+proc need-hosts/build-xtf {} { return BUILD }
 
 proc run-job/build {} {
 run-ts . = ts-xen-build
@@ -498,6 +499,10 @@ proc run-job/build-rumpuserxen {} {
 run-ts . = ts-xen-build + host tools
 }
 
+proc run-job/build-xtf {} {
+run-ts . = ts-xtf-build
+}
+
 proc prepare-build-host {} {
 global jobinfo
 run-ts broken = ts-hosts-allocate + host
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 05/14] BuildSupport: move buildcmd_stamped_logged here

[Wei Liu]

... so that other build scripts can use it, too.

It now accepts one more parameter called "component" to be useful in
other build scripts.

No functional change.

Signed-off-by: Wei Liu 
---
v2: new
---
 Osstest/BuildSupport.pm | 15 +++
 ts-xen-build| 18 ++
 2 files changed, 17 insertions(+), 16 deletions(-)

diff --git a/Osstest/BuildSupport.pm b/Osstest/BuildSupport.pm
index a183546..dfdf2e4 100644
--- a/Osstest/BuildSupport.pm
+++ b/Osstest/BuildSupport.pm
@@ -36,6 +36,7 @@ BEGIN {
   $whhost $ho
 
   builddirsprops
+  buildcmd_stamped_logged
   $builddir $makeflags
 
   prepbuilddirs
@@ -56,6 +57,20 @@ our ($whhost,$ho);
 our ($builddir,$makeflags);
 our ($xendist);
 
+sub buildcmd_stamped_logged ($$) {
+my ($timeout, $component, $stampname, $prefix, $cmd, $suffix) = @_;
+target_cmd_build($ho, $timeout, $builddir, <&1 && touch ../$stampname-ok-stamp
+) |tee ../$stampname-log
+test -f ../$stampname-ok-stamp
+$suffix
+echo ok.
+END
+#/;
+}
+
 sub selectbuildhost {
 # pass \@ARGV
 my ($av) = @_;
diff --git a/ts-xen-build b/ts-xen-build
index 58670f1..5e076d7 100755
--- a/ts-xen-build
+++ b/ts-xen-build
@@ -95,20 +95,6 @@ END
);
 }
 
-sub buildcmd_stamped_logged ($) {
-my ($timeout, $stampname, $prefix, $cmd, $suffix) = @_;
-target_cmd_build($ho, $timeout, $builddir, <&1 && touch ../$stampname-ok-stamp
-) |tee ../$stampname-log
-test -f ../$stampname-ok-stamp
-$suffix
-echo ok.
-END
-#/;
-}
-
 sub build () {
 my $xend_opt= $r{enable_xend} =~ m/true/ ? "--enable-xend" : 
"--disable-xend";
 my $ovmf_opt= $r{enable_ovmf} =~ m/true/ ? "--enable-ovmf" : 
"--disable-ovmf";
@@ -116,7 +102,7 @@ sub build () {
 my $configure_prefix = $r{cmdprefix_configure} // '';
 my $make_prefix =  $r{cmdprefix_make}  // '';
 
-buildcmd_stamped_logged(600, 'configure', 

[Xen-devel] [OSSTEST PATCH RFC v2 09/14] mfi-common: create xtf build job for 4.8 onwards

[Wei Liu]

Signed-off-by: Wei Liu 
---
 mfi-common | 29 +
 1 file changed, 29 insertions(+)

diff --git a/mfi-common b/mfi-common
index 971ded3..b1e8dab 100644
--- a/mfi-common
+++ b/mfi-common
@@ -67,6 +67,21 @@ xenbranch_xsm_variants () {
 esac
 }
 
+xenbranch_wants_xtf_tests () {
+case "$xenbranch" in
+xen-3.*-testing) return 1;;
+xen-4.0-testing) return 1;;
+xen-4.1-testing) return 1;;
+xen-4.2-testing) return 1;;
+xen-4.3-testing) return 1;;
+xen-4.4-testing) return 1;;
+xen-4.5-testing) return 1;;
+xen-4.6-testing) return 1;;
+xen-4.7-testing) return 1;;
+*) return 0;;
+esac
+}
+
 job_create_build () {
   job_create_build_filter_callback "$@" || return 0
 
@@ -265,6 +280,20 @@ create_build_jobs () {
 
 fi
 
+if xenbranch_wants_xtf_tests; then
+# Only x86, build once for amd64 and use the same result for
+# both amd64 and i386
+if [ x$arch = xamd64 ] ; then
+job_create_build build-$arch-xtf build-xtf   \
+arch=$arch   \
+$RUNVARS $BUILD_RUNVARS $BUILD_XTF_RUNVARS $arch_runvars \
+$hostos_runvars  \
+host_hostflags=$build_hostflags  \
+tree_xtf=$TREE_XTF   \
+revision_xtf=$REVISION_XTF
+fi
+fi
+
 if branch_wants_rumpkernel_tests; then
 
 case $arch in
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 01/14] ts-xen-build: always compile in FEP support

[Wei Liu]

By default FEP depends on debug flag. When we are near release the debug
flag will be turned off. In order to test a release build, we explicitly
enable FEP in build configuration.

Since we target Xen versions that already have Kconfig support, only a
Kconfig option is created for now.

We can easily add config option for older Xen when necessary.

Note that this only compiles in FEP support. To enable it a user needs
to explicitly specify fep=1 in hypervisor command line.

Signed-off-by: Wei Liu 
---
 ts-xen-build | 7 ++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/ts-xen-build b/ts-xen-build
index 5933dd4..58670f1 100755
--- a/ts-xen-build
+++ b/ts-xen-build
@@ -84,9 +84,14 @@ END
(nonempty($earlyprintk) ? <>.config CONFIG_EARLY_PRINTK=$earlyprintk
 END
-   ($ho->{Suite} =~ m/squeeze|wheezy/ ? <{Suite} =~ m/squeeze|wheezy/ ? <>.config PYTHON_PREFIX_ARG=
 END
+<>xen/.config CONFIG_HVM_FEP=y
+   fi
+END
);
 }
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 08/14] Introduce ts-xtf-install

[Wei Liu]

Extract XTF to the desire location.

Signed-off-by: Wei Liu 
---
v2: remove reference of runvar xtfdir
---
 ts-xtf-install | 37 +
 1 file changed, 37 insertions(+)
 create mode 100755 ts-xtf-install

diff --git a/ts-xtf-install b/ts-xtf-install
new file mode 100755
index 000..040772e
--- /dev/null
+++ b/ts-xtf-install
@@ -0,0 +1,37 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2016 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+our ($whhost) = @ARGV;
+$whhost ||= 'host';
+
+our $ho= selecthost($whhost);
+
+sub extract () {
+my %distpath;
+
+target_extract_jobdistpath($ho, "xtf", "path_xtfdist",
+   $r{xtfbuildjob}, \%distpath);
+}
+
+extract();
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 00/14] Integrate XTF into OSSTest

[Wei Liu]

Hi all

This patch series integrates XTF into OSSTest. It still depends on
quite a few things (listed below) but I think it has gotten to point
that it can be posted for review.

It depends on having a canonical location for xtf.git. Currently this
series contains a patch to point to my own xtf.git tree.

It depends on Ian's work:

1. Add a "skip" state to OSSTest.
2. Add diverse flag.

To make the XTf jobs reliable, we also need to:

1. Fix synchronisation issues in xenconsole related code in Xen.
2. Fix synchronisation issue in xtf-runner.

The effect of  synchronisations can be seen in [0]. Test 1 and 5
failed. I've posted patches for them ([2], [3]).

I created a dummy test case that always fails. We expect ts-xtf-run to
continue running after that. The result is at [1].

The test flight in [4] contained a hack that removed "skip" from accepcted
states, so test-pv64-xsa-167 became "never pass" (should be "skip" in normal
case). In job 4 test-pv32-pae-pv-iopl~vmassist failed due to xenconsole race.
The code correctly caused rest of the job to be skipped. The same race also
caused xtf-fep to fail in job 2, but that test won't cause the job to be
aborted anymore.
 

Wei.


[0] http://osstest.xs.citrite.net/~osstest/testlogs/logs/66910/
[1] http://osstest.xs.citrite.net/~osstest/testlogs/logs/66916/
[2] <1470418894-11358-1-git-send-email-wei.l...@citrix.com>
[3] <1470060258-20084-1-git-send-email-wei.l...@citrix.com>
[4] http://osstest.xs.citrite.net/~osstest/testlogs/logs/66926/

[0] and [1] are from v1 of this series, [4] is from v2 (this version).

* Currently all logs are only accessible from within Citrix. Sorry. :-/

Wei Liu (14):
  ts-xen-build: always compile in FEP support
  DO NOT APPLY ts-leak-check: sleep 5 seconds before collecting stuff
  ap-common: add xtf tree
  DO NOT APPLY point xtf to my personal tree
  BuildSupport: move buildcmd_stamped_logged here
  Introduce ts-xtf-build
  sg-run-job: create xtf build recipe
  Introduce ts-xtf-install
  mfi-common: create xtf build job for 4.8 onwards
  Introduce ts-xtf-fep
  Introduce ts-xtf-run
  sg-run-job: test-xtf recipe
  make-flight: create 5 xtf jobs
  Create XTF branch

 Osstest/BuildSupport.pm |  15 ++
 ap-common   |   4 ++
 ap-fetch-version|   4 ++
 ap-print-url|   3 ++
 ap-push |   5 ++
 cr-daily-branch |   8 
 cr-for-branches |   2 +-
 cri-common  |   1 +
 make-flight |  40 
 mfi-common  |  29 
 sg-run-job  |  15 ++
 ts-leak-check   |   3 ++
 ts-xen-build|  25 --
 ts-xtf-build|  54 +
 ts-xtf-fep  |  37 +++
 ts-xtf-install  |  37 +++
 ts-xtf-run  | 122 
 17 files changed, 386 insertions(+), 18 deletions(-)
 create mode 100755 ts-xtf-build
 create mode 100755 ts-xtf-fep
 create mode 100755 ts-xtf-install
 create mode 100755 ts-xtf-run

-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 06/14] Introduce ts-xtf-build

[Wei Liu]

Clone, build and package XTF for later use.

Signed-off-by: Wei Liu 
---
v2:
1. use buildcmd_stamped_logged
2. pass @ARGV to make
3. use /home/xtf
---
 ts-xtf-build | 54 ++
 1 file changed, 54 insertions(+)
 create mode 100755 ts-xtf-build

diff --git a/ts-xtf-build b/ts-xtf-build
new file mode 100755
index 000..1af1b4e
--- /dev/null
+++ b/ts-xtf-build
@@ -0,0 +1,54 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2016 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+use strict qw(vars);
+use DBI;
+use Osstest;
+use File::Path;
+use POSIX;
+use Osstest::TestSupport;
+use Osstest::BuildSupport;
+
+tsreadconfig();
+selectbuildhost(\@ARGV);
+# remaining arguments are passed as targets to "make"
+builddirsprops();
+
+sub checkout () {
+prepbuilddirs();
+
+build_clone($ho, 'xtf', $builddir, 'xtf');
+}
+
+sub build_and_install () {
+
+buildcmd_stamped_logged(300, 'xtf', 'build', '', 

[Xen-devel] [ovmf test] 100328: all pass - PUSHED

[osstest service owner]

flight 100328 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100328/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 74bbe31b8d485da26ec7ffad5e78b8384a9eb9a5
baseline version:
 ovmf 7e9cf612056ed2667eaad7f0e901f0a37f6ddd5b

Last test of basis8  2016-08-06 17:14:25 Z1 days
Testing same since   100328  2016-08-08 03:15:26 Z0 days1 attempts


People who touched revisions under test:
  Dandan Bi 
  Hao Wu 
  Yonghong Zhu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=ovmf
+ revision=74bbe31b8d485da26ec7ffad5e78b8384a9eb9a5
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push ovmf 
74bbe31b8d485da26ec7ffad5e78b8384a9eb9a5
+ branch=ovmf
+ revision=74bbe31b8d485da26ec7ffad5e78b8384a9eb9a5
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=ovmf
+ xenbranch=xen-unstable
+ '[' xovmf = xlinux ']'
+ linuxbranch=
+ '[' x = x ']'
+ qemuubranch=qemu-upstream-unstable
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable
+ prevxenbranch=xen-4.7-testing
+ '[' x74bbe31b8d485da26ec7ffad5e78b8384a9eb9a5 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src 
'[fetch=try]'
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local 'options=[fetch=try]'
 getconfig GitCacheProxy
 perl -e '
use Osstest;
readglobalconfig();
print $c{"GitCacheProxy"} or die $!;
'
+++ local cache=git://cache:9419/
+++ '[' xgit://cache:9419/ '!=' x ']'
+++ echo 
'git://cache:9419/https://github.com/rumpkernel/rumpkernel-netbsd-src%20[fetch=tr

Re: [Xen-devel] [PATCH] tools: xenalyze: kill spurious sched_switch output in non dump mode.

[George Dunlap]

On 04/08/16 09:59, Dario Faggioli wrote:
> In fact, 52cf096df7 ("xenalyze: handle scheduling event"),
> when dealing with TRC_SCHED_SWITCH, forgot to check whether
> we actually are in dump mode, causing the printf() in
> dump_sched_switch() to always produce its output, which
> is not what we want.
> 
> Signed-off-by: Dario Faggioli 

Acked-by: George Dunlap 

And queued.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [Design RFC] Towards work-conserving RTDS scheduler

[Dario Faggioli]

On Thu, 2016-08-04 at 01:15 -0400, Meng Xu wrote:
> Hi Dario,
> 
Hi,

> I'm thinking about changing the current RTDS scheduler to
> work-conserving version as we briefly discussed before.
> Below is a design of the work-conserving RTDS.
> I'm hoping to get your feedback about the design ideas first before I
> start writing it in code.
> 
Here I am, sorry for the delay.

> I think the code change should not be a lot as long as we don't
> provide the functionality of switching between work-conserving and
> non-work-conserving. Because the following design will keep the
> real-time property of the current RTDS scheduler, I don't see the
> reason why we should let users switch to non-work-conserving version.
> :-)
> 
Oh, but there's a bit one: _money_! :-O

If you're a service/cloud provided you may or may not want that a
customers that pays for a 40% utilization VM to be able to use more
than that. In particular, you may want to ask more money to them, in
order to enable that possibility! :-P

Anyway, I don't think --with this design of yours-- that it is such a
big deal to make it possible to switch work-conserving*ness on and off
(see below). Actually, I think it's even possible to to that on a per-
vcpu basis, which I think would be quite cool!

> --- Below is the design ---
> 
> [...]
>
> *** Requirement of the work-conserving RTDS scheduler ***
> 1) The new RTDS scheduler should be work-conserving, of course.
> 2) The new RTDS scheduler should not break any real-time guarantee
> provided by the current RTDS scheduler.
> 
> *** Design of  Work-Conserving RTDS Scheduler ***
> VCPU model
> 1) (Period, Budget): Guaranteed  time for each 
> 2) Priority index: It indicates the current  priority level of the
> VCPU. When a VCPU’s budget is depleted in the current period, its
> priority index will increase by 1 and its budget will be replenished.
> 3) A VCPU’s budget and priority index will be reset at the beginning
> of each period
> 
Ok, I think I see what you mean and it looks to make sense to me.

Just one question/observation. As you know, I come from a CBS mindset.
CBS postpones a task/vcpu's deadline when it runs out of budget, and it
can, natively, work in work conserving or non-work conserving mode
(just by wither continue to consider the vcpu runnable, with the later
deadline which mean demoted priority, or block it until the next
period, respectively).

The nice thing about this is that the scheduling analysis that has been
developed works for both modes. Of course, what it says is that you can
only guarantee to each vcpu the reserved utilization, and you should
not rely on the additional capacity that you may be getting because
you're in work conserving mode and the system happened to be idle for a
few time this or that other period (so, very similar to what you're
proposing). _HOWEVER_, there are evolutions of CBS (called GRUB and
SHRUB, I'm sure you'll be able to find the papers), where the 'unused
bandwidth' (i.e., the otherwise idle time that you're making use of iff
you're in work conserving mode) is distributed in a precise way
(according to some weights, IIRC) to the various vcpus, hence making
scheduling analysis both possible and useful again.

Now, I'm not at all saying that we (you! :-D) should RTDS into using
CBS(ish) or anything like that. I'm just thinking out loud and
wondering:
 - could it be useful to have a scheduling analysis in place for the 
   scheduler in work conserving mode (one, of course, that takes into 
   account and give guarantees on the otherwise idle bandwidth... I 
   know that the existing one holds! :-P) ?
 - if yes, do you already have one --or do you think it will be 
   possible to develop one-- for your priority-index based model?

Note that I'm not saying you should, and I'd be perfectly fine with a
"no analysis, but let's keep things simple for now"... This just came
to my mind, and I'm just pointing it ouy, to make sure we consider and
think about it, and make a conscious decision.

> Scheduling policy: modified gEDF
> 1) Priority comparison:
>    a) VCPUs with lower priority index has higher priority than VCPUs
> with higher priority index
>    b) VCPUs with same priority index uses gEDF policy to decide the
> priority order
> 2) Scheduling point
>    a) VCPU’s budget is depleted for the current priority index
>    b) VCPU starts a new period
>    c) VCPU is blocked or waked up
> 3) Scheduling decision is made when scheduler is invoked
> a) Always pick the current M highest-priority VCPUs to run on the
> M cores.
> 
So, still about the analysis point above, and just out of the top of my
head (and without being used to do this things any longer!!), it looks
like it's possible think at some analysis for this.

In fact, since:
 - vcpus with different priority indexes are totally disjoint sets,
 - there's a strict ordering between priority indexes,
 - vcpus sort of use their scheduling parameters at each priority index

This looks to me like vcpus are subj

Re: [Xen-devel] live migration to qemu.git fails

[Olaf Hering]

On Fri, Aug 05, Olaf Hering wrote:

> Not sure what change was made to hide the error.

On the receiving side, /usr/bin/qemu-system-i386 works while
/usr/bin/qemu-system-x86_64 fails. Since the sending side is
/usr/lib/xen/bin/qemu-system-i386 there is appearenly an incompatibality
between the i386 and the x86_64 binary.

Not sure if there is any gain in running qemu-system-x86_64, so I think
its best to adjust our --with-system-qemu to use qemu-system-i386.

Olaf


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-4.5-testing bisection] complete test-amd64-amd64-xl-pvh-intel

[osstest service owner]

branch xen-4.5-testing
xenbranch xen-4.5-testing
job test-amd64-amd64-xl-pvh-intel
testid xen-boot

Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git

*** Found and reproduced problem changeset ***

  Bug is in tree:  xen git://xenbits.xen.org/xen.git
  Bug introduced:  c421378a8d14c811e5467d535bc71adc0328a316
  Bug not present: b1f4e86aa3bd224bde62f18cf51381e6fe731a2f
  Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/100337/


  commit c421378a8d14c811e5467d535bc71adc0328a316
  Author: George Dunlap 
  Date:   Fri Aug 5 14:07:27 2016 +0200
  
  xen: Have schedulers revise initial placement
  
  The generic domain creation logic in
  xen/common/domctl.c:default_vcpu0_location() attempts to try to do
  initial placement load-balancing by placing vcpu 0 on the least-busy
  non-primary hyperthread available.  Unfortunately, the logic can end
  up picking a pcpu that's not in the online mask.  When this is passed
  to a scheduler such which assumes that the initial assignment is
  valid, it causes a null pointer dereference looking up the runqueue.
  
  Furthermore, this initial placement doesn't take into account hard or
  soft affinity, or any scheduler-specific knowledge (such as historic
  runqueue load, as in credit2).
  
  To solve this, when inserting a vcpu, always call the per-scheduler
  "pick" function to revise the initial placement.  This will
  automatically take all knowledge the scheduler has into account.
  
  csched2_cpu_pick ASSERTs that the vcpu's pcpu scheduler lock has been
  taken.  Grab and release the lock to minimize time spend with irqs
  disabled.
  
  Signed-off-by: George Dunlap 
  Reviewed-by: Meng Xu 
  Reviwed-by: Dario Faggioli 
  master commit: 9f358ddd69463fa8fb65cf67beb5f6f0d3350e32
  master date: 2016-07-26 10:42:49 +0100


For bisection revision-tuple graph see:
   
http://logs.test-lab.xenproject.org/osstest/results/bisect/xen-4.5-testing/test-amd64-amd64-xl-pvh-intel.xen-boot.html
Revision IDs in each graph node refer, respectively, to the Trees above.


Running cs-bisection-step 
--graph-out=/home/logs/results/bisect/xen-4.5-testing/test-amd64-amd64-xl-pvh-intel.xen-boot
 --summary-out=tmp/100337.bisection-summary --basis-template=99752 
--blessings=real,real-bisect xen-4.5-testing test-amd64-amd64-xl-pvh-intel 
xen-boot
Searching for failure / basis pass:
 99963 fail [host=godello1] / 99752 [host=godello0] 96516 ok.
Failure / basis pass flights: 99963 / 96516
(tree with no url: ovmf)
(tree with no url: seabios)
Tree: linux git://xenbits.xen.org/linux-pvops.git
Tree: linuxfirmware git://xenbits.xen.org/osstest/linux-firmware.git
Tree: qemu git://xenbits.xen.org/qemu-xen-traditional.git
Tree: qemuu git://xenbits.xen.org/qemu-xen.git
Tree: xen git://xenbits.xen.org/xen.git
Latest da99423b3cd3e48c42c0d64b79aba58d828f9648 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
28c21388c2a32259cff37fc578684f994dca8c9f 
835c204f1196ab8f5213a9dc5299ed76e748cdca 
c18c1456c48f23d9b31e7a32a21aa1ae9c53df93
Basis pass 44dd5e6b1cf505485d839bd62d47e29a36232d67 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
28c21388c2a32259cff37fc578684f994dca8c9f 
5e40cec825a2582d8a91119c485f5130cc2648e9 
eadd6636fae2abe1608207569e32c8457e37c653
Generating revisions with ./adhoc-revtuple-generator  
git://xenbits.xen.org/linux-pvops.git#44dd5e6b1cf505485d839bd62d47e29a36232d67-da99423b3cd3e48c42c0d64b79aba58d828f9648
 
git://xenbits.xen.org/osstest/linux-firmware.git#c530a75c1e6a472b0eb9558310b518f0dfcd8860-c530a75c1e6a472b0eb9558310b518f0dfcd8860
 
git://xenbits.xen.org/qemu-xen-traditional.git#28c21388c2a32259cff37fc578684f994dca8c9f-28c21388c2a32259cff37fc578684f994dca8c9f
 
git://xenbits.xen.org/qemu-xen.git#5e40cec825a2582d8a91119c485f5130cc2648e9-835c204f1196ab8f5213a9dc5299ed76e748cdca
 
git://xenbits.xen.org/xen.git#eadd6636fae2abe1608207569e32c8457e37c653-c18c1456c48f23d9b31e7a32a21aa1ae9c53df93
From git://cache:9419/git://xenbits.xen.org/xen
   7fb0a87..7279829  staging-> origin/staging
Loaded 3006 nodes in revision graph
Searching for test results:
 96511 pass 44dd5e6b1cf505485d839bd62d47e29a36232d67 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
28c21388c2a32259cff37fc578684f994dca8c9f 
5e40cec825a2582d8a91119c485f5130cc2648e9 
eadd6636fae2abe1608207569e32c8457e37c653
 96516 pass 44dd5e6b1cf505485d839bd62d47e29a36232d67 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
28c21388c2a32259cff37fc578684f994dca8c9f 
5e40cec825a2582d8a91119c485f5130cc2648e9 
eadd6636fae2abe1608207569e32c8457e37c653
 99752 [host=godello0]
 99963 fail da99423b3cd3e48c42c0d64b79aba58d828f9648 
c530a75c1e6a472b0eb9558310b518f0dfcd8860 
28c21388c2a32259cff37fc578684f

[Xen-devel] [OSSTEST PATCH RFC v2 11/14] Introduce ts-xtf-run

[Wei Liu]

This is the main script for running XTF.  It will first perform
selftest, and then run each XTF test case as a substep.

It does the following things:

1. Run self tests for individual environment and record the result.
2. Collect tests according to available environments.
3. Run the collected tests one by one.

The script may exit early if it detects the test host is down or
xtf-runner returns non-recognisable exit code.

Signed-off-by: Wei Liu 
---
 ts-xtf-run | 122 +
 1 file changed, 122 insertions(+)
 create mode 100755 ts-xtf-run

diff --git a/ts-xtf-run b/ts-xtf-run
new file mode 100755
index 000..c72ac18
--- /dev/null
+++ b/ts-xtf-run
@@ -0,0 +1,122 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2016 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+use strict qw(vars);
+use Osstest;
+use POSIX;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+our $ho = selecthost('host');
+
+our $runner = "/home/xtf/xtf-runner";
+our @tests;
+our @all_environments;
+our @environments;
+
+# XTF results (runner returned numeric values) and OSStest results:
+#
+# SUCCESS(0)  -> pass
+# SKIP(3) -> skip
+# ERROR(4)-> fail
+# FAILURE(5)  -> fail
+#
+sub xtf_result_to_osstest_result ($) {
+my ($xret) = @_;
+
+return "pass" if $xret == 0;
+return "skip" if $xret == 3;
+return "fail" if $xret == 4;
+return "fail" if $xret == 5;
+die "xtf runner gave unexpected result $xret";
+}
+
+sub get_environments () {
+my $output = target_cmd_output_root($ho, <&2; echo \$?
+END
+$output =~ m/^(\d+)$/ or die "$output ?";
+$ret = $1;
+
+target_cmd_output_root($ho, 'echo ok.');
+$osstest_result = xtf_result_to_osstest_result($ret);
+1;
+}) {
+logm("XTF test threw exception:\n$@");
+substep_finish($tid, "fail");
+exit 0;
+}
+
+logm("Test $name, XTF returned $ret -> OSSTest $osstest_result");
+substep_finish($tid, $osstest_result);
+
+return $ret;
+}
+
+# Run selftest for each environment, record the ones that are
+# functional to get maximum coverage.
+sub do_selftest () {
+foreach my $e (sort @all_environments) {
+my $output = target_cmd_output_root($ho, 

[Xen-devel] [OSSTEST PATCH RFC v2 10/14] Introduce ts-xtf-fep

[Wei Liu]

Test the availability of FEP during runtime.

Signed-off-by: Wei Liu 
---
v2:
1. use target_cmd_output_root to get output
2. die if the output is not in expected format
3. use fep test result as exit code of the script
4. remove the use of  xtfdir runvar
---
 ts-xtf-fep | 37 +
 1 file changed, 37 insertions(+)
 create mode 100755 ts-xtf-fep

diff --git a/ts-xtf-fep b/ts-xtf-fep
new file mode 100755
index 000..c5222e0
--- /dev/null
+++ b/ts-xtf-fep
@@ -0,0 +1,37 @@
+#!/usr/bin/perl -w
+# This is part of "osstest", an automated testing framework for Xen.
+# Copyright (C) 2016 Citrix Inc.
+#
+# This program is free software: you can redistribute it and/or modify
+# it under the terms of the GNU Affero General Public License as published by
+# the Free Software Foundation, either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU Affero General Public License for more details.
+#
+# You should have received a copy of the GNU Affero General Public License
+# along with this program.  If not, see .
+
+use strict qw(vars);
+use Osstest;
+use POSIX;
+use Osstest::TestSupport;
+
+tsreadconfig();
+
+our $ho = selecthost('host');
+
+sub fep_test () {
+my $output = target_cmd_output_root($ho, <&2; echo \$?
+END
+
+$output =~ m/^(\d+)$/ or die "$output ?";
+
+return $1;
+}
+
+exit fep_test();
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 13/14] make-flight: create 5 xtf jobs

[Wei Liu]

Create jobs only for x86 and set host flag diverse-xtf-x86.

Signed-off-by: Wei Liu 
---
 make-flight | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/make-flight b/make-flight
index 18e5bc3..613b3d0 100755
--- a/make-flight
+++ b/make-flight
@@ -422,6 +422,26 @@ do_rtds_tests () {
 $debian_runvars all_hostflags=$most_hostflags
 }
 
+do_xtf_tests () {
+  if ! xenbranch_wants_xtf_tests; then
+  return
+  fi
+
+  # xtf is only meaningful to x86. And amd64 and i386 should be the same,
+  # create only for one subarch is good enough.
+  if [ x$xenarch != xamd64 -o x$dom0arch != xamd64 ]; then
+  return;
+  fi
+
+  for i in `seq 1 5`; do
+  job_create_test test-xtf-$xenarch-$dom0arch-$i \
+   test-xtf xl $xenarch $dom0arch\
+xtfbuildjob=build-$xenarch-xtf   \
+xen_boot_append='hvm_fep=1'  \
+all_hostflags=$most_hostflags,diverse-xtf-x86
+  done
+}
+
 do_multivcpu_tests () {
   if [ $xenarch != $dom0arch ]; then
 return
@@ -705,6 +725,8 @@ test_matrix_do_one () {
 
   do_pygrub_tests
   do_pvgrub_tests
+
+  do_xtf_tests
 }
 
 if [ x$buildflight = x ]; then
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 14/14] Create XTF branch

[Wei Liu]

This branch contains Xen and Linux build jobs and all jobs which contain
xtf in their name.

Signed-off-by: Wei Liu 
---
I'm not sure if everything is ok because I can't test all changes.
---
 ap-fetch-version |  4 
 ap-print-url |  3 +++
 ap-push  |  5 +
 cr-daily-branch  |  8 
 cr-for-branches  |  2 +-
 cri-common   |  1 +
 make-flight  | 18 ++
 7 files changed, 40 insertions(+), 1 deletion(-)

diff --git a/ap-fetch-version b/ap-fetch-version
index f26d60a..fa78d51 100755
--- a/ap-fetch-version
+++ b/ap-fetch-version
@@ -98,6 +98,10 @@ seabios)
repo_tree_rev_fetch_git seabios \
$TREE_SEABIOS_UPSTREAM master $LOCALREV_SEABIOS
;;
+xtf)
+   repo_tree_rev_fetch_git xtf \
+   $TREE_XTF_UPSTREAM master $LOCALREV_XTF
+   ;;
 ovmf)
repo_tree_rev_fetch_git ovmf \
$TREE_OVMF_UPSTREAM master $LOCALREV_OVMF
diff --git a/ap-print-url b/ap-print-url
index 4088852..8c5e604 100755
--- a/ap-print-url
+++ b/ap-print-url
@@ -58,6 +58,9 @@ rumpuserxen)
 seabios)
echo $TREE_SEABIOS_UPSTREAM
;;
+xtf)
+   echo $TREE_XTF_UPSTREAM
+   ;;
 ovmf)
echo $TREE_OVMF_UPSTREAM
;;
diff --git a/ap-push b/ap-push
index e33a803..63b7f80 100755
--- a/ap-push
+++ b/ap-push
@@ -40,6 +40,7 @@ TREE_LIBVIRT=$PUSH_TREE_LIBVIRT
 TREE_RUMPUSERXEN=$PUSH_TREE_RUMPUSERXEN
 TREE_SEABIOS=$PUSH_TREE_SEABIOS
 TREE_OVMF=$PUSH_TREE_OVMF
+TREE_XTF=$PUSH_TREE_XTF
 
 if info_linux_tree "$branch"; then
cd $repos/linux
@@ -120,6 +121,10 @@ seabios)
cd $repos/seabios
git push $TREE_SEABIOS $revision:refs/heads/xen-tested-master
;;
+xtf)
+   cd $repos/xtf
+   git push $TREE_XTF $revision:refs/heads/xen-tested-master
+   ;;
 ovmf)
cd $repos/ovmf
git push $TREE_OVMF $revision:refs/heads/xen-tested-master
diff --git a/cr-daily-branch b/cr-daily-branch
index 21780b8..3f74e2e 100755
--- a/cr-daily-branch
+++ b/cr-daily-branch
@@ -195,6 +195,10 @@ if [ "x$REVISION_LIBVIRT" = x ]; then
determine_version REVISION_LIBVIRT libvirt LIBVIRT
export REVISION_LIBVIRT
 fi
+if [ "x$REVISION_XTF" = x ]; then
+   determine_version REVISION_XTF xtf XTF
+   export REVISION_XTF
+fi
 if [ "x$REVISION_RUMPUSERXEN" = x ]; then
determine_version REVISION_RUMPUSERXEN rumpuserxen RUMPUSERXEN
export REVISION_RUMPUSERXEN
@@ -248,6 +252,10 @@ seabios)
realtree=seabios
NEW_REVISION=$REVISION_SEABIOS
;;
+xtf)
+   realtree=xtf
+   NEW_REVISION=$REVISION_XTF
+   ;;
 ovmf)
realtree=ovmf
NEW_REVISION=$REVISION_OVMF
diff --git a/cr-for-branches b/cr-for-branches
index 6a161d4..5e8b1a4 100755
--- a/cr-for-branches
+++ b/cr-for-branches
@@ -31,7 +31,7 @@ scriptoptions="$1"; shift
 LOGFILE=tmp/cr-for-branches.log
 export LOGFILE
 
-: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing 
xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-4.7-testing 
xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing 
qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing 
qemu-upstream-4.6-testing qemu-upstream-4.7-testing linux-4.1 linux-3.18 
linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf 
${EXTRA_BRANCHES}}
+: ${BRANCHES:=osstest xen-4.0-testing xen-4.1-testing xen-4.2-testing 
xen-4.3-testing xen-4.4-testing xen-4.5-testing xen-4.6-testing xen-4.7-testing 
xen-unstable qemu-mainline qemu-upstream-unstable qemu-upstream-4.2-testing 
qemu-upstream-4.3-testing qemu-upstream-4.4-testing qemu-upstream-4.5-testing 
qemu-upstream-4.6-testing qemu-upstream-4.7-testing linux-4.1 linux-3.18 
linux-3.16 linux-3.14 linux-3.10 linux-3.4 linux-arm-xen seabios ovmf xtf 
${EXTRA_BRANCHES}}
 export BRANCHES
 
 fetchwlem=$wlem
diff --git a/cri-common b/cri-common
index cdee48d..6d5545c 100644
--- a/cri-common
+++ b/cri-common
@@ -78,6 +78,7 @@ select_xenbranch () {
libvirt)tree=libvirt;   xenbranch=xen-unstable ;;
rumpuserxen)  tree=rumpuserxen; xenbranch=xen-unstable ;;
seabios)tree=seabios;   xenbranch=xen-unstable ;;
+   xtf)tree=xtf;   xenbranch=xen-unstable ;;
ovmf)   tree=ovmf;  xenbranch=xen-unstable ;;
distros-*)  tree=none;  xenbranch=xen-unstable ;;
osstest)tree=osstest;   xenbranch=xen-unstable ;;
diff --git a/make-flight b/make-flight
index 613b3d0..8aec1a7 100755
--- a/make-flight
+++ b/make-flight
@@ -75,6 +75,14 @@ job_create_build_filter_callback () {
 *) return 1 ;;
   esac
 ;;
+xtf)
+  case "$job" in
+build-amd64)   ;;
+build-amd64-pvops) ;;
+build-*-xtf)   ;;
+*) return 1 ;;
+  esac
+;;
   esac
   return 0
 }
@@ -11

Re: [Xen-devel] Xen 4.6.1 crash with altp2m enabledbydefault

[Kevin.Mayer]

vmx_vmenter_helper is not part of the call stack. The address is simply the 
location of the ud2 to which the 
__vmwrite(HOST_CR0, v->arch.hvm_vmx.host_cr0);
In
static void vmx_fpu_leave(struct vcpu *v)
jumps.
There are two vmwrites in vmx_vcpu_update_eptp (called by altp2m_vcpu_destroy):
__vmwrite(EPT_POINTER, ept_get_eptp(ept));
__vmwrite(EPTP_INDEX, vcpu_altp2m(v).p2midx);

And four in vmx_vcpu_update_vmfunc_ve (also called by altp2m_vcpu_destroy)
__vmwrite(VM_FUNCTION_CONTROL, VMX_VMFUNC_EPTP_SWITCHING);
__vmwrite(EPTP_LIST_ADDR, virt_to_maddr(d->arch.altp2m_eptp));
__vmwrite(VIRT_EXCEPTION_INFO, mfn_x(mfn) << PAGE_SHIFT);
__vmwrite(SECONDARY_VM_EXEC_CONTROL,  v->arch.hvm_vmx.secondary_exec_control);

After the altp2m-part hvm_vcpu_destroy also calls nestedhvm_vcpu_destroy(v), 
but this code path is executed unconditionally so I assume that the error lies 
somewhere in the altp2m_vcpu_destroy(v).

What exactly are the vmx_vmcs_enter / exit required for? I often see the 
vmx_vmcs_enter; __vmwrite; vmx_vmcs_exit combination. Need the __vmwrites be 
guarded by an enter / exit ( which Is not the case in the static void 
vmx_fpu_leave(struct vcpu *v) )?
Is it possible that the 
altp2m_vcpu_destroy->vmx_vcpu_update_eptp->vmx_vmcs_exit->vmx_clear_vmcs 
invalidates the vmcs for the current vcpu?

Cheers

Kevin

> -Ursprüngliche Nachricht-
> Von: Jan Beulich [mailto:jbeul...@suse.com]
> Gesendet: Freitag, 5. August 2016 16:49
> An: Mayer, Kevin 
> Cc: andrew.coop...@citrix.com; xen-devel@lists.xen.org
> Betreff: Re: AW: AW: AW: [Xen-devel] Xen 4.6.1 crash with altp2m enabled
> bydefault
> 
> >>> On 05.08.16 at 14:51,  wrote:
> > According to the xen dmesg
> >
> > (XEN) RIP:e008:[]
> vmx_vmenter_helper+0x27e/0x30a
> > (XEN) RFLAGS: 00010003   CONTEXT: hypervisor
> > (XEN) rax: 8005003b   rbx: 8300e72fc000   rcx:
> 
> > (XEN) rdx: 6c00   rsi: 830617fd7fc0   rdi: 8300e6fc
> > (XEN) rbp: 830617fd7c40   rsp: 830617fd7c30   r8:  
> > (XEN) r9:  830be8dc9310   r10:    r11: 3475e9cf85d0
> > (XEN) r12: 0006   r13: 830c14ee1000   r14: 8300e6fc
> > (XEN) r15: 830617fd   cr0: 8005003b   cr4:
> 26e0
> > (XEN) cr3: 0001bd665000   cr2: 0451
> > (XEN) ds:    es:    fs:    gs:    ss:    cs: e008
> >
> > 0x82d0801fa0c3 :mov$0x6c00,%edx
> > 0x82d0801fa0c8 :vmwrite %rax,%rdx
> >
> > The vmwrite tries to write 0x8005003b   to 0x6c00.
> > But the active VCPU has a 0-vmcs-pointer.
> 
> Which likely means altp2m manages to confuse some of VMX'es VMCS
> management - vmx_vmenter_helper() being on the path back to the guest,
> it should be impossible for the VMCS pointer to be zero here. Can you
> perhaps identify the most recent vmread or vmwrite which worked fine?
> There ought to be many on that path, and the state corruption could then
> perhaps be narrowed to quite small a range of code.
> 
> Jan

Virus checked by G Data MailSecurity
Version: AVA 25.7794 dated 08.08.2016
Virus news: www.antiviruslab.com

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [OSSTEST PATCH RFC v2 12/14] sg-run-job: test-xtf recipe

[Wei Liu]

Install XTF, run FEP test and then run all available tests.

Signed-off-by: Wei Liu 
---
v2: spawn and reap ts-xtf-fep to make it non-fatal
---
 sg-run-job | 10 ++
 1 file changed, 10 insertions(+)

diff --git a/sg-run-job b/sg-run-job
index 240b265..c9773ab 100755
--- a/sg-run-job
+++ b/sg-run-job
@@ -411,6 +411,16 @@ proc run-job/test-nested {} {
 run-ts . = ts-guest-stop l1 l2
 }
 
+proc need-hosts/test-xtf {} { return host }
+proc run-job/test-xtf {} {
+run-ts . = ts-xtf-install
+
+# Spawn and reap fep test to make it non-fatal
+reap-ts [spawn-ts . = ts-xtf-fep]
+
+run-ts . = ts-xtf-run
+}
+
 proc test-guest-migr {g} {
 set to_reap [spawn-ts . = ts-migrate-support-check + host $g 1]
 set can_migrate [reap-ts $to_reap]
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH] x86/traps: Drop use_error_code parameter from do_{, guest_}trap()

[Andrew Cooper]

Whether or not an error code is needed can be determinted entirely from the
trapnr paramter, as error codes are architecturally specified.

Introduce TRAP_HAVE_EC as a bitmap of reserved vectors which have error codes,
and drop the use_error_code from all callsites.

As a result, the DO_ERROR{,_NOCODE}() macros become entirely superflouous and
can be dropped.  Update the exception_table to point straight at do_trap().

Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 

Bloat-o-meter shows a modest net code decrease of 264 bytes.
---
 xen/arch/x86/traps.c| 91 +
 xen/arch/x86/x86_64/entry.S | 18 
 xen/include/asm-x86/processor.h |  6 +++
 3 files changed, 52 insertions(+), 63 deletions(-)

diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index b042976..c228b45 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -625,12 +625,17 @@ void fatal_trap(const struct cpu_user_regs *regs, bool_t 
show_remote)
   (regs->eflags & X86_EFLAGS_IF) ? "" : ", IN INTERRUPT CONTEXT");
 }
 
-static void do_guest_trap(
-int trapnr, const struct cpu_user_regs *regs, int use_error_code)
+static void do_guest_trap(unsigned int trapnr,
+  const struct cpu_user_regs *regs)
 {
 struct vcpu *v = current;
 struct trap_bounce *tb;
 const struct trap_info *ti;
+bool_t use_error_code;
+
+ASSERT(trapnr < 32);
+
+use_error_code = (TRAP_HAVE_EC & (1u << trapnr));
 
 trace_pv_trap(trapnr, regs->eip, use_error_code, regs->error_code);
 
@@ -666,7 +671,7 @@ static void instruction_done(
 current->arch.debugreg[6] |= bpmatch | DR_STATUS_RESERVED_ONE;
 if ( regs->eflags & X86_EFLAGS_TF )
 current->arch.debugreg[6] |= DR_STEP;
-do_guest_trap(TRAP_debug, regs, 0);
+do_guest_trap(TRAP_debug, regs);
 }
 }
 
@@ -714,7 +719,7 @@ int set_guest_machinecheck_trapbounce(void)
 struct vcpu *v = current;
 struct trap_bounce *tb = &v->arch.pv_vcpu.trap_bounce;
  
-do_guest_trap(TRAP_machine_check, guest_cpu_user_regs(), 0);
+do_guest_trap(TRAP_machine_check, guest_cpu_user_regs());
 tb->flags &= ~TBF_EXCEPTION; /* not needed for MCE delivery path */
 return !null_trap_bounce(v, tb);
 }
@@ -727,7 +732,7 @@ int set_guest_nmi_trapbounce(void)
 {
 struct vcpu *v = current;
 struct trap_bounce *tb = &v->arch.pv_vcpu.trap_bounce;
-do_guest_trap(TRAP_nmi, guest_cpu_user_regs(), 0);
+do_guest_trap(TRAP_nmi, guest_cpu_user_regs());
 tb->flags &= ~TBF_EXCEPTION; /* not needed for NMI delivery path */
 return !null_trap_bounce(v, tb);
 }
@@ -743,7 +748,7 @@ void do_reserved_trap(struct cpu_user_regs *regs)
 panic("FATAL RESERVED TRAP %#x: %s", trapnr, trapstr(trapnr));
 }
 
-static void do_trap(struct cpu_user_regs *regs, int use_error_code)
+void do_trap(struct cpu_user_regs *regs)
 {
 struct vcpu *curr = current;
 unsigned int trapnr = regs->entry_vector;
@@ -757,7 +762,7 @@ static void do_trap(struct cpu_user_regs *regs, int 
use_error_code)
 
 if ( guest_mode(regs) )
 {
-do_guest_trap(trapnr, regs, use_error_code);
+do_guest_trap(trapnr, regs);
 return;
 }
 
@@ -789,28 +794,6 @@ static void do_trap(struct cpu_user_regs *regs, int 
use_error_code)
   trapnr, trapstr(trapnr), regs->error_code);
 }
 
-#define DO_ERROR_NOCODE(name)   \
-void do_##name(struct cpu_user_regs *regs)  \
-{   \
-do_trap(regs, 0);   \
-}
-
-#define DO_ERROR(name)  \
-void do_##name(struct cpu_user_regs *regs)  \
-{   \
-do_trap(regs, 1);   \
-}
-
-DO_ERROR_NOCODE(divide_error)
-DO_ERROR_NOCODE(overflow)
-DO_ERROR_NOCODE(bounds)
-DO_ERROR(   invalid_TSS)
-DO_ERROR(   segment_not_present)
-DO_ERROR(   stack_segment)
-DO_ERROR_NOCODE(coprocessor_error)
-DO_ERROR(   alignment_check)
-DO_ERROR_NOCODE(simd_coprocessor_error)
-
 /* Returns 0 if not handled, and non-0 for success. */
 int rdmsr_hypervisor_regs(uint32_t idx, uint64_t *val)
 {
@@ -1318,7 +1301,7 @@ void do_invalid_op(struct cpu_user_regs *regs)
 {
 if ( !emulate_invalid_rdtscp(regs) &&
  !emulate_forced_invalid_op(regs) )
-do_guest_trap(TRAP_invalid_op, regs, 0);
+do_guest_trap(TRAP_invalid_op, regs);
 return;
 }
 
@@ -1432,7 +1415,7 @@ void do_int3(struct cpu_user_regs *regs)
 return;
 } 
 
-do_guest_trap(TRAP_int3, regs, 0);
+do_guest_trap(TRAP_int3, regs);
 }
 
 static void reserved_bit_page_fault(
@@ -2604,7 +2587,7 @@ static int emulate_privileged_op(struct cpu_user_regs 
*regs)
 if ( lock || rep_prefix || opsize_prefix
  || !(v->arch.pv_vcpu.ctrlreg[4] & X86_CR4_O

[Xen-devel] [PATCH 1/4] tools/xenalyze: Remove bogus library dependencies

[George Dunlap]

xenalyze was inheriting LDLIBS of xentrace; but it doesn't need them.

Remove this dependency, which allows xenalyze to be built without the libraries
having been built, and run without the libraries being installed.

Signed-off-by: George Dunlap 
---
CC: Ian Jackson 
CC: Wei Liu 
CC: Dario Faggioli 
CC: Anshul Makkar 
---
 tools/xentrace/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/xentrace/Makefile b/tools/xentrace/Makefile
index 0157be2..a85ed33 100644
--- a/tools/xentrace/Makefile
+++ b/tools/xentrace/Makefile
@@ -50,7 +50,7 @@ xentrace_setsize: setsize.o
$(CC) $(LDFLAGS) -o $@ $< $(LDLIBS) $(APPEND_LDFLAGS)
 
 xenalyze: xenalyze.o mread.o
-   $(CC) $(LDFLAGS) -o $@ $^ $(LDLIBS) $(APPEND_LDFLAGS)
+   $(CC) $(LDFLAGS) -o $@ $^ $(APPEND_LDFLAGS)
 
 -include $(DEPS)
 
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 2/4] tools/xenalyze: Remove weighted cpi summaries

[George Dunlap]

At the moment these structures are not used, and half of the code for
collecting it is commented out.  To be used they require further
support for collecting hardware instruction counter data inside of
Xen.

Remove the code entirely; when they're wanted again they will be here
in the git log.

Signed-off-by: George Dunlap 
---
CC: Ian Jackson 
CC: Wei Liu 
CC: Dario Faggioli 
CC: Anshul Makkar 
---
 tools/xentrace/xenalyze.c | 121 --
 1 file changed, 121 deletions(-)

diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c
index d223de6..d455a52 100644
--- a/tools/xentrace/xenalyze.c
+++ b/tools/xentrace/xenalyze.c
@@ -147,7 +147,6 @@ int verbosity = 5;
 struct {
 unsigned
 scatterplot_interrupt_eip:1,
-scatterplot_cpi:1,
 scatterplot_unpin_promote:1,
 scatterplot_cr3_switch:1,
 scatterplot_wake_to_halt:1,
@@ -223,7 +222,6 @@ struct {
 } interval;
 } opt = {
 .scatterplot_interrupt_eip=0,
-.scatterplot_cpi=0,
 .scatterplot_unpin_promote=0,
 .scatterplot_cr3_switch=0,
 .scatterplot_wake_to_halt=0,
@@ -290,15 +288,6 @@ struct cycle_summary {
 struct interval_element interval;
 };
 
-struct weighted_cpi_summary {
-int count;
-unsigned long long instructions;
-unsigned long long cycles;
-float *cpi;
-unsigned long long *cpi_weight;
-struct interval_element interval;
-};
-
 /* -- Symbol list information -- */
 #define SYMBOL_ENTRIES_PER_STRUCT 1023
 #define SYMBOL_NAME_SIZE 124
@@ -1672,7 +1661,6 @@ struct vcpu_data {
 struct cycle_framework f;
 struct cycle_summary runstates[RUNSTATE_MAX];
 struct cycle_summary runnable_states[RUNNABLE_STATE_MAX];
-struct weighted_cpi_summary cpi;
 struct cycle_summary cpu_affinity_all,
 cpu_affinity_pcpu[MAX_CPUS];
 enum {
@@ -2297,54 +2285,6 @@ static inline void clear_interval_cycles(struct 
interval_element *e) {
 e->instructions = 0;
 }
 
-#if 0
-static inline void update_cpi(struct weighted_cpi_summary *s,
-  unsigned long long i,
-  unsigned long long c) {
-/* We don't know ahead of time how many samples there are, and working
- * with dynamic stuff is a pain, and unnecessary.  This algorithm will
- * generate a sample set that approximates an even sample.  We can
- * then take the percentiles on this, and get an approximate value. */
-int lap, index;
-
-if ( opt.sample_size ) {
-lap = (s->count/opt.sample_size)+1;
-index =s->count % opt.sample_size;
-
-if((index - (lap/3))%lap == 0) {
-if(!s->cpi) {
-assert(!s->cpi_weight);
-
-s->cpi = malloc(sizeof(*s->cpi) * opt.sample_size);
-s->cpi_weight = malloc(sizeof(*s->cpi_weight) * 
opt.sample_size);
-if(!s->cpi || !s->cpi_weight) {
-fprintf(stderr, "%s: malloc failed!\n", __func__);
-error(ERR_SYSTEM, NULL);
-}
-}
-assert(s->cpi_weight);
-
-s->cpi[index] = (float) c / i;
-s->cpi_weight[index]=c;
-}
-}
-
-s->instructions += i;
-s->cycles += c;
-s->count++;
-
-s->interval.instructions += i;
-s->interval.cycles += c;
-s->interval.count++;
-}
-
-static inline void clear_interval_cpi(struct weighted_cpi_summary *s) {
-s->interval.cycles = 0;
-s->interval.count = 0;
-s->interval.instructions = 0;
-}
-#endif
-
 static inline void print_cpu_affinity(struct cycle_summary *s, char *p) {
 if(s->count) {
 long long avg;
@@ -2370,31 +2310,6 @@ static inline void print_cpu_affinity(struct 
cycle_summary *s, char *p) {
 }
 }
 
-static inline void print_cpi_summary(struct weighted_cpi_summary *s) {
-if(s->count) {
-float avg;
-
-avg = (float)s->cycles / s->instructions;
-
-if ( opt.sample_size ) {
-float p5, p50, p95;
-int data_size = s->count;
-
-if(data_size > opt.sample_size)
-data_size = opt.sample_size;
-
-p50 = weighted_percentile(s->cpi, s->cpi_weight, data_size, 50);
-p5 = weighted_percentile(s->cpi, s->cpi_weight, data_size, 5);
-p95 = weighted_percentile(s->cpi, s->cpi_weight, data_size, 95);
-
-printf("  CPI summary: %2.2f {%2.2f|%2.2f|%2.2f}\n",
-   avg, p5, p50, p95);
-} else {
-printf("  CPI summary: %2.2f\n", avg);
-}
-}
-}
-
 static inline void print_cycle_percent_summary(struct cycle_summary *s,
tsc_t total, char *p) {
 if(s->count) {
@@ -7324,31 +7239,6 @@ update:
 else if ( sevt.old_runstate == RUNSTATE_RUNNING
   || v->runstate.state == RUNSTATE_RUNNING )
 {
-#if 0
-/* A lot of traces include cpi that shouldn't... */
-if(perfctrs && v->runstate.tsc) {

[Xen-devel] [PATCH 4/4] tools/xenalyze: Allow automatic resizing of sample buffers

[George Dunlap]

Rather than have large fixed-size buffers, start with smaller buffers
and allow them to grow as needed (doubling each time), with a fairly
large maximum.  Allow this maximum to be set by a command-line
parameter.

Signed-off-by: George Dunlap 
---
CC: Ian Jackson 
CC: Wei Liu 
CC: Dario Faggioli 
CC: Anshul Makkar 
---
 tools/xentrace/xenalyze.c | 95 +--
 1 file changed, 68 insertions(+), 27 deletions(-)

diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c
index 455cbdf..a4d8823 100644
--- a/tools/xentrace/xenalyze.c
+++ b/tools/xentrace/xenalyze.c
@@ -44,7 +44,8 @@ struct mread_ctrl;
 #define QHZ_FROM_HZ(_hz) (((_hz) << 10)/ 10)
 
 #define ADDR_SPACE_BITS 48
-#define DEFAULT_SAMPLE_SIZE 10240
+#define DEFAULT_SAMPLE_SIZE 1024
+#define DEFAULT_SAMPLE_MAX  1024*1024*32
 #define DEFAULT_INTERVAL_LENGTH 1000
 
 struct array_struct {
@@ -187,7 +188,7 @@ struct {
 unsigned long long histogram_interrupt_increment;
 int interrupt_eip_enumeration_vector;
 int default_guest_paging_levels;
-int sample_size;
+int sample_size, sample_max;
 enum error_level tolerance; /* Tolerate up to this level of error */
 struct {
 tsc_t cycles;
@@ -257,6 +258,7 @@ struct {
 .cpu_qhz = QHZ_FROM_HZ(DEFAULT_CPU_HZ),
 .default_guest_paging_levels = 2,
 .sample_size = DEFAULT_SAMPLE_SIZE,
+.sample_max = DEFAULT_SAMPLE_MAX,
 .tolerance = ERR_SANITY,
 .interval = { .msec = DEFAULT_INTERVAL_LENGTH },
 };
@@ -275,7 +277,7 @@ struct interval_element {
 };
 
 struct cycle_summary {
-int event_count, count;
+int event_count, count, sample_size;
 unsigned long long cycles;
 long long *sample;
 struct interval_element interval;
@@ -2203,28 +2205,51 @@ static inline double summary_percent_global(struct 
cycle_summary *s) {
 }
 
 static inline void update_cycles(struct cycle_summary *s, long long c) {
-/* We don't know ahead of time how many samples there are, and working
- * with dynamic stuff is a pain, and unnecessary.  This algorithm will
- * generate a sample set that approximates an even sample.  We can
- * then take the percentiles on this, and get an approximate value. */
 s->event_count++;
 
 if (!c)
 return;
 
 if(opt.sample_size) {
-int lap = (s->count/opt.sample_size)+1,
-index =s->count % opt.sample_size;
-if((index - (lap/3))%lap == 0) {
-if(!s->sample) {
-s->sample = malloc(sizeof(*s->sample) * opt.sample_size);
-if(!s->sample) {
-fprintf(stderr, "%s: malloc failed!\n", __func__);
-error(ERR_SYSTEM, NULL);
-}
+if (s->count >= s->sample_size
+&& (s->count == 0
+|| opt.sample_max == 0
+|| s->sample_size < opt.sample_max)) {
+int new_size;
+void * new_sample = NULL;
+
+new_size = s->sample_size << 1;
+
+if (new_size == 0)
+new_size = opt.sample_size;
+
+if (opt.sample_max != 0 && new_size > opt.sample_max)
+new_size = opt.sample_max;
+
+//printf("New size: %d\n", new_size);
+
+new_sample = realloc(s->sample, sizeof(*s->sample) * new_size);
+
+if (new_sample) {
+s->sample = new_sample;
+s->sample_size = new_size;
 }
-s->sample[index]=c;
 }
+
+if (s->count < s->sample_size) {
+s->sample[s->count]=c;
+} else {
+/* 
+ * If we run out of space for samples, start taking only a
+ * subset of samples.
+ */
+int lap, index;
+lap = (s->count/s->sample_size)+1;
+index =s->count % s->sample_size;
+if((index - (lap/3))%lap == 0) {
+s->sample[index]=c;
+ }
+ }
 }
 s->count++;
 s->cycles += c;
@@ -2248,8 +2273,8 @@ static inline void print_cpu_affinity(struct 
cycle_summary *s, char *p) {
 if ( opt.sample_size ) {
 long long  p5, p50, p95;
 int data_size = s->count;
-   if(data_size > opt.sample_size)
-data_size = opt.sample_size;
+   if(data_size > s->sample_size)
+   data_size = s->sample_size;
 
 p50 = percentile(s->sample, data_size, 50);
 p5 = percentile(s->sample, data_size, 5);
@@ -2280,8 +2305,8 @@ static inline void print_cycle_percent_summary(struct 
cycle_summary *s,
 long long p5, p50, p95;
 int data_size = s->count;
 
-if(data_size > opt.sample_size)
-data_size = opt.sample_size;
+if(data_size > s->sample_size)
+data_size = s->sample_size;
 
 p50 = self_weighted_percentile(s->sample, data_size, 50);

[Xen-devel] [PATCH 3/4] tools/xenalyze: Get rid of extraneous data structure

[George Dunlap]

The only difference between event_cycle_summary and cycle_summary was
that the former has a separate counter for "events" which had
zero-cycle events.  But a lot of the code dealing with them had to be
duplicated with slightly different fields.

Remove event_cycle_summary, add an "event_count" field to
cycle_symmary, and use cycle_summary for everything.

Signed-off-by: George Dunlap 
---
CC: Ian Jackson 
CC: Wei Liu 
CC: Dario Faggioli 
CC: Anshul Makkar 
---
 tools/xentrace/xenalyze.c | 208 ++
 1 file changed, 81 insertions(+), 127 deletions(-)

diff --git a/tools/xentrace/xenalyze.c b/tools/xentrace/xenalyze.c
index d455a52..455cbdf 100644
--- a/tools/xentrace/xenalyze.c
+++ b/tools/xentrace/xenalyze.c
@@ -274,15 +274,8 @@ struct interval_element {
 long long instructions;
 };
 
-struct event_cycle_summary {
-int count, cycles_count;
-long long cycles;
-long long *cycles_sample;
-struct interval_element interval;
-};
-
 struct cycle_summary {
-int count;
+int event_count, count;
 unsigned long long cycles;
 long long *sample;
 struct interval_element interval;
@@ -397,7 +390,7 @@ enum {
 struct eip_list_struct {
 struct eip_list_struct *next;
 unsigned long long eip;
-struct event_cycle_summary summary;
+struct cycle_summary summary;
 int type;
 void * extra;
 };
@@ -1314,21 +1307,21 @@ struct hvm_data {
 
 /* Information about particular exit reasons */
 struct {
-struct event_cycle_summary exit_reason[HVM_EXIT_REASON_MAX];
+struct cycle_summary exit_reason[HVM_EXIT_REASON_MAX];
 int extint[EXTERNAL_INTERRUPT_MAX+1];
 int *extint_histogram;
-struct event_cycle_summary trap[HVM_TRAP_MAX];
-struct event_cycle_summary pf_xen[PF_XEN_MAX];
-struct event_cycle_summary pf_xen_emul[PF_XEN_EMUL_MAX];
-struct event_cycle_summary pf_xen_emul_early_unshadow[5];
-struct event_cycle_summary pf_xen_non_emul[PF_XEN_NON_EMUL_MAX];
-struct event_cycle_summary pf_xen_fixup[PF_XEN_FIXUP_MAX];
-struct event_cycle_summary 
pf_xen_fixup_unsync_resync[PF_XEN_FIXUP_UNSYNC_RESYNC_MAX+1];
-struct event_cycle_summary cr_write[CR_MAX];
-struct event_cycle_summary cr3_write_resyncs[RESYNCS_MAX+1];
-struct event_cycle_summary vmcall[HYPERCALL_MAX+1];
-struct event_cycle_summary generic[HVM_EVENT_HANDLER_MAX];
-struct event_cycle_summary mmio[NONPF_MMIO_MAX];
+struct cycle_summary trap[HVM_TRAP_MAX];
+struct cycle_summary pf_xen[PF_XEN_MAX];
+struct cycle_summary pf_xen_emul[PF_XEN_EMUL_MAX];
+struct cycle_summary pf_xen_emul_early_unshadow[5];
+struct cycle_summary pf_xen_non_emul[PF_XEN_NON_EMUL_MAX];
+struct cycle_summary pf_xen_fixup[PF_XEN_FIXUP_MAX];
+struct cycle_summary 
pf_xen_fixup_unsync_resync[PF_XEN_FIXUP_UNSYNC_RESYNC_MAX+1];
+struct cycle_summary cr_write[CR_MAX];
+struct cycle_summary cr3_write_resyncs[RESYNCS_MAX+1];
+struct cycle_summary vmcall[HYPERCALL_MAX+1];
+struct cycle_summary generic[HVM_EVENT_HANDLER_MAX];
+struct cycle_summary mmio[NONPF_MMIO_MAX];
 struct hvm_gi_struct {
 int count;
 struct cycle_summary runtime[GUEST_INTERRUPT_CASE_MAX];
@@ -1337,7 +1330,7 @@ struct hvm_data {
 tsc_t start_tsc;
 } guest_interrupt[GUEST_INTERRUPT_MAX + 1];
 /* IPI Latency */
-struct event_cycle_summary ipi_latency;
+struct cycle_summary ipi_latency;
 int ipi_count[256];
 struct {
 struct io_address *mmio, *pio;
@@ -2200,62 +2193,28 @@ static inline double __cycles_percent(long long cycles, 
long long total) {
 return (double)(cycles*100) / total;
 }
 
-static inline double __summary_percent(struct event_cycle_summary *s,
+static inline double __summary_percent(struct cycle_summary *s,
struct cycle_framework *f) {
 return __cycles_percent(s->cycles, f->total_cycles);
 }
 
-static inline double summary_percent_global(struct event_cycle_summary *s) {
+static inline double summary_percent_global(struct cycle_summary *s) {
 return __summary_percent(s, &P.f);
 }
 
-static inline void update_summary(struct event_cycle_summary *s, long long c) {
-/* We don't know ahead of time how many samples there are, and working
- * with dynamic stuff is a pain, and unnecessary.  This algorithm will
- * generate a sample set that approximates an even sample.  We can
- * then take the percentiles on this, and get an approximate value. */
-if(c) {
-if(opt.sample_size) {
-int lap = (s->cycles_count/opt.sample_size)+1,
-index =s->cycles_count % opt.sample_size;
-if((index - (lap/3))%lap == 0) {
-if(!s->cycles_sample) {
-s->cycles_sample = malloc(sizeof(*s->cycles_sample) * 

Re: [Xen-devel] [RFC PATCH 8/8] tools/console: remove 5s bodge in console client

[Ian Jackson]

Wei Liu writes ("Re: [RFC PATCH 8/8] tools/console: remove 5s bodge in console 
client"):
> On Fri, Aug 05, 2016 at 05:36:08PM +0100, Wei Liu wrote:
> > On Fri, Aug 05, 2016 at 05:32:30PM +0100, Ian Jackson wrote:
> > > My point is that xenconsole seems to have the functionality already.
> > > You are moving it to libxl.
> > 
> > Yes, that's what I did.

... but my implicit question was: why ?

> > > The sole visible effect of this is to make xenconsole unreliable for
> > > people who invoke it directly.
> > 
> > No, it should (at least by design) still be reliable since the libxl now
> > waits before exec xenconsole.

And, when someone invokes xenconsole directly, libxl is not involved.

> Now I have decided to drop patch 4 -- the less patch the better. :-)

Good, thanks.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] CODING_STYLE: Allow single-sentence comments without full stops

[George Dunlap]

On Fri, Aug 5, 2016 at 2:56 PM, George Dunlap  wrote:
> On Thu, Aug 4, 2016 at 11:25 AM, Ian Jackson  
> wrote:
>> George Dunlap writes ("[PATCH] CODING_STYLE: Allow single-sentence comments 
>> without full stops"):
>>> One of the common ways in which contributors trip up over the
>>> CODING_STYLE guides is by not putting a full stop at the end of a
>>> comment when there is only a single sentence.  Calling these out is a
>>> waste of everybody's time: The full stop at the end of a comment with
>>> a single sentence (or a single phrase) adds absolutely nothing to the
>>> legibility of the code.
>>>
>>> Modify CODING_STYLE to allow comments with a single sentence or
>>> sentence fragment to either have a full stop or not, while making it
>>> clear that comments with multiple sentences must have a full stop at
>>> the end of each sentence.
>>>
>>> Signed-off-by: George Dunlap 
>>
>> Seriously ?
>>
>> Yes, the bike shed can be purple with occasional mauve.
>>
>> Acked-by: Ian Jackson 
>
> Thanks Ian.
>
> The other comments so far don't seem to be objections to this
> relaxation, but about whether standards should be relaxed further.  If
> I don't hear any objections by Monday I'll check this patch in and let
> others submit follow-on patches if they want.

And pushed.

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 2/6] tools/console: introduce --start-notify-fd option for console client

[Ian Jackson]

Wei Liu writes ("[PATCH v2 2/6] tools/console: introduce --start-notify-fd 
option for console client"):
> The console client will write 0x00 to that fd before entering console
> loop to indicate its readiness.

Acked-by: Ian Jackson 

> + do {
> + r = write(start_notify_fd, msg, 1);
> + } while ((r == -1 && errno == EINTR) || r == 0);

I don't think write(,,1) can return 0.  If I wrote this code I would
either call abort, or crash, in that case.  But spinning is OK too.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 3/6] libxl: factor out libxl__console_tty_path

[Ian Jackson]

Wei Liu writes ("[PATCH v2 3/6] libxl: factor out libxl__console_tty_path"):
> No other user yet.
> 
> Signed-off-by: Wei Liu 
> Acked-by: Ian Jackson 

Of course the proximate reason for this has been dropped from this
series.

However, I think this patch should be retained, because it's a step
towards implementing libxl_console_getfd.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] libxl: libxl_{primary_, }console_exec now take notify_fd argument

[Ian Jackson]

Wei Liu writes ("[PATCH v2 4/6] libxl: libxl_{primary_,}console_exec now take 
notify_fd argument"):
> The new argument will be passed down to xenconsole process, which then
> uses it to notify readiness.
...
> -execl(p, p, domid_s, "--num", cons_num_s, "--type", cons_type_s, (void 
> *)NULL);
> +if (notify_fd != -1) {
> +notify_fd_s = GCSPRINTF("%d", notify_fd);
> +execl(p, p, domid_s, "--num", cons_num_s, "--type", cons_type_s,
> +  "--start-notify-fd", notify_fd_s, (void *)NULL);
> +} else
> +execl(p, p, domid_s, "--num", cons_num_s, "--type", cons_type_s,
> +  (void *)NULL);

Technically this braces-then-non-braces conforms to the spec in
CODING_STYLE but not to the style elsewhere in libxl.  Could you put
braces around the else clause too ?

Sorry for not noticing this earlier.  We can fix this on checkin if
you like.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] libxl: libxl_{primary_, }console_exec now take notify_fd argument

[Wei Liu]

On Mon, Aug 08, 2016 at 11:15:01AM +0100, Ian Jackson wrote:
> Wei Liu writes ("[PATCH v2 4/6] libxl: libxl_{primary_,}console_exec now take 
> notify_fd argument"):
> > The new argument will be passed down to xenconsole process, which then
> > uses it to notify readiness.
> ...
> > -execl(p, p, domid_s, "--num", cons_num_s, "--type", cons_type_s, (void 
> > *)NULL);
> > +if (notify_fd != -1) {
> > +notify_fd_s = GCSPRINTF("%d", notify_fd);
> > +execl(p, p, domid_s, "--num", cons_num_s, "--type", cons_type_s,
> > +  "--start-notify-fd", notify_fd_s, (void *)NULL);
> > +} else
> > +execl(p, p, domid_s, "--num", cons_num_s, "--type", cons_type_s,
> > +  (void *)NULL);
> 
> Technically this braces-then-non-braces conforms to the spec in
> CODING_STYLE but not to the style elsewhere in libxl.  Could you put
> braces around the else clause too ?
> 
> Sorry for not noticing this earlier.  We can fix this on checkin if
> you like.
> 

Sure.

Wei.

> Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 6/6] xl: use xenconsole startup protocol

[Ian Jackson]

Wei Liu writes ("[PATCH v2 6/6] xl: use xenconsole startup protocol"):
> If user asks xl to automatically connect to console when creating a
> guest, use the new startup protocol before trying to unpause domain so
> that we don't lose any console output.
...
>  if ( dom_info->console_autoconnect ) {
> +if (libxl_pipe(ctx, notify_pipe)) {
> +fprintf(stderr,
> +"Failed to create console notification pipe, errno %d\n",
> +errno);

Is it really necessary to print to stderr, when libxl_pipe has already
logged a message ?  That seems the compelling advantage of
libxl_pipe...

> +else if (r == 1 && buf[0] != 0x00)
> +fprintf(stderr, "Got unexpected response from xenconsole: %x\n",
> +buf[0]);

I would generally prefer %#x, but up to you.

Acked-by: Ian Jackson 

Thanks,
Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH] libxl: CODING_STYLE: Forbid if (...) { stmt; } else stmt;

[Ian Jackson]

And clarify that the rule about omitting braces for single statements
is optional (it is even contradicted by the example).

Signed-off-by: Ian Jackson 
CC: Wei Liu 
---
 tools/libxl/CODING_STYLE | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/tools/libxl/CODING_STYLE b/tools/libxl/CODING_STYLE
index 01ee25b..32170ef 100644
--- a/tools/libxl/CODING_STYLE
+++ b/tools/libxl/CODING_STYLE
@@ -294,8 +294,10 @@ Libxenlight coding style is super simple.  Avoid tricky 
expressions.
 
 5. Block structure
 
-Every indented statement is braced, except blocks that contain just
-one statement.
+Every indented statement is braced, but blocks that contain just one
+statement may have the braces omitted.  To avoid confusion, either all
+the blocks in an if...else chain have braces, or none of them do.
+
 The opening brace is on the line that contains the control flow
 statement that introduces the new block; the closing brace is on the
 same line as the else keyword, or on a line by itself if there is no
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] PVH hypercall interface

[Marek Marczykowski-Górecki]

Hi,

What hypercalls are available for PVH guests? How is it different for
HVM guests? Is it documented somewhere?
For example I'd expect that do_mmu_update is available only for PV
guests, but looking at the code I can't find anything preventing other
guest types from using it (no, some obscure conditions deep in execution
path doesn't count).

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Xen 4.6.1 crash with altp2m enabledbydefault

[Jan Beulich]

>>> On 08.08.16 at 11:48,  wrote:
> vmx_vmenter_helper is not part of the call stack. The address is simply the 
> location of the ud2 to which the 
> __vmwrite(HOST_CR0, v->arch.hvm_vmx.host_cr0);
> In
> static void vmx_fpu_leave(struct vcpu *v)
> jumps.
> There are two vmwrites in vmx_vcpu_update_eptp (called by 
> altp2m_vcpu_destroy):
> __vmwrite(EPT_POINTER, ept_get_eptp(ept));
> __vmwrite(EPTP_INDEX, vcpu_altp2m(v).p2midx);
> 
> And four in vmx_vcpu_update_vmfunc_ve (also called by altp2m_vcpu_destroy)
> __vmwrite(VM_FUNCTION_CONTROL, VMX_VMFUNC_EPTP_SWITCHING);
> __vmwrite(EPTP_LIST_ADDR, virt_to_maddr(d->arch.altp2m_eptp));
> __vmwrite(VIRT_EXCEPTION_INFO, mfn_x(mfn) << PAGE_SHIFT);
> __vmwrite(SECONDARY_VM_EXEC_CONTROL,  
> v->arch.hvm_vmx.secondary_exec_control);
> 
> After the altp2m-part hvm_vcpu_destroy also calls nestedhvm_vcpu_destroy(v), 
> but this code path is executed unconditionally so I assume that the error 
> lies somewhere in the altp2m_vcpu_destroy(v).
> 
> What exactly are the vmx_vmcs_enter / exit required for? I often see the 
> vmx_vmcs_enter; __vmwrite; vmx_vmcs_exit combination. Need the __vmwrites be 
> guarded by an enter / exit ( which Is not the case in the static void 
> vmx_fpu_leave(struct vcpu *v) )?

On code paths where the correct VMCS may not be the current one
it is necessary to frame vmread / vmwrite accordingly.

> Is it possible that the 
> altp2m_vcpu_destroy->vmx_vcpu_update_eptp->vmx_vmcs_exit->vmx_clear_vmcs 
> invalidates the vmcs for the current vcpu?

I certainly can't exclude this possibility.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] VM_EVENT_FLAG_SET_REGISTERS and sync_vcpu_execstate()

[Razvan Cojocaru]

Hello,

We've been mostly setting registers by using xc_vcpu_setcontext():

https://github.com/razvan-cojocaru/libbdvmi/blob/master/src/bdvmixendriver.cpp#L504

but lately trying to push as much of that as possible to the
VM_EVENT_FLAG_SET_REGISTERS-related code (i.e. via the vm_event replies)
to save processing time.

So with the xc_vcpu_setcontext() call removed, I've found that there are
cases where vm_event_set_registers() won't work properly unless I kept
the xc_vcpu_getcontext() call. This would appear to be not because of
anything that arch_get_info_guest() does (please see the implementation
for XEN_DOMCTL_getvcpucontext), but because of the vcpu_pause() call, or
more specifically, because of its calling sync_vcpu_execstate().

So it would appear that a sync_vcpu_execstate(v) call is necessary at
the start of vm_event_set_registers() for the vcpu struct instance to be
synchronized with the current VCPU state.

Any objections to a patch with this simple modification?


Thanks,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] libxl: CODING_STYLE: Forbid if (...) { stmt; } else stmt;

[Wei Liu]

On Mon, Aug 08, 2016 at 11:21:31AM +0100, Ian Jackson wrote:
> And clarify that the rule about omitting braces for single statements
> is optional (it is even contradicted by the example).
> 
> Signed-off-by: Ian Jackson 
> CC: Wei Liu 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 6/6] xl: use xenconsole startup protocol

[Wei Liu]

On Mon, Aug 08, 2016 at 11:17:20AM +0100, Ian Jackson wrote:
> Wei Liu writes ("[PATCH v2 6/6] xl: use xenconsole startup protocol"):
> > If user asks xl to automatically connect to console when creating a
> > guest, use the new startup protocol before trying to unpause domain so
> > that we don't lose any console output.
> ...
> >  if ( dom_info->console_autoconnect ) {
> > +if (libxl_pipe(ctx, notify_pipe)) {
> > +fprintf(stderr,
> > +"Failed to create console notification pipe, errno 
> > %d\n",
> > +errno);
> 
> Is it really necessary to print to stderr, when libxl_pipe has already
> logged a message ?  That seems the compelling advantage of
> libxl_pipe...
> 

OK. I can delete this fprintf.

> > +else if (r == 1 && buf[0] != 0x00)
> > +fprintf(stderr, "Got unexpected response from xenconsole: 
> > %x\n",
> > +buf[0]);
> 
> I would generally prefer %#x, but up to you.
> 

Done.

> Acked-by: Ian Jackson 
> 

Thanks.

> Thanks,
> Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] libxl: libxl_{primary_, }console_exec now take notify_fd argument

[Wei Liu]

On Fri, Aug 05, 2016 at 06:41:32PM +0100, Wei Liu wrote:
> The new argument will be passed down to xenconsole process, which then
> uses it to notify readiness.
> 
> Signed-off-by: Wei Liu 

Forgot to carry over the ack from previous version.

  Acked-by: Ian Jackson 

I think the ack still stands because there is no change in this patch.

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCHv2 1/2] xen/prvicmd: use ENOTTY if the IOCTL is not supported

[Juergen Gross]

On 04/08/16 17:16, David Vrabel wrote:
> The standard return value for ioctl(2) where the cmd is not supported
> is ENOTTY, not EINVAL.
> 
> Signed-off-by: David Vrabel 

Reviewed-by: Juergen Gross 


Juergen

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] PVH guess crash with memory < maxmem

[Marek Marczykowski-Górecki]

Hi,

I'm having troubles with starting PVH guest when memory < maxmem. Exact
crash message:

(XEN) d2v0 EPT violation 0x182 (-w-/---) gpa 0x003e7ff000 
mfn 0x type 4
(XEN) d2v0 Walking EPT tables for GFN 3e7ff:
(XEN) d2v0  gfn exceeds max_mapped_pfn 19000
(XEN) d2v0  --- GLA 0x88003e7ff000
(XEN) domain_crash called from vmx.c:3022
(XEN) Domain 2 (vcpu#0) crashed on cpu#2:
(XEN) [ Xen-4.7.0  x86_64  debug=n  Not tainted ]
(XEN) CPU:2
(XEN) RIP:0010:[]
(XEN) RFLAGS: 00010016   CONTEXT: hvm guest (d2v0)
(XEN) rax:    rbx: 0001   rcx: 003f
(XEN) rdx: 3e7ff000   rsi: 02b96000   rdi: 88003e7ff000
(XEN) rbp: 81c03b68   rsp: 81c03b40   r8: 0200
(XEN) r9:     r10: 02b96000   r11: 
(XEN) r12: 0001   r13: 0003e7ff   r14: 8800
(XEN) r15: 880001c10828   cr0: 80050033   cr4: 0020
(XEN) cr3: 01c0e000   cr2: 
(XEN) ds:    es:    fs:    gs:    ss:    cs: 0010
(XEN) Guest stack trace from rsp=81c03b40:
(XEN)   Fault while accessing guest memory.

Guest kernel: 4.1.13, guest config attached.

Is there any way around it, or such configuration isn't supported yet?

Or maybe it is possible to increase maxmem later using some trick
(memory hotplug or something)?

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
name = "pvhtest"
uuid = "ac823572-853b-4c8c-ba77-47aa515ccd63"
maxmem = 400
memory = 1000
pvh = 1
vcpus = 4
localtime = 0
on_poweroff = "destroy"
on_reboot = "destroy"
on_crash = "destroy"
kernel = "/var/lib/qubes/vm-kernels/4.1.13-6/vmlinuz"
ramdisk = "/var/lib/qubes/vm-kernels/4.1.13-6/initramfs"
extra = "root=/dev/mapper/dmroot ro nomodeset console=hvc0 rd_NO_PLYMOUTH 3 
nopat"
disk = [ 
"/var/lib/qubes/vm-templates/fedora-23-clone/root.img,raw,xvda,r,backendtype=phy",
 "/var/lib/qubes/appvms/pvhtest/private.img,raw,xvdb,w,backendtype=phy",
"/var/lib/qubes/appvms/pvhtest/volatile.img,raw,xvdc,w,backendtype=phy",

"/var/lib/qubes/vm-kernels/4.1.13-6/modules.img,raw,xvdd,r,backendtype=phy" ]



signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] PVH hypercall interface

[Andrew Cooper]

On 08/08/16 11:21, Marek Marczykowski-Górecki wrote:
> Hi,
>
> What hypercalls are available for PVH guests?

First of all, be aware that there is currently some chaos in the
codebase which is soon to be resolved.

What you are looking for is HVMLite, which will likely be renamed to PVH
once it is complete.  Original PVH will then be filed in /dev/null,
leaving only a set of lessons in git history.

> How is it different for HVM guests?

HVMLite guests are just HVM guests without qemu.

Roger (CC'd) is leading the effort.  You select an HVMLite guest by
choosing device-model = None in the xl configuration file, rather than
setting pvh=1

> Is it documented somewhere?

There are patches on the list.

Also, docs/misc/hvmlite.markdown

> For example I'd expect that do_mmu_update is available only for PV
> guests, but looking at the code I can't find anything preventing other
> guest types from using it (no, some obscure conditions deep in execution
> path doesn't count).

HVM guests use the hvm_hypercall_table, not the hypercall tables in entry.S

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 0/6] Fix console synchronisation issues

[Wei Liu]

Adjusted some patches as required and pushed.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/4] tools/xenalyze: Remove bogus library dependencies

[Wei Liu]

On Mon, Aug 08, 2016 at 10:54:28AM +0100, George Dunlap wrote:
> xenalyze was inheriting LDLIBS of xentrace; but it doesn't need them.
> 
> Remove this dependency, which allows xenalyze to be built without the 
> libraries
> having been built, and run without the libraries being installed.
> 
> Signed-off-by: George Dunlap 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 4/4] tools/xenalyze: Allow automatic resizing of sample buffers

[Wei Liu]

On Mon, Aug 08, 2016 at 10:54:31AM +0100, George Dunlap wrote:
> +
> +new_size = s->sample_size << 1;
> +
> +if (new_size == 0)
> +new_size = opt.sample_size;
> +
> +if (opt.sample_max != 0 && new_size > opt.sample_max)
> +new_size = opt.sample_max;
> +
> +//printf("New size: %d\n", new_size);

Maybe this needs to be deleted?

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 3/4] tools/xenalyze: Get rid of extraneous data structure

[Wei Liu]

On Mon, Aug 08, 2016 at 10:54:30AM +0100, George Dunlap wrote:
> The only difference between event_cycle_summary and cycle_summary was
> that the former has a separate counter for "events" which had
> zero-cycle events.  But a lot of the code dealing with them had to be
> duplicated with slightly different fields.
> 
> Remove event_cycle_summary, add an "event_count" field to
> cycle_symmary, and use cycle_summary for everything.
> 
> Signed-off-by: George Dunlap 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/4] tools/xenalyze: Remove weighted cpi summaries

[Wei Liu]

On Mon, Aug 08, 2016 at 10:54:29AM +0100, George Dunlap wrote:
> At the moment these structures are not used, and half of the code for
> collecting it is commented out.  To be used they require further
> support for collecting hardware instruction counter data inside of
> Xen.
> 
> Remove the code entirely; when they're wanted again they will be here
> in the git log.
> 
> Signed-off-by: George Dunlap 

Acked-by: Wei Liu 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 4/6] libxl: libxl_{primary_, }console_exec now take notify_fd argument

[Ian Jackson]

Wei Liu writes ("Re: [PATCH v2 4/6] libxl: libxl_{primary_,}console_exec now 
take notify_fd argument"):
> On Fri, Aug 05, 2016 at 06:41:32PM +0100, Wei Liu wrote:
> > The new argument will be passed down to xenconsole process, which then
> > uses it to notify readiness.
> > 
> > Signed-off-by: Wei Liu 
> 
> Forgot to carry over the ack from previous version.
> 
>   Acked-by: Ian Jackson 
> 
> I think the ack still stands because there is no change in this patch.

Yes.  I think this series is done now ?  The minor comments can be
fixed during checkin.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] libxl: CODING_STYLE: Forbid if (...) { stmt; } else stmt;

[Wei Liu]

On Mon, Aug 08, 2016 at 11:31:46AM +0100, Wei Liu wrote:
> On Mon, Aug 08, 2016 at 11:21:31AM +0100, Ian Jackson wrote:
> > And clarify that the rule about omitting braces for single statements
> > is optional (it is even contradicted by the example).
> > 
> > Signed-off-by: Ian Jackson 
> > CC: Wei Liu 
> 
> Acked-by: Wei Liu 

Pushed.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 4/4] tools/xenalyze: Allow automatic resizing of sample buffers

[Wei Liu]

On Mon, Aug 08, 2016 at 12:05:58PM +0100, Wei Liu wrote:
> On Mon, Aug 08, 2016 at 10:54:31AM +0100, George Dunlap wrote:
> > +
> > +new_size = s->sample_size << 1;
> > +
> > +if (new_size == 0)
> > +new_size = opt.sample_size;
> > +
> > +if (opt.sample_max != 0 && new_size > opt.sample_max)
> > +new_size = opt.sample_max;
> > +
> > +//printf("New size: %d\n", new_size);
> 
> Maybe this needs to be deleted?
> 

FAOD I don't think it is worthy of resending just because of this
cosmetic change. Whatever adjustment can be done while committing.

Acked-by: Wei Liu 

> Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 2/2] qdisk - hw/block/xen_disk: grant copy implementation

[Roger Pau Monné]

On Tue, Aug 02, 2016 at 04:06:30PM +0200, Paulina Szubarczyk wrote:
> Copy data operated on during request from/to local buffers to/from
> the grant references.
> 
> Before grant copy operation local buffers must be allocated what is
> done by calling ioreq_init_copy_buffers. For the 'read' operation,
> first, the qemu device invokes the read operation on local buffers
> and on the completion grant copy is called and buffers are freed.
> For the 'write' operation grant copy is performed before invoking
> write by qemu device.
> 
> A new value 'feature_grant_copy' is added to recognize when the
> grant copy operation is supported by a guest.
> 
> Signed-off-by: Paulina Szubarczyk 

Thanks, this is looking good, just a couple of minor comments below.

> ---
> Changes since v3:
> - qemu_memalign/qemu_free is used instead function allocating
>   memory from xc.
> - removed the get_buffer function instead there is a direct call
>   to qemu_memalign.
> - moved ioreq_copy for write operation to ioreq_runio_qemu_aio.
> - added struct xengnttab_grant_copy_segment_t and stub in
>   xen_common.h for version of xen earlier then 480.
> - added checking for version 480 to configure. The test repeats
>   all the operation that are required for version < 480 and
>   checks if xengnttab_grant_copy() is implemented.
> 
> * I did not change the way of testing if grant_copy operation is
>   implemented. As far as I understand if the code from
>   gnttab_unimp.c is used then the gnttab device is unavailable
>   and the handler to gntdev would be invalid. But if the handler
>   is valid then the ioctl should return operation unimplemented
>   if the gntdev does not implement the operation.
> ---
>  configure   |  56 +
>  hw/block/xen_disk.c | 142 
> ++--
>  include/hw/xen/xen_common.h |  25 
>  3 files changed, 218 insertions(+), 5 deletions(-)
> 
> diff --git a/configure b/configure
> index f57fcc6..b5bf7d4 100755
> --- a/configure
> +++ b/configure
> @@ -1956,6 +1956,62 @@ EOF
>  /*
>   * If we have stable libs the we don't want the libxc compat
>   * layers, regardless of what CFLAGS we may have been given.
> + *
> + * Also, check if xengnttab_grant_copy_segment_t is defined and
> + * grant copy operation is implemented.
> + */
> +#undef XC_WANT_COMPAT_EVTCHN_API
> +#undef XC_WANT_COMPAT_GNTTAB_API
> +#undef XC_WANT_COMPAT_MAP_FOREIGN_API
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#include 
> +#if !defined(HVM_MAX_VCPUS)
> +# error HVM_MAX_VCPUS not defined
> +#endif
> +int main(void) {
> +  xc_interface *xc = NULL;
> +  xenforeignmemory_handle *xfmem;
> +  xenevtchn_handle *xe;
> +  xengnttab_handle *xg;
> +  xen_domain_handle_t handle;
> +  xengnttab_grant_copy_segment_t* seg = NULL;
> +
> +  xs_daemon_open();
> +
> +  xc = xc_interface_open(0, 0, 0);
> +  xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
> +  xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
> +  xc_hvm_inject_msi(xc, 0, 0xf000, 0x);
> +  xc_hvm_create_ioreq_server(xc, 0, HVM_IOREQSRV_BUFIOREQ_ATOMIC, NULL);
> +  xc_domain_create(xc, 0, handle, 0, NULL, NULL);
> +
> +  xfmem = xenforeignmemory_open(0, 0);
> +  xenforeignmemory_map(xfmem, 0, 0, 0, 0, 0);
> +
> +  xe = xenevtchn_open(0, 0);
> +  xenevtchn_fd(xe);
> +
> +  xg = xengnttab_open(0, 0);
> +  xengnttab_map_grant_ref(xg, 0, 0, 0);

I don't think you need to check xengnttab_map_grant_ref, just 
xengnttab_grant_copy should be enough? Since xengnttab_grant_copy was added 
later you can assume xengnttab_map_grant_ref will be there.

> +  xengnttab_grant_copy(xg, 0, seg);
> +
> +  return 0;
> +}
> +EOF
> +  compile_prog "" "$xen_libs $xen_stable_libs"
> +then
> +xen_ctrl_version=480
> +xen=yes
> +  elif
> +  cat > $TMPC < +/*
> + * If we have stable libs the we don't want the libxc compat
> + * layers, regardless of what CFLAGS we may have been given.
>   */
>  #undef XC_WANT_COMPAT_EVTCHN_API
>  #undef XC_WANT_COMPAT_GNTTAB_API
> diff --git a/hw/block/xen_disk.c b/hw/block/xen_disk.c
> index 3b8ad33..2dd1464 100644
> --- a/hw/block/xen_disk.c
> +++ b/hw/block/xen_disk.c
> @@ -119,6 +119,9 @@ struct XenBlkDev {
>  unsigned intpersistent_gnt_count;
>  unsigned intmax_grants;
>  
> +/* Grant copy */
> +gbooleanfeature_grant_copy;
> +
>  /* qemu block driver */
>  DriveInfo   *dinfo;
>  BlockBackend*blk;
> @@ -489,6 +492,95 @@ static int ioreq_map(struct ioreq *ioreq)
>  return 0;
>  }
>  
> +static void free_buffers(struct ioreq *ioreq)
> +{
> +int i;
> +
> +for (i = 0; i < ioreq->v.niov; i++) {
> +ioreq->page[i] = NULL;
> +}
> +
> +qemu_vfree(ioreq->pages);
> +}
> +
> +static int ioreq_init_copy_buffers(struct ioreq *ioreq)
> +{
> +int i;
> +
> +if (ioreq->v.niov == 0) {
> +return 0;
> +}
> +
> +ioreq->pages = qemu_memalig

Re: [Xen-devel] Device model operation hypercall (DMOP, re qemu depriv)

[Jan Beulich]

>>> On 05.08.16 at 18:28,  wrote:
> Jan Beulich writes ("Re: Device model operation hypercall (DMOP, re qemu 
> depriv)"):
>> On 04.08.16 at 13:21,  wrote:
>> > What we cannot do is audit every HVMCTL, fix the class 2 problems, and
>> > then declare HVMCTL to have the relevant security property, and
>> > implement corresponding code in dom0's privcmd drivers which relies on
>> > the security property.  This is because the dom0 privcmd driver
>> > doesn't know whether the HVMCTLs it is allowing not-fully-trusted
>> > userspace to make are actually trustworthy (with the specific
>> > hypervisor version in question.)
>> 
>> I continue to not really understand this argumentation: Dom0's
>> privcmd driver doesn't really matter here. If there's a bug in
>> something qemu uses, this is a problem no matter whether that
>> operation gets called though the to-be-added privcmd logic, or
>> straight from a stubdom qemu. Both are less than fully privileged.
>> What do I continue to be missing?
> 
> Let me try again.  Earlier I wrote:
> 
>   AFAICT there are two kinds of possible bug:
> 
>   1. An HVMCTL (or hvmop) that can have an adverse affect on the whole
>   system.  Such bugs would already be security bugs, covered by our
>   security support policy.  Such a bug would be a security bug whether
>   the operation is an hvmop or an HVMCTL or a DMOP.
> 
>   2. An HVMCTL (or hvmop) that can have an adverse effect on the calling
>   domain.  Such bugs are not currently security bugs.  But the of qemu
>   depriv project requires them to be removed.  Such such a bug is a
>   security bug if it is a DMOP[1] but not otherwise.
> 
> Bugs of class 1 are already security bugs.  They can already be
> exploited by stub device models.
> 
> Bugs of class 2 are only security bugs if we allow unprivileged
> callers within a privileged domain to use the corresponding hypercall.
> 
> That is, a bug of class 2 would allow the unprivileged qemu process in
> dom0 to cause damage to other parts of dom0.
> 
> Bugs of class 2 are of no interest to an attacker who has control of a
> stub device model, because all it allows them to do is damage the
> device domain domain (which they already control).

Ah, okay, I think I finally understand. I'm sorry for being dense.

I'm having, however, a hard time imagining a class 2 bug for any
of the hvmop-s that are being converted by the hvmctl series:
These act on the target domain, so would not touch the calling
ones state other than for copying argument structures to/from
guest memory (and I don't view mixing up domain pointers as
a likely source of problems). Any other problem they might
reasonably have would then affect the system as a whole (class
1) or just the target domain (non-security bug).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] PVH hypercall interface

[Roger Pau Monne]

On Mon, Aug 08, 2016 at 11:42:03AM +0100, Andrew Cooper wrote:
> On 08/08/16 11:21, Marek Marczykowski-Górecki wrote:
> > Hi,
> >
> > What hypercalls are available for PVH guests?
> 
> First of all, be aware that there is currently some chaos in the
> codebase which is soon to be resolved.
> 
> What you are looking for is HVMLite, which will likely be renamed to PVH
> once it is complete.  Original PVH will then be filed in /dev/null,
> leaving only a set of lessons in git history.
> 
> > How is it different for HVM guests?
> 
> HVMLite guests are just HVM guests without qemu.
> 
> Roger (CC'd) is leading the effort.  You select an HVMLite guest by
> choosing device-model = None in the xl configuration file, rather than
> setting pvh=1
> 
> > Is it documented somewhere?
> 
> There are patches on the list.
> 
> Also, docs/misc/hvmlite.markdown
> 
> > For example I'd expect that do_mmu_update is available only for PV
> > guests, but looking at the code I can't find anything preventing other
> > guest types from using it (no, some obscure conditions deep in execution
> > path doesn't count).
> 
> HVM guests use the hvm_hypercall_table, not the hypercall tables in entry.S

They have access to the same set of hypercalls as HVM guests, in fact the 
hypercall table is shared between HVM and HVMlite guests (to become PVH, 
just using HVMlite here to avoid confusion). Classic PVH guests had their 
own hypercall table, but as Andrew said you should not look at those.

Roger.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [XTF PATCH v2] xtf-runner: use xl create -Fc directly

[Wei Liu]

Now that xl create -c is fixed in xen-unstable, there is no need to keep
the hack to get guest console output anymore.

Use xl create -Fc directly, then wait for the xl process to exit.  Print
any error as it occurs.

Signed-off-by: Wei Liu 
---
Rebased on top of master
---
 xtf-runner | 34 +++---
 1 file changed, 15 insertions(+), 19 deletions(-)

diff --git a/xtf-runner b/xtf-runner
index c063699..a87bde6 100755
--- a/xtf-runner
+++ b/xtf-runner
@@ -425,30 +425,22 @@ def list_tests(opts):
 for sel in opts.selection:
 print sel
 
+create_list = []
 
 def run_test(test):
 """ Run a specific test """
 
-cmd = ['xl', 'create', '-p', test.cfg_path()]
+cmd = ['xl', 'create', '-Fc', test.cfg_path()]
 print "Executing '%s'" % (" ".join(cmd), )
-rc = subproc_call(cmd)
-if rc:
-raise RunnerError("Failed to create VM")
+guest = Popen(cmd, stdout = PIPE, stderr = PIPE)
+create_list.append(guest)
 
-cmd = ['xl', 'console', test.vm_name()]
-print "Executing '%s'" % (" ".join(cmd), )
-console = Popen(cmd, stdout = PIPE)
-
-cmd = ['xl', 'unpause', test.vm_name()]
-print "Executing '%s'" % (" ".join(cmd), )
-rc = subproc_call(cmd)
-if rc:
-raise RunnerError("Failed to unpause VM")
+# stdout is console output, stderr is xl output
+stdout, stderr = guest.communicate()
 
-stdout, _ = console.communicate()
-
-if console.returncode:
-raise RunnerError("Failed to obtain VM console")
+if guest.returncode:
+print stderr
+raise RunnerError("Failed to communicate with guest")
 
 lines = stdout.splitlines()
 
@@ -603,10 +595,14 @@ def main():
 opts.selection = interpret_selection(opts)
 
 if opts.list_tests:
-return list_tests(opts)
+ret = list_tests(opts)
 else:
-return run_tests(opts)
+ret = run_tests(opts)
+
+for child in create_list:
+child.wait()
 
+return ret
 
 if __name__ == "__main__":
 try:
-- 
2.1.4


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] x86/traps: Drop use_error_code parameter from do_{, guest_}trap()

[Jan Beulich]

>>> On 08.08.16 at 11:51,  wrote:
> Whether or not an error code is needed can be determinted entirely from the
> trapnr paramter, as error codes are architecturally specified.
> 
> Introduce TRAP_HAVE_EC as a bitmap of reserved vectors which have error codes,
> and drop the use_error_code from all callsites.
> 
> As a result, the DO_ERROR{,_NOCODE}() macros become entirely superflouous and
> can be dropped.  Update the exception_table to point straight at do_trap().
> 
> Signed-off-by: Andrew Cooper 

Reviewed-by: Jan Beulich 


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH V2] x86/hvm: Allow guest_request vm_events coming from userspace

[Jan Beulich]

>>> On 08.08.16 at 10:06,  wrote:
> Allow guest userspace code to request that a vm_event be sent out
> via VMCALL. This functionality seems to be handy for a number of
> Xen developers, as stated on the mailing list (thread "[Xen-devel]
> HVMOP_guest_request_vm_event only works from guest in ring0").
> 
> Signed-off-by: Razvan Cojocaru 
> 
> ---
> Changes since V1:
>  - No longer repeating the check when mode == 8.

Technically this looks correct to me now. Albeit I'm still not really
convinced we actually want to start making exceptions here.

> --- a/xen/arch/x86/hvm/hvm.c
> +++ b/xen/arch/x86/hvm/hvm.c
> @@ -4146,15 +4146,25 @@ int hvm_do_hypercall(struct cpu_user_regs *regs)
>  switch ( mode )
>  {
>  case 8:
> +if ( eax == __HYPERVISOR_hvm_op &&
> + regs->rdi == HVMOP_guest_request_vm_event )
> +break;
> +/* Fallthrough */
>  case 4:
> +/* Fallthrough */
>  case 2:

At least this one annotation is pointless, but if we decide to commit
the change this can of course be taken care of while committing.

> +if ( mode != 8 && eax == __HYPERVISOR_hvm_op &&
> + regs->_ebx == HVMOP_guest_request_vm_event )
> +break;
>  hvm_get_segment_register(curr, x86_seg_ss, &sreg);
>  if ( unlikely(sreg.attr.fields.dpl) )
>  {
> +/* Fallthrough */
>  default:

I would hope this annotation to be pointless too, but that would
need to be clarified by someone more familiar with Coverity.

>  regs->eax = -EPERM;
>  return HVM_HCALL_completed;
>  }
> +/* Fallthrough */
>  case 0:

This one, otoh, looks like it was indeed missing (and Coverity
should have complained).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 2/2] qdisk - hw/block/xen_disk: grant copy implementation

[Paulina Szubarczyk]

On 08/08/2016 01:11 PM, Roger Pau Monné wrote:

On Tue, Aug 02, 2016 at 04:06:30PM +0200, Paulina Szubarczyk wrote:

Copy data operated on during request from/to local buffers to/from
the grant references.

Before grant copy operation local buffers must be allocated what is
done by calling ioreq_init_copy_buffers. For the 'read' operation,
first, the qemu device invokes the read operation on local buffers
and on the completion grant copy is called and buffers are freed.
For the 'write' operation grant copy is performed before invoking
write by qemu device.

A new value 'feature_grant_copy' is added to recognize when the
grant copy operation is supported by a guest.

Signed-off-by: Paulina Szubarczyk 


Thanks, this is looking good, just a couple of minor comments below.



Thank you for the review, I will apply the comments.


---
Changes since v3:
- qemu_memalign/qemu_free is used instead function allocating
   memory from xc.
- removed the get_buffer function instead there is a direct call
   to qemu_memalign.
- moved ioreq_copy for write operation to ioreq_runio_qemu_aio.
- added struct xengnttab_grant_copy_segment_t and stub in
   xen_common.h for version of xen earlier then 480.
- added checking for version 480 to configure. The test repeats
   all the operation that are required for version < 480 and
   checks if xengnttab_grant_copy() is implemented.

* I did not change the way of testing if grant_copy operation is
   implemented. As far as I understand if the code from
   gnttab_unimp.c is used then the gnttab device is unavailable
   and the handler to gntdev would be invalid. But if the handler
   is valid then the ioctl should return operation unimplemented
   if the gntdev does not implement the operation.
---
  configure   |  56 +
  hw/block/xen_disk.c | 142 ++--
  include/hw/xen/xen_common.h |  25 
  3 files changed, 218 insertions(+), 5 deletions(-)

diff --git a/configure b/configure
index f57fcc6..b5bf7d4 100755
--- a/configure
+++ b/configure
@@ -1956,6 +1956,62 @@ EOF
  /*
   * If we have stable libs the we don't want the libxc compat
   * layers, regardless of what CFLAGS we may have been given.
+ *
+ * Also, check if xengnttab_grant_copy_segment_t is defined and
+ * grant copy operation is implemented.
+ */
+#undef XC_WANT_COMPAT_EVTCHN_API
+#undef XC_WANT_COMPAT_GNTTAB_API
+#undef XC_WANT_COMPAT_MAP_FOREIGN_API
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#if !defined(HVM_MAX_VCPUS)
+# error HVM_MAX_VCPUS not defined
+#endif
+int main(void) {
+  xc_interface *xc = NULL;
+  xenforeignmemory_handle *xfmem;
+  xenevtchn_handle *xe;
+  xengnttab_handle *xg;
+  xen_domain_handle_t handle;
+  xengnttab_grant_copy_segment_t* seg = NULL;
+
+  xs_daemon_open();
+
+  xc = xc_interface_open(0, 0, 0);
+  xc_hvm_set_mem_type(0, 0, HVMMEM_ram_ro, 0, 0);
+  xc_domain_add_to_physmap(0, 0, XENMAPSPACE_gmfn, 0, 0);
+  xc_hvm_inject_msi(xc, 0, 0xf000, 0x);
+  xc_hvm_create_ioreq_server(xc, 0, HVM_IOREQSRV_BUFIOREQ_ATOMIC, NULL);
+  xc_domain_create(xc, 0, handle, 0, NULL, NULL);
+
+  xfmem = xenforeignmemory_open(0, 0);
+  xenforeignmemory_map(xfmem, 0, 0, 0, 0, 0);
+
+  xe = xenevtchn_open(0, 0);
+  xenevtchn_fd(xe);
+
+  xg = xengnttab_open(0, 0);
+  xengnttab_map_grant_ref(xg, 0, 0, 0);


I don't think you need to check xengnttab_map_grant_ref, just
xengnttab_grant_copy should be enough? Since xengnttab_grant_copy was added
later you can assume xengnttab_map_grant_ref will be there.


+  xengnttab_grant_copy(xg, 0, seg);
+
+  return 0;
+}
+EOF
+  compile_prog "" "$xen_libs $xen_stable_libs"
+then
+xen_ctrl_version=480
+xen=yes
+  elif
+  cat > $TMPC page[i] = NULL;
+}
+
+qemu_vfree(ioreq->pages);
+}
+
+static int ioreq_init_copy_buffers(struct ioreq *ioreq)
+{
+int i;
+
+if (ioreq->v.niov == 0) {
+return 0;
+}
+
+ioreq->pages = qemu_memalign(XC_PAGE_SIZE, ioreq->v.niov * XC_PAGE_SIZE);
+if (!ioreq->pages) {
+return -1;
+}
+
+for (i = 0; i < ioreq->v.niov; i++) {
+ioreq->page[i] = ioreq->pages + i * XC_PAGE_SIZE;
+ioreq->v.iov[i].iov_base = ioreq->page[i];
+}
+
+return 0;
+}
+
+static int ioreq_copy(struct ioreq *ioreq)
+{
+xengnttab_handle *gnt = ioreq->blkdev->xendev.gnttabdev;
+xengnttab_grant_copy_segment_t segs[BLKIF_MAX_SEGMENTS_PER_REQUEST];
+int i, count = 0, r, rc;


Why do you need to initialize count to 0 here? A couple of lines below you
inconditionally set it to ioreq->v.niov.


+int64_t file_blk = ioreq->blkdev->file_blk;
+
+if (ioreq->v.niov == 0) {
+return 0;
+}
+
+count = ioreq->v.niov;
+
+for (i = 0; i < count; i++) {
+
+if (ioreq->req.operation == BLKIF_OP_READ) {
+segs[i].flags = GNTCOPY_dest_gref;
+segs[i].dest.foreign.ref = ioreq->refs[i];
+   

Re: [Xen-devel] PVH guess crash with memory < maxmem

[Jan Beulich]

>>> On 08.08.16 at 12:40,  wrote:
> I'm having troubles with starting PVH guest when memory < maxmem. Exact
> crash message:
> 
> (XEN) d2v0 EPT violation 0x182 (-w-/---) gpa 0x003e7ff000 mfn 
> 0x type 4
> (XEN) d2v0 Walking EPT tables for GFN 3e7ff:
> (XEN) d2v0  gfn exceeds max_mapped_pfn 19000
> (XEN) d2v0  --- GLA 0x88003e7ff000
> (XEN) domain_crash called from vmx.c:3022
> (XEN) Domain 2 (vcpu#0) crashed on cpu#2:
> (XEN) [ Xen-4.7.0  x86_64  debug=n  Not tainted ]
> (XEN) CPU:2
> (XEN) RIP:0010:[]
> (XEN) RFLAGS: 00010016   CONTEXT: hvm guest (d2v0)
> (XEN) rax:    rbx: 0001   rcx: 
> 003f
> (XEN) rdx: 3e7ff000   rsi: 02b96000   rdi: 
> 88003e7ff000
> (XEN) rbp: 81c03b68   rsp: 81c03b40   r8: 0200
> (XEN) r9:     r10: 02b96000   r11: 
> 
> (XEN) r12: 0001   r13: 0003e7ff   r14: 
> 8800
> (XEN) r15: 880001c10828   cr0: 80050033   cr4: 
> 0020
> (XEN) cr3: 01c0e000   cr2: 
> (XEN) ds:    es:    fs:    gs:    ss:    cs: 0010
> (XEN) Guest stack trace from rsp=81c03b40:
> (XEN)   Fault while accessing guest memory.
> 
> Guest kernel: 4.1.13, guest config attached.
> 
> Is there any way around it, or such configuration isn't supported yet?

Indeed iirc PVHv1 and PoD are incompatible with one another,
and I don't know any possible workaround. And then I'm not
sure it's worth your time playing with PVHv1 in the first place.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Lars Kurth]

Hi,

as part of a number of tasks to move Xen Project websites to https, we 
investigated whether we can move our tarballs to a new Xen Project owned domain 
to download tarballs. Currently tarballs are stored on 
http://bits.xensource.com, which is a http site only. We do not have sufficient 
control of bits.xensource.com (which is an Akamai site) to convert the site to 
https, and are thus potentially exposed to MiM attacks. 

To fix this, the current plan of record is to
- Copy existing tarballs to an existing or new VM
- To expose that VM via the new public URL ftp.xenproject.org (this is 
non-browsable, thus ftp - we also already have 
https://downloads.xenproject.org/ to host legacy content)
- To only publish new tarballs on https://ftp.xenproject.org
- To update http://xenproject.org/downloads/xen-archives.html to use the new VM

In most cases, the ftp.xenproject.org site would *not* be exposed directly to 
users, but via the download manager on xenproject.org. The exception are blog 
posts and xen-devel@/etc. mails such as 
https://blog.xenproject.org/2016/05/11/announcing-xen-project-4-7-rc-and-test-day-schedule/

We would either keep existing tarballs on bits.xensource.com OR - if we have 
sufficient control - implement a 301 redirect to the new site. This would 
ensure that 3rd party links to tarballs are not broken. 

Does anyone have any objection regarding the name of the site and/or proposal. 
I am assuming this is non-controversial: if I don't get any objections by end 
of day Friday 12th, Aug assume we can go ahead with the change.

Best Regards
Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v4 2/2] qdisk - hw/block/xen_disk: grant copy implementation

[Paulina Szubarczyk]

On 08/02/2016 04:06 PM, Paulina Szubarczyk wrote:

Copy data operated on during request from/to local buffers to/from
the grant references.

Before grant copy operation local buffers must be allocated what is
done by calling ioreq_init_copy_buffers. For the 'read' operation,
first, the qemu device invokes the read operation on local buffers
and on the completion grant copy is called and buffers are freed.
For the 'write' operation grant copy is performed before invoking
write by qemu device.

A new value 'feature_grant_copy' is added to recognize when the
grant copy operation is supported by a guest.

Signed-off-by: Paulina Szubarczyk 
---
Changes since v3:
- qemu_memalign/qemu_free is used instead function allocating
   memory from xc.
- removed the get_buffer function instead there is a direct call
   to qemu_memalign.
- moved ioreq_copy for write operation to ioreq_runio_qemu_aio.
- added struct xengnttab_grant_copy_segment_t and stub in
   xen_common.h for version of xen earlier then 480.
- added checking for version 480 to configure. The test repeats
   all the operation that are required for version < 480 and
   checks if xengnttab_grant_copy() is implemented.

* I did not change the way of testing if grant_copy operation is
   implemented. As far as I understand if the code from
   gnttab_unimp.c is used then the gnttab device is unavailable
   and the handler to gntdev would be invalid. But if the handler
   is valid then the ioctl should return operation unimplemented
   if the gntdev does not implement the operation.
---
  configure   |  56 +
  hw/block/xen_disk.c | 142 ++--
  include/hw/xen/xen_common.h |  25 
  3 files changed, 218 insertions(+), 5 deletions(-)



  /* qemu block driver */
  DriveInfo   *dinfo;
  BlockBackend*blk;
@@ -489,6 +492,95 @@ static int ioreq_map(struct ioreq *ioreq)
  return 0;
  }



  static void qemu_aio_complete(void *opaque, int ret)
@@ -511,8 +603,29 @@ static void qemu_aio_complete(void *opaque, int ret)
  return;
  }

+if (ioreq->blkdev->feature_grant_copy) {
+switch (ioreq->req.operation) {
+case BLKIF_OP_READ:
+/* in case of failure ioreq->aio_errors is increased */
+ioreq_copy(ioreq);


I would add a condition to invoke the grant copy only if the ret 
argument with which the callback from BlockBackend 
'qemu_aio_complete(void *opaque, int ret)' is called is equal to 0

to not unnecessary copy invalid data.


+free_buffers(ioreq);
+break;
+case BLKIF_OP_WRITE:
+case BLKIF_OP_FLUSH_DISKCACHE:
+if (!ioreq->req.nr_segments) {
+break;
+}
+free_buffers(ioreq);
+break;
+default:
+break;
+}
+}
+
  ioreq->status = ioreq->aio_errors ? BLKIF_RSP_ERROR : BLKIF_RSP_OKAY;
-ioreq_unmap(ioreq);
+if (!ioreq->blkdev->feature_grant_copy) {
+ioreq_unmap(ioreq);
+}
  ioreq_finish(ioreq);
  switch (ioreq->req.operation) {
  case BLKIF_OP_WRITE:


Paulina

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] VM_EVENT_FLAG_SET_REGISTERS and sync_vcpu_execstate()

[Andrew Cooper]

On 08/08/16 11:31, Razvan Cojocaru wrote:
> Hello,
>
> We've been mostly setting registers by using xc_vcpu_setcontext():
>
> https://github.com/razvan-cojocaru/libbdvmi/blob/master/src/bdvmixendriver.cpp#L504
>
> but lately trying to push as much of that as possible to the
> VM_EVENT_FLAG_SET_REGISTERS-related code (i.e. via the vm_event replies)
> to save processing time.
>
> So with the xc_vcpu_setcontext() call removed, I've found that there are
> cases where vm_event_set_registers() won't work properly unless I kept
> the xc_vcpu_getcontext() call. This would appear to be not because of
> anything that arch_get_info_guest() does (please see the implementation
> for XEN_DOMCTL_getvcpucontext), but because of the vcpu_pause() call, or
> more specifically, because of its calling sync_vcpu_execstate().
>
> So it would appear that a sync_vcpu_execstate(v) call is necessary at
> the start of vm_event_set_registers() for the vcpu struct instance to be
> synchronized with the current VCPU state.
>
> Any objections to a patch with this simple modification?

It would be helpful to identify exactly what is currently going wrong,
and why sync_vcpu_execstate(v) helps.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] PVH hypercall interface

[Marek Marczykowski-Górecki]

On Mon, Aug 08, 2016 at 01:19:43PM +0200, Roger Pau Monne wrote:
> On Mon, Aug 08, 2016 at 11:42:03AM +0100, Andrew Cooper wrote:
> > On 08/08/16 11:21, Marek Marczykowski-Górecki wrote:
> > > Hi,
> > >
> > > What hypercalls are available for PVH guests?
> > 
> > First of all, be aware that there is currently some chaos in the
> > codebase which is soon to be resolved.
> > 
> > What you are looking for is HVMLite, which will likely be renamed to PVH
> > once it is complete.  Original PVH will then be filed in /dev/null,
> > leaving only a set of lessons in git history.
> > 
> > > How is it different for HVM guests?
> > 
> > HVMLite guests are just HVM guests without qemu.
> > 
> > Roger (CC'd) is leading the effort.  You select an HVMLite guest by
> > choosing device-model = None in the xl configuration file, rather than
> > setting pvh=1
> > 
> > > Is it documented somewhere?
> > 
> > There are patches on the list.
> > 
> > Also, docs/misc/hvmlite.markdown
> > 
> > > For example I'd expect that do_mmu_update is available only for PV
> > > guests, but looking at the code I can't find anything preventing other
> > > guest types from using it (no, some obscure conditions deep in execution
> > > path doesn't count).
> > 
> > HVM guests use the hvm_hypercall_table, not the hypercall tables in entry.S
> 
> They have access to the same set of hypercalls as HVM guests, in fact the 
> hypercall table is shared between HVM and HVMlite guests (to become PVH, 
> just using HVMlite here to avoid confusion). Classic PVH guests had their 
> own hypercall table, but as Andrew said you should not look at those.

Thanks for the explanation. What is the current state of HVMlite? I see
"none" is valid value for "device_model_version" already, but if I try
to start such guest, it fails at x86_compat call (tries to boot it as PV
guest). I guess it's because of missing kernel support. Is "[PATCH v2
00/11] HVMlite domU support" sent in Feb the latest version available?
When it is planned to have it in vanilla kernel?


-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] PVH guess crash with memory < maxmem

[Marek Marczykowski-Górecki]

On Mon, Aug 08, 2016 at 05:38:56AM -0600, Jan Beulich wrote:
> Indeed iirc PVHv1 and PoD are incompatible with one another,
> and I don't know any possible workaround. And then I'm not
> sure it's worth your time playing with PVHv1 in the first place.

Thanks.

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?


signature.asc
Description: PGP signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable-smoke test] 100341: tolerable all pass - PUSHED

[osstest service owner]

flight 100341 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100341/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  72798298c751a3df21473a0d86d05ec7bb46e258
baseline version:
 xen  7fb0a87d97201f9c3639f85615eacd93110dc1c5

Last test of basis99970  2016-08-05 17:03:22 Z2 days
Testing same since   100341  2016-08-08 10:02:49 Z0 days1 attempts


People who touched revisions under test:
  Dario Faggioli 
  George Dunlap 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

+ branch=xen-unstable-smoke
+ revision=72798298c751a3df21473a0d86d05ec7bb46e258
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x '!=' x/home/osstest/repos/lock ']'
++ OSSTEST_REPOS_LOCK_LOCKED=/home/osstest/repos/lock
++ exec with-lock-ex -w /home/osstest/repos/lock ./ap-push xen-unstable-smoke 
72798298c751a3df21473a0d86d05ec7bb46e258
+ branch=xen-unstable-smoke
+ revision=72798298c751a3df21473a0d86d05ec7bb46e258
+ . ./cri-lock-repos
++ . ./cri-common
+++ . ./cri-getconfig
+++ umask 002
+++ getrepos
 getconfig Repos
 perl -e '
use Osstest;
readglobalconfig();
print $c{"Repos"} or die $!;
'
+++ local repos=/home/osstest/repos
+++ '[' -z /home/osstest/repos ']'
+++ '[' '!' -d /home/osstest/repos ']'
+++ echo /home/osstest/repos
++ repos=/home/osstest/repos
++ repos_lock=/home/osstest/repos/lock
++ '[' x/home/osstest/repos/lock '!=' x/home/osstest/repos/lock ']'
+ . ./cri-common
++ . ./cri-getconfig
++ umask 002
+ select_xenbranch
+ case "$branch" in
+ tree=xen
+ xenbranch=xen-unstable-smoke
+ qemuubranch=qemu-upstream-unstable
+ '[' xxen = xlinux ']'
+ linuxbranch=
+ '[' xqemu-upstream-unstable = x ']'
+ select_prevxenbranch
++ ./cri-getprevxenbranch xen-unstable-smoke
+ prevxenbranch=xen-4.7-testing
+ '[' x72798298c751a3df21473a0d86d05ec7bb46e258 = x ']'
+ : tested/2.6.39.x
+ . ./ap-common
++ : osst...@xenbits.xen.org
+++ getconfig OsstestUpstream
+++ perl -e '
use Osstest;
readglobalconfig();
print $c{"OsstestUpstream"} or die $!;
'
++ :
++ : git://xenbits.xen.org/xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/xen.git
++ : git://xenbits.xen.org/qemu-xen-traditional.git
++ : git://git.kernel.org
++ : git://git.kernel.org/pub/scm/linux/kernel/git
++ : git
++ : git://xenbits.xen.org/libvirt.git
++ : osst...@xenbits.xen.org:/home/xen/git/libvirt.git
++ : git://xenbits.xen.org/libvirt.git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : git
++ : git://xenbits.xen.org/rumpuser-xen.git
++ : osst...@xenbits.xen.org:/home/xen/git/rumpuser-xen.git
+++ besteffort_repo https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ cached_repo https://github.com/rumpkernel/rumpkernel-netbsd-src 
'[fetch=try]'
+++ local repo=https://github.com/rumpkernel/rumpkernel-netbsd-src
+++ local 'options=[fetch=try]'
 getconfig GitCacheProxy
 perl -e '
use Osstest;
readglobalconfig();
print $c{"GitCacheProxy"} or die $!;
'
+++ local cache=git://cache:9419/
+++ '[' xgit://cache:9419/ '!=' x ']'
+++ echo 
'git://cach

[Xen-devel] [ovmf baseline-only test] 66938: all pass

[Platform Team regression test user]

This run is configured for baseline tests only.

flight 66938 ovmf real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/66938/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 74bbe31b8d485da26ec7ffad5e78b8384a9eb9a5
baseline version:
 ovmf 7e9cf612056ed2667eaad7f0e901f0a37f6ddd5b

Last test of basis66934  2016-08-07 18:19:47 Z0 days
Testing same since66938  2016-08-08 09:48:59 Z0 days1 attempts


People who touched revisions under test:
  Dandan Bi 
  Hao Wu 
  Yonghong Zhu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.

(No revision log; it would be 363 lines long.)

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/4] tools/xenalyze: Remove bogus library dependencies

[Dario Faggioli]

On Mon, 2016-08-08 at 10:54 +0100, George Dunlap wrote:
> xenalyze was inheriting LDLIBS of xentrace; but it doesn't need them.
> 
> Remove this dependency, which allows xenalyze to be built without the
> libraries
> having been built, and run without the libraries being installed.
> 
Which is great... I did put exactly something like this in my TODO list
as well! :-)

> Signed-off-by: George Dunlap 
> ---
> CC: Ian Jackson 
> CC: Wei Liu 
> CC: Dario Faggioli 
> CC: Anshul Makkar 
>
FWI (makefiles are not my filed, but as said, I've been digging a bit
myself this one already:

Reviewed-by: Dario Faggioli 

Thanks and Regards,
Dario
-- 
<> (Raistlin Majere)
-
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



signature.asc
Description: This is a digitally signed message part
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] VM_EVENT_FLAG_SET_REGISTERS and sync_vcpu_execstate()

[Razvan Cojocaru]

On 08/08/2016 03:01 PM, Andrew Cooper wrote:
> On 08/08/16 11:31, Razvan Cojocaru wrote:
>> Hello,
>>
>> We've been mostly setting registers by using xc_vcpu_setcontext():
>>
>> https://github.com/razvan-cojocaru/libbdvmi/blob/master/src/bdvmixendriver.cpp#L504
>>
>> but lately trying to push as much of that as possible to the
>> VM_EVENT_FLAG_SET_REGISTERS-related code (i.e. via the vm_event replies)
>> to save processing time.
>>
>> So with the xc_vcpu_setcontext() call removed, I've found that there are
>> cases where vm_event_set_registers() won't work properly unless I kept
>> the xc_vcpu_getcontext() call. This would appear to be not because of
>> anything that arch_get_info_guest() does (please see the implementation
>> for XEN_DOMCTL_getvcpucontext), but because of the vcpu_pause() call, or
>> more specifically, because of its calling sync_vcpu_execstate().
>>
>> So it would appear that a sync_vcpu_execstate(v) call is necessary at
>> the start of vm_event_set_registers() for the vcpu struct instance to be
>> synchronized with the current VCPU state.
>>
>> Any objections to a patch with this simple modification?
> 
> It would be helpful to identify exactly what is currently going wrong,
> and why sync_vcpu_execstate(v) helps.

I've placed a

printk("EIP: 0x%016lx, 0x%016lx\n", v->arch.user_regs.eip,
rsp->data.regs.x86.rip);

at the top of vm_event_set_registers(), so I could see what the old
value is (v->arch.user_regs.eip) vs. the new value (rsp->data.regs.x86.rip).

I'm also logging these in my application, the difference there being
that the old value is the value that came with the vm_event request,
which has been obtained with guest_cpu_user_regs()->eip (where in
vm_event_set_registers() we set v->arch.user_regs.eip, since v != current).

Here's a short test run, with xl dmesg:

(XEN) [  395.739543] EIP: 0xf80001be5957, 0xf80001be595b
(XEN) [  409.795311] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  416.675023] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  421.475567] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  428.275125] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  435.507009] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  441.318224] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  445.514807] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  452.539190] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  454.762810] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  459.538651] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  462.027222] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  481.770470] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  483.298493] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  486.522344] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  491.042325] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  494.874468] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  497.450765] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  500.562738] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  509.179754] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  510.826236] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  512.106206] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  518.658092] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  520.450156] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  521.882088] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  523.250092] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  524.577987] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  525.962042] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  527.353942] EIP: 0xf80001be4812, 0xf80001be4812
(XEN) [  528.714089] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  530.065994] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  531.357762] EIP: 0xf80001be4812, 0xf80001be4812
(XEN) [  532.594016] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  533.849886] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  535.145879] EIP: 0xf80001be4812, 0xf80001be4812
(XEN) [  536.546846] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  537.905756] EIP: 0xf80001be480f, 0xf80001be4812
(XEN) [  539.209454] EIP: 0xf80001be4812, 0xf80001be4812

and the corresponding part in the userspace application's log:

GET EIP: 0xf80001be5957 SET EIP: 0xf80001be595b
GET EIP: 0xf80001be480f SET EIP: 0xf80001be4812
GET EIP: 0xf80001be480f SET EIP: 0xf80001be4812
GET EIP: 0xf80001be480f SET EIP: 0xf80001be4812
GET EIP: 0xf80001be480f SET EIP: 0xf80001be4812
GET EIP: 0xf80001be480f SET EIP: 0xf80001be4812
GET EIP: 0xf80001be480f SET EIP: 0xf80001be4812
GET EIP: 0xf80001be480f SET EIP: 0xf80001be4812
GET EIP: 0xf80001be480f SET EIP: 0xf80001be4812
GET EIP: 0xf80001be480f SET EIP: 0xf80001be4812
GET EIP: 0xf80001be480f SET EIP: 0xf80001be4812
GET EIP: 0xf80001be48

[Xen-devel] [distros-debian-sid test] 66937: tolerable FAIL

[Platform Team regression test user]

flight 66937 distros-debian-sid real [real]
http://osstest.xs.citrite.net/~osstest/testlogs/logs/66937/

Failures :-/ but no regressions.

Regressions which are regarded as allowable (not blocking):
 test-amd64-i386-i386-sid-netboot-pvgrub 9 debian-di-install fail blocked in 
66876
 test-amd64-i386-amd64-sid-netboot-pygrub 9 debian-di-install fail blocked in 
66876
 test-amd64-amd64-i386-sid-netboot-pygrub 9 debian-di-install fail blocked in 
66876
 test-armhf-armhf-armhf-sid-netboot-pygrub 10 guest-start   fail like 66876
 test-amd64-amd64-amd64-sid-netboot-pvgrub  9 debian-di-install fail like 66876

baseline version:
 flight   66876

jobs:
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-amd64-sid-netboot-pvgrubfail
 test-amd64-i386-i386-sid-netboot-pvgrub  fail
 test-amd64-i386-amd64-sid-netboot-pygrub fail
 test-armhf-armhf-armhf-sid-netboot-pygrubfail
 test-amd64-amd64-i386-sid-netboot-pygrub fail



sg-report-flight on osstest.xs.citrite.net
logs: /home/osstest/logs
images: /home/osstest/images

Logs, config files, etc. are available at
http://osstest.xs.citrite.net/~osstest/testlogs/logs

Test harness code can be found at
http://xenbits.xensource.com/gitweb?p=osstest.git;a=summary


Push not applicable.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH v2] xtf-runner: use xl create -Fc directly

[Andrew Cooper]

On 08/08/16 12:24, Wei Liu wrote:
> Now that xl create -c is fixed in xen-unstable, there is no need to keep
> the hack to get guest console output anymore.
>
> Use xl create -Fc directly, then wait for the xl process to exit.  Print
> any error as it occurs.
>
> Signed-off-by: Wei Liu 

Sadly, now I think about this further, it does re-introduce the
serialisation problem I was trying specifically trying to avoid.

You need to run `xl create -F` so you can sensibly wait on the create
list to avoid tripping up the leak detection.

However, the guest.communicate() call will wait for the guest process to
terminate, which includes all output.

Therefore, I think we still need the `xl create -Fp`, `xl console`, `xl
unpause` dance, where the create process gets put on the create_list,
and it is the console process which gets communicated with.

This also has the advantage that it doesn't cause ./xtf-runner to break
against all non-staging trees.

> ---
> Rebased on top of master
> ---
>  xtf-runner | 34 +++---
>  1 file changed, 15 insertions(+), 19 deletions(-)
>
> diff --git a/xtf-runner b/xtf-runner
> index c063699..a87bde6 100755
> --- a/xtf-runner
> +++ b/xtf-runner
> @@ -425,30 +425,22 @@ def list_tests(opts):
>  for sel in opts.selection:
>  print sel
>  
> +create_list = []

I would call this the xl_create_list to give its name a bit more
context, and I would also suggest a comment along the lines of:

# List of `xl create` processes which may still be cleaning up after
dead domains

~Andrew

>  
>  def run_test(test):
>  """ Run a specific test """
>  
> -cmd = ['xl', 'create', '-p', test.cfg_path()]
> +cmd = ['xl', 'create', '-Fc', test.cfg_path()]
>  print "Executing '%s'" % (" ".join(cmd), )
> -rc = subproc_call(cmd)
> -if rc:
> -raise RunnerError("Failed to create VM")
> +guest = Popen(cmd, stdout = PIPE, stderr = PIPE)
> +create_list.append(guest)
>  
> -cmd = ['xl', 'console', test.vm_name()]
> -print "Executing '%s'" % (" ".join(cmd), )
> -console = Popen(cmd, stdout = PIPE)
> -
> -cmd = ['xl', 'unpause', test.vm_name()]
> -print "Executing '%s'" % (" ".join(cmd), )
> -rc = subproc_call(cmd)
> -if rc:
> -raise RunnerError("Failed to unpause VM")
> +# stdout is console output, stderr is xl output
> +stdout, stderr = guest.communicate()
>  
> -stdout, _ = console.communicate()
> -
> -if console.returncode:
> -raise RunnerError("Failed to obtain VM console")
> +if guest.returncode:
> +print stderr
> +raise RunnerError("Failed to communicate with guest")
>  
>  lines = stdout.splitlines()
>  
> @@ -603,10 +595,14 @@ def main():
>  opts.selection = interpret_selection(opts)
>  
>  if opts.list_tests:
> -return list_tests(opts)
> +ret = list_tests(opts)
>  else:
> -return run_tests(opts)
> +ret = run_tests(opts)
> +
> +for child in create_list:
> +child.wait()
>  
> +return ret
>  
>  if __name__ == "__main__":
>  try:


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[George Dunlap]

On 08/08/16 12:43, Lars Kurth wrote:
> Hi,
> 
> as part of a number of tasks to move Xen Project websites to https, we 
> investigated whether we can move our tarballs to a new Xen Project owned 
> domain to download tarballs. Currently tarballs are stored on 
> http://bits.xensource.com, which is a http site only. We do not have 
> sufficient control of bits.xensource.com (which is an Akamai site) to convert 
> the site to https, and are thus potentially exposed to MiM attacks. 
> 
> To fix this, the current plan of record is to
> - Copy existing tarballs to an existing or new VM
> - To expose that VM via the new public URL ftp.xenproject.org (this is 
> non-browsable, thus ftp - we also already have 
> https://downloads.xenproject.org/ to host legacy content)
> - To only publish new tarballs on https://ftp.xenproject.org

The pedant in me thinks it's strange to have a server titled "ftp" that
is speaking https rather than ftp.  But we're apparently already using
"downloads.xenproject.org" for something else, and I don't have a better
name, so I suppose it will have to do. :-)

 -George


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [XTF PATCH v2] xtf-runner: use xl create -Fc directly

[Wei Liu]

On Mon, Aug 08, 2016 at 02:06:37PM +0100, Andrew Cooper wrote:
> On 08/08/16 12:24, Wei Liu wrote:
> > Now that xl create -c is fixed in xen-unstable, there is no need to keep
> > the hack to get guest console output anymore.
> >
> > Use xl create -Fc directly, then wait for the xl process to exit.  Print
> > any error as it occurs.
> >
> > Signed-off-by: Wei Liu 
> 
> Sadly, now I think about this further, it does re-introduce the
> serialisation problem I was trying specifically trying to avoid.
> 

Can you give an example of the race you wanted to avoid?

I thought with the xenconsole work in place I had solved all races I was
aware of, but maybe I missed something obvious.

> You need to run `xl create -F` so you can sensibly wait on the create
> list to avoid tripping up the leak detection.
> 
> However, the guest.communicate() call will wait for the guest process to
> terminate, which includes all output.
> 

Is there a problem with that?

> Therefore, I think we still need the `xl create -Fp`, `xl console`, `xl
> unpause` dance, where the create process gets put on the create_list,
> and it is the console process which gets communicated with.
> 
> This also has the advantage that it doesn't cause ./xtf-runner to break
> against all non-staging trees.
> 

I thought we decided to grep log file for that?

Wei.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 2/2] AMD/VPMU: Keep reserved MSR bits untouched but allow the rest to be written

[Boris Ostrovsky]

While AMD APM suggests that reserved MSR bits are not supposed to be
touched, it is not clear how (or whether) HW enforces this for PMU
registers. At least on some family 10h processors writes of these bits
are apparently ignored: guests (such as Linux) assume that the bits
are zero and write the MSRs with that assumption in mind even though
the bits are set by the time OS/hypervisor starts runnning.

With that in mind, let's stop reporting an error to the guest if it
tries to modify those bits and instead adjust the value to be written
safely.

Signed-off-by: Boris Ostrovsky 
---
 xen/arch/x86/cpu/vpmu_amd.c | 16 
 1 file changed, 4 insertions(+), 12 deletions(-)

diff --git a/xen/arch/x86/cpu/vpmu_amd.c b/xen/arch/x86/cpu/vpmu_amd.c
index 55d03b3..6147ec4 100644
--- a/xen/arch/x86/cpu/vpmu_amd.c
+++ b/xen/arch/x86/cpu/vpmu_amd.c
@@ -247,15 +247,7 @@ static int amd_vpmu_load(struct vcpu *v, bool_t from_guest)
 
 for ( i = 0; i < num_counters; i++ )
 {
-if ( (ctrl_regs[i] & CTRL_RSVD_MASK) != ctrl_rsvd[i] )
-{
-/*
- * Not necessary to re-init context since we should never load
- * it until guest provides valid values. But just to be safe.
- */
-amd_vpmu_init_regs(ctxt);
-return -EINVAL;
-}
+ctrl_regs[i] = (ctrl_regs[i] & ~CTRL_RSVD_MASK) | ctrl_rsvd[i];
 
 if ( is_pmu_enabled(ctrl_regs[i]) )
 is_running = 1;
@@ -363,9 +355,9 @@ static int amd_vpmu_do_wrmsr(unsigned int msr, uint64_t 
msr_content,
 
 ASSERT(!supported);
 
-if ( (type == MSR_TYPE_CTRL ) &&
- ((msr_content & CTRL_RSVD_MASK) != ctrl_rsvd[idx]) )
-return -EINVAL;
+/* Make sure reserved bits are not touched */
+if ( type == MSR_TYPE_CTRL )
+msr_content = (msr_content & ~CTRL_RSVD_MASK) | ctrl_rsvd[idx];
 
 /* For all counters, enable guest only mode for HVM guest */
 if ( has_hvm_container_vcpu(v) && (type == MSR_TYPE_CTRL) &&
-- 
1.8.3.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 0/2] AMD VPMU fixes

[Boris Ostrovsky]

A couple of AMD VPMU fixes
* Handle original (pre-family 15h) MSR range reserved for PMU use
* Stop reporting error back to the guest when reserved PMU MSR bits are
  modified since apparently guests (Linux at least) may assume those bits
  to be zero. Just make sure those bits are set/cleared prior to being
  written according to values discovered during initialization.

Boris Ostrovsky (2):
  AMD/VPMU: 0xc001 - 0xc001007 MSRs are in PMU range
  AMD/VPMU: Keep reserved MSR bits untouched but allow the rest to be
written


 xen/arch/x86/cpu/vpmu_amd.c | 16 
 xen/arch/x86/traps.c|  2 ++
 2 files changed, 6 insertions(+), 12 deletions(-)

-- 
1.8.3.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH 1/2] AMD/VPMU: 0xc0010000 - 0xc001007 MSRs are in PMU range

[Boris Ostrovsky]

We need to check for older PMU MSR range when emulating MSR
accesses for PV guests.

Signed-off-by: Boris Ostrovsky 
---
 xen/arch/x86/traps.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/xen/arch/x86/traps.c b/xen/arch/x86/traps.c
index 767d0b0..79a3516 100644
--- a/xen/arch/x86/traps.c
+++ b/xen/arch/x86/traps.c
@@ -2903,6 +2903,7 @@ static int emulate_privileged_op(struct cpu_user_regs 
*regs)
 {
 vpmu_msr = 1;
 case MSR_AMD_FAM15H_EVNTSEL0...MSR_AMD_FAM15H_PERFCTR5:
+case MSR_K7_EVNTSEL0...MSR_K7_PERFCTR3:
 if ( vpmu_msr || (boot_cpu_data.x86_vendor == X86_VENDOR_AMD) )
 {
 if ( (vpmu_mode & XENPMU_MODE_ALL) &&
@@ -3030,6 +3031,7 @@ static int emulate_privileged_op(struct cpu_user_regs 
*regs)
 {
 vpmu_msr = 1;
 case MSR_AMD_FAM15H_EVNTSEL0...MSR_AMD_FAM15H_PERFCTR5:
+case MSR_K7_EVNTSEL0...MSR_K7_PERFCTR3:
 if ( vpmu_msr || (boot_cpu_data.x86_vendor == X86_VENDOR_AMD) )
 {
 
-- 
1.8.3.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 1/3] livepach: Add .livepatch.hooks functions and test-case

[George Dunlap]

On 05/08/16 16:35, Jan Beulich wrote:
 On 04.08.16 at 17:49,  wrote:
>> In general, the hooks provide flexibility when having to deal with
>> unforeseen cases, but their application should be rarely required (<
>> 10%)."
> 
> But the greater flexibility of course comes with increased chances
> of screwing things up. I'm therefore still not entirely convinced that
> such XSAs wouldn't then better not be live patched.

But this functionality is optional, right?  So although I tend to agree
with you, we can let the person / organization preparing the patch take
that risk if they want to?

 -George


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] PVH hypercall interface

[Boris Ostrovsky]

On 08/08/2016 08:13 AM, Marek Marczykowski-Górecki wrote:
> Thanks for the explanation. What is the current state of HVMlite? I see
> "none" is valid value for "device_model_version" already, but if I try
> to start such guest, it fails at x86_compat call (tries to boot it as PV
> guest). I guess it's because of missing kernel support. Is "[PATCH v2
> 00/11] HVMlite domU support" sent in Feb the latest version available?
> When it is planned to have it in vanilla kernel?

This has been delayed by ACPI rework in Xen. I will post Linux domU
patches as soon as that is done.

-boris




signature.asc
Description: OpenPGP digital signature
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [libvirt test] 100331: regressions - trouble: blocked/broken/fail/pass

[osstest service owner]

flight 100331 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100331/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-i386-pvops  3 host-install(3) broken REGR. vs. 99955
 test-armhf-armhf-libvirt-raw  6 xen-boot  fail REGR. vs. 99955

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-armhf-armhf-libvirt 14 guest-saverestorefail   never pass
 test-armhf-armhf-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 12 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 11 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-qcow2 13 guest-saverestorefail never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 10 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 11 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 12 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-xsm 14 guest-saverestorefail   never pass

version targeted for testing:
 libvirt  492a383e583da63faf2693860bf4ff925a9da35f
baseline version:
 libvirt  d5813d72ad2bd8f9735b96bafe2ba350f74b3994

Last test of basis99955  2016-08-05 04:21:10 Z3 days
Failing since 99982  2016-08-06 04:23:44 Z2 days2 attempts
Testing same since   100331  2016-08-08 04:21:57 Z0 days1 attempts


People who touched revisions under test:
  Eric Blake 
  Erik Skultety 
  Jiri Denemark 
  John Ferlan 
  Ján Tomko 
  Michal Privoznik 
  Nikolay Shirokovskiy 
  Peter Krempa 

jobs:
 build-amd64-xsm  pass
 build-armhf-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-armhf  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-armhf-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-armhf-pvopspass
 build-i386-pvops broken  
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm   pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsmblocked 
 test-amd64-amd64-libvirt-xsm pass
 test-armhf-armhf-libvirt-xsm fail
 test-amd64-i386-libvirt-xsm  blocked 
 test-amd64-amd64-libvirt pass
 test-armhf-armhf-libvirt fail
 test-amd64-i386-libvirt  blocked 
 test-amd64-amd64-libvirt-pairpass
 test-amd64-i386-libvirt-pair blocked 
 test-armhf-armhf-libvirt-qcow2   fail
 test-armhf-armhf-libvirt-raw fail
 test-amd64-amd64-libvirt-vhd pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary

broken-step build-i386-pvops host-install(3)

Not pushing.


commit 492a383e583da63faf2693860bf4ff925a9da35f
Author: Nikolay Shirokovskiy 
Date:   Mon Jun 20 13:08:15 2016 +0300

vz: add vzDomainGetJobStats

Signed-off-by: Nikolay Shirokovskiy 

commit dd7b0a388f529259ff130c988d38d6cec34cb862
Author: Nikolay Shirokovskiy 
Date:   Mon Jun 20 13:08:14 2016 +0300

vz: add getting job info for migration

Unf

Re: [Xen-devel] Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Ian Jackson]

Lars Kurth writes ("Proposed plan and URL name for new VM to download xen 
tarballs (ftp.xenproject.org)"):
> To fix this, the current plan of record is to
> - Copy existing tarballs to an existing or new VM
> - To expose that VM via the new public URL ftp.xenproject.org (this is 
> non-browsable, thus ftp - we also already have 
> https://downloads.xenproject.org/ to host legacy content)
> - To only publish new tarballs on https://ftp.xenproject.org
> - To update http://xenproject.org/downloads/xen-archives.html to use the new 
> VM

We should consider whether the release tarballs (and ancient Xen 3.x
docs archive) could be put on https://downloads.xenproject.org/.

Looking at the webserver directory listing there suggests that we
could fit the `new' content alongside the old.

> We would either keep existing tarballs on bits.xensource.com OR - if we have 
> sufficient control - implement a 301 redirect to the new site. This would 
> ensure that 3rd party links to tarballs are not broken. 

We might be able to manage that.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] AMD/VPMU: 0xc0010000 - 0xc001007 MSRs are in PMU range

[Jan Beulich]

>>> On 08.08.16 at 15:41,  wrote:
> --- a/xen/arch/x86/traps.c
> +++ b/xen/arch/x86/traps.c
> @@ -2903,6 +2903,7 @@ static int emulate_privileged_op(struct cpu_user_regs 
> *regs)
>  {
>  vpmu_msr = 1;
>  case MSR_AMD_FAM15H_EVNTSEL0...MSR_AMD_FAM15H_PERFCTR5:
> +case MSR_K7_EVNTSEL0...MSR_K7_PERFCTR3:
>  if ( vpmu_msr || (boot_cpu_data.x86_vendor == 
> X86_VENDOR_AMD) )
>  {
>  if ( (vpmu_mode & XENPMU_MODE_ALL) &&
> @@ -3030,6 +3031,7 @@ static int emulate_privileged_op(struct cpu_user_regs 
> *regs)
>  {
>  vpmu_msr = 1;
>  case MSR_AMD_FAM15H_EVNTSEL0...MSR_AMD_FAM15H_PERFCTR5:
> +case MSR_K7_EVNTSEL0...MSR_K7_PERFCTR3:
>  if ( vpmu_msr || (boot_cpu_data.x86_vendor == 
> X86_VENDOR_AMD) )
>  {
>  

And all the logic in vpmu_amd.c is already in suitable shape? Namely
I'm wondering whether CTRL_RSVD_MASK indeed applies uniformly
to both the pre-Fam15 and Fam15+ MSRs.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Device model operation hypercall (DMOP, re qemu depriv)

[Ian Jackson]

Jan Beulich writes ("Re: Device model operation hypercall (DMOP, re qemu 
depriv)"):
> On 05.08.16 at 18:28,  wrote:
> > That is, a bug of class 2 would allow the unprivileged qemu process in
> > dom0 to cause damage to other parts of dom0.
...
> Ah, okay, I think I finally understand. [...]
> 
> I'm having, however, a hard time imagining a class 2 bug for any
> of the hvmop-s that are being converted by the hvmctl series:
> These act on the target domain, so would not touch the calling
> ones state other than for copying argument structures to/from
> guest memory (and I don't view mixing up domain pointers as
> a likely source of problems).

Right.  AIUI all the hypercall arguments are passed using
calling-guest-virtual addresses, which the dom0 privcmd arrangements
are capable of ensuring are sane.

> Any other problem they might
> reasonably have would then affect the system as a whole (class
> 1) or just the target domain (non-security bug).

So would it therefore be OK to introduce the enhanced security promise
- the lack of `class 2' bugs - for HVMCTL from the beginning ?

This would involve a small amount of extra thought for each invididual
hypercall, just to check that the assumptions we are relying on (as
you put them above) are not violated.

If so then good, but I think we still need to have a proper
conversation about how we are going to provide ABI stability to qemu
in practice.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/2] AMD/VPMU: Keep reserved MSR bits untouched but allow the rest to be written

[Jan Beulich]

>>> On 08.08.16 at 15:41,  wrote:
> While AMD APM suggests that reserved MSR bits are not supposed to be
> touched, it is not clear how (or whether) HW enforces this for PMU
> registers. At least on some family 10h processors writes of these bits
> are apparently ignored: guests (such as Linux) assume that the bits
> are zero and write the MSRs with that assumption in mind even though
> the bits are set by the time OS/hypervisor starts runnning.

So how did these bits become non-zero then?

Independent of that I think the relaxation would better only be done
for those older CPUs.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 1/3] livepach: Add .livepatch.hooks functions and test-case

[Jan Beulich]

>>> On 08.08.16 at 15:42,  wrote:
> On 05/08/16 16:35, Jan Beulich wrote:
> On 04.08.16 at 17:49,  wrote:
>>> In general, the hooks provide flexibility when having to deal with
>>> unforeseen cases, but their application should be rarely required (<
>>> 10%)."
>> 
>> But the greater flexibility of course comes with increased chances
>> of screwing things up. I'm therefore still not entirely convinced that
>> such XSAs wouldn't then better not be live patched.
> 
> But this functionality is optional, right?  So although I tend to agree
> with you, we can let the person / organization preparing the patch take
> that risk if they want to?

That's a valid point, albeit I think patch producer and patch consumer
might have different views, and the patch consumer may not know
(or even know to care) whether (s)he's got handed a patch with or
without use of hooks.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Lars Kurth]

> On 8 Aug 2016, at 14:51, Ian Jackson  wrote:
> 
> Lars Kurth writes ("Proposed plan and URL name for new VM to download xen 
> tarballs (ftp.xenproject.org)"):
>> To fix this, the current plan of record is to
>> - Copy existing tarballs to an existing or new VM
>> - To expose that VM via the new public URL ftp.xenproject.org (this is 
>> non-browsable, thus ftp - we also already have 
>> https://downloads.xenproject.org/ to host legacy content)
>> - To only publish new tarballs on https://ftp.xenproject.org
>> - To update http://xenproject.org/downloads/xen-archives.html to use the new 
>> VM
> 
> We should consider whether the release tarballs (and ancient Xen 3.x
> docs archive) could be put on https://downloads.xenproject.org/.
> 
> Looking at the webserver directory listing there suggests that we
> could fit the `new' content alongside the old.

I don't mind. I can ask Credativ whether that is doable from a load 
perspective. But the answer is probably yes.

I am assuming this would not create any extra complexities for the workflow of 
creating tarballs? Please confirm.

>> We would either keep existing tarballs on bits.xensource.com OR - if we have 
>> sufficient control - implement a 301 redirect to the new site. This would 
>> ensure that 3rd party links to tarballs are not broken. 
> 
> We might be able to manage that.

What I thought: a simple change to .htaccess should be all that is needed

Lars
___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Proposed plan and URL name for new VM to download xen tarballs (ftp.xenproject.org)

[Ian Jackson]

Lars Kurth writes ("Re: Proposed plan and URL name for new VM to download xen 
tarballs (ftp.xenproject.org)"):
> I don't mind. I can ask Credativ whether that is doable from a load 
> perspective. But the answer is probably yes.

If it isn't doable then moving the release tarballs load elsewhere is
no help, since it's all coming out of the same rackspace account.

Currently downloads.xenproject.org is hosted on the mail-and-dns VM.

> I am assuming this would not create any extra complexities for the workflow 
> of creating tarballs? Please confirm.

No.

> > We might be able to manage that.
> 
> What I thought: a simple change to .htaccess should be all that is needed

I doubt that.  Akamai are not using Apache.  But we can investigate.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] AMD/VPMU: 0xc0010000 - 0xc001007 MSRs are in PMU range

[Boris Ostrovsky]

On 08/08/2016 09:53 AM, Jan Beulich wrote:
 On 08.08.16 at 15:41,  wrote:
>> --- a/xen/arch/x86/traps.c
>> +++ b/xen/arch/x86/traps.c
>> @@ -2903,6 +2903,7 @@ static int emulate_privileged_op(struct cpu_user_regs 
>> *regs)
>>  {
>>  vpmu_msr = 1;
>>  case MSR_AMD_FAM15H_EVNTSEL0...MSR_AMD_FAM15H_PERFCTR5:
>> +case MSR_K7_EVNTSEL0...MSR_K7_PERFCTR3:
>>  if ( vpmu_msr || (boot_cpu_data.x86_vendor == 
>> X86_VENDOR_AMD) )
>>  {
>>  if ( (vpmu_mode & XENPMU_MODE_ALL) &&
>> @@ -3030,6 +3031,7 @@ static int emulate_privileged_op(struct cpu_user_regs 
>> *regs)
>>  {
>>  vpmu_msr = 1;
>>  case MSR_AMD_FAM15H_EVNTSEL0...MSR_AMD_FAM15H_PERFCTR5:
>> +case MSR_K7_EVNTSEL0...MSR_K7_PERFCTR3:
>>  if ( vpmu_msr || (boot_cpu_data.x86_vendor == 
>> X86_VENDOR_AMD) )
>>  {
>>  
> And all the logic in vpmu_amd.c is already in suitable shape? Namely
> I'm wondering whether CTRL_RSVD_MASK indeed applies uniformly
> to both the pre-Fam15 and Fam15+ MSRs.

The reserved bits look the same on all supported families --- bits
63:42, 39:36, 21 and 19. Except apparently on family 12h bit 19 is MBZ.

-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] Device model operation hypercall (DMOP, re qemu depriv)

[Jan Beulich]

>>> On 08.08.16 at 15:46,  wrote:
> Jan Beulich writes ("Re: Device model operation hypercall (DMOP, re qemu 
> depriv)"):
>> I'm having, however, a hard time imagining a class 2 bug for any
>> of the hvmop-s that are being converted by the hvmctl series:
>> These act on the target domain, so would not touch the calling
>> ones state other than for copying argument structures to/from
>> guest memory (and I don't view mixing up domain pointers as
>> a likely source of problems).
> 
> Right.  AIUI all the hypercall arguments are passed using
> calling-guest-virtual addresses, which the dom0 privcmd arrangements
> are capable of ensuring are sane.
> 
>> Any other problem they might
>> reasonably have would then affect the system as a whole (class
>> 1) or just the target domain (non-security bug).
> 
> So would it therefore be OK to introduce the enhanced security promise
> - the lack of `class 2' bugs - for HVMCTL from the beginning ?

I think so, since ...

> This would involve a small amount of extra thought for each invididual
> hypercall, just to check that the assumptions we are relying on (as
> you put them above) are not violated.

... this looks to be a manageable amount of code auditing (albeit
I'd like to see whether someone else can perhaps come up with
some potential, more realistic kind of bug that could fall into class
2 before volunteering to make an attempt at doing such an audit).

> If so then good, but I think we still need to have a proper
> conversation about how we are going to provide ABI stability to qemu
> in practice.

Yes, the ABI stability aspect still needs coming to an agreement.
Since relaxation is easier than tightening, I'm going to retain the
current way the patches are in this regard, i.e. assume an
unstable hypercall ABI.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 1/2] AMD/VPMU: 0xc0010000 - 0xc001007 MSRs are in PMU range

[Jan Beulich]

>>> On 08.08.16 at 16:06,  wrote:
> On 08/08/2016 09:53 AM, Jan Beulich wrote:
> On 08.08.16 at 15:41,  wrote:
>>> --- a/xen/arch/x86/traps.c
>>> +++ b/xen/arch/x86/traps.c
>>> @@ -2903,6 +2903,7 @@ static int emulate_privileged_op(struct cpu_user_regs 
> *regs)
>>>  {
>>>  vpmu_msr = 1;
>>>  case MSR_AMD_FAM15H_EVNTSEL0...MSR_AMD_FAM15H_PERFCTR5:
>>> +case MSR_K7_EVNTSEL0...MSR_K7_PERFCTR3:
>>>  if ( vpmu_msr || (boot_cpu_data.x86_vendor == 
> X86_VENDOR_AMD) )
>>>  {
>>>  if ( (vpmu_mode & XENPMU_MODE_ALL) &&
>>> @@ -3030,6 +3031,7 @@ static int emulate_privileged_op(struct cpu_user_regs 
> *regs)
>>>  {
>>>  vpmu_msr = 1;
>>>  case MSR_AMD_FAM15H_EVNTSEL0...MSR_AMD_FAM15H_PERFCTR5:
>>> +case MSR_K7_EVNTSEL0...MSR_K7_PERFCTR3:
>>>  if ( vpmu_msr || (boot_cpu_data.x86_vendor == 
> X86_VENDOR_AMD) )
>>>  {
>>>  
>> And all the logic in vpmu_amd.c is already in suitable shape? Namely
>> I'm wondering whether CTRL_RSVD_MASK indeed applies uniformly
>> to both the pre-Fam15 and Fam15+ MSRs.
> 
> The reserved bits look the same on all supported families --- bits
> 63:42, 39:36, 21 and 19. Except apparently on family 12h bit 19 is MBZ.

Isn't MBZ == reserved for all practical purposes? In any event the
patch is fine then afaic.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH 2/2] AMD/VPMU: Keep reserved MSR bits untouched but allow the rest to be written

[Boris Ostrovsky]

On 08/08/2016 09:56 AM, Jan Beulich wrote:
 On 08.08.16 at 15:41,  wrote:
>> While AMD APM suggests that reserved MSR bits are not supposed to be
>> touched, it is not clear how (or whether) HW enforces this for PMU
>> registers. At least on some family 10h processors writes of these bits
>> are apparently ignored: guests (such as Linux) assume that the bits
>> are zero and write the MSRs with that assumption in mind even though
>> the bits are set by the time OS/hypervisor starts runnning.
> So how did these bits become non-zero then?

Apparently one of the systems in our lab always had somewhat strange
post-BIOS state of these registers, with some bits (I think 19, can't
remember now) set. I never picked this up before since we don't really
test VPMU, we just initialize it and failure to initialize is non-fatal.

>
> Independent of that I think the relaxation would better only be done
> for those older CPUs.

Why? This can easily happen on any family.

-boris

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 1/3] livepach: Add .livepatch.hooks functions and test-case

[George Dunlap]

On 08/08/16 15:01, Jan Beulich wrote:
 On 08.08.16 at 15:42,  wrote:
>> On 05/08/16 16:35, Jan Beulich wrote:
>> On 04.08.16 at 17:49,  wrote:
 In general, the hooks provide flexibility when having to deal with
 unforeseen cases, but their application should be rarely required (<
 10%)."
>>>
>>> But the greater flexibility of course comes with increased chances
>>> of screwing things up. I'm therefore still not entirely convinced that
>>> such XSAs wouldn't then better not be live patched.
>>
>> But this functionality is optional, right?  So although I tend to agree
>> with you, we can let the person / organization preparing the patch take
>> that risk if they want to?
> 
> That's a valid point, albeit I think patch producer and patch consumer
> might have different views, and the patch consumer may not know
> (or even know to care) whether (s)he's got handed a patch with or
> without use of hooks.

But the patch consumer probably doesn't know for sure if the patch was
even tested, or if it even fixes the things it was meant to fix.  The
producer / consumer relationship will always have an aspect of trust and
some mechanism for feedback.

I suppose there is a sort of "prisoner's dilemma" aspect to this though:
suppose provider A thinks that hooks are too dangerous, and doesn't
provide a hotfix for a certain XSA as a result, but provider B, seeing
an opportunity, ignores the risks and provides a hotpatch for their
product instead.  If the consumers can't evaluate the danger, provider A
will be pressured into providing a patch even though they don't think
it's a good idea.

So if this really is too dangerous to be used regularly, there is an
argument to be made not to accept the functionality.  (I'm not making an
argument either way at the moment.)

 -George

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [xen-unstable-smoke test] 100344: regressions - FAIL

[osstest service owner]

flight 100344 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/100344/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt   5 libvirt-buildfail REGR. vs. 100341

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-xl  12 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  13 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  2e426d6eecfd358b6a78553e63fcb24548010537
baseline version:
 xen  72798298c751a3df21473a0d86d05ec7bb46e258

Last test of basis   100341  2016-08-08 10:02:49 Z0 days
Testing same since   100344  2016-08-08 13:01:47 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  George Dunlap 
  Ian Jackson 
  Ian Jackson 
  Wei Liu 

jobs:
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  fail
 test-armhf-armhf-xl  pass
 test-amd64-amd64-xl-qemuu-debianhvm-i386 pass
 test-amd64-amd64-libvirt blocked 



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.


commit 2e426d6eecfd358b6a78553e63fcb24548010537
Author: Andrew Cooper 
Date:   Wed Aug 3 16:56:56 2016 +

x86/traps: Drop use_error_code parameter from do_{,guest_}trap()

Whether or not an error code is needed can be determinted entirely from the
trapnr paramter, as error codes are architecturally specified.

Introduce TRAP_HAVE_EC as a bitmap of reserved vectors which have error 
codes,
and drop the use_error_code from all callsites.

As a result, the DO_ERROR{,_NOCODE}() macros become entirely superflouous 
and
can be dropped.  Update the exception_table to point straight at do_trap().

Signed-off-by: Andrew Cooper 
Reviewed-by: Jan Beulich 

commit 9ee1d0395576283b24c87f8d5ec7bb123cadc27c
Author: Ian Jackson 
Date:   Mon Aug 8 11:21:31 2016 +0100

libxl: CODING_STYLE: Forbid if (...) { stmt; } else stmt;

And clarify that the rule about omitting braces for single statements
is optional (it is even contradicted by the example).

Signed-off-by: Ian Jackson 
Acked-by: Wei Liu 

commit 0cd19069c88744c6558015934633727a69988bce
Author: Wei Liu 
Date:   Mon Aug 1 10:55:59 2016 +0100

xl: use xenconsole startup protocol

If user asks xl to automatically connect to console when creating a
guest, use the new startup protocol before trying to unpause domain so
that we don't lose any console output.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 

commit f2333d5f9f9fd8e823bad7bf8804c9423b97eb08
Author: Wei Liu 
Date:   Mon Aug 1 10:36:57 2016 +0100

docs: document xenconsole startup protocol

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 

commit 731df782267983d06c929fb513f420a1579021f9
Author: Wei Liu 
Date:   Mon Aug 1 10:28:00 2016 +0100

libxl: libxl_{primary_,}console_exec now take notify_fd argument

The new argument will be passed down to xenconsole process, which then
uses it to notify readiness.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 

commit 88502c075a393508a6315c4f905132b03be42347
Author: Wei Liu 
Date:   Mon Aug 1 12:20:09 2016 +0100

libxl: factor out libxl__console_tty_path

No other user yet.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 

commit 373f99d2eafd084073731486a20a3a5a2e120548
Author: Wei Liu 
Date:   Fri Jul 29 18:24:25 2016 +0100

tools/console: introduce --start-notify-fd option for console client

The console client will write 0x00 to that fd before entering console
loop to indicate its readiness.

Signed-off-by: Wei Liu 
Acked-by: Ian Jackson 

commit 9fc57406d5b629ad0190b3b3d2d034f88896d5cb
Author: Wei Liu 
Date:   Fri Jul 29 18:22:26 2016 +0100

tools/console: fix help string in client

There is no short '-t' option.

Signed-off-by: Wei Liu 
 

Re: [Xen-devel] [PATCH V2] x86/hvm: Allow guest_request vm_events coming from userspace

[Razvan Cojocaru]

On 08/08/2016 02:34 PM, Jan Beulich wrote:
 On 08.08.16 at 10:06,  wrote:
>> Allow guest userspace code to request that a vm_event be sent out
>> via VMCALL. This functionality seems to be handy for a number of
>> Xen developers, as stated on the mailing list (thread "[Xen-devel]
>> HVMOP_guest_request_vm_event only works from guest in ring0").
>>
>> Signed-off-by: Razvan Cojocaru 
>>
>> ---
>> Changes since V1:
>>  - No longer repeating the check when mode == 8.
> 
> Technically this looks correct to me now. Albeit I'm still not really
> convinced we actually want to start making exceptions here.

I am in any case happy that it's been properly reviewed (thank you!).

The exception here is IMO warranted by the fact that this particular
event has been especially designed so as to be able to signal from the
guest to an introspection application, hence the special status.
Considering also that, as Andrew has pointed out, alternatives are
possible, and that there are several interested users, I thought it
would be worth a shot.

But in the end it is, understandably, up to the maintainers.


Thanks again,
Razvan

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] pci passthrough kills qemu-xen

[Olaf Hering]

On Wed, Jul 27, Konrad Rzeszutek Wilk wrote:

> >  Xen 4.7.20160704T103633.a492556-1.xen47
> > (XEN) Xen version 4.7.20160704T103633.a492556-1.xen47 (abuild@) (gcc (SUSE 
> > Linux) 4.8.5) debug=n Mon Jul  4 12:36:33 UTC 2016
> 
> Could you recompile it with debug=y?

Its attached, nothing special AFAICS.

> > (XEN) HVM2 restore: CPU 0
> 
> .. Huh? Where is hvmloader?
> 
> Is it even being loaded? 

No idea, how can I tell?


Olaf
 Xen 4.7.20160726T130829.899495b-1.xen47
(XEN) Xen version 4.7.20160726T130829.899495b-1.xen47 (abuild@) (gcc (SUSE 
Linux) 4.8.5) debug=y Tue Jul 26 14:08:29 UTC 2016
(XEN) Latest ChangeSet: 2016-07-26 14:08:29 +0100 git:899495b
(XEN) Bootloader: GRUB2 2.02~beta2
(XEN) Command line: Xcrashkernel=512M-:256M console=com1 com1=115200 loglvl=all 
guest_loglvl=all
(XEN) Video information:
(XEN)  VGA is text mode 80x25, font 8x16
(XEN)  VBE/DDC methods: V2; EDID transfer time: 1 seconds
(XEN) Disc information:
(XEN)  Found 1 MBR signatures
(XEN)  Found 1 EDD information structures
(XEN) Xen-e820 RAM map:
(XEN)   - 0009ec00 (usable)
(XEN)  000f - 0010 (reserved)
(XEN)  0010 - db6cfc00 (usable)
(XEN)  db6cfc00 - db723c00 (ACPI NVS)
(XEN)  db723c00 - db725c00 (ACPI data)
(XEN)  db725c00 - dc00 (reserved)
(XEN)  f800 - fc00 (reserved)
(XEN)  fec0 - fed00400 (reserved)
(XEN)  fed2 - feda (reserved)
(XEN)  fee0 - fef0 (reserved)
(XEN)  ffb0 - 0001 (reserved)
(XEN)  0001 - 00022000 (usable)
(XEN) ACPI: RSDP 000FEC00, 0024 (r2 DELL  )
(XEN) ACPI: XSDT 000FC7E0, 007C (r1 DELLB11K  15 ASL61)
(XEN) ACPI: FACP 000FC8D0, 00F4 (r3 DELLB11K  15 ASL61)
(XEN) ACPI: DSDT FFDCBEE5, 47BC (r1   DELLdt_ex 1000 INTL 20050624)
(XEN) ACPI: FACS DB6CFC00, 0040
(XEN) ACPI: SSDT FFDD07C2, 00AC (r1   DELLst_ex 1000 INTL 20050624)
(XEN) ACPI: APIC 000FC9C4, 0092 (r1 DELLB11K  15 ASL61)
(XEN) ACPI: BOOT 000FCA56, 0028 (r1 DELLB11K  15 ASL61)
(XEN) ACPI: ASF! 000FCA7E, 0096 (r32 DELLB11K  15 ASL61)
(XEN) ACPI: MCFG 000FCB14, 003C (r1 DELLB11K  15 ASL61)
(XEN) ACPI: HPET 000FCB50, 0038 (r1 DELLB11K  15 ASL61)
(XEN) ACPI: TCPA 000FCDAC, 0032 (r1 DELLB11K  15 ASL61)
(XEN) ACPI: DMAR 000FCDDE, 0068 (r1 DELLB11K  15 ASL61)
(XEN) ACPI: SLIC 000FCB88, 0176 (r1 DELLB11K  15 ASL61)
(XEN) ACPI: SSDT DB725C00, 2874 (r1  INTEL PPM RCM  8001 INTL 20061109)
(XEN) System RAM: 8118MB (8313268kB)
(XEN) No NUMA configuration found
(XEN) Faking a node at -00022000
(XEN) Domain heap initialised
(XEN) found SMP MP-table at 000fe710
(XEN) DMI 2.6 present.
(XEN) Using APIC driver default
(XEN) ACPI: PM-Timer IO Port: 0x808
(XEN) ACPI: SLEEP INFO: pm1x_cnt[1:804,1:0], pm1x_evt[1:800,1:0]
(XEN) ACPI: wakeup_vec[db6cfc0c], vec_size[20]
(XEN) ACPI: Local APIC address 0xfee0
(XEN) ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled)
(XEN) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled)
(XEN) ACPI: LAPIC_NMI (acpi_id[0xff] high level lint[0x1])
(XEN) ACPI: IOAPIC (id[0x08] address[0xfec0] gsi_base[0])
(XEN) IOAPIC[0]: apic_id 8, version 32, address 0xfec0, GSI 0-23
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
(XEN) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
(XEN) ACPI: IRQ0 used by override.
(XEN) ACPI: IRQ2 used by override.
(XEN) ACPI: IRQ9 used by override.
(XEN) Enabling APIC mode:  Flat.  Using 1 I/O APICs
(XEN) ACPI: HPET id: 0x8086a701 base: 0xfed0
(XEN) ERST table was not found
(XEN) Using ACPI (MADT) for SMP configuration information
(XEN) SMP: Allowing 8 CPUs (0 hotplug CPUs)
(XEN) IRQ limits: 24 GSI, 1528 MSI/MSI-X
(XEN) mce_intel.c:735: MCA Capability: BCAST 1 SER 0 CMCI 1 firstbank 0 
extended MCE MSR 0
(XEN) Intel machine check reporting enabled
(XEN) Using scheduler: SMP Credit Scheduler (credit)
(XEN) Detected 2793.056 MHz processor.
(XEN) Initing memory sharing.
(XEN) alt table 82d0802de590 -> 82d0802dfa90
(XEN) PCI: MCFG configuration 0: base f800 segment  buses 00 - 3f
(XEN) PCI: MCFG area at f800 reserved in E820
(XEN) PCI: Using MCFG for segment  bus 00-3f
(XEN) Intel VT-d iommu 0 supported page sizes: 4kB.
(XEN) Intel VT-d Snoop Control enabled.
(XEN) Intel VT-d Dom0 DMA Passthrough not enabled.
(XEN) Intel VT-

Re: [Xen-devel] Build problems with xen 4.7

[Peng Fan]

Hi,

On Fri, May 13, 2016 at 11:23:29AM -0400, Konrad Rzeszutek Wilk wrote:
>On Fri, May 13, 2016 at 03:25:52PM +0100, M A Young wrote:
>> On Fri, 13 May 2016, Jan Beulich wrote:
>> 
>> > >>> On 13.05.16 at 15:49,  wrote:
>> > > ...
>> > > 
>> > > Still an issue - with 4.7.0-rc1.
>> > 
>> > And I don't recall anyone having contributed a fix/workaround.
>> > 
>> > > If I do:
>> > > 
>> > > $export CFLAGS=" "'
>> > > $make
>> > > 
>> > > I end up with:
>> > > gcc -E -O1 -fno-omit-frame-pointer -m64 -DBUILD_ID -g 
>> > > -fno-strict-aliasing -std=gnu99 
>> > >[...]
>> > > :0:0: error: "__OBJECT_FILE__" redefined [-Werror]
>> > > :0:0: note: this is the location of the previous definition
>> > > :0:0: error: "__OBJECT_LABEL__" redefined [-Werror]
>> > > :0:0: note: this is the location of the previous definition
>> > > cc1: all warnings being treated as errors
>> > > Makefile:62: recipe for target 'compat/callback.i' failed

I met similar issue when cross compile xen for ARM64 on X86 PC using yocto poky 
4.9.3
toolchain.

aarch64-poky-linux-gcc -O2 -pipe -g -feliminate-unused-debug-types -O1 
-fno-omit-frame-pointer -DBUILD_ID -g -fno-strict-aliasing -std=gnu99 -Wall 
-Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-variable 
-Wno-unused-local-typedefs -O1 -fno-omit-frame-pointer -DBUILD_ID -g 
-fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes 
-Wdeclaration-after-statement -Wno-unused-but-set-variable 
-Wno-unused-local-typedefs -O1 -fno-omit-frame-pointer -DBUILD_ID -g 
-fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes 
-Wdeclaration-after-statement -Wno-unused-but-set-variable 
-Wno-unused-local-typedefs -nostdinc -fno-builtin -fno-common -Werror 
-Wredundant-decls -Wno-pointer-arith -pipe -g -D__XEN__ -include 
/home/Freenix/work/sw-stash/imx8/8qm/xen/xen/xen/include/xen/config.h 
'-D__OBJECT_FILE__="/home/Freenix/work/sw-stash/imx8/8qm/xen/xen/xen/xen"' 
-Wa,--strip-local-absolute -fno-optimize-sibling-calls -fno-omit-frame-pointer 
-MMD -MF /home/Freenix/work/sw-stash/imx8/8qm/xen/xen/xen/.xen.d -mcpu=generic 
-mgeneral-regs-only -DCONFIG_EARLY_PRINTK 
-DEARLY_PRINTK_INC=\"debug-imx8qm.inc\" -DEARLY_PRINTK_BAUD= 
-DEARLY_UART_BASE_ADDRESS=0x5a06 -DEARLY_UART_REG_SHIFT= 
-fno-optimize-sibling-calls 
-I/home/Freenix/work/sw-stash/imx8/8qm/xen/xen/xen/include -fno-stack-protector 
-fno-exceptions -Wnested-externs -DGCC_HAS_VISIBILITY_ATTRIBUTE -O1 
-fno-omit-frame-pointer -DBUILD_ID -g -fno-strict-aliasing -std=gnu99 -Wall 
-Wstrict-prototypes -Wdeclaration-after-statement -Wno-unused-but-set-variable 
-Wno-unused-local-typedefs -O1 -fno-omit-frame-pointer -DBUILD_ID -g 
-fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes 
-Wdeclaration-after-statement -Wno-unused-but-set-variable 
-Wno-unused-local-typedefs -nostdinc -fno-builtin -fno-common -Werror 
-Wredundant-decls -Wno-pointer-arith -pipe -g -D__XEN__ -include 
/home/Freenix/work/sw-stash/imx8/8qm/xen/xen/xen/include/xen/config.h 
'-D__OBJECT_FILE__="asm-offsets.s"' -Wa,--strip-local-absolute 
-fno-optimize-sibling-calls -fno-omit-frame-pointer -MMD -MF ./.asm-offsets.s.d 
-mcpu=generic -mgeneral-regs-only -DCONFIG_EARLY_PRINTK 
-DEARLY_PRINTK_INC=\"debug-imx8qm.inc\" -DEARLY_PRINTK_BAUD= 
-DEARLY_UART_BASE_ADDRESS=0x5a06 -DEARLY_UART_REG_SHIFT= 
-I/home/Freenix/work/sw-stash/imx8/8qm/xen/xen/xen/include -fno-stack-protector 
-fno-exceptions -Wnested-externs -DGCC_HAS_VISIBILITY_ATTRIBUTE -S -o 
asm-offsets.s arm64/asm-offsets.c
:0:0: error: "__OBJECT_FILE__" redefined [-Werror]
:0:0: note: this is the location of the previous definition
cc1: all warnings being treated as errors


>> > 
>> > My previous recommendation stands: Then just don't do this.
>> 
>> I hacked around it for my test builds (eg. see my test build at
>> https://copr.fedorainfracloud.org/coprs/myoung/xentest/build/204111/
>> ) by not setting CFLAGS in the environment, but by instead adding the 
>> recommended Fedora RPM settings into config/StdGNU.mk via a different 
>> environment variable.
>
>ah:
>
>--- xen-4.7.0/config/StdGNU.mk.orig2016-04-15 22:56:52.191227591 +0100
>+++ xen-4.7.0/config/StdGNU.mk 2016-04-15 23:01:40.978829756 +0100
>@@ -37,6 +37,12 @@
> 
> ifneq ($(debug),y)
> CFLAGS += -O2 -fomit-frame-pointer -fno-tree-coalesce-vars
>+ifeq ($(XEN_TARGET_ARCH),x86_64)
>+#might be cross-compiling so strip out possible x86_32 options
>+CFLAGS += $(shell echo $(CFLAGS_EXTRA) | sed -e 's/-m32//g' -e 
>'s/-march=i686//g' -e 's/-mtune=atom//g')
>+else
>+CFLAGS += $(CFLAGS_EXTRA)
>+endif
> else
> # Less than -O1 produces bad code and large stack frames
> CFLAGS += -O1 -fno-omit-frame-pointer
>
>
>And in the spec file:
>export CFLAGS_EXTRA="$RPM_OPT_FLAGS"
>make %{?_smp_mflags} %{?efi_flags} prefix=/usr dist-xen
>

Does this patch work when cross compile for ARM64?

Thanks,
Peng.

>
>> 
>> Another thing you might need to know if you are building xen on Fedora 24 
>> is that you need to add -fno-tree-coalesce-vars if you are on a g