[Xen-devel] [libvirt bisection] complete build-armhf-libvirt
branch xen-unstable xenbranch xen-unstable job build-armhf-libvirt testid libvirt-build Tree: libvirt git://libvirt.org/libvirt.git Tree: libvirt_gnulib https://git.savannah.gnu.org/git/gnulib.git/ Tree: libvirt_keycodemapdb https://gitlab.com/keycodemap/keycodemapdb.git Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git Tree: qemuu git://xenbits.xen.org/qemu-xen.git Tree: seabios git://xenbits.xen.org/osstest/seabios.git Tree: xen git://xenbits.xen.org/xen.git *** Found and reproduced problem changeset *** Bug is in tree: libvirt git://libvirt.org/libvirt.git Bug introduced: c7f75bf04d07506bd4d9e862b9b38a1e423d88b6 Bug not present: bfe9f25b49827f02027b5a5e88226ce933e1bd7c Last fail repro: http://logs.test-lab.xenproject.org/osstest/logs/144632/ commit c7f75bf04d07506bd4d9e862b9b38a1e423d88b6 Author: Daniel P. Berrangé Date: Fri Oct 18 14:18:36 2019 +0100 docs: introduce rst2html as a mandatory tool for building docs The rst2html tool is provided by python docutils, and as the name suggests, it converts RST documents into HTML. Basic rules are added for integrating RST docs into the website build process. This enables us to start writing docs on our website in RST format instead of HTML, without changing the rest of our website templating system away from XSLT yet. Reviewed-by: Michal Privoznik Signed-off-by: Daniel P. Berrangé For bisection revision-tuple graph see: http://logs.test-lab.xenproject.org/osstest/results/bisect/libvirt/build-armhf-libvirt.libvirt-build.html Revision IDs in each graph node refer, respectively, to the Trees above. Running cs-bisection-step --graph-out=/home/logs/results/bisect/libvirt/build-armhf-libvirt.libvirt-build --summary-out=tmp/144632.bisection-summary --basis-template=144517 --blessings=real,real-bisect libvirt build-armhf-libvirt libvirt-build Searching for failure / basis pass: 144615 fail [host=cubietruck-gleizes] / 144517 ok. Failure / basis pass flights: 144615 / 144517 Tree: libvirt git://libvirt.org/libvirt.git Tree: libvirt_gnulib https://git.savannah.gnu.org/git/gnulib.git/ Tree: libvirt_keycodemapdb https://gitlab.com/keycodemap/keycodemapdb.git Tree: ovmf git://xenbits.xen.org/osstest/ovmf.git Tree: qemuu git://xenbits.xen.org/qemu-xen.git Tree: seabios git://xenbits.xen.org/osstest/seabios.git Tree: xen git://xenbits.xen.org/xen.git Latest fda14dd7821d3e20b1416b90525242b7d4306fe9 1f6fb368c04919243e2c70f2aa514a5f88e95309 6280c94f306df6a20bbc100ba15a5a81af0366e6 49054b6bb66d35484e92c65f27584c4283a60986 933ebad2470a169504799a1d95b8e410bd9847ef c9ba5276e3217ac6a1ec772dbebf568ba3a8a55d 8359dde71826bfbcf04412bda001903f809571c9 Basis pass d0d728c7c00fd3a62731e50c7bc646df323c0622 1f6fb368c04919243e2c70f2aa514a5f88e95309 6280c94f306df6a20bbc100ba15a5a81af0366e6 4d613feee57ebd4680f3c23398a9b33723f29fd6 933ebad2470a169504799a1d95b8e410bd9847ef c9ba5276e3217ac6a1ec772dbebf568ba3a8a55d 42c8cdc039d6dc7d6aea8008bb24622eaf4b7bc8 Generating revisions with ./adhoc-revtuple-generator git://libvirt.org/libvirt.git#d0d728c7c00fd3a62731e50c7bc646df323c0622-fda14dd7821d3e20b1416b90525242b7d4306fe9 https://git.savannah.gnu.org/git/gnulib.git/#1f6fb368c04919243e2c70f2aa514a5f88e95309-1f6fb368c04919243e2c70f2aa514a5f88e95309 https://gitlab.com/keycodemap/keycodemapdb.git#6280c94f306df6a20bbc100ba15a5a81af0366e6-6280c94f306df6a20bbc100ba15a5a81af0366e6 git://xenbits.xen.org/osstest/ovmf.git#4d613feee57ebd4680f3c23398a9b33723f29fd\ 6-49054b6bb66d35484e92c65f27584c4283a60986 git://xenbits.xen.org/qemu-xen.git#933ebad2470a169504799a1d95b8e410bd9847ef-933ebad2470a169504799a1d95b8e410bd9847ef git://xenbits.xen.org/osstest/seabios.git#c9ba5276e3217ac6a1ec772dbebf568ba3a8a55d-c9ba5276e3217ac6a1ec772dbebf568ba3a8a55d git://xenbits.xen.org/xen.git#42c8cdc039d6dc7d6aea8008bb24622eaf4b7bc8-8359dde71826bfbcf04412bda001903f809571c9 Auto packing the repository in background for optimum performance. See "git help gc" for manual housekeeping. error: The last gc run reported the following. Please correct the root cause and remove gc.log. Automatic cleanup will not be performed until the file is removed. warning: There are too many unreachable loose objects; run 'git prune' to remove them. Auto packing the repository in background for optimum performance. See "git help gc" for manual housekeeping. error: The last gc run reported the following. Please correct the root cause and remove gc.log. Automatic cleanup will not be performed until the file is removed. warning: There are too many unreachable loose objects; run 'git prune' to remove them. Loaded 15001 nodes in revision graph Searching for test results: 144517 pass d0d728c7c00fd3a62731e50c7bc646df323c0622 1f6fb368c04919243e2c70f2aa514a5f88e95309 6280c94f306df6a20bbc100ba15a5a81af0366e6 4d613feee57ebd4680f3c23398a9b33723f29fd6
Re: [Xen-devel] [PATCH 1/2] x86/mm: factor out the code for shattering an l3 PTE
On 07/12/2019 19:37, Andrew Cooper wrote:
On 07/12/2019 19:04, Julien Grall wrote:
Hi Hongyan,
On 06/12/2019 15:53, Hongyan Xia wrote:
map_pages_to_xen and modify_xen_mappings are performing almost exactly
the same operations when shattering an l3 PTE, the only difference
being whether we want to flush.
Signed-off-by: Hongyan Xia
---
xen/arch/x86/mm.c | 85 ++-
1 file changed, 40 insertions(+), 45 deletions(-)
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 7d4dd80a85..421083 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -5151,6 +5151,43 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v)
flush_area_local((const void *)v, f) : \
flush_area_all((const void *)v, f))
+/* Shatter an l3 entry and populate l2. If virt is passed in, also
do flush. */
+static void shatter_l3e(l3_pgentry_t *pl3e, l2_pgentry_t *l2t,
+ unsigned long virt, bool locking)
+{
+ unsigned int i;
+ l3_pgentry_t ol3e = *pl3e;
+
+ for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
+ l2e_write(l2t + i,
+ l2e_from_pfn(l3e_get_pfn(ol3e) +
+ (i << PAGETABLE_ORDER),
+ l3e_get_flags(ol3e)));
I understand this is just a factor out of the current code, but the
code feels wrong (at least in theory) and wasteful.
You would allocate the L2 table, prepare it and then free it if the L3
entry has _PAGE_PRESENT or _PAGE_PSE cleared.
Also, in theory, there is nothing preventing the l3 flags to be
modified behind your back. So you could end up to generate the l2
entries with the old l3 flags.
In nutshell, it feels to me that the shattering/allocation should
happen with the lock taken. This would also allow you to not allocate
the l2 table when you are removing the page.
The existing behaviour is deliberate and most likely wants to remain.
It makes adjustments safe to concurrent modifications, while reducing
the critical section of the lock to only the alteration of the live PTEs.
We don't expect concurrent conflicting updates to these pagetables at
all, but extending the locked region around the memory allocation and
writing the new pagetable is a bottlekneck to parallel updates of
independent addresses.
I am quite interested to know a bit more about the potential
bottlenecks. When I rewrote the Xen PT helpers for Arm I didn't see many
access to the Xen PT.
To make a comparison, I see much more update to the P2M. So I would
expect such optimization to be more a concern there. Yet we take the
lock for the full update. Maybe this was an oversight when the P2M was
created?
Cheers,
--
Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [GIT PULL] xen: branch for v5.5-rc1
The pull request you sent on Fri, 6 Dec 2019 17:55:11 +0100: > git://git.kernel.org/pub/scm/linux/kernel/git/xen/tip.git > for-linus-5.5b-rc1-tag has been merged into torvalds/linux.git: https://git.kernel.org/torvalds/c/f74fd13f4585e418a3e630a82468be58bf1d98c1 Thank you! -- Deet-doot-dot, I am a bot. https://korg.wiki.kernel.org/userdoc/prtracker ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH v3 1/3] xen/flask: Drop the gen-policy.py script
The script is Python 2 specific, and fails with string/binary issues with
Python 3:
Traceback (most recent call last):
File "gen-policy.py", line 14, in
for char in sys.stdin.read():
File "/usr/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8c in position 0:
invalid start byte
Fixing the script to be compatible isn't hard, but using python here is
wasteful. Drop the script entirely, and write a short flask-policy.S instead.
Signed-off-by: Andrew Cooper
---
CC: Daniel De Graaf
CC: Juergen Gross
v2:
* Fix tabs vs spaces issues
v3:
* Use % rather than @ for progbits/object, for Arm32 build.
* Spotted by https://travis-ci.org/andyhhp/xen/builds/622085138
For 4.13. This is a blocker to our intent to by Py3-clean in this release.
Discovered entirely accidently when testing the final patch.
---
xen/xsm/flask/Makefile | 6 ++
xen/xsm/flask/flask-policy.S | 20
xen/xsm/flask/gen-policy.py | 23 ---
3 files changed, 22 insertions(+), 27 deletions(-)
create mode 100644 xen/xsm/flask/flask-policy.S
delete mode 100644 xen/xsm/flask/gen-policy.py
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index f5ffab1226..7c3f381287 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -27,7 +27,8 @@ $(FLASK_H_FILES): $(FLASK_H_DEPEND)
$(AV_H_FILES): $(AV_H_DEPEND)
$(CONFIG_SHELL) policy/mkaccess_vector.sh $(AWK) $(AV_H_DEPEND)
-obj-$(CONFIG_XSM_FLASK_POLICY) += policy.o
+obj-bin-$(CONFIG_XSM_FLASK_POLICY) += flask-policy.o
+flask-policy.o: policy.bin
FLASK_BUILD_DIR := $(CURDIR)
POLICY_SRC := $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
@@ -36,9 +37,6 @@ policy.bin: FORCE
$(MAKE) -f $(XEN_ROOT)/tools/flask/policy/Makefile.common -C
$(XEN_ROOT)/tools/flask/policy FLASK_BUILD_DIR=$(FLASK_BUILD_DIR)
cmp -s $(POLICY_SRC) $@ || cp $(POLICY_SRC) $@
-policy.c: policy.bin gen-policy.py
- $(PYTHON) gen-policy.py < $< > $@
-
.PHONY: clean
clean::
rm -f $(ALL_H_FILES) *.o $(DEPS_RM) policy.* $(POLICY_SRC)
diff --git a/xen/xsm/flask/flask-policy.S b/xen/xsm/flask/flask-policy.S
new file mode 100644
index 00..81bfc09ec2
--- /dev/null
+++ b/xen/xsm/flask/flask-policy.S
@@ -0,0 +1,20 @@
+.section .init.rodata, "a", %progbits
+
+/* const unsigned char xsm_flask_init_policy[] __initconst */
+.align 4
+.global xsm_flask_init_policy
+xsm_flask_init_policy:
+.incbin "policy.bin"
+.Lend:
+
+.type xsm_flask_init_policy, %object
+.size xsm_flask_init_policy, . - xsm_flask_init_policy
+
+/* const unsigned int __initconst xsm_flask_init_policy_size */
+.align 4
+.global xsm_flask_init_policy_size
+xsm_flask_init_policy_size:
+.long .Lend - xsm_flask_init_policy
+
+.type xsm_flask_init_policy_size, %object
+.size xsm_flask_init_policy_size, . - xsm_flask_init_policy_size
diff --git a/xen/xsm/flask/gen-policy.py b/xen/xsm/flask/gen-policy.py
deleted file mode 100644
index c7501e4614..00
--- a/xen/xsm/flask/gen-policy.py
+++ /dev/null
@@ -1,23 +0,0 @@
-#!/usr/bin/env python
-import sys
-
-policy_size = 0
-
-sys.stdout.write("""
-/* This file is autogenerated by gen_policy.py */
-#include
-#include
-
-const unsigned char xsm_flask_init_policy[] __initconst = {
-""")
-
-for char in sys.stdin.read():
-sys.stdout.write(" 0x%02x," % ord(char))
-policy_size = policy_size + 1
-if policy_size % 13 == 0:
-sys.stdout.write("\n")
-
-sys.stdout.write("""
-};
-const unsigned int __initconst xsm_flask_init_policy_size = %d;
-""" % policy_size)
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH for-4.13 v3 0/3] xen: Build fixes related to Python3
Patch 3 is a fix for a problem reported on IRC. It is a very-nice-to-have considering our attempt to make Xen 4.13 Py3-clean. While testing patch 3, it became apparent that XSM/Flask isn't Py3-clean, and this is a blocker. It is addressed in patch 1. Patch 2 addresses a bug spotted by Gitlab while testing v1 of this series. It isn't strictly a Py3 bug, but is a build system robustness fix. v3 of this series fixes all Travis and Gitlab identified issues: https://travis-ci.org/andyhhp/xen/builds/622092503 https://gitlab.com/xen-project/people/andyhhp/xen/pipelines/101417861 Andrew Cooper (3): xen/flask: Drop the gen-policy.py script xen/banner: Drop the fig-to-oct.py script xen/build: Automatically locate a suitable python interpreter xen/Makefile | 7 ++- xen/tools/fig-to-oct.py | 18 -- xen/tools/process-banner.sed | 14 ++ xen/xsm/flask/Makefile | 6 ++ xen/xsm/flask/flask-policy.S | 20 xen/xsm/flask/gen-policy.py | 23 --- 6 files changed, 42 insertions(+), 46 deletions(-) delete mode 100644 xen/tools/fig-to-oct.py create mode 100755 xen/tools/process-banner.sed create mode 100644 xen/xsm/flask/flask-policy.S delete mode 100644 xen/xsm/flask/gen-policy.py -- 2.11.0 ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH v3 2/3] xen/banner: Drop the fig-to-oct.py script
The script is 664 rather than 775, so the banner conversion doesn't actually
work if $(PYTHON) is empty:
/bin/sh: tools/fig-to-oct.py: Permission denied
make[3]: *** [include/xen/compile.h] Error 126
make[3]: Leaving directory `/builds/xen-project/people/andyhhp/xen/xen'
Fixing this is easy, but using python here is wasteful. compile.h doesn't
need XEN_BANNER rendering in octal, and text is much more simple to handle.
Replace fig-to-oct.py with a smaller sed script. This could be a shell
one-liner, but it is much more simple to comment sensibly, and doesn't need to
include the added cognative load of makefile and shell escaping.
While changing this logic, take the opportunity to optimise the banner
space (and time on the serial port) by dropping trailing whitespace, which is
84 characters for current staging.
Signed-off-by: Andrew Cooper
---
CC: George Dunlap
CC: Ian Jackson
CC: Jan Beulich
CC: Konrad Rzeszutek Wilk
CC: Stefano Stabellini
CC: Wei Liu
CC: Julien Grall
CC: Juergen Gross
v2:
* New
v3:
* Fix error: backslash-newline at end of file [-Werror]
* Implement as a sed script.
This (v3) is how happy in the CI:
https://gitlab.com/xen-project/people/andyhhp/xen/pipelines/101409945
Spotted by Gitlab CI, caused by `which` not being present in some of the
CentOS containers. While this is more of a container bug than anything else,
it does highlight that the build ought to cope.
---
xen/Makefile | 2 +-
xen/tools/fig-to-oct.py | 18 --
xen/tools/process-banner.sed | 14 ++
3 files changed, 15 insertions(+), 19 deletions(-)
delete mode 100644 xen/tools/fig-to-oct.py
create mode 100755 xen/tools/process-banner.sed
diff --git a/xen/Makefile b/xen/Makefile
index 99701e3165..949ca6eb03 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -176,7 +176,7 @@ include/xen/compile.h: include/xen/compile.h.in .banner
-e 's!@@changeset@@!$(shell tools/scmversion $(XEN_ROOT) || echo
"unavailable")!g' \
< include/xen/compile.h.in > $@.new
@cat .banner
- @$(PYTHON) tools/fig-to-oct.py < .banner >> $@.new
+ @sed -rf tools/process-banner.sed < .banner >> $@.new
@mv -f $@.new $@
include/asm-$(TARGET_ARCH)/asm-offsets.h: arch/$(TARGET_ARCH)/asm-offsets.s
diff --git a/xen/tools/fig-to-oct.py b/xen/tools/fig-to-oct.py
deleted file mode 100644
index db4fd32159..00
--- a/xen/tools/fig-to-oct.py
+++ /dev/null
@@ -1,18 +0,0 @@
-#!/usr/bin/env python
-import sys
-
-chars_per_line = 18
-chars_so_far = 0
-
-sys.stdout.write('"')
-
-for char in sys.stdin.read():
-
-sys.stdout.write("\\%03o" % ord(char))
-chars_so_far = chars_so_far + 1
-
-if chars_so_far == chars_per_line:
-chars_so_far = 0
-sys.stdout.write('" \\\n"')
-
-sys.stdout.write('"\n')
diff --git a/xen/tools/process-banner.sed b/xen/tools/process-banner.sed
new file mode 100755
index 00..56c76558bc
--- /dev/null
+++ b/xen/tools/process-banner.sed
@@ -0,0 +1,14 @@
+#!/bin/sed -rf
+# Process a text input, to turn it into a C string for the XEN_BANNER macro.
+
+# Strip trailing whitespace.
+s_ *$__
+
+# Escape backslashes.
+s_\\__g
+
+# Enclose the line in "...\n".
+s_(.*)_"\1\\n"_
+
+# Trailing \ on all but the final line.
+$!s_$_ \\_
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH v3 3/3] xen/build: Automatically locate a suitable python interpreter
Needing to pass PYTHON=python3 into hypervisor builds is irritating and unnecessary. Locate a suitable interpreter automatically, defaulting to Py3 if it is available. Reported-by: Steven Haigh Signed-off-by: Andrew Cooper --- CC: George Dunlap CC: Ian Jackson CC: Jan Beulich CC: Konrad Rzeszutek Wilk CC: Stefano Stabellini CC: Wei Liu CC: Julien Grall CC: Steven Haigh CC: Juergen Gross v2: * Cope with `which` not being present in the system. * Only evaulate the shell command one, rather than once per $(PTHON) usage For 4.13. This is a very-nice-to-have WRT our Py3-clean intention. --- xen/Makefile | 5 + 1 file changed, 5 insertions(+) diff --git a/xen/Makefile b/xen/Makefile index 949ca6eb03..f36a5bc6c0 100644 --- a/xen/Makefile +++ b/xen/Makefile @@ -13,6 +13,11 @@ export XEN_BUILD_TIME?= $(shell LC_ALL=C date +%T) export XEN_BUILD_HOST ?= $(shell hostname) export XEN_CONFIG_EXPERT ?= n +# Best effort attempt to find a python interpreter, defaulting to Python 3 if +# available. Fall back to just `python` if `which` is nowhere to be found. +PYTHON_INTERPRETER := $(word 1,$(shell which python3 python python2 2>/dev/null) python) +export PYTHON ?= $(PYTHON_INTERPRETER) + export BASEDIR := $(CURDIR) export XEN_ROOT := $(BASEDIR)/.. -- 2.11.0 ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH v4 3/6] arm64: remove uaccess_ttbr0 asm macros from cache functions
Hi Pavel,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on next-20191206]
[cannot apply to arm64/for-next/core xen-tip/linux-next v5.4-rc8 arm/for-next
v5.4]
[if your patch is applied to the wrong git tree, please drop us a note to help
improve the system. BTW, we also suggest to use '--base' option to specify the
base tree in git format-patch, please see https://stackoverflow.com/a/37406982]
url:
https://github.com/0day-ci/linux/commits/Pavel-Tatashin/Use-C-inlines-for-uaccess/20191207-044947
base:838333c80c4f64a4ef9f5486f8bbc73312cd3abf
config: arm64-defconfig (attached as .config)
compiler: aarch64-linux-gcc (GCC) 7.5.0
reproduce:
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O
~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
GCC_VERSION=7.5.0 make.cross ARCH=arm64
If you fix the issue, kindly add following tag
Reported-by: kbuild test robot
All errors (new ones prefixed by >>):
arch/arm64/kernel/sys_compat.c: In function '__do_compat_cache_op':
>> arch/arm64/kernel/sys_compat.c:44:7: error: void value not ignored as it
>> ought to be
ret = __flush_cache_user_range(start, start + chunk);
^
vim +44 arch/arm64/kernel/sys_compat.c
3dd681d944f6d8 Will Deacon 2012-03-05 23
a2d25a5391ca21 Vladimir Murzin 2014-12-01 24 static long
a2d25a5391ca21 Vladimir Murzin 2014-12-01 25 __do_compat_cache_op(unsigned
long start, unsigned long end)
3dd681d944f6d8 Will Deacon 2012-03-05 26 {
a2d25a5391ca21 Vladimir Murzin 2014-12-01 27 long ret;
3dd681d944f6d8 Will Deacon 2012-03-05 28
a2d25a5391ca21 Vladimir Murzin 2014-12-01 29 do {
a2d25a5391ca21 Vladimir Murzin 2014-12-01 30 unsigned long chunk =
min(PAGE_SIZE, end - start);
3dd681d944f6d8 Will Deacon 2012-03-05 31
a2d25a5391ca21 Vladimir Murzin 2014-12-01 32 if
(fatal_signal_pending(current))
a2d25a5391ca21 Vladimir Murzin 2014-12-01 33 return 0;
a2d25a5391ca21 Vladimir Murzin 2014-12-01 34
222fc0c8503d98 James Morse 2019-10-17 35 if
(cpus_have_const_cap(ARM64_WORKAROUND_1542419)) {
222fc0c8503d98 James Morse 2019-10-17 36 /*
222fc0c8503d98 James Morse 2019-10-17 37* The
workaround requires an inner-shareable tlbi.
222fc0c8503d98 James Morse 2019-10-17 38* We pick the
reserved-ASID to minimise the impact.
222fc0c8503d98 James Morse 2019-10-17 39*/
27a22fbdeedd6c Catalin Marinas 2019-10-28 40
__tlbi(aside1is, __TLBI_VADDR(0, 0));
222fc0c8503d98 James Morse 2019-10-17 41 dsb(ish);
222fc0c8503d98 James Morse 2019-10-17 42 }
222fc0c8503d98 James Morse 2019-10-17 43
a2d25a5391ca21 Vladimir Murzin 2014-12-01 @44 ret =
__flush_cache_user_range(start, start + chunk);
a2d25a5391ca21 Vladimir Murzin 2014-12-01 45 if (ret)
a2d25a5391ca21 Vladimir Murzin 2014-12-01 46 return ret;
a2d25a5391ca21 Vladimir Murzin 2014-12-01 47
a2d25a5391ca21 Vladimir Murzin 2014-12-01 48 cond_resched();
a2d25a5391ca21 Vladimir Murzin 2014-12-01 49 start += chunk;
a2d25a5391ca21 Vladimir Murzin 2014-12-01 50 } while (start < end);
a2d25a5391ca21 Vladimir Murzin 2014-12-01 51
a2d25a5391ca21 Vladimir Murzin 2014-12-01 52 return 0;
3dd681d944f6d8 Will Deacon 2012-03-05 53 }
3dd681d944f6d8 Will Deacon 2012-03-05 54
:: The code at line 44 was first introduced by commit
:: a2d25a5391ca219f196f9fee7b535c40d201c6bf arm64: compat: align cacheflush
syscall with arch/arm
:: TO: Vladimir Murzin
:: CC: Will Deacon
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/hyperkitty/list/kbuild-...@lists.01.org Intel Corporation
.config.gz
Description: application/gzip
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 1/2] x86/mm: factor out the code for shattering an l3 PTE
On 07/12/2019 19:04, Julien Grall wrote:
> Hi Hongyan,
>
> On 06/12/2019 15:53, Hongyan Xia wrote:
>> map_pages_to_xen and modify_xen_mappings are performing almost exactly
>> the same operations when shattering an l3 PTE, the only difference
>> being whether we want to flush.
>>
>> Signed-off-by: Hongyan Xia
>> ---
>> xen/arch/x86/mm.c | 85 ++-
>> 1 file changed, 40 insertions(+), 45 deletions(-)
>>
>> diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
>> index 7d4dd80a85..421083 100644
>> --- a/xen/arch/x86/mm.c
>> +++ b/xen/arch/x86/mm.c
>> @@ -5151,6 +5151,43 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v)
>> flush_area_local((const void *)v, f) : \
>> flush_area_all((const void *)v, f))
>> +/* Shatter an l3 entry and populate l2. If virt is passed in, also
>> do flush. */
>> +static void shatter_l3e(l3_pgentry_t *pl3e, l2_pgentry_t *l2t,
>> + unsigned long virt, bool locking)
>> +{
>> + unsigned int i;
>> + l3_pgentry_t ol3e = *pl3e;
>> +
>> + for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
>> + l2e_write(l2t + i,
>> + l2e_from_pfn(l3e_get_pfn(ol3e) +
>> + (i << PAGETABLE_ORDER),
>> + l3e_get_flags(ol3e)));
>
> I understand this is just a factor out of the current code, but the
> code feels wrong (at least in theory) and wasteful.
>
> You would allocate the L2 table, prepare it and then free it if the L3
> entry has _PAGE_PRESENT or _PAGE_PSE cleared.
>
> Also, in theory, there is nothing preventing the l3 flags to be
> modified behind your back. So you could end up to generate the l2
> entries with the old l3 flags.
>
> In nutshell, it feels to me that the shattering/allocation should
> happen with the lock taken. This would also allow you to not allocate
> the l2 table when you are removing the page.
The existing behaviour is deliberate and most likely wants to remain.
It makes adjustments safe to concurrent modifications, while reducing
the critical section of the lock to only the alteration of the live PTEs.
We don't expect concurrent conflicting updates to these pagetables at
all, but extending the locked region around the memory allocation and
writing the new pagetable is a bottlekneck to parallel updates of
independent addresses.
~Andrew
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 1/2] x86/mm: factor out the code for shattering an l3 PTE
Hi Hongyan,
On 06/12/2019 15:53, Hongyan Xia wrote:
map_pages_to_xen and modify_xen_mappings are performing almost exactly
the same operations when shattering an l3 PTE, the only difference
being whether we want to flush.
Signed-off-by: Hongyan Xia
---
xen/arch/x86/mm.c | 85 ++-
1 file changed, 40 insertions(+), 45 deletions(-)
diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
index 7d4dd80a85..421083 100644
--- a/xen/arch/x86/mm.c
+++ b/xen/arch/x86/mm.c
@@ -5151,6 +5151,43 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long v)
flush_area_local((const void *)v, f) : \
flush_area_all((const void *)v, f))
+/* Shatter an l3 entry and populate l2. If virt is passed in, also do flush. */
+static void shatter_l3e(l3_pgentry_t *pl3e, l2_pgentry_t *l2t,
+unsigned long virt, bool locking)
+{
+unsigned int i;
+l3_pgentry_t ol3e = *pl3e;
+
+for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
+l2e_write(l2t + i,
+ l2e_from_pfn(l3e_get_pfn(ol3e) +
+ (i << PAGETABLE_ORDER),
+ l3e_get_flags(ol3e)));
I understand this is just a factor out of the current code, but the code
feels wrong (at least in theory) and wasteful.
You would allocate the L2 table, prepare it and then free it if the L3
entry has _PAGE_PRESENT or _PAGE_PSE cleared.
Also, in theory, there is nothing preventing the l3 flags to be modified
behind your back. So you could end up to generate the l2 entries with
the old l3 flags.
In nutshell, it feels to me that the shattering/allocation should happen
with the lock taken. This would also allow you to not allocate the l2
table when you are removing the page.
Cheers,
--
Julien Grall
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 1/2] x86/mm: factor out the code for shattering an l3 PTE
Hi, On 07/12/2019 18:20, Xia, Hongyan wrote: hmm... if we want to make the nullification visible to the caller we might need to pass &. I wonder if it makes sense to simply move the memory allocation of pl2e into shatter_l3e as well, so that the caller cannot have any ideas. AFAICT, the allocation is done when shattering the page-tables. So it would make sense to move it withing the shatter function. Note that you will need to propagate the error if there is any. Cheers, ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
Re: [Xen-devel] [PATCH 1/2] x86/mm: factor out the code for shattering an l3 PTE
Hi Andrew,
On Fri, 2019-12-06 at 17:50 +, Andrew Cooper wrote:
> On 06/12/2019 15:53, Hongyan Xia wrote:
> > map_pages_to_xen and modify_xen_mappings are performing almost
> > exactly
> > the same operations when shattering an l3 PTE, the only difference
> > being whether we want to flush.
> >
> > Signed-off-by: Hongyan Xia
>
> Just for reviewing purposes, how does this relate to your other
> posted
> series. Is it independent, a prerequisite, or does it depend on that
> series?
This is independent. Other series I posted will touch a lot of PTE
functions, and as Jan suggested, it would be nice to refactor some of
the long and complicated ones before changing them, which could also
prepare us for 5-level paging in the future. Although if these
refactoring patches get in, I need to rebase the series I posted
before.
>
> > ---
> > xen/arch/x86/mm.c | 85 ++-
> >
> > 1 file changed, 40 insertions(+), 45 deletions(-)
> >
> > diff --git a/xen/arch/x86/mm.c b/xen/arch/x86/mm.c
> > index 7d4dd80a85..421083 100644
> > --- a/xen/arch/x86/mm.c
> > +++ b/xen/arch/x86/mm.c
> > @@ -5151,6 +5151,43 @@ l1_pgentry_t *virt_to_xen_l1e(unsigned long
> > v)
> > flush_area_local((const void *)v, f) : \
> > flush_area_all((const void *)v, f))
> >
> > +/* Shatter an l3 entry and populate l2. If virt is passed in, also
> > do flush. */
> > +static void shatter_l3e(l3_pgentry_t *pl3e, l2_pgentry_t *l2t,
> > +unsigned long virt, bool locking)
> > +{
> > +unsigned int i;
> > +l3_pgentry_t ol3e = *pl3e;
> > +
> > +for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++ )
> > +l2e_write(l2t + i,
> > + l2e_from_pfn(l3e_get_pfn(ol3e) +
> > + (i << PAGETABLE_ORDER),
> > + l3e_get_flags(ol3e)));
>
> The PTE macros are especially poor for generated asm, and in cases
> like
> this, I'd like to improve things.
>
> In particular, I believe the following code has identical behaviour:
>
> l2_pgentry_t nl2e = l2e_from_intpte(l3e_get_intpte(ol3e));
>
> for ( i = 0; i < L2_PAGETABLE_ENTRIES; i++, nl2e.l2 +=
> PAGETABLE_ORDER )
> l2e_write(l2t + i, nl2e);
>
> (although someone please double check my logic) and rather better asm
> generation. (I also expect there to be some discussion on whether
> using
> n2le.l2 directly is something we'd want to start doing.)
>
I believe it should be nl2e.l2 += 1<<(PAGETABLE_ORDER+PAGE_SHIFT) ?
Although the code rarely touches the field (.l2) directly, so maybe use
the macros (get_intpte -> add -> from_intpte) for consistency? They
should produce the same code if the compiler is not too stupid.
> > +
> > +if ( locking )
> > +spin_lock(_pgdir_lock);
> > +if ( (l3e_get_flags(ol3e) & _PAGE_PRESENT) &&
> > + (l3e_get_flags(ol3e) & _PAGE_PSE) )
>
> There is a subtle difference between the original code, and the
> refactored code, and it depends on the memory barrier from
> spin_lock().
>
> Previously, it was re-read from memory after the lock, whereas now it
> is
> likely the stale version from before map_pgdir was locked.
>
> Either you can go back to the old version and use *pl3e, or
> alternatively use:
>
> if ( locking )
> spin_lock(_pgdir_lock);
> ol3e = ACCESS_ONCE(*pl3e);
> if ( ...
>
> to make it clear that a reread from memory is required.
>
Good point. Thanks.
> > +{
> > +l3e_write_atomic(pl3e, l3e_from_mfn(virt_to_mfn(l2t),
>
> This would probably generate better asm by using the maddr variants
> so
> we don't have a double shift.
>
Right. I can change that.
> > +__PAGE_HYPERVISOR));
> > +l2t = NULL;
> > +}
> > +if ( locking )
> > +spin_unlock(_pgdir_lock);
> > +if ( virt )
> > +{
> > +unsigned int flush_flags =
> > +FLUSH_TLB | FLUSH_ORDER(2 * PAGETABLE_ORDER);
> > +
> > +if ( (l3e_get_flags(ol3e) & _PAGE_GLOBAL) )
> > +flush_flags |= FLUSH_TLB_GLOBAL;
> > +flush_area(virt, flush_flags);
> > +}
> > +if ( l2t )
> > +free_xen_pagetable(l2t);
>
> This surely needs to NULL out l2t, just so the caller doesn't get any
> clever ideas and ends up with a use-after-free?
>
> ~Andrew
hmm... if we want to make the nullification visible to the caller we
might need to pass &. I wonder if it makes sense to simply move the
memory allocation of pl2e into shatter_l3e as well, so that the caller
cannot have any ideas.
Thanks,
Hongyan
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH v2 2/3] xen/banner: Drop the fig-to-oct.py script
The script is 664 rather than 775, so the banner conversion doesn't actually
work if $(PYTHON) is empty:
/bin/sh: tools/fig-to-oct.py: Permission denied
make[3]: *** [include/xen/compile.h] Error 126
make[3]: Leaving directory `/builds/xen-project/people/andyhhp/xen/xen'
Fixing this is easy, but using python here is wasteful. compile.h doesn't
need XEN_BANNER rendering in octal, and text is much more simple to handle.
Replace fig-to-oct.py with a sed oneliner.
While changing this logic, take the opportunity to optimise the banner
space (and time on the serial port) by dropping trailing whitespace, which is
84 characters for current staging.
Signed-off-by: Andrew Cooper
---
CC: George Dunlap
CC: Ian Jackson
CC: Jan Beulich
CC: Konrad Rzeszutek Wilk
CC: Stefano Stabellini
CC: Wei Liu
CC: Julien Grall
CC: Juergen Gross
v2:
* New
Spotted by Gitlab CI, caused by `which` not being present in some of the
CentOS containers. While this is more of a container bug than anything else,
it does highlight that the build ought to cope.
---
xen/Makefile| 5 -
xen/tools/fig-to-oct.py | 18 --
2 files changed, 4 insertions(+), 19 deletions(-)
delete mode 100644 xen/tools/fig-to-oct.py
diff --git a/xen/Makefile b/xen/Makefile
index 99701e3165..13ae1b6011 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -163,6 +163,9 @@ delete-unfresh-files:
@mv -f $@.tmp $@
# compile.h contains dynamic build info. Rebuilt on every 'make' invocation.
+#
+# For .banner sed-ary, strip trailing whitespace, escape backslashes, and wrap
+# each line in '"...\n" \' to become a valid C string
include/xen/compile.h: include/xen/compile.h.in .banner
@sed -e 's/@@date@@/$(XEN_BUILD_DATE)/g' \
-e 's/@@time@@/$(XEN_BUILD_TIME)/g' \
@@ -176,7 +179,7 @@ include/xen/compile.h: include/xen/compile.h.in .banner
-e 's!@@changeset@@!$(shell tools/scmversion $(XEN_ROOT) || echo
"unavailable")!g' \
< include/xen/compile.h.in > $@.new
@cat .banner
- @$(PYTHON) tools/fig-to-oct.py < .banner >> $@.new
+ @sed -e 's_[ ]*$$__' -e 's_\\__g' -e 's_\(.*\)_"\1\\n" \\_' <
.banner >> $@.new
@mv -f $@.new $@
include/asm-$(TARGET_ARCH)/asm-offsets.h: arch/$(TARGET_ARCH)/asm-offsets.s
diff --git a/xen/tools/fig-to-oct.py b/xen/tools/fig-to-oct.py
deleted file mode 100644
index db4fd32159..00
--- a/xen/tools/fig-to-oct.py
+++ /dev/null
@@ -1,18 +0,0 @@
-#!/usr/bin/env python
-import sys
-
-chars_per_line = 18
-chars_so_far = 0
-
-sys.stdout.write('"')
-
-for char in sys.stdin.read():
-
-sys.stdout.write("\\%03o" % ord(char))
-chars_so_far = chars_so_far + 1
-
-if chars_so_far == chars_per_line:
-chars_so_far = 0
-sys.stdout.write('" \\\n"')
-
-sys.stdout.write('"\n')
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH for-4.13 v2 0/3] xen: Build fixes related to Python3
Patch 3 is a fix for a problem reported on IRC. It is a very-nice-to-have considering our attempt to make Xen 4.13 Py3-clean. While testing patch 3, it became apparent that XSM/Flask isn't Py3-clean, and this is a blocker. It is addressed in patch 1. Patch 2 addresses a bug spotted by Gitlab while testing v1 of this series. It isn't strictly a Py3 bug, but is a build system robustness fix. Andrew Cooper (3): xen/flask: Drop the gen-policy.py script xen/banner: Drop the fig-to-oct.py script xen/build: Automatically locate a suitable python interpreter xen/Makefile | 10 +- xen/tools/fig-to-oct.py | 18 -- xen/xsm/flask/Makefile | 6 ++ xen/xsm/flask/flask-policy.S | 20 xen/xsm/flask/gen-policy.py | 23 --- 5 files changed, 31 insertions(+), 46 deletions(-) delete mode 100644 xen/tools/fig-to-oct.py create mode 100644 xen/xsm/flask/flask-policy.S delete mode 100644 xen/xsm/flask/gen-policy.py -- 2.11.0 ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH v2 3/3] xen/build: Automatically locate a suitable python interpreter
Needing to pass PYTHON=python3 into hypervisor builds is irritating and unnecessary. Locate a suitable interpreter automatically, defaulting to Py3 if it is available. Reported-by: Steven Haigh Signed-off-by: Andrew Cooper --- CC: George Dunlap CC: Ian Jackson CC: Jan Beulich CC: Konrad Rzeszutek Wilk CC: Stefano Stabellini CC: Wei Liu CC: Julien Grall CC: Steven Haigh CC: Juergen Gross v2: * Cope with `which` not being present in the system. * Only evaulate the shell command one, rather than once per $(PTHON) usage For 4.13. This is a very-nice-to-have WRT our Py3-clean intention. --- xen/Makefile | 5 + 1 file changed, 5 insertions(+) diff --git a/xen/Makefile b/xen/Makefile index 13ae1b6011..4f02aca3ac 100644 --- a/xen/Makefile +++ b/xen/Makefile @@ -13,6 +13,11 @@ export XEN_BUILD_TIME?= $(shell LC_ALL=C date +%T) export XEN_BUILD_HOST ?= $(shell hostname) export XEN_CONFIG_EXPERT ?= n +# Best effort attempt to find a python interpreter, defaulting to Python 3 if +# available. Fall back to just `python` if `which` is nowhere to be found. +PYTHON_INTERPRETER := $(word 1,$(shell which python3 python python2 2>/dev/null) python) +export PYTHON ?= $(PYTHON_INTERPRETER) + export BASEDIR := $(CURDIR) export XEN_ROOT := $(BASEDIR)/.. -- 2.11.0 ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH v2 1/3] xen/flask: Drop the gen-policy.py script
The script is Python 2 specific, and fails with string/binary issues with
Python 3:
Traceback (most recent call last):
File "gen-policy.py", line 14, in
for char in sys.stdin.read():
File "/usr/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8c in position 0:
invalid start byte
Fixing the script to be compatible isn't hard, but using python here is
wasteful. Drop the script entirely, and write a short flask-policy.S instead.
Signed-off-by: Andrew Cooper
---
CC: Daniel De Graaf
CC: Juergen Gross
v2:
* Fix tabs vs spaces issues
For 4.13. This is a blocker to our intent to by Py3-clean in this release.
Discovered entirely accidently when testing the final patch.
---
xen/xsm/flask/Makefile | 6 ++
xen/xsm/flask/flask-policy.S | 20
xen/xsm/flask/gen-policy.py | 23 ---
3 files changed, 22 insertions(+), 27 deletions(-)
create mode 100644 xen/xsm/flask/flask-policy.S
delete mode 100644 xen/xsm/flask/gen-policy.py
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index f5ffab1226..7c3f381287 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -27,7 +27,8 @@ $(FLASK_H_FILES): $(FLASK_H_DEPEND)
$(AV_H_FILES): $(AV_H_DEPEND)
$(CONFIG_SHELL) policy/mkaccess_vector.sh $(AWK) $(AV_H_DEPEND)
-obj-$(CONFIG_XSM_FLASK_POLICY) += policy.o
+obj-bin-$(CONFIG_XSM_FLASK_POLICY) += flask-policy.o
+flask-policy.o: policy.bin
FLASK_BUILD_DIR := $(CURDIR)
POLICY_SRC := $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
@@ -36,9 +37,6 @@ policy.bin: FORCE
$(MAKE) -f $(XEN_ROOT)/tools/flask/policy/Makefile.common -C
$(XEN_ROOT)/tools/flask/policy FLASK_BUILD_DIR=$(FLASK_BUILD_DIR)
cmp -s $(POLICY_SRC) $@ || cp $(POLICY_SRC) $@
-policy.c: policy.bin gen-policy.py
- $(PYTHON) gen-policy.py < $< > $@
-
.PHONY: clean
clean::
rm -f $(ALL_H_FILES) *.o $(DEPS_RM) policy.* $(POLICY_SRC)
diff --git a/xen/xsm/flask/flask-policy.S b/xen/xsm/flask/flask-policy.S
new file mode 100644
index 00..b63a14851d
--- /dev/null
+++ b/xen/xsm/flask/flask-policy.S
@@ -0,0 +1,20 @@
+.section .init.rodata, "a", @progbits
+
+/* const unsigned char xsm_flask_init_policy[] __initconst */
+.align 4
+.global xsm_flask_init_policy
+xsm_flask_init_policy:
+.incbin "policy.bin"
+.Lend:
+
+.type xsm_flask_init_policy, @object
+.size xsm_flask_init_policy, . - xsm_flask_init_policy
+
+/* const unsigned int __initconst xsm_flask_init_policy_size */
+.align 4
+.global xsm_flask_init_policy_size
+xsm_flask_init_policy_size:
+.long .Lend - xsm_flask_init_policy
+
+.type xsm_flask_init_policy_size, @object
+.size xsm_flask_init_policy_size, . - xsm_flask_init_policy_size
diff --git a/xen/xsm/flask/gen-policy.py b/xen/xsm/flask/gen-policy.py
deleted file mode 100644
index c7501e4614..00
--- a/xen/xsm/flask/gen-policy.py
+++ /dev/null
@@ -1,23 +0,0 @@
-#!/usr/bin/env python
-import sys
-
-policy_size = 0
-
-sys.stdout.write("""
-/* This file is autogenerated by gen_policy.py */
-#include
-#include
-
-const unsigned char xsm_flask_init_policy[] __initconst = {
-""")
-
-for char in sys.stdin.read():
-sys.stdout.write(" 0x%02x," % ord(char))
-policy_size = policy_size + 1
-if policy_size % 13 == 0:
-sys.stdout.write("\n")
-
-sys.stdout.write("""
-};
-const unsigned int __initconst xsm_flask_init_policy_size = %d;
-""" % policy_size)
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [xen-unstable test] 144619: tolerable FAIL
flight 144619 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/144619/ Failures :-/ but no regressions. Tests which are failing intermittently (not blocking): test-amd64-amd64-xl-rtds 16 guest-localmigrate fail pass in 144613 Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-rtds 18 guest-localmigrate/x10 fail in 144613 blocked in 144619 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 144613 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 144613 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 144613 test-armhf-armhf-libvirt 14 saverestore-support-checkfail like 144613 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 144613 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail like 144613 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 144613 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 144613 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 144613 test-amd64-i386-xl-pvshim12 guest-start fail never pass test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail never pass test-amd64-i386-libvirt-xsm 13 migrate-support-checkfail never pass test-amd64-i386-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-seattle 13 migrate-support-checkfail never pass test-arm64-arm64-xl-seattle 14 saverestore-support-checkfail never pass test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check fail never pass test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check fail never pass test-arm64-arm64-xl-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-xl 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit2 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-credit2 14 saverestore-support-checkfail never pass test-arm64-arm64-xl 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-credit1 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit1 14 saverestore-support-checkfail never pass test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail never pass test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-arndale 13 migrate-support-checkfail never pass test-armhf-armhf-xl-arndale 14 saverestore-support-checkfail never pass test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2 fail never pass test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail never pass test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail never pass test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-credit2 13 migrate-support-checkfail never pass test-armhf-armhf-xl-credit2 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass test-armhf-armhf-xl 13 migrate-support-checkfail never pass test-armhf-armhf-xl 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-credit1 13 migrate-support-checkfail never pass test-armhf-armhf-xl-credit1 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-rtds 13 migrate-support-checkfail never pass test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail never pass test-armhf-armhf-libvirt 13 migrate-support-checkfail never pass test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail never pass test-armhf-armhf-xl-vhd 12 migrate-support-checkfail never pass test-armhf-armhf-xl-vhd 13 saverestore-support-checkfail never pass test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail never pass version targeted for testing: xen ae25407faaaddf4abe44137ebf0e177a8c8f9858 baseline version: xen ae25407faaaddf4abe44137ebf0e177a8c8f9858 Last test of basis 144619 2019-12-07 09:25:36 Z0 days Testing same since (not found) 0 attempts jobs: build-amd64-xsm pass build-arm64-xsm pass build-i386-xsm
[Xen-devel] [PATCH 2/2] xen/build: Automatically locate a suitable python interpreter
Needing to pass PYTHON=python3 into hypervisor builds is irritating and unnecessary. Locate a suitable interpreter automatically, defaulting to Py3 if it is available. Reported-by: Steven Haigh Signed-off-by: Andrew Cooper --- CC: George Dunlap CC: Ian Jackson CC: Jan Beulich CC: Konrad Rzeszutek Wilk CC: Stefano Stabellini CC: Wei Liu CC: Julien Grall CC: Steven Haigh CC: Juergen Gross For 4.13. This is a very-nice-to-have WRT our Py3-clean intention. --- xen/Makefile | 2 ++ 1 file changed, 2 insertions(+) diff --git a/xen/Makefile b/xen/Makefile index 99701e3165..b936d1812b 100644 --- a/xen/Makefile +++ b/xen/Makefile @@ -12,6 +12,8 @@ export XEN_BUILD_DATE ?= $(shell LC_ALL=C date) export XEN_BUILD_TIME ?= $(shell LC_ALL=C date +%T) export XEN_BUILD_HOST ?= $(shell hostname) export XEN_CONFIG_EXPERT ?= n +# Best effort attempt to find a python interpreter +export PYTHON ?= $(word 1,$(shell which python3 python python2)) export BASEDIR := $(CURDIR) export XEN_ROOT := $(BASEDIR)/.. -- 2.11.0 ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH 1/2] xen/flask: Fix Python 3 problems with gen-policy.py
The script is Python 2 specific, and fails with string/binary issues with
Python 3:
Traceback (most recent call last):
File "gen-policy.py", line 14, in
for char in sys.stdin.read():
File "/usr/lib/python3.5/codecs.py", line 321, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8c in position 0:
invalid start byte
Fixing the script to be compatible isn't hard, but using python here is
wasteful. Drop the script entirely, and write a short flask-policy.S instead.
Signed-off-by: Andrew Cooper
---
CC: Daniel De Graaf
CC: Juergen Gross
For 4.13. This is a blocker to our intent to by Py3-clean in this release
Discovered entirely accidently when testing the following patch.
---
xen/xsm/flask/Makefile | 6 ++
xen/xsm/flask/flask-policy.S | 20
xen/xsm/flask/gen-policy.py | 23 ---
3 files changed, 22 insertions(+), 27 deletions(-)
create mode 100644 xen/xsm/flask/flask-policy.S
delete mode 100644 xen/xsm/flask/gen-policy.py
diff --git a/xen/xsm/flask/Makefile b/xen/xsm/flask/Makefile
index f5ffab1226..7c3f381287 100644
--- a/xen/xsm/flask/Makefile
+++ b/xen/xsm/flask/Makefile
@@ -27,7 +27,8 @@ $(FLASK_H_FILES): $(FLASK_H_DEPEND)
$(AV_H_FILES): $(AV_H_DEPEND)
$(CONFIG_SHELL) policy/mkaccess_vector.sh $(AWK) $(AV_H_DEPEND)
-obj-$(CONFIG_XSM_FLASK_POLICY) += policy.o
+obj-bin-$(CONFIG_XSM_FLASK_POLICY) += flask-policy.o
+flask-policy.o: policy.bin
FLASK_BUILD_DIR := $(CURDIR)
POLICY_SRC := $(FLASK_BUILD_DIR)/xenpolicy-$(XEN_FULLVERSION)
@@ -36,9 +37,6 @@ policy.bin: FORCE
$(MAKE) -f $(XEN_ROOT)/tools/flask/policy/Makefile.common -C
$(XEN_ROOT)/tools/flask/policy FLASK_BUILD_DIR=$(FLASK_BUILD_DIR)
cmp -s $(POLICY_SRC) $@ || cp $(POLICY_SRC) $@
-policy.c: policy.bin gen-policy.py
- $(PYTHON) gen-policy.py < $< > $@
-
.PHONY: clean
clean::
rm -f $(ALL_H_FILES) *.o $(DEPS_RM) policy.* $(POLICY_SRC)
diff --git a/xen/xsm/flask/flask-policy.S b/xen/xsm/flask/flask-policy.S
new file mode 100644
index 00..d78ce77fd6
--- /dev/null
+++ b/xen/xsm/flask/flask-policy.S
@@ -0,0 +1,20 @@
+.section .init.rodata, "a", @progbits
+
+/* const unsigned char xsm_flask_init_policy[] __initconst */
+ .align 4
+.global xsm_flask_init_policy
+xsm_flask_init_policy:
+.incbin "policy.bin"
+.Lend:
+
+.type xsm_flask_init_policy, @object
+.size xsm_flask_init_policy, . - xsm_flask_init_policy
+
+/* const unsigned int __initconst xsm_flask_init_policy_size */
+ .align 4
+.global xsm_flask_init_policy_size
+xsm_flask_init_policy_size:
+.long .Lend - xsm_flask_init_policy
+
+.type xsm_flask_init_policy_size, @object
+.size xsm_flask_init_policy_size, . - xsm_flask_init_policy_size
diff --git a/xen/xsm/flask/gen-policy.py b/xen/xsm/flask/gen-policy.py
deleted file mode 100644
index c7501e4614..00
--- a/xen/xsm/flask/gen-policy.py
+++ /dev/null
@@ -1,23 +0,0 @@
-#!/usr/bin/env python
-import sys
-
-policy_size = 0
-
-sys.stdout.write("""
-/* This file is autogenerated by gen_policy.py */
-#include
-#include
-
-const unsigned char xsm_flask_init_policy[] __initconst = {
-""")
-
-for char in sys.stdin.read():
-sys.stdout.write(" 0x%02x," % ord(char))
-policy_size = policy_size + 1
-if policy_size % 13 == 0:
-sys.stdout.write("\n")
-
-sys.stdout.write("""
-};
-const unsigned int __initconst xsm_flask_init_policy_size = %d;
-""" % policy_size)
--
2.11.0
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [PATCH for-4.13 0/2] xen: Build fixes related to Python3
Patch 2 is a fix for a problem reported on IRC. It is a very-nice-to-have considering our attempt to make Xen 4.13 Py3-clean. While testing patch 2, it became apparent that XSM/Flask isn't Py3-clean, and this is a blocker. It is addressed in patch 1. Andrew Cooper (2): xen/flask: Fix Python 3 problems with gen-policy.py xen/build: Automatically locate a suitable python interpreter xen/Makefile | 2 ++ xen/xsm/flask/Makefile | 6 ++ xen/xsm/flask/flask-policy.S | 20 xen/xsm/flask/gen-policy.py | 23 --- 4 files changed, 24 insertions(+), 27 deletions(-) create mode 100644 xen/xsm/flask/flask-policy.S delete mode 100644 xen/xsm/flask/gen-policy.py -- 2.11.0 ___ Xen-devel mailing list Xen-devel@lists.xenproject.org https://lists.xenproject.org/mailman/listinfo/xen-devel
[Xen-devel] [xen-unstable test] 144613: tolerable FAIL - PUSHED
flight 144613 xen-unstable real [real] http://logs.test-lab.xenproject.org/osstest/logs/144613/ Failures :-/ but no regressions. Tests which did not succeed, but are not blocking: test-amd64-amd64-xl-rtds 18 guest-localmigrate/x10 fail blocked in 144588 test-amd64-amd64-xl-qemut-win7-amd64 17 guest-stopfail like 144588 test-amd64-i386-xl-qemuu-win7-amd64 17 guest-stop fail like 144588 test-amd64-amd64-xl-qemuu-win7-amd64 17 guest-stopfail like 144588 test-armhf-armhf-libvirt 14 saverestore-support-checkfail like 144588 test-amd64-i386-xl-qemut-win7-amd64 17 guest-stop fail like 144588 test-armhf-armhf-libvirt-raw 13 saverestore-support-checkfail like 144588 test-amd64-amd64-xl-qemuu-ws16-amd64 17 guest-stopfail like 144588 test-amd64-amd64-xl-qemut-ws16-amd64 17 guest-stopfail like 144588 test-amd64-i386-xl-qemuu-ws16-amd64 17 guest-stop fail like 144588 test-amd64-i386-xl-pvshim12 guest-start fail never pass test-amd64-amd64-libvirt-xsm 13 migrate-support-checkfail never pass test-amd64-i386-libvirt-xsm 13 migrate-support-checkfail never pass test-amd64-i386-libvirt 13 migrate-support-checkfail never pass test-arm64-arm64-xl-seattle 13 migrate-support-checkfail never pass test-arm64-arm64-xl-seattle 14 saverestore-support-checkfail never pass test-amd64-amd64-libvirt 13 migrate-support-checkfail never pass test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check fail never pass test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 11 migrate-support-check fail never pass test-arm64-arm64-xl-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-xl 13 migrate-support-checkfail never pass test-arm64-arm64-xl-xsm 14 saverestore-support-checkfail never pass test-arm64-arm64-xl 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-credit2 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit1 13 migrate-support-checkfail never pass test-arm64-arm64-xl-credit1 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-credit2 14 saverestore-support-checkfail never pass test-arm64-arm64-libvirt-xsm 13 migrate-support-checkfail never pass test-arm64-arm64-libvirt-xsm 14 saverestore-support-checkfail never pass test-arm64-arm64-xl-thunderx 13 migrate-support-checkfail never pass test-arm64-arm64-xl-thunderx 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-arndale 13 migrate-support-checkfail never pass test-armhf-armhf-xl-arndale 14 saverestore-support-checkfail never pass test-amd64-amd64-qemuu-nested-amd 17 debian-hvm-install/l1/l2 fail never pass test-amd64-amd64-libvirt-vhd 12 migrate-support-checkfail never pass test-armhf-armhf-xl-multivcpu 13 migrate-support-checkfail never pass test-armhf-armhf-xl-multivcpu 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-credit2 13 migrate-support-checkfail never pass test-armhf-armhf-xl-credit2 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-cubietruck 13 migrate-support-checkfail never pass test-armhf-armhf-xl-cubietruck 14 saverestore-support-checkfail never pass test-armhf-armhf-xl 13 migrate-support-checkfail never pass test-armhf-armhf-xl 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-credit1 13 migrate-support-checkfail never pass test-armhf-armhf-xl-credit1 14 saverestore-support-checkfail never pass test-armhf-armhf-xl-rtds 13 migrate-support-checkfail never pass test-armhf-armhf-xl-rtds 14 saverestore-support-checkfail never pass test-armhf-armhf-libvirt 13 migrate-support-checkfail never pass test-armhf-armhf-libvirt-raw 12 migrate-support-checkfail never pass test-armhf-armhf-xl-vhd 12 migrate-support-checkfail never pass test-armhf-armhf-xl-vhd 13 saverestore-support-checkfail never pass test-amd64-i386-xl-qemut-ws16-amd64 17 guest-stop fail never pass version targeted for testing: xen ae25407faaaddf4abe44137ebf0e177a8c8f9858 baseline version: xen 8359dde71826bfbcf04412bda001903f809571c9 Last test of basis 144588 2019-12-06 13:36:41 Z0 days Testing same since 144613 2019-12-07 00:07:17 Z0 days1 attempts People who touched revisions under test: Andrew Cooper jobs: build-amd64-xsm pass build-arm64-xsm pass build-i386-xsm
