Re: [PATCH v6 7/9] docs: Change Makefile and sphinx configuration for doxygen

2021-07-02 Thread Luca Fancellu
Hi Stefano,


> On 1 Jul 2021, at 18:43, Stefano Stabellini  wrote:
> 
> On Thu, 1 Jul 2021, Luca Fancellu wrote:
>>> On 24 Jun 2021, at 00:33, Stefano Stabellini  wrote:
>>> 
>>> On Mon, 10 May 2021, Luca Fancellu wrote:
 Modify docs/Makefile to call doxygen and generate sphinx
 html documentation given the doxygen XML output.
 
 Modify docs/conf.py sphinx configuration file to setup
 the breathe extension that works as bridge between
 sphinx and doxygen.
 
 Add some files to the .gitignore to ignore some
 generated files for doxygen.
 
 Signed-off-by: Luca Fancellu 
 ---
 .gitignore|  6 ++
 docs/Makefile | 42 +++---
 docs/conf.py  | 48 +---
 3 files changed, 90 insertions(+), 6 deletions(-)
 
 diff --git a/.gitignore b/.gitignore
 index 1c2fa1530b..d271e0ce6a 100644
 --- a/.gitignore
 +++ b/.gitignore
 @@ -58,6 +58,12 @@ docs/man7/
 docs/man8/
 docs/pdf/
 docs/txt/
 +docs/doxygen-output
 +docs/sphinx
 +docs/xen.doxyfile
 +docs/xen.doxyfile.tmp
 +docs/xen-doxygen/doxygen_include.h
 +docs/xen-doxygen/doxygen_include.h.tmp
 extras/mini-os*
 install/*
 stubdom/*-minios-config.mk
 diff --git a/docs/Makefile b/docs/Makefile
 index 8de1efb6f5..2f784c36ce 100644
 --- a/docs/Makefile
 +++ b/docs/Makefile
 @@ -17,6 +17,18 @@ TXTSRC-y := $(sort $(shell find misc -name '*.txt' 
 -print))
 
 PANDOCSRC-y := $(sort $(shell find designs/ features/ misc/ process/ 
 specs/ \( -name '*.pandoc' -o -name '*.md' \) -print))
 
 +# Directory in which the doxygen documentation is created
 +# This must be kept in sync with breathe_projects value in conf.py
 +DOXYGEN_OUTPUT = doxygen-output
 +
 +# Doxygen input headers from xen-doxygen/doxy_input.list file
 +DOXY_LIST_SOURCES != cat "xen-doxygen/doxy_input.list"
 +DOXY_LIST_SOURCES := $(realpath $(addprefix 
 $(XEN_ROOT)/,$(DOXY_LIST_SOURCES)))
>> 
>> Hi Stefano,
>> 
>>> 
>>> I cannot find exactly who is populating doxy_input.list. I can see it is
>>> empty in patch #6. Does it get populated during the build?
>> 
>> doxy_input.list is the only file that should be modified by the developer 
>> when he/she wants to add documentation
>> for a new file to be parsed by Doxygen, in my patch about documenting 
>> grant_tables.h you can see I add
>> there the path “xen/include/public/grant_table.h"
> 
> OK, thanks. I missed that addition.
> 
> 
>>> 
 +DOXY_DEPS := xen.doxyfile \
 +   xen-doxygen/mainpage.md \
 +   xen-doxygen/doxygen_include.h
 +
 # Documentation targets
 $(foreach i,$(MAN_SECTIONS), \
  $(eval DOC_MAN$(i) := $(patsubst man/%.$(i),man$(i)/%.$(i), \
 @@ -46,8 +58,28 @@ all: build
 build: html txt pdf man-pages figs
 
 .PHONY: sphinx-html
 -sphinx-html:
 -  sphinx-build -b html . sphinx/html
 +sphinx-html: $(DOXY_DEPS) $(DOXY_LIST_SOURCES)
 +ifneq ($(SPHINXBUILD),no)
>>> 
>>> This check on SPHINXBUILD is new, it wasn't there before. Why do we need
>>> it now? We are not really changing anything in regards to Sphinx, just
>>> adding Doxygen support. Or was it a mistake that it was missing even
>>> before this patch?
>> 
>> Yes this is new, I saw that we didn’t look if sphinx was installed in the 
>> system, so now we did
> 
> In that case, I think anything related to SPHINXBUILD and whether sphinx
> is installed or not, should be a separate patch at the beginning of the
> series. It could be committed independently before the rest of the
> series. When we get to this patch, SPHINXBUILD should be already there.

I’ve introduced SPHINXBUILD in this patch: [PATCH v6 5/9] docs: add checks to 
configure for sphinx and doxygen,
In your commend do you mean that you would like it to be outside this serie and 
this serie to be based on top of that one?


> 
> 
 +  $(DOXYGEN) xen.doxyfile
 +  XEN_ROOT=$(realpath $(XEN_ROOT)) $(SPHINXBUILD) -b html . sphinx/html
 +else
 +  @echo "Sphinx is not installed; skipping sphinx-html documentation."
 +endif
 +
 +xen.doxyfile: xen.doxyfile.in xen-doxygen/doxy_input.list
 +  @echo "Generating $@"
 +  @sed -e "s,@XEN_BASE@,$(realpath $(XEN_ROOT)),g" $< \
 +  | sed -e "s,@DOXY_OUT@,$(DOXYGEN_OUTPUT),g" > $@.tmp
 +  @$(foreach inc,\
 +  $(DOXY_LIST_SOURCES),\
 +  echo "INPUT += \"$(inc)\"" >> $@.tmp; \
 +  )
 +  mv $@.tmp $@
 +
 +xen-doxygen/doxygen_include.h: xen-doxygen/doxygen_include.h.in
 +  @echo "Generating $@"
 +  @sed -e "s,@XEN_BASE@,$(realpath $(XEN_ROOT)),g" $< > $@.tmp
 +  @mv $@.tmp $@
>>> 
>>> Is the absolute path required? If not, we can probably get rid of this
>>> generation step and simply have the relative path in
>>> xen-doxygen/doxygen_include.h

Re: [PATCH] xen/arm: smmuv1: Fixed stream matching register allocation

2021-07-02 Thread Rahul Singh
Hi Julien,

> On 30 Jun 2021, at 7:00 pm, Julien Grall  wrote:
> 
> 
> 
> On 30/06/2021 18:49, Rahul Singh wrote:
>> Hi Julien,
> 
> Hi,
> 
>>> On 30 Jun 2021, at 10:09 am, Julien Grall  wrote:
>>> 
>>> Hi Rahul,
>>> 
>>> On 25/06/2021 17:37, Rahul Singh wrote:
 SMR allocation should be based on the number of supported stream
 matching register for each SMMU device.
 Issue introduced by commit 5e08586afbb90b2e2d56c175c07db77a4afa873c
 when backported the patches from Linux to XEN to fix the stream match
 conflict issue when two devices have the same stream-id.
 Acked-by: Stefano Stabellini 
 Tested-by: Stefano Stabellini 
 Signed-off-by: Rahul Singh 
 ---
  xen/drivers/passthrough/arm/smmu.c | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)
 diff --git a/xen/drivers/passthrough/arm/smmu.c 
 b/xen/drivers/passthrough/arm/smmu.c
 index d9a3a0cbf6..da2cd457d7 100644
 --- a/xen/drivers/passthrough/arm/smmu.c
 +++ b/xen/drivers/passthrough/arm/smmu.c
 @@ -149,6 +149,7 @@ typedef enum irqreturn irqreturn_t;
  #define kzalloc(size, flags)  _xzalloc(size, sizeof(void *))
  #define devm_kzalloc(dev, size, flags)_xzalloc(size, sizeof(void *))
  #define kmalloc_array(size, n, flags) _xmalloc_array(size, 
 sizeof(void *), n)
 +#define kzalloc_array(size, n, flags) _xzalloc_array(size, 
 sizeof(void *), n)
static void __iomem *devm_ioremap_resource(struct device *dev,
   struct resource *res)
 @@ -2221,7 +,7 @@ static int arm_smmu_device_cfg_probe(struct 
 arm_smmu_device *smmu)
smmu->smr_mask_mask = smr >> SMR_MASK_SHIFT;
/* Zero-initialised to mark as invalid */
 -  smmu->smrs = devm_kzalloc(smmu->dev, sizeof(*smmu->smrs), 
 GFP_KERNEL);
 +  smmu->smrs = kzalloc_array(sizeof(*smmu->smrs), size, 
 GFP_KERNEL);
>>> 
>>> I noticed this is already in... However, I am a bit puzzled into why this 
>>> was switched devm_kzalloc() to kzalloc_array(). This doesn't matter for Xen 
>>> as they are just wrappers to x*alloc() but a mention in the commit message 
>>> would have been useful.
>> Yes we can use the devm_kzalloc(..) but then we have to pass 
>> (sizeof(*smmu->smrs) * size ) as size argument to devm_kzalloc(..)
>> I thought for better code readability I will use kzalloc_array() as the 
>> function name suggests we are allocating memory for an array.
> 
> My point is devm_k*alloc() and k*alloc() are quite different on the paper. 
> One will allocate memory for a given device while the other is unknown memory.
> 
> It would have been better to call the function devm_kzalloc_array() to keep 
> to keep the code coherent. Can you please send a patch to make the switch?

Ok. I will modify the code as per your request as below . I will use 
devm_kcalloc(..) as this will be more coherent.

diff --git a/xen/drivers/passthrough/arm/smmu.c 
b/xen/drivers/passthrough/arm/smmu.c
index da2cd457d7..658c40433c 100644
--- a/xen/drivers/passthrough/arm/smmu.c
+++ b/xen/drivers/passthrough/arm/smmu.c
@@ -149,7 +149,8 @@ typedef enum irqreturn irqreturn_t;
 #define kzalloc(size, flags)   _xzalloc(size, sizeof(void *))
 #define devm_kzalloc(dev, size, flags) _xzalloc(size, sizeof(void *))
 #define kmalloc_array(size, n, flags)  _xmalloc_array(size, sizeof(void *), n)
-#define kzalloc_array(size, n, flags)  _xzalloc_array(size, sizeof(void *), n)
+#define devm_kcalloc(dev, n, size, flags)  \
+   _xzalloc_array(size, sizeof(void *), n)
 
 static void __iomem *devm_ioremap_resource(struct device *dev,
   struct resource *res)
@@ -,7 +2223,8 @@ static int arm_smmu_device_cfg_probe(struct 
arm_smmu_device *smmu)
smmu->smr_mask_mask = smr >> SMR_MASK_SHIFT;
 
/* Zero-initialised to mark as invalid */
-   smmu->smrs = kzalloc_array(sizeof(*smmu->smrs), size, 
GFP_KERNEL);
+   smmu->smrs = devm_kcalloc(smmu->dev, size, sizeof(*smmu->smrs),
+   GFP_KERNEL);
if (!smmu->smrs)
return -ENOMEM;

Regards,
Rahul

> 
> Cheers,
> 
> -- 
> Julien Grall




[libvirt test] 163223: regressions - FAIL

2021-07-02 Thread osstest service owner
flight 163223 libvirt real [real]
http://logs.test-lab.xenproject.org/osstest/logs/163223/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 build-amd64-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-armhf-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-arm64-libvirt   6 libvirt-buildfail REGR. vs. 151777
 build-i386-libvirt6 libvirt-buildfail REGR. vs. 151777

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-amd64-libvirt-vhd  1 build-check(1)   blocked  n/a
 test-amd64-amd64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt   1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-pair  1 build-check(1)   blocked  n/a
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 1 build-check(1) blocked n/a
 test-amd64-i386-libvirt-xsm   1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-qcow2  1 build-check(1)   blocked  n/a
 test-arm64-arm64-libvirt-xsm  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt  1 build-check(1)   blocked  n/a
 test-armhf-armhf-libvirt-raw  1 build-check(1)   blocked  n/a

version targeted for testing:
 libvirt  dfa1e9b3eb438866c122e4b8d0b4f99679f47cd2
baseline version:
 libvirt  2c846fa6bcc11929c9fb857a22430fb9945654ad

Last test of basis   151777  2020-07-10 04:19:19 Z  357 days
Failing since151818  2020-07-11 04:18:52 Z  356 days  348 attempts
Testing same since   163223  2021-07-02 04:20:05 Z0 days1 attempts


People who touched revisions under test:
Adolfo Jayme Barrientos 
  Aleksandr Alekseev 
  Aleksei Zakharov 
  Andika Triwidada 
  Andrea Bolognani 
  Balázs Meskó 
  Barrett Schonefeld 
  Bastian Germann 
  Bastien Orivel 
  BiaoXiang Ye 
  Bihong Yu 
  Binfeng Wu 
  Bjoern Walk 
  Boris Fiuczynski 
  Brian Turek 
  Bruno Haible 
  Chris Mayo 
  Christian Ehrhardt 
  Christian Schoenebeck 
  Cole Robinson 
  Collin Walling 
  Cornelia Huck 
  Cédric Bosdonnat 
  Côme Borsoi 
  Daniel Henrique Barboza 
  Daniel Letai 
  Daniel P. Berrange 
  Daniel P. Berrangé 
  Dmytro Linkin 
  Eiichi Tsukata 
  Eric Farman 
  Erik Skultety 
  Fabian Affolter 
  Fabian Freyer 
  Fabiano Fidêncio 
  Fangge Jin 
  Farhan Ali 
  Fedora Weblate Translation 
  gongwei 
  Guoyi Tu
  Göran Uddeborg 
  Halil Pasic 
  Han Han 
  Hao Wang 
  Hela Basa 
  Helmut Grohne 
  Ian Wienand 
  Jakob Meng 
  Jamie Strandboge 
  Jamie Strandboge 
  Jan Kuparinen 
  Jean-Baptiste Holcroft 
  Jianan Gao 
  Jim Fehlig 
  Jin Yan 
  Jiri Denemark 
  John Ferlan 
  Jonathan Watt 
  Jonathon Jongsma 
  Julio Faracco 
  Ján Tomko 
  Kashyap Chamarthy 
  Kevin Locke 
  Kristina Hanicova 
  Laine Stump 
  Laszlo Ersek 
  Lee Yarwood 
  Liao Pingfang 
  Lin Ma 
  Lin Ma 
  Lin Ma 
  Liu Yiding 
  Luke Yue 
  Luyao Zhong 
  Marc Hartmayer 
  Marc-André Lureau 
  Marek Marczykowski-Górecki 
  Markus Schade 
  Martin Kletzander 
  Masayoshi Mizuma 
  Matt Coleman 
  Matt Coleman 
  Mauro Matteo Cascella 
  Meina Li 
  Michal Privoznik 
  Michał Smyk 
  Milo Casagrande 
  Moshe Levi 
  Muha Aliss 
  Nathan 
  Neal Gompa 
  Nick Shyrokovskiy 
  Nickys Music Group 
  Nico Pache 
  Nikolay Shirokovskiy 
  Olaf Hering 
  Olesya Gerasimenko 
  Orion Poplawski 
  Pany 
  Patrick Magauran 
  Paulo de Rezende Pinatti 
  Pavel Hrdina 
  Peng Liang 
  Peter Krempa 
  Pino Toscano 
  Pino Toscano 
  Piotr Drąg 
  Prathamesh Chavan 
  Ricky Tigg 
  Roman Bogorodskiy 
  Roman Bolshakov 
  Ryan Gahagan 
  Ryan Schmidt 
  Sam Hartman 
  Scott Shambarger 
  Sebastian Mitterle 
  SeongHyun Jo 
  Shalini Chellathurai Saroja 
  Shaojun Yang 
  Shi Lei 
  simmon 
  Simon Chopin 
  Simon Gaiser 
  Stefan Bader 
  Stefan Berger 
  Stefan Berger 
  Stefan Hajnoczi 
  Szymon Scholz 
  Thomas Huth 
  Tim Wiederhake 
  Tomáš Golembiovský 
  Tomáš Janoušek 
  Tuguoyi 
  Ville Skyttä 
  Wang Xin 
  WangJian 
  Weblate 
  Wei Liu 
  Wei Liu 
  William Douglas 
  Yalei Li <274268...@qq.com>
  Yalei Li 
  Yang Hang 
  Yanqiu Zhang 
  Yaroslav Kargin 
  Yi Li 
  Yi Wang 
  Yuri Chornoivan 
  Zbigniew Jędrzejewski-Szmek 
  Zheng Chuan 
  zhenwei pi 
  Zhenyu Zheng 

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pa

Re: [PATCH v6 9/9] docs/doxygen: doxygen documentation for grant_table.h

2021-07-02 Thread Luca Fancellu



> On 1 Jul 2021, at 18:44, Stefano Stabellini  wrote:
> 
> On Thu, 1 Jul 2021, Luca Fancellu wrote:
>> Hi Stefano,
>> 
>>> On 24 Jun 2021, at 00:34, Stefano Stabellini  wrote:
>>> 
>>> On Mon, 10 May 2021, Luca Fancellu wrote:
 Modification to include/public/grant_table.h:
 
 1) Add doxygen tags to:
 - Create Grant tables section
 - include variables in the generated documentation
 - Used @keepindent/@endkeepindent to enclose comment
  section that are indented using spaces, to keep
  the indentation.
 2) Add .rst file for grant table for Arm64
>>> 
>>> Why only arm64?
>> 
>> This is a mistake, it should be just “Add .rst file for grant table"
>> 
>>> 
>>> 
 Signed-off-by: Luca Fancellu 
 ---
 v6 changes:
 - Fix misaligned comment
 - Moved comments to make them display in the docs
 - Included more documentation in the docs
 (see output here: 
 https://luca.fancellu.gitlab.io/xen-docs/hypercall-interfaces/common/grant_tables.html)
>>> 
>>> It looks much much better. All the info we care about seems to be there.
>>> The only things that I noticed missing and might be good to keep is the
>>> small comment about HYPERVISOR_grant_table_op:
>>> 
>>> /* ` enum neg_errnoval
>>> * ` HYPERVISOR_grant_table_op(enum grant_table_op cmd,
>>> * `   void *args,
>>> * `   unsigned int count)
>>> * `
>>> *
>>> * @args points to an array of a per-command data structure. The array
>>> * has @count members
>> 
>> Where do you want me to put this comment in the html page? In the end of the 
>> description in the top of the page?
> 
> Yeah, that looks like a good place

Great, for a preview, have a look on this: 
https://luca.fancellu.gitlab.io/xen-docs/hypercall-interfaces/common/grant_tables.html




[ovmf test] 163221: regressions - FAIL

2021-07-02 Thread osstest service owner
flight 163221 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/163221/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 162359
 test-amd64-amd64-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 
162359

version targeted for testing:
 ovmf fea7901dba72eeac526f3ef12a4ad4c539622373
baseline version:
 ovmf c410ad4da4b7785170d3d42a3ba190c2caac6feb

Last test of basis   162359  2021-06-04 03:40:08 Z   28 days
Failing since162368  2021-06-04 15:42:59 Z   27 days   73 attempts
Testing same since   163216  2021-07-01 22:42:29 Z0 days3 attempts


People who touched revisions under test:
  Abner Chang 
  Agrawal, Sachin 
  Alexandru Elisei 
  Anthony PERARD 
  Ard Biesheuvel 
  Daniel Schaefer 
  Daoxiang Li 
  Dov Murik 
  DunTan 
  gaoliming 
  Guo Dong 
  Hao A Wu 
  Jian J Wang 
  Kaaira Gupta 
  Ken Lautner 
  Kenneth Lautner 
  Kun Qin 
  Laszlo Ersek 
  Leif Lindholm 
  Liming Gao 
  Loo Tung Lun 
  Loo, Tung Lun 
  Manickavasakam Karpagavinayagam 
  Maurice Ma 
  Ni, Ray 
  Patrick Rudolph 
  Pierre Gondois 
  Ray Ni 
  Rebecca Cran 
  Rebecca Cran 
  Sachin Agrawal 
  Sami Mujawar 
  Scottie Kuo 
  Sean Brogan 
  Sean Brogan 
  Sumana Venur 
  Sunil V L 
  xueshengfeng 
  Zhiguang Liu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3224 lines long.)



Re: [PATCH V7 01/18] perf/core: Use static_call to optimize perf_guest_info_callbacks

2021-07-02 Thread Peter Zijlstra
On Tue, Jun 22, 2021 at 05:42:49PM +0800, Zhu Lingshan wrote:
> diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
> index 8f71dd72ef95..c71af4cfba9b 100644
> --- a/arch/x86/events/core.c
> +++ b/arch/x86/events/core.c
> @@ -90,6 +90,27 @@ DEFINE_STATIC_CALL_NULL(x86_pmu_pebs_aliases, 
> *x86_pmu.pebs_aliases);
>   */
>  DEFINE_STATIC_CALL_RET0(x86_pmu_guest_get_msrs, *x86_pmu.guest_get_msrs);
>  
> +DEFINE_STATIC_CALL_RET0(x86_guest_state, *(perf_guest_cbs->state));
> +DEFINE_STATIC_CALL_RET0(x86_guest_get_ip, *(perf_guest_cbs->get_ip));
> +DEFINE_STATIC_CALL_RET0(x86_guest_handle_intel_pt_intr, 
> *(perf_guest_cbs->handle_intel_pt_intr));
> +
> +void arch_perf_update_guest_cbs(void)
> +{
> + static_call_update(x86_guest_state, (void *)&__static_call_return0);
> + static_call_update(x86_guest_get_ip, (void *)&__static_call_return0);
> + static_call_update(x86_guest_handle_intel_pt_intr, (void 
> *)&__static_call_return0);
> +
> + if (perf_guest_cbs && perf_guest_cbs->state)
> + static_call_update(x86_guest_state, perf_guest_cbs->state);
> +
> + if (perf_guest_cbs && perf_guest_cbs->get_ip)
> + static_call_update(x86_guest_get_ip, perf_guest_cbs->get_ip);
> +
> + if (perf_guest_cbs && perf_guest_cbs->handle_intel_pt_intr)
> + static_call_update(x86_guest_handle_intel_pt_intr,
> +perf_guest_cbs->handle_intel_pt_intr);
> +}

Coding style wants { } on that last if().



[xen-unstable test] 163219: tolerable FAIL

2021-07-02 Thread osstest service owner
flight 163219 xen-unstable real [real]
http://logs.test-lab.xenproject.org/osstest/logs/163219/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-armhf-armhf-libvirt-raw  8 xen-boot fail in 163206 pass in 163219
 test-amd64-i386-libvirt   8 xen-boot   fail pass in 163206

Tests which did not succeed, but are not blocking:
 test-amd64-i386-libvirt 15 migrate-support-check fail in 163206 never pass
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 163190
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 163206
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 163206
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 163206
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 163206
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 163206
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 163206
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 163206
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 163206
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 163206
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 163206
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass

version targeted for testing:
 xen  f95b7b37cfc6d4613721df9357090d14712013c0
baseline version:
 xen  f95b7b37cfc6d4613721df9357090d14712013c0

Last test of basis   163219  2021-07-02 01:52:38 Z0 days
Testing same since  (not found) 0 attempts

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm  

Re: [PATCH v15 12/12] of: Add plumbing for restricted DMA pool

2021-07-02 Thread Robin Murphy

On 2021-07-02 04:08, Guenter Roeck wrote:

Hi,

On Thu, Jun 24, 2021 at 11:55:26PM +0800, Claire Chang wrote:

If a device is not behind an IOMMU, we look up the device node and set
up the restricted DMA when the restricted-dma-pool is presented.

Signed-off-by: Claire Chang 
Tested-by: Stefano Stabellini 
Tested-by: Will Deacon 


With this patch in place, all sparc and sparc64 qemu emulations
fail to boot. Symptom is that the root file system is not found.
Reverting this patch fixes the problem. Bisect log is attached.


Ah, OF_ADDRESS depends on !SPARC, so of_dma_configure_id() is presumably 
returning an unexpected -ENODEV from the of_dma_set_restricted_buffer() 
stub. That should probably be returning 0 instead, since either way it's 
not an error condition for it to simply do nothing.


Robin.



Guenter

---
# bad: [fb0ca446157a86b75502c1636b0d81e642fe6bf1] Add linux-next specific files 
for 20210701
# good: [62fb9874f5da54fdb243003b386128037319b219] Linux 5.13
git bisect start 'HEAD' 'v5.13'
# bad: [f63c4fda987a19b1194cc45cb72fd5bf968d9d90] Merge remote-tracking branch 
'rdma/for-next'
git bisect bad f63c4fda987a19b1194cc45cb72fd5bf968d9d90
# good: [46bb5dd1d2a63e906e374e97dfd4a5e33934b1c4] Merge remote-tracking branch 
'ipsec/master'
git bisect good 46bb5dd1d2a63e906e374e97dfd4a5e33934b1c4
# good: [43ba6969cfb8185353a7a6fc79070f13b9e3d6d3] Merge remote-tracking branch 
'clk/clk-next'
git bisect good 43ba6969cfb8185353a7a6fc79070f13b9e3d6d3
# good: [1ca5eddcf8dca1d6345471c6404e7364af0d7019] Merge remote-tracking branch 
'fuse/for-next'
git bisect good 1ca5eddcf8dca1d6345471c6404e7364af0d7019
# good: [8f6d7b3248705920187263a4e7147b0752ec7dcf] Merge remote-tracking branch 
'pci/next'
git bisect good 8f6d7b3248705920187263a4e7147b0752ec7dcf
# good: [df1885a755784da3ef285f36d9230c1d090ef186] RDMA/rtrs_clt: Alloc less 
memory with write path fast memory registration
git bisect good df1885a755784da3ef285f36d9230c1d090ef186
# good: [93d31efb58c8ad4a66bbedbc2d082df458c04e45] Merge remote-tracking branch 
'cpufreq-arm/cpufreq/arm/linux-next'
git bisect good 93d31efb58c8ad4a66bbedbc2d082df458c04e45
# good: [46308965ae6fdc7c25deb2e8c048510ae51bbe66] RDMA/irdma: Check contents 
of user-space irdma_mem_reg_req object
git bisect good 46308965ae6fdc7c25deb2e8c048510ae51bbe66
# good: [6de7a1d006ea9db235492b288312838d6878385f] 
thermal/drivers/int340x/processor_thermal: Split enumeration and processing part
git bisect good 6de7a1d006ea9db235492b288312838d6878385f
# good: [081bec2577cda3d04f6559c60b6f4e2242853520] dt-bindings: of: Add 
restricted DMA pool
git bisect good 081bec2577cda3d04f6559c60b6f4e2242853520
# good: [bf95ac0bcd69979af146852f6a617a60285ebbc1] Merge remote-tracking branch 
'thermal/thermal/linux-next'
git bisect good bf95ac0bcd69979af146852f6a617a60285ebbc1
# good: [3d8287544223a3d2f37981c1f9ffd94d0b5e9ffc] RDMA/core: Always release 
restrack object
git bisect good 3d8287544223a3d2f37981c1f9ffd94d0b5e9ffc
# bad: [cff1f23fad6e0bd7d671acce0d15285c709f259c] Merge remote-tracking branch 
'swiotlb/linux-next'
git bisect bad cff1f23fad6e0bd7d671acce0d15285c709f259c
# bad: [b655006619b7bccd0dc1e055bd72de5d613e7b5c] of: Add plumbing for 
restricted DMA pool
git bisect bad b655006619b7bccd0dc1e055bd72de5d613e7b5c
# first bad commit: [b655006619b7bccd0dc1e055bd72de5d613e7b5c] of: Add plumbing 
for restricted DMA pool





Re: [PATCH] libxl/x86: check return value of SHADOW_OP_SET_ALLOCATION domctl

2021-07-02 Thread Jan Beulich
On 01.07.2021 11:36, Anthony PERARD wrote:
> On Mon, Jun 28, 2021 at 01:47:03PM +0200, Jan Beulich wrote:
>> --- a/tools/libs/light/libxl_x86.c
>> +++ b/tools/libs/light/libxl_x86.c
>> @@ -531,8 +531,18 @@ int libxl__arch_domain_create(libxl__gc
>>  if (d_config->b_info.type != LIBXL_DOMAIN_TYPE_PV) {
>>  unsigned long shadow = DIV_ROUNDUP(d_config->b_info.shadow_memkb,
>> 1024);
>> -xc_shadow_control(ctx->xch, domid, 
>> XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION,
>> -  NULL, 0, &shadow, 0, NULL);
>> +int rc = xc_shadow_control(ctx->xch, domid,
> 
> Could you use 'r' instead of 'rc' ? The later is reserved for libxl
> error codes while the former is for system and libxc calls.

Of course I can, but I did look at the rest of the function and
found that it uses "ret" for the purpose of what you now say
"rc" ought to be used for. Seeing "ret", I decided to avoid it
(knowing you use different names for different kinds of return
values). While I've switched to "r" for now, I'd be rather
inclined to re-use "ret" instead. (Or actually, as per the
remark further down, I can get away without any local variable
then.)

>> +   XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION,
>> +   NULL, 0, &shadow, 0, NULL);
>> +
>> +if (rc) {
> 
> xc_shadow_control seems to return "domctl.u.shadow_op.pages" in some
> cases, are all non-zero return value errors?

Indeed it does, but (a) we pass in zero here and (b) this
operation doesn't alter (nor even care about) the value. So I'd
prefer to stick to what I have, but if you tell me to switch to
"... < 0", I will.

>> +LOGED(ERROR, domid,
>> +  "Failed to set %s allocation: %d (errno:%d)\n",
> 
> LOGED already prints prints the meaning of the "errno" value, so we
> don't need to log it.

I see. Please note that again I took neighboring code (a few lines
down) for reference. Judging from other call sites (not the one
right below here) I infer I also shouldn't have \n in the format
string?

>> +  libxl_defbool_val(d_config->c_info.hap) ? "HAP" : 
>> "shadow",
>> +  rc, errno);
> 
> Is the return value of xc_shadow_control() actually useful when errno is
> already logged?

I don't know. Again what I had matches what can be found a few
lines down in the same function. But looking at other uses (in
other files) I'm getting the impression that it's useless -
dropped.

Jan




Re: [PATCH v15 12/12] of: Add plumbing for restricted DMA pool

2021-07-02 Thread Will Deacon
On Fri, Jul 02, 2021 at 12:39:41PM +0100, Robin Murphy wrote:
> On 2021-07-02 04:08, Guenter Roeck wrote:
> > On Thu, Jun 24, 2021 at 11:55:26PM +0800, Claire Chang wrote:
> > > If a device is not behind an IOMMU, we look up the device node and set
> > > up the restricted DMA when the restricted-dma-pool is presented.
> > > 
> > > Signed-off-by: Claire Chang 
> > > Tested-by: Stefano Stabellini 
> > > Tested-by: Will Deacon 
> > 
> > With this patch in place, all sparc and sparc64 qemu emulations
> > fail to boot. Symptom is that the root file system is not found.
> > Reverting this patch fixes the problem. Bisect log is attached.
> 
> Ah, OF_ADDRESS depends on !SPARC, so of_dma_configure_id() is presumably
> returning an unexpected -ENODEV from the of_dma_set_restricted_buffer()
> stub. That should probably be returning 0 instead, since either way it's not
> an error condition for it to simply do nothing.

Something like below?

Will

--->8

>From 4d9dcb9210c1f37435b6088284e04b6b36ee8c4d Mon Sep 17 00:00:00 2001
From: Will Deacon 
Date: Fri, 2 Jul 2021 14:13:28 +0100
Subject: [PATCH] of: Return success from of_dma_set_restricted_buffer() when
 !OF_ADDRESS

When CONFIG_OF_ADDRESS=n, of_dma_set_restricted_buffer() returns -ENODEV
and breaks the boot for sparc[64] machines. Return 0 instead, since the
function is essentially a glorified NOP in this configuration.

Cc: Claire Chang 
Cc: Konrad Rzeszutek Wilk 
Reported-by: Guenter Roeck 
Suggested-by: Robin Murphy 
Link: https://lore.kernel.org/r/20210702030807.ga2685...@roeck-us.net
Signed-off-by: Will Deacon 
---
 drivers/of/of_private.h | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/of/of_private.h b/drivers/of/of_private.h
index 8fde97565d11..34dd548c5eac 100644
--- a/drivers/of/of_private.h
+++ b/drivers/of/of_private.h
@@ -173,7 +173,8 @@ static inline int of_dma_get_range(struct device_node *np,
 static inline int of_dma_set_restricted_buffer(struct device *dev,
   struct device_node *np)
 {
-   return -ENODEV;
+   /* Do nothing, successfully. */
+   return 0;
 }
 #endif
 
-- 
2.32.0.93.g670b81a890-goog




Re: [PATCH v15 12/12] of: Add plumbing for restricted DMA pool

2021-07-02 Thread Guenter Roeck

On 7/2/21 6:18 AM, Will Deacon wrote:

On Fri, Jul 02, 2021 at 12:39:41PM +0100, Robin Murphy wrote:

On 2021-07-02 04:08, Guenter Roeck wrote:

On Thu, Jun 24, 2021 at 11:55:26PM +0800, Claire Chang wrote:

If a device is not behind an IOMMU, we look up the device node and set
up the restricted DMA when the restricted-dma-pool is presented.

Signed-off-by: Claire Chang 
Tested-by: Stefano Stabellini 
Tested-by: Will Deacon 


With this patch in place, all sparc and sparc64 qemu emulations
fail to boot. Symptom is that the root file system is not found.
Reverting this patch fixes the problem. Bisect log is attached.


Ah, OF_ADDRESS depends on !SPARC, so of_dma_configure_id() is presumably
returning an unexpected -ENODEV from the of_dma_set_restricted_buffer()
stub. That should probably be returning 0 instead, since either way it's not
an error condition for it to simply do nothing.


Something like below?



Yes, that does the trick.


Will

--->8


From 4d9dcb9210c1f37435b6088284e04b6b36ee8c4d Mon Sep 17 00:00:00 2001

From: Will Deacon 
Date: Fri, 2 Jul 2021 14:13:28 +0100
Subject: [PATCH] of: Return success from of_dma_set_restricted_buffer() when
  !OF_ADDRESS

When CONFIG_OF_ADDRESS=n, of_dma_set_restricted_buffer() returns -ENODEV
and breaks the boot for sparc[64] machines. Return 0 instead, since the
function is essentially a glorified NOP in this configuration.

Cc: Claire Chang 
Cc: Konrad Rzeszutek Wilk 
Reported-by: Guenter Roeck 
Suggested-by: Robin Murphy 
Link: https://lore.kernel.org/r/20210702030807.ga2685...@roeck-us.net
Signed-off-by: Will Deacon 


Tested-by: Guenter Roeck 


---
  drivers/of/of_private.h | 3 ++-
  1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/drivers/of/of_private.h b/drivers/of/of_private.h
index 8fde97565d11..34dd548c5eac 100644
--- a/drivers/of/of_private.h
+++ b/drivers/of/of_private.h
@@ -173,7 +173,8 @@ static inline int of_dma_get_range(struct device_node *np,
  static inline int of_dma_set_restricted_buffer(struct device *dev,
   struct device_node *np)
  {
-   return -ENODEV;
+   /* Do nothing, successfully. */
+   return 0;
  }
  #endif
  






Re: [PATCH v15 06/12] swiotlb: Use is_swiotlb_force_bounce for swiotlb data bouncing

2021-07-02 Thread Will Deacon
Hi Nathan,

On Thu, Jul 01, 2021 at 12:52:20AM -0700, Nathan Chancellor wrote:
> On 7/1/2021 12:40 AM, Will Deacon wrote:
> > On Wed, Jun 30, 2021 at 08:56:51AM -0700, Nathan Chancellor wrote:
> > > On Wed, Jun 30, 2021 at 12:43:48PM +0100, Will Deacon wrote:
> > > > On Wed, Jun 30, 2021 at 05:17:27PM +0800, Claire Chang wrote:
> > > > > `BUG: unable to handle page fault for address: 003a8290` and
> > > > > the fact it crashed at `_raw_spin_lock_irqsave` look like the memory
> > > > > (maybe dev->dma_io_tlb_mem) was corrupted?
> > > > > The dev->dma_io_tlb_mem should be set here
> > > > > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/pci/probe.c#n2528)
> > > > > through device_initialize.
> > > > 
> > > > I'm less sure about this. 'dma_io_tlb_mem' should be pointing at
> > > > 'io_tlb_default_mem', which is a page-aligned allocation from memblock.
> > > > The spinlock is at offset 0x24 in that structure, and looking at the
> > > > register dump from the crash:
> > > > 
> > > > Jun 29 18:28:42 hp-4300G kernel: RSP: 0018:adb4013db9e8 EFLAGS: 
> > > > 00010006
> > > > Jun 29 18:28:42 hp-4300G kernel: RAX: 003a8290 RBX: 
> > > >  RCX: 8900572ad580
> > > > Jun 29 18:28:42 hp-4300G kernel: RDX: 89005653f024 RSI: 
> > > > 000c RDI: 1d17
> > > > Jun 29 18:28:42 hp-4300G kernel: RBP: 0a20d000 R08: 
> > > > 000c R09: 
> > > > Jun 29 18:28:42 hp-4300G kernel: R10: 0a20d000 R11: 
> > > > 89005653f000 R12: 0212
> > > > Jun 29 18:28:42 hp-4300G kernel: R13: 1000 R14: 
> > > > 0002 R15: 0020
> > > > Jun 29 18:28:42 hp-4300G kernel: FS:  7f1f8898ea40() 
> > > > GS:89005728() knlGS:
> > > > Jun 29 18:28:42 hp-4300G kernel: CS:  0010 DS:  ES:  CR0: 
> > > > 80050033
> > > > Jun 29 18:28:42 hp-4300G kernel: CR2: 003a8290 CR3: 
> > > > 0001020d CR4: 00350ee0
> > > > Jun 29 18:28:42 hp-4300G kernel: Call Trace:
> > > > Jun 29 18:28:42 hp-4300G kernel:  _raw_spin_lock_irqsave+0x39/0x50
> > > > Jun 29 18:28:42 hp-4300G kernel:  swiotlb_tbl_map_single+0x12b/0x4c0
> > > > 
> > > > Then that correlates with R11 holding the 'dma_io_tlb_mem' pointer and
> > > > RDX pointing at the spinlock. Yet RAX is holding junk :/
> > > > 
> > > > I agree that enabling KASAN would be a good idea, but I also think we
> > > > probably need to get some more information out of 
> > > > swiotlb_tbl_map_single()
> > > > to see see what exactly is going wrong in there.
> > > 
> > > I can certainly enable KASAN and if there is any debug print I can add
> > > or dump anything, let me know!
> > 
> > I bit the bullet and took v5.13 with swiotlb/for-linus-5.14 merged in, built
> > x86 defconfig and ran it on my laptop. However, it seems to work fine!
> > 
> > Please can you share your .config?
> 
> Sure thing, it is attached. It is just Arch Linux's config run through
> olddefconfig. The original is below in case you need to diff it.
> 
> https://raw.githubusercontent.com/archlinux/svntogit-packages/9045405dc835527164f3034b3ceb9a67c7a53cd4/trunk/config
> 
> If there is anything more that I can provide, please let me know.

I eventually got this booting (for some reason it was causing LD to SEGV
trying to link it for a while...) and sadly it works fine on my laptop. Hmm.

Did you manage to try again with KASAN?

It might also be worth taking the IOMMU out of the equation, since that
interfaces differently with SWIOTLB and I couldn't figure out the code path
from the log you provided. What happens if you boot with "amd_iommu=off
swiotlb=force"?

(although word of warning here: i915 dies horribly on my laptop if I pass
swiotlb=force, even with the distro 5.10 kernel)

Will



Re: [XEN PATCH] Config.mk: re-pin OVMF changeset and unpin qemu-xen

2021-07-02 Thread Julien Grall

Hi,

On 28/06/2021 14:48, Ian Jackson wrote:

Anthony PERARD writes ("[XEN PATCH] Config.mk: re-pin OVMF changeset and unpin 
qemu-xen"):

qemu-xen tree have a osstest gate and doesn't need to be pinned.

On the other hand, OVMF's xen repository doesn't have a gate and needs
to be pinned. The "master" branch correspond now to the tag
"edk2-stable202105", so pin to that commit.

Fixes: a04509d34d72 ("Branching: Update version files etc. for newly unstable")
Signed-off-by: Anthony PERARD 


Acked-by: Ian Jackson 

Looks like I adjusted the wrong line in a04509d34d72.  Sorry.


I have committed it.

Cheers,

--
Julien Grall



Re: Some more QEMU 6.0 fixes

2021-07-02 Thread Julien Grall

Hi,

On 28/06/2021 11:01, Anthony PERARD wrote:

Follow-up of
 [XEN PATCH v2 0/8] Fix libxl with QEMU 6.0 + remove some more deprecated 
usages
to fix few missing bits.


I have committed the series.


To be backported to Xen 4.15 as well.


@Ian can you queue it for backport?

Cheers,

--
Julien Grall



[PATCH-4.15] tools/libs/ctrl: fix xc_core_arch_map_p2m() to support linear p2m table

2021-07-02 Thread Juergen Gross
The core of a pv linux guest produced via "xl dump-core" is not usable
as since kernel 4.14 only the linear p2m table is kept if Xen indicates
it is supporting that. Unfortunately xc_core_arch_map_p2m() is still
supporting the 3-level p2m tree only.

Fix that by copying the functionality of map_p2m() from libxenguest to
libxenctrl.

Additionally the mapped p2m isn't of a fixed length now, so the
interface to the mapping functions needs to be adapted. In order not to
add even more parameters, expand struct domain_info_context and use a
pointer to that as a parameter.

This is a backport of upstream commit bd7a29c3d0b937ab542a.

As the original patch includes a modification of a data structure
passed via pointer to a library function, the related function in the
library is renamed in order to be able to spot any external users of
that function. Note that it is extremely unlikely any such users
outside the Xen git tree are existing, so the risk to break any
existing programs is very unlikely. In case such a user is existing,
changing the name of xc_map_domain_meminfo() will at least avoid
silent breakage.

Fixes: dc6d60937121 ("libxc: set flag for support of linear p2m list in domain 
builder")
Signed-off-by: Juergen Gross 
---
 tools/include/xenguest.h  |   2 +
 tools/libs/ctrl/xc_core.c |   5 +-
 tools/libs/ctrl/xc_core.h |   8 +-
 tools/libs/ctrl/xc_core_arm.c |  23 +--
 tools/libs/ctrl/xc_core_x86.c | 256 --
 tools/libs/ctrl/xc_private.h  |   1 +
 tools/libs/guest/xg_domain.c  |  17 +--
 7 files changed, 234 insertions(+), 78 deletions(-)

diff --git a/tools/include/xenguest.h b/tools/include/xenguest.h
index 217022b6e7..36a26deba4 100644
--- a/tools/include/xenguest.h
+++ b/tools/include/xenguest.h
@@ -700,8 +700,10 @@ struct xc_domain_meminfo {
 xen_pfn_t *pfn_type;
 xen_pfn_t *p2m_table;
 unsigned long p2m_size;
+unsigned int p2m_frames;
 };
 
+#define xc_map_domain_meminfo xc_map_domain_meminfo_mod
 int xc_map_domain_meminfo(xc_interface *xch, uint32_t domid,
   struct xc_domain_meminfo *minfo);
 
diff --git a/tools/libs/ctrl/xc_core.c b/tools/libs/ctrl/xc_core.c
index b47ab2f6d8..9576bec5a3 100644
--- a/tools/libs/ctrl/xc_core.c
+++ b/tools/libs/ctrl/xc_core.c
@@ -574,8 +574,7 @@ xc_domain_dumpcore_via_callback(xc_interface *xch,
 goto out;
 }
 
-sts = xc_core_arch_map_p2m(xch, dinfo->guest_width, &info, live_shinfo,
-   &p2m, &dinfo->p2m_size);
+sts = xc_core_arch_map_p2m(xch, dinfo, &info, live_shinfo, &p2m);
 if ( sts != 0 )
 goto out;
 
@@ -945,7 +944,7 @@ out:
 if ( memory_map != NULL )
 free(memory_map);
 if ( p2m != NULL )
-munmap(p2m, PAGE_SIZE * P2M_FL_ENTRIES);
+munmap(p2m, PAGE_SIZE * dinfo->p2m_frames);
 if ( p2m_array != NULL )
 free(p2m_array);
 if ( pfn_array != NULL )
diff --git a/tools/libs/ctrl/xc_core.h b/tools/libs/ctrl/xc_core.h
index 36fb755da2..8ea1f93a10 100644
--- a/tools/libs/ctrl/xc_core.h
+++ b/tools/libs/ctrl/xc_core.h
@@ -138,14 +138,14 @@ int xc_core_arch_memory_map_get(xc_interface *xch,
 xc_dominfo_t *info, shared_info_any_t 
*live_shinfo,
 xc_core_memory_map_t **mapp,
 unsigned int *nr_entries);
-int xc_core_arch_map_p2m(xc_interface *xch, unsigned int guest_width,
+int xc_core_arch_map_p2m(xc_interface *xch, struct domain_info_context *dinfo,
  xc_dominfo_t *info, shared_info_any_t *live_shinfo,
- xen_pfn_t **live_p2m, unsigned long *pfnp);
+ xen_pfn_t **live_p2m);
 
-int xc_core_arch_map_p2m_writable(xc_interface *xch, unsigned int guest_width,
+int xc_core_arch_map_p2m_writable(xc_interface *xch, struct 
domain_info_context *dinfo,
   xc_dominfo_t *info,
   shared_info_any_t *live_shinfo,
-  xen_pfn_t **live_p2m, unsigned long *pfnp);
+  xen_pfn_t **live_p2m);
 
 int xc_core_arch_get_scratch_gpfn(xc_interface *xch, uint32_t domid,
   xen_pfn_t *gpfn);
diff --git a/tools/libs/ctrl/xc_core_arm.c b/tools/libs/ctrl/xc_core_arm.c
index 7b587b4cc5..93765a565f 100644
--- a/tools/libs/ctrl/xc_core_arm.c
+++ b/tools/libs/ctrl/xc_core_arm.c
@@ -66,33 +66,24 @@ xc_core_arch_memory_map_get(xc_interface *xch, struct 
xc_core_arch_context *unus
 
 static int
 xc_core_arch_map_p2m_rw(xc_interface *xch, struct domain_info_context *dinfo, 
xc_dominfo_t *info,
-shared_info_any_t *live_shinfo, xen_pfn_t **live_p2m,
-unsigned long *pfnp, int rw)
+shared_info_any_t *live_shinfo, xen_pfn_t **live_p2m, 
int rw)
 {
 errno = ENOSYS;
 return -1;
 }
 
 int
-xc_core_arch_map_

Re: [PATCH] libxl/x86: check return value of SHADOW_OP_SET_ALLOCATION domctl

2021-07-02 Thread Anthony PERARD
On Fri, Jul 02, 2021 at 02:29:31PM +0200, Jan Beulich wrote:
> On 01.07.2021 11:36, Anthony PERARD wrote:
> > On Mon, Jun 28, 2021 at 01:47:03PM +0200, Jan Beulich wrote:
> >> --- a/tools/libs/light/libxl_x86.c
> >> +++ b/tools/libs/light/libxl_x86.c
> >> @@ -531,8 +531,18 @@ int libxl__arch_domain_create(libxl__gc
> >>  if (d_config->b_info.type != LIBXL_DOMAIN_TYPE_PV) {
> >>  unsigned long shadow = DIV_ROUNDUP(d_config->b_info.shadow_memkb,
> >> 1024);
> >> -xc_shadow_control(ctx->xch, domid, 
> >> XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION,
> >> -  NULL, 0, &shadow, 0, NULL);
> >> +int rc = xc_shadow_control(ctx->xch, domid,
> > 
> > Could you use 'r' instead of 'rc' ? The later is reserved for libxl
> > error codes while the former is for system and libxc calls.
> 
> Of course I can, but I did look at the rest of the function and
> found that it uses "ret" for the purpose of what you now say
> "rc" ought to be used for. Seeing "ret", I decided to avoid it
> (knowing you use different names for different kinds of return
> values). While I've switched to "r" for now, I'd be rather
> inclined to re-use "ret" instead. (Or actually, as per the
> remark further down, I can get away without any local variable
> then.)

I know there's quite a few (many?) coding style issue in libxl. I'm
trying to prevent new issue without asking to fix the existing one.
The use of "ret" is an already existing issue, so I'm fine with it been
use in this patch for libxl error code in the function.

BTW, you still need to store the return value of xc_shadow_control()
into a "r" variable before checking it for error.

> 
> >> +   XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION,
> >> +   NULL, 0, &shadow, 0, NULL);
> >> +
> >> +if (rc) {
> > 
> > xc_shadow_control seems to return "domctl.u.shadow_op.pages" in some
> > cases, are all non-zero return value errors?
> 
> Indeed it does, but (a) we pass in zero here and (b) this
> operation doesn't alter (nor even care about) the value. So I'd
> prefer to stick to what I have, but if you tell me to switch to
> "... < 0", I will.

That's fine, no need to change.

> 
> >> +LOGED(ERROR, domid,
> >> +  "Failed to set %s allocation: %d (errno:%d)\n",
> > 
> > LOGED already prints prints the meaning of the "errno" value, so we
> > don't need to log it.
> 
> I see. Please note that again I took neighboring code (a few lines
> down) for reference. Judging from other call sites (not the one
> right below here) I infer I also shouldn't have \n in the format
> string?

Ah, indeed, the '\n' isn't needed.

> >> +  libxl_defbool_val(d_config->c_info.hap) ? "HAP" : 
> >> "shadow",
> >> +  rc, errno);
> > 
> > Is the return value of xc_shadow_control() actually useful when errno is
> > already logged?
> 
> I don't know. Again what I had matches what can be found a few
> lines down in the same function. But looking at other uses (in
> other files) I'm getting the impression that it's useless -
> dropped.

Whether or not the return value is useful to be logged depends only on
xc_shadow_control(). But thanks.

Thanks,

-- 
Anthony PERARD



Re: [PATCH v20210701 04/40] tools: use integer division in convert-legacy-stream

2021-07-02 Thread Andrew Cooper
On 01/07/2021 10:55, Olaf Hering wrote:
> A single slash gives a float, a double slash gives an int.
>
> bitmap = unpack_exact("Q" * ((max_id/64) + 1))
> TypeError: can't multiply sequence by non-int of type 'float'
>
> Signed-off-by: Olaf Hering 

Reviewed-by: Andrew Cooper 



Re: [PATCH] libxl/x86: check return value of SHADOW_OP_SET_ALLOCATION domctl

2021-07-02 Thread Jan Beulich
On 02.07.2021 16:46, Anthony PERARD wrote:
> On Fri, Jul 02, 2021 at 02:29:31PM +0200, Jan Beulich wrote:
>> On 01.07.2021 11:36, Anthony PERARD wrote:
>>> On Mon, Jun 28, 2021 at 01:47:03PM +0200, Jan Beulich wrote:
 --- a/tools/libs/light/libxl_x86.c
 +++ b/tools/libs/light/libxl_x86.c
 @@ -531,8 +531,18 @@ int libxl__arch_domain_create(libxl__gc
  if (d_config->b_info.type != LIBXL_DOMAIN_TYPE_PV) {
  unsigned long shadow = DIV_ROUNDUP(d_config->b_info.shadow_memkb,
 1024);
 -xc_shadow_control(ctx->xch, domid, 
 XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION,
 -  NULL, 0, &shadow, 0, NULL);
 +int rc = xc_shadow_control(ctx->xch, domid,
>>>
>>> Could you use 'r' instead of 'rc' ? The later is reserved for libxl
>>> error codes while the former is for system and libxc calls.
>>
>> Of course I can, but I did look at the rest of the function and
>> found that it uses "ret" for the purpose of what you now say
>> "rc" ought to be used for. Seeing "ret", I decided to avoid it
>> (knowing you use different names for different kinds of return
>> values). While I've switched to "r" for now, I'd be rather
>> inclined to re-use "ret" instead. (Or actually, as per the
>> remark further down, I can get away without any local variable
>> then.)
> 
> I know there's quite a few (many?) coding style issue in libxl. I'm
> trying to prevent new issue without asking to fix the existing one.
> The use of "ret" is an already existing issue, so I'm fine with it been
> use in this patch for libxl error code in the function.
> 
> BTW, you still need to store the return value of xc_shadow_control()
> into a "r" variable before checking it for error.

Are you saying that

if (xc_shadow_control(ctx->xch, domid,
  XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION,
  NULL, 0, &shadow_mb, 0, NULL)) {

is not acceptable, style-wise? If indeed you are, please disambiguate
your statement above regarding the use of "ret": May I or may I not
use it? IOW do I need to introduce "r", or can I get away with the
existing local variables.

Jan




Re: [PATCH v15 06/12] swiotlb: Use is_swiotlb_force_bounce for swiotlb data bouncing

2021-07-02 Thread Robin Murphy

On 2021-07-02 14:58, Will Deacon wrote:

Hi Nathan,

On Thu, Jul 01, 2021 at 12:52:20AM -0700, Nathan Chancellor wrote:

On 7/1/2021 12:40 AM, Will Deacon wrote:

On Wed, Jun 30, 2021 at 08:56:51AM -0700, Nathan Chancellor wrote:

On Wed, Jun 30, 2021 at 12:43:48PM +0100, Will Deacon wrote:

On Wed, Jun 30, 2021 at 05:17:27PM +0800, Claire Chang wrote:

`BUG: unable to handle page fault for address: 003a8290` and
the fact it crashed at `_raw_spin_lock_irqsave` look like the memory
(maybe dev->dma_io_tlb_mem) was corrupted?
The dev->dma_io_tlb_mem should be set here
(https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/pci/probe.c#n2528)
through device_initialize.


I'm less sure about this. 'dma_io_tlb_mem' should be pointing at
'io_tlb_default_mem', which is a page-aligned allocation from memblock.
The spinlock is at offset 0x24 in that structure, and looking at the
register dump from the crash:

Jun 29 18:28:42 hp-4300G kernel: RSP: 0018:adb4013db9e8 EFLAGS: 00010006
Jun 29 18:28:42 hp-4300G kernel: RAX: 003a8290 RBX:  
RCX: 8900572ad580
Jun 29 18:28:42 hp-4300G kernel: RDX: 89005653f024 RSI: 000c 
RDI: 1d17
Jun 29 18:28:42 hp-4300G kernel: RBP: 0a20d000 R08: 000c 
R09: 
Jun 29 18:28:42 hp-4300G kernel: R10: 0a20d000 R11: 89005653f000 
R12: 0212
Jun 29 18:28:42 hp-4300G kernel: R13: 1000 R14: 0002 
R15: 0020
Jun 29 18:28:42 hp-4300G kernel: FS:  7f1f8898ea40() 
GS:89005728() knlGS:
Jun 29 18:28:42 hp-4300G kernel: CS:  0010 DS:  ES:  CR0: 
80050033
Jun 29 18:28:42 hp-4300G kernel: CR2: 003a8290 CR3: 0001020d 
CR4: 00350ee0
Jun 29 18:28:42 hp-4300G kernel: Call Trace:
Jun 29 18:28:42 hp-4300G kernel:  _raw_spin_lock_irqsave+0x39/0x50
Jun 29 18:28:42 hp-4300G kernel:  swiotlb_tbl_map_single+0x12b/0x4c0

Then that correlates with R11 holding the 'dma_io_tlb_mem' pointer and
RDX pointing at the spinlock. Yet RAX is holding junk :/

I agree that enabling KASAN would be a good idea, but I also think we
probably need to get some more information out of swiotlb_tbl_map_single()
to see see what exactly is going wrong in there.


I can certainly enable KASAN and if there is any debug print I can add
or dump anything, let me know!


I bit the bullet and took v5.13 with swiotlb/for-linus-5.14 merged in, built
x86 defconfig and ran it on my laptop. However, it seems to work fine!

Please can you share your .config?


Sure thing, it is attached. It is just Arch Linux's config run through
olddefconfig. The original is below in case you need to diff it.

https://raw.githubusercontent.com/archlinux/svntogit-packages/9045405dc835527164f3034b3ceb9a67c7a53cd4/trunk/config

If there is anything more that I can provide, please let me know.


I eventually got this booting (for some reason it was causing LD to SEGV
trying to link it for a while...) and sadly it works fine on my laptop. Hmm.

Did you manage to try again with KASAN?

It might also be worth taking the IOMMU out of the equation, since that
interfaces differently with SWIOTLB and I couldn't figure out the code path
from the log you provided. What happens if you boot with "amd_iommu=off
swiotlb=force"?


Oh, now there's a thing... the chat from the IOMMU API in the boot log 
implies that the IOMMU *should* be in the picture - we see that default 
domains are IOMMU_DOMAIN_DMA default and the GPU :0c:00.0 was added 
to a group. That means dev->dma_ops should be set and DMA API calls 
should be going through iommu-dma, yet the callstack in the crash says 
we've gone straight from dma_map_page_attrs() to swiotlb_map(), implying 
the inline dma_direct_map_page() path.


If dev->dma_ops didn't look right in the first place, it's perhaps less 
surprising that dev->dma_io_tlb_mem might be wild as well. It doesn't 
seem plausible that we should have a race between initialising the 
device and probing its driver, so maybe the whole dev pointer is getting 
trampled earlier in the callchain (or is fundamentally wrong to begin 
with, but from a quick skim of the amdgpu code it did look like 
adev->dev and adev->pdev are appropriately set early on by 
amdgpu_pci_probe()).



(although word of warning here: i915 dies horribly on my laptop if I pass
swiotlb=force, even with the distro 5.10 kernel)


FWIW I'd imagine you probably need to massively increase the SWIOTLB 
buffer size to have hope of that working.


Robin.



Re: [PATCH] libxl/x86: check return value of SHADOW_OP_SET_ALLOCATION domctl

2021-07-02 Thread Jan Beulich
On 02.07.2021 17:12, Jan Beulich wrote:
> On 02.07.2021 16:46, Anthony PERARD wrote:
>> On Fri, Jul 02, 2021 at 02:29:31PM +0200, Jan Beulich wrote:
>>> On 01.07.2021 11:36, Anthony PERARD wrote:
 On Mon, Jun 28, 2021 at 01:47:03PM +0200, Jan Beulich wrote:
> --- a/tools/libs/light/libxl_x86.c
> +++ b/tools/libs/light/libxl_x86.c
> @@ -531,8 +531,18 @@ int libxl__arch_domain_create(libxl__gc
>  if (d_config->b_info.type != LIBXL_DOMAIN_TYPE_PV) {
>  unsigned long shadow = DIV_ROUNDUP(d_config->b_info.shadow_memkb,
> 1024);
> -xc_shadow_control(ctx->xch, domid, 
> XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION,
> -  NULL, 0, &shadow, 0, NULL);
> +int rc = xc_shadow_control(ctx->xch, domid,

 Could you use 'r' instead of 'rc' ? The later is reserved for libxl
 error codes while the former is for system and libxc calls.
>>>
>>> Of course I can, but I did look at the rest of the function and
>>> found that it uses "ret" for the purpose of what you now say
>>> "rc" ought to be used for. Seeing "ret", I decided to avoid it
>>> (knowing you use different names for different kinds of return
>>> values). While I've switched to "r" for now, I'd be rather
>>> inclined to re-use "ret" instead. (Or actually, as per the
>>> remark further down, I can get away without any local variable
>>> then.)
>>
>> I know there's quite a few (many?) coding style issue in libxl. I'm
>> trying to prevent new issue without asking to fix the existing one.
>> The use of "ret" is an already existing issue, so I'm fine with it been
>> use in this patch for libxl error code in the function.
>>
>> BTW, you still need to store the return value of xc_shadow_control()
>> into a "r" variable before checking it for error.
> 
> Are you saying that
> 
> if (xc_shadow_control(ctx->xch, domid,
>   XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION,
>   NULL, 0, &shadow_mb, 0, NULL)) {
> 
> is not acceptable, style-wise?

Oh, there is indeed such a rule under "ERROR HANDLING". Which means ...

> If indeed you are, please disambiguate
> your statement above regarding the use of "ret": May I or may I not
> use it? IOW do I need to introduce "r", or can I get away with the
> existing local variables.

... I need this to be clarified.

Jan




[ovmf test] 163224: regressions - FAIL

2021-07-02 Thread osstest service owner
flight 163224 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/163224/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 162359
 test-amd64-amd64-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 
162359

version targeted for testing:
 ovmf fea7901dba72eeac526f3ef12a4ad4c539622373
baseline version:
 ovmf c410ad4da4b7785170d3d42a3ba190c2caac6feb

Last test of basis   162359  2021-06-04 03:40:08 Z   28 days
Failing since162368  2021-06-04 15:42:59 Z   27 days   74 attempts
Testing same since   163216  2021-07-01 22:42:29 Z0 days4 attempts


People who touched revisions under test:
  Abner Chang 
  Agrawal, Sachin 
  Alexandru Elisei 
  Anthony PERARD 
  Ard Biesheuvel 
  Daniel Schaefer 
  Daoxiang Li 
  Dov Murik 
  DunTan 
  gaoliming 
  Guo Dong 
  Hao A Wu 
  Jian J Wang 
  Kaaira Gupta 
  Ken Lautner 
  Kenneth Lautner 
  Kun Qin 
  Laszlo Ersek 
  Leif Lindholm 
  Liming Gao 
  Loo Tung Lun 
  Loo, Tung Lun 
  Manickavasakam Karpagavinayagam 
  Maurice Ma 
  Ni, Ray 
  Patrick Rudolph 
  Pierre Gondois 
  Ray Ni 
  Rebecca Cran 
  Rebecca Cran 
  Sachin Agrawal 
  Sami Mujawar 
  Scottie Kuo 
  Sean Brogan 
  Sean Brogan 
  Sumana Venur 
  Sunil V L 
  xueshengfeng 
  Zhiguang Liu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3224 lines long.)



Re: [PATCH v20210701 05/40] tools: handle libxl__physmap_info.name properly in convert-legacy-stream

2021-07-02 Thread Andrew Cooper
On 01/07/2021 10:56, Olaf Hering wrote:
> diff --git a/tools/python/scripts/convert-legacy-stream 
> b/tools/python/scripts/convert-legacy-stream
> index 66ee3d2f5d..9003ac4f6d 100755
> --- a/tools/python/scripts/convert-legacy-stream
> +++ b/tools/python/scripts/convert-legacy-stream
> @@ -336,20 +336,21 @@ def read_libxl_toolstack(vm, data):
>  if len(data) < namelen:
>  raise StreamError("Remaining data too short for physmap name")
>  
> -name = data[:namelen]
> +c_string = data[:namelen]
>  data = data[namelen:]
>  
>  # Strip padding off the end of name
>  if twidth == 64:
> -name = name[:-4]
> +c_string = c_string[:-4]
>  
> -if name[-1] != b'\x00':
> +name, nil = unpack("={0}sB".format(len(c_string) - 1), c_string)

This is rather invasive.  How about simply:

diff --git a/tools/python/scripts/convert-legacy-stream
b/tools/python/scripts/convert-legacy-stream
index ca93a93848ec..d4ae94c02f21 100755
--- a/tools/python/scripts/convert-legacy-stream
+++ b/tools/python/scripts/convert-legacy-stream
@@ -342,7 +342,7 @@ def read_libxl_toolstack(vm, data):
 if twidth == 64:
 name = name[:-4]
 
-    if name[-1] != b'\x00':
+    if bytearray(name)[-1] != 0:
 raise StreamError("physmap name not NUL terminated")
 
 root = b"physmap/%x" % (phys, )

which is rather more contained, and looks to work from Py2.6 and later?

~Andrew



Re: [PATCH] libxl/x86: check return value of SHADOW_OP_SET_ALLOCATION domctl

2021-07-02 Thread Anthony PERARD
On Fri, Jul 02, 2021 at 05:14:40PM +0200, Jan Beulich wrote:
> On 02.07.2021 17:12, Jan Beulich wrote:
> > On 02.07.2021 16:46, Anthony PERARD wrote:
> >> On Fri, Jul 02, 2021 at 02:29:31PM +0200, Jan Beulich wrote:
> >>> On 01.07.2021 11:36, Anthony PERARD wrote:
>  On Mon, Jun 28, 2021 at 01:47:03PM +0200, Jan Beulich wrote:
> > --- a/tools/libs/light/libxl_x86.c
> > +++ b/tools/libs/light/libxl_x86.c
> > @@ -531,8 +531,18 @@ int libxl__arch_domain_create(libxl__gc
> >  if (d_config->b_info.type != LIBXL_DOMAIN_TYPE_PV) {
> >  unsigned long shadow = 
> > DIV_ROUNDUP(d_config->b_info.shadow_memkb,
> > 1024);
> > -xc_shadow_control(ctx->xch, domid, 
> > XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION,
> > -  NULL, 0, &shadow, 0, NULL);
> > +int rc = xc_shadow_control(ctx->xch, domid,
> 
>  Could you use 'r' instead of 'rc' ? The later is reserved for libxl
>  error codes while the former is for system and libxc calls.
> >>>
> >>> Of course I can, but I did look at the rest of the function and
> >>> found that it uses "ret" for the purpose of what you now say
> >>> "rc" ought to be used for. Seeing "ret", I decided to avoid it
> >>> (knowing you use different names for different kinds of return
> >>> values). While I've switched to "r" for now, I'd be rather
> >>> inclined to re-use "ret" instead. (Or actually, as per the
> >>> remark further down, I can get away without any local variable
> >>> then.)
> >>
> >> I know there's quite a few (many?) coding style issue in libxl. I'm
> >> trying to prevent new issue without asking to fix the existing one.
> >> The use of "ret" is an already existing issue, so I'm fine with it been
> >> use in this patch for libxl error code in the function.
> >>
> >> BTW, you still need to store the return value of xc_shadow_control()
> >> into a "r" variable before checking it for error.
> > 
> > Are you saying that
> > 
> > if (xc_shadow_control(ctx->xch, domid,
> >   XEN_DOMCTL_SHADOW_OP_SET_ALLOCATION,
> >   NULL, 0, &shadow_mb, 0, NULL)) {
> > 
> > is not acceptable, style-wise?
> 
> Oh, there is indeed such a rule under "ERROR HANDLING". Which means ...
> 
> > If indeed you are, please disambiguate
> > your statement above regarding the use of "ret": May I or may I not
> > use it? IOW do I need to introduce "r", or can I get away with the
> > existing local variables.
> 
> ... I need this to be clarified.

You need to introduce the "r" local variable, to store xc_shadow_control
return value.
Then, set "ret" to ERROR_FAIL before "goto out;".

Hope that's clearer.

Cheers,

-- 
Anthony PERARD



Re: [PATCH V7 01/18] perf/core: Use static_call to optimize perf_guest_info_callbacks

2021-07-02 Thread Joe Perches
On Fri, 2021-07-02 at 13:22 +0200, Peter Zijlstra wrote:
> On Tue, Jun 22, 2021 at 05:42:49PM +0800, Zhu Lingshan wrote:
> > diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
[]
> > @@ -90,6 +90,27 @@ DEFINE_STATIC_CALL_NULL(x86_pmu_pebs_aliases, 
> > *x86_pmu.pebs_aliases);
> >   */
> >  DEFINE_STATIC_CALL_RET0(x86_pmu_guest_get_msrs, *x86_pmu.guest_get_msrs);
> >  
> > 
> > +DEFINE_STATIC_CALL_RET0(x86_guest_state, *(perf_guest_cbs->state));
> > +DEFINE_STATIC_CALL_RET0(x86_guest_get_ip, *(perf_guest_cbs->get_ip));
> > +DEFINE_STATIC_CALL_RET0(x86_guest_handle_intel_pt_intr, 
> > *(perf_guest_cbs->handle_intel_pt_intr));
> > +
> > +void arch_perf_update_guest_cbs(void)
> > +{
> > +   static_call_update(x86_guest_state, (void *)&__static_call_return0);
> > +   static_call_update(x86_guest_get_ip, (void *)&__static_call_return0);
> > +   static_call_update(x86_guest_handle_intel_pt_intr, (void 
> > *)&__static_call_return0);
> > +
> > +   if (perf_guest_cbs && perf_guest_cbs->state)
> > +   static_call_update(x86_guest_state, perf_guest_cbs->state);
> > +
> > +   if (perf_guest_cbs && perf_guest_cbs->get_ip)
> > +   static_call_update(x86_guest_get_ip, perf_guest_cbs->get_ip);
> > +
> > +   if (perf_guest_cbs && perf_guest_cbs->handle_intel_pt_intr)
> > +   static_call_update(x86_guest_handle_intel_pt_intr,
> > +  perf_guest_cbs->handle_intel_pt_intr);
> > +}
> 
> Coding style wants { } on that last if().

That's just your personal preference.

The coding-style document doesn't require that.

It just says single statement.  It's not the number of
vertical lines or characters required for the statement.

--

Do not unnecessarily use braces where a single statement will do.

.. code-block:: c

if (condition)
action();

and

.. code-block:: none

if (condition)
do_this();
else
do_that();

This does not apply if only one branch of a conditional statement is a single
statement; in the latter case use braces in both branches:





Re: [PATCH v20210701 06/40] tools: fix Python3.4 TypeError in format string

2021-07-02 Thread Marek Marczykowski-Górecki
On Thu, Jul 01, 2021 at 11:56:01AM +0200, Olaf Hering wrote:
> Using the first element of a tuple for a format specifier fails with
> python3.4 as included in SLE12:
> b = b"string/%x" % (i, )
> TypeError: unsupported operand type(s) for %: 'bytes' and 'tuple'
> 
> It happens to work with python 2.7 and 3.6.
> Use a syntax that is handled by all three variants.
> 
> Signed-off-by: Olaf Hering 
> ---
>  tools/python/scripts/convert-legacy-stream | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/tools/python/scripts/convert-legacy-stream 
> b/tools/python/scripts/convert-legacy-stream
> index 9003ac4f6d..235b922ff5 100755
> --- a/tools/python/scripts/convert-legacy-stream
> +++ b/tools/python/scripts/convert-legacy-stream
> @@ -347,9 +347,9 @@ def read_libxl_toolstack(vm, data):
>  if nil != 0:
>  raise StreamError("physmap name not NUL terminated")
>  
> -root = b"physmap/%x" % (phys, )
> -kv = [root + b"/start_addr", b"%x" % (start, ),
> -  root + b"/size",   b"%x" % (size, ),
> +root = bytes(("physmap/%x" % phys).encode('utf-8'))
> +kv = [root + b"/start_addr", bytes(("%x" % start).encode('utf-8')),
> +  root + b"/size",   bytes(("%x" % size).encode('utf-8')),

Why bytes()? Encode does already return bytes type.

>root + b"/name",   name]
>  
>  for key, val in zip(kv[0::2], kv[1::2]):

-- 
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab


signature.asc
Description: PGP signature


Re: [PATCH V7 01/18] perf/core: Use static_call to optimize perf_guest_info_callbacks

2021-07-02 Thread Peter Zijlstra
On Fri, Jul 02, 2021 at 09:00:22AM -0700, Joe Perches wrote:
> On Fri, 2021-07-02 at 13:22 +0200, Peter Zijlstra wrote:
> > On Tue, Jun 22, 2021 at 05:42:49PM +0800, Zhu Lingshan wrote:
> > > diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
> []
> > > @@ -90,6 +90,27 @@ DEFINE_STATIC_CALL_NULL(x86_pmu_pebs_aliases, 
> > > *x86_pmu.pebs_aliases);
> > >   */
> > >  DEFINE_STATIC_CALL_RET0(x86_pmu_guest_get_msrs, *x86_pmu.guest_get_msrs);
> > >  
> > > 
> > > +DEFINE_STATIC_CALL_RET0(x86_guest_state, *(perf_guest_cbs->state));
> > > +DEFINE_STATIC_CALL_RET0(x86_guest_get_ip, *(perf_guest_cbs->get_ip));
> > > +DEFINE_STATIC_CALL_RET0(x86_guest_handle_intel_pt_intr, 
> > > *(perf_guest_cbs->handle_intel_pt_intr));
> > > +
> > > +void arch_perf_update_guest_cbs(void)
> > > +{
> > > + static_call_update(x86_guest_state, (void *)&__static_call_return0);
> > > + static_call_update(x86_guest_get_ip, (void *)&__static_call_return0);
> > > + static_call_update(x86_guest_handle_intel_pt_intr, (void 
> > > *)&__static_call_return0);
> > > +
> > > + if (perf_guest_cbs && perf_guest_cbs->state)
> > > + static_call_update(x86_guest_state, perf_guest_cbs->state);
> > > +
> > > + if (perf_guest_cbs && perf_guest_cbs->get_ip)
> > > + static_call_update(x86_guest_get_ip, perf_guest_cbs->get_ip);
> > > +
> > > + if (perf_guest_cbs && perf_guest_cbs->handle_intel_pt_intr)
> > > + static_call_update(x86_guest_handle_intel_pt_intr,
> > > +perf_guest_cbs->handle_intel_pt_intr);
> > > +}
> > 
> > Coding style wants { } on that last if().
> 
> That's just your personal preference.

As a maintainer, those carry weight, also that's tip rules:

  https://lore.kernel.org/lkml/20181107171149.165693...@linutronix.de/



Re: [PATCH V7 01/18] perf/core: Use static_call to optimize perf_guest_info_callbacks

2021-07-02 Thread Mark Rutland
On Fri, Jul 02, 2021 at 09:00:22AM -0700, Joe Perches wrote:
> On Fri, 2021-07-02 at 13:22 +0200, Peter Zijlstra wrote:
> > On Tue, Jun 22, 2021 at 05:42:49PM +0800, Zhu Lingshan wrote:
> > > diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
> []
> > > @@ -90,6 +90,27 @@ DEFINE_STATIC_CALL_NULL(x86_pmu_pebs_aliases, 
> > > *x86_pmu.pebs_aliases);
> > >   */
> > >  DEFINE_STATIC_CALL_RET0(x86_pmu_guest_get_msrs, *x86_pmu.guest_get_msrs);
> > >  
> > > 
> > > +DEFINE_STATIC_CALL_RET0(x86_guest_state, *(perf_guest_cbs->state));
> > > +DEFINE_STATIC_CALL_RET0(x86_guest_get_ip, *(perf_guest_cbs->get_ip));
> > > +DEFINE_STATIC_CALL_RET0(x86_guest_handle_intel_pt_intr, 
> > > *(perf_guest_cbs->handle_intel_pt_intr));
> > > +
> > > +void arch_perf_update_guest_cbs(void)
> > > +{
> > > + static_call_update(x86_guest_state, (void *)&__static_call_return0);
> > > + static_call_update(x86_guest_get_ip, (void *)&__static_call_return0);
> > > + static_call_update(x86_guest_handle_intel_pt_intr, (void 
> > > *)&__static_call_return0);
> > > +
> > > + if (perf_guest_cbs && perf_guest_cbs->state)
> > > + static_call_update(x86_guest_state, perf_guest_cbs->state);
> > > +
> > > + if (perf_guest_cbs && perf_guest_cbs->get_ip)
> > > + static_call_update(x86_guest_get_ip, perf_guest_cbs->get_ip);
> > > +
> > > + if (perf_guest_cbs && perf_guest_cbs->handle_intel_pt_intr)
> > > + static_call_update(x86_guest_handle_intel_pt_intr,
> > > +perf_guest_cbs->handle_intel_pt_intr);
> > > +}
> > 
> > Coding style wants { } on that last if().
> 
> That's just your personal preference.
> 
> The coding-style document doesn't require that.
> 
> It just says single statement.  It's not the number of
> vertical lines or characters required for the statement.
> 
> --
> 
> Do not unnecessarily use braces where a single statement will do.
> 
> .. code-block:: c
> 
>   if (condition)
>   action();
> 
> and
> 
> .. code-block:: none
> 
>   if (condition)
>   do_this();
>   else
>   do_that();
> 
> This does not apply if only one branch of a conditional statement is a single
> statement; in the latter case use braces in both branches:

Immediately after this, we say:

| Also, use braces when a loop contains more than a single simple statement:
|
| .. code-block:: c
| 
| while (condition) {
| if (test)
| do_something();
| }
| 

... and while that says "a loop", the principle is obviously supposed to
apply to conditionals too; structurally they're no different. We should
just fix the documentation to say "a loop or conditional", or something
to that effect.

Mark.



Re: [PATCH v20210701 06/40] tools: fix Python3.4 TypeError in format string

2021-07-02 Thread Andrew Cooper
On 02/07/2021 17:19, Marek Marczykowski-Górecki wrote:
> On Thu, Jul 01, 2021 at 11:56:01AM +0200, Olaf Hering wrote:
>> Using the first element of a tuple for a format specifier fails with
>> python3.4 as included in SLE12:
>> b = b"string/%x" % (i, )
>> TypeError: unsupported operand type(s) for %: 'bytes' and 'tuple'
>>
>> It happens to work with python 2.7 and 3.6.
>> Use a syntax that is handled by all three variants.
>>
>> Signed-off-by: Olaf Hering 
>> ---
>>  tools/python/scripts/convert-legacy-stream | 6 +++---
>>  1 file changed, 3 insertions(+), 3 deletions(-)
>>
>> diff --git a/tools/python/scripts/convert-legacy-stream 
>> b/tools/python/scripts/convert-legacy-stream
>> index 9003ac4f6d..235b922ff5 100755
>> --- a/tools/python/scripts/convert-legacy-stream
>> +++ b/tools/python/scripts/convert-legacy-stream
>> @@ -347,9 +347,9 @@ def read_libxl_toolstack(vm, data):
>>  if nil != 0:
>>  raise StreamError("physmap name not NUL terminated")
>>  
>> -root = b"physmap/%x" % (phys, )
>> -kv = [root + b"/start_addr", b"%x" % (start, ),
>> -  root + b"/size",   b"%x" % (size, ),
>> +root = bytes(("physmap/%x" % phys).encode('utf-8'))
>> +kv = [root + b"/start_addr", bytes(("%x" % start).encode('utf-8')),
>> +  root + b"/size",   bytes(("%x" % size).encode('utf-8')),
> Why bytes()? Encode does already return bytes type.

Yes - I've just tried this out on various version of python (including
https://www.onlinegdb.com/online_python_interpreter which is the only
place I can find Python 3.4 easily available)

.encode() does return bytes (Py3) or str (Py2) so doesn't need the
surrounding bytes().

However, the % (phys, ) with the trailing comma is deliberate to work
around a common python error, so wants to remain if you're keeping the
%-formatting.

~Andrew



Re: [PATCH V7 01/18] perf/core: Use static_call to optimize perf_guest_info_callbacks

2021-07-02 Thread Joe Perches
On Fri, 2021-07-02 at 18:19 +0200, Peter Zijlstra wrote:
> On Fri, Jul 02, 2021 at 09:00:22AM -0700, Joe Perches wrote:
> > On Fri, 2021-07-02 at 13:22 +0200, Peter Zijlstra wrote:
> > > On Tue, Jun 22, 2021 at 05:42:49PM +0800, Zhu Lingshan wrote:
> > > > diff --git a/arch/x86/events/core.c b/arch/x86/events/core.c
> > []
> > > > +   if (perf_guest_cbs && perf_guest_cbs->handle_intel_pt_intr)
> > > > +   static_call_update(x86_guest_handle_intel_pt_intr,
> > > > +  
> > > > perf_guest_cbs->handle_intel_pt_intr);
> > > > +}
> > > 
> > > Coding style wants { } on that last if().
> > 
> > That's just your personal preference.
> 
> As a maintainer, those carry weight, also that's tip rules:
> 
>   https://lore.kernel.org/lkml/20181107171149.165693...@linutronix.de/

Right, definitely so.

But merely referencing 'coding style' is ambiguous at best.

btw:

ASCII commonly refers to '{' and '}', the curly brackets, to be braces
and '[' and ']', the square brackets, to be brackets.

It might be clearer to use that terminology.

belts and braces, etc...

cheers, Joe



+Bracket rules
+^
+
+Brackets should be omitted only if the statement which follows 'if', 'for',
+'while' etc. is truly a single line::
+
+   if (foo)
+   do_something();
+
+The following is not considered to be a single line statement even
+though C does not require brackets::
+
+   for (i = 0; i < end; i++)
+   if (foo[i])
+   do_something(foo[i]);
+
+Adding brackets around the outer loop enhances the reading flow::
+
+   for (i = 0; i < end; i++) {
+   if (foo[i])
+   do_something(foo[i]);
+   }





[qemu-mainline test] 163220: regressions - FAIL

2021-07-02 Thread osstest service owner
flight 163220 qemu-mainline real [real]
flight 163226 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/163220/
http://logs.test-lab.xenproject.org/osstest/logs/163226/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-qemuu-nested-amd 16 xen-boot/l1 fail REGR. vs. 152631
 test-amd64-amd64-qemuu-nested-intel 16 xen-boot/l1   fail REGR. vs. 152631

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 152631
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 152631
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 152631
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 152631
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 152631
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 152631
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass

version targeted for testing:
 qemuudd62bf14b756821fa293e3465955a41e9d460deb
baseline version:
 qemuu1d806cef0e38b5db8347a8e12f214d543204a314

Last test of basis   152631  2020-08-20 09:07:46 Z  316 days
Failing since152659  2020-08-21 14:07:39 Z  315 days  578 attempts
Testing same since   163220  2021-07-02 02:27:48 Z0 days1 attempts


552 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm  pass
 build-arm64-xsm  pass
 build-i386-xsm   pass
 build-amd64

Re: [PATCH V7 01/18] perf/core: Use static_call to optimize perf_guest_info_callbacks

2021-07-02 Thread Joe Perches
On Fri, 2021-07-02 at 17:38 +0100, Mark Rutland wrote:
> On Fri, Jul 02, 2021 at 09:00:22AM -0700, Joe Perches wrote:
> > On Fri, 2021-07-02 at 13:22 +0200, Peter Zijlstra wrote:
> > > On Tue, Jun 22, 2021 at 05:42:49PM +0800, Zhu Lingshan wrote:
[]
> > > > +   if (perf_guest_cbs && perf_guest_cbs->handle_intel_pt_intr)
> > > > +   static_call_update(x86_guest_handle_intel_pt_intr,
> > > > +  
> > > > perf_guest_cbs->handle_intel_pt_intr);
> > > > +}
> > > 
> > > Coding style wants { } on that last if().
> > 
> > That's just your personal preference.
> > 
> > The coding-style document doesn't require that.
> > 
> > It just says single statement.  It's not the number of
> > vertical lines or characters required for the statement.
> > 
> > --
> > 
> > Do not unnecessarily use braces where a single statement will do.
> > 
> > .. code-block:: c
> > 
> > if (condition)
> > action();
> > 
> > and
> > 
> > .. code-block:: none
> > 
> > if (condition)
> > do_this();
> > else
> > do_that();
> > 
> > This does not apply if only one branch of a conditional statement is a 
> > single
> > statement; in the latter case use braces in both branches:
> 
> Immediately after this, we say:
> 
> > Also, use braces when a loop contains more than a single simple statement:
> > 
> > .. code-block:: c
> > 
> > while (condition) {
> > if (test)
> > do_something();
> > }
> > 
> 
> ... and while that says "a loop", the principle is obviously supposed to
> apply to conditionals too; structurally they're no different. We should
> just fix the documentation to say "a loop or conditional", or something
> to that effect.

  Maybe.

I think there are _way_ too many existing obvious uses where the
statement that follows a conditional is multi-line.

if (foo)
printk(fmt,
   args...);

where the braces wouldn't add anything other than more vertical space.

I don't much care one way or another other than Peter's somewhat ambiguous
use of the phrase "coding style".

checkpatch doesn't emit a message either way.
-
$ cat t_multiline.c
// SPDX-License-Identifier: GPL-2.0-only

void foo(void)
{
if (foo) {
pr_info(fmt,
args);
}

if (foo)
pr_info(fmt,
args);

if (foo)
pr_info(fmt, args);
}

$ ./scripts/checkpatch.pl -f --strict t_multiline.c
total: 0 errors, 0 warnings, 0 checks, 16 lines checked

t_multiline.c has no obvious style problems and is ready for submission.
-

cheers, Joe





[ovmf test] 163227: regressions - FAIL

2021-07-02 Thread osstest service owner
flight 163227 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/163227/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 162359
 test-amd64-amd64-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 
162359

version targeted for testing:
 ovmf fea7901dba72eeac526f3ef12a4ad4c539622373
baseline version:
 ovmf c410ad4da4b7785170d3d42a3ba190c2caac6feb

Last test of basis   162359  2021-06-04 03:40:08 Z   28 days
Failing since162368  2021-06-04 15:42:59 Z   28 days   75 attempts
Testing same since   163216  2021-07-01 22:42:29 Z0 days5 attempts


People who touched revisions under test:
  Abner Chang 
  Agrawal, Sachin 
  Alexandru Elisei 
  Anthony PERARD 
  Ard Biesheuvel 
  Daniel Schaefer 
  Daoxiang Li 
  Dov Murik 
  DunTan 
  gaoliming 
  Guo Dong 
  Hao A Wu 
  Jian J Wang 
  Kaaira Gupta 
  Ken Lautner 
  Kenneth Lautner 
  Kun Qin 
  Laszlo Ersek 
  Leif Lindholm 
  Liming Gao 
  Loo Tung Lun 
  Loo, Tung Lun 
  Manickavasakam Karpagavinayagam 
  Maurice Ma 
  Ni, Ray 
  Patrick Rudolph 
  Pierre Gondois 
  Ray Ni 
  Rebecca Cran 
  Rebecca Cran 
  Sachin Agrawal 
  Sami Mujawar 
  Scottie Kuo 
  Sean Brogan 
  Sean Brogan 
  Sumana Venur 
  Sunil V L 
  xueshengfeng 
  Zhiguang Liu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3224 lines long.)



[PATCH] xen/Makefile: drop -Werror

2021-07-02 Thread Fabrice Fontaine
Drop -Werror to avoid the following build failure with -DNDEBUG:

In file included from :0:0:
/usr/lfs/hdd_v1/rc-buildroot-test/scripts/instance-0/output-1/build/xen-4.14.2/xen/include/xen/config.h:94:0:
 error: "NDEBUG" redefined [-Werror]
 #define NDEBUG

:0:0: note: this is the location of the previous definition

Fixes:
 - 
http://autobuild.buildroot.org/results/66573ad0abc4244c0dfeac8b684a7bfcc31c0d4d

Signed-off-by: Fabrice Fontaine 
---
 xen/Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/xen/Makefile b/xen/Makefile
index 89879fad4c..cf9f83b1fb 100644
--- a/xen/Makefile
+++ b/xen/Makefile
@@ -210,7 +210,7 @@ CFLAGS += -fomit-frame-pointer
 endif
 
 CFLAGS += -nostdinc -fno-builtin -fno-common
-CFLAGS += -Werror -Wredundant-decls -Wno-pointer-arith
+CFLAGS += -Wredundant-decls -Wno-pointer-arith
 $(call cc-option-add,CFLAGS,CC,-Wvla)
 CFLAGS += -pipe -D__XEN__ -include $(BASEDIR)/include/xen/config.h
 CFLAGS-$(CONFIG_DEBUG_INFO) += -g
-- 
2.30.2




Re: [PATCH] xen/Makefile: drop -Werror

2021-07-02 Thread Andrew Cooper
On 02/07/2021 18:06, Fabrice Fontaine wrote:
> Drop -Werror to avoid the following build failure with -DNDEBUG:
>
> In file included from :0:0:
> /usr/lfs/hdd_v1/rc-buildroot-test/scripts/instance-0/output-1/build/xen-4.14.2/xen/include/xen/config.h:94:0:
>  error: "NDEBUG" redefined [-Werror]
>  #define NDEBUG
>
> :0:0: note: this is the location of the previous definition
>
> Fixes:
>  - 
> http://autobuild.buildroot.org/results/66573ad0abc4244c0dfeac8b684a7bfcc31c0d4d
>
> Signed-off-by: Fabrice Fontaine 

For better or worse, It is Xen's policy that -Werror will remain.  95%
of the time, it is the right thing.  We will however build failures
whenever they crop up.

This one is weird though.  How is NDEBUG getting in twice?  What does
the rest of this build environment look like?

~Andrew



Re: [PATCH] xen/Makefile: drop -Werror

2021-07-02 Thread Fabrice Fontaine
Le ven. 2 juil. 2021 à 19:34, Andrew Cooper
 a écrit :
>
> On 02/07/2021 18:06, Fabrice Fontaine wrote:
> > Drop -Werror to avoid the following build failure with -DNDEBUG:
> >
> > In file included from :0:0:
> > /usr/lfs/hdd_v1/rc-buildroot-test/scripts/instance-0/output-1/build/xen-4.14.2/xen/include/xen/config.h:94:0:
> >  error: "NDEBUG" redefined [-Werror]
> >  #define NDEBUG
> >
> > :0:0: note: this is the location of the previous definition
> >
> > Fixes:
> >  - 
> > http://autobuild.buildroot.org/results/66573ad0abc4244c0dfeac8b684a7bfcc31c0d4d
> >
> > Signed-off-by: Fabrice Fontaine 
>
> For better or worse, It is Xen's policy that -Werror will remain.  95%
> of the time, it is the right thing.  We will however build failures
> whenever they crop up.
>
> This one is weird though.  How is NDEBUG getting in twice?  What does
> the rest of this build environment look like?
NDEBUG is added by buildroot in the command line if the user sets
BR2_ENABLE_RUNTIME_DEBUG to false since
https://git.buildroot.net/buildroot/commit/?id=5a8c50fe05afacc3cbe8e7347e238da9f242fab0

I do agree that setting -Werror is generally perfectly valid for upstream.
However, for downstream packager, it is generally seen as an issue as
it will always raise unexepected build failures with older, newer, or
exotic toolchains, see
https://embeddedartistry.com/blog/2017/05/22/werror-is-not-your-friend.
It would be good to, at least, have an option to disable -Werror for
example through a XEN_DISABLE_WERROR.
>
> ~Andrew
Best Regards,

Fabrice



[xen-unstable-smoke test] 163225: tolerable all pass - PUSHED

2021-07-02 Thread osstest service owner
flight 163225 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/163225/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  3bc3be978fd61f8099797864136c5f447c0e4aae
baseline version:
 xen  f95b7b37cfc6d4613721df9357090d14712013c0

Last test of basis   163183  2021-06-29 13:01:32 Z3 days
Testing same since   163225  2021-07-02 15:01:35 Z0 days1 attempts


People who touched revisions under test:
  Anthony PERARD 
  Ian Jackson 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   f95b7b37cf..3bc3be978f  3bc3be978fd61f8099797864136c5f447c0e4aae -> smoke



[linux-linus test] 163222: regressions - FAIL

2021-07-02 Thread osstest service owner
flight 163222 linux-linus real [real]
http://logs.test-lab.xenproject.org/osstest/logs/163222/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-xsm7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-intel  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64-shadow 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-xl-qemuu-dmrestrict-amd64-dmrestrict 7 xen-install fail REGR. 
vs. 152332
 test-amd64-i386-libvirt   7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-coresched-i386-xl  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-ws16-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-xl7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-debianhvm-amd64  7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-amd64-i386-qemut-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-qemuu-rhel6hvm-amd  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-examine   6 xen-install  fail REGR. vs. 152332
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 7 xen-install fail REGR. vs. 
152332
 test-amd64-i386-freebsd10-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-pvshim 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-raw7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-xl-shadow 7 xen-install  fail REGR. vs. 152332
 test-amd64-i386-freebsd10-i386  7 xen-installfail REGR. vs. 152332
 test-amd64-i386-xl-qemut-debianhvm-i386-xsm 7 xen-install fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-ovmf-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemuu-win7-amd64  7 xen-install   fail REGR. vs. 152332
 test-amd64-i386-xl-qemut-stubdom-debianhvm-amd64-xsm 7 xen-install fail REGR. 
vs. 152332
 test-amd64-i386-libvirt-xsm   7 xen-install  fail REGR. vs. 152332
 test-arm64-arm64-xl-seattle  12 debian-install   fail REGR. vs. 152332
 test-amd64-amd64-libvirt-vhd 13 guest-start  fail REGR. vs. 152332
 test-arm64-arm64-xl-credit1  12 debian-install   fail REGR. vs. 152332
 test-arm64-arm64-xl-credit2  12 debian-install   fail REGR. vs. 152332
 test-arm64-arm64-xl-thunderx 12 debian-install   fail REGR. vs. 152332
 test-arm64-arm64-xl-xsm  12 debian-install   fail REGR. vs. 152332
 test-armhf-armhf-examine  8 reboot   fail REGR. vs. 152332
 test-arm64-arm64-xl  12 debian-install   fail REGR. vs. 152332
 test-arm64-arm64-libvirt-xsm 12 debian-install   fail REGR. vs. 152332
 test-amd64-amd64-amd64-pvgrub 20 guest-stop  fail REGR. vs. 152332
 test-armhf-armhf-xl-credit1   8 xen-boot fail REGR. vs. 152332
 test-amd64-amd64-i386-pvgrub 20 guest-stop   fail REGR. vs. 152332
 test-armhf-armhf-xl   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl-credit2   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-libvirt  8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-libvirt-raw  8 xen-boot fail REGR. vs. 152332
 test-amd64-amd64-xl-qcow213 guest-start  fail REGR. vs. 152332
 test-armhf-armhf-xl-cubietruck 12 debian-install fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 10 xen-install/src_host fail REGR. vs. 152332
 test-amd64-i386-libvirt-pair 11 xen-install/dst_host fail REGR. vs. 152332
 test-armhf-armhf-xl-arndale   8 xen-boot fail REGR. vs. 152332
 test-armhf-armhf-xl-vhd  12 debian-di-installfail REGR. vs. 152332
 test-armhf-armhf-xl-multivcpu 12 debian-install  fail REGR. vs. 152332

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds  8 xen-boot fail REGR. vs. 152332

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 152332
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 152332
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 152332
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 152332
 test-amd

Re: [PATCH] xen/Makefile: drop -Werror

2021-07-02 Thread Elliott Mitchell
On Fri, Jul 02, 2021 at 07:51:55PM +0200, Fabrice Fontaine wrote:
> 
> I do agree that setting -Werror is generally perfectly valid for upstream.
> However, for downstream packager, it is generally seen as an issue as
> it will always raise unexepected build failures with older, newer, or
> exotic toolchains, see
> https://embeddedartistry.com/blog/2017/05/22/werror-is-not-your-friend.
> It would be good to, at least, have an option to disable -Werror for
> example through a XEN_DISABLE_WERROR.

Two people don't make it a majority opinion, but if this was a meeting
this opinion would get a second.

I don't know where everyone is on the spectrum, but I also strongly
dislike -Werror yet do like -Wall and tend to get rid of warnings.
-Werror is good for continuous integration systems, not so great for
releases or active development.

-Werror kind of seems like Stack Ranking, good for use during brief
periods, but poor for long term continuous use.


-- 
(\___(\___(\__  --=> 8-) EHM <=--  __/)___/)___/)
 \BS (| ehem+sig...@m5p.com  PGP 87145445 |)   /
  \_CS\   |  _  -O #include  O-   _  |   /  _/
8A19\___\_|_/58D2 7E3D DDF4 7BA6 <-PGP-> 41D1 B375 37D0 8714\_|_/___/5445





[PATCH] tools/libxenguest: Fix max_extd_leaf calculation for legacy restore

2021-07-02 Thread Andrew Cooper
0x1c is lower than any value which will actually be observed in
p->extd.max_leaf, but higher than the logical 9 leaves worth of extended data
on Intel systems, causing x86_cpuid_copy_to_buffer() to fail with -ENOBUFS.

Correct the calculation.

The problem was first noticed in c/s 34990446ca9 "libxl: don't ignore the
return value from xc_cpuid_apply_policy" but introduced earlier.

Fixes: 34990446ca91 ("libxl: don't ignore the return value from 
xc_cpuid_apply_policy")
Reported-by: Olaf Hering 
Signed-off-by: Andrew Cooper 
---
CC: Jan Beulich 
CC: Roger Pau Monné 
CC: Wei Liu 
CC: Ian Jackson 
CC: Olaf Hering 

Olaf - as I've changed the fix, I haven't included your T-by tag, but I'm
confident that this will suitably address the issue.
---
 tools/libs/guest/xg_cpuid_x86.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libs/guest/xg_cpuid_x86.c b/tools/libs/guest/xg_cpuid_x86.c
index e01d657e0394..0c9c4fefc1ef 100644
--- a/tools/libs/guest/xg_cpuid_x86.c
+++ b/tools/libs/guest/xg_cpuid_x86.c
@@ -513,7 +513,7 @@ int xc_cpuid_apply_policy(xc_interface *xch, uint32_t 
domid, bool restore,
 /* Clamp maximum leaves to the ones supported on 4.12. */
 p->basic.max_leaf = min(p->basic.max_leaf, 0xdu);
 p->feat.max_subleaf = 0;
-p->extd.max_leaf = min(p->extd.max_leaf, 0x1cu);
+p->extd.max_leaf = min(p->extd.max_leaf, 0x801c);
 }
 
 if ( featureset )
-- 
2.11.0




[PATCH] tools/libxenguest: Fix migration's debug option

2021-07-02 Thread Andrew Cooper
The code has gone through many refactors, but the first refactor was the one
which broke it by inverting the check with respect to checkpointed streams.

Fixes: 7449fb36c6c8 ("migration/save: pass checkpointed_stream from libxl to 
libxc")
Reported-by: Olaf Hering 
Signed-off-by: Andrew Cooper 
---
CC: Ian Jackson 
CC: Wei Liu 
CC: Olaf Hering 

`xl migrate --debug` might not be perfect, but this at least brings it back to
mostly working.

I don't think dropping it is a sensible move.  In particular, it is invaluable
for testing the logdirty infrastructure when migrating a memtest VM.

If anyone has a clever idea to fix the grant problem, then we can.  It is
after all a debug option, without any specific prescribed behaviour.
---
 tools/libs/guest/xg_sr_save.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/libs/guest/xg_sr_save.c b/tools/libs/guest/xg_sr_save.c
index 2ba7c3200cd5..f0e2bd048d37 100644
--- a/tools/libs/guest/xg_sr_save.c
+++ b/tools/libs/guest/xg_sr_save.c
@@ -752,7 +752,7 @@ static int send_domain_memory_live(struct xc_sr_context 
*ctx)
 if ( rc )
 goto out;
 
-if ( ctx->save.debug && ctx->stream_type != XC_STREAM_PLAIN )
+if ( ctx->save.debug && ctx->stream_type == XC_STREAM_PLAIN )
 {
 rc = verify_frames(ctx);
 if ( rc )
-- 
2.11.0




Re: [PATCH v20210701 10/40] tools: add xc_is_known_page_type to libxenctrl

2021-07-02 Thread Andrew Cooper
On 01/07/2021 10:56, Olaf Hering wrote:
> Users of xc_get_pfn_type_batch may want to sanity check the data
> returned by Xen. Add a simple helper for this purpose.
>
> Signed-off-by: Olaf Hering 

Subject needs correcting after v2.

However, given that this is in the save/restore common header, does it
really need a prefix?  Simply is_known_page_type() seems good enough.

>
> v02:
> - rename xc_is_known_page_type to sr_is_known_page_type
> - move from ctrl/xc_private.h to saverestore/common.h (jgross)
> ---
>  tools/libs/saverestore/common.h | 33 +
>  1 file changed, 33 insertions(+)
>
> diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h
> index ca2eb47a4f..07c506360c 100644
> --- a/tools/libs/saverestore/common.h
> +++ b/tools/libs/saverestore/common.h
> @@ -467,6 +467,39 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned 
> int count,
>  /* Handle a STATIC_DATA_END record. */
>  int handle_static_data_end(struct xc_sr_context *ctx);
>  
> +/* Sanitiy check for types returned by Xen */
> +static inline bool sr_is_known_page_type(xen_pfn_t type)

uint32_t

> +{
> +bool ret;

The logic will be rather shorter and cleaner to read by dropping ret and
using return directly out of the switch.

> +
> +switch (type)

Spaces.

I can fix up everything on commit if you're happy with the suggestions.

~Andrew



Re: [PATCH v20210701 11/40] tools: use sr_is_known_page_type

2021-07-02 Thread Andrew Cooper
On 01/07/2021 10:56, Olaf Hering wrote:
> Verify pfn type on sending side, also verify incoming batch of pfns.
>
> Signed-off-by: Olaf Hering 
> Reviewed-by: Juergen Gross 

Any reason this isn't folded into the previous patch, like your
subsequent two page type helper patches are?

> diff --git a/tools/libs/saverestore/save.c b/tools/libs/saverestore/save.c
> index ae3e8797d0..6f820ea432 100644
> --- a/tools/libs/saverestore/save.c
> +++ b/tools/libs/saverestore/save.c
> @@ -147,6 +147,12 @@ static int write_batch(struct xc_sr_context *ctx)
>  
>  for ( i = 0; i < nr_pfns; ++i )
>  {
> +if ( sr_is_known_page_type(types[i]) == false )
> +{
> +ERROR("Wrong type %#"PRIpfn" for pfn %#"PRIpfn, types[i], 
> mfns[i]);

"Unknown type" would be more accurate.

~Andrew




Re: [PATCH v20210701 12/40] tools: unify type checking for data pfns in migration stream

2021-07-02 Thread Andrew Cooper
On 01/07/2021 10:56, Olaf Hering wrote:
> Introduce a helper which decides if a given pfn in the migration
> stream is backed by memory.
>
> This specifically deals with type XEN_DOMCTL_PFINFO_XALLOC, which was
> a synthetic toolstack-only type used in Xen 4.2 to 4.5. It indicated a
> dirty page on the sending side for which no data will be send in the
> initial iteration.
>
> No change in behavior intended.
>
> Signed-off-by: Olaf Hering 
> ---
>  tools/libs/saverestore/common.h  | 17 +
>  tools/libs/saverestore/restore.c |  5 ++---
>  2 files changed, 19 insertions(+), 3 deletions(-)
>
> diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h
> index 07c506360c..fa242e808d 100644
> --- a/tools/libs/saverestore/common.h
> +++ b/tools/libs/saverestore/common.h
> @@ -500,6 +500,23 @@ static inline bool sr_is_known_page_type(xen_pfn_t type)
>  return ret;
>  }
>  
> +static inline bool page_type_to_populate(uint32_t type)
> +{
> +bool ret;
> +
> +switch (type)
> +{

Same style comments as before.

> +case XEN_DOMCTL_PFINFO_XTAB:
> +case XEN_DOMCTL_PFINFO_BROKEN:
> +ret = false;
> +break;
> +case XEN_DOMCTL_PFINFO_XALLOC:
> +default:
> +ret = true;
> +break;

I know you're replacing the logic as-was, but in hindsight, I'm not sure
it was great to begin with.  It defaults the unallocated types to being
considered populated, which isn't a clever idea.

Anyone adding a new page type is going to have to audit/edit each of
these helpers.  I think it would be better to write all the true cases
explicitly.

> +}
> +return ret;
> +}
>  #endif
>  /*
>   * Local variables:
> diff --git a/tools/libs/saverestore/restore.c 
> b/tools/libs/saverestore/restore.c
> index 324b9050e2..477b7527a1 100644
> --- a/tools/libs/saverestore/restore.c
> +++ b/tools/libs/saverestore/restore.c
> @@ -152,9 +152,8 @@ int populate_pfns(struct xc_sr_context *ctx, unsigned int 
> count,
>  
>  for ( i = 0; i < count; ++i )
>  {
> -if ( (!types || (types &&
> - (types[i] != XEN_DOMCTL_PFINFO_XTAB &&
> -  types[i] != XEN_DOMCTL_PFINFO_BROKEN))) &&
> +if ( (!types ||
> +  (types && page_type_to_populate(types[i]) == true)) &&

I'm surprised not to have seen a compiler or static analysis complaint
about this.

!A || (A && B) is redundant, and simplifies to !A || B.

Clearly need to blame whichever numpty wrote this code to begin with.

~Andrew




Re: [PATCH v20210701 13/40] tools: unify type checking for data pfns in migration stream

2021-07-02 Thread Andrew Cooper
On 01/07/2021 10:56, Olaf Hering wrote:
> diff --git a/tools/libs/saverestore/common.h b/tools/libs/saverestore/common.h
> index fa242e808d..905b4078f6 100644
> --- a/tools/libs/saverestore/common.h
> +++ b/tools/libs/saverestore/common.h
> @@ -517,6 +517,24 @@ static inline bool page_type_to_populate(uint32_t type)
>  }
>  return ret;
>  }
> +
> +static inline bool page_type_has_stream_data(uint32_t type)
> +{
> +bool ret;
> +
> +switch (type)
> +{
> +case XEN_DOMCTL_PFINFO_BROKEN:
> +case XEN_DOMCTL_PFINFO_XALLOC:
> +case XEN_DOMCTL_PFINFO_XTAB:
> +ret = false;
> +break;
> +default:
> +ret = true;
> +break;

As with page_type_to_populate(), we shouldn't really default the
unallocated types to having stream data.

Subject to this and the other style concerned, Reviewed-by: Andrew
Cooper 

I'm happy to fix up all the issue for the page type helpers on commit,
if you're happy.

~Andrew




Re: [PATCH v6 9/9] docs/doxygen: doxygen documentation for grant_table.h

2021-07-02 Thread Stefano Stabellini
On Fri, 2 Jul 2021, Luca Fancellu wrote:
> > On 1 Jul 2021, at 18:44, Stefano Stabellini  wrote:
> > 
> > On Thu, 1 Jul 2021, Luca Fancellu wrote:
> >> Hi Stefano,
> >> 
> >>> On 24 Jun 2021, at 00:34, Stefano Stabellini  
> >>> wrote:
> >>> 
> >>> On Mon, 10 May 2021, Luca Fancellu wrote:
>  Modification to include/public/grant_table.h:
>  
>  1) Add doxygen tags to:
>  - Create Grant tables section
>  - include variables in the generated documentation
>  - Used @keepindent/@endkeepindent to enclose comment
>   section that are indented using spaces, to keep
>   the indentation.
>  2) Add .rst file for grant table for Arm64
> >>> 
> >>> Why only arm64?
> >> 
> >> This is a mistake, it should be just “Add .rst file for grant table"
> >> 
> >>> 
> >>> 
>  Signed-off-by: Luca Fancellu 
>  ---
>  v6 changes:
>  - Fix misaligned comment
>  - Moved comments to make them display in the docs
>  - Included more documentation in the docs
>  (see output here: 
>  https://luca.fancellu.gitlab.io/xen-docs/hypercall-interfaces/common/grant_tables.html)
> >>> 
> >>> It looks much much better. All the info we care about seems to be there.
> >>> The only things that I noticed missing and might be good to keep is the
> >>> small comment about HYPERVISOR_grant_table_op:
> >>> 
> >>> /* ` enum neg_errnoval
> >>> * ` HYPERVISOR_grant_table_op(enum grant_table_op cmd,
> >>> * `   void *args,
> >>> * `   unsigned int count)
> >>> * `
> >>> *
> >>> * @args points to an array of a per-command data structure. The array
> >>> * has @count members
> >> 
> >> Where do you want me to put this comment in the html page? In the end of 
> >> the description in the top of the page?
> > 
> > Yeah, that looks like a good place
> 
> Great, for a preview, have a look on this: 
> https://luca.fancellu.gitlab.io/xen-docs/hypercall-interfaces/common/grant_tables.html

Looks good!

[xen-unstable-smoke test] 163230: tolerable all pass - PUSHED

2021-07-02 Thread osstest service owner
flight 163230 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/163230/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  74d044d51b19bb697eac5c3deafa140f6afafec8
baseline version:
 xen  3bc3be978fd61f8099797864136c5f447c0e4aae

Last test of basis   163225  2021-07-02 15:01:35 Z0 days
Testing same since   163230  2021-07-02 18:01:38 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Olaf Hering 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   3bc3be978f..74d044d51b  74d044d51b19bb697eac5c3deafa140f6afafec8 -> smoke



Re: [PATCH v6 5/9] docs: add checks to configure for sphinx and doxygen

2021-07-02 Thread Stefano Stabellini
On Mon, 10 May 2021, Luca Fancellu wrote:
> Add checks in the configure files to see if the system
> is capable of generate the sphinx html docs using
^ of generating

> doxygen and sphinx-breathe tools.

I take you updated the configure script but running autoconf again,
right?

Assuming that is the case:

Acked-by: Stefano Stabellini 


> Signed-off-by: Luca Fancellu 
> ---
>  config/Docs.mk.in |   2 +
>  docs/configure| 258 ++
>  docs/configure.ac |  15 +++
>  3 files changed, 275 insertions(+)
> 
> diff --git a/config/Docs.mk.in b/config/Docs.mk.in
> index e76e5cd5ff..dfd4a02838 100644
> --- a/config/Docs.mk.in
> +++ b/config/Docs.mk.in
> @@ -7,3 +7,5 @@ POD2HTML:= @POD2HTML@
>  POD2TEXT:= @POD2TEXT@
>  PANDOC  := @PANDOC@
>  PERL:= @PERL@
> +SPHINXBUILD := @SPHINXBUILD@
> +DOXYGEN := @DOXYGEN@
> diff --git a/docs/configure b/docs/configure
> index 569bd4c2ff..0ebf046a79 100755
> --- a/docs/configure
> +++ b/docs/configure
> @@ -588,6 +588,8 @@ ac_unique_file="misc/xen-command-line.pandoc"
>  ac_subst_vars='LTLIBOBJS
>  LIBOBJS
>  PERL
> +DOXYGEN
> +SPHINXBUILD
>  PANDOC
>  POD2TEXT
>  POD2HTML
> @@ -673,6 +675,7 @@ POD2MAN
>  POD2HTML
>  POD2TEXT
>  PANDOC
> +DOXYGEN
>  PERL'
>  
>  
> @@ -1318,6 +1321,7 @@ Some influential environment variables:
>POD2HTMLPath to pod2html tool
>POD2TEXTPath to pod2text tool
>PANDOC  Path to pandoc tool
> +  DOXYGEN Path to doxygen tool
>PERLPath to Perl parser
>  
>  Use these variables to override the choices made by `configure' or to help
> @@ -1800,6 +1804,7 @@ ac_configure="$SHELL $ac_aux_dir/configure"  # Please 
> don't use this var.
>  
>  
>  
> +
>  case "$host_os" in
>  *freebsd*) XENSTORED_KVA=/dev/xen/xenstored ;;
>  *) XENSTORED_KVA=/proc/xen/xsd_kva ;;
> @@ -1812,6 +1817,53 @@ case "$host_os" in
>  esac
>  
>  
> +# ===
> +# https://www.gnu.org/software/autoconf-archive/ax_python_module.html
> +# ===
> +#
> +# SYNOPSIS
> +#
> +#   AX_PYTHON_MODULE(modname[, fatal, python])
> +#
> +# DESCRIPTION
> +#
> +#   Checks for Python module.
> +#
> +#   If fatal is non-empty then absence of a module will trigger an error.
> +#   The third parameter can either be "python" for Python 2 or "python3" for
> +#   Python 3; defaults to Python 3.
> +#
> +# LICENSE
> +#
> +#   Copyright (c) 2008 Andrew Collier
> +#
> +#   Copying and distribution of this file, with or without modification, are
> +#   permitted in any medium without royalty provided the copyright notice
> +#   and this notice are preserved. This file is offered as-is, without any
> +#   warranty.
> +
> +#serial 9
> +
> +# This is what autoupdate's m4 run will expand.  It fires
> +# the warning (with _au_warn_XXX), outputs it into the
> +# updated configure.ac (with AC_DIAGNOSE), and then outputs
> +# the replacement expansion.
> +
> +
> +# This is an auxiliary macro that is also run when
> +# autoupdate runs m4.  It simply calls m4_warning, but
> +# we need a wrapper so that each warning is emitted only
> +# once.  We break the quoting in m4_warning's argument in
> +# order to expand this macro's arguments, not AU_DEFUN's.
> +
> +
> +# Finally, this is the expansion that is picked up by
> +# autoconf.  It tells the user to run autoupdate, and
> +# then outputs the replacement expansion.  We do not care
> +# about autoupdate's warning because that contains
> +# information on what to do *after* running autoupdate.
>  
>  
>  test "x$prefix" = "xNONE" && prefix=$ac_default_prefix
> @@ -2232,6 +2284,212 @@ $as_echo "$as_me: WARNING: pandoc is not available so 
> some documentation won't b
>  fi
>  
>  
> +# If sphinx is installed, make sure to have also the dependencies to build
> +# Sphinx documentation.
> +for ac_prog in sphinx-build
> +do
> +  # Extract the first word of "$ac_prog", so it can be a program name with 
> args.
> +set dummy $ac_prog; ac_word=$2
> +{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for $ac_word" >&5
> +$as_echo_n "checking for $ac_word... " >&6; }
> +if ${ac_cv_prog_SPHINXBUILD+:} false; then :
> +  $as_echo_n "(cached) " >&6
> +else
> +  if test -n "$SPHINXBUILD"; then
> +  ac_cv_prog_SPHINXBUILD="$SPHINXBUILD" # Let the user override the test.
> +else
> +as_save_IFS=$IFS; IFS=$PATH_SEPARATOR
> +for as_dir in $PATH
> +do
> +  IFS=$as_save_IFS
> +  test -z "$as_dir" && as_dir=.
> +for ac_exec_ext in '' $ac_executable_extensions; do
> +  if as_fn_executable_p "$as_dir/$ac_word$ac_exec_ext"; then
> +ac_cv_prog_SPHINXBUILD="$ac_prog"
> +$as_echo "$as_me:${as_lineno-$LINENO}: found 
> $as_dir/$ac_word$ac_exec_ext" >&5
> +break 2
> +  fi
> +done
> +  done
> +IFS=$as_save_IFS
> +
> +fi
> +fi
> +SPHINXBUILD=$ac_cv_prog_SPHINXBUILD
> +if test -n "$SPHINXBUIL

Re: [PATCH v6 4/9] m4/python: add function to docs_tool.m4 and new m4 module

2021-07-02 Thread Stefano Stabellini
On Mon, 10 May 2021, Luca Fancellu wrote:
> Add ax_python_module.m4 to have a way to check if
> a python module is installed in the system.
> 
> Add a function to docs_tool.m4 to throw an error if the
> required docs tool is missing.
> 
> Signed-off-by: Luca Fancellu 

Acked-by: Stefano Stabellini 


> ---
>  m4/ax_python_module.m4 | 56 ++
>  m4/docs_tool.m4|  9 +++
>  2 files changed, 65 insertions(+)
>  create mode 100644 m4/ax_python_module.m4
> 
> diff --git a/m4/ax_python_module.m4 b/m4/ax_python_module.m4
> new file mode 100644
> index 00..107d88264a
> --- /dev/null
> +++ b/m4/ax_python_module.m4
> @@ -0,0 +1,56 @@
> +# ===
> +# https://www.gnu.org/software/autoconf-archive/ax_python_module.html
> +# ===
> +#
> +# SYNOPSIS
> +#
> +#   AX_PYTHON_MODULE(modname[, fatal, python])
> +#
> +# DESCRIPTION
> +#
> +#   Checks for Python module.
> +#
> +#   If fatal is non-empty then absence of a module will trigger an error.
> +#   The third parameter can either be "python" for Python 2 or "python3" for
> +#   Python 3; defaults to Python 3.
> +#
> +# LICENSE
> +#
> +#   Copyright (c) 2008 Andrew Collier
> +#
> +#   Copying and distribution of this file, with or without modification, are
> +#   permitted in any medium without royalty provided the copyright notice
> +#   and this notice are preserved. This file is offered as-is, without any
> +#   warranty.
> +
> +#serial 9
> +
> +AU_ALIAS([AC_PYTHON_MODULE], [AX_PYTHON_MODULE])
> +AC_DEFUN([AX_PYTHON_MODULE],[
> +if test -z $PYTHON;
> +then
> +if test -z "$3";
> +then
> +PYTHON="python3"
> +else
> +PYTHON="$3"
> +fi
> +fi
> +PYTHON_NAME=`basename $PYTHON`
> +AC_MSG_CHECKING($PYTHON_NAME module: $1)
> +$PYTHON -c "import $1" 2>/dev/null
> +if test $? -eq 0;
> +then
> +AC_MSG_RESULT(yes)
> +eval AS_TR_CPP(HAVE_PYMOD_$1)=yes
> +else
> +AC_MSG_RESULT(no)
> +eval AS_TR_CPP(HAVE_PYMOD_$1)=no
> +#
> +if test -n "$2"
> +then
> +AC_MSG_ERROR(failed to find required module $1)
> +exit 1
> +fi
> +fi
> +])
> \ No newline at end of file
> diff --git a/m4/docs_tool.m4 b/m4/docs_tool.m4
> index 3e8814ac8d..39aa348026 100644
> --- a/m4/docs_tool.m4
> +++ b/m4/docs_tool.m4
> @@ -15,3 +15,12 @@ dnl
>  AC_MSG_WARN([$2 is not available so some documentation won't be 
> built])
>  ])
>  ])
> +
> +AC_DEFUN([AX_DOCS_TOOL_REQ_PROG], [
> +dnl
> +AC_ARG_VAR([$1], [Path to $2 tool])
> +AC_PATH_PROG([$1], [$2])
> +AS_IF([! test -x "$ac_cv_path_$1"], [
> +AC_MSG_ERROR([$2 is needed])
> +])
> +])
> \ No newline at end of file
> -- 
> 2.17.1
> 
> 



Re: [PATCH v6 7/9] docs: Change Makefile and sphinx configuration for doxygen

2021-07-02 Thread Stefano Stabellini
On Fri, 2 Jul 2021, Luca Fancellu wrote:
> > On 1 Jul 2021, at 18:43, Stefano Stabellini  wrote:
> > 
> > On Thu, 1 Jul 2021, Luca Fancellu wrote:
> >>> On 24 Jun 2021, at 00:33, Stefano Stabellini  
> >>> wrote:
> >>> 
> >>> On Mon, 10 May 2021, Luca Fancellu wrote:
>  Modify docs/Makefile to call doxygen and generate sphinx
>  html documentation given the doxygen XML output.
>  
>  Modify docs/conf.py sphinx configuration file to setup
>  the breathe extension that works as bridge between
>  sphinx and doxygen.
>  
>  Add some files to the .gitignore to ignore some
>  generated files for doxygen.
>  
>  Signed-off-by: Luca Fancellu 
>  ---
>  .gitignore|  6 ++
>  docs/Makefile | 42 +++---
>  docs/conf.py  | 48 +---
>  3 files changed, 90 insertions(+), 6 deletions(-)
>  
>  diff --git a/.gitignore b/.gitignore
>  index 1c2fa1530b..d271e0ce6a 100644
>  --- a/.gitignore
>  +++ b/.gitignore
>  @@ -58,6 +58,12 @@ docs/man7/
>  docs/man8/
>  docs/pdf/
>  docs/txt/
>  +docs/doxygen-output
>  +docs/sphinx
>  +docs/xen.doxyfile
>  +docs/xen.doxyfile.tmp
>  +docs/xen-doxygen/doxygen_include.h
>  +docs/xen-doxygen/doxygen_include.h.tmp
>  extras/mini-os*
>  install/*
>  stubdom/*-minios-config.mk
>  diff --git a/docs/Makefile b/docs/Makefile
>  index 8de1efb6f5..2f784c36ce 100644
>  --- a/docs/Makefile
>  +++ b/docs/Makefile
>  @@ -17,6 +17,18 @@ TXTSRC-y := $(sort $(shell find misc -name '*.txt' 
>  -print))
>  
>  PANDOCSRC-y := $(sort $(shell find designs/ features/ misc/ process/ 
>  specs/ \( -name '*.pandoc' -o -name '*.md' \) -print))
>  
>  +# Directory in which the doxygen documentation is created
>  +# This must be kept in sync with breathe_projects value in conf.py
>  +DOXYGEN_OUTPUT = doxygen-output
>  +
>  +# Doxygen input headers from xen-doxygen/doxy_input.list file
>  +DOXY_LIST_SOURCES != cat "xen-doxygen/doxy_input.list"
>  +DOXY_LIST_SOURCES := $(realpath $(addprefix 
>  $(XEN_ROOT)/,$(DOXY_LIST_SOURCES)))
> >> 
> >> Hi Stefano,
> >> 
> >>> 
> >>> I cannot find exactly who is populating doxy_input.list. I can see it is
> >>> empty in patch #6. Does it get populated during the build?
> >> 
> >> doxy_input.list is the only file that should be modified by the developer 
> >> when he/she wants to add documentation
> >> for a new file to be parsed by Doxygen, in my patch about documenting 
> >> grant_tables.h you can see I add
> >> there the path “xen/include/public/grant_table.h"
> > 
> > OK, thanks. I missed that addition.
> > 
> > 
> >>> 
>  +DOXY_DEPS := xen.doxyfile \
>  + xen-doxygen/mainpage.md \
>  + xen-doxygen/doxygen_include.h
>  +
>  # Documentation targets
>  $(foreach i,$(MAN_SECTIONS), \
>   $(eval DOC_MAN$(i) := $(patsubst man/%.$(i),man$(i)/%.$(i), \
>  @@ -46,8 +58,28 @@ all: build
>  build: html txt pdf man-pages figs
>  
>  .PHONY: sphinx-html
>  -sphinx-html:
>  -sphinx-build -b html . sphinx/html
>  +sphinx-html: $(DOXY_DEPS) $(DOXY_LIST_SOURCES)
>  +ifneq ($(SPHINXBUILD),no)
> >>> 
> >>> This check on SPHINXBUILD is new, it wasn't there before. Why do we need
> >>> it now? We are not really changing anything in regards to Sphinx, just
> >>> adding Doxygen support. Or was it a mistake that it was missing even
> >>> before this patch?
> >> 
> >> Yes this is new, I saw that we didn’t look if sphinx was installed in the 
> >> system, so now we did
> > 
> > In that case, I think anything related to SPHINXBUILD and whether sphinx
> > is installed or not, should be a separate patch at the beginning of the
> > series. It could be committed independently before the rest of the
> > series. When we get to this patch, SPHINXBUILD should be already there.
> 
> I’ve introduced SPHINXBUILD in this patch: [PATCH v6 5/9] docs: add checks to 
> configure for sphinx and doxygen,
> In your commend do you mean that you would like it to be outside this serie 
> and this serie to be based on top of that one?

I totally missed patches 4 and 5. Can you please CC me to the whole
series next time?

I meant as a separate patch, like you have done in patch #5. It doesn't
necessarily need to be at the beginning of the series so what you have
already done is OK.

[ovmf test] 163229: regressions - FAIL

2021-07-02 Thread osstest service owner
flight 163229 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/163229/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 162359
 test-amd64-amd64-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 
162359

version targeted for testing:
 ovmf fea7901dba72eeac526f3ef12a4ad4c539622373
baseline version:
 ovmf c410ad4da4b7785170d3d42a3ba190c2caac6feb

Last test of basis   162359  2021-06-04 03:40:08 Z   28 days
Failing since162368  2021-06-04 15:42:59 Z   28 days   76 attempts
Testing same since   163216  2021-07-01 22:42:29 Z0 days6 attempts


People who touched revisions under test:
  Abner Chang 
  Agrawal, Sachin 
  Alexandru Elisei 
  Anthony PERARD 
  Ard Biesheuvel 
  Daniel Schaefer 
  Daoxiang Li 
  Dov Murik 
  DunTan 
  gaoliming 
  Guo Dong 
  Hao A Wu 
  Jian J Wang 
  Kaaira Gupta 
  Ken Lautner 
  Kenneth Lautner 
  Kun Qin 
  Laszlo Ersek 
  Leif Lindholm 
  Liming Gao 
  Loo Tung Lun 
  Loo, Tung Lun 
  Manickavasakam Karpagavinayagam 
  Maurice Ma 
  Ni, Ray 
  Patrick Rudolph 
  Pierre Gondois 
  Ray Ni 
  Rebecca Cran 
  Rebecca Cran 
  Sachin Agrawal 
  Sami Mujawar 
  Scottie Kuo 
  Sean Brogan 
  Sean Brogan 
  Sumana Venur 
  Sunil V L 
  xueshengfeng 
  Zhiguang Liu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3224 lines long.)



[qemu-mainline test] 163228: regressions - FAIL

2021-07-02 Thread osstest service owner
flight 163228 qemu-mainline real [real]
flight 163234 qemu-mainline real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/163228/
http://logs.test-lab.xenproject.org/osstest/logs/163234/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-qemuu-nested-amd 16 xen-boot/l1 fail REGR. vs. 152631
 test-amd64-amd64-qemuu-nested-intel 16 xen-boot/l1   fail REGR. vs. 152631

Tests which are failing intermittently (not blocking):
 test-amd64-i386-xl-xsm20 guest-localmigrate/x10 fail pass in 163234-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-xl-rtds 19 guest-start.2   fail blocked in 152631
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 152631
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 152631
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 152631
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 152631
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 152631
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 152631
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass

version targeted for testing:
 qemuu9c2647f75004c4f7d64c9c0ec55f8c6f0739a8b1
baseline version:
 qemuu1d806cef0e38b5db8347a8e12f214d543204a314

Last test of basis   152631  2020-08-20 09:07:46 Z  316 days
Failing since152659  2020-08-21 14:07:39 Z  315 days  579 attempts
Testing same since   163228  2021-07-02 17:08:27 Z0 days1 attempts


554 people touched revisions under test,
not listing them all

jobs:
 build-amd64-xsm 

Re: [PATCH v15 06/12] swiotlb: Use is_swiotlb_force_bounce for swiotlb data bouncing

2021-07-02 Thread Nathan Chancellor
Hi Will and Robin,

On Fri, Jul 02, 2021 at 04:13:50PM +0100, Robin Murphy wrote:
> On 2021-07-02 14:58, Will Deacon wrote:
> > Hi Nathan,
> > 
> > On Thu, Jul 01, 2021 at 12:52:20AM -0700, Nathan Chancellor wrote:
> > > On 7/1/2021 12:40 AM, Will Deacon wrote:
> > > > On Wed, Jun 30, 2021 at 08:56:51AM -0700, Nathan Chancellor wrote:
> > > > > On Wed, Jun 30, 2021 at 12:43:48PM +0100, Will Deacon wrote:
> > > > > > On Wed, Jun 30, 2021 at 05:17:27PM +0800, Claire Chang wrote:
> > > > > > > `BUG: unable to handle page fault for address: 003a8290` 
> > > > > > > and
> > > > > > > the fact it crashed at `_raw_spin_lock_irqsave` look like the 
> > > > > > > memory
> > > > > > > (maybe dev->dma_io_tlb_mem) was corrupted?
> > > > > > > The dev->dma_io_tlb_mem should be set here
> > > > > > > (https://git.kernel.org/pub/scm/linux/kernel/git/next/linux-next.git/tree/drivers/pci/probe.c#n2528)
> > > > > > > through device_initialize.
> > > > > > 
> > > > > > I'm less sure about this. 'dma_io_tlb_mem' should be pointing at
> > > > > > 'io_tlb_default_mem', which is a page-aligned allocation from 
> > > > > > memblock.
> > > > > > The spinlock is at offset 0x24 in that structure, and looking at the
> > > > > > register dump from the crash:
> > > > > > 
> > > > > > Jun 29 18:28:42 hp-4300G kernel: RSP: 0018:adb4013db9e8 EFLAGS: 
> > > > > > 00010006
> > > > > > Jun 29 18:28:42 hp-4300G kernel: RAX: 003a8290 RBX: 
> > > > > >  RCX: 8900572ad580
> > > > > > Jun 29 18:28:42 hp-4300G kernel: RDX: 89005653f024 RSI: 
> > > > > > 000c RDI: 1d17
> > > > > > Jun 29 18:28:42 hp-4300G kernel: RBP: 0a20d000 R08: 
> > > > > > 000c R09: 
> > > > > > Jun 29 18:28:42 hp-4300G kernel: R10: 0a20d000 R11: 
> > > > > > 89005653f000 R12: 0212
> > > > > > Jun 29 18:28:42 hp-4300G kernel: R13: 1000 R14: 
> > > > > > 0002 R15: 0020
> > > > > > Jun 29 18:28:42 hp-4300G kernel: FS:  7f1f8898ea40() 
> > > > > > GS:89005728() knlGS:
> > > > > > Jun 29 18:28:42 hp-4300G kernel: CS:  0010 DS:  ES:  CR0: 
> > > > > > 80050033
> > > > > > Jun 29 18:28:42 hp-4300G kernel: CR2: 003a8290 CR3: 
> > > > > > 0001020d CR4: 00350ee0
> > > > > > Jun 29 18:28:42 hp-4300G kernel: Call Trace:
> > > > > > Jun 29 18:28:42 hp-4300G kernel:  _raw_spin_lock_irqsave+0x39/0x50
> > > > > > Jun 29 18:28:42 hp-4300G kernel:  swiotlb_tbl_map_single+0x12b/0x4c0
> > > > > > 
> > > > > > Then that correlates with R11 holding the 'dma_io_tlb_mem' pointer 
> > > > > > and
> > > > > > RDX pointing at the spinlock. Yet RAX is holding junk :/
> > > > > > 
> > > > > > I agree that enabling KASAN would be a good idea, but I also think 
> > > > > > we
> > > > > > probably need to get some more information out of 
> > > > > > swiotlb_tbl_map_single()
> > > > > > to see see what exactly is going wrong in there.
> > > > > 
> > > > > I can certainly enable KASAN and if there is any debug print I can add
> > > > > or dump anything, let me know!
> > > > 
> > > > I bit the bullet and took v5.13 with swiotlb/for-linus-5.14 merged in, 
> > > > built
> > > > x86 defconfig and ran it on my laptop. However, it seems to work fine!
> > > > 
> > > > Please can you share your .config?
> > > 
> > > Sure thing, it is attached. It is just Arch Linux's config run through
> > > olddefconfig. The original is below in case you need to diff it.
> > > 
> > > https://raw.githubusercontent.com/archlinux/svntogit-packages/9045405dc835527164f3034b3ceb9a67c7a53cd4/trunk/config
> > > 
> > > If there is anything more that I can provide, please let me know.
> > 
> > I eventually got this booting (for some reason it was causing LD to SEGV
> > trying to link it for a while...) and sadly it works fine on my laptop. Hmm.

Seems like it might be something specific to the amdgpu module?

> > Did you manage to try again with KASAN?

Yes, it took a few times to reproduce the issue but I did manage to get
a dmesg, please find it attached. I build from commit 7d31f1c65cc9 ("swiotlb:
fix implicit debugfs declarations") in Konrad's tree.

> > It might also be worth taking the IOMMU out of the equation, since that
> > interfaces differently with SWIOTLB and I couldn't figure out the code path
> > from the log you provided. What happens if you boot with "amd_iommu=off
> > swiotlb=force"?
> 
> Oh, now there's a thing... the chat from the IOMMU API in the boot log
> implies that the IOMMU *should* be in the picture - we see that default
> domains are IOMMU_DOMAIN_DMA default and the GPU :0c:00.0 was added to a
> group. That means dev->dma_ops should be set and DMA API calls should be
> going through iommu-dma, yet the callstack in the crash says we've gone
> straight from dma_map_page_attrs() to swiotlb_map(), implying the inline
> dma_direct_map_page() path.
> 
> If dev->dma_ops didn't loo

[ovmf test] 163233: regressions - FAIL

2021-07-02 Thread osstest service owner
flight 163233 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/163233/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-i386-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 162359
 test-amd64-amd64-xl-qemuu-ovmf-amd64 12 debian-hvm-install fail REGR. vs. 
162359

version targeted for testing:
 ovmf fea7901dba72eeac526f3ef12a4ad4c539622373
baseline version:
 ovmf c410ad4da4b7785170d3d42a3ba190c2caac6feb

Last test of basis   162359  2021-06-04 03:40:08 Z   29 days
Failing since162368  2021-06-04 15:42:59 Z   28 days   77 attempts
Testing same since   163216  2021-07-01 22:42:29 Z1 days7 attempts


People who touched revisions under test:
  Abner Chang 
  Agrawal, Sachin 
  Alexandru Elisei 
  Anthony PERARD 
  Ard Biesheuvel 
  Daniel Schaefer 
  Daoxiang Li 
  Dov Murik 
  DunTan 
  gaoliming 
  Guo Dong 
  Hao A Wu 
  Jian J Wang 
  Kaaira Gupta 
  Ken Lautner 
  Kenneth Lautner 
  Kun Qin 
  Laszlo Ersek 
  Leif Lindholm 
  Liming Gao 
  Loo Tung Lun 
  Loo, Tung Lun 
  Manickavasakam Karpagavinayagam 
  Maurice Ma 
  Ni, Ray 
  Patrick Rudolph 
  Pierre Gondois 
  Ray Ni 
  Rebecca Cran 
  Rebecca Cran 
  Sachin Agrawal 
  Sami Mujawar 
  Scottie Kuo 
  Sean Brogan 
  Sean Brogan 
  Sumana Venur 
  Sunil V L 
  xueshengfeng 
  Zhiguang Liu 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 fail
 test-amd64-i386-xl-qemuu-ovmf-amd64  fail



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Not pushing.

(No revision log; it would be 3224 lines long.)



[xen-unstable test] 163231: regressions - FAIL

2021-07-02 Thread osstest service owner
flight 163231 xen-unstable real [real]
flight 163237 xen-unstable real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/163231/
http://logs.test-lab.xenproject.org/osstest/logs/163237/

Regressions :-(

Tests which did not succeed and are blocking,
including tests which could not be run:
 test-amd64-amd64-examine  4 memdisk-try-append   fail REGR. vs. 163219

Regressions which are regarded as allowable (not blocking):
 test-armhf-armhf-xl-rtds18 guest-start/debian.repeat fail REGR. vs. 163219

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 163219
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 163219
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 163219
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 163219
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 163219
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 163219
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 163219
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 163219
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 163219
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 163219
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 163219
 test-arm64-arm64-xl-seattle  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-seattle  16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-cubietruck 15 migrate-support-checkfail never pass
 test-armhf-armhf-xl-cubietruck 16 saverestore-support-checkfail never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass

version targeted for testing:
 xen  3bc3be978fd61f8099797864136c5f447c0e4aae
baseline version:
 xen  f95b7b37cfc6d4613721df9357090d14712013c0

Last test of basis   163219  2021-07-02 01:52:38 Z1 days
Testing same since   163231  2021-07-02 18:06:46 Z0 days1 attempts

---