Re: [PATCH] docs/misra: add R11.1 R11.2 R11.3 R11.6

[Jan Beulich]

On 14.11.2023 23:37, Stefano Stabellini wrote:
> [PATCH v2] docs/misra: add R11.1 R11.2 R11.3 R11.6
> 
> Add MISRA C Rules 11.1, 11.2, 11.3, 11.6 as discussed.
> 
> Explicitly add in the notes that conversions to integer types are
> permitted if the destination type has enough bits to hold the entire
> value. GCC gives enough guarantees in terms of preserving the bit
> content in such situations.
> 
> Also allow for bool conversions (e.g. to check if a function point is
> valid).
> 
> Signed-off-by: Stefano Stabellini 

Acked-by: Jan Beulich 

Re: [PATCH 7/7] xen/events: remove some info_for_irq() calls in pirq handling

[Juergen Gross]

On 14.11.23 19:16, Oleksandr Tyshchenko wrote:



On 16.10.23 09:28, Juergen Gross wrote:

Hello Juergen



Instead of the IRQ number user the struct irq_info pointer as parameter
in the internal pirq related functions. This allows to drop some calls
of info_for_irq().

Signed-off-by: Juergen Gross 



Looks good, so

Reviewed-by: Oleksandr Tyshchenko 


Just one NIT below ...


[snip]

   
-static void pirq_query_unmask(int irq)

+static void pirq_query_unmask(struct irq_info *info)
   {
struct physdev_irq_status_query irq_status;
-   struct irq_info *info = info_for_irq(irq);
   
   	BUG_ON(info->type != IRQT_PIRQ);
   
-	irq_status.irq = pirq_from_irq(irq);

+   irq_status.irq = info->u.pirq.pirq;



   ... what is the reason to open-code pirq_from_irq() here?
For example, __startup_pirq() continues to use helper in almost the same
situation ...


Good catch. I'll change that, especially as it allows to remove the BUG_ON()
which is in the helper already.


Juergen


OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

[ovmf test] 183758: all pass - PUSHED

[osstest service owner]

flight 183758 ovmf real [real]
http://logs.test-lab.xenproject.org/osstest/logs/183758/

Perfect :-)
All tests in this flight passed as required
version targeted for testing:
 ovmf 3db76e6476e493d3cda45b81bba99a645180cf35
baseline version:
 ovmf c9cce5a00591dc437c258f0365b5d434c628b1e0

Last test of basis   183741  2023-11-13 05:42:55 Z2 days
Testing same since   183758  2023-11-15 04:11:08 Z0 days1 attempts


People who touched revisions under test:
  Igor Kulchytskyy 

jobs:
 build-amd64-xsm  pass
 build-i386-xsm   pass
 build-amd64  pass
 build-i386   pass
 build-amd64-libvirt  pass
 build-i386-libvirt   pass
 build-amd64-pvopspass
 build-i386-pvops pass
 test-amd64-amd64-xl-qemuu-ovmf-amd64 pass
 test-amd64-i386-xl-qemuu-ovmf-amd64  pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/osstest/ovmf.git
   c9cce5a005..3db76e6476  3db76e6476e493d3cda45b81bba99a645180cf35 -> 
xen-tested-master

Re: [PATCH v2 28/29] tools/xenstored: support complete log capabilities in stubdom

[Juergen Gross]

On 14.11.23 21:57, Julien Grall wrote:

Hi Juergen,

On 14/11/2023 09:20, Juergen Gross wrote:

On 14.11.23 10:05, Julien Grall wrote:

Hi,

On 14/11/2023 06:45, Juergen Gross wrote:

On 13.11.23 23:40, Julien Grall wrote:

Hi Juergen,

On 10/11/2023 16:08, Juergen Gross wrote:

With 9pfs being fully available in Xenstore-stubdom now, there is no
reason to not fully support all logging capabilities in stubdom.

Open the logfile on stubdom only after the 9pfs file system has been
mounted.

Signed-off-by: Juergen Gross 
Reviewed-by: Jason Andryuk 
---
  tools/hotplug/Linux/launch-xenstore.in |  1 +
  tools/xenstored/control.c  | 30 +-
  tools/xenstored/minios.c   |  3 +++
  3 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/tools/hotplug/Linux/launch-xenstore.in 
b/tools/hotplug/Linux/launch-xenstore.in

index e854ca1eb8..da4eeca7c5 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -98,6 +98,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons && . 
@CONFIG_DIR@/@CONFIG_LEAF

  [ -z "$XENSTORE_DOMAIN_SIZE" ] && XENSTORE_DOMAIN_SIZE=8
  XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --memory 
$XENSTORE_DOMAIN_SIZE"
  [ -z "$XENSTORE_MAX_DOMAIN_SIZE" ] || 
XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --maxmem 
$XENSTORE_MAX_DOMAIN_SIZE"
+    [ -z "$XENSTORED_TRACE" ] || 
XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS -T xenstored-trace.log"


I am probably missing something, but any reason to not use 
@XEN_LOG_DIR@/xenstored-trace.log as we do for xenstored?


Yes. Stubdom has no access to XEN_LOG_DIR.


Ok. This restriction is not that obvious... The documentation in 
sysconfig.xencommons.in implies that it will be written in XEN_LOG_DIR:


## Type: string
## Default: ""
#
# Additional commandline arguments to start xenstored,
# like "--trace-file @XEN_LOG_DIR@/xenstored-trace.log"
# See "@sbindir@/xenstored --help" for possible options.
# Only evaluated if XENSTORETYPE is "daemon".
XENSTORED_ARGS=

Also, we are saying this is only supported when Xenstored is daemonized. So I 
think there are some documentation update necessary in this patch.


This is for the daemon. And the documentation example here is using an
absolute filename, so this is very clear where the trace file will be written.


Ah yes. Sorry I got confused.



For stubdom a related parameter "XENSTORE_DOMAIN_ARGS" is available. I can add
a sentence to the explanation of that parameter describing setting of a
possible trace file specification.


Do you mean in the following comment:

## Type: string
## Default: Not defined, tracing off
#
# Log xenstored messages
# Only evaluated if XENSTORETYPE is "daemon".
#XENSTORED_TRACE=[yes|on|1]

I think here we need to remove the "Only...".


Oh, I missed that one. It needs to be considered in the stubdom case, too.


Juergen


OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

Re: [PATCH v2 27/29] tools/xenstored: add helpers for filename handling

[Juergen Gross]

On 14.11.23 21:53, Julien Grall wrote:

Hi Juergen,

On 14/11/2023 09:26, Juergen Gross wrote:

On 14.11.23 10:10, Julien Grall wrote:

Hi Juergen,

On 14/11/2023 06:45, Juergen Gross wrote:

On 13.11.23 23:25, Julien Grall wrote:

Hi Juergen,

On 10/11/2023 16:08, Juergen Gross wrote:

Add some helpers for handling filenames which might need different
implementations between stubdom and daemon environments:

- expansion of relative filenames (those are not really defined today,
   just expand them to be relative to /var/lib/xen/xenstore)
- expansion of xenstore_daemon_rundir() (used e.g. for saving the state
   file in case of live update - needs to be unchanged in the daemon
   case, but should result in /var/lib/xen/xenstore for stubdom)

Signed-off-by: Juergen Gross 
Reviewed-by: Jason Andryuk 
---
  tools/xenstored/core.c  | 9 -
  tools/xenstored/core.h  | 3 +++
  tools/xenstored/lu_daemon.c | 4 ++--
  tools/xenstored/minios.c    | 5 +
  tools/xenstored/posix.c | 5 +
  5 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 4a9d874f17..77befd24c9 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -158,6 +158,13 @@ void trace_destroy(const void *data, const char *type)
  trace("obj: DESTROY %s %p\n", type, data);
  }
+char *absolute_filename(const void *ctx, const char *filename)


Can the return be const?


I'll have a look.




+{
+    if (filename[0] != '/')
+    return talloc_asprintf(ctx, XENSTORE_LIB_DIR "/%s", filename);


Looking at the rest of patch, you will be using xenstore_rundir(). I wonder 
whether it would not be better to switch to xenstore_rundir() so...



+    return talloc_strdup(ctx, filename);
+}
+
  /**
   * Signal handler for SIGHUP, which requests that the trace log is reopened
   * (in the main loop).  A single byte is written to reopen_log_pipe, to 
awaken

@@ -2983,7 +2990,7 @@ int main(int argc, char *argv[])
  signal(SIGHUP, trigger_reopen_log);
  if (tracefile)
-    tracefile = talloc_strdup(NULL, tracefile);
+    tracefile = absolute_filename(NULL, tracefile);
  #ifndef NO_LIVE_UPDATE
  /* Read state in case of live update. */
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index a0d3592961..51e1dddb22 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -393,6 +393,9 @@ void early_init(void);
  void mount_9pfs(void);
  #endif
+const char *xenstore_rundir(void);
+char *absolute_filename(const void *ctx, const char *filename);
+
  /* Write out the pidfile */
  void write_pidfile(const char *pidfile);
diff --git a/tools/xenstored/lu_daemon.c b/tools/xenstored/lu_daemon.c
index 71bcabadd3..635ab0 100644
--- a/tools/xenstored/lu_daemon.c
+++ b/tools/xenstored/lu_daemon.c
@@ -24,7 +24,7 @@ void lu_get_dump_state(struct lu_dump_state *state)
  state->size = 0;
  state->filename = talloc_asprintf(NULL, "%s/state_dump",
-  xenstore_daemon_rundir());
+  xenstore_rundir());


... call and ...


  if (!state->filename)
  barf("Allocation failure");
@@ -65,7 +65,7 @@ FILE *lu_dump_open(const void *ctx)
  int fd;
  filename = talloc_asprintf(ctx, "%s/state_dump",
-   xenstore_daemon_rundir());
+   xenstore_rundir());


... this one could be replaced with absolute_filename().


No, I don't think this is a good idea.

I don't want the daemon to store trace files specified as relative files
to be stored in /var/run/xen, while I want all files of the stubdom to be
stored under /var/lib/xen.


Why? This is a bit odd to have a different behavior between stubdom and 
daemon. It would be much easier for the user if they knew all the files would 
be at the same place regardless the version used.


The main difference is that stubdom has access to only _one_ directory in dom0.


Would you be able to explain why we can only give access to a single directory? 
Is this because of the 9pfs protocol?


Yes. I can mount a specific dom0 directory in the guest.




I /could/ give it access to "/", but do we really want that?


I don't think we want to give access to "/".



We can't get rid of the /var/run/xen access in xenstored, as it is used for
live update, so the old and the new xenstored need to agree on the location of
the state file.


Well, Xenstored Live-Update is technically a tech preview feature. So we *could* 
break it and allow the state file to be specified on the command line.


Anyway, I am ok if you want them to have the live-update state in a separate 
however...




And we can't drop the possibility of specifying absolute paths
for e.g. trace files, as those might be in use by users already.

Also, regarding the background of my question. You are introducing a function 
call absolute_filename(). From the name, there is nothing indicating that 
this should only be used for trace files. If this is only indented 

Re: [PATCH v2 25/29] tools/xenstored: map stubdom interface

[Juergen Gross]

On 14.11.23 21:48, Julien Grall wrote:

Hi,

On 14/11/2023 09:12, Juergen Gross wrote:

On 14.11.23 09:56, Julien Grall wrote:

Hi,

On 14/11/2023 06:33, Juergen Gross wrote:

On 13.11.23 23:04, Julien Grall wrote:

Hi Juergen,

On 10/11/2023 16:08, Juergen Gross wrote:

When running as stubdom, map the stubdom's Xenstore ring page in order
to support using the 9pfs frontend.

Signed-off-by: Juergen Gross 
---
  tools/xenstored/core.c   |  2 ++
  tools/xenstored/domain.c | 27 ++-
  tools/xenstored/domain.h |  1 +
  3 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index b9ec50b34c..4a9d874f17 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -2991,6 +2991,8 @@ int main(int argc, char *argv[])
  lu_read_state();
  #endif
+    stubdom_init();
+
  check_store();
  /* Get ready to listen to the tools. */
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index fa17f68618..162b87b460 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -37,6 +37,10 @@
  #include 
  #include 
+#ifdef __MINIOS__
+#include 
+#endif
+
  static xc_interface **xc_handle;
  xengnttab_handle **xgt_handle;
  static evtchn_port_t virq_port;
@@ -500,6 +504,11 @@ static void *map_interface(domid_t domid)
  if (domid == xenbus_master_domid())
  return xenbus_map();
+#ifdef __MINIOS__
+    if (domid == stub_domid)
+    return xenstore_buf;
+#endif
+
  return xengnttab_map_grant_ref(*xgt_handle, domid,
 GNTTAB_RESERVED_XENSTORE,
 PROT_READ|PROT_WRITE);
@@ -509,7 +518,7 @@ static void unmap_interface(domid_t domid, void 
*interface)

  {
  if (domid == xenbus_master_domid())
  unmap_xenbus(interface);
-    else
+    else if (domid != stub_domid)
  xengnttab_unmap(*xgt_handle, interface, 1);
  }
@@ -1214,6 +1223,22 @@ void dom0_init(void)
  xenevtchn_notify(xce_handle, dom0->port);
  }
+void stubdom_init(void)
+{
+#ifdef __MINIOS__
+    struct domain *stubdom;
+
+    if (stub_domid < 0)
+    return;
+
+    stubdom = introduce_domain(NULL, stub_domid, xenbus_evtchn, false);
+    if (!stubdom)
+    barf_perror("Failed to initialize stubdom");
+
+    xenevtchn_notify(xce_handle, stubdom->port);


If I compare to introduce_domain(), it is not entirely clear to me why the 
notification is done unconditionally here. Can you clarify?


There is no reason to do it conditionally, as we can be sure the event channel
is existing and the ring page is accessible.


Hmmm... But there is a second part in the condition:

domain->interface->connection == XENSTORE_RECONNECT

Why isn't it necessary here? What I am looking for particularly is some 
in-code comment (or at least in the commit message) because this is not 100% 
clear why we are diverging.


The test of XENSTORE_RECONNECT is done for domUs started before or together with
dom0 in order to give them a signal that they can start to use Xenstore.

Here we are initializing our own connection, so there is no need to test whether
the other end is waiting for us. We know there is no possible problem sending
the event, so we can just do it. The main instruction guarded by the test of
XENSTORE_RECONNECT is the setting of XENSTORE_CONNECTED, which then needs the
event to be sent to signal that change in the connection state.

In the end we are _not_ diverging. You should just compare the code to the more
comparable dom0_init() code. There the event is being sent unconditionally, too.


Ok. Can this be documented somewhere?


I'll add something to the commit message.


Juergen



OpenPGP_0xB0DE9DD628BF132F.asc
Description: OpenPGP public key


OpenPGP_signature.asc
Description: OpenPGP digital signature

[xen-4.17-testing test] 183753: tolerable FAIL - PUSHED

[osstest service owner]

flight 183753 xen-4.17-testing real [real]
flight 183757 xen-4.17-testing real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/183753/
http://logs.test-lab.xenproject.org/osstest/logs/183757/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-qemuu-freebsd12-amd64 18 guest-saverestore.2 fail pass in 
183757-retest

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 183343
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 183343
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 183343
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 183343
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 183343
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 183343
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 183343
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 183343
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 183343
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 183343
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 183343
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 183343
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-vhd  14 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  0527bab0901b800e3f1be2e7836c5346b6e08809
baseline version:
 xen  0b56bed864ca9b572473957f0254aefa797216f2

Last test of basis   183343  

xen | Successful pipeline for stable-4.16 | 4dfe9517

[GitLab]

Pipeline #1072738507 has passed!

Project: xen ( https://gitlab.com/xen-project/xen )
Branch: stable-4.16 ( https://gitlab.com/xen-project/xen/-/commits/stable-4.16 )

Commit: 4dfe9517 ( 
https://gitlab.com/xen-project/xen/-/commit/4dfe95177b948d1f3ed27a801f603ed7f1bc36e8
 )
Commit Message: x86/spec-ctrl: Remove conditional IRQs-on-ness ...
Commit Author: Andrew Cooper ( https://gitlab.com/andyhhp )



Pipeline #1072738507 ( 
https://gitlab.com/xen-project/xen/-/pipelines/1072738507 ) triggered by Ganis 
( https://gitlab.com/ganis )
successfully completed 5 jobs in 1 stage.

-- 
You're receiving this email because of your account on gitlab.com.

Re: [PATCH v1 4/7] xen_pvdev: Do not assume Dom0 when creating a directrory

[Volodymyr Babchuk]

Hi David,

David Woodhouse  writes:

> [[S/MIME Signed Part:Undecided]]
> On Fri, 2023-11-10 at 20:42 +, Volodymyr Babchuk wrote:
>> From: Oleksandr Tyshchenko 
>> 
>> Instead of forcing the owner to domid 0, use XS_PRESERVE_OWNER to save
>> the previous owner of the directory.
>> 
>
> You're missing the words "... if it already exists" from that sentence.
>
> If the directory *didn't* already exist, it gets created with dom0 as
> the owner still, right? Assuming XenStore allows QEMU to do that.

If it didn't already exist, it is created and it inherits access rights
from the parent node.

> Strictly, the node gets created (if permitted) and *then*
> xs_set_permissions() attempts to set dom0 as the owner (if permitted).

Yes. I'll rephrase this as "Instead of forcing the owner to domid 0, use
 XS_PRESERVE_OWNER to save the inherited owner of the directory."

will it be okay?

>
>> Note that for other than Dom0 domain (non toolstack domain) the
>> "driver_domain" property should be set in domain config file for the
>> toolstack to create required directories in advance.
>> 
>> Signed-off-by: Oleksandr Tyshchenko 
>> Signed-off-by: Volodymyr Babchuk 
>> ---
>>  hw/xen/xen_pvdev.c | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>> 
>> diff --git a/hw/xen/xen_pvdev.c b/hw/xen/xen_pvdev.c
>> index c5ad71e8dc..42bdd4f6c8 100644
>> --- a/hw/xen/xen_pvdev.c
>> +++ b/hw/xen/xen_pvdev.c
>> @@ -60,7 +60,8 @@ void xen_config_cleanup(void)
>>  
>>  int xenstore_mkdir(char *path, int p)
>>  {
>> -    if (!qemu_xen_xs_create(xenstore, 0, 0, xen_domid, p, path)) {
>> +    if (!qemu_xen_xs_create(xenstore, 0, XS_PRESERVE_OWNER,
>> +    xen_domid, p, path)) {
>>  xen_pv_printf(NULL, 0, "xs_mkdir %s: failed\n", path);
>>  return -1;
>>  }
>
> Why bother with xenstore_mkdir()? AFAICT it's *only* used from the
> legacy XenLegacyDevice stuff, and can't we just finish the job of
> moving from that to the XenDevice model? I've done console and net
> recently; want to keep going?

Well, I am not so familiar with QEMU to accomplish this in a short
time. If you really need help, I can take alook at 9p driver, it looks
simplest of them all...

>
> And even then... the xenstore_mkdir() function is called twice from
> xen_config_dev_dirs() in hw/xen/xen_devconfig.c to create the frontend
> and backend directories — which is what the rest of your patch series
> is trying to eliminate because a driver domain doesn't have permissions
> to do that anyway.
>
> It's also called from xen_be_register() in hw/xen/xen_devconfig.c to
> create device-model/${GUEST_DOMID}/backends/${DEVICE_TYPE} (using a
> relative path, so in the driver domain's XenStore). That one presumably
> *won't* exist already, and so XS_PRESERVE_OWNER won't even have any
> effect?

As I said, it will inherit driver's domain access rights. So yeah,
Oleksandr's patch covers this case, mostly.

I agree, it would be better to drop xenstore_mkdir() at all, but this is
tangent to my task of adding virtio-pci support for ARM guests. Those
Oleksandr's patches for drive domain, that you are seeing, come to life
only because our system happens to use a separate driver domain.

-- 
WBR, Volodymyr

[xen-4.16-testing test] 183752: tolerable FAIL - PUSHED

[osstest service owner]

flight 183752 xen-4.16-testing real [real]
http://logs.test-lab.xenproject.org/osstest/logs/183752/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 183350
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 183357
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 183357
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 183357
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 183357
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 183357
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 183357
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 183357
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 183357
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 183357
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 183357
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 183357
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-vhd  14 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-checkfail never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass

version targeted for testing:
 xen  4dfe95177b948d1f3ed27a801f603ed7f1bc36e8
baseline version:
 xen  29efce0f8f10e381417a61f2f9988b40d4f6bcf0

Last test of basis   183357  2023-10-12 10:08:24 Z   33 days
Testing same since   183752  2023-11-14 13:06:51 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Roger Pau Monne 
  Roger Pau 

Re: Values generated by the ViryaOS uboot-script-gen do not work correctly on the Chromebook Snow

[Mario Marietto]

I hope that the informations below are correct :

- the imagebuilder config file :

MEMORY_START="0x0"
MEMORY_END="0x8000"
LOAD_CMD="ext2load mmc 1:3"
BOOT_CMD="bootm"
DEVICE_TREE="exynos5250-snow.dtb"
XEN="xen-4.17-armhf"
XEN_CMD="console=dtuart dtuart=serial0 dom0_mem=1152M dom0_max_vcpus=2
bootscrub=0 vwfi=native sched=null"
DOM0_KERNEL="zImage-6.6.0-xen-iommu-dma-on-xen"
DOM0_CMD="console=tty earlycon=xen earlyprintk=xen root=/dev/mmcblk1p4 rw
rootwait clk_ignore_unused"
UBOOT_SOURCE="xen.source"
UBOOT_SCRIPT="xen.scr"

xen.source : (that does not work)

mmc dev 1
ext2load mmc 1:3 0xE0 zImage-6.6.0-xen-iommu-dma-on-xen
ext2load mmc 1:3 0x180 xen-4.17-armhf.ub
ext2load mmc 1:3 0x1A0 exynos5250-snow.dtb
fdt addr 0x1A0
fdt resize 1024
fdt set /chosen \#address-cells <0x2>
fdt set /chosen \#size-cells <0x2>
fdt set /chosen xen,xen-bootargs "console=dtuart dtuart=serial0
dom0_mem=1152M dom0_max_vcpus=2 bootscrub=0 vwfi=native sched=null"
fdt mknod /chosen dom0
fdt set /chosen/dom0 compatible  "xen,linux-zimage" "xen,multiboot-module"
"multiboot,module"
fdt set /chosen/dom0 reg <0x0 0xE0 0x0 0x87C200 >
fdt set /chosen xen,dom0-bootargs "console=tty earlycon=xen earlyprintk=xen
root=/dev/mmcblk1p4 rw rootwait clk_ignore_unused"
setenv fdt_high 0x
bootm 0x180 - 0x1A0

xen.source : (created by chuck and that works)

mmc dev 1
ext2load mmc 1:3 0x4200 zImage-6.6.0-xen-iommu-dma-on-xen
ext2load mmc 1:3 0x5100 xen-4.17-armhf-armmp-0x51004000.ub
ext2load mmc 1:3 0x5ffec000 exynos5250-snow.dtb
fdt addr 0x5ffec000
fdt resize 1024
fdt set /chosen \#address-cells <0x2>
fdt set /chosen \#size-cells <0x2>
fdt set /chosen xen,xen-bootargs "console=dtuart dtuart=serial0
dom0_mem=1152M dom0_max_vcpus=2 bootscrub=0 vwfi=native sched=null"
fdt mknod /chosen dom0
fdt set /chosen/dom0 compatible  "xen,linux-zimage" "xen,multiboot-module"
"multiboot,module"
fdt set /chosen/dom0 reg <0x0 0x4200 0x0 0x87C200 >
fdt set /chosen xen,dom0-bootargs "console=tty1 root=/dev/mmcblk1p4 rw
rootwait clk_ignore_unused --no-log"
bootm 0x5100 - 0x5ffec000

all the values that you see in this conf. files have been calculated by
chuck by hand,because the values generated by the imagebuilder are wrong.
The only value that's well calculated by the imagebuilder is 0x87C200

- the size of all the binaries specified in the imagebuilder config file :

exynos5250-snow.dtb = 46.6 KiB (47,769 byte)
zImage-6.6.0-xen-iommu-dma-on-xen = 8.5 MiB (8,897,024 byte)



On Wed, Nov 15, 2023 at 12:17 AM Stefano Stabellini 
wrote:

> Hi Mario,
>
> I think we misunderstood each other :-)
>
> MEMORY_START-MEMORY_END is not supposed to be computed: it is supposed
> to come from the memory node in device tree tree (/memory) of the
> platform. The idea is that you should not have to do any computations,
> but only reuse the same address range specified there.
>
> Similarly in regards to "please post the size of all the binaries",
> this is just for debugging, so that I can see if there are any bugs with
> uboot-script-gen. I cannot debug the script unless I figure out what the
> problem is and the only way I can do that is with the binary sizes and
> redoing all the steps by hand.
>
> The expected outcome is that once we resolve the problem you should be
> able to use uboot-script-gen without any additional computation needed.
>
> Of course using static values is also OK.
>
>
> On Wed, 15 Nov 2023, Mario Marietto wrote:
> > ---> uboot-script-gen assumes that the memory range specified by
> MEMORY_START-MEMORY_END is valid and correct.
> >
> > Actually Chuck chose 0 as MEMORY_START and 0x80 as MEMORY_END and
> these are stable values,they don't change. If you ask me to calculate
> > those values,it means that we need to compute these values. I imagine
> that to calculate these values is not easy.
> >
> > ---> To debug this kind of issues please post the size of all the
> binaries specified in the imagebuilder config file
> >
> > I imagine that I should also calculate those values. And again,I see a
> complication.
> >
> > I'm realizing that the method used by Chuck is easier because he uses
> stable values. In the end,there aren't any calculations to do and
> > since I'm looking for an easier method,not a more complicated one,I
> think that Chuck's method is good as is.
> >
> > On Tue, Nov 14, 2023 at 11:51 PM Stefano Stabellini <
> sstabell...@kernel.org> wrote:
> >   Hi Mario,
> >
> >   It is difficult to know how to change uboot-script-gen if we don't
> know
> >   why it is currently going wrong.
> >
> >   uboot-script-gen assumes that the memory range specified by
> >   MEMORY_START-MEMORY_END is valid and correct.
> >
> >   So if you specified a valid and correct memory range in your
> config file
> >   (0x41e0-0x6000) why is it failing?
> >
> >   The only thing uboot-script-gen does is choosing aligned addresses
> >   within the MEMORY_START-MEMORY_END 

Re: Values generated by the ViryaOS uboot-script-gen do not work correctly on the Chromebook Snow

[Stefano Stabellini]

Hi Mario,

I think we misunderstood each other :-)

MEMORY_START-MEMORY_END is not supposed to be computed: it is supposed
to come from the memory node in device tree tree (/memory) of the
platform. The idea is that you should not have to do any computations,
but only reuse the same address range specified there.

Similarly in regards to "please post the size of all the binaries",
this is just for debugging, so that I can see if there are any bugs with
uboot-script-gen. I cannot debug the script unless I figure out what the
problem is and the only way I can do that is with the binary sizes and
redoing all the steps by hand.

The expected outcome is that once we resolve the problem you should be
able to use uboot-script-gen without any additional computation needed.

Of course using static values is also OK.


On Wed, 15 Nov 2023, Mario Marietto wrote:
> ---> uboot-script-gen assumes that the memory range specified by 
> MEMORY_START-MEMORY_END is valid and correct.
> 
> Actually Chuck chose 0 as MEMORY_START and 0x80 as MEMORY_END and these 
> are stable values,they don't change. If you ask me to calculate
> those values,it means that we need to compute these values. I imagine that to 
> calculate these values is not easy.
> 
> ---> To debug this kind of issues please post the size of all the binaries 
> specified in the imagebuilder config file
> 
> I imagine that I should also calculate those values. And again,I see a 
> complication.
> 
> I'm realizing that the method used by Chuck is easier because he uses stable 
> values. In the end,there aren't any calculations to do and
> since I'm looking for an easier method,not a more complicated one,I think 
> that Chuck's method is good as is. 
> 
> On Tue, Nov 14, 2023 at 11:51 PM Stefano Stabellini  
> wrote:
>   Hi Mario,
> 
>   It is difficult to know how to change uboot-script-gen if we don't know
>   why it is currently going wrong.
> 
>   uboot-script-gen assumes that the memory range specified by
>   MEMORY_START-MEMORY_END is valid and correct.
> 
>   So if you specified a valid and correct memory range in your config file
>   (0x41e0-0x6000) why is it failing?
> 
>   The only thing uboot-script-gen does is choosing aligned addresses
>   within the MEMORY_START-MEMORY_END range. The addresses are supposed not
>   to overlap (meaning the initrd will not overwrite part of the kernel
>   when loaded). If the issue is a bug in uboot-script-gen, such as the
>   generated addresses overlap or they are not aligned, then we can fix the
>   alignment or overlap bug. To debug this kind of issues please post:
>   - the imagebuilder config file
>   - the generate boot.source script
>   - the size of all the binaries specified in the imagebuilder config file
> 
>   On the other hand if 0x41e0-0x6000 is not a safe memory range to
>   use, then you need to specify a different memory range.
> 
>   Cheers,
> 
>   Stefano
> 
> 
> 
>   On Mon, 13 Nov 2023, Mario Marietto wrote:
>   > Hello.
>   >
>   > I'm trying to find an easier way to the problem that you can read 
> here :
>   >
>   > 
> https://github.com/mobile-virt/u-boot-chromebook-xe303c12/tree/chromebook/xen#starting-a-domu-guest
>   >
>   > where Chuck says :
>   >
>   >  6. Create the u-boot shell commands that will be used to boot Xen 
> and dom0.
>   >
>   > Create a file in /home/user (or any other directory) named 
> bootxen.source with these contents :
>   >
>   >
>   > mmc dev 1 && mmc rescan 1
>   > ext2load mmc 1:3 0x4200 zImage-6.1.61-stb-xen-cbe+
>   > ext2load mmc 1:3 0x5100 xen-4.17-armhf-armmp-0x51004000.ub
>   > ext2load mmc 1:3 0x5ffec000 exynos5250-snow-6.1.61-stb-xen-cbe+.dtb
>   > fdt addr 0x5ffec000
>   > fdt resize 1024
>   > fdt set /chosen \#address-cells <0x2>
>   > fdt set /chosen \#size-cells <0x2>
>   > fdt set /chosen xen,xen-bootargs "console=dtuart dtuart=serial0 
> dom0_mem=1G dom0_max_vcpus=2 bootscrub=0 vwfi=native"
>   > fdt mknod /chosen dom0
>   > fdt set /chosen/dom0 compatible  "xen,linux-zimage" 
> "xen,multiboot-module" "multiboot,module"
>   > fdt set /chosen/dom0 reg <0x0 0x4200 0x0 0x7D7200 >
>   > fdt set /chosen xen,dom0-bootargs "console=tty1 root=/dev/mmcblk1p4 
> rw rootwait clk_ignore_unused"
>   > bootm 0x5100 - 0x5ffec000
>   >
>   > The hex value 0x7D7200 is the size of the zImage-6.1.61-stb-xen-cbe+ 
> file, and that value is computed from the
>   uboot-script-gen script
>   > available from here :
>   >
>   >
>   > https://gitlab.com/ViryaOS/imagebuilder
>   >
>   >
>   > This is the interesting point :
>   >
>   >
>   > Please note that most of the other values in the script generated by 
> the ViryaOS uboot-script-gen do not work correctly with
>   the Chromebook
>   > 

[PATCH] docs/misra: add R16.2 and R16.3

[Stefano Stabellini]

For Rule 16.2 deviate xen/arch/x86/x86_emulate.
For Rule 16.2 allow control flow statements and terminals. For the rest,
request the "fallthrough" psedo-keyword to be used.

Signed-off-by: Stefano Stabellini 
---
 docs/misra/rules.rst | 14 ++
 1 file changed, 14 insertions(+)

diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
index 375a886607..926ecc9d82 100644
--- a/docs/misra/rules.rst
+++ b/docs/misra/rules.rst
@@ -462,6 +462,20 @@ maintainers if you want to suggest a change.
 
while(0) and while(1) and alike are allowed.
 
+   * - `Rule 16.2 
`_
+ - Required
+ - A switch label shall only be used when the most closely-enclosing
+   compound statement is the body of a switch statement
+ - xen/arch/x86/x86_emulate/ doesn't follow this guideline
+
+   * - `Rule 16.3 
`_
+ - Required
+ - An unconditional break statement shall terminate every
+   switch-clause
+ - Control flow statements (goto, return, continue) and terminals
+   (BUG()) are permitted. Other cases shall have the pseudo-keyword
+   "fallthrough".
+
* - `Rule 16.7 
`_
  - Required
  - A switch-expression shall not have essentially Boolean type
-- 
2.25.1

Re: Values generated by the ViryaOS uboot-script-gen do not work correctly on the Chromebook Snow

[Mario Marietto]

---> uboot-script-gen assumes that the memory range specified by
MEMORY_START-MEMORY_END is valid and correct.

Actually Chuck chose 0 as MEMORY_START and 0x80 as MEMORY_END and these
are stable values,they don't change. If you ask me to calculate those
values,it means that we need to compute these values. I imagine that to
calculate these values is not easy.

---> To debug this kind of issues please post the size of all the binaries
specified in the imagebuilder config file

I imagine that I should also calculate those values. And again,I see a
complication.

I'm realizing that the method used by Chuck is easier because he uses
stable values. In the end,there aren't any calculations to do and since I'm
looking for an easier method,not a more complicated one,I think that
Chuck's method is good as is.

On Tue, Nov 14, 2023 at 11:51 PM Stefano Stabellini 
wrote:

> Hi Mario,
>
> It is difficult to know how to change uboot-script-gen if we don't know
> why it is currently going wrong.
>
> uboot-script-gen assumes that the memory range specified by
> MEMORY_START-MEMORY_END is valid and correct.
>
> So if you specified a valid and correct memory range in your config file
> (0x41e0-0x6000) why is it failing?
>
> The only thing uboot-script-gen does is choosing aligned addresses
> within the MEMORY_START-MEMORY_END range. The addresses are supposed not
> to overlap (meaning the initrd will not overwrite part of the kernel
> when loaded). If the issue is a bug in uboot-script-gen, such as the
> generated addresses overlap or they are not aligned, then we can fix the
> alignment or overlap bug. To debug this kind of issues please post:
> - the imagebuilder config file
> - the generate boot.source script
> - the size of all the binaries specified in the imagebuilder config file
>
> On the other hand if 0x41e0-0x6000 is not a safe memory range to
> use, then you need to specify a different memory range.
>
> Cheers,
>
> Stefano
>
>
>
> On Mon, 13 Nov 2023, Mario Marietto wrote:
> > Hello.
> >
> > I'm trying to find an easier way to the problem that you can read here :
> >
> >
> https://github.com/mobile-virt/u-boot-chromebook-xe303c12/tree/chromebook/xen#starting-a-domu-guest
> >
> > where Chuck says :
> >
> >  6. Create the u-boot shell commands that will be used to boot Xen and
> dom0.
> >
> > Create a file in /home/user (or any other directory) named
> bootxen.source with these contents :
> >
> >
> > mmc dev 1 && mmc rescan 1
> > ext2load mmc 1:3 0x4200 zImage-6.1.61-stb-xen-cbe+
> > ext2load mmc 1:3 0x5100 xen-4.17-armhf-armmp-0x51004000.ub
> > ext2load mmc 1:3 0x5ffec000 exynos5250-snow-6.1.61-stb-xen-cbe+.dtb
> > fdt addr 0x5ffec000
> > fdt resize 1024
> > fdt set /chosen \#address-cells <0x2>
> > fdt set /chosen \#size-cells <0x2>
> > fdt set /chosen xen,xen-bootargs "console=dtuart dtuart=serial0
> dom0_mem=1G dom0_max_vcpus=2 bootscrub=0 vwfi=native"
> > fdt mknod /chosen dom0
> > fdt set /chosen/dom0 compatible  "xen,linux-zimage"
> "xen,multiboot-module" "multiboot,module"
> > fdt set /chosen/dom0 reg <0x0 0x4200 0x0 0x7D7200 >
> > fdt set /chosen xen,dom0-bootargs "console=tty1 root=/dev/mmcblk1p4 rw
> rootwait clk_ignore_unused"
> > bootm 0x5100 - 0x5ffec000
> >
> > The hex value 0x7D7200 is the size of the zImage-6.1.61-stb-xen-cbe+
> file, and that value is computed from the uboot-script-gen script
> > available from here :
> >
> >
> > https://gitlab.com/ViryaOS/imagebuilder
> >
> >
> > This is the interesting point :
> >
> >
> > Please note that most of the other values in the script generated by the
> ViryaOS uboot-script-gen do not work correctly with the Chromebook
> > Snow, but the script does correctly calculate the size of the dom0 Linux
> kernel image.
> >
> >
> > Some time ago Stefano suggested to put the values below for MEMORY_START
> and MEMORY_END inside the xen-config file :
> >
> >
> > nano xen-config file :
> >
> >
> > MEMORY_START="0x41e0"
> > MEMORY_END="0x6000"
> > LOAD_CMD="ext2load mmc 1:3"
> > BOOT_CMD="bootm"
> > DEVICE_TREE="exynos5250-snow.dtb"
> > XEN="xen-4.17-armhf"
> > XEN_CMD="console=dtuart dtuart=serial0 dom0_mem=768M dom0_max_vcpus=2
> bootscrub=0 vwfi=native sched=null"
> > DOM0_KERNEL="zImage-6.6.0-xen-dma-mapping"
> > DOM0_CMD="console=tty earlycon=xen earlyprintk=xen root=/dev/mmcblk1p4
> rw rootwait clk_ignore_unused"
> > UBOOT_SOURCE="xen.source"
> >
> >
> > bash ./uboot-script-gen -c xen-config -d .
> >
> >
> > Image Name:
> > Created:  Thu Nov  2 20:59:24 2023
> > Image Type:   ARM Linux Kernel Image (uncompressed)
> > Data Size:884744 Bytes = 864.01 KiB = 0.84 MiB
> > Load Address: 42c0
> > Entry Point:  42c0
> >
> >
> > Generated uboot script xen.scr, to be loaded at address 0x4200:
> > ext2load mmc 1:3 0x4200 xen.scr; source 0x4200
> >
> >
> > and I tried to boot Xen and Linux 6.6 as dom0 :
> >
> > SMDK5250 # mmc dev 1
> > SMDK5250 # ext2load mmc 1:3 0x4200 xen.scr; source 0x4200

[PATCH v2] misra: add R21.1 R21.2

[Stefano Stabellini]

Add 21.1 and 21.2, with a longer comment to explain how strategy with
leading underscores and why we think we are safe today.

Signed-off-by: Stefano Stabellini 
---
Changes in v2:
- remove R14.4
- update note section of 21.1
---
 docs/misra/rules.rst | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
index da343ab3ac..375a886607 100644
--- a/docs/misra/rules.rst
+++ b/docs/misra/rules.rst
@@ -519,6 +519,28 @@ maintainers if you want to suggest a change.
they are related
  -
 
+   * - `Rule 21.1 
`_
+ - Required
+ - #define and #undef shall not be used on a reserved identifier or
+   reserved macro name
+ - Identifiers starting with an underscore followed by another underscore
+   or an upper-case letter are reserved. Today Xen uses many, such as
+   header guards and bitwise manipulation functions. Upon analysis it turns
+   out Xen identifiers do not clash with the identifiers used by modern
+   GCC, but that is not a guarantee that there won't be a naming clash in
+   the future or with another compiler.  For these reasons we discourage
+   the introduction of new reserved identifiers in Xen, and we see it as
+   positive the reduction of reserved identifiers. At the same time,
+   certain identifiers starting with an underscore are also commonly used
+   in Linux (e.g. __set_bit) and we don't think it would be an improvement
+   to rename them.
+
+   * - `Rule 21.2 
`_
+ - Required
+ - A reserved identifier or reserved macro name shall not be
+   declared
+ - See comment for Rule 21.1
+
* - `Rule 21.13 
`_
  - Mandatory
  - Any value passed to a function in  shall be representable as an
-- 
2.25.1

Re: Values generated by the ViryaOS uboot-script-gen do not work correctly on the Chromebook Snow

[Stefano Stabellini]

Hi Mario,

It is difficult to know how to change uboot-script-gen if we don't know
why it is currently going wrong.

uboot-script-gen assumes that the memory range specified by
MEMORY_START-MEMORY_END is valid and correct.

So if you specified a valid and correct memory range in your config file
(0x41e0-0x6000) why is it failing?

The only thing uboot-script-gen does is choosing aligned addresses
within the MEMORY_START-MEMORY_END range. The addresses are supposed not
to overlap (meaning the initrd will not overwrite part of the kernel
when loaded). If the issue is a bug in uboot-script-gen, such as the
generated addresses overlap or they are not aligned, then we can fix the
alignment or overlap bug. To debug this kind of issues please post:
- the imagebuilder config file
- the generate boot.source script
- the size of all the binaries specified in the imagebuilder config file

On the other hand if 0x41e0-0x6000 is not a safe memory range to
use, then you need to specify a different memory range.

Cheers,

Stefano



On Mon, 13 Nov 2023, Mario Marietto wrote:
> Hello.
> 
> I'm trying to find an easier way to the problem that you can read here :
> 
> https://github.com/mobile-virt/u-boot-chromebook-xe303c12/tree/chromebook/xen#starting-a-domu-guest
> 
> where Chuck says :
> 
>  6. Create the u-boot shell commands that will be used to boot Xen and dom0.
> 
> Create a file in /home/user (or any other directory) named bootxen.source 
> with these contents :
> 
> 
> mmc dev 1 && mmc rescan 1
> ext2load mmc 1:3 0x4200 zImage-6.1.61-stb-xen-cbe+
> ext2load mmc 1:3 0x5100 xen-4.17-armhf-armmp-0x51004000.ub
> ext2load mmc 1:3 0x5ffec000 exynos5250-snow-6.1.61-stb-xen-cbe+.dtb
> fdt addr 0x5ffec000
> fdt resize 1024
> fdt set /chosen \#address-cells <0x2>
> fdt set /chosen \#size-cells <0x2>
> fdt set /chosen xen,xen-bootargs "console=dtuart dtuart=serial0 dom0_mem=1G 
> dom0_max_vcpus=2 bootscrub=0 vwfi=native"
> fdt mknod /chosen dom0
> fdt set /chosen/dom0 compatible  "xen,linux-zimage" "xen,multiboot-module" 
> "multiboot,module"
> fdt set /chosen/dom0 reg <0x0 0x4200 0x0 0x7D7200 >
> fdt set /chosen xen,dom0-bootargs "console=tty1 root=/dev/mmcblk1p4 rw 
> rootwait clk_ignore_unused"
> bootm 0x5100 - 0x5ffec000
> 
> The hex value 0x7D7200 is the size of the zImage-6.1.61-stb-xen-cbe+ file, 
> and that value is computed from the uboot-script-gen script
> available from here :
> 
> 
> https://gitlab.com/ViryaOS/imagebuilder
> 
> 
> This is the interesting point :
> 
> 
> Please note that most of the other values in the script generated by the 
> ViryaOS uboot-script-gen do not work correctly with the Chromebook
> Snow, but the script does correctly calculate the size of the dom0 Linux 
> kernel image.
> 
> 
> Some time ago Stefano suggested to put the values below for MEMORY_START and 
> MEMORY_END inside the xen-config file :
> 
> 
> nano xen-config file :
> 
> 
> MEMORY_START="0x41e0"
> MEMORY_END="0x6000"
> LOAD_CMD="ext2load mmc 1:3"
> BOOT_CMD="bootm"
> DEVICE_TREE="exynos5250-snow.dtb"
> XEN="xen-4.17-armhf"
> XEN_CMD="console=dtuart dtuart=serial0 dom0_mem=768M dom0_max_vcpus=2 
> bootscrub=0 vwfi=native sched=null"
> DOM0_KERNEL="zImage-6.6.0-xen-dma-mapping"
> DOM0_CMD="console=tty earlycon=xen earlyprintk=xen root=/dev/mmcblk1p4 rw 
> rootwait clk_ignore_unused"
> UBOOT_SOURCE="xen.source"
> 
> 
> bash ./uboot-script-gen -c xen-config -d .
> 
> 
> Image Name:    
> Created:  Thu Nov  2 20:59:24 2023
> Image Type:   ARM Linux Kernel Image (uncompressed)
> Data Size:    884744 Bytes = 864.01 KiB = 0.84 MiB
> Load Address: 42c0
> Entry Point:  42c0
> 
> 
> Generated uboot script xen.scr, to be loaded at address 0x4200:
> ext2load mmc 1:3 0x4200 xen.scr; source 0x4200
> 
> 
> and I tried to boot Xen and Linux 6.6 as dom0 :
> 
> SMDK5250 # mmc dev 1
> SMDK5250 # ext2load mmc 1:3 0x4200 xen.scr; source 0x4200
> but it did not work : it reboots on the verification screen.
> 
> --
> Mario.
> 
> 

Re: [PATCH] docs/misra: add R11.1 R11.2 R11.3 R11.6

[Stefano Stabellini]

On Tue, 14 Nov 2023, Jan Beulich wrote:
> On 14.11.2023 00:44, Stefano Stabellini wrote:
> > --- a/docs/misra/rules.rst
> > +++ b/docs/misra/rules.rst
> > @@ -383,6 +383,38 @@ maintainers if you want to suggest a change.
> >  
> > CFLAGS="-Warith-conversion -Wno-error=arith-conversion" make -C xen
> >  
> > +   * - `Rule 11.1 
> > `_
> > + - Required
> > + - Conversions shall not be performed between a pointer to a
> > +   function and any other type
> > + - All conversions to integer types are permitted if the destination
> > +   type has enough bits to hold the entire value. Conversions to
> > +   bool and void* are permitted.
> > +
> > +   * - `Rule 11.2 
> > `_
> > + - Required
> > + - Conversions shall not be performed between a pointer to an
> > +   incomplete type and any other type
> > + - All conversions to integer types are permitted if the destination
> > +   type has enough bits to hold the entire value. Conversions to
> > +   bool and void* are permitted.
> 
> Assuming the rule's exception 2 really means void, not void*, the mentioning
> of void* here is still fine.
> 
> > +   * - `Rule 11.3 
> > `_
> > + - Required
> > + - A cast shall not be performed between a pointer to object type
> > +   and a pointer to a different object type
> > + - All conversions to integer types are permitted if the destination
> > +   type has enough bits to hold the entire value. Conversions to
> > +   bool and void* are permitted.
> 
> Here and ...
> 
> > +   * - `Rule 11.6 
> > `_
> > + - Required
> > + - A cast shall not be performed between pointer to void and an
> > +   arithmetic type
> > + - All conversions to integer types are permitted if the destination
> > +   type has enough bits to hold the entire value. Conversions to
> > +   bool and void* are permitted.
> 
> ... here, however I consider its mentioning misleading: "void" is neither
> an object type nor an arithmetic one.

Yes I think here it is misleading. I'll remove the mention of void* for
Rule 11.6. I'll keep it for Rule 11.1 and 11.2.

For Rule 11.3, I think I should remove the entire note as it is not
helpful.


---
[PATCH v2] docs/misra: add R11.1 R11.2 R11.3 R11.6

Add MISRA C Rules 11.1, 11.2, 11.3, 11.6 as discussed.

Explicitly add in the notes that conversions to integer types are
permitted if the destination type has enough bits to hold the entire
value. GCC gives enough guarantees in terms of preserving the bit
content in such situations.

Also allow for bool conversions (e.g. to check if a function point is
valid).

Signed-off-by: Stefano Stabellini 

diff --git a/docs/misra/rules.rst b/docs/misra/rules.rst
index aa65eb4dd0..da343ab3ac 100644
--- a/docs/misra/rules.rst
+++ b/docs/misra/rules.rst
@@ -383,6 +383,36 @@ maintainers if you want to suggest a change.
 
CFLAGS="-Warith-conversion -Wno-error=arith-conversion" make -C xen
 
+   * - `Rule 11.1 
`_
+ - Required
+ - Conversions shall not be performed between a pointer to a
+   function and any other type
+ - All conversions to integer types are permitted if the destination
+   type has enough bits to hold the entire value. Conversions to
+   bool and void* are permitted.
+
+   * - `Rule 11.2 
`_
+ - Required
+ - Conversions shall not be performed between a pointer to an
+   incomplete type and any other type
+ - All conversions to integer types are permitted if the destination
+   type has enough bits to hold the entire value. Conversions to
+   bool and void* are permitted.
+
+   * - `Rule 11.3 
`_
+ - Required
+ - A cast shall not be performed between a pointer to object type
+   and a pointer to a different object type
+ -
+
+   * - `Rule 11.6 
`_
+ - Required
+ - A cast shall not be performed between pointer to void and an
+   arithmetic type
+ - All conversions to integer types are permitted if the destination
+   type has enough bits to hold the entire value. Conversions to
+   bool are permitted.
+
* - `Rule 11.7 
`_
  - Required
  - A cast shall not be performed between pointer to object and a 
noninteger 

Re: [XEN PATCH][for-4.19 v4] xen: address violations of Rule 11.9

[Stefano Stabellini]

On Tue, 14 Nov 2023, Nicola Vetrini wrote:
> On 2023-11-14 08:19, Jan Beulich wrote:
> > On 14.11.2023 00:58, Stefano Stabellini wrote:
> > > On Mon, 13 Nov 2023, Jan Beulich wrote:
> > > > On 19.10.2023 09:55, Nicola Vetrini wrote:
> > > > > The constant 0 is used instead of NULL in '__ACCESS_ONCE' as a
> > > > > compile-time check to detect non-scalar types; its usage for this
> > > > > purpose is deviated.
> > > > > 
> > > > > Furthermore, the 'typeof_field' macro is introduced as a general way
> > > > > to access the type of a struct member without declaring a variable
> > > > > of struct type. Both this macro and 'sizeof_field' are moved to
> > > > > 'xen/macros.h'.
> > > > > 
> > > > > No functional change intended.
> > > > > 
> > > > > Signed-off-by: Nicola Vetrini 
> > > > > Reviewed-by: Jan Beulich 
> > > > > Reviewed-by: Stefano Stabellini 
> > > > > ---
> > > > > Changes in v2:
> > > > > - added entry in deviations.rst
> > > > > Changes in v3:
> > > > > - dropped access_field
> > > > > - moved macro to macros.h
> > > > > ---
> > > > > Changes in v4:
> > > > > - Amend deviation record.
> > > > > ---
> > > > >  automation/eclair_analysis/ECLAIR/deviations.ecl |  9 +
> > > > >  docs/misra/deviations.rst|  6 ++
> > > > >  xen/include/xen/compiler.h   |  8 
> > > > >  xen/include/xen/kernel.h |  2 +-
> > > > >  xen/include/xen/macros.h | 16
> > > > > 
> > > > >  5 files changed, 32 insertions(+), 9 deletions(-)
> > > > 
> > > > I tried to commit this patch, but ...
> > > > 
> > > > > diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl
> > > > > b/automation/eclair_analysis/ECLAIR/deviations.ecl
> > > > > index fa56e5c00a27..28d9c37bedb2 100644
> > > > > --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> > > > > +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> > > > > @@ -246,6 +246,15 @@ constant expressions are required.\""
> > > > >"any()"}
> > > > >  -doc_end
> > > > > 
> > > > > +#
> > > > > +# Series 11
> > > > > +#
> > > > > +
> > > > > +-doc_begin="This construct is used to check if the type is scalar,
> > > > > and for this purpose the use of 0 as a null pointer constant is
> > > > > deliberate."
> > > > > +-config=MC3R1.R11.9,reports+={deliberate,
> > > > > "any_area(any_loc(any_exp(macro(^__ACCESS_ONCE$"
> > > > > +}
> > > > > +-doc_end
> > > > > +
> > > > >  #
> > > > >  # Series 13
> > > > >  #
> > > > 
> > > > ... this change doesn't apply, and I also can't see how it would. There
> > > > were
> > > > no dependencies specified, so it's also not clear on top of which other
> > > > (ready to be committed) patch(es) this might have been meant to apply.
> > > > Please
> > > > resubmit in a shape applicable to the staging branch.
> > > 
> > > The order doesn't matter in that file, you can place # Series 11 just
> > > after # Series 10
> > 
> > That would have been one of the possible guesses. Yet then ...
> > 
> > > --- a/automation/eclair_analysis/ECLAIR/deviations.ecl
> > > +++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
> > > @@ -274,6 +274,15 @@ still non-negative."
> > >  -config=MC3R1.R10.1,etypes+={safe,
> > > "stmt(operator(logical)||node(conditional_operator||binary_conditional_operator))",
> > > "dst_type(ebool||boolean)"}
> > >  -doc_end
> > > 
> > > +#
> > > +# Series 11
> > > +#
> > > +
> > > +-doc_begin="This construct is used to check if the type is scalar, and
> > > for this purpose the use of 0 as a null pointer constant is deliberate."
> > > +-config=MC3R1.R11.9,reports+={deliberate,
> > > "any_area(any_loc(any_exp(macro(^__ACCESS_ONCE$"
> > > +}
> > > +-doc_end
> > > +
> > >  ### Set 3 ###
> > 
> > ... there is this grouping by sets as well.
> > 
> 
> +Stefano
> 
> I think the culprit here is that this commit 699899f44d09
> was never merged with 4.19. Is that correct? My own tree is rebased
> against that.

699899f44d09 is "automation/eclair: update deviations and accepted
guidelines" and should go in.

I'll wait until the release which should be imminent and then commit
both commits (if Jan doesn't beat me to it)

Re: [XEN PATCH v2] automation/eclair: add a deviation for MISRA C:2012 Rule 8.6

[Stefano Stabellini]

On Tue, 14 Nov 2023, Federico Serafini wrote:
> Update ECLAIR configuration to take into account the search
> procedure adopted by Unix linkers.
> Update deviations.rst accordingly and tag Rule 8.6 as "clean".
> 
> Signed-off-by: Federico Serafini 

Reviewed-by: Stefano Stabellini 

Re: [ImageBuilder] Add 'bootz' to a list of supported boot commands

[Stefano Stabellini]

On Tue, 14 Nov 2023, Michal Orzel wrote:
> At the moment, in order to boot 32-bit images, we need to set BOOT_CMD
> to 'bootm' which results in adding a u-boot header on top of an image.
> Add 'bootz' to a list of supported boot commands, so that we can skip
> this extra step. In most cases, specifying 'bootm' should only be needed
> for images without Linux kernel Image/zImage header. Otherwise, we can
> use 'booti'/'bootz'.
> 
> Signed-off-by: Michal Orzel 

Reviewed-by: Stefano Stabellini 


> ---
>  README.md| 4 ++--
>  scripts/uboot-script-gen | 2 +-
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/README.md b/README.md
> index fe5d2052cc69..3b4b16f1f7e4 100644
> --- a/README.md
> +++ b/README.md
> @@ -80,8 +80,8 @@ Where:
>though.
>  
>  - BOOT_CMD specifies the u-boot command used to boot the binaries.
> -  By default, it is 'booti'. The acceptable values are 'booti', 'bootm'
> -  and 'bootefi' and 'none'. If the value is 'none', the BOOT_CMD is not
> +  By default, it is 'booti'. The acceptable values are 'booti', 'bootm',
> +  'bootz' and 'bootefi' and 'none'. If the value is 'none', the BOOT_CMD is 
> not
>added to the boot script, and the addresses for the Xen binary and the
>DTB are stored in 'host_kernel_addr' and 'host_fdt_addr' u-boot
>env variables respectively, to be used manually when booting.
> diff --git a/scripts/uboot-script-gen b/scripts/uboot-script-gen
> index 6e52da5e2aca..9e3d86e4743a 100755
> --- a/scripts/uboot-script-gen
> +++ b/scripts/uboot-script-gen
> @@ -1041,7 +1041,7 @@ function check_depends()
>  
>  function check_boot_cmd()
>  {
> -if ! [[ " bootm booti bootefi none " =~ " ${BOOT_CMD}" ]]
> +if ! [[ " bootm booti bootz bootefi none " =~ " ${BOOT_CMD}" ]]
>  then
>  echo "\"BOOT_CMD=$BOOT_CMD\" is not valid"
>  exit 1
> -- 
> 2.25.1
> 

Re: [PATCH] arm/mm: add option to prefer IOMMU ops for DMA on Xen

[Stefano Stabellini]

On Tue, 14 Nov 2023, Robin Murphy wrote:
> On 11/11/2023 6:45 pm, Chuck Zmudzinski wrote:
> > Enabling the new option, ARM_DMA_USE_IOMMU_XEN, fixes this error when
> > attaching the Exynos mixer in Linux dom0 on Xen on the Chromebook Snow
> > (and probably on other devices that use the Exynos mixer):
> > 
> > [drm] Exynos DRM: using 1440.fimd device for DMA mapping operations
> > exynos-drm exynos-drm: bound 1440.fimd (ops 0xc0d96354)
> > exynos-mixer 1445.mixer: [drm:exynos_drm_register_dma] *ERROR* Device
> >   1445.mixer lacks support for IOMMU
> > exynos-drm exynos-drm: failed to bind 1445.mixer (ops 0xc0d97554): -22
> > exynos-drm exynos-drm: adev bind failed: -22
> > exynos-dp: probe of 145b.dp-controller failed with error -22
> > 
> > Linux normally uses xen_swiotlb_dma_ops for DMA for all devices when
> > xen_swiotlb is detected even when Xen exposes an IOMMU to Linux. Enabling
> > the new config option allows devices such as the Exynos mixer to use the
> > IOMMU instead of xen_swiotlb_dma_ops for DMA and this fixes the error.
> > 
> > The new config option is not set by default because it is likely some
> > devices that use IOMMU for DMA on Xen will cause DMA errors and memory
> > corruption when Xen PV block and network drivers are in use on the system.
> > 
> > Link:
> > https://lore.kernel.org/xen-devel/acfab1c5-eed1-4930-8c70-8681e256c...@netscape.net/
> > 
> > Signed-off-by: Chuck Zmudzinski 
> > ---
> > The reported error with the Exynos mixer is not fixed by default by adding
> > a second patch to select the new option in the Kconfig definition for the
> > Exynos mixer if EXYNOS_IOMMU and SWIOTLB_XEN are enabled because it is
> > not certain setting the config option is suitable for all cases. So it is
> > necessary to explicitly select the new config option during the config
> > stage of the Linux kernel build to fix the reported error or similar
> > errors that have the same cause of lack of support for IOMMU on Xen. This
> > is necessary to avoid any regressions that might be caused by enabling the
> > new option by default for the Exynos mixer.
> >   arch/arm/mm/dma-mapping.c |  6 ++
> >   drivers/xen/Kconfig   | 16 
> >   2 files changed, 22 insertions(+)
> > 
> > diff --git a/arch/arm/mm/dma-mapping.c b/arch/arm/mm/dma-mapping.c
> > index 5409225b4abc..ca04fdf01be3 100644
> > --- a/arch/arm/mm/dma-mapping.c
> > +++ b/arch/arm/mm/dma-mapping.c
> > @@ -1779,6 +1779,12 @@ void arch_setup_dma_ops(struct device *dev, u64
> > dma_base, u64 size,
> > if (iommu)
> > arm_setup_iommu_dma_ops(dev, dma_base, size, iommu, coherent);
> >   +#ifdef CONFIG_ARM_DMA_USE_IOMMU_XEN
> 
> FWIW I don't think this really needs a config option - if Xen *has* made an
> IOMMU available, then there isn't really much reason not to use it, and if for
> some reason someone really didn't want to then they could simply disable the
> IOMMU driver anyway.

The fact that the Exynos IOMMU is exposed to Linux is a mistake. Xen
doesn't recognize the Exynos IOMMU (it is not one of the IOMMUs Xen has
a driver for) so it assigns the IOMMU to Dom0. It doesn't happen on
purpose, it happens by accident. Certain things are going to break,
specifically I am fairly certain PV drivers are going to break.

If Xen recognized the Exynos IOMMU as an IOMMU it would probably hide it
from Dom0. (Today Xen doesn't have a list of IOMMUs Xen recognizes but
doesn't have a driver for.)

I think it is OK for Chuck and others to play around with this
configuration but I wouldn't add a new kconfig option to Linux to
support it.

If we do want a kconfig option, I would add a kconfig option or Linux
command line option to enable/disable swiotlb-xen. Basically a way to
force-enable or force-disable xen_swiotlb_detect(). That could be
generally useful for debugging and would also solve the problem here as
it could be used to force-disable swiotlb-xen. I would imagine that the
end result is the same: the default ops (iommu_ops) are used.



> > +   if (dev->dma_ops == _ops) {
> > +   dev->archdata.dma_ops_setup = true;
> 
> The existing assignment is effectively unconditional by this point anyway, so
> could probably just be moved earlier to save duplicating it (or perhaps just
> make the xen_setup_dma_ops() call conditional instead to save the early return
> as well).
> 
> However, are the IOMMU DMA ops really compatible with Xen? The comments about
> hypercalls and foreign memory in xen_arch_need_swiotlb() leave me concerned
> that assuming non-coherent DMA to any old Dom0 page is OK might not actually
> work in general :/

Xen has (not yet upstreaming) support for nested IOMMU (Xen uses the
IOMMU while also it exposes a virtual IOMMU to guests.) In those cases
the iommu_ops should be compatible with Xen.

swiotlb-xen is useful in cases where there is no IOMMU on the platform
(or the IOMMU doesn't cover all DMA-capable devices) and Dom0 is 1:1
mapped. 

[xen-4.15-testing test] 183751: tolerable FAIL - PUSHED

[osstest service owner]

flight 183751 xen-4.15-testing real [real]
flight 183756 xen-4.15-testing real-retest [real]
http://logs.test-lab.xenproject.org/osstest/logs/183751/
http://logs.test-lab.xenproject.org/osstest/logs/183756/

Failures :-/ but no regressions.

Tests which are failing intermittently (not blocking):
 test-amd64-amd64-xl-qemut-debianhvm-i386-xsm 12 debian-hvm-install fail pass 
in 183756-retest

Tests which did not succeed, but are not blocking:
 test-armhf-armhf-libvirt 16 saverestore-support-checkfail  like 183342
 test-amd64-amd64-xl-qemuu-win7-amd64 19 guest-stopfail like 183342
 test-amd64-amd64-xl-qemut-win7-amd64 19 guest-stopfail like 183342
 test-amd64-amd64-qemuu-nested-amd 20 debian-hvm-install/l1/l2 fail like 183342
 test-amd64-i386-xl-qemut-win7-amd64 19 guest-stop fail like 183342
 test-armhf-armhf-libvirt-raw 15 saverestore-support-checkfail  like 183342
 test-armhf-armhf-libvirt-qcow2 15 saverestore-support-check   fail like 183342
 test-amd64-amd64-xl-qemuu-ws16-amd64 19 guest-stopfail like 183342
 test-amd64-amd64-xl-qemut-ws16-amd64 19 guest-stopfail like 183342
 test-amd64-i386-xl-qemuu-ws16-amd64 19 guest-stop fail like 183342
 test-amd64-i386-xl-qemuu-win7-amd64 19 guest-stop fail like 183342
 test-amd64-i386-xl-qemut-ws16-amd64 19 guest-stop fail like 183342
 test-amd64-i386-libvirt-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-multivcpu 15 migrate-support-checkfail  never pass
 test-armhf-armhf-xl-multivcpu 16 saverestore-support-checkfail  never pass
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-amd64-i386-libvirt  15 migrate-support-checkfail   never pass
 test-amd64-amd64-libvirt-vhd 14 migrate-support-checkfail   never pass
 test-amd64-i386-xl-pvshim14 guest-start  fail   never pass
 test-amd64-amd64-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-amd64-i386-libvirt-qemuu-debianhvm-amd64-xsm 13 migrate-support-check 
fail never pass
 test-arm64-arm64-xl-thunderx 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-thunderx 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-amd64-i386-libvirt-raw  14 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 15 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-xsm 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-arndale  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit2  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-credit1  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-rtds 16 saverestore-support-checkfail   never pass
 test-arm64-arm64-libvirt-raw 14 migrate-support-checkfail   never pass
 test-arm64-arm64-libvirt-raw 15 saverestore-support-checkfail   never pass
 test-arm64-arm64-xl-vhd  14 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-vhd  15 saverestore-support-checkfail   never pass
 test-armhf-armhf-libvirt-raw 14 migrate-support-checkfail   never pass
 test-armhf-armhf-libvirt-qcow2 14 migrate-support-checkfail never pass
 test-armhf-armhf-xl-vhd  14 migrate-support-checkfail   never pass
 test-armhf-armhf-xl-vhd  15 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  b918c4cdc7ab2c1c9e9a9b54fa9d9c595913e028
baseline version:
 xen  4a4daf6bddbe8a741329df5cc8768f7dec664aed

Last test of basis   183342  

xen | Successful pipeline for stable-4.15 | b918c4cd

[GitLab]

Pipeline #1072669753 has passed!

Project: xen ( https://gitlab.com/xen-project/xen )
Branch: stable-4.15 ( https://gitlab.com/xen-project/xen/-/commits/stable-4.15 )

Commit: b918c4cd ( 
https://gitlab.com/xen-project/xen/-/commit/b918c4cdc7ab2c1c9e9a9b54fa9d9c595913e028
 )
Commit Message: x86/spec-ctrl: Remove conditional IRQs-on-ness ...
Commit Author: Andrew Cooper ( https://gitlab.com/andyhhp )



Pipeline #1072669753 ( 
https://gitlab.com/xen-project/xen/-/pipelines/1072669753 ) triggered by Ganis 
( https://gitlab.com/ganis )
successfully completed 3 jobs in 1 stage.

-- 
You're receiving this email because of your account on gitlab.com.

Re: [XEN PATCH v4 2/2] docs/misra: add guidance on the format of Dir 4.1 docs for ECLAIR

[Julien Grall]

Hi Nicola,

On 14/11/2023 15:36, Nicola Vetrini wrote:

Additional guidance on the formatting of the document for ECLAIR
is supplied.

Signed-off-by: Nicola Vetrini 


Acked-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [XEN PATCH v4 1/2] automation/eclair: make the docs for MISRA C:2012 Dir 4.1 visible to ECLAIR

[Julien Grall]

Hi,

On 14/11/2023 15:36, Nicola Vetrini wrote:

To be able to check for the existence of the necessary subsections in
the documentation for MISRA C:2012 Dir 4.1, ECLAIR needs to have a source
file that is built.

This file is generated from 'C-runtime-failures.rst' in docs/misra
and the configuration is updated accordingly.

Signed-off-by: Nicola Vetrini 
---
Changes from RFC:
- Dropped unused/useless code
- Revised the sed command
- Revised the clean target

Changes in v2:
- Added explanative comment to the makefile
- printf instead of echo

Changes in v3:
- Terminate the generated file with a newline
- Build it with -std=c99, so that the documentation
   for D1.1 applies.
Changes in v5:
- Transform and build the file directly in the eclair-specific directory
---
  automation/eclair_analysis/build.sh   | 21 +++--
  automation/eclair_analysis/prepare.sh |  7 ---
  2 files changed, 23 insertions(+), 5 deletions(-)

diff --git a/automation/eclair_analysis/build.sh 
b/automation/eclair_analysis/build.sh
index ec087dd822fa..f24292ed0643 100755
--- a/automation/eclair_analysis/build.sh
+++ b/automation/eclair_analysis/build.sh
@@ -33,12 +33,29 @@ else
PROCESSORS=6
  fi
  
+runtime_failures_docs() {

+  doc="C-runtime-failures.rst"
+  builddir="automation/eclair_analysis"
+
+  cp "docs/misra/${doc}" "${builddir}"


Is it necessary to copy the .rst? IOW, would it be sufficient to use...


+  cd "${builddir}"
+  printf "/*\n\n" >"${doc}.tmp"
+  sed -e 's|\*/|*//*|g' "${doc}" >>"${doc}.tmp"


... docs/misc/${doc} here?


+  printf "\n\n*/\n" >>"${doc}.tmp"
+  mv "${doc}.tmp" "${doc}.c"


NIT: I am not sure why you need to first create .tmp and then create.c.


+
+  # Cannot redirect to /dev/null because it would be excluded from the analysis
+  "${CROSS_COMPILE}gcc-12" -std=c99 -c "${doc}.c" -o "${doc}.o"


NIT: It would be helpful to specify why -std=c99 is used. Above, you 
suggest this is to enable D1.1.


NIT: Can we define CC and use here and ...


+  cd -
+}
+
  (
-  cd xen
+  runtime_failures_docs
  
make "-j${PROCESSORS}" "-l${PROCESSORS}.0"\

 "CROSS_COMPILE=${CROSS_COMPILE}" \
 "CC=${CROSS_COMPILE}gcc-12"  \


...? This would make easier to re-use the code.


 "CXX=${CROSS_COMPILE}g++-12" \
-   "XEN_TARGET_ARCH=${XEN_TARGET_ARCH}"
+   "XEN_TARGET_ARCH=${XEN_TARGET_ARCH}" \
+   -C xen
  )
diff --git a/automation/eclair_analysis/prepare.sh 
b/automation/eclair_analysis/prepare.sh
index 0cac5eba00ae..fe9d16e48ecc 100755
--- a/automation/eclair_analysis/prepare.sh
+++ b/automation/eclair_analysis/prepare.sh
@@ -35,11 +35,12 @@ else
  fi
  
  (

-cd xen
-cp "${CONFIG_FILE}" .config
+./configure
+cp "${CONFIG_FILE}" xen/.config
  make clean
  find . -type f -name "*.safparse" -print -delete
-make -f ${script_dir}/Makefile.prepare prepare
+cd xen
+make -f "${script_dir}/Makefile.prepare" prepare
  # Translate the /* SAF-n-safe */ comments into ECLAIR CBTs
  scripts/xen-analysis.py --run-eclair --no-build --no-clean
  )


Cheers,

--
Julien Grall

Re: [PATCH 4/5] tools/xenstored: remove "-N" command line option

[Andrew Cooper]

On 13/11/2023 12:43 pm, Juergen Gross wrote:
> The "-N" (do not daemonize) command line option is of questionable use:
> its sole purpose seems to be to aid debugging of xenstored by making it
> easier to start xenstored under gdb, or to see any debug messages
> easily.
>
> Debug messages can as well be sent to syslog(), while gdb can be
> attached to the daemon easily. The only not covered case is an error
> while initializing xenstored, but this could be handled e.g. by saving
> a core dump, which can be analyzed later.
>
> The call of talloc_enable_leak_report_full() done only with "-N"
> specified is no longer needed, as the same can be achieved via
> "xenstore-control memreport".
>
> Signed-off-by: Juergen Gross 

CC Edvin.

There's a patch out to specifically use this option (under systemd) to
fix a bug we found.

I can't recall the details, but I seem to recall it wasn't specific to
oxenstored.

~Andrew

Re: [PATCH v1 1/7] xen-block: Do not write frontend nodes

[David Woodhouse]

On Tue, 2023-11-14 at 21:32 +, Volodymyr Babchuk wrote:
> 
> > I think we want to kill the xen_backend_set_device() function and
> > instead set the backend as a property of the XenDevice *before*
> > realizing it.
> 
> Not sure that I got this. Right now device is property of
> XenBackendInstance. Are you proposing to make this other way around?
> 
> Right now I am looking for a place where to store the information of
> XenDevice creator. My plan was to add "found_in_xenbus" property to
> XenDevice and set it in xen_backend_device_create.

A bit like this?

https://lore.kernel.org/qemu-devel/916e6f41e2da700375f84a2fda7b85d4b58dfb31.ca...@infradead.org



smime.p7s
Description: S/MIME cryptographic signature

Re: [XEN PATCH v4 2/2] docs/misra: add guidance on the format of Dir 4.1 docs for ECLAIR

[Stefano Stabellini]

On Tue, 14 Nov 2023, Nicola Vetrini wrote:
> Additional guidance on the formatting of the document for ECLAIR
> is supplied.
> 
> Signed-off-by: Nicola Vetrini 

Reviewed-by: Stefano Stabellini 

Re: [XEN PATCH v4 1/2] automation/eclair: make the docs for MISRA C:2012 Dir 4.1 visible to ECLAIR

[Stefano Stabellini]

On Tue, 14 Nov 2023, Nicola Vetrini wrote:
> To be able to check for the existence of the necessary subsections in
> the documentation for MISRA C:2012 Dir 4.1, ECLAIR needs to have a source
> file that is built.
> 
> This file is generated from 'C-runtime-failures.rst' in docs/misra
> and the configuration is updated accordingly.
> 
> Signed-off-by: Nicola Vetrini 

This is actually match better than before, thanks!

Reviewed-by: Stefano Stabellini 

Re: [PATCH] automation: set architecture in docker files

[Stefano Stabellini]

On Tue, 14 Nov 2023, Roger Pau Monné wrote:
> On Tue, Nov 14, 2023 at 03:00:17PM +, Anthony PERARD wrote:
> > On Tue, Nov 14, 2023 at 10:01:06AM +0100, Roger Pau Monné wrote:
> > > On Mon, Nov 13, 2023 at 04:10:24PM -0800, Stefano Stabellini wrote:
> > > > On Mon, 13 Nov 2023, Roger Pau Monne wrote:
> > > > > Pass the desired architecture of the image in the FROM instruction if 
> > > > > the
> > > > > image is possibly multi-platform.
> > > > > 
> > > > > This allows using the x86 Dockerfiles on OS X on arm64 hardware.
> > > > > 
> > > > > No functional change intended.
> > > > > 
> > > > > Signed-off-by: Roger Pau Monné 
> > > > 
> > > > Although I am not opposed to this change, so far we have been using:
> > > > arm64v8/alpine:3.18
> > > > 
> > > > for x86 it is not specified but it would be:
> > > > amd64/alpine:3.18
> > > > 
> > > > Two options:
> > > > 1) we add amd64/ everywhere and leave the arm containers alone
> > > > 2) we change all containers, including the arm containers, to use the
> > > > --platform option
> > > > 
> > > > I don't think is a good idea to have 2 different ways to specify the
> > > > architecture for x86 and arm containers
> > > 
> > > I'm not an expert on this, but attempting to use (2):
> > > 
> > > -FROM arm64v8/alpine:3.18
> > > +FROM --platform=linux/arm64v8 alpine:3.18
> > > 
> > > Does not work for me:
> > > 
> > > % make -C automation/build alpine/3.18-arm64v8
> > > docker build --pull -t 
> > > registry.gitlab.com/xen-project/xen/alpine:3.18-arm64v8 -f 
> > > alpine/3.18-arm64v8.dockerfile alpine
> > > [+] Building 1.4s (3/3) FINISHED  
> > > docker:desktop-linux
> > >  => [internal] load .dockerignore 
> > > 0.0s
> > >  => => transferring context: 2B   
> > > 0.0s
> > >  => [internal] load build definition from 3.18-arm64v8.dockerfile 
> > > 0.0s
> > >  => => transferring dockerfile: 818B  
> > > 0.0s
> > >  => ERROR [internal] load metadata for docker.io/library/alpine:3.18  
> > > 1.4s
> > > --
> > >  > [internal] load metadata for docker.io/library/alpine:3.18:
> > > --
> > > 3.18-arm64v8.dockerfile:1
> > > 
> > >1 | >>> FROM --platform=linux/arm64v8 alpine:3.18
> > >2 | LABEL maintainer.name="The Xen Project" \
> > >3 |   maintainer.email="xen-devel@lists.xenproject.org"
> > > 
> > > ERROR: failed to solve: alpine:3.18: no match for platform in manifest 
> > > sha256:eece025e432126ce23f223450a0326fbebde39cdf496a85d8c016293fc851978: 
> > > not found
> > > make: *** [alpine/3.18-arm64v8] Error 1
> > > 
> > > That's why I've left the prefixed images alone.
> > > 
> > > I could prefix the x86 images with amd64/ if that's preferred, I
> > > didn't try that option, as the Docker manual suggested using
> > > --platform.
> > 
> > So a few things to know, "--platform=linux/amd64" just select a
> > different build of one container. For example, for the "alpine"
> > containers, you can see all the different builds available on the docker
> > hub, here a few links:
> > - Official Docker, Alpine images, home:
> >   https://hub.docker.com/_/alpine
> > - The different builds: 
> >   https://hub.docker.com/_/alpine/tags
> > 
> > So, for amd64v8, you probably want --platform=linux/arm64/v8
> 
> Interesting, I guess I was looking at an outdated documentation that
> stated the tag as arm64v8 instead of arm64/v8.
> 
> > 
> > Then, they are per-architecture repository that make it easier to deal
> > with foreign architecture, and probably maintained by a different
> > community. e.g. for alpine arm64v8:
> > https://hub.docker.com/r/arm64v8/alpine/
> > 
> > Those provide a build for a single architecture.
> 
> Right, so those two are not actually the same image.  I wonder whether
> we would want to uniformly switch to using --platform when possible,
> in order to make sure we are using the same (multi arch) image to
> avoid surprises.
> 
> > 
> > 
> > Sometime, you actually need to "--platform=*" to select a particular
> > architecture, like I did for "jessie-i386.dockerfile".
> > 
> > 
> > One thing I've notice when using --platform is that, if for example I
> > use the container "--platform=linux/amd64 alpine:3" then
> > "--platform=linux/arm/v6 alpine:3"; later when I only specify
> > "alpine:3", it's going to be the armv6, and I think docker is going to
> > complain if I try tu use "--platform=linux/amd64 alpine:3" without
> > "docker pull" first (or I guess docker build --pull).
> > 
> > Hope that help.
> > 
> > So I guess using containers "amd64/*" or "arm64v8/*" is fine, but
> > sometime will need to use "--platform=*".
> 
> My take is that it's better to use --platform when possible, as then
> all 

Re: [PATCH 5/5] tools/xenstored: remove the "-R" command line option

[Julien Grall]

Hi Juergen,

On 13/11/2023 12:43, Juergen Gross wrote:

The "-R" (no recovery) command line option enables to omit fixing the
node store in case of detected inconsistencies.

This might have been of interest in the past, when the node data base
was kept in a file, but now the usability of this option is zero.

Remove the "-R" option.

Signed-off-by: Juergen Gross 


Reviewed-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH v1 1/7] xen-block: Do not write frontend nodes

[Volodymyr Babchuk]

Hi David,

David Woodhouse  writes:

> On 11 November 2023 16:51:22 GMT-05:00, Andrew Cooper 
>  wrote:
>>On 11/11/2023 8:18 pm, David Woodhouse wrote:
>>> On 11 November 2023 08:43:40 GMT-05:00, Andrew Cooper 
>>>  wrote:
 Furthermore, the control domain doesn't always have the domid of 0.

 If qemu wants/needs to make changes like this, the control domain has to
 arrange for qemu's domain to have appropriate permissions on the nodes.
>>> Right. And that's simple enough: if you are running QEMU in a
>>> domain which doesn't have permission to create the backend
>>> directory and/or the frontend nodes, don't ask it to *create*
>>> devices. In that case it is only able to connect as the backend for
>>> devices which were created *for* it by the toolstack.
>>>
>>> The criterion used in this patch series should be "did QEMU create this 
>>> device, or discover it".
>>>
>>
>>Yeah, that sounds like the right approach.
>
> I think we want to kill the xen_backend_set_device() function and
> instead set the backend as a property of the XenDevice *before*
> realizing it.

Not sure that I got this. Right now device is property of
XenBackendInstance. Are you proposing to make this other way around?

Right now I am looking for a place where to store the information of
XenDevice creator. My plan was to add "found_in_xenbus" property to
XenDevice and set it in xen_backend_device_create.

-- 
WBR, Volodymyr

Re: [PATCH-for-9.0 v2 06/19] hw/pci/msi: Restrict xen_is_pirq_msi() call to Xen

[David Woodhouse]

On Tue, 2023-11-14 at 10:44 -0500, David Woodhouse wrote:
> 
> I believe that if you push your branch to a gitlab tree with the
> right CI variables defined, it'll run all the CI? And I *hope* it
> fails with this patch. It's precisely the kind of thing I was
> *intending* to catch with the testing!

So it doesn't fail. Because virtio-net-pci doesn't use MSI, we need to
test with E1000E or something like that. And actually I'm not 100%
convinced the avocado tests are even running in the gitlab CI.

And also, if I test it manually... it doesn't fail because you didn't
break it :)

You didn't break the xen_evtchn_snoop_msi() call; that still works.

What you were changing is the part which makes msi_is_masked() lie and
unconditionally return false when it's mapped to a Xen PIRQ. And
actually, even in the Xen-emulation case, xen_is_pirq_msi() is always
returning zero. So your patch isn't changing anything.

It *does*, however, make hw/pci/msi.c do this exactly the same way as
hw/pci/msix.c does. That one does check xen_enabled() first.

I need to double-check whether the msi{x,}_is_masked() checks ought to
be doing the same for Xen-emu as they do on real Xen. That whole thing
is a complete abomination. But that isn't your problem. For your patch,

Reviewed-by: David Woodhouse 







smime.p7s
Description: S/MIME cryptographic signature

Re: [PATCH 4/5] tools/xenstored: remove "-N" command line option

[Julien Grall]

Hi,

On 13/11/2023 12:43, Juergen Gross wrote:

The "-N" (do not daemonize) command line option is of questionable use:
its sole purpose seems to be to aid debugging of xenstored by making it
easier to start xenstored under gdb, or to see any debug messages
easily.

Debug messages can as well be sent to syslog(), while gdb can be
attached to the daemon easily. The only not covered case is an error
while initializing xenstored, but this could be handled e.g. by saving
a core dump, which can be analyzed later.


The slight trouble is a debug message in the syslog (such as from 
barf()) is often not enough. What you want is a coredump or attach gdb 
to analyze.


For the former, I found quite useful when debugging the Live-Update code 
to replace exit() with abort() in bar*(). So a coredump will be 
generated anytime barf*() is called.




The call of talloc_enable_leak_report_full() done only with "-N"
specified is no longer needed, as the same can be achieved via
"xenstore-control memreport".

Signed-off-by: Juergen Gross 
---
Slightly RFC, as this is making debugging a little bit harder in
specific cases. OTOH I didn't use this option since years, in spite of
having done a _lot_ of xenstore hacking.


I have used a few times but when trying to run Xenstored with ASAN but 
this wasn't trivial to use.




Signed-off-by: Juergen Gross 


Reviewed-by: Julien Grall 

Re: [PATCH 3/5] tools/xenstored: remove the "-P" command line option

[Julien Grall]

Hi Juergen,

On 13/11/2023 12:43, Juergen Gross wrote:

The "-P" command line option just results in printing the PID of the
xenstored daemon to stdout before stdout is being closed. The same
information can be retrieved from the PID file via the "-F" option.


I looked at the git history to figure out whether -P was added before 
-F. But it looks like they were added together. It is not entirely clear 
to me the author thought the both options were necessary.


Anyway, I agree this is a bit redundant.



Remove the redundant "-P" option.

Signed-off-by: Juergen Gross 


Reviewed-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH 2/5] tools/xenstored: remove "-V" command line option

[Julien Grall]

Hi Juergen,

On 13/11/2023 12:43, Juergen Gross wrote:

The "-V" (verbose) command line option is nearly completely redundant
with "io" tracing. Just the time of the printed data is a little bit
different, while the tracing is more informative.


The current position of trace_io() is a bit annoying in write_messages() 
because it will only be called once the message has been fully written 
to the ring. Can we consider adding a trace_*() when we are about to 
write? (Or maybe queue the message?)



In any case, I think it is odd to enable verbose and tracing separately. 
So with or without:


Reviewed-by: Julien Grall 

Cheers,

--
Julien Grall

Re: [PATCH v2 28/29] tools/xenstored: support complete log capabilities in stubdom

[Julien Grall]

Hi Juergen,

On 14/11/2023 09:20, Juergen Gross wrote:

On 14.11.23 10:05, Julien Grall wrote:

Hi,

On 14/11/2023 06:45, Juergen Gross wrote:

On 13.11.23 23:40, Julien Grall wrote:

Hi Juergen,

On 10/11/2023 16:08, Juergen Gross wrote:

With 9pfs being fully available in Xenstore-stubdom now, there is no
reason to not fully support all logging capabilities in stubdom.

Open the logfile on stubdom only after the 9pfs file system has been
mounted.

Signed-off-by: Juergen Gross 
Reviewed-by: Jason Andryuk 
---
  tools/hotplug/Linux/launch-xenstore.in |  1 +
  tools/xenstored/control.c  | 30 
+-

  tools/xenstored/minios.c   |  3 +++
  3 files changed, 19 insertions(+), 15 deletions(-)

diff --git a/tools/hotplug/Linux/launch-xenstore.in 
b/tools/hotplug/Linux/launch-xenstore.in

index e854ca1eb8..da4eeca7c5 100644
--- a/tools/hotplug/Linux/launch-xenstore.in
+++ b/tools/hotplug/Linux/launch-xenstore.in
@@ -98,6 +98,7 @@ test -f @CONFIG_DIR@/@CONFIG_LEAF_DIR@/xencommons 
&& . @CONFIG_DIR@/@CONFIG_LEAF

  [ -z "$XENSTORE_DOMAIN_SIZE" ] && XENSTORE_DOMAIN_SIZE=8
  XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --memory 
$XENSTORE_DOMAIN_SIZE"
  [ -z "$XENSTORE_MAX_DOMAIN_SIZE" ] || 
XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS --maxmem 
$XENSTORE_MAX_DOMAIN_SIZE"
+    [ -z "$XENSTORED_TRACE" ] || 
XENSTORE_DOMAIN_ARGS="$XENSTORE_DOMAIN_ARGS -T xenstored-trace.log"


I am probably missing something, but any reason to not use 
@XEN_LOG_DIR@/xenstored-trace.log as we do for xenstored?


Yes. Stubdom has no access to XEN_LOG_DIR.


Ok. This restriction is not that obvious... The documentation in 
sysconfig.xencommons.in implies that it will be written in XEN_LOG_DIR:


## Type: string
## Default: ""
#
# Additional commandline arguments to start xenstored,
# like "--trace-file @XEN_LOG_DIR@/xenstored-trace.log"
# See "@sbindir@/xenstored --help" for possible options.
# Only evaluated if XENSTORETYPE is "daemon".
XENSTORED_ARGS=

Also, we are saying this is only supported when Xenstored is 
daemonized. So I think there are some documentation update necessary 
in this patch.


This is for the daemon. And the documentation example here is using an
absolute filename, so this is very clear where the trace file will be 
written.


Ah yes. Sorry I got confused.



For stubdom a related parameter "XENSTORE_DOMAIN_ARGS" is available. I 
can add

a sentence to the explanation of that parameter describing setting of a
possible trace file specification.


Do you mean in the following comment:

## Type: string
## Default: Not defined, tracing off
#
# Log xenstored messages
# Only evaluated if XENSTORETYPE is "daemon".
#XENSTORED_TRACE=[yes|on|1]

I think here we need to remove the "Only...".

Cheers,

--
Julien Grall

Re: [PATCH v2 27/29] tools/xenstored: add helpers for filename handling

[Julien Grall]

Hi Juergen,

On 14/11/2023 09:26, Juergen Gross wrote:

On 14.11.23 10:10, Julien Grall wrote:

Hi Juergen,

On 14/11/2023 06:45, Juergen Gross wrote:

On 13.11.23 23:25, Julien Grall wrote:

Hi Juergen,

On 10/11/2023 16:08, Juergen Gross wrote:

Add some helpers for handling filenames which might need different
implementations between stubdom and daemon environments:

- expansion of relative filenames (those are not really defined today,
   just expand them to be relative to /var/lib/xen/xenstore)
- expansion of xenstore_daemon_rundir() (used e.g. for saving the 
state

   file in case of live update - needs to be unchanged in the daemon
   case, but should result in /var/lib/xen/xenstore for stubdom)

Signed-off-by: Juergen Gross 
Reviewed-by: Jason Andryuk 
---
  tools/xenstored/core.c  | 9 -
  tools/xenstored/core.h  | 3 +++
  tools/xenstored/lu_daemon.c | 4 ++--
  tools/xenstored/minios.c    | 5 +
  tools/xenstored/posix.c | 5 +
  5 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index 4a9d874f17..77befd24c9 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -158,6 +158,13 @@ void trace_destroy(const void *data, const 
char *type)

  trace("obj: DESTROY %s %p\n", type, data);
  }
+char *absolute_filename(const void *ctx, const char *filename)


Can the return be const?


I'll have a look.




+{
+    if (filename[0] != '/')
+    return talloc_asprintf(ctx, XENSTORE_LIB_DIR "/%s", 
filename);


Looking at the rest of patch, you will be using xenstore_rundir(). I 
wonder whether it would not be better to switch to xenstore_rundir() 
so...



+    return talloc_strdup(ctx, filename);
+}
+
  /**
   * Signal handler for SIGHUP, which requests that the trace log 
is reopened
   * (in the main loop).  A single byte is written to 
reopen_log_pipe, to awaken

@@ -2983,7 +2990,7 @@ int main(int argc, char *argv[])
  signal(SIGHUP, trigger_reopen_log);
  if (tracefile)
-    tracefile = talloc_strdup(NULL, tracefile);
+    tracefile = absolute_filename(NULL, tracefile);
  #ifndef NO_LIVE_UPDATE
  /* Read state in case of live update. */
diff --git a/tools/xenstored/core.h b/tools/xenstored/core.h
index a0d3592961..51e1dddb22 100644
--- a/tools/xenstored/core.h
+++ b/tools/xenstored/core.h
@@ -393,6 +393,9 @@ void early_init(void);
  void mount_9pfs(void);
  #endif
+const char *xenstore_rundir(void);
+char *absolute_filename(const void *ctx, const char *filename);
+
  /* Write out the pidfile */
  void write_pidfile(const char *pidfile);
diff --git a/tools/xenstored/lu_daemon.c b/tools/xenstored/lu_daemon.c
index 71bcabadd3..635ab0 100644
--- a/tools/xenstored/lu_daemon.c
+++ b/tools/xenstored/lu_daemon.c
@@ -24,7 +24,7 @@ void lu_get_dump_state(struct lu_dump_state *state)
  state->size = 0;
  state->filename = talloc_asprintf(NULL, "%s/state_dump",
-  xenstore_daemon_rundir());
+  xenstore_rundir());


... call and ...


  if (!state->filename)
  barf("Allocation failure");
@@ -65,7 +65,7 @@ FILE *lu_dump_open(const void *ctx)
  int fd;
  filename = talloc_asprintf(ctx, "%s/state_dump",
-   xenstore_daemon_rundir());
+   xenstore_rundir());


... this one could be replaced with absolute_filename().


No, I don't think this is a good idea.

I don't want the daemon to store trace files specified as relative files
to be stored in /var/run/xen, while I want all files of the stubdom 
to be

stored under /var/lib/xen.


Why? This is a bit odd to have a different behavior between stubdom 
and daemon. It would be much easier for the user if they knew all the 
files would be at the same place regardless the version used.


The main difference is that stubdom has access to only _one_ directory 
in dom0.


Would you be able to explain why we can only give access to a single 
directory? Is this because of the 9pfs protocol?



I /could/ give it access to "/", but do we really want that?


I don't think we want to give access to "/".



We can't get rid of the /var/run/xen access in xenstored, as it is used for
live update, so the old and the new xenstored need to agree on the 
location of

the state file.


Well, Xenstored Live-Update is technically a tech preview feature. So we 
*could* break it and allow the state file to be specified on the command 
line.


Anyway, I am ok if you want them to have the live-update state in a 
separate however...



And we can't drop the possibility of specifying absolute 
paths

for e.g. trace files, as those might be in use by users already.

Also, regarding the background of my question. You are introducing a 
function call absolute_filename(). From the name, there is nothing 
indicating that this should only be used for trace files. If this is 
only indented for tracefile, then I think this should be renamed to 
something like 

Re: [PATCH v2 25/29] tools/xenstored: map stubdom interface

[Julien Grall]

Hi,

On 14/11/2023 09:12, Juergen Gross wrote:

On 14.11.23 09:56, Julien Grall wrote:

Hi,

On 14/11/2023 06:33, Juergen Gross wrote:

On 13.11.23 23:04, Julien Grall wrote:

Hi Juergen,

On 10/11/2023 16:08, Juergen Gross wrote:

When running as stubdom, map the stubdom's Xenstore ring page in order
to support using the 9pfs frontend.

Signed-off-by: Juergen Gross 
---
  tools/xenstored/core.c   |  2 ++
  tools/xenstored/domain.c | 27 ++-
  tools/xenstored/domain.h |  1 +
  3 files changed, 29 insertions(+), 1 deletion(-)

diff --git a/tools/xenstored/core.c b/tools/xenstored/core.c
index b9ec50b34c..4a9d874f17 100644
--- a/tools/xenstored/core.c
+++ b/tools/xenstored/core.c
@@ -2991,6 +2991,8 @@ int main(int argc, char *argv[])
  lu_read_state();
  #endif
+    stubdom_init();
+
  check_store();
  /* Get ready to listen to the tools. */
diff --git a/tools/xenstored/domain.c b/tools/xenstored/domain.c
index fa17f68618..162b87b460 100644
--- a/tools/xenstored/domain.c
+++ b/tools/xenstored/domain.c
@@ -37,6 +37,10 @@
  #include 
  #include 
+#ifdef __MINIOS__
+#include 
+#endif
+
  static xc_interface **xc_handle;
  xengnttab_handle **xgt_handle;
  static evtchn_port_t virq_port;
@@ -500,6 +504,11 @@ static void *map_interface(domid_t domid)
  if (domid == xenbus_master_domid())
  return xenbus_map();
+#ifdef __MINIOS__
+    if (domid == stub_domid)
+    return xenstore_buf;
+#endif
+
  return xengnttab_map_grant_ref(*xgt_handle, domid,
 GNTTAB_RESERVED_XENSTORE,
 PROT_READ|PROT_WRITE);
@@ -509,7 +518,7 @@ static void unmap_interface(domid_t domid, void 
*interface)

  {
  if (domid == xenbus_master_domid())
  unmap_xenbus(interface);
-    else
+    else if (domid != stub_domid)
  xengnttab_unmap(*xgt_handle, interface, 1);
  }
@@ -1214,6 +1223,22 @@ void dom0_init(void)
  xenevtchn_notify(xce_handle, dom0->port);
  }
+void stubdom_init(void)
+{
+#ifdef __MINIOS__
+    struct domain *stubdom;
+
+    if (stub_domid < 0)
+    return;
+
+    stubdom = introduce_domain(NULL, stub_domid, xenbus_evtchn, 
false);

+    if (!stubdom)
+    barf_perror("Failed to initialize stubdom");
+
+    xenevtchn_notify(xce_handle, stubdom->port);


If I compare to introduce_domain(), it is not entirely clear to me 
why the notification is done unconditionally here. Can you clarify?


There is no reason to do it conditionally, as we can be sure the 
event channel

is existing and the ring page is accessible.


Hmmm... But there is a second part in the condition:

domain->interface->connection == XENSTORE_RECONNECT

Why isn't it necessary here? What I am looking for particularly is 
some in-code comment (or at least in the commit message) because this 
is not 100% clear why we are diverging.


The test of XENSTORE_RECONNECT is done for domUs started before or 
together with

dom0 in order to give them a signal that they can start to use Xenstore.

Here we are initializing our own connection, so there is no need to test 
whether
the other end is waiting for us. We know there is no possible problem 
sending
the event, so we can just do it. The main instruction guarded by the 
test of
XENSTORE_RECONNECT is the setting of XENSTORE_CONNECTED, which then 
needs the

event to be sent to signal that change in the connection state.

In the end we are _not_ diverging. You should just compare the code to 
the more
comparable dom0_init() code. There the event is being sent 
unconditionally, too.


Ok. Can this be documented somewhere?

Cheers,

--
Julien Grall

xen | Failed pipeline for staging-4.17 | 28f44b60

[GitLab]

Pipeline #1072370735 has failed!

Project: xen ( https://gitlab.com/xen-project/xen )
Branch: staging-4.17 ( 
https://gitlab.com/xen-project/xen/-/commits/staging-4.17 )

Commit: 28f44b60 ( 
https://gitlab.com/xen-project/xen/-/commit/28f44b603fd86c233726bdc2a11b6325f102471a
 )
Commit Message: xen/grant: Fix build in PV_SHIM

There was a va...
Commit Author: Andrew Cooper ( https://gitlab.com/andyhhp )


Pipeline #1072370735 ( 
https://gitlab.com/xen-project/xen/-/pipelines/1072370735 ) triggered by Ganis 
( https://gitlab.com/ganis )
had 4 failed jobs.

Job #5534997591 ( https://gitlab.com/xen-project/xen/-/jobs/5534997591/raw )

Stage: build
Name: ubuntu-focal-gcc-debug
Job #5534997608 ( https://gitlab.com/xen-project/xen/-/jobs/5534997608/raw )

Stage: build
Name: alpine-3.12-gcc-debug
Job #5534997597 ( https://gitlab.com/xen-project/xen/-/jobs/5534997597/raw )

Stage: build
Name: opensuse-leap-clang-debug
Job #5534997599 ( https://gitlab.com/xen-project/xen/-/jobs/5534997599/raw )

Stage: build
Name: opensuse-leap-gcc-debug

-- 
You're receiving this email because of your account on gitlab.com.

Re: [XEN PATCH 0/9] x86: parallelize AP bring-up during boot

[Julien Grall]

Hi Krystian,

Thanks you for sending the series! Just posting some extra data point.

On 14/11/2023 17:49, Krystian Hebel wrote:

Patch series available on this branch:
https://github.com/TrenchBoot/xen/tree/smp_cleanup_upstreaming

This series makes AP bring-up parallel on x86. This reduces number of
IPIs (and more importantly, delays between them) required to start all
logical processors significantly.

In order to make it possible, some state variables that were global
had to be made per-CPU. In most cases, accesses to those variables can
be performed through helper macros, some of them existed before in a
different form.

In addition to work required for parallel initialization, I've fixed
issues in error path around `machine_restart()` that were discovered
during implementation and testing.

CPU hotplug should not be affected, but I had no way of testing it.
During wakeup from S3 APs are started one by one. It should be possible
to enable parallel execution there as well, but I don't have a way of
testing it as of now.

To measure the improvement, I added output lines (identical for before
and after changes so there is no impact of printing over serial) just
above and below `if ( !pv_shim )` block. `console_timestamps=raw` was
used to get as accurate timestamp as possible, and average over 3 boots
was taken into account for each measurement. The final improvement was
calculated as (1 - after/before) * 100%, rounded to 0.01%. These are
the results:

* Dell OptiPlex 9010 with Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
   (4 cores, 4 threads): 48.83%
* MSI PRO Z790-P with 13th Gen Intel(R) Core(TM) i5-13600K (14 cores,
   20 threads, 6P+8E) `smt=on`: 36.16%
* MSI PRO Z790-P with 13th Gen Intel(R) Core(TM) i5-13600K (14 cores,
   20 threads, 6P+8E) `smt=off`: 0.25% (parking takes a lot of additional
   time)
* HP t630 Thin Client with AMD Embedded G-Series GX-420GI Radeon R7E
   (4 cores, 4 threads): 68.00%


Your series reminded me some optimization we did at AWS in the SMP boot 
code. This hasn't yet been sent upstrema so I thought I would compare to 
your series to see if there are any differences.


Instead of adding support for parallel SMP boot, we decided to optimize 
some of the wait call (see diff below).


This was tested on a nested environment (KVM/QEMU) on c5 metal with 
x2apic disabled. The numbers are for bringing-up 95 CPUs:


  * Currently: 2s
  * With AWS change only: 300ms
  * With your change only: 100ms

So your approach is superior :). I see you are dropping the loop in 
smp_callin(). So the first hunk in the below diff is not necessary 
anymore. The second hunk probably still makes sense for CPU hotplug (and 
maybe S3 bring-up) even though it would only save 10ms. I will look to 
send the patch.


Diff:

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 3a1a659082c6..86fd5500e1ea 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -172,9 +172,9 @@ static void smp_callin(void)
 Dprintk("Waiting for CALLOUT.\n");
 for ( i = 0; cpu_state != CPU_STATE_CALLOUT; i++ )
 {
-BUG_ON(i >= 200);
+BUG_ON(i >= 20);
 cpu_relax();
-mdelay(10);
+udelay(10);
 }

 /*
@@ -430,6 +430,10 @@ static int wakeup_secondary_cpu(int phys_apicid, 
unsigned long start_eip)

  * Refer to AMD APM Vol2 15.27 "Secure Startup with SKINIT".
  */
 bool send_INIT = ap_boot_method != AP_BOOT_SKINIT;
+bool modern_cpu = ((boot_cpu_data.x86_vendor == X86_VENDOR_INTEL) &&
+   (boot_cpu_data.x86 == 6)) ||
+  ((boot_cpu_data.x86_vendor == X86_VENDOR_AMD) &&
+   (boot_cpu_data.x86 >= 0xF));

 /*
  * Some versions of tboot might be able to handle the entire wake 
sequence
@@ -464,7 +468,15 @@ static int wakeup_secondary_cpu(int phys_apicid, 
unsigned long start_eip)

 send_status = apic_read(APIC_ICR) & APIC_ICR_BUSY;
 } while ( send_status && (timeout++ < 1000) );

-mdelay(10);
+/*
+ * The Multiprocessor Specification 1.4 (1997) example code 
suggests
+ * that there should be a 10ms delay between the BSP 
asserting INIT

+ * and de-asserting INIT, when starting a remote processor.
+ * But that slows boot and resume on modern processors, 
which include

+ * many cores and don't require that delay.
+ */
+if ( !modern_cpu )
+mdelay(10);

 Dprintk("Deasserting INIT.\n");

Cheers,

--
Julien Grall

Re: [PATCH 01/10] x86/spec-ctrl: Remove conditional IRQs-on-ness for INT $0x80/0x82 paths

[Krystian Hebel]

Please ignore this series and review/reply to the other one
([XEN PATCH 0/9] x86: parallelize AP bring-up during boot) instead.
`git send-email` did what I asked it to do instead of what I wanted,
so I accidentally sent WIP version of patches.

Sorry for the noise.

--
Krystian Hebel
Firmware Engineer
https://3mdeb.com | @3mdeb_com

Re: [PATCH 5/7] xen/events: drop xen_allocate_irqs_dynamic()

[Oleksandr Tyshchenko]

On 14.11.23 10:35, Juergen Gross wrote:


Hello Juergen


> On 14.11.23 09:20, Oleksandr Tyshchenko wrote:
>>
>>
>> On 16.10.23 09:28, Juergen Gross wrote:
>>
>>
>> Hello Juergen
>>
>>> Instead of having a common function for allocating a single IRQ or a
>>> consecutive number of IRQs, split up the functionality into the callers
>>> of xen_allocate_irqs_dynamic().
>>>
>>> This allows to handle any allocation error in xen_irq_init() gracefully
>>> instead of panicing the system. Let xen_irq_init() return the irq_info
>>> pointer or NULL in case of an allocation error.
>>>
>>> Additionally set the IRQ into irq_info already at allocation time, as
>>> otherwise the IRQ would be '0' (which is a valid IRQ number) until
>>> being set.
>>>
>>> Signed-off-by: Juergen Gross 
>>> ---
>>>    drivers/xen/events/events_base.c | 74 
>>> +++-
>>>    1 file changed, 44 insertions(+), 30 deletions(-)
>>>
>>
>> [snip]
>>
>>> @@ -1725,6 +1738,7 @@ void rebind_evtchn_irq(evtchn_port_t evtchn, 
>>> int irq)
>>>   so there should be a proper type */
>>>    BUG_ON(info->type == IRQT_UNBOUND);
>>> +    info->irq = irq;
>>
>>
>> I failed to understand why this is added here. Doesn't irq remain the
>> same, and info->irq remains valid? Could you please clarify.
> 
> The IRQ remains the same, but the event channel could change.
> 
> This setting of info->irq compensates for the related removal in
> xen_irq_info_common_setup().


Thanks for the clarification, you can add my

Reviewed-by: Oleksandr Tyshchenko 

> 
>>
>> Other changes lgtm.
> 
> 
> Juergen
> 

Re: [PATCH 7/7] xen/events: remove some info_for_irq() calls in pirq handling

[Oleksandr Tyshchenko]

On 16.10.23 09:28, Juergen Gross wrote:

Hello Juergen


> Instead of the IRQ number user the struct irq_info pointer as parameter
> in the internal pirq related functions. This allows to drop some calls
> of info_for_irq().
> 
> Signed-off-by: Juergen Gross 


Looks good, so

Reviewed-by: Oleksandr Tyshchenko 


Just one NIT below ...


[snip]

>   
> -static void pirq_query_unmask(int irq)
> +static void pirq_query_unmask(struct irq_info *info)
>   {
>   struct physdev_irq_status_query irq_status;
> - struct irq_info *info = info_for_irq(irq);
>   
>   BUG_ON(info->type != IRQT_PIRQ);
>   
> - irq_status.irq = pirq_from_irq(irq);
> + irq_status.irq = info->u.pirq.pirq;


  ... what is the reason to open-code pirq_from_irq() here?
For example, __startup_pirq() continues to use helper in almost the same 
situation ...


[snip]

>   
> -static unsigned int __startup_pirq(unsigned int irq)
> +static unsigned int __startup_pirq(struct irq_info *info)
>   {
>   struct evtchn_bind_pirq bind_pirq;
> - struct irq_info *info = info_for_irq(irq);
> - evtchn_port_t evtchn = evtchn_from_irq(irq);
> + evtchn_port_t evtchn = info->evtchn;
>   int rc;
>   
>   BUG_ON(info->type != IRQT_PIRQ);
> @@ -851,20 +868,20 @@ static unsigned int __startup_pirq(unsigned int irq)
>   if (VALID_EVTCHN(evtchn))
>   goto out;
>   
> - bind_pirq.pirq = pirq_from_irq(irq);
> + bind_pirq.pirq = pirq_from_irq(info);

... here



[snip]

[PATCH 10/10] x86/smp: start APs in parallel during boot

[Krystian Hebel]

Multiple delays are required when sending IPIs and waiting for
responses. During boot, 4 such IPIs were sent per each AP. With this
change, only one set of broadcast IPIs is sent. This reduces boot time,
especially for platforms with large number of cores.

Single CPU initialization is still possible, it is used for hotplug.

During wakeup from S3 APs are started one by one. It should be possible
to enable parallel execution there as well, but I don't have a way of
properly testing it as of now.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/include/asm/smp.h |  1 +
 xen/arch/x86/setup.c   |  2 +
 xen/arch/x86/smpboot.c | 68 --
 3 files changed, 51 insertions(+), 20 deletions(-)

diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
index 98739028a6ed..6ca0158a368d 100644
--- a/xen/arch/x86/include/asm/smp.h
+++ b/xen/arch/x86/include/asm/smp.h
@@ -31,6 +31,7 @@ DECLARE_PER_CPU(cpumask_var_t, send_ipi_cpumask);
 extern bool park_offline_cpus;
 
 void smp_send_nmi_allbutself(void);
+void smp_send_init_sipi_sipi_allbutself(void);
 
 void send_IPI_mask(const cpumask_t *, int vector);
 void send_IPI_self(int vector);
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index b2c0679725ea..42a9067b81eb 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1963,6 +1963,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 cpu_data[i].stack_base = cpu_alloc_stack(i);
 }
 
+smp_send_init_sipi_sipi_allbutself();
+
 for_each_present_cpu ( i )
 {
 if ( (park_offline_cpus || num_online_cpus() < max_cpus) &&
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index cbea2d45f70d..e9a7f78a5a6f 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -425,7 +425,7 @@ void start_secondary(unsigned int cpu)
 
 static int wakeup_secondary_cpu(int phys_apicid, unsigned long start_eip)
 {
-unsigned long send_status = 0, accept_status = 0;
+unsigned long send_status = 0, accept_status = 0, sh = 0;
 int maxlvt, timeout, i;
 
 /*
@@ -445,6 +445,12 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
 if ( tboot_in_measured_env() && !tboot_wake_ap(phys_apicid, start_eip) )
 return 0;
 
+/*
+ * Use destination shorthand for broadcasting IPIs during boot.
+ */
+if ( phys_apicid == BAD_APICID )
+sh = APIC_DEST_ALLBUT;
+
 /*
  * Be paranoid about clearing APIC errors.
  */
@@ -458,7 +464,7 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
 /*
  * Turn INIT on target chip via IPI
  */
-apic_icr_write(APIC_INT_LEVELTRIG | APIC_INT_ASSERT | APIC_DM_INIT,
+apic_icr_write(APIC_INT_LEVELTRIG | APIC_INT_ASSERT | APIC_DM_INIT | 
sh,
phys_apicid);
 
 if ( !x2apic_enabled )
@@ -475,7 +481,7 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
 
 Dprintk("Deasserting INIT.\n");
 
-apic_icr_write(APIC_INT_LEVELTRIG | APIC_DM_INIT, phys_apicid);
+apic_icr_write(APIC_INT_LEVELTRIG | APIC_DM_INIT | sh, 
phys_apicid);
 
 Dprintk("Waiting for send to finish...\n");
 timeout = 0;
@@ -512,7 +518,7 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
  * STARTUP IPI
  * Boot on the stack
  */
-apic_icr_write(APIC_DM_STARTUP | (start_eip >> 12), phys_apicid);
+apic_icr_write(APIC_DM_STARTUP | (start_eip >> 12) | sh, phys_apicid);
 
 if ( !x2apic_enabled )
 {
@@ -565,7 +571,6 @@ int alloc_cpu_id(void)
 static int do_boot_cpu(int apicid, int cpu)
 {
 int timeout, boot_error = 0, rc = 0;
-unsigned long start_eip;
 
 /*
  * Save current MTRR state in case it was changed since early boot
@@ -573,21 +578,31 @@ static int do_boot_cpu(int apicid, int cpu)
  */
 mtrr_save_state();
 
-start_eip = bootsym_phys(trampoline_realmode_entry);
+/* Check if AP is already up. */
+if ( cpu_data[cpu].cpu_state != CPU_STATE_INIT )
+{
+/* This grunge runs the startup process for the targeted processor. */
+unsigned long start_eip;
+start_eip = bootsym_phys(trampoline_realmode_entry);
 
-/* start_eip needs be page aligned, and below the 1M boundary. */
-if ( start_eip & ~0xff000 )
-panic("AP trampoline %#lx not suitably positioned\n", start_eip);
+/* start_eip needs be page aligned, and below the 1M boundary. */
+if ( start_eip & ~0xff000 )
+panic("AP trampoline %#lx not suitably positioned\n", start_eip);
 
-/* So we see what's up   */
-if ( opt_cpu_info )
-printk("Booting processor %d/%d eip %lx\n",
-   cpu, apicid, start_eip);
+/* So we see what's up   */
+if ( opt_cpu_info )
+printk("AP 

[XEN PATCH 8/9] x86/smp: make cpu_state per-CPU

[Krystian Hebel]

This will be used for parallel AP bring-up.

CPU_STATE_INIT changed direction. It was previously set by BSP and never
consumed by AP. Now it signals that AP got through assembly part of
initialization and waits for BSP to call notifiers that set up data
structures required for further initialization.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/include/asm/cpufeature.h |  1 +
 xen/arch/x86/smpboot.c| 80 ---
 2 files changed, 49 insertions(+), 32 deletions(-)

diff --git a/xen/arch/x86/include/asm/cpufeature.h 
b/xen/arch/x86/include/asm/cpufeature.h
index ff0e18864cc7..1831b5fb368f 100644
--- a/xen/arch/x86/include/asm/cpufeature.h
+++ b/xen/arch/x86/include/asm/cpufeature.h
@@ -38,6 +38,7 @@ struct cpuinfo_x86 {
 unsigned int cpu_core_id;  /* core ID of each logical CPU */
 unsigned int compute_unit_id;  /* AMD compute unit ID of each logical 
CPU */
 void *stack_base;
+unsigned int cpu_state;
 unsigned short x86_clflush_size;
 } __cacheline_aligned;
 
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 39ffd356dbbc..cbea2d45f70d 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -65,15 +65,18 @@ struct cpuinfo_x86 cpu_data[NR_CPUS] =
 { [0 ... NR_CPUS-1] .apicid = BAD_APICID };
 
 static int cpu_error;
-static enum cpu_state {
+enum cpu_state {
 CPU_STATE_DYING,/* slave -> master: I am dying */
 CPU_STATE_DEAD, /* slave -> master: I am completely dead */
-CPU_STATE_INIT, /* master -> slave: Early bringup phase 1 */
-CPU_STATE_CALLOUT,  /* master -> slave: Early bringup phase 2 */
+CPU_STATE_INIT, /* slave -> master: Early bringup phase 1 completed */
+CPU_STATE_CALLOUT,  /* master -> slave: Start early bringup phase 2 */
 CPU_STATE_CALLIN,   /* slave -> master: Completed phase 2 */
 CPU_STATE_ONLINE/* master -> slave: Go fully online now. */
-} cpu_state;
-#define set_cpu_state(state) do { smp_mb(); cpu_state = (state); } while (0)
+};
+#define set_cpu_state(cpu, state) do { \
+smp_mb(); \
+cpu_data[cpu].cpu_state = (state); \
+} while (0)
 
 void initialize_cpu_data(unsigned int cpu)
 {
@@ -168,16 +171,7 @@ static void synchronize_tsc_slave(unsigned int slave)
 static void smp_callin(void)
 {
 unsigned int cpu = smp_processor_id();
-int i, rc;
-
-/* Wait 2s total for startup. */
-Dprintk("Waiting for CALLOUT.\n");
-for ( i = 0; cpu_state != CPU_STATE_CALLOUT; i++ )
-{
-BUG_ON(i >= 200);
-cpu_relax();
-mdelay(10);
-}
+int rc;
 
 /*
  * The boot CPU has finished the init stage and is spinning on cpu_state
@@ -213,12 +207,12 @@ static void smp_callin(void)
 }
 
 /* Allow the master to continue. */
-set_cpu_state(CPU_STATE_CALLIN);
+set_cpu_state(cpu, CPU_STATE_CALLIN);
 
 synchronize_tsc_slave(cpu);
 
 /* And wait for our final Ack. */
-while ( cpu_state != CPU_STATE_ONLINE )
+while ( cpu_data[cpu].cpu_state != CPU_STATE_ONLINE )
 cpu_relax();
 }
 
@@ -313,6 +307,9 @@ void start_secondary(unsigned int cpu)
 {
 struct cpu_info *info = get_cpu_info();
 
+/* Tell BSP that we are awake. */
+set_cpu_state(cpu, CPU_STATE_INIT);
+
 /*
  * Don't put anything before smp_callin(), SMP booting is so fragile that 
we
  * want to limit the things done here to the most necessary things.
@@ -320,6 +317,10 @@ void start_secondary(unsigned int cpu)
 
 /* Critical region without IDT or TSS.  Any fault is deadly! */
 
+/* Wait until data set up by CPU_UP_PREPARE notifiers is ready. */
+while ( cpu_data[cpu].cpu_state != CPU_STATE_CALLOUT )
+cpu_relax();
+
 set_current(idle_vcpu[cpu]);
 this_cpu(curr_vcpu) = idle_vcpu[cpu];
 rdmsrl(MSR_EFER, this_cpu(efer));
@@ -585,26 +586,35 @@ static int do_boot_cpu(int apicid, int cpu)
 
 /* This grunge runs the startup process for the targeted processor. */
 
-set_cpu_state(CPU_STATE_INIT);
-
 /* Starting actual IPI sequence... */
 boot_error = wakeup_secondary_cpu(apicid, start_eip);
 
 if ( !boot_error )
 {
-/* Allow AP to start initializing. */
-set_cpu_state(CPU_STATE_CALLOUT);
-Dprintk("After Callout %d.\n", cpu);
-
-/* Wait 5s total for a response. */
-for ( timeout = 0; timeout < 5; timeout++ )
+/* Wait 2s total for a response. */
+for ( timeout = 0; timeout < 2; timeout++ )
 {
-if ( cpu_state != CPU_STATE_CALLOUT )
+if ( cpu_data[cpu].cpu_state == CPU_STATE_INIT )
 break;
 udelay(100);
 }
 
-if ( cpu_state == CPU_STATE_CALLIN )
+if ( cpu_data[cpu].cpu_state == CPU_STATE_INIT )
+{
+/* Allow AP to start initializing. */
+set_cpu_state(cpu, CPU_STATE_CALLOUT);
+Dprintk("After Callout %d.\n", cpu);
+
+/* Wait 5s total for a 

[PATCH 08/10] x86/smp: drop booting_cpu variable

[Krystian Hebel]

CPU id is obtained as a side effect of searching for appropriate
stack for AP. It can be used as a parameter to start_secondary().
Coincidentally this also makes further work on making AP bring-up
code parallel easier.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/boot/x86_64.S | 13 +
 xen/arch/x86/smpboot.c | 15 +--
 2 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index 8d61f270761f..ad01f20d548d 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -20,20 +20,24 @@ ENTRY(__high_start)
 jz  .L_stack_set
 
 /* APs only: get stack base from APIC ID saved in %esp. */
-mov $0, %rax
+mov $0, %rbx
 lea cpu_data(%rip), %rcx
 /* cpu_data[0] is BSP, skip it. */
 1:
-add $1, %rax
+add $1, %rbx
 add $CPUINFO_X86_sizeof, %rcx
-cmp $NR_CPUS, %eax
+cmp $NR_CPUS, %rbx
 jb  2f
 hlt
 2:
 cmp %esp, CPUINFO_X86_apicid(%rcx)
 jne 1b
 
-/* %rcx is now cpu_data[cpu], read stack base from it. */
+/*
+ * At this point:
+ * - %rcx is cpu_data[cpu], read stack base from it,
+ * - %rbx (callee-save) is Xen cpu number, pass it to 
start_secondary().
+ */
 mov CPUINFO_X86_stack_base(%rcx), %rsp
 
 test%rsp,%rsp
@@ -101,6 +105,7 @@ ENTRY(__high_start)
 .L_ap_cet_done:
 #endif /* CONFIG_XEN_SHSTK || CONFIG_XEN_IBT */
 
+mov %rbx, %rdi
 tailcall start_secondary
 
 .L_bsp:
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index a3895dafa267..39ffd356dbbc 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -222,8 +222,6 @@ static void smp_callin(void)
 cpu_relax();
 }
 
-static int booting_cpu;
-
 /* CPUs for which sibling maps can be computed. */
 static cpumask_t cpu_sibling_setup_map;
 
@@ -311,15 +309,14 @@ static void set_cpu_sibling_map(unsigned int cpu)
 }
 }
 
-void start_secondary(void *unused)
+void start_secondary(unsigned int cpu)
 {
 struct cpu_info *info = get_cpu_info();
 
 /*
- * Dont put anything before smp_callin(), SMP booting is so fragile that we
+ * Don't put anything before smp_callin(), SMP booting is so fragile that 
we
  * want to limit the things done here to the most necessary things.
  */
-unsigned int cpu = booting_cpu;
 
 /* Critical region without IDT or TSS.  Any fault is deadly! */
 
@@ -346,9 +343,9 @@ void start_secondary(void *unused)
  */
 spin_debug_disable();
 
-get_cpu_info()->use_pv_cr3 = false;
-get_cpu_info()->xen_cr3 = 0;
-get_cpu_info()->pv_cr3 = 0;
+info->use_pv_cr3 = false;
+info->xen_cr3 = 0;
+info->pv_cr3 = 0;
 
 /*
  * BUG_ON() used in load_system_tables() and later code may end up calling
@@ -575,8 +572,6 @@ static int do_boot_cpu(int apicid, int cpu)
  */
 mtrr_save_state();
 
-booting_cpu = cpu;
-
 start_eip = bootsym_phys(trampoline_realmode_entry);
 
 /* start_eip needs be page aligned, and below the 1M boundary. */
-- 
2.41.0

[XEN PATCH 9/9] x86/smp: start APs in parallel during boot

[Krystian Hebel]

Multiple delays are required when sending IPIs and waiting for
responses. During boot, 4 such IPIs were sent per each AP. With this
change, only one set of broadcast IPIs is sent. This reduces boot time,
especially for platforms with large number of cores.

Single CPU initialization is still possible, it is used for hotplug.

During wakeup from S3 APs are started one by one. It should be possible
to enable parallel execution there as well, but I don't have a way of
properly testing it as of now.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/include/asm/smp.h |  1 +
 xen/arch/x86/setup.c   |  2 +
 xen/arch/x86/smpboot.c | 68 --
 3 files changed, 51 insertions(+), 20 deletions(-)

diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
index 98739028a6ed..6ca0158a368d 100644
--- a/xen/arch/x86/include/asm/smp.h
+++ b/xen/arch/x86/include/asm/smp.h
@@ -31,6 +31,7 @@ DECLARE_PER_CPU(cpumask_var_t, send_ipi_cpumask);
 extern bool park_offline_cpus;
 
 void smp_send_nmi_allbutself(void);
+void smp_send_init_sipi_sipi_allbutself(void);
 
 void send_IPI_mask(const cpumask_t *, int vector);
 void send_IPI_self(int vector);
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index b2c0679725ea..42a9067b81eb 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1963,6 +1963,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 cpu_data[i].stack_base = cpu_alloc_stack(i);
 }
 
+smp_send_init_sipi_sipi_allbutself();
+
 for_each_present_cpu ( i )
 {
 if ( (park_offline_cpus || num_online_cpus() < max_cpus) &&
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index cbea2d45f70d..e9a7f78a5a6f 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -425,7 +425,7 @@ void start_secondary(unsigned int cpu)
 
 static int wakeup_secondary_cpu(int phys_apicid, unsigned long start_eip)
 {
-unsigned long send_status = 0, accept_status = 0;
+unsigned long send_status = 0, accept_status = 0, sh = 0;
 int maxlvt, timeout, i;
 
 /*
@@ -445,6 +445,12 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
 if ( tboot_in_measured_env() && !tboot_wake_ap(phys_apicid, start_eip) )
 return 0;
 
+/*
+ * Use destination shorthand for broadcasting IPIs during boot.
+ */
+if ( phys_apicid == BAD_APICID )
+sh = APIC_DEST_ALLBUT;
+
 /*
  * Be paranoid about clearing APIC errors.
  */
@@ -458,7 +464,7 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
 /*
  * Turn INIT on target chip via IPI
  */
-apic_icr_write(APIC_INT_LEVELTRIG | APIC_INT_ASSERT | APIC_DM_INIT,
+apic_icr_write(APIC_INT_LEVELTRIG | APIC_INT_ASSERT | APIC_DM_INIT | 
sh,
phys_apicid);
 
 if ( !x2apic_enabled )
@@ -475,7 +481,7 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
 
 Dprintk("Deasserting INIT.\n");
 
-apic_icr_write(APIC_INT_LEVELTRIG | APIC_DM_INIT, phys_apicid);
+apic_icr_write(APIC_INT_LEVELTRIG | APIC_DM_INIT | sh, 
phys_apicid);
 
 Dprintk("Waiting for send to finish...\n");
 timeout = 0;
@@ -512,7 +518,7 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
  * STARTUP IPI
  * Boot on the stack
  */
-apic_icr_write(APIC_DM_STARTUP | (start_eip >> 12), phys_apicid);
+apic_icr_write(APIC_DM_STARTUP | (start_eip >> 12) | sh, phys_apicid);
 
 if ( !x2apic_enabled )
 {
@@ -565,7 +571,6 @@ int alloc_cpu_id(void)
 static int do_boot_cpu(int apicid, int cpu)
 {
 int timeout, boot_error = 0, rc = 0;
-unsigned long start_eip;
 
 /*
  * Save current MTRR state in case it was changed since early boot
@@ -573,21 +578,31 @@ static int do_boot_cpu(int apicid, int cpu)
  */
 mtrr_save_state();
 
-start_eip = bootsym_phys(trampoline_realmode_entry);
+/* Check if AP is already up. */
+if ( cpu_data[cpu].cpu_state != CPU_STATE_INIT )
+{
+/* This grunge runs the startup process for the targeted processor. */
+unsigned long start_eip;
+start_eip = bootsym_phys(trampoline_realmode_entry);
 
-/* start_eip needs be page aligned, and below the 1M boundary. */
-if ( start_eip & ~0xff000 )
-panic("AP trampoline %#lx not suitably positioned\n", start_eip);
+/* start_eip needs be page aligned, and below the 1M boundary. */
+if ( start_eip & ~0xff000 )
+panic("AP trampoline %#lx not suitably positioned\n", start_eip);
 
-/* So we see what's up   */
-if ( opt_cpu_info )
-printk("Booting processor %d/%d eip %lx\n",
-   cpu, apicid, start_eip);
+/* So we see what's up   */
+if ( opt_cpu_info )
+printk("AP 

[PATCH 07/10] x86/shutdown: protect against recurrent machine_restart()

[Krystian Hebel]

If multiple CPUs called machine_restart() before actual restart took
place, but after boot CPU declared itself not online, ASSERT in
on_selected_cpus() will fail. Few calls later execution would end up
in machine_restart() again, with another frame on call stack for new
exception.

To protect against running out of stack, code checks if boot CPU is
still online before calling on_selected_cpus().

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/shutdown.c | 20 +---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/shutdown.c b/xen/arch/x86/shutdown.c
index 7619544d14da..32c70505ed77 100644
--- a/xen/arch/x86/shutdown.c
+++ b/xen/arch/x86/shutdown.c
@@ -577,9 +577,23 @@ void machine_restart(unsigned int delay_millisecs)
 /* Ensure we are the boot CPU. */
 if ( get_apic_id() != boot_cpu_physical_apicid )
 {
-/* Send IPI to the boot CPU (logical cpu 0). */
-on_selected_cpus(cpumask_of(0), __machine_restart,
- _millisecs, 0);
+/*
+ * Send IPI to the boot CPU (logical cpu 0).
+ *
+ * If multiple CPUs called machine_restart() before actual restart
+ * took place, but after boot CPU declared itself not online, 
ASSERT
+ * in on_selected_cpus() will fail. Few calls later we would end up
+ * here again, with another frame on call stack for new exception.
+ * To protect against running out of stack, check if boot CPU is
+ * online.
+ *
+ * Note this is not an atomic operation, so it is possible for
+ * on_selected_cpus() to be called once after boot CPU is offline
+ * before we hit halt() below.
+ */
+if ( cpu_online(0) )
+on_selected_cpus(cpumask_of(0), __machine_restart,
+ _millisecs, 0);
 for ( ; ; )
 halt();
 }
-- 
2.41.0

[XEN PATCH 5/9] x86/smp: call x2apic_ap_setup() earlier

[Krystian Hebel]

It used to be called from smp_callin(), however BUG_ON() was invoked on
multiple occasions before that. It may end up calling machine_restart()
which tries to get APIC ID for CPU running this code. If BSP detected
that x2APIC is enabled, get_apic_id() will try to use it for all CPUs.
Enabling x2APIC on secondary CPUs earlier protects against an endless
loop of #GP exceptions caused by attempts to read IA32_X2APIC_APICID
MSR while x2APIC is disabled in IA32_APIC_BASE.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/smpboot.c | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 8ae65ab1769f..a3895dafa267 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -184,7 +184,6 @@ static void smp_callin(void)
  * update until we finish. We are free to set up this CPU: first the APIC.
  */
 Dprintk("CALLIN, before setup_local_APIC().\n");
-x2apic_ap_setup();
 setup_local_APIC(false);
 
 /* Save our processor parameters. */
@@ -351,6 +350,14 @@ void start_secondary(void *unused)
 get_cpu_info()->xen_cr3 = 0;
 get_cpu_info()->pv_cr3 = 0;
 
+/*
+ * BUG_ON() used in load_system_tables() and later code may end up calling
+ * machine_restart() which tries to get APIC ID for CPU running this code.
+ * If BSP detected that x2APIC is enabled, get_apic_id() will try to use it
+ * for _all_ CPUs. Enable x2APIC on secondary CPUs now so we won't end up
+ * with endless #GP loop.
+ */
+x2apic_ap_setup();
 load_system_tables();
 
 /* Full exception support from here on in. */
-- 
2.41.0

[PATCH 06/10] x86/smp: call x2apic_ap_setup() earlier

[Krystian Hebel]

It used to be called from smp_callin(), however BUG_ON() was invoked on
multiple occasions before that. It may end up calling machine_restart()
which tries to get APIC ID for CPU running this code. If BSP detected
that x2APIC is enabled, get_apic_id() will try to use it for all CPUs.
Enabling x2APIC on secondary CPUs earlier protects against an endless
loop of #GP exceptions caused by attempts to read IA32_X2APIC_APICID
MSR while x2APIC is disabled in IA32_APIC_BASE.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/smpboot.c | 9 -
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 8ae65ab1769f..a3895dafa267 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -184,7 +184,6 @@ static void smp_callin(void)
  * update until we finish. We are free to set up this CPU: first the APIC.
  */
 Dprintk("CALLIN, before setup_local_APIC().\n");
-x2apic_ap_setup();
 setup_local_APIC(false);
 
 /* Save our processor parameters. */
@@ -351,6 +350,14 @@ void start_secondary(void *unused)
 get_cpu_info()->xen_cr3 = 0;
 get_cpu_info()->pv_cr3 = 0;
 
+/*
+ * BUG_ON() used in load_system_tables() and later code may end up calling
+ * machine_restart() which tries to get APIC ID for CPU running this code.
+ * If BSP detected that x2APIC is enabled, get_apic_id() will try to use it
+ * for _all_ CPUs. Enable x2APIC on secondary CPUs now so we won't end up
+ * with endless #GP loop.
+ */
+x2apic_ap_setup();
 load_system_tables();
 
 /* Full exception support from here on in. */
-- 
2.41.0

[XEN PATCH 3/9] x86/smp: drop x86_cpu_to_apicid, use cpu_data[cpu].apicid instead

[Krystian Hebel]

Both fields held the same data.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/boot/x86_64.S   |  8 +---
 xen/arch/x86/include/asm/asm_defns.h |  2 +-
 xen/arch/x86/include/asm/processor.h |  2 ++
 xen/arch/x86/include/asm/smp.h   |  4 
 xen/arch/x86/numa.c  | 15 +++
 xen/arch/x86/smpboot.c   |  8 
 xen/arch/x86/x86_64/asm-offsets.c|  4 +++-
 7 files changed, 22 insertions(+), 21 deletions(-)

diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index b85b47b5c1a0..195550b5c0ea 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -20,15 +20,17 @@ ENTRY(__high_start)
 jz  .L_stack_set
 
 /* APs only: get stack base from APIC ID saved in %esp. */
-mov $-1, %rax
-lea x86_cpu_to_apicid(%rip), %rcx
+mov $0, %rax
+lea cpu_data(%rip), %rcx
+/* cpu_data[0] is BSP, skip it. */
 1:
 add $1, %rax
+add $CPUINFO_X86_sizeof, %rcx
 cmp $NR_CPUS, %eax
 jb  2f
 hlt
 2:
-cmp %esp, (%rcx, %rax, 4)
+cmp %esp, CPUINFO_X86_apicid(%rcx)
 jne 1b
 
 /* %eax is now Xen CPU index. */
diff --git a/xen/arch/x86/include/asm/asm_defns.h 
b/xen/arch/x86/include/asm/asm_defns.h
index baaaccb26e17..6b05d9d140b8 100644
--- a/xen/arch/x86/include/asm/asm_defns.h
+++ b/xen/arch/x86/include/asm/asm_defns.h
@@ -158,7 +158,7 @@ register unsigned long current_stack_pointer asm("rsp");
 #endif
 
 #define CPUINFO_FEATURE_OFFSET(feature)   \
-(CPUINFO_features + (cpufeat_word(feature) * 4))
+(CPUINFO_X86_features + (cpufeat_word(feature) * 4))
 
 #else
 
diff --git a/xen/arch/x86/include/asm/processor.h 
b/xen/arch/x86/include/asm/processor.h
index b0d2a62c075f..8345d58094da 100644
--- a/xen/arch/x86/include/asm/processor.h
+++ b/xen/arch/x86/include/asm/processor.h
@@ -92,6 +92,8 @@ struct x86_cpu_id {
 extern struct cpuinfo_x86 cpu_data[];
 #define current_cpu_data cpu_data[smp_processor_id()]
 
+#define cpu_physical_id(cpu)   cpu_data[cpu].apicid
+
 extern bool probe_cpuid_faulting(void);
 extern void ctxt_switch_levelling(const struct vcpu *next);
 extern void (*ctxt_switch_masking)(const struct vcpu *next);
diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
index c0b5d7cdd8dd..94c557491860 100644
--- a/xen/arch/x86/include/asm/smp.h
+++ b/xen/arch/x86/include/asm/smp.h
@@ -39,10 +39,6 @@ extern void (*mtrr_hook) (void);
 
 extern void zap_low_mappings(void);
 
-extern u32 x86_cpu_to_apicid[];
-
-#define cpu_physical_id(cpu)   x86_cpu_to_apicid[cpu]
-
 #define cpu_is_offline(cpu) unlikely(!cpu_online(cpu))
 extern void cpu_exit_clear(unsigned int cpu);
 extern void cpu_uninit(unsigned int cpu);
diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 39e131cb4f35..91527be5b406 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -54,14 +54,13 @@ bool __init arch_numa_unavailable(void)
 /*
  * Setup early cpu_to_node.
  *
- * Populate cpu_to_node[] only if x86_cpu_to_apicid[],
- * and apicid_to_node[] tables have valid entries for a CPU.
- * This means we skip cpu_to_node[] initialisation for NUMA
- * emulation and faking node case (when running a kernel compiled
- * for NUMA on a non NUMA box), which is OK as cpu_to_node[]
- * is already initialized in a round robin manner at numa_init_array,
- * prior to this call, and this initialization is good enough
- * for the fake NUMA cases.
+ * Populate cpu_to_node[] only if cpu_data[], and apicid_to_node[]
+ * tables have valid entries for a CPU. This means we skip
+ * cpu_to_node[] initialisation for NUMA emulation and faking node
+ * case (when running a kernel compiled for NUMA on a non NUMA box),
+ * which is OK as cpu_to_node[] is already initialized in a round
+ * robin manner at numa_init_array, prior to this call, and this
+ * initialization is good enough for the fake NUMA cases.
  */
 void __init init_cpu_to_node(void)
 {
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index de87c5a41926..f061486e56eb 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -61,10 +61,8 @@ unsigned int __read_mostly nr_sockets;
 cpumask_t **__read_mostly socket_cpumask;
 static cpumask_t *secondary_socket_cpumask;
 
-struct cpuinfo_x86 cpu_data[NR_CPUS];
-
-u32 x86_cpu_to_apicid[NR_CPUS] __read_mostly =
-   { [0 ... NR_CPUS-1] = BAD_APICID };
+struct cpuinfo_x86 cpu_data[NR_CPUS] =
+{ [0 ... NR_CPUS-1] .apicid = BAD_APICID };
 
 static int cpu_error;
 static enum cpu_state {
@@ -81,7 +79,9 @@ void *stack_base[NR_CPUS];
 
 void initialize_cpu_data(unsigned int cpu)
 {
+uint32_t apicid = cpu_physical_id(cpu);
 cpu_data[cpu] = boot_cpu_data;
+cpu_physical_id(cpu) = apicid;
 }
 
 static bool smp_store_cpu_info(unsigned int id)
diff --git a/xen/arch/x86/x86_64/asm-offsets.c 
b/xen/arch/x86/x86_64/asm-offsets.c
index 

[PATCH 2/2] x86/smp: start APs in parallel during boot

[Krystian Hebel]

Multiple delays are required when sending IPIs and waiting for
responses. During boot, 4 such IPIs were sent per each AP. With this
change, only one set of broadcast IPIs is sent. This reduces boot time,
especially for platforms with large number of cores.

Single CPU initialization is still possible, it is used for hotplug.

During wakeup from S3 APs are started one by one. It should be possible
to enable parallel execution there as well, but I don't have a way of
properly testing it as of now.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/include/asm/smp.h |  1 +
 xen/arch/x86/setup.c   |  2 +
 xen/arch/x86/smpboot.c | 68 --
 3 files changed, 51 insertions(+), 20 deletions(-)

diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
index 98739028a6ed..6ca0158a368d 100644
--- a/xen/arch/x86/include/asm/smp.h
+++ b/xen/arch/x86/include/asm/smp.h
@@ -31,6 +31,7 @@ DECLARE_PER_CPU(cpumask_var_t, send_ipi_cpumask);
 extern bool park_offline_cpus;
 
 void smp_send_nmi_allbutself(void);
+void smp_send_init_sipi_sipi_allbutself(void);
 
 void send_IPI_mask(const cpumask_t *, int vector);
 void send_IPI_self(int vector);
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index b2c0679725ea..42a9067b81eb 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1963,6 +1963,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 cpu_data[i].stack_base = cpu_alloc_stack(i);
 }
 
+smp_send_init_sipi_sipi_allbutself();
+
 for_each_present_cpu ( i )
 {
 if ( (park_offline_cpus || num_online_cpus() < max_cpus) &&
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index cbea2d45f70d..e9a7f78a5a6f 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -425,7 +425,7 @@ void start_secondary(unsigned int cpu)
 
 static int wakeup_secondary_cpu(int phys_apicid, unsigned long start_eip)
 {
-unsigned long send_status = 0, accept_status = 0;
+unsigned long send_status = 0, accept_status = 0, sh = 0;
 int maxlvt, timeout, i;
 
 /*
@@ -445,6 +445,12 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
 if ( tboot_in_measured_env() && !tboot_wake_ap(phys_apicid, start_eip) )
 return 0;
 
+/*
+ * Use destination shorthand for broadcasting IPIs during boot.
+ */
+if ( phys_apicid == BAD_APICID )
+sh = APIC_DEST_ALLBUT;
+
 /*
  * Be paranoid about clearing APIC errors.
  */
@@ -458,7 +464,7 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
 /*
  * Turn INIT on target chip via IPI
  */
-apic_icr_write(APIC_INT_LEVELTRIG | APIC_INT_ASSERT | APIC_DM_INIT,
+apic_icr_write(APIC_INT_LEVELTRIG | APIC_INT_ASSERT | APIC_DM_INIT | 
sh,
phys_apicid);
 
 if ( !x2apic_enabled )
@@ -475,7 +481,7 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
 
 Dprintk("Deasserting INIT.\n");
 
-apic_icr_write(APIC_INT_LEVELTRIG | APIC_DM_INIT, phys_apicid);
+apic_icr_write(APIC_INT_LEVELTRIG | APIC_DM_INIT | sh, 
phys_apicid);
 
 Dprintk("Waiting for send to finish...\n");
 timeout = 0;
@@ -512,7 +518,7 @@ static int wakeup_secondary_cpu(int phys_apicid, unsigned 
long start_eip)
  * STARTUP IPI
  * Boot on the stack
  */
-apic_icr_write(APIC_DM_STARTUP | (start_eip >> 12), phys_apicid);
+apic_icr_write(APIC_DM_STARTUP | (start_eip >> 12) | sh, phys_apicid);
 
 if ( !x2apic_enabled )
 {
@@ -565,7 +571,6 @@ int alloc_cpu_id(void)
 static int do_boot_cpu(int apicid, int cpu)
 {
 int timeout, boot_error = 0, rc = 0;
-unsigned long start_eip;
 
 /*
  * Save current MTRR state in case it was changed since early boot
@@ -573,21 +578,31 @@ static int do_boot_cpu(int apicid, int cpu)
  */
 mtrr_save_state();
 
-start_eip = bootsym_phys(trampoline_realmode_entry);
+/* Check if AP is already up. */
+if ( cpu_data[cpu].cpu_state != CPU_STATE_INIT )
+{
+/* This grunge runs the startup process for the targeted processor. */
+unsigned long start_eip;
+start_eip = bootsym_phys(trampoline_realmode_entry);
 
-/* start_eip needs be page aligned, and below the 1M boundary. */
-if ( start_eip & ~0xff000 )
-panic("AP trampoline %#lx not suitably positioned\n", start_eip);
+/* start_eip needs be page aligned, and below the 1M boundary. */
+if ( start_eip & ~0xff000 )
+panic("AP trampoline %#lx not suitably positioned\n", start_eip);
 
-/* So we see what's up   */
-if ( opt_cpu_info )
-printk("Booting processor %d/%d eip %lx\n",
-   cpu, apicid, start_eip);
+/* So we see what's up   */
+if ( opt_cpu_info )
+printk("AP 

[PATCH 04/10] x86/smp: drop x86_cpu_to_apicid, use cpu_data[cpu].apicid instead

[Krystian Hebel]

Both fields held the same data.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/boot/x86_64.S   |  8 +---
 xen/arch/x86/include/asm/asm_defns.h |  2 +-
 xen/arch/x86/include/asm/processor.h |  2 ++
 xen/arch/x86/include/asm/smp.h   |  4 
 xen/arch/x86/numa.c  | 15 +++
 xen/arch/x86/smpboot.c   |  8 
 xen/arch/x86/x86_64/asm-offsets.c|  4 +++-
 7 files changed, 22 insertions(+), 21 deletions(-)

diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index b85b47b5c1a0..195550b5c0ea 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -20,15 +20,17 @@ ENTRY(__high_start)
 jz  .L_stack_set
 
 /* APs only: get stack base from APIC ID saved in %esp. */
-mov $-1, %rax
-lea x86_cpu_to_apicid(%rip), %rcx
+mov $0, %rax
+lea cpu_data(%rip), %rcx
+/* cpu_data[0] is BSP, skip it. */
 1:
 add $1, %rax
+add $CPUINFO_X86_sizeof, %rcx
 cmp $NR_CPUS, %eax
 jb  2f
 hlt
 2:
-cmp %esp, (%rcx, %rax, 4)
+cmp %esp, CPUINFO_X86_apicid(%rcx)
 jne 1b
 
 /* %eax is now Xen CPU index. */
diff --git a/xen/arch/x86/include/asm/asm_defns.h 
b/xen/arch/x86/include/asm/asm_defns.h
index baaaccb26e17..6b05d9d140b8 100644
--- a/xen/arch/x86/include/asm/asm_defns.h
+++ b/xen/arch/x86/include/asm/asm_defns.h
@@ -158,7 +158,7 @@ register unsigned long current_stack_pointer asm("rsp");
 #endif
 
 #define CPUINFO_FEATURE_OFFSET(feature)   \
-(CPUINFO_features + (cpufeat_word(feature) * 4))
+(CPUINFO_X86_features + (cpufeat_word(feature) * 4))
 
 #else
 
diff --git a/xen/arch/x86/include/asm/processor.h 
b/xen/arch/x86/include/asm/processor.h
index b0d2a62c075f..8345d58094da 100644
--- a/xen/arch/x86/include/asm/processor.h
+++ b/xen/arch/x86/include/asm/processor.h
@@ -92,6 +92,8 @@ struct x86_cpu_id {
 extern struct cpuinfo_x86 cpu_data[];
 #define current_cpu_data cpu_data[smp_processor_id()]
 
+#define cpu_physical_id(cpu)   cpu_data[cpu].apicid
+
 extern bool probe_cpuid_faulting(void);
 extern void ctxt_switch_levelling(const struct vcpu *next);
 extern void (*ctxt_switch_masking)(const struct vcpu *next);
diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
index c0b5d7cdd8dd..94c557491860 100644
--- a/xen/arch/x86/include/asm/smp.h
+++ b/xen/arch/x86/include/asm/smp.h
@@ -39,10 +39,6 @@ extern void (*mtrr_hook) (void);
 
 extern void zap_low_mappings(void);
 
-extern u32 x86_cpu_to_apicid[];
-
-#define cpu_physical_id(cpu)   x86_cpu_to_apicid[cpu]
-
 #define cpu_is_offline(cpu) unlikely(!cpu_online(cpu))
 extern void cpu_exit_clear(unsigned int cpu);
 extern void cpu_uninit(unsigned int cpu);
diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 39e131cb4f35..91527be5b406 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -54,14 +54,13 @@ bool __init arch_numa_unavailable(void)
 /*
  * Setup early cpu_to_node.
  *
- * Populate cpu_to_node[] only if x86_cpu_to_apicid[],
- * and apicid_to_node[] tables have valid entries for a CPU.
- * This means we skip cpu_to_node[] initialisation for NUMA
- * emulation and faking node case (when running a kernel compiled
- * for NUMA on a non NUMA box), which is OK as cpu_to_node[]
- * is already initialized in a round robin manner at numa_init_array,
- * prior to this call, and this initialization is good enough
- * for the fake NUMA cases.
+ * Populate cpu_to_node[] only if cpu_data[], and apicid_to_node[]
+ * tables have valid entries for a CPU. This means we skip
+ * cpu_to_node[] initialisation for NUMA emulation and faking node
+ * case (when running a kernel compiled for NUMA on a non NUMA box),
+ * which is OK as cpu_to_node[] is already initialized in a round
+ * robin manner at numa_init_array, prior to this call, and this
+ * initialization is good enough for the fake NUMA cases.
  */
 void __init init_cpu_to_node(void)
 {
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index de87c5a41926..f061486e56eb 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -61,10 +61,8 @@ unsigned int __read_mostly nr_sockets;
 cpumask_t **__read_mostly socket_cpumask;
 static cpumask_t *secondary_socket_cpumask;
 
-struct cpuinfo_x86 cpu_data[NR_CPUS];
-
-u32 x86_cpu_to_apicid[NR_CPUS] __read_mostly =
-   { [0 ... NR_CPUS-1] = BAD_APICID };
+struct cpuinfo_x86 cpu_data[NR_CPUS] =
+{ [0 ... NR_CPUS-1] .apicid = BAD_APICID };
 
 static int cpu_error;
 static enum cpu_state {
@@ -81,7 +79,9 @@ void *stack_base[NR_CPUS];
 
 void initialize_cpu_data(unsigned int cpu)
 {
+uint32_t apicid = cpu_physical_id(cpu);
 cpu_data[cpu] = boot_cpu_data;
+cpu_physical_id(cpu) = apicid;
 }
 
 static bool smp_store_cpu_info(unsigned int id)
diff --git a/xen/arch/x86/x86_64/asm-offsets.c 
b/xen/arch/x86/x86_64/asm-offsets.c
index 

[PATCH 09/10] x86/smp: make cpu_state per-CPU

[Krystian Hebel]

This will be used for parallel AP bring-up.

CPU_STATE_INIT changed direction. It was previously set by BSP and never
consumed by AP. Now it signals that AP got through assembly part of
initialization and waits for BSP to call notifiers that set up data
structures required for further initialization.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/include/asm/cpufeature.h |  1 +
 xen/arch/x86/smpboot.c| 80 ---
 2 files changed, 49 insertions(+), 32 deletions(-)

diff --git a/xen/arch/x86/include/asm/cpufeature.h 
b/xen/arch/x86/include/asm/cpufeature.h
index ff0e18864cc7..1831b5fb368f 100644
--- a/xen/arch/x86/include/asm/cpufeature.h
+++ b/xen/arch/x86/include/asm/cpufeature.h
@@ -38,6 +38,7 @@ struct cpuinfo_x86 {
 unsigned int cpu_core_id;  /* core ID of each logical CPU */
 unsigned int compute_unit_id;  /* AMD compute unit ID of each logical 
CPU */
 void *stack_base;
+unsigned int cpu_state;
 unsigned short x86_clflush_size;
 } __cacheline_aligned;
 
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 39ffd356dbbc..cbea2d45f70d 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -65,15 +65,18 @@ struct cpuinfo_x86 cpu_data[NR_CPUS] =
 { [0 ... NR_CPUS-1] .apicid = BAD_APICID };
 
 static int cpu_error;
-static enum cpu_state {
+enum cpu_state {
 CPU_STATE_DYING,/* slave -> master: I am dying */
 CPU_STATE_DEAD, /* slave -> master: I am completely dead */
-CPU_STATE_INIT, /* master -> slave: Early bringup phase 1 */
-CPU_STATE_CALLOUT,  /* master -> slave: Early bringup phase 2 */
+CPU_STATE_INIT, /* slave -> master: Early bringup phase 1 completed */
+CPU_STATE_CALLOUT,  /* master -> slave: Start early bringup phase 2 */
 CPU_STATE_CALLIN,   /* slave -> master: Completed phase 2 */
 CPU_STATE_ONLINE/* master -> slave: Go fully online now. */
-} cpu_state;
-#define set_cpu_state(state) do { smp_mb(); cpu_state = (state); } while (0)
+};
+#define set_cpu_state(cpu, state) do { \
+smp_mb(); \
+cpu_data[cpu].cpu_state = (state); \
+} while (0)
 
 void initialize_cpu_data(unsigned int cpu)
 {
@@ -168,16 +171,7 @@ static void synchronize_tsc_slave(unsigned int slave)
 static void smp_callin(void)
 {
 unsigned int cpu = smp_processor_id();
-int i, rc;
-
-/* Wait 2s total for startup. */
-Dprintk("Waiting for CALLOUT.\n");
-for ( i = 0; cpu_state != CPU_STATE_CALLOUT; i++ )
-{
-BUG_ON(i >= 200);
-cpu_relax();
-mdelay(10);
-}
+int rc;
 
 /*
  * The boot CPU has finished the init stage and is spinning on cpu_state
@@ -213,12 +207,12 @@ static void smp_callin(void)
 }
 
 /* Allow the master to continue. */
-set_cpu_state(CPU_STATE_CALLIN);
+set_cpu_state(cpu, CPU_STATE_CALLIN);
 
 synchronize_tsc_slave(cpu);
 
 /* And wait for our final Ack. */
-while ( cpu_state != CPU_STATE_ONLINE )
+while ( cpu_data[cpu].cpu_state != CPU_STATE_ONLINE )
 cpu_relax();
 }
 
@@ -313,6 +307,9 @@ void start_secondary(unsigned int cpu)
 {
 struct cpu_info *info = get_cpu_info();
 
+/* Tell BSP that we are awake. */
+set_cpu_state(cpu, CPU_STATE_INIT);
+
 /*
  * Don't put anything before smp_callin(), SMP booting is so fragile that 
we
  * want to limit the things done here to the most necessary things.
@@ -320,6 +317,10 @@ void start_secondary(unsigned int cpu)
 
 /* Critical region without IDT or TSS.  Any fault is deadly! */
 
+/* Wait until data set up by CPU_UP_PREPARE notifiers is ready. */
+while ( cpu_data[cpu].cpu_state != CPU_STATE_CALLOUT )
+cpu_relax();
+
 set_current(idle_vcpu[cpu]);
 this_cpu(curr_vcpu) = idle_vcpu[cpu];
 rdmsrl(MSR_EFER, this_cpu(efer));
@@ -585,26 +586,35 @@ static int do_boot_cpu(int apicid, int cpu)
 
 /* This grunge runs the startup process for the targeted processor. */
 
-set_cpu_state(CPU_STATE_INIT);
-
 /* Starting actual IPI sequence... */
 boot_error = wakeup_secondary_cpu(apicid, start_eip);
 
 if ( !boot_error )
 {
-/* Allow AP to start initializing. */
-set_cpu_state(CPU_STATE_CALLOUT);
-Dprintk("After Callout %d.\n", cpu);
-
-/* Wait 5s total for a response. */
-for ( timeout = 0; timeout < 5; timeout++ )
+/* Wait 2s total for a response. */
+for ( timeout = 0; timeout < 2; timeout++ )
 {
-if ( cpu_state != CPU_STATE_CALLOUT )
+if ( cpu_data[cpu].cpu_state == CPU_STATE_INIT )
 break;
 udelay(100);
 }
 
-if ( cpu_state == CPU_STATE_CALLIN )
+if ( cpu_data[cpu].cpu_state == CPU_STATE_INIT )
+{
+/* Allow AP to start initializing. */
+set_cpu_state(cpu, CPU_STATE_CALLOUT);
+Dprintk("After Callout %d.\n", cpu);
+
+/* Wait 5s total for a 

[XEN PATCH 2/9] x86: don't access x86_cpu_to_apicid[] directly, use cpu_physical_id(cpu)

[Krystian Hebel]

This is done in preparation to move data from x86_cpu_to_apicid[]
elsewhere.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/acpi/cpu_idle.c  |  4 ++--
 xen/arch/x86/acpi/lib.c   |  2 +-
 xen/arch/x86/apic.c   |  2 +-
 xen/arch/x86/cpu/mwait-idle.c |  4 ++--
 xen/arch/x86/domain.c |  2 +-
 xen/arch/x86/mpparse.c|  6 +++---
 xen/arch/x86/numa.c   |  2 +-
 xen/arch/x86/platform_hypercall.c |  2 +-
 xen/arch/x86/setup.c  | 14 +++---
 xen/arch/x86/smpboot.c|  4 ++--
 xen/arch/x86/spec_ctrl.c  |  2 +-
 xen/arch/x86/sysctl.c |  2 +-
 12 files changed, 23 insertions(+), 23 deletions(-)

diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
index cfce4cc0408f..d03e54bcef38 100644
--- a/xen/arch/x86/acpi/cpu_idle.c
+++ b/xen/arch/x86/acpi/cpu_idle.c
@@ -1256,7 +1256,7 @@ int get_cpu_id(u32 acpi_id)
 
 for ( i = 0; i < nr_cpu_ids; i++ )
 {
-if ( apic_id == x86_cpu_to_apicid[i] )
+if ( apic_id == cpu_physical_id(i) )
 return i;
 }
 
@@ -1362,7 +1362,7 @@ long set_cx_pminfo(uint32_t acpi_id, struct 
xen_processor_power *power)
 
 if ( !cpu_online(cpu_id) )
 {
-uint32_t apic_id = x86_cpu_to_apicid[cpu_id];
+uint32_t apic_id = cpu_physical_id(cpu_id);
 
 /*
  * If we've just learned of more available C states, wake the CPU if
diff --git a/xen/arch/x86/acpi/lib.c b/xen/arch/x86/acpi/lib.c
index 51cb082ca02a..87a1f38f3f5a 100644
--- a/xen/arch/x86/acpi/lib.c
+++ b/xen/arch/x86/acpi/lib.c
@@ -91,7 +91,7 @@ unsigned int acpi_get_processor_id(unsigned int cpu)
 {
unsigned int acpiid, apicid;
 
-   if ((apicid = x86_cpu_to_apicid[cpu]) == BAD_APICID)
+   if ((apicid = cpu_physical_id(cpu)) == BAD_APICID)
return INVALID_ACPIID;
 
for (acpiid = 0; acpiid < ARRAY_SIZE(x86_acpiid_to_apicid); acpiid++)
diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
index 6acdd0ec1468..63e18968e54c 100644
--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -950,7 +950,7 @@ __next:
  */
 if (boot_cpu_physical_apicid == -1U)
 boot_cpu_physical_apicid = get_apic_id();
-x86_cpu_to_apicid[0] = get_apic_id();
+cpu_physical_id(0) = get_apic_id();
 
 ioapic_init();
 }
diff --git a/xen/arch/x86/cpu/mwait-idle.c b/xen/arch/x86/cpu/mwait-idle.c
index ff5c808bc952..88168da8a0ca 100644
--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -1202,8 +1202,8 @@ static void __init ivt_idle_state_table_update(void)
unsigned int cpu, max_apicid = boot_cpu_physical_apicid;
 
for_each_present_cpu(cpu)
-   if (max_apicid < x86_cpu_to_apicid[cpu])
-   max_apicid = x86_cpu_to_apicid[cpu];
+   if (max_apicid < cpu_physical_id(cpu))
+   max_apicid = cpu_physical_id(cpu);
switch (apicid_to_socket(max_apicid)) {
case 0: case 1:
/* 1 and 2 socket systems use default ivt_cstates */
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 3712e36df930..f45437511a46 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -1615,7 +1615,7 @@ long do_vcpu_op(int cmd, unsigned int vcpuid, 
XEN_GUEST_HANDLE_PARAM(void) arg)
 break;
 
 cpu_id.phys_id =
-(uint64_t)x86_cpu_to_apicid[v->vcpu_id] |
+(uint64_t)cpu_physical_id(v->vcpu_id) |
 ((uint64_t)acpi_get_processor_id(v->vcpu_id) << 32);
 
 rc = -EFAULT;
diff --git a/xen/arch/x86/mpparse.c b/xen/arch/x86/mpparse.c
index d8ccab2449c6..b8cabebe7bf9 100644
--- a/xen/arch/x86/mpparse.c
+++ b/xen/arch/x86/mpparse.c
@@ -187,7 +187,7 @@ static int MP_processor_info_x(struct mpc_config_processor 
*m,
   " Processor with apicid %i ignored\n", apicid);
return cpu;
}
-   x86_cpu_to_apicid[cpu] = apicid;
+   cpu_physical_id(cpu) = apicid;
cpumask_set_cpu(cpu, _present_map);
}
 
@@ -822,12 +822,12 @@ void mp_unregister_lapic(uint32_t apic_id, uint32_t cpu)
if (!cpu || (apic_id == boot_cpu_physical_apicid))
return;
 
-   if (x86_cpu_to_apicid[cpu] != apic_id)
+   if (cpu_physical_id(cpu) != apic_id)
return;
 
physid_clear(apic_id, phys_cpu_present_map);
 
-   x86_cpu_to_apicid[cpu] = BAD_APICID;
+   cpu_physical_id(cpu) = BAD_APICID;
cpumask_clear_cpu(cpu, _present_map);
 }
 
diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 4b0b297c7e09..39e131cb4f35 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -70,7 +70,7 @@ void __init init_cpu_to_node(void)
 
 for ( i = 0; i < nr_cpu_ids; i++ )
 {
-u32 apicid = x86_cpu_to_apicid[i];
+u32 apicid = cpu_physical_id(i);
 if ( apicid == BAD_APICID )
 continue;
 

[PATCH 02/10] x86/boot: choose AP stack based on APIC ID

[Krystian Hebel]

This is made as first step of making parallel AP bring-up possible. It
should be enough for pre-C code.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/boot/trampoline.S | 20 
 xen/arch/x86/boot/x86_64.S | 28 +++-
 xen/arch/x86/setup.c   |  7 +++
 3 files changed, 54 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/boot/trampoline.S b/xen/arch/x86/boot/trampoline.S
index b8ab0ffdcbb0..ec254125016d 100644
--- a/xen/arch/x86/boot/trampoline.S
+++ b/xen/arch/x86/boot/trampoline.S
@@ -72,6 +72,26 @@ trampoline_protmode_entry:
 mov $X86_CR4_PAE,%ecx
 mov %ecx,%cr4
 
+/*
+ * Get APIC ID while we're in non-paged mode. Start by checking if
+ * x2APIC is enabled.
+ */
+mov $MSR_APIC_BASE, %ecx
+rdmsr
+and $APIC_BASE_EXTD, %eax
+jnz .Lx2apic
+
+/* Not x2APIC, read from MMIO */
+mov 0xfee00020, %esp
+shr $24, %esp
+jmp 1f
+
+.Lx2apic:
+mov $(MSR_X2APIC_FIRST + (0x20 >> 4)), %ecx
+rdmsr
+mov %eax, %esp
+1:
+
 /* Load pagetable base register. */
 mov $sym_offs(idle_pg_table),%eax
 add bootsym_rel(trampoline_xen_phys_start,4,%eax)
diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index 04bb62ae8680..b85b47b5c1a0 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -15,7 +15,33 @@ ENTRY(__high_start)
 mov $XEN_MINIMAL_CR4,%rcx
 mov %rcx,%cr4
 
-mov stack_start(%rip),%rsp
+test%ebx,%ebx
+cmovz   stack_start(%rip), %rsp
+jz  .L_stack_set
+
+/* APs only: get stack base from APIC ID saved in %esp. */
+mov $-1, %rax
+lea x86_cpu_to_apicid(%rip), %rcx
+1:
+add $1, %rax
+cmp $NR_CPUS, %eax
+jb  2f
+hlt
+2:
+cmp %esp, (%rcx, %rax, 4)
+jne 1b
+
+/* %eax is now Xen CPU index. */
+lea stack_base(%rip), %rcx
+mov (%rcx, %rax, 8), %rsp
+
+test%rsp,%rsp
+jnz 1f
+hlt
+1:
+add $(STACK_SIZE - CPUINFO_sizeof), %rsp
+
+.L_stack_set:
 
 /* Reset EFLAGS (subsumes CLI and CLD). */
 pushq   $0
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index a3d3f797bb1e..1285969901e0 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1951,6 +1951,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
  */
 if ( !pv_shim )
 {
+/* Separate loop to make parallel AP bringup possible. */
 for_each_present_cpu ( i )
 {
 /* Set up cpu_to_node[]. */
@@ -1958,6 +1959,12 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 /* Set up node_to_cpumask based on cpu_to_node[]. */
 numa_add_cpu(i);
 
+if ( stack_base[i] == NULL )
+stack_base[i] = cpu_alloc_stack(i);
+}
+
+for_each_present_cpu ( i )
+{
 if ( (park_offline_cpus || num_online_cpus() < max_cpus) &&
  !cpu_online(i) )
 {
-- 
2.41.0

[XEN PATCH 0/9] x86: parallelize AP bring-up during boot

[Krystian Hebel]

Patch series available on this branch:
https://github.com/TrenchBoot/xen/tree/smp_cleanup_upstreaming

This series makes AP bring-up parallel on x86. This reduces number of
IPIs (and more importantly, delays between them) required to start all
logical processors significantly.

In order to make it possible, some state variables that were global
had to be made per-CPU. In most cases, accesses to those variables can
be performed through helper macros, some of them existed before in a
different form.

In addition to work required for parallel initialization, I've fixed
issues in error path around `machine_restart()` that were discovered
during implementation and testing.

CPU hotplug should not be affected, but I had no way of testing it.
During wakeup from S3 APs are started one by one. It should be possible
to enable parallel execution there as well, but I don't have a way of
testing it as of now.

To measure the improvement, I added output lines (identical for before
and after changes so there is no impact of printing over serial) just
above and below `if ( !pv_shim )` block. `console_timestamps=raw` was
used to get as accurate timestamp as possible, and average over 3 boots
was taken into account for each measurement. The final improvement was
calculated as (1 - after/before) * 100%, rounded to 0.01%. These are
the results:

* Dell OptiPlex 9010 with Intel(R) Core(TM) i5-3470 CPU @ 3.20GHz
  (4 cores, 4 threads): 48.83%
* MSI PRO Z790-P with 13th Gen Intel(R) Core(TM) i5-13600K (14 cores,
  20 threads, 6P+8E) `smt=on`: 36.16%
* MSI PRO Z790-P with 13th Gen Intel(R) Core(TM) i5-13600K (14 cores,
  20 threads, 6P+8E) `smt=off`: 0.25% (parking takes a lot of additional
  time)
* HP t630 Thin Client with AMD Embedded G-Series GX-420GI Radeon R7E
  (4 cores, 4 threads): 68.00%

Krystian Hebel (9):
  x86/boot: choose AP stack based on APIC ID
  x86: don't access x86_cpu_to_apicid[] directly, use
cpu_physical_id(cpu)
  x86/smp: drop x86_cpu_to_apicid, use cpu_data[cpu].apicid instead
  x86/smp: move stack_base to cpu_data
  x86/smp: call x2apic_ap_setup() earlier
  x86/shutdown: protect against recurrent machine_restart()
  x86/smp: drop booting_cpu variable
  x86/smp: make cpu_state per-CPU
  x86/smp: start APs in parallel during boot

 xen/arch/x86/acpi/cpu_idle.c  |   4 +-
 xen/arch/x86/acpi/lib.c   |   2 +-
 xen/arch/x86/apic.c   |   2 +-
 xen/arch/x86/boot/trampoline.S|  20 +++
 xen/arch/x86/boot/x86_64.S|  34 -
 xen/arch/x86/cpu/mwait-idle.c |   4 +-
 xen/arch/x86/domain.c |   2 +-
 xen/arch/x86/include/asm/asm_defns.h  |   2 +-
 xen/arch/x86/include/asm/cpufeature.h |   2 +
 xen/arch/x86/include/asm/processor.h  |   2 +
 xen/arch/x86/include/asm/smp.h|   7 +-
 xen/arch/x86/mpparse.c|   6 +-
 xen/arch/x86/numa.c   |  17 +--
 xen/arch/x86/platform_hypercall.c |   2 +-
 xen/arch/x86/setup.c  |  25 +++-
 xen/arch/x86/shutdown.c   |  20 ++-
 xen/arch/x86/smpboot.c| 207 --
 xen/arch/x86/spec_ctrl.c  |   2 +-
 xen/arch/x86/sysctl.c |   2 +-
 xen/arch/x86/traps.c  |   4 +-
 xen/arch/x86/x86_64/asm-offsets.c |   5 +-
 xen/include/xen/smp.h |   2 -
 22 files changed, 248 insertions(+), 125 deletions(-)


base-commit: fb41228ececea948c7953c8c16fe28fd65c6536b
prerequisite-patch-id: 142a87c707411d49e136c3fb76f1b14963ec6dc8
prerequisite-patch-id: f155cb7e2761deec26b76f1b8b587bc56a404c80
-- 
2.41.0

[XEN PATCH 7/9] x86/smp: drop booting_cpu variable

[Krystian Hebel]

CPU id is obtained as a side effect of searching for appropriate
stack for AP. It can be used as a parameter to start_secondary().
Coincidentally this also makes further work on making AP bring-up
code parallel easier.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/boot/x86_64.S | 13 +
 xen/arch/x86/smpboot.c | 15 +--
 2 files changed, 14 insertions(+), 14 deletions(-)

diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index 8d61f270761f..ad01f20d548d 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -20,20 +20,24 @@ ENTRY(__high_start)
 jz  .L_stack_set
 
 /* APs only: get stack base from APIC ID saved in %esp. */
-mov $0, %rax
+mov $0, %rbx
 lea cpu_data(%rip), %rcx
 /* cpu_data[0] is BSP, skip it. */
 1:
-add $1, %rax
+add $1, %rbx
 add $CPUINFO_X86_sizeof, %rcx
-cmp $NR_CPUS, %eax
+cmp $NR_CPUS, %rbx
 jb  2f
 hlt
 2:
 cmp %esp, CPUINFO_X86_apicid(%rcx)
 jne 1b
 
-/* %rcx is now cpu_data[cpu], read stack base from it. */
+/*
+ * At this point:
+ * - %rcx is cpu_data[cpu], read stack base from it,
+ * - %rbx (callee-save) is Xen cpu number, pass it to 
start_secondary().
+ */
 mov CPUINFO_X86_stack_base(%rcx), %rsp
 
 test%rsp,%rsp
@@ -101,6 +105,7 @@ ENTRY(__high_start)
 .L_ap_cet_done:
 #endif /* CONFIG_XEN_SHSTK || CONFIG_XEN_IBT */
 
+mov %rbx, %rdi
 tailcall start_secondary
 
 .L_bsp:
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index a3895dafa267..39ffd356dbbc 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -222,8 +222,6 @@ static void smp_callin(void)
 cpu_relax();
 }
 
-static int booting_cpu;
-
 /* CPUs for which sibling maps can be computed. */
 static cpumask_t cpu_sibling_setup_map;
 
@@ -311,15 +309,14 @@ static void set_cpu_sibling_map(unsigned int cpu)
 }
 }
 
-void start_secondary(void *unused)
+void start_secondary(unsigned int cpu)
 {
 struct cpu_info *info = get_cpu_info();
 
 /*
- * Dont put anything before smp_callin(), SMP booting is so fragile that we
+ * Don't put anything before smp_callin(), SMP booting is so fragile that 
we
  * want to limit the things done here to the most necessary things.
  */
-unsigned int cpu = booting_cpu;
 
 /* Critical region without IDT or TSS.  Any fault is deadly! */
 
@@ -346,9 +343,9 @@ void start_secondary(void *unused)
  */
 spin_debug_disable();
 
-get_cpu_info()->use_pv_cr3 = false;
-get_cpu_info()->xen_cr3 = 0;
-get_cpu_info()->pv_cr3 = 0;
+info->use_pv_cr3 = false;
+info->xen_cr3 = 0;
+info->pv_cr3 = 0;
 
 /*
  * BUG_ON() used in load_system_tables() and later code may end up calling
@@ -575,8 +572,6 @@ static int do_boot_cpu(int apicid, int cpu)
  */
 mtrr_save_state();
 
-booting_cpu = cpu;
-
 start_eip = bootsym_phys(trampoline_realmode_entry);
 
 /* start_eip needs be page aligned, and below the 1M boundary. */
-- 
2.41.0

[PATCH 05/10] x86/smp: move stack_base to cpu_data

[Krystian Hebel]

This location is easier to access from assembly. Having it close to
other data required during initialization has also positive (although
rather small) impact on prefetching data from RAM.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/boot/x86_64.S|  5 ++---
 xen/arch/x86/include/asm/cpufeature.h |  1 +
 xen/arch/x86/include/asm/smp.h|  2 +-
 xen/arch/x86/setup.c  |  6 +++---
 xen/arch/x86/smpboot.c| 25 +
 xen/arch/x86/traps.c  |  4 ++--
 xen/arch/x86/x86_64/asm-offsets.c |  1 +
 xen/include/xen/smp.h |  2 --
 8 files changed, 23 insertions(+), 23 deletions(-)

diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index 195550b5c0ea..8d61f270761f 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -33,9 +33,8 @@ ENTRY(__high_start)
 cmp %esp, CPUINFO_X86_apicid(%rcx)
 jne 1b
 
-/* %eax is now Xen CPU index. */
-lea stack_base(%rip), %rcx
-mov (%rcx, %rax, 8), %rsp
+/* %rcx is now cpu_data[cpu], read stack base from it. */
+mov CPUINFO_X86_stack_base(%rcx), %rsp
 
 test%rsp,%rsp
 jnz 1f
diff --git a/xen/arch/x86/include/asm/cpufeature.h 
b/xen/arch/x86/include/asm/cpufeature.h
index 06e1dd7f3332..ff0e18864cc7 100644
--- a/xen/arch/x86/include/asm/cpufeature.h
+++ b/xen/arch/x86/include/asm/cpufeature.h
@@ -37,6 +37,7 @@ struct cpuinfo_x86 {
 unsigned int phys_proc_id; /* package ID of each logical CPU */
 unsigned int cpu_core_id;  /* core ID of each logical CPU */
 unsigned int compute_unit_id;  /* AMD compute unit ID of each logical 
CPU */
+void *stack_base;
 unsigned short x86_clflush_size;
 } __cacheline_aligned;
 
diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
index 94c557491860..98739028a6ed 100644
--- a/xen/arch/x86/include/asm/smp.h
+++ b/xen/arch/x86/include/asm/smp.h
@@ -69,7 +69,7 @@ extern cpumask_t **socket_cpumask;
  * by certain scheduling code only.
  */
 #define get_cpu_current(cpu) \
-(get_cpu_info_from_stack((unsigned long)stack_base[cpu])->current_vcpu)
+(get_cpu_info_from_stack((unsigned 
long)cpu_data[cpu].stack_base)->current_vcpu)
 
 extern unsigned int disabled_cpus;
 extern bool unaccounted_cpus;
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index a19fe219bbf6..b2c0679725ea 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -798,7 +798,7 @@ static void __init noreturn reinit_bsp_stack(void)
 /* Update SYSCALL trampolines */
 percpu_traps_init();
 
-stack_base[0] = stack;
+cpu_data[0].stack_base = stack;
 
 rc = setup_cpu_root_pgt(0);
 if ( rc )
@@ -1959,8 +1959,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 /* Set up node_to_cpumask based on cpu_to_node[]. */
 numa_add_cpu(i);
 
-if ( stack_base[i] == NULL )
-stack_base[i] = cpu_alloc_stack(i);
+if ( cpu_data[i].stack_base == NULL )
+cpu_data[i].stack_base = cpu_alloc_stack(i);
 }
 
 for_each_present_cpu ( i )
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index f061486e56eb..8ae65ab1769f 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -75,13 +75,15 @@ static enum cpu_state {
 } cpu_state;
 #define set_cpu_state(state) do { smp_mb(); cpu_state = (state); } while (0)
 
-void *stack_base[NR_CPUS];
-
 void initialize_cpu_data(unsigned int cpu)
 {
 uint32_t apicid = cpu_physical_id(cpu);
+void *stack = cpu_data[cpu].stack_base;
+
 cpu_data[cpu] = boot_cpu_data;
+
 cpu_physical_id(cpu) = apicid;
+cpu_data[cpu].stack_base = stack;
 }
 
 static bool smp_store_cpu_info(unsigned int id)
@@ -579,8 +581,6 @@ static int do_boot_cpu(int apicid, int cpu)
 printk("Booting processor %d/%d eip %lx\n",
cpu, apicid, start_eip);
 
-stack_start = stack_base[cpu] + STACK_SIZE - sizeof(struct cpu_info);
-
 /* This grunge runs the startup process for the targeted processor. */
 
 set_cpu_state(CPU_STATE_INIT);
@@ -856,7 +856,7 @@ int setup_cpu_root_pgt(unsigned int cpu)
 
 /* Install direct map page table entries for stack, IDT, and TSS. */
 for ( off = rc = 0; !rc && off < STACK_SIZE; off += PAGE_SIZE )
-rc = clone_mapping(__va(__pa(stack_base[cpu])) + off, rpt);
+rc = clone_mapping(__va(__pa(cpu_data[cpu].stack_base)) + off, rpt);
 
 if ( !rc )
 rc = clone_mapping(idt_tables[cpu], rpt);
@@ -1007,10 +1007,10 @@ static void cpu_smpboot_free(unsigned int cpu, bool 
remove)
 FREE_XENHEAP_PAGE(per_cpu(gdt, cpu));
 FREE_XENHEAP_PAGE(idt_tables[cpu]);
 
-if ( stack_base[cpu] )
+if ( cpu_data[cpu].stack_base )
 {
-memguard_unguard_stack(stack_base[cpu]);
-FREE_XENHEAP_PAGES(stack_base[cpu], 

[XEN PATCH 6/9] x86/shutdown: protect against recurrent machine_restart()

[Krystian Hebel]

If multiple CPUs called machine_restart() before actual restart took
place, but after boot CPU declared itself not online, ASSERT in
on_selected_cpus() will fail. Few calls later execution would end up
in machine_restart() again, with another frame on call stack for new
exception.

To protect against running out of stack, code checks if boot CPU is
still online before calling on_selected_cpus().

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/shutdown.c | 20 +---
 1 file changed, 17 insertions(+), 3 deletions(-)

diff --git a/xen/arch/x86/shutdown.c b/xen/arch/x86/shutdown.c
index 7619544d14da..32c70505ed77 100644
--- a/xen/arch/x86/shutdown.c
+++ b/xen/arch/x86/shutdown.c
@@ -577,9 +577,23 @@ void machine_restart(unsigned int delay_millisecs)
 /* Ensure we are the boot CPU. */
 if ( get_apic_id() != boot_cpu_physical_apicid )
 {
-/* Send IPI to the boot CPU (logical cpu 0). */
-on_selected_cpus(cpumask_of(0), __machine_restart,
- _millisecs, 0);
+/*
+ * Send IPI to the boot CPU (logical cpu 0).
+ *
+ * If multiple CPUs called machine_restart() before actual restart
+ * took place, but after boot CPU declared itself not online, 
ASSERT
+ * in on_selected_cpus() will fail. Few calls later we would end up
+ * here again, with another frame on call stack for new exception.
+ * To protect against running out of stack, check if boot CPU is
+ * online.
+ *
+ * Note this is not an atomic operation, so it is possible for
+ * on_selected_cpus() to be called once after boot CPU is offline
+ * before we hit halt() below.
+ */
+if ( cpu_online(0) )
+on_selected_cpus(cpumask_of(0), __machine_restart,
+ _millisecs, 0);
 for ( ; ; )
 halt();
 }
-- 
2.41.0

[PATCH 03/10] x86: don't access x86_cpu_to_apicid[] directly, use cpu_physical_id(cpu)

[Krystian Hebel]

This is done in preparation to move data from x86_cpu_to_apicid[]
elsewhere.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/acpi/cpu_idle.c  |  4 ++--
 xen/arch/x86/acpi/lib.c   |  2 +-
 xen/arch/x86/apic.c   |  2 +-
 xen/arch/x86/cpu/mwait-idle.c |  4 ++--
 xen/arch/x86/domain.c |  2 +-
 xen/arch/x86/mpparse.c|  6 +++---
 xen/arch/x86/numa.c   |  2 +-
 xen/arch/x86/platform_hypercall.c |  2 +-
 xen/arch/x86/setup.c  | 14 +++---
 xen/arch/x86/smpboot.c|  4 ++--
 xen/arch/x86/spec_ctrl.c  |  2 +-
 xen/arch/x86/sysctl.c |  2 +-
 12 files changed, 23 insertions(+), 23 deletions(-)

diff --git a/xen/arch/x86/acpi/cpu_idle.c b/xen/arch/x86/acpi/cpu_idle.c
index cfce4cc0408f..d03e54bcef38 100644
--- a/xen/arch/x86/acpi/cpu_idle.c
+++ b/xen/arch/x86/acpi/cpu_idle.c
@@ -1256,7 +1256,7 @@ int get_cpu_id(u32 acpi_id)
 
 for ( i = 0; i < nr_cpu_ids; i++ )
 {
-if ( apic_id == x86_cpu_to_apicid[i] )
+if ( apic_id == cpu_physical_id(i) )
 return i;
 }
 
@@ -1362,7 +1362,7 @@ long set_cx_pminfo(uint32_t acpi_id, struct 
xen_processor_power *power)
 
 if ( !cpu_online(cpu_id) )
 {
-uint32_t apic_id = x86_cpu_to_apicid[cpu_id];
+uint32_t apic_id = cpu_physical_id(cpu_id);
 
 /*
  * If we've just learned of more available C states, wake the CPU if
diff --git a/xen/arch/x86/acpi/lib.c b/xen/arch/x86/acpi/lib.c
index 51cb082ca02a..87a1f38f3f5a 100644
--- a/xen/arch/x86/acpi/lib.c
+++ b/xen/arch/x86/acpi/lib.c
@@ -91,7 +91,7 @@ unsigned int acpi_get_processor_id(unsigned int cpu)
 {
unsigned int acpiid, apicid;
 
-   if ((apicid = x86_cpu_to_apicid[cpu]) == BAD_APICID)
+   if ((apicid = cpu_physical_id(cpu)) == BAD_APICID)
return INVALID_ACPIID;
 
for (acpiid = 0; acpiid < ARRAY_SIZE(x86_acpiid_to_apicid); acpiid++)
diff --git a/xen/arch/x86/apic.c b/xen/arch/x86/apic.c
index 6acdd0ec1468..63e18968e54c 100644
--- a/xen/arch/x86/apic.c
+++ b/xen/arch/x86/apic.c
@@ -950,7 +950,7 @@ __next:
  */
 if (boot_cpu_physical_apicid == -1U)
 boot_cpu_physical_apicid = get_apic_id();
-x86_cpu_to_apicid[0] = get_apic_id();
+cpu_physical_id(0) = get_apic_id();
 
 ioapic_init();
 }
diff --git a/xen/arch/x86/cpu/mwait-idle.c b/xen/arch/x86/cpu/mwait-idle.c
index ff5c808bc952..88168da8a0ca 100644
--- a/xen/arch/x86/cpu/mwait-idle.c
+++ b/xen/arch/x86/cpu/mwait-idle.c
@@ -1202,8 +1202,8 @@ static void __init ivt_idle_state_table_update(void)
unsigned int cpu, max_apicid = boot_cpu_physical_apicid;
 
for_each_present_cpu(cpu)
-   if (max_apicid < x86_cpu_to_apicid[cpu])
-   max_apicid = x86_cpu_to_apicid[cpu];
+   if (max_apicid < cpu_physical_id(cpu))
+   max_apicid = cpu_physical_id(cpu);
switch (apicid_to_socket(max_apicid)) {
case 0: case 1:
/* 1 and 2 socket systems use default ivt_cstates */
diff --git a/xen/arch/x86/domain.c b/xen/arch/x86/domain.c
index 3712e36df930..f45437511a46 100644
--- a/xen/arch/x86/domain.c
+++ b/xen/arch/x86/domain.c
@@ -1615,7 +1615,7 @@ long do_vcpu_op(int cmd, unsigned int vcpuid, 
XEN_GUEST_HANDLE_PARAM(void) arg)
 break;
 
 cpu_id.phys_id =
-(uint64_t)x86_cpu_to_apicid[v->vcpu_id] |
+(uint64_t)cpu_physical_id(v->vcpu_id) |
 ((uint64_t)acpi_get_processor_id(v->vcpu_id) << 32);
 
 rc = -EFAULT;
diff --git a/xen/arch/x86/mpparse.c b/xen/arch/x86/mpparse.c
index d8ccab2449c6..b8cabebe7bf9 100644
--- a/xen/arch/x86/mpparse.c
+++ b/xen/arch/x86/mpparse.c
@@ -187,7 +187,7 @@ static int MP_processor_info_x(struct mpc_config_processor 
*m,
   " Processor with apicid %i ignored\n", apicid);
return cpu;
}
-   x86_cpu_to_apicid[cpu] = apicid;
+   cpu_physical_id(cpu) = apicid;
cpumask_set_cpu(cpu, _present_map);
}
 
@@ -822,12 +822,12 @@ void mp_unregister_lapic(uint32_t apic_id, uint32_t cpu)
if (!cpu || (apic_id == boot_cpu_physical_apicid))
return;
 
-   if (x86_cpu_to_apicid[cpu] != apic_id)
+   if (cpu_physical_id(cpu) != apic_id)
return;
 
physid_clear(apic_id, phys_cpu_present_map);
 
-   x86_cpu_to_apicid[cpu] = BAD_APICID;
+   cpu_physical_id(cpu) = BAD_APICID;
cpumask_clear_cpu(cpu, _present_map);
 }
 
diff --git a/xen/arch/x86/numa.c b/xen/arch/x86/numa.c
index 4b0b297c7e09..39e131cb4f35 100644
--- a/xen/arch/x86/numa.c
+++ b/xen/arch/x86/numa.c
@@ -70,7 +70,7 @@ void __init init_cpu_to_node(void)
 
 for ( i = 0; i < nr_cpu_ids; i++ )
 {
-u32 apicid = x86_cpu_to_apicid[i];
+u32 apicid = cpu_physical_id(i);
 if ( apicid == BAD_APICID )
 continue;
 

[PATCH 1/2] x86/smp: make cpu_state per-CPU

[Krystian Hebel]

This will be used for parallel AP bring-up.

CPU_STATE_INIT changed direction. It was previously set by BSP and never
consumed by AP. Now it signals that AP got through assembly part of
initialization and waits for BSP to call notifiers that set up data
structures required for further initialization.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/include/asm/cpufeature.h |  1 +
 xen/arch/x86/smpboot.c| 80 ---
 2 files changed, 49 insertions(+), 32 deletions(-)

diff --git a/xen/arch/x86/include/asm/cpufeature.h 
b/xen/arch/x86/include/asm/cpufeature.h
index ff0e18864cc7..1831b5fb368f 100644
--- a/xen/arch/x86/include/asm/cpufeature.h
+++ b/xen/arch/x86/include/asm/cpufeature.h
@@ -38,6 +38,7 @@ struct cpuinfo_x86 {
 unsigned int cpu_core_id;  /* core ID of each logical CPU */
 unsigned int compute_unit_id;  /* AMD compute unit ID of each logical 
CPU */
 void *stack_base;
+unsigned int cpu_state;
 unsigned short x86_clflush_size;
 } __cacheline_aligned;
 
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 39ffd356dbbc..cbea2d45f70d 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -65,15 +65,18 @@ struct cpuinfo_x86 cpu_data[NR_CPUS] =
 { [0 ... NR_CPUS-1] .apicid = BAD_APICID };
 
 static int cpu_error;
-static enum cpu_state {
+enum cpu_state {
 CPU_STATE_DYING,/* slave -> master: I am dying */
 CPU_STATE_DEAD, /* slave -> master: I am completely dead */
-CPU_STATE_INIT, /* master -> slave: Early bringup phase 1 */
-CPU_STATE_CALLOUT,  /* master -> slave: Early bringup phase 2 */
+CPU_STATE_INIT, /* slave -> master: Early bringup phase 1 completed */
+CPU_STATE_CALLOUT,  /* master -> slave: Start early bringup phase 2 */
 CPU_STATE_CALLIN,   /* slave -> master: Completed phase 2 */
 CPU_STATE_ONLINE/* master -> slave: Go fully online now. */
-} cpu_state;
-#define set_cpu_state(state) do { smp_mb(); cpu_state = (state); } while (0)
+};
+#define set_cpu_state(cpu, state) do { \
+smp_mb(); \
+cpu_data[cpu].cpu_state = (state); \
+} while (0)
 
 void initialize_cpu_data(unsigned int cpu)
 {
@@ -168,16 +171,7 @@ static void synchronize_tsc_slave(unsigned int slave)
 static void smp_callin(void)
 {
 unsigned int cpu = smp_processor_id();
-int i, rc;
-
-/* Wait 2s total for startup. */
-Dprintk("Waiting for CALLOUT.\n");
-for ( i = 0; cpu_state != CPU_STATE_CALLOUT; i++ )
-{
-BUG_ON(i >= 200);
-cpu_relax();
-mdelay(10);
-}
+int rc;
 
 /*
  * The boot CPU has finished the init stage and is spinning on cpu_state
@@ -213,12 +207,12 @@ static void smp_callin(void)
 }
 
 /* Allow the master to continue. */
-set_cpu_state(CPU_STATE_CALLIN);
+set_cpu_state(cpu, CPU_STATE_CALLIN);
 
 synchronize_tsc_slave(cpu);
 
 /* And wait for our final Ack. */
-while ( cpu_state != CPU_STATE_ONLINE )
+while ( cpu_data[cpu].cpu_state != CPU_STATE_ONLINE )
 cpu_relax();
 }
 
@@ -313,6 +307,9 @@ void start_secondary(unsigned int cpu)
 {
 struct cpu_info *info = get_cpu_info();
 
+/* Tell BSP that we are awake. */
+set_cpu_state(cpu, CPU_STATE_INIT);
+
 /*
  * Don't put anything before smp_callin(), SMP booting is so fragile that 
we
  * want to limit the things done here to the most necessary things.
@@ -320,6 +317,10 @@ void start_secondary(unsigned int cpu)
 
 /* Critical region without IDT or TSS.  Any fault is deadly! */
 
+/* Wait until data set up by CPU_UP_PREPARE notifiers is ready. */
+while ( cpu_data[cpu].cpu_state != CPU_STATE_CALLOUT )
+cpu_relax();
+
 set_current(idle_vcpu[cpu]);
 this_cpu(curr_vcpu) = idle_vcpu[cpu];
 rdmsrl(MSR_EFER, this_cpu(efer));
@@ -585,26 +586,35 @@ static int do_boot_cpu(int apicid, int cpu)
 
 /* This grunge runs the startup process for the targeted processor. */
 
-set_cpu_state(CPU_STATE_INIT);
-
 /* Starting actual IPI sequence... */
 boot_error = wakeup_secondary_cpu(apicid, start_eip);
 
 if ( !boot_error )
 {
-/* Allow AP to start initializing. */
-set_cpu_state(CPU_STATE_CALLOUT);
-Dprintk("After Callout %d.\n", cpu);
-
-/* Wait 5s total for a response. */
-for ( timeout = 0; timeout < 5; timeout++ )
+/* Wait 2s total for a response. */
+for ( timeout = 0; timeout < 2; timeout++ )
 {
-if ( cpu_state != CPU_STATE_CALLOUT )
+if ( cpu_data[cpu].cpu_state == CPU_STATE_INIT )
 break;
 udelay(100);
 }
 
-if ( cpu_state == CPU_STATE_CALLIN )
+if ( cpu_data[cpu].cpu_state == CPU_STATE_INIT )
+{
+/* Allow AP to start initializing. */
+set_cpu_state(cpu, CPU_STATE_CALLOUT);
+Dprintk("After Callout %d.\n", cpu);
+
+/* Wait 5s total for a 

[XEN PATCH 1/9] x86/boot: choose AP stack based on APIC ID

[Krystian Hebel]

This is made as first step of making parallel AP bring-up possible. It
should be enough for pre-C code.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/boot/trampoline.S | 20 
 xen/arch/x86/boot/x86_64.S | 28 +++-
 xen/arch/x86/setup.c   |  7 +++
 3 files changed, 54 insertions(+), 1 deletion(-)

diff --git a/xen/arch/x86/boot/trampoline.S b/xen/arch/x86/boot/trampoline.S
index b8ab0ffdcbb0..ec254125016d 100644
--- a/xen/arch/x86/boot/trampoline.S
+++ b/xen/arch/x86/boot/trampoline.S
@@ -72,6 +72,26 @@ trampoline_protmode_entry:
 mov $X86_CR4_PAE,%ecx
 mov %ecx,%cr4
 
+/*
+ * Get APIC ID while we're in non-paged mode. Start by checking if
+ * x2APIC is enabled.
+ */
+mov $MSR_APIC_BASE, %ecx
+rdmsr
+and $APIC_BASE_EXTD, %eax
+jnz .Lx2apic
+
+/* Not x2APIC, read from MMIO */
+mov 0xfee00020, %esp
+shr $24, %esp
+jmp 1f
+
+.Lx2apic:
+mov $(MSR_X2APIC_FIRST + (0x20 >> 4)), %ecx
+rdmsr
+mov %eax, %esp
+1:
+
 /* Load pagetable base register. */
 mov $sym_offs(idle_pg_table),%eax
 add bootsym_rel(trampoline_xen_phys_start,4,%eax)
diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index 04bb62ae8680..b85b47b5c1a0 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -15,7 +15,33 @@ ENTRY(__high_start)
 mov $XEN_MINIMAL_CR4,%rcx
 mov %rcx,%cr4
 
-mov stack_start(%rip),%rsp
+test%ebx,%ebx
+cmovz   stack_start(%rip), %rsp
+jz  .L_stack_set
+
+/* APs only: get stack base from APIC ID saved in %esp. */
+mov $-1, %rax
+lea x86_cpu_to_apicid(%rip), %rcx
+1:
+add $1, %rax
+cmp $NR_CPUS, %eax
+jb  2f
+hlt
+2:
+cmp %esp, (%rcx, %rax, 4)
+jne 1b
+
+/* %eax is now Xen CPU index. */
+lea stack_base(%rip), %rcx
+mov (%rcx, %rax, 8), %rsp
+
+test%rsp,%rsp
+jnz 1f
+hlt
+1:
+add $(STACK_SIZE - CPUINFO_sizeof), %rsp
+
+.L_stack_set:
 
 /* Reset EFLAGS (subsumes CLI and CLD). */
 pushq   $0
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index a3d3f797bb1e..1285969901e0 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -1951,6 +1951,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
  */
 if ( !pv_shim )
 {
+/* Separate loop to make parallel AP bringup possible. */
 for_each_present_cpu ( i )
 {
 /* Set up cpu_to_node[]. */
@@ -1958,6 +1959,12 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 /* Set up node_to_cpumask based on cpu_to_node[]. */
 numa_add_cpu(i);
 
+if ( stack_base[i] == NULL )
+stack_base[i] = cpu_alloc_stack(i);
+}
+
+for_each_present_cpu ( i )
+{
 if ( (park_offline_cpus || num_online_cpus() < max_cpus) &&
  !cpu_online(i) )
 {
-- 
2.41.0

[PATCH 01/10] x86/spec-ctrl: Remove conditional IRQs-on-ness for INT $0x80/0x82 paths

[Krystian Hebel]

From: Andrew Cooper 

Before speculation defences, some paths in Xen could genuinely get away with
being IRQs-on at entry.  But XPTI invalidated this property on most paths, and
attempting to maintain it on the remaining paths was a mistake.

Fast forward, and DO_SPEC_CTRL_COND_IBPB (protection for AMD BTC/SRSO) is not
IRQ-safe, running with IRQs enabled in some cases.  The other actions taken on
these paths happen to be IRQ-safe.

Make entry_int82() and int80_direct_trap() unconditionally Interrupt Gates
rather than Trap Gates.  Remove the conditional re-adjustment of
int80_direct_trap() in smp_prepare_cpus(), and have entry_int82() explicitly
enable interrupts when safe to do so.

In smp_prepare_cpus(), with the conditional re-adjustment removed, the
clearing of pv_cr3 is the only remaining action gated on XPTI, and it is out
of place anyway, repeating work already done by smp_prepare_boot_cpu().  Drop
the entire if() condition to avoid leaving an incorrect vestigial remnant.

Also drop comments which make incorrect statements about when its safe to
enable interrupts.

This is XSA-446 / CVE-2023-46836

Signed-off-by: Andrew Cooper 
Reviewed-by: Roger Pau Monné 
---
 xen/arch/x86/pv/traps.c|  4 ++--
 xen/arch/x86/smpboot.c | 14 --
 xen/arch/x86/x86_64/compat/entry.S |  2 ++
 xen/arch/x86/x86_64/entry.S|  1 -
 4 files changed, 4 insertions(+), 17 deletions(-)

diff --git a/xen/arch/x86/pv/traps.c b/xen/arch/x86/pv/traps.c
index 74f333da7e1c..240d1a2db7a3 100644
--- a/xen/arch/x86/pv/traps.c
+++ b/xen/arch/x86/pv/traps.c
@@ -139,11 +139,11 @@ void __init pv_trap_init(void)
 #ifdef CONFIG_PV32
 /* The 32-on-64 hypercall vector is only accessible from ring 1. */
 _set_gate(idt_table + HYPERCALL_VECTOR,
-  SYS_DESC_trap_gate, 1, entry_int82);
+  SYS_DESC_irq_gate, 1, entry_int82);
 #endif
 
 /* Fast trap for int80 (faster than taking the #GP-fixup path). */
-_set_gate(idt_table + LEGACY_SYSCALL_VECTOR, SYS_DESC_trap_gate, 3,
+_set_gate(idt_table + LEGACY_SYSCALL_VECTOR, SYS_DESC_irq_gate, 3,
   _direct_trap);
 
 open_softirq(NMI_SOFTIRQ, nmi_softirq);
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index 3a1a659082c6..4c54ecbc91d7 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -1158,20 +1158,6 @@ void __init smp_prepare_cpus(void)
 
 stack_base[0] = (void *)((unsigned long)stack_start & ~(STACK_SIZE - 1));
 
-if ( opt_xpti_hwdom || opt_xpti_domu )
-{
-get_cpu_info()->pv_cr3 = 0;
-
-#ifdef CONFIG_PV
-/*
- * All entry points which may need to switch page tables have to start
- * with interrupts off. Re-write what pv_trap_init() has put there.
- */
-_set_gate(idt_table + LEGACY_SYSCALL_VECTOR, SYS_DESC_irq_gate, 3,
-  _direct_trap);
-#endif
-}
-
 set_nr_sockets();
 
 socket_cpumask = xzalloc_array(cpumask_t *, nr_sockets);
diff --git a/xen/arch/x86/x86_64/compat/entry.S 
b/xen/arch/x86/x86_64/compat/entry.S
index bd5abd8040bd..fcc3a721f147 100644
--- a/xen/arch/x86/x86_64/compat/entry.S
+++ b/xen/arch/x86/x86_64/compat/entry.S
@@ -21,6 +21,8 @@ ENTRY(entry_int82)
 SPEC_CTRL_ENTRY_FROM_PV /* Req: %rsp=regs/cpuinfo, %rdx=0, Clob: acd */
 /* WARNING! `ret`, `call *`, `jmp *` not safe before this point. */
 
+sti
+
 CR4_PV32_RESTORE
 
 GET_CURRENT(bx)
diff --git a/xen/arch/x86/x86_64/entry.S b/xen/arch/x86/x86_64/entry.S
index 5ca74f5f62b2..9a7b129aa7e4 100644
--- a/xen/arch/x86/x86_64/entry.S
+++ b/xen/arch/x86/x86_64/entry.S
@@ -327,7 +327,6 @@ ENTRY(sysenter_entry)
 #ifdef CONFIG_XEN_SHSTK
 ALTERNATIVE "", "setssbsy", X86_FEATURE_XEN_SHSTK
 #endif
-/* sti could live here when we don't switch page tables below. */
 pushq $FLAT_USER_SS
 pushq $0
 pushfq
-- 
2.41.0

[XEN PATCH 4/9] x86/smp: move stack_base to cpu_data

[Krystian Hebel]

This location is easier to access from assembly. Having it close to
other data required during initialization has also positive (although
rather small) impact on prefetching data from RAM.

Signed-off-by: Krystian Hebel 
---
 xen/arch/x86/boot/x86_64.S|  5 ++---
 xen/arch/x86/include/asm/cpufeature.h |  1 +
 xen/arch/x86/include/asm/smp.h|  2 +-
 xen/arch/x86/setup.c  |  6 +++---
 xen/arch/x86/smpboot.c| 25 +
 xen/arch/x86/traps.c  |  4 ++--
 xen/arch/x86/x86_64/asm-offsets.c |  1 +
 xen/include/xen/smp.h |  2 --
 8 files changed, 23 insertions(+), 23 deletions(-)

diff --git a/xen/arch/x86/boot/x86_64.S b/xen/arch/x86/boot/x86_64.S
index 195550b5c0ea..8d61f270761f 100644
--- a/xen/arch/x86/boot/x86_64.S
+++ b/xen/arch/x86/boot/x86_64.S
@@ -33,9 +33,8 @@ ENTRY(__high_start)
 cmp %esp, CPUINFO_X86_apicid(%rcx)
 jne 1b
 
-/* %eax is now Xen CPU index. */
-lea stack_base(%rip), %rcx
-mov (%rcx, %rax, 8), %rsp
+/* %rcx is now cpu_data[cpu], read stack base from it. */
+mov CPUINFO_X86_stack_base(%rcx), %rsp
 
 test%rsp,%rsp
 jnz 1f
diff --git a/xen/arch/x86/include/asm/cpufeature.h 
b/xen/arch/x86/include/asm/cpufeature.h
index 06e1dd7f3332..ff0e18864cc7 100644
--- a/xen/arch/x86/include/asm/cpufeature.h
+++ b/xen/arch/x86/include/asm/cpufeature.h
@@ -37,6 +37,7 @@ struct cpuinfo_x86 {
 unsigned int phys_proc_id; /* package ID of each logical CPU */
 unsigned int cpu_core_id;  /* core ID of each logical CPU */
 unsigned int compute_unit_id;  /* AMD compute unit ID of each logical 
CPU */
+void *stack_base;
 unsigned short x86_clflush_size;
 } __cacheline_aligned;
 
diff --git a/xen/arch/x86/include/asm/smp.h b/xen/arch/x86/include/asm/smp.h
index 94c557491860..98739028a6ed 100644
--- a/xen/arch/x86/include/asm/smp.h
+++ b/xen/arch/x86/include/asm/smp.h
@@ -69,7 +69,7 @@ extern cpumask_t **socket_cpumask;
  * by certain scheduling code only.
  */
 #define get_cpu_current(cpu) \
-(get_cpu_info_from_stack((unsigned long)stack_base[cpu])->current_vcpu)
+(get_cpu_info_from_stack((unsigned 
long)cpu_data[cpu].stack_base)->current_vcpu)
 
 extern unsigned int disabled_cpus;
 extern bool unaccounted_cpus;
diff --git a/xen/arch/x86/setup.c b/xen/arch/x86/setup.c
index a19fe219bbf6..b2c0679725ea 100644
--- a/xen/arch/x86/setup.c
+++ b/xen/arch/x86/setup.c
@@ -798,7 +798,7 @@ static void __init noreturn reinit_bsp_stack(void)
 /* Update SYSCALL trampolines */
 percpu_traps_init();
 
-stack_base[0] = stack;
+cpu_data[0].stack_base = stack;
 
 rc = setup_cpu_root_pgt(0);
 if ( rc )
@@ -1959,8 +1959,8 @@ void __init noreturn __start_xen(unsigned long mbi_p)
 /* Set up node_to_cpumask based on cpu_to_node[]. */
 numa_add_cpu(i);
 
-if ( stack_base[i] == NULL )
-stack_base[i] = cpu_alloc_stack(i);
+if ( cpu_data[i].stack_base == NULL )
+cpu_data[i].stack_base = cpu_alloc_stack(i);
 }
 
 for_each_present_cpu ( i )
diff --git a/xen/arch/x86/smpboot.c b/xen/arch/x86/smpboot.c
index f061486e56eb..8ae65ab1769f 100644
--- a/xen/arch/x86/smpboot.c
+++ b/xen/arch/x86/smpboot.c
@@ -75,13 +75,15 @@ static enum cpu_state {
 } cpu_state;
 #define set_cpu_state(state) do { smp_mb(); cpu_state = (state); } while (0)
 
-void *stack_base[NR_CPUS];
-
 void initialize_cpu_data(unsigned int cpu)
 {
 uint32_t apicid = cpu_physical_id(cpu);
+void *stack = cpu_data[cpu].stack_base;
+
 cpu_data[cpu] = boot_cpu_data;
+
 cpu_physical_id(cpu) = apicid;
+cpu_data[cpu].stack_base = stack;
 }
 
 static bool smp_store_cpu_info(unsigned int id)
@@ -579,8 +581,6 @@ static int do_boot_cpu(int apicid, int cpu)
 printk("Booting processor %d/%d eip %lx\n",
cpu, apicid, start_eip);
 
-stack_start = stack_base[cpu] + STACK_SIZE - sizeof(struct cpu_info);
-
 /* This grunge runs the startup process for the targeted processor. */
 
 set_cpu_state(CPU_STATE_INIT);
@@ -856,7 +856,7 @@ int setup_cpu_root_pgt(unsigned int cpu)
 
 /* Install direct map page table entries for stack, IDT, and TSS. */
 for ( off = rc = 0; !rc && off < STACK_SIZE; off += PAGE_SIZE )
-rc = clone_mapping(__va(__pa(stack_base[cpu])) + off, rpt);
+rc = clone_mapping(__va(__pa(cpu_data[cpu].stack_base)) + off, rpt);
 
 if ( !rc )
 rc = clone_mapping(idt_tables[cpu], rpt);
@@ -1007,10 +1007,10 @@ static void cpu_smpboot_free(unsigned int cpu, bool 
remove)
 FREE_XENHEAP_PAGE(per_cpu(gdt, cpu));
 FREE_XENHEAP_PAGE(idt_tables[cpu]);
 
-if ( stack_base[cpu] )
+if ( cpu_data[cpu].stack_base )
 {
-memguard_unguard_stack(stack_base[cpu]);
-FREE_XENHEAP_PAGES(stack_base[cpu], 

Re: [PATCH v3] xen: ifdef inclusion of in

[Jan Beulich]

On 14.11.2023 16:16, Oleksii Kurochko wrote:
> Ifdef-ing inclusion of  allows to avoid
> generation of empty  for cases when
> CONFIG_GRANT_TABLE is not enabled.
> 
> The following changes were done for Arm:
>  should be included directly because it contains
> gnttab_dom0_frames() macros which is unique for Arm and is used in
> arch/arm/domain_build.c.
>  is #ifdef-ed with CONFIG_GRANT_TABLE in
>  so in case of !CONFIG_GRANT_TABLE gnttab_dom0_frames
> won't be available for use in arch/arm/domain_build.c.
> 
> Suggested-by: Jan Beulich 
> Signed-off-by: Oleksii Kurochko 

Same two comments here as for the mem_access.h patch.

Jan

Re: [PATCH v3] xen/asm-generic: ifdef inclusion of

[Jan Beulich]

On 14.11.2023 16:13, Oleksii Kurochko wrote:
> ifdefing inclusion of  in 
> allows to avoid generation of empty  header
> for the case when !CONFIG_MEM_ACCESS.
> 
> For Arm it was explicitly added inclusion of  for p2m.c
> and traps.c because they require some functions from  which
> aren't available in case of !CONFIG_MEM_ACCESS.
> 
> Suggested-by: Jan Beulich 
> Signed-off-by: Oleksii Kurochko 
> 
> ---
> This patch was part of patch series:
> https://lore.kernel.org/xen-devel/cover.1699633310.git.oleksii.kuroc...@gmail.com/
> 
> The patch series hasn't been reviewed all yet so send this path
> separately.
> ---
>  xen/arch/arm/p2m.c   | 6 ++
>  xen/arch/arm/traps.c | 6 ++
>  xen/include/xen/mem_access.h | 2 ++
>  3 files changed, 14 insertions(+)

Also drop PPC's then dead header, please.

> --- a/xen/arch/arm/p2m.c
> +++ b/xen/arch/arm/p2m.c
> @@ -11,6 +11,12 @@
>  #include 
>  #include 
>  #include 
> +/*
> + * Inclusion of  in  is #ifdef-ed with
> + * CONFIG_MEM_ACCESS so in case of !CONFIG_MEM_ACCESS will cause a 
> compilation
> + * issue "implicit declaration of functions 'p2m_mem_access*'.
> + */
> +#include 

Personally I'm against such comments (they simply don't scale), but this
is Arm code, so Arm folks will need to judge.

Jan

Re: [PATCH] automation: set architecture in docker files

[Roger Pau Monné]

On Tue, Nov 14, 2023 at 03:00:17PM +, Anthony PERARD wrote:
> On Tue, Nov 14, 2023 at 10:01:06AM +0100, Roger Pau Monné wrote:
> > On Mon, Nov 13, 2023 at 04:10:24PM -0800, Stefano Stabellini wrote:
> > > On Mon, 13 Nov 2023, Roger Pau Monne wrote:
> > > > Pass the desired architecture of the image in the FROM instruction if 
> > > > the
> > > > image is possibly multi-platform.
> > > > 
> > > > This allows using the x86 Dockerfiles on OS X on arm64 hardware.
> > > > 
> > > > No functional change intended.
> > > > 
> > > > Signed-off-by: Roger Pau Monné 
> > > 
> > > Although I am not opposed to this change, so far we have been using:
> > > arm64v8/alpine:3.18
> > > 
> > > for x86 it is not specified but it would be:
> > > amd64/alpine:3.18
> > > 
> > > Two options:
> > > 1) we add amd64/ everywhere and leave the arm containers alone
> > > 2) we change all containers, including the arm containers, to use the
> > > --platform option
> > > 
> > > I don't think is a good idea to have 2 different ways to specify the
> > > architecture for x86 and arm containers
> > 
> > I'm not an expert on this, but attempting to use (2):
> > 
> > -FROM arm64v8/alpine:3.18
> > +FROM --platform=linux/arm64v8 alpine:3.18
> > 
> > Does not work for me:
> > 
> > % make -C automation/build alpine/3.18-arm64v8
> > docker build --pull -t 
> > registry.gitlab.com/xen-project/xen/alpine:3.18-arm64v8 -f 
> > alpine/3.18-arm64v8.dockerfile alpine
> > [+] Building 1.4s (3/3) FINISHED
> >   docker:desktop-linux
> >  => [internal] load .dockerignore   
> >   0.0s
> >  => => transferring context: 2B 
> >   0.0s
> >  => [internal] load build definition from 3.18-arm64v8.dockerfile   
> >   0.0s
> >  => => transferring dockerfile: 818B
> >   0.0s
> >  => ERROR [internal] load metadata for docker.io/library/alpine:3.18
> >   1.4s
> > --
> >  > [internal] load metadata for docker.io/library/alpine:3.18:
> > --
> > 3.18-arm64v8.dockerfile:1
> > 
> >1 | >>> FROM --platform=linux/arm64v8 alpine:3.18
> >2 | LABEL maintainer.name="The Xen Project" \
> >3 |   maintainer.email="xen-devel@lists.xenproject.org"
> > 
> > ERROR: failed to solve: alpine:3.18: no match for platform in manifest 
> > sha256:eece025e432126ce23f223450a0326fbebde39cdf496a85d8c016293fc851978: 
> > not found
> > make: *** [alpine/3.18-arm64v8] Error 1
> > 
> > That's why I've left the prefixed images alone.
> > 
> > I could prefix the x86 images with amd64/ if that's preferred, I
> > didn't try that option, as the Docker manual suggested using
> > --platform.
> 
> So a few things to know, "--platform=linux/amd64" just select a
> different build of one container. For example, for the "alpine"
> containers, you can see all the different builds available on the docker
> hub, here a few links:
> - Official Docker, Alpine images, home:
>   https://hub.docker.com/_/alpine
> - The different builds: 
>   https://hub.docker.com/_/alpine/tags
> 
> So, for amd64v8, you probably want --platform=linux/arm64/v8

Interesting, I guess I was looking at an outdated documentation that
stated the tag as arm64v8 instead of arm64/v8.

> 
> Then, they are per-architecture repository that make it easier to deal
> with foreign architecture, and probably maintained by a different
> community. e.g. for alpine arm64v8:
> https://hub.docker.com/r/arm64v8/alpine/
> 
> Those provide a build for a single architecture.

Right, so those two are not actually the same image.  I wonder whether
we would want to uniformly switch to using --platform when possible,
in order to make sure we are using the same (multi arch) image to
avoid surprises.

> 
> 
> Sometime, you actually need to "--platform=*" to select a particular
> architecture, like I did for "jessie-i386.dockerfile".
> 
> 
> One thing I've notice when using --platform is that, if for example I
> use the container "--platform=linux/amd64 alpine:3" then
> "--platform=linux/arm/v6 alpine:3"; later when I only specify
> "alpine:3", it's going to be the armv6, and I think docker is going to
> complain if I try tu use "--platform=linux/amd64 alpine:3" without
> "docker pull" first (or I guess docker build --pull).
> 
> Hope that help.
> 
> So I guess using containers "amd64/*" or "arm64v8/*" is fine, but
> sometime will need to use "--platform=*".

My take is that it's better to use --platform when possible, as then
all platforms share the same image, and the contents of the image
should be more consistent.

I guess we could see about switching some of the image that currently
use a prefix (like the Alpine one) in order to instead use --platform
and share the same image.  I wouldn't want to do it in 

[PATCH-for-9.0 9/9] hw/xen/hvm: Inline xen_arch_set_memory()

[Philippe Mathieu-Daudé]

xen_arch_set_memory() is not arch-specific anymore. Being
called once, inline it in xen_set_memory().

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/hw/xen/xen-hvm-common.h |   3 -
 hw/xen/xen-hvm-common.c | 104 
 2 files changed, 51 insertions(+), 56 deletions(-)

diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-common.h
index 536712dc83..a1b8a2783b 100644
--- a/include/hw/xen/xen-hvm-common.h
+++ b/include/hw/xen/xen-hvm-common.h
@@ -99,8 +99,5 @@ void cpu_ioreq_pio(ioreq_t *req);
 
 void xen_read_physmap(XenIOState *state);
 void xen_arch_handle_ioreq(XenIOState *state, ioreq_t *req);
-void xen_arch_set_memory(XenIOState *state,
- MemoryRegionSection *section,
- bool add);
 
 #endif /* HW_XEN_HVM_COMMON_H */
diff --git a/hw/xen/xen-hvm-common.c b/hw/xen/xen-hvm-common.c
index 50ce45effc..789c6b4b7a 100644
--- a/hw/xen/xen-hvm-common.c
+++ b/hw/xen/xen-hvm-common.c
@@ -426,50 +426,6 @@ void qmp_xen_set_global_dirty_log(bool enable, Error 
**errp)
 }
 }
 
-void xen_arch_set_memory(XenIOState *state, MemoryRegionSection *section,
- bool add)
-{
-unsigned target_page_bits = qemu_target_page_bits();
-int page_size = qemu_target_page_size();
-int page_mask = -page_size;
-hwaddr start_addr = section->offset_within_address_space;
-ram_addr_t size = int128_get64(section->size);
-bool log_dirty = memory_region_is_logging(section->mr, DIRTY_MEMORY_VGA);
-hvmmem_type_t mem_type;
-
-if (!memory_region_is_ram(section->mr)) {
-return;
-}
-
-if (log_dirty != add) {
-return;
-}
-
-trace_xen_client_set_memory(start_addr, size, log_dirty);
-
-start_addr &= page_mask;
-size = ROUND_UP(size, page_size);
-
-if (add) {
-if (!memory_region_is_rom(section->mr)) {
-xen_add_to_physmap(state, start_addr, size,
-   section->mr, section->offset_within_region);
-} else {
-mem_type = HVMMEM_ram_ro;
-if (xen_set_mem_type(xen_domid, mem_type,
- start_addr >> target_page_bits,
- size >> target_page_bits)) {
-DPRINTF("xen_set_mem_type error, addr: "HWADDR_FMT_plx"\n",
-start_addr);
-}
-}
-} else {
-if (xen_remove_from_physmap(state, start_addr, size) < 0) {
-DPRINTF("physmapping does not exist at "HWADDR_FMT_plx"\n", 
start_addr);
-}
-}
-}
-
 void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size, MemoryRegion *mr,
Error **errp)
 {
@@ -512,20 +468,62 @@ static void xen_set_memory(struct MemoryListener 
*listener,
bool add)
 {
 XenIOState *state = container_of(listener, XenIOState, memory_listener);
+unsigned target_page_bits = qemu_target_page_bits();
+int page_size = qemu_target_page_size();
+int page_mask = -page_size;
+hwaddr start_addr;
+ram_addr_t size;
+bool log_dirty;
+hvmmem_type_t mem_type;
+
 
 if (section->mr == _memory) {
 return;
-} else {
-if (add) {
-xen_map_memory_section(xen_domid, state->ioservid,
-   section);
-} else {
-xen_unmap_memory_section(xen_domid, state->ioservid,
- section);
-}
 }
 
-xen_arch_set_memory(state, section, add);
+if (add) {
+xen_map_memory_section(xen_domid, state->ioservid,
+   section);
+} else {
+xen_unmap_memory_section(xen_domid, state->ioservid,
+ section);
+}
+
+if (!memory_region_is_ram(section->mr)) {
+return;
+}
+
+log_dirty = memory_region_is_logging(section->mr, DIRTY_MEMORY_VGA);
+
+if (log_dirty != add) {
+return;
+}
+
+start_addr = section->offset_within_address_space;
+size = int128_get64(section->size);
+trace_xen_client_set_memory(start_addr, size, log_dirty);
+
+start_addr &= page_mask;
+size = ROUND_UP(size, page_size);
+
+if (add) {
+if (!memory_region_is_rom(section->mr)) {
+xen_add_to_physmap(state, start_addr, size,
+   section->mr, section->offset_within_region);
+} else {
+mem_type = HVMMEM_ram_ro;
+if (xen_set_mem_type(xen_domid, mem_type,
+ start_addr >> target_page_bits,
+ size >> target_page_bits)) {
+DPRINTF("xen_set_mem_type error, addr: "HWADDR_FMT_plx"\n",
+start_addr);
+}
+}
+} else {
+if (xen_remove_from_physmap(state, start_addr, size) < 0) {
+DPRINTF("physmapping does not exist at "HWADDR_FMT_plx"\n", 
start_addr);
+

[XEN PATCH v2] automation/eclair: add a deviation for MISRA C:2012 Rule 8.6

[Federico Serafini]

Update ECLAIR configuration to take into account the search
procedure adopted by Unix linkers.
Update deviations.rst accordingly and tag Rule 8.6 as "clean".

Signed-off-by: Federico Serafini 
---
Changes is v2:
  - deviation is based on xen/lib/*;
  - justification improved;
  - reflected changes to deviations.ecl;
  - tagged rule as "clean".
---
 automation/eclair_analysis/ECLAIR/deviations.ecl | 11 +++
 automation/eclair_analysis/ECLAIR/tagging.ecl|  2 +-
 docs/misra/deviations.rst| 12 
 3 files changed, 24 insertions(+), 1 deletion(-)

diff --git a/automation/eclair_analysis/ECLAIR/deviations.ecl 
b/automation/eclair_analysis/ECLAIR/deviations.ecl
index d8170106b4..d12ac2e28e 100644
--- a/automation/eclair_analysis/ECLAIR/deviations.ecl
+++ b/automation/eclair_analysis/ECLAIR/deviations.ecl
@@ -214,6 +214,17 @@ definition is compiled-out or optimized-out by the 
compiler)"
 -config=MC3R1.R8.6,reports+={deliberate, "first_area(^.*has no definition$)"}
 -doc_end
 
+-doc_begin="The search procedure for Unix linkers is well defined, see ld(1)
+manual: \"The linker will search an archive only once, at the location where it
+is specified on the command line. If the archive defines a symbol which was
+undefined in some object which appeared before the archive on the command line,
+the linker will include the appropriate file(s) from the archive\".
+In Xen, thanks to the order in which file names appear in the build commands,
+if arch-specific definitions are present, they get always linked in before
+searching in the lib.a archive resulting from xen/lib."
+-config=MC3R1.R8.6,declarations+={deliberate, "loc(file(^xen/lib/.*$))"}
+-doc_end
+
 -doc_begin="The gnu_inline attribute without static is deliberately allowed."
 -config=MC3R1.R8.10,declarations+={deliberate,"property(gnu_inline)"}
 -doc_end
diff --git a/automation/eclair_analysis/ECLAIR/tagging.ecl 
b/automation/eclair_analysis/ECLAIR/tagging.ecl
index 78a0bc948b..d0824830d7 100644
--- a/automation/eclair_analysis/ECLAIR/tagging.ecl
+++ b/automation/eclair_analysis/ECLAIR/tagging.ecl
@@ -30,7 +30,7 @@
 
 -doc_begin="Clean guidelines: new violations for these guidelines are not 
accepted."
 
--service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R2.2||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R8.1||MC3R1.R8.5||MC3R1.R8.8||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R9.2||MC3R1.R9.4||MC3R1.R9.5||MC3R1.R12.5||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.6||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6"
+-service_selector={clean_guidelines_common,"MC3R1.D1.1||MC3R1.D2.1||MC3R1.D4.11||MC3R1.D4.14||MC3R1.R1.1||MC3R1.R1.3||MC3R1.R1.4||MC3R1.R2.2||MC3R1.R3.1||MC3R1.R3.2||MC3R1.R4.1||MC3R1.R5.1||MC3R1.R5.2||MC3R1.R5.4||MC3R1.R6.1||MC3R1.R6.2||MC3R1.R7.1||MC3R1.R8.1||MC3R1.R8.5||MC3R1.R8.6||MC3R1.R8.8||MC3R1.R8.10||MC3R1.R8.12||MC3R1.R8.14||MC3R1.R9.2||MC3R1.R9.4||MC3R1.R9.5||MC3R1.R12.5||MC3R1.R17.3||MC3R1.R17.4||MC3R1.R17.6||MC3R1.R21.13||MC3R1.R21.19||MC3R1.R21.21||MC3R1.R22.2||MC3R1.R22.4||MC3R1.R22.5||MC3R1.R22.6"
 }
 
 -setq=target,getenv("XEN_TARGET_ARCH")
diff --git a/docs/misra/deviations.rst b/docs/misra/deviations.rst
index 8511a18925..0d18e33780 100644
--- a/docs/misra/deviations.rst
+++ b/docs/misra/deviations.rst
@@ -147,6 +147,18 @@ Deviations related to MISRA C:2012 Rules:
definition is compiled-out or optimized-out by the compiler).
  - Tagged as `deliberate` in ECLAIR.
 
+   * - R8.6
+ - The search procedure for Unix linkers is well defined, see ld(1) manual:
+   "The linker will search an archive only once, at the location where it
+   is specified on the command line. If the archive defines a symbol which
+   was undefined in some object which appeared before the archive on the
+   command line, the linker will include the appropriate file(s) from the
+   archive".
+   In Xen, thanks to the order in which file names appear in the build
+   commands, if arch-specific definitions are present, they get always
+   linked in before searching in the lib.a archive resulting from xen/lib.
+ - Tagged as `deliberate` for ECLAIR.
+
* - R8.10
  - The gnu_inline attribute without static is deliberately allowed.
  - Tagged as `deliberate` for ECLAIR.
-- 
2.34.1

xen | Failed pipeline for staging-4.17 | 0527bab0

[GitLab]

Pipeline #1071977342 has failed!

Project: xen ( https://gitlab.com/xen-project/xen )
Branch: staging-4.17 ( 
https://gitlab.com/xen-project/xen/-/commits/staging-4.17 )

Commit: 0527bab0 ( 
https://gitlab.com/xen-project/xen/-/commit/0527bab0901b800e3f1be2e7836c5346b6e08809
 )
Commit Message: x86/spec-ctrl: Add SRSO whitepaper URL

... now...
Commit Author: Andrew Cooper ( https://gitlab.com/andyhhp )
Committed by: Jan Beulich ( https://gitlab.com/jbeulich )


Pipeline #1071977342 ( 
https://gitlab.com/xen-project/xen/-/pipelines/1071977342 ) triggered by Ganis 
( https://gitlab.com/ganis )
had 1 failed job.

Job #5532419311 ( https://gitlab.com/xen-project/xen/-/jobs/5532419311/raw )

Stage: build
Name: debian-buster-gcc-ibt

-- 
You're receiving this email because of your account on gitlab.com.

[RFC PATCH-for-9.0 8/9] hw/xen/hvm: Merge xen-hvm-common.c files

[Philippe Mathieu-Daudé]

hw/i386/xen/xen-hvm-common.c content is target agnostic,
and should be common to all targets. Merge both files.
Remove the now unnecessary xen_register_framebuffer() stub.

ARM targets now inherit the common xen_memory_listener.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/arm/xen_arm.c |  24 --
 hw/i386/xen/xen-hvm-common.c | 473 ---
 hw/xen/xen-hvm-common.c  | 458 +
 stubs/xen-hw-stub.c  |   4 -
 hw/i386/xen/meson.build  |   1 -
 5 files changed, 458 insertions(+), 502 deletions(-)
 delete mode 100644 hw/i386/xen/xen-hvm-common.c

diff --git a/hw/arm/xen_arm.c b/hw/arm/xen_arm.c
index 39dcd74d07..0ead84c9e1 100644
--- a/hw/arm/xen_arm.c
+++ b/hw/arm/xen_arm.c
@@ -38,17 +38,6 @@
 #define TYPE_XEN_ARM  MACHINE_TYPE_NAME("xenpvh")
 OBJECT_DECLARE_SIMPLE_TYPE(XenArmState, XEN_ARM)
 
-const MemoryListener xen_memory_listener = {
-.region_add = xen_region_add,
-.region_del = xen_region_del,
-.log_start = NULL,
-.log_stop = NULL,
-.log_sync = NULL,
-.log_global_start = NULL,
-.log_global_stop = NULL,
-.priority = MEMORY_LISTENER_PRIORITY_ACCEL,
-};
-
 struct XenArmState {
 /*< private >*/
 MachineState parent;
@@ -136,19 +125,6 @@ void xen_arch_handle_ioreq(XenIOState *state, ioreq_t *req)
 return;
 }
 
-void xen_arch_set_memory(XenIOState *state, MemoryRegionSection *section,
- bool add)
-{
-}
-
-void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t length)
-{
-}
-
-void qmp_xen_set_global_dirty_log(bool enable, Error **errp)
-{
-}
-
 #ifdef CONFIG_TPM
 static void xen_enable_tpm(XenArmState *xam)
 {
diff --git a/hw/i386/xen/xen-hvm-common.c b/hw/i386/xen/xen-hvm-common.c
deleted file mode 100644
index e8ef0e0c94..00
--- a/hw/i386/xen/xen-hvm-common.c
+++ /dev/null
@@ -1,473 +0,0 @@
-/*
- * Copyright (C) 2010   Citrix Ltd.
- *
- * This work is licensed under the terms of the GNU GPL, version 2.  See
- * the COPYING file in the top-level directory.
- *
- * Contributions after 2012-01-13 are licensed under the terms of the
- * GNU GPL, version 2 or (at your option) any later version.
- */
-
-#include "qemu/osdep.h"
-#include "qemu/range.h"
-#include "qapi/qapi-commands-migration.h"
-#include "exec/target_page.h"
-#include "hw/xen/xen-hvm-common.h"
-#include "trace.h"
-
-static MemoryRegion *framebuffer;
-static bool xen_in_migration;
-
-static QLIST_HEAD(, XenPhysmap) xen_physmap;
-static const XenPhysmap *log_for_dirtybit;
-/* Buffer used by xen_sync_dirty_bitmap */
-static unsigned long *dirty_bitmap;
-
-static XenPhysmap *get_physmapping(hwaddr start_addr, ram_addr_t size,
-   int page_mask)
-{
-XenPhysmap *physmap = NULL;
-
-start_addr &= -page_mask;
-
-QLIST_FOREACH(physmap, _physmap, list) {
-if (range_covers_byte(physmap->start_addr, physmap->size, start_addr)) 
{
-return physmap;
-}
-}
-return NULL;
-}
-
-static hwaddr xen_phys_offset_to_gaddr(hwaddr phys_offset, ram_addr_t size,
-   int page_mask)
-{
-hwaddr addr = phys_offset & -page_mask;
-XenPhysmap *physmap = NULL;
-
-QLIST_FOREACH(physmap, _physmap, list) {
-if (range_covers_byte(physmap->phys_offset, physmap->size, addr)) {
-return physmap->start_addr + (phys_offset - physmap->phys_offset);
-}
-}
-
-return phys_offset;
-}
-
-#ifdef XEN_COMPAT_PHYSMAP
-static int xen_save_physmap(XenIOState *state, XenPhysmap *physmap)
-{
-char path[80], value[17];
-
-snprintf(path, sizeof(path),
-"/local/domain/0/device-model/%d/physmap/%"PRIx64"/start_addr",
-xen_domid, (uint64_t)physmap->phys_offset);
-snprintf(value, sizeof(value), "%"PRIx64, (uint64_t)physmap->start_addr);
-if (!xs_write(state->xenstore, 0, path, value, strlen(value))) {
-return -1;
-}
-snprintf(path, sizeof(path),
-"/local/domain/0/device-model/%d/physmap/%"PRIx64"/size",
-xen_domid, (uint64_t)physmap->phys_offset);
-snprintf(value, sizeof(value), "%"PRIx64, (uint64_t)physmap->size);
-if (!xs_write(state->xenstore, 0, path, value, strlen(value))) {
-return -1;
-}
-if (physmap->name) {
-snprintf(path, sizeof(path),
-"/local/domain/0/device-model/%d/physmap/%"PRIx64"/name",
-xen_domid, (uint64_t)physmap->phys_offset);
-if (!xs_write(state->xenstore, 0, path,
-  physmap->name, strlen(physmap->name))) {
-return -1;
-}
-}
-return 0;
-}
-#else
-static int xen_save_physmap(XenIOState *state, XenPhysmap *physmap)
-{
-return 0;
-}
-#endif
-
-static int xen_add_to_physmap(XenIOState *state,
-  hwaddr start_addr,
-  ram_addr_t size,
-  MemoryRegion *mr,
-  

[RFC PATCH-for-9.0 6/9] hw/xen/hvm: Initialize xen_physmap QLIST in xen_read_physmap()

[Philippe Mathieu-Daudé]

xen_read_physmap() is the first function requiring
xen_physmap QLIST being initialized. Move the init
call there.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/i386/xen/xen-hvm.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c
index 789779d02c..3b9c31c1c8 100644
--- a/hw/i386/xen/xen-hvm.c
+++ b/hw/i386/xen/xen-hvm.c
@@ -532,6 +532,8 @@ void xen_read_physmap(XenIOState *state)
 char path[80], *value = NULL;
 char **entries = NULL;
 
+QLIST_INIT(_physmap);
+
 snprintf(path, sizeof(path),
 "/local/domain/0/device-model/%d/physmap", xen_domid);
 entries = xs_directory(state->xenstore, 0, path, );
@@ -575,6 +577,7 @@ void xen_read_physmap(XenIOState *state)
 #else
 void xen_read_physmap(XenIOState *state)
 {
+QLIST_INIT(_physmap);
 }
 #endif
 
@@ -595,7 +598,6 @@ void xen_hvm_init_pc(PCMachineState *pcms, MemoryRegion 
**ram_memory)
 
 xen_register_ioreq(state, max_cpus, _memory_listener);
 
-QLIST_INIT(_physmap);
 xen_read_physmap(state);
 
 suspend.notify = xen_suspend_notifier;
-- 
2.41.0

[PATCH-for-9.0 7/9] hw/xen/hvm: Extract common code to xen-hvm-common.c

[Philippe Mathieu-Daudé]

Extract non-x86 specific code out of xen-hvm.c,
to xen-hvm-common.c. For now this new file is
only build for x86 targets.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/i386/xen/xen-hvm-common.c | 473 +++
 hw/i386/xen/xen-hvm.c| 459 -
 hw/i386/xen/meson.build  |   1 +
 3 files changed, 474 insertions(+), 459 deletions(-)
 create mode 100644 hw/i386/xen/xen-hvm-common.c

diff --git a/hw/i386/xen/xen-hvm-common.c b/hw/i386/xen/xen-hvm-common.c
new file mode 100644
index 00..e8ef0e0c94
--- /dev/null
+++ b/hw/i386/xen/xen-hvm-common.c
@@ -0,0 +1,473 @@
+/*
+ * Copyright (C) 2010   Citrix Ltd.
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2.  See
+ * the COPYING file in the top-level directory.
+ *
+ * Contributions after 2012-01-13 are licensed under the terms of the
+ * GNU GPL, version 2 or (at your option) any later version.
+ */
+
+#include "qemu/osdep.h"
+#include "qemu/range.h"
+#include "qapi/qapi-commands-migration.h"
+#include "exec/target_page.h"
+#include "hw/xen/xen-hvm-common.h"
+#include "trace.h"
+
+static MemoryRegion *framebuffer;
+static bool xen_in_migration;
+
+static QLIST_HEAD(, XenPhysmap) xen_physmap;
+static const XenPhysmap *log_for_dirtybit;
+/* Buffer used by xen_sync_dirty_bitmap */
+static unsigned long *dirty_bitmap;
+
+static XenPhysmap *get_physmapping(hwaddr start_addr, ram_addr_t size,
+   int page_mask)
+{
+XenPhysmap *physmap = NULL;
+
+start_addr &= -page_mask;
+
+QLIST_FOREACH(physmap, _physmap, list) {
+if (range_covers_byte(physmap->start_addr, physmap->size, start_addr)) 
{
+return physmap;
+}
+}
+return NULL;
+}
+
+static hwaddr xen_phys_offset_to_gaddr(hwaddr phys_offset, ram_addr_t size,
+   int page_mask)
+{
+hwaddr addr = phys_offset & -page_mask;
+XenPhysmap *physmap = NULL;
+
+QLIST_FOREACH(physmap, _physmap, list) {
+if (range_covers_byte(physmap->phys_offset, physmap->size, addr)) {
+return physmap->start_addr + (phys_offset - physmap->phys_offset);
+}
+}
+
+return phys_offset;
+}
+
+#ifdef XEN_COMPAT_PHYSMAP
+static int xen_save_physmap(XenIOState *state, XenPhysmap *physmap)
+{
+char path[80], value[17];
+
+snprintf(path, sizeof(path),
+"/local/domain/0/device-model/%d/physmap/%"PRIx64"/start_addr",
+xen_domid, (uint64_t)physmap->phys_offset);
+snprintf(value, sizeof(value), "%"PRIx64, (uint64_t)physmap->start_addr);
+if (!xs_write(state->xenstore, 0, path, value, strlen(value))) {
+return -1;
+}
+snprintf(path, sizeof(path),
+"/local/domain/0/device-model/%d/physmap/%"PRIx64"/size",
+xen_domid, (uint64_t)physmap->phys_offset);
+snprintf(value, sizeof(value), "%"PRIx64, (uint64_t)physmap->size);
+if (!xs_write(state->xenstore, 0, path, value, strlen(value))) {
+return -1;
+}
+if (physmap->name) {
+snprintf(path, sizeof(path),
+"/local/domain/0/device-model/%d/physmap/%"PRIx64"/name",
+xen_domid, (uint64_t)physmap->phys_offset);
+if (!xs_write(state->xenstore, 0, path,
+  physmap->name, strlen(physmap->name))) {
+return -1;
+}
+}
+return 0;
+}
+#else
+static int xen_save_physmap(XenIOState *state, XenPhysmap *physmap)
+{
+return 0;
+}
+#endif
+
+static int xen_add_to_physmap(XenIOState *state,
+  hwaddr start_addr,
+  ram_addr_t size,
+  MemoryRegion *mr,
+  hwaddr offset_within_region)
+{
+unsigned target_page_bits = qemu_target_page_bits();
+int page_size = qemu_target_page_size();
+int page_mask = -page_size;
+unsigned long nr_pages;
+int rc = 0;
+XenPhysmap *physmap = NULL;
+hwaddr pfn, start_gpfn;
+hwaddr phys_offset = memory_region_get_ram_addr(mr);
+const char *mr_name;
+
+if (get_physmapping(start_addr, size, page_mask)) {
+return 0;
+}
+if (size <= 0) {
+return -1;
+}
+
+/* Xen can only handle a single dirty log region for now and we want
+ * the linear framebuffer to be that region.
+ * Avoid tracking any regions that is not videoram and avoid tracking
+ * the legacy vga region. */
+if (mr == framebuffer && start_addr > 0xb) {
+goto go_physmap;
+}
+return -1;
+
+go_physmap:
+DPRINTF("mapping vram to %"HWADDR_PRIx" - %"HWADDR_PRIx"\n",
+start_addr, start_addr + size);
+
+mr_name = memory_region_name(mr);
+
+physmap = g_new(XenPhysmap, 1);
+
+physmap->start_addr = start_addr;
+physmap->size = size;
+physmap->name = mr_name;
+physmap->phys_offset = phys_offset;
+
+QLIST_INSERT_HEAD(_physmap, physmap, list);
+
+if 

[PATCH-for-9.0 4/9] hw/xen/hvm: Expose xen_memory_listener declaration

[Philippe Mathieu-Daudé]

There can only be a single xen_memory_listener definition
in a qemu-system binary.

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/hw/xen/xen-hvm-common.h | 1 +
 hw/arm/xen_arm.c| 2 +-
 hw/i386/xen/xen-hvm.c   | 2 +-
 3 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-common.h
index 83ed16f425..0fed15ed04 100644
--- a/include/hw/xen/xen-hvm-common.h
+++ b/include/hw/xen/xen-hvm-common.h
@@ -18,6 +18,7 @@
 extern MemoryRegion xen_memory;
 extern MemoryListener xen_io_listener;
 extern DeviceListener xen_device_listener;
+extern const MemoryListener xen_memory_listener;
 
 //#define DEBUG_XEN_HVM
 
diff --git a/hw/arm/xen_arm.c b/hw/arm/xen_arm.c
index b478d74ea0..39dcd74d07 100644
--- a/hw/arm/xen_arm.c
+++ b/hw/arm/xen_arm.c
@@ -38,7 +38,7 @@
 #define TYPE_XEN_ARM  MACHINE_TYPE_NAME("xenpvh")
 OBJECT_DECLARE_SIMPLE_TYPE(XenArmState, XEN_ARM)
 
-static const MemoryListener xen_memory_listener = {
+const MemoryListener xen_memory_listener = {
 .region_add = xen_region_add,
 .region_del = xen_region_del,
 .log_start = NULL,
diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c
index b64204ea94..a65a96f0de 100644
--- a/hw/i386/xen/xen-hvm.c
+++ b/hw/i386/xen/xen-hvm.c
@@ -469,7 +469,7 @@ static void xen_log_global_stop(MemoryListener *listener)
 xen_in_migration = false;
 }
 
-static const MemoryListener xen_memory_listener = {
+const MemoryListener xen_memory_listener = {
 .name = "xen-memory",
 .region_add = xen_region_add,
 .region_del = xen_region_del,
-- 
2.41.0

[PATCH-for-9.0 2/9] hw/xen/hvm: Propagate page_mask to a pair of functions

[Philippe Mathieu-Daudé]

We are going to replace TARGET_PAGE_MASK by a
runtime variable. In order to reduce code duplication,
propagate TARGET_PAGE_MASK to get_physmapping() and
xen_phys_offset_to_gaddr().

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/i386/xen/xen-hvm.c | 18 ++
 1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c
index 8aa6a1ec3b..3b10425986 100644
--- a/hw/i386/xen/xen-hvm.c
+++ b/hw/i386/xen/xen-hvm.c
@@ -174,11 +174,12 @@ static void xen_ram_init(PCMachineState *pcms,
 }
 }
 
-static XenPhysmap *get_physmapping(hwaddr start_addr, ram_addr_t size)
+static XenPhysmap *get_physmapping(hwaddr start_addr, ram_addr_t size,
+   int page_mask)
 {
 XenPhysmap *physmap = NULL;
 
-start_addr &= TARGET_PAGE_MASK;
+start_addr &= page_mask;
 
 QLIST_FOREACH(physmap, _physmap, list) {
 if (range_covers_byte(physmap->start_addr, physmap->size, start_addr)) 
{
@@ -188,9 +189,10 @@ static XenPhysmap *get_physmapping(hwaddr start_addr, 
ram_addr_t size)
 return NULL;
 }
 
-static hwaddr xen_phys_offset_to_gaddr(hwaddr phys_offset, ram_addr_t size)
+static hwaddr xen_phys_offset_to_gaddr(hwaddr phys_offset, ram_addr_t size,
+   int page_mask)
 {
-hwaddr addr = phys_offset & TARGET_PAGE_MASK;
+hwaddr addr = phys_offset & page_mask;
 XenPhysmap *physmap = NULL;
 
 QLIST_FOREACH(physmap, _physmap, list) {
@@ -252,7 +254,7 @@ static int xen_add_to_physmap(XenIOState *state,
 hwaddr phys_offset = memory_region_get_ram_addr(mr);
 const char *mr_name;
 
-if (get_physmapping(start_addr, size)) {
+if (get_physmapping(start_addr, size, TARGET_PAGE_MASK)) {
 return 0;
 }
 if (size <= 0) {
@@ -325,7 +327,7 @@ static int xen_remove_from_physmap(XenIOState *state,
 XenPhysmap *physmap = NULL;
 hwaddr phys_offset = 0;
 
-physmap = get_physmapping(start_addr, size);
+physmap = get_physmapping(start_addr, size, TARGET_PAGE_MASK);
 if (physmap == NULL) {
 return -1;
 }
@@ -373,7 +375,7 @@ static void xen_sync_dirty_bitmap(XenIOState *state,
 int rc, i, j;
 const XenPhysmap *physmap = NULL;
 
-physmap = get_physmapping(start_addr, size);
+physmap = get_physmapping(start_addr, size, TARGET_PAGE_MASK);
 if (physmap == NULL) {
 /* not handled */
 return;
@@ -633,7 +635,7 @@ void xen_hvm_modified_memory(ram_addr_t start, ram_addr_t 
length)
 int rc;
 ram_addr_t start_pfn, nb_pages;
 
-start = xen_phys_offset_to_gaddr(start, length);
+start = xen_phys_offset_to_gaddr(start, length, TARGET_PAGE_MASK);
 
 if (length == 0) {
 length = TARGET_PAGE_SIZE;
-- 
2.41.0

[PATCH-for-9.0 5/9] hw/xen/hvm: Expose xen_read_physmap() prototype

[Philippe Mathieu-Daudé]

In a pair of commit we are going to call xen_read_physmap()
out of hw/i386/xen/xen-hvm.c.

Signed-off-by: Philippe Mathieu-Daudé 
---
 include/hw/xen/xen-hvm-common.h | 1 +
 hw/i386/xen/xen-hvm.c   | 4 ++--
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/include/hw/xen/xen-hvm-common.h b/include/hw/xen/xen-hvm-common.h
index 0fed15ed04..536712dc83 100644
--- a/include/hw/xen/xen-hvm-common.h
+++ b/include/hw/xen/xen-hvm-common.h
@@ -97,6 +97,7 @@ void xen_register_ioreq(XenIOState *state, unsigned int 
max_cpus,
 
 void cpu_ioreq_pio(ioreq_t *req);
 
+void xen_read_physmap(XenIOState *state);
 void xen_arch_handle_ioreq(XenIOState *state, ioreq_t *req);
 void xen_arch_set_memory(XenIOState *state,
  MemoryRegionSection *section,
diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c
index a65a96f0de..789779d02c 100644
--- a/hw/i386/xen/xen-hvm.c
+++ b/hw/i386/xen/xen-hvm.c
@@ -525,7 +525,7 @@ static void handle_vmport_ioreq(XenIOState *state, ioreq_t 
*req)
 }
 
 #ifdef XEN_COMPAT_PHYSMAP
-static void xen_read_physmap(XenIOState *state)
+void xen_read_physmap(XenIOState *state)
 {
 XenPhysmap *physmap = NULL;
 unsigned int len, num, i;
@@ -573,7 +573,7 @@ static void xen_read_physmap(XenIOState *state)
 free(entries);
 }
 #else
-static void xen_read_physmap(XenIOState *state)
+void xen_read_physmap(XenIOState *state)
 {
 }
 #endif
-- 
2.41.0

[RFC PATCH-for-9.0 0/9] hw/xen: Have ARM targets use common xen_memory_listener

[Philippe Mathieu-Daudé]

Hi,

While looking at Xen target-specific code, I noticed some
generic code used by x86 which is not implemented for ARM.

Maybe ARM machines don't need it, I don't know. But I
wanted to see if I can get this common code target agnostic
and build it once, possibly bringing smth useful to ARM.

The changes don't break CI testing and Avocado local tests.
If this xen_memory_listener feature isn't required for ARM,
I'll send follow up patch that keep this series with common
xen_memory_listener but with a runtime flag to disable.

Patches do the usual "change target-specific API to a
target-agnostic one", in particular using "exec/target_page.h"
at runtime. Then non-x86 code is extracted, then merged to
the generic xen/xen-hvm-common.c.

Thoughts?

Regards,

Phil.

Based-on: <20231114143816.71079-1-phi...@linaro.org>

Philippe Mathieu-Daudé (9):
  hw/xen/hvm: Inline TARGET_PAGE_ALIGN() macro
  hw/xen/hvm: Propagate page_mask to a pair of functions
  hw/xen/hvm: Get target page size at runtime
  hw/xen/hvm: Expose xen_memory_listener declaration
  hw/xen/hvm: Expose xen_read_physmap() prototype
  hw/xen/hvm: Initialize xen_physmap QLIST in xen_read_physmap()
  hw/xen/hvm: Extract common code to xen-hvm-common.c
  hw/xen/hvm: Merge xen-hvm-common.c files
  hw/xen/hvm: Inline xen_arch_set_memory()

 include/hw/xen/xen-hvm-common.h |   5 +-
 hw/arm/xen_arm.c|  24 --
 hw/i386/xen/xen-hvm.c   | 441 +
 hw/xen/xen-hvm-common.c | 474 +++-
 stubs/xen-hw-stub.c |   4 -
 5 files changed, 468 insertions(+), 480 deletions(-)

-- 
2.41.0

[PATCH-for-9.0 3/9] hw/xen/hvm: Get target page size at runtime

[Philippe Mathieu-Daudé]

In order to build this file once for all targets, replace:

  TARGET_PAGE_BITS -> qemu_target_page_bits()
  TARGET_PAGE_SIZE -> qemu_target_page_size()
  TARGET_PAGE_MASK -> -qemu_target_page_size()

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/i386/xen/xen-hvm.c | 62 +++
 1 file changed, 39 insertions(+), 23 deletions(-)

diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c
index 3b10425986..b64204ea94 100644
--- a/hw/i386/xen/xen-hvm.c
+++ b/hw/i386/xen/xen-hvm.c
@@ -22,6 +22,7 @@
 
 #include "hw/xen/xen-hvm-common.h"
 #include 
+#include "exec/target_page.h"
 #include "cpu.h"
 
 static MemoryRegion ram_640k, ram_lo, ram_hi;
@@ -247,6 +248,9 @@ static int xen_add_to_physmap(XenIOState *state,
   MemoryRegion *mr,
   hwaddr offset_within_region)
 {
+unsigned target_page_bits = qemu_target_page_bits();
+int page_size = qemu_target_page_size();
+int page_mask = -page_size;
 unsigned long nr_pages;
 int rc = 0;
 XenPhysmap *physmap = NULL;
@@ -254,7 +258,7 @@ static int xen_add_to_physmap(XenIOState *state,
 hwaddr phys_offset = memory_region_get_ram_addr(mr);
 const char *mr_name;
 
-if (get_physmapping(start_addr, size, TARGET_PAGE_MASK)) {
+if (get_physmapping(start_addr, size, page_mask)) {
 return 0;
 }
 if (size <= 0) {
@@ -294,9 +298,9 @@ go_physmap:
 return 0;
 }
 
-pfn = phys_offset >> TARGET_PAGE_BITS;
-start_gpfn = start_addr >> TARGET_PAGE_BITS;
-nr_pages = size >> TARGET_PAGE_BITS;
+pfn = phys_offset >> target_page_bits;
+start_gpfn = start_addr >> target_page_bits;
+nr_pages = size >> target_page_bits;
 rc = xendevicemodel_relocate_memory(xen_dmod, xen_domid, nr_pages, pfn,
 start_gpfn);
 if (rc) {
@@ -310,8 +314,8 @@ go_physmap:
 }
 
 rc = xendevicemodel_pin_memory_cacheattr(xen_dmod, xen_domid,
-   start_addr >> TARGET_PAGE_BITS,
-   (start_addr + size - 1) >> TARGET_PAGE_BITS,
+   start_addr >> target_page_bits,
+   (start_addr + size - 1) >> target_page_bits,
XEN_DOMCTL_MEM_CACHEATTR_WB);
 if (rc) {
 error_report("pin_memory_cacheattr failed: %s", strerror(errno));
@@ -323,11 +327,14 @@ static int xen_remove_from_physmap(XenIOState *state,
hwaddr start_addr,
ram_addr_t size)
 {
+unsigned target_page_bits = qemu_target_page_bits();
+int page_size = qemu_target_page_size();
+int page_mask = -page_size;
 int rc = 0;
 XenPhysmap *physmap = NULL;
 hwaddr phys_offset = 0;
 
-physmap = get_physmapping(start_addr, size, TARGET_PAGE_MASK);
+physmap = get_physmapping(start_addr, size, page_mask);
 if (physmap == NULL) {
 return -1;
 }
@@ -338,9 +345,9 @@ static int xen_remove_from_physmap(XenIOState *state,
 DPRINTF("unmapping vram to %"HWADDR_PRIx" - %"HWADDR_PRIx", at "
 "%"HWADDR_PRIx"\n", start_addr, start_addr + size, phys_offset);
 
-size >>= TARGET_PAGE_BITS;
-start_addr >>= TARGET_PAGE_BITS;
-phys_offset >>= TARGET_PAGE_BITS;
+size >>= target_page_bits;
+start_addr >>= target_page_bits;
+phys_offset >>= target_page_bits;
 rc = xendevicemodel_relocate_memory(xen_dmod, xen_domid, size, start_addr,
 phys_offset);
 if (rc) {
@@ -369,13 +376,16 @@ static void xen_sync_dirty_bitmap(XenIOState *state,
   hwaddr start_addr,
   ram_addr_t size)
 {
-hwaddr npages = size >> TARGET_PAGE_BITS;
+unsigned target_page_bits = qemu_target_page_bits();
+int page_size = qemu_target_page_size();
+int page_mask = -page_size;
+hwaddr npages = size >> target_page_bits;
 const int width = sizeof(unsigned long) * 8;
 size_t bitmap_size = DIV_ROUND_UP(npages, width);
 int rc, i, j;
 const XenPhysmap *physmap = NULL;
 
-physmap = get_physmapping(start_addr, size, TARGET_PAGE_MASK);
+physmap = get_physmapping(start_addr, size, page_mask);
 if (physmap == NULL) {
 /* not handled */
 return;
@@ -389,7 +399,7 @@ static void xen_sync_dirty_bitmap(XenIOState *state,
 return;
 }
 
-rc = xen_track_dirty_vram(xen_domid, start_addr >> TARGET_PAGE_BITS,
+rc = xen_track_dirty_vram(xen_domid, start_addr >> target_page_bits,
   npages, dirty_bitmap);
 if (rc < 0) {
 #ifndef ENODATA
@@ -410,8 +420,7 @@ static void xen_sync_dirty_bitmap(XenIOState *state,
 j = ctzl(map);
 map &= ~(1ul << j);
 memory_region_set_dirty(framebuffer,
-(i * width + j) * 

[PATCH-for-9.0 1/9] hw/xen/hvm: Inline TARGET_PAGE_ALIGN() macro

[Philippe Mathieu-Daudé]

Use TARGET_PAGE_SIZE to calculate TARGET_PAGE_ALIGN.

Signed-off-by: Philippe Mathieu-Daudé 
---
 hw/i386/xen/xen-hvm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/i386/xen/xen-hvm.c b/hw/i386/xen/xen-hvm.c
index f1c30d1384..8aa6a1ec3b 100644
--- a/hw/i386/xen/xen-hvm.c
+++ b/hw/i386/xen/xen-hvm.c
@@ -678,7 +678,7 @@ void xen_arch_set_memory(XenIOState *state, 
MemoryRegionSection *section,
 trace_xen_client_set_memory(start_addr, size, log_dirty);
 
 start_addr &= TARGET_PAGE_MASK;
-size = TARGET_PAGE_ALIGN(size);
+size = ROUND_UP(size, TARGET_PAGE_SIZE);
 
 if (add) {
 if (!memory_region_is_rom(section->mr)) {
-- 
2.41.0

[xen-unstable-smoke test] 183750: tolerable all pass - PUSHED

[osstest service owner]

flight 183750 xen-unstable-smoke real [real]
http://logs.test-lab.xenproject.org/osstest/logs/183750/

Failures :-/ but no regressions.

Tests which did not succeed, but are not blocking:
 test-amd64-amd64-libvirt 15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  15 migrate-support-checkfail   never pass
 test-arm64-arm64-xl-xsm  16 saverestore-support-checkfail   never pass
 test-armhf-armhf-xl  15 migrate-support-checkfail   never pass
 test-armhf-armhf-xl  16 saverestore-support-checkfail   never pass

version targeted for testing:
 xen  a48bb129f1b9ff55c22cf6d2b589247c8ba3b10e
baseline version:
 xen  fb41228ececea948c7953c8c16fe28fd65c6536b

Last test of basis   183743  2023-11-13 15:02:04 Z1 days
Testing same since   183750  2023-11-14 13:03:54 Z0 days1 attempts


People who touched revisions under test:
  Andrew Cooper 
  Roger Pau Monne 
  Roger Pau Monné 

jobs:
 build-arm64-xsm  pass
 build-amd64  pass
 build-armhf  pass
 build-amd64-libvirt  pass
 test-armhf-armhf-xl  pass
 test-arm64-arm64-xl-xsm  pass
 test-amd64-amd64-xl-qemuu-debianhvm-amd64pass
 test-amd64-amd64-libvirt pass



sg-report-flight on osstest.test-lab.xenproject.org
logs: /home/logs/logs
images: /home/logs/images

Logs, config files, etc. are available at
http://logs.test-lab.xenproject.org/osstest/logs

Explanation of these reports, and of osstest in general, is at
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README.email;hb=master
http://xenbits.xen.org/gitweb/?p=osstest.git;a=blob;f=README;hb=master

Test harness code can be found at
http://xenbits.xen.org/gitweb?p=osstest.git;a=summary


Pushing revision :

To xenbits.xen.org:/home/xen/git/xen.git
   fb41228ece..a48bb129f1  a48bb129f1b9ff55c22cf6d2b589247c8ba3b10e -> smoke

Re: [PATCH-for-9.0 v2 06/19] hw/pci/msi: Restrict xen_is_pirq_msi() call to Xen

[David Woodhouse]

On Tue, 2023-11-14 at 16:22 +0100, Philippe Mathieu-Daudé wrote:
> 
> If so, possibly few places incorrectly check 'xen_enabled()'
> instead of this 'xen_guest()'.

Sorry, I meant to respond to that one directly. I don't *think* there
are any cases of that. As I added the CONFIG_XEN_EMU support, I moved a
bunch of stuff to live under CONFIG_XEN_BUS instead of CONFIG_XEN,
fixing them up (and implementing the emulated versions of grant table
operations, event channel operations etc. which no longer come from the
actual Xen libraries).

The existing cases of xen_enabled() really *are* being used to mean
'xen_accel_enabled()', AFAIK. Apart from that one you just tried to add
:)


smime.p7s
Description: S/MIME cryptographic signature

Re: [XEN PATCH][for-4.19] domain: add ASSERT to help static analysis tools

[Nicola Vetrini]

On 2023-11-11 02:13, Stefano Stabellini wrote:

On Fri, 10 Nov 2023, Nicola Vetrini wrote:

Hi everyone,

I trimmed the thread a bit, to make this more readable.

> > > > > IMHO, the only viable option would be to have a configuration to
> > > > > keep
> > > > > ASSERT in production build for scanning tools.
> > > >
> > > > But wouldn't that then likely mean scanning to be done on builds not
> > > > also
> > > > used in production? Would doing so even be permitted when
> > > > certification
> > > > is a requirement? Or do you expect such production builds to be used
> > > > with
> > > > the assertions left in place (increasing the risk of a crash; recall
> > > > that
> > > > assertions themselves may also be wrong, and hence one triggering in
> > > > rare
> > > > cases may not really be a reason to bring down the system)?
> > >
> > > I will leave Stefano/Nicola to answer from the certification
> > > perspective. But
> > > I don't really see how we could get away unless we replace most of the
> > > ASSERT() with proper runtime check (which may not be desirable for
> > > ASSERT()s
> > > like this one).
> >
> > For sure we don't want to replace ASSERTs with runtime checks.
> >
> > Nicola, do we really need the ASSERT to be implemented as a check, or
> > would the presence of the ASSERT alone suffice as a tag, the same way we
> > would be using /* SAF-xx-safe */ or asmlinkage?
> >
> > If we only need ASSERT as a deviation tag, then production builds vs.
> > debug build doesn't matter.
> >
> > If ECLAIR actually needs ASSERT to be implemented as a check, could we
> > have a special #define to define ASSERT in a special way for static
> > analysis tools in production builds? For instance:
> >
> > #ifdef STATIC_ANALYSIS
> > #define ASSERT(p) \
> >  do { if ( unlikely(!(p)) ) printk("ASSERT triggered %s:%d",
> > __file__,__LINE__); } while (0)
> > #endif
>
> Just to make 100% clear, you are saying that assessor will be happy if we
> analyze it with ASSERT enabled but in production we use it wout them
> enabled? The assumption here is that they should have *never* been triggered
> so they surely should not happen in production.
>
> Cheers,

First of all, Andrew is experimenting with an alternate solution, so 
we should

wait making
any decision here until he can share the outcome of his findings.
However, from a certification perspective, the fact that the codebase 
is

tested with
asserts enabled is a strong enough claim for a justification to be 
based on an

assertion;
the code path just needs to be exercised by the tests.
Getting into the business of how to define asserts for static analysis 
is

likely to
just cause more trouble.


That's great. Also given Andrew's reply, then can we just go ahead with
adding the ASSERT as done in this patch (with the added in-code comment
as requested by Jan)?


Yes, and sorry for the late reply.

--
Nicola Vetrini, BSc
Software Engineer, BUGSENG srl (https://bugseng.com)

Re: Clang-format configuration discussion - pt 1

[Luca Fancellu]

> On 14 Nov 2023, at 15:59, Jan Beulich  wrote:
> 
> On 14.11.2023 15:59, Luca Fancellu wrote:
>> 
>> 
>>> On 13 Nov 2023, at 16:27, Jan Beulich  wrote:
>>> 
>>> On 13.11.2023 16:20, Luca Fancellu wrote:
> On 13 Nov 2023, at 11:31, Jan Beulich  wrote:
> On 08.11.2023 10:53, Luca Fancellu wrote:
> --
>> 
>> Standard: C++03
>> 
>> ---
>> From the documentation: Parse and format C++ constructs compatible with 
>> this standard.
> 
> Since I continue to be puzzled - iirc you said this is because of lack
> of availability of "C99" as a value here. What's entirely unclear to
> me is: How does this matter to a tool checking coding style (which is
> largely about formatting, not any lexical or syntactical aspects)?
> 
>> This value is used also in Linux.
> 
> Considering how different the two styles are, I don't think this is
> overly relevant.
 
 Ok, maybe I understand your point, you are looking for a reason to declare 
 this configurable instead
 of not specifying it at all?
>>> 
>>> Not really, no. Here I was merely saying that with the styles being
>>> sufficiently different, what Linux uses is probably not very significant
>>> for our own decision.
>>> 
 If it’s that, from what I understand clang-format will use the default 
 value if we don’t specify anything
 for this one, so it will take ‘Latest’. I think we should put a value for 
 this one to fix it and don’t have
 surprises if that behaviour changes and seeing that also in Linux that 
 value is fixed increased my
 confidence.
 
 However, if you feel that we should not specify it, I’ve done a test and 
 not specifying it is not changing
 the current output. I can’t say that for a different clang-format version 
 though or if changes happen in the
 future.
>>> 
>>> It's fine to set values. All I'm saying is that at least I would prefer
>>> if it was also clear what exact effect the setting of a value has,
>>> especially when that does not really match the language we use in the
>>> project.
>> 
>> Yes I agree, I think Alejandro’s reply to this configurable reflects my 
>> thoughts about it.
>> 
>> So if we all agree that we should set this parameter, do we all agree that 
>> it should be the
>> value above?
>> 
>> Do you have other concerns regarding this or the other parameters in this 
>> thread?
> 
> I did raise what was occurring to me. This doesn't mean that down the
> road yet something else might not pop up.

Sure, thanks for raising your concern, I’ve asked so that at the deadline if no 
other concern
are raised, we can move on with another set of configurable to discuss.

Re: Clang-format configuration discussion - pt 1

[Jan Beulich]

On 14.11.2023 15:59, Luca Fancellu wrote:
> 
> 
>> On 13 Nov 2023, at 16:27, Jan Beulich  wrote:
>>
>> On 13.11.2023 16:20, Luca Fancellu wrote:
 On 13 Nov 2023, at 11:31, Jan Beulich  wrote:
 On 08.11.2023 10:53, Luca Fancellu wrote:
 --
>
> Standard: C++03
>
> ---
> From the documentation: Parse and format C++ constructs compatible with 
> this standard.

 Since I continue to be puzzled - iirc you said this is because of lack
 of availability of "C99" as a value here. What's entirely unclear to
 me is: How does this matter to a tool checking coding style (which is
 largely about formatting, not any lexical or syntactical aspects)?

> This value is used also in Linux.

 Considering how different the two styles are, I don't think this is
 overly relevant.
>>>
>>> Ok, maybe I understand your point, you are looking for a reason to declare 
>>> this configurable instead
>>> of not specifying it at all?
>>
>> Not really, no. Here I was merely saying that with the styles being
>> sufficiently different, what Linux uses is probably not very significant
>> for our own decision.
>>
>>> If it’s that, from what I understand clang-format will use the default 
>>> value if we don’t specify anything
>>> for this one, so it will take ‘Latest’. I think we should put a value for 
>>> this one to fix it and don’t have
>>> surprises if that behaviour changes and seeing that also in Linux that 
>>> value is fixed increased my
>>> confidence.
>>>
>>> However, if you feel that we should not specify it, I’ve done a test and 
>>> not specifying it is not changing
>>> the current output. I can’t say that for a different clang-format version 
>>> though or if changes happen in the
>>> future.
>>
>> It's fine to set values. All I'm saying is that at least I would prefer
>> if it was also clear what exact effect the setting of a value has,
>> especially when that does not really match the language we use in the
>> project.
> 
> Yes I agree, I think Alejandro’s reply to this configurable reflects my 
> thoughts about it.
> 
> So if we all agree that we should set this parameter, do we all agree that it 
> should be the
> value above?
> 
> Do you have other concerns regarding this or the other parameters in this 
> thread?

I did raise what was occurring to me. This doesn't mean that down the
road yet something else might not pop up.

Jan

Re: [PATCH-for-9.0 v2 18/19] hw/i386/xen: Compile 'xen-hvm.c' with Xen CPPFLAGS

[David Woodhouse]

On 14 November 2023 09:38:14 GMT-05:00, "Philippe Mathieu-Daudé" 
 wrote:
>xen-hvm.c calls xc_set_hvm_param() from ,
>so better compile it with Xen CPPFLAGS.
>
>Signed-off-by: Philippe Mathieu-Daudé 

Reviewed-by: David Woodhouse 

Re: [PATCH-for-9.0 v2 16/19] hw/xen/xen_pt: Add missing license

[David Woodhouse]

On 14 November 2023 09:38:12 GMT-05:00, "Philippe Mathieu-Daudé" 
 wrote:
>Commit eaab4d60d3 ("Introduce Xen PCI Passthrough, qdevice")
>introduced both xen_pt.[ch], but only added the license to
>xen_pt.c. Use the same license for xen_pt.h.
>
>Suggested-by: David Woodhouse 
>Signed-off-by: Philippe Mathieu-Daudé 


Reviewed-by: David Woodhouse 

Re: [PATCH-for-9.0 v2 10/19] hw/xen: Rename 'ram_memory' global variable as 'xen_memory'

[David Woodhouse]

On 14 November 2023 09:38:06 GMT-05:00, "Philippe Mathieu-Daudé" 
 wrote:
>To avoid a potential global variable shadow in
>hw/i386/pc_piix.c::pc_init1(), rename Xen's
>"ram_memory" as "xen_memory".
>
>Signed-off-by: Philippe Mathieu-Daudé 

Well OK, but aren't you going to be coming back later to eliminate global 
variables which are actually per-VM?

Or is that the point, because *then* the conflicting name will actually matter?

Reviewed-by: David Woodhouse 

Re: [PATCH-for-9.0 v2 06/19] hw/pci/msi: Restrict xen_is_pirq_msi() call to Xen

[David Woodhouse]

On 14 November 2023 10:22:23 GMT-05:00, "Philippe Mathieu-Daudé" 
 wrote:
>On 14/11/23 16:13, David Woodhouse wrote:
>> On 14 November 2023 09:38:02 GMT-05:00, "Philippe Mathieu-Daudé" 
>>  wrote:
>>> Similarly to the restriction in hw/pci/msix.c (see commit
>>> e1e4bf2252 "msix: fix msix_vector_masked"), restrict the
>>> xen_is_pirq_msi() call in msi_is_masked() to Xen.
>>> 
>>> Signed-off-by: Philippe Mathieu-Daudé 
>> 
>> Hm, we do also support the Xen abomination of snooping on MSI table writes 
>> to see if they're targeted at a Xen PIRQ, then actually unmasking the MSI 
>> from QEMU when the guest binds the corresponding event channel to that PIRQ.
>> 
>> I think this is going to break in CI as kvm_xen_guest.py does deliberately 
>> exercise that use case, doesn't it?
>
>Hmmm I see what you mean.
>
>So you mentioned these checks:
>
>- host Xen accel
>- Xen accel emulated to guest via KVM host accel
>
>Maybe we need here:
>
>- guest expected to run Xen
>
>  Being (
>Xen accel emulated to guest via KVM host accel
>   OR
>host Xen accel
>)
>
>If so, possibly few places incorrectly check 'xen_enabled()'
>instead of this 'xen_guest()'.

I think xen_is_pirq_msi() had that test built in, didn't it? Adding a 
'xen_enabled() &&' prefix was technically redundant? 

What's the actual problem we're trying to solve here? That we had two separate 
implementations of xen_is_pirq_msi() (three if you count an empty stub?) which 
are resolved at link time and prevent you from running Xen-accel and KVM-accel 
VMs within the same QEMU process?

>"Xen on KVM" is a tricky case...
>
>> I deliberately *didn't* switch to testing the Xen PV net device, with a 
>> comment that testing MSI and irqchip permutations was far more entertaining. 
>> So I hope it should catch this?
>
>¯\_(ツ)_/¯

I believe that if you push your branch to a gitlab tree with the right CI 
variables defined, it'll run all the CI? And I *hope* it fails with this patch. 
It's precisely the kind of thing I was *intending* to catch with the testing!

Re: [PATCH-for-9.0 v2 01/19] tests/avocado: Add 'guest:xen' tag to tests running Xen guest

[Philippe Mathieu-Daudé]

On 14/11/23 16:19, David Woodhouse wrote:

On 14 November 2023 10:13:14 GMT-05:00, "Philippe Mathieu-Daudé" 
 wrote:

On 14/11/23 16:08, David Woodhouse wrote:

On 14 November 2023 10:00:09 GMT-05:00, "Philippe Mathieu-Daudé" 
 wrote:

On 14/11/23 15:50, David Woodhouse wrote:

On 14 November 2023 09:37:57 GMT-05:00, "Philippe Mathieu-Daudé" 
 wrote:

Add a tag to run all Xen-specific tests using:

$ make check-avocado AVOCADO_TAGS='guest:xen'

Signed-off-by: Philippe Mathieu-Daudé 
---
tests/avocado/boot_xen.py  | 3 +++
tests/avocado/kvm_xen_guest.py | 1 +
2 files changed, 4 insertions(+)


Those two are very different. One runs on Xen, the other on KVM. Do we want to 
use the same tag for both?


My understanding is,
- boot_xen.py runs Xen on TCG
- kvm_xen_guest.py runs Xen on KVM
so both runs Xen guests.


Does boot_xen.py actually boot *Xen*? And presumably at least one Xen guest 
*within* Xen?


I'll let Alex confirm, but yes, I expect Xen guest within Xen guest within TCG. So the tags 
"accel:tcg" (already present) and "guest:xen".


kvm_xen_guest.py boots a "Xen guest" under KVM directly without any real Xen 
being present. It's *emulating* Xen.


Yes, so the tag "guest:xen" is correct.


They do both run Xen guests (or at least guests which use Xen hypercalls and 
*think* they're running under Xen). But is that the important classification 
for lumping them together?


The idea of AVOCADO_TAGS is to restrict testing to what you want to cover. So 
here this allow running 'anything that can run Xen guest'
in a single command, for example it is handy on my macOS aarch64 host.


Ok, that makes sense then. Thanks for your patience.


No problem, I'll add a better description in v3.


Reviewed-by: David Woodhouse 


Thanks!

[XEN PATCH v4 2/2] docs/misra: add guidance on the format of Dir 4.1 docs for ECLAIR

[Nicola Vetrini]

Additional guidance on the formatting of the document for ECLAIR
is supplied.

Signed-off-by: Nicola Vetrini 
---
 docs/misra/C-runtime-failures.rst | 8 
 1 file changed, 8 insertions(+)

diff --git a/docs/misra/C-runtime-failures.rst 
b/docs/misra/C-runtime-failures.rst
index 1052b2afca13..cac51d6b2596 100644
--- a/docs/misra/C-runtime-failures.rst
+++ b/docs/misra/C-runtime-failures.rst
@@ -12,6 +12,14 @@ built-in checks that are present in the language.
 The presence of such documentation is requested by MISRA C:2012 Directive 4.1,
 whose headline states: "Run-time failures shall be minimized".
 
+The ECLAIR checker for MISRA C:2012 Directive 4.1 requires the documentation
+to be supplied using the following format:
+
+``Documentation for MISRA C:2012 Dir 4.1:  ``
+
+The matched categories are the ones listed below (e.g., ``overflow`` and
+``unexpected wrapping``). The content of the description is not checked and can
+span multiple lines.
 
 Documentation for MISRA C:2012 Dir 4.1: overflow
 
-- 
2.34.1

[XEN PATCH v4 0/2] use the documentation for MISRA C:2012 Dir 4.1

[Nicola Vetrini]

This series addresses some concerns raised on patches 2 and 3 from [1].
Note that patch 1 from that series has already been applied.

Patch 1 comprises a modified version of patches 2 and 3 of the previous series.
Patch 2 is brand new, as it merely clarifies how to write such documentation.

[1] 
https://lore.kernel.org/xen-devel/cover.1696231870.git.nicola.vetr...@bugseng.com/

Nicola Vetrini (2):
  automation/eclair: make the docs for MISRA C:2012 Dir 4.1 visible to
ECLAIR
  docs/misra: add guidance on the format of  Dir 4.1 docs for ECLAIR

 automation/eclair_analysis/build.sh   | 21 +++--
 automation/eclair_analysis/prepare.sh |  7 ---
 docs/misra/C-runtime-failures.rst |  8 
 3 files changed, 31 insertions(+), 5 deletions(-)

-- 
2.34.1

[XEN PATCH v4 1/2] automation/eclair: make the docs for MISRA C:2012 Dir 4.1 visible to ECLAIR

[Nicola Vetrini]

To be able to check for the existence of the necessary subsections in
the documentation for MISRA C:2012 Dir 4.1, ECLAIR needs to have a source
file that is built.

This file is generated from 'C-runtime-failures.rst' in docs/misra
and the configuration is updated accordingly.

Signed-off-by: Nicola Vetrini 
---
Changes from RFC:
- Dropped unused/useless code
- Revised the sed command
- Revised the clean target

Changes in v2:
- Added explanative comment to the makefile
- printf instead of echo

Changes in v3:
- Terminate the generated file with a newline
- Build it with -std=c99, so that the documentation
  for D1.1 applies.
Changes in v5:
- Transform and build the file directly in the eclair-specific directory
---
 automation/eclair_analysis/build.sh   | 21 +++--
 automation/eclair_analysis/prepare.sh |  7 ---
 2 files changed, 23 insertions(+), 5 deletions(-)

diff --git a/automation/eclair_analysis/build.sh 
b/automation/eclair_analysis/build.sh
index ec087dd822fa..f24292ed0643 100755
--- a/automation/eclair_analysis/build.sh
+++ b/automation/eclair_analysis/build.sh
@@ -33,12 +33,29 @@ else
   PROCESSORS=6
 fi
 
+runtime_failures_docs() {
+  doc="C-runtime-failures.rst"
+  builddir="automation/eclair_analysis"
+  
+  cp "docs/misra/${doc}" "${builddir}"
+  cd "${builddir}"
+  printf "/*\n\n" >"${doc}.tmp"
+  sed -e 's|\*/|*//*|g' "${doc}" >>"${doc}.tmp"
+  printf "\n\n*/\n" >>"${doc}.tmp"
+  mv "${doc}.tmp" "${doc}.c"
+  
+  # Cannot redirect to /dev/null because it would be excluded from the analysis
+  "${CROSS_COMPILE}gcc-12" -std=c99 -c "${doc}.c" -o "${doc}.o"
+  cd -
+}
+
 (
-  cd xen
+  runtime_failures_docs
 
   make "-j${PROCESSORS}" "-l${PROCESSORS}.0"\
"CROSS_COMPILE=${CROSS_COMPILE}" \
"CC=${CROSS_COMPILE}gcc-12"  \
"CXX=${CROSS_COMPILE}g++-12" \
-   "XEN_TARGET_ARCH=${XEN_TARGET_ARCH}"
+   "XEN_TARGET_ARCH=${XEN_TARGET_ARCH}" \
+   -C xen
 )
diff --git a/automation/eclair_analysis/prepare.sh 
b/automation/eclair_analysis/prepare.sh
index 0cac5eba00ae..fe9d16e48ecc 100755
--- a/automation/eclair_analysis/prepare.sh
+++ b/automation/eclair_analysis/prepare.sh
@@ -35,11 +35,12 @@ else
 fi
 
 (
-cd xen
-cp "${CONFIG_FILE}" .config
+./configure
+cp "${CONFIG_FILE}" xen/.config
 make clean
 find . -type f -name "*.safparse" -print -delete
-make -f ${script_dir}/Makefile.prepare prepare
+cd xen
+make -f "${script_dir}/Makefile.prepare" prepare
 # Translate the /* SAF-n-safe */ comments into ECLAIR CBTs
 scripts/xen-analysis.py --run-eclair --no-build --no-clean
 )
-- 
2.34.1

Re: [RFC PATCH-for-9.0 v2 09/19] hw/block/xen_blkif: Align structs with QEMU_ALIGNED() instead of #pragma

[David Woodhouse]

On 14 November 2023 09:38:05 GMT-05:00, "Philippe Mathieu-Daudé" 
 wrote:
>Except imported source files, QEMU code base uses
>the QEMU_ALIGNED() macro to align its structures.
>
>Signed-off-by: Philippe Mathieu-Daudé 

Can we have a BUILD_BUG_ON(sizeof==) for these please?

Re: [PATCH-for-9.0 v2 07/19] hw/xen: Remove unnecessary xen_hvm_inject_msi() stub

[David Woodhouse]

On 14 November 2023 09:38:03 GMT-05:00, "Philippe Mathieu-Daudé" 
 wrote:
>Since commit 04b0de0ee8 ("xen: factor out common functions")
>xen_hvm_inject_msi() stub is not required.
>
>Signed-off-by: Philippe Mathieu-Daudé 

Reviewed-by: David Woodhouse 

Re: Clang-format configuration discussion - pt 1

[Alejandro Vallejo]

Hi,

On Tue, Nov 14, 2023 at 02:59:35PM +, Luca Fancellu wrote:
> 
> 
> > On 13 Nov 2023, at 16:27, Jan Beulich  wrote:
> > 
> > On 13.11.2023 16:20, Luca Fancellu wrote:
> >>> On 13 Nov 2023, at 11:31, Jan Beulich  wrote:
> >>> On 08.11.2023 10:53, Luca Fancellu wrote:
> >>> --
>  
>  Standard: C++03
>  
>  ---
>  From the documentation: Parse and format C++ constructs compatible with 
>  this standard.
> >>> 
> >>> Since I continue to be puzzled - iirc you said this is because of lack
> >>> of availability of "C99" as a value here. What's entirely unclear to
> >>> me is: How does this matter to a tool checking coding style (which is
> >>> largely about formatting, not any lexical or syntactical aspects)?
> >>> 
>  This value is used also in Linux.
> >>> 
> >>> Considering how different the two styles are, I don't think this is
> >>> overly relevant.
> >> 
> >> Ok, maybe I understand your point, you are looking for a reason to declare 
> >> this configurable instead
> >> of not specifying it at all?
> > 
> > Not really, no. Here I was merely saying that with the styles being
> > sufficiently different, what Linux uses is probably not very significant
> > for our own decision.
> > 
> >> If it’s that, from what I understand clang-format will use the default 
> >> value if we don’t specify anything
> >> for this one, so it will take ‘Latest’. I think we should put a value for 
> >> this one to fix it and don’t have
> >> surprises if that behaviour changes and seeing that also in Linux that 
> >> value is fixed increased my
> >> confidence.
> >> 
> >> However, if you feel that we should not specify it, I’ve done a test and 
> >> not specifying it is not changing
> >> the current output. I can’t say that for a different clang-format version 
> >> though or if changes happen in the
> >> future.
> > 
> > It's fine to set values. All I'm saying is that at least I would prefer
> > if it was also clear what exact effect the setting of a value has,
> > especially when that does not really match the language we use in the
> > project.
On C, allegedly, none. It ought to control defaults for things like
SpaceBeforeCpp11BracedList, SpacesInAngles and other C++-specific things,
because the C++ language sticks syntactical extensions every other Tuesday.
Alas, whatever it does (there's no full list). I'd feel a lot more
comfortable knowing it won't change under our feet.

For reference, clang-format's docs state as an example:

```
c++03: latest:
vector > x;   vs. vector> x;
```

> 
> Yes I agree, I think Alejandro’s reply to this configurable reflects my 
> thoughts about it.
> 
> So if we all agree that we should set this parameter, do we all agree that it 
> should be the
> value above?
> 
> Do you have other concerns regarding this or the other parameters in this 
> thread?
> 
> 

Cheers,
Alejandro

Re: [PATCH v2 02/15] xen/asm-generic: introduce generic device.h

[Oleksii]

On Mon, 2023-11-13 at 17:43 +0100, Jan Beulich wrote:
> On 10.11.2023 17:30, Oleksii Kurochko wrote:
> > --- /dev/null
> > +++ b/xen/include/asm-generic/device.h
> > @@ -0,0 +1,140 @@
> > +/* SPDX-License-Identifier: GPL-2.0-only */
> > +#ifndef __ASM_GENERIC_DEVICE_H__
> > +#define __ASM_GENERIC_DEVICE_H__
> > +
> > +enum device_type
> > +{
> > +    DEV_DT,
> > +#ifdef HAS_PCI
> > +    DEV_PCI,
> > +#endif
> > +};
> > +
> > +struct dev_archdata {
> > +    void *iommu;    /* IOMMU private data */
> > +};
> > +
> > +/* struct device - The basic device structure */
> > +struct device
> > +{
> > +    enum device_type type;
> > +#ifdef CONFIG_HAS_DEVICE_TREE
> > +    struct dt_device_node *of_node; /* Used by drivers imported
> > from Linux */
> > +#endif
> 
> There's just this instance where HAS_DEVICE_TREE is checked. Why not
> elsewhere?
> Imo, if you really want this header in asm-generic/, then it wants to
> be truly
> generic (i.e. not implying DT just like you're not implying PCI or
> ACPI).
Mostly copied this file from Arm and only that one part was #ifdef-ed
with HAS_DEVICE_TREE.
But it makes sense to #ifdef DEV_DT, inclusion of 
and definiotn of dev_is_dt() macros.

I'll made the changes in next version of patch series.

Thanks for comments.

~ Oleksii

Re: [PATCH-for-9.0 v2 06/19] hw/pci/msi: Restrict xen_is_pirq_msi() call to Xen

[Philippe Mathieu-Daudé]

On 14/11/23 16:13, David Woodhouse wrote:

On 14 November 2023 09:38:02 GMT-05:00, "Philippe Mathieu-Daudé" 
 wrote:

Similarly to the restriction in hw/pci/msix.c (see commit
e1e4bf2252 "msix: fix msix_vector_masked"), restrict the
xen_is_pirq_msi() call in msi_is_masked() to Xen.

Signed-off-by: Philippe Mathieu-Daudé 


Hm, we do also support the Xen abomination of snooping on MSI table writes to 
see if they're targeted at a Xen PIRQ, then actually unmasking the MSI from 
QEMU when the guest binds the corresponding event channel to that PIRQ.

I think this is going to break in CI as kvm_xen_guest.py does deliberately 
exercise that use case, doesn't it?


Hmmm I see what you mean.

So you mentioned these checks:

- host Xen accel
- Xen accel emulated to guest via KVM host accel

Maybe we need here:

- guest expected to run Xen

  Being (
Xen accel emulated to guest via KVM host accel
OR
host Xen accel
)

If so, possibly few places incorrectly check 'xen_enabled()'
instead of this 'xen_guest()'.

"Xen on KVM" is a tricky case...


I deliberately *didn't* switch to testing the Xen PV net device, with a comment 
that testing MSI and irqchip permutations was far more entertaining. So I hope 
it should catch this?


¯\_(ツ)_/¯

Re: [PATCH-for-9.0 v2 01/19] tests/avocado: Add 'guest:xen' tag to tests running Xen guest

[David Woodhouse]

On 14 November 2023 10:13:14 GMT-05:00, "Philippe Mathieu-Daudé" 
 wrote:
>On 14/11/23 16:08, David Woodhouse wrote:
>> On 14 November 2023 10:00:09 GMT-05:00, "Philippe Mathieu-Daudé" 
>>  wrote:
>>> On 14/11/23 15:50, David Woodhouse wrote:
 On 14 November 2023 09:37:57 GMT-05:00, "Philippe Mathieu-Daudé" 
  wrote:
> Add a tag to run all Xen-specific tests using:
> 
>$ make check-avocado AVOCADO_TAGS='guest:xen'
> 
> Signed-off-by: Philippe Mathieu-Daudé 
> ---
> tests/avocado/boot_xen.py  | 3 +++
> tests/avocado/kvm_xen_guest.py | 1 +
> 2 files changed, 4 insertions(+)
 
 Those two are very different. One runs on Xen, the other on KVM. Do we 
 want to use the same tag for both?
>>> 
>>> My understanding is,
>>> - boot_xen.py runs Xen on TCG
>>> - kvm_xen_guest.py runs Xen on KVM
>>> so both runs Xen guests.
>> 
>> Does boot_xen.py actually boot *Xen*? And presumably at least one Xen guest 
>> *within* Xen?
>
>I'll let Alex confirm, but yes, I expect Xen guest within Xen guest within 
>TCG. So the tags "accel:tcg" (already present) and "guest:xen".
>
>> kvm_xen_guest.py boots a "Xen guest" under KVM directly without any real Xen 
>> being present. It's *emulating* Xen.
>
>Yes, so the tag "guest:xen" is correct.
>
>> They do both run Xen guests (or at least guests which use Xen hypercalls and 
>> *think* they're running under Xen). But is that the important classification 
>> for lumping them together?
>
>The idea of AVOCADO_TAGS is to restrict testing to what you want to cover. So 
>here this allow running 'anything that can run Xen guest'
>in a single command, for example it is handy on my macOS aarch64 host.

Ok, that makes sense then. Thanks for your patience.

Reviewed-by: David Woodhouse