Re: [PATCH 2/3] migration: Drop unnecessary check in ram's pending_exact()
On Wed, 2024-03-20 at 14:57 -0400, Peter Xu wrote:
> On Wed, Mar 20, 2024 at 06:51:26PM +0100, Nina Schoetterl-Glausch wrote:
> > On Wed, 2024-01-17 at 15:58 +0800, pet...@redhat.com wrote:
> > > From: Peter Xu
> > >
> > > When the migration frameworks fetches the exact pending sizes, it means
> > > this check:
> > >
> > > remaining_size < s->threshold_size
> > >
> > > Must have been done already, actually at migration_iteration_run():
> > >
> > > if (must_precopy <= s->threshold_size) {
> > > qemu_savevm_state_pending_exact(_precopy, _postcopy);
> > >
> > > That should be after one round of ram_state_pending_estimate(). It makes
> > > the 2nd check meaningless and can be dropped.
> > >
> > > To say it in another way, when reaching ->state_pending_exact(), we
> > > unconditionally sync dirty bits for precopy.
> > >
> > > Then we can drop migrate_get_current() there too.
> > >
> > > Signed-off-by: Peter Xu
> >
> > Hi Peter,
>
> Hi, Nina,
>
> >
> > could you have a look at this issue:
> > https://gitlab.com/qemu-project/qemu/-/issues/1565
> >
> > which I reopened. Previous thread here:
> >
> > https://lore.kernel.org/qemu-devel/20230324184129.3119575-1-...@linux.ibm.com/
> >
> > I'm seeing migration failures with s390x TCG again, which look the same to
> > me
> > as those a while back.
>
> I'm still quite confused how that could be caused of this.
>
> What you described in the previous bug report seems to imply some page was
> leftover in migration so some page got corrupted after migrated.
>
> However what this patch mostly does is it can sync more than before even if
> I overlooked the condition check there (I still think the check is
> redundant, there's one outlier when remaining_size == threshold_size, but I
> don't think it should matter here as of now). It'll make more sense if
> this patch made the sync less, but that's not the case but vice versa.
[...]
> In the previous discussion, you mentioned that you bisected to the commit
> and also verified the fix. Now you also mentioned in the bz that you can't
> reporduce this bug manually.
>
> Is it still possible to be reproduced with some scripts? Do you also mean
> that it's harder to reproduce comparing to before? In all cases, some way
> to reproduce it would definitely be helpful.
I tried running the kvm-unit-test a bunch of times in a loop and couldn't
trigger a failure. I just tried again on a different system and managed just
fine, yay. No idea why it wouldn't on the first system tho.
>
> Even if we want to revert this change, we'll need to know whether this will
> fix your case so we need something to verify it before a revert. I'll
> consider that the last though as I had a feeling this is papering over
> something else.
I can check if I can reproduce the issue before & after b0504edd ("migration:
Drop unnecessary check in ram's pending_exact()").
I can also check if I can reproduce it on x86, that worked last time.
Anything else? Ideas on how to pinpoint where the corruption happens?
>
> Thanks,
>
Re: [PATCH 2/3] migration: Drop unnecessary check in ram's pending_exact()
I cc'ed Juan, but it looks like he is no longer with Redhat.
Re: [PATCH 2/3] migration: Drop unnecessary check in ram's pending_exact()
On Wed, 2024-01-17 at 15:58 +0800, pet...@redhat.com wrote:
> From: Peter Xu
>
> When the migration frameworks fetches the exact pending sizes, it means
> this check:
>
> remaining_size < s->threshold_size
>
> Must have been done already, actually at migration_iteration_run():
>
> if (must_precopy <= s->threshold_size) {
> qemu_savevm_state_pending_exact(_precopy, _postcopy);
>
> That should be after one round of ram_state_pending_estimate(). It makes
> the 2nd check meaningless and can be dropped.
>
> To say it in another way, when reaching ->state_pending_exact(), we
> unconditionally sync dirty bits for precopy.
>
> Then we can drop migrate_get_current() there too.
>
> Signed-off-by: Peter Xu
Hi Peter,
could you have a look at this issue:
https://gitlab.com/qemu-project/qemu/-/issues/1565
which I reopened. Previous thread here:
https://lore.kernel.org/qemu-devel/20230324184129.3119575-1-...@linux.ibm.com/
I'm seeing migration failures with s390x TCG again, which look the same to me
as those a while back.
> ---
> migration/ram.c | 9 -
> 1 file changed, 4 insertions(+), 5 deletions(-)
>
> diff --git a/migration/ram.c b/migration/ram.c
> index c0cdcccb75..d5b7cd5ac2 100644
> --- a/migration/ram.c
> +++ b/migration/ram.c
> @@ -3213,21 +3213,20 @@ static void ram_state_pending_estimate(void *opaque,
> uint64_t *must_precopy,
> static void ram_state_pending_exact(void *opaque, uint64_t *must_precopy,
> uint64_t *can_postcopy)
> {
> -MigrationState *s = migrate_get_current();
> RAMState **temp = opaque;
> RAMState *rs = *temp;
> +uint64_t remaining_size;
>
> -uint64_t remaining_size = rs->migration_dirty_pages * TARGET_PAGE_SIZE;
> -
> -if (!migration_in_postcopy() && remaining_size < s->threshold_size) {
> +if (!migration_in_postcopy()) {
> bql_lock();
> WITH_RCU_READ_LOCK_GUARD() {
> migration_bitmap_sync_precopy(rs, false);
> }
> bql_unlock();
> -remaining_size = rs->migration_dirty_pages * TARGET_PAGE_SIZE;
> }
>
> +remaining_size = rs->migration_dirty_pages * TARGET_PAGE_SIZE;
> +
> if (migrate_postcopy_ram()) {
> /* We can do postcopy, and all the data is postcopiable */
> *can_postcopy += remaining_size;
This basically reverts 28ef5339c3 ("migration: fix ram_state_pending_exact()"),
which originally
made the issue disappear.
Any thoughts on the matter appreciated.
Thanks,
Nina
Re: [PATCH v2] target/s390x: improve cpu compatibility check error message
On Thu, 2024-03-14 at 22:37 +0100, Claudio Fontana wrote:
> some users were confused by this message showing under TCG:
>
> Selected CPU generation is too new. Maximum supported model
> in the configuration: 'xyz'
>
> Clarify that the maximum can depend on the accel, and add a
> hint to try a different one.
>
> Also add a hint for features mismatch to suggest trying
> different accel, QEMU and kernel versions.
>
> Signed-off-by: Claudio Fontana
Reviewed-by: Nina Schoetterl-Glausch
> ---
> target/s390x/cpu_models.c | 22 +++---
> 1 file changed, 15 insertions(+), 7 deletions(-)
>
> diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
> index 1a1c096122..8ed3bb6a27 100644
> --- a/target/s390x/cpu_models.c
> +++ b/target/s390x/cpu_models.c
> @@ -500,6 +500,16 @@ static void error_prepend_missing_feat(const char *name,
> void *opaque)
> error_prepend((Error **) opaque, "%s ", name);
> }
>
> +static void check_compat_model_failed(Error **errp,
> + const S390CPUModel *max_model,
> + const char *msg)
> +{
> +error_setg(errp, "%s. Maximum supported model in the current
> configuration: \'%s\'",
> + msg, max_model->def->name);
> +error_append_hint(errp, "Consider a different accelerator, try \"-accel
> help\"\n");
> +return;
> +}
> +
> static void check_compatibility(const S390CPUModel *max_model,
> const S390CPUModel *model, Error **errp)
> {
> @@ -507,15 +517,11 @@ static void check_compatibility(const S390CPUModel
> *max_model,
> S390FeatBitmap missing;
>
> if (model->def->gen > max_model->def->gen) {
> -error_setg(errp, "Selected CPU generation is too new. Maximum "
> - "supported model in the configuration: \'%s\'",
> - max_model->def->name);
> +check_compat_model_failed(errp, max_model, "Selected CPU generation
> is too new");
> return;
> } else if (model->def->gen == max_model->def->gen &&
> model->def->ec_ga > max_model->def->ec_ga) {
> -error_setg(errp, "Selected CPU GA level is too new. Maximum "
> - "supported model in the configuration: \'%s\'",
> - max_model->def->name);
> +check_compat_model_failed(errp, max_model, "Selected CPU GA level is
> too new");
> return;
> }
>
> @@ -537,7 +543,9 @@ static void check_compatibility(const S390CPUModel
> *max_model,
> error_setg(errp, " ");
> s390_feat_bitmap_to_ascii(missing, errp, error_prepend_missing_feat);
> error_prepend(errp, "Some features requested in the CPU model are not "
> - "available in the configuration: ");
> + "available in the current configuration: ");
> +error_append_hint(errp,
> + "Consider a different accelerator, QEMU, or kernel
> version\n");
If I'm reading the regex right, this ^, a string literal on a separate line, is
excluded
from the line length check.
> }
>
> S390CPUModel *get_max_cpu_model(Error **errp)
Re: [PATCH] target/s390x: improve cpu compatibility check error message
On Thu, 2024-03-14 at 20:00 +0100, Claudio Fontana wrote:
> some users were confused by this message showing under TCG:
>
> Selected CPU generation is too new. Maximum supported model
> in the configuration: 'xyz'
>
> Try to clarify that the maximum can depend on the accel by
> adding also the current accelerator to the message as such:
>
> Selected CPU generation is too new. Maximum supported model
> in the accelerator 'tcg' configuration: 'xyz'
>
> Signed-off-by: Claudio Fontana
> ---
> target/s390x/cpu_models.c | 11 ++-
> 1 file changed, 6 insertions(+), 5 deletions(-)
>
> diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
> index 1a1c096122..0d6d8fc727 100644
> --- a/target/s390x/cpu_models.c
> +++ b/target/s390x/cpu_models.c
> @@ -508,14 +508,14 @@ static void check_compatibility(const S390CPUModel
> *max_model,
>
> if (model->def->gen > max_model->def->gen) {
> error_setg(errp, "Selected CPU generation is too new. Maximum "
> - "supported model in the configuration: \'%s\'",
> - max_model->def->name);
> + "supported model in the accelerator \'%s\' configuration:
> \'%s\'",
> + current_accel_name(), max_model->def->name);
> return;
> } else if (model->def->gen == max_model->def->gen &&
> model->def->ec_ga > max_model->def->ec_ga) {
> error_setg(errp, "Selected CPU GA level is too new. Maximum "
> - "supported model in the configuration: \'%s\'",
> - max_model->def->name);
> + "supported model in the accelerator \'%s\' configuration:
> \'%s\'",
> + current_accel_name(), max_model->def->name);
> return;
> }
>
> @@ -537,7 +537,8 @@ static void check_compatibility(const S390CPUModel
> *max_model,
> error_setg(errp, " ");
> s390_feat_bitmap_to_ascii(missing, errp, error_prepend_missing_feat);
> error_prepend(errp, "Some features requested in the CPU model are not "
> - "available in the configuration: ");
> + "available in the accelerator \'%s\' configuration: ",
> + current_accel_name());
> }
I wonder if these might not be confusing in other circumstances, e.g. when
running with KVM and the Linux version lacks support for some feature.
I think something along the lines of:
error_...(errp, "... supported by the current configuration ...", ...);
error_append_hint(errp, "Consider using a different accelerator, a different
QEMU version or, when using KVM, a different kernel");
would be better.
I'm not sure about line breaks in error message, I like the better grepability
of unbroken lines but the coding style guide doesn't mention anything.
>
> S390CPUModel *get_max_cpu_model(Error **errp)
Re: [PATCH] docs/s390: clarify even more that cpu-topology is KVM-only
On Thu, 2024-03-14 at 18:22 +0100, Claudio Fontana wrote: > At least for now cpu-topology is implemented only for KVM. > > We already say this, but this tries to be more explicit, > and also show it in the examples. > > This adds a new reference in the introduction that we can point to, > whenever we need to reference accelerators and how to select them. > > Signed-off-by: Claudio Fontana Reviewed-by: Nina Schoetterl-Glausch Tested-by: Nina Schoetterl-Glausch (meaning I ran make html) > --- > docs/system/introduction.rst | 2 ++ > docs/system/s390x/cpu-topology.rst | 14 -- > 2 files changed, 10 insertions(+), 6 deletions(-) > > diff --git a/docs/system/introduction.rst b/docs/system/introduction.rst > index 51ac132d6c..746707eb00 100644 > --- a/docs/system/introduction.rst > +++ b/docs/system/introduction.rst > @@ -1,6 +1,8 @@ > Introduction > > > +.. _Accelerators: > + > Virtualisation Accelerators > --- > > diff --git a/docs/system/s390x/cpu-topology.rst > b/docs/system/s390x/cpu-topology.rst > index 5133fdc362..ca344e273c 100644 > --- a/docs/system/s390x/cpu-topology.rst > +++ b/docs/system/s390x/cpu-topology.rst > @@ -25,17 +25,19 @@ monitor polarization changes, see > ``docs/devel/s390-cpu-topology.rst``. > Prerequisites > - > > -To use the CPU topology, you need to run with KVM on a s390x host that > -uses the Linux kernel v6.0 or newer (which provide the so-called > +To use the CPU topology, you currently need to choose the KVM accelerator. > +See :ref:`Accelerators` for more details about accelerators and how to > select them. > + > +The s390x host needs to use a Linux kernel v6.0 or newer (which provides the > so-called > ``KVM_CAP_S390_CPU_TOPOLOGY`` capability that allows QEMU to signal the > CPU topology facility via the so-called STFLE bit 11 to the VM). > > Enabling CPU topology > - > > -Currently, CPU topology is only enabled in the host model by default. > +Currently, CPU topology is enabled by default only in the "host" cpu model. > > -Enabling CPU topology in a CPU model is done by setting the CPU flag > +Enabling CPU topology in another CPU model is done by setting the CPU flag > ``ctop`` to ``on`` as in: > > .. code-block:: bash > @@ -132,7 +134,7 @@ In the following machine we define 8 sockets with 4 cores > each. > > .. code-block:: bash > > - $ qemu-system-s390x -m 2G \ > + $ qemu-system-s390x -accel kvm -m 2G \ > -cpu gen16b,ctop=on \ > -smp cpus=5,sockets=8,cores=4,maxcpus=32 \ > -device host-s390x-cpu,core-id=14 \ > @@ -227,7 +229,7 @@ with vertical high entitlement. > > .. code-block:: bash > > - $ qemu-system-s390x -m 2G \ > + $ qemu-system-s390x -accel kvm -m 2G \ > -cpu gen16b,ctop=on \ > -smp cpus=1,sockets=8,cores=4,maxcpus=32 \ > \
Re: question on s390x topology: KVM only, or also TCG?
On Thu, 2024-03-14 at 16:54 +0100, Thomas Huth wrote: > On 14/03/2024 16.49, Claudio Fontana wrote: > > Hello Pierre, Ilya, > > > > I have a question on the s390x "topology" feature and examples. > > > > Mainly, is this feature supposed to be KVM accelerator-only, or also > > available when using the TCG accelerator? > > Hi Claudio! > > Pierre left IBM, please CC: Nina with regards to s390x topology instead. > > But with regards to your question, I think I can answer that, too: The > topology feature is currently working with KVM only, yes. It hasn't been > implemented for TCG yet. > > > (docs/devel/s390-cpu-topology.rst vs > > https://www.qemu.org/docs/master/system/s390x/cpu-topology.html) > > > > I see stsi-topology.c in target/s390x/kvm/ , so that part is clearly > > KVM-specific, > > > > but in hw/s390x/cpu-topology.c I read: > > > > " > > * - The first part in this file is taking care of all common functions > > * used by KVM and TCG to create and modify the topology. What Thomas said. Read this as the code in file being independent with respect to the accelerator, it's just that TCG support is missing. [...] > > > > So I would assume this is KVM-only, but then in the "Examples" section > > below I see the example: > > > > " > > $ qemu-system-s390x -m 2G \ > >-cpu gen16b,ctop=on \ TCG doesn't support this cpu ^ and so will refuse to run. > >-smp cpus=5,sockets=8,cores=4,maxcpus=32 \ When running with TCG, drawers & books are supported by -smp also, but well, you cannot do anything with that. [...] >
Re: [RFC] docs/s390: Fix wrong command example in s390-cpu-topology.rst
On Mon, 2023-11-27 at 17:39 +0800, Zhao Liu wrote:
> From: Zhao Liu
>
> From s390_possible_cpu_arch_ids() in hw/s390x/s390-virtio-ccw.c, the
> "core-id" is the index of pssible_cpus->cpus[], so it should only be
s/pssible_cpus/possible_cpus/
> less than possible_cpus->len, which is equal to ms->smp.max_cpus.
>
> Fix the wrong "core-id" 112 because it is greater than maxcpus (36) in
Maybe s/is greater/isn't less/ since the valid ids are 0-35 inclusive.
> -smp.
>
> Cc: Nina Schoetterl-Glausch
> Signed-off-by: Zhao Liu
Reviewed-by: Nina Schoetterl-Glausch
Thanks!
> ---
> RFC: Not tested on S390 machine, just code reading.
> ---
> docs/devel/s390-cpu-topology.rst | 6 +++---
> 1 file changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/docs/devel/s390-cpu-topology.rst
> b/docs/devel/s390-cpu-topology.rst
> index 9eab28d5e5d8..48313b92d417 100644
> --- a/docs/devel/s390-cpu-topology.rst
> +++ b/docs/devel/s390-cpu-topology.rst
> @@ -15,7 +15,7 @@ have default values:
> -smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
> -device z14-s390x-cpu,core-id=19,entitlement=high \
> -device z14-s390x-cpu,core-id=11,entitlement=low \
> --device z14-s390x-cpu,core-id=112,entitlement=high \
> +-device z14-s390x-cpu,core-id=12,entitlement=high \
> ...
>
> Additions to query-cpus-fast
> @@ -78,7 +78,7 @@ modifiers for all configured vCPUs.
>"dedicated": true,
>"thread-id": 537005,
>"props": {
> -"core-id": 112,
> +"core-id": 12,
> "socket-id": 0,
> "drawer-id": 3,
> "book-id": 2
> @@ -86,7 +86,7 @@ modifiers for all configured vCPUs.
>"cpu-state": "operating",
>"entitlement": "high",
>"qom-path": "/machine/peripheral-anon/device[2]",
> - "cpu-index": 112,
> + "cpu-index": 12,
>"target": "s390x"
> }
>]
[PATCH] target/s390x/cpu topology: Fix ordering and creation of TLEs
In case of horizontal polarization entitlement has no effect on
ordering.
Moreover, since the comparison is used to insert CPUs at the correct
position in the TLE list, this affects the creation of TLEs and now
correctly collapses horizontally polarized CPUs into one TLE.
Fixes: f4f54b582f ("target/s390x/cpu topology: handle STSI(15) and build the
SYSIB")
Signed-off-by: Nina Schoetterl-Glausch
---
Unfortunately this wasn't caught by the kvm-unit-test because that also
had a bug :(
target/s390x/kvm/stsi-topology.c | 8 +---
1 file changed, 5 insertions(+), 3 deletions(-)
diff --git a/target/s390x/kvm/stsi-topology.c b/target/s390x/kvm/stsi-topology.c
index efd2aa71f1..c8d6389cd8 100644
--- a/target/s390x/kvm/stsi-topology.c
+++ b/target/s390x/kvm/stsi-topology.c
@@ -210,6 +210,9 @@ static S390TopologyId s390_topology_from_cpu(S390CPU *cpu)
static int s390_topology_id_cmp(const S390TopologyId *l,
const S390TopologyId *r)
{
+int l_polarization = l->vertical ? l->entitlement : 0;
+int r_polarization = r->vertical ? r->entitlement : 0;
+
/*
* lexical order, compare less significant values only if more significant
* ones are equal
@@ -219,9 +222,8 @@ static int s390_topology_id_cmp(const S390TopologyId *l,
l->book - r->book ?:
l->socket - r->socket ?:
l->type - r->type ?:
- /* logic is inverted for the next three */
- r->vertical - l->vertical ?:
- r->entitlement - l->entitlement ?:
+ /* logic is inverted for the next two */
+ r_polarization - l_polarization ?:
r->dedicated - l->dedicated ?:
l->origin - r->origin;
}
base-commit: c60be6e3e38cb36dc66129e757ec4b34152232be
--
2.39.2
Re: [PULL 06/25] s390x/cpu topology: resetting the Topology-Change-Report
On Thu, 2023-10-19 at 21:32 +0200, Thomas Huth wrote: [...] > Thanks, that seems to fix the issue with --enable-debug, too. > I'll squash that into the related patch (also fixing the indentation in > s390_has_topology()) and respin the pull request. > > Thomas > Thanks!
Re: [PULL 06/25] s390x/cpu topology: resetting the Topology-Change-Report
On Thu, 2023-10-19 at 09:35 -0700, Stefan Hajnoczi wrote:
> On Wed, 18 Oct 2023 at 06:09, Thomas Huth wrote:
> >
> > From: Pierre Morel
> >
> > During a subsystem reset the Topology-Change-Report is cleared
> > by the machine.
> > Let's ask KVM to clear the Modified Topology Change Report (MTCR)
> > bit of the SCA in the case of a subsystem reset.
> >
> > Signed-off-by: Pierre Morel
> > Reviewed-by: Thomas Huth
> > Reviewed-by: Nina Schoetterl-Glausch
> > Co-developed-by: Nina Schoetterl-Glausch
> > Signed-off-by: Nina Schoetterl-Glausch
> > Message-ID: <20231016183925.2384704-7-...@linux.ibm.com>
> > Signed-off-by: Thomas Huth
> > ---
> > include/hw/s390x/cpu-topology.h | 1 +
> > target/s390x/cpu.h | 1 +
> > target/s390x/kvm/kvm_s390x.h| 1 +
> > hw/s390x/cpu-topology.c | 11 +++
> > hw/s390x/s390-virtio-ccw.c | 3 +++
> > target/s390x/cpu-sysemu.c | 13 +
> > target/s390x/kvm/kvm.c | 17 +
> > 7 files changed, 47 insertions(+)
> >
> > diff --git a/include/hw/s390x/cpu-topology.h
> > b/include/hw/s390x/cpu-topology.h
> > index f95d26d37c..e33e7c66df 100644
> > --- a/include/hw/s390x/cpu-topology.h
> > +++ b/include/hw/s390x/cpu-topology.h
> > @@ -56,6 +56,7 @@ static inline void s390_topology_setup_cpu(MachineState
> > *ms,
> > #endif
> >
> > extern S390Topology s390_topology;
> > +void s390_topology_reset(void);
>
> Please take a look at the following CI failure:
>
> /usr/bin/ld: libqemu-s390x-softmmu.fa.p/hw_s390x_s390-virtio-ccw.c.o:
> in function `subsystem_reset':
> /home/gitlab-runner/builds/E8PpwMky/0/qemu-project/qemu/build/../hw/s390x/s390-virtio-ccw.c:128:
> undefined reference to `s390_topology_reset'
>
> https://gitlab.com/qemu-project/qemu/-/jobs/5330218593
I can replicate this with --disable-kvm, tho I don't think that's what the CI
does.
Fix looks something like this (copy pasted):
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -45,6 +45,7 @@ typedef QTAILQ_HEAD(, S390TopologyEntry) S390TopologyList;
#ifdef CONFIG_KVM
bool s390_has_topology(void);
void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
+void s390_topology_reset(void);
#else
static inline bool s390_has_topology(void)
{
@@ -53,10 +54,14 @@ static inline bool s390_has_topology(void)
static inline void s390_topology_setup_cpu(MachineState *ms,
S390CPU *cpu,
Error **errp) {}
+static inline void s390_topology_reset(void)
+{
+/* Unreachable, CPU topology not implemented for TCG */
+assert(false);
+}
#endif
extern S390Topology s390_topology;
-void s390_topology_reset(void);
static inline int s390_std_socket(int n, CpuTopology *smp)
{
Re: [PULL 03/25] s390x/cpu topology: add topology entries on CPU hotplug
On Wed, 2023-10-18 at 15:06 +0200, Thomas Huth wrote:
> From: Pierre Morel
>
> The topology information are attributes of the CPU and are
> specified during the CPU device creation.
>
> On hot plug we:
> - calculate the default values for the topology for drawers,
> books and sockets in the case they are not specified.
> - verify the CPU attributes
> - check that we have still room on the desired socket
>
> The possibility to insert a CPU in a mask is dependent on the
> number of cores allowed in a socket, a book or a drawer, the
> checking is done during the hot plug of the CPU to have an
> immediate answer.
>
> If the complete topology is not specified, the core is added
> in the physical topology based on its core ID and it gets
> defaults values for the modifier attributes.
>
> This way, starting QEMU without specifying the topology can
> still get some advantage of the CPU topology.
>
> Signed-off-by: Pierre Morel
> Reviewed-by: Nina Schoetterl-Glausch
> Co-developed-by: Nina Schoetterl-Glausch
> Reviewed-by: Thomas Huth
> Signed-off-by: Nina Schoetterl-Glausch
> Message-ID: <20231016183925.2384704-4-...@linux.ibm.com>
> Signed-off-by: Thomas Huth
> ---
> MAINTAINERS | 6 +
> include/hw/s390x/cpu-topology.h | 54 +++
> hw/s390x/cpu-topology.c | 259
> hw/s390x/s390-virtio-ccw.c | 22 ++-
> hw/s390x/meson.build| 1 +
> 5 files changed, 340 insertions(+), 2 deletions(-)
> create mode 100644 include/hw/s390x/cpu-topology.h
> create mode 100644 hw/s390x/cpu-topology.c
[...]
> --- /dev/null
> +++ b/include/hw/s390x/cpu-topology.h
> @@ -0,0 +1,54 @@
> +/* SPDX-License-Identifier: GPL-2.0-or-later */
> +/*
> + * CPU Topology
> + *
> + * Copyright IBM Corp. 2022, 2023
> + * Author(s): Pierre Morel
> + *
> + */
> +#ifndef HW_S390X_CPU_TOPOLOGY_H
> +#define HW_S390X_CPU_TOPOLOGY_H
> +
> +#ifndef CONFIG_USER_ONLY
> +
> +#include "qemu/queue.h"
> +#include "hw/boards.h"
> +#include "qapi/qapi-types-machine-target.h"
> +
> +typedef struct S390Topology {
> +uint8_t *cores_per_socket;
> +} S390Topology;
> +
> +#ifdef CONFIG_KVM
> +bool s390_has_topology(void);
> +void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
> +#else
> +static inline bool s390_has_topology(void)
> +{
> + return false;
Just noticed that the indent is off here :(
> +}
[...]
Re: [PATCH v26 01/21] qapi: machine.json: change docs regarding CPU topology
On Tue, 2023-10-17 at 13:10 +0200, Thomas Huth wrote: > On 17/10/2023 11.40, Markus Armbruster wrote: > > Nina Schoetterl-Glausch writes: > > > > > Clarify roles of different architectures. > > > Also change things a bit in anticipation of additional members being > > > added. > > > > > > Suggested-by: Markus Armbruster > > > Signed-off-by: Nina Schoetterl-Glausch > > > --- > > > qapi/machine.json | 58 +++ > > > 1 file changed, 38 insertions(+), 20 deletions(-) > ... > > > > Address my nitpicks one way or the other, and you may add > > Acked-by: Markus Armbruster > > > > Thanks Markus! > > Nina, if you agree, I can fix the nitpicks while picking the patches up, so > you don't have to respin again? Thanks for going ahead with it! I was out sick the last couple of days. > > Thomas >
[PATCH v26 10/21] machine: adding s390 topology to query-cpu-fast
From: Pierre Morel
S390x provides two more topology attributes, entitlement and dedication.
Let's add these CPU attributes to the QAPI command query-cpu-fast.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Acked-by: Markus Armbruster
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine.json | 9 -
target/s390x/cpu.c | 9 +
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/qapi/machine.json b/qapi/machine.json
index e73416d086..0915efdee8 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -57,9 +57,16 @@
#
# @cpu-state: the virtual CPU's state
#
+# @dedicated: the virtual CPU's dedication (since 8.2)
+#
+# @entitlement: the virtual CPU's entitlement (since 8.2)
+#
# Since: 2.12
##
-{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
+{ 'struct': 'CpuInfoS390',
+ 'data': { 'cpu-state': 'CpuS390State',
+'*dedicated': 'bool',
+'*entitlement': 'CpuS390Entitlement' } }
##
# @CpuInfoFast:
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index ba646461a1..6acfa1c91b 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -38,6 +38,7 @@
#ifndef CONFIG_USER_ONLY
#include "sysemu/reset.h"
#endif
+#include "hw/s390x/cpu-topology.h"
#define CR0_RESET 0xE0UL
#define CR14_RESET 0xC200UL;
@@ -146,6 +147,14 @@ static void s390_query_cpu_fast(CPUState *cpu, CpuInfoFast
*value)
S390CPU *s390_cpu = S390_CPU(cpu);
value->u.s390x.cpu_state = s390_cpu->env.cpu_state;
+#if !defined(CONFIG_USER_ONLY)
+if (s390_has_topology()) {
+value->u.s390x.has_dedicated = true;
+value->u.s390x.dedicated = s390_cpu->env.dedicated;
+value->u.s390x.has_entitlement = true;
+value->u.s390x.entitlement = s390_cpu->env.entitlement;
+}
+#endif
}
/* S390CPUClass::reset() */
--
2.39.2
[PATCH v26 19/21] tests/avocado: s390x cpu topology test socket full
From: Pierre Morel
This test verifies that QMP set-cpu-topology does not accept
to overload a socket.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
tests/avocado/s390_topology.py | 26 ++
1 file changed, 26 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 3661048f4c..a63c2b2923 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -338,3 +338,29 @@ def test_dedicated(self):
self.guest_set_dispatching('0');
self.check_topology(0, 0, 0, 0, 'high', True)
self.check_polarization("horizontal")
+
+
+def test_socket_full(self):
+"""
+This test verifies that QEMU does not accept to overload a socket.
+The socket-id 0 on book-id 0 already contains CPUs 0 and 1 and can
+not accept any new CPU while socket-id 0 on book-id 1 is free.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.add_args('-smp',
+ '3,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'socket-id': 0, 'book-id': 0})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'socket-id': 0, 'book-id': 1})
+self.assertEqual(res['return'], {})
--
2.39.2
[PATCH v26 20/21] tests/avocado: s390x cpu topology dedicated errors
From: Pierre Morel
Let's test that QEMU refuses to setup a dedicated CPU with
low or medium entitlement.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
tests/avocado/s390_topology.py | 48 ++
1 file changed, 48 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index a63c2b2923..d3e6556c0f 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -364,3 +364,51 @@ def test_socket_full(self):
res = self.vm.qmp('set-cpu-topology',
{'core-id': 2, 'socket-id': 0, 'book-id': 1})
self.assertEqual(res['return'], {})
+
+def test_dedicated_error(self):
+"""
+This test verifies that QEMU refuses to lower the entitlement
+of a dedicated CPU
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+self.assertEqual(res['return'], {})
+
+self.check_topology(0, 0, 0, 0, 'high', True)
+
+self.guest_set_dispatching('1');
+
+self.check_topology(0, 0, 0, 0, 'high', True)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low', 'dedicated':
True})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low'})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium', 'dedicated':
True})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium'})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low', 'dedicated':
False})
+self.assertEqual(res['return'], {})
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium', 'dedicated':
False})
+self.assertEqual(res['return'], {})
--
2.39.2
[PATCH v26 04/21] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
From: Pierre Morel
On interception of STSI(15.1.x) the System Information Block
(SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Acked-by: Markus Armbruster
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS | 1 +
qapi/machine-target.json | 14 ++
include/hw/s390x/cpu-topology.h | 23 +++
include/hw/s390x/sclp.h | 1 +
target/s390x/cpu.h | 75 +++
hw/s390x/cpu-topology.c | 2 +
target/s390x/kvm/kvm.c | 5 +-
target/s390x/kvm/stsi-topology.c | 334 +++
target/s390x/kvm/meson.build | 3 +-
9 files changed, 456 insertions(+), 2 deletions(-)
create mode 100644 target/s390x/kvm/stsi-topology.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 7c68725c44..f3951e0443 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1717,6 +1717,7 @@ M: Nina Schoetterl-Glausch
S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
+F: target/s390x/kvm/stsi-topology.c
X86 Machines
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index f0a6b72414..93cbf1c128 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -361,3 +361,17 @@
'TARGET_MIPS',
'TARGET_LOONGARCH64',
'TARGET_RISCV' ] } }
+
+##
+# @CpuS390Polarization:
+#
+# An enumeration of CPU polarization that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.2
+##
+{ 'enum': 'CpuS390Polarization',
+ 'prefix': 'S390_CPU_POLARIZATION',
+ 'data': [ 'horizontal', 'vertical' ],
+ 'if': 'TARGET_S390X'
+}
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 97b0af2795..f95d26d37c 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -15,10 +15,33 @@
#include "hw/boards.h"
#include "qapi/qapi-types-machine-target.h"
+#define S390_TOPOLOGY_CPU_IFL 0x03
+
+typedef struct S390TopologyId {
+uint8_t sentinel;
+uint8_t drawer;
+uint8_t book;
+uint8_t socket;
+uint8_t type;
+uint8_t vertical:1;
+uint8_t entitlement:2;
+uint8_t dedicated;
+uint8_t origin;
+} S390TopologyId;
+
+typedef struct S390TopologyEntry {
+QTAILQ_ENTRY(S390TopologyEntry) next;
+S390TopologyId id;
+uint64_t mask;
+} S390TopologyEntry;
+
typedef struct S390Topology {
uint8_t *cores_per_socket;
+CpuS390Polarization polarization;
} S390Topology;
+typedef QTAILQ_HEAD(, S390TopologyEntry) S390TopologyList;
+
#ifdef CONFIG_KVM
bool s390_has_topology(void);
void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index cf1f2efae2..c49051e17e 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,6 +112,7 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
+#define SCLP_READ_SCP_INFO_MNEST2
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index 56f9340914..09bff39fe4 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -570,6 +570,29 @@ typedef struct SysIB_322 {
} SysIB_322;
QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096);
+/*
+ * Topology Magnitude fields (MAG) indicates the maximum number of
+ * topology list entries (TLE) at the corresponding nesting level.
+ */
+#define S390_TOPOLOGY_MAG 6
+#define S390_TOPOLOGY_MAG6 0
+#define S390_TOPOLOGY_MAG5 1
+#define S390_TOPOLOGY_MAG4 2
+#define S390_TOPOLOGY_MAG3 3
+#define S390_TOPOLOGY_MAG2 4
+#define S390_TOPOLOGY_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+uint8_t reserved0[2];
+uint16_t length;
+uint8_t mag[S390_TOPOLOGY_MAG];
+uint8_t reserved1;
+uint8_t mnest;
+uint32_t reserved2;
+char tle[];
+} SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
typedef union SysIB {
SysIB_111 sysib_111;
SysIB_121 sysib_121;
@@ -577,9 +600,61 @@ typedef union SysIB {
SysIB_221 sysib_221;
SysIB_222 sysib_222;
SysIB_322 sysib_322;
+SysIB_151x sysib_151x;
} SysIB;
QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ * Defines a container to contain other Topology List Entries
+ * of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ * Specifies the CPUs position, type, entitlement and polarization
+ * of the CPUs contained in the last container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only three levels, the drawer's, book'
[PATCH v26 08/21] target/s390x/cpu topology: activate CPU topology
From: Pierre Morel
The KVM capability KVM_CAP_S390_CPU_TOPOLOGY is used to
activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and
the topology facility in the host CPU model for the guest
in the case the topology is available in QEMU and in KVM.
The feature is disabled by default and fenced for SE
(secure execution).
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
hw/s390x/cpu-topology.c | 2 +-
target/s390x/cpu_models.c | 1 +
target/s390x/kvm/kvm.c| 9 +
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index a00a4ce4df..4e4c35f6da 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -69,7 +69,7 @@ static int s390_socket_nb(S390CPU *cpu)
*/
bool s390_has_topology(void)
{
-return false;
+return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY);
}
/**
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index 98f14c09c2..4dead48650 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -255,6 +255,7 @@ bool s390_has_feat(S390Feat feat)
case S390_FEAT_SIE_CMMA:
case S390_FEAT_SIE_PFMFI:
case S390_FEAT_SIE_IBS:
+case S390_FEAT_CONFIGURATION_TOPOLOGY:
return false;
break;
default:
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 4d84dcc018..2e43dc746f 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -366,6 +366,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_SIGP, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_VECTOR_REGISTERS, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_STSI, 0);
+kvm_vm_enable_cap(s, KVM_CAP_S390_CPU_TOPOLOGY, 0);
if (ri_allowed()) {
if (kvm_vm_enable_cap(s, KVM_CAP_S390_RI, 0) == 0) {
cap_ri = 1;
@@ -2509,6 +2510,14 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model,
Error **errp)
set_bit(S390_FEAT_UNPACK, model->features);
}
+/*
+ * If we have kernel support for CPU Topology indicate the
+ * configuration-topology facility.
+ */
+if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) {
+set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features);
+}
+
/* We emulate a zPCI bus and AEN, therefore we don't need HW support */
set_bit(S390_FEAT_ZPCI, model->features);
set_bit(S390_FEAT_ADAPTER_EVENT_NOTIFICATION, model->features);
--
2.39.2
[PATCH v26 15/21] tests/avocado: s390x cpu topology core
From: Pierre Morel
Introduction of the s390x cpu topology core functions and
basic tests.
We test the correlation between the command line and
the QMP results in query-cpus-fast for various CPU topology.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Tested-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS| 1 +
tests/avocado/s390_topology.py | 200 +
2 files changed, 201 insertions(+)
create mode 100644 tests/avocado/s390_topology.py
diff --git a/MAINTAINERS b/MAINTAINERS
index e236dd181d..e21a8c8097 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1720,6 +1720,7 @@ F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/stsi-topology.c
F: docs/devel/s390-cpu-topology.rst
F: docs/system/s390x/cpu-topology.rst
+F: tests/avocado/s390_topology.py
X86 Machines
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
new file mode 100644
index 00..9078b45281
--- /dev/null
+++ b/tests/avocado/s390_topology.py
@@ -0,0 +1,200 @@
+# Functional test that boots a Linux kernel and checks the console
+#
+# Copyright IBM Corp. 2023
+#
+# Author:
+# Pierre Morel
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+import shutil
+import time
+
+from avocado_qemu import QemuSystemTest
+from avocado_qemu import exec_command
+from avocado_qemu import exec_command_and_wait_for_pattern
+from avocado_qemu import interrupt_interactive_console_until_pattern
+from avocado_qemu import wait_for_console_pattern
+from avocado.utils import process
+from avocado.utils import archive
+
+
+class S390CPUTopology(QemuSystemTest):
+"""
+S390x CPU topology consists of 4 topology layers, from bottom to top,
+the cores, sockets, books and drawers and 2 modifiers attributes,
+the entitlement and the dedication.
+See: docs/system/s390x/cpu-topology.rst.
+
+S390x CPU topology is setup in different ways:
+- implicitly from the '-smp' argument by completing each topology
+ level one after the other beginning with drawer 0, book 0 and
+ socket 0.
+- explicitly from the '-device' argument on the QEMU command line
+- explicitly by hotplug of a new CPU using QMP or HMP
+- it is modified by using QMP 'set-cpu-topology'
+
+The S390x modifier attribute entitlement depends on the machine
+polarization, which can be horizontal or vertical.
+The polarization is changed on a request from the guest.
+"""
+timeout = 90
+
+KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
+ 'root=/dev/ram '
+ 'selinux=0 '
+ 'rdinit=/bin/sh')
+
+def wait_until_booted(self):
+wait_for_console_pattern(self, 'no job control',
+ failure_message='Kernel panic - not syncing',
+ vm=None)
+
+def check_topology(self, c, s, b, d, e, t):
+res = self.vm.qmp('query-cpus-fast')
+cpus = res['return']
+for cpu in cpus:
+core = cpu['props']['core-id']
+socket = cpu['props']['socket-id']
+book = cpu['props']['book-id']
+drawer = cpu['props']['drawer-id']
+entitlement = cpu.get('entitlement')
+dedicated = cpu.get('dedicated')
+if core == c:
+self.assertEqual(drawer, d)
+self.assertEqual(book, b)
+self.assertEqual(socket, s)
+self.assertEqual(entitlement, e)
+self.assertEqual(dedicated, t)
+
+def kernel_init(self):
+"""
+We need a VM that supports CPU topology,
+currently this only the case when using KVM, not TCG.
+We need a kernel supporting the CPU topology.
+We need a minimal root filesystem with a shell.
+"""
+self.require_accelerator("kvm")
+kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/kernel.img')
+kernel_hash = '0d1aaaf303f07cf0160c8c48e56fe638'
+kernel_path = self.fetch_asset(kernel_url, algorithm='md5',
+ asset_hash=kernel_hash)
+
+initrd_url = ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/initrd.img')
+initrd_hash = 'a122057d95725ac030e2ec51df46e172'
+initrd_path_xz = self.fetch_asset(initrd_url, algorithm='md5',
+ asset_hash=initrd_hash)
+initrd_path = os.path.join(self.workdir, 'initrd-
[PATCH v26 12/21] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE QAPI event
From: Pierre Morel
When the guest asks to change the polarization this change
is forwarded to the upper layer using QAPI.
The upper layer is supposed to take according decisions concerning
CPU provisioning.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Acked-by: Markus Armbruster
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-target.json | 35 +++
hw/s390x/cpu-topology.c | 2 ++
2 files changed, 37 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 7688f32ffa..ac93a5f82c 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -417,3 +417,38 @@
'features': [ 'unstable' ],
'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
}
+
+##
+# @CPU_POLARIZATION_CHANGE:
+#
+# Emitted when the guest asks to change the polarization.
+#
+# The guest can tell the host (via the PTF instruction) whether the
+# CPUs should be provisioned using horizontal or vertical polarization.
+#
+# On horizontal polarization the host is expected to provision all vCPUs
+# equally.
+#
+# On vertical polarization the host can provision each vCPU differently.
+# The guest will get information on the details of the provisioning
+# the next time it uses the STSI(15) instruction.
+#
+# @polarization: polarization specified by the guest
+#
+# Features:
+#
+# @unstable: This event is experimental.
+#
+# Since: 8.2
+#
+# Example:
+#
+# <- { "event": "CPU_POLARIZATION_CHANGE",
+# "data": { "polarization": "horizontal" },
+# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
+##
+{ 'event': 'CPU_POLARIZATION_CHANGE',
+ 'data': { 'polarization': 'CpuS390Polarization' },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index f3771f5045..327bccea4f 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -24,6 +24,7 @@
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
#include "qapi/qapi-commands-machine-target.h"
+#include "qapi/qapi-events-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -136,6 +137,7 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
} else {
s390_topology.polarization = polarization;
s390_cpu_topology_set_changed(true);
+qapi_event_send_cpu_polarization_change(polarization);
setcc(cpu, 0);
}
break;
--
2.39.2
[PATCH v26 03/21] s390x/cpu topology: add topology entries on CPU hotplug
From: Pierre Morel
The topology information are attributes of the CPU and are
specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers,
books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socket
The possibility to insert a CPU in a mask is dependent on the
number of cores allowed in a socket, a book or a drawer, the
checking is done during the hot plug of the CPU to have an
immediate answer.
If the complete topology is not specified, the core is added
in the physical topology based on its core ID and it gets
defaults values for the modifier attributes.
This way, starting QEMU without specifying the topology can
still get some advantage of the CPU topology.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS | 6 +
include/hw/s390x/cpu-topology.h | 54 +++
hw/s390x/cpu-topology.c | 259
hw/s390x/s390-virtio-ccw.c | 22 ++-
hw/s390x/meson.build| 1 +
5 files changed, 340 insertions(+), 2 deletions(-)
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 hw/s390x/cpu-topology.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 0fd1721190..7c68725c44 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1712,6 +1712,12 @@ F: hw/s390x/event-facility.c
F: hw/s390x/sclp*.c
L: qemu-s3...@nongnu.org
+S390 CPU topology
+M: Nina Schoetterl-Glausch
+S: Supported
+F: include/hw/s390x/cpu-topology.h
+F: hw/s390x/cpu-topology.c
+
X86 Machines
PC
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
new file mode 100644
index 00..97b0af2795
--- /dev/null
+++ b/include/hw/s390x/cpu-topology.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel
+ *
+ */
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#ifndef CONFIG_USER_ONLY
+
+#include "qemu/queue.h"
+#include "hw/boards.h"
+#include "qapi/qapi-types-machine-target.h"
+
+typedef struct S390Topology {
+uint8_t *cores_per_socket;
+} S390Topology;
+
+#ifdef CONFIG_KVM
+bool s390_has_topology(void);
+void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
+#else
+static inline bool s390_has_topology(void)
+{
+ return false;
+}
+static inline void s390_topology_setup_cpu(MachineState *ms,
+ S390CPU *cpu,
+ Error **errp) {}
+#endif
+
+extern S390Topology s390_topology;
+
+static inline int s390_std_socket(int n, CpuTopology *smp)
+{
+return (n / smp->cores) % smp->sockets;
+}
+
+static inline int s390_std_book(int n, CpuTopology *smp)
+{
+return (n / (smp->cores * smp->sockets)) % smp->books;
+}
+
+static inline int s390_std_drawer(int n, CpuTopology *smp)
+{
+return (n / (smp->cores * smp->sockets * smp->books)) % smp->drawers;
+}
+
+#endif /* CONFIG_USER_ONLY */
+
+#endif
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
new file mode 100644
index 00..cabd42e779
--- /dev/null
+++ b/hw/s390x/cpu-topology.c
@@ -0,0 +1,259 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel
+ *
+ * S390 topology handling can be divided in two parts:
+ *
+ * - The first part in this file is taking care of all common functions
+ * used by KVM and TCG to create and modify the topology.
+ *
+ * - The second part, building the topology information data for the
+ * guest with CPU and KVM specificity will be implemented inside
+ * the target/s390/kvm sub tree.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/qdev-properties.h"
+#include "hw/boards.h"
+#include "target/s390x/cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
+#include "hw/s390x/cpu-topology.h"
+
+/*
+ * s390_topology is used to keep the topology information.
+ * .cores_per_socket: tracks information on the count of cores
+ *per socket.
+ */
+S390Topology s390_topology = {
+/* will be initialized after the CPU model is realized */
+.cores_per_socket = NULL,
+};
+
+/**
+ * s390_socket_nb:
+ * @cpu: s390x CPU
+ *
+ * Returns the socket number used inside the cores_per_socket array
+ * for a topology tree entry
+ */
+static int s390_socket_nb_from_ids(int drawer_id, int book_id, int socket_id)
+{
+return (drawer_id * current_machine->smp.books + book_id) *
+ current_machine->smp.sockets + socket_id;
+}
+
+/**
+
[PATCH v26 13/21] qapi/s390x/cpu topology: add query-s390x-cpu-polarization command
From: Pierre Morel
The query-s390x-cpu-polarization qmp command returns the current
CPU polarization of the machine.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Acked-by: Markus Armbruster
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-target.json | 30 ++
hw/s390x/cpu-topology.c | 8
2 files changed, 38 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index ac93a5f82c..4e55adbe00 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -452,3 +452,33 @@
'features': [ 'unstable' ],
'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
}
+
+##
+# @CpuPolarizationInfo:
+#
+# The result of a CPU polarization query.
+#
+# @polarization: the CPU polarization
+#
+# Since: 8.2
+##
+{ 'struct': 'CpuPolarizationInfo',
+ 'data': { 'polarization': 'CpuS390Polarization' },
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
+
+##
+# @query-s390x-cpu-polarization:
+#
+# Features:
+#
+# @unstable: This command is experimental.
+#
+# Returns: the machine's CPU polarization
+#
+# Since: 8.2
+##
+{ 'command': 'query-s390x-cpu-polarization', 'returns': 'CpuPolarizationInfo',
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 327bccea4f..f16bdf65fa 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -459,3 +459,11 @@ void qmp_set_cpu_topology(uint16_t core,
has_drawer, drawer, has_entitlement, entitlement,
has_dedicated, dedicated, errp);
}
+
+CpuPolarizationInfo *qmp_query_s390x_cpu_polarization(Error **errp)
+{
+CpuPolarizationInfo *info = g_new0(CpuPolarizationInfo, 1);
+
+info->polarization = s390_topology.polarization;
+return info;
+}
--
2.39.2
[PATCH v26 01/21] qapi: machine.json: change docs regarding CPU topology
Clarify roles of different architectures.
Also change things a bit in anticipation of additional members being
added.
Suggested-by: Markus Armbruster
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine.json | 58 +++
1 file changed, 38 insertions(+), 20 deletions(-)
diff --git a/qapi/machine.json b/qapi/machine.json
index a08b6576ca..058e884fd2 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -71,8 +71,8 @@
#
# @thread-id: ID of the underlying host thread
#
-# @props: properties describing to which node/socket/core/thread
-# virtual CPU belongs to, provided if supported by board
+# @props: properties of type CpuInstanceProperties associated with a
+# virtual CPU, e.g. the socket id
#
# @target: the QEMU system emulation target, which determines which
# additional fields will be listed (since 3.0)
@@ -899,28 +899,34 @@
# should be passed by management with device_add command when a CPU is
# being hotplugged.
#
+# Which members are optional and which mandatory depends on the
+# architecture and board.
+#
+# The ids other than the node-id specify the position of the CPU
+# within the CPU topology as defined by @SMPConfiguration.
+#
# @node-id: NUMA node ID the CPU belongs to
#
-# @socket-id: socket number within node/board the CPU belongs to
+# @socket-id: socket number within CPU topology the CPU belongs to
#
-# @die-id: die number within socket the CPU belongs to (since 4.1)
+# @die-id: die number within the parent container the CPU belongs to
+#(since 4.1)
#
-# @cluster-id: cluster number within die the CPU belongs to (since
-# 7.1)
+# @cluster-id: cluster number within the parent container the CPU
+# belongs to (since 7.1)
#
-# @core-id: core number within cluster the CPU belongs to
+# @core-id: core number within the parent container the CPU
+# belongs to
#
-# @thread-id: thread number within core the CPU belongs to
+# @thread-id: thread number within the core the CPU belongs to
#
-# Note: currently there are 6 properties that could be present but
-# management should be prepared to pass through other properties
-# with device_add command to allow for future interface extension.
-# This also requires the filed names to be kept in sync with the
-# properties passed to -device/device_add.
+# Note: management should be prepared to pass through additional
+# properties with device_add.
#
# Since: 2.7
##
{ 'struct': 'CpuInstanceProperties',
+ # Keep these in sync with the properties device_add accepts
'data': { '*node-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
@@ -1478,21 +1484,33 @@
# Schema for CPU topology configuration. A missing value lets QEMU
# figure out a suitable value based on the ones that are provided.
#
+# The members other than @cpus and @maxcpus define topology
+# containers.
+#
+# The ordering from highest/coarsest to lowest/finest is:
+# @sockets, @dies, @clusters, @cores, @threads.
+#
+# Different architectures support different subsets of topology
+# containers.
+#
+# For examples, s390x does not have clusters and dies, the socket
+# is the parent container of cores.
+#
# @cpus: number of virtual CPUs in the virtual machine
#
+# @maxcpus: maximum number of hotpluggable virtual CPUs in the virtual
+# machine
+#
# @sockets: number of sockets in the CPU topology
#
-# @dies: number of dies per socket in the CPU topology
+# @dies: number of dies per parent container
#
-# @clusters: number of clusters per die in the CPU topology (since
+# @clusters: number of clusters per parent container (since
# 7.0)
#
-# @cores: number of cores per cluster in the CPU topology
+# @cores: number of cores per parent container
#
-# @threads: number of threads per core in the CPU topology
-#
-# @maxcpus: maximum number of hotpluggable virtual CPUs in the virtual
-# machine
+# @threads: number of threads per core
#
# Since: 6.1
##
--
2.39.2
[PATCH v26 00/21] s390x: CPU Topology
g new CPU creation (Thomas) - Improving codding style and argument usages (Thomas) - since v6 - Changes on smp args in qemu-options (Daniel) - changed comments in machine.jason (Daniel) - Added reset (Janosch) - since v5 - rebasing on newer QEMU version - reworked most lines above 80 characters. - since v4 - Added drawer and books to topology - Added numa topology - Added documentation - since v3 - Added migration (Thomas) - Separated STSI instruction from KVM to prepare TCG (Thomas) - Take care of endianess to prepare TCG (Thomas) - Added comments on STSI CPU container and PFT instruction (Thomas) - Moved enabling the instructions as the last patch (Thomas) Nina Schoetterl-Glausch (1): qapi: machine.json: change docs regarding CPU topology Pierre Morel (20): CPU topology: extend with s390 specifics s390x/cpu topology: add topology entries on CPU hotplug target/s390x/cpu topology: handle STSI(15) and build the SYSIB s390x/sclp: reporting the maximum nested topology entries s390x/cpu topology: resetting the Topology-Change-Report s390x/cpu topology: interception of PTF instruction target/s390x/cpu topology: activate CPU topology qapi/s390x/cpu topology: set-cpu-topology qmp command machine: adding s390 topology to query-cpu-fast machine: adding s390 topology to info hotpluggable-cpus qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE QAPI event qapi/s390x/cpu topology: add query-s390x-cpu-polarization command docs/s390x/cpu topology: document s390x cpu topology tests/avocado: s390x cpu topology core tests/avocado: s390x cpu topology polarization tests/avocado: s390x cpu topology entitlement tests tests/avocado: s390x cpu topology test dedicated CPU tests/avocado: s390x cpu topology test socket full tests/avocado: s390x cpu topology dedicated errors tests/avocado: s390x cpu topology bad move MAINTAINERS | 11 + docs/devel/index-internals.rst | 1 + docs/devel/s390-cpu-topology.rst| 170 ++ docs/system/s390x/cpu-topology.rst | 244 +++ docs/system/target-s390x.rst| 1 + qapi/machine-common.json| 21 ++ qapi/machine-target.json| 121 +++ qapi/machine.json | 86 +++-- qapi/qapi-schema.json | 1 + include/hw/boards.h | 10 +- include/hw/qdev-properties-system.h | 4 + include/hw/s390x/cpu-topology.h | 78 + include/hw/s390x/s390-virtio-ccw.h | 6 + include/hw/s390x/sclp.h | 4 +- target/s390x/cpu.h | 82 + target/s390x/kvm/kvm_s390x.h| 1 + hw/core/machine-hmp-cmds.c | 6 + hw/core/machine-smp.c | 48 ++- hw/core/machine.c | 4 + hw/core/qdev-properties-system.c| 13 + hw/s390x/cpu-topology.c | 469 hw/s390x/s390-virtio-ccw.c | 29 +- hw/s390x/sclp.c | 5 + system/vl.c | 6 + target/s390x/cpu-sysemu.c | 13 + target/s390x/cpu.c | 16 + target/s390x/cpu_models.c | 1 + target/s390x/kvm/kvm.c | 42 ++- target/s390x/kvm/stsi-topology.c| 334 hw/s390x/meson.build| 1 + qapi/meson.build| 1 + qemu-options.hx | 7 +- target/s390x/kvm/meson.build| 3 +- tests/avocado/s390_topology.py | 439 ++ 34 files changed, 2241 insertions(+), 37 deletions(-) create mode 100644 docs/devel/s390-cpu-topology.rst create mode 100644 docs/system/s390x/cpu-topology.rst create mode 100644 qapi/machine-common.json create mode 100644 include/hw/s390x/cpu-topology.h create mode 100644 hw/s390x/cpu-topology.c create mode 100644 target/s390x/kvm/stsi-topology.c create mode 100644 tests/avocado/s390_topology.py Range-diff against v25: 1: 578a12c6e9 ! 1: cd428e36e1 qapi: machine.json: change docs regarding CpuInstanceProperties @@ Metadata Author: Nina Schoetterl-Glausch ## Commit message ## -qapi: machine.json: change docs regarding CpuInstanceProperties +qapi: machine.json: change docs regarding CPU topology Clarify roles of different architectures. Also change things a bit in anticipation of additional members being @@ qapi/machine.json # should be passed by management with device_add command when a CPU is # being hotplugged. # -+# Which members are optional and which mandatory depends on the architecture -+# and board. ++# Which members are optional and which mandatory depends on the ++# architecture and board. ++# ++# The ids other than the node-id specify the position of the CPU ++# within the CPU topology as defined by @SMPConfiguration. +# # @node-id: NUMA node ID the CPU belongs to
[PATCH v26 06/21] s390x/cpu topology: resetting the Topology-Change-Report
From: Pierre Morel
During a subsystem reset the Topology-Change-Report is cleared
by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
include/hw/s390x/cpu-topology.h | 1 +
target/s390x/cpu.h | 1 +
target/s390x/kvm/kvm_s390x.h| 1 +
hw/s390x/cpu-topology.c | 11 +++
hw/s390x/s390-virtio-ccw.c | 3 +++
target/s390x/cpu-sysemu.c | 13 +
target/s390x/kvm/kvm.c | 17 +
7 files changed, 47 insertions(+)
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index f95d26d37c..e33e7c66df 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -56,6 +56,7 @@ static inline void s390_topology_setup_cpu(MachineState *ms,
#endif
extern S390Topology s390_topology;
+void s390_topology_reset(void);
static inline int s390_std_socket(int n, CpuTopology *smp)
{
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index 09bff39fe4..40c5cedd0e 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -654,6 +654,7 @@ typedef struct SysIBCPUListEntry {
QEMU_BUILD_BUG_ON(sizeof(SysIBCPUListEntry) != 16);
void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar,
uintptr_t ra);
+void s390_cpu_topology_set_changed(bool changed);
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin
*/
diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
index f9785564d0..649dae5948 100644
--- a/target/s390x/kvm/kvm_s390x.h
+++ b/target/s390x/kvm/kvm_s390x.h
@@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
void kvm_s390_restart_interrupt(S390CPU *cpu);
void kvm_s390_stop_interrupt(S390CPU *cpu);
void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
+int kvm_s390_topology_set_mtcr(uint64_t attr);
#endif /* KVM_S390X_H */
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 13168341b6..7ec9319272 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -90,6 +90,17 @@ static void s390_topology_init(MachineState *ms)
smp->books * smp->drawers);
}
+/**
+ * s390_topology_reset:
+ *
+ * Generic reset for CPU topology, calls s390_topology_reset()
+ * to reset the kernel Modified Topology Change Record.
+ */
+void s390_topology_reset(void)
+{
+s390_cpu_topology_set_changed(false);
+}
+
/**
* s390_topology_cpu_default:
* @cpu: pointer to a S390CPU
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 7fe2bce20c..6012165d41 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -124,6 +124,9 @@ static void subsystem_reset(void)
device_cold_reset(dev);
}
}
+if (s390_has_topology()) {
+s390_topology_reset();
+}
}
static int virtio_ccw_hcall_notify(const uint64_t *args)
diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c
index 8112561e5e..1cd30c1d84 100644
--- a/target/s390x/cpu-sysemu.c
+++ b/target/s390x/cpu-sysemu.c
@@ -307,3 +307,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data
arg)
kvm_s390_set_diag318(cs, arg.host_ulong);
}
}
+
+void s390_cpu_topology_set_changed(bool changed)
+{
+int ret;
+
+if (kvm_enabled()) {
+ret = kvm_s390_topology_set_mtcr(changed);
+if (ret) {
+error_report("Failed to set Modified Topology Change Report: %s",
+ strerror(-ret));
+}
+}
+}
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 53d6300809..d6bda3a2a8 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -2664,6 +2664,23 @@ int kvm_s390_get_zpci_op(void)
return cap_zpci_op;
}
+int kvm_s390_topology_set_mtcr(uint64_t attr)
+{
+struct kvm_device_attr attribute = {
+.group = KVM_S390_VM_CPU_TOPOLOGY,
+.attr = attr,
+};
+
+if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+return 0;
+}
+if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) {
+return -ENOTSUP;
+}
+
+return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, );
+}
+
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
--
2.39.2
[PATCH v26 14/21] docs/s390x/cpu topology: document s390x cpu topology
From: Pierre Morel
Add some basic examples for the definition of cpu topology
in s390x.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS| 2 +
docs/devel/index-internals.rst | 1 +
docs/devel/s390-cpu-topology.rst | 170
docs/system/s390x/cpu-topology.rst | 244 +
docs/system/target-s390x.rst | 1 +
qapi/machine.json | 2 +
6 files changed, 420 insertions(+)
create mode 100644 docs/devel/s390-cpu-topology.rst
create mode 100644 docs/system/s390x/cpu-topology.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index f3951e0443..e236dd181d 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1718,6 +1718,8 @@ S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/stsi-topology.c
+F: docs/devel/s390-cpu-topology.rst
+F: docs/system/s390x/cpu-topology.rst
X86 Machines
diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst
index e1a93df263..6f81df92bc 100644
--- a/docs/devel/index-internals.rst
+++ b/docs/devel/index-internals.rst
@@ -14,6 +14,7 @@ Details about QEMU's various subsystems including how to add
features to them.
migration
multi-process
reset
+ s390-cpu-topology
s390-dasd-ipl
tracing
vfio-migration
diff --git a/docs/devel/s390-cpu-topology.rst b/docs/devel/s390-cpu-topology.rst
new file mode 100644
index 00..9eab28d5e5
--- /dev/null
+++ b/docs/devel/s390-cpu-topology.rst
@@ -0,0 +1,170 @@
+QAPI interface for S390 CPU topology
+
+
+The following sections will explain the QAPI interface for S390 CPU topology
+with the help of exemplary output.
+For this, let's assume that QEMU has been started with the following
+command, defining 4 CPUs, where CPU[0] is defined by the -smp argument and will
+have default values:
+
+.. code-block:: bash
+
+ qemu-system-s390x \
+-enable-kvm \
+-cpu z14,ctop=on \
+-smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
+-device z14-s390x-cpu,core-id=19,entitlement=high \
+-device z14-s390x-cpu,core-id=11,entitlement=low \
+-device z14-s390x-cpu,core-id=112,entitlement=high \
+ ...
+
+Additions to query-cpus-fast
+
+
+The command query-cpus-fast allows querying the topology tree and
+modifiers for all configured vCPUs.
+
+.. code-block:: QMP
+
+ { "execute": "query-cpus-fast" }
+ {
+ "return": [
+{
+ "dedicated": false,
+ "thread-id": 536993,
+ "props": {
+"core-id": 0,
+"socket-id": 0,
+"drawer-id": 0,
+"book-id": 0
+ },
+ "cpu-state": "operating",
+ "entitlement": "medium",
+ "qom-path": "/machine/unattached/device[0]",
+ "cpu-index": 0,
+ "target": "s390x"
+},
+{
+ "dedicated": false,
+ "thread-id": 537003,
+ "props": {
+"core-id": 19,
+"socket-id": 1,
+"drawer-id": 0,
+"book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[0]",
+ "cpu-index": 19,
+ "target": "s390x"
+},
+{
+ "dedicated": false,
+ "thread-id": 537004,
+ "props": {
+"core-id": 11,
+"socket-id": 1,
+"drawer-id": 0,
+"book-id": 1
+ },
+ "cpu-state": "operating",
+ "entitlement": "low",
+ "qom-path": "/machine/peripheral-anon/device[1]",
+ "cpu-index": 11,
+ "target": "s390x"
+},
+{
+ "dedicated": true,
+ "thread-id": 537005,
+ "props": {
+"core-id": 112,
+"socket-id": 0,
+"drawer-id": 3,
+"book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[2]",
+ "cpu-index": 112,
+ "target": "s390x"
+}
+ ]
+ }
+
+
+QAPI command: set-cpu-topology
+--
+
+The command set-cpu-topology allows modifying the topology tree
+or the topology modifiers of a vCPU in the configuration.
+
+.. code-block:: QMP
+
+{ "execute": "set-cpu-topology",
+ &
[PATCH v26 18/21] tests/avocado: s390x cpu topology test dedicated CPU
From: Pierre Morel
A dedicated CPU in vertical polarization can only have
a high entitlement.
Let's check this from both host and guest point of view.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
tests/avocado/s390_topology.py | 33 +
1 file changed, 33 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 24fac9a54d..3661048f4c 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -305,3 +305,36 @@ def test_entitlement(self):
self.guest_set_dispatching('0');
self.check_polarization("horizontal")
self.check_topology(0, 0, 0, 0, 'high', False)
+
+
+def test_dedicated(self):
+"""
+This test verifies that QEMU adjusts the entitlement correctly when a
+CPU is made dedicated.
+QEMU retains the entitlement value when horizontal polarization is in
effect.
+For the guest, the field shows the effective value of the entitlement.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+self.check_polarization("horizontal")
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+self.assertEqual(res['return'], {})
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("horizontal")
+
+self.guest_set_dispatching('1');
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("vertical:high")
+
+self.guest_set_dispatching('0');
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("horizontal")
--
2.39.2
[PATCH v26 09/21] qapi/s390x/cpu topology: set-cpu-topology qmp command
From: Pierre Morel
The modification of the CPU attributes are done through a monitor
command.
It allows to move the core inside the topology tree to optimize
the cache usage in the case the host's hypervisor previously
moved the CPU.
The same command allows to modify the CPU attributes modifiers
like polarization entitlement and the dedicated attribute to notify
the guest if the host admin modified scheduling or dedication of a vCPU.
With this knowledge the guest has the possibility to optimize the
usage of the vCPUs.
The command has a feature unstable for the moment.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-target.json | 42 +
hw/s390x/cpu-topology.c | 132 +++
2 files changed, 174 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 93cbf1c128..7688f32ffa 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -4,6 +4,8 @@
# This work is licensed under the terms of the GNU GPL, version 2 or later.
# See the COPYING file in the top-level directory.
+{ 'include': 'machine-common.json' }
+
##
# @CpuModelInfo:
#
@@ -375,3 +377,43 @@
'data': [ 'horizontal', 'vertical' ],
'if': 'TARGET_S390X'
}
+
+##
+# @set-cpu-topology:
+#
+# Modify the topology by moving the CPU inside the topology tree,
+# or by changing a modifier attribute of a CPU.
+# Absent values will not be modified.
+#
+# @core-id: the vCPU ID to be moved
+#
+# @socket-id: destination socket to move the vCPU to
+#
+# @book-id: destination book to move the vCPU to
+#
+# @drawer-id: destination drawer to move the vCPU to
+#
+# @entitlement: entitlement to set
+#
+# @dedicated: whether the provisioning of real to virtual CPU is dedicated
+#
+# Features:
+#
+# @unstable: This command is experimental.
+#
+# Returns: Nothing on success.
+#
+# Since: 8.2
+##
+{ 'command': 'set-cpu-topology',
+ 'data': {
+ 'core-id': 'uint16',
+ '*socket-id': 'uint16',
+ '*book-id': 'uint16',
+ '*drawer-id': 'uint16',
+ '*entitlement': 'CpuS390Entitlement',
+ '*dedicated': 'bool'
+ },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 4e4c35f6da..f3771f5045 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -23,6 +23,7 @@
#include "target/s390x/cpu.h"
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
+#include "qapi/qapi-commands-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -257,6 +258,29 @@ static bool s390_topology_check(uint16_t socket_id,
uint16_t book_id,
return true;
}
+/**
+ * s390_topology_need_report
+ * @cpu: Current cpu
+ * @drawer_id: future drawer ID
+ * @book_id: future book ID
+ * @socket_id: future socket ID
+ * @entitlement: future entitlement
+ * @dedicated: future dedicated
+ *
+ * A modified topology change report is needed if the topology
+ * tree or the topology attributes change.
+ */
+static bool s390_topology_need_report(S390CPU *cpu, int drawer_id,
+ int book_id, int socket_id,
+ uint16_t entitlement, bool dedicated)
+{
+return cpu->env.drawer_id != drawer_id ||
+ cpu->env.book_id != book_id ||
+ cpu->env.socket_id != socket_id ||
+ cpu->env.entitlement != entitlement ||
+ cpu->env.dedicated != dedicated;
+}
+
/**
* s390_update_cpu_props:
* @ms: the machine state
@@ -325,3 +349,111 @@ void s390_topology_setup_cpu(MachineState *ms, S390CPU
*cpu, Error **errp)
/* topology tree is reflected in props */
s390_update_cpu_props(ms, cpu);
}
+
+static void s390_change_topology(uint16_t core_id,
+ bool has_socket_id, uint16_t socket_id,
+ bool has_book_id, uint16_t book_id,
+ bool has_drawer_id, uint16_t drawer_id,
+ bool has_entitlement,
+ CpuS390Entitlement entitlement,
+ bool has_dedicated, bool dedicated,
+ Error **errp)
+{
+MachineState *ms = current_machine;
+int old_socket_entry;
+int new_socket_entry;
+bool report_needed;
+S390CPU *cpu;
+
+cpu = s390_cpu_addr2state(core_id);
+if (!cpu) {
+error_setg(errp, "Core-id %d does not exist!", core_id);
+return;
+}
+
+/* Get attributes not provided from cpu and verify the new topology */
+if (!has_socket_id) {
+socket_id = cpu->env.socket_id;
+}
+if (!has_book_id) {
+book_id = cpu->env.book_id;
+
[PATCH v26 17/21] tests/avocado: s390x cpu topology entitlement tests
From: Pierre Morel
Test changes in the entitlement from both a guest and a host point of
view, depending on the polarization.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Tested-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
tests/avocado/s390_topology.py | 62 ++
1 file changed, 62 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 8166cee134..24fac9a54d 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -243,3 +243,65 @@ def test_polarization(self):
res = self.vm.qmp('query-s390x-cpu-polarization')
self.assertEqual(res['return']['polarization'], 'horizontal')
self.check_topology(0, 0, 0, 0, 'medium', False)
+
+
+def check_polarization(self, polarization):
+#We need to wait for the change to have been propagated to the kernel
+exec_command_and_wait_for_pattern(self,
+"\n".join([
+"timeout 1 sh -c 'while true",
+'do',
+'syspath="/sys/devices/system/cpu/cpu0/polarization"',
+'polarization="$(cat "$syspath")" || exit',
+ f'if [ "$polarization" = "{polarization}" ]; then',
+'exit 0',
+'fi',
+'sleep 0.01',
+#searched for strings mustn't show up in command, '' to
obfuscate
+"done' && echo succ''ess || echo fail''ure",
+]),
+"success", "failure")
+
+
+def test_entitlement(self):
+"""
+This test verifies that QEMU modifies the entitlement
+after a guest request and that the guest sees the change.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+self.check_polarization('horizontal')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+self.guest_set_dispatching('1')
+self.check_polarization('vertical:medium')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low'})
+self.assertEqual(res['return'], {})
+self.check_polarization('vertical:low')
+self.check_topology(0, 0, 0, 0, 'low', False)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium'})
+self.assertEqual(res['return'], {})
+self.check_polarization('vertical:medium')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'high'})
+self.assertEqual(res['return'], {})
+self.check_polarization('vertical:high')
+self.check_topology(0, 0, 0, 0, 'high', False)
+
+self.guest_set_dispatching('0');
+self.check_polarization("horizontal")
+self.check_topology(0, 0, 0, 0, 'high', False)
--
2.39.2
[PATCH v26 07/21] s390x/cpu topology: interception of PTF instruction
From: Pierre Morel
When the host supports the CPU topology facility, the PTF
instruction with function code 2 is interpreted by the SIE,
provided that the userland hypervisor activates the interpretation
by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
The PTF instructions with function code 0 and 1 are intercepted
and must be emulated by the userland hypervisor.
During RESET all CPU of the configuration are placed in
horizontal polarity.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
include/hw/s390x/s390-virtio-ccw.h | 6
hw/s390x/cpu-topology.c| 55 ++
target/s390x/kvm/kvm.c | 11 ++
3 files changed, 72 insertions(+)
diff --git a/include/hw/s390x/s390-virtio-ccw.h
b/include/hw/s390x/s390-virtio-ccw.h
index 9bba21a916..c1d46e78af 100644
--- a/include/hw/s390x/s390-virtio-ccw.h
+++ b/include/hw/s390x/s390-virtio-ccw.h
@@ -30,6 +30,12 @@ struct S390CcwMachineState {
uint8_t loadparm[8];
};
+#define S390_PTF_REASON_NONE (0x00 << 8)
+#define S390_PTF_REASON_DONE (0x01 << 8)
+#define S390_PTF_REASON_BUSY (0x02 << 8)
+#define S390_TOPO_FC_MASK 0xffUL
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra);
+
struct S390CcwMachineClass {
/*< private >*/
MachineClass parent_class;
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 7ec9319272..a00a4ce4df 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -90,6 +90,60 @@ static void s390_topology_init(MachineState *ms)
smp->books * smp->drawers);
}
+/*
+ * s390_handle_ptf:
+ *
+ * @register 1: contains the function code
+ *
+ * Function codes 0 (horizontal) and 1 (vertical) define the CPU
+ * polarization requested by the guest.
+ *
+ * Function code 2 is handling topology changes and is interpreted
+ * by the SIE.
+ */
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
+{
+CpuS390Polarization polarization;
+CPUS390XState *env = >env;
+uint64_t reg = env->regs[r1];
+int fc = reg & S390_TOPO_FC_MASK;
+
+if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+s390_program_interrupt(env, PGM_OPERATION, ra);
+return;
+}
+
+if (env->psw.mask & PSW_MASK_PSTATE) {
+s390_program_interrupt(env, PGM_PRIVILEGED, ra);
+return;
+}
+
+if (reg & ~S390_TOPO_FC_MASK) {
+s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+return;
+}
+
+polarization = S390_CPU_POLARIZATION_VERTICAL;
+switch (fc) {
+case 0:
+polarization = S390_CPU_POLARIZATION_HORIZONTAL;
+/* fallthrough */
+case 1:
+if (s390_topology.polarization == polarization) {
+env->regs[r1] |= S390_PTF_REASON_DONE;
+setcc(cpu, 2);
+} else {
+s390_topology.polarization = polarization;
+s390_cpu_topology_set_changed(true);
+setcc(cpu, 0);
+}
+break;
+default:
+/* Note that fc == 2 is interpreted by the SIE */
+s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+}
+}
+
/**
* s390_topology_reset:
*
@@ -99,6 +153,7 @@ static void s390_topology_init(MachineState *ms)
void s390_topology_reset(void)
{
s390_cpu_topology_set_changed(false);
+s390_topology.polarization = S390_CPU_POLARIZATION_HORIZONTAL;
}
/**
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index d6bda3a2a8..4d84dcc018 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -86,6 +86,7 @@
#define PRIV_B9_EQBS0x9c
#define PRIV_B9_CLP 0xa0
+#define PRIV_B9_PTF 0xa2
#define PRIV_B9_PCISTG 0xd0
#define PRIV_B9_PCILG 0xd2
#define PRIV_B9_RPCIT 0xd3
@@ -1457,6 +1458,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct
kvm_run *run)
}
}
+static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run)
+{
+uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f;
+
+s390_handle_ptf(cpu, r1, RA_IGNORED);
+}
+
static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
{
int r = 0;
@@ -1474,6 +1482,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run,
uint8_t ipa1)
case PRIV_B9_RPCIT:
r = kvm_rpcit_service_call(cpu, run);
break;
+case PRIV_B9_PTF:
+kvm_handle_ptf(cpu, run);
+break;
case PRIV_B9_EQBS:
/* just inject exception */
r = -1;
--
2.39.2
[PATCH v26 21/21] tests/avocado: s390x cpu topology bad move
From: Pierre Morel
This test verifies that QEMU refuses to move a CPU to an
nonexistent location.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
tests/avocado/s390_topology.py | 25 +
1 file changed, 25 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index d3e6556c0f..9154ac8776 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -412,3 +412,28 @@ def test_dedicated_error(self):
res = self.vm.qmp('set-cpu-topology',
{'core-id': 0, 'entitlement': 'medium', 'dedicated':
False})
self.assertEqual(res['return'], {})
+
+def test_move_error(self):
+"""
+This test verifies that QEMU refuses to move a CPU to an
+nonexistent location
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'drawer-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'book-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'socket-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.39.2
[PATCH v26 05/21] s390x/sclp: reporting the maximum nested topology entries
From: Pierre Morel
The maximum nested topology entries is used by the guest to
know how many nested topology are available on the machine.
Let change the MNEST value from 2 to 4 in the SCLP READ INFO
structure now that we support books and drawers.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
---
include/hw/s390x/sclp.h | 5 +++--
hw/s390x/sclp.c | 5 +
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index c49051e17e..9aef6d9370 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,12 +112,13 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
-#define SCLP_READ_SCP_INFO_MNEST2
+#define SCLP_READ_SCP_INFO_MNEST4
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
uint8_t rnsize;
-uint8_t _reserved1[16 - 11]; /* 11-15 */
+uint8_t _reserved1[15 - 11]; /* 11-14 */
+uint8_t stsi_parm; /* 15-15 */
uint16_t entries_cpu; /* 16-17 */
uint16_t offset_cpu;/* 18-19 */
uint8_t _reserved2[24 - 20]; /* 20-23 */
diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index eff74479f4..d339cbb7e4 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -20,6 +20,7 @@
#include "hw/s390x/event-facility.h"
#include "hw/s390x/s390-pci-bus.h"
#include "hw/s390x/ipl.h"
+#include "hw/s390x/cpu-topology.h"
static inline SCLPDevice *get_sclp_device(void)
{
@@ -123,6 +124,10 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
return;
}
+if (s390_has_topology()) {
+read_info->stsi_parm = SCLP_READ_SCP_INFO_MNEST;
+}
+
/* CPU information */
prepare_cpu_entries(machine, entries_start, _count);
read_info->entries_cpu = cpu_to_be16(cpu_count);
--
2.39.2
[PATCH v26 11/21] machine: adding s390 topology to info hotpluggable-cpus
From: Pierre Morel
S390 topology adds books and drawers topology containers.
Let's add these to the HMP information for hotpluggable cpus.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
---
hw/core/machine-hmp-cmds.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c
index c3e55ef9e9..9a4b59c6f2 100644
--- a/hw/core/machine-hmp-cmds.c
+++ b/hw/core/machine-hmp-cmds.c
@@ -71,6 +71,12 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)
if (c->has_node_id) {
monitor_printf(mon, "node-id: \"%" PRIu64 "\"\n", c->node_id);
}
+if (c->has_drawer_id) {
+monitor_printf(mon, "drawer-id: \"%" PRIu64 "\"\n",
c->drawer_id);
+}
+if (c->has_book_id) {
+monitor_printf(mon, "book-id: \"%" PRIu64 "\"\n", c->book_id);
+}
if (c->has_socket_id) {
monitor_printf(mon, "socket-id: \"%" PRIu64 "\"\n",
c->socket_id);
}
--
2.39.2
[PATCH v26 02/21] CPU topology: extend with s390 specifics
From: Pierre Morel
S390 adds two new SMP levels, drawers and books to the CPU
topology.
S390 CPUs have specific topology features like dedication and
entitlement. These indicate to the guest information on host
vCPU scheduling and help the guest make better scheduling decisions.
Add the new levels to the relevant QAPI structs.
Add all the supported topology levels, dedication and entitlement
as properties to S390 CPUs.
Create machine-common.json so we can later include it in
machine-target.json also.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS | 1 +
qapi/machine-common.json| 21 +
qapi/machine.json | 21 +++--
qapi/qapi-schema.json | 1 +
include/hw/boards.h | 10 +-
include/hw/qdev-properties-system.h | 4 +++
target/s390x/cpu.h | 6
hw/core/machine-smp.c | 48 -
hw/core/machine.c | 4 +++
hw/core/qdev-properties-system.c| 13
hw/s390x/s390-virtio-ccw.c | 4 +++
system/vl.c | 6
target/s390x/cpu.c | 7 +
qapi/meson.build| 1 +
qemu-options.hx | 7 +++--
15 files changed, 141 insertions(+), 13 deletions(-)
create mode 100644 qapi/machine-common.json
diff --git a/MAINTAINERS b/MAINTAINERS
index ceea4c2bf2..0fd1721190 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1795,6 +1795,7 @@ F: hw/core/null-machine.c
F: hw/core/numa.c
F: hw/cpu/cluster.c
F: qapi/machine.json
+F: qapi/machine-common.json
F: qapi/machine-target.json
F: include/hw/boards.h
F: include/hw/core/cpu.h
diff --git a/qapi/machine-common.json b/qapi/machine-common.json
new file mode 100644
index 00..fa6bd71d12
--- /dev/null
+++ b/qapi/machine-common.json
@@ -0,0 +1,21 @@
+# -*- Mode: Python -*-
+# vim: filetype=python
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+##
+# = Machines S390 data types
+##
+
+##
+# @CpuS390Entitlement:
+#
+# An enumeration of CPU entitlements that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.2
+##
+{ 'enum': 'CpuS390Entitlement',
+ 'prefix': 'S390_CPU_ENTITLEMENT',
+ 'data': [ 'auto', 'low', 'medium', 'high' ] }
diff --git a/qapi/machine.json b/qapi/machine.json
index 058e884fd2..e73416d086 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -9,6 +9,7 @@
##
{ 'include': 'common.json' }
+{ 'include': 'machine-common.json' }
##
# @SysEmuTarget:
@@ -907,7 +908,13 @@
#
# @node-id: NUMA node ID the CPU belongs to
#
-# @socket-id: socket number within CPU topology the CPU belongs to
+# @drawer-id: drawer number within CPU topology the CPU belongs to
+# (since 8.2)
+#
+# @book-id: book number within parent container the CPU belongs to
+# (since 8.2)
+#
+# @socket-id: socket number within parent container the CPU belongs to
#
# @die-id: die number within the parent container the CPU belongs to
#(since 4.1)
@@ -928,6 +935,8 @@
{ 'struct': 'CpuInstanceProperties',
# Keep these in sync with the properties device_add accepts
'data': { '*node-id': 'int',
+'*drawer-id': 'int',
+'*book-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
'*cluster-id': 'int',
@@ -1488,7 +1497,7 @@
# containers.
#
# The ordering from highest/coarsest to lowest/finest is:
-# @sockets, @dies, @clusters, @cores, @threads.
+# @drawers, @books, @sockets, @dies, @clusters, @cores, @threads.
#
# Different architectures support different subsets of topology
# containers.
@@ -1501,7 +1510,11 @@
# @maxcpus: maximum number of hotpluggable virtual CPUs in the virtual
# machine
#
-# @sockets: number of sockets in the CPU topology
+# @drawers: number of drawers in the CPU topology (since 8.2)
+#
+# @books: number of books in the CPU topology (since 8.2)
+#
+# @sockets: number of sockets per parent container
#
# @dies: number of dies per parent container
#
@@ -1516,6 +1529,8 @@
##
{ 'struct': 'SMPConfiguration', 'data': {
'*cpus': 'int',
+ '*drawers': 'int',
+ '*books': 'int',
'*sockets': 'int',
'*dies': 'int',
'*clusters': 'int',
diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json
index 6594afba31..c01ec335e6 100644
--- a/qapi/qapi-schema.json
+++ b/qapi/qapi-schema.json
@@ -66,6 +66,7 @@
{ 'include': 'introspect.json' }
{ 'include': 'qom.json' }
{ 'include': 'qdev.json' }
+{ 'include': 'machine-common.json' }
{ 'include': 'machine.json' }
{ 'include': 'machine-target.json' }
{ 'include': 'replay.json' }
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 55a64a13fd..6237ea39d9 100644
--- a/include/hw/boards.h
+++ b/include
[PATCH v26 16/21] tests/avocado: s390x cpu topology polarization
From: Pierre Morel
Polarization is changed on a request from the guest.
Let's verify the polarization is accordingly set by QEMU.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
tests/avocado/s390_topology.py | 45 ++
1 file changed, 45 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 9078b45281..8166cee134 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -41,6 +41,7 @@ class S390CPUTopology(QemuSystemTest):
The polarization is changed on a request from the guest.
"""
timeout = 90
+event_timeout = 10
KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
'root=/dev/ram '
@@ -103,6 +104,14 @@ def kernel_init(self):
'-initrd', initrd_path,
'-append', kernel_command_line)
+def system_init(self):
+self.log.info("System init")
+exec_command_and_wait_for_pattern(self,
+""" mount proc -t proc /proc;
+mount sys -t sysfs /sys;
+cat /sys/devices/system/cpu/dispatching """,
+'0')
+
def test_single(self):
"""
This test checks the simplest topology with a single CPU.
@@ -198,3 +207,39 @@ def test_dash_device(self):
self.check_topology(3, 1, 1, 1, 'high', False)
self.check_topology(4, 1, 1, 1, 'medium', False)
self.check_topology(5, 2, 1, 1, 'high', True)
+
+
+def guest_set_dispatching(self, dispatching):
+exec_command(self,
+f'echo {dispatching} > /sys/devices/system/cpu/dispatching')
+self.vm.event_wait('CPU_POLARIZATION_CHANGE', self.event_timeout)
+exec_command_and_wait_for_pattern(self,
+'cat /sys/devices/system/cpu/dispatching', dispatching)
+
+
+def test_polarization(self):
+"""
+This test verifies that QEMU modifies the entitlement change after
+several guest polarization change requests.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'horizontal')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+self.guest_set_dispatching('1');
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'vertical')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+self.guest_set_dispatching('0');
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'horizontal')
+self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.39.2
Re: [PATCH v25 02/21] CPU topology: extend with s390 specifics
On Thu, 2023-10-12 at 13:02 +0200, Markus Armbruster wrote: > Nina Schoetterl-Glausch writes: > > > From: Pierre Morel > > > > S390 adds two new SMP levels, drawers and books to the CPU > > topology. > > S390 CPUs have specific topology features like dedication and > > entitlement. These indicate to the guest information on host > > vCPU scheduling and help the guest make better scheduling decisions. > > > > Let us provide the SMP properties with books and drawers levels > > and S390 CPU with dedication and entitlement, > > This is vague. Peeking at the patch, I can see it adds properties > "socket-id", "book-id", "drawer-id", "dedicated", and "entitlement" to > "s390x-cpu" objects. Suggest to spell that out here. > > > Add machine-common.json so we can later include it in > > machine-target.json also. > > > > Signed-off-by: Pierre Morel > > Reviewed-by: Nina Schoetterl-Glausch > > Co-developed-by: Nina Schoetterl-Glausch > > Reviewed-by: Thomas Huth > > Signed-off-by: Nina Schoetterl-Glausch > > --- > > MAINTAINERS | 1 + > > qapi/machine-common.json| 21 + > > qapi/machine.json | 17 +- > > qapi/qapi-schema.json | 1 + > > include/hw/boards.h | 10 +- > > include/hw/qdev-properties-system.h | 4 +++ > > target/s390x/cpu.h | 6 > > hw/core/machine-smp.c | 48 - > > hw/core/machine.c | 4 +++ > > hw/core/qdev-properties-system.c| 13 > > hw/s390x/s390-virtio-ccw.c | 4 +++ > > softmmu/vl.c| 6 > > target/s390x/cpu.c | 7 + > > qapi/meson.build| 1 + > > qemu-options.hx | 7 +++-- > > 15 files changed, 139 insertions(+), 11 deletions(-) > > create mode 100644 qapi/machine-common.json > > > > diff --git a/MAINTAINERS b/MAINTAINERS > > index 81625f036b..3f6888aa86 100644 > > --- a/MAINTAINERS > > +++ b/MAINTAINERS > > @@ -1775,6 +1775,7 @@ F: hw/core/null-machine.c > > F: hw/core/numa.c > > F: hw/cpu/cluster.c > > F: qapi/machine.json > > +F: qapi/machine-common.json > > F: qapi/machine-target.json > > F: include/hw/boards.h > > F: include/hw/core/cpu.h > > diff --git a/qapi/machine-common.json b/qapi/machine-common.json > > new file mode 100644 > > index 00..fa6bd71d12 [...] > > ## > > # @SysEmuTarget: > > @@ -904,7 +905,13 @@ > > # > > # @node-id: NUMA node ID the CPU belongs to > > # > > -# @socket-id: socket number within node/board the CPU belongs to > > +# @drawer-id: drawer number within node/board the CPU belongs to > > +# (since 8.2) > > +# > > +# @book-id: book number within drawer/node/board the CPU belongs to > > +# (since 8.2) > > +# > > +# @socket-id: socket number within book/node/board the CPU belongs to > > # > > # @die-id: die number within socket the CPU belongs to (since 4.1) > > # ># @cluster-id: cluster number within die the CPU belongs to (since ># 7.1) ># ># @core-id: core number within cluster the CPU belongs to ># ># @thread-id: thread number within core the CPU belongs to > > So... > > * A thread can only be within a core > > * A core can only be within a cluster > > * A cluster can only be within a die > > * A die can only be within a socket > > * A socket can be within a book, node, or board > > * A book can be within a drawer, node, or board > > * A drawer can be within a node, or board > > * A node is a NUMA node > > * A board is what exactly? can we have more than one? is node always > within a/the board? Yeah, the description is confusing. > > Asked differently: what are the possible hierarchies of things? The way I understand things is: * Different architectures have different hierarchies, say 1. (thread, core, cluster, die, socket) 2. (thread, core, socket, book, drawer) We define a qemu artificial ordered super set (thread, core, cluster, die, socket, book, drawer) where architectures can choose a subset of, specifying that they support a certain level or not. Now if for example x86 wanted to support a book level between thread and core, we'd need to change a bunch of code and make things more complicated. The NUMA node-id maps a hierarchy tuple to a node, I don't think it's part o
Re: [PATCH v24 01/21] qapi: machine.json: change docs regarding CpuInstanceProperties
On Thu, 2023-10-12 at 07:59 +0200, Markus Armbruster wrote:
> Nina Schoetterl-Glausch writes:
>
> > Clarify roles of different architectures.
> > Also change things a bit in anticipation of additional members being
> > added.
> >
> > Suggested-by: Markus Armbruster
> > Signed-off-by: Nina Schoetterl-Glausch
> > ---
> >
> >
> > Reference to s390x docs added in patch 14
> >
> >
> > qapi/machine.json | 17 +
> > 1 file changed, 9 insertions(+), 8 deletions(-)
> >
> > diff --git a/qapi/machine.json b/qapi/machine.json
> > index a08b6576ca..3c074c9902 100644
> > --- a/qapi/machine.json
> > +++ b/qapi/machine.json
> > @@ -70,9 +70,9 @@
> > # @qom-path: path to the CPU object in the QOM tree
> > #
> > # @thread-id: ID of the underlying host thread
> > -#
> > -# @props: properties describing to which node/socket/core/thread
> > -# virtual CPU belongs to, provided if supported by board
> > +
>
> Keep the '#', please.
>
> > +# @props: properties of type CpuInstanceProperties associated with a
> > +# virtual CPU, e.g. the socket id
> > #
> > # @target: the QEMU system emulation target, which determines which
> > # additional fields will be listed (since 3.0)
> > @@ -899,6 +899,9 @@
> > # should be passed by management with device_add command when a CPU is
> > # being hotplugged.
> > #
> > +# Which members are optional and which mandatory depends on the
> > architecture
> > +# and board.
>
> Long line. Wrap like this:
>
># Which members are optional and which mandatory depends on the
># architecture and board.
>
> Would "depends on the machine type" be more precise?
Depends on the definition of "machine type" I guess.
"architecture and board" seems more descriptive to me.
>
> > +#
> > # @node-id: NUMA node ID the CPU belongs to
> > #
> > # @socket-id: socket number within node/board the CPU belongs to
> > @@ -912,15 +915,13 @@
> > #
> > # @thread-id: thread number within core the CPU belongs to
> > #
> > -# Note: currently there are 6 properties that could be present but
> > -# management should be prepared to pass through other properties
> > -# with device_add command to allow for future interface extension.
> > -# This also requires the filed names to be kept in sync with the
> > -# properties passed to -device/device_add.
> > +# Note: management should be prepared to pass through additional
> > +# properties with device_add.
>
> Indent the second line for consistency with other Note: sections:
>
># Note: management should be prepared to pass through additional
># properties with device_add.
>
> > #
> > # Since: 2.7
> > ##
> > { 'struct': 'CpuInstanceProperties',
> > + # Keep these in sync with the properties device_add accepts
> >'data': { '*node-id': 'int',
> > '*socket-id': 'int',
> > '*die-id': 'int',
>
[PATCH v25 19/21] tests/avocado: s390x cpu topology test socket full
From: Pierre Morel
This test verifies that QMP set-cpu-topology does not accept
to overload a socket.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
tests/avocado/s390_topology.py | 26 ++
1 file changed, 26 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 3661048f4c..a63c2b2923 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -338,3 +338,29 @@ def test_dedicated(self):
self.guest_set_dispatching('0');
self.check_topology(0, 0, 0, 0, 'high', True)
self.check_polarization("horizontal")
+
+
+def test_socket_full(self):
+"""
+This test verifies that QEMU does not accept to overload a socket.
+The socket-id 0 on book-id 0 already contains CPUs 0 and 1 and can
+not accept any new CPU while socket-id 0 on book-id 1 is free.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.add_args('-smp',
+ '3,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'socket-id': 0, 'book-id': 0})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'socket-id': 0, 'book-id': 1})
+self.assertEqual(res['return'], {})
--
2.39.2
[PATCH v25 20/21] tests/avocado: s390x cpu topology dedicated errors
From: Pierre Morel
Let's test that QEMU refuses to setup a dedicated CPU with
low or medium entitlement.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
tests/avocado/s390_topology.py | 48 ++
1 file changed, 48 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index a63c2b2923..d3e6556c0f 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -364,3 +364,51 @@ def test_socket_full(self):
res = self.vm.qmp('set-cpu-topology',
{'core-id': 2, 'socket-id': 0, 'book-id': 1})
self.assertEqual(res['return'], {})
+
+def test_dedicated_error(self):
+"""
+This test verifies that QEMU refuses to lower the entitlement
+of a dedicated CPU
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+self.assertEqual(res['return'], {})
+
+self.check_topology(0, 0, 0, 0, 'high', True)
+
+self.guest_set_dispatching('1');
+
+self.check_topology(0, 0, 0, 0, 'high', True)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low', 'dedicated':
True})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low'})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium', 'dedicated':
True})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium'})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low', 'dedicated':
False})
+self.assertEqual(res['return'], {})
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium', 'dedicated':
False})
+self.assertEqual(res['return'], {})
--
2.39.2
[PATCH v25 06/21] s390x/cpu topology: resetting the Topology-Change-Report
From: Pierre Morel
During a subsystem reset the Topology-Change-Report is cleared
by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
include/hw/s390x/cpu-topology.h | 1 +
target/s390x/cpu.h | 1 +
target/s390x/kvm/kvm_s390x.h| 1 +
hw/s390x/cpu-topology.c | 11 +++
hw/s390x/s390-virtio-ccw.c | 3 +++
target/s390x/cpu-sysemu.c | 13 +
target/s390x/kvm/kvm.c | 17 +
7 files changed, 47 insertions(+)
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index f95d26d37c..e33e7c66df 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -56,6 +56,7 @@ static inline void s390_topology_setup_cpu(MachineState *ms,
#endif
extern S390Topology s390_topology;
+void s390_topology_reset(void);
static inline int s390_std_socket(int n, CpuTopology *smp)
{
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index a93139b854..2c47ce4ec0 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -655,6 +655,7 @@ typedef struct SysIBCPUListEntry {
QEMU_BUILD_BUG_ON(sizeof(SysIBCPUListEntry) != 16);
void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar,
uintptr_t ra);
+void s390_cpu_topology_set_changed(bool changed);
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin
*/
diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
index f9785564d0..649dae5948 100644
--- a/target/s390x/kvm/kvm_s390x.h
+++ b/target/s390x/kvm/kvm_s390x.h
@@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
void kvm_s390_restart_interrupt(S390CPU *cpu);
void kvm_s390_stop_interrupt(S390CPU *cpu);
void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
+int kvm_s390_topology_set_mtcr(uint64_t attr);
#endif /* KVM_S390X_H */
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 13168341b6..7ec9319272 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -90,6 +90,17 @@ static void s390_topology_init(MachineState *ms)
smp->books * smp->drawers);
}
+/**
+ * s390_topology_reset:
+ *
+ * Generic reset for CPU topology, calls s390_topology_reset()
+ * to reset the kernel Modified Topology Change Record.
+ */
+void s390_topology_reset(void)
+{
+s390_cpu_topology_set_changed(false);
+}
+
/**
* s390_topology_cpu_default:
* @cpu: pointer to a S390CPU
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 7fe2bce20c..6012165d41 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -124,6 +124,9 @@ static void subsystem_reset(void)
device_cold_reset(dev);
}
}
+if (s390_has_topology()) {
+s390_topology_reset();
+}
}
static int virtio_ccw_hcall_notify(const uint64_t *args)
diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c
index 8112561e5e..1cd30c1d84 100644
--- a/target/s390x/cpu-sysemu.c
+++ b/target/s390x/cpu-sysemu.c
@@ -307,3 +307,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data
arg)
kvm_s390_set_diag318(cs, arg.host_ulong);
}
}
+
+void s390_cpu_topology_set_changed(bool changed)
+{
+int ret;
+
+if (kvm_enabled()) {
+ret = kvm_s390_topology_set_mtcr(changed);
+if (ret) {
+error_report("Failed to set Modified Topology Change Report: %s",
+ strerror(-ret));
+}
+}
+}
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 53d6300809..d6bda3a2a8 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -2664,6 +2664,23 @@ int kvm_s390_get_zpci_op(void)
return cap_zpci_op;
}
+int kvm_s390_topology_set_mtcr(uint64_t attr)
+{
+struct kvm_device_attr attribute = {
+.group = KVM_S390_VM_CPU_TOPOLOGY,
+.attr = attr,
+};
+
+if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+return 0;
+}
+if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) {
+return -ENOTSUP;
+}
+
+return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, );
+}
+
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
--
2.39.2
[PATCH v25 01/21] qapi: machine.json: change docs regarding CpuInstanceProperties
Clarify roles of different architectures.
Also change things a bit in anticipation of additional members being
added.
Suggested-by: Markus Armbruster
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine.json | 15 ---
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/qapi/machine.json b/qapi/machine.json
index a08b6576ca..40b835e28e 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -71,8 +71,8 @@
#
# @thread-id: ID of the underlying host thread
#
-# @props: properties describing to which node/socket/core/thread
-# virtual CPU belongs to, provided if supported by board
+# @props: properties of type CpuInstanceProperties associated with a
+# virtual CPU, e.g. the socket id
#
# @target: the QEMU system emulation target, which determines which
# additional fields will be listed (since 3.0)
@@ -899,6 +899,9 @@
# should be passed by management with device_add command when a CPU is
# being hotplugged.
#
+# Which members are optional and which mandatory depends on the architecture
+# and board.
+#
# @node-id: NUMA node ID the CPU belongs to
#
# @socket-id: socket number within node/board the CPU belongs to
@@ -912,15 +915,13 @@
#
# @thread-id: thread number within core the CPU belongs to
#
-# Note: currently there are 6 properties that could be present but
-# management should be prepared to pass through other properties
-# with device_add command to allow for future interface extension.
-# This also requires the filed names to be kept in sync with the
-# properties passed to -device/device_add.
+# Note: management should be prepared to pass through additional
+# properties with device_add.
#
# Since: 2.7
##
{ 'struct': 'CpuInstanceProperties',
+ # Keep these in sync with the properties device_add accepts
'data': { '*node-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
--
2.39.2
[PATCH v25 10/21] machine: adding s390 topology to query-cpu-fast
From: Pierre Morel
S390x provides two more topology attributes, entitlement and dedication.
Let's add these CPU attributes to the QAPI command query-cpu-fast.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine.json | 11 ++-
target/s390x/cpu.c | 9 +
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/qapi/machine.json b/qapi/machine.json
index 7aacd26af0..072a12d168 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -57,9 +57,18 @@
#
# @cpu-state: the virtual CPU's state
#
+# @dedicated: the virtual CPU's dedication (since 8.2)
+#
+# @entitlement: the virtual CPU's entitlement (since 8.2)
+#
# Since: 2.12
##
-{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
+{ 'struct': 'CpuInfoS390',
+ 'data': { 'cpu-state': 'CpuS390State',
+'*dedicated': 'bool',
+'*entitlement': 'CpuS390Entitlement'
+ }
+}
##
# @CpuInfoFast:
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index 74405beb51..5967e34a85 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -38,6 +38,7 @@
#ifndef CONFIG_USER_ONLY
#include "sysemu/reset.h"
#endif
+#include "hw/s390x/cpu-topology.h"
#define CR0_RESET 0xE0UL
#define CR14_RESET 0xC200UL;
@@ -146,6 +147,14 @@ static void s390_query_cpu_fast(CPUState *cpu, CpuInfoFast
*value)
S390CPU *s390_cpu = S390_CPU(cpu);
value->u.s390x.cpu_state = s390_cpu->env.cpu_state;
+#if !defined(CONFIG_USER_ONLY)
+if (s390_has_topology()) {
+value->u.s390x.has_dedicated = true;
+value->u.s390x.dedicated = s390_cpu->env.dedicated;
+value->u.s390x.has_entitlement = true;
+value->u.s390x.entitlement = s390_cpu->env.entitlement;
+}
+#endif
}
/* S390CPUClass::reset() */
--
2.39.2
[PATCH v25 09/21] qapi/s390x/cpu topology: set-cpu-topology qmp command
From: Pierre Morel
The modification of the CPU attributes are done through a monitor
command.
It allows to move the core inside the topology tree to optimize
the cache usage in the case the host's hypervisor previously
moved the CPU.
The same command allows to modify the CPU attributes modifiers
like polarization entitlement and the dedicated attribute to notify
the guest if the host admin modified scheduling or dedication of a vCPU.
With this knowledge the guest has the possibility to optimize the
usage of the vCPUs.
The command has a feature unstable for the moment.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-target.json | 42 +
hw/s390x/cpu-topology.c | 132 +++
2 files changed, 174 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 93cbf1c128..3fd8390922 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -4,6 +4,8 @@
# This work is licensed under the terms of the GNU GPL, version 2 or later.
# See the COPYING file in the top-level directory.
+{ 'include': 'machine-common.json' }
+
##
# @CpuModelInfo:
#
@@ -375,3 +377,43 @@
'data': [ 'horizontal', 'vertical' ],
'if': 'TARGET_S390X'
}
+
+##
+# @set-cpu-topology:
+#
+# Modifies the topology by moving the CPU inside the topology
+# tree or by changing a modifier attribute of a CPU.
+# Absent values will not be modified.
+#
+# @core-id: the vCPU ID to be moved
+#
+# @socket-id: destination socket to move the vCPU to
+#
+# @book-id: destination book to move the vCPU to
+#
+# @drawer-id: destination drawer to move the vCPU to
+#
+# @entitlement: entitlement to set
+#
+# @dedicated: whether the provisioning of real to virtual CPU is dedicated
+#
+# Features:
+#
+# @unstable: This command may still be modified.
+#
+# Returns: Nothing on success, the reason on failure.
+#
+# Since: 8.2
+##
+{ 'command': 'set-cpu-topology',
+ 'data': {
+ 'core-id': 'uint16',
+ '*socket-id': 'uint16',
+ '*book-id': 'uint16',
+ '*drawer-id': 'uint16',
+ '*entitlement': 'CpuS390Entitlement',
+ '*dedicated': 'bool'
+ },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 4e4c35f6da..f3771f5045 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -23,6 +23,7 @@
#include "target/s390x/cpu.h"
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
+#include "qapi/qapi-commands-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -257,6 +258,29 @@ static bool s390_topology_check(uint16_t socket_id,
uint16_t book_id,
return true;
}
+/**
+ * s390_topology_need_report
+ * @cpu: Current cpu
+ * @drawer_id: future drawer ID
+ * @book_id: future book ID
+ * @socket_id: future socket ID
+ * @entitlement: future entitlement
+ * @dedicated: future dedicated
+ *
+ * A modified topology change report is needed if the topology
+ * tree or the topology attributes change.
+ */
+static bool s390_topology_need_report(S390CPU *cpu, int drawer_id,
+ int book_id, int socket_id,
+ uint16_t entitlement, bool dedicated)
+{
+return cpu->env.drawer_id != drawer_id ||
+ cpu->env.book_id != book_id ||
+ cpu->env.socket_id != socket_id ||
+ cpu->env.entitlement != entitlement ||
+ cpu->env.dedicated != dedicated;
+}
+
/**
* s390_update_cpu_props:
* @ms: the machine state
@@ -325,3 +349,111 @@ void s390_topology_setup_cpu(MachineState *ms, S390CPU
*cpu, Error **errp)
/* topology tree is reflected in props */
s390_update_cpu_props(ms, cpu);
}
+
+static void s390_change_topology(uint16_t core_id,
+ bool has_socket_id, uint16_t socket_id,
+ bool has_book_id, uint16_t book_id,
+ bool has_drawer_id, uint16_t drawer_id,
+ bool has_entitlement,
+ CpuS390Entitlement entitlement,
+ bool has_dedicated, bool dedicated,
+ Error **errp)
+{
+MachineState *ms = current_machine;
+int old_socket_entry;
+int new_socket_entry;
+bool report_needed;
+S390CPU *cpu;
+
+cpu = s390_cpu_addr2state(core_id);
+if (!cpu) {
+error_setg(errp, "Core-id %d does not exist!", core_id);
+return;
+}
+
+/* Get attributes not provided from cpu and verify the new topology */
+if (!has_socket_id) {
+socket_id = cpu->env.socket_id;
+}
+if (!has_book_id) {
+book_id = cpu->env.book_id;
+
[PATCH v25 07/21] s390x/cpu topology: interception of PTF instruction
From: Pierre Morel
When the host supports the CPU topology facility, the PTF
instruction with function code 2 is interpreted by the SIE,
provided that the userland hypervisor activates the interpretation
by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
The PTF instructions with function code 0 and 1 are intercepted
and must be emulated by the userland hypervisor.
During RESET all CPU of the configuration are placed in
horizontal polarity.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
include/hw/s390x/s390-virtio-ccw.h | 6
hw/s390x/cpu-topology.c| 55 ++
target/s390x/kvm/kvm.c | 11 ++
3 files changed, 72 insertions(+)
diff --git a/include/hw/s390x/s390-virtio-ccw.h
b/include/hw/s390x/s390-virtio-ccw.h
index 9bba21a916..c1d46e78af 100644
--- a/include/hw/s390x/s390-virtio-ccw.h
+++ b/include/hw/s390x/s390-virtio-ccw.h
@@ -30,6 +30,12 @@ struct S390CcwMachineState {
uint8_t loadparm[8];
};
+#define S390_PTF_REASON_NONE (0x00 << 8)
+#define S390_PTF_REASON_DONE (0x01 << 8)
+#define S390_PTF_REASON_BUSY (0x02 << 8)
+#define S390_TOPO_FC_MASK 0xffUL
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra);
+
struct S390CcwMachineClass {
/*< private >*/
MachineClass parent_class;
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 7ec9319272..a00a4ce4df 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -90,6 +90,60 @@ static void s390_topology_init(MachineState *ms)
smp->books * smp->drawers);
}
+/*
+ * s390_handle_ptf:
+ *
+ * @register 1: contains the function code
+ *
+ * Function codes 0 (horizontal) and 1 (vertical) define the CPU
+ * polarization requested by the guest.
+ *
+ * Function code 2 is handling topology changes and is interpreted
+ * by the SIE.
+ */
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
+{
+CpuS390Polarization polarization;
+CPUS390XState *env = >env;
+uint64_t reg = env->regs[r1];
+int fc = reg & S390_TOPO_FC_MASK;
+
+if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+s390_program_interrupt(env, PGM_OPERATION, ra);
+return;
+}
+
+if (env->psw.mask & PSW_MASK_PSTATE) {
+s390_program_interrupt(env, PGM_PRIVILEGED, ra);
+return;
+}
+
+if (reg & ~S390_TOPO_FC_MASK) {
+s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+return;
+}
+
+polarization = S390_CPU_POLARIZATION_VERTICAL;
+switch (fc) {
+case 0:
+polarization = S390_CPU_POLARIZATION_HORIZONTAL;
+/* fallthrough */
+case 1:
+if (s390_topology.polarization == polarization) {
+env->regs[r1] |= S390_PTF_REASON_DONE;
+setcc(cpu, 2);
+} else {
+s390_topology.polarization = polarization;
+s390_cpu_topology_set_changed(true);
+setcc(cpu, 0);
+}
+break;
+default:
+/* Note that fc == 2 is interpreted by the SIE */
+s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+}
+}
+
/**
* s390_topology_reset:
*
@@ -99,6 +153,7 @@ static void s390_topology_init(MachineState *ms)
void s390_topology_reset(void)
{
s390_cpu_topology_set_changed(false);
+s390_topology.polarization = S390_CPU_POLARIZATION_HORIZONTAL;
}
/**
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index d6bda3a2a8..4d84dcc018 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -86,6 +86,7 @@
#define PRIV_B9_EQBS0x9c
#define PRIV_B9_CLP 0xa0
+#define PRIV_B9_PTF 0xa2
#define PRIV_B9_PCISTG 0xd0
#define PRIV_B9_PCILG 0xd2
#define PRIV_B9_RPCIT 0xd3
@@ -1457,6 +1458,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct
kvm_run *run)
}
}
+static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run)
+{
+uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f;
+
+s390_handle_ptf(cpu, r1, RA_IGNORED);
+}
+
static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
{
int r = 0;
@@ -1474,6 +1482,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run,
uint8_t ipa1)
case PRIV_B9_RPCIT:
r = kvm_rpcit_service_call(cpu, run);
break;
+case PRIV_B9_PTF:
+kvm_handle_ptf(cpu, run);
+break;
case PRIV_B9_EQBS:
/* just inject exception */
r = -1;
--
2.39.2
[PATCH v25 04/21] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
From: Pierre Morel
On interception of STSI(15.1.x) the System Information Block
(SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS | 1 +
qapi/machine-target.json | 14 ++
include/hw/s390x/cpu-topology.h | 23 +++
include/hw/s390x/sclp.h | 1 +
target/s390x/cpu.h | 75 +++
hw/s390x/cpu-topology.c | 2 +
target/s390x/kvm/kvm.c | 5 +-
target/s390x/kvm/stsi-topology.c | 334 +++
target/s390x/kvm/meson.build | 3 +-
9 files changed, 456 insertions(+), 2 deletions(-)
create mode 100644 target/s390x/kvm/stsi-topology.c
diff --git a/MAINTAINERS b/MAINTAINERS
index ee1cd6e581..db982cea3f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1696,6 +1696,7 @@ M: Nina Schoetterl-Glausch
S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
+F: target/s390x/kvm/stsi-topology.c
X86 Machines
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index f0a6b72414..93cbf1c128 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -361,3 +361,17 @@
'TARGET_MIPS',
'TARGET_LOONGARCH64',
'TARGET_RISCV' ] } }
+
+##
+# @CpuS390Polarization:
+#
+# An enumeration of CPU polarization that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.2
+##
+{ 'enum': 'CpuS390Polarization',
+ 'prefix': 'S390_CPU_POLARIZATION',
+ 'data': [ 'horizontal', 'vertical' ],
+ 'if': 'TARGET_S390X'
+}
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 97b0af2795..f95d26d37c 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -15,10 +15,33 @@
#include "hw/boards.h"
#include "qapi/qapi-types-machine-target.h"
+#define S390_TOPOLOGY_CPU_IFL 0x03
+
+typedef struct S390TopologyId {
+uint8_t sentinel;
+uint8_t drawer;
+uint8_t book;
+uint8_t socket;
+uint8_t type;
+uint8_t vertical:1;
+uint8_t entitlement:2;
+uint8_t dedicated;
+uint8_t origin;
+} S390TopologyId;
+
+typedef struct S390TopologyEntry {
+QTAILQ_ENTRY(S390TopologyEntry) next;
+S390TopologyId id;
+uint64_t mask;
+} S390TopologyEntry;
+
typedef struct S390Topology {
uint8_t *cores_per_socket;
+CpuS390Polarization polarization;
} S390Topology;
+typedef QTAILQ_HEAD(, S390TopologyEntry) S390TopologyList;
+
#ifdef CONFIG_KVM
bool s390_has_topology(void);
void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index cf1f2efae2..c49051e17e 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,6 +112,7 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
+#define SCLP_READ_SCP_INFO_MNEST2
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index dfcc1aa1fc..a93139b854 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -571,6 +571,29 @@ typedef struct SysIB_322 {
} SysIB_322;
QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096);
+/*
+ * Topology Magnitude fields (MAG) indicates the maximum number of
+ * topology list entries (TLE) at the corresponding nesting level.
+ */
+#define S390_TOPOLOGY_MAG 6
+#define S390_TOPOLOGY_MAG6 0
+#define S390_TOPOLOGY_MAG5 1
+#define S390_TOPOLOGY_MAG4 2
+#define S390_TOPOLOGY_MAG3 3
+#define S390_TOPOLOGY_MAG2 4
+#define S390_TOPOLOGY_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+uint8_t reserved0[2];
+uint16_t length;
+uint8_t mag[S390_TOPOLOGY_MAG];
+uint8_t reserved1;
+uint8_t mnest;
+uint32_t reserved2;
+char tle[];
+} SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
typedef union SysIB {
SysIB_111 sysib_111;
SysIB_121 sysib_121;
@@ -578,9 +601,61 @@ typedef union SysIB {
SysIB_221 sysib_221;
SysIB_222 sysib_222;
SysIB_322 sysib_322;
+SysIB_151x sysib_151x;
} SysIB;
QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ * Defines a container to contain other Topology List Entries
+ * of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ * Specifies the CPUs position, type, entitlement and polarization
+ * of the CPUs contained in the last container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only three levels, the drawer's, book's and socket's level.
+ *
+ * A container with a nestin
[PATCH v25 03/21] s390x/cpu topology: add topology entries on CPU hotplug
From: Pierre Morel
The topology information are attributes of the CPU and are
specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers,
books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socket
The possibility to insert a CPU in a mask is dependent on the
number of cores allowed in a socket, a book or a drawer, the
checking is done during the hot plug of the CPU to have an
immediate answer.
If the complete topology is not specified, the core is added
in the physical topology based on its core ID and it gets
defaults values for the modifier attributes.
This way, starting QEMU without specifying the topology can
still get some advantage of the CPU topology.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS | 6 +
include/hw/s390x/cpu-topology.h | 54 +++
hw/s390x/cpu-topology.c | 259
hw/s390x/s390-virtio-ccw.c | 22 ++-
hw/s390x/meson.build| 1 +
5 files changed, 340 insertions(+), 2 deletions(-)
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 hw/s390x/cpu-topology.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 3f6888aa86..ee1cd6e581 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1691,6 +1691,12 @@ F: hw/s390x/event-facility.c
F: hw/s390x/sclp*.c
L: qemu-s3...@nongnu.org
+S390 CPU topology
+M: Nina Schoetterl-Glausch
+S: Supported
+F: include/hw/s390x/cpu-topology.h
+F: hw/s390x/cpu-topology.c
+
X86 Machines
PC
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
new file mode 100644
index 00..97b0af2795
--- /dev/null
+++ b/include/hw/s390x/cpu-topology.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel
+ *
+ */
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#ifndef CONFIG_USER_ONLY
+
+#include "qemu/queue.h"
+#include "hw/boards.h"
+#include "qapi/qapi-types-machine-target.h"
+
+typedef struct S390Topology {
+uint8_t *cores_per_socket;
+} S390Topology;
+
+#ifdef CONFIG_KVM
+bool s390_has_topology(void);
+void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
+#else
+static inline bool s390_has_topology(void)
+{
+ return false;
+}
+static inline void s390_topology_setup_cpu(MachineState *ms,
+ S390CPU *cpu,
+ Error **errp) {}
+#endif
+
+extern S390Topology s390_topology;
+
+static inline int s390_std_socket(int n, CpuTopology *smp)
+{
+return (n / smp->cores) % smp->sockets;
+}
+
+static inline int s390_std_book(int n, CpuTopology *smp)
+{
+return (n / (smp->cores * smp->sockets)) % smp->books;
+}
+
+static inline int s390_std_drawer(int n, CpuTopology *smp)
+{
+return (n / (smp->cores * smp->sockets * smp->books)) % smp->drawers;
+}
+
+#endif /* CONFIG_USER_ONLY */
+
+#endif
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
new file mode 100644
index 00..cabd42e779
--- /dev/null
+++ b/hw/s390x/cpu-topology.c
@@ -0,0 +1,259 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel
+ *
+ * S390 topology handling can be divided in two parts:
+ *
+ * - The first part in this file is taking care of all common functions
+ * used by KVM and TCG to create and modify the topology.
+ *
+ * - The second part, building the topology information data for the
+ * guest with CPU and KVM specificity will be implemented inside
+ * the target/s390/kvm sub tree.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/qdev-properties.h"
+#include "hw/boards.h"
+#include "target/s390x/cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
+#include "hw/s390x/cpu-topology.h"
+
+/*
+ * s390_topology is used to keep the topology information.
+ * .cores_per_socket: tracks information on the count of cores
+ *per socket.
+ */
+S390Topology s390_topology = {
+/* will be initialized after the CPU model is realized */
+.cores_per_socket = NULL,
+};
+
+/**
+ * s390_socket_nb:
+ * @cpu: s390x CPU
+ *
+ * Returns the socket number used inside the cores_per_socket array
+ * for a topology tree entry
+ */
+static int s390_socket_nb_from_ids(int drawer_id, int book_id, int socket_id)
+{
+return (drawer_id * current_machine->smp.books + book_id) *
+ current_machine->smp.sockets + socket_id;
+}
+
+/**
+
[PATCH v25 11/21] machine: adding s390 topology to info hotpluggable-cpus
From: Pierre Morel
S390 topology adds books and drawers topology containers.
Let's add these to the HMP information for hotpluggable cpus.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
---
hw/core/machine-hmp-cmds.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c
index c3e55ef9e9..9a4b59c6f2 100644
--- a/hw/core/machine-hmp-cmds.c
+++ b/hw/core/machine-hmp-cmds.c
@@ -71,6 +71,12 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)
if (c->has_node_id) {
monitor_printf(mon, "node-id: \"%" PRIu64 "\"\n", c->node_id);
}
+if (c->has_drawer_id) {
+monitor_printf(mon, "drawer-id: \"%" PRIu64 "\"\n",
c->drawer_id);
+}
+if (c->has_book_id) {
+monitor_printf(mon, "book-id: \"%" PRIu64 "\"\n", c->book_id);
+}
if (c->has_socket_id) {
monitor_printf(mon, "socket-id: \"%" PRIu64 "\"\n",
c->socket_id);
}
--
2.39.2
[PATCH v25 17/21] tests/avocado: s390x cpu topology entitlement tests
From: Pierre Morel
Test changes in the entitlement from both a guest and a host point of
view, depending on the polarization.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Tested-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
tests/avocado/s390_topology.py | 62 ++
1 file changed, 62 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 8166cee134..24fac9a54d 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -243,3 +243,65 @@ def test_polarization(self):
res = self.vm.qmp('query-s390x-cpu-polarization')
self.assertEqual(res['return']['polarization'], 'horizontal')
self.check_topology(0, 0, 0, 0, 'medium', False)
+
+
+def check_polarization(self, polarization):
+#We need to wait for the change to have been propagated to the kernel
+exec_command_and_wait_for_pattern(self,
+"\n".join([
+"timeout 1 sh -c 'while true",
+'do',
+'syspath="/sys/devices/system/cpu/cpu0/polarization"',
+'polarization="$(cat "$syspath")" || exit',
+ f'if [ "$polarization" = "{polarization}" ]; then',
+'exit 0',
+'fi',
+'sleep 0.01',
+#searched for strings mustn't show up in command, '' to
obfuscate
+"done' && echo succ''ess || echo fail''ure",
+]),
+"success", "failure")
+
+
+def test_entitlement(self):
+"""
+This test verifies that QEMU modifies the entitlement
+after a guest request and that the guest sees the change.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+self.check_polarization('horizontal')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+self.guest_set_dispatching('1')
+self.check_polarization('vertical:medium')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low'})
+self.assertEqual(res['return'], {})
+self.check_polarization('vertical:low')
+self.check_topology(0, 0, 0, 0, 'low', False)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium'})
+self.assertEqual(res['return'], {})
+self.check_polarization('vertical:medium')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'high'})
+self.assertEqual(res['return'], {})
+self.check_polarization('vertical:high')
+self.check_topology(0, 0, 0, 0, 'high', False)
+
+self.guest_set_dispatching('0');
+self.check_polarization("horizontal")
+self.check_topology(0, 0, 0, 0, 'high', False)
--
2.39.2
[PATCH v25 16/21] tests/avocado: s390x cpu topology polarization
From: Pierre Morel
Polarization is changed on a request from the guest.
Let's verify the polarization is accordingly set by QEMU.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
tests/avocado/s390_topology.py | 45 ++
1 file changed, 45 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 9078b45281..8166cee134 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -41,6 +41,7 @@ class S390CPUTopology(QemuSystemTest):
The polarization is changed on a request from the guest.
"""
timeout = 90
+event_timeout = 10
KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
'root=/dev/ram '
@@ -103,6 +104,14 @@ def kernel_init(self):
'-initrd', initrd_path,
'-append', kernel_command_line)
+def system_init(self):
+self.log.info("System init")
+exec_command_and_wait_for_pattern(self,
+""" mount proc -t proc /proc;
+mount sys -t sysfs /sys;
+cat /sys/devices/system/cpu/dispatching """,
+'0')
+
def test_single(self):
"""
This test checks the simplest topology with a single CPU.
@@ -198,3 +207,39 @@ def test_dash_device(self):
self.check_topology(3, 1, 1, 1, 'high', False)
self.check_topology(4, 1, 1, 1, 'medium', False)
self.check_topology(5, 2, 1, 1, 'high', True)
+
+
+def guest_set_dispatching(self, dispatching):
+exec_command(self,
+f'echo {dispatching} > /sys/devices/system/cpu/dispatching')
+self.vm.event_wait('CPU_POLARIZATION_CHANGE', self.event_timeout)
+exec_command_and_wait_for_pattern(self,
+'cat /sys/devices/system/cpu/dispatching', dispatching)
+
+
+def test_polarization(self):
+"""
+This test verifies that QEMU modifies the entitlement change after
+several guest polarization change requests.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'horizontal')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+self.guest_set_dispatching('1');
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'vertical')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+self.guest_set_dispatching('0');
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'horizontal')
+self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.39.2
[PATCH v25 15/21] tests/avocado: s390x cpu topology core
From: Pierre Morel
Introduction of the s390x cpu topology core functions and
basic tests.
We test the correlation between the command line and
the QMP results in query-cpus-fast for various CPU topology.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Tested-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS| 1 +
tests/avocado/s390_topology.py | 200 +
2 files changed, 201 insertions(+)
create mode 100644 tests/avocado/s390_topology.py
diff --git a/MAINTAINERS b/MAINTAINERS
index da4f099877..9cd6973ace 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1699,6 +1699,7 @@ F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/stsi-topology.c
F: docs/devel/s390-cpu-topology.rst
F: docs/system/s390x/cpu-topology.rst
+F: tests/avocado/s390_topology.py
X86 Machines
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
new file mode 100644
index 00..9078b45281
--- /dev/null
+++ b/tests/avocado/s390_topology.py
@@ -0,0 +1,200 @@
+# Functional test that boots a Linux kernel and checks the console
+#
+# Copyright IBM Corp. 2023
+#
+# Author:
+# Pierre Morel
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+import shutil
+import time
+
+from avocado_qemu import QemuSystemTest
+from avocado_qemu import exec_command
+from avocado_qemu import exec_command_and_wait_for_pattern
+from avocado_qemu import interrupt_interactive_console_until_pattern
+from avocado_qemu import wait_for_console_pattern
+from avocado.utils import process
+from avocado.utils import archive
+
+
+class S390CPUTopology(QemuSystemTest):
+"""
+S390x CPU topology consists of 4 topology layers, from bottom to top,
+the cores, sockets, books and drawers and 2 modifiers attributes,
+the entitlement and the dedication.
+See: docs/system/s390x/cpu-topology.rst.
+
+S390x CPU topology is setup in different ways:
+- implicitly from the '-smp' argument by completing each topology
+ level one after the other beginning with drawer 0, book 0 and
+ socket 0.
+- explicitly from the '-device' argument on the QEMU command line
+- explicitly by hotplug of a new CPU using QMP or HMP
+- it is modified by using QMP 'set-cpu-topology'
+
+The S390x modifier attribute entitlement depends on the machine
+polarization, which can be horizontal or vertical.
+The polarization is changed on a request from the guest.
+"""
+timeout = 90
+
+KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
+ 'root=/dev/ram '
+ 'selinux=0 '
+ 'rdinit=/bin/sh')
+
+def wait_until_booted(self):
+wait_for_console_pattern(self, 'no job control',
+ failure_message='Kernel panic - not syncing',
+ vm=None)
+
+def check_topology(self, c, s, b, d, e, t):
+res = self.vm.qmp('query-cpus-fast')
+cpus = res['return']
+for cpu in cpus:
+core = cpu['props']['core-id']
+socket = cpu['props']['socket-id']
+book = cpu['props']['book-id']
+drawer = cpu['props']['drawer-id']
+entitlement = cpu.get('entitlement')
+dedicated = cpu.get('dedicated')
+if core == c:
+self.assertEqual(drawer, d)
+self.assertEqual(book, b)
+self.assertEqual(socket, s)
+self.assertEqual(entitlement, e)
+self.assertEqual(dedicated, t)
+
+def kernel_init(self):
+"""
+We need a VM that supports CPU topology,
+currently this only the case when using KVM, not TCG.
+We need a kernel supporting the CPU topology.
+We need a minimal root filesystem with a shell.
+"""
+self.require_accelerator("kvm")
+kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/kernel.img')
+kernel_hash = '0d1aaaf303f07cf0160c8c48e56fe638'
+kernel_path = self.fetch_asset(kernel_url, algorithm='md5',
+ asset_hash=kernel_hash)
+
+initrd_url = ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/initrd.img')
+initrd_hash = 'a122057d95725ac030e2ec51df46e172'
+initrd_path_xz = self.fetch_asset(initrd_url, algorithm='md5',
+ asset_hash=initrd_hash)
+initrd_path = os.path.join(self.workdir, 'initrd-
[PATCH v25 08/21] target/s390x/cpu topology: activate CPU topology
From: Pierre Morel
The KVM capability KVM_CAP_S390_CPU_TOPOLOGY is used to
activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and
the topology facility in the host CPU model for the guest
in the case the topology is available in QEMU and in KVM.
The feature is disabled by default and fenced for SE
(secure execution).
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
hw/s390x/cpu-topology.c | 2 +-
target/s390x/cpu_models.c | 1 +
target/s390x/kvm/kvm.c| 9 +
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index a00a4ce4df..4e4c35f6da 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -69,7 +69,7 @@ static int s390_socket_nb(S390CPU *cpu)
*/
bool s390_has_topology(void)
{
-return false;
+return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY);
}
/**
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index 98f14c09c2..4dead48650 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -255,6 +255,7 @@ bool s390_has_feat(S390Feat feat)
case S390_FEAT_SIE_CMMA:
case S390_FEAT_SIE_PFMFI:
case S390_FEAT_SIE_IBS:
+case S390_FEAT_CONFIGURATION_TOPOLOGY:
return false;
break;
default:
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 4d84dcc018..2e43dc746f 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -366,6 +366,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_SIGP, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_VECTOR_REGISTERS, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_STSI, 0);
+kvm_vm_enable_cap(s, KVM_CAP_S390_CPU_TOPOLOGY, 0);
if (ri_allowed()) {
if (kvm_vm_enable_cap(s, KVM_CAP_S390_RI, 0) == 0) {
cap_ri = 1;
@@ -2509,6 +2510,14 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model,
Error **errp)
set_bit(S390_FEAT_UNPACK, model->features);
}
+/*
+ * If we have kernel support for CPU Topology indicate the
+ * configuration-topology facility.
+ */
+if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) {
+set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features);
+}
+
/* We emulate a zPCI bus and AEN, therefore we don't need HW support */
set_bit(S390_FEAT_ZPCI, model->features);
set_bit(S390_FEAT_ADAPTER_EVENT_NOTIFICATION, model->features);
--
2.39.2
[PATCH v25 14/21] docs/s390x/cpu topology: document s390x cpu topology
From: Pierre Morel
Add some basic examples for the definition of cpu topology
in s390x.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS| 2 +
docs/devel/index-internals.rst | 1 +
docs/devel/s390-cpu-topology.rst | 170
docs/system/s390x/cpu-topology.rst | 244 +
docs/system/target-s390x.rst | 1 +
qapi/machine.json | 2 +
6 files changed, 420 insertions(+)
create mode 100644 docs/devel/s390-cpu-topology.rst
create mode 100644 docs/system/s390x/cpu-topology.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index db982cea3f..da4f099877 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1697,6 +1697,8 @@ S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/stsi-topology.c
+F: docs/devel/s390-cpu-topology.rst
+F: docs/system/s390x/cpu-topology.rst
X86 Machines
diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst
index e1a93df263..6f81df92bc 100644
--- a/docs/devel/index-internals.rst
+++ b/docs/devel/index-internals.rst
@@ -14,6 +14,7 @@ Details about QEMU's various subsystems including how to add
features to them.
migration
multi-process
reset
+ s390-cpu-topology
s390-dasd-ipl
tracing
vfio-migration
diff --git a/docs/devel/s390-cpu-topology.rst b/docs/devel/s390-cpu-topology.rst
new file mode 100644
index 00..9eab28d5e5
--- /dev/null
+++ b/docs/devel/s390-cpu-topology.rst
@@ -0,0 +1,170 @@
+QAPI interface for S390 CPU topology
+
+
+The following sections will explain the QAPI interface for S390 CPU topology
+with the help of exemplary output.
+For this, let's assume that QEMU has been started with the following
+command, defining 4 CPUs, where CPU[0] is defined by the -smp argument and will
+have default values:
+
+.. code-block:: bash
+
+ qemu-system-s390x \
+-enable-kvm \
+-cpu z14,ctop=on \
+-smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
+-device z14-s390x-cpu,core-id=19,entitlement=high \
+-device z14-s390x-cpu,core-id=11,entitlement=low \
+-device z14-s390x-cpu,core-id=112,entitlement=high \
+ ...
+
+Additions to query-cpus-fast
+
+
+The command query-cpus-fast allows querying the topology tree and
+modifiers for all configured vCPUs.
+
+.. code-block:: QMP
+
+ { "execute": "query-cpus-fast" }
+ {
+ "return": [
+{
+ "dedicated": false,
+ "thread-id": 536993,
+ "props": {
+"core-id": 0,
+"socket-id": 0,
+"drawer-id": 0,
+"book-id": 0
+ },
+ "cpu-state": "operating",
+ "entitlement": "medium",
+ "qom-path": "/machine/unattached/device[0]",
+ "cpu-index": 0,
+ "target": "s390x"
+},
+{
+ "dedicated": false,
+ "thread-id": 537003,
+ "props": {
+"core-id": 19,
+"socket-id": 1,
+"drawer-id": 0,
+"book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[0]",
+ "cpu-index": 19,
+ "target": "s390x"
+},
+{
+ "dedicated": false,
+ "thread-id": 537004,
+ "props": {
+"core-id": 11,
+"socket-id": 1,
+"drawer-id": 0,
+"book-id": 1
+ },
+ "cpu-state": "operating",
+ "entitlement": "low",
+ "qom-path": "/machine/peripheral-anon/device[1]",
+ "cpu-index": 11,
+ "target": "s390x"
+},
+{
+ "dedicated": true,
+ "thread-id": 537005,
+ "props": {
+"core-id": 112,
+"socket-id": 0,
+"drawer-id": 3,
+"book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[2]",
+ "cpu-index": 112,
+ "target": "s390x"
+}
+ ]
+ }
+
+
+QAPI command: set-cpu-topology
+--
+
+The command set-cpu-topology allows modifying the topology tree
+or the topology modifiers of a vCPU in the configuration.
+
+.. code-block:: QMP
+
+{ "execute": "set-cpu-topology",
+ &
[PATCH v25 21/21] tests/avocado: s390x cpu topology bad move
From: Pierre Morel
This test verifies that QEMU refuses to move a CPU to an
nonexistent location.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
tests/avocado/s390_topology.py | 25 +
1 file changed, 25 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index d3e6556c0f..9154ac8776 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -412,3 +412,28 @@ def test_dedicated_error(self):
res = self.vm.qmp('set-cpu-topology',
{'core-id': 0, 'entitlement': 'medium', 'dedicated':
False})
self.assertEqual(res['return'], {})
+
+def test_move_error(self):
+"""
+This test verifies that QEMU refuses to move a CPU to an
+nonexistent location
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'drawer-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'book-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'socket-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.39.2
[PATCH v25 02/21] CPU topology: extend with s390 specifics
From: Pierre Morel
S390 adds two new SMP levels, drawers and books to the CPU
topology.
S390 CPUs have specific topology features like dedication and
entitlement. These indicate to the guest information on host
vCPU scheduling and help the guest make better scheduling decisions.
Let us provide the SMP properties with books and drawers levels
and S390 CPU with dedication and entitlement,
Add machine-common.json so we can later include it in
machine-target.json also.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS | 1 +
qapi/machine-common.json| 21 +
qapi/machine.json | 17 +-
qapi/qapi-schema.json | 1 +
include/hw/boards.h | 10 +-
include/hw/qdev-properties-system.h | 4 +++
target/s390x/cpu.h | 6
hw/core/machine-smp.c | 48 -
hw/core/machine.c | 4 +++
hw/core/qdev-properties-system.c| 13
hw/s390x/s390-virtio-ccw.c | 4 +++
softmmu/vl.c| 6
target/s390x/cpu.c | 7 +
qapi/meson.build| 1 +
qemu-options.hx | 7 +++--
15 files changed, 139 insertions(+), 11 deletions(-)
create mode 100644 qapi/machine-common.json
diff --git a/MAINTAINERS b/MAINTAINERS
index 81625f036b..3f6888aa86 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1775,6 +1775,7 @@ F: hw/core/null-machine.c
F: hw/core/numa.c
F: hw/cpu/cluster.c
F: qapi/machine.json
+F: qapi/machine-common.json
F: qapi/machine-target.json
F: include/hw/boards.h
F: include/hw/core/cpu.h
diff --git a/qapi/machine-common.json b/qapi/machine-common.json
new file mode 100644
index 00..fa6bd71d12
--- /dev/null
+++ b/qapi/machine-common.json
@@ -0,0 +1,21 @@
+# -*- Mode: Python -*-
+# vim: filetype=python
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+##
+# = Machines S390 data types
+##
+
+##
+# @CpuS390Entitlement:
+#
+# An enumeration of CPU entitlements that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.2
+##
+{ 'enum': 'CpuS390Entitlement',
+ 'prefix': 'S390_CPU_ENTITLEMENT',
+ 'data': [ 'auto', 'low', 'medium', 'high' ] }
diff --git a/qapi/machine.json b/qapi/machine.json
index 40b835e28e..7aacd26af0 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -9,6 +9,7 @@
##
{ 'include': 'common.json' }
+{ 'include': 'machine-common.json' }
##
# @SysEmuTarget:
@@ -904,7 +905,13 @@
#
# @node-id: NUMA node ID the CPU belongs to
#
-# @socket-id: socket number within node/board the CPU belongs to
+# @drawer-id: drawer number within node/board the CPU belongs to
+# (since 8.2)
+#
+# @book-id: book number within drawer/node/board the CPU belongs to
+# (since 8.2)
+#
+# @socket-id: socket number within book/node/board the CPU belongs to
#
# @die-id: die number within socket the CPU belongs to (since 4.1)
#
@@ -923,6 +930,8 @@
{ 'struct': 'CpuInstanceProperties',
# Keep these in sync with the properties device_add accepts
'data': { '*node-id': 'int',
+'*drawer-id': 'int',
+'*book-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
'*cluster-id': 'int',
@@ -1481,6 +1490,10 @@
#
# @cpus: number of virtual CPUs in the virtual machine
#
+# @drawers: number of drawers in the CPU topology (since 8.2)
+#
+# @books: number of books in the CPU topology (since 8.2)
+#
# @sockets: number of sockets in the CPU topology
#
# @dies: number of dies per socket in the CPU topology
@@ -1499,6 +1512,8 @@
##
{ 'struct': 'SMPConfiguration', 'data': {
'*cpus': 'int',
+ '*drawers': 'int',
+ '*books': 'int',
'*sockets': 'int',
'*dies': 'int',
'*clusters': 'int',
diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json
index 6594afba31..c01ec335e6 100644
--- a/qapi/qapi-schema.json
+++ b/qapi/qapi-schema.json
@@ -66,6 +66,7 @@
{ 'include': 'introspect.json' }
{ 'include': 'qom.json' }
{ 'include': 'qdev.json' }
+{ 'include': 'machine-common.json' }
{ 'include': 'machine.json' }
{ 'include': 'machine-target.json' }
{ 'include': 'replay.json' }
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 55a64a13fd..6237ea39d9 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -135,12 +135,16 @@ typedef struct {
* @clusters_supported - whether clusters are supported by the machine
* @has_clusters - whether clusters are explicitly specified in the user
* provided SMP configuration
+ * @books_supported - whether books are supported by the machine
+ * @drawers_supported - whether drawers are supported by the machine
*/
typedef struct {
bool prefer_sockets
[PATCH v25 05/21] s390x/sclp: reporting the maximum nested topology entries
From: Pierre Morel
The maximum nested topology entries is used by the guest to
know how many nested topology are available on the machine.
Let change the MNEST value from 2 to 4 in the SCLP READ INFO
structure now that we support books and drawers.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
---
include/hw/s390x/sclp.h | 5 +++--
hw/s390x/sclp.c | 5 +
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index c49051e17e..9aef6d9370 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,12 +112,13 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
-#define SCLP_READ_SCP_INFO_MNEST2
+#define SCLP_READ_SCP_INFO_MNEST4
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
uint8_t rnsize;
-uint8_t _reserved1[16 - 11]; /* 11-15 */
+uint8_t _reserved1[15 - 11]; /* 11-14 */
+uint8_t stsi_parm; /* 15-15 */
uint16_t entries_cpu; /* 16-17 */
uint16_t offset_cpu;/* 18-19 */
uint8_t _reserved2[24 - 20]; /* 20-23 */
diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index eff74479f4..d339cbb7e4 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -20,6 +20,7 @@
#include "hw/s390x/event-facility.h"
#include "hw/s390x/s390-pci-bus.h"
#include "hw/s390x/ipl.h"
+#include "hw/s390x/cpu-topology.h"
static inline SCLPDevice *get_sclp_device(void)
{
@@ -123,6 +124,10 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
return;
}
+if (s390_has_topology()) {
+read_info->stsi_parm = SCLP_READ_SCP_INFO_MNEST;
+}
+
/* CPU information */
prepare_cpu_entries(machine, entries_start, _count);
read_info->entries_cpu = cpu_to_be16(cpu_count);
--
2.39.2
[PATCH v25 12/21] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
From: Pierre Morel
When the guest asks to change the polarization this change
is forwarded to the upper layer using QAPI.
The upper layer is supposed to take according decisions concerning
CPU provisioning.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-target.json | 34 ++
hw/s390x/cpu-topology.c | 2 ++
2 files changed, 36 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 3fd8390922..02f1de5d02 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -417,3 +417,37 @@
'features': [ 'unstable' ],
'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
}
+
+##
+# @CPU_POLARIZATION_CHANGE:
+#
+# Emitted when the guest asks to change the polarization.
+#
+# The guest can tell the host (via the PTF instruction) whether the
+# CPUs should be provisioned using horizontal or vertical polarization.
+#
+# On horizontal polarization the host is expected to provision all vCPUs
+# equally.
+# On vertical polarization the host can provision each vCPU differently.
+# The guest will get information on the details of the provisioning
+# the next time it uses the STSI(15) instruction.
+#
+# @polarization: polarization specified by the guest
+#
+# Features:
+#
+# @unstable: This event is experimental.
+#
+# Since: 8.2
+#
+# Example:
+#
+# <- { "event": "CPU_POLARIZATION_CHANGE",
+# "data": { "polarization": "horizontal" },
+# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
+##
+{ 'event': 'CPU_POLARIZATION_CHANGE',
+ 'data': { 'polarization': 'CpuS390Polarization' },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index f3771f5045..327bccea4f 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -24,6 +24,7 @@
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
#include "qapi/qapi-commands-machine-target.h"
+#include "qapi/qapi-events-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -136,6 +137,7 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
} else {
s390_topology.polarization = polarization;
s390_cpu_topology_set_changed(true);
+qapi_event_send_cpu_polarization_change(polarization);
setcc(cpu, 0);
}
break;
--
2.39.2
[PATCH v25 18/21] tests/avocado: s390x cpu topology test dedicated CPU
From: Pierre Morel
A dedicated CPU in vertical polarization can only have
a high entitlement.
Let's check this from both host and guest point of view.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
tests/avocado/s390_topology.py | 33 +
1 file changed, 33 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 24fac9a54d..3661048f4c 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -305,3 +305,36 @@ def test_entitlement(self):
self.guest_set_dispatching('0');
self.check_polarization("horizontal")
self.check_topology(0, 0, 0, 0, 'high', False)
+
+
+def test_dedicated(self):
+"""
+This test verifies that QEMU adjusts the entitlement correctly when a
+CPU is made dedicated.
+QEMU retains the entitlement value when horizontal polarization is in
effect.
+For the guest, the field shows the effective value of the entitlement.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+self.check_polarization("horizontal")
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+self.assertEqual(res['return'], {})
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("horizontal")
+
+self.guest_set_dispatching('1');
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("vertical:high")
+
+self.guest_set_dispatching('0');
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("horizontal")
--
2.39.2
[PATCH v25 13/21] qapi/s390x/cpu topology: add query-s390x-cpu-polarization command
From: Pierre Morel
The query-s390x-cpu-polarization qmp command returns the current
CPU polarization of the machine.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-target.json | 30 ++
hw/s390x/cpu-topology.c | 8
2 files changed, 38 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 02f1de5d02..10289835fb 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -451,3 +451,33 @@
'features': [ 'unstable' ],
'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
}
+
+##
+# @CpuPolarizationInfo:
+#
+# The result of a CPU polarization query.
+#
+# @polarization: the CPU polarization
+#
+# Since: 8.2
+##
+{ 'struct': 'CpuPolarizationInfo',
+ 'data': { 'polarization': 'CpuS390Polarization' },
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
+
+##
+# @query-s390x-cpu-polarization:
+#
+# Features:
+#
+# @unstable: This command is experimental.
+#
+# Returns: the machine's CPU polarization
+#
+# Since: 8.2
+##
+{ 'command': 'query-s390x-cpu-polarization', 'returns': 'CpuPolarizationInfo',
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 327bccea4f..f16bdf65fa 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -459,3 +459,11 @@ void qmp_set_cpu_topology(uint16_t core,
has_drawer, drawer, has_entitlement, entitlement,
has_dedicated, dedicated, errp);
}
+
+CpuPolarizationInfo *qmp_query_s390x_cpu_polarization(Error **errp)
+{
+CpuPolarizationInfo *info = g_new0(CpuPolarizationInfo, 1);
+
+info->polarization = s390_topology.polarization;
+return info;
+}
--
2.39.2
[PATCH v25 00/21] s390x: CPU Topology
l)
- changed comments in machine.jason
(Daniel)
- Added reset
(Janosch)
- since v5
- rebasing on newer QEMU version
- reworked most lines above 80 characters.
- since v4
- Added drawer and books to topology
- Added numa topology
- Added documentation
- since v3
- Added migration
(Thomas)
- Separated STSI instruction from KVM to prepare TCG
(Thomas)
- Take care of endianess to prepare TCG
(Thomas)
- Added comments on STSI CPU container and PFT instruction
(Thomas)
- Moved enabling the instructions as the last patch
(Thomas)
Nina Schoetterl-Glausch (1):
qapi: machine.json: change docs regarding CpuInstanceProperties
Pierre Morel (20):
CPU topology: extend with s390 specifics
s390x/cpu topology: add topology entries on CPU hotplug
target/s390x/cpu topology: handle STSI(15) and build the SYSIB
s390x/sclp: reporting the maximum nested topology entries
s390x/cpu topology: resetting the Topology-Change-Report
s390x/cpu topology: interception of PTF instruction
target/s390x/cpu topology: activate CPU topology
qapi/s390x/cpu topology: set-cpu-topology qmp command
machine: adding s390 topology to query-cpu-fast
machine: adding s390 topology to info hotpluggable-cpus
qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
qapi/s390x/cpu topology: add query-s390x-cpu-polarization command
docs/s390x/cpu topology: document s390x cpu topology
tests/avocado: s390x cpu topology core
tests/avocado: s390x cpu topology polarization
tests/avocado: s390x cpu topology entitlement tests
tests/avocado: s390x cpu topology test dedicated CPU
tests/avocado: s390x cpu topology test socket full
tests/avocado: s390x cpu topology dedicated errors
tests/avocado: s390x cpu topology bad move
MAINTAINERS | 11 +
docs/devel/index-internals.rst | 1 +
docs/devel/s390-cpu-topology.rst| 170 ++
docs/system/s390x/cpu-topology.rst | 244 +++
docs/system/target-s390x.rst| 1 +
qapi/machine-common.json| 21 ++
qapi/machine-target.json| 120 +++
qapi/machine.json | 45 ++-
qapi/qapi-schema.json | 1 +
include/hw/boards.h | 10 +-
include/hw/qdev-properties-system.h | 4 +
include/hw/s390x/cpu-topology.h | 78 +
include/hw/s390x/s390-virtio-ccw.h | 6 +
include/hw/s390x/sclp.h | 4 +-
target/s390x/cpu.h | 82 +
target/s390x/kvm/kvm_s390x.h| 1 +
hw/core/machine-hmp-cmds.c | 6 +
hw/core/machine-smp.c | 48 ++-
hw/core/machine.c | 4 +
hw/core/qdev-properties-system.c| 13 +
hw/s390x/cpu-topology.c | 469
hw/s390x/s390-virtio-ccw.c | 29 +-
hw/s390x/sclp.c | 5 +
softmmu/vl.c| 6 +
target/s390x/cpu-sysemu.c | 13 +
target/s390x/cpu.c | 16 +
target/s390x/cpu_models.c | 1 +
target/s390x/kvm/kvm.c | 42 ++-
target/s390x/kvm/stsi-topology.c| 334
hw/s390x/meson.build| 1 +
qapi/meson.build| 1 +
qemu-options.hx | 7 +-
target/s390x/kvm/meson.build| 3 +-
tests/avocado/s390_topology.py | 439 ++
34 files changed, 2212 insertions(+), 24 deletions(-)
create mode 100644 docs/devel/s390-cpu-topology.rst
create mode 100644 docs/system/s390x/cpu-topology.rst
create mode 100644 qapi/machine-common.json
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 hw/s390x/cpu-topology.c
create mode 100644 target/s390x/kvm/stsi-topology.c
create mode 100644 tests/avocado/s390_topology.py
Range-diff against v24:
1: 2c7552ab60 ! 1: 578a12c6e9 qapi: machine.json: change docs regarding
CpuInstanceProperties
@@ Commit message
## qapi/machine.json ##
@@
- # @qom-path: path to the CPU object in the QOM tree
#
# @thread-id: ID of the underlying host thread
--#
+ #
-# @props: properties describing to which node/socket/core/thread
-# virtual CPU belongs to, provided if supported by board
-+
+# @props: properties of type CpuInstanceProperties associated with a
+# virtual CPU, e.g. the socket id
#
2: 3ea5ee1016 ! 2: 4a6314be55 CPU topology: extend with s390 specifics
@@ Commit message
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
+ Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
+ ## MAINTAINERS ##
+@@ MAINTAINERS: F: hw/core/null-machine.c
+ F: hw/core/numa.c
+ F: hw/cpu/cluster.c
+ F: qapi/machine.json
++F: qapi/machine-common.json
+ F: qapi/machine-target.json
+ F: include/hw/boards.h
Re: [PATCH v24 02/21] CPU topology: extend with s390 specifics
On Tue, 2023-09-26 at 14:15 +0200, Nina Schoetterl-Glausch wrote: > From: Pierre Morel > > S390 adds two new SMP levels, drawers and books to the CPU > topology. > S390 CPUs have specific topology features like dedication and > entitlement. These indicate to the guest information on host > vCPU scheduling and help the guest make better scheduling decisions. > > Let us provide the SMP properties with books and drawers levels > and S390 CPU with dedication and entitlement, > > Add machine-common.json so we can later include it in > machine-target.json also. > > Signed-off-by: Pierre Morel > Reviewed-by: Nina Schoetterl-Glausch > Co-developed-by: Nina Schoetterl-Glausch > Signed-off-by: Nina Schoetterl-Glausch > --- > qapi/machine-common.json| 21 + I guess I should also add this new file to MAINTAINERS under Machine core, right?
[PATCH v24 04/21] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
From: Pierre Morel
On interception of STSI(15.1.x) the System Information Block
(SYSIB) is built from the list of pre-ordered topology entries.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS | 1 +
qapi/machine-target.json | 14 ++
include/hw/s390x/cpu-topology.h | 23 +++
include/hw/s390x/sclp.h | 1 +
target/s390x/cpu.h | 75 +++
hw/s390x/cpu-topology.c | 2 +
target/s390x/kvm/kvm.c | 5 +-
target/s390x/kvm/stsi-topology.c | 339 +++
target/s390x/kvm/meson.build | 3 +-
9 files changed, 461 insertions(+), 2 deletions(-)
create mode 100644 target/s390x/kvm/stsi-topology.c
diff --git a/MAINTAINERS b/MAINTAINERS
index c5a5303f61..447bf8ac20 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1696,6 +1696,7 @@ M: Nina Schoetterl-Glausch
S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
+F: target/s390x/kvm/stsi-topology.c
X86 Machines
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index f0a6b72414..93cbf1c128 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -361,3 +361,17 @@
'TARGET_MIPS',
'TARGET_LOONGARCH64',
'TARGET_RISCV' ] } }
+
+##
+# @CpuS390Polarization:
+#
+# An enumeration of CPU polarization that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.2
+##
+{ 'enum': 'CpuS390Polarization',
+ 'prefix': 'S390_CPU_POLARIZATION',
+ 'data': [ 'horizontal', 'vertical' ],
+ 'if': 'TARGET_S390X'
+}
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 97b0af2795..350c7ea8aa 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -15,10 +15,33 @@
#include "hw/boards.h"
#include "qapi/qapi-types-machine-target.h"
+#define S390_TOPOLOGY_CPU_IFL 0x03
+
+typedef struct s390_topology_id {
+uint8_t sentinel;
+uint8_t drawer;
+uint8_t book;
+uint8_t socket;
+uint8_t type;
+uint8_t vertical:1;
+uint8_t entitlement:2;
+uint8_t dedicated;
+uint8_t origin;
+} s390_topology_id;
+
+typedef struct S390TopologyEntry {
+QTAILQ_ENTRY(S390TopologyEntry) next;
+s390_topology_id id;
+uint64_t mask;
+} S390TopologyEntry;
+
typedef struct S390Topology {
uint8_t *cores_per_socket;
+CpuS390Polarization polarization;
} S390Topology;
+typedef QTAILQ_HEAD(, S390TopologyEntry) S390TopologyList;
+
#ifdef CONFIG_KVM
bool s390_has_topology(void);
void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index cf1f2efae2..c49051e17e 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,6 +112,7 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
+#define SCLP_READ_SCP_INFO_MNEST2
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index dfcc1aa1fc..c1ba5c46d6 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -571,6 +571,29 @@ typedef struct SysIB_322 {
} SysIB_322;
QEMU_BUILD_BUG_ON(sizeof(SysIB_322) != 4096);
+/*
+ * Topology Magnitude fields (MAG) indicates the maximum number of
+ * topology list entries (TLE) at the corresponding nesting level.
+ */
+#define S390_TOPOLOGY_MAG 6
+#define S390_TOPOLOGY_MAG6 0
+#define S390_TOPOLOGY_MAG5 1
+#define S390_TOPOLOGY_MAG4 2
+#define S390_TOPOLOGY_MAG3 3
+#define S390_TOPOLOGY_MAG2 4
+#define S390_TOPOLOGY_MAG1 5
+/* Configuration topology */
+typedef struct SysIB_151x {
+uint8_t reserved0[2];
+uint16_t length;
+uint8_t mag[S390_TOPOLOGY_MAG];
+uint8_t reserved1;
+uint8_t mnest;
+uint32_t reserved2;
+char tle[];
+} SysIB_151x;
+QEMU_BUILD_BUG_ON(sizeof(SysIB_151x) != 16);
+
typedef union SysIB {
SysIB_111 sysib_111;
SysIB_121 sysib_121;
@@ -578,9 +601,61 @@ typedef union SysIB {
SysIB_221 sysib_221;
SysIB_222 sysib_222;
SysIB_322 sysib_322;
+SysIB_151x sysib_151x;
} SysIB;
QEMU_BUILD_BUG_ON(sizeof(SysIB) != 4096);
+/*
+ * CPU Topology List provided by STSI with fc=15 provides a list
+ * of two different Topology List Entries (TLE) types to specify
+ * the topology hierarchy.
+ *
+ * - Container Topology List Entry
+ * Defines a container to contain other Topology List Entries
+ * of any type, nested containers or CPU.
+ * - CPU Topology List Entry
+ * Specifies the CPUs position, type, entitlement and polarization
+ * of the CPUs contained in the last Container TLE.
+ *
+ * There can be theoretically up to five levels of containers, QEMU
+ * uses only three levels, the drawer's, book's and socket's level.
+ *
+ * A container with
[PATCH v24 13/21] qapi/s390x/cpu topology: add query-s390x-cpu-polarization command
From: Pierre Morel
The query-s390x-cpu-polarization qmp command returns the current
CPU polarization of the machine.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-target.json | 30 ++
hw/s390x/cpu-topology.c | 8
2 files changed, 38 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 02f1de5d02..10289835fb 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -451,3 +451,33 @@
'features': [ 'unstable' ],
'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
}
+
+##
+# @CpuPolarizationInfo:
+#
+# The result of a CPU polarization query.
+#
+# @polarization: the CPU polarization
+#
+# Since: 8.2
+##
+{ 'struct': 'CpuPolarizationInfo',
+ 'data': { 'polarization': 'CpuS390Polarization' },
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
+
+##
+# @query-s390x-cpu-polarization:
+#
+# Features:
+#
+# @unstable: This command is experimental.
+#
+# Returns: the machine's CPU polarization
+#
+# Since: 8.2
+##
+{ 'command': 'query-s390x-cpu-polarization', 'returns': 'CpuPolarizationInfo',
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 18274db74c..1561a8ab44 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -459,3 +459,11 @@ void qmp_set_cpu_topology(uint16_t core,
has_drawer, drawer, has_entitlement, entitlement,
has_dedicated, dedicated, errp);
}
+
+CpuPolarizationInfo *qmp_query_s390x_cpu_polarization(Error **errp)
+{
+CpuPolarizationInfo *info = g_new0(CpuPolarizationInfo, 1);
+
+info->polarization = s390_topology.polarization;
+return info;
+}
--
2.39.2
[PATCH v24 06/21] s390x/cpu topology: resetting the Topology-Change-Report
From: Pierre Morel
During a subsystem reset the Topology-Change-Report is cleared
by the machine.
Let's ask KVM to clear the Modified Topology Change Report (MTCR)
bit of the SCA in the case of a subsystem reset.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
include/hw/s390x/cpu-topology.h | 1 +
target/s390x/cpu.h | 1 +
target/s390x/kvm/kvm_s390x.h| 1 +
hw/s390x/cpu-topology.c | 11 +++
hw/s390x/s390-virtio-ccw.c | 3 +++
target/s390x/cpu-sysemu.c | 13 +
target/s390x/kvm/kvm.c | 17 +
7 files changed, 47 insertions(+)
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
index 350c7ea8aa..196c9afb3f 100644
--- a/include/hw/s390x/cpu-topology.h
+++ b/include/hw/s390x/cpu-topology.h
@@ -56,6 +56,7 @@ static inline void s390_topology_setup_cpu(MachineState *ms,
#endif
extern S390Topology s390_topology;
+void s390_topology_reset(void);
static inline int s390_std_socket(int n, CpuTopology *smp)
{
diff --git a/target/s390x/cpu.h b/target/s390x/cpu.h
index c1ba5c46d6..2722e7c450 100644
--- a/target/s390x/cpu.h
+++ b/target/s390x/cpu.h
@@ -655,6 +655,7 @@ typedef struct SysIBCPUListEntry {
QEMU_BUILD_BUG_ON(sizeof(SysIBCPUListEntry) != 16);
void insert_stsi_15_1_x(S390CPU *cpu, int sel2, uint64_t addr, uint8_t ar,
uintptr_t ra);
+void s390_cpu_topology_set_changed(bool changed);
/* MMU defines */
#define ASCE_ORIGIN (~0xfffULL) /* segment table origin
*/
diff --git a/target/s390x/kvm/kvm_s390x.h b/target/s390x/kvm/kvm_s390x.h
index f9785564d0..649dae5948 100644
--- a/target/s390x/kvm/kvm_s390x.h
+++ b/target/s390x/kvm/kvm_s390x.h
@@ -47,5 +47,6 @@ void kvm_s390_crypto_reset(void);
void kvm_s390_restart_interrupt(S390CPU *cpu);
void kvm_s390_stop_interrupt(S390CPU *cpu);
void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info);
+int kvm_s390_topology_set_mtcr(uint64_t attr);
#endif /* KVM_S390X_H */
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 9bf13a6ebf..03de3d3d10 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -90,6 +90,17 @@ static void s390_topology_init(MachineState *ms)
smp->books * smp->drawers);
}
+/**
+ * s390_topology_reset:
+ *
+ * Generic reset for CPU topology, calls s390_topology_reset()
+ * to reset the kernel Modified Topology Change Record.
+ */
+void s390_topology_reset(void)
+{
+s390_cpu_topology_set_changed(false);
+}
+
/**
* s390_topology_cpu_default:
* @cpu: pointer to a S390CPU
diff --git a/hw/s390x/s390-virtio-ccw.c b/hw/s390x/s390-virtio-ccw.c
index 7fe2bce20c..6012165d41 100644
--- a/hw/s390x/s390-virtio-ccw.c
+++ b/hw/s390x/s390-virtio-ccw.c
@@ -124,6 +124,9 @@ static void subsystem_reset(void)
device_cold_reset(dev);
}
}
+if (s390_has_topology()) {
+s390_topology_reset();
+}
}
static int virtio_ccw_hcall_notify(const uint64_t *args)
diff --git a/target/s390x/cpu-sysemu.c b/target/s390x/cpu-sysemu.c
index 8112561e5e..1cd30c1d84 100644
--- a/target/s390x/cpu-sysemu.c
+++ b/target/s390x/cpu-sysemu.c
@@ -307,3 +307,16 @@ void s390_do_cpu_set_diag318(CPUState *cs, run_on_cpu_data
arg)
kvm_s390_set_diag318(cs, arg.host_ulong);
}
}
+
+void s390_cpu_topology_set_changed(bool changed)
+{
+int ret;
+
+if (kvm_enabled()) {
+ret = kvm_s390_topology_set_mtcr(changed);
+if (ret) {
+error_report("Failed to set Modified Topology Change Report: %s",
+ strerror(-ret));
+}
+}
+}
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 53d6300809..d6bda3a2a8 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -2664,6 +2664,23 @@ int kvm_s390_get_zpci_op(void)
return cap_zpci_op;
}
+int kvm_s390_topology_set_mtcr(uint64_t attr)
+{
+struct kvm_device_attr attribute = {
+.group = KVM_S390_VM_CPU_TOPOLOGY,
+.attr = attr,
+};
+
+if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+return 0;
+}
+if (!kvm_vm_check_attr(kvm_state, KVM_S390_VM_CPU_TOPOLOGY, attr)) {
+return -ENOTSUP;
+}
+
+return kvm_vm_ioctl(kvm_state, KVM_SET_DEVICE_ATTR, );
+}
+
void kvm_arch_accel_class_init(ObjectClass *oc)
{
}
--
2.39.2
[PATCH v24 19/21] tests/avocado: s390x cpu topology test socket full
From: Pierre Morel
This test verifies that QMP set-cpu-topology does not accept
to overload a socket.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
tests/avocado/s390_topology.py | 26 ++
1 file changed, 26 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 3661048f4c..a63c2b2923 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -338,3 +338,29 @@ def test_dedicated(self):
self.guest_set_dispatching('0');
self.check_topology(0, 0, 0, 0, 'high', True)
self.check_polarization("horizontal")
+
+
+def test_socket_full(self):
+"""
+This test verifies that QEMU does not accept to overload a socket.
+The socket-id 0 on book-id 0 already contains CPUs 0 and 1 and can
+not accept any new CPU while socket-id 0 on book-id 1 is free.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.add_args('-smp',
+ '3,drawers=2,books=2,sockets=3,cores=2,maxcpus=24')
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'socket-id': 0, 'book-id': 0})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 2, 'socket-id': 0, 'book-id': 1})
+self.assertEqual(res['return'], {})
--
2.39.2
[PATCH v24 21/21] tests/avocado: s390x cpu topology bad move
From: Pierre Morel
This test verifies that QEMU refuses to move a CPU to an
nonexistent location.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
tests/avocado/s390_topology.py | 25 +
1 file changed, 25 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index d3e6556c0f..9154ac8776 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -412,3 +412,28 @@ def test_dedicated_error(self):
res = self.vm.qmp('set-cpu-topology',
{'core-id': 0, 'entitlement': 'medium', 'dedicated':
False})
self.assertEqual(res['return'], {})
+
+def test_move_error(self):
+"""
+This test verifies that QEMU refuses to move a CPU to an
+nonexistent location
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'drawer-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'book-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'socket-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.39.2
[PATCH v24 05/21] s390x/sclp: reporting the maximum nested topology entries
From: Pierre Morel
The maximum nested topology entries is used by the guest to
know how many nested topology are available on the machine.
Let change the MNEST value from 2 to 4 in the SCLP READ INFO
structure now that we support books and drawers.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
---
include/hw/s390x/sclp.h | 5 +++--
hw/s390x/sclp.c | 5 +
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index c49051e17e..9aef6d9370 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,12 +112,13 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
-#define SCLP_READ_SCP_INFO_MNEST2
+#define SCLP_READ_SCP_INFO_MNEST4
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
uint8_t rnsize;
-uint8_t _reserved1[16 - 11]; /* 11-15 */
+uint8_t _reserved1[15 - 11]; /* 11-14 */
+uint8_t stsi_parm; /* 15-15 */
uint16_t entries_cpu; /* 16-17 */
uint16_t offset_cpu;/* 18-19 */
uint8_t _reserved2[24 - 20]; /* 20-23 */
diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index eff74479f4..d339cbb7e4 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -20,6 +20,7 @@
#include "hw/s390x/event-facility.h"
#include "hw/s390x/s390-pci-bus.h"
#include "hw/s390x/ipl.h"
+#include "hw/s390x/cpu-topology.h"
static inline SCLPDevice *get_sclp_device(void)
{
@@ -123,6 +124,10 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
return;
}
+if (s390_has_topology()) {
+read_info->stsi_parm = SCLP_READ_SCP_INFO_MNEST;
+}
+
/* CPU information */
prepare_cpu_entries(machine, entries_start, _count);
read_info->entries_cpu = cpu_to_be16(cpu_count);
--
2.39.2
[PATCH v24 09/21] qapi/s390x/cpu topology: set-cpu-topology qmp command
From: Pierre Morel
The modification of the CPU attributes are done through a monitor
command.
It allows to move the core inside the topology tree to optimize
the cache usage in the case the host's hypervisor previously
moved the CPU.
The same command allows to modify the CPU attributes modifiers
like polarization entitlement and the dedicated attribute to notify
the guest if the host admin modified scheduling or dedication of a vCPU.
With this knowledge the guest has the possibility to optimize the
usage of the vCPUs.
The command has a feature unstable for the moment.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-target.json | 42 +
hw/s390x/cpu-topology.c | 132 +++
2 files changed, 174 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 93cbf1c128..3fd8390922 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -4,6 +4,8 @@
# This work is licensed under the terms of the GNU GPL, version 2 or later.
# See the COPYING file in the top-level directory.
+{ 'include': 'machine-common.json' }
+
##
# @CpuModelInfo:
#
@@ -375,3 +377,43 @@
'data': [ 'horizontal', 'vertical' ],
'if': 'TARGET_S390X'
}
+
+##
+# @set-cpu-topology:
+#
+# Modifies the topology by moving the CPU inside the topology
+# tree or by changing a modifier attribute of a CPU.
+# Absent values will not be modified.
+#
+# @core-id: the vCPU ID to be moved
+#
+# @socket-id: destination socket to move the vCPU to
+#
+# @book-id: destination book to move the vCPU to
+#
+# @drawer-id: destination drawer to move the vCPU to
+#
+# @entitlement: entitlement to set
+#
+# @dedicated: whether the provisioning of real to virtual CPU is dedicated
+#
+# Features:
+#
+# @unstable: This command may still be modified.
+#
+# Returns: Nothing on success, the reason on failure.
+#
+# Since: 8.2
+##
+{ 'command': 'set-cpu-topology',
+ 'data': {
+ 'core-id': 'uint16',
+ '*socket-id': 'uint16',
+ '*book-id': 'uint16',
+ '*drawer-id': 'uint16',
+ '*entitlement': 'CpuS390Entitlement',
+ '*dedicated': 'bool'
+ },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 13f404a0d7..28adfb3f84 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -23,6 +23,7 @@
#include "target/s390x/cpu.h"
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
+#include "qapi/qapi-commands-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -257,6 +258,29 @@ static bool s390_topology_check(uint16_t socket_id,
uint16_t book_id,
return true;
}
+/**
+ * s390_topology_need_report
+ * @cpu: Current cpu
+ * @drawer_id: future drawer ID
+ * @book_id: future book ID
+ * @socket_id: future socket ID
+ * @entitlement: future entitlement
+ * @dedicated: future dedicated
+ *
+ * A modified topology change report is needed if the topology
+ * tree or the topology attributes change.
+ */
+static bool s390_topology_need_report(S390CPU *cpu, int drawer_id,
+ int book_id, int socket_id,
+ uint16_t entitlement, bool dedicated)
+{
+return cpu->env.drawer_id != drawer_id ||
+ cpu->env.book_id != book_id ||
+ cpu->env.socket_id != socket_id ||
+ cpu->env.entitlement != entitlement ||
+ cpu->env.dedicated != dedicated;
+}
+
/**
* s390_update_cpu_props:
* @ms: the machine state
@@ -325,3 +349,111 @@ void s390_topology_setup_cpu(MachineState *ms, S390CPU
*cpu, Error **errp)
/* topology tree is reflected in props */
s390_update_cpu_props(ms, cpu);
}
+
+static void s390_change_topology(uint16_t core_id,
+ bool has_socket_id, uint16_t socket_id,
+ bool has_book_id, uint16_t book_id,
+ bool has_drawer_id, uint16_t drawer_id,
+ bool has_entitlement,
+ CpuS390Entitlement entitlement,
+ bool has_dedicated, bool dedicated,
+ Error **errp)
+{
+MachineState *ms = current_machine;
+int old_socket_entry;
+int new_socket_entry;
+bool report_needed;
+S390CPU *cpu;
+
+cpu = s390_cpu_addr2state(core_id);
+if (!cpu) {
+error_setg(errp, "Core-id %d does not exist!", core_id);
+return;
+}
+
+/* Get attributes not provided from cpu and verify the new topology */
+if (!has_socket_id) {
+socket_id = cpu->env.socket_id;
+}
+if (!has_book_id) {
+book_id = cpu->env.book_id;
+
[PATCH v24 03/21] s390x/cpu topology: add topology entries on CPU hotplug
From: Pierre Morel
The topology information are attributes of the CPU and are
specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers,
books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socket
The possibility to insert a CPU in a mask is dependent on the
number of cores allowed in a socket, a book or a drawer, the
checking is done during the hot plug of the CPU to have an
immediate answer.
If the complete topology is not specified, the core is added
in the physical topology based on its core ID and it gets
defaults values for the modifier attributes.
This way, starting QEMU without specifying the topology can
still get some advantage of the CPU topology.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS | 6 +
include/hw/s390x/cpu-topology.h | 54 +++
hw/s390x/cpu-topology.c | 259
hw/s390x/s390-virtio-ccw.c | 22 ++-
hw/s390x/meson.build| 1 +
5 files changed, 340 insertions(+), 2 deletions(-)
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 hw/s390x/cpu-topology.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 355b1960ce..c5a5303f61 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1691,6 +1691,12 @@ F: hw/s390x/event-facility.c
F: hw/s390x/sclp*.c
L: qemu-s3...@nongnu.org
+S390 CPU topology
+M: Nina Schoetterl-Glausch
+S: Supported
+F: include/hw/s390x/cpu-topology.h
+F: hw/s390x/cpu-topology.c
+
X86 Machines
PC
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
new file mode 100644
index 00..97b0af2795
--- /dev/null
+++ b/include/hw/s390x/cpu-topology.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel
+ *
+ */
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#ifndef CONFIG_USER_ONLY
+
+#include "qemu/queue.h"
+#include "hw/boards.h"
+#include "qapi/qapi-types-machine-target.h"
+
+typedef struct S390Topology {
+uint8_t *cores_per_socket;
+} S390Topology;
+
+#ifdef CONFIG_KVM
+bool s390_has_topology(void);
+void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
+#else
+static inline bool s390_has_topology(void)
+{
+ return false;
+}
+static inline void s390_topology_setup_cpu(MachineState *ms,
+ S390CPU *cpu,
+ Error **errp) {}
+#endif
+
+extern S390Topology s390_topology;
+
+static inline int s390_std_socket(int n, CpuTopology *smp)
+{
+return (n / smp->cores) % smp->sockets;
+}
+
+static inline int s390_std_book(int n, CpuTopology *smp)
+{
+return (n / (smp->cores * smp->sockets)) % smp->books;
+}
+
+static inline int s390_std_drawer(int n, CpuTopology *smp)
+{
+return (n / (smp->cores * smp->sockets * smp->books)) % smp->drawers;
+}
+
+#endif /* CONFIG_USER_ONLY */
+
+#endif
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
new file mode 100644
index 00..189fcc5334
--- /dev/null
+++ b/hw/s390x/cpu-topology.c
@@ -0,0 +1,259 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel
+ *
+ * S390 topology handling can be divided in two parts:
+ *
+ * - The first part in this file is taking care of all common functions
+ * used by KVM and TCG to create and modify the topology.
+ *
+ * - The second part, building the topology information data for the
+ * guest with CPU and KVM specificity will be implemented inside
+ * the target/s390/kvm sub tree.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/qdev-properties.h"
+#include "hw/boards.h"
+#include "target/s390x/cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
+#include "hw/s390x/cpu-topology.h"
+
+/*
+ * s390_topology is used to keep the topology information.
+ * .cores_per_socket: tracks information on the count of cores
+ *per socket.
+ */
+S390Topology s390_topology = {
+/* will be initialized after the CPU model is realized */
+.cores_per_socket = NULL,
+};
+
+/**
+ * s390_socket_nb:
+ * @cpu: s390x CPU
+ *
+ * Returns the socket number used inside the cores_per_socket array
+ * for a topology tree entry
+ */
+static int s390_socket_nb_from_ids(int drawer_id, int book_id, int socket_id)
+{
+return (drawer_id * current_machine->smp.books + book_id) *
+ current_machine->smp.sockets + socket_id;
+}
+
+/**
+
[PATCH v24 08/21] target/s390x/cpu topology: activate CPU topology
From: Pierre Morel
The KVM capability KVM_CAP_S390_CPU_TOPOLOGY is used to
activate the S390_FEAT_CONFIGURATION_TOPOLOGY feature and
the topology facility in the host CPU model for the guest
in the case the topology is available in QEMU and in KVM.
The feature is disabled by default and fenced for SE
(secure execution).
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
hw/s390x/cpu-topology.c | 2 +-
target/s390x/cpu_models.c | 1 +
target/s390x/kvm/kvm.c| 9 +
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index c34581964f..13f404a0d7 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -69,7 +69,7 @@ static int s390_socket_nb(S390CPU *cpu)
*/
bool s390_has_topology(void)
{
-return false;
+return s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY);
}
/**
diff --git a/target/s390x/cpu_models.c b/target/s390x/cpu_models.c
index 98f14c09c2..4dead48650 100644
--- a/target/s390x/cpu_models.c
+++ b/target/s390x/cpu_models.c
@@ -255,6 +255,7 @@ bool s390_has_feat(S390Feat feat)
case S390_FEAT_SIE_CMMA:
case S390_FEAT_SIE_PFMFI:
case S390_FEAT_SIE_IBS:
+case S390_FEAT_CONFIGURATION_TOPOLOGY:
return false;
break;
default:
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index 4d84dcc018..2e43dc746f 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -366,6 +366,7 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_SIGP, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_VECTOR_REGISTERS, 0);
kvm_vm_enable_cap(s, KVM_CAP_S390_USER_STSI, 0);
+kvm_vm_enable_cap(s, KVM_CAP_S390_CPU_TOPOLOGY, 0);
if (ri_allowed()) {
if (kvm_vm_enable_cap(s, KVM_CAP_S390_RI, 0) == 0) {
cap_ri = 1;
@@ -2509,6 +2510,14 @@ void kvm_s390_get_host_cpu_model(S390CPUModel *model,
Error **errp)
set_bit(S390_FEAT_UNPACK, model->features);
}
+/*
+ * If we have kernel support for CPU Topology indicate the
+ * configuration-topology facility.
+ */
+if (kvm_check_extension(kvm_state, KVM_CAP_S390_CPU_TOPOLOGY)) {
+set_bit(S390_FEAT_CONFIGURATION_TOPOLOGY, model->features);
+}
+
/* We emulate a zPCI bus and AEN, therefore we don't need HW support */
set_bit(S390_FEAT_ZPCI, model->features);
set_bit(S390_FEAT_ADAPTER_EVENT_NOTIFICATION, model->features);
--
2.39.2
[PATCH v24 10/21] machine: adding s390 topology to query-cpu-fast
From: Pierre Morel
S390x provides two more topology attributes, entitlement and dedication.
Let's add these CPU attributes to the QAPI command query-cpu-fast.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine.json | 11 ++-
target/s390x/cpu.c | 9 +
2 files changed, 19 insertions(+), 1 deletion(-)
diff --git a/qapi/machine.json b/qapi/machine.json
index 8499d40269..eb2de7acc7 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -57,9 +57,18 @@
#
# @cpu-state: the virtual CPU's state
#
+# @dedicated: the virtual CPU's dedication (since 8.2)
+#
+# @entitlement: the virtual CPU's entitlement (since 8.2)
+#
# Since: 2.12
##
-{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
+{ 'struct': 'CpuInfoS390',
+ 'data': { 'cpu-state': 'CpuS390State',
+'*dedicated': 'bool',
+'*entitlement': 'CpuS390Entitlement'
+ }
+}
##
# @CpuInfoFast:
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index 74405beb51..5967e34a85 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -38,6 +38,7 @@
#ifndef CONFIG_USER_ONLY
#include "sysemu/reset.h"
#endif
+#include "hw/s390x/cpu-topology.h"
#define CR0_RESET 0xE0UL
#define CR14_RESET 0xC200UL;
@@ -146,6 +147,14 @@ static void s390_query_cpu_fast(CPUState *cpu, CpuInfoFast
*value)
S390CPU *s390_cpu = S390_CPU(cpu);
value->u.s390x.cpu_state = s390_cpu->env.cpu_state;
+#if !defined(CONFIG_USER_ONLY)
+if (s390_has_topology()) {
+value->u.s390x.has_dedicated = true;
+value->u.s390x.dedicated = s390_cpu->env.dedicated;
+value->u.s390x.has_entitlement = true;
+value->u.s390x.entitlement = s390_cpu->env.entitlement;
+}
+#endif
}
/* S390CPUClass::reset() */
--
2.39.2
[PATCH v24 12/21] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
From: Pierre Morel
When the guest asks to change the polarization this change
is forwarded to the upper layer using QAPI.
The upper layer is supposed to take according decisions concerning
CPU provisioning.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-target.json | 34 ++
hw/s390x/cpu-topology.c | 2 ++
2 files changed, 36 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 3fd8390922..02f1de5d02 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -417,3 +417,37 @@
'features': [ 'unstable' ],
'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
}
+
+##
+# @CPU_POLARIZATION_CHANGE:
+#
+# Emitted when the guest asks to change the polarization.
+#
+# The guest can tell the host (via the PTF instruction) whether the
+# CPUs should be provisioned using horizontal or vertical polarization.
+#
+# On horizontal polarization the host is expected to provision all vCPUs
+# equally.
+# On vertical polarization the host can provision each vCPU differently.
+# The guest will get information on the details of the provisioning
+# the next time it uses the STSI(15) instruction.
+#
+# @polarization: polarization specified by the guest
+#
+# Features:
+#
+# @unstable: This event is experimental.
+#
+# Since: 8.2
+#
+# Example:
+#
+# <- { "event": "CPU_POLARIZATION_CHANGE",
+# "data": { "polarization": "horizontal" },
+# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
+##
+{ 'event': 'CPU_POLARIZATION_CHANGE',
+ 'data': { 'polarization': 'CpuS390Polarization' },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 28adfb3f84..18274db74c 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -24,6 +24,7 @@
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
#include "qapi/qapi-commands-machine-target.h"
+#include "qapi/qapi-events-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -136,6 +137,7 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
} else {
s390_topology.polarization = polarization;
s390_cpu_topology_set_changed(true);
+qapi_event_send_cpu_polarization_change(polarization);
setcc(cpu, 0);
}
break;
--
2.39.2
[PATCH v24 17/21] tests/avocado: s390x cpu topology entitlement tests
From: Pierre Morel
Test changes in the entitlement from both a guest and a host point of
view, depending on the polarization.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
tests/avocado/s390_topology.py | 62 ++
1 file changed, 62 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 8166cee134..24fac9a54d 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -243,3 +243,65 @@ def test_polarization(self):
res = self.vm.qmp('query-s390x-cpu-polarization')
self.assertEqual(res['return']['polarization'], 'horizontal')
self.check_topology(0, 0, 0, 0, 'medium', False)
+
+
+def check_polarization(self, polarization):
+#We need to wait for the change to have been propagated to the kernel
+exec_command_and_wait_for_pattern(self,
+"\n".join([
+"timeout 1 sh -c 'while true",
+'do',
+'syspath="/sys/devices/system/cpu/cpu0/polarization"',
+'polarization="$(cat "$syspath")" || exit',
+ f'if [ "$polarization" = "{polarization}" ]; then',
+'exit 0',
+'fi',
+'sleep 0.01',
+#searched for strings mustn't show up in command, '' to
obfuscate
+"done' && echo succ''ess || echo fail''ure",
+]),
+"success", "failure")
+
+
+def test_entitlement(self):
+"""
+This test verifies that QEMU modifies the entitlement
+after a guest request and that the guest sees the change.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+self.check_polarization('horizontal')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+self.guest_set_dispatching('1')
+self.check_polarization('vertical:medium')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low'})
+self.assertEqual(res['return'], {})
+self.check_polarization('vertical:low')
+self.check_topology(0, 0, 0, 0, 'low', False)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium'})
+self.assertEqual(res['return'], {})
+self.check_polarization('vertical:medium')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'high'})
+self.assertEqual(res['return'], {})
+self.check_polarization('vertical:high')
+self.check_topology(0, 0, 0, 0, 'high', False)
+
+self.guest_set_dispatching('0');
+self.check_polarization("horizontal")
+self.check_topology(0, 0, 0, 0, 'high', False)
--
2.39.2
[PATCH v24 07/21] s390x/cpu topology: interception of PTF instruction
From: Pierre Morel
When the host supports the CPU topology facility, the PTF
instruction with function code 2 is interpreted by the SIE,
provided that the userland hypervisor activates the interpretation
by using the KVM_CAP_S390_CPU_TOPOLOGY KVM extension.
The PTF instructions with function code 0 and 1 are intercepted
and must be emulated by the userland hypervisor.
During RESET all CPU of the configuration are placed in
horizontal polarity.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
include/hw/s390x/s390-virtio-ccw.h | 6
hw/s390x/cpu-topology.c| 55 ++
target/s390x/kvm/kvm.c | 11 ++
3 files changed, 72 insertions(+)
diff --git a/include/hw/s390x/s390-virtio-ccw.h
b/include/hw/s390x/s390-virtio-ccw.h
index 9bba21a916..c1d46e78af 100644
--- a/include/hw/s390x/s390-virtio-ccw.h
+++ b/include/hw/s390x/s390-virtio-ccw.h
@@ -30,6 +30,12 @@ struct S390CcwMachineState {
uint8_t loadparm[8];
};
+#define S390_PTF_REASON_NONE (0x00 << 8)
+#define S390_PTF_REASON_DONE (0x01 << 8)
+#define S390_PTF_REASON_BUSY (0x02 << 8)
+#define S390_TOPO_FC_MASK 0xffUL
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra);
+
struct S390CcwMachineClass {
/*< private >*/
MachineClass parent_class;
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 03de3d3d10..c34581964f 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -90,6 +90,60 @@ static void s390_topology_init(MachineState *ms)
smp->books * smp->drawers);
}
+/*
+ * s390_handle_ptf:
+ *
+ * @register 1: contains the function code
+ *
+ * Function codes 0 (horizontal) and 1 (vertical) define the CPU
+ * polarization requested by the guest.
+ *
+ * Function code 2 is handling topology changes and is interpreted
+ * by the SIE.
+ */
+void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
+{
+CpuS390Polarization polarization;
+CPUS390XState *env = >env;
+uint64_t reg = env->regs[r1];
+int fc = reg & S390_TOPO_FC_MASK;
+
+if (!s390_has_feat(S390_FEAT_CONFIGURATION_TOPOLOGY)) {
+s390_program_interrupt(env, PGM_OPERATION, ra);
+return;
+}
+
+if (env->psw.mask & PSW_MASK_PSTATE) {
+s390_program_interrupt(env, PGM_PRIVILEGED, ra);
+return;
+}
+
+if (reg & ~S390_TOPO_FC_MASK) {
+s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+return;
+}
+
+polarization = S390_CPU_POLARIZATION_VERTICAL;
+switch (fc) {
+case 0:
+polarization = S390_CPU_POLARIZATION_HORIZONTAL;
+/* fallthrough */
+case 1:
+if (s390_topology.polarization == polarization) {
+env->regs[r1] |= S390_PTF_REASON_DONE;
+setcc(cpu, 2);
+} else {
+s390_topology.polarization = polarization;
+s390_cpu_topology_set_changed(true);
+setcc(cpu, 0);
+}
+break;
+default:
+/* Note that fc == 2 is interpreted by the SIE */
+s390_program_interrupt(env, PGM_SPECIFICATION, ra);
+}
+}
+
/**
* s390_topology_reset:
*
@@ -99,6 +153,7 @@ static void s390_topology_init(MachineState *ms)
void s390_topology_reset(void)
{
s390_cpu_topology_set_changed(false);
+s390_topology.polarization = S390_CPU_POLARIZATION_HORIZONTAL;
}
/**
diff --git a/target/s390x/kvm/kvm.c b/target/s390x/kvm/kvm.c
index d6bda3a2a8..4d84dcc018 100644
--- a/target/s390x/kvm/kvm.c
+++ b/target/s390x/kvm/kvm.c
@@ -86,6 +86,7 @@
#define PRIV_B9_EQBS0x9c
#define PRIV_B9_CLP 0xa0
+#define PRIV_B9_PTF 0xa2
#define PRIV_B9_PCISTG 0xd0
#define PRIV_B9_PCILG 0xd2
#define PRIV_B9_RPCIT 0xd3
@@ -1457,6 +1458,13 @@ static int kvm_mpcifc_service_call(S390CPU *cpu, struct
kvm_run *run)
}
}
+static void kvm_handle_ptf(S390CPU *cpu, struct kvm_run *run)
+{
+uint8_t r1 = (run->s390_sieic.ipb >> 20) & 0x0f;
+
+s390_handle_ptf(cpu, r1, RA_IGNORED);
+}
+
static int handle_b9(S390CPU *cpu, struct kvm_run *run, uint8_t ipa1)
{
int r = 0;
@@ -1474,6 +1482,9 @@ static int handle_b9(S390CPU *cpu, struct kvm_run *run,
uint8_t ipa1)
case PRIV_B9_RPCIT:
r = kvm_rpcit_service_call(cpu, run);
break;
+case PRIV_B9_PTF:
+kvm_handle_ptf(cpu, run);
+break;
case PRIV_B9_EQBS:
/* just inject exception */
r = -1;
--
2.39.2
[PATCH v24 18/21] tests/avocado: s390x cpu topology test dedicated CPU
From: Pierre Morel
A dedicated CPU in vertical polarization can only have
a high entitlement.
Let's check this from both host and guest point of view.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
tests/avocado/s390_topology.py | 33 +
1 file changed, 33 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 24fac9a54d..3661048f4c 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -305,3 +305,36 @@ def test_entitlement(self):
self.guest_set_dispatching('0');
self.check_polarization("horizontal")
self.check_topology(0, 0, 0, 0, 'high', False)
+
+
+def test_dedicated(self):
+"""
+This test verifies that QEMU adjusts the entitlement correctly when a
+CPU is made dedicated.
+QEMU retains the entitlement value when horizontal polarization is in
effect.
+For the guest, the field shows the effective value of the entitlement.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+self.check_polarization("horizontal")
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+self.assertEqual(res['return'], {})
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("horizontal")
+
+self.guest_set_dispatching('1');
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("vertical:high")
+
+self.guest_set_dispatching('0');
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("horizontal")
--
2.39.2
[PATCH v24 01/21] qapi: machine.json: change docs regarding CpuInstanceProperties
Clarify roles of different architectures.
Also change things a bit in anticipation of additional members being
added.
Suggested-by: Markus Armbruster
Signed-off-by: Nina Schoetterl-Glausch
---
Reference to s390x docs added in patch 14
qapi/machine.json | 17 +
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/qapi/machine.json b/qapi/machine.json
index a08b6576ca..3c074c9902 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -70,9 +70,9 @@
# @qom-path: path to the CPU object in the QOM tree
#
# @thread-id: ID of the underlying host thread
-#
-# @props: properties describing to which node/socket/core/thread
-# virtual CPU belongs to, provided if supported by board
+
+# @props: properties of type CpuInstanceProperties associated with a
+# virtual CPU, e.g. the socket id
#
# @target: the QEMU system emulation target, which determines which
# additional fields will be listed (since 3.0)
@@ -899,6 +899,9 @@
# should be passed by management with device_add command when a CPU is
# being hotplugged.
#
+# Which members are optional and which mandatory depends on the architecture
+# and board.
+#
# @node-id: NUMA node ID the CPU belongs to
#
# @socket-id: socket number within node/board the CPU belongs to
@@ -912,15 +915,13 @@
#
# @thread-id: thread number within core the CPU belongs to
#
-# Note: currently there are 6 properties that could be present but
-# management should be prepared to pass through other properties
-# with device_add command to allow for future interface extension.
-# This also requires the filed names to be kept in sync with the
-# properties passed to -device/device_add.
+# Note: management should be prepared to pass through additional
+# properties with device_add.
#
# Since: 2.7
##
{ 'struct': 'CpuInstanceProperties',
+ # Keep these in sync with the properties device_add accepts
'data': { '*node-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
--
2.39.2
[PATCH v24 15/21] tests/avocado: s390x cpu topology core
From: Pierre Morel
Introduction of the s390x cpu topology core functions and
basic tests.
We test the correlation between the command line and
the QMP results in query-cpus-fast for various CPU topology.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS| 1 +
tests/avocado/s390_topology.py | 200 +
2 files changed, 201 insertions(+)
create mode 100644 tests/avocado/s390_topology.py
diff --git a/MAINTAINERS b/MAINTAINERS
index c31cec0c26..2798d02448 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1699,6 +1699,7 @@ F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/stsi-topology.c
F: docs/devel/s390-cpu-topology.rst
F: docs/system/s390x/cpu-topology.rst
+F: tests/avocado/s390_topology.py
X86 Machines
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
new file mode 100644
index 00..9078b45281
--- /dev/null
+++ b/tests/avocado/s390_topology.py
@@ -0,0 +1,200 @@
+# Functional test that boots a Linux kernel and checks the console
+#
+# Copyright IBM Corp. 2023
+#
+# Author:
+# Pierre Morel
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or
+# later. See the COPYING file in the top-level directory.
+
+import os
+import shutil
+import time
+
+from avocado_qemu import QemuSystemTest
+from avocado_qemu import exec_command
+from avocado_qemu import exec_command_and_wait_for_pattern
+from avocado_qemu import interrupt_interactive_console_until_pattern
+from avocado_qemu import wait_for_console_pattern
+from avocado.utils import process
+from avocado.utils import archive
+
+
+class S390CPUTopology(QemuSystemTest):
+"""
+S390x CPU topology consists of 4 topology layers, from bottom to top,
+the cores, sockets, books and drawers and 2 modifiers attributes,
+the entitlement and the dedication.
+See: docs/system/s390x/cpu-topology.rst.
+
+S390x CPU topology is setup in different ways:
+- implicitly from the '-smp' argument by completing each topology
+ level one after the other beginning with drawer 0, book 0 and
+ socket 0.
+- explicitly from the '-device' argument on the QEMU command line
+- explicitly by hotplug of a new CPU using QMP or HMP
+- it is modified by using QMP 'set-cpu-topology'
+
+The S390x modifier attribute entitlement depends on the machine
+polarization, which can be horizontal or vertical.
+The polarization is changed on a request from the guest.
+"""
+timeout = 90
+
+KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
+ 'root=/dev/ram '
+ 'selinux=0 '
+ 'rdinit=/bin/sh')
+
+def wait_until_booted(self):
+wait_for_console_pattern(self, 'no job control',
+ failure_message='Kernel panic - not syncing',
+ vm=None)
+
+def check_topology(self, c, s, b, d, e, t):
+res = self.vm.qmp('query-cpus-fast')
+cpus = res['return']
+for cpu in cpus:
+core = cpu['props']['core-id']
+socket = cpu['props']['socket-id']
+book = cpu['props']['book-id']
+drawer = cpu['props']['drawer-id']
+entitlement = cpu.get('entitlement')
+dedicated = cpu.get('dedicated')
+if core == c:
+self.assertEqual(drawer, d)
+self.assertEqual(book, b)
+self.assertEqual(socket, s)
+self.assertEqual(entitlement, e)
+self.assertEqual(dedicated, t)
+
+def kernel_init(self):
+"""
+We need a VM that supports CPU topology,
+currently this only the case when using KVM, not TCG.
+We need a kernel supporting the CPU topology.
+We need a minimal root filesystem with a shell.
+"""
+self.require_accelerator("kvm")
+kernel_url = ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/kernel.img')
+kernel_hash = '0d1aaaf303f07cf0160c8c48e56fe638'
+kernel_path = self.fetch_asset(kernel_url, algorithm='md5',
+ asset_hash=kernel_hash)
+
+initrd_url = ('https://archives.fedoraproject.org/pub/archive'
+ '/fedora-secondary/releases/35/Server/s390x/os'
+ '/images/initrd.img')
+initrd_hash = 'a122057d95725ac030e2ec51df46e172'
+initrd_path_xz = self.fetch_asset(initrd_url, algorithm='md5',
+ asset_hash=initrd_hash)
+initrd_path = os.path.join(self.workdir, 'initrd-raw.img')
+archive.lzma_
[PATCH v24 16/21] tests/avocado: s390x cpu topology polarization
From: Pierre Morel
Polarization is changed on a request from the guest.
Let's verify the polarization is accordingly set by QEMU.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
tests/avocado/s390_topology.py | 45 ++
1 file changed, 45 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 9078b45281..8166cee134 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -41,6 +41,7 @@ class S390CPUTopology(QemuSystemTest):
The polarization is changed on a request from the guest.
"""
timeout = 90
+event_timeout = 10
KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
'root=/dev/ram '
@@ -103,6 +104,14 @@ def kernel_init(self):
'-initrd', initrd_path,
'-append', kernel_command_line)
+def system_init(self):
+self.log.info("System init")
+exec_command_and_wait_for_pattern(self,
+""" mount proc -t proc /proc;
+mount sys -t sysfs /sys;
+cat /sys/devices/system/cpu/dispatching """,
+'0')
+
def test_single(self):
"""
This test checks the simplest topology with a single CPU.
@@ -198,3 +207,39 @@ def test_dash_device(self):
self.check_topology(3, 1, 1, 1, 'high', False)
self.check_topology(4, 1, 1, 1, 'medium', False)
self.check_topology(5, 2, 1, 1, 'high', True)
+
+
+def guest_set_dispatching(self, dispatching):
+exec_command(self,
+f'echo {dispatching} > /sys/devices/system/cpu/dispatching')
+self.vm.event_wait('CPU_POLARIZATION_CHANGE', self.event_timeout)
+exec_command_and_wait_for_pattern(self,
+'cat /sys/devices/system/cpu/dispatching', dispatching)
+
+
+def test_polarization(self):
+"""
+This test verifies that QEMU modifies the entitlement change after
+several guest polarization change requests.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'horizontal')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+self.guest_set_dispatching('1');
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'vertical')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+self.guest_set_dispatching('0');
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'horizontal')
+self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.39.2
[PATCH v24 14/21] docs/s390x/cpu topology: document s390x cpu topology
From: Pierre Morel
Add some basic examples for the definition of cpu topology
in s390x.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS| 2 +
docs/devel/index-internals.rst | 1 +
docs/devel/s390-cpu-topology.rst | 170
docs/system/s390x/cpu-topology.rst | 244 +
docs/system/target-s390x.rst | 1 +
qapi/machine.json | 2 +
6 files changed, 420 insertions(+)
create mode 100644 docs/devel/s390-cpu-topology.rst
create mode 100644 docs/system/s390x/cpu-topology.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index 447bf8ac20..c31cec0c26 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1697,6 +1697,8 @@ S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/stsi-topology.c
+F: docs/devel/s390-cpu-topology.rst
+F: docs/system/s390x/cpu-topology.rst
X86 Machines
diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst
index e1a93df263..6f81df92bc 100644
--- a/docs/devel/index-internals.rst
+++ b/docs/devel/index-internals.rst
@@ -14,6 +14,7 @@ Details about QEMU's various subsystems including how to add
features to them.
migration
multi-process
reset
+ s390-cpu-topology
s390-dasd-ipl
tracing
vfio-migration
diff --git a/docs/devel/s390-cpu-topology.rst b/docs/devel/s390-cpu-topology.rst
new file mode 100644
index 00..9eab28d5e5
--- /dev/null
+++ b/docs/devel/s390-cpu-topology.rst
@@ -0,0 +1,170 @@
+QAPI interface for S390 CPU topology
+
+
+The following sections will explain the QAPI interface for S390 CPU topology
+with the help of exemplary output.
+For this, let's assume that QEMU has been started with the following
+command, defining 4 CPUs, where CPU[0] is defined by the -smp argument and will
+have default values:
+
+.. code-block:: bash
+
+ qemu-system-s390x \
+-enable-kvm \
+-cpu z14,ctop=on \
+-smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
+-device z14-s390x-cpu,core-id=19,entitlement=high \
+-device z14-s390x-cpu,core-id=11,entitlement=low \
+-device z14-s390x-cpu,core-id=112,entitlement=high \
+ ...
+
+Additions to query-cpus-fast
+
+
+The command query-cpus-fast allows querying the topology tree and
+modifiers for all configured vCPUs.
+
+.. code-block:: QMP
+
+ { "execute": "query-cpus-fast" }
+ {
+ "return": [
+{
+ "dedicated": false,
+ "thread-id": 536993,
+ "props": {
+"core-id": 0,
+"socket-id": 0,
+"drawer-id": 0,
+"book-id": 0
+ },
+ "cpu-state": "operating",
+ "entitlement": "medium",
+ "qom-path": "/machine/unattached/device[0]",
+ "cpu-index": 0,
+ "target": "s390x"
+},
+{
+ "dedicated": false,
+ "thread-id": 537003,
+ "props": {
+"core-id": 19,
+"socket-id": 1,
+"drawer-id": 0,
+"book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[0]",
+ "cpu-index": 19,
+ "target": "s390x"
+},
+{
+ "dedicated": false,
+ "thread-id": 537004,
+ "props": {
+"core-id": 11,
+"socket-id": 1,
+"drawer-id": 0,
+"book-id": 1
+ },
+ "cpu-state": "operating",
+ "entitlement": "low",
+ "qom-path": "/machine/peripheral-anon/device[1]",
+ "cpu-index": 11,
+ "target": "s390x"
+},
+{
+ "dedicated": true,
+ "thread-id": 537005,
+ "props": {
+"core-id": 112,
+"socket-id": 0,
+"drawer-id": 3,
+"book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[2]",
+ "cpu-index": 112,
+ "target": "s390x"
+}
+ ]
+ }
+
+
+QAPI command: set-cpu-topology
+--
+
+The command set-cpu-topology allows modifying the topology tree
+or the topology modifiers of a vCPU in the configuration.
+
+.. code-block:: QMP
+
+{ "execute": "set-cpu-topology",
+ &
[PATCH v24 20/21] tests/avocado: s390x cpu topology dedicated errors
From: Pierre Morel
Let's test that QEMU refuses to setup a dedicated CPU with
low or medium entitlement.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
tests/avocado/s390_topology.py | 48 ++
1 file changed, 48 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index a63c2b2923..d3e6556c0f 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -364,3 +364,51 @@ def test_socket_full(self):
res = self.vm.qmp('set-cpu-topology',
{'core-id': 2, 'socket-id': 0, 'book-id': 1})
self.assertEqual(res['return'], {})
+
+def test_dedicated_error(self):
+"""
+This test verifies that QEMU refuses to lower the entitlement
+of a dedicated CPU
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+self.assertEqual(res['return'], {})
+
+self.check_topology(0, 0, 0, 0, 'high', True)
+
+self.guest_set_dispatching('1');
+
+self.check_topology(0, 0, 0, 0, 'high', True)
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low', 'dedicated':
True})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low'})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium', 'dedicated':
True})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium'})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'low', 'dedicated':
False})
+self.assertEqual(res['return'], {})
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'entitlement': 'medium', 'dedicated':
False})
+self.assertEqual(res['return'], {})
--
2.39.2
[PATCH v24 11/21] machine: adding s390 topology to info hotpluggable-cpus
From: Pierre Morel
S390 topology adds books and drawers topology containers.
Let's add these to the HMP information for hotpluggable cpus.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
---
hw/core/machine-hmp-cmds.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c
index c3e55ef9e9..9a4b59c6f2 100644
--- a/hw/core/machine-hmp-cmds.c
+++ b/hw/core/machine-hmp-cmds.c
@@ -71,6 +71,12 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)
if (c->has_node_id) {
monitor_printf(mon, "node-id: \"%" PRIu64 "\"\n", c->node_id);
}
+if (c->has_drawer_id) {
+monitor_printf(mon, "drawer-id: \"%" PRIu64 "\"\n",
c->drawer_id);
+}
+if (c->has_book_id) {
+monitor_printf(mon, "book-id: \"%" PRIu64 "\"\n", c->book_id);
+}
if (c->has_socket_id) {
monitor_printf(mon, "socket-id: \"%" PRIu64 "\"\n",
c->socket_id);
}
--
2.39.2
[PATCH v24 02/21] CPU topology: extend with s390 specifics
From: Pierre Morel
S390 adds two new SMP levels, drawers and books to the CPU
topology.
S390 CPUs have specific topology features like dedication and
entitlement. These indicate to the guest information on host
vCPU scheduling and help the guest make better scheduling decisions.
Let us provide the SMP properties with books and drawers levels
and S390 CPU with dedication and entitlement,
Add machine-common.json so we can later include it in
machine-target.json also.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-common.json| 21 +
qapi/machine.json | 17 +-
qapi/qapi-schema.json | 1 +
include/hw/boards.h | 10 +-
include/hw/qdev-properties-system.h | 4 +++
target/s390x/cpu.h | 6
hw/core/machine-smp.c | 48 -
hw/core/machine.c | 4 +++
hw/core/qdev-properties-system.c| 13
hw/s390x/s390-virtio-ccw.c | 4 +++
softmmu/vl.c| 6
target/s390x/cpu.c | 7 +
qapi/meson.build| 1 +
qemu-options.hx | 7 +++--
14 files changed, 138 insertions(+), 11 deletions(-)
create mode 100644 qapi/machine-common.json
diff --git a/qapi/machine-common.json b/qapi/machine-common.json
new file mode 100644
index 00..fa6bd71d12
--- /dev/null
+++ b/qapi/machine-common.json
@@ -0,0 +1,21 @@
+# -*- Mode: Python -*-
+# vim: filetype=python
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+##
+# = Machines S390 data types
+##
+
+##
+# @CpuS390Entitlement:
+#
+# An enumeration of CPU entitlements that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.2
+##
+{ 'enum': 'CpuS390Entitlement',
+ 'prefix': 'S390_CPU_ENTITLEMENT',
+ 'data': [ 'auto', 'low', 'medium', 'high' ] }
diff --git a/qapi/machine.json b/qapi/machine.json
index 3c074c9902..8499d40269 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -9,6 +9,7 @@
##
{ 'include': 'common.json' }
+{ 'include': 'machine-common.json' }
##
# @SysEmuTarget:
@@ -904,7 +905,13 @@
#
# @node-id: NUMA node ID the CPU belongs to
#
-# @socket-id: socket number within node/board the CPU belongs to
+# @drawer-id: drawer number within node/board the CPU belongs to
+# (since 8.2)
+#
+# @book-id: book number within drawer/node/board the CPU belongs to
+# (since 8.2)
+#
+# @socket-id: socket number within book/node/board the CPU belongs to
#
# @die-id: die number within socket the CPU belongs to (since 4.1)
#
@@ -923,6 +930,8 @@
{ 'struct': 'CpuInstanceProperties',
# Keep these in sync with the properties device_add accepts
'data': { '*node-id': 'int',
+'*drawer-id': 'int',
+'*book-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
'*cluster-id': 'int',
@@ -1481,6 +1490,10 @@
#
# @cpus: number of virtual CPUs in the virtual machine
#
+# @drawers: number of drawers in the CPU topology (since 8.2)
+#
+# @books: number of books in the CPU topology (since 8.2)
+#
# @sockets: number of sockets in the CPU topology
#
# @dies: number of dies per socket in the CPU topology
@@ -1499,6 +1512,8 @@
##
{ 'struct': 'SMPConfiguration', 'data': {
'*cpus': 'int',
+ '*drawers': 'int',
+ '*books': 'int',
'*sockets': 'int',
'*dies': 'int',
'*clusters': 'int',
diff --git a/qapi/qapi-schema.json b/qapi/qapi-schema.json
index 6594afba31..c01ec335e6 100644
--- a/qapi/qapi-schema.json
+++ b/qapi/qapi-schema.json
@@ -66,6 +66,7 @@
{ 'include': 'introspect.json' }
{ 'include': 'qom.json' }
{ 'include': 'qdev.json' }
+{ 'include': 'machine-common.json' }
{ 'include': 'machine.json' }
{ 'include': 'machine-target.json' }
{ 'include': 'replay.json' }
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 6c67af196a..6dcfc879eb 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -134,12 +134,16 @@ typedef struct {
* @clusters_supported - whether clusters are supported by the machine
* @has_clusters - whether clusters are explicitly specified in the user
* provided SMP configuration
+ * @books_supported - whether books are supported by the machine
+ * @drawers_supported - whether drawers are supported by the machine
*/
typedef struct {
bool prefer_sockets;
bool dies_supported;
bool clusters_supported;
bool has_clusters;
+bool books_supported;
+bool drawers_supported;
} SMPCompatProps;
/**
@@ -310,7 +314,9 @@ typedef struct DeviceMemoryState {
/**
* CpuTopology:
* @cpus: the number of present logical processors on the machine
- * @sockets: the number of sockets on the machine
+ * @drawers: the number of drawers
[PATCH v24 00/21] s390x: CPU Topology
nce v4
- Added drawer and books to topology
- Added numa topology
- Added documentation
- since v3
- Added migration
(Thomas)
- Separated STSI instruction from KVM to prepare TCG
(Thomas)
- Take care of endianess to prepare TCG
(Thomas)
- Added comments on STSI CPU container and PFT instruction
(Thomas)
- Moved enabling the instructions as the last patch
(Thomas)
Nina Schoetterl-Glausch (1):
qapi: machine.json: change docs regarding CpuInstanceProperties
Pierre Morel (20):
CPU topology: extend with s390 specifics
s390x/cpu topology: add topology entries on CPU hotplug
target/s390x/cpu topology: handle STSI(15) and build the SYSIB
s390x/sclp: reporting the maximum nested topology entries
s390x/cpu topology: resetting the Topology-Change-Report
s390x/cpu topology: interception of PTF instruction
target/s390x/cpu topology: activate CPU topology
qapi/s390x/cpu topology: set-cpu-topology qmp command
machine: adding s390 topology to query-cpu-fast
machine: adding s390 topology to info hotpluggable-cpus
qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
qapi/s390x/cpu topology: add query-s390x-cpu-polarization command
docs/s390x/cpu topology: document s390x cpu topology
tests/avocado: s390x cpu topology core
tests/avocado: s390x cpu topology polarization
tests/avocado: s390x cpu topology entitlement tests
tests/avocado: s390x cpu topology test dedicated CPU
tests/avocado: s390x cpu topology test socket full
tests/avocado: s390x cpu topology dedicated errors
tests/avocado: s390x cpu topology bad move
MAINTAINERS | 10 +
docs/devel/index-internals.rst | 1 +
docs/devel/s390-cpu-topology.rst| 170 ++
docs/system/s390x/cpu-topology.rst | 244 +++
docs/system/target-s390x.rst| 1 +
qapi/machine-common.json| 21 ++
qapi/machine-target.json| 120 +++
qapi/machine.json | 47 ++-
qapi/qapi-schema.json | 1 +
include/hw/boards.h | 10 +-
include/hw/qdev-properties-system.h | 4 +
include/hw/s390x/cpu-topology.h | 78 +
include/hw/s390x/s390-virtio-ccw.h | 6 +
include/hw/s390x/sclp.h | 4 +-
target/s390x/cpu.h | 82 +
target/s390x/kvm/kvm_s390x.h| 1 +
hw/core/machine-hmp-cmds.c | 6 +
hw/core/machine-smp.c | 48 ++-
hw/core/machine.c | 4 +
hw/core/qdev-properties-system.c| 13 +
hw/s390x/cpu-topology.c | 469
hw/s390x/s390-virtio-ccw.c | 29 +-
hw/s390x/sclp.c | 5 +
softmmu/vl.c| 6 +
target/s390x/cpu-sysemu.c | 13 +
target/s390x/cpu.c | 16 +
target/s390x/cpu_models.c | 1 +
target/s390x/kvm/kvm.c | 42 ++-
target/s390x/kvm/stsi-topology.c| 339
hw/s390x/meson.build| 1 +
qapi/meson.build| 1 +
qemu-options.hx | 7 +-
target/s390x/kvm/meson.build| 3 +-
tests/avocado/s390_topology.py | 439 ++
34 files changed, 2217 insertions(+), 25 deletions(-)
create mode 100644 docs/devel/s390-cpu-topology.rst
create mode 100644 docs/system/s390x/cpu-topology.rst
create mode 100644 qapi/machine-common.json
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 hw/s390x/cpu-topology.c
create mode 100644 target/s390x/kvm/stsi-topology.c
create mode 100644 tests/avocado/s390_topology.py
Range-diff against v23:
-: -- > 1: 2c7552ab60 qapi: machine.json: change docs regarding
CpuInstanceProperties
1: f2219b7583 ! 2: 3ea5ee1016 CPU topology: extend with s390 specifics
@@ Commit message
Let us provide the SMP properties with books and drawers levels
and S390 CPU with dedication and entitlement,
+Add machine-common.json so we can later include it in
+machine-target.json also.
+
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
@@ qapi/machine-common.json (new)
+##
+# @CpuS390Entitlement:
+#
-+# An enumeration of cpu entitlements that can be assumed by a virtual
++# An enumeration of CPU entitlements that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.2
@@ qapi/machine.json
##
# @SysEmuTarget:
-@@
- #
- # @thread-id: ID of the underlying host thread
- #
--# @props: properties describing to which node/socket/core/thread
-+# @props: properties describing to which
node/drawer/book/socket/core/thread
- # virtual CPU belongs to, provided if supported by board
- #
- # @target: the QEMU system emulation target, which determines which
@@
#
# @node-id: NUMA n
Re: [PATCH v23 01/20] CPU topology: extend with s390 specifics
On Tue, 2023-09-19 at 14:47 +0200, Markus Armbruster wrote:
> Nina Schoetterl-Glausch writes:
>
> > From: Pierre Morel
> >
> > S390 adds two new SMP levels, drawers and books to the CPU
> > topology.
> > S390 CPUs have specific topology features like dedication and
> > entitlement. These indicate to the guest information on host
> > vCPU scheduling and help the guest make better scheduling decisions.
> >
> > Let us provide the SMP properties with books and drawers levels
> > and S390 CPU with dedication and entitlement,
> >
> > Signed-off-by: Pierre Morel
> > Reviewed-by: Nina Schoetterl-Glausch
> > Co-developed-by: Nina Schoetterl-Glausch
> > Signed-off-by: Nina Schoetterl-Glausch
> > ---
> > qapi/machine-common.json| 21 +
> > qapi/machine.json | 19 ++--
> > include/hw/boards.h | 10 +-
> > include/hw/qdev-properties-system.h | 4 +++
> > target/s390x/cpu.h | 6
> > hw/core/machine-smp.c | 48 -
> > hw/core/machine.c | 4 +++
> > hw/core/qdev-properties-system.c| 13
> > hw/s390x/s390-virtio-ccw.c | 4 +++
> > softmmu/vl.c| 6
> > target/s390x/cpu.c | 7 +
> > qapi/meson.build| 1 +
> > qemu-options.hx | 7 +++--
> > 13 files changed, 137 insertions(+), 13 deletions(-)
> > create mode 100644 qapi/machine-common.json
> >
> > diff --git a/qapi/machine-common.json b/qapi/machine-common.json
> > new file mode 100644
> > index 00..e40421bb37
> > --- /dev/null
> > +++ b/qapi/machine-common.json
>
> Why do you need a separate QAPI sub-module?
>
> > @@ -0,0 +1,21 @@
> > +# -*- Mode: Python -*-
> > +# vim: filetype=python
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> > +# See the COPYING file in the top-level directory.
> > +
> > +##
> > +# = Machines S390 data types
> > +##
> > +
> > +##
> > +# @CpuS390Entitlement:
> > +#
> > +# An enumeration of cpu entitlements that can be assumed by a virtual
> > +# S390 CPU
> > +#
> > +# Since: 8.2
> > +##
> > +{ 'enum': 'CpuS390Entitlement',
> > + 'prefix': 'S390_CPU_ENTITLEMENT',
> > + 'data': [ 'auto', 'low', 'medium', 'high' ] }
> > diff --git a/qapi/machine.json b/qapi/machine.json
> > index a08b6576ca..a63cb951d2 100644
> > --- a/qapi/machine.json
> > +++ b/qapi/machine.json
> > @@ -9,6 +9,7 @@
>##
># = Machines
> > ##
> >
> > { 'include': 'common.json' }
> > +{ 'include': 'machine-common.json' }
>
> Section structure is borked :)
>
> Existing section "Machine" now ends at the new "Machines S390 data
> types" you pull in here. The contents of below moves from "Machines" to
> "Machines S390 data types".
>
> Before I explain how to avoid this, I'd like to understand why we need a
> new sub-module.
Should I just move the include statements above the section header?
I assume I could also include it in qapi-schema.json before the machine.json
include and that that is the reason we don't have
the same problem with e.g. migration.json.
But just moving the includes seems cleaner.
[...]
Re: [PATCH v23 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command
On Wed, 2023-09-20 at 13:36 +0200, Markus Armbruster wrote:
> Nina Schoetterl-Glausch writes:
>
> > From: Pierre Morel
> >
> > The modification of the CPU attributes are done through a monitor
> > command.
> >
> > It allows to move the core inside the topology tree to optimize
> > the cache usage in the case the host's hypervisor previously
> > moved the CPU.
> >
> > The same command allows to modify the CPU attributes modifiers
> > like polarization entitlement and the dedicated attribute to notify
> > the guest if the host admin modified scheduling or dedication of a vCPU.
> >
> > With this knowledge the guest has the possibility to optimize the
> > usage of the vCPUs.
> >
> > The command has a feature unstable for the moment.
> >
> > Signed-off-by: Pierre Morel
> > Reviewed-by: Nina Schoetterl-Glausch
> > Co-developed-by: Nina Schoetterl-Glausch
> > Signed-off-by: Nina Schoetterl-Glausch
> > ---
> > qapi/machine-target.json | 37 +++
> > hw/s390x/cpu-topology.c | 132 +++
> > 2 files changed, 169 insertions(+)
> >
> > diff --git a/qapi/machine-target.json b/qapi/machine-target.json
> > index 0d45a590ce..e47a252bd9 100644
> > --- a/qapi/machine-target.json
> > +++ b/qapi/machine-target.json
[...]
> > +# Modifies the topology by moving the CPU inside the topology
> > +# tree or by changing a modifier attribute of a CPU.
> > +# Default value for optional parameter is the current value
> > +# used by the CPU.
>
> So, anything absent will not be changed. Maybe that's a clearer way to
> put it. What do you think?
Yes.
>
> > +#
> > +# Returns: Nothing on success, the reason on failure.
> > +#
> > +# Since: 8.2
> > +##
> > +{ 'command': 'set-cpu-topology',
> > + 'data': {
> > + 'core-id': 'uint16',
> > + '*socket-id': 'uint16',
> > + '*book-id': 'uint16',
> > + '*drawer-id': 'uint16',
>
> CpuInstanceProperties uses 'int' for these. Any particular reason for
> the difference?
unsigned -> no need to check if >0
16bit is also the width the hardware uses for these values on s390.
> > + '*entitlement': 'CpuS390Entitlement',
> > + '*dedicated': 'bool'
> > + },
> > + 'features': [ 'unstable' ],
> > + 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
> > +}
>
> [...]
>
Re: [PATCH v23 01/20] CPU topology: extend with s390 specifics
On Wed, 2023-09-20 at 13:11 +0200, Markus Armbruster wrote:
> Nina Schoetterl-Glausch writes:
>
> > From: Pierre Morel
> >
> > S390 adds two new SMP levels, drawers and books to the CPU
> > topology.
> > S390 CPUs have specific topology features like dedication and
> > entitlement. These indicate to the guest information on host
> > vCPU scheduling and help the guest make better scheduling decisions.
> >
> > Let us provide the SMP properties with books and drawers levels
> > and S390 CPU with dedication and entitlement,
> >
> > Signed-off-by: Pierre Morel
> > Reviewed-by: Nina Schoetterl-Glausch
> > Co-developed-by: Nina Schoetterl-Glausch
> > Signed-off-by: Nina Schoetterl-Glausch
> > diff --git a/qapi/machine-common.json b/qapi/machine-common.json
> > new file mode 100644
> > index 00..e40421bb37
> > --- /dev/null
> > +++ b/qapi/machine-common.json
> > @@ -0,0 +1,21 @@
> > +# -*- Mode: Python -*-
> > +# vim: filetype=python
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> > +# See the COPYING file in the top-level directory.
> > +
> > +##
> > +# = Machines S390 data types
> > +##
> > +
> > +##
> > +# @CpuS390Entitlement:
> > +#
> > +# An enumeration of cpu entitlements that can be assumed by a virtual
> > +# S390 CPU
>
> CPU entitlements
>
> Would someone reasonably familiar with S390 understand this? Because
Well, someone familiar with s390 topology would, otherwise probably not tbh.
> I'm not and I don't; I wonder what "a virtual CPU assuming an
> entitlement" means.
Basically, on s390x the OS is always running on some hypervisor.
Even without KVM or z/VM you can slice up the machine, namely into logical
partitions (LPARs). Therefore, there is a scheduling of virtual CPUs to the
actual physical ones. "Entitlement" is a statement about how that scheduling
works for a virtual CPU. The same concepts can then also be applied to KVM.
>
> > +#
> > +# Since: 8.2
> > +##
> > +{ 'enum': 'CpuS390Entitlement',
> > + 'prefix': 'S390_CPU_ENTITLEMENT',
> > + 'data': [ 'auto', 'low', 'medium', 'high' ] }
>
> [...]
>
Re: [PATCH v23 01/20] CPU topology: extend with s390 specifics
On Wed, 2023-09-20 at 12:57 +0200, Markus Armbruster wrote:
> Nina Schoetterl-Glausch writes:
>
> > On Tue, 2023-09-19 at 14:47 +0200, Markus Armbruster wrote:
> > > Nina Schoetterl-Glausch writes:
> > >
> > > > From: Pierre Morel
> > > >
> > > > S390 adds two new SMP levels, drawers and books to the CPU
> > > > topology.
> > > > S390 CPUs have specific topology features like dedication and
> > > > entitlement. These indicate to the guest information on host
> > > > vCPU scheduling and help the guest make better scheduling decisions.
> > > >
> > > > Let us provide the SMP properties with books and drawers levels
> > > > and S390 CPU with dedication and entitlement,
> > > >
> > > > Signed-off-by: Pierre Morel
> > > > Reviewed-by: Nina Schoetterl-Glausch
> > > > Co-developed-by: Nina Schoetterl-Glausch
> > > > Signed-off-by: Nina Schoetterl-Glausch
> > > > ---
> > > > qapi/machine-common.json| 21 +
> > > > qapi/machine.json | 19 ++--
> > > > include/hw/boards.h | 10 +-
> > > > include/hw/qdev-properties-system.h | 4 +++
> > > > target/s390x/cpu.h | 6
> > > > hw/core/machine-smp.c | 48 -
> > > > hw/core/machine.c | 4 +++
> > > > hw/core/qdev-properties-system.c| 13
> > > > hw/s390x/s390-virtio-ccw.c | 4 +++
> > > > softmmu/vl.c| 6
> > > > target/s390x/cpu.c | 7 +
> > > > qapi/meson.build| 1 +
> > > > qemu-options.hx | 7 +++--
> > > > 13 files changed, 137 insertions(+), 13 deletions(-)
> > > > create mode 100644 qapi/machine-common.json
> > > >
> > > > diff --git a/qapi/machine-common.json b/qapi/machine-common.json
> > > > new file mode 100644
> > > > index 00..e40421bb37
> > > > --- /dev/null
> > > > +++ b/qapi/machine-common.json
> > >
> > > Why do you need a separate QAPI sub-module?
> >
> > See here
> > https://lore.kernel.org/qemu-devel/d8da6f7d1e3addcb63614f548ed77ac1b8895e63.ca...@linux.ibm.com/
>
> Quote:
>
> CpuS390Entitlement would be useful in both machine.json and
> machine-target.json
>
> This is not obvious from this patch. I figure this patch could add it
> to machine.json just fine. The use in machine-target.json in appears
> only in PATCH 08.
Want me to add the rational to the commit message?
>
> because query-cpu-fast is defined in machine.json and set-cpu-topology is
> defined
> in machine-target.json.
>
> So then the question is where best to define CpuS390Entitlement.
> In machine.json and include machine.json in machine-target.json?
> Or define it in another file and include it from both?
>
> You do the latter in this patch.
>
> I figure the former would be tolerable, too.
>
> That said, having target-specific stuff in machine.json feels... odd.
> Before this series, we have CpuInfoS390 and CpuS390State there, for
> query-cpus-fast. That command returns a list of objects where common
> members are target-independent, and the variant members are
> target-dependent. qmp_query_cpus_fast() uses a CPU method to populate
> the target-dependent members.
>
> I'm not sure splitting query-cpus-fast into a target-dependent and a
> target-independent part is worth the bother.
>
> In this patch, you work with the structure you found. Can't fault you
> for that :)
>
> > > > @@ -0,0 +1,21 @@
> > > > +# -*- Mode: Python -*-
> > > > +# vim: filetype=python
> > > > +#
> > > > +# This work is licensed under the terms of the GNU GPL, version 2 or
> > > > later.
> > > > +# See the COPYING file in the top-level directory.
> > > > +
> > > > +##
> > > > +# = Machines S390 data types
> > > > +##
> > > > +
> > > > +##
> > > > +# @CpuS390Entitlement:
> > > > +#
> > > > +# An enumeration of cpu entitlements that can be assumed by a virtual
> > > > +# S390 CPU
> > > > +#
> > > > +# Since: 8.2
> > > > +##
> > > > +{ 'enum': 'CpuS390Entitlement',
> > > > + 'prefix': 'S390_CPU_ENTITLEMENT',
> > &
Re: [PATCH v23 01/20] CPU topology: extend with s390 specifics
On Tue, 2023-09-19 at 14:47 +0200, Markus Armbruster wrote:
> Nina Schoetterl-Glausch writes:
>
> > From: Pierre Morel
> >
> > S390 adds two new SMP levels, drawers and books to the CPU
> > topology.
> > S390 CPUs have specific topology features like dedication and
> > entitlement. These indicate to the guest information on host
> > vCPU scheduling and help the guest make better scheduling decisions.
> >
> > Let us provide the SMP properties with books and drawers levels
> > and S390 CPU with dedication and entitlement,
> >
> > Signed-off-by: Pierre Morel
> > Reviewed-by: Nina Schoetterl-Glausch
> > Co-developed-by: Nina Schoetterl-Glausch
> > Signed-off-by: Nina Schoetterl-Glausch
> > ---
> > qapi/machine-common.json| 21 +
> > qapi/machine.json | 19 ++--
> > include/hw/boards.h | 10 +-
> > include/hw/qdev-properties-system.h | 4 +++
> > target/s390x/cpu.h | 6
> > hw/core/machine-smp.c | 48 -
> > hw/core/machine.c | 4 +++
> > hw/core/qdev-properties-system.c| 13
> > hw/s390x/s390-virtio-ccw.c | 4 +++
> > softmmu/vl.c| 6
> > target/s390x/cpu.c | 7 +
> > qapi/meson.build| 1 +
> > qemu-options.hx | 7 +++--
> > 13 files changed, 137 insertions(+), 13 deletions(-)
> > create mode 100644 qapi/machine-common.json
> >
> > diff --git a/qapi/machine-common.json b/qapi/machine-common.json
> > new file mode 100644
> > index 00..e40421bb37
> > --- /dev/null
> > +++ b/qapi/machine-common.json
>
> Why do you need a separate QAPI sub-module?
See here
https://lore.kernel.org/qemu-devel/d8da6f7d1e3addcb63614f548ed77ac1b8895e63.ca...@linux.ibm.com/
>
> > @@ -0,0 +1,21 @@
> > +# -*- Mode: Python -*-
> > +# vim: filetype=python
> > +#
> > +# This work is licensed under the terms of the GNU GPL, version 2 or later.
> > +# See the COPYING file in the top-level directory.
> > +
> > +##
> > +# = Machines S390 data types
> > +##
> > +
> > +##
> > +# @CpuS390Entitlement:
> > +#
> > +# An enumeration of cpu entitlements that can be assumed by a virtual
> > +# S390 CPU
> > +#
> > +# Since: 8.2
> > +##
> > +{ 'enum': 'CpuS390Entitlement',
> > + 'prefix': 'S390_CPU_ENTITLEMENT',
> > + 'data': [ 'auto', 'low', 'medium', 'high' ] }
> > diff --git a/qapi/machine.json b/qapi/machine.json
> > index a08b6576ca..a63cb951d2 100644
> > --- a/qapi/machine.json
> > +++ b/qapi/machine.json
> > @@ -9,6 +9,7 @@
>##
># = Machines
> > ##
> >
> > { 'include': 'common.json' }
> > +{ 'include': 'machine-common.json' }
>
> Section structure is borked :)
>
> Existing section "Machine" now ends at the new "Machines S390 data
> types" you pull in here. The contents of below moves from "Machines" to
> "Machines S390 data types".
>
> Before I explain how to avoid this, I'd like to understand why we need a
> new sub-module.
>
> >
> > ##
> > # @SysEmuTarget:
> > @@ -71,7 +72,7 @@
>##
># @CpuInfoFast:
>#
># Information about a virtual CPU
>#
># @cpu-index: index of the virtual CPU
>#
># @qom-path: path to the CPU object in the QOM tree
> > #
> > # @thread-id: ID of the underlying host thread
> > #
> > -# @props: properties describing to which node/socket/core/thread
> > +# @props: properties describing to which
> > node/drawer/book/socket/core/thread
> > # virtual CPU belongs to, provided if supported by board
>
> Is this description accurate?
Kinda, although the wording might not be the best.
All the CpuInstanceProperties fields are optional, it's like a superset of
possible
properties across architectures.
Only a subset might be returned by query-cpus-fast.
Also die and cluster are missing.
>
> @props is of type CpuInstanceProperties, shown below. Its documentation
> describes it as "properties to be used for hotplugging a CPU instance,
> it should be passed by management with device_add command when a CPU is
> being hotplugged." Hmm.
>
> I figure details ("node/drawer/book/socket/core/thread") are better left
> to CpuInstanceProperties.
>
> The "provided if supported by board" part makes no sense to me. If
> @props is there, it lists the
Re: [PATCH v23 03/20] target/s390x/cpu topology: handle STSI(15) and build the SYSIB
On Thu, 2023-09-14 at 14:06 +0200, Nina Schoetterl-Glausch wrote:
> From: Pierre Morel
>
> On interception of STSI(15.1.x) the System Information Block
> (SYSIB) is built from the list of pre-ordered topology entries.
>
> Signed-off-by: Pierre Morel
> Reviewed-by: Nina Schoetterl-Glausch
> Co-developed-by: Nina Schoetterl-Glausch
> Signed-off-by: Nina Schoetterl-Glausch
> ---
> MAINTAINERS | 1 +
> qapi/machine-target.json | 14 ++
> include/hw/s390x/cpu-topology.h | 23 +++
> include/hw/s390x/sclp.h | 1 +
> target/s390x/cpu.h | 75 +++
> hw/s390x/cpu-topology.c | 2 +
> target/s390x/kvm/kvm.c | 5 +-
> target/s390x/kvm/stsi-topology.c | 338 +++
> target/s390x/kvm/meson.build | 3 +-
> 9 files changed, 460 insertions(+), 2 deletions(-)
> create mode 100644 target/s390x/kvm/stsi-topology.c
[...]
> diff --git a/target/s390x/kvm/stsi-topology.c
> b/target/s390x/kvm/stsi-topology.c
> new file mode 100644
> index 00..22bac2b834
> --- /dev/null
> +++ b/target/s390x/kvm/stsi-topology.c
[...]
> +/**
> + * s390_topology_id_cmp:
> + * @l: first s390_topology_id
> + * @r: second s390_topology_id
> + *
> + * Compare two topology ids according to the sorting order specified by the
> PoP.
> + *
> + * Returns a negative number if the first id is less than, 0 if it is equal
> to
> + * and positive if it is larger than the second id.
> + */
> +static int s390_topology_id_cmp(const s390_topology_id *l,
> +const s390_topology_id *r)
> +{
> +/*
> + * lexical order, compare less significant values only if more
> significant
> + * ones are equal
> + */
> +return l->sentinel - r->sentinel ?:
> + l->drawer - r->drawer ?:
> + l->book - r->book ?:
> + l->socket - r->socket ?:
> + l->type - r->type ?:
> + -1 * (l->vertical - r->vertical) ?:
> + -1 * (l->entitlement - r->entitlement) ?:
> + -1 * (l->dedicated - r->dedicated) ?:
I'll switch around those expressions manually, since gcc complains:
error: ‘*’ in boolean context, suggest ‘&&’ instead
[-Werror=int-in-bool-context]
222 |-1 * (l->vertical - r->vertical) ?:
|~~~^
I'll add a comment also, since the inverted logic is not as obvious as before.
> + l->origin - r->origin;
> +}
[...]
[PATCH v23 13/20] docs/s390x/cpu topology: document s390x cpu topology
From: Pierre Morel
Add some basic examples for the definition of cpu topology
in s390x.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS| 2 +
docs/devel/index-internals.rst | 1 +
docs/devel/s390-cpu-topology.rst | 170
docs/system/s390x/cpu-topology.rst | 242 +
docs/system/target-s390x.rst | 1 +
5 files changed, 416 insertions(+)
create mode 100644 docs/devel/s390-cpu-topology.rst
create mode 100644 docs/system/s390x/cpu-topology.rst
diff --git a/MAINTAINERS b/MAINTAINERS
index 17b92fe3ce..0cba0cb2d1 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1703,6 +1703,8 @@ S: Supported
F: include/hw/s390x/cpu-topology.h
F: hw/s390x/cpu-topology.c
F: target/s390x/kvm/stsi-topology.c
+F: docs/devel/s390-cpu-topology.rst
+F: docs/system/s390x/cpu-topology.rst
X86 Machines
diff --git a/docs/devel/index-internals.rst b/docs/devel/index-internals.rst
index e1a93df263..6f81df92bc 100644
--- a/docs/devel/index-internals.rst
+++ b/docs/devel/index-internals.rst
@@ -14,6 +14,7 @@ Details about QEMU's various subsystems including how to add
features to them.
migration
multi-process
reset
+ s390-cpu-topology
s390-dasd-ipl
tracing
vfio-migration
diff --git a/docs/devel/s390-cpu-topology.rst b/docs/devel/s390-cpu-topology.rst
new file mode 100644
index 00..9eab28d5e5
--- /dev/null
+++ b/docs/devel/s390-cpu-topology.rst
@@ -0,0 +1,170 @@
+QAPI interface for S390 CPU topology
+
+
+The following sections will explain the QAPI interface for S390 CPU topology
+with the help of exemplary output.
+For this, let's assume that QEMU has been started with the following
+command, defining 4 CPUs, where CPU[0] is defined by the -smp argument and will
+have default values:
+
+.. code-block:: bash
+
+ qemu-system-s390x \
+-enable-kvm \
+-cpu z14,ctop=on \
+-smp 1,drawers=3,books=3,sockets=2,cores=2,maxcpus=36 \
+-device z14-s390x-cpu,core-id=19,entitlement=high \
+-device z14-s390x-cpu,core-id=11,entitlement=low \
+-device z14-s390x-cpu,core-id=112,entitlement=high \
+ ...
+
+Additions to query-cpus-fast
+
+
+The command query-cpus-fast allows querying the topology tree and
+modifiers for all configured vCPUs.
+
+.. code-block:: QMP
+
+ { "execute": "query-cpus-fast" }
+ {
+ "return": [
+{
+ "dedicated": false,
+ "thread-id": 536993,
+ "props": {
+"core-id": 0,
+"socket-id": 0,
+"drawer-id": 0,
+"book-id": 0
+ },
+ "cpu-state": "operating",
+ "entitlement": "medium",
+ "qom-path": "/machine/unattached/device[0]",
+ "cpu-index": 0,
+ "target": "s390x"
+},
+{
+ "dedicated": false,
+ "thread-id": 537003,
+ "props": {
+"core-id": 19,
+"socket-id": 1,
+"drawer-id": 0,
+"book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[0]",
+ "cpu-index": 19,
+ "target": "s390x"
+},
+{
+ "dedicated": false,
+ "thread-id": 537004,
+ "props": {
+"core-id": 11,
+"socket-id": 1,
+"drawer-id": 0,
+"book-id": 1
+ },
+ "cpu-state": "operating",
+ "entitlement": "low",
+ "qom-path": "/machine/peripheral-anon/device[1]",
+ "cpu-index": 11,
+ "target": "s390x"
+},
+{
+ "dedicated": true,
+ "thread-id": 537005,
+ "props": {
+"core-id": 112,
+"socket-id": 0,
+"drawer-id": 3,
+"book-id": 2
+ },
+ "cpu-state": "operating",
+ "entitlement": "high",
+ "qom-path": "/machine/peripheral-anon/device[2]",
+ "cpu-index": 112,
+ "target": "s390x"
+}
+ ]
+ }
+
+
+QAPI command: set-cpu-topology
+--
+
+The command set-cpu-topology allows modifying the topology tree
+or the topology modifiers of a vCPU in the configuration.
+
+.. code-block:: QMP
+
+{ "execute": "set-cpu-topology",
+ "arguments": {
+ "core-id
[PATCH v23 08/20] qapi/s390x/cpu topology: set-cpu-topology qmp command
From: Pierre Morel
The modification of the CPU attributes are done through a monitor
command.
It allows to move the core inside the topology tree to optimize
the cache usage in the case the host's hypervisor previously
moved the CPU.
The same command allows to modify the CPU attributes modifiers
like polarization entitlement and the dedicated attribute to notify
the guest if the host admin modified scheduling or dedication of a vCPU.
With this knowledge the guest has the possibility to optimize the
usage of the vCPUs.
The command has a feature unstable for the moment.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-target.json | 37 +++
hw/s390x/cpu-topology.c | 132 +++
2 files changed, 169 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index 0d45a590ce..e47a252bd9 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -4,6 +4,8 @@
# This work is licensed under the terms of the GNU GPL, version 2 or later.
# See the COPYING file in the top-level directory.
+{ 'include': 'machine-common.json' }
+
##
# @CpuModelInfo:
#
@@ -375,3 +377,38 @@
'data': [ 'horizontal', 'vertical' ],
'if': 'TARGET_S390X'
}
+
+##
+# @set-cpu-topology:
+#
+# @core-id: the vCPU ID to be moved
+# @socket-id: optional destination socket where to move the vCPU
+# @book-id: optional destination book where to move the vCPU
+# @drawer-id: optional destination drawer where to move the vCPU
+# @entitlement: optional entitlement
+# @dedicated: optional, if the vCPU is dedicated to a real CPU
+#
+# Features:
+# @unstable: This command may still be modified.
+#
+# Modifies the topology by moving the CPU inside the topology
+# tree or by changing a modifier attribute of a CPU.
+# Default value for optional parameter is the current value
+# used by the CPU.
+#
+# Returns: Nothing on success, the reason on failure.
+#
+# Since: 8.2
+##
+{ 'command': 'set-cpu-topology',
+ 'data': {
+ 'core-id': 'uint16',
+ '*socket-id': 'uint16',
+ '*book-id': 'uint16',
+ '*drawer-id': 'uint16',
+ '*entitlement': 'CpuS390Entitlement',
+ '*dedicated': 'bool'
+ },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 13f404a0d7..28adfb3f84 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -23,6 +23,7 @@
#include "target/s390x/cpu.h"
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
+#include "qapi/qapi-commands-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -257,6 +258,29 @@ static bool s390_topology_check(uint16_t socket_id,
uint16_t book_id,
return true;
}
+/**
+ * s390_topology_need_report
+ * @cpu: Current cpu
+ * @drawer_id: future drawer ID
+ * @book_id: future book ID
+ * @socket_id: future socket ID
+ * @entitlement: future entitlement
+ * @dedicated: future dedicated
+ *
+ * A modified topology change report is needed if the topology
+ * tree or the topology attributes change.
+ */
+static bool s390_topology_need_report(S390CPU *cpu, int drawer_id,
+ int book_id, int socket_id,
+ uint16_t entitlement, bool dedicated)
+{
+return cpu->env.drawer_id != drawer_id ||
+ cpu->env.book_id != book_id ||
+ cpu->env.socket_id != socket_id ||
+ cpu->env.entitlement != entitlement ||
+ cpu->env.dedicated != dedicated;
+}
+
/**
* s390_update_cpu_props:
* @ms: the machine state
@@ -325,3 +349,111 @@ void s390_topology_setup_cpu(MachineState *ms, S390CPU
*cpu, Error **errp)
/* topology tree is reflected in props */
s390_update_cpu_props(ms, cpu);
}
+
+static void s390_change_topology(uint16_t core_id,
+ bool has_socket_id, uint16_t socket_id,
+ bool has_book_id, uint16_t book_id,
+ bool has_drawer_id, uint16_t drawer_id,
+ bool has_entitlement,
+ CpuS390Entitlement entitlement,
+ bool has_dedicated, bool dedicated,
+ Error **errp)
+{
+MachineState *ms = current_machine;
+int old_socket_entry;
+int new_socket_entry;
+bool report_needed;
+S390CPU *cpu;
+
+cpu = s390_cpu_addr2state(core_id);
+if (!cpu) {
+error_setg(errp, "Core-id %d does not exist!", core_id);
+return;
+}
+
+/* Get attributes not provided from cpu and verify the new topology */
+if (!has_socket_id) {
+socket_id = cpu->env.socket_id;
+}
+if (!has_book_id) {
[PATCH v23 04/20] s390x/sclp: reporting the maximum nested topology entries
From: Pierre Morel
The maximum nested topology entries is used by the guest to
know how many nested topology are available on the machine.
Let change the MNEST value from 2 to 4 in the SCLP READ INFO
structure now that we support books and drawers.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
---
include/hw/s390x/sclp.h | 5 +++--
hw/s390x/sclp.c | 5 +
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/include/hw/s390x/sclp.h b/include/hw/s390x/sclp.h
index c49051e17e..9aef6d9370 100644
--- a/include/hw/s390x/sclp.h
+++ b/include/hw/s390x/sclp.h
@@ -112,12 +112,13 @@ typedef struct CPUEntry {
} QEMU_PACKED CPUEntry;
#define SCLP_READ_SCP_INFO_FIXED_CPU_OFFSET 128
-#define SCLP_READ_SCP_INFO_MNEST2
+#define SCLP_READ_SCP_INFO_MNEST4
typedef struct ReadInfo {
SCCBHeader h;
uint16_t rnmax;
uint8_t rnsize;
-uint8_t _reserved1[16 - 11]; /* 11-15 */
+uint8_t _reserved1[15 - 11]; /* 11-14 */
+uint8_t stsi_parm; /* 15-15 */
uint16_t entries_cpu; /* 16-17 */
uint16_t offset_cpu;/* 18-19 */
uint8_t _reserved2[24 - 20]; /* 20-23 */
diff --git a/hw/s390x/sclp.c b/hw/s390x/sclp.c
index eff74479f4..d339cbb7e4 100644
--- a/hw/s390x/sclp.c
+++ b/hw/s390x/sclp.c
@@ -20,6 +20,7 @@
#include "hw/s390x/event-facility.h"
#include "hw/s390x/s390-pci-bus.h"
#include "hw/s390x/ipl.h"
+#include "hw/s390x/cpu-topology.h"
static inline SCLPDevice *get_sclp_device(void)
{
@@ -123,6 +124,10 @@ static void read_SCP_info(SCLPDevice *sclp, SCCB *sccb)
return;
}
+if (s390_has_topology()) {
+read_info->stsi_parm = SCLP_READ_SCP_INFO_MNEST;
+}
+
/* CPU information */
prepare_cpu_entries(machine, entries_start, _count);
read_info->entries_cpu = cpu_to_be16(cpu_count);
--
2.39.2
[PATCH v23 10/20] machine: adding s390 topology to info hotpluggable-cpus
From: Pierre Morel
S390 topology adds books and drawers topology containers.
Let's add these to the HMP information for hotpluggable cpus.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
---
hw/core/machine-hmp-cmds.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/hw/core/machine-hmp-cmds.c b/hw/core/machine-hmp-cmds.c
index c3e55ef9e9..9a4b59c6f2 100644
--- a/hw/core/machine-hmp-cmds.c
+++ b/hw/core/machine-hmp-cmds.c
@@ -71,6 +71,12 @@ void hmp_hotpluggable_cpus(Monitor *mon, const QDict *qdict)
if (c->has_node_id) {
monitor_printf(mon, "node-id: \"%" PRIu64 "\"\n", c->node_id);
}
+if (c->has_drawer_id) {
+monitor_printf(mon, "drawer-id: \"%" PRIu64 "\"\n",
c->drawer_id);
+}
+if (c->has_book_id) {
+monitor_printf(mon, "book-id: \"%" PRIu64 "\"\n", c->book_id);
+}
if (c->has_socket_id) {
monitor_printf(mon, "socket-id: \"%" PRIu64 "\"\n",
c->socket_id);
}
--
2.39.2
[PATCH v23 01/20] CPU topology: extend with s390 specifics
From: Pierre Morel
S390 adds two new SMP levels, drawers and books to the CPU
topology.
S390 CPUs have specific topology features like dedication and
entitlement. These indicate to the guest information on host
vCPU scheduling and help the guest make better scheduling decisions.
Let us provide the SMP properties with books and drawers levels
and S390 CPU with dedication and entitlement,
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-common.json| 21 +
qapi/machine.json | 19 ++--
include/hw/boards.h | 10 +-
include/hw/qdev-properties-system.h | 4 +++
target/s390x/cpu.h | 6
hw/core/machine-smp.c | 48 -
hw/core/machine.c | 4 +++
hw/core/qdev-properties-system.c| 13
hw/s390x/s390-virtio-ccw.c | 4 +++
softmmu/vl.c| 6
target/s390x/cpu.c | 7 +
qapi/meson.build| 1 +
qemu-options.hx | 7 +++--
13 files changed, 137 insertions(+), 13 deletions(-)
create mode 100644 qapi/machine-common.json
diff --git a/qapi/machine-common.json b/qapi/machine-common.json
new file mode 100644
index 00..e40421bb37
--- /dev/null
+++ b/qapi/machine-common.json
@@ -0,0 +1,21 @@
+# -*- Mode: Python -*-
+# vim: filetype=python
+#
+# This work is licensed under the terms of the GNU GPL, version 2 or later.
+# See the COPYING file in the top-level directory.
+
+##
+# = Machines S390 data types
+##
+
+##
+# @CpuS390Entitlement:
+#
+# An enumeration of cpu entitlements that can be assumed by a virtual
+# S390 CPU
+#
+# Since: 8.2
+##
+{ 'enum': 'CpuS390Entitlement',
+ 'prefix': 'S390_CPU_ENTITLEMENT',
+ 'data': [ 'auto', 'low', 'medium', 'high' ] }
diff --git a/qapi/machine.json b/qapi/machine.json
index a08b6576ca..a63cb951d2 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -9,6 +9,7 @@
##
{ 'include': 'common.json' }
+{ 'include': 'machine-common.json' }
##
# @SysEmuTarget:
@@ -71,7 +72,7 @@
#
# @thread-id: ID of the underlying host thread
#
-# @props: properties describing to which node/socket/core/thread
+# @props: properties describing to which node/drawer/book/socket/core/thread
# virtual CPU belongs to, provided if supported by board
#
# @target: the QEMU system emulation target, which determines which
@@ -901,7 +902,11 @@
#
# @node-id: NUMA node ID the CPU belongs to
#
-# @socket-id: socket number within node/board the CPU belongs to
+# @drawer-id: drawer number within node/board the CPU belongs to (since 8.2)
+#
+# @book-id: book number within drawer/node/board the CPU belongs to (since 8.2)
+#
+# @socket-id: socket number within book/node/board the CPU belongs to
#
# @die-id: die number within socket the CPU belongs to (since 4.1)
#
@@ -912,7 +917,7 @@
#
# @thread-id: thread number within core the CPU belongs to
#
-# Note: currently there are 6 properties that could be present but
+# Note: currently there are 8 properties that could be present but
# management should be prepared to pass through other properties
# with device_add command to allow for future interface extension.
# This also requires the filed names to be kept in sync with the
@@ -922,6 +927,8 @@
##
{ 'struct': 'CpuInstanceProperties',
'data': { '*node-id': 'int',
+'*drawer-id': 'int',
+'*book-id': 'int',
'*socket-id': 'int',
'*die-id': 'int',
'*cluster-id': 'int',
@@ -1480,6 +1487,10 @@
#
# @cpus: number of virtual CPUs in the virtual machine
#
+# @drawers: number of drawers in the CPU topology (since 8.2)
+#
+# @books: number of books in the CPU topology (since 8.2)
+#
# @sockets: number of sockets in the CPU topology
#
# @dies: number of dies per socket in the CPU topology
@@ -1498,6 +1509,8 @@
##
{ 'struct': 'SMPConfiguration', 'data': {
'*cpus': 'int',
+ '*drawers': 'int',
+ '*books': 'int',
'*sockets': 'int',
'*dies': 'int',
'*clusters': 'int',
diff --git a/include/hw/boards.h b/include/hw/boards.h
index 6c67af196a..6dcfc879eb 100644
--- a/include/hw/boards.h
+++ b/include/hw/boards.h
@@ -134,12 +134,16 @@ typedef struct {
* @clusters_supported - whether clusters are supported by the machine
* @has_clusters - whether clusters are explicitly specified in the user
* provided SMP configuration
+ * @books_supported - whether books are supported by the machine
+ * @drawers_supported - whether drawers are supported by the machine
*/
typedef struct {
bool prefer_sockets;
bool dies_supported;
bool clusters_supported;
bool has_clusters;
+bool books_supported;
+bool drawers_supported;
} SMPCompatProps;
/**
@@ -310,7 +314,9 @@ typedef struct
[PATCH v23 11/20] qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
From: Pierre Morel
When the guest asks to change the polarization this change
is forwarded to the upper layer using QAPI.
The upper layer is supposed to take according decisions concerning
CPU provisioning.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine-target.json | 33 +
hw/s390x/cpu-topology.c | 2 ++
2 files changed, 35 insertions(+)
diff --git a/qapi/machine-target.json b/qapi/machine-target.json
index e47a252bd9..276c3bf9d1 100644
--- a/qapi/machine-target.json
+++ b/qapi/machine-target.json
@@ -412,3 +412,36 @@
'features': [ 'unstable' ],
'if': { 'all': [ 'TARGET_S390X' , 'CONFIG_KVM' ] }
}
+
+##
+# @CPU_POLARIZATION_CHANGE:
+#
+# Emitted when the guest asks to change the polarization.
+#
+# @polarization: polarization specified by the guest
+#
+# Features:
+# @unstable: This command may still be modified.
+#
+# The guest can tell the host (via the PTF instruction) whether the
+# CPUs should be provisioned using horizontal or vertical polarization.
+#
+# On horizontal polarization the host is expected to provision all vCPUs
+# equally.
+# On vertical polarization the host can provision each vCPU differently.
+# The guest will get information on the details of the provisioning
+# the next time it uses the STSI(15) instruction.
+#
+# Since: 8.2
+#
+# Example:
+#
+# <- { "event": "CPU_POLARIZATION_CHANGE",
+# "data": { "polarization": "horizontal" },
+# "timestamp": { "seconds": 1401385907, "microseconds": 422329 } }
+##
+{ 'event': 'CPU_POLARIZATION_CHANGE',
+ 'data': { 'polarization': 'CpuS390Polarization' },
+ 'features': [ 'unstable' ],
+ 'if': { 'all': [ 'TARGET_S390X', 'CONFIG_KVM' ] }
+}
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
index 28adfb3f84..18274db74c 100644
--- a/hw/s390x/cpu-topology.c
+++ b/hw/s390x/cpu-topology.c
@@ -24,6 +24,7 @@
#include "hw/s390x/s390-virtio-ccw.h"
#include "hw/s390x/cpu-topology.h"
#include "qapi/qapi-commands-machine-target.h"
+#include "qapi/qapi-events-machine-target.h"
/*
* s390_topology is used to keep the topology information.
@@ -136,6 +137,7 @@ void s390_handle_ptf(S390CPU *cpu, uint8_t r1, uintptr_t ra)
} else {
s390_topology.polarization = polarization;
s390_cpu_topology_set_changed(true);
+qapi_event_send_cpu_polarization_change(polarization);
setcc(cpu, 0);
}
break;
--
2.39.2
[PATCH v23 02/20] s390x/cpu topology: add topology entries on CPU hotplug
From: Pierre Morel
The topology information are attributes of the CPU and are
specified during the CPU device creation.
On hot plug we:
- calculate the default values for the topology for drawers,
books and sockets in the case they are not specified.
- verify the CPU attributes
- check that we have still room on the desired socket
The possibility to insert a CPU in a mask is dependent on the
number of cores allowed in a socket, a book or a drawer, the
checking is done during the hot plug of the CPU to have an
immediate answer.
If the complete topology is not specified, the core is added
in the physical topology based on its core ID and it gets
defaults values for the modifier attributes.
This way, starting QEMU without specifying the topology can
still get some advantage of the CPU topology.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
MAINTAINERS | 6 +
include/hw/s390x/cpu-topology.h | 54 +++
hw/s390x/cpu-topology.c | 259
hw/s390x/s390-virtio-ccw.c | 22 ++-
hw/s390x/meson.build| 1 +
5 files changed, 340 insertions(+), 2 deletions(-)
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 hw/s390x/cpu-topology.c
diff --git a/MAINTAINERS b/MAINTAINERS
index 00562f924f..9c6599a55b 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -1697,6 +1697,12 @@ F: hw/s390x/event-facility.c
F: hw/s390x/sclp*.c
L: qemu-s3...@nongnu.org
+S390 CPU topology
+M: Nina Schoetterl-Glausch
+S: Supported
+F: include/hw/s390x/cpu-topology.h
+F: hw/s390x/cpu-topology.c
+
X86 Machines
PC
diff --git a/include/hw/s390x/cpu-topology.h b/include/hw/s390x/cpu-topology.h
new file mode 100644
index 00..97b0af2795
--- /dev/null
+++ b/include/hw/s390x/cpu-topology.h
@@ -0,0 +1,54 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel
+ *
+ */
+#ifndef HW_S390X_CPU_TOPOLOGY_H
+#define HW_S390X_CPU_TOPOLOGY_H
+
+#ifndef CONFIG_USER_ONLY
+
+#include "qemu/queue.h"
+#include "hw/boards.h"
+#include "qapi/qapi-types-machine-target.h"
+
+typedef struct S390Topology {
+uint8_t *cores_per_socket;
+} S390Topology;
+
+#ifdef CONFIG_KVM
+bool s390_has_topology(void);
+void s390_topology_setup_cpu(MachineState *ms, S390CPU *cpu, Error **errp);
+#else
+static inline bool s390_has_topology(void)
+{
+ return false;
+}
+static inline void s390_topology_setup_cpu(MachineState *ms,
+ S390CPU *cpu,
+ Error **errp) {}
+#endif
+
+extern S390Topology s390_topology;
+
+static inline int s390_std_socket(int n, CpuTopology *smp)
+{
+return (n / smp->cores) % smp->sockets;
+}
+
+static inline int s390_std_book(int n, CpuTopology *smp)
+{
+return (n / (smp->cores * smp->sockets)) % smp->books;
+}
+
+static inline int s390_std_drawer(int n, CpuTopology *smp)
+{
+return (n / (smp->cores * smp->sockets * smp->books)) % smp->drawers;
+}
+
+#endif /* CONFIG_USER_ONLY */
+
+#endif
diff --git a/hw/s390x/cpu-topology.c b/hw/s390x/cpu-topology.c
new file mode 100644
index 00..189fcc5334
--- /dev/null
+++ b/hw/s390x/cpu-topology.c
@@ -0,0 +1,259 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * CPU Topology
+ *
+ * Copyright IBM Corp. 2022, 2023
+ * Author(s): Pierre Morel
+ *
+ * S390 topology handling can be divided in two parts:
+ *
+ * - The first part in this file is taking care of all common functions
+ * used by KVM and TCG to create and modify the topology.
+ *
+ * - The second part, building the topology information data for the
+ * guest with CPU and KVM specificity will be implemented inside
+ * the target/s390/kvm sub tree.
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "qemu/error-report.h"
+#include "hw/qdev-properties.h"
+#include "hw/boards.h"
+#include "target/s390x/cpu.h"
+#include "hw/s390x/s390-virtio-ccw.h"
+#include "hw/s390x/cpu-topology.h"
+
+/*
+ * s390_topology is used to keep the topology information.
+ * .cores_per_socket: tracks information on the count of cores
+ *per socket.
+ */
+S390Topology s390_topology = {
+/* will be initialized after the CPU model is realized */
+.cores_per_socket = NULL,
+};
+
+/**
+ * s390_socket_nb:
+ * @cpu: s390x CPU
+ *
+ * Returns the socket number used inside the cores_per_socket array
+ * for a topology tree entry
+ */
+static int s390_socket_nb_from_ids(int drawer_id, int book_id, int socket_id)
+{
+return (drawer_id * current_machine->smp.books + book_id) *
+ current_machine->smp.sockets + socket_id;
+}
+
+/**
+
[PATCH v23 20/20] tests/avocado: s390x cpu topology bad move
From: Pierre Morel
This test verifies that QEMU refuses to move a CPU to an
nonexistent location.
Signed-off-by: Pierre Morel
Reviewed-by: Thomas Huth
---
tests/avocado/s390_topology.py | 25 +
1 file changed, 25 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index d3e6556c0f..9154ac8776 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -412,3 +412,28 @@ def test_dedicated_error(self):
res = self.vm.qmp('set-cpu-topology',
{'core-id': 0, 'entitlement': 'medium', 'dedicated':
False})
self.assertEqual(res['return'], {})
+
+def test_move_error(self):
+"""
+This test verifies that QEMU refuses to move a CPU to an
+nonexistent location
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'drawer-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'book-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+res = self.vm.qmp('set-cpu-topology', {'core-id': 0, 'socket-id': 1})
+self.assertEqual(res['error']['class'], 'GenericError')
+
+self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.39.2
[PATCH v23 15/20] tests/avocado: s390x cpu topology polarization
From: Pierre Morel
Polarization is changed on a request from the guest.
Let's verify the polarization is accordingly set by QEMU.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
tests/avocado/s390_topology.py | 45 ++
1 file changed, 45 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 9078b45281..8166cee134 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -41,6 +41,7 @@ class S390CPUTopology(QemuSystemTest):
The polarization is changed on a request from the guest.
"""
timeout = 90
+event_timeout = 10
KERNEL_COMMON_COMMAND_LINE = ('printk.time=0 '
'root=/dev/ram '
@@ -103,6 +104,14 @@ def kernel_init(self):
'-initrd', initrd_path,
'-append', kernel_command_line)
+def system_init(self):
+self.log.info("System init")
+exec_command_and_wait_for_pattern(self,
+""" mount proc -t proc /proc;
+mount sys -t sysfs /sys;
+cat /sys/devices/system/cpu/dispatching """,
+'0')
+
def test_single(self):
"""
This test checks the simplest topology with a single CPU.
@@ -198,3 +207,39 @@ def test_dash_device(self):
self.check_topology(3, 1, 1, 1, 'high', False)
self.check_topology(4, 1, 1, 1, 'medium', False)
self.check_topology(5, 2, 1, 1, 'high', True)
+
+
+def guest_set_dispatching(self, dispatching):
+exec_command(self,
+f'echo {dispatching} > /sys/devices/system/cpu/dispatching')
+self.vm.event_wait('CPU_POLARIZATION_CHANGE', self.event_timeout)
+exec_command_and_wait_for_pattern(self,
+'cat /sys/devices/system/cpu/dispatching', dispatching)
+
+
+def test_polarization(self):
+"""
+This test verifies that QEMU modifies the entitlement change after
+several guest polarization change requests.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'horizontal')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+self.guest_set_dispatching('1');
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'vertical')
+self.check_topology(0, 0, 0, 0, 'medium', False)
+
+self.guest_set_dispatching('0');
+res = self.vm.qmp('query-s390x-cpu-polarization')
+self.assertEqual(res['return']['polarization'], 'horizontal')
+self.check_topology(0, 0, 0, 0, 'medium', False)
--
2.39.2
[PATCH v23 00/20] s390x: CPU Topology
- Separated STSI instruction from KVM to prepare TCG
(Thomas)
- Take care of endianess to prepare TCG
(Thomas)
- Added comments on STSI CPU container and PFT instruction
(Thomas)
- Moved enabling the instructions as the last patch
(Thomas)
Pierre Morel (20):
CPU topology: extend with s390 specifics
s390x/cpu topology: add topology entries on CPU hotplug
target/s390x/cpu topology: handle STSI(15) and build the SYSIB
s390x/sclp: reporting the maximum nested topology entries
s390x/cpu topology: resetting the Topology-Change-Report
s390x/cpu topology: interception of PTF instruction
target/s390x/cpu topology: activate CPU topology
qapi/s390x/cpu topology: set-cpu-topology qmp command
machine: adding s390 topology to query-cpu-fast
machine: adding s390 topology to info hotpluggable-cpus
qapi/s390x/cpu topology: CPU_POLARIZATION_CHANGE qapi event
qapi/s390x/cpu topology: query-cpu-polarization qmp command
docs/s390x/cpu topology: document s390x cpu topology
tests/avocado: s390x cpu topology core
tests/avocado: s390x cpu topology polarization
tests/avocado: s390x cpu topology entitlement tests
tests/avocado: s390x cpu topology test dedicated CPU
tests/avocado: s390x cpu topology test socket full
tests/avocado: s390x cpu topology dedicated errors
tests/avocado: s390x cpu topology bad move
MAINTAINERS | 10 +
docs/devel/index-internals.rst | 1 +
docs/devel/s390-cpu-topology.rst| 170 ++
docs/system/s390x/cpu-topology.rst | 242 ++
docs/system/target-s390x.rst| 1 +
qapi/machine-common.json| 21 ++
qapi/machine-target.json| 113 +++
qapi/machine.json | 28 +-
include/hw/boards.h | 10 +-
include/hw/qdev-properties-system.h | 4 +
include/hw/s390x/cpu-topology.h | 78 +
include/hw/s390x/s390-virtio-ccw.h | 6 +
include/hw/s390x/sclp.h | 4 +-
target/s390x/cpu.h | 82 +
target/s390x/kvm/kvm_s390x.h| 1 +
hw/core/machine-hmp-cmds.c | 6 +
hw/core/machine-smp.c | 48 ++-
hw/core/machine.c | 4 +
hw/core/qdev-properties-system.c| 13 +
hw/s390x/cpu-topology.c | 469
hw/s390x/s390-virtio-ccw.c | 29 +-
hw/s390x/sclp.c | 5 +
softmmu/vl.c| 6 +
target/s390x/cpu-sysemu.c | 13 +
target/s390x/cpu.c | 16 +
target/s390x/cpu_models.c | 1 +
target/s390x/kvm/kvm.c | 42 ++-
target/s390x/kvm/stsi-topology.c| 338
hw/s390x/meson.build| 1 +
qapi/meson.build| 1 +
qemu-options.hx | 7 +-
target/s390x/kvm/meson.build| 3 +-
tests/avocado/s390_topology.py | 439 ++
33 files changed, 2193 insertions(+), 19 deletions(-)
create mode 100644 docs/devel/s390-cpu-topology.rst
create mode 100644 docs/system/s390x/cpu-topology.rst
create mode 100644 qapi/machine-common.json
create mode 100644 include/hw/s390x/cpu-topology.h
create mode 100644 hw/s390x/cpu-topology.c
create mode 100644 target/s390x/kvm/stsi-topology.c
create mode 100644 tests/avocado/s390_topology.py
Range-diff against v22:
1: c26d3df9f9 ! 1: f2219b7583 CPU topology: extend with s390 specifics
@@ qapi/machine.json
# @thread-id: ID of the underlying host thread
#
-# @props: properties describing to which node/socket/core/thread
--# virtual CPU belongs to, provided if supported by board
+# @props: properties describing to which
node/drawer/book/socket/core/thread
-+# virtual CPU belongs to, provided if supported by board
+ # virtual CPU belongs to, provided if supported by board
#
# @target: the QEMU system emulation target, which determines which
- # additional fields will be listed (since 3.0)
@@
#
# @node-id: NUMA node ID the CPU belongs to
@@ hw/s390x/s390-virtio-ccw.c: static void
ccw_machine_class_init(ObjectClass *oc,
assert(!mc->get_hotplug_handler);
mc->get_hotplug_handler = s390_get_hotplug_handler;
mc->cpu_index_to_instance_props = s390_cpu_index_to_props;
+@@ hw/s390x/s390-virtio-ccw.c: static void
ccw_machine_8_1_class_options(MachineClass *mc)
+ {
+ ccw_machine_8_2_class_options(mc);
+ compat_props_add(mc->compat_props, hw_compat_8_1, hw_compat_8_1_len);
++mc->smp_props.drawers_supported = false;
++mc->smp_props.books_supported = false;
+ }
+ DEFINE_CCW_MACHINE(8_1, "8.1", false);
+
## softmmu/vl.c ##
@@ softmmu/vl.c: static QemuOptsList qemu_smp_opts = {
2: 6d32f1e3d4 ! 2: 5fae55e5a1 s390x/cpu topology: add topology entries on
CPU hotplug
@@ Commit message
[PATCH v23 09/20] machine: adding s390 topology to query-cpu-fast
From: Pierre Morel
S390x provides two more topology attributes, entitlement and dedication.
Let's add these CPU attributes to the QAPI command query-cpu-fast.
Signed-off-by: Pierre Morel
Reviewed-by: Nina Schoetterl-Glausch
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Signed-off-by: Nina Schoetterl-Glausch
---
qapi/machine.json | 9 -
target/s390x/cpu.c | 9 +
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/qapi/machine.json b/qapi/machine.json
index a63cb951d2..be1d70d8cd 100644
--- a/qapi/machine.json
+++ b/qapi/machine.json
@@ -56,10 +56,17 @@
# Additional information about a virtual S390 CPU
#
# @cpu-state: the virtual CPU's state
+# @dedicated: the virtual CPU's dedication (since 8.2)
+# @entitlement: the virtual CPU's entitlement (since 8.2)
#
# Since: 2.12
##
-{ 'struct': 'CpuInfoS390', 'data': { 'cpu-state': 'CpuS390State' } }
+{ 'struct': 'CpuInfoS390',
+ 'data': { 'cpu-state': 'CpuS390State',
+'*dedicated': 'bool',
+'*entitlement': 'CpuS390Entitlement'
+ }
+}
##
# @CpuInfoFast:
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index 74405beb51..5967e34a85 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -38,6 +38,7 @@
#ifndef CONFIG_USER_ONLY
#include "sysemu/reset.h"
#endif
+#include "hw/s390x/cpu-topology.h"
#define CR0_RESET 0xE0UL
#define CR14_RESET 0xC200UL;
@@ -146,6 +147,14 @@ static void s390_query_cpu_fast(CPUState *cpu, CpuInfoFast
*value)
S390CPU *s390_cpu = S390_CPU(cpu);
value->u.s390x.cpu_state = s390_cpu->env.cpu_state;
+#if !defined(CONFIG_USER_ONLY)
+if (s390_has_topology()) {
+value->u.s390x.has_dedicated = true;
+value->u.s390x.dedicated = s390_cpu->env.dedicated;
+value->u.s390x.has_entitlement = true;
+value->u.s390x.entitlement = s390_cpu->env.entitlement;
+}
+#endif
}
/* S390CPUClass::reset() */
--
2.39.2
[PATCH v23 17/20] tests/avocado: s390x cpu topology test dedicated CPU
From: Pierre Morel
A dedicated CPU in vertical polarization can only have
a high entitlement.
Let's check this from both host and guest point of view.
Signed-off-by: Pierre Morel
Co-developed-by: Nina Schoetterl-Glausch
Reviewed-by: Thomas Huth
Reviewed-by: Nina Schoetterl-Glausch
Signed-off-by: Nina Schoetterl-Glausch
---
tests/avocado/s390_topology.py | 33 +
1 file changed, 33 insertions(+)
diff --git a/tests/avocado/s390_topology.py b/tests/avocado/s390_topology.py
index 24fac9a54d..3661048f4c 100644
--- a/tests/avocado/s390_topology.py
+++ b/tests/avocado/s390_topology.py
@@ -305,3 +305,36 @@ def test_entitlement(self):
self.guest_set_dispatching('0');
self.check_polarization("horizontal")
self.check_topology(0, 0, 0, 0, 'high', False)
+
+
+def test_dedicated(self):
+"""
+This test verifies that QEMU adjusts the entitlement correctly when a
+CPU is made dedicated.
+QEMU retains the entitlement value when horizontal polarization is in
effect.
+For the guest, the field shows the effective value of the entitlement.
+
+:avocado: tags=arch:s390x
+:avocado: tags=machine:s390-ccw-virtio
+"""
+self.kernel_init()
+self.vm.launch()
+self.wait_until_booted()
+
+self.system_init()
+
+self.check_polarization("horizontal")
+
+res = self.vm.qmp('set-cpu-topology',
+ {'core-id': 0, 'dedicated': True})
+self.assertEqual(res['return'], {})
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("horizontal")
+
+self.guest_set_dispatching('1');
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("vertical:high")
+
+self.guest_set_dispatching('0');
+self.check_topology(0, 0, 0, 0, 'high', True)
+self.check_polarization("horizontal")
--
2.39.2
