Re: [PULL 15/22] x86: Grant AMX permission for guest
On Wed, Mar 16, 2022 at 04:57:39PM +0100, Peter Krempa wrote:
> On Tue, Mar 08, 2022 at 12:34:38 +0100, Paolo Bonzini wrote:
> > From: Yang Zhong
> >
> > Kernel allocates 4K xstate buffer by default. For XSAVE features
> > which require large state component (e.g. AMX), Linux kernel
> > dynamically expands the xstate buffer only after the process has
> > acquired the necessary permissions. Those are called dynamically-
> > enabled XSAVE features (or dynamic xfeatures).
> >
> > There are separate permissions for native tasks and guests.
> >
> > Qemu should request the guest permissions for dynamic xfeatures
> > which will be exposed to the guest. This only needs to be done
> > once before the first vcpu is created.
> >
> > KVM implemented one new ARCH_GET_XCOMP_SUPP system attribute API to
> > get host side supported_xcr0 and Qemu can decide if it can request
> > dynamically enabled XSAVE features permission.
> > https://lore.kernel.org/all/20220126152210.3044876-1-pbonz...@redhat.com/
> >
> > Suggested-by: Paolo Bonzini
> > Signed-off-by: Yang Zhong
> > Signed-off-by: Jing Liu
> > Message-Id: <20220217060434.52460-4-yang.zh...@intel.com>
> > Signed-off-by: Paolo Bonzini
> > ---
> > target/i386/cpu.c | 7 +
> > target/i386/cpu.h | 4 +++
> > target/i386/kvm/kvm-cpu.c | 12
> > target/i386/kvm/kvm.c | 57 ++
> > target/i386/kvm/kvm_i386.h | 1 +
> > 5 files changed, 75 insertions(+), 6 deletions(-)
>
> With this commit qemu crashes for me when invoking the following
> QMP command:
>
> $ ~pipo/git/qemu.git/build/qemu-system-x86_64 -S -no-user-config -nodefaults
> -nographic -machine none,accel=kvm -qmp stdio
> {"QMP": {"version": {"qemu": {"micro": 90, "minor": 2, "major": 6},
> "package": "v7.0.0-rc0-8-g1d60bb4b14"}, "capabilities": ["oob"]}}
> {'execute':'qmp_capabilities'}
> {"return": {}}
> {"execute":"qom-list-properties","arguments":{"typename":"max-x86_64-cpu"},"id":"libvirt-41"}
> qemu-system-x86_64: ../target/i386/kvm/kvm-cpu.c:105: kvm_cpu_xsave_init:
> Assertion `esa->size == eax' failed.
> Aborted (core dumped)
>
> Note that the above is on a box with an 'AMD Ryzen 9 3900X'.
>
> Curiously on a laptop with an Intel chip (Intel(R) Core(TM) i7-10610U)
> it seems to work.
Paolo, I debugged this issue and found this issue is caused by xstate feature
bit9
(MPK, which like pku in intel) in the some AMD platforms.
#AMD Spec, p409
https://www.amd.com/system/files/TechDocs/24593.pdf
I checked the cpuid info from AMD EPYC 7402P server and ECX=0x9, the eax is
0x40,
which is different with eax=0x0008 in Intel platform. So, the ASSERT is
generated
by AMX changes.
##AMD host
0x000d 0x00: eax=0x0207 ebx=0x0340 ecx=0x0380 edx=0x
0x000d 0x01: eax=0x000f ebx=0x0340 ecx=0x edx=0x
0x000d 0x02: eax=0x0100 ebx=0x0240 ecx=0x edx=0x
0x000d 0x09: eax=0x0040 ebx=0x0340 ecx=0x edx=0x
##Intel host
0x000d 0x00: eax=0x000602e7 ebx=0x2b00 ecx=0x2b00 edx=0x
0x000d 0x01: eax=0x001f ebx=0x2d00 ecx=0xdd00 edx=0x
0x000d 0x02: eax=0x0100 ebx=0x0240 ecx=0x edx=0x
0x000d 0x05: eax=0x0040 ebx=0x0440 ecx=0x edx=0x
0x000d 0x06: eax=0x0200 ebx=0x0480 ecx=0x edx=0x
0x000d 0x07: eax=0x0400 ebx=0x0680 ecx=0x edx=0x
0x000d 0x08: eax=0x0080 ebx=0x ecx=0x0001 edx=0x
0x000d 0x09: eax=0x0008 ebx=0x0a80 ecx=0x edx=0x
0x000d 0x0a: eax=0x0008 ebx=0x ecx=0x0001 edx=0x
0x000d 0x0b: eax=0x0010 ebx=0x ecx=0x0001 edx=0x
0x000d 0x0c: eax=0x0018 ebx=0x ecx=0x0001 edx=0x
0x000d 0x0e: eax=0x0030 ebx=0x ecx=0x0001 edx=0x
0x000d 0x0f: eax=0x0328 ebx=0x ecx=0x0001 edx=0x
0x000d 0x11: eax=0x0040 ebx=0x0ac0 ecx=0x0002 edx=0x
0x000d 0x12: eax=0x2000 ebx=0x0b00 ecx=0x0006 edx=0x
But I also checked same cpuid info from AMD MILAN server, the eax=0x0008
in ECX=0x9.
So, for this ECX=0x9, the eax values in different AMD server are different.
How can we handle those different value since we have used host_cpuid() to
read host's
registers? thanks!
Yang
Re: [PULL 15/22] x86: Grant AMX permission for guest
On Fri, Mar 18, 2022 at 11:13:56AM +0100, Michal Prívozník wrote:
> On 3/16/22 16:57, Peter Krempa wrote:
> > On Tue, Mar 08, 2022 at 12:34:38 +0100, Paolo Bonzini wrote:
> >> From: Yang Zhong
> >>
> >> Kernel allocates 4K xstate buffer by default. For XSAVE features
> >> which require large state component (e.g. AMX), Linux kernel
> >> dynamically expands the xstate buffer only after the process has
> >> acquired the necessary permissions. Those are called dynamically-
> >> enabled XSAVE features (or dynamic xfeatures).
> >>
> >> There are separate permissions for native tasks and guests.
> >>
> >> Qemu should request the guest permissions for dynamic xfeatures
> >> which will be exposed to the guest. This only needs to be done
> >> once before the first vcpu is created.
> >>
> >> KVM implemented one new ARCH_GET_XCOMP_SUPP system attribute API to
> >> get host side supported_xcr0 and Qemu can decide if it can request
> >> dynamically enabled XSAVE features permission.
> >> https://lore.kernel.org/all/20220126152210.3044876-1-pbonz...@redhat.com/
> >>
> >> Suggested-by: Paolo Bonzini
> >> Signed-off-by: Yang Zhong
> >> Signed-off-by: Jing Liu
> >> Message-Id: <20220217060434.52460-4-yang.zh...@intel.com>
> >> Signed-off-by: Paolo Bonzini
> >> ---
> >> target/i386/cpu.c | 7 +
> >> target/i386/cpu.h | 4 +++
> >> target/i386/kvm/kvm-cpu.c | 12
> >> target/i386/kvm/kvm.c | 57 ++
> >> target/i386/kvm/kvm_i386.h | 1 +
> >> 5 files changed, 75 insertions(+), 6 deletions(-)
> >
> > With this commit qemu crashes for me when invoking the following
> > QMP command:
> >
> > $ ~pipo/git/qemu.git/build/qemu-system-x86_64 -S -no-user-config
> > -nodefaults -nographic -machine none,accel=kvm -qmp stdio
> > {"QMP": {"version": {"qemu": {"micro": 90, "minor": 2, "major": 6},
> > "package": "v7.0.0-rc0-8-g1d60bb4b14"}, "capabilities": ["oob"]}}
> > {'execute':'qmp_capabilities'}
> > {"return": {}}
> > {"execute":"qom-list-properties","arguments":{"typename":"max-x86_64-cpu"},"id":"libvirt-41"}
> > qemu-system-x86_64: ../target/i386/kvm/kvm-cpu.c:105: kvm_cpu_xsave_init:
> > Assertion `esa->size == eax' failed.
> > Aborted (core dumped)
> >
> > Note that the above is on a box with an 'AMD Ryzen 9 3900X'.
> >
> > Curiously on a laptop with an Intel chip (Intel(R) Core(TM) i7-10610U)
> > it seems to work.
> >
> >
>
> Not trying to beat a dead horse here, but I've just found another
> problem with this patch. On my laptop (Linux maggie
> 5.15.26-gentoo-x86_64 #1 SMP Thu Mar 10 08:55:28 CET 2022 x86_64
> Intel(R) Core(TM) i7-10610U CPU @ 1.80GHz GenuineIntel GNU/Linux), when
> I start a guest it no longer sees AVX instructions:
>
> qemu.git $ ./build/qemu-system-x86_64 -accel kvm -cpu host ...
>
Thanks Michal, this issue is caused by compatibility with older kernel
version.
The Qemu will report below logs:
emu-system-x86_64: warning: cannot get sys attribute capabilities 0
qemu-system-x86_64: warning: cannot get sys attribute capabilities 0
qemu-system-x86_64: warning: cannot get sys attribute capabilities 0
qemu-system-x86_64: warning: host doesn't support requested feature:
CPUID.0DH:EAX [bit 5]
qemu-system-x86_64: warning: host doesn't support requested feature:
CPUID.0DH:EAX [bit 6]
qemu-system-x86_64: warning: host doesn't support requested feature:
CPUID.0DH:EAX [bit 9]
..
Since the AMX changes in Qemu need read ARCH_GET_XCOMP_SUPP attribute to get
host supported_xcr0
value, and new kernel release add this new API. So the older kernel can't
report right xcr0 value.
I made one new patch to fix this issue, please try this patch. thanks!
https://lists.nongnu.org/archive/html/qemu-devel/2022-03/msg04732.html
Paolo, this patch only fix this compatibility issue, but the issue caused by
AMD cpu is still not
fixed from my side because no AMD platform can be used in my side. If you
have no time to check
this issue, maybe I need lookfor this platform from our internal. thanks!
Yang
> Michal
Re: [PULL 15/22] x86: Grant AMX permission for guest
On 3/16/22 16:57, Peter Krempa wrote:
> On Tue, Mar 08, 2022 at 12:34:38 +0100, Paolo Bonzini wrote:
>> From: Yang Zhong
>>
>> Kernel allocates 4K xstate buffer by default. For XSAVE features
>> which require large state component (e.g. AMX), Linux kernel
>> dynamically expands the xstate buffer only after the process has
>> acquired the necessary permissions. Those are called dynamically-
>> enabled XSAVE features (or dynamic xfeatures).
>>
>> There are separate permissions for native tasks and guests.
>>
>> Qemu should request the guest permissions for dynamic xfeatures
>> which will be exposed to the guest. This only needs to be done
>> once before the first vcpu is created.
>>
>> KVM implemented one new ARCH_GET_XCOMP_SUPP system attribute API to
>> get host side supported_xcr0 and Qemu can decide if it can request
>> dynamically enabled XSAVE features permission.
>> https://lore.kernel.org/all/20220126152210.3044876-1-pbonz...@redhat.com/
>>
>> Suggested-by: Paolo Bonzini
>> Signed-off-by: Yang Zhong
>> Signed-off-by: Jing Liu
>> Message-Id: <20220217060434.52460-4-yang.zh...@intel.com>
>> Signed-off-by: Paolo Bonzini
>> ---
>> target/i386/cpu.c | 7 +
>> target/i386/cpu.h | 4 +++
>> target/i386/kvm/kvm-cpu.c | 12
>> target/i386/kvm/kvm.c | 57 ++
>> target/i386/kvm/kvm_i386.h | 1 +
>> 5 files changed, 75 insertions(+), 6 deletions(-)
>
> With this commit qemu crashes for me when invoking the following
> QMP command:
>
> $ ~pipo/git/qemu.git/build/qemu-system-x86_64 -S -no-user-config -nodefaults
> -nographic -machine none,accel=kvm -qmp stdio
> {"QMP": {"version": {"qemu": {"micro": 90, "minor": 2, "major": 6},
> "package": "v7.0.0-rc0-8-g1d60bb4b14"}, "capabilities": ["oob"]}}
> {'execute':'qmp_capabilities'}
> {"return": {}}
> {"execute":"qom-list-properties","arguments":{"typename":"max-x86_64-cpu"},"id":"libvirt-41"}
> qemu-system-x86_64: ../target/i386/kvm/kvm-cpu.c:105: kvm_cpu_xsave_init:
> Assertion `esa->size == eax' failed.
> Aborted (core dumped)
>
> Note that the above is on a box with an 'AMD Ryzen 9 3900X'.
>
> Curiously on a laptop with an Intel chip (Intel(R) Core(TM) i7-10610U)
> it seems to work.
>
>
Not trying to beat a dead horse here, but I've just found another
problem with this patch. On my laptop (Linux maggie
5.15.26-gentoo-x86_64 #1 SMP Thu Mar 10 08:55:28 CET 2022 x86_64
Intel(R) Core(TM) i7-10610U CPU @ 1.80GHz GenuineIntel GNU/Linux), when
I start a guest it no longer sees AVX instructions:
qemu.git $ ./build/qemu-system-x86_64 -accel kvm -cpu host ...
Michal
Re: [PULL 15/22] x86: Grant AMX permission for guest
On Wed, 2022-03-16 at 17:50 +, Daniel P. Berrangé wrote: > On Wed, Mar 16, 2022 at 05:48:04PM +, David Edmondson wrote: > > On Wednesday, 2022-03-16 at 16:05:01 GMT, Daniel P. Berrangé wrote: > > > > > On Wed, Mar 16, 2022 at 04:57:39PM +0100, Peter Krempa wrote: > > > > On Tue, Mar 08, 2022 at 12:34:38 +0100, Paolo Bonzini wrote: > > > > > From: Yang Zhong > > > > > > > > > > Kernel allocates 4K xstate buffer by default. For XSAVE features > > > > > which require large state component (e.g. AMX), Linux kernel > > > > > dynamically expands the xstate buffer only after the process has > > > > > acquired the necessary permissions. Those are called dynamically- > > > > > enabled XSAVE features (or dynamic xfeatures). > > > > > > > > > > There are separate permissions for native tasks and guests. > > > > > > > > > > Qemu should request the guest permissions for dynamic xfeatures > > > > > which will be exposed to the guest. This only needs to be done > > > > > once before the first vcpu is created. > > > > > > > > > > KVM implemented one new ARCH_GET_XCOMP_SUPP system attribute API to > > > > > get host side supported_xcr0 and Qemu can decide if it can request > > > > > dynamically enabled XSAVE features permission. > > > > > https://lore.kernel.org/all/20220126152210.3044876-1-pbonz...@redhat.com/ > > > > > > > > > > Suggested-by: Paolo Bonzini > > > > > Signed-off-by: Yang Zhong > > > > > Signed-off-by: Jing Liu > > > > > Message-Id: <20220217060434.52460-4-yang.zh...@intel.com> > > > > > Signed-off-by: Paolo Bonzini > > > > > --- > > > > > target/i386/cpu.c | 7 + > > > > > target/i386/cpu.h | 4 +++ > > > > > target/i386/kvm/kvm-cpu.c | 12 > > > > > target/i386/kvm/kvm.c | 57 > > > > > ++ > > > > > target/i386/kvm/kvm_i386.h | 1 + > > > > > 5 files changed, 75 insertions(+), 6 deletions(-) > > > > > > > > With this commit qemu crashes for me when invoking the following > > > > QMP command: > > > > > > It is way worse than that even. If you remove '-S' you get an > > > immediate kaboom on startup on AMD hosts > > > > Which AMD CPU is in this host? > > AMD EPYC 7302P > > > With regards, > Daniel my 3970X - same issue. Best regards, Maxim Levitsky
Re: [PULL 15/22] x86: Grant AMX permission for guest
On Wed, Mar 16, 2022 at 04:57:39PM +0100, Peter Krempa wrote:
> On Tue, Mar 08, 2022 at 12:34:38 +0100, Paolo Bonzini wrote:
> > From: Yang Zhong
> >
> > Kernel allocates 4K xstate buffer by default. For XSAVE features
> > which require large state component (e.g. AMX), Linux kernel
> > dynamically expands the xstate buffer only after the process has
> > acquired the necessary permissions. Those are called dynamically-
> > enabled XSAVE features (or dynamic xfeatures).
> >
> > There are separate permissions for native tasks and guests.
> >
> > Qemu should request the guest permissions for dynamic xfeatures
> > which will be exposed to the guest. This only needs to be done
> > once before the first vcpu is created.
> >
> > KVM implemented one new ARCH_GET_XCOMP_SUPP system attribute API to
> > get host side supported_xcr0 and Qemu can decide if it can request
> > dynamically enabled XSAVE features permission.
> > https://lore.kernel.org/all/20220126152210.3044876-1-pbonz...@redhat.com/
> >
> > Suggested-by: Paolo Bonzini
> > Signed-off-by: Yang Zhong
> > Signed-off-by: Jing Liu
> > Message-Id: <20220217060434.52460-4-yang.zh...@intel.com>
> > Signed-off-by: Paolo Bonzini
> > ---
> > target/i386/cpu.c | 7 +
> > target/i386/cpu.h | 4 +++
> > target/i386/kvm/kvm-cpu.c | 12
> > target/i386/kvm/kvm.c | 57 ++
> > target/i386/kvm/kvm_i386.h | 1 +
> > 5 files changed, 75 insertions(+), 6 deletions(-)
>
> With this commit qemu crashes for me when invoking the following
> QMP command:
>
> $ ~pipo/git/qemu.git/build/qemu-system-x86_64 -S -no-user-config -nodefaults
> -nographic -machine none,accel=kvm -qmp stdio
> {"QMP": {"version": {"qemu": {"micro": 90, "minor": 2, "major": 6},
> "package": "v7.0.0-rc0-8-g1d60bb4b14"}, "capabilities": ["oob"]}}
> {'execute':'qmp_capabilities'}
> {"return": {}}
> {"execute":"qom-list-properties","arguments":{"typename":"max-x86_64-cpu"},"id":"libvirt-41"}
> qemu-system-x86_64: ../target/i386/kvm/kvm-cpu.c:105: kvm_cpu_xsave_init:
> Assertion `esa->size == eax' failed.
> Aborted (core dumped)
>
> Note that the above is on a box with an 'AMD Ryzen 9 3900X'.
>
> Curiously on a laptop with an Intel chip (Intel(R) Core(TM) i7-10610U)
> it seems to work.
Thanks for pointing this out!
In my side, no AMD machine can be used to try this issue, I listed the
FPU info from host kernel dmesg for reference.
root@984fee00bf64:~# dmesg | grep fpu
[0.00] x86/fpu: Supporting XSAVE feature 0x001: 'x87 floating point
registers'
[0.00] x86/fpu: Supporting XSAVE feature 0x002: 'SSE registers'
[0.00] x86/fpu: Supporting XSAVE feature 0x004: 'AVX registers'
[0.00] x86/fpu: Supporting XSAVE feature 0x020: 'AVX-512 opmask'
[0.00] x86/fpu: Supporting XSAVE feature 0x040: 'AVX-512 Hi256'
[0.00] x86/fpu: Supporting XSAVE feature 0x080: 'AVX-512 ZMM_Hi256'
[0.00] x86/fpu: Supporting XSAVE feature 0x200: 'Protection Keys User
registers'
[0.00] x86/fpu: Supporting XSAVE feature 0x400: 'PASID state'
[0.00] x86/fpu: Supporting XSAVE feature 0x2: 'AMX Tile config'
[0.00] x86/fpu: Supporting XSAVE feature 0x4: 'AMX Tile data'
[0.00] x86/fpu: xstate_offset[2]: 576, xstate_sizes[2]: 256
[0.00] x86/fpu: xstate_offset[5]: 832, xstate_sizes[5]: 64
[0.00] x86/fpu: xstate_offset[6]: 896, xstate_sizes[6]: 512
[0.00] x86/fpu: xstate_offset[7]: 1408, xstate_sizes[7]: 1024
[0.00] x86/fpu: xstate_offset[9]: 2432, xstate_sizes[9]:8
[0.00] x86/fpu: xstate_offset[10]: 2440, xstate_sizes[10]:8
[0.00] x86/fpu: xstate_offset[17]: 2496, xstate_sizes[17]: 64
[0.00] x86/fpu: xstate_offset[18]: 2560, xstate_sizes[18]: 8192
[0.00] x86/fpu: Enabled xstate features 0x606e7, context size is
10752 bytes, using 'compacted' format.
Paolo, if you have fix patch, I can double check this from Intel SPR server.
thanks!
Yang
Re: [PULL 15/22] x86: Grant AMX permission for guest
On Wed, Mar 16, 2022 at 05:48:04PM +, David Edmondson wrote: > On Wednesday, 2022-03-16 at 16:05:01 GMT, Daniel P. Berrangé wrote: > > > On Wed, Mar 16, 2022 at 04:57:39PM +0100, Peter Krempa wrote: > >> On Tue, Mar 08, 2022 at 12:34:38 +0100, Paolo Bonzini wrote: > >> > From: Yang Zhong > >> > > >> > Kernel allocates 4K xstate buffer by default. For XSAVE features > >> > which require large state component (e.g. AMX), Linux kernel > >> > dynamically expands the xstate buffer only after the process has > >> > acquired the necessary permissions. Those are called dynamically- > >> > enabled XSAVE features (or dynamic xfeatures). > >> > > >> > There are separate permissions for native tasks and guests. > >> > > >> > Qemu should request the guest permissions for dynamic xfeatures > >> > which will be exposed to the guest. This only needs to be done > >> > once before the first vcpu is created. > >> > > >> > KVM implemented one new ARCH_GET_XCOMP_SUPP system attribute API to > >> > get host side supported_xcr0 and Qemu can decide if it can request > >> > dynamically enabled XSAVE features permission. > >> > https://lore.kernel.org/all/20220126152210.3044876-1-pbonz...@redhat.com/ > >> > > >> > Suggested-by: Paolo Bonzini > >> > Signed-off-by: Yang Zhong > >> > Signed-off-by: Jing Liu > >> > Message-Id: <20220217060434.52460-4-yang.zh...@intel.com> > >> > Signed-off-by: Paolo Bonzini > >> > --- > >> > target/i386/cpu.c | 7 + > >> > target/i386/cpu.h | 4 +++ > >> > target/i386/kvm/kvm-cpu.c | 12 > >> > target/i386/kvm/kvm.c | 57 ++ > >> > target/i386/kvm/kvm_i386.h | 1 + > >> > 5 files changed, 75 insertions(+), 6 deletions(-) > >> > >> With this commit qemu crashes for me when invoking the following > >> QMP command: > > > > It is way worse than that even. If you remove '-S' you get an > > immediate kaboom on startup on AMD hosts > > Which AMD CPU is in this host? AMD EPYC 7302P With regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [PULL 15/22] x86: Grant AMX permission for guest
On Wednesday, 2022-03-16 at 16:05:01 GMT, Daniel P. Berrangé wrote: > On Wed, Mar 16, 2022 at 04:57:39PM +0100, Peter Krempa wrote: >> On Tue, Mar 08, 2022 at 12:34:38 +0100, Paolo Bonzini wrote: >> > From: Yang Zhong >> > >> > Kernel allocates 4K xstate buffer by default. For XSAVE features >> > which require large state component (e.g. AMX), Linux kernel >> > dynamically expands the xstate buffer only after the process has >> > acquired the necessary permissions. Those are called dynamically- >> > enabled XSAVE features (or dynamic xfeatures). >> > >> > There are separate permissions for native tasks and guests. >> > >> > Qemu should request the guest permissions for dynamic xfeatures >> > which will be exposed to the guest. This only needs to be done >> > once before the first vcpu is created. >> > >> > KVM implemented one new ARCH_GET_XCOMP_SUPP system attribute API to >> > get host side supported_xcr0 and Qemu can decide if it can request >> > dynamically enabled XSAVE features permission. >> > https://lore.kernel.org/all/20220126152210.3044876-1-pbonz...@redhat.com/ >> > >> > Suggested-by: Paolo Bonzini >> > Signed-off-by: Yang Zhong >> > Signed-off-by: Jing Liu >> > Message-Id: <20220217060434.52460-4-yang.zh...@intel.com> >> > Signed-off-by: Paolo Bonzini >> > --- >> > target/i386/cpu.c | 7 + >> > target/i386/cpu.h | 4 +++ >> > target/i386/kvm/kvm-cpu.c | 12 >> > target/i386/kvm/kvm.c | 57 ++ >> > target/i386/kvm/kvm_i386.h | 1 + >> > 5 files changed, 75 insertions(+), 6 deletions(-) >> >> With this commit qemu crashes for me when invoking the following >> QMP command: > > It is way worse than that even. If you remove '-S' you get an > immediate kaboom on startup on AMD hosts Which AMD CPU is in this host? > $ ./build/qemu-system-x86_64 -accel kvm > Unable to init server: Could not connect: Connection refused > qemu-system-x86_64: ../target/i386/kvm/kvm-cpu.c:105: kvm_cpu_xsave_init: > Assertion `esa->size == eax' failed. > Aborted (core dumped) > > With regards, > Daniel dme. -- My girl Friday, she no square.
Re: [PULL 15/22] x86: Grant AMX permission for guest
On 3/16/22 17:05, Daniel P. Berrangé wrote: On Wed, Mar 16, 2022 at 04:57:39PM +0100, Peter Krempa wrote: On Tue, Mar 08, 2022 at 12:34:38 +0100, Paolo Bonzini wrote: From: Yang Zhong Kernel allocates 4K xstate buffer by default. For XSAVE features which require large state component (e.g. AMX), Linux kernel dynamically expands the xstate buffer only after the process has acquired the necessary permissions. Those are called dynamically- enabled XSAVE features (or dynamic xfeatures). There are separate permissions for native tasks and guests. Qemu should request the guest permissions for dynamic xfeatures which will be exposed to the guest. This only needs to be done once before the first vcpu is created. KVM implemented one new ARCH_GET_XCOMP_SUPP system attribute API to get host side supported_xcr0 and Qemu can decide if it can request dynamically enabled XSAVE features permission. https://lore.kernel.org/all/20220126152210.3044876-1-pbonz...@redhat.com/ Suggested-by: Paolo Bonzini Signed-off-by: Yang Zhong Signed-off-by: Jing Liu Message-Id: <20220217060434.52460-4-yang.zh...@intel.com> Signed-off-by: Paolo Bonzini --- target/i386/cpu.c | 7 + target/i386/cpu.h | 4 +++ target/i386/kvm/kvm-cpu.c | 12 target/i386/kvm/kvm.c | 57 ++ target/i386/kvm/kvm_i386.h | 1 + 5 files changed, 75 insertions(+), 6 deletions(-) With this commit qemu crashes for me when invoking the following QMP command: It is way worse than that even. If you remove '-S' you get an immediate kaboom on startup on AMD hosts $ ./build/qemu-system-x86_64 -accel kvm Unable to init server: Could not connect: Connection refused qemu-system-x86_64: ../target/i386/kvm/kvm-cpu.c:105: kvm_cpu_xsave_init: Assertion `esa->size == eax' failed. Aborted (core dumped) I'll check it tomorrow, thanks. Paolo
Re: [PULL 15/22] x86: Grant AMX permission for guest
On Wed, Mar 16, 2022 at 04:57:39PM +0100, Peter Krempa wrote: > On Tue, Mar 08, 2022 at 12:34:38 +0100, Paolo Bonzini wrote: > > From: Yang Zhong > > > > Kernel allocates 4K xstate buffer by default. For XSAVE features > > which require large state component (e.g. AMX), Linux kernel > > dynamically expands the xstate buffer only after the process has > > acquired the necessary permissions. Those are called dynamically- > > enabled XSAVE features (or dynamic xfeatures). > > > > There are separate permissions for native tasks and guests. > > > > Qemu should request the guest permissions for dynamic xfeatures > > which will be exposed to the guest. This only needs to be done > > once before the first vcpu is created. > > > > KVM implemented one new ARCH_GET_XCOMP_SUPP system attribute API to > > get host side supported_xcr0 and Qemu can decide if it can request > > dynamically enabled XSAVE features permission. > > https://lore.kernel.org/all/20220126152210.3044876-1-pbonz...@redhat.com/ > > > > Suggested-by: Paolo Bonzini > > Signed-off-by: Yang Zhong > > Signed-off-by: Jing Liu > > Message-Id: <20220217060434.52460-4-yang.zh...@intel.com> > > Signed-off-by: Paolo Bonzini > > --- > > target/i386/cpu.c | 7 + > > target/i386/cpu.h | 4 +++ > > target/i386/kvm/kvm-cpu.c | 12 > > target/i386/kvm/kvm.c | 57 ++ > > target/i386/kvm/kvm_i386.h | 1 + > > 5 files changed, 75 insertions(+), 6 deletions(-) > > With this commit qemu crashes for me when invoking the following > QMP command: It is way worse than that even. If you remove '-S' you get an immediate kaboom on startup on AMD hosts $ ./build/qemu-system-x86_64 -accel kvm Unable to init server: Could not connect: Connection refused qemu-system-x86_64: ../target/i386/kvm/kvm-cpu.c:105: kvm_cpu_xsave_init: Assertion `esa->size == eax' failed. Aborted (core dumped) With regards, Daniel -- |: https://berrange.com -o-https://www.flickr.com/photos/dberrange :| |: https://libvirt.org -o-https://fstop138.berrange.com :| |: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|
Re: [PULL 15/22] x86: Grant AMX permission for guest
On Tue, Mar 08, 2022 at 12:34:38 +0100, Paolo Bonzini wrote:
> From: Yang Zhong
>
> Kernel allocates 4K xstate buffer by default. For XSAVE features
> which require large state component (e.g. AMX), Linux kernel
> dynamically expands the xstate buffer only after the process has
> acquired the necessary permissions. Those are called dynamically-
> enabled XSAVE features (or dynamic xfeatures).
>
> There are separate permissions for native tasks and guests.
>
> Qemu should request the guest permissions for dynamic xfeatures
> which will be exposed to the guest. This only needs to be done
> once before the first vcpu is created.
>
> KVM implemented one new ARCH_GET_XCOMP_SUPP system attribute API to
> get host side supported_xcr0 and Qemu can decide if it can request
> dynamically enabled XSAVE features permission.
> https://lore.kernel.org/all/20220126152210.3044876-1-pbonz...@redhat.com/
>
> Suggested-by: Paolo Bonzini
> Signed-off-by: Yang Zhong
> Signed-off-by: Jing Liu
> Message-Id: <20220217060434.52460-4-yang.zh...@intel.com>
> Signed-off-by: Paolo Bonzini
> ---
> target/i386/cpu.c | 7 +
> target/i386/cpu.h | 4 +++
> target/i386/kvm/kvm-cpu.c | 12
> target/i386/kvm/kvm.c | 57 ++
> target/i386/kvm/kvm_i386.h | 1 +
> 5 files changed, 75 insertions(+), 6 deletions(-)
With this commit qemu crashes for me when invoking the following
QMP command:
$ ~pipo/git/qemu.git/build/qemu-system-x86_64 -S -no-user-config -nodefaults
-nographic -machine none,accel=kvm -qmp stdio
{"QMP": {"version": {"qemu": {"micro": 90, "minor": 2, "major": 6}, "package":
"v7.0.0-rc0-8-g1d60bb4b14"}, "capabilities": ["oob"]}}
{'execute':'qmp_capabilities'}
{"return": {}}
{"execute":"qom-list-properties","arguments":{"typename":"max-x86_64-cpu"},"id":"libvirt-41"}
qemu-system-x86_64: ../target/i386/kvm/kvm-cpu.c:105: kvm_cpu_xsave_init:
Assertion `esa->size == eax' failed.
Aborted (core dumped)
Note that the above is on a box with an 'AMD Ryzen 9 3900X'.
Curiously on a laptop with an Intel chip (Intel(R) Core(TM) i7-10610U)
it seems to work.
