Re: [edk2-devel] GuestPhysAddrSize questions
On Mon, Mar 4, 2024 at 6:24 PM Tom Lendacky wrote: > > On 3/4/24 07:09, Gerd Hoffmann wrote: > >Hi, > > > >>> 23:16 GuestPhysAddrSize Maximum guest physical address size in bits. > >>> This number applies only to guests using > >>> nested > >>> paging. When this field is zero, refer to the > >>> PhysAddrSize field for the maximum guest > >>> physical address size. See “Secure Virtual > >>> Machine” in APM Volume 2. > > > >> I believe the main purpose of GuestPhysAddrSize was for software use (for > >> nested virtualization) and that the hardware itself has always returned > >> zero > >> for that value. So you should be able to use that field. Adding @Paolo for > >> his thoughts. > > > > Reviewers mentioned this is meant for nested guests, i.e. (if I > > understand this correctly) the l0 hypervisor can use that to tell > > the l1 hypervisor what the l2 guest phys-bits should be. > > > > Is this nested virtualization use documented somewhere? Tried to > > search for GuestPhysAddrSize or Fn8000_0008_EAX in APM Volume 2, > > found nothing. > > Right, and I don't think you'll see anything added to the APM that will > state how it can be used by software. The APM is an architectural > definition and won't talk about hypervisors and using nested paging, etc. I don't think that's a problem. The problem is that the definition in the APM is ambiguous and can mean one of three things: 1) it can be a suggested value for PhysAddrSize (bits 7:0) of guests that use nested paging. This would imply that in a nested page guest, with GuestPhysAddrSize=48, setting bits 51:48 would cause a #PF(reserved) exception. 2) it can be equivalent to LinAddrSize (bits 15:8) but for nested page tables. This would imply that, with GuestPhysAddrSize=48, VMRUN would fail if hCR4.LA57=1. 3) it can be what I propose above: the architecture defined a situation that can only happen when using nested paging (on AMD: host_CR4.LA57=0, PhysAddrSize=52), and GuestPhysAddrSize is an architectural way to explain the situation to guests. The message above suggests that the intended meaning is (1). That is because "the l0 hypervisor can use that to tell the l1 hypervisor what the l2 guest phys-bits should be" is exactly the same as "the processor can use that to tell the hypervisor what the guest phys-bits should be" (just shifted one level down). However, there are no processors that implement (1) or (2), so my suggestion is to clarify that the intended meaning is (3). Do you agree that the above proposal is a plausible interpretation of what is already in the APM, but clearer? And do you think there is a way for the clarification to make it into the APM? Paolo -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#116465): https://edk2.groups.io/g/devel/message/116465 Mute This Topic: https://groups.io/mt/104510523/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] GuestPhysAddrSize questions
On 3/4/24 07:09, Gerd Hoffmann wrote: Hi, 23:16 GuestPhysAddrSize Maximum guest physical address size in bits. This number applies only to guests using nested paging. When this field is zero, refer to the PhysAddrSize field for the maximum guest physical address size. See “Secure Virtual Machine” in APM Volume 2. I believe the main purpose of GuestPhysAddrSize was for software use (for nested virtualization) and that the hardware itself has always returned zero for that value. So you should be able to use that field. Adding @Paolo for his thoughts. Posted patches for kernel https://lore.kernel.org/kvm/20240301101410.356007-1-kra...@redhat.com/ and qemu https://lore.kernel.org/kvm/20240301101713.356759-1-kra...@redhat.com/ (sorry forgot to Cc you). Reviewers mentioned this is meant for nested guests, i.e. (if I understand this correctly) the l0 hypervisor can use that to tell the l1 hypervisor what the l2 guest phys-bits should be. Is this nested virtualization use documented somewhere? Tried to search for GuestPhysAddrSize or Fn8000_0008_EAX in APM Volume 2, found nothing. Right, and I don't think you'll see anything added to the APM that will state how it can be used by software. The APM is an architectural definition and won't talk about hypervisors and using nested paging, etc. Is there any case where the phys-bits limits for an l1 guest and l2 guest would be different? I haven't really thought about this before and all the implications that may or may not be in play, so I don't think I can really answer that. Thanks, Tom thanks & take care, Gerd -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#116332): https://edk2.groups.io/g/devel/message/116332 Mute This Topic: https://groups.io/mt/104510523/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] GuestPhysAddrSize questions
Hi, > >23:16 GuestPhysAddrSize Maximum guest physical address size in bits. > >This number applies only to guests using nested > >paging. When this field is zero, refer to the > >PhysAddrSize field for the maximum guest > >physical address size. See “Secure Virtual > >Machine” in APM Volume 2. > I believe the main purpose of GuestPhysAddrSize was for software use (for > nested virtualization) and that the hardware itself has always returned zero > for that value. So you should be able to use that field. Adding @Paolo for > his thoughts. Posted patches for kernel https://lore.kernel.org/kvm/20240301101410.356007-1-kra...@redhat.com/ and qemu https://lore.kernel.org/kvm/20240301101713.356759-1-kra...@redhat.com/ (sorry forgot to Cc you). Reviewers mentioned this is meant for nested guests, i.e. (if I understand this correctly) the l0 hypervisor can use that to tell the l1 hypervisor what the l2 guest phys-bits should be. Is this nested virtualization use documented somewhere? Tried to search for GuestPhysAddrSize or Fn8000_0008_EAX in APM Volume 2, found nothing. Is there any case where the phys-bits limits for an l1 guest and l2 guest would be different? thanks & take care, Gerd -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#116313): https://edk2.groups.io/g/devel/message/116313 Mute This Topic: https://groups.io/mt/104510523/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] GuestPhysAddrSize questions
On Thu, Feb 22, 2024 at 5:13 PM Paolo Bonzini wrote: > Also, to clarify the hardware behavior, if hCR4.LA57=0 and host > PhysAddrSize==52, then will guest physical addresses above 2^48 > > 1) cause a reserved #PF in the guest, or > > 2) cause a non-present NPF exit in the hypervisor? > > I remember that several years ago we had a discussion on hCR4.LA57=0 > reducing the address space compared to MAXPHYADDR, but I cannot find the > emails and also at the time I didn't notice GuestPhysAddrSize. Found them! They say that "according to the design document, CPU will report #NPF if the guest references a PA that is greater than 48 bits while the hypervisor is in 4-level nested paging mode". That's nice, because it's the same behavior as the affected Intel processors. Paolo > Anyhow, basically we would like GuestPhysAddrSize to be "redefined" as > > Maximum usable physical address size in bits. Physical addresses > above this size should not be used, but will not produce a "reserved" > page fault. When this field is zero, all bits up to PhysAddrSize are > usable. This field is expected to be nonzero only on guests where > the hypervisor is using nested paging. -=-=-=-=-=-=-=-=-=-=-=- Groups.io Links: You receive all messages sent to this group. View/Reply Online (#115859): https://edk2.groups.io/g/devel/message/115859 Mute This Topic: https://groups.io/mt/104510523/21656 Group Owner: devel+ow...@edk2.groups.io Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] GuestPhysAddrSize questions
On 2/22/24 16:44, Tom Lendacky wrote:
On 2/22/24 05:24, Gerd Hoffmann wrote:
Hi,
+ if (Cr4.Bits.LA57) {
+ if (PhysBits > 48) {
+ /*
+ * Some Intel CPUs support 5-level paging, have more than 48
+ * phys-bits but support only 4-level EPT, which effectively
+ * limits guest phys-bits to 48.
+ *
+ * AMD Processors have a different but somewhat related
+ * problem: They can handle guest phys-bits larger than 48
+ * only in case the host runs in 5-level paging mode.
+ *
+ * Until we have some way to communicate that kind of
+ * limitations from hypervisor to guest, limit phys-bits
+ * to 48 unconditionally.
+ */
So I'm looking for some communication path. One option would be to use
some bits in the KVM cpuid leaves. Another possible candidate is cpuid
leaf 0x8008.
From the AMD APM (revision 3.35):
CPUID Fn8000_0008_EAX Long Mode Size Identifiers
The value returned in EAX provides information about the maximum host
and guest physical and linear address width (in bits) supported by the
processor.
Bits FieldName Description
31:24 — Reserved
23:16 GuestPhysAddrSize Maximum guest physical address size in bits.
This number applies only to guests using
nested
paging. When this field is zero, refer to the
PhysAddrSize field for the maximum guest
physical address size. See “Secure Virtual
Machine” in APM Volume 2.
15:8 LinAddrSize Maximum linear address size in bits.
7:0 PhysAddrSize Maximum physical address size in bits. When
GuestPhysAddrSize is zero, this field also
indicates the maximum guest physical address
size.
The description of the GuestPhysAddrSize is somewhat vague. Is this a
value the hypervisor should use to figure how much address space it can
give to guests? Or is this a value the hypervisor can set to inform the
guest about the available address space (which would be a solution to
the problem outlined in the comment above)? Or both?
I believe the main purpose of GuestPhysAddrSize was for software use
(for nested virtualization) and that the hardware itself has always
returned zero for that value. So you should be able to use that field.
Adding @Paolo for his thoughts.
I have already discussed this with Gerd, so I don't really have many
thoughts to add.
Anyhow, basically we would like GuestPhysAddrSize to be "redefined" as
Maximum usable physical address size in bits. Physical addresses
above this size should not be used, but will not produce a "reserved"
page fault. When this field is zero, all bits up to PhysAddrSize are
usable. This field is expected to be nonzero only on guests where
the hypervisor is using nested paging.
Also, to clarify the hardware behavior, if hCR4.LA57=0 and host
PhysAddrSize==52, then will guest physical addresses above 2^48
1) cause a reserved #PF in the guest, or
2) cause a non-present NPF exit in the hypervisor?
I remember that several years ago we had a discussion on hCR4.LA57=0
reducing the address space compared to MAXPHYADDR, but I cannot find the
emails and also at the time I didn't notice GuestPhysAddrSize.
Paolo
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#115832): https://edk2.groups.io/g/devel/message/115832
Mute This Topic: https://groups.io/mt/104510523/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [edk2-devel] GuestPhysAddrSize questions
On 2/22/24 05:24, Gerd Hoffmann wrote:
Hi,
+if (Cr4.Bits.LA57) {
+ if (PhysBits > 48) {
+/*
+ * Some Intel CPUs support 5-level paging, have more than 48
+ * phys-bits but support only 4-level EPT, which effectively
+ * limits guest phys-bits to 48.
+ *
+ * AMD Processors have a different but somewhat related
+ * problem: They can handle guest phys-bits larger than 48
+ * only in case the host runs in 5-level paging mode.
+ *
+ * Until we have some way to communicate that kind of
+ * limitations from hypervisor to guest, limit phys-bits
+ * to 48 unconditionally.
+ */
So I'm looking for some communication path. One option would be to use
some bits in the KVM cpuid leaves. Another possible candidate is cpuid
leaf 0x8008.
From the AMD APM (revision 3.35):
CPUID Fn8000_0008_EAX Long Mode Size Identifiers
The value returned in EAX provides information about the maximum host
and guest physical and linear address width (in bits) supported by the
processor.
Bits FieldNameDescription
31:24 —Reserved
23:16 GuestPhysAddrSize Maximum guest physical address size in bits.
This number applies only to guests using nested
paging. When this field is zero, refer to the
PhysAddrSize field for the maximum guest
physical address size. See “Secure Virtual
Machine” in APM Volume 2.
15:8 LinAddrSize Maximum linear address size in bits.
7:0 PhysAddrSize Maximum physical address size in bits. When
GuestPhysAddrSize is zero, this field also
indicates the maximum guest physical address
size.
The description of the GuestPhysAddrSize is somewhat vague. Is this a
value the hypervisor should use to figure how much address space it can
give to guests? Or is this a value the hypervisor can set to inform the
guest about the available address space (which would be a solution to
the problem outlined in the comment above)? Or both?
I believe the main purpose of GuestPhysAddrSize was for software use (for
nested virtualization) and that the hardware itself has always returned
zero for that value. So you should be able to use that field. Adding
@Paolo for his thoughts.
Thanks,
Tom
In case GuestPhysAddrSize should not be used this way: Is it possible
allocate the currently reserved bits 31:24 for that purpose?
Tom? Michael?
thanks & take care,
Gerd
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#115824): https://edk2.groups.io/g/devel/message/115824
Mute This Topic: https://groups.io/mt/104510523/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
[edk2-devel] GuestPhysAddrSize questions (was: Re: [PATCH v4 3/3] OvmfPkg/PlatformInitLib: add 5-level paging) support
Hi,
> +if (Cr4.Bits.LA57) {
> + if (PhysBits > 48) {
> +/*
> + * Some Intel CPUs support 5-level paging, have more than 48
> + * phys-bits but support only 4-level EPT, which effectively
> + * limits guest phys-bits to 48.
> + *
> + * AMD Processors have a different but somewhat related
> + * problem: They can handle guest phys-bits larger than 48
> + * only in case the host runs in 5-level paging mode.
> + *
> + * Until we have some way to communicate that kind of
> + * limitations from hypervisor to guest, limit phys-bits
> + * to 48 unconditionally.
> + */
So I'm looking for some communication path. One option would be to use
some bits in the KVM cpuid leaves. Another possible candidate is cpuid
leaf 0x8008.
>From the AMD APM (revision 3.35):
CPUID Fn8000_0008_EAX Long Mode Size Identifiers
The value returned in EAX provides information about the maximum host
and guest physical and linear address width (in bits) supported by the
processor.
Bits FieldNameDescription
31:24 —Reserved
23:16 GuestPhysAddrSize Maximum guest physical address size in bits.
This number applies only to guests using nested
paging. When this field is zero, refer to the
PhysAddrSize field for the maximum guest
physical address size. See “Secure Virtual
Machine” in APM Volume 2.
15:8 LinAddrSize Maximum linear address size in bits.
7:0 PhysAddrSize Maximum physical address size in bits. When
GuestPhysAddrSize is zero, this field also
indicates the maximum guest physical address
size.
The description of the GuestPhysAddrSize is somewhat vague. Is this a
value the hypervisor should use to figure how much address space it can
give to guests? Or is this a value the hypervisor can set to inform the
guest about the available address space (which would be a solution to
the problem outlined in the comment above)? Or both?
In case GuestPhysAddrSize should not be used this way: Is it possible
allocate the currently reserved bits 31:24 for that purpose?
Tom? Michael?
thanks & take care,
Gerd
-=-=-=-=-=-=-=-=-=-=-=-
Groups.io Links: You receive all messages sent to this group.
View/Reply Online (#115801): https://edk2.groups.io/g/devel/message/115801
Mute This Topic: https://groups.io/mt/104506481/21656
Group Owner: devel+ow...@edk2.groups.io
Unsubscribe: https://edk2.groups.io/g/devel/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
