Re: [Xen-devel] [PATCH] xen/arm: p2m: Don't use default access permission when shattering a superpage

[Stefano Stabellini]

On Fri, 29 Jul 2016, Julien Grall wrote:
> The following message flood the console when memaccess is enabled on
> various platforms:
> 
> traps.c:2510:d1v0 HSR=0x9383004f pc=0x08b7d4c4 gva=0x08eeb8e0 
> gpa=0x004903f8e0
> 
> This is because a data abort from a guest was received due to a
> permission fault but memaccess thought there are no permission fault.
> 
> On ARM, memaccess permissions are stored in a radix tree because there
> are not enough available bits in the p2m entry to store the access
> restriction. When memaccess is restricting the access (i.e any other
> access than p2m_access_rwx), the access will be added in the radix tree
> using the GFN as a key. This will be done for all 4KB pages.
> 
> This means that memaccess has to shatter all the superpages in a given
> region to set the permission on a 4KB granularity. Currently, when a
> superpage is shattered, the new entries are using the value
> p2m->default_access which will restrict permission (because memaccess
> has been enabled). However the radix tree does not yet contain
> an entry for this GFN.
> 
> If a guest VCPU is running at the same time and trying to access the
> modified region, it will result to a stage-2 permission fault. As
> the radix tree does not yet contain an entry for the GFN, memaccess will
> deduce that the fault was not valid and a data abort will be injecting
> to the guest (and crash it).
> 
> Furthermore, the permission may be restricted outside of the requested
> region if it is only a subset of a 1GB/2MB superpage.
> 
> The two issues can be fixed by re-using the permission of the superpage
> entry and override the necessary fields. This is not a problem because
> memaccess cannot work on superpage.
> 
> Lastly, document the code which call mfn_to_p2m_entry when creating a
> the p2m entry for a table to explain that create the p2m entry to page table
> to explain that permission are ignored by the hardware (See D4.3.1 in ARM DDI
> 0487A.j). so the value of the parameter 'access' of mfn_to_p2m_entry does
> not matter.
> 
> Signed-off-by: Julien Grall 

Reviewed-by: Stefano Stabellini 


> ---
> This patch needs to be backported up to Xen 4.6 (first release
> which introduced memaccess on ARM). This may require few adjustement
> because the code has changed a bit.
> 
> Without it, the guest will randomly crash because the permission has
> been overriden whilst shattering a superpage and before adding the access
> permission in the radix tree.
> 
> Also I am wondering if it would be better to pass p2m_access_rwx
> to mfn_to_p2m_entry when creating a table entry. This would save
> a couple of instructions.
> ---
>  xen/arch/arm/p2m.c | 18 +-
>  1 file changed, 13 insertions(+), 5 deletions(-)
> 
> diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
> index 40a0b80..d60fbbf 100644
> --- a/xen/arch/arm/p2m.c
> +++ b/xen/arch/arm/p2m.c
> @@ -434,7 +434,6 @@ static int p2m_create_table(struct p2m_domain *p2m, 
> lpae_t *entry,
>  p = __map_domain_page(page);
>  if ( splitting )
>  {
> -p2m_type_t t = entry->p2m.type;
>  mfn_t mfn = _mfn(entry->p2m.base);
>  int i;
>  
> @@ -444,15 +443,20 @@ static int p2m_create_table(struct p2m_domain *p2m, 
> lpae_t *entry,
>   */
>   for ( i=0 ; i < LPAE_ENTRIES; i++ )
>   {
> - pte = mfn_to_p2m_entry(mfn_add(mfn, i << (level_shift - 
> LPAE_SHIFT)),
> -t, p2m->default_access);
> + /*
> +  * Use the content of the superpage entry and override
> +  * the necessary fields. So the correct permissions are
> +  * kept.
> +  */
> + pte = *entry;
> + pte.p2m.base = mfn_x(mfn_add(mfn,
> +  i << (level_shift - LPAE_SHIFT)));
>  
>   /*
>* First and second level super pages set p2m.table = 0, but
>* third level entries set table = 1.
>*/
> - if ( level_shift - LPAE_SHIFT )
> - pte.p2m.table = 0;
> + pte.p2m.table = !(level_shift - LPAE_SHIFT);
>  
>   write_pte([i], pte);
>   }
> @@ -467,6 +471,10 @@ static int p2m_create_table(struct p2m_domain *p2m, 
> lpae_t *entry,
>  
>  unmap_domain_page(p);
>  
> +/*
> + * The access value does not matter because the hardware will ignore
> + * the permission fields for table entry.
> + */
>  pte = mfn_to_p2m_entry(_mfn(page_to_mfn(page)), p2m_invalid,
> p2m->default_access);
>  
> -- 
> 1.9.1
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: p2m: Don't use default access permission when shattering a superpage

[Julien Grall]

On 30/07/16 00:21, Tamas K Lengyel wrote:

On Fri, Jul 29, 2016 at 12:53 PM, Julien Grall  wrote:

The following message flood the console when memaccess is enabled on
various platforms:

traps.c:2510:d1v0 HSR=0x9383004f pc=0x08b7d4c4 gva=0x08eeb8e0 
gpa=0x004903f8e0

This is because a data abort from a guest was received due to a
permission fault but memaccess thought there are no permission fault.

On ARM, memaccess permissions are stored in a radix tree because there
are not enough available bits in the p2m entry to store the access
restriction. When memaccess is restricting the access (i.e any other
access than p2m_access_rwx), the access will be added in the radix tree
using the GFN as a key. This will be done for all 4KB pages.

This means that memaccess has to shatter all the superpages in a given
region to set the permission on a 4KB granularity. Currently, when a
superpage is shattered, the new entries are using the value
p2m->default_access which will restrict permission (because memaccess
has been enabled). However the radix tree does not yet contain
an entry for this GFN.

If a guest VCPU is running at the same time and trying to access the
modified region, it will result to a stage-2 permission fault. As
the radix tree does not yet contain an entry for the GFN, memaccess will
deduce that the fault was not valid and a data abort will be injecting
to the guest (and crash it).

Furthermore, the permission may be restricted outside of the requested
region if it is only a subset of a 1GB/2MB superpage.

The two issues can be fixed by re-using the permission of the superpage
entry and override the necessary fields. This is not a problem because
memaccess cannot work on superpage.

Lastly, document the code which call mfn_to_p2m_entry when creating a
the p2m entry for a table to explain that create the p2m entry to page table
to explain that permission are ignored by the hardware (See D4.3.1 in ARM DDI
0487A.j). so the value of the parameter 'access' of mfn_to_p2m_entry does
not matter.

Signed-off-by: Julien Grall 


Thanks for looking into this, it is very strange that this issue has
not surfaced before as mem_access was extensively tested on the
Arndale and the XU during the 4.6 merge. Maybe it just happened that
the superpage shattering path was never hit during the tests. It still
does work fine on my Cubietruck as of the latest staging..


Unless the memory is very fragmented, you will likely use 2MB superpages 
(you can find the number of 1GB, 2MB, 4KB mappings via the Xen console, 
key 'q'):


(XEN) p2m mappings for domain 0 (vmid 1):
(XEN)   1G mappings: 7 (shattered 0)
(XEN)   2M mappings: 196 (shattered 0)
(XEN)   4K mappings: 170

My setup was very basic, DOM0 has the same number of vCPUs as the number 
of pCPUs. A Linux guest [1] is started and then xen-access is called 
(see script [2]).


I have tested with various Linux guest: full distribution (Debian) and 
buildroot initramfs. The platform was Foundation model [3] and Juno-r2, 
although the processors are ARMv8 but I don't expect this to be a reason.


Regards,

[1]  42sh# cat guest.cfg
name="guest"
memory= "256"
kernel= "/root/Image"
extra= "console=hvc0 root=/dev/ram1"
ramdisk= "/root/initramfs"

[2] 42sh# cat init.sh
#! /bin/sh
mount -t proc proc  /proc
mount -t devtmpfs dev /dev

export HOME=/root

/etc/init.d/xencommons start
xl -vvv create /root/guest.cfg
sleep 2

echo "Calling memaccess"
/root/xen/tools/tests/xen-access/xen-access 1 write

exec /bin/bash

[3] http://www.arm.com/products/tools/models/foundation-model.php



Tamas



--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH] xen/arm: p2m: Don't use default access permission when shattering a superpage

[Tamas K Lengyel]

On Fri, Jul 29, 2016 at 12:53 PM, Julien Grall  wrote:
> The following message flood the console when memaccess is enabled on
> various platforms:
>
> traps.c:2510:d1v0 HSR=0x9383004f pc=0x08b7d4c4 gva=0x08eeb8e0 
> gpa=0x004903f8e0
>
> This is because a data abort from a guest was received due to a
> permission fault but memaccess thought there are no permission fault.
>
> On ARM, memaccess permissions are stored in a radix tree because there
> are not enough available bits in the p2m entry to store the access
> restriction. When memaccess is restricting the access (i.e any other
> access than p2m_access_rwx), the access will be added in the radix tree
> using the GFN as a key. This will be done for all 4KB pages.
>
> This means that memaccess has to shatter all the superpages in a given
> region to set the permission on a 4KB granularity. Currently, when a
> superpage is shattered, the new entries are using the value
> p2m->default_access which will restrict permission (because memaccess
> has been enabled). However the radix tree does not yet contain
> an entry for this GFN.
>
> If a guest VCPU is running at the same time and trying to access the
> modified region, it will result to a stage-2 permission fault. As
> the radix tree does not yet contain an entry for the GFN, memaccess will
> deduce that the fault was not valid and a data abort will be injecting
> to the guest (and crash it).
>
> Furthermore, the permission may be restricted outside of the requested
> region if it is only a subset of a 1GB/2MB superpage.
>
> The two issues can be fixed by re-using the permission of the superpage
> entry and override the necessary fields. This is not a problem because
> memaccess cannot work on superpage.
>
> Lastly, document the code which call mfn_to_p2m_entry when creating a
> the p2m entry for a table to explain that create the p2m entry to page table
> to explain that permission are ignored by the hardware (See D4.3.1 in ARM DDI
> 0487A.j). so the value of the parameter 'access' of mfn_to_p2m_entry does
> not matter.
>
> Signed-off-by: Julien Grall 

Thanks for looking into this, it is very strange that this issue has
not surfaced before as mem_access was extensively tested on the
Arndale and the XU during the 4.6 merge. Maybe it just happened that
the superpage shattering path was never hit during the tests. It still
does work fine on my Cubietruck as of the latest staging..

Tamas

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

[Xen-devel] [PATCH] xen/arm: p2m: Don't use default access permission when shattering a superpage

[Julien Grall]

The following message flood the console when memaccess is enabled on
various platforms:

traps.c:2510:d1v0 HSR=0x9383004f pc=0x08b7d4c4 gva=0x08eeb8e0 
gpa=0x004903f8e0

This is because a data abort from a guest was received due to a
permission fault but memaccess thought there are no permission fault.

On ARM, memaccess permissions are stored in a radix tree because there
are not enough available bits in the p2m entry to store the access
restriction. When memaccess is restricting the access (i.e any other
access than p2m_access_rwx), the access will be added in the radix tree
using the GFN as a key. This will be done for all 4KB pages.

This means that memaccess has to shatter all the superpages in a given
region to set the permission on a 4KB granularity. Currently, when a
superpage is shattered, the new entries are using the value
p2m->default_access which will restrict permission (because memaccess
has been enabled). However the radix tree does not yet contain
an entry for this GFN.

If a guest VCPU is running at the same time and trying to access the
modified region, it will result to a stage-2 permission fault. As
the radix tree does not yet contain an entry for the GFN, memaccess will
deduce that the fault was not valid and a data abort will be injecting
to the guest (and crash it).

Furthermore, the permission may be restricted outside of the requested
region if it is only a subset of a 1GB/2MB superpage.

The two issues can be fixed by re-using the permission of the superpage
entry and override the necessary fields. This is not a problem because
memaccess cannot work on superpage.

Lastly, document the code which call mfn_to_p2m_entry when creating a
the p2m entry for a table to explain that create the p2m entry to page table
to explain that permission are ignored by the hardware (See D4.3.1 in ARM DDI
0487A.j). so the value of the parameter 'access' of mfn_to_p2m_entry does
not matter.

Signed-off-by: Julien Grall 

---
This patch needs to be backported up to Xen 4.6 (first release
which introduced memaccess on ARM). This may require few adjustement
because the code has changed a bit.

Without it, the guest will randomly crash because the permission has
been overriden whilst shattering a superpage and before adding the access
permission in the radix tree.

Also I am wondering if it would be better to pass p2m_access_rwx
to mfn_to_p2m_entry when creating a table entry. This would save
a couple of instructions.
---
 xen/arch/arm/p2m.c | 18 +-
 1 file changed, 13 insertions(+), 5 deletions(-)

diff --git a/xen/arch/arm/p2m.c b/xen/arch/arm/p2m.c
index 40a0b80..d60fbbf 100644
--- a/xen/arch/arm/p2m.c
+++ b/xen/arch/arm/p2m.c
@@ -434,7 +434,6 @@ static int p2m_create_table(struct p2m_domain *p2m, lpae_t 
*entry,
 p = __map_domain_page(page);
 if ( splitting )
 {
-p2m_type_t t = entry->p2m.type;
 mfn_t mfn = _mfn(entry->p2m.base);
 int i;
 
@@ -444,15 +443,20 @@ static int p2m_create_table(struct p2m_domain *p2m, 
lpae_t *entry,
  */
  for ( i=0 ; i < LPAE_ENTRIES; i++ )
  {
- pte = mfn_to_p2m_entry(mfn_add(mfn, i << (level_shift - 
LPAE_SHIFT)),
-t, p2m->default_access);
+ /*
+  * Use the content of the superpage entry and override
+  * the necessary fields. So the correct permissions are
+  * kept.
+  */
+ pte = *entry;
+ pte.p2m.base = mfn_x(mfn_add(mfn,
+  i << (level_shift - LPAE_SHIFT)));
 
  /*
   * First and second level super pages set p2m.table = 0, but
   * third level entries set table = 1.
   */
- if ( level_shift - LPAE_SHIFT )
- pte.p2m.table = 0;
+ pte.p2m.table = !(level_shift - LPAE_SHIFT);
 
  write_pte([i], pte);
  }
@@ -467,6 +471,10 @@ static int p2m_create_table(struct p2m_domain *p2m, lpae_t 
*entry,
 
 unmap_domain_page(p);
 
+/*
+ * The access value does not matter because the hardware will ignore
+ * the permission fields for table entry.
+ */
 pte = mfn_to_p2m_entry(_mfn(page_to_mfn(page)), p2m_invalid,
p2m->default_access);
 
-- 
1.9.1


___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel