subject:"\[Xen\-devel\] Design doc of adding ACPI support for arm64 on Xen \- version 2"

 On 12.08.15 at 18:18, julien.gr...@citrix.com wrote:
 On 12/08/15 16:58, Jan Beulich wrote:
 On 12.08.15 at 17:51, christoffer.d...@linaro.org wrote:
 On Wed, Aug 12, 2015 at 5:45 PM, Jan Beulich jbeul...@suse.com wrote:
 On 07.08.15 at 04:11, zhaoshengl...@huawei.com wrote:
 All these tables will be copied to Dom0 memory except that the reused
 tables(DSDT, SPCR, etc) will be mapped to Dom0.

 Which again seems odd - such tables should be considered MMIO
 (despite living in RAM), and hence not be part of Dom0's memory
 assignment.

 not sure if this applies to the context of your comment, but we had
 issues before when trying to deal with this data as device memory
 (MMIO), because ARM doesn't allow unaligned accesses etc. on device
 memory, and so Dom0 would crash at memory abort exceptions during
 boot, so this really does have to be mapped as RAM to Dom0.  If you
 meant some internal Xen bookkeeping thing, then just ignore me.
 
 Hmm, how would native Linux avoid such unaligned accesses then?
 
 ACPI table are living on RAM on ARM. So there is no issue with Linux
 baremetal.

I'm sure our interpretation of the meaning of RAM here differs:
While from a physical perspective the tables live in RAM too on
x86, from a memory map perspective they don't. And since iirc
ACPI implies UEFI, and since UEFI has special memory types
for such tables (to prevent the OS from using the space as
normal memory), I can't believe this to be different under ARM.

 The problem is from Xen where we are mapping memory region with Device
 attribute. This is because until now we never had a reason to directly
 map other thing than MMIO to the domain.
 
 This could be fixed by adding new helper in Xen to directly map RAM region.

Which would seem to be the correct route.

 Nonetheless, we still have to copy some table in Xen in order to modify
 them and/or new one. I have in mind the FADT table to set the hypervisor
 field and hiding the hypervisor specific data (GIC hyp, timer hyp...) to
 avoid the kernel thinks there is hyp mode available.

Have to? Or rather would like to? As said in another reply to
Shannon, I didn't see any rationale spelled out for this fundamental
difference to x86.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

 On 13.08.15 at 10:01, christoffer.d...@linaro.org wrote:
 On Thu, Aug 13, 2015 at 8:41 AM, Jan Beulich jbeul...@suse.com wrote:
 On 12.08.15 at 18:18, julien.gr...@citrix.com wrote:
 Nonetheless, we still have to copy some table in Xen in order to modify
 them and/or new one. I have in mind the FADT table to set the hypervisor
 field and hiding the hypervisor specific data (GIC hyp, timer hyp...) to
 avoid the kernel thinks there is hyp mode available.

 Have to? Or rather would like to? As said in another reply to
 Shannon, I didn't see any rationale spelled out for this fundamental
 difference to x86.

 So the suggestion is to just edit the tables in place rather than
 copying them and relying on Xen having already read whatever
 'original' information it needs before modifying them?

And just to clarify / amend my reply just sent: I also don't see why
you would need to expose the System Table to Dom0. Did anyone
of you look at how we draw the boundary between Xen and Dom0
on x86? If not, I'd suggest doing so. If so, I'd like to ask for
proper reasoning for any item of data exposed to Dom0 beyond
what we currently expose.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Thu, 2015-08-13 at 00:41 -0600, Jan Beulich wrote:
  
  Nonetheless, we still have to copy some table in Xen in order to modify
  them and/or new one. I have in mind the FADT table to set the hypervisor
  field and hiding the hypervisor specific data (GIC hyp, timer hyp...) to
  avoid the kernel thinks there is hyp mode available.
 
 Have to? Or rather would like to? As said in another reply to
 Shannon, I didn't see any rationale spelled out for this fundamental
 difference to x86.

I think the fundamental difference is that h/w virt features are not
(always) discoverable through h/w interfaces on ARM whereas they are
visible in e.g. cpuid on x86 and not described in ACPI (x86 is only just
gaining hardware support for virtualised interrupts so perhaps this will
change? I think it doesn't have any hypervisor dedicated timer sources).

So on ARM the firmware tables contain things like the additional
virtualisation register regions in the interrupt controller description and
the hypervisor timer interrupt in the timer block description. So we would
like to hide these from dom0.

Perhaps instead we should teach dom0 to notice that it was launched in
Kernel mode rather than Hyp mode (which is detectable) and therefore ignore
these unusable resources. Now you mention it that does sound sensible and I
imagine is even already (close to) true for a Linux kernel to handle e.g.
older firmware with newer device tree.

The other reason for modification is to hide the Xen console device (i.e.
one UART) from the dom0 kernel, since it is unusable. How does that work on
x86? Do you just not bother and you expect the admin to arrange the
configuration to work or is there some other trick?

BTW, IIRC x86 does modify at least one ACPI table which is the DMAR (I
think), to hide the IOMMU from the guest? That's another table we would
want to frob on ARM I think (or it's equivalent, which I think is IORT).

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

 On 13.08.15 at 10:01, christoffer.d...@linaro.org wrote:
 On Thu, Aug 13, 2015 at 8:41 AM, Jan Beulich jbeul...@suse.com wrote:
 On 12.08.15 at 18:18, julien.gr...@citrix.com wrote:
 Nonetheless, we still have to copy some table in Xen in order to modify
 them and/or new one. I have in mind the FADT table to set the hypervisor
 field and hiding the hypervisor specific data (GIC hyp, timer hyp...) to
 avoid the kernel thinks there is hyp mode available.

 Have to? Or rather would like to? As said in another reply to
 Shannon, I didn't see any rationale spelled out for this fundamental
 difference to x86.

 So the suggestion is to just edit the tables in place rather than
 copying them and relying on Xen having already read whatever
 'original' information it needs before modifying them?

No, the suggestion is to leave the tables alone and teach Dom0
to ignore parts it has no interest in.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-13 Thread Christoffer Dall

On Thu, Aug 13, 2015 at 11:22:19AM +0100, Ian Campbell wrote:
 On Thu, 2015-08-13 at 11:13 +0100, Stefano Stabellini wrote:
  
For example it is only natural for the kernel to try to use the GIC 
hyp
functionalities if they are described, while actually they are not
emulated by Xen at all.
   
   See Ian's earlier reply: It can also be considered natural for it to
   be aware that when run in EL2 to not use EL1 functionality.
 
 NB EL2 == Hyp and EL1 == Kernel, so it's the other way round, FWIW.
 
  It is not just about the GIC Hyp functionalities.
 
 What else is there which is not subject to this logic? Timers are too, it
 even applies to IOMMU's which have both stage1 and stage2 bits.
 
 BTW, I think kernels _already_ need to deal with a lot of this because in
 reality nobody modifies the DTB when they use a firmware which launches the
 kernel in EL1. IOW I think the kernel is already aware of which resources
 can be used by which privilege level.
 
Yes, for resources specific to EL2 I believe that is indeed the case
(the GIC driver doesn't look at the hypervisor control register address,
and KVM does not even get that far if you're not booted in EL2, and the
timer only uses the virtual timer if not booted in EL2 - we never
attempt to use the hyp timer until Marc's VHE patches land, but they
also depend on being booted in hyp mode).

However, what about for other resources?  Having code somewhere that
says hide this random piece of hardware if you're Xen dom0 sounds
awful to me.  I know it's only the serial port right now, but still.

-Christoffer

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Thu, 2015-08-13 at 12:29 +0200, Christoffer Dall wrote:
 On Thu, Aug 13, 2015 at 11:22:19AM +0100, Ian Campbell wrote:
  On Thu, 2015-08-13 at 11:13 +0100, Stefano Stabellini wrote:
   
 For example it is only natural for the kernel to try to use the 
 GIC 
 hyp
 functionalities if they are described, while actually they are 
 not
 emulated by Xen at all.

See Ian's earlier reply: It can also be considered natural for it 
to
be aware that when run in EL2 to not use EL1 functionality.
  
  NB EL2 == Hyp and EL1 == Kernel, so it's the other way round, FWIW.
  
   It is not just about the GIC Hyp functionalities.
  
  What else is there which is not subject to this logic? Timers are too, 
  it
  even applies to IOMMU's which have both stage1 and stage2 bits.
  
  BTW, I think kernels _already_ need to deal with a lot of this because 
  in
  reality nobody modifies the DTB when they use a firmware which launches 
  the
  kernel in EL1. IOW I think the kernel is already aware of which 
  resources
  can be used by which privilege level.
  
 Yes, for resources specific to EL2 I believe that is indeed the case
 (the GIC driver doesn't look at the hypervisor control register address,
 and KVM does not even get that far if you're not booted in EL2, and the
 timer only uses the virtual timer if not booted in EL2 - we never
 attempt to use the hyp timer until Marc's VHE patches land, but they
 also depend on being booted in hyp mode).

Right, and I think that's almost always going to be the case by virtue of
the architecture. You can't use these resources from EL1 (however you got
there) so you need checks.

 However, what about for other resources?  Having code somewhere that
 says hide this random piece of hardware if you're Xen dom0 sounds
 awful to me.  I know it's only the serial port right now, but still.

Right, that's the only bit I'm aware of that I'm not sure about, which is
why I was curious how x86 managed it (I've seen but not digested Jan's
answer).

At least this has the virtue of being an extra table, and tagging on a new
one of those has more limited impact on the tables I think.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

 On 13.08.15 at 11:05, ian.campb...@citrix.com wrote:
 The other reason for modification is to hide the Xen console device (i.e.
 one UART) from the dom0 kernel, since it is unusable. How does that work on
 x86? Do you just not bother and you expect the admin to arrange the
 configuration to work or is there some other trick?

When it's I/O port based we simply disallow access to the ports (i.e.
Dom0 reads return all ones, Dom0 writes get dropped). When it's
MMIO based (which iirc became an option only in the not so distant
past) we can't always do that, since there could be other devices
on the same MMIO page. In any event we then pci_hide_devices() or
pci_ro_device() the device (see ns16550_init_postirq()) to limit the
damage Dom0 can do.

 BTW, IIRC x86 does modify at least one ACPI table which is the DMAR (I
 think), to hide the IOMMU from the guest? That's another table we would
 want to frob on ARM I think (or it's equivalent, which I think is IORT).

Eliminating that hack is supposed to be on the VT-d maintainers'
TODO list(s) - Dom0 has no business looking at that table (and its
AMD counterpart already isn't being fiddled with in the same way).

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Thu, 13 Aug 2015, Jan Beulich wrote:
  On 12.08.15 at 18:18, julien.gr...@citrix.com wrote:
  On 12/08/15 16:58, Jan Beulich wrote:
  On 12.08.15 at 17:51, christoffer.d...@linaro.org wrote:
  On Wed, Aug 12, 2015 at 5:45 PM, Jan Beulich jbeul...@suse.com wrote:
  On 07.08.15 at 04:11, zhaoshengl...@huawei.com wrote:
  All these tables will be copied to Dom0 memory except that the reused
  tables(DSDT, SPCR, etc) will be mapped to Dom0.
 
  Which again seems odd - such tables should be considered MMIO
  (despite living in RAM), and hence not be part of Dom0's memory
  assignment.
 
  not sure if this applies to the context of your comment, but we had
  issues before when trying to deal with this data as device memory
  (MMIO), because ARM doesn't allow unaligned accesses etc. on device
  memory, and so Dom0 would crash at memory abort exceptions during
  boot, so this really does have to be mapped as RAM to Dom0.  If you
  meant some internal Xen bookkeeping thing, then just ignore me.
  
  Hmm, how would native Linux avoid such unaligned accesses then?
  
  ACPI table are living on RAM on ARM. So there is no issue with Linux
  baremetal.
 
 I'm sure our interpretation of the meaning of RAM here differs:
 While from a physical perspective the tables live in RAM too on
 x86, from a memory map perspective they don't. And since iirc
 ACPI implies UEFI, and since UEFI has special memory types
 for such tables (to prevent the OS from using the space as
 normal memory), I can't believe this to be different under ARM.
 
  The problem is from Xen where we are mapping memory region with Device
  attribute. This is because until now we never had a reason to directly
  map other thing than MMIO to the domain.
  
  This could be fixed by adding new helper in Xen to directly map RAM region.
 
 Which would seem to be the correct route.
 
  Nonetheless, we still have to copy some table in Xen in order to modify
  them and/or new one. I have in mind the FADT table to set the hypervisor
  field and hiding the hypervisor specific data (GIC hyp, timer hyp...) to
  avoid the kernel thinks there is hyp mode available.
 
 Have to? Or rather would like to? As said in another reply to
 Shannon, I didn't see any rationale spelled out for this fundamental
 difference to x86.

Just because it was done this way before, it doesn't mean that it is the
right way of doing it.

I think is bad design to expose the presence of certain functionalities
in ACPI tables and then expect that the Dom0 kernel won't use them.
ACPI tables should describe only and all the hardware which is exposed
to Dom0. Anything else is a mistake.

For example it is only natural for the kernel to try to use the GIC hyp
functionalities if they are described, while actually they are not
emulated by Xen at all.

I would rather teach Xen to fix the tables now, than later try teach
every possible Dom0 kernel (Linux, FreeBSD, etc) to ignore a set of info
which are wrong but still passed to them anyway. Moreover the list of
things to ignore can change over time. It is just a bad ABI to maintain.

In terms of code we could share in Linux between Xen x86 and Xen ARM
regarding dealing with ACPI info we want to ignore, unfortunately there
isn't much of it, because we are mostly talking about new ARM specific
tables here (GIC, arch_timer, etc).

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Thu, 2015-08-13 at 11:13 +0100, Stefano Stabellini wrote:
 
   For example it is only natural for the kernel to try to use the GIC 
   hyp
   functionalities if they are described, while actually they are not
   emulated by Xen at all.
  
  See Ian's earlier reply: It can also be considered natural for it to
  be aware that when run in EL2 to not use EL1 functionality.

NB EL2 == Hyp and EL1 == Kernel, so it's the other way round, FWIW.

 It is not just about the GIC Hyp functionalities.

What else is there which is not subject to this logic? Timers are too, it
even applies to IOMMU's which have both stage1 and stage2 bits.

BTW, I think kernels _already_ need to deal with a lot of this because in
reality nobody modifies the DTB when they use a firmware which launches the
kernel in EL1. IOW I think the kernel is already aware of which resources
can be used by which privilege level.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-13 Thread Julien Grall

On 13/08/15 11:54, Jan Beulich wrote:
 On 13.08.15 at 12:48, zhaoshengl...@huawei.com wrote:
 On 2015/8/13 18:29, Christoffer Dall wrote:
 However, what about for other resources?  Having code somewhere that
 says hide this random piece of hardware if you're Xen dom0 sounds
 awful to me.  I know it's only the serial port right now, but still.

 It needs to modify MADT table according to dom0_max_vcpus and modify EFI
 MMAP table according to dom0_mem. And modify FADT table to set
 hypervisor_id as XenVMM to tell Dom0 that it runs on Xen hypervisor.
 
 None of this should be required:
 
 Unaltered MADT may (or should I say will) be needed to drive P- or
 C-states.

We have to alter MADT for different reasons:
- restricting the number of vCPUs
- update the CPU bring up method
- changing the CPUID

Maybe we should pass a backup but we don't support cpufreq driver right
now and it would need quite a look of work to do it (see [1]).

 UEFI memmap shouldn't be exposed to Dom0 at all.

We have to craft a UEFI memmap in order to notify Dom0 where are the RAM
regions.

 Detecting running on Xen should (hopefully) be possible by other
 means for Dom0.

How there is no cpuid like on ARM. So no possibility to know whether you
are running on Xen or another hypervisor...

Modifying the FADT is the only viable solution we have today in order to
tell that it's running on Xen.

Regards,

[1]http://lists.xen.org/archives/html/xen-devel/2014-10/msg02883.html

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-13 Thread Shannon Zhao



On 2015/8/13 18:29, Christoffer Dall wrote:
 On Thu, Aug 13, 2015 at 11:22:19AM +0100, Ian Campbell wrote:
 On Thu, 2015-08-13 at 11:13 +0100, Stefano Stabellini wrote:

 For example it is only natural for the kernel to try to use the GIC 
 hyp
 functionalities if they are described, while actually they are not
 emulated by Xen at all.

 See Ian's earlier reply: It can also be considered natural for it to
 be aware that when run in EL2 to not use EL1 functionality.

 NB EL2 == Hyp and EL1 == Kernel, so it's the other way round, FWIW.

 It is not just about the GIC Hyp functionalities.

 What else is there which is not subject to this logic? Timers are too, it
 even applies to IOMMU's which have both stage1 and stage2 bits.

 BTW, I think kernels _already_ need to deal with a lot of this because in
 reality nobody modifies the DTB when they use a firmware which launches the
 kernel in EL1. IOW I think the kernel is already aware of which resources
 can be used by which privilege level.

 Yes, for resources specific to EL2 I believe that is indeed the case
 (the GIC driver doesn't look at the hypervisor control register address,
 and KVM does not even get that far if you're not booted in EL2, and the
 timer only uses the virtual timer if not booted in EL2 - we never
 attempt to use the hyp timer until Marc's VHE patches land, but they
 also depend on being booted in hyp mode).
 
 However, what about for other resources?  Having code somewhere that
 says hide this random piece of hardware if you're Xen dom0 sounds
 awful to me.  I know it's only the serial port right now, but still.
 
It needs to modify MADT table according to dom0_max_vcpus and modify EFI
MMAP table according to dom0_mem. And modify FADT table to set
hypervisor_id as XenVMM to tell Dom0 that it runs on Xen hypervisor.

--
Shannon



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Thu, 13 Aug 2015, Christoffer Dall wrote:
 On Thu, Aug 13, 2015 at 11:22:19AM +0100, Ian Campbell wrote:
  On Thu, 2015-08-13 at 11:13 +0100, Stefano Stabellini wrote:
   
 For example it is only natural for the kernel to try to use the GIC 
 hyp
 functionalities if they are described, while actually they are not
 emulated by Xen at all.

See Ian's earlier reply: It can also be considered natural for it to
be aware that when run in EL2 to not use EL1 functionality.
  
  NB EL2 == Hyp and EL1 == Kernel, so it's the other way round, FWIW.
  
   It is not just about the GIC Hyp functionalities.
  
  What else is there which is not subject to this logic? Timers are too, it
  even applies to IOMMU's which have both stage1 and stage2 bits.
  
  BTW, I think kernels _already_ need to deal with a lot of this because in
  reality nobody modifies the DTB when they use a firmware which launches the
  kernel in EL1. IOW I think the kernel is already aware of which resources
  can be used by which privilege level.
  
 Yes, for resources specific to EL2 I believe that is indeed the case
 (the GIC driver doesn't look at the hypervisor control register address,
 and KVM does not even get that far if you're not booted in EL2, and the
 timer only uses the virtual timer if not booted in EL2 - we never
 attempt to use the hyp timer until Marc's VHE patches land, but they
 also depend on being booted in hyp mode).

 However, what about for other resources?  Having code somewhere that
 says hide this random piece of hardware if you're Xen dom0 sounds
 awful to me.  I know it's only the serial port right now, but still.

I completely agree. Non-EL2 specific resources should defenitely be
removed. I can see that EL2 stuff could be left there, but I think that
removing it now would avoid us trouble in the future.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Thu, 13 Aug 2015, Julien Grall wrote:
 On 13/08/15 11:54, Jan Beulich wrote:
  On 13.08.15 at 12:48, zhaoshengl...@huawei.com wrote:
  On 2015/8/13 18:29, Christoffer Dall wrote:
  However, what about for other resources?  Having code somewhere that
  says hide this random piece of hardware if you're Xen dom0 sounds
  awful to me.  I know it's only the serial port right now, but still.
 
  It needs to modify MADT table according to dom0_max_vcpus and modify EFI
  MMAP table according to dom0_mem. And modify FADT table to set
  hypervisor_id as XenVMM to tell Dom0 that it runs on Xen hypervisor.
  
  None of this should be required:
  
  Unaltered MADT may (or should I say will) be needed to drive P- or
  C-states.
 
 We have to alter MADT for different reasons:
   - restricting the number of vCPUs
   - update the CPU bring up method
   - changing the CPUID
 
 Maybe we should pass a backup but we don't support cpufreq driver right
 now and it would need quite a look of work to do it (see [1]).
 
  UEFI memmap shouldn't be exposed to Dom0 at all.
 
 We have to craft a UEFI memmap in order to notify Dom0 where are the RAM
 regions.

Right. Keep in mind that there are no legacy interfaces to get these
info on ARM. All we have is ACPI and EFI.


  Detecting running on Xen should (hopefully) be possible by other
  means for Dom0.
 
 How there is no cpuid like on ARM. So no possibility to know whether you
 are running on Xen or another hypervisor...
 
 Modifying the FADT is the only viable solution we have today in order to
 tell that it's running on Xen.

We also have the XENV table.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Thu, 13 Aug 2015, Jan Beulich wrote:
  On 13.08.15 at 11:43, stefano.stabell...@eu.citrix.com wrote:
  On Thu, 13 Aug 2015, Jan Beulich wrote:
   On 12.08.15 at 18:18, julien.gr...@citrix.com wrote:
   On 12/08/15 16:58, Jan Beulich wrote:
   On 12.08.15 at 17:51, christoffer.d...@linaro.org wrote:
   On Wed, Aug 12, 2015 at 5:45 PM, Jan Beulich jbeul...@suse.com wrote:
   On 07.08.15 at 04:11, zhaoshengl...@huawei.com wrote:
   All these tables will be copied to Dom0 memory except that the reused
   tables(DSDT, SPCR, etc) will be mapped to Dom0.
  
   Which again seems odd - such tables should be considered MMIO
   (despite living in RAM), and hence not be part of Dom0's memory
   assignment.
  
   not sure if this applies to the context of your comment, but we had
   issues before when trying to deal with this data as device memory
   (MMIO), because ARM doesn't allow unaligned accesses etc. on device
   memory, and so Dom0 would crash at memory abort exceptions during
   boot, so this really does have to be mapped as RAM to Dom0.  If you
   meant some internal Xen bookkeeping thing, then just ignore me.
   
   Hmm, how would native Linux avoid such unaligned accesses then?
   
   ACPI table are living on RAM on ARM. So there is no issue with Linux
   baremetal.
  
  I'm sure our interpretation of the meaning of RAM here differs:
  While from a physical perspective the tables live in RAM too on
  x86, from a memory map perspective they don't. And since iirc
  ACPI implies UEFI, and since UEFI has special memory types
  for such tables (to prevent the OS from using the space as
  normal memory), I can't believe this to be different under ARM.
  
   The problem is from Xen where we are mapping memory region with Device
   attribute. This is because until now we never had a reason to directly
   map other thing than MMIO to the domain.
   
   This could be fixed by adding new helper in Xen to directly map RAM 
   region.
  
  Which would seem to be the correct route.
  
   Nonetheless, we still have to copy some table in Xen in order to modify
   them and/or new one. I have in mind the FADT table to set the hypervisor
   field and hiding the hypervisor specific data (GIC hyp, timer hyp...) to
   avoid the kernel thinks there is hyp mode available.
  
  Have to? Or rather would like to? As said in another reply to
  Shannon, I didn't see any rationale spelled out for this fundamental
  difference to x86.
  
  Just because it was done this way before, it doesn't mean that it is the
  right way of doing it.
 
 Correct. Which is why I'm not saying outright no but asking for
 justification.
 
  I think is bad design to expose the presence of certain functionalities
  in ACPI tables and then expect that the Dom0 kernel won't use them.
  ACPI tables should describe only and all the hardware which is exposed
  to Dom0. Anything else is a mistake.
 
 That depends on the perspective you take: Especially for a PV
 Dom0 it is quite reasonable for the kernel to be aware of certain
 implications from running under a (or the) hypervisor it knows of.
 I agree that the perspective may shift when it comes to HVM-like
 Dom0 (albeit in the spectrum I'd rather see ARM near PVH, which
 again is supposed to be aware).

PVH has always been the wrong analogy for ARM guests. Guest kernels boot
following native boot paths. They don't require emulation because the
ARM architecture is much more flexible and legacy free compared to x86,
not because we instructed the kernel to ignore certain pieces of
hardware. So they are like PV on HVM without any need for QEMU because
there is nothing to emulate. Maybe the right analogy is HVMLite.


  For example it is only natural for the kernel to try to use the GIC hyp
  functionalities if they are described, while actually they are not
  emulated by Xen at all.
 
 See Ian's earlier reply: It can also be considered natural for it to
 be aware that when run in EL2 to not use EL1 functionality.

It is not just about the GIC Hyp functionalities.


  I would rather teach Xen to fix the tables now, than later try teach
  every possible Dom0 kernel (Linux, FreeBSD, etc) to ignore a set of info
  which are wrong but still passed to them anyway. Moreover the list of
  things to ignore can change over time. It is just a bad ABI to maintain.
 
 I don't think there's a clear advantage to teaching Xen of new
 tables to hide over teaching Dom0 of new tables to ignore - it
 much depends on release cycles and such which one would be
 more beneficial.

There are a couple of very obvious advantages:

- there is one Xen, there are many Dom0 kernels
- if Xen changes ACPI tables, Dom0 won't be surprised by the fact that
actually they don't match the hardware. Keep in mind that we don't have
10 years of PV history on ARM. People still think of this as confusing
and counter-intuitive.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-13 Thread Julien Grall

On 13/08/15 10:20, Jan Beulich wrote:
 BTW, IIRC x86 does modify at least one ACPI table which is the DMAR (I
 think), to hide the IOMMU from the guest? That's another table we would
 want to frob on ARM I think (or it's equivalent, which I think is IORT).
 
 Eliminating that hack is supposed to be on the VT-d maintainers'
 TODO list(s) - Dom0 has no business looking at that table (and its
 AMD counterpart already isn't being fiddled with in the same way).

ARM SMMU is supporting 2 stage in order to protect the device also by
the domain.

At some point we will expose it to DOM0 and therefore may need to modify
IORT.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

 On 13.08.15 at 12:48, zhaoshengl...@huawei.com wrote:
 On 2015/8/13 18:29, Christoffer Dall wrote:
 However, what about for other resources?  Having code somewhere that
 says hide this random piece of hardware if you're Xen dom0 sounds
 awful to me.  I know it's only the serial port right now, but still.
 
 It needs to modify MADT table according to dom0_max_vcpus and modify EFI
 MMAP table according to dom0_mem. And modify FADT table to set
 hypervisor_id as XenVMM to tell Dom0 that it runs on Xen hypervisor.

None of this should be required:

Unaltered MADT may (or should I say will) be needed to drive P- or
C-states.

UEFI memmap shouldn't be exposed to Dom0 at all.

Detecting running on Xen should (hopefully) be possible by other
means for Dom0.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

 On 13.08.15 at 13:00, julien.gr...@citrix.com wrote:
 On 13/08/15 11:54, Jan Beulich wrote:
 On 13.08.15 at 12:48, zhaoshengl...@huawei.com wrote:
 On 2015/8/13 18:29, Christoffer Dall wrote:
 However, what about for other resources?  Having code somewhere that
 says hide this random piece of hardware if you're Xen dom0 sounds
 awful to me.  I know it's only the serial port right now, but still.

 It needs to modify MADT table according to dom0_max_vcpus and modify EFI
 MMAP table according to dom0_mem. And modify FADT table to set
 hypervisor_id as XenVMM to tell Dom0 that it runs on Xen hypervisor.
 
 None of this should be required:
 
 Unaltered MADT may (or should I say will) be needed to drive P- or
 C-states.
 
 We have to alter MADT for different reasons:
   - restricting the number of vCPUs
   - update the CPU bring up method
   - changing the CPUID
 
 Maybe we should pass a backup but we don't support cpufreq driver right
 now and it would need quite a look of work to do it (see [1]).

I.e. you intend to not even leave open the road there.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-13 Thread Julien Grall

On 13/08/15 12:55, Jan Beulich wrote:
 On 13.08.15 at 13:00, julien.gr...@citrix.com wrote:
 On 13/08/15 11:54, Jan Beulich wrote:
 On 13.08.15 at 12:48, zhaoshengl...@huawei.com wrote:
 On 2015/8/13 18:29, Christoffer Dall wrote:
 However, what about for other resources?  Having code somewhere that
 says hide this random piece of hardware if you're Xen dom0 sounds
 awful to me.  I know it's only the serial port right now, but still.

 It needs to modify MADT table according to dom0_max_vcpus and modify EFI
 MMAP table according to dom0_mem. And modify FADT table to set
 hypervisor_id as XenVMM to tell Dom0 that it runs on Xen hypervisor.

 None of this should be required:

 Unaltered MADT may (or should I say will) be needed to drive P- or
 C-states.

 We have to alter MADT for different reasons:
  - restricting the number of vCPUs
  - update the CPU bring up method
  - changing the CPUID

 Maybe we should pass a backup but we don't support cpufreq driver right
 now and it would need quite a look of work to do it (see [1]).
 
 I.e. you intend to not even leave open the road there.

Please beware that currently porting a guest to Xen on ARM is only a
couple of hundreds lines which is mainly detecting Xen and initializing
the Xen code. Other than that everything is the same as booting on
baremetal. I don't take into account the drivers as they usually make
usage of the kernel framework and don't touch boot code.

This is a strong advantage for Xen because developer would not have to
hack the internal OS in order to get running on Xen.

We are trying  to aim the same for ACPI. Any change compare to
baremetal boot will result to more work for the OS developer.

The MADT has to be modified in order to let DOM0 knows about his CPU
topology. If ever need the original MADT for power management, then we
should find a Xen specific way to do it (even if it's passing the
orginal MADT as a backup).

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Thu, 2015-08-13 at 13:08 +0100, Julien Grall wrote:
 
 The MADT has to be modified in order to let DOM0 knows about his CPU
 topology. If ever need the original MADT for power management, then we
 should find a Xen specific way to do it (even if it's passing the
 orginal MADT as a backup).

Right, I was about to say something along these lines, the default set of
tables should be the ones which are expected for the guest's (including
dom0's) actual configuration and own use.

Any additional magic stuff (like host-cpufreq in dom0) already needs to do
Xen specific magic to bring itself up in a world which doesn't match that
and should request whatever real host information it needs itself, not
expect to find it in the proper places.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-13 Thread Shannon Zhao



On 2015/8/13 18:29, Christoffer Dall wrote:
 On Thu, Aug 13, 2015 at 11:22:19AM +0100, Ian Campbell wrote:
  On Thu, 2015-08-13 at 11:13 +0100, Stefano Stabellini wrote:
   
 For example it is only natural for the kernel to try to use the 
 GIC 
 hyp
 functionalities if they are described, while actually they are not
 emulated by Xen at all.

See Ian's earlier reply: It can also be considered natural for it to
be aware that when run in EL2 to not use EL1 functionality.
  
  NB EL2 == Hyp and EL1 == Kernel, so it's the other way round, FWIW.
  
   It is not just about the GIC Hyp functionalities.
  
  What else is there which is not subject to this logic? Timers are too, it
  even applies to IOMMU's which have both stage1 and stage2 bits.
  
  BTW, I think kernels _already_ need to deal with a lot of this because in
  reality nobody modifies the DTB when they use a firmware which launches the
  kernel in EL1. IOW I think the kernel is already aware of which resources
  can be used by which privilege level.
  
 Yes, for resources specific to EL2 I believe that is indeed the case
 (the GIC driver doesn't look at the hypervisor control register address,
 and KVM does not even get that far if you're not booted in EL2, and the
 timer only uses the virtual timer if not booted in EL2 - we never
 attempt to use the hyp timer until Marc's VHE patches land, but they
 also depend on being booted in hyp mode).
 
 However, what about for other resources?  Having code somewhere that
 says hide this random piece of hardware if you're Xen dom0 sounds
 awful to me.  I know it's only the serial port right now, but still.

If we remove the entry of SPCR table from XSDT table, would it work for
Dom0 to ignore the uart? And it doesn't need the STAO table?

-- 
Shannon


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-12 Thread Jan Beulich

 On 07.08.15 at 04:11, zhaoshengl...@huawei.com wrote:
 1. Copy and change some EFI and ACPI tables
 ---

A key thing I'm missing here is reasoning of why this copying approach
is needed in the first place. Remember that on the x86 side we get
away without any such copying-and-changing. Yet the farther you
diverge from x86's model, the more changes you'll need to common
Xen code in e.g. Linux.

 All these tables will be copied to Dom0 memory except that the reused
 tables(DSDT, SPCR, etc) will be mapped to Dom0.

Which again seems odd - such tables should be considered MMIO
(despite living in RAM), and hence not be part of Dom0's memory
assignment.

 3. Dom0 gets grant table and event channel irq information
 ---
 As said above, we assign the hypervisor_id be XenVMM to tell Dom0 that
 it runs on Xen hypervisor.
 
 For grant table, add two new HVM_PARAMs: HVM_PARAM_GNTTAB_START_ADDRESS
 and HVM_PARAM_GNTTAB_SIZE.

Mind explaining why you need this on ARM when x86 gets away without?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-12 Thread Jan Beulich

 On 11.08.15 at 16:12, ian.campb...@citrix.com wrote:
 On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:
 
 [...]
 3. Dom0 gets grant table and event channel irq information
 ---
 As said above, we assign the hypervisor_id be XenVMM to tell Dom0 that
 it runs on Xen hypervisor.
 
 For grant table, add two new HVM_PARAMs: HVM_PARAM_GNTTAB_START_ADDRESS
 and HVM_PARAM_GNTTAB_SIZE.
 
 The reason we expose this range is essentially to allow OS authors to take
 a short cut by telling them about an IPA range which is unused, so it is
 available for remapping the grant table into. On x86 there is a BAR on the
 Xen platform PCI device which serves a similar purpose.

Ah, this answers a question I just raised, but means the items are
mis-named and badly described.

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Wed, 2015-08-12 at 10:47 +0800, Shannon Zhao wrote:
 
 On 2015/8/11 23:52, Boris Ostrovsky wrote:
  On 08/11/2015 11:35 AM, Ian Campbell wrote:
   On Tue, 2015-08-11 at 11:29 -0400, Boris Ostrovsky wrote:
On 08/11/2015 10:21 AM, Ian Campbell wrote:
 On Tue, 2015-08-11 at 15:19 +0100, Ian Campbell wrote:
  On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:
   This document is going to explain the design details of Xen 
   booting
   with
   ACPI on ARM. Maybe parts of it may not be appropriate. Any 
   comments
   are
   welcome.
  Some small subsets of this seem like they might overlap with 
  what
  will be
  required for PVH on x86 (a new x86 guest mode not dissimilar to 
  the
  sole
  ARM guest mode). If so then it would be preferable IMHO if PVH 
  x86
  could
  use the same interfaces.
  
  I've trimmed the quotes to just those bits and CCd some of the 
  PVH
  people
  (Boris and Roger[0]) in case they have any thoughts.
  
  Actually, having done the trimming there is only one such bit:
  
  [...]
   4. Map MMIO regions
   ---
   Register a bus_notifier for platform and amba bus in Linux.
 Previously PCI was included in this scheme, which is why I 
 thought of
 PVH,
 having missed that PCI wasn't mentioned here now.
 
 Is it handled some other way now or is that just an accidental
 omission?
PCI already has a bus notifier which is probably why it's not 
explicitly
called out here.
   Right, but I was more specifically thinking of the bit which followed
   regarding the use of the notifier to map the MMIO, which is the more
   important bit.
  
  This notifier calls PHYSDEVOP_pci_device_add and as far as I can tell
  MMIO is mapped there (for IOMMU, which IIUIC is the main issue here).
 
 Right, at this moment we only add platform and amba bus bus_notifier.
 The PCI device can reuse existing bus_notifier.

Super. Perhaps add a sentence to that effect so I remember that this
already exists next time?

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-12 Thread Shannon Zhao

Hi,

I'm working on re-spinning this patchset while encountering a werid
problem about xzalloc_bytes.

Since I need to copy some ACPI tables, I need to allocate some memory
for it. So there are a few places calling xzalloc_bytes. And it fails at
the fifth one. The log is shown as following:

(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading kernel from boot module @ 0008fa315000
(XEN) Allocating 1:1 mappings totalling 256MB for dom0:
(XEN) BANK[0] 0x009000-0x00a000 (256MB)
(XEN) Grant table range: 0x0087e0-0x0087e5b000
(XEN) Loading zImage from 0008fa315000 to
9008-909e0ec8
(XEN) Hypervisor Trap. HSR=0x9644 EC=0x25 IL=1 Syndrome=0x44
(XEN) CPU0: Unexpected Trap: Hypervisor
(XEN) [ Xen-4.6-unstable  arm64  debug=y  Not tainted ]
(XEN) CPU:0
(XEN) PC: 00238b78 xmem_pool_alloc+0x348/0x4b0
(XEN) LR: 00238960
(XEN) SP: 002bf4e0
(XEN) CPSR:   2249 MODE:64-bit EL2h (Hypervisor, handler)

(XEN) Xen call trace:
(XEN)[00238b78] xmem_pool_alloc+0x348/0x4b0 (PC)
(XEN)[00238960] xmem_pool_alloc+0x130/0x4b0 (LR)
(XEN)[00239100] _xmalloc+0xf4/0x290
(XEN)[002392b0] _xzalloc+0x14/0x38
(XEN)[00245430] construct_dom0+0xbc0/0x14cc
(XEN)[0028c4c4] start_xen+0xbf4/0xcb0
(XEN)[0020060c] paging+0x84/0xbc
(XEN)
(XEN)
(XEN) 
(XEN) Panic on CPU 0:
(XEN) CPU0: Unexpected Trap: Hypervisor
(XEN)
(XEN) 
(XEN)
(XEN) Reboot in five seconds...

-- 
Shannon


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Tue, 2015-08-11 at 17:01 +0100, Julien Grall wrote:
 On 11/08/15 16:19, Ian Campbell wrote:
   IIRC we talked about it few months ago and you said that using 
   balloon
   page will split in 4K the 1G/2M mapping done in the stage-2 p2m.
  
  Did I? Odd because I'm also of the opinion that alloc_ballooned_pages
  should operate in chunks of 2M at the hypercall layer and keep any
  resulting spare 4K pages on a free list to use for future such 
  allocations.
 
 That from what I recall from an IRL talk.
 
 Anyway, I've looked in my archive to see why we decided to keep the
 grant table parameters (in Xen ACPI table at this point). We were not
 sure that the domain as all the key in hand in order to find memory hole.
 
 I think it's quite important to not think only about Linux but all other
 Operating Systems. If we ever require a parameters later, it would mean
 that the OS won't be able to run as DOM0 on older Xen.
 
 Linux is using ballooned page, which means loosing ~128KB

It's not loosing anything. It is _using_ 128KB for a legitimate data
structure needed to support the platform (the fact that it happens to not
really be using the underlying RAM is irrelevant here).

It's not really inherently any different to loosing memory in order to
allocate e.g. page tables.

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Wed, 2015-08-12 at 10:42 +0800, Shannon Zhao wrote:
 
 On 2015/8/12 0:01, Julien Grall wrote:
  On 11/08/15 16:19, Ian Campbell wrote:
IIRC we talked about it few months ago and you said that using 
balloon
page will split in 4K the 1G/2M mapping done in the stage-2 p2m.
   
   Did I? Odd because I'm also of the opinion that alloc_ballooned_pages
   should operate in chunks of 2M at the hypercall layer and keep any
   resulting spare 4K pages on a free list to use for future such 
   allocations.
  
  That from what I recall from an IRL talk.
  
  Anyway, I've looked in my archive to see why we decided to keep the
  grant table parameters (in Xen ACPI table at this point). We were not
  sure that the domain as all the key in hand in order to find memory 
  hole.
  
  I think it's quite important to not think only about Linux but all 
  other
  Operating Systems. If we ever require a parameters later, it would mean
  that the OS won't be able to run as DOM0 on older Xen.
  
  Linux is using ballooned page, which means loosing ~128KB (default of
  the grant table on ARM) of memory because we never give back the page 
  to
  Xen due the 1:1 mapping. Although I guess this is not a big deal as 
  it's
  quite small and Linux, as said by David, will support memory hotplug 
  soon.
  
  FreeBSD is using memory hole in the address space so there is no issue 
  here.
  
  So I guess we could skip this parameters as 128KB doesn't seem to be a
  big deal.
  
   IOW it should avoid such shattering where it can.
  
  That would work too.
  
 
 So use xlated_setup_gnttab_pages for both DT and ACPI?

I think we might as well, yes.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Wed, 2015-08-12 at 11:04 +0800, Shannon Zhao wrote:
 Hi Julien,
 
 On 2015/8/12 0:19, Julien Grall wrote:
  Hi Shannon,
  
  On 07/08/15 03:11, Shannon Zhao wrote:
   2. Create minimal DT to pass required information to Dom0
   --
   The minimal DT mainly passes Dom0 bootargs, address and size of 
   initrd
   (if available), address and size of uefi system table, address and 
   size
   of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.
   
   An example of the minimal DT:
   / {
   #address-cells = 2;
   #size-cells = 1;
   chosen {
   bootargs = kernel=Image console=hvc0 
   earlycon=pl011,0x1c09
   root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force;
   linux,initrd-start = 0x;
   linux,initrd-end = 0x;
   linux,uefi-system-table = 0x;
   linux,uefi-mmap-start = 0x;
   linux,uefi-mmap-size = 0x;
   linux,uefi-mmap-desc-size = 0x;
   linux,uefi-mmap-desc-ver = 0x;
   };
   };
   
   For details loook at
   https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.
   txt
  
  AFAICT, the device tree properties in this documentation are only used
  in order to communicate between the UEFI stub and Linux.
  
  This means that those properties are not standardize and can change at
  any time by Linux folks. They don't even live in 
  Documentation/devicetree/
  
  I would also expect to see the same needs for FreeBSD running as DOM0
  with ACPI.
  
 I'm not very clear about how FreeBSD communicates with UEFI. And when
 booting with DT, how does FreeBSD communicate with UEFI? Not through
 these properties?

These properties are in effect a Linux internal interface defined between
the Linux UEFI stub and the Linux kernel proper. The stub and the
kernel are notionally separate entities, although they are in the same tree
etc there is a well defined transition/entry point between the two. Since
they are in the same tree even though they are in theory separate I
expect they will tend to co-evolve.

IIRC we discussed with some of the maintainers (at Connect?) making this a
more formal interface, i.e. exposing the entry point to Linux kernel
proper which understands these properties to other than just the Linux
UEFI stub specifically to external entities such as Xen.

Probably part of this work needs to formalise that, such as by moving this
binding into the proper external bindings dir.

At which point BSD can (hopefully!) choose to support the same interface.

Ian.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2


On 12/08/2015 08:22, Shannon Zhao wrote:

Hi,


Hi Shannon,

It's not part of the design discussion and we are avoiding to mix 
discussion. Can you please create another thread (or at least renaming 
the subject)?



I'm working on re-spinning this patchset while encountering a werid
problem about xzalloc_bytes.

Since I need to copy some ACPI tables, I need to allocate some memory
for it. So there are a few places calling xzalloc_bytes. And it fails at
the fifth one. The log is shown as following:


Do you copy data in the newly allocated memory between 2 xzalloc_bytes?

The only thing I have in mind based on the log below is your are 
overriding the metadata of the memory allocator.



(XEN) *** LOADING DOMAIN 0 ***
(XEN) Loading kernel from boot module @ 0008fa315000
(XEN) Allocating 1:1 mappings totalling 256MB for dom0:
(XEN) BANK[0] 0x009000-0x00a000 (256MB)
(XEN) Grant table range: 0x0087e0-0x0087e5b000
(XEN) Loading zImage from 0008fa315000 to
9008-909e0ec8
(XEN) Hypervisor Trap. HSR=0x9644 EC=0x25 IL=1 Syndrome=0x44
(XEN) CPU0: Unexpected Trap: Hypervisor
(XEN) [ Xen-4.6-unstable  arm64  debug=y  Not tainted ]
(XEN) CPU:0
(XEN) PC: 00238b78 xmem_pool_alloc+0x348/0x4b0
(XEN) LR: 00238960
(XEN) SP: 002bf4e0
(XEN) CPSR:   2249 MODE:64-bit EL2h (Hypervisor, handler)

(XEN) Xen call trace:
(XEN)[00238b78] xmem_pool_alloc+0x348/0x4b0 (PC)
(XEN)[00238960] xmem_pool_alloc+0x130/0x4b0 (LR)
(XEN)[00239100] _xmalloc+0xf4/0x290
(XEN)[002392b0] _xzalloc+0x14/0x38
(XEN)[00245430] construct_dom0+0xbc0/0x14cc
(XEN)[0028c4c4] start_xen+0xbf4/0xcb0
(XEN)[0020060c] paging+0x84/0xbc
(XEN)
(XEN)
(XEN) 
(XEN) Panic on CPU 0:
(XEN) CPU0: Unexpected Trap: Hypervisor
(XEN)
(XEN) 
(XEN)
(XEN) Reboot in five seconds...



Regards,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-12 Thread Stefano Stabellini

On Wed, 12 Aug 2015, Ian Campbell wrote:
 On Wed, 2015-08-12 at 11:04 +0800, Shannon Zhao wrote:
  Hi Julien,
  
  On 2015/8/12 0:19, Julien Grall wrote:
   Hi Shannon,
   
   On 07/08/15 03:11, Shannon Zhao wrote:
2. Create minimal DT to pass required information to Dom0
--
The minimal DT mainly passes Dom0 bootargs, address and size of 
initrd
(if available), address and size of uefi system table, address and 
size
of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.

An example of the minimal DT:
/ {
#address-cells = 2;
#size-cells = 1;
chosen {
bootargs = kernel=Image console=hvc0 
earlycon=pl011,0x1c09
root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force;
linux,initrd-start = 0x;
linux,initrd-end = 0x;
linux,uefi-system-table = 0x;
linux,uefi-mmap-start = 0x;
linux,uefi-mmap-size = 0x;
linux,uefi-mmap-desc-size = 0x;
linux,uefi-mmap-desc-ver = 0x;
};
};

For details loook at
https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.
txt
   
   AFAICT, the device tree properties in this documentation are only used
   in order to communicate between the UEFI stub and Linux.
   
   This means that those properties are not standardize and can change at
   any time by Linux folks. They don't even live in 
   Documentation/devicetree/
   
   I would also expect to see the same needs for FreeBSD running as DOM0
   with ACPI.
   
  I'm not very clear about how FreeBSD communicates with UEFI. And when
  booting with DT, how does FreeBSD communicate with UEFI? Not through
  these properties?
 
 These properties are in effect a Linux internal interface defined between
 the Linux UEFI stub and the Linux kernel proper. The stub and the
 kernel are notionally separate entities, although they are in the same tree
 etc there is a well defined transition/entry point between the two. Since
 they are in the same tree even though they are in theory separate I
 expect they will tend to co-evolve.
 
 IIRC we discussed with some of the maintainers (at Connect?) making this a
 more formal interface, i.e. exposing the entry point to Linux kernel
 proper which understands these properties to other than just the Linux
 UEFI stub specifically to external entities such as Xen.
 
 Probably part of this work needs to formalise that, such as by moving this
 binding into the proper external bindings dir.
 
 At which point BSD can (hopefully!) choose to support the same interface.

CCing Ard, who originally proposed this.

Yes, I think it would be good to submit a patch to Linux to formalize
the bindings and clear out that the interface has become external.

As a reference, this was the original thread:

http://marc.info/?l=linux-kernelm=142362266626403w=2

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2




On 12/08/2015 09:46, Ian Campbell wrote:

On Tue, 2015-08-11 at 17:01 +0100, Julien Grall wrote:

On 11/08/15 16:19, Ian Campbell wrote:

IIRC we talked about it few months ago and you said that using
balloon
page will split in 4K the 1G/2M mapping done in the stage-2 p2m.


Did I? Odd because I'm also of the opinion that alloc_ballooned_pages
should operate in chunks of 2M at the hypercall layer and keep any
resulting spare 4K pages on a free list to use for future such
allocations.


That from what I recall from an IRL talk.

Anyway, I've looked in my archive to see why we decided to keep the
grant table parameters (in Xen ACPI table at this point). We were not
sure that the domain as all the key in hand in order to find memory hole.

I think it's quite important to not think only about Linux but all other
Operating Systems. If we ever require a parameters later, it would mean
that the OS won't be able to run as DOM0 on older Xen.

Linux is using ballooned page, which means loosing ~128KB


It's not loosing anything. It is _using_ 128KB for a legitimate data
structure needed to support the platform (the fact that it happens to not
really be using the underlying RAM is irrelevant here).

It's not really inherently any different to loosing memory in order to
allocate e.g. page tables.


If you read my sentence until the end, you would have noticed why I say 
loosing. It's because we never give back this memory to Xen due to the 
1:1 mapping...


The grant table are already allocated in Xen separately which means 
doubling the size of memory used for DOM0 grant table.


Anyway, that's small amount of data and can optimized later using the 
hotplug.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Wed, 2015-08-12 at 11:48 +0100, Stefano Stabellini wrote:
 On Wed, 12 Aug 2015, Andrew Turner wrote:
  On Wed, 12 Aug 2015 10:21:55 +0100
  Julien Grall julien.gr...@citrix.com wrote:
  
   Hi,
   
   (Cc Andrew Turner who worked on the ACPI port for FreeBSD ARM64)
   
   On 12/08/2015 09:52, Ian Campbell wrote:
On Wed, 2015-08-12 at 11:04 +0800, Shannon Zhao wrote:
 Hi Julien,
 
 On 2015/8/12 0:19, Julien Grall wrote:
  Hi Shannon,
  
  On 07/08/15 03:11, Shannon Zhao wrote:
   2. Create minimal DT to pass required information to Dom0
   --
   The minimal DT mainly passes Dom0 bootargs, address and size 
   of
   initrd
   (if available), address and size of uefi system table, 
   address
   and size
   of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc
   -ver.
   
   An example of the minimal DT:
   / {
#address-cells = 2;
#size-cells = 1;
chosen {
bootargs = kernel=Image console=hvc0
   earlycon=pl011,0x1c09
   root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force;
linux,initrd-start = 0x;
linux,initrd-end = 0x;
linux,uefi-system-table = 0x;
linux,uefi-mmap-start = 0x;
linux,uefi-mmap-size = 0x;
linux,uefi-mmap-desc-size = 0x;
linux,uefi-mmap-desc-ver = 0x;
};
  
  Would it be possible to add a stdout property and node for the hvc0
  device? It would help FreeBSD as we use this to find the kernel
  console. We check for the stdout-path, linux,stdout-path, stdout,
  stdin-path, and stdin properties, in that order, with the first
  property selected as the console. If none are found we fall back to
  searching for a serial0 device. You can see how we find the device at
  [1].
  
   };
   
   For details loook at
   https://github.com/torvalds/linux/blob/master/Documentation/a
   rm/uefi.
   txt
  
  AFAICT, the device tree properties in this documentation are 
  only
  used in order to communicate between the UEFI stub and Linux.
  
  This means that those properties are not standardize and can
  change at any time by Linux folks. They don't even live in
  Documentation/devicetree/
  
  I would also expect to see the same needs for FreeBSD running 
  as
  DOM0 with ACPI.
  
 I'm not very clear about how FreeBSD communicates with UEFI. And
 when booting with DT, how does FreeBSD communicate with UEFI? Not
 through these properties?
  
  FreeBSD has a tool called loader.efi. It gets loaded by UEFI, and knows
  how to communicate with it. It then loads the kernel and reads any
  important data the kernel may need. Finally it puts this data into a
  format the kernel understands, exits the boot services, and boots the
  kernel. The kernel never communicates with UEFI, we have loaded any
  data we need (however this may change in the future).
  
  In the case of the memory may loader.efi calls GetMemoryMap from
  EFI_BOOT_SERVICES. It then passes this data directly to the kernel for
  the kernel to parse in the early boot code.
  

These properties are in effect a Linux internal interface defined
between the Linux UEFI stub and the Linux kernel proper. The
stub and the kernel are notionally separate entities, although they
are in the same tree etc there is a well defined transition/entry
point between the two. Since they are in the same tree even though
they are in theory separate I expect they will tend to co-evolve.

IIRC we discussed with some of the maintainers (at Connect?) making
this a more formal interface, i.e. exposing the entry point to
Linux kernel proper which understands these properties to other
than just the Linux UEFI stub specifically to external entities
such as Xen.

Probably part of this work needs to formalise that, such as by
moving this binding into the proper external bindings dir.

At which point BSD can (hopefully!) choose to support the same
interface.
  
  What are the advantages of these bindings over the existing UEFI calls
  to get the memory map?
 
 The UEFI calls couldn't be used because ExitBootServices has already
 been called.
 
 The Linux UEFI stub is a bit like your loader.efi, except that is part
 of the kernel.

Strictly it is considered a separate thing, much like loader.efi, despite
where it lives e.g. it is self contained and not allowed to call into the
kernel proper except via the formal interface provided for the hand-off.

That might seem like semantic quibbling, but I just want to clarify that
the Linux and BSD approaches here are basically the same.

Given that these device tree bindings are really just

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-12 Thread Stefano Stabellini

On Wed, 12 Aug 2015, Andrew Turner wrote:
 On Wed, 12 Aug 2015 10:21:55 +0100
 Julien Grall julien.gr...@citrix.com wrote:
 
  Hi,
  
  (Cc Andrew Turner who worked on the ACPI port for FreeBSD ARM64)
  
  On 12/08/2015 09:52, Ian Campbell wrote:
   On Wed, 2015-08-12 at 11:04 +0800, Shannon Zhao wrote:
   Hi Julien,
  
   On 2015/8/12 0:19, Julien Grall wrote:
   Hi Shannon,
  
   On 07/08/15 03:11, Shannon Zhao wrote:
   2. Create minimal DT to pass required information to Dom0
   --
   The minimal DT mainly passes Dom0 bootargs, address and size of
   initrd
   (if available), address and size of uefi system table, address
   and size
   of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.
  
   An example of the minimal DT:
   / {
#address-cells = 2;
#size-cells = 1;
chosen {
bootargs = kernel=Image console=hvc0
   earlycon=pl011,0x1c09
   root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force;
linux,initrd-start = 0x;
linux,initrd-end = 0x;
linux,uefi-system-table = 0x;
linux,uefi-mmap-start = 0x;
linux,uefi-mmap-size = 0x;
linux,uefi-mmap-desc-size = 0x;
linux,uefi-mmap-desc-ver = 0x;
};
 
 Would it be possible to add a stdout property and node for the hvc0
 device? It would help FreeBSD as we use this to find the kernel
 console. We check for the stdout-path, linux,stdout-path, stdout,
 stdin-path, and stdin properties, in that order, with the first
 property selected as the console. If none are found we fall back to
 searching for a serial0 device. You can see how we find the device at
 [1].
 
   };
  
   For details loook at
   https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.
   txt
  
   AFAICT, the device tree properties in this documentation are only
   used in order to communicate between the UEFI stub and Linux.
  
   This means that those properties are not standardize and can
   change at any time by Linux folks. They don't even live in
   Documentation/devicetree/
  
   I would also expect to see the same needs for FreeBSD running as
   DOM0 with ACPI.
  
   I'm not very clear about how FreeBSD communicates with UEFI. And
   when booting with DT, how does FreeBSD communicate with UEFI? Not
   through these properties?
 
 FreeBSD has a tool called loader.efi. It gets loaded by UEFI, and knows
 how to communicate with it. It then loads the kernel and reads any
 important data the kernel may need. Finally it puts this data into a
 format the kernel understands, exits the boot services, and boots the
 kernel. The kernel never communicates with UEFI, we have loaded any
 data we need (however this may change in the future).
 
 In the case of the memory may loader.efi calls GetMemoryMap from
 EFI_BOOT_SERVICES. It then passes this data directly to the kernel for
 the kernel to parse in the early boot code.
 
  
   These properties are in effect a Linux internal interface defined
   between the Linux UEFI stub and the Linux kernel proper. The
   stub and the kernel are notionally separate entities, although they
   are in the same tree etc there is a well defined transition/entry
   point between the two. Since they are in the same tree even though
   they are in theory separate I expect they will tend to co-evolve.
  
   IIRC we discussed with some of the maintainers (at Connect?) making
   this a more formal interface, i.e. exposing the entry point to
   Linux kernel proper which understands these properties to other
   than just the Linux UEFI stub specifically to external entities
   such as Xen.
  
   Probably part of this work needs to formalise that, such as by
   moving this binding into the proper external bindings dir.
  
   At which point BSD can (hopefully!) choose to support the same
   interface.
 
 What are the advantages of these bindings over the existing UEFI calls
 to get the memory map?

The UEFI calls couldn't be used because ExitBootServices has already
been called.

The Linux UEFI stub is a bit like your loader.efi, except that is part
of the kernel. These bindings are used to pass data from the Linux UEFI
stub to the kernel proper, after ExitBootServices is called.

We plan to use the same bindings in Xen to pass data to the Dom0 kernel,
again after ExitBootServices is called (by Xen in this case).

The question is whether FreeBSD could support these bindings too.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-12 Thread Jan Beulich

 On 12.08.15 at 17:51, christoffer.d...@linaro.org wrote:
 On Wed, Aug 12, 2015 at 5:45 PM, Jan Beulich jbeul...@suse.com wrote:
 On 07.08.15 at 04:11, zhaoshengl...@huawei.com wrote:
 All these tables will be copied to Dom0 memory except that the reused
 tables(DSDT, SPCR, etc) will be mapped to Dom0.

 Which again seems odd - such tables should be considered MMIO
 (despite living in RAM), and hence not be part of Dom0's memory
 assignment.

 not sure if this applies to the context of your comment, but we had
 issues before when trying to deal with this data as device memory
 (MMIO), because ARM doesn't allow unaligned accesses etc. on device
 memory, and so Dom0 would crash at memory abort exceptions during
 boot, so this really does have to be mapped as RAM to Dom0.  If you
 meant some internal Xen bookkeeping thing, then just ignore me.

Hmm, how would native Linux avoid such unaligned accesses then?

Jan


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-12 Thread Andrew Turner

On Wed, 12 Aug 2015 10:21:55 +0100
Julien Grall julien.gr...@citrix.com wrote:

Hi,

(Cc Andrew Turner who worked on the ACPI port for FreeBSD ARM64)

On 12/08/2015 09:52, Ian Campbell wrote:
On Wed, 2015-08-12 at 11:04 +0800, Shannon Zhao wrote:
Hi Julien,

On 2015/8/12 0:19, Julien Grall wrote:
Hi Shannon,

On 07/08/15 03:11, Shannon Zhao wrote:
2. Create minimal DT to pass required information to Dom0
--
The minimal DT mainly passes Dom0 bootargs, address and size of
initrd
(if available), address and size of uefi system table, address
and size
of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.

An example of the minimal DT:
/ {
#address-cells = 2;
#size-cells = 1;
chosen {
bootargs = kernel=Image console=hvc0
earlycon=pl011,0x1c09
root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force;
linux,initrd-start = 0x;
linux,initrd-end = 0x;
linux,uefi-system-table = 0x;
linux,uefi-mmap-start = 0x;
linux,uefi-mmap-size = 0x;
linux,uefi-mmap-desc-size = 0x;
linux,uefi-mmap-desc-ver = 0x;
};

Would it be possible to add a stdout property and node for the hvc0
device? It would help FreeBSD as we use this to find the kernel
console. We check for the stdout-path, linux,stdout-path, stdout,
stdin-path, and stdin properties, in that order, with the first
property selected as the console. If none are found we fall back to
searching for a serial0 device. You can see how we find the device at
[1].

};

For details loook at
https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.
txt

AFAICT, the device tree properties in this documentation are only
used in order to communicate between the UEFI stub and Linux.

This means that those properties are not standardize and can
change at any time by Linux folks. They don't even live in
Documentation/devicetree/

I would also expect to see the same needs for FreeBSD running as
DOM0 with ACPI.

I'm not very clear about how FreeBSD communicates with UEFI. And
when booting with DT, how does FreeBSD communicate with UEFI? Not
through these properties?

FreeBSD has a tool called loader.efi. It gets loaded by UEFI, and knows
how to communicate with it. It then loads the kernel and reads any
important data the kernel may need. Finally it puts this data into a
format the kernel understands, exits the boot services, and boots the
kernel. The kernel never communicates with UEFI, we have loaded any
data we need (however this may change in the future).

In the case of the memory may loader.efi calls GetMemoryMap from
EFI_BOOT_SERVICES. It then passes this data directly to the kernel for
the kernel to parse in the early boot code.

These properties are in effect a Linux internal interface defined
between the Linux UEFI stub and the Linux kernel proper. The
stub and the kernel are notionally separate entities, although they
are in the same tree etc there is a well defined transition/entry
point between the two. Since they are in the same tree even though
they are in theory separate I expect they will tend to co-evolve.

IIRC we discussed with some of the maintainers (at Connect?) making
this a more formal interface, i.e. exposing the entry point to
Linux kernel proper which understands these properties to other
than just the Linux UEFI stub specifically to external entities
such as Xen.

Probably part of this work needs to formalise that, such as by
moving this binding into the proper external bindings dir.

At which point BSD can (hopefully!) choose to support the same
interface.

What are the advantages of these bindings over the existing UEFI calls
to get the memory map?

Andrew

[1]
https://svnweb.freebsd.org/base/head/sys/dev/uart/uart_cpu_fdt.c?revision=281438view=markup#l127

--
ABT Systems Ltd
Unit 11, Hove Business Centre, Fonthill Road, Hove, BN3 6HA
Registered in England and Wales, No. 9285513

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On 12/08/15 16:58, Jan Beulich wrote:
 On 12.08.15 at 17:51, christoffer.d...@linaro.org wrote:
 On Wed, Aug 12, 2015 at 5:45 PM, Jan Beulich jbeul...@suse.com wrote:
 On 07.08.15 at 04:11, zhaoshengl...@huawei.com wrote:
 All these tables will be copied to Dom0 memory except that the reused
 tables(DSDT, SPCR, etc) will be mapped to Dom0.

 Which again seems odd - such tables should be considered MMIO
 (despite living in RAM), and hence not be part of Dom0's memory
 assignment.

 not sure if this applies to the context of your comment, but we had
 issues before when trying to deal with this data as device memory
 (MMIO), because ARM doesn't allow unaligned accesses etc. on device
 memory, and so Dom0 would crash at memory abort exceptions during
 boot, so this really does have to be mapped as RAM to Dom0.  If you
 meant some internal Xen bookkeeping thing, then just ignore me.
 
 Hmm, how would native Linux avoid such unaligned accesses then?

ACPI table are living on RAM on ARM. So there is no issue with Linux
baremetal.

The problem is from Xen where we are mapping memory region with Device
attribute. This is because until now we never had a reason to directly
map other thing than MMIO to the domain.

This could be fixed by adding new helper in Xen to directly map RAM region.

Nonetheless, we still have to copy some table in Xen in order to modify
them and/or new one. I have in mind the FADT table to set the hypervisor
field and hiding the hypervisor specific data (GIC hyp, timer hyp...) to
avoid the kernel thinks there is hyp mode available.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On 12/08/15 11:36, Andrew Turner wrote:
 Would it be possible to add a stdout property and node for the hvc0
 device? It would help FreeBSD as we use this to find the kernel
 console. We check for the stdout-path, linux,stdout-path, stdout,
 stdin-path, and stdin properties, in that order, with the first
 property selected as the console. If none are found we fall back to
 searching for a serial0 device. You can see how we find the device at
 [1].

the stdout-path property is used by Xen in order to get the UART. The
property will be removed from the device tree passed to DOM0.

The Xen console is not UART a driver so having a property with hvc
wouldn't help here.

Although, when running on Xen, FreeBSD is creating a Xen console by
default with the higher priority. It will be grabbed by FreeBSD as the
default one.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On 12/08/15 12:23, Ian Campbell wrote:
 Strictly it is considered a separate thing, much like loader.efi, despite
 where it lives e.g. it is self contained and not allowed to call into the
 kernel proper except via the formal interface provided for the hand-off.
 
 That might seem like semantic quibbling, but I just want to clarify that
 the Linux and BSD approaches here are basically the same.
 
 Given that these device tree bindings are really just Linux's equivalent of
 the a format the kernel understands which BSD uses as described above. I
 don't know what format BSD uses, Linux just happened to have a DTB library
 handy...

IIRC, on FreeBSD, the loader and the kernel is talking through a custom
format called metadata. There is no modification of device tree by the
loader.

Although, I would prefer to see a common interface between Xen and DOM0
rather than implementing a custom one for each OS we will support.

I have to think how everything will work together. AFAIK, on x86, the
loader is loading Xen and the FreeBSD kernel in the memory. The metadata
necessary for the kernel is passed as a multiboot entry.

I will speak with Royger to see what we can do here.

In the meantime, I think we should drop linux, when we standardize
them to show that they are generic and not linux specific.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

Hi Julien,

On 2015/8/12 0:19, Julien Grall wrote:
 Hi Shannon,
 
 On 07/08/15 03:11, Shannon Zhao wrote:
 2. Create minimal DT to pass required information to Dom0
 --
 The minimal DT mainly passes Dom0 bootargs, address and size of initrd
 (if available), address and size of uefi system table, address and size
 of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.

 An example of the minimal DT:
 / {
 #address-cells = 2;
 #size-cells = 1;
 chosen {
 bootargs = kernel=Image console=hvc0 earlycon=pl011,0x1c09
 root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force;
 linux,initrd-start = 0x;
 linux,initrd-end = 0x;
 linux,uefi-system-table = 0x;
 linux,uefi-mmap-start = 0x;
 linux,uefi-mmap-size = 0x;
 linux,uefi-mmap-desc-size = 0x;
 linux,uefi-mmap-desc-ver = 0x;
 };
 };

 For details loook at
 https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.txt
 
 AFAICT, the device tree properties in this documentation are only used
 in order to communicate between the UEFI stub and Linux.
 
 This means that those properties are not standardize and can change at
 any time by Linux folks. They don't even live in Documentation/devicetree/
 
 I would also expect to see the same needs for FreeBSD running as DOM0
 with ACPI.

I'm not very clear about how FreeBSD communicates with UEFI. And when
booting with DT, how does FreeBSD communicate with UEFI? Not through
these properties?

 So it looks like to me that a generic name would be better for all those
 properties.
 
If we change these name, it needs change some functions in Linux. Will
it impact the use of Linux with UEFI not on Xen?

Thanks,
-- 
Shannon


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On 11/08/15 03:09, Shannon Zhao wrote:
 Hi Julien,

Hi Shannon,

 On 2015/8/7 18:33, Julien Grall wrote:
 Hi Shannon,

 Just some clarification questions.

 On 07/08/15 03:11, Shannon Zhao wrote:
 3. Dom0 gets grant table and event channel irq information
 ---
 As said above, we assign the hypervisor_id be XenVMM to tell Dom0 that
 it runs on Xen hypervisor.

 For grant table, add two new HVM_PARAMs: HVM_PARAM_GNTTAB_START_ADDRESS
 and HVM_PARAM_GNTTAB_SIZE.

 For event channel irq, reuse HVM_PARAM_CALLBACK_IRQ and add a new
 delivery type:
 val[63:56] == 3: val[15:8] is flag: val[7:0] is a PPI (ARM and ARM64
 only)

 Can you describe the content of flag?

 
 This needs definition as well. I think it could use the definition of
 xenv table. Bit 0 stands interrupt mode and bit 1 stands interrupt
 polarity. And explain it in the comment of HVM_PARAM_CALLBACK_IRQ.

That would be fine for me.

 When constructing Dom0 in Xen, save these values. Then Dom0 could get
 them through hypercall HVMOP_get_param.

 4. Map MMIO regions
 ---
 Register a bus_notifier for platform and amba bus in Linux. Add a new
 XENMAPSPACE XENMAPSPACE_dev_mmio. Within the register, check if the
 device is newly added, then call hypercall XENMEM_add_to_physmap to map
 the mmio regions.

 5. Route device interrupts to Dom0
 --
 Route all the SPI interrupts to Dom0 before Dom0 booting.

 Not all the SPI will be routed to DOM0. Some are used by Xen and should
 never be used by any guest. I have in mind the UART and SMMU interrupts.

 You will have to find away to skip them nicely. Note that not all the
 IRQs used by Xen are properly registered when we build DOM0 (see the SMMU).

 To uart, we can get the interrupt information from SPCR table and hide
 it from Dom0.

Can you clarify your meaning of hide from DOM0? Did you mean avoid to
route the SPI to DOM0?

 IIUC, currently Xen (as well as Linux) doesn't support use SMMU when
 booting with ACPI. When it supports, it could read the interrupts
 information from IORT table and Hide them from Dom0.

Well for Xen we don't even have ACPI supported upstream ;). For Linux
there is some on-going work. Anyway, this is not important right now.

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2



On 2015/8/11 17:46, Julien Grall wrote:
 On 11/08/15 03:09, Shannon Zhao wrote:
 Hi Julien,
 
 Hi Shannon,
 
 On 2015/8/7 18:33, Julien Grall wrote:
 Hi Shannon,

 Just some clarification questions.

 On 07/08/15 03:11, Shannon Zhao wrote:
 3. Dom0 gets grant table and event channel irq information
 ---
 As said above, we assign the hypervisor_id be XenVMM to tell Dom0 that
 it runs on Xen hypervisor.

 For grant table, add two new HVM_PARAMs: HVM_PARAM_GNTTAB_START_ADDRESS
 and HVM_PARAM_GNTTAB_SIZE.

 For event channel irq, reuse HVM_PARAM_CALLBACK_IRQ and add a new
 delivery type:
 val[63:56] == 3: val[15:8] is flag: val[7:0] is a PPI (ARM and ARM64
 only)

 Can you describe the content of flag?


 This needs definition as well. I think it could use the definition of
 xenv table. Bit 0 stands interrupt mode and bit 1 stands interrupt
 polarity. And explain it in the comment of HVM_PARAM_CALLBACK_IRQ.
 
 That would be fine for me.
 
 When constructing Dom0 in Xen, save these values. Then Dom0 could get
 them through hypercall HVMOP_get_param.

 4. Map MMIO regions
 ---
 Register a bus_notifier for platform and amba bus in Linux. Add a new
 XENMAPSPACE XENMAPSPACE_dev_mmio. Within the register, check if the
 device is newly added, then call hypercall XENMEM_add_to_physmap to map
 the mmio regions.

 5. Route device interrupts to Dom0
 --
 Route all the SPI interrupts to Dom0 before Dom0 booting.

 Not all the SPI will be routed to DOM0. Some are used by Xen and should
 never be used by any guest. I have in mind the UART and SMMU interrupts.

 You will have to find away to skip them nicely. Note that not all the
 IRQs used by Xen are properly registered when we build DOM0 (see the SMMU).

 To uart, we can get the interrupt information from SPCR table and hide
 it from Dom0.
 
 Can you clarify your meaning of hide from DOM0? Did you mean avoid to
 route the SPI to DOM0?
 
Yes.

 IIUC, currently Xen (as well as Linux) doesn't support use SMMU when
 booting with ACPI. When it supports, it could read the interrupts
 information from IORT table and Hide them from Dom0.
 
 Well for Xen we don't even have ACPI supported upstream ;). For Linux
 there is some on-going work. Anyway, this is not important right now.
 

Yeah, that could be done after this patchset upstream.

Thanks,
-- 
Shannon


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:
 This document is going to explain the design details of Xen booting with
 ACPI on ARM. Maybe parts of it may not be appropriate. Any comments are
 welcome.

Some small subsets of this seem like they might overlap with what will be
required for PVH on x86 (a new x86 guest mode not dissimilar to the sole
ARM guest mode). If so then it would be preferable IMHO if PVH x86 could
use the same interfaces.

I've trimmed the quotes to just those bits and CCd some of the PVH people
(Boris and Roger[0]) in case they have any thoughts.

Actually, having done the trimming there is only one such bit:

[...]
 4. Map MMIO regions
 ---
 Register a bus_notifier for platform and amba bus in Linux. Add a new
 XENMAPSPACE XENMAPSPACE_dev_mmio. Within the register, check if the
 device is newly added, then call hypercall XENMEM_add_to_physmap to map
 the mmio regions.

Ian.

[0] Roger is away for a week or so, but I'm expect feedback to be of the
we could use one extra field type rather than this needs to be done some
totally different way for x86/PVH (in which case we wouldn't want to share
the interface anyway I suppose) so need to block on awaiting that feedback.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-11 Thread David Vrabel

On 11/08/15 15:12, Ian Campbell wrote:
 On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:

 [...]
 3. Dom0 gets grant table and event channel irq information
 ---
 As said above, we assign the hypervisor_id be XenVMM to tell Dom0 that
 it runs on Xen hypervisor.

 For grant table, add two new HVM_PARAMs: HVM_PARAM_GNTTAB_START_ADDRESS
 and HVM_PARAM_GNTTAB_SIZE.
 
 The reason we expose this range is essentially to allow OS authors to take
 a short cut by telling them about an IPA range which is unused, so it is
 available for remapping the grant table into. On x86 there is a BAR on the
 Xen platform PCI device which serves a similar purpose.
 
 IIRC somebody (perhaps David V, CCd) had proposed at some point to make it
 so that Linux was able to pick such an IPA itself by examining the memory
 map or by some other scheme.

PVH in Linux uses ballooned pages which are vmap()'d into a virtually
contiguous region.

See xlated_setup_gnttab_pages().

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Tue, 2015-08-11 at 15:51 +0100, David Vrabel wrote:
 On 11/08/15 15:12, Ian Campbell wrote:
  On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:
   
  [...]
   3. Dom0 gets grant table and event channel irq information
   ---
   As said above, we assign the hypervisor_id be XenVMM to tell Dom0 
   that
   it runs on Xen hypervisor.
   
   For grant table, add two new HVM_PARAMs: 
   HVM_PARAM_GNTTAB_START_ADDRESS
   and HVM_PARAM_GNTTAB_SIZE.
  
  The reason we expose this range is essentially to allow OS authors to 
  take
  a short cut by telling them about an IPA range which is unused, so it 
  is
  available for remapping the grant table into. On x86 there is a BAR on 
  the
  Xen platform PCI device which serves a similar purpose.
  
  IIRC somebody (perhaps David V, CCd) had proposed at some point to make 
  it
  so that Linux was able to pick such an IPA itself by examining the 
  memory
  map or by some other scheme.
 
 PVH in Linux uses ballooned pages which are vmap()'d into a virtually
 contiguous region.
 
 See xlated_setup_gnttab_pages().

So somewhat more concrete than a proposal then ;-)

I don't see anything there which would be a problem on ARM, so we should
probably go that route there too (at least for ACPI, if not globally for
all ARM guests).

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-11 Thread David Vrabel

On 11/08/15 15:59, Ian Campbell wrote:
 On Tue, 2015-08-11 at 15:51 +0100, David Vrabel wrote:
 On 11/08/15 15:12, Ian Campbell wrote:
 On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:

 [...]
 3. Dom0 gets grant table and event channel irq information
 ---
 As said above, we assign the hypervisor_id be XenVMM to tell Dom0 
 that
 it runs on Xen hypervisor.

 For grant table, add two new HVM_PARAMs: 
 HVM_PARAM_GNTTAB_START_ADDRESS
 and HVM_PARAM_GNTTAB_SIZE.

 The reason we expose this range is essentially to allow OS authors to 
 take
 a short cut by telling them about an IPA range which is unused, so it 
 is
 available for remapping the grant table into. On x86 there is a BAR on 
 the
 Xen platform PCI device which serves a similar purpose.

 IIRC somebody (perhaps David V, CCd) had proposed at some point to make 
 it
 so that Linux was able to pick such an IPA itself by examining the 
 memory
 map or by some other scheme.

 PVH in Linux uses ballooned pages which are vmap()'d into a virtually
 contiguous region.

 See xlated_setup_gnttab_pages().
 
 So somewhat more concrete than a proposal then ;-)
 
 I don't see anything there which would be a problem on ARM, so we should
 probably go that route there too (at least for ACPI, if not globally for
 all ARM guests).

If someone does this please move xlated_setup_gnttab_pages() into
drivers/xen/xlate_mmu.c, and not copy it into an arm specific file.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:
 
[...]
 3. Dom0 gets grant table and event channel irq information
 ---
 As said above, we assign the hypervisor_id be XenVMM to tell Dom0 that
 it runs on Xen hypervisor.
 
 For grant table, add two new HVM_PARAMs: HVM_PARAM_GNTTAB_START_ADDRESS
 and HVM_PARAM_GNTTAB_SIZE.

The reason we expose this range is essentially to allow OS authors to take
a short cut by telling them about an IPA range which is unused, so it is
available for remapping the grant table into. On x86 there is a BAR on the
Xen platform PCI device which serves a similar purpose.

IIRC somebody (perhaps David V, CCd) had proposed at some point to make it
so that Linux was able to pick such an IPA itself by examining the memory
map or by some other scheme.

Any _if_ there was a viable proposal along those lines then we could use it
and avoid the need for these HVM params perhaps?

Ian.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Tue, 2015-08-11 at 15:19 +0100, Ian Campbell wrote:
 On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:
  This document is going to explain the design details of Xen booting 
  with
  ACPI on ARM. Maybe parts of it may not be appropriate. Any comments are
  welcome.
 
 Some small subsets of this seem like they might overlap with what will be
 required for PVH on x86 (a new x86 guest mode not dissimilar to the sole
 ARM guest mode). If so then it would be preferable IMHO if PVH x86 could
 use the same interfaces.
 
 I've trimmed the quotes to just those bits and CCd some of the PVH people
 (Boris and Roger[0]) in case they have any thoughts.
 
 Actually, having done the trimming there is only one such bit:
 
 [...]
  4. Map MMIO regions
  ---
  Register a bus_notifier for platform and amba bus in Linux.

Previously PCI was included in this scheme, which is why I thought of PVH,
having missed that PCI wasn't mentioned here now.

Is it handled some other way now or is that just an accidental omission?

   Add a new
  XENMAPSPACE XENMAPSPACE_dev_mmio. Within the register, check if the
  device is newly added, then call hypercall XENMEM_add_to_physmap to map
  the mmio regions.
 
 Ian.
 
 [0] Roger is away for a week or so, but I'm expect feedback to be of the
 we could use one extra field type rather than this needs to be done 
 some
 totally different way for x86/PVH (in which case we wouldn't want to 
 share
 the interface anyway I suppose) so need to block on awaiting that 
 feedback.

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On 11/08/15 15:59, Ian Campbell wrote:
 On Tue, 2015-08-11 at 15:51 +0100, David Vrabel wrote:
 On 11/08/15 15:12, Ian Campbell wrote:
 On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:

 [...]
 3. Dom0 gets grant table and event channel irq information
 ---
 As said above, we assign the hypervisor_id be XenVMM to tell Dom0 
 that
 it runs on Xen hypervisor.

 For grant table, add two new HVM_PARAMs: 
 HVM_PARAM_GNTTAB_START_ADDRESS
 and HVM_PARAM_GNTTAB_SIZE.

 The reason we expose this range is essentially to allow OS authors to 
 take
 a short cut by telling them about an IPA range which is unused, so it 
 is
 available for remapping the grant table into. On x86 there is a BAR on 
 the
 Xen platform PCI device which serves a similar purpose.

 IIRC somebody (perhaps David V, CCd) had proposed at some point to make 
 it
 so that Linux was able to pick such an IPA itself by examining the 
 memory
 map or by some other scheme.

 PVH in Linux uses ballooned pages which are vmap()'d into a virtually
 contiguous region.

 See xlated_setup_gnttab_pages().
 
 So somewhat more concrete than a proposal then ;-)
 
 I don't see anything there which would be a problem on ARM, so we should
 probably go that route there too (at least for ACPI, if not globally for
 all ARM guests).

IIRC we talked about it few months ago and you said that using balloon
page will split in 4K the 1G/2M mapping done in the stage-2 p2m.

Note that for DOM0 we never give back the ballooned page to Xen. This is
because we have to respect the 1:1 mapping when DOM0 want to get back
the page.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Tue, 2015-08-11 at 16:11 +0100, Julien Grall wrote:
 On 11/08/15 15:59, Ian Campbell wrote:
  On Tue, 2015-08-11 at 15:51 +0100, David Vrabel wrote:
   On 11/08/15 15:12, Ian Campbell wrote:
On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:
 
[...]
 3. Dom0 gets grant table and event channel irq information
 ---
 As said above, we assign the hypervisor_id be XenVMM to tell 
 Dom0 
 that
 it runs on Xen hypervisor.
 
 For grant table, add two new HVM_PARAMs: 
 HVM_PARAM_GNTTAB_START_ADDRESS
 and HVM_PARAM_GNTTAB_SIZE.

The reason we expose this range is essentially to allow OS authors 
to 
take
a short cut by telling them about an IPA range which is unused, so 
it 
is
available for remapping the grant table into. On x86 there is a BAR 
on 
the
Xen platform PCI device which serves a similar purpose.

IIRC somebody (perhaps David V, CCd) had proposed at some point to 
make 
it
so that Linux was able to pick such an IPA itself by examining the 
memory
map or by some other scheme.
   
   PVH in Linux uses ballooned pages which are vmap()'d into a virtually
   contiguous region.
   
   See xlated_setup_gnttab_pages().
  
  So somewhat more concrete than a proposal then ;-)
  
  I don't see anything there which would be a problem on ARM, so we 
  should
  probably go that route there too (at least for ACPI, if not globally 
  for
  all ARM guests).
 
 IIRC we talked about it few months ago and you said that using balloon
 page will split in 4K the 1G/2M mapping done in the stage-2 p2m.

Did I? Odd because I'm also of the opinion that alloc_ballooned_pages
should operate in chunks of 2M at the hypercall layer and keep any
resulting spare 4K pages on a free list to use for future such allocations.

IOW it should avoid such shattering where it can.

 Note that for DOM0 we never give back the ballooned page to Xen. This is
 because we have to respect the 1:1 mapping when DOM0 want to get back
 the page.
 
 Regards,
 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-11 Thread David Vrabel

On 11/08/15 16:19, Ian Campbell wrote:
 On Tue, 2015-08-11 at 16:11 +0100, Julien Grall wrote:
 On 11/08/15 15:59, Ian Campbell wrote:
 On Tue, 2015-08-11 at 15:51 +0100, David Vrabel wrote:
 On 11/08/15 15:12, Ian Campbell wrote:
 On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:

 [...]
 3. Dom0 gets grant table and event channel irq information
 ---
 As said above, we assign the hypervisor_id be XenVMM to tell 
 Dom0 
 that
 it runs on Xen hypervisor.

 For grant table, add two new HVM_PARAMs: 
 HVM_PARAM_GNTTAB_START_ADDRESS
 and HVM_PARAM_GNTTAB_SIZE.

 The reason we expose this range is essentially to allow OS authors 
 to 
 take
 a short cut by telling them about an IPA range which is unused, so 
 it 
 is
 available for remapping the grant table into. On x86 there is a BAR 
 on 
 the
 Xen platform PCI device which serves a similar purpose.

 IIRC somebody (perhaps David V, CCd) had proposed at some point to 
 make 
 it
 so that Linux was able to pick such an IPA itself by examining the 
 memory
 map or by some other scheme.

 PVH in Linux uses ballooned pages which are vmap()'d into a virtually
 contiguous region.

 See xlated_setup_gnttab_pages().

 So somewhat more concrete than a proposal then ;-)

 I don't see anything there which would be a problem on ARM, so we 
 should
 probably go that route there too (at least for ACPI, if not globally 
 for
 all ARM guests).

 IIRC we talked about it few months ago and you said that using balloon
 page will split in 4K the 1G/2M mapping done in the stage-2 p2m.
 
 Did I? Odd because I'm also of the opinion that alloc_ballooned_pages
 should operate in chunks of 2M at the hypercall layer and keep any
 resulting spare 4K pages on a free list to use for future such allocations.
 
 IOW it should avoid such shattering where it can.

You can also (soon) enable memory hotplug and get to hotplug new (empty)
memory sections to avoid having to release any frames back to Xen.

David

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-11 Thread Boris Ostrovsky


On 08/11/2015 10:21 AM, Ian Campbell wrote:

On Tue, 2015-08-11 at 15:19 +0100, Ian Campbell wrote:

On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:

This document is going to explain the design details of Xen booting
with
ACPI on ARM. Maybe parts of it may not be appropriate. Any comments are
welcome.

Some small subsets of this seem like they might overlap with what will be
required for PVH on x86 (a new x86 guest mode not dissimilar to the sole
ARM guest mode). If so then it would be preferable IMHO if PVH x86 could
use the same interfaces.

I've trimmed the quotes to just those bits and CCd some of the PVH people
(Boris and Roger[0]) in case they have any thoughts.

Actually, having done the trimming there is only one such bit:

[...]

4. Map MMIO regions
---
Register a bus_notifier for platform and amba bus in Linux.

Previously PCI was included in this scheme, which is why I thought of PVH,
having missed that PCI wasn't mentioned here now.

Is it handled some other way now or is that just an accidental omission?


PCI already has a bus notifier which is probably why it's not explicitly 
called out here.


-borsi




  Add a new
XENMAPSPACE XENMAPSPACE_dev_mmio. Within the register, check if the
device is newly added, then call hypercall XENMEM_add_to_physmap to map
the mmio regions.

Ian.

[0] Roger is away for a week or so, but I'm expect feedback to be of the
we could use one extra field type rather than this needs to be done
some
totally different way for x86/PVH (in which case we wouldn't want to
share
the interface anyway I suppose) so need to block on awaiting that
feedback.



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

2015-08-11 Thread Boris Ostrovsky


On 08/11/2015 11:35 AM, Ian Campbell wrote:

On Tue, 2015-08-11 at 11:29 -0400, Boris Ostrovsky wrote:

On 08/11/2015 10:21 AM, Ian Campbell wrote:

On Tue, 2015-08-11 at 15:19 +0100, Ian Campbell wrote:

On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:

This document is going to explain the design details of Xen booting
with
ACPI on ARM. Maybe parts of it may not be appropriate. Any comments
are
welcome.

Some small subsets of this seem like they might overlap with what
will be
required for PVH on x86 (a new x86 guest mode not dissimilar to the
sole
ARM guest mode). If so then it would be preferable IMHO if PVH x86
could
use the same interfaces.

I've trimmed the quotes to just those bits and CCd some of the PVH
people
(Boris and Roger[0]) in case they have any thoughts.

Actually, having done the trimming there is only one such bit:

[...]

4. Map MMIO regions
---
Register a bus_notifier for platform and amba bus in Linux.

Previously PCI was included in this scheme, which is why I thought of
PVH,
having missed that PCI wasn't mentioned here now.

Is it handled some other way now or is that just an accidental
omission?

PCI already has a bus notifier which is probably why it's not explicitly
called out here.

Right, but I was more specifically thinking of the bit which followed
regarding the use of the notifier to map the MMIO, which is the more
important bit.


This notifier calls PHYSDEVOP_pci_device_add and as far as I can tell 
MMIO is mapped there (for IOMMU, which IIUIC is the main issue here).


-boris




   Add a new
XENMAPSPACE XENMAPSPACE_dev_mmio. Within the register, check if
the
device is newly added, then call hypercall XENMEM_add_to_physmap to
map
the mmio regions.

Ian.

[0] Roger is away for a week or so, but I'm expect feedback to be of
the
we could use one extra field type rather than this needs to be
done
some
totally different way for x86/PVH (in which case we wouldn't want to
share
the interface anyway I suppose) so need to block on awaiting that
feedback.


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel



___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On 11/08/15 16:19, Ian Campbell wrote:
 IIRC we talked about it few months ago and you said that using balloon
 page will split in 4K the 1G/2M mapping done in the stage-2 p2m.
 
 Did I? Odd because I'm also of the opinion that alloc_ballooned_pages
 should operate in chunks of 2M at the hypercall layer and keep any
 resulting spare 4K pages on a free list to use for future such allocations.

That from what I recall from an IRL talk.

Anyway, I've looked in my archive to see why we decided to keep the
grant table parameters (in Xen ACPI table at this point). We were not
sure that the domain as all the key in hand in order to find memory hole.

I think it's quite important to not think only about Linux but all other
Operating Systems. If we ever require a parameters later, it would mean
that the OS won't be able to run as DOM0 on older Xen.

Linux is using ballooned page, which means loosing ~128KB (default of
the grant table on ARM) of memory because we never give back the page to
Xen due the 1:1 mapping. Although I guess this is not a big deal as it's
quite small and Linux, as said by David, will support memory hotplug soon.

FreeBSD is using memory hole in the address space so there is no issue here.

So I guess we could skip this parameters as 128KB doesn't seem to be a
big deal.

 IOW it should avoid such shattering where it can.

That would work too.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

Hi Shannon,

On 07/08/15 03:11, Shannon Zhao wrote:
 2. Create minimal DT to pass required information to Dom0
 --
 The minimal DT mainly passes Dom0 bootargs, address and size of initrd
 (if available), address and size of uefi system table, address and size
 of uefi memory table, uefi-mmap-desc-size and uefi-mmap-desc-ver.
 
 An example of the minimal DT:
 / {
 #address-cells = 2;
 #size-cells = 1;
 chosen {
 bootargs = kernel=Image console=hvc0 earlycon=pl011,0x1c09
 root=/dev/vda2 rw rootfstype=ext4 init=/bin/sh acpi=force;
 linux,initrd-start = 0x;
 linux,initrd-end = 0x;
 linux,uefi-system-table = 0x;
 linux,uefi-mmap-start = 0x;
 linux,uefi-mmap-size = 0x;
 linux,uefi-mmap-desc-size = 0x;
 linux,uefi-mmap-desc-ver = 0x;
 };
 };
 
 For details loook at
 https://github.com/torvalds/linux/blob/master/Documentation/arm/uefi.txt

AFAICT, the device tree properties in this documentation are only used
in order to communicate between the UEFI stub and Linux.

This means that those properties are not standardize and can change at
any time by Linux folks. They don't even live in Documentation/devicetree/

I would also expect to see the same needs for FreeBSD running as DOM0
with ACPI.

So it looks like to me that a generic name would be better for all those
properties.

Regards,

-- 
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2

On Tue, 2015-08-11 at 11:29 -0400, Boris Ostrovsky wrote:
 On 08/11/2015 10:21 AM, Ian Campbell wrote:
  On Tue, 2015-08-11 at 15:19 +0100, Ian Campbell wrote:
   On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:
This document is going to explain the design details of Xen booting
with
ACPI on ARM. Maybe parts of it may not be appropriate. Any comments 
are
welcome.
   Some small subsets of this seem like they might overlap with what 
   will be
   required for PVH on x86 (a new x86 guest mode not dissimilar to the 
   sole
   ARM guest mode). If so then it would be preferable IMHO if PVH x86 
   could
   use the same interfaces.
   
   I've trimmed the quotes to just those bits and CCd some of the PVH 
   people
   (Boris and Roger[0]) in case they have any thoughts.
   
   Actually, having done the trimming there is only one such bit:
   
   [...]
4. Map MMIO regions
---
Register a bus_notifier for platform and amba bus in Linux.
  Previously PCI was included in this scheme, which is why I thought of 
  PVH,
  having missed that PCI wasn't mentioned here now.
  
  Is it handled some other way now or is that just an accidental 
  omission?
 
 PCI already has a bus notifier which is probably why it's not explicitly 
 called out here.

Right, but I was more specifically thinking of the bit which followed
regarding the use of the notifier to map the MMIO, which is the more
important bit.

 
  Add a new
XENMAPSPACE XENMAPSPACE_dev_mmio. Within the register, check if 
the
device is newly added, then call hypercall XENMEM_add_to_physmap to 
map
the mmio regions.
   Ian.
   
   [0] Roger is away for a week or so, but I'm expect feedback to be of 
   the
   we could use one extra field type rather than this needs to be 
   done
   some
   totally different way for x86/PVH (in which case we wouldn't want to
   share
   the interface anyway I suppose) so need to block on awaiting that
   feedback.
 
 
 ___
 Xen-devel mailing list
 Xen-devel@lists.xen.org
 http://lists.xen.org/xen-devel

___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2



On 2015/8/11 23:52, Boris Ostrovsky wrote:
 On 08/11/2015 11:35 AM, Ian Campbell wrote:
 On Tue, 2015-08-11 at 11:29 -0400, Boris Ostrovsky wrote:
 On 08/11/2015 10:21 AM, Ian Campbell wrote:
 On Tue, 2015-08-11 at 15:19 +0100, Ian Campbell wrote:
 On Fri, 2015-08-07 at 10:11 +0800, Shannon Zhao wrote:
 This document is going to explain the design details of Xen booting
 with
 ACPI on ARM. Maybe parts of it may not be appropriate. Any comments
 are
 welcome.
 Some small subsets of this seem like they might overlap with what
 will be
 required for PVH on x86 (a new x86 guest mode not dissimilar to the
 sole
 ARM guest mode). If so then it would be preferable IMHO if PVH x86
 could
 use the same interfaces.

 I've trimmed the quotes to just those bits and CCd some of the PVH
 people
 (Boris and Roger[0]) in case they have any thoughts.

 Actually, having done the trimming there is only one such bit:

 [...]
 4. Map MMIO regions
 ---
 Register a bus_notifier for platform and amba bus in Linux.
 Previously PCI was included in this scheme, which is why I thought of
 PVH,
 having missed that PCI wasn't mentioned here now.

 Is it handled some other way now or is that just an accidental
 omission?
 PCI already has a bus notifier which is probably why it's not explicitly
 called out here.
 Right, but I was more specifically thinking of the bit which followed
 regarding the use of the notifier to map the MMIO, which is the more
 important bit.
 
 This notifier calls PHYSDEVOP_pci_device_add and as far as I can tell
 MMIO is mapped there (for IOMMU, which IIUIC is the main issue here).

Right, at this moment we only add platform and amba bus bus_notifier.
The PCI device can reuse existing bus_notifier.

-- 
Shannon


___
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Re: [Xen-devel] Design doc of adding ACPI support for arm64 on Xen - version 2