On Thu, 21 Jul 2016, Julien Grall wrote: > On 21/07/2016 19:54, Stefano Stabellini wrote: > > On Thu, 21 Jul 2016, Julien Grall wrote: > > > Hi Stefano, > > > > > > On 21/07/16 18:53, Stefano Stabellini wrote: > > > > On Wed, 20 Jul 2016, Boris Ostrovsky wrote: > > > > > On 07/20/2016 01:28 PM, Stefano Stabellini wrote: > > > > > > On Wed, 20 Jul 2016, Boris Ostrovsky wrote: > > > > > > > On 07/20/2016 09:41 AM, Julien Grall wrote: > > > > > > > > > > > > > > > > On 20/07/2016 14:33, Boris Ostrovsky wrote: > > > > > > > > > On 07/20/2016 08:33 AM, Julien Grall wrote: > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > On 14/07/16 14:37, Stefano Stabellini wrote: > > > > > > > > > > > On Wed, 13 Jul 2016, Julien Grall wrote: > > > > > > > > > > > > Hello, > > > > > > > > > > > > > > > > > > > > > > > > On 12/07/2016 17:58, Boris Ostrovsky wrote: > > > > > > > > > > > > > On 07/12/2016 12:10 PM, Julien Grall wrote: > > > > > > > > > > > > > > On 12/07/2016 16:08, Boris Ostrovsky wrote: > > > > > > > > > > > > > > > On 07/12/2016 10:57 AM, Shannon Zhao wrote: > > > > > > > > > > > > > > It will affect some others part of the guest if we > > > > > > > > > > > > > > don't > > > > > > > > > > > > > > increment > > > > > > > > > > > > > > the > > > > > > > > > > > > > > "maxmem" requested by the user. For ARM the ACPI > > > > > > > > > > > > > > blob > > > > > > > > > > > > > > will be > > > > > > > > > > > > > > exposed > > > > > > > > > > > > > > at a specific address that is outside of the guest > > > > > > > > > > > > > > RAM > > > > > > > > > > > > > > (see the > > > > > > > > > > > > > > guest > > > > > > > > > > > > > > memory layout in public/arch-arm.h). > > > > > > > > > > > > > > > > > > > > > > > > > > > > We chose this solution over putting in the RAM > > > > > > > > > > > > > > because > > > > > > > > > > > > > > the ACPI > > > > > > > > > > > > > > tables > > > > > > > > > > > > > > are not easily relocatable (compare to the device > > > > > > > > > > > > > > tree, > > > > > > > > > > > > > > initrd and > > > > > > > > > > > > > > kernel) so we could not take advantage of superpage > > > > > > > > > > > > > > in > > > > > > > > > > > > > > both stage-2 > > > > > > > > > > > > > > (hypervisor) and stage-1 (kernel) page table. > > > > > > > > > > > > > Maybe this is something ARM-specific then. For x86 we > > > > > > > > > > > > > will > > > > > > > > > > > > > want to > > > > > > > > > > > > > keep > > > > > > > > > > > > > maxmem unchanged. > > > > > > > > > > > > I don't think what I described in my previous mail is > > > > > > > > > > > > ARM-specific. The > > > > > > > > > > > > pressure will be more important on the TLBs, if Xen does > > > > > > > > > > > > not > > > > > > > > > > > > use > > > > > > > > > > > > superpage in > > > > > > > > > > > > the stage 2 page tables (i.e EPT for x86) no matter the > > > > > > > > > > > > architecture. > > > > > > > > > > > > > > > > > > > > > > > > IHMO, this seems to be a bigger drawback compare to add > > > > > > > > > > > > few > > > > > > > > > > > > more > > > > > > > > > > > > kilobytes to > > > > > > > > > > > > maxmem in the toolstack for the ACPI blob. You will > > > > > > > > > > > > loose > > > > > > > > > > > > them when > > > > > > > > > > > > creating > > > > > > > > > > > > the intermediate page table in any case. > > > > > > > > > > > I agree with Julien. On ARM we have to increase maxmem > > > > > > > > > > > because > > > > > > > > > > > I don't > > > > > > > > > > > think that shattering a superpage is acceptable for just a > > > > > > > > > > > few > > > > > > > > > > > KBs. In > > > > > > > > > > > fact, it's not much about increasing maxmem, but it's > > > > > > > > > > > about > > > > > > > > > > > keeping > > > > > > > > > > > the > > > > > > > > > > > allocation of guest memory to the value passed by the user > > > > > > > > > > > in > > > > > > > > > > > "memory", > > > > > > > > > > > so that it can be done in the most efficient way possible. > > > > > > > > > > > (I > > > > > > > > > > > am > > > > > > > > > > > assuming users are going to allocate VMs of 2048MB, rather > > > > > > > > > > > than > > > > > > > > > > > 2049MB.) > > > > > > > > > > > > > > > > > > > > > > I wouldn't want to end up adding to the performance tuning > > > > > > > > > > > page on the > > > > > > > > > > > wiki "Make sure to add 1 more MB to the memory of your VM > > > > > > > > > > > to > > > > > > > > > > > get the > > > > > > > > > > > most out of the system." > > > > > > > > > > > > > > > > > > > > > > I know that the location of the ACPI blob on x86 is > > > > > > > > > > > different > > > > > > > > > > > in guest > > > > > > > > > > > memory space, but it seems to me that the problem would be > > > > > > > > > > > the > > > > > > > > > > > same. Do > > > > > > > > > > > you have 1 gigabyte pages in stage-2 on x86? If so, I > > > > > > > > > > > would > > > > > > > > > > > think > > > > > > > > > > > twice > > > > > > > > > > > about this. Otherwise, if you only have 4K and 2MB > > > > > > > > > > > allocations, > > > > > > > > > > > then it > > > > > > > > > > > might not make that much of a difference. > > > > > > > > > > Looking at the x86 code, 1 gigabyte pages seems to be > > > > > > > > > > supported. > > > > > > > > > > > > > > > > > > > > Boris, do you have any opinions on this? > > > > > > > > > > > > > > > > > > I don't think I understand the superpage shattering argument. > > > > > > > > > In > > > > > > > > > x86 > > > > > > > > > the tables live in regular RAM and a guest is free to use > > > > > > > > > those > > > > > > > > > addresses as regular memory. > > > > > > > > > > > > > > > > > > This apparently is different from how ARM manages the tables > > > > > > > > > (you > > > > > > > > > said > > > > > > > > > in an earlier message that they are not part of RAM) so I can > > > > > > > > > see > > > > > > > > > that > > > > > > > > > taking memory from RAM allocation to store the tables may > > > > > > > > > affect > > > > > > > > > how > > > > > > > > > mapping is done, potentially causing GB pages to be broken. > > > > > > > > > > > > > > > > > > In fact (and I am totally speculating here) padding memory for > > > > > > > > > x86 > > > > > > > > > may > > > > > > > > > actually *cause* shattering because we will have (for example) > > > > > > > > > 2049MB of > > > > > > > > > RAM to deal with. > > > > > > > > Correct me if I am wrong. On your series you are populating the > > > > > > > > page > > > > > > > > at a specific address for the ACPI tables separately to the RAM > > > > > > > > allocation. So you will shatter GB pages if the user provides > > > > > > > > 2048MB > > > > > > > > because the ACPI tables is accounted in the 2048MB. > > > > > > > And to be honest I am not convinced this was a well selected > > > > > > > address > > > > > > > (0xfc000000). I am actually thinking about moving it down (this > > > > > > > may > > > > > > > require changing dsdt.asl). I don't know whether I will actually > > > > > > > do it > > > > > > > in this version but it is certainly a possibility. > > > > > > I don't understand how this statement fits in the discussion. > > > > > > > > > > > > The memory allocation for the ACPI blob is done by the toolstack > > > > > > separately from the rest of guest memory, leading to two separate > > > > > > stage-2 pagetable allocations of less than 1-gigabyte pages. Is that > > > > > > correct? > > > > > > > > > > > > > > > If I move the table lower into memory we won't have to do any extra > > > > > allocation. The memory will have been already allocated for the guest, > > > > > we just map it and copy the tables. > > > > > > > > I see, thanks for the explanation. I think this could work for ARM too > > > > and should avoid the stage-2 shattering issue described above. > > > > > > But you will end up to have stage-1 shattering issue if you put the ACPI > > > tables lower into the guest RAM, reducing the overall performance. It is > > > why I > > > first asked Shannon to put the ACPI outside of the guest RAM. > > > > I am not sure about this actually: even with the ACPI blob in the middle > > of guest memory, the guest OS could still use a single superpage for its > > own stage-1 memory mappings. I don't know if Linux does it that way, but > > it should be possible. > > You are assuming that the guest will map the ACPI blob with the same > attributes as the rest of the superpage. > > IHMO, a sane operating system will want to map the ACPI blob read-only.
That's true. But there are other things which might be mapped differently and could shatter a stage-1 superpage mapping (especially on x86 that has a much more complex memory map than ARM). Obviously adding one more is not doing it any good, but it might not make a difference in practice. Anyway, I agree with Julien that his suggestion is the best for ARM. If the libxl maintainers are willing to accept two different code paths for this on ARM and x86, then I am fine with it too. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
