Re: [Android-virt] [PATCH v5 04/13] ARM: KVM: Memory virtualization setup
On Tue, Dec 13, 2011 at 12:10 PM, Antonios Motakis wrote: > On 12/12/2011 04:49 PM, Avi Kivity wrote: >> On 12/12/2011 05:25 PM, Peter Maydell wrote: >>> On 12 December 2011 15:15, Avi Kivity wrote: We need to differentiate in how Linux-as-a-guest acts and how the cpu is supposed to work. A guest operating system can theoretically assign the ASID x to process A running on vcpu 0, and the same ASID x to process B running on vcpu 1 >>> That would be a guest bug. From the ARM ARM: >>> "For a symmetric multiprocessor cluster where a single operating system >>> is running on the set of processing elements, ARMv7 requires all ASID >>> values to be assigned uniquely within any single Inner Shareable domain. >>> In other words, each ASID value must have the same meaning to all >>> processing elements in the system." >> Thanks. So per-vm vmids should work. >> > We where playing with a VMID recycling patch based on the assumption of > per-cpu VMIDs being possible, which would have the advantage of > recycling VMIDs without much complicated locking (inspired by the KVM > SVM implementation). However we killed it with fire and hot plasma when > it became clear it violated the ARM spec... > > On the other hand, maybe we could do something with per vcpu VMIDs, but > with proper synchronization accross physical CPUs in order to be > compatible with the spec, but at the same time potentially allow a buggy > guest to run? Since in practice a lot of CPUs will not share TLB (and > instruction cache) structures, maybe it's possible that there is > software out there that violates the spec, without having problems on > the real hw. > > Anyway VMID reuse will be available soon, and the difference between a > per vm and per vcpu implementation is a couple of trivial lines of code. > yes, this is going to be a simple per-vm implementation that flushes TLBs on roll-over for the next patch series, let's leave it at that for now! -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Android-virt] [PATCH v5 04/13] ARM: KVM: Memory virtualization setup
On 12/12/2011 04:49 PM, Avi Kivity wrote: On 12/12/2011 05:25 PM, Peter Maydell wrote: On 12 December 2011 15:15, Avi Kivity wrote: We need to differentiate in how Linux-as-a-guest acts and how the cpu is supposed to work. A guest operating system can theoretically assign the ASID x to process A running on vcpu 0, and the same ASID x to process B running on vcpu 1 That would be a guest bug. From the ARM ARM: "For a symmetric multiprocessor cluster where a single operating system is running on the set of processing elements, ARMv7 requires all ASID values to be assigned uniquely within any single Inner Shareable domain. In other words, each ASID value must have the same meaning to all processing elements in the system." Thanks. So per-vm vmids should work. We where playing with a VMID recycling patch based on the assumption of per-cpu VMIDs being possible, which would have the advantage of recycling VMIDs without much complicated locking (inspired by the KVM SVM implementation). However we killed it with fire and hot plasma when it became clear it violated the ARM spec... On the other hand, maybe we could do something with per vcpu VMIDs, but with proper synchronization accross physical CPUs in order to be compatible with the spec, but at the same time potentially allow a buggy guest to run? Since in practice a lot of CPUs will not share TLB (and instruction cache) structures, maybe it's possible that there is software out there that violates the spec, without having problems on the real hw. Anyway VMID reuse will be available soon, and the difference between a per vm and per vcpu implementation is a couple of trivial lines of code. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Android-virt] [PATCH v5 04/13] ARM: KVM: Memory virtualization setup
On Mon, Dec 12, 2011 at 10:49 AM, Avi Kivity wrote: > On 12/12/2011 05:25 PM, Peter Maydell wrote: >> On 12 December 2011 15:15, Avi Kivity wrote: >> > We need to differentiate in how Linux-as-a-guest acts and how the cpu is >> > supposed to work. A guest operating system can theoretically assign the >> > ASID x to process A running on vcpu 0, and the same ASID x to process B >> > running on vcpu 1 >> >> That would be a guest bug. From the ARM ARM: >> "For a symmetric multiprocessor cluster where a single operating system >> is running on the set of processing elements, ARMv7 requires all ASID >> values to be assigned uniquely within any single Inner Shareable domain. >> In other words, each ASID value must have the same meaning to all >> processing elements in the system." > > Thanks. So per-vm vmids should work. > yep. -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Android-virt] [PATCH v5 04/13] ARM: KVM: Memory virtualization setup
On 12/12/2011 05:25 PM, Peter Maydell wrote: > On 12 December 2011 15:15, Avi Kivity wrote: > > We need to differentiate in how Linux-as-a-guest acts and how the cpu is > > supposed to work. A guest operating system can theoretically assign the > > ASID x to process A running on vcpu 0, and the same ASID x to process B > > running on vcpu 1 > > That would be a guest bug. From the ARM ARM: > "For a symmetric multiprocessor cluster where a single operating system > is running on the set of processing elements, ARMv7 requires all ASID > values to be assigned uniquely within any single Inner Shareable domain. > In other words, each ASID value must have the same meaning to all > processing elements in the system." Thanks. So per-vm vmids should work. -- error compiling committee.c: too many arguments to function -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Android-virt] [PATCH v5 04/13] ARM: KVM: Memory virtualization setup
On 12 December 2011 15:15, Avi Kivity wrote: > We need to differentiate in how Linux-as-a-guest acts and how the cpu is > supposed to work. A guest operating system can theoretically assign the > ASID x to process A running on vcpu 0, and the same ASID x to process B > running on vcpu 1 That would be a guest bug. From the ARM ARM: "For a symmetric multiprocessor cluster where a single operating system is running on the set of processing elements, ARMv7 requires all ASID values to be assigned uniquely within any single Inner Shareable domain. In other words, each ASID value must have the same meaning to all processing elements in the system." -- PMM -- To unsubscribe from this list: send the line "unsubscribe kvm" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Android-virt] [PATCH v5 04/13] ARM: KVM: Memory virtualization setup
On 12/12/2011 05:09 PM, Christoffer Dall wrote:
> On Mon, Dec 12, 2011 at 9:40 AM, Avi Kivity wrote:
> > On 12/11/2011 12:24 PM, Christoffer Dall wrote:
> >> This commit introduces the framework for guest memory management
> >> through the use of 2nd stage translation. Each VM has a pointer
> >> to a level-1 tabled (the pgd field in struct kvm_arch) which is
> >> used for the 2nd stage translations. Entries are added when handling
> >> guest faults (later patch) and the table itself can be allocated and
> >> freed through the following functions implemented in
> >> arch/arm/kvm/arm_mmu.c:
> >> - kvm_alloc_stage2_pgd(struct kvm *kvm);
> >> - kvm_free_stage2_pgd(struct kvm *kvm);
> >>
> >> Further, each entry in TLBs and caches are tagged with a VMID
> >> identifier in addition to ASIDs. The VMIDs are managed using
> >> a bitmap and assigned when creating the VM in kvm_arch_init_vm()
> >> where the 2nd stage pgd is also allocated. The table is freed in
> >> kvm_arch_destroy_vm(). Both functions are called from the main
> >> KVM code.
> >>
> >>
> >> struct kvm_arch {
> >> - pgd_t *pgd; /* 1-level 2nd stage table */
> >> + u32vmid;/* The VMID used for the virt. memory system */
> >> + pgd_t *pgd; /* 1-level 2nd stage table */
> >> + u64vttbr; /* VTTBR value associated with above pgd and vmid */
> >> };
> >>
> >
> > I can't say I have a solid grasp here, but my feeling is that vmid needs
> > to be per-vcpu. Otherwise vcpu 1 can migrate to a cpu that previously
> > ran vcpu 0, and reuse its tlb since they have the same vmid.
> >
> According to the ARM guys, the same rules apply as for ASIDs which
> results into 1 vmid per VM. The vmid is checked along with a
> guest-specific asid, so they would only share TLB entries when running
> the same guest process, which I assume is fine then. Actually
> desirable. But, granted, my SMP knowledge is limited so far, so if
> Marc or Catalin feels like chipping in here...
We need to differentiate in how Linux-as-a-guest acts and how the cpu is
supposed to work. A guest operating system can theoretically assign the
ASID x to process A running on vcpu 0, and the same ASID x to process B
running on vcpu 1, and be sure that TLB entries don't leak from A to B
since the TLB is cpu local (is that in fact correct?). Does the ARM
arch allow this, or does it forbid running different contexts on
different vcpus with the same ASID?
--
error compiling committee.c: too many arguments to function
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [Android-virt] [PATCH v5 04/13] ARM: KVM: Memory virtualization setup
On Mon, Dec 12, 2011 at 9:40 AM, Avi Kivity wrote:
> On 12/11/2011 12:24 PM, Christoffer Dall wrote:
>> This commit introduces the framework for guest memory management
>> through the use of 2nd stage translation. Each VM has a pointer
>> to a level-1 tabled (the pgd field in struct kvm_arch) which is
>> used for the 2nd stage translations. Entries are added when handling
>> guest faults (later patch) and the table itself can be allocated and
>> freed through the following functions implemented in
>> arch/arm/kvm/arm_mmu.c:
>> - kvm_alloc_stage2_pgd(struct kvm *kvm);
>> - kvm_free_stage2_pgd(struct kvm *kvm);
>>
>> Further, each entry in TLBs and caches are tagged with a VMID
>> identifier in addition to ASIDs. The VMIDs are managed using
>> a bitmap and assigned when creating the VM in kvm_arch_init_vm()
>> where the 2nd stage pgd is also allocated. The table is freed in
>> kvm_arch_destroy_vm(). Both functions are called from the main
>> KVM code.
>>
>>
>> struct kvm_arch {
>> - pgd_t *pgd; /* 1-level 2nd stage table */
>> + u32 vmid; /* The VMID used for the virt. memory system */
>> + pgd_t *pgd; /* 1-level 2nd stage table */
>> + u64 vttbr; /* VTTBR value associated with above pgd and vmid */
>> };
>>
>
> I can't say I have a solid grasp here, but my feeling is that vmid needs
> to be per-vcpu. Otherwise vcpu 1 can migrate to a cpu that previously
> ran vcpu 0, and reuse its tlb since they have the same vmid.
>
According to the ARM guys, the same rules apply as for ASIDs which
results into 1 vmid per VM. The vmid is checked along with a
guest-specific asid, so they would only share TLB entries when running
the same guest process, which I assume is fine then. Actually
desirable. But, granted, my SMP knowledge is limited so far, so if
Marc or Catalin feels like chipping in here...
--
To unsubscribe from this list: send the line "unsubscribe kvm" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
