On Mon, Mar 07, 2016 at 11:21:33AM +0000, George Dunlap wrote:
> On Fri, Mar 4, 2016 at 10:00 PM, Konrad Rzeszutek Wilk
> <konrad.w...@oracle.com> wrote:
> >> +/* Handle VT-d posted-interrupt when VCPU is blocked. */
> >> +static void pi_wakeup_interrupt(struct cpu_user_regs *regs)
> >> +{
> >> + struct arch_vmx_struct *vmx, *tmp;
> >> + spinlock_t *lock = &per_cpu(vmx_pi_blocking, smp_processor_id()).lock;
> >> + struct list_head *blocked_vcpus =
> >> + &per_cpu(vmx_pi_blocking, smp_processor_id()).list;
> >> +
> >> + ack_APIC_irq();
> >> + this_cpu(irq_count)++;
> >> +
> >> + spin_lock(lock);
> >> +
> >> + /*
> >> + * XXX: The length of the list depends on how many vCPU is current
> >> + * blocked on this specific pCPU. This may hurt the interrupt latency
> >> + * if the list grows to too many entries.
> >> + */
> >> + list_for_each_entry_safe(vmx, tmp, blocked_vcpus, pi_blocking.list)
> >> + {
> >
> >
> > My recollection of the 'most-horrible' case of this being really bad is when
> > the scheduler puts the vCPU0 and VCPU1 of the guest on the same pCPU (as an
> > example)
> > and they round-robin all the time.
> >
> > <handwaving>
> > Would it be perhaps possible to have an anti-affinity flag to deter the
> > scheduler from this? That is whichever struct vcpu has 'anti-affinity' flag
> > set - the scheduler will try as much as it can _to not_ schedule the
> > 'struct vcpu'
> > if the previous 'struct vcpu' had this flag as well on this pCPU?
>
> Well having vcpus from the same guest on the same pcpu is problematic
> for a number of reasons -- spinlocks first and foremost. So in
> general trying to avoid that would be useful for most guests.
PV ticketlocks in HVM and PV guests make this "manageable".
>
> The thing with scheduling is that it's a bit like economics: it seems
> simple but it's actually not at all obvious what the emergent behavior
> will be from adding a simple rule. :-)
<nods>
>
> On the whole it seems unlikely that having two vcpus on a single pcpu
> is a "stable" situation -- it's likely to be pretty transient, and
> thus not have a major impact on performance.
Except that we are concerned with it - in fact we are disabling this
feature because it may happen. How do we make sure it does not happen
all the time? Or at least do some back-off if things do get
in this situation.
>
> That said, the load balancing code from credit2 *should*, in theory,
> make it easier to implement this sort of thing; it has the concept of
> a "cost" that it's trying to minimize; so you could in theory add a
> "cost" to configurations where vcpus from the same processor share the
> same pcpu. Then it's not a hard-and-fast rule: if you have more vcpus
> than pcpus, the scheduler will just deal. :-)
>
> But I think some profiling is in order before anyone does serious work on
> this.
I appreciate your response being 'profiling' instead of 'Are you
NUTS!?' :-)
>
> -George
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
