Hi,
On 12/12/2025 10:01, Mykyta Poturai wrote:
On 16.11.25 13:24, Julien Grall wrote:
Hi,
On 12/11/2025 10:51, Mykyta Poturai wrote:
Move IRQs from dying CPU to the online ones.
Guest-bound IRQs are already handled by scheduler in the process of
moving vCPUs to active pCPUs, so we only need to handle IRQs used by Xen
itself.
If IRQ is to be migrated, it's affinity is set to a mask of all online
CPUs. With current GIC implementation, this means they are routed to a
random online CPU. This may cause extra moves if multiple cores are
disabled in sequence, but should prevent all interrupts from piling up
on CPU0 in case of repeated up-down cycles on different cores.
Wouldn't they eventually all move to CPU0 in the case of suspend/resume
or if all the CPUs but CPU0 are turned off and then off? If so,
shouldn't we try to rebalance the interrupts?
In case of disabling/enabling all cores in a sequence, yes. This was
designed with the idea of achieving some balancing when
enabling/disabling some cores for power saving reasons.
I understand how this may balance when disabling some cores. But I don't
understand how it helps for enabling cores. Can you provide more details?
I agree that
proper balancing should be implemented, but it is a complex task on its
own and requires a substantial amount of testing on different hardware
to prove it is close to optimal. So I think implementing it is out of
scope for what I’m trying to do here.
Can you provide some details about what you are trying and why it would
be ok to avoid the balancing?
> > If this would be okay, I can implement a relatively simple solution of
just adding onlined CPUs back to the affinity mask for now. I think it
should improve the situation for the “switching all cores” case.
Do you mean calling gic_irq_set_affinity() with the CPU? If so, Xen is
still going to get one CPU from the affinity mask.
IRQs from CPU 0 are never migrated, as dying CPU 0 means we are either
shutting down compeletely or entering system suspend.
I can't find a place where __cpu_disable() is called on CPU0. Do you
have any pointer? In any case, I am not sure I want to bake that
assumption in more places of the code.
I assume it would be called when suspend is implemented. In any case, I
will rework this to replace the hard check for CPU 0 with the “is it the
last CPU online” one.
AFAIK __cpu_disable() is not going to be called during suspend/resume
for CPU0. So the only case is shutting down the platform.
Considering that all Xen-used IRQs are currently allocated during init
on CPU 0, and setup_irq uses smp_processor_id for the initial affinity.
Looking at the SMMU driver, we seems to request IRQs at the time the
device is attached. So are you sure about this?
Indeed, I have missed that one. I will remove those statements then.
I think you need to have something in the commit message explaining why
you are ignoring the balancing for the SMMU.
Cheers,
--
Julien Grall
