Hi Dario,
On 06/12/2016 22:39, Dario Faggioli wrote:
On Tue, 2016-12-06 at 13:53 -0800, Stefano Stabellini wrote:
On Tue, 6 Dec 2016, Dario Faggioli wrote:
Sorry if I can't be more useful than this for now. :-/
We don't need scheduler support to implement interrupt migration. The
question was much simpler than that: moving a vCPU with interrupts
assigned to it is slower than moving a vCPU without interrupts
assigned
to it. You could say that the slowness is directly proportional do
the
number of interrupts assigned to the vCPU. Does the scheduler know
that?
Or blindly moves vCPUs around? Also see below.
Ah, ok, it is indeed a simpler question than I thought! :-)
Answer: no, the scheduler does not use the information of how many or
what interrupts are being routed to a vCPU in any way.
Just for the sake of correctness and precision, it does not "blindly
moves vCPUs around", as in, it follows some criteria for deciding
whether or not to move a vCPU, and if yes, where to, but among those
criteria, there is no trace of anything related to routed interrupts.
Let me also add that the criteria are scheduler specific, so they're
different, e.g., between Credit and Credit2.
Starting considering routed interrupt as a migration criteria in Credit
would be rather difficult. Credit use a 'best effort' approach for
migrating vCPUs, which is hard to augment.
Starting considering routed interrupt as a migration criteria in
Credit2 would be much easier. Credit2's load balancer is specifically
designed for being extendible with things like that. It would require
some thinking, though, in order to figure out how important this
particular aspect would be, wrt others that are considered.
E.g., if I have pCPU 0 loaded at 75% and pCPU 1 loaded at 25%, vCPU A
has a lot of routed interrupts, and moving it gives me perfect load
balancing (i.e., load will become 50% on pCPU 0 and 50% on pCPU 1)
should I move it or not?
Well, it depends if whether or not we think that the overhead we save
by not migrating outweights the benefit of a perfectly balanced system.
Something like that...
This idea to migrate IRQ while migrating vCPU is already wrong to me as
this is really expensive, I am talking in term of milliseconds easily as
in some case Xen will have to interact with a shared command queue.
Spreading the load is much better and avoid to migrate directly IRQ that
are disabled when the vCPU is been migrated.
I cannot see any reason to not do that as changing the interrupt
affinity may be not taken into account directly. By that, I mean it
might be possible to receive another interrupt on the wrong pCPU.
I really think we should make the vCPU migration much simpler (e.g avoid
this big loop over interrupt). In fine, if we really expect the
scheduler to migrate the vCPU on a different pCPU. We should also expect
receiving the interrupt on the wrong pCPU may not happen often.
Regards,
--
Julien Grall
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel