[ ... ]
I as a patch writer always like to do that when it's essential. Normally
the case is I don't have enough reviewer resources to help me get a better
design, or discuss about it.
Right, but we can't keep providing a moving target. See the thread pool
discussion for an example. It's hard to work that way. The discussion
here is similar, we introduced the union, now we're moving to the
struct. And you're right that the changes here are small, so let's not
get caught in that.
What's your suggestion on the thread pool? Should we merge the change
where vfio creates the threads on its own (assuming vfio maintainers are ok
with it)?
I would say no, that's what I suggested. I'd start with reusing
ThreadPool, then we found issue when Stefan reported worry on abusing the
API. All these discussions seem sensible to me so far. We can't rush on
these until we figure things out step by step. I don't see a way.
I saw Cedric suggesting to not even create a thread on recv side. I am not
sure whether that's easy, but I'd agree with Cedric if possible. I think
Maciej could have a point where it can block mutlifd threads, aka, IO
threads, which might be unwanted.
Sorry, If I am adding noise on this topic. I made this suggestion
because I spotted some asymmetry in the proposal.
The send and recv implementation in VFIO relies on different
interfaces with different level of complexity. The send part is
using a set of multifd callbacks called from multifd threads,
if I am correct. Whereas the recv part is directly implemented
in VFIO with local thread(s?) doing their own state receive cookery.
I was expecting a common interface to minimize assumptions on both
ends. It doesn't have to be callback based. It could be a set of
services a subsystem could use to transfer state in parallel.
<side note>
VFIO migration is driven by numerous callbacks and it is
difficult to understand the context in which these are called.
Adding more callbacks might not be the best approach.
</side note>
The other comment was on optimisation. If this is an optimisation
then I would expect, first, a non-optimized version not using threads
(on the recv side).
VFIO Migration is a "new" feature which needs some more run-in.
That said, it is stable, MLX5 VFs devices have good support, you
can rely on me to evaluate the future respins.
Thanks,
C.
