On Tue, Aug 18, 2026 at 09:57:12PM +0800, Yanfei Xu wrote: > > On 2026/8/18 21:01, Peter Xu wrote: > > On Tue, Aug 18, 2026 at 07:57:06PM +0800, Yanfei Xu wrote: > > > Hi Peter, > > Hi, Yanfei, > > > > > No objection to removing the dead code — it clearly never worked > > > > > > I do have one question about the direction, though. The removed logic > > > was the only in-tree attempt at MR unregistration for the non-pin-all > > > path. Without it, registered MRs grow monotonically over a migration, > > > and with large, widely-spread dirty memory over chunks the accumulated > > > MR metadata (user + kernel) can cost more than pin-all and even perform > > > worse — which rather defeats the purpose of not pinning everything. > > > > > > do we still intend to keep and improve the non-pin-all path going > > > forward? If so, some form of dynamic MR unregistration will eventually > > > be needed and it might be worth keeping this code，or at least leaving > > > a TODO to mark the gap? > > Thanks for taking a look. This is a valid question to ask. > > > > Though it was there for 13 years without being "enhanced", it means the > > possibility we leverage it in the next couple of years is low. > > > > You also discussed the other side of things: I am not a frequent RDMA user, > > but my understanding is frequent MR reg operations already slow down > > migration quite a bit. It means dynamic management including unregisters > > will be even worse. AFAICT, it'll be a challenging task if we want to keep > > the performance in bar and add a hard throttle to pinned memory. > > One advantage of non-pin-all is that it neither sends the all-zero chunk > nor registers the corresponding MRs. For guests with a low dirty-page > workload and a large number of zero pages, this lets it migrate faster > than pin-all and pin less guest memory during the migration.
I actually don't know why RDMA_CONTROL_COMPRESS is only used in !pin_all, do you know? I can only guess RDMA WRITEs were fast and need no round-robin chats, so it's faster than RDMA_CONTROL_COMPRESS, but you seem to say it's not true. Meanwhile, I would expect pin-all=off ultimately should meet the same perf over pin-all=on.. so I don't really know how needs pin-all=on... maybe it's useful when one is looking for minimum total migration time when VMs are required to be evicted from one host? Feel free to share if you have more data points; I'm almost speaking from reading the code, so it could be wrong. Thanks, -- Peter Xu
