On 17/12/2025 22:59, Peter Xu wrote: > Right, it will, because any time used for sync has the vCPUs running, so > that will contributes to the total dirtied pages, hence partly increase D, > as you pointed out. > > But my point is, if you _really_ have R=B all right, you should e.g. on a > 10Gbps NIC seeing R~=10Gbps. If R is not wire speed, it means the R is not > really correctly measured..
In my experience, the bandwidth of live migration usually doesn't reach the nic's bandwidth limit (my test environment's nic bandwidth limit is 200Gbps). This could be due to various reasons: for example, the live migration main thread's ability to search for dirty pages may have reached a bottleneck; the nic's interrupt binding range might limit the softirq's processing capacity; there might be too few multifd threads; or there might be overhead in synchronizing between the live migration main thread and the multifd thread. > > I think it's likely impossible to measure the correct R so that it'll equal > to B, however IMHO we can still think about something that makes the R > getting much closer to B, then when normally y is a constant (default > 300ms, for example) it'll start to converge where it used to not be able to. Yes, there are always various factors that can cause measurement errors. We can only try to make the calculated value as close as possible to the actual value. > E.g. QEMU can currently report R as low as 10Mbps even if on 10Gbps, IMHO > it'll be much better and start solving a lot of such problems if it can > start to report at least a few Gbps based on all kinds of methods > (e.g. excluding sync, as you experimented), then even if it's not reporting > 10Gbps it'll help. > After I applied these optimizations, typically the bandwidth statistics from QEMU and the real-time nic bandwidth monitored by atop are close. Those extremely low bandwidth(but consistent with atop monitoring) is usually caused by zero pages or dirty pages with extremely high compression rates. In these cases, QEMU uses very little nic bandwidth to transmit a large number of dirty pages, but the bandwidth is only calculated based on the actual amount of data transmitted. If we want to use the actual number of dirty pages transmitted to calculate bandwidth, we face another risk: if the dirty pages transmitted before the downtime have a high compression ratio, and the dirty pages to be transmitted after the downtime have a low compression ratio, then the downtime will far exceed expectations. This may have strayed a bit, but just providing some potentially useful information from my perspective. Thanks!
