On Mon, Jun 20, 2022 at 11:23:53AM +0200, Juan Quintela wrote: > Once discussed this, what I asked in the past is that you are having too > much dirty memory on zero_copy. When you have a Multiterabyte guest, in > a single round you have a "potentially" dirty memory on each channel of: > > total_amount_memory / number of channels. > > In a Multiterabyte guest, this is going to be more that probably in the > dozens of gigabytes. As far as I know there is no card/driver that will > benefit for so many pages in zero_copy, and kernel will move to > synchronous copy at some point. (In older threads, daniel showed how to > test for this case).
I was wondering whether the kernel needs to cache a lot of messages for zero copy if we don't flush it for a long time, as recvmsg(MSG_ERRQUEUE) seems to be fetching one message from the kernel one at a time. And, whether that queue has a limit in length or something. Does it mean that when the kernel could have cached enough of these messages then it'll fallback to the no-zero-copy mode? And probably that's the way how kernel protects itself from using too much buffer for the error msgs? This reminded me - Leo, have you considered adding the patch altogether to detect the "fallback to non-zero-copy" condition? Because when with it and when the fallback happens at some point (e.g. when the guest memory is larger than some value) we'll know. Thanks, -- Peter Xu
