On Wed, Nov 15, 2023 at 2:09 PM Akihiko Odaki <akihiko.od...@daynix.com> wrote: > > > > On 2023/11/15 7:09, Yuri Benditovich wrote: > > > > > > On Tue, Nov 14, 2023 at 9:03 AM Akihiko Odaki <akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>> wrote: > > > > On 2023/11/14 2:26, Yuri Benditovich wrote: > > > > > > > > > On Mon, Nov 13, 2023 at 2:44 PM Akihiko Odaki > > <akihiko.od...@daynix.com <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>>> wrote: > > > > > > On 2023/11/13 20:44, Yuri Benditovich wrote: > > > > > > > > > > > > On Sat, Nov 11, 2023 at 5:28 PM Akihiko Odaki > > > <akihiko.od...@daynix.com <mailto:akihiko.od...@daynix.com> > > <mailto:akihiko.od...@daynix.com <mailto:akihiko.od...@daynix.com>> > > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>>>> wrote: > > > > > > > > On 2023/11/03 22:14, Yuri Benditovich wrote: > > > > > > > > > > > > > > > On Fri, Nov 3, 2023 at 11:55 AM Akihiko Odaki > > > > <akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>>> > > > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>> > > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>>>>> wrote: > > > > > > > > > > On 2023/11/03 18:35, Yuri Benditovich wrote: > > > > > > > > > > > > > > > > > > On Thu, Nov 2, 2023 at 4:56 PM Akihiko Odaki > > > > > <akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>> <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>>> > > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>> <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>>>> > > > > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>> > > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>>> > > > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>> > > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com> > > > <mailto:akihiko.od...@daynix.com > > <mailto:akihiko.od...@daynix.com>>>>>> wrote: > > > > > > > > > > > > On 2023/11/02 19:20, Yuri Benditovich wrote: > > > > > > > > > > > > > > > > > > > > > On Thu, Nov 2, 2023 at 11:33 AM > > Michael S. > > > Tsirkin > > > > > > <m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>> > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>>> > > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>> > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>>>> > > > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>> > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>>> > > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>> > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>>>>> > > > > > > > <mailto:m...@redhat.com > > <mailto:m...@redhat.com> > > > <mailto:m...@redhat.com <mailto:m...@redhat.com>> > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>>> > > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>> > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>>>> > > > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>> > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>>> > > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>> > > > <mailto:m...@redhat.com <mailto:m...@redhat.com> > > <mailto:m...@redhat.com <mailto:m...@redhat.com>>>>>>> wrote: > > > > > > > > > > > > > > On Thu, Nov 02, 2023 at 11:09:27AM > > > +0200, Yuri > > > > > Benditovich wrote: > > > > > > > > Probably we mix two different > > patches > > > in this > > > > > discussion. > > > > > > > > Focusing on the patch in the > > e-mail > > > header: > > > > > > > > > > > > > > > > IMO it is not acceptable to > > fail QEMU run > > > > for one > > > > > feature > > > > > > that we > > > > > > > can't make > > > > > > > > active when we silently drop > > all other > > > > features in > > > > > such a > > > > > > case. > > > > > > > > > > > > > > If the feature is off by default > > then it > > > seems more > > > > > reasonable > > > > > > > and silent masking can be seen as > > a bug. > > > > > > > Most virtio features are on by > > default > > > this is > > > > why it's > > > > > > > reasonable to mask them. > > > > > > > > > > > > > > > > > > > > > If we are talking about RSS: setting it > > > initially > > > > off is the > > > > > > development > > > > > > > time decision. > > > > > > > When it will be completely stable > > there is > > > no reason to > > > > > keep it > > > > > > off by > > > > > > > default, so this is more a question > > of time > > > and of a > > > > > readiness of > > > > > > libvirt. > > > > > > > > > > > > It is not ok to make "on" the default; > > that will > > > > enable RSS > > > > > even when > > > > > > eBPF steering support is not present and can > > > result in > > > > > performance > > > > > > degradation. > > > > > > > > > > > > > > > > > > Exactly as it is today - with vhost=on the host > > > does not > > > > suggest RSS > > > > > > without eBPF. > > > > > > I do not understand what you call "performance > > > > degradation", can you > > > > > > describe the scenario? > > > > > > > > > > I was not clear, but I was talking about the > > case of > > > > vhost=off or peers > > > > > other than tap (e.g., user). rss=on employs > > in-qemu RSS, > > > > which incurs > > > > > overheads for such configurations. > > > > > > > > > > > > > > > So, vhost=off OR peers other than tap: > > > > > > > > > > In the case of peers other than tap (IMO) we're not > > > talking about > > > > > performance at all. > > > > > Backends like "user" (without vnet_hdr) do not > > support _many_ > > > > > performance-oriented features. > > > > > If RSS is somehow "supported" for such backends this is > > > rather a > > > > > misunderstanding (IMO again). > > > > > > > > We do not need to ensure good performance when RSS is > > enabled > > > by the > > > > guest for backends without eBPF steering program as > > you say. > > > In-QEMU > > > > RSS > > > > is only useful for testing and not meant to improve the > > > performance. > > > > > > > > However, if you set rss=on, QEMU will advertise the > > > availability of RSS > > > > feature. The guest will have no mean to know if it's > > > implemented in a > > > > way not performance-wise so it may decide to use the > > feature > > > to improve > > > > the performance, which can result in performance > > degradation. > > > > Therefore, > > > > it's better not to set rss=on for such backends. > > > > > > > > > > > > I still do not understand what is the scenario where you > > see or > > > suspect > > > > the mentioned "performance degradation". > > > > We can discuss whether such a problem exists as soon as you > > > explain it. > > > > > > The scenario is that: > > > - rss=on, > > > - A backend without eBPF steering support is in use, and > > > - The guest expects VIRTIO_NET_F_RSS has little overheads as > > hardware > > > RSS implementations do. > > > > > > I consider the risk of the performance degradation in such a > > situation > > > is the reason why virtio-net emits a warning ("Can't load > > eBPF RSS - > > > fallback to software RSS") when in-QEMU RSS is in use. > > > > > > > > > In a described scenario (vhost=off) I do not see why the performance > > > degradation should happen: > > > the SW RSS (if activated) will place each packet into proper > > queue (even > > > if the auto_mq in kernel is not able to do that) and such a way the > > > guest will not need to reschedule the packet to proper CPU > > > > > > > The scenario I'm concerned is that the guest has its own packet > > steering > > mechanism which is feature-wise superior to RSS. For example, Linux has > > such a mechanism called RPS, which has some advantages due to its > > extensible nature according to: > > > > https://www.kernel.org/doc/html/v6.6/networking/scaling.html#rps-receive-packet-steering > > > > <https://www.kernel.org/doc/html/v6.6/networking/scaling.html#rps-receive-packet-steering> > > > > Such a guest may still prefer hardware RSS if available since hardware > > RSS is expected to have less overheads. However, it is not true for > > in-qemu RSS, and using in-QEMU RSS instead of the guest-side steering > > mechanism may just hide useful features the guest-side steering > > mechanism has and result in performance degradation. > > > > > > Note that in terms of per-packet computation for RSS the in-QEMU RSS > > does exactly the same operations in native code that the eBPF does in > > the emulation. > > So, I wouldn't say that SW RSS brings some "performance degradation". > > We prefer eBPF as it can serve both vhost and non-vhost setups. > > I see the eBPF steering program in a different way. > > tuntap can have several queues,
Note that RSS can benefit for single queue as well. > and the assignment of packets is > automatically done by tuntap by default. However, the default assignment > policy may not be good for all applications, and some may need a > different policy. Such an application can still re-assign packets to > application-internal queues that reside on different CPUs and that's > what 'in-qemu' RSS does. However, that incurs communication overheads > across CPUs. The eBPF steering program can eliminate the overheads by > allowing the userspace to override the packet assignment policy with eBPF. > Yes, eBPF steering can do more than just RSS. For VM use cases, it just needs to wait for the spec support. For example, spec will support receiving flow director soon, we can leverage the eBPF support for tuntap to do that. > The eBPF steering program feature of tuntap and its benefit are > independent of the vhost usage. In fact, the feature predates the usage > in QEMU that combines vhost and eBPF steering program. I think they should coexist. For example, for many reasons eBPF might not be available on the host. > The initial > proposal of the eBPF steering program I found is submitted by Jason and > available at: > https://lore.kernel.org/lkml/1506500637-13881-1-git-send-email-jasow...@redhat.com/ > > Jason, please point out if you find something wrong in my understanding > with the eBPF steering program feature or something to add. Thanks >
