20/11/2024 00:23, Thea Corinne Rossman: > Conceptually, for two containers on the same host, how would exchanging > traffic work under-the-hood? Specifically, how is the physical NIC > involved, if at all?
Traffic between containers on the same host does not need to use a PCI device. You can check virtio-user for this usage: https://doc.dpdk.org/guides/howto/virtio_user_for_container_networking.html > For example, on a physical host: for TX, a userspace application writes a > packet to host memory and pushes its physical address/metadata to the > appropriate NIC's TX queue. The NIC uses the physical address + DMA to > avoid a copy when serializing/sending. (Similar for RX in the other > direction, where the NIC writes to memory.) > > I'm not sure how this would translate to a containerized case on a single > host, since traffic shouldn't need to exit and the container network has > its own namespace. Say that two different containers have NICs mapped to > different PCI addresses (split device). If container A appends to its TX > queue, what happens next? > > Thanks again for your help. > > On Tue, Nov 19, 2024 at 2:14 PM Thomas Monjalon <tho...@monjalon.net> wrote: > > > 19/11/2024 22:39, Thea Corinne Rossman: > > > This is SO helpful -- thank you so much. > > > > > > One follow-up question regarding NICs: can multiple containers on the > > same > > > host share the same PCI device? If I have a host NIC with (say) VFIO > > driver > > > binding, do I have to split it with some kind of SR-IOV so that each > > > container has its own "NIC" binding? Or, when running DPDK's "devbind" > > > script, can I set up each one with the same PCI address? > > > > You need to split devices. > > SR-IOV VF may help, or you can use SF which was designed exactly for this, > > with the help of the Linux auxiliary bus.
