Hi Richard, I understand your motivations, and that's why I suggested to slow down your generator and see what happens since I had the same suspect introduced by Luigi Rizzo, (receive livelock). Probably if you increase the packet rate progressively, you should be able to clarify this. I would also try (during these tests) to assign a predefined cpu core to the vm and follow its load. This seems similar to my brief experience with Click virtualized within Linux Containers, I couldn't overcome 60.000pps. When I rised up the rate further, the forwarding performance went down very quickly.
A curiosity: which hardware platform are you using exactly ? Ciao, Giorgio On Thu, Mar 21, 2013 at 12:31 PM, Richard Neumann <[email protected]>wrote: > Hello again, > > @Giorgio Calarco: > Hi Richard, what happens if you decrease the input rate at lower > values? For instance at 40000pps? (maybe you have to change your > packet generator to perform this experiment ). And, where is > your brctl-based bridge created ? Within dom0? > Let me know, > > I'll try to run a test with a slower packet generator and will let you > know how it performs then. But the problem is that we actually do want > to do high-speed packet processing here. > The brige is created inside the domU environment, connecting the two 10G > interfaces which have been passed through from the dom0. > > @Luigi Rizzo: > > At first sight it seems a case of receive livelock, which does not occur > > so badly with the linux bridge as it (probably) is helped by NAPI. > > > > It is not clear if your userspace click is also using netmap (which > > may have its own bugs) and/or whether the two interfaces eth1 and > > eth2 are using pci-passthrough or are emulated/paravirtualized. > > in the respective setup, click is is not using netmap and even was > compiled without the --with-netmap stuff. > The 10G Cards are using pci-passthrough, because I expected better > performance from this than from emulation or paravirtualization (I did > not actually compare it yet). > > > In any case the 530Kpps you get with linux bridge is probably close > > to the peak performance you can get, unless (a) Xen gives you access > > to the real hw (via virtual functions and/or pci-passthrough), and > > So, for I am actually am using PCI passthrough, the performance should > be better even using a linux bridge. > > > (b) you are using netmap or some very fast os stack bypass to talk > > to the network interfaces within the virtual machine. > > > > I managed to go quite fast within qemu+kvm, see below > > > > http://info.iet.unipi.it/~luigi/netmap/talk-google-2013.html > > http://info.iet.unipi.it/~luigi/papers/20130206-qemu.pdf > > > > but that needed a little bit of tweaking here and there > > (not that i doubt that the Xen folks are smart enough to do > > something similar). > > Thank you both for your help so far. > > Best regards, > > Richard > > > _______________________________________________ > click mailing list > [email protected] > https://amsterdam.lcs.mit.edu/mailman/listinfo/click > _______________________________________________ click mailing list [email protected] https://amsterdam.lcs.mit.edu/mailman/listinfo/click
