On Sat, 2007-09-06 at 10:58 -0400, Leonid Grossman wrote: > IMHO, in addition to current Intel and Neterion NICs, some/most upcoming > NICs are likely to be multiqueue, since virtualization emerges as a > major driver for hw designs (there are other things of course that drive > hw, but these are complimentary to multiqueue). > > PCI-SIG IOV extensions for pci spec are almost done, and a typical NIC > (at least, typical 10GbE NIC that supports some subset of IOV) in the > near future is likely to have at least 8 independent channels with its > own tx/rx queue, MAC address, msi-x vector(s), reset that doesn't affect > other channels, etc.
Leonid - any relation between that and data center ethernet? i.e http://www.ieee802.org/3/ar/public/0503/wadekar_1_0503.pdf It seems to desire to do virtualization as well. Is there any open spec for PCI-SIG IOV? > Basically, each channel could be used as an independent NIC that just > happens to share pci bus and 10GbE PHY with other channels (but has > per-channel QoS and throughput guarantees). Sounds very similar to data centre ethernet - except data centre ethernet seems to map "channels" to rings; whereas the scheme you describe maps a channel essentially to a virtual nic which seems to read in the common case as a single tx, single rx ring. Is that right? If yes, we should be able to do the virtual nics today without any changes really since each one appears as a separate NIC. It will be a matter of probably boot time partitioning and parametrization to create virtual nics (ex of priorities of each virtual NIC etc). > In a non-virtualized system, such NICs could be used in a mode when each > channel runs on one core; this may eliminate some locking... This mode > will require btw deterministic session steering, current hashing > approach in the patch is not sufficient; this is something we can > contribute once Peter's code is in. I can actually see how the PCI-SIG approach using virtual NIC approach could run on multiple CPUs (since each is no different from a NIC that we have today). And our current Linux steering would also work just fine. In the case of non-virtual NICs, i am afraid i dont think it is as easy as simple session steering - if you want to be generic that is; you may wanna consider a more complex connection tracking i.e a grouping of sessions as the basis for steering to a tx ring (and therefore tying to a specific CPU). If you are an ISP or a data center with customers partitioned based on simple subnets, then i can see a simple classification based on subnets being tied to a hw ring/CPU. And in such cases simple flow control on a per ring basis makes sense. Have you guys experimented on the the non-virtual case? And are you doing the virtual case as a pair of tx/rx being a single virtual nic? > In general, a consensus on kernel support for multiqueue NICs will be > beneficial since multiqueue HW is here and other stacks already taking > advantage of it. My main contention with the Peters approach has been to do with the propagating of flow control back to the qdisc queues. However, if this PCI SIG standard is also desiring such an approach then it will shed a different light. cheers, jamal - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
