I’m now posting the sfxge driver for code review. I don’t have webrev/review board (yet), but here’s a pull-request that you can comment on, or merge into a private tree (yes I know illumos doesn’t merge PRs.):
https://github.com/illumos/illumos-gate/pull/16 So this is the sfxge driver which supports modern Solarflare NICs, including 10 and 40 GbE. I’ve only tested 10GbE using the Solarflare Flareon Ultra 7000 Series. My testing is that this driver, when tuned properly, gives significantly lower latency than the ixgbe driver. I’ve not benchmarked for throughput, or CPU utilization. In my world latency is king. I have however put rather severe load in terms of packets per second on the NIC… I’ve run stress runs with TCP where I push around 1M pkts/sec (TCP messages at 200 bytes per message with Nagle disabled) across this NIC; at that rate the TCP stack and the soft rings from GLDv3 become concerns, although I ran out of test client capacity before I saturated the capabilities of this NIC. (My testing involved an in-kernel TCP proxy that I developed for our SDN at Lucera.) Round trip latencies with this driver and combination are typically several usec (or more!) better than I could achieve with ixgbe. The driver lacks support for Crossbow virtualization, which means it uses soft rings. Even with this deficiency, it out performs the ixgbe NIC in my configuration. (I will work on addressing Crossbow support with Solarflare in a future update.) SFC9300 NICs are not enabled, but this can be done with a compile time option; if anyone has such NICs I’d be grateful for help testing. SFC 9200 work fine. (I’m not sure SFC 9300 are actually available for purchase yet.) The older SFC 4000 family is *not* supported — the support for this was present in earlier versions of the code, but Solarflare have removed the code from their common code base, and so such NICs should not be used. This driver was contributed directly for illumos integration by Solarflare, and their engineering staff have been enormously helpful in this effort. Please join me in thanking Solarflare. The driver has some illumos-specific improvements (mostly cleaning up unused Solaris 10 support for legacy NDD, some other minor fixes and improvements, man page, packaging, etc.) that were performed by me, as well as lint cleanups. Note that the driver was already in very good shape, and is filled with rich dtrace probes, etc. If you want to see the gory details of the changes I made, have a look at my github branch here: https://github.com/gdamore/illumos-core/tree/sfxge We are using this driver and NIC at Lucera. That said, there may be sharp edges still, and I’ve not yet run the entire thing through NICDRV. (Mostly because setting up NICDRV is a major PITA.) Again, if you can help with testing (especially with 40GbE!), I’d be grateful. And review feedback is most welcome as well! - Garrett ------------------------------------------- illumos-discuss Archives: https://www.listbox.com/member/archive/182180/=now RSS Feed: https://www.listbox.com/member/archive/rss/182180/21175430-2e6923be Modify Your Subscription: https://www.listbox.com/member/?member_id=21175430&id_secret=21175430-6a77cda4 Powered by Listbox: http://www.listbox.com
