Thanks everyone, for your comments. I now see (never used it nor looked at it before) that lagg/lacp pins traffic for a TCP connection to an interface (not infallibly for Peter).
I also now learned a little bit w.r.t. multiple queues for fast NICs. Sounds like the "nconnect" option is worth implementing. Thanks for your input, rick ________________________________________ From: Peter Eriksson <[email protected]> Sent: Tuesday, June 29, 2021 5:11 AM To: Rick Macklem Cc: freebsd-net Subject: Re: RFC: NFS trunking (multiple TCP connections for a mount CAUTION: This email originated from outside of the University of Guelph. Do not click links or open attachments unless you recognize the sender and know the content is safe. If in doubt, forward suspicious emails to [email protected] > I don't understand how multiple TCP connections to the same > server IP address will distribute the load across multiple network > interfaces? > I thought that lagg would have handled this? A lagg typically keeps all data in a TCP stream on a specific lagg member (depending on how the lagg is set up, unless you select the “roundrobin” option in freebsd - don’t do that unless you like out-of-order packets…) Network equipment with laggs typically hash the IP streams over the lagg members based on MAC addresses (source&target), IP addresses (source&target) and port numbers. (We have been diagnosing a fun problem locally where we see packet losses/performance drops over our internal backbone network for certain combinations of odd/even IP addresses/port numbers when things pass certain SPB “routers” (which typically hash the streams over many “channels” between routers)… Fun fun. :-) I think the multiple NFS TCP streams could make for some nice performance improvements in certain cases. And it would be a more generalisation of having multiple streams between two hosts - one-or-many over IPv4 and one-or-many over IPv6 at the same time. Windows SMB has a similar feature. Just avoid the Linux NFS mounting deadlock issue with “down” servers please :-) - Peter
