Re: TFO for NFS
Scheffenegger, Richard wrote: >I know, NFS TCP sessions are some of the most long-lived sessions in regular >use. Ok, so I'll admit I can't wrap my head around this. It is way out of my area of expertise (so I've added freebsd-net@ to the cc), but it seems to me that NFS is the about the least appropriate use fot TFO. It seems that, for TFO to be useful, the application needs to be doing frequent short lived TCP connections and often across WAN/Internet. NFS mounts do neither of the above. - They, as we've noted, only normally do a TCP connect at mount time. Usually run on low latency LAN environments. (High latency connections hammer NFS performance, due to its frequent small RPCs that the client must wait for replies to sychronously.) All you might save is one RTT. Take a look at how many RPCs (each with a RTT) happen on an active NFS mount. >My rationale is two-fold: > >First, having a relatively high-profile use of the TFO option in the core OS >modules >will definitely expose that feature to at least some use. Well, I don't think it is NFS's job to expose a feature that is not useful for it. (If you were to implement this and benchmarking showed a significant improvement in elapsed time to do an NFS mount, then that could be a different story.) >Second, in case of a network disconnect (or, something with my company does, >>that would be most comparable to unassigning and reassigning the server IP >>address between different physical ports), while there is IO load, TFO may >reduce >(ever so slightly) the latency impact of the enqueued IOs. I'm not sure I understand this. NFS always uses port# 2049. If you are referring to the host IP address, then wouldn't that be handled via. Arp and routing? (Does this require a fresh TCP connection to the same server IP address?) >My plan is first to simply enable the socket option - that should result in >TFO to >get negotiated for, but no actual latency improvement, while the >traditional >connect() sequence to set up a TCP session is done., from the >client side; the >server side will not need to change, and can send out >initial data right away with >the syn/ack (at least in theory, if the syn >contained a full NFS request that can be >responded to). > >Changing the client to make use of the SYN+data facilities would be a 2nd step. Well, during an NFS mount, there is first a TCP connection made by mount_nfs im userspace and it is only used for a single Null RPC. --> This checks that the server is up and running. Then mount_nfs does nmount(2), which will create a second TCP connection which is normally used until unmount. --> All you save is the RTT for the one first RPC of many. >Also, I shall make this a configurable, since some network devices may inhibit >TFO >packets, incurring a delay (but that's mostly public internet, not >private networks >where NFS is being used). Ideally with TFO default to on >(once it's working >properly), but able to explicitly disable it for certain >mounts. NFS suffered TSO related bugs for several (> 5) years (and I wouldn't be surprised if there are still net device drivers broken such that TSO must be disabled to make NFS work ok on them. As such, I get very nervous about this kind of thing. Reliability always trumps performance when it comes to file system work. Now, if you are interested in improving NFS performance over TCP, that could be a very interesting project, but I doubt TFO would be relevant. Especially when you look at long fat pipes (TCP connections with a large delay * bandwidth), there is probably a lot that could be done. --> Read-ahead, write-back algorithm changes. Read/Write data size. Throttling/congestion avoidance/window sizing in TCP. And the list goes on and on... I do hope that NFS over TLS allows more use of NFS across the Internet, so performance work related to NFS running on WAN/Internet connections would be a great thing to do. (I'm not conversant with the current TCP stack, so I'm not the guy to tackle this.) rick Richard Scheffenegger -Original Message- From: Rick Macklem Sent: Freitag, 28. August 2020 04:35 To: Scheffenegger, Richard ; rmack...@freebsd.org Cc: Michael Tuexen Subject: Re: TFO for NFS NetApp Security WARNING: This is an external email. Do not click links or open attachments unless you recognize the sender and know the content is safe. Well, you'll find the soconnect() stuff in sys/rpc/clnt_vc.c. If you just want to play around with it, have fun. As for this being useful in practice, that seems unlikely. When the kernel RPC code uses TCP it establishes one TCP connection at mount time and uses that connection until unmount unless the connection breaks somehow. (A server will often disconnect after about 5 minutes of no activity on the connection. This almost never happens for NFSv4, since the NFSv4 client does an RPC every 30sec to maintain the lease against the server.) --> A new TCP connection
[Bug 246885] if_igb: Loader tunables/LEDs gone after iflib conversion?
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246885 Jose Luis Duran changed: What|Removed |Added Attachment #217603|0 |1 is obsolete|| --- Comment #7 from Jose Luis Duran --- Created attachment 217604 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=217604=edit Remove LED-related sections Sorry, I was reading the on-line version of the man page and got confused. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 246885] if_igb: Loader tunables/LEDs gone after iflib conversion?
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246885 --- Comment #6 from Jose Luis Duran --- (In reply to Jose Luis Durhttps://bugs.freebsd.org/bugzilla/an from comment #5) Wait... I think I have to create a separate igb.4/if_igb.4 man page (revert base r350075), because if_em do have these tunables. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 246885] if_igb: Loader tunables/LEDs gone after iflib conversion?
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246885 --- Comment #5 from Jose Luis Duran --- Created attachment 217603 --> https://bugs.freebsd.org/bugzilla/attachment.cgi?id=217603=edit em.4: Remove non-applicable sections Update manual to reflect current state. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 246003] em(4) Intel I219-V6 on NUC8i5BEH randomly loses carrier or fails over to 100Mbit
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246003 --- Comment #2 from Joshua Kinard --- (In reply to Kevin Bowling from comment #1) > Can you test stable/12 snapshots? Not easily. The system is currently running 12.1-RELEASE-p8. I can, however, test patches if you have a specific commit that may address the issue and works w/ 12.1-RELEASE (or mostly applies to the source). Currently using the out-of-tree intel-em-kmod driver on this platform, version 7.7.8 from Intel's download center. The latest version in the ports tree is 7.7.5 and that one also exhibited issues w/ jumbo frames. I ended up giving up and went back to mtu 1500, and have not attempted jumbo frames on 7.7.8 yet. -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 246885] if_igb: Loader tunables/LEDs gone after iflib conversion?
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246885 Kevin Bowling changed: What|Removed |Added CC||kbowl...@freebsd.org --- Comment #4 from Kevin Bowling --- (In reply to Jose Luis Duran from comment #3) yes, can you submit a patch? -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 246003] em(4) Intel I219-V6 on NUC8i5BEH randomly loses carrier or fails over to 100Mbit
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=246003 Kevin Bowling changed: What|Removed |Added CC||kbowl...@freebsd.org --- Comment #1 from Kevin Bowling --- Can you test stable/12 snapshots? -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 248652] netmap: pkt-gen TX huge pps difference between 11-STABLE and 12-STABLE/CURRENT on ix & ixl NIC
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248652 --- Comment #8 from Sylvain Galliano --- After looking at iflib_netmap_timer_adjust() & iflib_netmap_txsync() in sys/net/iflib.c, I made some tuning on kern.hz: Still using X520 with 1 queue ix0: PCI Express Bus: Speed 5.0GT/s Width x8 ix0: netmap queues/slots: TX 1/2048, RX 1/2048 * /boot/loader.conf: kern.hz=1000 (default) pkt-gen result: 204.153802 main_thread [2639] 2.562 Mpps (2.567 Mpkts 1.230 Gbps in 1001994 usec) 465.32 avg_batch 9 min_space 205.155321 main_thread [2639] 2.561 Mpps (2.565 Mpkts 1.229 Gbps in 1001519 usec) 465.45 avg_batch 9 min_space 5500 irq/s: * /boot/loader.conf: kern.hz=1999 pkt-gen result: 41.375049 main_thread [2639] 5.117 Mpps (5.222 Mpkts 2.456 Gbps in 1020510 usec) 465.45 avg_batch 9 min_space 42.375546 main_thread [2639] 5.118 Mpps (5.121 Mpkts 2.457 Gbps in 1000497 usec) 465.42 avg_batch 9 min_space 11000 irq/s X2 performance & irq/s * /boot/loader.conf: kern.hz=2000 pkt-gen result: 797.608080 main_thread [2639] 2.560 Mpps (2.563 Mpkts 1.229 Gbps in 1001001 usec) 465.50 avg_batch 9 min_space 798.609079 main_thread [2639] 2.560 Mpps (2.563 Mpkts 1.229 Gbps in 1000999 usec) 465.41 avg_batch 9 min_space 5500 irq/s Same performance & irq/s as kern.hz=1000 (due to limit at 2000 in iflib_netmap_timer_adjust & iflib_netmap_txsync) * Last test, this one I forced 'ticks' parameter to '1' in callout_reset_on on iflib_netmap_timer_adjust & iflib_netmap_txsync by increasing the 2000 limit to 2 in both functions and put an insame value for kern.hz /boot/loader.conf: kern.hz=1 pkt-gen result: 345.415939 main_thread [2639] 14.880 Mpps (14.890 Mpkts 7.142 Gbps in 1000699 usec) 430.97 avg_batch 9 min_space 346.429134 main_thread [2639] 14.880 Mpps (15.076 Mpkts 7.142 Gbps in 1013196 usec) 432.17 avg_batch 9 min_space 29000 irq/s Same performance as FreeBSD 11 Looks like callout_reset_on to iflib_timer have a look high delay. -- You are receiving this mail because: You are the assignee for the bug. You are on the CC list for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
How change RSS Hash function for igb ?
Hi, I need to change RSS Hash function to 2-tuple for igb driver. How can I do it ? I didn't see any sysctl. If there is any way even by modifying driver code I want to try. Regards ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 248958] pptpd + vlan panic
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248958 Mark Linimon changed: What|Removed |Added Assignee|b...@freebsd.org|n...@freebsd.org -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
[Bug 248958] pptpd + vlan panic
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=248958 Mark Linimon changed: What|Removed |Added Keywords||panic -- You are receiving this mail because: You are the assignee for the bug. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
CARP over VLAN over LAGG
Hello, I have a "highly available" router/firewall with the following configuration (1). Those are plugged in two 2930F (with VSF) using LACP. It works well, except that I have some weird issues with the CARP demotion counter when I'm unplugging some interfaces involved in the lagg/carp setup, for example if I unplug/replug igb0 and igb1 in this case: (dmesg): igb0: link state changed to DOWN igb1: link state changed to DOWN carp: demoted by 240 to 240 (send error 50 on vlan11) carp: 11@vlan11: MASTER -> BACKUP (more frequent advertisement received) vlan11: deletion failed: 3 igb1: link state changed to UP igb0: link state changed to UP then the CARP status stays to BACKUP unless I demote the CARP demotion counter manually with: sudo sysctl net.inet.carp.demotion=-240: (dmesg): carp: demoted by -240 to 0 (sysctl) carp: 11@vlan11: BACKUP -> MASTER (preempting a slower master) I guess this is because it takes some time for lagg/lacp to converge and thus carp thinks that there is a problematic condition as it experiences problems with sending announcements.. What it the best way to handle this? Thanks, Julien (1) https://gist.github.com/silenius/577606b596ff1d220bbfd9956d05baef -- Julien Cigar Belgian Biodiversity Platform (http://www.biodiversity.be) PGP fingerprint: EEF9 F697 4B68 D275 7B11 6A25 B2BB 3710 A204 23C0 No trees were killed in the creation of this message. However, many electrons were terribly inconvenienced. ___ freebsd-net@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "freebsd-net-unsubscr...@freebsd.org"
