Fortville (700) has always been a bit of a disaster (https://cdrdv2.intel.com/v1/dl/getContent/331430?explicitVersion=true), I'd see if you can press your Intel reps into getting you the 550s or the 800-series NICs for the unnecessary troubles it's a much nicer design.
It's surprising they are shipping new cards with that old of a firmware, you should be on 8.50 for the driver you are running (https://www.intel.com/content/www/us/en/download/18635/non-volatile-memory-nvm-update-utility-for-intel-ethernet-adapters-700-series-linux.html). Doing the FW update is worth a shot but most issues I've seen have been driver related and you are running a pretty recent driver. Regards, Kevin On Mon, Mar 14, 2022 at 7:44 AM Matthew Weiner <mlwei...@lakelandschools.org> wrote: > > I'm at my wits end with this, Citrix is stumped, Dell is stumped, and with > the supply chain issues the way they are we can't just yank these out in > favor of X550s. The problem is we have a group of Dell R740s with X710 > dual-port NICs and the performance is, in a word, awful. Like 5-6 megabit > upload and 250 megabit download awful. However, identical server hardware > with any other card, be it a Broadcom or an Intel X550T, no issues. We can > get line rate all day long. The latest attempt was swapping the X710 for a > newer X710-T2L-t, which performed maybe 5-10 percent better. We've tried > three different driver revisions, firmware, BIOS, all the available > Hypervisor updates, it still performs the same. > > The servers in question have X550s on the motherboard mezzanine card which > perform fine, and a single dual-port X710 in the PCIe riser. The X710 is > set up with an LACP pair trunked with three VLANs tagged across it. In > this pool we also have servers with X550s on the PCIe cards, and > Broadcoms. All those with an identical configuration perform without > issue, it's only the X710s that show this problem. > > LSPCI info for the cards (if you want the full output please let me know - > it's extremely long so I clipped it) > > af:00.0 0200: 8086:15ff (rev 02) > Subsystem: 8086:0005 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 32 bytes > Interrupt: pin A routed to IRQ 88 > Region 0: Memory at e2000000 (64-bit, prefetchable) [size=16M] > Region 3: Memory at e3008000 (64-bit, prefetchable) [size=32K] > Expansion ROM at e3200000 [disabled] [size=512K] > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA > PME(D0+,D1-,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME- > Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ > Address: 0000000000000000 Data: 0000 > Masking: 00000000 Pending: 00000000 > Capabilities: [70] MSI-X: Enable+ Count=129 Masked- > Vector table: BAR=3 offset=00000000 > PBA: BAR=3 offset=00001000 > Capabilities: [a0] Express (v2) Endpoint, MSI 00 > DevCap: MaxPayload 2048 bytes, PhantFunc 0, Latency L0s > <512ns, L1 <64us > ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ > SlotPowerLimit 0.000W > DevCtl: Report errors: Correctable- Non-Fatal+ Fatal+ > Unsupported+ > RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- > FLReset- > MaxPayload 256 bytes, MaxReadReq 512 bytes > DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- > TransPend- > LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM L1, Exit > Latency L0s <2us, L1 <16us > ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+ > LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 8GT/s, Width x8, TrErr- Train- SlotClk+ > DLActive- BWMgmt- ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, > OBFF Not Supported > DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-, > LTR-, OBFF Disabled > LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- > SpeedDis- > Transmit Margin: Normal Operating Range, > EnterModifiedCompliance- ComplianceSOS- > Compliance De-emphasis: -6dB > LnkSta2: Current De-emphasis Level: -3.5dB, > EqualizationComplete+, EqualizationPhase1+ > EqualizationPhase2+, EqualizationPhase3+, > LinkEqualizationRequest- > Capabilities: [e0] Vital Product Data > Product Name: X710 10GbE Controller > Read-only fields: > [V0] Vendor specific: FFV19.5.0 > [PN] Part number: 1WYFT > [MN] Manufacture ID: 31 30 32 38 > [V1] Vendor specific: DSV1028VPDR.VER2.1 > [V3] Vendor specific: DTINIC > [V4] Vendor specific: DCM1001FFFFFF2101FFFFFF > [V5] Vendor specific: NPY2 > [V6] Vendor specific: PMT1 > [V7] Vendor specific: NMVIntel Corp > [V8] Vendor specific: L1D0 > [RV] Reserved: checksum good, 1 byte(s) reserved > Read/write fields: > [Y1] System specific: CCF1 > End > Capabilities: [100 v2] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- > RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ > RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UESvrt: DLP+ SDES+ TLP+ FCP+ CmpltTO+ CmpltAbrt+ UnxCmplt- > RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol- > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > NonFatalErr+ > CEMsk: RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ > NonFatalErr+ > AERCap: First Error Pointer: 00, GenCap+ CGenEn+ ChkCap+ > ChkEn+ > Capabilities: [140 v1] Device Serial Number aa-a4-d5-ff-ff-91-96-b4 > Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) > ARICap: MFVC- ACS-, Next Function: 1 > ARICtl: MFVC- ACS-, Function Group: 0 > Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV) > IOVCap: Migration-, Interrupt Message Number: 000 > IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+ > IOVSta: Migration- > Initial VFs: 64, Total VFs: 64, Number of VFs: 0, Function > Dependency Link: 00 > VF offset: 16, stride: 1, Device ID: 154c > Supported Page Size: 00000553, System Page Size: 00000001 > Region 0: Memory at 00000000e3300000 (64-bit, prefetchable) > Region 3: Memory at 00000000e3b00000 (64-bit, prefetchable) > VF Migration: offset: 00000000, BIR: 0 > Capabilities: [1a0 v1] Transaction Processing Hints > Device specific mode supported > No steering table available > Capabilities: [1b0 v1] Access Control Services > ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- > UpstreamFwd- EgressCtrl- DirectTrans- > ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- > UpstreamFwd- EgressCtrl- DirectTrans- > Capabilities: [1d0 v1] #19 > Kernel driver in use: i40e > Kernel modules: i40e > > af:00.1 0200: 8086:15ff (rev 02) > Subsystem: 8086:0000 > Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- > Stepping- SERR- FastB2B- DisINTx+ > Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- > <TAbort- <MAbort- >SERR- <PERR- INTx- > Latency: 0, Cache Line Size: 32 bytes > Interrupt: pin A routed to IRQ 88 > Region 0: Memory at e1000000 (64-bit, prefetchable) [size=16M] > Region 3: Memory at e3000000 (64-bit, prefetchable) [size=32K] > Expansion ROM at e3280000 [disabled] [size=512K] > Capabilities: [40] Power Management version 3 > Flags: PMEClk- DSI+ D1- D2- AuxCurrent=0mA > PME(D0+,D1-,D2-,D3hot+,D3cold+) > Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME- > Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+ > Address: 0000000000000000 Data: 0000 > Masking: 00000000 Pending: 00000000 > Capabilities: [70] MSI-X: Enable+ Count=129 Masked- > Vector table: BAR=3 offset=00000000 > PBA: BAR=3 offset=00001000 > Capabilities: [a0] Express (v2) Endpoint, MSI 00 > DevCap: MaxPayload 2048 bytes, PhantFunc 0, Latency L0s > <512ns, L1 <64us > ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ > SlotPowerLimit 0.000W > DevCtl: Report errors: Correctable- Non-Fatal+ Fatal+ > Unsupported+ > RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop- > FLReset- > MaxPayload 256 bytes, MaxReadReq 512 bytes > DevSta: CorrErr+ UncorrErr- FatalErr- UnsuppReq+ AuxPwr- > TransPend- > LnkCap: Port #0, Speed 8GT/s, Width x8, ASPM L1, Exit > Latency L0s <2us, L1 <16us > ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+ > LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+ > ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- > LnkSta: Speed 8GT/s, Width x8, TrErr- Train- SlotClk+ > DLActive- BWMgmt- ABWMgmt- > DevCap2: Completion Timeout: Range ABCD, TimeoutDis+, LTR-, > OBFF Not Supported > DevCtl2: Completion Timeout: 65ms to 210ms, TimeoutDis-, > LTR-, OBFF Disabled > LnkSta2: Current De-emphasis Level: -3.5dB, > EqualizationComplete-, EqualizationPhase1- > EqualizationPhase2-, EqualizationPhase3-, > LinkEqualizationRequest- > Capabilities: [e0] Vital Product Data > Product Name: X710 10GbE Controller > Read-only fields: > [V0] Vendor specific: FFV19.5.0 > [PN] Part number: 1WYFT > [MN] Manufacture ID: 31 30 32 38 > [V1] Vendor specific: DSV1028VPDR.VER2.1 > [V3] Vendor specific: DTINIC > [V4] Vendor specific: DCM1001FFFFFF2101FFFFFF > [V5] Vendor specific: NPY2 > [V6] Vendor specific: PMT1 > [V7] Vendor specific: NMVIntel Corp > [V8] Vendor specific: L1D0 > [RV] Reserved: checksum good, 1 byte(s) reserved > Read/write fields: > [Y1] System specific: CCF1 > End > Capabilities: [100 v2] Advanced Error Reporting > UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- > RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt+ UnxCmplt+ > RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- > UESvrt: DLP+ SDES+ TLP+ FCP+ CmpltTO+ CmpltAbrt+ UnxCmplt- > RxOF+ MalfTLP+ ECRC+ UnsupReq- ACSViol- > CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- > NonFatalErr+ > CEMsk: RxErr+ BadTLP+ BadDLLP+ Rollover+ Timeout+ > NonFatalErr+ > AERCap: First Error Pointer: 00, GenCap+ CGenEn+ ChkCap+ > ChkEn+ > Capabilities: [140 v1] Device Serial Number aa-a4-d5-ff-ff-91-96-b4 > Capabilities: [150 v1] Alternative Routing-ID Interpretation (ARI) > ARICap: MFVC- ACS-, Next Function: 0 > ARICtl: MFVC- ACS-, Function Group: 0 > Capabilities: [160 v1] Single Root I/O Virtualization (SR-IOV) > IOVCap: Migration-, Interrupt Message Number: 000 > IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy- > IOVSta: Migration- > Initial VFs: 64, Total VFs: 64, Number of VFs: 0, Function > Dependency Link: 01 > VF offset: 79, stride: 1, Device ID: 154c > Supported Page Size: 00000553, System Page Size: 00000001 > Region 0: Memory at 00000000e3700000 (64-bit, prefetchable) > Region 3: Memory at 00000000e3c00000 (64-bit, prefetchable) > VF Migration: offset: 00000000, BIR: 0 > Capabilities: [1a0 v1] Transaction Processing Hints > Device specific mode supported > No steering table available > Capabilities: [1b0 v1] Access Control Services > ACSCap: SrcValid- TransBlk- ReqRedir- CmpltRedir- > UpstreamFwd- EgressCtrl- DirectTrans- > ACSCtl: SrcValid- TransBlk- ReqRedir- CmpltRedir- > UpstreamFwd- EgressCtrl- DirectTrans- > Kernel driver in use: i40e > Kernel modules: i40e > > ethtool -k output > Features for eth4: > rx-checksumming: on > tx-checksumming: on > tx-checksum-ipv4: on > tx-checksum-ip-generic: off [fixed] > tx-checksum-ipv6: on > tx-checksum-fcoe-crc: off [fixed] > tx-checksum-sctp: on > scatter-gather: on > tx-scatter-gather: on > tx-scatter-gather-fraglist: off [fixed] > tcp-segmentation-offload: on > tx-tcp-segmentation: on > tx-tcp-ecn-segmentation: on > tx-tcp-mangleid-segmentation: off > tx-tcp6-segmentation: on > udp-fragmentation-offload: off > generic-segmentation-offload: on > generic-receive-offload: on > large-receive-offload: off [fixed] > rx-vlan-offload: on > tx-vlan-offload: on > ntuple-filters: on > receive-hashing: on > highdma: on > rx-vlan-filter: on [fixed] > vlan-challenged: off [fixed] > tx-lockless: off [fixed] > netns-local: off [fixed] > tx-gso-robust: off [fixed] > tx-fcoe-segmentation: off [fixed] > tx-gre-segmentation: on > tx-gre-csum-segmentation: on > tx-ipxip4-segmentation: on > tx-ipxip6-segmentation: on > tx-udp_tnl-segmentation: on > tx-udp_tnl-csum-segmentation: on > tx-gso-partial: on > tx-sctp-segmentation: off [fixed] > tx-esp-segmentation: off [fixed] > tx-udp-segmentation: off [fixed] > fcoe-mtu: off [fixed] > tx-nocache-copy: off > loopback: off [fixed] > rx-fcs: off [fixed] > rx-all: off [fixed] > tx-vlan-stag-hw-insert: off [fixed] > rx-vlan-stag-hw-parse: off [fixed] > rx-vlan-stag-filter: on [fixed] > l2-fwd-offload: off [fixed] > hw-tc-offload: on > esp-hw-offload: off [fixed] > esp-tx-csum-hw-offload: off [fixed] > rx-udp_tunnel-port-offload: on > tls-hw-tx-offload: off [fixed] > tls-hw-rx-offload: off [fixed] > rx-gro-hw: off [fixed] > tls-hw-record: off [fixed] > > ethtool -i output > driver: i40e > version: 2.17.4 > firmware-version: 7.10 0x800075d6 19.5.9 > expansion-rom-version: > bus-info: 0000:af:00.0 > supports-statistics: yes > supports-test: yes > supports-eeprom-access: yes > supports-register-dump: yes > supports-priv-flags: yes > > This is a new card, the X710-T2L we swapped in, so I haven't flashed its > firmware yet, the others have version 20 on them with no change. > > Any pointers or assistance is greatly appreciated. Otherwise we're looking > at July for availability of X550s on a PCIe card from Dell. > > -Matt Weiner > > _______________________________________________ > E1000-devel mailing list > E1000-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/e1000-devel > To learn more about Intel Ethernet, visit > https://forums.intel.com/s/topic/0TO0P00000018NbWAI/intel-ethernet _______________________________________________ E1000-devel mailing list E1000-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/e1000-devel To learn more about Intel Ethernet, visit https://forums.intel.com/s/topic/0TO0P00000018NbWAI/intel-ethernet
