https://bugs.dpdk.org/show_bug.cgi?id=791
Bug ID: 791
Summary: Broadcom N2100G NIC does not reset after use
Product: DPDK
Version: 21.08
Hardware: x86
OS: Linux
Status: UNCONFIRMED
Severity: normal
Priority: Normal
Component: core
Assignee: dev@dpdk.org
Reporter: c...@hpe.com
Target Milestone: ---
The Broadcom N2100G NIC (2x100G OCP 3.0 version) can only be used once after a
reboot. It appears that when testpmd shuts down, it leaves the NIC in a state
that subsequent uses show it to be in use. A reboot clears this condition.
To duplicate:
Bind the vfio-pci driver to the NIC
Run a test using testpmd (interactive or non-interactive - I was using
non-interactive)
Stop the test
Run the test a second time
Error -22 will occur when trying to open the ports
Output from test:
Configuring Port 0 (socket 0)
bnxt_hwrm_port_phy_cfg(): error 2:0:00000000:0000
bnxt_set_hwrm_link_config(): Set link config failed with rc -22
bnxt_update_phy_setting(): Failed to update PHY settings
Fail to start port 0: Invalid argument
Configuring Port 1 (socket 0)
bnxt_hwrm_port_phy_cfg(): error 2:0:00000000:0000
bnxt_set_hwrm_link_config(): Set link config failed with rc -22
bnxt_update_phy_setting(): Failed to update PHY settings
Fail to start port 1: Invalid argument
Please stop the ports first
Done
No commandline core given, start packet forwarding
Not all ports were started
I have tried unbinding vfio-pci and binding bnxt_en.
I have also tried stopping the ports from testpmd in interactive mode.
Neither clears the condition, only a reboot does.
Comments from previous patches indicate the OCP card can only be modified by
the BMC.
This is not an issue with the P2100G (standup PCIe NIC)
This has been verified on DPDK 21.02, 21.05, and 21.08
Test system is an HPE DL385 dual AMD Milan processors
F/W on the NIC is current (218.0.219.21)
lspci output for port 0:
43:00.0 Ethernet controller: Broadcom Inc. and subsidiaries BCM57508
NetXtreme-E 10Gb/25Gb/40Gb/50Gb/100Gb/200Gb Ethernet (rev 11)
Subsystem: Broadcom Inc. and subsidiaries NetXtreme-E Dual-port 100G
QSFP56 Ethernet OCP 3.0 Adapter (BCM957508-N2100G)
Physical Slot: 10
Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr+
Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort-
<MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 179
NUMA node: 0
Region 0: Memory at ae210000 (64-bit, prefetchable) [size=64K]
Region 2: Memory at ae100000 (64-bit, prefetchable) [size=1M]
Region 4: Memory at ae228000 (64-bit, prefetchable) [size=32K]
Expansion ROM at ae500000 [virtual] [disabled] [size=256K]
Capabilities: [48] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA
PME(D0+,D1-,D2-,D3hot+,D3cold+)
Status: D3 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME-
Capabilities: [50] Vital Product Data
Product Name: Broadcom NetXtreme E-Series Dual-port 100Gb OCP
3.0 Ethernet Adapter
Read-only fields:
[PN] Part number: BCM957508-N2100G
[MN] Manufacture ID: 14E4
[V0] Vendor specific: 218.0.219.21
[V1] Vendor specific: 218.0.219.7
[V3] Vendor specific: 218.0.219.13
[V6] Vendor specific: 218.0.219.1
[V7] Vendor specific: 218.0.219.2
[V9] Vendor specific: 218.0.3.0
[SN] Serial number: N2100200300053CQ
[RV] Reserved: checksum good, 112 byte(s) reserved
End
Capabilities: [58] MSI: Enable- Count=1/8 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Capabilities: [a0] MSI-X: Enable- Count=74 Masked-
Vector table: BAR=4 offset=00000000
PBA: BAR=4 offset=00004000
Capabilities: [ac] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 512 bytes, PhantFunc 0, Latency L0s <4us, L1
<64us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+
SlotPowerLimit 75.000W
DevCtl: CorrErr- NonFatalErr+ FatalErr+ UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr+ NoSnoop+ FLReset-
MaxPayload 512 bytes, MaxReadReq 4096 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr+
TransPend-
LnkCap: Port #0, Speed 16GT/s, Width x16, ASPM not supported
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk+
ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 16GT/s (ok), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range ABCD, TimeoutDis+,
NROPrPrP-, LTR+
10BitTagComp+, 10BitTagReq+, OBFF Via WAKE#, ExtFmt-,
EETLPPrefix-
EmergencyPowerReduction Not Supported,
EmergencyPowerReductionInit-
FRS-, TPHComp-, ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR+,
OBFF Disabled
AtomicOpsCtl: ReqEn-
LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range,
EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB,
EqualizationComplete+, EqualizationPhase1+
EqualizationPhase2+, EqualizationPhase3+,
LinkEqualizationRequest-
Capabilities: [100 v1] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt-
RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt-
RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP- SDES+ TLP+ FCP- CmpltTO- CmpltAbrt- UnxCmplt-
RxOF- MalfTLP- ECRC+ UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout-
AdvNonFatalErr+
CEMsk: RxErr- BadTLP+ BadDLLP+ Rollover+ Timeout+
AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap+ ECRCGenEn+
ECRCChkCap+ ECRCChkEn+
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [13c v1] Device Serial Number bc-97-e1-ff-fe-46-75-34
Capabilities: [150 v1] Power Budgeting <?>
Capabilities: [160 v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=ff
Status: NegoPending- InProgress-
Capabilities: [180 v1] Vendor Specific Information: ID=0000 Rev=0
Len=020 <?>
Capabilities: [1b0 v1] Latency Tolerance Reporting
Max snoop latency: 1048576ns
Max no snoop latency: 1048576ns
Capabilities: [1b8 v1] Alternative Routing-ID Interpretation (ARI)
ARICap: MFVC- ACS-, Next Function: 1
ARICtl: MFVC- ACS-, Function Group: 0
Capabilities: [1c0 v1] Single Root I/O Virtualization (SR-IOV)
IOVCap: Migration-, Interrupt Message Number: 000
IOVCtl: Enable- Migration- Interrupt- MSE- ARIHierarchy+
IOVSta: Migration-
Initial VFs: 8, Total VFs: 8, Number of VFs: 0, Function
Dependency Link: 00
VF offset: 256, stride: 1, Device ID: 1806
Supported Page Size: 000005ff, System Page Size: 00000001
Region 0: Memory at 00000000ae2b0000 (64-bit, prefetchable)
Region 2: Memory at 0000000000000000 (64-bit, prefetchable)
Region 4: Memory at 00000000ae230000 (64-bit, prefetchable)
VF Migration: offset: 00000000, BIR: 0
Capabilities: [230 v1] Transaction Processing Hints
Interrupt vector mode supported
Device specific mode supported
Steering table in MSI-X table
Capabilities: [300 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn-, PerformEqu-
LaneErrStat: 0
Capabilities: [200 v1] Precision Time Measurement
PTMCap: Requester:+ Responder:- Root:-
PTMClockGranularity: Unimplemented
PTMControl: Enabled:- RootSelected:-
PTMEffectiveGranularity: Unknown
Capabilities: [358 v1] Physical Layer 16.0 GT/s <?>
Capabilities: [388 v1] Lane Margining at the Receiver <?>
Kernel driver in use: vfio-pci
Kernel modules: bnxt_en
--
You are receiving this mail because:
You are the assignee for the bug.
