W dniu 01.11.2018 o 21:37, Saeed Mahameed pisze:
On Thu, 2018-11-01 at 12:09 +0100, Paweł Staszewski wrote:
W dniu 01.11.2018 o 10:50, Saeed Mahameed pisze:
On Wed, 2018-10-31 at 22:57 +0100, Paweł Staszewski wrote:
Hi
So maybee someone will be interested how linux kernel handles
normal
traffic (not pktgen :) )
Server HW configuration:
CPU : Intel(R) Xeon(R) Gold 6132 CPU @ 2.60GHz
NIC's: 2x 100G Mellanox ConnectX-4 (connected to x16 pcie 8GT)
Server software:
FRR - as routing daemon
enp175s0f0 (100G) - 16 vlans from upstreams (28 RSS binded to
local
numa
node)
enp175s0f1 (100G) - 343 vlans to clients (28 RSS binded to local
numa
node)
Maximum traffic that server can handle:
Bandwidth
bwm-ng v0.6.1 (probing every 1.000s), press 'h' for help
input: /proc/net/dev type: rate
\ iface Rx Tx Total
=================================================================
====
=========
enp175s0f1: 28.51 Gb/s 37.24
Gb/s
65.74 Gb/s
enp175s0f0: 38.07 Gb/s 28.44
Gb/s
66.51 Gb/s
---------------------------------------------------------------
----
-----------
total: 66.58 Gb/s 65.67
Gb/s
132.25 Gb/s
Packets per second:
bwm-ng v0.6.1 (probing every 1.000s), press 'h' for help
input: /proc/net/dev type: rate
- iface Rx Tx Total
=================================================================
====
=========
enp175s0f1: 5248589.00 P/s 3486617.75 P/s
8735207.00 P/s
enp175s0f0: 3557944.25 P/s 5232516.00 P/s
8790460.00 P/s
---------------------------------------------------------------
----
-----------
total: 8806533.00 P/s 8719134.00 P/s
17525668.00 P/s
After reaching that limits nics on the upstream side (more RX
traffic)
start to drop packets
I just dont understand that server can't handle more bandwidth
(~40Gbit/s is limit where all cpu's are 100% util) - where pps on
RX
side are increasing.
Where do you see 40 Gb/s ? you showed that both ports on the same
NIC (
same pcie link) are doing 66.58 Gb/s (RX) + 65.67 Gb/s (TX) =
132.25
Gb/s which aligns with your pcie link limit, what am i missing ?
hmm yes that was my concern also - cause cant find anywhere
informations
about that bandwidth is uni or bidirectional - so if 126Gbit for x16
8GT
is unidir - then bidir will be 126/2 ~68Gbit - which will fit total
bw
on both ports
i think it is bidir
This can explain maybee also why cpuload is rising rapidly from
120Gbit/s in total to 132Gbit (counters of bwmng are from /proc/net -
so
there can be some error in reading them when offloading (gro/gso/tso)
on
nic's is enabled that is why
Was thinking that maybee reached some pcie x16 limit - but x16
8GT
is
126Gbit - and also when testing with pktgen i can reach more bw
and
pps
(like 4x more comparing to normal internet traffic)
Are you forwarding when using pktgen as well or you just testing
the RX
side pps ?
Yes pktgen was tested on single port RX
Can check also forwarding to eliminate pciex limits
So this explains why you have more RX pps, since tx is idle and pcie
will be free to do only rx.
[...]
ethtool -S enp175s0f1
NIC statistics:
rx_packets: 173730800927
rx_bytes: 99827422751332
tx_packets: 142532009512
tx_bytes: 184633045911222
tx_tso_packets: 25989113891
tx_tso_bytes: 132933363384458
tx_tso_inner_packets: 0
tx_tso_inner_bytes: 0
tx_added_vlan_packets: 74630239613
tx_nop: 2029817748
rx_lro_packets: 0
rx_lro_bytes: 0
rx_ecn_mark: 0
rx_removed_vlan_packets: 173730800927
rx_csum_unnecessary: 0
rx_csum_none: 434357
rx_csum_complete: 173730366570
rx_csum_unnecessary_inner: 0
rx_xdp_drop: 0
rx_xdp_redirect: 0
rx_xdp_tx_xmit: 0
rx_xdp_tx_full: 0
rx_xdp_tx_err: 0
rx_xdp_tx_cqe: 0
tx_csum_none: 38260960853
tx_csum_partial: 36369278774
tx_csum_partial_inner: 0
tx_queue_stopped: 1
tx_queue_dropped: 0
tx_xmit_more: 748638099
tx_recover: 0
tx_cqes: 73881645031
tx_queue_wake: 1
tx_udp_seg_rem: 0
tx_cqe_err: 0
tx_xdp_xmit: 0
tx_xdp_full: 0
tx_xdp_err: 0
tx_xdp_cqes: 0
rx_wqe_err: 0
rx_mpwqe_filler_cqes: 0
rx_mpwqe_filler_strides: 0
rx_buff_alloc_err: 0
rx_cqe_compress_blks: 0
rx_cqe_compress_pkts: 0
If this is a pcie bottleneck it might be useful to enable CQE
compression (to reduce PCIe completion descriptors transactions)
you should see the above rx_cqe_compress_pkts increasing when
enabled.
$ ethtool --set-priv-flags enp175s0f1 rx_cqe_compress on
$ ethtool --show-priv-flags enp175s0f1
Private flags for p6p1:
rx_cqe_moder : on
cqe_moder : off
rx_cqe_compress : on
...
try this on both interfaces.
Done
ethtool --show-priv-flags enp175s0f1
Private flags for enp175s0f1:
rx_cqe_moder : on
tx_cqe_moder : off
rx_cqe_compress : on
rx_striding_rq : off
rx_no_csum_complete: off
ethtool --show-priv-flags enp175s0f0
Private flags for enp175s0f0:
rx_cqe_moder : on
tx_cqe_moder : off
rx_cqe_compress : on
rx_striding_rq : off
rx_no_csum_complete: off
did it help reduce the load on the pcie ? do you see more pps ?
what is the ratio between rx_cqe_compress_pkts and over all rx packets
?
So - a little more pps
Before change top - graph / after bottom -> image with graph stats from
proc/net/dev
cqe_compress enabled at 11:55
Sorry - but for real life traffic it is hard to do any counter
differences - cause traffic just rising alone from minute to minute :)
But for that time the change is visible on graph - cause was almost same
for past 20minutes before change.
full ethtool below:
NIC statistics:
rx_packets: 516522465438
rx_bytes: 680052911258729
tx_packets: 677697545586
tx_bytes: 413647643141709
tx_tso_packets: 42530913279
tx_tso_bytes: 235655668554142
tx_tso_inner_packets: 0
tx_tso_inner_bytes: 0
tx_added_vlan_packets: 551156530885
tx_nop: 8536823558
rx_lro_packets: 0
rx_lro_bytes: 0
rx_ecn_mark: 0
rx_removed_vlan_packets: 516522465438
rx_csum_unnecessary: 0
rx_csum_none: 50382868
rx_csum_complete: 516472082570
rx_csum_unnecessary_inner: 0
rx_xdp_drop: 0
rx_xdp_redirect: 0
rx_xdp_tx_xmit: 0
rx_xdp_tx_full: 0
rx_xdp_tx_err: 0
rx_xdp_tx_cqe: 0
tx_csum_none: 494075047017
tx_csum_partial: 57081483898
tx_csum_partial_inner: 0
tx_queue_stopped: 518624
tx_queue_dropped: 0
tx_xmit_more: 1717880628
tx_recover: 0
tx_cqes: 549438869029
tx_queue_wake: 518627
tx_udp_seg_rem: 0
tx_cqe_err: 0
tx_xdp_xmit: 0
tx_xdp_full: 0
tx_xdp_err: 0
tx_xdp_cqes: 0
rx_wqe_err: 0
rx_mpwqe_filler_cqes: 0
rx_mpwqe_filler_strides: 0
rx_buff_alloc_err: 0
rx_cqe_compress_blks: 11483228712
rx_cqe_compress_pkts: 25794213324
rx_page_reuse: 0
rx_cache_reuse: 63610249810
rx_cache_full: 194650916511
rx_cache_empty: 1118208
rx_cache_busy: 194650982430
rx_cache_waive: 0
rx_congst_umr: 0
rx_arfs_err: 0
ch_events: 119556002196
ch_poll: 121107424977
ch_arm: 115856746008
ch_aff_change: 31
ch_eq_rearm: 0
rx_out_of_buffer: 6880325
rx_if_down_packets: 2062529
rx_vport_unicast_packets: 517433716795
rx_vport_unicast_bytes: 683464347301443
tx_vport_unicast_packets: 677697453738
tx_vport_unicast_bytes: 415788589663315
rx_vport_multicast_packets: 208258309
rx_vport_multicast_bytes: 14224046052
tx_vport_multicast_packets: 21689
tx_vport_multicast_bytes: 2158334
rx_vport_broadcast_packets: 75838646
rx_vport_broadcast_bytes: 4697944695
tx_vport_broadcast_packets: 68730
tx_vport_broadcast_bytes: 4123800
rx_vport_rdma_unicast_packets: 0
rx_vport_rdma_unicast_bytes: 0
tx_vport_rdma_unicast_packets: 0
tx_vport_rdma_unicast_bytes: 0
rx_vport_rdma_multicast_packets: 0
rx_vport_rdma_multicast_bytes: 0
tx_vport_rdma_multicast_packets: 0
tx_vport_rdma_multicast_bytes: 0
tx_packets_phy: 677697543252
rx_packets_phy: 521319491878
rx_crc_errors_phy: 0
tx_bytes_phy: 418499385791411
rx_bytes_phy: 690322537017274
tx_multicast_phy: 21689
tx_broadcast_phy: 68730
rx_multicast_phy: 208258305
rx_broadcast_phy: 75838646
rx_in_range_len_errors_phy: 4
rx_out_of_range_len_phy: 0
rx_oversize_pkts_phy: 0
rx_symbol_err_phy: 0
tx_mac_control_phy: 0
rx_mac_control_phy: 0
rx_unsupported_op_phy: 0
rx_pause_ctrl_phy: 0
tx_pause_ctrl_phy: 0
rx_discards_phy: 3601449265
tx_discards_phy: 0
tx_errors_phy: 0
rx_undersize_pkts_phy: 0
rx_fragments_phy: 0
rx_jabbers_phy: 0
rx_64_bytes_phy: 1416456771
rx_65_to_127_bytes_phy: 40750434737
rx_128_to_255_bytes_phy: 11518110310
rx_256_to_511_bytes_phy: 7055850637
rx_512_to_1023_bytes_phy: 7811550424
rx_1024_to_1518_bytes_phy: 265547564845
rx_1519_to_2047_bytes_phy: 187219522899
rx_2048_to_4095_bytes_phy: 0
rx_4096_to_8191_bytes_phy: 0
rx_8192_to_10239_bytes_phy: 0
link_down_events_phy: 0
rx_pcs_symbol_err_phy: 0
rx_corrected_bits_phy: 0
rx_pci_signal_integrity: 0
tx_pci_signal_integrity: 48
rx_prio0_bytes: 688807632117485
rx_prio0_packets: 516310309931
tx_prio0_bytes: 418499382756025
tx_prio0_packets: 677697534982
rx_prio1_bytes: 1497701612877
rx_prio1_packets: 1206768094
tx_prio1_bytes: 0
tx_prio1_packets: 0
rx_prio2_bytes: 112271227
rx_prio2_packets: 337295
tx_prio2_bytes: 0
tx_prio2_packets: 0
rx_prio3_bytes: 1165455555
rx_prio3_packets: 1544310
tx_prio3_bytes: 0
tx_prio3_packets: 0
rx_prio4_bytes: 161857240
rx_prio4_packets: 341392
tx_prio4_bytes: 0
tx_prio4_packets: 0
rx_prio5_bytes: 455031612
rx_prio5_packets: 2861469
tx_prio5_bytes: 0
tx_prio5_packets: 0
rx_prio6_bytes: 1873928697
rx_prio6_packets: 5146981
tx_prio6_bytes: 0
tx_prio6_packets: 0
rx_prio7_bytes: 13423452430
rx_prio7_packets: 190724796
tx_prio7_bytes: 0
tx_prio7_packets: 0
module_unplug: 0
module_bus_stuck: 0
module_high_temp: 0
module_bad_shorted: 0
ch0_events: 4252266777
ch0_poll: 4330804273
ch0_arm: 4120233182
ch0_aff_change: 2
ch0_eq_rearm: 0
ch1_events: 3938415938
ch1_poll: 4012621322
ch1_arm: 3810131188
ch1_aff_change: 2
ch1_eq_rearm: 0
ch2_events: 3897428860
ch2_poll: 3973886848
ch2_arm: 3773019397
ch2_aff_change: 1
ch2_eq_rearm: 0
ch3_events: 4108000541
ch3_poll: 4180139872
ch3_arm: 3982093366
ch3_aff_change: 1
ch3_eq_rearm: 0
ch4_events: 4652570079
ch4_poll: 4720541090
ch4_arm: 4524475054
ch4_aff_change: 2
ch4_eq_rearm: 0
ch5_events: 3899177385
ch5_poll: 3974274186
ch5_arm: 3772299186
ch5_aff_change: 2
ch5_eq_rearm: 0
ch6_events: 3915161350
ch6_poll: 3992338199
ch6_arm: 3794710989
ch6_aff_change: 0
ch6_eq_rearm: 0
ch7_events: 4008175631
ch7_poll: 4081321248
ch7_arm: 3882263723
ch7_aff_change: 0
ch7_eq_rearm: 0
ch8_events: 4207422352
ch8_poll: 4276465449
ch8_arm: 4077650366
ch8_aff_change: 0
ch8_eq_rearm: 0
ch9_events: 4036491879
ch9_poll: 4108975987
ch9_arm: 3914493694
ch9_aff_change: 0
ch9_eq_rearm: 0
ch10_events: 4066261595
ch10_poll: 4134419606
ch10_arm: 3936637711
ch10_aff_change: 1
ch10_eq_rearm: 0
ch11_events: 4440494043
ch11_poll: 4507578730
ch11_arm: 4318629438
ch11_aff_change: 0
ch11_eq_rearm: 0
ch12_events: 4066958252
ch12_poll: 4130191506
ch12_arm: 3934337782
ch12_aff_change: 0
ch12_eq_rearm: 0
ch13_events: 4051309159
ch13_poll: 4118864120
ch13_arm: 3921011919
ch13_aff_change: 0
ch13_eq_rearm: 0
ch14_events: 4321664800
ch14_poll: 4382433680
ch14_arm: 4186130552
ch14_aff_change: 0
ch14_eq_rearm: 0
ch15_events: 4701102075
ch15_poll: 4760373932
ch15_arm: 4570151468
ch15_aff_change: 0
ch15_eq_rearm: 0
ch16_events: 4311052687
ch16_poll: 4345937129
ch16_arm: 4170883819
ch16_aff_change: 0
ch16_eq_rearm: 0
ch17_events: 4647570931
ch17_poll: 4680218533
ch17_arm: 4509426288
ch17_aff_change: 0
ch17_eq_rearm: 0
ch18_events: 4598195702
ch18_poll: 4631314898
ch18_arm: 4457267084
ch18_aff_change: 0
ch18_eq_rearm: 0
ch19_events: 4808094560
ch19_poll: 4841368340
ch19_arm: 4670604358
ch19_aff_change: 0
ch19_eq_rearm: 0
ch20_events: 4240910605
ch20_poll: 4276531502
ch20_arm: 4101767278
ch20_aff_change: 1
ch20_eq_rearm: 0
ch21_events: 4389371472
ch21_poll: 4426870311
ch21_arm: 4249339045
ch21_aff_change: 2
ch21_eq_rearm: 0
ch22_events: 4282958754
ch22_poll: 4319228073
ch22_arm: 4145102991
ch22_aff_change: 2
ch22_eq_rearm: 0
ch23_events: 4440196528
ch23_poll: 4474090188
ch23_arm: 4300837147
ch23_aff_change: 2
ch23_eq_rearm: 0
ch24_events: 4326875785
ch24_poll: 4364971263
ch24_arm: 4186404526
ch24_aff_change: 2
ch24_eq_rearm: 0
ch25_events: 4286528453
ch25_poll: 4324089445
ch25_arm: 4147222616
ch25_aff_change: 3
ch25_eq_rearm: 0
ch26_events: 4098043104
ch26_poll: 4138133745
ch26_arm: 3967438971
ch26_aff_change: 4
ch26_eq_rearm: 0
ch27_events: 4563302840
ch27_poll: 4599441446
ch27_arm: 4432182806
ch27_aff_change: 4
ch27_eq_rearm: 0
ch28_events: 4
ch28_poll: 4
ch28_arm: 4
ch28_aff_change: 0
ch28_eq_rearm: 0
ch29_events: 6
ch29_poll: 6
ch29_arm: 6
ch29_aff_change: 0
ch29_eq_rearm: 0
ch30_events: 4
ch30_poll: 4
ch30_arm: 4
ch30_aff_change: 0
ch30_eq_rearm: 0
ch31_events: 4
ch31_poll: 4
ch31_arm: 4
ch31_aff_change: 0
ch31_eq_rearm: 0
ch32_events: 4
ch32_poll: 4
ch32_arm: 4
ch32_aff_change: 0
ch32_eq_rearm: 0
ch33_events: 4
ch33_poll: 4
ch33_arm: 4
ch33_aff_change: 0
ch33_eq_rearm: 0
ch34_events: 4
ch34_poll: 4
ch34_arm: 4
ch34_aff_change: 0
ch34_eq_rearm: 0
ch35_events: 4
ch35_poll: 4
ch35_arm: 4
ch35_aff_change: 0
ch35_eq_rearm: 0
ch36_events: 4
ch36_poll: 4
ch36_arm: 4
ch36_aff_change: 0
ch36_eq_rearm: 0
ch37_events: 4
ch37_poll: 4
ch37_arm: 4
ch37_aff_change: 0
ch37_eq_rearm: 0
ch38_events: 4
ch38_poll: 4
ch38_arm: 4
ch38_aff_change: 0
ch38_eq_rearm: 0
ch39_events: 4
ch39_poll: 4
ch39_arm: 4
ch39_aff_change: 0
ch39_eq_rearm: 0
ch40_events: 4
ch40_poll: 4
ch40_arm: 4
ch40_aff_change: 0
ch40_eq_rearm: 0
ch41_events: 4
ch41_poll: 4
ch41_arm: 4
ch41_aff_change: 0
ch41_eq_rearm: 0
ch42_events: 4
ch42_poll: 4
ch42_arm: 4
ch42_aff_change: 0
ch42_eq_rearm: 0
ch43_events: 4
ch43_poll: 4
ch43_arm: 4
ch43_aff_change: 0
ch43_eq_rearm: 0
ch44_events: 4
ch44_poll: 4
ch44_arm: 4
ch44_aff_change: 0
ch44_eq_rearm: 0
ch45_events: 4
ch45_poll: 4
ch45_arm: 4
ch45_aff_change: 0
ch45_eq_rearm: 0
ch46_events: 4
ch46_poll: 4
ch46_arm: 4
ch46_aff_change: 0
ch46_eq_rearm: 0
ch47_events: 4
ch47_poll: 4
ch47_arm: 4
ch47_aff_change: 0
ch47_eq_rearm: 0
ch48_events: 4
ch48_poll: 4
ch48_arm: 4
ch48_aff_change: 0
ch48_eq_rearm: 0
ch49_events: 4
ch49_poll: 4
ch49_arm: 4
ch49_aff_change: 0
ch49_eq_rearm: 0
ch50_events: 4
ch50_poll: 4
ch50_arm: 4
ch50_aff_change: 0
ch50_eq_rearm: 0
ch51_events: 4
ch51_poll: 4
ch51_arm: 4
ch51_aff_change: 0
ch51_eq_rearm: 0
ch52_events: 4
ch52_poll: 4
ch52_arm: 4
ch52_aff_change: 0
ch52_eq_rearm: 0
ch53_events: 4
ch53_poll: 4
ch53_arm: 4
ch53_aff_change: 0
ch53_eq_rearm: 0
ch54_events: 4
ch54_poll: 4
ch54_arm: 4
ch54_aff_change: 0
ch54_eq_rearm: 0
ch55_events: 4
ch55_poll: 4
ch55_arm: 4
ch55_aff_change: 0
ch55_eq_rearm: 0
rx0_packets: 21390033774
rx0_bytes: 27326856299122
rx0_csum_complete: 21339650906
rx0_csum_unnecessary: 0
rx0_csum_unnecessary_inner: 0
rx0_csum_none: 50382868
rx0_xdp_drop: 0
rx0_xdp_redirect: 0
rx0_lro_packets: 0
rx0_lro_bytes: 0
rx0_ecn_mark: 0
rx0_removed_vlan_packets: 21390033774
rx0_wqe_err: 0
rx0_mpwqe_filler_cqes: 0
rx0_mpwqe_filler_strides: 0
rx0_buff_alloc_err: 0
rx0_cqe_compress_blks: 481077641
rx0_cqe_compress_pkts: 1085647489
rx0_page_reuse: 0
rx0_cache_reuse: 19050049
rx0_cache_full: 10675964285
rx0_cache_empty: 37376
rx0_cache_busy: 10675966819
rx0_cache_waive: 0
rx0_congst_umr: 0
rx0_arfs_err: 0
rx0_xdp_tx_xmit: 0
rx0_xdp_tx_full: 0
rx0_xdp_tx_err: 0
rx0_xdp_tx_cqes: 0
rx1_packets: 19868919527
rx1_bytes: 26149716991561
rx1_csum_complete: 19868919527
rx1_csum_unnecessary: 0
rx1_csum_unnecessary_inner: 0
rx1_csum_none: 0
rx1_xdp_drop: 0
rx1_xdp_redirect: 0
rx1_lro_packets: 0
rx1_lro_bytes: 0
rx1_ecn_mark: 0
rx1_removed_vlan_packets: 19868919527
rx1_wqe_err: 0
rx1_mpwqe_filler_cqes: 0
rx1_mpwqe_filler_strides: 0
rx1_buff_alloc_err: 0
rx1_cqe_compress_blks: 420210560
rx1_cqe_compress_pkts: 941233388
rx1_page_reuse: 0
rx1_cache_reuse: 46200002
rx1_cache_full: 9888257242
rx1_cache_empty: 37376
rx1_cache_busy: 9888259746
rx1_cache_waive: 0
rx1_congst_umr: 0
rx1_arfs_err: 0
rx1_xdp_tx_xmit: 0
rx1_xdp_tx_full: 0
rx1_xdp_tx_err: 0
rx1_xdp_tx_cqes: 0
rx2_packets: 19575013662
rx2_bytes: 25759818417945
rx2_csum_complete: 19575013662
rx2_csum_unnecessary: 0
rx2_csum_unnecessary_inner: 0
rx2_csum_none: 0
rx2_xdp_drop: 0
rx2_xdp_redirect: 0
rx2_lro_packets: 0
rx2_lro_bytes: 0
rx2_ecn_mark: 0
rx2_removed_vlan_packets: 19575013662
rx2_wqe_err: 0
rx2_mpwqe_filler_cqes: 0
rx2_mpwqe_filler_strides: 0
rx2_buff_alloc_err: 0
rx2_cqe_compress_blks: 412345511
rx2_cqe_compress_pkts: 923376167
rx2_page_reuse: 0
rx2_cache_reuse: 38837731
rx2_cache_full: 9748666548
rx2_cache_empty: 37376
rx2_cache_busy: 9748669093
rx2_cache_waive: 0
rx2_congst_umr: 0
rx2_arfs_err: 0
rx2_xdp_tx_xmit: 0
rx2_xdp_tx_full: 0
rx2_xdp_tx_err: 0
rx2_xdp_tx_cqes: 0
rx3_packets: 19795911749
rx3_bytes: 25969475566905
rx3_csum_complete: 19795911749
rx3_csum_unnecessary: 0
rx3_csum_unnecessary_inner: 0
rx3_csum_none: 0
rx3_xdp_drop: 0
rx3_xdp_redirect: 0
rx3_lro_packets: 0
rx3_lro_bytes: 0
rx3_ecn_mark: 0
rx3_removed_vlan_packets: 19795911749
rx3_wqe_err: 0
rx3_mpwqe_filler_cqes: 0
rx3_mpwqe_filler_strides: 0
rx3_buff_alloc_err: 0
rx3_cqe_compress_blks: 416658765
rx3_cqe_compress_pkts: 934986266
rx3_page_reuse: 0
rx3_cache_reuse: 34542124
rx3_cache_full: 9863411232
rx3_cache_empty: 37376
rx3_cache_busy: 9863413732
rx3_cache_waive: 0
rx3_congst_umr: 0
rx3_arfs_err: 0
rx3_xdp_tx_xmit: 0
rx3_xdp_tx_full: 0
rx3_xdp_tx_err: 0
rx3_xdp_tx_cqes: 0
rx4_packets: 20445652378
rx4_bytes: 26949065110265
rx4_csum_complete: 20445652378
rx4_csum_unnecessary: 0
rx4_csum_unnecessary_inner: 0
rx4_csum_none: 0
rx4_xdp_drop: 0
rx4_xdp_redirect: 0
rx4_lro_packets: 0
rx4_lro_bytes: 0
rx4_ecn_mark: 0
rx4_removed_vlan_packets: 20445652378
rx4_wqe_err: 0
rx4_mpwqe_filler_cqes: 0
rx4_mpwqe_filler_strides: 0
rx4_buff_alloc_err: 0
rx4_cqe_compress_blks: 506085858
rx4_cqe_compress_pkts: 1147860328
rx4_page_reuse: 0
rx4_cache_reuse: 10122542864
rx4_cache_full: 100281206
rx4_cache_empty: 37376
rx4_cache_busy: 100283304
rx4_cache_waive: 0
rx4_congst_umr: 0
rx4_arfs_err: 0
rx4_xdp_tx_xmit: 0
rx4_xdp_tx_full: 0
rx4_xdp_tx_err: 0
rx4_xdp_tx_cqes: 0
rx5_packets: 19622362246
rx5_bytes: 25843450982982
rx5_csum_complete: 19622362246
rx5_csum_unnecessary: 0
rx5_csum_unnecessary_inner: 0
rx5_csum_none: 0
rx5_xdp_drop: 0
rx5_xdp_redirect: 0
rx5_lro_packets: 0
rx5_lro_bytes: 0
rx5_ecn_mark: 0
rx5_removed_vlan_packets: 19622362246
rx5_wqe_err: 0
rx5_mpwqe_filler_cqes: 0
rx5_mpwqe_filler_strides: 0
rx5_buff_alloc_err: 0
rx5_cqe_compress_blks: 422840924
rx5_cqe_compress_pkts: 948005878
rx5_page_reuse: 0
rx5_cache_reuse: 31285453
rx5_cache_full: 9779893117
rx5_cache_empty: 37376
rx5_cache_busy: 9779895647
rx5_cache_waive: 0
rx5_congst_umr: 0
rx5_arfs_err: 0
rx5_xdp_tx_xmit: 0
rx5_xdp_tx_full: 0
rx5_xdp_tx_err: 0
rx5_xdp_tx_cqes: 0
rx6_packets: 19788231278
rx6_bytes: 25985783006486
rx6_csum_complete: 19788231278
rx6_csum_unnecessary: 0
rx6_csum_unnecessary_inner: 0
rx6_csum_none: 0
rx6_xdp_drop: 0
rx6_xdp_redirect: 0
rx6_lro_packets: 0
rx6_lro_bytes: 0
rx6_ecn_mark: 0
rx6_removed_vlan_packets: 19788231278
rx6_wqe_err: 0
rx6_mpwqe_filler_cqes: 0
rx6_mpwqe_filler_strides: 0
rx6_buff_alloc_err: 0
rx6_cqe_compress_blks: 418799056
rx6_cqe_compress_pkts: 938282685
rx6_page_reuse: 0
rx6_cache_reuse: 18114793
rx6_cache_full: 9875998295
rx6_cache_empty: 37376
rx6_cache_busy: 9876000831
rx6_cache_waive: 0
rx6_congst_umr: 0
rx6_arfs_err: 0
rx6_xdp_tx_xmit: 0
rx6_xdp_tx_full: 0
rx6_xdp_tx_err: 0
rx6_xdp_tx_cqes: 0
rx7_packets: 19795759168
rx7_bytes: 26085056586860
rx7_csum_complete: 19795759168
rx7_csum_unnecessary: 0
rx7_csum_unnecessary_inner: 0
rx7_csum_none: 0
rx7_xdp_drop: 0
rx7_xdp_redirect: 0
rx7_lro_packets: 0
rx7_lro_bytes: 0
rx7_ecn_mark: 0
rx7_removed_vlan_packets: 19795759168
rx7_wqe_err: 0
rx7_mpwqe_filler_cqes: 0
rx7_mpwqe_filler_strides: 0
rx7_buff_alloc_err: 0
rx7_cqe_compress_blks: 413959224
rx7_cqe_compress_pkts: 927675936
rx7_page_reuse: 0
rx7_cache_reuse: 23902990
rx7_cache_full: 9873974042
rx7_cache_empty: 37376
rx7_cache_busy: 9873976574
rx7_cache_waive: 0
rx7_congst_umr: 0
rx7_arfs_err: 0
rx7_xdp_tx_xmit: 0
rx7_xdp_tx_full: 0
rx7_xdp_tx_err: 0
rx7_xdp_tx_cqes: 0
rx8_packets: 19963477439
rx8_bytes: 26384640501789
rx8_csum_complete: 19963477439
rx8_csum_unnecessary: 0
rx8_csum_unnecessary_inner: 0
rx8_csum_none: 0
rx8_xdp_drop: 0
rx8_xdp_redirect: 0
rx8_lro_packets: 0
rx8_lro_bytes: 0
rx8_ecn_mark: 0
rx8_removed_vlan_packets: 19963477439
rx8_wqe_err: 0
rx8_mpwqe_filler_cqes: 0
rx8_mpwqe_filler_strides: 0
rx8_buff_alloc_err: 0
rx8_cqe_compress_blks: 420422857
rx8_cqe_compress_pkts: 942720292
rx8_page_reuse: 0
rx8_cache_reuse: 88181713
rx8_cache_full: 9893554525
rx8_cache_empty: 37376
rx8_cache_busy: 9893556983
rx8_cache_waive: 0
rx8_congst_umr: 0
rx8_arfs_err: 0
rx8_xdp_tx_xmit: 0
rx8_xdp_tx_full: 0
rx8_xdp_tx_err: 0
rx8_xdp_tx_cqes: 0
rx9_packets: 19726642138
rx9_bytes: 26063924286499
rx9_csum_complete: 19726642138
rx9_csum_unnecessary: 0
rx9_csum_unnecessary_inner: 0
rx9_csum_none: 0
rx9_xdp_drop: 0
rx9_xdp_redirect: 0
rx9_lro_packets: 0
rx9_lro_bytes: 0
rx9_ecn_mark: 0
rx9_removed_vlan_packets: 19726642138
rx9_wqe_err: 0
rx9_mpwqe_filler_cqes: 0
rx9_mpwqe_filler_strides: 0
rx9_buff_alloc_err: 0
rx9_cqe_compress_blks: 424227411
rx9_cqe_compress_pkts: 951534873
rx9_page_reuse: 0
rx9_cache_reuse: 482901440
rx9_cache_full: 9380417487
rx9_cache_empty: 37376
rx9_cache_busy: 9380419608
rx9_cache_waive: 0
rx9_congst_umr: 0
rx9_arfs_err: 0
rx9_xdp_tx_xmit: 0
rx9_xdp_tx_full: 0
rx9_xdp_tx_err: 0
rx9_xdp_tx_cqes: 0
rx10_packets: 19901229170
rx10_bytes: 26300854495044
rx10_csum_complete: 19901229170
rx10_csum_unnecessary: 0
rx10_csum_unnecessary_inner: 0
rx10_csum_none: 0
rx10_xdp_drop: 0
rx10_xdp_redirect: 0
rx10_lro_packets: 0
rx10_lro_bytes: 0
rx10_ecn_mark: 0
rx10_removed_vlan_packets: 19901229170
rx10_wqe_err: 0
rx10_mpwqe_filler_cqes: 0
rx10_mpwqe_filler_strides: 0
rx10_buff_alloc_err: 0
rx10_cqe_compress_blks: 419082938
rx10_cqe_compress_pkts: 940791347
rx10_page_reuse: 0
rx10_cache_reuse: 14896055
rx10_cache_full: 9935715977
rx10_cache_empty: 37376
rx10_cache_busy: 9935718513
rx10_cache_waive: 0
rx10_congst_umr: 0
rx10_arfs_err: 0
rx10_xdp_tx_xmit: 0
rx10_xdp_tx_full: 0
rx10_xdp_tx_err: 0
rx10_xdp_tx_cqes: 0
rx11_packets: 20352190494
rx11_bytes: 26851034425372
rx11_csum_complete: 20352190494
rx11_csum_unnecessary: 0
rx11_csum_unnecessary_inner: 0
rx11_csum_none: 0
rx11_xdp_drop: 0
rx11_xdp_redirect: 0
rx11_lro_packets: 0
rx11_lro_bytes: 0
rx11_ecn_mark: 0
rx11_removed_vlan_packets: 20352190494
rx11_wqe_err: 0
rx11_mpwqe_filler_cqes: 0
rx11_mpwqe_filler_strides: 0
rx11_buff_alloc_err: 0
rx11_cqe_compress_blks: 501992147
rx11_cqe_compress_pkts: 1140398610
rx11_page_reuse: 0
rx11_cache_reuse: 10071721531
rx11_cache_full: 104371621
rx11_cache_empty: 37376
rx11_cache_busy: 104373697
rx11_cache_waive: 0
rx11_congst_umr: 0
rx11_arfs_err: 0
rx11_xdp_tx_xmit: 0
rx11_xdp_tx_full: 0
rx11_xdp_tx_err: 0
rx11_xdp_tx_cqes: 0
rx12_packets: 19934747149
rx12_bytes: 26296478787829
rx12_csum_complete: 19934747149
rx12_csum_unnecessary: 0
rx12_csum_unnecessary_inner: 0
rx12_csum_none: 0
rx12_xdp_drop: 0
rx12_xdp_redirect: 0
rx12_lro_packets: 0
rx12_lro_bytes: 0
rx12_ecn_mark: 0
rx12_removed_vlan_packets: 19934747149
rx12_wqe_err: 0
rx12_mpwqe_filler_cqes: 0
rx12_mpwqe_filler_strides: 0
rx12_buff_alloc_err: 0
rx12_cqe_compress_blks: 443350570
rx12_cqe_compress_pkts: 995997220
rx12_page_reuse: 0
rx12_cache_reuse: 9864934174
rx12_cache_full: 102437428
rx12_cache_empty: 37376
rx12_cache_busy: 102439382
rx12_cache_waive: 0
rx12_congst_umr: 0
rx12_arfs_err: 0
rx12_xdp_tx_xmit: 0
rx12_xdp_tx_full: 0
rx12_xdp_tx_err: 0
rx12_xdp_tx_cqes: 0
rx13_packets: 19866908096
rx13_bytes: 26160931936186
rx13_csum_complete: 19866908096
rx13_csum_unnecessary: 0
rx13_csum_unnecessary_inner: 0
rx13_csum_none: 0
rx13_xdp_drop: 0
rx13_xdp_redirect: 0
rx13_lro_packets: 0
rx13_lro_bytes: 0
rx13_ecn_mark: 0
rx13_removed_vlan_packets: 19866908096
rx13_wqe_err: 0
rx13_mpwqe_filler_cqes: 0
rx13_mpwqe_filler_strides: 0
rx13_buff_alloc_err: 0
rx13_cqe_compress_blks: 413640141
rx13_cqe_compress_pkts: 926175066
rx13_page_reuse: 0
rx13_cache_reuse: 36358610
rx13_cache_full: 9897092921
rx13_cache_empty: 37376
rx13_cache_busy: 9897095422
rx13_cache_waive: 0
rx13_congst_umr: 0
rx13_arfs_err: 0
rx13_xdp_tx_xmit: 0
rx13_xdp_tx_full: 0
rx13_xdp_tx_err: 0
rx13_xdp_tx_cqes: 0
rx14_packets: 20229035746
rx14_bytes: 26655092809172
rx14_csum_complete: 20229035746
rx14_csum_unnecessary: 0
rx14_csum_unnecessary_inner: 0
rx14_csum_none: 0
rx14_xdp_drop: 0
rx14_xdp_redirect: 0
rx14_lro_packets: 0
rx14_lro_bytes: 0
rx14_ecn_mark: 0
rx14_removed_vlan_packets: 20229035746
rx14_wqe_err: 0
rx14_mpwqe_filler_cqes: 0
rx14_mpwqe_filler_strides: 0
rx14_buff_alloc_err: 0
rx14_cqe_compress_blks: 460990337
rx14_cqe_compress_pkts: 1041287948
rx14_page_reuse: 0
rx14_cache_reuse: 25649275
rx14_cache_full: 10088866045
rx14_cache_empty: 37376
rx14_cache_busy: 10088868574
rx14_cache_waive: 0
rx14_congst_umr: 0
rx14_arfs_err: 0
rx14_xdp_tx_xmit: 0
rx14_xdp_tx_full: 0
rx14_xdp_tx_err: 0
rx14_xdp_tx_cqes: 0
rx15_packets: 20528177154
rx15_bytes: 27029263893264
rx15_csum_complete: 20528177154
rx15_csum_unnecessary: 0
rx15_csum_unnecessary_inner: 0
rx15_csum_none: 0
rx15_xdp_drop: 0
rx15_xdp_redirect: 0
rx15_lro_packets: 0
rx15_lro_bytes: 0
rx15_ecn_mark: 0
rx15_removed_vlan_packets: 20528177154
rx15_wqe_err: 0
rx15_mpwqe_filler_cqes: 0
rx15_mpwqe_filler_strides: 0
rx15_buff_alloc_err: 0
rx15_cqe_compress_blks: 476776176
rx15_cqe_compress_pkts: 1076153263
rx15_page_reuse: 0
rx15_cache_reuse: 48426735
rx15_cache_full: 10215659289
rx15_cache_empty: 37376
rx15_cache_busy: 10215661817
rx15_cache_waive: 0
rx15_congst_umr: 0
rx15_arfs_err: 0
rx15_xdp_tx_xmit: 0
rx15_xdp_tx_full: 0
rx15_xdp_tx_err: 0
rx15_xdp_tx_cqes: 0
rx16_packets: 16104078098
rx16_bytes: 21256361789679
rx16_csum_complete: 16104078098
rx16_csum_unnecessary: 0
rx16_csum_unnecessary_inner: 0
rx16_csum_none: 0
rx16_xdp_drop: 0
rx16_xdp_redirect: 0
rx16_lro_packets: 0
rx16_lro_bytes: 0
rx16_ecn_mark: 0
rx16_removed_vlan_packets: 16104078098
rx16_wqe_err: 0
rx16_mpwqe_filler_cqes: 0
rx16_mpwqe_filler_strides: 0
rx16_buff_alloc_err: 0
rx16_cqe_compress_blks: 352082054
rx16_cqe_compress_pkts: 787161670
rx16_page_reuse: 0
rx16_cache_reuse: 25912567
rx16_cache_full: 8026124051
rx16_cache_empty: 37376
rx16_cache_busy: 8026126465
rx16_cache_waive: 0
rx16_congst_umr: 0
rx16_arfs_err: 0
rx16_xdp_tx_xmit: 0
rx16_xdp_tx_full: 0
rx16_xdp_tx_err: 0
rx16_xdp_tx_cqes: 0
rx17_packets: 16314055017
rx17_bytes: 21589139030173
rx17_csum_complete: 16314055017
rx17_csum_unnecessary: 0
rx17_csum_unnecessary_inner: 0
rx17_csum_none: 0
rx17_xdp_drop: 0
rx17_xdp_redirect: 0
rx17_lro_packets: 0
rx17_lro_bytes: 0
rx17_ecn_mark: 0
rx17_removed_vlan_packets: 16314055017
rx17_wqe_err: 0
rx17_mpwqe_filler_cqes: 0
rx17_mpwqe_filler_strides: 0
rx17_buff_alloc_err: 0
rx17_cqe_compress_blks: 387834541
rx17_cqe_compress_pkts: 871851081
rx17_page_reuse: 0
rx17_cache_reuse: 24021313
rx17_cache_full: 8133003829
rx17_cache_empty: 37376
rx17_cache_busy: 8133006175
rx17_cache_waive: 0
rx17_congst_umr: 0
rx17_arfs_err: 0
rx17_xdp_tx_xmit: 0
rx17_xdp_tx_full: 0
rx17_xdp_tx_err: 0
rx17_xdp_tx_cqes: 0
rx18_packets: 16439016814
rx18_bytes: 21648651917475
rx18_csum_complete: 16439016814
rx18_csum_unnecessary: 0
rx18_csum_unnecessary_inner: 0
rx18_csum_none: 0
rx18_xdp_drop: 0
rx18_xdp_redirect: 0
rx18_lro_packets: 0
rx18_lro_bytes: 0
rx18_ecn_mark: 0
rx18_removed_vlan_packets: 16439016814
rx18_wqe_err: 0
rx18_mpwqe_filler_cqes: 0
rx18_mpwqe_filler_strides: 0
rx18_buff_alloc_err: 0
rx18_cqe_compress_blks: 375066666
rx18_cqe_compress_pkts: 843563974
rx18_page_reuse: 0
rx18_cache_reuse: 8151064266
rx18_cache_full: 68442025
rx18_cache_empty: 37376
rx18_cache_busy: 68444122
rx18_cache_waive: 0
rx18_congst_umr: 0
rx18_arfs_err: 0
rx18_xdp_tx_xmit: 0
rx18_xdp_tx_full: 0
rx18_xdp_tx_err: 0
rx18_xdp_tx_cqes: 0
rx19_packets: 16641223506
rx19_bytes: 21964749940935
rx19_csum_complete: 16641223506
rx19_csum_unnecessary: 0
rx19_csum_unnecessary_inner: 0
rx19_csum_none: 0
rx19_xdp_drop: 0
rx19_xdp_redirect: 0
rx19_lro_packets: 0
rx19_lro_bytes: 0
rx19_ecn_mark: 0
rx19_removed_vlan_packets: 16641223506
rx19_wqe_err: 0
rx19_mpwqe_filler_cqes: 0
rx19_mpwqe_filler_strides: 0
rx19_buff_alloc_err: 0
rx19_cqe_compress_blks: 387825932
rx19_cqe_compress_pkts: 872266355
rx19_page_reuse: 0
rx19_cache_reuse: 116433620
rx19_cache_full: 8204175954
rx19_cache_empty: 37376
rx19_cache_busy: 8204178120
rx19_cache_waive: 0
rx19_congst_umr: 0
rx19_arfs_err: 0
rx19_xdp_tx_xmit: 0
rx19_xdp_tx_full: 0
rx19_xdp_tx_err: 0
rx19_xdp_tx_cqes: 0
rx20_packets: 16206927741
rx20_bytes: 21387447038430
rx20_csum_complete: 16206927741
rx20_csum_unnecessary: 0
rx20_csum_unnecessary_inner: 0
rx20_csum_none: 0
rx20_xdp_drop: 0
rx20_xdp_redirect: 0
rx20_lro_packets: 0
rx20_lro_bytes: 0
rx20_ecn_mark: 0
rx20_removed_vlan_packets: 16206927741
rx20_wqe_err: 0
rx20_mpwqe_filler_cqes: 0
rx20_mpwqe_filler_strides: 0
rx20_buff_alloc_err: 0
rx20_cqe_compress_blks: 370144620
rx20_cqe_compress_pkts: 829122671
rx20_page_reuse: 0
rx20_cache_reuse: 8053733744
rx20_cache_full: 49728026
rx20_cache_empty: 37376
rx20_cache_busy: 49730116
rx20_cache_waive: 0
rx20_congst_umr: 0
rx20_arfs_err: 0
rx20_xdp_tx_xmit: 0
rx20_xdp_tx_full: 0
rx20_xdp_tx_err: 0
rx20_xdp_tx_cqes: 0
rx21_packets: 16562361314
rx21_bytes: 21856653284356
rx21_csum_complete: 16562361314
rx21_csum_unnecessary: 0
rx21_csum_unnecessary_inner: 0
rx21_csum_none: 0
rx21_xdp_drop: 0
rx21_xdp_redirect: 0
rx21_lro_packets: 0
rx21_lro_bytes: 0
rx21_ecn_mark: 0
rx21_removed_vlan_packets: 16562361314
rx21_wqe_err: 0
rx21_mpwqe_filler_cqes: 0
rx21_mpwqe_filler_strides: 0
rx21_buff_alloc_err: 0
rx21_cqe_compress_blks: 350790425
rx21_cqe_compress_pkts: 783850729
rx21_page_reuse: 0
rx21_cache_reuse: 28077493
rx21_cache_full: 8253100706
rx21_cache_empty: 37376
rx21_cache_busy: 8253103147
rx21_cache_waive: 0
rx21_congst_umr: 0
rx21_arfs_err: 0
rx21_xdp_tx_xmit: 0
rx21_xdp_tx_full: 0
rx21_xdp_tx_err: 0
rx21_xdp_tx_cqes: 0
rx22_packets: 16350307571
rx22_bytes: 21408575325592
rx22_csum_complete: 16350307571
rx22_csum_unnecessary: 0
rx22_csum_unnecessary_inner: 0
rx22_csum_none: 0
rx22_xdp_drop: 0
rx22_xdp_redirect: 0
rx22_lro_packets: 0
rx22_lro_bytes: 0
rx22_ecn_mark: 0
rx22_removed_vlan_packets: 16350307571
rx22_wqe_err: 0
rx22_mpwqe_filler_cqes: 0
rx22_mpwqe_filler_strides: 0
rx22_buff_alloc_err: 0
rx22_cqe_compress_blks: 353531065
rx22_cqe_compress_pkts: 790814415
rx22_page_reuse: 0
rx22_cache_reuse: 16934343
rx22_cache_full: 8158216889
rx22_cache_empty: 37376
rx22_cache_busy: 8158219417
rx22_cache_waive: 0
rx22_congst_umr: 0
rx22_arfs_err: 0
rx22_xdp_tx_xmit: 0
rx22_xdp_tx_full: 0
rx22_xdp_tx_err: 0
rx22_xdp_tx_cqes: 0
rx23_packets: 16019811764
rx23_bytes: 21137182570985
rx23_csum_complete: 16019811764
rx23_csum_unnecessary: 0
rx23_csum_unnecessary_inner: 0
rx23_csum_none: 0
rx23_xdp_drop: 0
rx23_xdp_redirect: 0
rx23_lro_packets: 0
rx23_lro_bytes: 0
rx23_ecn_mark: 0
rx23_removed_vlan_packets: 16019811764
rx23_wqe_err: 0
rx23_mpwqe_filler_cqes: 0
rx23_mpwqe_filler_strides: 0
rx23_buff_alloc_err: 0
rx23_cqe_compress_blks: 349733033
rx23_cqe_compress_pkts: 781248862
rx23_page_reuse: 0
rx23_cache_reuse: 33422343
rx23_cache_full: 7976481152
rx23_cache_empty: 37376
rx23_cache_busy: 7976483525
rx23_cache_waive: 0
rx23_congst_umr: 0
rx23_arfs_err: 0
rx23_xdp_tx_xmit: 0
rx23_xdp_tx_full: 0
rx23_xdp_tx_err: 0
rx23_xdp_tx_cqes: 0
rx24_packets: 16212040646
rx24_bytes: 21393399325700
rx24_csum_complete: 16212040646
rx24_csum_unnecessary: 0
rx24_csum_unnecessary_inner: 0
rx24_csum_none: 0
rx24_xdp_drop: 0
rx24_xdp_redirect: 0
rx24_lro_packets: 0
rx24_lro_bytes: 0
rx24_ecn_mark: 0
rx24_removed_vlan_packets: 16212040646
rx24_wqe_err: 0
rx24_mpwqe_filler_cqes: 0
rx24_mpwqe_filler_strides: 0
rx24_buff_alloc_err: 0
rx24_cqe_compress_blks: 379833752
rx24_cqe_compress_pkts: 852020179
rx24_page_reuse: 0
rx24_cache_reuse: 8033552512
rx24_cache_full: 72465843
rx24_cache_empty: 37376
rx24_cache_busy: 72467789
rx24_cache_waive: 0
rx24_congst_umr: 0
rx24_arfs_err: 0
rx24_xdp_tx_xmit: 0
rx24_xdp_tx_full: 0
rx24_xdp_tx_err: 0
rx24_xdp_tx_cqes: 0
rx25_packets: 16412186257
rx25_bytes: 21651198388407
rx25_csum_complete: 16412186257
rx25_csum_unnecessary: 0
rx25_csum_unnecessary_inner: 0
rx25_csum_none: 0
rx25_xdp_drop: 0
rx25_xdp_redirect: 0
rx25_lro_packets: 0
rx25_lro_bytes: 0
rx25_ecn_mark: 0
rx25_removed_vlan_packets: 16412186257
rx25_wqe_err: 0
rx25_mpwqe_filler_cqes: 0
rx25_mpwqe_filler_strides: 0
rx25_buff_alloc_err: 0
rx25_cqe_compress_blks: 383979685
rx25_cqe_compress_pkts: 861985772
rx25_page_reuse: 0
rx25_cache_reuse: 8129807841
rx25_cache_full: 76283342
rx25_cache_empty: 37376
rx25_cache_busy: 76285271
rx25_cache_waive: 0
rx25_congst_umr: 0
rx25_arfs_err: 0
rx25_xdp_tx_xmit: 0
rx25_xdp_tx_full: 0
rx25_xdp_tx_err: 0
rx25_xdp_tx_cqes: 0
rx26_packets: 16304310003
rx26_bytes: 21571217538721
rx26_csum_complete: 16304310003
rx26_csum_unnecessary: 0
rx26_csum_unnecessary_inner: 0
rx26_csum_none: 0
rx26_xdp_drop: 0
rx26_xdp_redirect: 0
rx26_lro_packets: 0
rx26_lro_bytes: 0
rx26_ecn_mark: 0
rx26_removed_vlan_packets: 16304310003
rx26_wqe_err: 0
rx26_mpwqe_filler_cqes: 0
rx26_mpwqe_filler_strides: 0
rx26_buff_alloc_err: 0
rx26_cqe_compress_blks: 353314041
rx26_cqe_compress_pkts: 788838424
rx26_page_reuse: 0
rx26_cache_reuse: 19673790
rx26_cache_full: 8132478659
rx26_cache_empty: 37376
rx26_cache_busy: 8132481198
rx26_cache_waive: 0
rx26_congst_umr: 0
rx26_arfs_err: 0
rx26_xdp_tx_xmit: 0
rx26_xdp_tx_full: 0
rx26_xdp_tx_err: 0
rx26_xdp_tx_cqes: 0
rx27_packets: 16171856079
rx27_bytes: 21376891736540
rx27_csum_complete: 16171856079
rx27_csum_unnecessary: 0
rx27_csum_unnecessary_inner: 0
rx27_csum_none: 0
rx27_xdp_drop: 0
rx27_xdp_redirect: 0
rx27_lro_packets: 0
rx27_lro_bytes: 0
rx27_ecn_mark: 0
rx27_removed_vlan_packets: 16171856079
rx27_wqe_err: 0
rx27_mpwqe_filler_cqes: 0
rx27_mpwqe_filler_strides: 0
rx27_buff_alloc_err: 0
rx27_cqe_compress_blks: 386632845
rx27_cqe_compress_pkts: 869362576
rx27_page_reuse: 0
rx27_cache_reuse: 10070560
rx27_cache_full: 8075854928
rx27_cache_empty: 37376
rx27_cache_busy: 8075857468
rx27_cache_waive: 0
rx27_congst_umr: 0
rx27_arfs_err: 0
rx27_xdp_tx_xmit: 0
rx27_xdp_tx_full: 0
rx27_xdp_tx_err: 0
rx27_xdp_tx_cqes: 0
rx28_packets: 0
rx28_bytes: 0
rx28_csum_complete: 0
rx28_csum_unnecessary: 0
rx28_csum_unnecessary_inner: 0
rx28_csum_none: 0
rx28_xdp_drop: 0
rx28_xdp_redirect: 0
rx28_lro_packets: 0
rx28_lro_bytes: 0
rx28_ecn_mark: 0
rx28_removed_vlan_packets: 0
rx28_wqe_err: 0
rx28_mpwqe_filler_cqes: 0
rx28_mpwqe_filler_strides: 0
rx28_buff_alloc_err: 0
rx28_cqe_compress_blks: 0
rx28_cqe_compress_pkts: 0
rx28_page_reuse: 0
rx28_cache_reuse: 0
rx28_cache_full: 0
rx28_cache_empty: 2560
rx28_cache_busy: 0
rx28_cache_waive: 0
rx28_congst_umr: 0
rx28_arfs_err: 0
rx28_xdp_tx_xmit: 0
rx28_xdp_tx_full: 0
rx28_xdp_tx_err: 0
rx28_xdp_tx_cqes: 0
rx29_packets: 0
rx29_bytes: 0
rx29_csum_complete: 0
rx29_csum_unnecessary: 0
rx29_csum_unnecessary_inner: 0
rx29_csum_none: 0
rx29_xdp_drop: 0
rx29_xdp_redirect: 0
rx29_lro_packets: 0
rx29_lro_bytes: 0
rx29_ecn_mark: 0
rx29_removed_vlan_packets: 0
rx29_wqe_err: 0
rx29_mpwqe_filler_cqes: 0
rx29_mpwqe_filler_strides: 0
rx29_buff_alloc_err: 0
rx29_cqe_compress_blks: 0
rx29_cqe_compress_pkts: 0
rx29_page_reuse: 0
rx29_cache_reuse: 0
rx29_cache_full: 0
rx29_cache_empty: 2560
rx29_cache_busy: 0
rx29_cache_waive: 0
rx29_congst_umr: 0
rx29_arfs_err: 0
rx29_xdp_tx_xmit: 0
rx29_xdp_tx_full: 0
rx29_xdp_tx_err: 0
rx29_xdp_tx_cqes: 0
rx30_packets: 0
rx30_bytes: 0
rx30_csum_complete: 0
rx30_csum_unnecessary: 0
rx30_csum_unnecessary_inner: 0
rx30_csum_none: 0
rx30_xdp_drop: 0
rx30_xdp_redirect: 0
rx30_lro_packets: 0
rx30_lro_bytes: 0
rx30_ecn_mark: 0
rx30_removed_vlan_packets: 0
rx30_wqe_err: 0
rx30_mpwqe_filler_cqes: 0
rx30_mpwqe_filler_strides: 0
rx30_buff_alloc_err: 0
rx30_cqe_compress_blks: 0
rx30_cqe_compress_pkts: 0
rx30_page_reuse: 0
rx30_cache_reuse: 0
rx30_cache_full: 0
rx30_cache_empty: 2560
rx30_cache_busy: 0
rx30_cache_waive: 0
rx30_congst_umr: 0
rx30_arfs_err: 0
rx30_xdp_tx_xmit: 0
rx30_xdp_tx_full: 0
rx30_xdp_tx_err: 0
rx30_xdp_tx_cqes: 0
rx31_packets: 0
rx31_bytes: 0
rx31_csum_complete: 0
rx31_csum_unnecessary: 0
rx31_csum_unnecessary_inner: 0
rx31_csum_none: 0
rx31_xdp_drop: 0
rx31_xdp_redirect: 0
rx31_lro_packets: 0
rx31_lro_bytes: 0
rx31_ecn_mark: 0
rx31_removed_vlan_packets: 0
rx31_wqe_err: 0
rx31_mpwqe_filler_cqes: 0
rx31_mpwqe_filler_strides: 0
rx31_buff_alloc_err: 0
rx31_cqe_compress_blks: 0
rx31_cqe_compress_pkts: 0
rx31_page_reuse: 0
rx31_cache_reuse: 0
rx31_cache_full: 0
rx31_cache_empty: 2560
rx31_cache_busy: 0
rx31_cache_waive: 0
rx31_congst_umr: 0
rx31_arfs_err: 0
rx31_xdp_tx_xmit: 0
rx31_xdp_tx_full: 0
rx31_xdp_tx_err: 0
rx31_xdp_tx_cqes: 0
rx32_packets: 0
rx32_bytes: 0
rx32_csum_complete: 0
rx32_csum_unnecessary: 0
rx32_csum_unnecessary_inner: 0
rx32_csum_none: 0
rx32_xdp_drop: 0
rx32_xdp_redirect: 0
rx32_lro_packets: 0
rx32_lro_bytes: 0
rx32_ecn_mark: 0
rx32_removed_vlan_packets: 0
rx32_wqe_err: 0
rx32_mpwqe_filler_cqes: 0
rx32_mpwqe_filler_strides: 0
rx32_buff_alloc_err: 0
rx32_cqe_compress_blks: 0
rx32_cqe_compress_pkts: 0
rx32_page_reuse: 0
rx32_cache_reuse: 0
rx32_cache_full: 0
rx32_cache_empty: 2560
rx32_cache_busy: 0
rx32_cache_waive: 0
rx32_congst_umr: 0
rx32_arfs_err: 0
rx32_xdp_tx_xmit: 0
rx32_xdp_tx_full: 0
rx32_xdp_tx_err: 0
rx32_xdp_tx_cqes: 0
rx33_packets: 0
rx33_bytes: 0
rx33_csum_complete: 0
rx33_csum_unnecessary: 0
rx33_csum_unnecessary_inner: 0
rx33_csum_none: 0
rx33_xdp_drop: 0
rx33_xdp_redirect: 0
rx33_lro_packets: 0
rx33_lro_bytes: 0
rx33_ecn_mark: 0
rx33_removed_vlan_packets: 0
rx33_wqe_err: 0
rx33_mpwqe_filler_cqes: 0
rx33_mpwqe_filler_strides: 0
rx33_buff_alloc_err: 0
rx33_cqe_compress_blks: 0
rx33_cqe_compress_pkts: 0
rx33_page_reuse: 0
rx33_cache_reuse: 0
rx33_cache_full: 0
rx33_cache_empty: 2560
rx33_cache_busy: 0
rx33_cache_waive: 0
rx33_congst_umr: 0
rx33_arfs_err: 0
rx33_xdp_tx_xmit: 0
rx33_xdp_tx_full: 0
rx33_xdp_tx_err: 0
rx33_xdp_tx_cqes: 0
rx34_packets: 0
rx34_bytes: 0
rx34_csum_complete: 0
rx34_csum_unnecessary: 0
rx34_csum_unnecessary_inner: 0
rx34_csum_none: 0
rx34_xdp_drop: 0
rx34_xdp_redirect: 0
rx34_lro_packets: 0
rx34_lro_bytes: 0
rx34_ecn_mark: 0
rx34_removed_vlan_packets: 0
rx34_wqe_err: 0
rx34_mpwqe_filler_cqes: 0
rx34_mpwqe_filler_strides: 0
rx34_buff_alloc_err: 0
rx34_cqe_compress_blks: 0
rx34_cqe_compress_pkts: 0
rx34_page_reuse: 0
rx34_cache_reuse: 0
rx34_cache_full: 0
rx34_cache_empty: 2560
rx34_cache_busy: 0
rx34_cache_waive: 0
rx34_congst_umr: 0
rx34_arfs_err: 0
rx34_xdp_tx_xmit: 0
rx34_xdp_tx_full: 0
rx34_xdp_tx_err: 0
rx34_xdp_tx_cqes: 0
rx35_packets: 0
rx35_bytes: 0
rx35_csum_complete: 0
rx35_csum_unnecessary: 0
rx35_csum_unnecessary_inner: 0
rx35_csum_none: 0
rx35_xdp_drop: 0
rx35_xdp_redirect: 0
rx35_lro_packets: 0
rx35_lro_bytes: 0
rx35_ecn_mark: 0
rx35_removed_vlan_packets: 0
rx35_wqe_err: 0
rx35_mpwqe_filler_cqes: 0
rx35_mpwqe_filler_strides: 0
rx35_buff_alloc_err: 0
rx35_cqe_compress_blks: 0
rx35_cqe_compress_pkts: 0
rx35_page_reuse: 0
rx35_cache_reuse: 0
rx35_cache_full: 0
rx35_cache_empty: 2560
rx35_cache_busy: 0
rx35_cache_waive: 0
rx35_congst_umr: 0
rx35_arfs_err: 0
rx35_xdp_tx_xmit: 0
rx35_xdp_tx_full: 0
rx35_xdp_tx_err: 0
rx35_xdp_tx_cqes: 0
rx36_packets: 0
rx36_bytes: 0
rx36_csum_complete: 0
rx36_csum_unnecessary: 0
rx36_csum_unnecessary_inner: 0
rx36_csum_none: 0
rx36_xdp_drop: 0
rx36_xdp_redirect: 0
rx36_lro_packets: 0
rx36_lro_bytes: 0
rx36_ecn_mark: 0
rx36_removed_vlan_packets: 0
rx36_wqe_err: 0
rx36_mpwqe_filler_cqes: 0
rx36_mpwqe_filler_strides: 0
rx36_buff_alloc_err: 0
rx36_cqe_compress_blks: 0
rx36_cqe_compress_pkts: 0
rx36_page_reuse: 0
rx36_cache_reuse: 0
rx36_cache_full: 0
rx36_cache_empty: 2560
rx36_cache_busy: 0
rx36_cache_waive: 0
rx36_congst_umr: 0
rx36_arfs_err: 0
rx36_xdp_tx_xmit: 0
rx36_xdp_tx_full: 0
rx36_xdp_tx_err: 0
rx36_xdp_tx_cqes: 0
rx37_packets: 0
rx37_bytes: 0
rx37_csum_complete: 0
rx37_csum_unnecessary: 0
rx37_csum_unnecessary_inner: 0
rx37_csum_none: 0
rx37_xdp_drop: 0
rx37_xdp_redirect: 0
rx37_lro_packets: 0
rx37_lro_bytes: 0
rx37_ecn_mark: 0
rx37_removed_vlan_packets: 0
rx37_wqe_err: 0
rx37_mpwqe_filler_cqes: 0
rx37_mpwqe_filler_strides: 0
rx37_buff_alloc_err: 0
rx37_cqe_compress_blks: 0
rx37_cqe_compress_pkts: 0
rx37_page_reuse: 0
rx37_cache_reuse: 0
rx37_cache_full: 0
rx37_cache_empty: 2560
rx37_cache_busy: 0
rx37_cache_waive: 0
rx37_congst_umr: 0
rx37_arfs_err: 0
rx37_xdp_tx_xmit: 0
rx37_xdp_tx_full: 0
rx37_xdp_tx_err: 0
rx37_xdp_tx_cqes: 0
rx38_packets: 0
rx38_bytes: 0
rx38_csum_complete: 0
rx38_csum_unnecessary: 0
rx38_csum_unnecessary_inner: 0
rx38_csum_none: 0
rx38_xdp_drop: 0
rx38_xdp_redirect: 0
rx38_lro_packets: 0
rx38_lro_bytes: 0
rx38_ecn_mark: 0
rx38_removed_vlan_packets: 0
rx38_wqe_err: 0
rx38_mpwqe_filler_cqes: 0
rx38_mpwqe_filler_strides: 0
rx38_buff_alloc_err: 0
rx38_cqe_compress_blks: 0
rx38_cqe_compress_pkts: 0
rx38_page_reuse: 0
rx38_cache_reuse: 0
rx38_cache_full: 0
rx38_cache_empty: 2560
rx38_cache_busy: 0
rx38_cache_waive: 0
rx38_congst_umr: 0
rx38_arfs_err: 0
rx38_xdp_tx_xmit: 0
rx38_xdp_tx_full: 0
rx38_xdp_tx_err: 0
rx38_xdp_tx_cqes: 0
rx39_packets: 0
rx39_bytes: 0
rx39_csum_complete: 0
rx39_csum_unnecessary: 0
rx39_csum_unnecessary_inner: 0
rx39_csum_none: 0
rx39_xdp_drop: 0
rx39_xdp_redirect: 0
rx39_lro_packets: 0
rx39_lro_bytes: 0
rx39_ecn_mark: 0
rx39_removed_vlan_packets: 0
rx39_wqe_err: 0
rx39_mpwqe_filler_cqes: 0
rx39_mpwqe_filler_strides: 0
rx39_buff_alloc_err: 0
rx39_cqe_compress_blks: 0
rx39_cqe_compress_pkts: 0
rx39_page_reuse: 0
rx39_cache_reuse: 0
rx39_cache_full: 0
rx39_cache_empty: 2560
rx39_cache_busy: 0
rx39_cache_waive: 0
rx39_congst_umr: 0
rx39_arfs_err: 0
rx39_xdp_tx_xmit: 0
rx39_xdp_tx_full: 0
rx39_xdp_tx_err: 0
rx39_xdp_tx_cqes: 0
rx40_packets: 0
rx40_bytes: 0
rx40_csum_complete: 0
rx40_csum_unnecessary: 0
rx40_csum_unnecessary_inner: 0
rx40_csum_none: 0
rx40_xdp_drop: 0
rx40_xdp_redirect: 0
rx40_lro_packets: 0
rx40_lro_bytes: 0
rx40_ecn_mark: 0
rx40_removed_vlan_packets: 0
rx40_wqe_err: 0
rx40_mpwqe_filler_cqes: 0
rx40_mpwqe_filler_strides: 0
rx40_buff_alloc_err: 0
rx40_cqe_compress_blks: 0
rx40_cqe_compress_pkts: 0
rx40_page_reuse: 0
rx40_cache_reuse: 0
rx40_cache_full: 0
rx40_cache_empty: 2560
rx40_cache_busy: 0
rx40_cache_waive: 0
rx40_congst_umr: 0
rx40_arfs_err: 0
rx40_xdp_tx_xmit: 0
rx40_xdp_tx_full: 0
rx40_xdp_tx_err: 0
rx40_xdp_tx_cqes: 0
rx41_packets: 0
rx41_bytes: 0
rx41_csum_complete: 0
rx41_csum_unnecessary: 0
rx41_csum_unnecessary_inner: 0
rx41_csum_none: 0
rx41_xdp_drop: 0
rx41_xdp_redirect: 0
rx41_lro_packets: 0
rx41_lro_bytes: 0
rx41_ecn_mark: 0
rx41_removed_vlan_packets: 0
rx41_wqe_err: 0
rx41_mpwqe_filler_cqes: 0
rx41_mpwqe_filler_strides: 0
rx41_buff_alloc_err: 0
rx41_cqe_compress_blks: 0
rx41_cqe_compress_pkts: 0
rx41_page_reuse: 0
rx41_cache_reuse: 0
rx41_cache_full: 0
rx41_cache_empty: 2560
rx41_cache_busy: 0
rx41_cache_waive: 0
rx41_congst_umr: 0
rx41_arfs_err: 0
rx41_xdp_tx_xmit: 0
rx41_xdp_tx_full: 0
rx41_xdp_tx_err: 0
rx41_xdp_tx_cqes: 0
rx42_packets: 0
rx42_bytes: 0
rx42_csum_complete: 0
rx42_csum_unnecessary: 0
rx42_csum_unnecessary_inner: 0
rx42_csum_none: 0
rx42_xdp_drop: 0
rx42_xdp_redirect: 0
rx42_lro_packets: 0
rx42_lro_bytes: 0
rx42_ecn_mark: 0
rx42_removed_vlan_packets: 0
rx42_wqe_err: 0
rx42_mpwqe_filler_cqes: 0
rx42_mpwqe_filler_strides: 0
rx42_buff_alloc_err: 0
rx42_cqe_compress_blks: 0
rx42_cqe_compress_pkts: 0
rx42_page_reuse: 0
rx42_cache_reuse: 0
rx42_cache_full: 0
rx42_cache_empty: 2560
rx42_cache_busy: 0
rx42_cache_waive: 0
rx42_congst_umr: 0
rx42_arfs_err: 0
rx42_xdp_tx_xmit: 0
rx42_xdp_tx_full: 0
rx42_xdp_tx_err: 0
rx42_xdp_tx_cqes: 0
rx43_packets: 0
rx43_bytes: 0
rx43_csum_complete: 0
rx43_csum_unnecessary: 0
rx43_csum_unnecessary_inner: 0
rx43_csum_none: 0
rx43_xdp_drop: 0
rx43_xdp_redirect: 0
rx43_lro_packets: 0
rx43_lro_bytes: 0
rx43_ecn_mark: 0
rx43_removed_vlan_packets: 0
rx43_wqe_err: 0
rx43_mpwqe_filler_cqes: 0
rx43_mpwqe_filler_strides: 0
rx43_buff_alloc_err: 0
rx43_cqe_compress_blks: 0
rx43_cqe_compress_pkts: 0
rx43_page_reuse: 0
rx43_cache_reuse: 0
rx43_cache_full: 0
rx43_cache_empty: 2560
rx43_cache_busy: 0
rx43_cache_waive: 0
rx43_congst_umr: 0
rx43_arfs_err: 0
rx43_xdp_tx_xmit: 0
rx43_xdp_tx_full: 0
rx43_xdp_tx_err: 0
rx43_xdp_tx_cqes: 0
rx44_packets: 0
rx44_bytes: 0
rx44_csum_complete: 0
rx44_csum_unnecessary: 0
rx44_csum_unnecessary_inner: 0
rx44_csum_none: 0
rx44_xdp_drop: 0
rx44_xdp_redirect: 0
rx44_lro_packets: 0
rx44_lro_bytes: 0
rx44_ecn_mark: 0
rx44_removed_vlan_packets: 0
rx44_wqe_err: 0
rx44_mpwqe_filler_cqes: 0
rx44_mpwqe_filler_strides: 0
rx44_buff_alloc_err: 0
rx44_cqe_compress_blks: 0
rx44_cqe_compress_pkts: 0
rx44_page_reuse: 0
rx44_cache_reuse: 0
rx44_cache_full: 0
rx44_cache_empty: 2560
rx44_cache_busy: 0
rx44_cache_waive: 0
rx44_congst_umr: 0
rx44_arfs_err: 0
rx44_xdp_tx_xmit: 0
rx44_xdp_tx_full: 0
rx44_xdp_tx_err: 0
rx44_xdp_tx_cqes: 0
rx45_packets: 0
rx45_bytes: 0
rx45_csum_complete: 0
rx45_csum_unnecessary: 0
rx45_csum_unnecessary_inner: 0
rx45_csum_none: 0
rx45_xdp_drop: 0
rx45_xdp_redirect: 0
rx45_lro_packets: 0
rx45_lro_bytes: 0
rx45_ecn_mark: 0
rx45_removed_vlan_packets: 0
rx45_wqe_err: 0
rx45_mpwqe_filler_cqes: 0
rx45_mpwqe_filler_strides: 0
rx45_buff_alloc_err: 0
rx45_cqe_compress_blks: 0
rx45_cqe_compress_pkts: 0
rx45_page_reuse: 0
rx45_cache_reuse: 0
rx45_cache_full: 0
rx45_cache_empty: 2560
rx45_cache_busy: 0
rx45_cache_waive: 0
rx45_congst_umr: 0
rx45_arfs_err: 0
rx45_xdp_tx_xmit: 0
rx45_xdp_tx_full: 0
rx45_xdp_tx_err: 0
rx45_xdp_tx_cqes: 0
rx46_packets: 0
rx46_bytes: 0
rx46_csum_complete: 0
rx46_csum_unnecessary: 0
rx46_csum_unnecessary_inner: 0
rx46_csum_none: 0
rx46_xdp_drop: 0
rx46_xdp_redirect: 0
rx46_lro_packets: 0
rx46_lro_bytes: 0
rx46_ecn_mark: 0
rx46_removed_vlan_packets: 0
rx46_wqe_err: 0
rx46_mpwqe_filler_cqes: 0
rx46_mpwqe_filler_strides: 0
rx46_buff_alloc_err: 0
rx46_cqe_compress_blks: 0
rx46_cqe_compress_pkts: 0
rx46_page_reuse: 0
rx46_cache_reuse: 0
rx46_cache_full: 0
rx46_cache_empty: 2560
rx46_cache_busy: 0
rx46_cache_waive: 0
rx46_congst_umr: 0
rx46_arfs_err: 0
rx46_xdp_tx_xmit: 0
rx46_xdp_tx_full: 0
rx46_xdp_tx_err: 0
rx46_xdp_tx_cqes: 0
rx47_packets: 0
rx47_bytes: 0
rx47_csum_complete: 0
rx47_csum_unnecessary: 0
rx47_csum_unnecessary_inner: 0
rx47_csum_none: 0
rx47_xdp_drop: 0
rx47_xdp_redirect: 0
rx47_lro_packets: 0
rx47_lro_bytes: 0
rx47_ecn_mark: 0
rx47_removed_vlan_packets: 0
rx47_wqe_err: 0
rx47_mpwqe_filler_cqes: 0
rx47_mpwqe_filler_strides: 0
rx47_buff_alloc_err: 0
rx47_cqe_compress_blks: 0
rx47_cqe_compress_pkts: 0
rx47_page_reuse: 0
rx47_cache_reuse: 0
rx47_cache_full: 0
rx47_cache_empty: 2560
rx47_cache_busy: 0
rx47_cache_waive: 0
rx47_congst_umr: 0
rx47_arfs_err: 0
rx47_xdp_tx_xmit: 0
rx47_xdp_tx_full: 0
rx47_xdp_tx_err: 0
rx47_xdp_tx_cqes: 0
rx48_packets: 0
rx48_bytes: 0
rx48_csum_complete: 0
rx48_csum_unnecessary: 0
rx48_csum_unnecessary_inner: 0
rx48_csum_none: 0
rx48_xdp_drop: 0
rx48_xdp_redirect: 0
rx48_lro_packets: 0
rx48_lro_bytes: 0
rx48_ecn_mark: 0
rx48_removed_vlan_packets: 0
rx48_wqe_err: 0
rx48_mpwqe_filler_cqes: 0
rx48_mpwqe_filler_strides: 0
rx48_buff_alloc_err: 0
rx48_cqe_compress_blks: 0
rx48_cqe_compress_pkts: 0
rx48_page_reuse: 0
rx48_cache_reuse: 0
rx48_cache_full: 0
rx48_cache_empty: 2560
rx48_cache_busy: 0
rx48_cache_waive: 0
rx48_congst_umr: 0
rx48_arfs_err: 0
rx48_xdp_tx_xmit: 0
rx48_xdp_tx_full: 0
rx48_xdp_tx_err: 0
rx48_xdp_tx_cqes: 0
rx49_packets: 0
rx49_bytes: 0
rx49_csum_complete: 0
rx49_csum_unnecessary: 0
rx49_csum_unnecessary_inner: 0
rx49_csum_none: 0
rx49_xdp_drop: 0
rx49_xdp_redirect: 0
rx49_lro_packets: 0
rx49_lro_bytes: 0
rx49_ecn_mark: 0
rx49_removed_vlan_packets: 0
rx49_wqe_err: 0
rx49_mpwqe_filler_cqes: 0
rx49_mpwqe_filler_strides: 0
rx49_buff_alloc_err: 0
rx49_cqe_compress_blks: 0
rx49_cqe_compress_pkts: 0
rx49_page_reuse: 0
rx49_cache_reuse: 0
rx49_cache_full: 0
rx49_cache_empty: 2560
rx49_cache_busy: 0
rx49_cache_waive: 0
rx49_congst_umr: 0
rx49_arfs_err: 0
rx49_xdp_tx_xmit: 0
rx49_xdp_tx_full: 0
rx49_xdp_tx_err: 0
rx49_xdp_tx_cqes: 0
rx50_packets: 0
rx50_bytes: 0
rx50_csum_complete: 0
rx50_csum_unnecessary: 0
rx50_csum_unnecessary_inner: 0
rx50_csum_none: 0
rx50_xdp_drop: 0
rx50_xdp_redirect: 0
rx50_lro_packets: 0
rx50_lro_bytes: 0
rx50_ecn_mark: 0
rx50_removed_vlan_packets: 0
rx50_wqe_err: 0
rx50_mpwqe_filler_cqes: 0
rx50_mpwqe_filler_strides: 0
rx50_buff_alloc_err: 0
rx50_cqe_compress_blks: 0
rx50_cqe_compress_pkts: 0
rx50_page_reuse: 0
rx50_cache_reuse: 0
rx50_cache_full: 0
rx50_cache_empty: 2560
rx50_cache_busy: 0
rx50_cache_waive: 0
rx50_congst_umr: 0
rx50_arfs_err: 0
rx50_xdp_tx_xmit: 0
rx50_xdp_tx_full: 0
rx50_xdp_tx_err: 0
rx50_xdp_tx_cqes: 0
rx51_packets: 0
rx51_bytes: 0
rx51_csum_complete: 0
rx51_csum_unnecessary: 0
rx51_csum_unnecessary_inner: 0
rx51_csum_none: 0
rx51_xdp_drop: 0
rx51_xdp_redirect: 0
rx51_lro_packets: 0
rx51_lro_bytes: 0
rx51_ecn_mark: 0
rx51_removed_vlan_packets: 0
rx51_wqe_err: 0
rx51_mpwqe_filler_cqes: 0
rx51_mpwqe_filler_strides: 0
rx51_buff_alloc_err: 0
rx51_cqe_compress_blks: 0
rx51_cqe_compress_pkts: 0
rx51_page_reuse: 0
rx51_cache_reuse: 0
rx51_cache_full: 0
rx51_cache_empty: 2560
rx51_cache_busy: 0
rx51_cache_waive: 0
rx51_congst_umr: 0
rx51_arfs_err: 0
rx51_xdp_tx_xmit: 0
rx51_xdp_tx_full: 0
rx51_xdp_tx_err: 0
rx51_xdp_tx_cqes: 0
rx52_packets: 0
rx52_bytes: 0
rx52_csum_complete: 0
rx52_csum_unnecessary: 0
rx52_csum_unnecessary_inner: 0
rx52_csum_none: 0
rx52_xdp_drop: 0
rx52_xdp_redirect: 0
rx52_lro_packets: 0
rx52_lro_bytes: 0
rx52_ecn_mark: 0
rx52_removed_vlan_packets: 0
rx52_wqe_err: 0
rx52_mpwqe_filler_cqes: 0
rx52_mpwqe_filler_strides: 0
rx52_buff_alloc_err: 0
rx52_cqe_compress_blks: 0
rx52_cqe_compress_pkts: 0
rx52_page_reuse: 0
rx52_cache_reuse: 0
rx52_cache_full: 0
rx52_cache_empty: 2560
rx52_cache_busy: 0
rx52_cache_waive: 0
rx52_congst_umr: 0
rx52_arfs_err: 0
rx52_xdp_tx_xmit: 0
rx52_xdp_tx_full: 0
rx52_xdp_tx_err: 0
rx52_xdp_tx_cqes: 0
rx53_packets: 0
rx53_bytes: 0
rx53_csum_complete: 0
rx53_csum_unnecessary: 0
rx53_csum_unnecessary_inner: 0
rx53_csum_none: 0
rx53_xdp_drop: 0
rx53_xdp_redirect: 0
rx53_lro_packets: 0
rx53_lro_bytes: 0
rx53_ecn_mark: 0
rx53_removed_vlan_packets: 0
rx53_wqe_err: 0
rx53_mpwqe_filler_cqes: 0
rx53_mpwqe_filler_strides: 0
rx53_buff_alloc_err: 0
rx53_cqe_compress_blks: 0
rx53_cqe_compress_pkts: 0
rx53_page_reuse: 0
rx53_cache_reuse: 0
rx53_cache_full: 0
rx53_cache_empty: 2560
rx53_cache_busy: 0
rx53_cache_waive: 0
rx53_congst_umr: 0
rx53_arfs_err: 0
rx53_xdp_tx_xmit: 0
rx53_xdp_tx_full: 0
rx53_xdp_tx_err: 0
rx53_xdp_tx_cqes: 0
rx54_packets: 0
rx54_bytes: 0
rx54_csum_complete: 0
rx54_csum_unnecessary: 0
rx54_csum_unnecessary_inner: 0
rx54_csum_none: 0
rx54_xdp_drop: 0
rx54_xdp_redirect: 0
rx54_lro_packets: 0
rx54_lro_bytes: 0
rx54_ecn_mark: 0
rx54_removed_vlan_packets: 0
rx54_wqe_err: 0
rx54_mpwqe_filler_cqes: 0
rx54_mpwqe_filler_strides: 0
rx54_buff_alloc_err: 0
rx54_cqe_compress_blks: 0
rx54_cqe_compress_pkts: 0
rx54_page_reuse: 0
rx54_cache_reuse: 0
rx54_cache_full: 0
rx54_cache_empty: 2560
rx54_cache_busy: 0
rx54_cache_waive: 0
rx54_congst_umr: 0
rx54_arfs_err: 0
rx54_xdp_tx_xmit: 0
rx54_xdp_tx_full: 0
rx54_xdp_tx_err: 0
rx54_xdp_tx_cqes: 0
rx55_packets: 0
rx55_bytes: 0
rx55_csum_complete: 0
rx55_csum_unnecessary: 0
rx55_csum_unnecessary_inner: 0
rx55_csum_none: 0
rx55_xdp_drop: 0
rx55_xdp_redirect: 0
rx55_lro_packets: 0
rx55_lro_bytes: 0
rx55_ecn_mark: 0
rx55_removed_vlan_packets: 0
rx55_wqe_err: 0
rx55_mpwqe_filler_cqes: 0
rx55_mpwqe_filler_strides: 0
rx55_buff_alloc_err: 0
rx55_cqe_compress_blks: 0
rx55_cqe_compress_pkts: 0
rx55_page_reuse: 0
rx55_cache_reuse: 0
rx55_cache_full: 0
rx55_cache_empty: 2560
rx55_cache_busy: 0
rx55_cache_waive: 0
rx55_congst_umr: 0
rx55_arfs_err: 0
rx55_xdp_tx_xmit: 0
rx55_xdp_tx_full: 0
rx55_xdp_tx_err: 0
rx55_xdp_tx_cqes: 0
tx0_packets: 24512439668
tx0_bytes: 15287569052791
tx0_tso_packets: 1536157106
tx0_tso_bytes: 8571753637944
tx0_tso_inner_packets: 0
tx0_tso_inner_bytes: 0
tx0_csum_partial: 2132156117
tx0_csum_partial_inner: 0
tx0_added_vlan_packets: 19906601448
tx0_nop: 308098536
tx0_csum_none: 17774445331
tx0_stopped: 19625
tx0_dropped: 0
tx0_xmit_more: 67864870
tx0_recover: 0
tx0_cqes: 19838744246
tx0_wake: 19624
tx0_cqe_err: 0
tx1_packets: 22598557053
tx1_bytes: 13568850145010
tx1_tso_packets: 1369529475
tx1_tso_bytes: 7661777265382
tx1_tso_inner_packets: 0
tx1_tso_inner_bytes: 0
tx1_csum_partial: 1884639496
tx1_csum_partial_inner: 0
tx1_added_vlan_packets: 18468333696
tx1_nop: 281301783
tx1_csum_none: 16583694200
tx1_stopped: 19457
tx1_dropped: 0
tx1_xmit_more: 55170875
tx1_recover: 0
tx1_cqes: 18413169824
tx1_wake: 19455
tx1_cqe_err: 0
tx2_packets: 22821611433
tx2_bytes: 13752535163683
tx2_tso_packets: 1396978825
tx2_tso_bytes: 7774704508463
tx2_tso_inner_packets: 0
tx2_tso_inner_bytes: 0
tx2_csum_partial: 1897834558
tx2_csum_partial_inner: 0
tx2_added_vlan_packets: 18641958085
tx2_nop: 286934891
tx2_csum_none: 16744123527
tx2_stopped: 13214
tx2_dropped: 0
tx2_xmit_more: 61749446
tx2_recover: 0
tx2_cqes: 18580215654
tx2_wake: 13214
tx2_cqe_err: 0
tx3_packets: 22580809948
tx3_bytes: 13730542936609
tx3_tso_packets: 1370434579
tx3_tso_bytes: 7605636711455
tx3_tso_inner_packets: 0
tx3_tso_inner_bytes: 0
tx3_csum_partial: 1865573748
tx3_csum_partial_inner: 0
tx3_added_vlan_packets: 18491873644
tx3_nop: 281195875
tx3_csum_none: 16626299896
tx3_stopped: 12542
tx3_dropped: 0
tx3_xmit_more: 57681647
tx3_recover: 0
tx3_cqes: 18434198757
tx3_wake: 12540
tx3_cqe_err: 0
tx4_packets: 27801801208
tx4_bytes: 17058453171137
tx4_tso_packets: 1740500105
tx4_tso_bytes: 9474905622036
tx4_tso_inner_packets: 0
tx4_tso_inner_bytes: 0
tx4_csum_partial: 2279225376
tx4_csum_partial_inner: 0
tx4_added_vlan_packets: 22744081633
tx4_nop: 349753979
tx4_csum_none: 20464856257
tx4_stopped: 14816
tx4_dropped: 0
tx4_xmit_more: 65469322
tx4_recover: 0
tx4_cqes: 22678618972
tx4_wake: 14816
tx4_cqe_err: 0
tx5_packets: 25099783024
tx5_bytes: 14917740698381
tx5_tso_packets: 1512988013
tx5_tso_bytes: 8571208921023
tx5_tso_inner_packets: 0
tx5_tso_inner_bytes: 0
tx5_csum_partial: 2078498561
tx5_csum_partial_inner: 0
tx5_added_vlan_packets: 20465533760
tx5_nop: 312614719
tx5_csum_none: 18387035199
tx5_stopped: 4605
tx5_dropped: 0
tx5_xmit_more: 64188936
tx5_recover: 0
tx5_cqes: 20401350718
tx5_wake: 4604
tx5_cqe_err: 0
tx6_packets: 25025504896
tx6_bytes: 14908021946070
tx6_tso_packets: 1515718977
tx6_tso_bytes: 8511442522461
tx6_tso_inner_packets: 0
tx6_tso_inner_bytes: 0
tx6_csum_partial: 2056378610
tx6_csum_partial_inner: 0
tx6_added_vlan_packets: 20434066400
tx6_nop: 310594020
tx6_csum_none: 18377687790
tx6_stopped: 15234
tx6_dropped: 0
tx6_xmit_more: 61130422
tx6_recover: 0
tx6_cqes: 20372943611
tx6_wake: 15234
tx6_cqe_err: 0
tx7_packets: 25457096169
tx7_bytes: 15456289446172
tx7_tso_packets: 1553342799
tx7_tso_bytes: 8764550988105
tx7_tso_inner_packets: 0
tx7_tso_inner_bytes: 0
tx7_csum_partial: 2105765233
tx7_csum_partial_inner: 0
tx7_added_vlan_packets: 20720382377
tx7_nop: 319044853
tx7_csum_none: 18614617145
tx7_stopped: 18745
tx7_dropped: 0
tx7_xmit_more: 57050107
tx7_recover: 0
tx7_cqes: 20663340775
tx7_wake: 18746
tx7_cqe_err: 0
tx8_packets: 25389771649
tx8_bytes: 15225503883962
tx8_tso_packets: 1563367648
tx8_tso_bytes: 8710384514258
tx8_tso_inner_packets: 0
tx8_tso_inner_bytes: 0
tx8_csum_partial: 2106586634
tx8_csum_partial_inner: 0
tx8_added_vlan_packets: 20704676274
tx8_nop: 318149261
tx8_csum_none: 18598089640
tx8_stopped: 4733
tx8_dropped: 0
tx8_xmit_more: 61014317
tx8_recover: 0
tx8_cqes: 20643667301
tx8_wake: 4735
tx8_cqe_err: 0
tx9_packets: 25521500166
tx9_bytes: 15302293145755
tx9_tso_packets: 1546316697
tx9_tso_bytes: 8770688145926
tx9_tso_inner_packets: 0
tx9_tso_inner_bytes: 0
tx9_csum_partial: 2097652880
tx9_csum_partial_inner: 0
tx9_added_vlan_packets: 20778408432
tx9_nop: 318538543
tx9_csum_none: 18680755556
tx9_stopped: 16118
tx9_dropped: 0
tx9_xmit_more: 68509728
tx9_recover: 0
tx9_cqes: 20709906498
tx9_wake: 16118
tx9_cqe_err: 0
tx10_packets: 25451605829
tx10_bytes: 15386896170792
tx10_tso_packets: 1576473520
tx10_tso_bytes: 8880888676383
tx10_tso_inner_packets: 0
tx10_tso_inner_bytes: 0
tx10_csum_partial: 2129796141
tx10_csum_partial_inner: 0
tx10_added_vlan_packets: 20659622590
tx10_nop: 319117433
tx10_csum_none: 18529826450
tx10_stopped: 20187
tx10_dropped: 0
tx10_xmit_more: 58892184
tx10_recover: 0
tx10_cqes: 20600737739
tx10_wake: 20188
tx10_cqe_err: 0
tx11_packets: 27008919793
tx11_bytes: 16587719213058
tx11_tso_packets: 1734884654
tx11_tso_bytes: 9475681471870
tx11_tso_inner_packets: 0
tx11_tso_inner_bytes: 0
tx11_csum_partial: 2296162292
tx11_csum_partial_inner: 0
tx11_added_vlan_packets: 21943096263
tx11_nop: 344188182
tx11_csum_none: 19646933971
tx11_stopped: 9703
tx11_dropped: 0
tx11_xmit_more: 66530718
tx11_recover: 0
tx11_cqes: 21876571667
tx11_wake: 9704
tx11_cqe_err: 0
tx12_packets: 25969493269
tx12_bytes: 15980767963416
tx12_tso_packets: 1671396456
tx12_tso_bytes: 9268973672821
tx12_tso_inner_packets: 0
tx12_tso_inner_bytes: 0
tx12_csum_partial: 2243809182
tx12_csum_partial_inner: 0
tx12_added_vlan_packets: 20980642456
tx12_nop: 330241007
tx12_csum_none: 18736833276
tx12_stopped: 10341
tx12_dropped: 0
tx12_xmit_more: 57834100
tx12_recover: 0
tx12_cqes: 20922815079
tx12_wake: 10342
tx12_cqe_err: 0
tx13_packets: 25332762261
tx13_bytes: 15353213283280
tx13_tso_packets: 1577433599
tx13_tso_bytes: 8785240284281
tx13_tso_inner_packets: 0
tx13_tso_inner_bytes: 0
tx13_csum_partial: 2110640515
tx13_csum_partial_inner: 0
tx13_added_vlan_packets: 20605670910
tx13_nop: 319805741
tx13_csum_none: 18495030395
tx13_stopped: 7006
tx13_dropped: 0
tx13_xmit_more: 58314402
tx13_recover: 0
tx13_cqes: 20547362770
tx13_wake: 7008
tx13_cqe_err: 0
tx14_packets: 26333743548
tx14_bytes: 16070719060573
tx14_tso_packets: 1677922970
tx14_tso_bytes: 9240299765487
tx14_tso_inner_packets: 0
tx14_tso_inner_bytes: 0
tx14_csum_partial: 2215668906
tx14_csum_partial_inner: 0
tx14_added_vlan_packets: 21384410786
tx14_nop: 332734939
tx14_csum_none: 19168741880
tx14_stopped: 13160
tx14_dropped: 0
tx14_xmit_more: 57650391
tx14_recover: 0
tx14_cqes: 21326767783
tx14_wake: 13161
tx14_cqe_err: 0
tx15_packets: 26824968971
tx15_bytes: 16687994233452
tx15_tso_packets: 1755745052
tx15_tso_bytes: 9533814012441
tx15_tso_inner_packets: 0
tx15_tso_inner_bytes: 0
tx15_csum_partial: 2304778064
tx15_csum_partial_inner: 0
tx15_added_vlan_packets: 21740906107
tx15_nop: 344143287
tx15_csum_none: 19436128058
tx15_stopped: 75
tx15_dropped: 0
tx15_xmit_more: 63325832
tx15_recover: 0
tx15_cqes: 21677585345
tx15_wake: 74
tx15_cqe_err: 0
tx16_packets: 24488158946
tx16_bytes: 15027415004570
tx16_tso_packets: 1559127391
tx16_tso_bytes: 8658691917845
tx16_tso_inner_packets: 0
tx16_tso_inner_bytes: 0
tx16_csum_partial: 2075856395
tx16_csum_partial_inner: 0
tx16_added_vlan_packets: 19835695731
tx16_nop: 308464189
tx16_csum_none: 17759839340
tx16_stopped: 4567
tx16_dropped: 0
tx16_xmit_more: 62631422
tx16_recover: 0
tx16_cqes: 19773070012
tx16_wake: 4568
tx16_cqe_err: 0
tx17_packets: 24700413784
tx17_bytes: 15216529713715
tx17_tso_packets: 1597555108
tx17_tso_bytes: 8773728661243
tx17_tso_inner_packets: 0
tx17_tso_inner_bytes: 0
tx17_csum_partial: 2127177297
tx17_csum_partial_inner: 0
tx17_added_vlan_packets: 20003144561
tx17_nop: 313356918
tx17_csum_none: 17875967264
tx17_stopped: 12572
tx17_dropped: 0
tx17_xmit_more: 62742980
tx17_recover: 0
tx17_cqes: 19940407615
tx17_wake: 12573
tx17_cqe_err: 0
tx18_packets: 24887710046
tx18_bytes: 15245034034664
tx18_tso_packets: 1582550520
tx18_tso_bytes: 8782692335483
tx18_tso_inner_packets: 0
tx18_tso_inner_bytes: 0
tx18_csum_partial: 2084514331
tx18_csum_partial_inner: 0
tx18_added_vlan_packets: 20173879181
tx18_nop: 314818702
tx18_csum_none: 18089364850
tx18_stopped: 21366
tx18_dropped: 0
tx18_xmit_more: 62485819
tx18_recover: 0
tx18_cqes: 20111400935
tx18_wake: 21366
tx18_cqe_err: 0
tx19_packets: 24831057648
tx19_bytes: 15164663890576
tx19_tso_packets: 1599135489
tx19_tso_bytes: 8756045449746
tx19_tso_inner_packets: 0
tx19_tso_inner_bytes: 0
tx19_csum_partial: 2119746608
tx19_csum_partial_inner: 0
tx19_added_vlan_packets: 20143573903
tx19_nop: 316966450
tx19_csum_none: 18023827295
tx19_stopped: 11431
tx19_dropped: 0
tx19_xmit_more: 57535904
tx19_recover: 0
tx19_cqes: 20086045325
tx19_wake: 11431
tx19_cqe_err: 0
tx20_packets: 21943735263
tx20_bytes: 13528749492187
tx20_tso_packets: 1390048103
tx20_tso_bytes: 7629058809637
tx20_tso_inner_packets: 0
tx20_tso_inner_bytes: 0
tx20_csum_partial: 1848533941
tx20_csum_partial_inner: 0
tx20_added_vlan_packets: 17861417651
tx20_nop: 276840365
tx20_csum_none: 16012883710
tx20_stopped: 38457
tx20_dropped: 0
tx20_xmit_more: 57042753
tx20_recover: 0
tx20_cqes: 17804384839
tx20_wake: 38457
tx20_cqe_err: 0
tx21_packets: 21476926958
tx21_bytes: 13096410597896
tx21_tso_packets: 1367724090
tx21_tso_bytes: 7568364585127
tx21_tso_inner_packets: 0
tx21_tso_inner_bytes: 0
tx21_csum_partial: 1830570727
tx21_csum_partial_inner: 0
tx21_added_vlan_packets: 17421087814
tx21_nop: 270611519
tx21_csum_none: 15590517087
tx21_stopped: 31213
tx21_dropped: 0
tx21_xmit_more: 60305389
tx21_recover: 0
tx21_cqes: 17360791205
tx21_wake: 31213
tx21_cqe_err: 0
tx22_packets: 21819106444
tx22_bytes: 13492871887100
tx22_tso_packets: 1387002018
tx22_tso_bytes: 7617705727669
tx22_tso_inner_packets: 0
tx22_tso_inner_bytes: 0
tx22_csum_partial: 1853632107
tx22_csum_partial_inner: 0
tx22_added_vlan_packets: 17743255447
tx22_nop: 274820992
tx22_csum_none: 15889623340
tx22_stopped: 24814
tx22_dropped: 0
tx22_xmit_more: 60811304
tx22_recover: 0
tx22_cqes: 17682451111
tx22_wake: 24815
tx22_cqe_err: 0
tx23_packets: 21830455800
tx23_bytes: 13427551902532
tx23_tso_packets: 1388556038
tx23_tso_bytes: 7604040587125
tx23_tso_inner_packets: 0
tx23_tso_inner_bytes: 0
tx23_csum_partial: 1850819694
tx23_csum_partial_inner: 0
tx23_added_vlan_packets: 17761271122
tx23_nop: 275142775
tx23_csum_none: 15910451428
tx23_stopped: 29899
tx23_dropped: 0
tx23_xmit_more: 58924909
tx23_recover: 0
tx23_cqes: 17702355187
tx23_wake: 29898
tx23_cqe_err: 0
tx24_packets: 21961484213
tx24_bytes: 13531373062497
tx24_tso_packets: 1394697504
tx24_tso_bytes: 7663866609308
tx24_tso_inner_packets: 0
tx24_tso_inner_bytes: 0
tx24_csum_partial: 1857072074
tx24_csum_partial_inner: 0
tx24_added_vlan_packets: 17856887568
tx24_nop: 276352855
tx24_csum_none: 15999815494
tx24_stopped: 33924
tx24_dropped: 0
tx24_xmit_more: 63992426
tx24_recover: 0
tx24_cqes: 17792905243
tx24_wake: 33923
tx24_cqe_err: 0
tx25_packets: 21853593838
tx25_bytes: 13357487830519
tx25_tso_packets: 1398822411
tx25_tso_bytes: 7691191518838
tx25_tso_inner_packets: 0
tx25_tso_inner_bytes: 0
tx25_csum_partial: 1869483109
tx25_csum_partial_inner: 0
tx25_added_vlan_packets: 17734634614
tx25_nop: 276327643
tx25_csum_none: 15865151505
tx25_stopped: 38651
tx25_dropped: 0
tx25_xmit_more: 56410535
tx25_recover: 0
tx25_cqes: 17678234537
tx25_wake: 38650
tx25_cqe_err: 0
tx26_packets: 21480261205
tx26_bytes: 13148973015935
tx26_tso_packets: 1348132284
tx26_tso_bytes: 7523489481775
tx26_tso_inner_packets: 0
tx26_tso_inner_bytes: 0
tx26_csum_partial: 1839740745
tx26_csum_partial_inner: 0
tx26_added_vlan_packets: 17430592911
tx26_nop: 270367836
tx26_csum_none: 15590852166
tx26_stopped: 34044
tx26_dropped: 0
tx26_xmit_more: 59870114
tx26_recover: 0
tx26_cqes: 17370736612
tx26_wake: 34043
tx26_cqe_err: 0
tx27_packets: 22694273108
tx27_bytes: 14135473431004
tx27_tso_packets: 1418371875
tx27_tso_bytes: 7784842263038
tx27_tso_inner_packets: 0
tx27_tso_inner_bytes: 0
tx27_csum_partial: 1919170584
tx27_csum_partial_inner: 0
tx27_added_vlan_packets: 18520826023
tx27_nop: 286296272
tx27_csum_none: 16601655439
tx27_stopped: 38125
tx27_dropped: 0
tx27_xmit_more: 72749775
tx27_recover: 0
tx27_cqes: 18448090270
tx27_wake: 38127
tx27_cqe_err: 0
tx28_packets: 0
tx28_bytes: 0
tx28_tso_packets: 0
tx28_tso_bytes: 0
tx28_tso_inner_packets: 0
tx28_tso_inner_bytes: 0
tx28_csum_partial: 0
tx28_csum_partial_inner: 0
tx28_added_vlan_packets: 0
tx28_nop: 0
tx28_csum_none: 0
tx28_stopped: 0
tx28_dropped: 0
tx28_xmit_more: 0
tx28_recover: 0
tx28_cqes: 0
tx28_wake: 0
tx28_cqe_err: 0
tx29_packets: 3
tx29_bytes: 266
tx29_tso_packets: 0
tx29_tso_bytes: 0
tx29_tso_inner_packets: 0
tx29_tso_inner_bytes: 0
tx29_csum_partial: 0
tx29_csum_partial_inner: 0
tx29_added_vlan_packets: 0
tx29_nop: 0
tx29_csum_none: 3
tx29_stopped: 0
tx29_dropped: 0
tx29_xmit_more: 1
tx29_recover: 0
tx29_cqes: 2
tx29_wake: 0
tx29_cqe_err: 0
tx30_packets: 0
tx30_bytes: 0
tx30_tso_packets: 0
tx30_tso_bytes: 0
tx30_tso_inner_packets: 0
tx30_tso_inner_bytes: 0
tx30_csum_partial: 0
tx30_csum_partial_inner: 0
tx30_added_vlan_packets: 0
tx30_nop: 0
tx30_csum_none: 0
tx30_stopped: 0
tx30_dropped: 0
tx30_xmit_more: 0
tx30_recover: 0
tx30_cqes: 0
tx30_wake: 0
tx30_cqe_err: 0
tx31_packets: 0
tx31_bytes: 0
tx31_tso_packets: 0
tx31_tso_bytes: 0
tx31_tso_inner_packets: 0
tx31_tso_inner_bytes: 0
tx31_csum_partial: 0
tx31_csum_partial_inner: 0
tx31_added_vlan_packets: 0
tx31_nop: 0
tx31_csum_none: 0
tx31_stopped: 0
tx31_dropped: 0
tx31_xmit_more: 0
tx31_recover: 0
tx31_cqes: 0
tx31_wake: 0
tx31_cqe_err: 0
tx32_packets: 0
tx32_bytes: 0
tx32_tso_packets: 0
tx32_tso_bytes: 0
tx32_tso_inner_packets: 0
tx32_tso_inner_bytes: 0
tx32_csum_partial: 0
tx32_csum_partial_inner: 0
tx32_added_vlan_packets: 0
tx32_nop: 0
tx32_csum_none: 0
tx32_stopped: 0
tx32_dropped: 0
tx32_xmit_more: 0
tx32_recover: 0
tx32_cqes: 0
tx32_wake: 0
tx32_cqe_err: 0
tx33_packets: 0
tx33_bytes: 0
tx33_tso_packets: 0
tx33_tso_bytes: 0
tx33_tso_inner_packets: 0
tx33_tso_inner_bytes: 0
tx33_csum_partial: 0
tx33_csum_partial_inner: 0
tx33_added_vlan_packets: 0
tx33_nop: 0
tx33_csum_none: 0
tx33_stopped: 0
tx33_dropped: 0
tx33_xmit_more: 0
tx33_recover: 0
tx33_cqes: 0
tx33_wake: 0
tx33_cqe_err: 0
tx34_packets: 0
tx34_bytes: 0
tx34_tso_packets: 0
tx34_tso_bytes: 0
tx34_tso_inner_packets: 0
tx34_tso_inner_bytes: 0
tx34_csum_partial: 0
tx34_csum_partial_inner: 0
tx34_added_vlan_packets: 0
tx34_nop: 0
tx34_csum_none: 0
tx34_stopped: 0
tx34_dropped: 0
tx34_xmit_more: 0
tx34_recover: 0
tx34_cqes: 0
tx34_wake: 0
tx34_cqe_err: 0
tx35_packets: 0
tx35_bytes: 0
tx35_tso_packets: 0
tx35_tso_bytes: 0
tx35_tso_inner_packets: 0
tx35_tso_inner_bytes: 0
tx35_csum_partial: 0
tx35_csum_partial_inner: 0
tx35_added_vlan_packets: 0
tx35_nop: 0
tx35_csum_none: 0
tx35_stopped: 0
tx35_dropped: 0
tx35_xmit_more: 0
tx35_recover: 0
tx35_cqes: 0
tx35_wake: 0
tx35_cqe_err: 0
tx36_packets: 0
tx36_bytes: 0
tx36_tso_packets: 0
tx36_tso_bytes: 0
tx36_tso_inner_packets: 0
tx36_tso_inner_bytes: 0
tx36_csum_partial: 0
tx36_csum_partial_inner: 0
tx36_added_vlan_packets: 0
tx36_nop: 0
tx36_csum_none: 0
tx36_stopped: 0
tx36_dropped: 0
tx36_xmit_more: 0
tx36_recover: 0
tx36_cqes: 0
tx36_wake: 0
tx36_cqe_err: 0
tx37_packets: 0
tx37_bytes: 0
tx37_tso_packets: 0
tx37_tso_bytes: 0
tx37_tso_inner_packets: 0
tx37_tso_inner_bytes: 0
tx37_csum_partial: 0
tx37_csum_partial_inner: 0
tx37_added_vlan_packets: 0
tx37_nop: 0
tx37_csum_none: 0
tx37_stopped: 0
tx37_dropped: 0
tx37_xmit_more: 0
tx37_recover: 0
tx37_cqes: 0
tx37_wake: 0
tx37_cqe_err: 0
tx38_packets: 0
tx38_bytes: 0
tx38_tso_packets: 0
tx38_tso_bytes: 0
tx38_tso_inner_packets: 0
tx38_tso_inner_bytes: 0
tx38_csum_partial: 0
tx38_csum_partial_inner: 0
tx38_added_vlan_packets: 0
tx38_nop: 0
tx38_csum_none: 0
tx38_stopped: 0
tx38_dropped: 0
tx38_xmit_more: 0
tx38_recover: 0
tx38_cqes: 0
tx38_wake: 0
tx38_cqe_err: 0
tx39_packets: 0
tx39_bytes: 0
tx39_tso_packets: 0
tx39_tso_bytes: 0
tx39_tso_inner_packets: 0
tx39_tso_inner_bytes: 0
tx39_csum_partial: 0
tx39_csum_partial_inner: 0
tx39_added_vlan_packets: 0
tx39_nop: 0
tx39_csum_none: 0
tx39_stopped: 0
tx39_dropped: 0
tx39_xmit_more: 0
tx39_recover: 0
tx39_cqes: 0
tx39_wake: 0
tx39_cqe_err: 0
tx40_packets: 0
tx40_bytes: 0
tx40_tso_packets: 0
tx40_tso_bytes: 0
tx40_tso_inner_packets: 0
tx40_tso_inner_bytes: 0
tx40_csum_partial: 0
tx40_csum_partial_inner: 0
tx40_added_vlan_packets: 0
tx40_nop: 0
tx40_csum_none: 0
tx40_stopped: 0
tx40_dropped: 0
tx40_xmit_more: 0
tx40_recover: 0
tx40_cqes: 0
tx40_wake: 0
tx40_cqe_err: 0
tx41_packets: 0
tx41_bytes: 0
tx41_tso_packets: 0
tx41_tso_bytes: 0
tx41_tso_inner_packets: 0
tx41_tso_inner_bytes: 0
tx41_csum_partial: 0
tx41_csum_partial_inner: 0
tx41_added_vlan_packets: 0
tx41_nop: 0
tx41_csum_none: 0
tx41_stopped: 0
tx41_dropped: 0
tx41_xmit_more: 0
tx41_recover: 0
tx41_cqes: 0
tx41_wake: 0
tx41_cqe_err: 0
tx42_packets: 0
tx42_bytes: 0
tx42_tso_packets: 0
tx42_tso_bytes: 0
tx42_tso_inner_packets: 0
tx42_tso_inner_bytes: 0
tx42_csum_partial: 0
tx42_csum_partial_inner: 0
tx42_added_vlan_packets: 0
tx42_nop: 0
tx42_csum_none: 0
tx42_stopped: 0
tx42_dropped: 0
tx42_xmit_more: 0
tx42_recover: 0
tx42_cqes: 0
tx42_wake: 0
tx42_cqe_err: 0
tx43_packets: 0
tx43_bytes: 0
tx43_tso_packets: 0
tx43_tso_bytes: 0
tx43_tso_inner_packets: 0
tx43_tso_inner_bytes: 0
tx43_csum_partial: 0
tx43_csum_partial_inner: 0
tx43_added_vlan_packets: 0
tx43_nop: 0
tx43_csum_none: 0
tx43_stopped: 0
tx43_dropped: 0
tx43_xmit_more: 0
tx43_recover: 0
tx43_cqes: 0
tx43_wake: 0
tx43_cqe_err: 0
tx44_packets: 0
tx44_bytes: 0
tx44_tso_packets: 0
tx44_tso_bytes: 0
tx44_tso_inner_packets: 0
tx44_tso_inner_bytes: 0
tx44_csum_partial: 0
tx44_csum_partial_inner: 0
tx44_added_vlan_packets: 0
tx44_nop: 0
tx44_csum_none: 0
tx44_stopped: 0
tx44_dropped: 0
tx44_xmit_more: 0
tx44_recover: 0
tx44_cqes: 0
tx44_wake: 0
tx44_cqe_err: 0
tx45_packets: 0
tx45_bytes: 0
tx45_tso_packets: 0
tx45_tso_bytes: 0
tx45_tso_inner_packets: 0
tx45_tso_inner_bytes: 0
tx45_csum_partial: 0
tx45_csum_partial_inner: 0
tx45_added_vlan_packets: 0
tx45_nop: 0
tx45_csum_none: 0
tx45_stopped: 0
tx45_dropped: 0
tx45_xmit_more: 0
tx45_recover: 0
tx45_cqes: 0
tx45_wake: 0
tx45_cqe_err: 0
tx46_packets: 0
tx46_bytes: 0
tx46_tso_packets: 0
tx46_tso_bytes: 0
tx46_tso_inner_packets: 0
tx46_tso_inner_bytes: 0
tx46_csum_partial: 0
tx46_csum_partial_inner: 0
tx46_added_vlan_packets: 0
tx46_nop: 0
tx46_csum_none: 0
tx46_stopped: 0
tx46_dropped: 0
tx46_xmit_more: 0
tx46_recover: 0
tx46_cqes: 0
tx46_wake: 0
tx46_cqe_err: 0
tx47_packets: 0
tx47_bytes: 0
tx47_tso_packets: 0
tx47_tso_bytes: 0
tx47_tso_inner_packets: 0
tx47_tso_inner_bytes: 0
tx47_csum_partial: 0
tx47_csum_partial_inner: 0
tx47_added_vlan_packets: 0
tx47_nop: 0
tx47_csum_none: 0
tx47_stopped: 0
tx47_dropped: 0
tx47_xmit_more: 0
tx47_recover: 0
tx47_cqes: 0
tx47_wake: 0
tx47_cqe_err: 0
tx48_packets: 0
tx48_bytes: 0
tx48_tso_packets: 0
tx48_tso_bytes: 0
tx48_tso_inner_packets: 0
tx48_tso_inner_bytes: 0
tx48_csum_partial: 0
tx48_csum_partial_inner: 0
tx48_added_vlan_packets: 0
tx48_nop: 0
tx48_csum_none: 0
tx48_stopped: 0
tx48_dropped: 0
tx48_xmit_more: 0
tx48_recover: 0
tx48_cqes: 0
tx48_wake: 0
tx48_cqe_err: 0
tx49_packets: 0
tx49_bytes: 0
tx49_tso_packets: 0
tx49_tso_bytes: 0
tx49_tso_inner_packets: 0
tx49_tso_inner_bytes: 0
tx49_csum_partial: 0
tx49_csum_partial_inner: 0
tx49_added_vlan_packets: 0
tx49_nop: 0
tx49_csum_none: 0
tx49_stopped: 0
tx49_dropped: 0
tx49_xmit_more: 0
tx49_recover: 0
tx49_cqes: 0
tx49_wake: 0
tx49_cqe_err: 0
tx50_packets: 0
tx50_bytes: 0
tx50_tso_packets: 0
tx50_tso_bytes: 0
tx50_tso_inner_packets: 0
tx50_tso_inner_bytes: 0
tx50_csum_partial: 0
tx50_csum_partial_inner: 0
tx50_added_vlan_packets: 0
tx50_nop: 0
tx50_csum_none: 0
tx50_stopped: 0
tx50_dropped: 0
tx50_xmit_more: 0
tx50_recover: 0
tx50_cqes: 0
tx50_wake: 0
tx50_cqe_err: 0
tx51_packets: 0
tx51_bytes: 0
tx51_tso_packets: 0
tx51_tso_bytes: 0
tx51_tso_inner_packets: 0
tx51_tso_inner_bytes: 0
tx51_csum_partial: 0
tx51_csum_partial_inner: 0
tx51_added_vlan_packets: 0
tx51_nop: 0
tx51_csum_none: 0
tx51_stopped: 0
tx51_dropped: 0
tx51_xmit_more: 0
tx51_recover: 0
tx51_cqes: 0
tx51_wake: 0
tx51_cqe_err: 0
tx52_packets: 0
tx52_bytes: 0
tx52_tso_packets: 0
tx52_tso_bytes: 0
tx52_tso_inner_packets: 0
tx52_tso_inner_bytes: 0
tx52_csum_partial: 0
tx52_csum_partial_inner: 0
tx52_added_vlan_packets: 0
tx52_nop: 0
tx52_csum_none: 0
tx52_stopped: 0
tx52_dropped: 0
tx52_xmit_more: 0
tx52_recover: 0
tx52_cqes: 0
tx52_wake: 0
tx52_cqe_err: 0
tx53_packets: 0
tx53_bytes: 0
tx53_tso_packets: 0
tx53_tso_bytes: 0
tx53_tso_inner_packets: 0
tx53_tso_inner_bytes: 0
tx53_csum_partial: 0
tx53_csum_partial_inner: 0
tx53_added_vlan_packets: 0
tx53_nop: 0
tx53_csum_none: 0
tx53_stopped: 0
tx53_dropped: 0
tx53_xmit_more: 0
tx53_recover: 0
tx53_cqes: 0
tx53_wake: 0
tx53_cqe_err: 0
tx54_packets: 0
tx54_bytes: 0
tx54_tso_packets: 0
tx54_tso_bytes: 0
tx54_tso_inner_packets: 0
tx54_tso_inner_bytes: 0
tx54_csum_partial: 0
tx54_csum_partial_inner: 0
tx54_added_vlan_packets: 0
tx54_nop: 0
tx54_csum_none: 0
tx54_stopped: 0
tx54_dropped: 0
tx54_xmit_more: 0
tx54_recover: 0
tx54_cqes: 0
tx54_wake: 0
tx54_cqe_err: 0
tx55_packets: 0
tx55_bytes: 0
tx55_tso_packets: 0
tx55_tso_bytes: 0
tx55_tso_inner_packets: 0
tx55_tso_inner_bytes: 0
tx55_csum_partial: 0
tx55_csum_partial_inner: 0
tx55_added_vlan_packets: 0
tx55_nop: 0
tx55_csum_none: 0
tx55_stopped: 0
tx55_dropped: 0
tx55_xmit_more: 0
tx55_recover: 0
tx55_cqes: 0
tx55_wake: 0
tx55_cqe_err: 0
tx0_xdp_xmit: 0
tx0_xdp_full: 0
tx0_xdp_err: 0
tx0_xdp_cqes: 0
tx1_xdp_xmit: 0
tx1_xdp_full: 0
tx1_xdp_err: 0
tx1_xdp_cqes: 0
tx2_xdp_xmit: 0
tx2_xdp_full: 0
tx2_xdp_err: 0
tx2_xdp_cqes: 0
tx3_xdp_xmit: 0
tx3_xdp_full: 0
tx3_xdp_err: 0
tx3_xdp_cqes: 0
tx4_xdp_xmit: 0
tx4_xdp_full: 0
tx4_xdp_err: 0
tx4_xdp_cqes: 0
tx5_xdp_xmit: 0
tx5_xdp_full: 0
tx5_xdp_err: 0
tx5_xdp_cqes: 0
tx6_xdp_xmit: 0
tx6_xdp_full: 0
tx6_xdp_err: 0
tx6_xdp_cqes: 0
tx7_xdp_xmit: 0
tx7_xdp_full: 0
tx7_xdp_err: 0
tx7_xdp_cqes: 0
tx8_xdp_xmit: 0
tx8_xdp_full: 0
tx8_xdp_err: 0
tx8_xdp_cqes: 0
tx9_xdp_xmit: 0
tx9_xdp_full: 0
tx9_xdp_err: 0
tx9_xdp_cqes: 0
tx10_xdp_xmit: 0
tx10_xdp_full: 0
tx10_xdp_err: 0
tx10_xdp_cqes: 0
tx11_xdp_xmit: 0
tx11_xdp_full: 0
tx11_xdp_err: 0
tx11_xdp_cqes: 0
tx12_xdp_xmit: 0
tx12_xdp_full: 0
tx12_xdp_err: 0
tx12_xdp_cqes: 0
tx13_xdp_xmit: 0
tx13_xdp_full: 0
tx13_xdp_err: 0
tx13_xdp_cqes: 0
tx14_xdp_xmit: 0
tx14_xdp_full: 0
tx14_xdp_err: 0
tx14_xdp_cqes: 0
tx15_xdp_xmit: 0
tx15_xdp_full: 0
tx15_xdp_err: 0
tx15_xdp_cqes: 0
tx16_xdp_xmit: 0
tx16_xdp_full: 0
tx16_xdp_err: 0
tx16_xdp_cqes: 0
tx17_xdp_xmit: 0
tx17_xdp_full: 0
tx17_xdp_err: 0
tx17_xdp_cqes: 0
tx18_xdp_xmit: 0
tx18_xdp_full: 0
tx18_xdp_err: 0
tx18_xdp_cqes: 0
tx19_xdp_xmit: 0
tx19_xdp_full: 0
tx19_xdp_err: 0
tx19_xdp_cqes: 0
tx20_xdp_xmit: 0
tx20_xdp_full: 0
tx20_xdp_err: 0
tx20_xdp_cqes: 0
tx21_xdp_xmit: 0
tx21_xdp_full: 0
tx21_xdp_err: 0
tx21_xdp_cqes: 0
tx22_xdp_xmit: 0
tx22_xdp_full: 0
tx22_xdp_err: 0
tx22_xdp_cqes: 0
tx23_xdp_xmit: 0
tx23_xdp_full: 0
tx23_xdp_err: 0
tx23_xdp_cqes: 0
tx24_xdp_xmit: 0
tx24_xdp_full: 0
tx24_xdp_err: 0
tx24_xdp_cqes: 0
tx25_xdp_xmit: 0
tx25_xdp_full: 0
tx25_xdp_err: 0
tx25_xdp_cqes: 0
tx26_xdp_xmit: 0
tx26_xdp_full: 0
tx26_xdp_err: 0
tx26_xdp_cqes: 0
tx27_xdp_xmit: 0
tx27_xdp_full: 0
tx27_xdp_err: 0
tx27_xdp_cqes: 0
tx28_xdp_xmit: 0
tx28_xdp_full: 0
tx28_xdp_err: 0
tx28_xdp_cqes: 0
tx29_xdp_xmit: 0
tx29_xdp_full: 0
tx29_xdp_err: 0
tx29_xdp_cqes: 0
tx30_xdp_xmit: 0
tx30_xdp_full: 0
tx30_xdp_err: 0
tx30_xdp_cqes: 0
tx31_xdp_xmit: 0
tx31_xdp_full: 0
tx31_xdp_err: 0
tx31_xdp_cqes: 0
tx32_xdp_xmit: 0
tx32_xdp_full: 0
tx32_xdp_err: 0
tx32_xdp_cqes: 0
tx33_xdp_xmit: 0
tx33_xdp_full: 0
tx33_xdp_err: 0
tx33_xdp_cqes: 0
tx34_xdp_xmit: 0
tx34_xdp_full: 0
tx34_xdp_err: 0
tx34_xdp_cqes: 0
tx35_xdp_xmit: 0
tx35_xdp_full: 0
tx35_xdp_err: 0
tx35_xdp_cqes: 0
tx36_xdp_xmit: 0
tx36_xdp_full: 0
tx36_xdp_err: 0
tx36_xdp_cqes: 0
tx37_xdp_xmit: 0
tx37_xdp_full: 0
tx37_xdp_err: 0
tx37_xdp_cqes: 0
tx38_xdp_xmit: 0
tx38_xdp_full: 0
tx38_xdp_err: 0
tx38_xdp_cqes: 0
tx39_xdp_xmit: 0
tx39_xdp_full: 0
tx39_xdp_err: 0
tx39_xdp_cqes: 0
tx40_xdp_xmit: 0
tx40_xdp_full: 0
tx40_xdp_err: 0
tx40_xdp_cqes: 0
tx41_xdp_xmit: 0
tx41_xdp_full: 0
tx41_xdp_err: 0
tx41_xdp_cqes: 0
tx42_xdp_xmit: 0
tx42_xdp_full: 0
tx42_xdp_err: 0
tx42_xdp_cqes: 0
tx43_xdp_xmit: 0
tx43_xdp_full: 0
tx43_xdp_err: 0
tx43_xdp_cqes: 0
tx44_xdp_xmit: 0
tx44_xdp_full: 0
tx44_xdp_err: 0
tx44_xdp_cqes: 0
tx45_xdp_xmit: 0
tx45_xdp_full: 0
tx45_xdp_err: 0
tx45_xdp_cqes: 0
tx46_xdp_xmit: 0
tx46_xdp_full: 0
tx46_xdp_err: 0
tx46_xdp_cqes: 0
tx47_xdp_xmit: 0
tx47_xdp_full: 0
tx47_xdp_err: 0
tx47_xdp_cqes: 0
tx48_xdp_xmit: 0
tx48_xdp_full: 0
tx48_xdp_err: 0
tx48_xdp_cqes: 0
tx49_xdp_xmit: 0
tx49_xdp_full: 0
tx49_xdp_err: 0
tx49_xdp_cqes: 0
tx50_xdp_xmit: 0
tx50_xdp_full: 0
tx50_xdp_err: 0
tx50_xdp_cqes: 0
tx51_xdp_xmit: 0
tx51_xdp_full: 0
tx51_xdp_err: 0
tx51_xdp_cqes: 0
tx52_xdp_xmit: 0
tx52_xdp_full: 0
tx52_xdp_err: 0
tx52_xdp_cqes: 0
tx53_xdp_xmit: 0
tx53_xdp_full: 0
tx53_xdp_err: 0
tx53_xdp_cqes: 0
tx54_xdp_xmit: 0
tx54_xdp_full: 0
tx54_xdp_err: 0
tx54_xdp_cqes: 0
tx55_xdp_xmit: 0
tx55_xdp_full: 0
tx55_xdp_err: 0
tx55_xdp_cqes: 0
[...]
ethtool -S enp175s0f0
NIC statistics:
rx_packets: 141574897253
rx_bytes: 184445040406258
tx_packets: 172569543894
tx_bytes: 99486882076365
tx_tso_packets: 9367664195
tx_tso_bytes: 56435233992948
tx_tso_inner_packets: 0
tx_tso_inner_bytes: 0
tx_added_vlan_packets: 141297671626
tx_nop: 2102916272
rx_lro_packets: 0
rx_lro_bytes: 0
rx_ecn_mark: 0
rx_removed_vlan_packets: 141574897252
rx_csum_unnecessary: 0
rx_csum_none: 23135854
rx_csum_complete: 141551761398
rx_csum_unnecessary_inner: 0
rx_xdp_drop: 0
rx_xdp_redirect: 0
rx_xdp_tx_xmit: 0
rx_xdp_tx_full: 0
rx_xdp_tx_err: 0
rx_xdp_tx_cqe: 0
tx_csum_none: 127934791664
It is a good idea to look into this, tx is not requesting hw tx
csumming for a lot of packets, maybe you are wasting a lot of cpu
on
calculating csum, or maybe this is just the rx csum complete..
tx_csum_partial: 13362879974
tx_csum_partial_inner: 0
tx_queue_stopped: 232561
TX queues are stalling, could be an indentation for the pcie
bottelneck.
tx_queue_dropped: 0
tx_xmit_more: 1266021946
tx_recover: 0
tx_cqes: 140031716469
tx_queue_wake: 232561
tx_udp_seg_rem: 0
tx_cqe_err: 0
tx_xdp_xmit: 0
tx_xdp_full: 0
tx_xdp_err: 0
tx_xdp_cqes: 0
rx_wqe_err: 0
rx_mpwqe_filler_cqes: 0
rx_mpwqe_filler_strides: 0
rx_buff_alloc_err: 0
rx_cqe_compress_blks: 0
rx_cqe_compress_pkts: 0
rx_page_reuse: 0
rx_cache_reuse: 16625975793
rx_cache_full: 54161465914
rx_cache_empty: 258048
rx_cache_busy: 54161472735
rx_cache_waive: 0
rx_congst_umr: 0
rx_arfs_err: 0
ch_events: 40572621887
ch_poll: 40885650979
ch_arm: 40429276692
ch_aff_change: 0
ch_eq_rearm: 0
rx_out_of_buffer: 2791690
rx_if_down_packets: 74
rx_vport_unicast_packets: 141843476308
rx_vport_unicast_bytes: 185421265403318
tx_vport_unicast_packets: 172569484005
tx_vport_unicast_bytes: 100019940094298
rx_vport_multicast_packets: 85122935
rx_vport_multicast_bytes: 5761316431
tx_vport_multicast_packets: 6452
tx_vport_multicast_bytes: 643540
rx_vport_broadcast_packets: 22423624
rx_vport_broadcast_bytes: 1390127090
tx_vport_broadcast_packets: 22024
tx_vport_broadcast_bytes: 1321440
rx_vport_rdma_unicast_packets: 0
rx_vport_rdma_unicast_bytes: 0
tx_vport_rdma_unicast_packets: 0
tx_vport_rdma_unicast_bytes: 0
rx_vport_rdma_multicast_packets: 0
rx_vport_rdma_multicast_bytes: 0
tx_vport_rdma_multicast_packets: 0
tx_vport_rdma_multicast_bytes: 0
tx_packets_phy: 172569501577
rx_packets_phy: 142871314588
rx_crc_errors_phy: 0
tx_bytes_phy: 100710212814151
rx_bytes_phy: 187209224289564
tx_multicast_phy: 6452
tx_broadcast_phy: 22024
rx_multicast_phy: 85122933
rx_broadcast_phy: 22423623
rx_in_range_len_errors_phy: 2
rx_out_of_range_len_phy: 0
rx_oversize_pkts_phy: 0
rx_symbol_err_phy: 0
tx_mac_control_phy: 0
rx_mac_control_phy: 0
rx_unsupported_op_phy: 0
rx_pause_ctrl_phy: 0
tx_pause_ctrl_phy: 0
rx_discards_phy: 920161423
Ok, this port seem to be suffering more, RX is congested, maybe due
to
the pcie bottleneck.
Yes this side is receiving more traffic - second port is +10G more tx
[...]
Average: 17 0.00 0.00 16.60 0.00 0.00 52.10
0.00 0.00 0.00 31.30
Average: 18 0.00 0.00 13.90 0.00 0.00 61.20
0.00 0.00 0.00 24.90
Average: 19 0.00 0.00 9.99 0.00 0.00 70.33
0.00 0.00 0.00 19.68
Average: 20 0.00 0.00 9.00 0.00 0.00 73.00
0.00 0.00 0.00 18.00
Average: 21 0.00 0.00 8.70 0.00 0.00 73.90
0.00 0.00 0.00 17.40
Average: 22 0.00 0.00 15.42 0.00 0.00 58.56
0.00 0.00 0.00 26.03
Average: 23 0.00 0.00 10.81 0.00 0.00 71.67
0.00 0.00 0.00 17.52
Average: 24 0.00 0.00 10.00 0.00 0.00 71.80
0.00 0.00 0.00 18.20
Average: 25 0.00 0.00 11.19 0.00 0.00 71.13
0.00 0.00 0.00 17.68
Average: 26 0.00 0.00 11.00 0.00 0.00 70.80
0.00 0.00 0.00 18.20
Average: 27 0.00 0.00 10.01 0.00 0.00 69.57
0.00 0.00 0.00 20.42
The numa cores are not at 100% util, you have around 20% of idle on
each one.
Yes - no 100% cpu - but the difference between 80% and 100% is like
push
aditional 1-2Gbit/s
yes but, it doens't look like the bottleneck is the cpu, although it is
close to be :)..
Average: 28 0.00 0.00 0.00 0.00 0.00 0.00
0.00
0.00 0.00 100.00
Average: 29 0.00 0.00 0.00 0.00 0.00 0.00
0.00
0.00 0.00 100.00
Average: 30 0.00 0.00 0.00 0.00 0.00 0.00
0.00
0.00 0.00 100.00
Average: 31 0.00 0.00 0.00 0.00 0.00 0.00
0.00
0.00 0.00 100.00
Average: 32 0.00 0.00 0.00 0.00 0.00 0.00
0.00
0.00 0.00 100.00
Average: 33 0.00 0.00 3.90 0.00 0.00 0.00
0.00
0.00 0.00 96.10
Average: 34 0.00 0.00 0.00 0.00 0.00 0.00
0.00
0.00 0.00 100.00
Average: 35 0.00 0.00 0.00 0.00 0.00 0.00
0.00
0.00 0.00 100.00
Average: 36 0.10 0.00 0.20 0.00 0.00 0.00
0.00
0.00 0.00 99.70
Average: 37 0.20 0.00 0.30 0.00 0.00 0.00
0.00
0.00 0.00 99.50
Average: 38 0.00 0.00 0.00 0.00 0.00 0.00
0.00
0.00 0.00 100.00
Average: 39 0.00 0.00 2.60 0.00 0.00 0.00
0.00
0.00 0.00 97.40
Average: 40 0.00 0.00 0.90 0.00 0.00 0.00
0.00
0.00 0.00 99.10
Average: 41 0.10 0.00 0.50 0.00 0.00 0.00
0.00
0.00 0.00 99.40
Average: 42 0.00 0.00 9.91 0.00 0.00 70.67
0.00 0.00 0.00 19.42
Average: 43 0.00 0.00 15.90 0.00 0.00 57.50
0.00 0.00 0.00 26.60
Average: 44 0.00 0.00 12.20 0.00 0.00 66.20
0.00 0.00 0.00 21.60
Average: 45 0.00 0.00 12.00 0.00 0.00 67.50
0.00 0.00 0.00 20.50
Average: 46 0.00 0.00 12.90 0.00 0.00 65.50
0.00 0.00 0.00 21.60
Average: 47 0.00 0.00 14.59 0.00 0.00 60.84
0.00 0.00 0.00 24.58
Average: 48 0.00 0.00 13.59 0.00 0.00 61.74
0.00 0.00 0.00 24.68
Average: 49 0.00 0.00 18.36 0.00 0.00 53.29
0.00 0.00 0.00 28.34
Average: 50 0.00 0.00 15.32 0.00 0.00 58.86
0.00 0.00 0.00 25.83
Average: 51 0.00 0.00 17.60 0.00 0.00 55.20
0.00 0.00 0.00 27.20
Average: 52 0.00 0.00 15.92 0.00 0.00 56.06
0.00 0.00 0.00 28.03
Average: 53 0.00 0.00 13.00 0.00 0.00 62.30
0.00 0.00 0.00 24.70
Average: 54 0.00 0.00 13.20 0.00 0.00 61.50
0.00 0.00 0.00 25.30
Average: 55 0.00 0.00 14.59 0.00 0.00 58.64
0.00 0.00 0.00 26.77
ethtool -k enp175s0f0
Features for enp175s0f0:
rx-checksumming: on
tx-checksumming: on
tx-checksum-ipv4: on
tx-checksum-ip-generic: off [fixed]
tx-checksum-ipv6: on
tx-checksum-fcoe-crc: off [fixed]
tx-checksum-sctp: off [fixed]
scatter-gather: on
tx-scatter-gather: on
tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
tx-tcp-segmentation: on
tx-tcp-ecn-segmentation: off [fixed]
tx-tcp-mangleid-segmentation: off
tx-tcp6-segmentation: on
udp-fragmentation-offload: off
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: off [fixed]
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off
receive-hashing: on
highdma: on [fixed]
rx-vlan-filter: on
vlan-challenged: off [fixed]
tx-lockless: off [fixed]
netns-local: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: on
tx-gre-csum-segmentation: on
tx-ipxip4-segmentation: off [fixed]
tx-ipxip6-segmentation: off [fixed]
tx-udp_tnl-segmentation: on
tx-udp_tnl-csum-segmentation: on
tx-gso-partial: on
tx-sctp-segmentation: off [fixed]
tx-esp-segmentation: off [fixed]
tx-udp-segmentation: on
fcoe-mtu: off [fixed]
tx-nocache-copy: off
loopback: off [fixed]
rx-fcs: off
rx-all: off
tx-vlan-stag-hw-insert: on
rx-vlan-stag-hw-parse: off [fixed]
rx-vlan-stag-filter: on [fixed]
l2-fwd-offload: off [fixed]
hw-tc-offload: off
esp-hw-offload: off [fixed]
esp-tx-csum-hw-offload: off [fixed]
rx-udp_tunnel-port-offload: on
tls-hw-tx-offload: off [fixed]
tls-hw-rx-offload: off [fixed]
rx-gro-hw: off [fixed]
tls-hw-record: off [fixed]
ethtool -c enp175s0f0
Coalesce parameters for enp175s0f0:
Adaptive RX: off TX: on
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0
dmac: 32703
rx-usecs: 256
rx-frames: 128
rx-usecs-irq: 0
rx-frames-irq: 0
tx-usecs: 8
tx-frames: 128
tx-usecs-irq: 0
tx-frames-irq: 0
rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0
rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0
ethtool -g enp175s0f0
Ring parameters for enp175s0f0:
Pre-set maximums:
RX: 8192
RX Mini: 0
RX Jumbo: 0
TX: 8192
Current hardware settings:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Also changed a little coalesce params - and best for this config are:
ethtool -c enp175s0f0
Coalesce parameters for enp175s0f0:
Adaptive RX: off TX: off
stats-block-usecs: 0
sample-interval: 0
pkt-rate-low: 0
pkt-rate-high: 0
dmac: 32573
rx-usecs: 40
rx-frames: 128
rx-usecs-irq: 0
rx-frames-irq: 0
tx-usecs: 8
tx-frames: 8
tx-usecs-irq: 0
tx-frames-irq: 0
rx-usecs-low: 0
rx-frame-low: 0
tx-usecs-low: 0
tx-frame-low: 0
rx-usecs-high: 0
rx-frame-high: 0
tx-usecs-high: 0
tx-frame-high: 0
Less drops on RX side - and more pps in overall forwarded.
how much improvement ? maybe we can improve our adaptive rx coal to be
efficient for this work load.
I can prepare more stats with ethtool maybee to compare - but normally
tested with simple icmp forwarded from interface to interface
- before change coalescence params:
adaptive-rx off rx-usecs 384 rx-frames 128
3% loss for icmp
- after change to:
adaptive-rx off rx-usecs 40 rx-frames 128 adaptive-tx off tx-usecs 8
tx-frames 8
2% loss for icmp
But yes - to know better will need to compare rx/tx counters from
ethtool + /proc/net/dev
Was trying to turn on adaptative-tx+rx - but 100% saturation at 43Gbit/s
RX / 43Gbit/s TX
