Re: [dpdk-dev] vmxnet3 TX TCP/UDP checksum not getting computed with L2_len > 14

[Padam Jeet Singh]

> On 17-Jun-2018, at 10:16 PM, Stephen Hemminger  
> wrote:
> 
> On Sun, 17 Jun 2018 14:55:06 +0530
> Padam Jeet Singh  wrote:
> 
>> Hello,
>> 
>> Issue observed when using vmxnet3 based interface on packet with following 
>> structure is sent:
>> 
>> Ethernet + PPPoE + PPP (22 bytes) as the Layer 2 header, 
>> IPv4 (20) 
>> UDP
>> DNS Payload
>> 
>> The tx offload value in this case is 0x0f0 (PKT_TX_IPV4  | 
>> PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM)
>> 
>> The checksum of the packet seen by the receiver shows incorrect checksum and 
>> it’s value is the pseudo checksum value that was set at the time of the TX. 
>> However the IP header checksum is correct.
>> 
>> The same issue is not seen when the L2 header is a just the Ethernet (14 
>> bytes).
>> 
>> Also, with the same setup on the same hardware if we switch the driver from 
>> vmxnet3 to e1000e, all checksums are computed correctly.
>> 
>> Is this a DPDK vmxnet3 driver bug or that of underlying esxi? The ESXi 
>> version is 6.0.0 (Build 3620759).
>> 
>> Thanks,
>> Padam
> 
> I don't think VMWare supports IP checksum offload. Since IP checksum is 
> trivial and in cache,
> the IP header checksum offload is usually not a speed up anyway. Linux for 
> example, never does
> IP header checksum offload.

It’s not the IP checksum - it’s the TCP/UDP checksum (L4) that’s not getting 
computed. In fact the IP checksum is coming correctly because we are handing it 
in software. The driver when queried returns TX offload flags with the IP 
header checksum bit off, but L4 checksum (TCP & UDP) are available. The UDP 
checksum gets computed correctly if and only if the l2_len is 14. In case of 
other encapsulations, e.g. pppoe (l2_len=22), this checksum computation is 
being skipped and the packet is simply sent out with the pseudo checksum value 
of the IP header. 

[dpdk-dev] vmxnet3 TX TCP/UDP checksum not getting computed with L2_len > 14

[Padam Jeet Singh]

Hello,

Issue observed when using vmxnet3 based interface on packet with following 
structure is sent:

Ethernet + PPPoE + PPP (22 bytes) as the Layer 2 header, 
IPv4 (20) 
UDP
DNS Payload

The tx offload value in this case is 0x0f0 (PKT_TX_IPV4  | 
PKT_TX_IP_CKSUM | PKT_TX_UDP_CKSUM)

The checksum of the packet seen by the receiver shows incorrect checksum and 
it’s value is the pseudo checksum value that was set at the time of the TX. 
However the IP header checksum is correct.

The same issue is not seen when the L2 header is a just the Ethernet (14 bytes).

Also, with the same setup on the same hardware if we switch the driver from 
vmxnet3 to e1000e, all checksums are computed correctly.

Is this a DPDK vmxnet3 driver bug or that of underlying esxi? The ESXi version 
is 6.0.0 (Build 3620759).

Thanks,
Padam

Re: [dpdk-dev] vmxnet3 RX VLAN Strip offload incorrect behaviour

[Padam Jeet Singh]

Hi,

Does anyone have a work-around for this? Should we debug the vmxnet3 driver 
since for sure this issue is even coming in standard l2fwd example.

Thanks,
Padam

> On 12-Apr-2018, at 11:52 AM, Padam Jeet Singh  
> wrote:
> 
> 
> 
>> On 10-Apr-2018, at 11:43 AM, Yong Wang  wrote:
>> 
>> When using 4095, I assume you enabled trunk mode on ESX vswitch.  What ESX 
>> version did you use? Is the vswitch standard switch or DVS? We have tested 
>> over ESX6.0 and onwards plus DPDK 17.05 and it should work.
> 
> Yes, the 4095 enables trunk mode. ESXi version is "6.0.0 (Build 3620759)”, 
> the switch is a “Standard vSwitch”.
> 
> I did a simple change to the standard l2fwd sample application. Then tried 
> the same app once with e1000e emulation driver, and then with the vmxnet3 
> driver. Patch for changes done to the l2fwd code are available here: 
> https://pastebin.com/0RMJYKr0
> 
> The change I did was enable hw_vlan_strip = 1 and also call 
> rte_eth_dev_set_vlan_offload both post configure and post start.
> 
> With e1000e driver, sample packets arriving with different VLANs:
> 
> mbuff->vlan_tci=500 vlan_tci_outer=0 ethertype=8 
> mbuff->vlan_tci=500 vlan_tci_outer=0 ethertype=8 
> mbuff->vlan_tci=105 vlan_tci_outer=0 ethertype=8 
> mbuff->vlan_tci=100 vlan_tci_outer=0 ethertype=8 
> 
> 
> With vmxnet3, packets arriving with different VLANs:
> 
> mbuff->vlan_tci=0 vlan_tci_outer=0 ethertype=81 
> mbuff->vlan_tci=0 vlan_tci_outer=0 ethertype=81 
> mbuff->vlan_tci=0 vlan_tci_outer=0 ethertype=81 
> mbuff->vlan_tci=0 vlan_tci_outer=0 ethertype=81 
> mbuff->vlan_tci=0 vlan_tci_outer=0 ethertype=81 
> 
> the ether type values are not printed byte-swapped …  so it’s 8100 (VLAN) and 
> 0800 (IPv4).
> 
> 
> 
> 
>> 
>> On 4/8/18, 11:22 PM, "Padam Jeet Singh"  wrote:
>> 
>> 
>>> On 06-Apr-2018, at 11:12 PM, Yong Wang  wrote:
>>> 
>>> Padam,
>>> 
>>> As far as I know, this feature works.  What DPDK version did you use?
>> 
>>   DPDK Version 17.05
>> 
>>> Is there any port reconfiguration (stop/start/mtu change, etc) that could 
>>> lose your vlan offload settings (a dump of the port config at runtime will 
>>> be able to confirm this)?  Can you also post a snippet of packet capture of 
>>> the vlan traffic received?
>>> 
>> 
>>   It’s a standard configure followed by start. No MTU change/Stop. Here are 
>> the port configuration post the device start:
>> 
>>   2018-04-09T05:37:45.332573+00:00 INFO inventum fpnas : APP: -- Ethernet 
>> Port Info (0) --
>>   2018-04-09T05:37:45.332729+00:00 INFO inventum fpnas : APP:   Driver   
>>   net_vmxnet3
>>   2018-04-09T05:37:45.332822+00:00 INFO inventum fpnas : APP:   if_index 
>>   0
>>   2018-04-09T05:37:45.332926+00:00 INFO inventum fpnas : APP:   
>> min_rx_bufsize 1646
>>   2018-04-09T05:37:45.333015+00:00 INFO inventum fpnas : APP:   
>> max_rx_pktlen  16384
>>   2018-04-09T05:37:45.333102+00:00 INFO inventum fpnas : APP:   
>> max_rx_queues supported 16
>>   2018-04-09T05:37:45.333218+00:00 INFO inventum fpnas : APP:   
>> max_tx_queues supported 8
>>   2018-04-09T05:37:45.05+00:00 INFO inventum fpnas : APP:   rx_queues 
>> configured 2
>>   2018-04-09T05:37:45.90+00:00 INFO inventum fpnas : APP:   tx_queues 
>> configured 2
>>   2018-04-09T05:37:45.333475+00:00 INFO inventum fpnas : APP:   
>> max_mac_addrs  1
>>   2018-04-09T05:37:45.333560+00:00 INFO inventum fpnas : APP:   max_vfs  
>>   0
>>   2018-04-09T05:37:45.333644+00:00 INFO inventum fpnas : APP:   
>> max_vmdq_pools 0
>>   2018-04-09T05:37:45.333728+00:00 INFO inventum fpnas : APP:   
>> rx_offload_cap 29
>>   2018-04-09T05:37:45.333832+00:00 INFO inventum fpnas : APP:   
>> tx_offload_cap 45
>>   2018-04-09T05:37:45.333944+00:00 INFO inventum fpnas : APP:   
>> vlan_offload_flags 1
>> 
>> 
>>   And the VLAN offload setting while the device is running (with the packet 
>> coming from the test PC):
>> 
>>   vlan_offload_flag: 1
>> 
>> 
>>   The configuration as is as follows:
>> 
>>   [vmxnet3 based interface on VM with DPDK app] —— [ vswitch with VLAN 4095 
>> and uplink using Intel igb based adaptor] —— [ test PC with VLAN interface ]
>> 
>>   Packet sent by the PC (ARP packet):
>> 
>>   Frame 1: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
>>   Ethernet II, Src: 00:25:64:cf:f2:30, Dst: ff:ff:ff:ff:ff:ff
>>   802.1Q Virtual LAN, PRI: 0, DEI: 0, ID:

Re: [dpdk-dev] vmxnet3 RX VLAN Strip offload incorrect behaviour

[Padam Jeet Singh]

> On 10-Apr-2018, at 11:43 AM, Yong Wang  wrote:
> 
> When using 4095, I assume you enabled trunk mode on ESX vswitch.  What ESX 
> version did you use? Is the vswitch standard switch or DVS? We have tested 
> over ESX6.0 and onwards plus DPDK 17.05 and it should work.

Yes, the 4095 enables trunk mode. ESXi version is "6.0.0 (Build 3620759)”, the 
switch is a “Standard vSwitch”.

I did a simple change to the standard l2fwd sample application. Then tried the 
same app once with e1000e emulation driver, and then with the vmxnet3 driver. 
Patch for changes done to the l2fwd code are available here: 
https://pastebin.com/0RMJYKr0

The change I did was enable hw_vlan_strip = 1 and also call 
rte_eth_dev_set_vlan_offload both post configure and post start.

With e1000e driver, sample packets arriving with different VLANs:

mbuff->vlan_tci=500 vlan_tci_outer=0 ethertype=8 
mbuff->vlan_tci=500 vlan_tci_outer=0 ethertype=8 
mbuff->vlan_tci=105 vlan_tci_outer=0 ethertype=8 
mbuff->vlan_tci=100 vlan_tci_outer=0 ethertype=8 


With vmxnet3, packets arriving with different VLANs:

mbuff->vlan_tci=0 vlan_tci_outer=0 ethertype=81 
mbuff->vlan_tci=0 vlan_tci_outer=0 ethertype=81 
mbuff->vlan_tci=0 vlan_tci_outer=0 ethertype=81 
mbuff->vlan_tci=0 vlan_tci_outer=0 ethertype=81 
mbuff->vlan_tci=0 vlan_tci_outer=0 ethertype=81 

the ether type values are not printed byte-swapped …  so it’s 8100 (VLAN) and 
0800 (IPv4).




> 
> On 4/8/18, 11:22 PM, "Padam Jeet Singh"  wrote:
> 
> 
>> On 06-Apr-2018, at 11:12 PM, Yong Wang  wrote:
>> 
>> Padam,
>> 
>> As far as I know, this feature works.  What DPDK version did you use?
> 
>DPDK Version 17.05
> 
>> Is there any port reconfiguration (stop/start/mtu change, etc) that could 
>> lose your vlan offload settings (a dump of the port config at runtime will 
>> be able to confirm this)?  Can you also post a snippet of packet capture of 
>> the vlan traffic received?
>> 
> 
>It’s a standard configure followed by start. No MTU change/Stop. Here are 
> the port configuration post the device start:
> 
>2018-04-09T05:37:45.332573+00:00 INFO inventum fpnas : APP: -- Ethernet 
> Port Info (0) --
>2018-04-09T05:37:45.332729+00:00 INFO inventum fpnas : APP:   Driver   
>   net_vmxnet3
>2018-04-09T05:37:45.332822+00:00 INFO inventum fpnas : APP:   if_index 
>   0
>2018-04-09T05:37:45.332926+00:00 INFO inventum fpnas : APP:   
> min_rx_bufsize 1646
>2018-04-09T05:37:45.333015+00:00 INFO inventum fpnas : APP:   
> max_rx_pktlen  16384
>2018-04-09T05:37:45.333102+00:00 INFO inventum fpnas : APP:   
> max_rx_queues supported 16
>2018-04-09T05:37:45.333218+00:00 INFO inventum fpnas : APP:   
> max_tx_queues supported 8
>2018-04-09T05:37:45.05+00:00 INFO inventum fpnas : APP:   rx_queues 
> configured 2
>2018-04-09T05:37:45.90+00:00 INFO inventum fpnas : APP:   tx_queues 
> configured 2
>2018-04-09T05:37:45.333475+00:00 INFO inventum fpnas : APP:   
> max_mac_addrs  1
>2018-04-09T05:37:45.333560+00:00 INFO inventum fpnas : APP:   max_vfs  
>   0
>2018-04-09T05:37:45.333644+00:00 INFO inventum fpnas : APP:   
> max_vmdq_pools 0
>2018-04-09T05:37:45.333728+00:00 INFO inventum fpnas : APP:   
> rx_offload_cap 29
>2018-04-09T05:37:45.333832+00:00 INFO inventum fpnas : APP:   
> tx_offload_cap 45
>2018-04-09T05:37:45.333944+00:00 INFO inventum fpnas : APP:   
> vlan_offload_flags 1
> 
> 
>And the VLAN offload setting while the device is running (with the packet 
> coming from the test PC):
> 
>vlan_offload_flag: 1
> 
> 
>The configuration as is as follows:
> 
>[vmxnet3 based interface on VM with DPDK app] —— [ vswitch with VLAN 4095 
> and uplink using Intel igb based adaptor] —— [ test PC with VLAN interface ]
> 
>Packet sent by the PC (ARP packet):
> 
>Frame 1: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
>Ethernet II, Src: 00:25:64:cf:f2:30, Dst: ff:ff:ff:ff:ff:ff
>802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 105
>Address Resolution Protocol (request)
> 
> 
>If I redo the test with the same physical and vmware setup but instead of 
> vmxnet3 I use e1000e emulation driver, all of this works fine. Packets come 
> properly stripped of VLAN with vlan_tci fields set correctly.
> 
> 
>> Yong 
>> 
>> On 4/6/18, 6:51 AM, "dev on behalf of Padam Jeet Singh" 
>>  wrote:
>> 
>>   Hi,
>> 
>>   When configuring the vmxnet3 based ethernet device, the RX VLAN Strip 
>> offload does not work as it usually does with other real NICs which support 
>> this function.
>> 
>>   When config

Re: [dpdk-dev] vmxnet3 RX VLAN Strip offload incorrect behaviour

[Padam Jeet Singh]

> On 06-Apr-2018, at 11:12 PM, Yong Wang  wrote:
> 
> Padam,
> 
> As far as I know, this feature works.  What DPDK version did you use?

DPDK Version 17.05

>  Is there any port reconfiguration (stop/start/mtu change, etc) that could 
> lose your vlan offload settings (a dump of the port config at runtime will be 
> able to confirm this)?  Can you also post a snippet of packet capture of the 
> vlan traffic received?
> 

It’s a standard configure followed by start. No MTU change/Stop. Here are the 
port configuration post the device start:

2018-04-09T05:37:45.332573+00:00 INFO inventum fpnas : APP: -- Ethernet Port 
Info (0) --
2018-04-09T05:37:45.332729+00:00 INFO inventum fpnas : APP:   Driver 
net_vmxnet3
2018-04-09T05:37:45.332822+00:00 INFO inventum fpnas : APP:   if_index   0
2018-04-09T05:37:45.332926+00:00 INFO inventum fpnas : APP:   min_rx_bufsize 
1646
2018-04-09T05:37:45.333015+00:00 INFO inventum fpnas : APP:   max_rx_pktlen  
16384
2018-04-09T05:37:45.333102+00:00 INFO inventum fpnas : APP:   max_rx_queues 
supported 16
2018-04-09T05:37:45.333218+00:00 INFO inventum fpnas : APP:   max_tx_queues 
supported 8
2018-04-09T05:37:45.05+00:00 INFO inventum fpnas : APP:   rx_queues 
configured 2
2018-04-09T05:37:45.90+00:00 INFO inventum fpnas : APP:   tx_queues 
configured 2
2018-04-09T05:37:45.333475+00:00 INFO inventum fpnas : APP:   max_mac_addrs  1
2018-04-09T05:37:45.333560+00:00 INFO inventum fpnas : APP:   max_vfs0
2018-04-09T05:37:45.333644+00:00 INFO inventum fpnas : APP:   max_vmdq_pools 0
2018-04-09T05:37:45.333728+00:00 INFO inventum fpnas : APP:   rx_offload_cap 29
2018-04-09T05:37:45.333832+00:00 INFO inventum fpnas : APP:   tx_offload_cap 45
2018-04-09T05:37:45.333944+00:00 INFO inventum fpnas : APP:   
vlan_offload_flags 1


And the VLAN offload setting while the device is running (with the packet 
coming from the test PC):

vlan_offload_flag: 1


The configuration as is as follows:

[vmxnet3 based interface on VM with DPDK app] —— [ vswitch with VLAN 4095 and 
uplink using Intel igb based adaptor] —— [ test PC with VLAN interface ]

Packet sent by the PC (ARP packet):

Frame 1: 46 bytes on wire (368 bits), 46 bytes captured (368 bits)
Ethernet II, Src: 00:25:64:cf:f2:30, Dst: ff:ff:ff:ff:ff:ff
802.1Q Virtual LAN, PRI: 0, DEI: 0, ID: 105
Address Resolution Protocol (request)


If I redo the test with the same physical and vmware setup but instead of 
vmxnet3 I use e1000e emulation driver, all of this works fine. Packets come 
properly stripped of VLAN with vlan_tci fields set correctly.


> Yong 
> 
> On 4/6/18, 6:51 AM, "dev on behalf of Padam Jeet Singh" 
>  wrote:
> 
>Hi,
> 
>When configuring the vmxnet3 based ethernet device, the RX VLAN Strip 
> offload does not work as it usually does with other real NICs which support 
> this function.
> 
>When configuring with rxmode.hw_vlan_strip = 1, the ipackets statistic 
> does not increase when a VLAN ether type frame is received on the port and it 
> is also not received on queue RX.
>However, when configuring rxmode.hw_vlan_strip = 0 ,the ipackets statistic 
> increases as well as frames arrives with VLAN header.
> 
>Though calling  rte_eth_dev_set_vlan_offload(port, ETH_VLAN_STRIP_OFFLOAD) 
> returns a success, however VLAN stripping on RX does not work.
> 
>TX VLAN INSERT offload on the other hand works just fine.
> 
>Is this a bug in vmxnet3 driver of dpdk?
> 
>Thanks,
>Padam
> 

Re: [dpdk-dev] vmxnet3 RX VLAN Strip offload incorrect behaviour

[Padam Jeet Singh]

+ Yong Wang - new maintainer for vmxnet3 pod

> On 06-Apr-2018, at 7:21 PM, Padam Jeet Singh  wrote:
> 
> Hi,
> 
> When configuring the vmxnet3 based ethernet device, the RX VLAN Strip offload 
> does not work as it usually does with other real NICs which support this 
> function.
> 
> When configuring with rxmode.hw_vlan_strip = 1, the ipackets statistic does 
> not increase when a VLAN ether type frame is received on the port and it is 
> also not received on queue RX.
> However, when configuring rxmode.hw_vlan_strip = 0 ,the ipackets statistic 
> increases as well as frames arrives with VLAN header.
> 
> Though calling  rte_eth_dev_set_vlan_offload(port, ETH_VLAN_STRIP_OFFLOAD) 
> returns a success, however VLAN stripping on RX does not work.
> 
> TX VLAN INSERT offload on the other hand works just fine.
> 
> Is this a bug in vmxnet3 driver of dpdk?
> 
> Thanks,
> Padam

[dpdk-dev] vmxnet3 RX VLAN Strip offload incorrect behaviour

[Padam Jeet Singh]

Hi,

When configuring the vmxnet3 based ethernet device, the RX VLAN Strip offload 
does not work as it usually does with other real NICs which support this 
function.

When configuring with rxmode.hw_vlan_strip = 1, the ipackets statistic does not 
increase when a VLAN ether type frame is received on the port and it is also 
not received on queue RX.
However, when configuring rxmode.hw_vlan_strip = 0 ,the ipackets statistic 
increases as well as frames arrives with VLAN header.

Though calling  rte_eth_dev_set_vlan_offload(port, ETH_VLAN_STRIP_OFFLOAD) 
returns a success, however VLAN stripping on RX does not work.

TX VLAN INSERT offload on the other hand works just fine.

Is this a bug in vmxnet3 driver of dpdk?

Thanks,
Padam

[dpdk-dev] vmxnet3 RX VLAN Strip offload incorrect behaviour

[Padam Jeet Singh]

When configuring the vmxnet3 based ethernet device, the VLAN Strip RX offload 
does not work as it usually does with other real NICs.

When configuring with rxmode.hw_vlan_strip = 1, the ipackets statistic does not 
increase when a VLAN ether type frame is received on the port and it is also 
not received on queue RX.
However, when configuring rxmode.hw_vlan_strip = 0 ,the ipackets statistic 
increases as well as frames arrives with VLAN header.

Though calling  rte_eth_dev_set_vlan_offload(port, ETH_VLAN_STRIP_OFFLOAD) 
returns a success, however VLAN stripping on RX does not work.

TX VLAN INSERT offload on the other hand works just fine.

Is this a bug in vmxnet3 driver of dpdk?

Thanks,
Padam

This email was sent from within Inventum Technologies Private Limited 
(https://www.inventum.net). This email (and any attachments or hyperlinks 
within it) may contain information that is confidential, legally privileged or 
otherwise protected from disclosure. If you are not the intended recipient of 
this email, you are not entitled to use, disclose, distribute, copy, print, 
disseminate or rely on this email in any way. If you have received this email 
in error, please notify the sender immediately by telephone or email and 
destroy it, and all copies of it.

[dpdk-dev] Missing Null check in ip_frag_find?

[Padam Jeet Singh]

Hi,

I have been trying to trace a segfault in ip_frag_find which I am seeing in 
production for some time now.

In the following code in function ip_frag_find from ip_frag_internal.c, the lru 
is not being validated for being not null. At this point the table is 
guaranteed to have an entry in the tail queue? Is a null check on lru required 
here?

/*
 * we found a free entry, check if we can use it.
 * If we run out of free entries in the table, then
 * check if we have a timed out entry to delete.
 */
} else if (free != NULL &&
tbl->max_entries <= tbl->use_entries) {
lru = TAILQ_FIRST(&tbl->lru);
if (max_cycles + lru->start < tms) {
ip_frag_tbl_del(tbl, dr, lru);
} else {
free = NULL;
IP_FRAG_TBL_STAT_UPDATE(&tbl->stat,
fail_nospace, 1);
}
}




Thanks,
Padam

[dpdk-dev] Unable to change source MAC address of packet

[Padam Jeet Singh]

> On 27-Oct-2016, at 7:37 pm, Wiles, Keith  wrote:
> 
> 
>> On Oct 27, 2016, at 6:33 AM, Padam Jeet Singh  
>> wrote:
>> 
>> Hi,
>> 
>> I am crafting a packet in which the source MAC address as set in the 
>> Ethernet header is different than the transmit port?s default MAC address. A 
>> packet capture of the packets coming out of this port however comes with 
>> source MAC address of the port?s default MAC address.
>> 
>> Altering the destination MAC address works fine and shows up correctly in 
>> packet capture.
>> 
>> The underlying network interface is an i210 and some logs added to the 
>> eth_igb_xmit_pkts function show that the packets I have crafted indeed are 
>> reaching the driver with the source MAC address set in the packet code of 
>> the application.
>> 
>> How can I disable this automatic source MAC address setting?
> 
> The packets sent with rte_eth_tx_burst() are not forced to a give MAC 
> address. If you are using something on top of DPDK like Pktgen or OVS or 
> something, then it may try to force a source MAC address.

No? not using pktgen or OVS. Plain simple code to take a packets from a KNI, 
change source mac address on all received packets,  and then tx_burst them to a 
port.

> Maybe the hardware does it, but we need to know the NIC being used and then 
> someone maybe able to answer. I do not know of any Intel NICs do that.

Intel i210 NIC (gigabit Ethernet) is being used. I have gone through the i210 
documentation and can?t see anything specific to setting of MAC address in 
hardware for TX side. For RX side there are validations like MAC filtering, but 
nothing over TX.

> 
> Is this what you are doing.

I agree that rte_eth_tx_burst does not overwrite the source MAC as I was able 
to trace all the way to the IGB driver that source mac makes it intact. There 
is no offload flags enabled in the mbuf. Yet the packets to the other side 
comes out as with source mac address of the port.

Is there any standard DPDK app which crafts packets with different source MAC 
than the port?s physical mac? (I checked the l2fwd example loads the port mac 
before transmitting and then uses the same in TX function).

> 
>> 
>> Thanks,
>> Padam
> 
> Regards,
> Keith
> 

[dpdk-dev] Unable to change source MAC address of packet

[Padam Jeet Singh]

Hi,

I am crafting a packet in which the source MAC address as set in the Ethernet 
header is different than the transmit port?s default MAC address. A packet 
capture of the packets coming out of this port however comes with source MAC 
address of the port?s default MAC address.

Altering the destination MAC address works fine and shows up correctly in 
packet capture.

The underlying network interface is an i210 and some logs added to the 
eth_igb_xmit_pkts function show that the packets I have crafted indeed are 
reaching the driver with the source MAC address set in the packet code of the 
application.

How can I disable this automatic source MAC address setting?

Thanks,
Padam

[dpdk-dev] I211, Double VLAN tags and VLAN Stripping not working

[Padam Jeet Singh]

Hello,

I am using the development branch of DPDK and am trying out double tagging 
using the DPDK PMD E1000 driver.

[dpdk-dev] Segmentation fault in ip_frag_find

[Padam Jeet Singh]

Hello,

We are using the DPDK framework and testing the IP reassembly code.

While reassembly is working fine, the application segfaults with the following 
(partially shown) stack trace. The segfault is not entirely reproducible - it 
happens quite randomly.

Program terminated with signal 11, Segmentation fault.
#0  0x004b3408 in ip_frag_find ()
(gdb) bt
#0  0x004b3408 in ip_frag_find ()
#1  0x004b0cde in rte_ipv4_frag_reassemble_packet ()

All the required checks before we call this function:

1. Death Row is initialised, and not null.
2. The reassemble table is initialised and not null.
3. Protect the reassembly code with a spin lock, so it?s not a thread issue.
4. Free the death row by calling rte_ip_frag_free_death_row post the reassembly 
call within the spin lock itself.
5. At transmit time, the reassembled packet is fragmented again to match the 
outbound MTU.

Version of DPDK is  1.7, however I have checked the git tree and not found any 
functional changes to till the stable version.

Any inputs/ideas on how the issue can be fixed?

Thanks,
Padam

[dpdk-dev] DPDK on Xeon Phi Processors

[Padam Jeet Singh]

Hi,

I was reading up on the Xeon Phi processors - up to 61 cores?.. looks very 
interesting. Has anyone tried offloading packet processing to such a 
co-processor utilising the existing code base? 

Thanks,
Padam

[dpdk-dev] Wrong TCP Checkum computed by hardware

[Padam Jeet Singh]

>> 
> Yes, you are correct, I just noticed you declare  it is static, If possible, 
> send more codes to me, I can help you with this.
>>> 
 Thanks,
 Padam
> 


I have the following code:


mbuf->ol_flags = (uint16_t) (mbuf->ol_flags &
(~PKT_TX_OFFLOAD_MASK));
mbuf->ol_flags |= PKT_TX_IP_CKSUM;
ipv4hdr->hdr_checksum = 0;
tcphdr = (struct tcp_hdr *)(pkt + sizeof(struct ether_hdr) +
sizeof(struct ipv4_hdr));
#ifdef L4CSUM_SW

tcphdr->cksum = 0;
tcphdr->cksum = get_ipv4_udptcp_checksum(ipv4hdr,   (uint16_t*)tcphdr);

#else

mbuf->ol_flags |= PKT_TX_TCP_CKSUM;
if(!(mbuf->ol_flags & PKT_TX_IPV4))
mbuf->ol_flags |= PKT_TX_IPV4;
tcphdr->cksum = 0;
tcphdr->cksum = rte_ipv4_phdr_cksum(ipv4hdr);

#endif

mbuf->pkt.vlan_macip.f.vlan_tci = 100;
mbuf->ol_flags |= PKT_TX_VLAN_PKT;

I have added a macro called L4CSUM_SW, if enabled does L4 Checksum computation 
in software. When I enable this, the packet received has the correct TCP 
checksum. It is important to note that the checksum is being computed, but is 
being computed wrong when TX_VLAN, IP_CKSUM and TCP_CKSUM are enabled.

When I do the same test with VLAN removed from the setup, the TCP checksum 
computation is correct. 

The system exhibiting this issue is a  Intel(R) Xeon(R) CPU E5-2620 v3 @ 
2.40GHz X 2, with 2 X 82599ES on a PCIe3 bus:
02:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ 
Network Connection (rev 01)
Flags: bus master, fast devsel, latency 0, IRQ 32
Memory at c7d2 (64-bit, non-prefetchable) [size=128K]
Memory at c7d44000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [70] MSI-X: Enable+ Count=64 Masked-
Capabilities: [a0] Express Endpoint, MSI 00
Capabilities: [e0] Vital Product Data
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Device Serial Number 00-90-0b-ff-ff-3f-19-d0
Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
Kernel driver in use: igb_uio
Kernel modules: ixgbe

[dpdk-dev] Wrong TCP Checkum computed by hardware

[Padam Jeet Singh]

> On 28-Oct-2015, at 1:46 pm, Liu, Jijiang  wrote:
> 
> 
> 
>> -Original Message-----
>> From: Padam Jeet Singh [mailto:padam.singh at inventum.net]
>> Sent: Wednesday, October 28, 2015 4:12 PM
>> To: Liu, Jijiang
>> Cc: dev at dpdk.org; Matthew Hall
>> Subject: Re: [dpdk-dev] Wrong TCP Checkum computed by hardware
>> 
>> 
>>> On 28-Oct-2015, at 1:31 pm, Liu, Jijiang  wrote:
>>> 
>>> 
>>>> -Original Message-
>>>> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Padam Jeet Singh
>>>> Sent: Wednesday, October 28, 2015 3:20 PM
>>>> To: Matthew Hall
>>>> Cc: dev at dpdk.org
>>>> Subject: Re: [dpdk-dev] Wrong TCP Checkum computed by hardware
>>>> 
>>> 
>>> Did you set the txq_flags?
>>> If the flag is not set, the default value is IXGBE_SIMPLE_FLAGS, and the any
>> TX offloads do not work.
>>> 
>>> See the following codes in ixgbe_rxtx.c file
>>> 
>>> /* Use a simple Tx queue (no offloads, no multi segs) if possible */
>>> if (((txq->txq_flags & IXGBE_SIMPLE_FLAGS) == IXGBE_SIMPLE_FLAGS)
>>> && (txq->tx_rs_thresh >=
>> RTE_PMD_IXGBE_TX_MAX_BURST)) {
>>> PMD_INIT_LOG(INFO, "Using simple tx code path");
>>> ...
>>> dev->tx_pkt_burst = ixgbe_xmit_pkts_simple;
>>> } else {
>>>  ...
>>> dev->tx_pkt_burst = ixgbe_xmit_pkts;
>>> }
>>> 
>>> 
>>> --Jijiang Liu
>> 
>> I initialise the queue with the following structure:
>> 
>> static const struct rte_eth_txconf tx_conf = {
>>  .tx_thresh = {
>>  .pthresh = 32,  /* Ring prefetch threshold */
>>  .hthresh = 0,   /* Ring host threshold */
>>  .wthresh = 0,   /* Ring writeback threshold */
>>  },
>>  .tx_free_thresh = 0,/* Use PMD default values */
>>  .tx_rs_thresh = 0,  /* Use PMD default values */
>> };
>> 
>> This would set the txq_flags to zero - so the tx_pkt_burst function would
>> always point to ixgbe_xmit_pkts. Also, as observed only TCP checksum is
>> computed wrong when there is VLAN TX Offload + IP Offload + TCP offload
>> bits set.  VLAN TX Offload + IP Offload + TCP CKSUM in software generates
>> correct packet on the wire.
> 
> I don't think the txq_flags is 0 if you just initialized the struct 
> rte_eth_txconf like that.

It?s declared as a global static, so it indeed is 0. I also added some debug 
around the init of the queue:

for (i = 0; i < tx; ++i) {
ret = rte_eth_tx_queue_setup(port, i, NB_TXD,
rte_eth_dev_socket_id(port), &tx_conf);
RTE_LOG(INFO, APP, "Port %u TXQ[%d] txflags = %d\n", (unsigned)port,
i, tx_conf.txq_flags);
if (ret < 0)
rte_exit(EXIT_FAILURE, "Could not setup up TX queue %d for "
"port%u (%d)", i, (unsigned)port, ret);
}

And got the following result:

Oct 28 13:55:34 localhost fpnas[1322]: APP: Port 0 TXQ[0] txflags = 0
Oct 28 13:55:34 localhost fpnas[1322]: APP: Port 0 TXQ[1] txflags = 0
Oct 28 13:55:34 localhost fpnas[1322]: APP: Port 1 TXQ[0] txflags = 0
Oct 28 13:55:34 localhost fpnas[1322]: APP: Port 1 TXQ[1] txflags = 0
Oct 28 13:55:35 localhost fpnas[1322]: APP: Port 2 TXQ[0] txflags = 0
Oct 28 13:55:35 localhost fpnas[1322]: APP: Port 3 TXQ[0] txflags = 0


> 
>> Thanks,
>> Padam

[dpdk-dev] Wrong TCP Checkum computed by hardware

[Padam Jeet Singh]

> On 28-Oct-2015, at 1:31 pm, Liu, Jijiang  wrote:
> 
> 
>> -Original Message-
>> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Padam Jeet Singh
>> Sent: Wednesday, October 28, 2015 3:20 PM
>> To: Matthew Hall
>> Cc: dev at dpdk.org
>> Subject: Re: [dpdk-dev] Wrong TCP Checkum computed by hardware
>> 
> 
> Did you set the txq_flags?
> If the flag is not set, the default value is IXGBE_SIMPLE_FLAGS, and the any 
> TX offloads do not work.
> 
> See the following codes in ixgbe_rxtx.c file
> 
> /* Use a simple Tx queue (no offloads, no multi segs) if possible */
>   if (((txq->txq_flags & IXGBE_SIMPLE_FLAGS) == IXGBE_SIMPLE_FLAGS)
>   && (txq->tx_rs_thresh >= RTE_PMD_IXGBE_TX_MAX_BURST)) {
>   PMD_INIT_LOG(INFO, "Using simple tx code path");
>  ...
>   dev->tx_pkt_burst = ixgbe_xmit_pkts_simple;
>   } else {
>...
>   dev->tx_pkt_burst = ixgbe_xmit_pkts;
>   }
> 
> 
> --Jijiang Liu

I initialise the queue with the following structure:

static const struct rte_eth_txconf tx_conf = {
.tx_thresh = {
.pthresh = 32,  /* Ring prefetch threshold */
.hthresh = 0,   /* Ring host threshold */
.wthresh = 0,   /* Ring writeback threshold */
},
.tx_free_thresh = 0,/* Use PMD default values */
.tx_rs_thresh = 0,  /* Use PMD default values */
};

This would set the txq_flags to zero - so the tx_pkt_burst function would 
always point to ixgbe_xmit_pkts. Also, as observed only TCP checksum is 
computed wrong when there is VLAN TX Offload + IP Offload + TCP offload bits 
set.  VLAN TX Offload + IP Offload + TCP CKSUM in software generates correct 
packet on the wire.

Thanks,
Padam

[dpdk-dev] Wrong TCP Checkum computed by hardware

[Padam Jeet Singh]

On 28-Oct-2015, at 12:27 pm, Matthew Hall  wrote:
> 
> On Wed, Oct 28, 2015 at 12:20:22PM +0530, Padam Jeet Singh wrote:
>> Any hint what could I be doing wrong here?
> 
> When this kind of stuff doesn't work it often will depend on the exact 
> version 
> of card, chip, etc. if there are any errata.
> 

82599ES or ixgbe PMD has not had any bug fixes related to offload - not at 
least what I can see in the git commits.

One important finding: Issue only comes when I do TX VLAN offload using 
PKT_TX_VLAN_PKT (fill vlan tci, l2_len, l3_len with the VLAN ID, sizeof(struct 
ether_hdr), sizeof(struct ipv4_hdr) respectively.

So basically VLAN OFFLOAD + IP CSUM OFFLOAD + TCP CSUM OFFLOAD causes the TCP 
checksum to be computed wrong. VLAN Offload + IP CSUM Offload + TCP CSUM in 
Software produces correct results. I am suspecting this to be ixgbe driver bug 
now as nothing in my code can trigger this behaviour.

> So you might want to collect the specifics of the board with lspci -v, 
> ethtool, and pulling it out to check the chip and board revisions.

02:00.0 Ethernet controller: Intel Corporation 82599ES 10-Gigabit SFI/SFP+ 
Network Connection (rev 01)
Flags: bus master, fast devsel, latency 0, IRQ 32
Memory at c7d2 (64-bit, non-prefetchable) [size=128K]
Memory at c7d44000 (64-bit, non-prefetchable) [size=16K]
Capabilities: [40] Power Management version 3
Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
Capabilities: [70] MSI-X: Enable+ Count=64 Masked-
Capabilities: [a0] Express Endpoint, MSI 00
Capabilities: [e0] Vital Product Data
Capabilities: [100] Advanced Error Reporting
Capabilities: [140] Device Serial Number 00-90-0b-ff-ff-3f-19-d0
Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
Kernel driver in use: igb_uio
Kernel modules: ixgbe

> 
> In addition check over the example apps and see how things work there 
> compared 
> with your own code. Often the DPDK interfaces are kind of complex and small 
> pointer or mbuf manipulation mistakes can cause very odd results.
> 

None of the sample code addresses the scenario which I have = VLAN offload + IP 
+ TCP offload.

> Matthew.

Thanks,
Padam

[dpdk-dev] Wrong TCP Checkum computed by hardware

[Padam Jeet Singh]

Hi,

While implementing a sample packet generation code, I am setting up the offload 
flags so that the hardware (82599ES in my case) computes the IP and TCP 
Checksums. However, it seems that only the IP checksum is computed whereas the 
TCP Checksum is computed, but computed wrong. Here is a snippet of what I doing:

/* First reset all offload flags */
mbuf->ol_flags = (uint16_t) (mbuf->ol_flags & (~PKT_TX_OFFLOAD_MASK));
/* Then enable IP Checksum offload */
mbuf->ol_flags |= PKT_TX_IP_CKSUM;
/* This is required to be set to 0 */
ipv4hdr->hdr_checksum = 0;
mbuf->ol_flags |= PKT_TX_TCP_CKSUM;
/* Compute the PSD header */
tcphdr->cksum = rte_ipv4_phdr_cksum(ipv4hdr);

When I replace the checksum computation from hardware-offload to a software 
implementation, the checksum computed is correct. I have verified both the 
cases with Wireshark on the system receiving the packets.

Any hint what could I be doing wrong here?

Thanks,
Padam

[dpdk-dev] Debugging memory leak of mbufs

[Padam Jeet Singh]

Hello,

I have an application which in which the rte_mempool for packets associated 
with the network interfaces using IXGBE and IGB drivers is depleting slowly. 
Each time I do a throughput test, the number of free_count keeps reducing by 
10-15 entries, and finally completely exhausts.

Any inputs on how to debug such a situation? 

Thanks,
Padam

[dpdk-dev] Packet Cloning

[Padam Jeet Singh]

Thanks Marc, Matt, Kyle, and Stephen for your inputs!

I have a possibly good solution of splitting the mbuf into parts to correctly 
mirror to multiple interfaces, else a fallback of copying the full payload into 
a new mbuf.

Though the library doesn?t offer a standard method to copy, I can submit a 
rte_mbuf_copy() patch that does a full deep copy.

Regards,
Padam


> On 28-May-2015, at 9:43 pm, Marc Sune  wrote:
> 
> 
> 
> On 28/05/15 18:06, Matt Laswell wrote:
>> Hey Kyle,
>> 
>> That's one way you can handle it, though I suspect you'll end up with some
>> complexity elsewhere in your code to deal with remembering whether you
>> should look at the original data or the copied and modified data.  Another
>> way is just to make a copy of the original mbuf, but have your copy API
>> stop after it reaches some particular point.  Perhaps just the L2-L4
>> headers, perhaps a few hundred bytes into payload, or perhaps something
>> else entirely. This all gets very application dependent, of course.  How
>> much is "enough" is going to depend heavily on what you're trying to
>> accomplish.
> 
> mbufs can be chained in multiple segments. So you could first split into two 
> segments leaving the big chunk in the original mbuf (chunk2) and copy chunk1 
> into the new mbuf (check prepend, adj and trim).
> 
> Marc
> 
> [1] http://dpdk.org/doc/api/rte__mbuf_8h.html
> 
>> 
> 

Padam J. Singh | VP Engg. | Inventum Technologies Private Limited
www.inventum.net 
+91.9810146640

Address:
C-70, Phase II Extension,
Noida 201305, UP,
India

Tel : +91.120.4647000

This email and any files transmitted with it are confidential and intended 
solely for the use of the individual or entity to whom they are addressed. If 
you have received this email in error please notify the system manager. This 
message contains confidential information and is intended only for the 
individual named. If you are not the named addressee you should not 
disseminate, distribute or copy this e-mail. Please notify the sender 
immediately by e-mail if you have received this e-mail by mistake and delete 
this e-mail from your system. If you are not the intended recipient you are 
notified that disclosing, copying, distributing or taking any action in 
reliance on the contents of this information is strictly prohibited. No 
employee or agent is authorized to conclude any binding agreement on behalf of 
[Inventum Technologies Private Limited] with another party by email without 
express written confirmation by a whole-time Director of the Company who has 
been authorized vide a valid board resolution to the effect. Our company 
accepts no liability for the content of this email, or for the consequences of 
any actions taken on the basis of the information provided, unless that 
information is subsequently confirmed in writing. If you are not the intended 
recipient you are notified that disclosing, copying, distributing or taking any 
action in reliance on the contents of this information is strictly prohibited. 
Any views or opinions presented in this email are solely those of the author 
and do not necessarily represent those of the company. Employees of [Inventum 
Technologies Private Limited] are expressly required not to make defamatory 
statements and not to infringe or authorize any infringement of copyright or 
any other legal right by email communications. Any such communication is 
contrary to company policy and outside the scope of the employment of the 
individual concerned. The company will not accept any liability in respect of 
such communication, and the employee responsible will be personally liable for 
any damages or other liability arising.

[dpdk-dev] Packet Cloning

[Padam Jeet Singh]

Hello,

Is there a function in DPDK to completely clone a pkt_mbuf including the 
segments? 

I am trying to build a packet mirroring application which sends packet out 
through two separate interfaces, but the packet payload needs to be altered 
before send.

Thanks,
Padam

[dpdk-dev] VLAN header insertion and removal

[Padam Jeet Singh]

Hello, 

I have done a simple mbuf adjust and prepend to achieve the removal and 
insertion of the vlan header and it works fine. The use case is something 
similar to the l3fwd example where one port has traffic coming in on multiple 
vlans and the other port has no vlans. The packet processing path in the middle 
inserts or removes the vlan. 

Is there an offload flag or mode where this can be done in hardware? Or is 
there a more optimized way to do this in dpdk? For sake of performance I want 
to avoid the copy ethernet header, modify mbuf, copy back the header steps. 

Thanks, 
Padam 
---
Sent from my mobile. Please excuse the brevity, spelling and punctuation. 

[dpdk-dev] [PATCH] kni: allow setting KNI device MAC address

[Padam Jeet Singh]

Added relevant callback function to change a KNI device's MAC address

Signed-off-by: Padam Jeet Singh 
---
 lib/librte_eal/linuxapp/kni/kni_net.c | 16 
 1 file changed, 16 insertions(+)

diff --git a/lib/librte_eal/linuxapp/kni/kni_net.c 
b/lib/librte_eal/linuxapp/kni/kni_net.c
index d3c0190..15e81c1 100644
--- a/lib/librte_eal/linuxapp/kni/kni_net.c
+++ b/lib/librte_eal/linuxapp/kni/kni_net.c
@@ -612,6 +612,21 @@ kni_net_rebuild_header(struct sk_buff *skb)
return 0;
 }

+/**
+ * kni_net_set_mac - Change the Ethernet Address of the KNI NIC
+ * @netdev: network interface device structure
+ * @p: pointer to an address structure
+ *
+ * Returns 0 on success, negative on failure
+ **/
+static int kni_net_set_mac(struct net_device *netdev, void *p)
+{
+   struct sockaddr *addr = p;
+   if (!is_valid_ether_addr((unsigned char *)(addr->sa_data)))
+   return -EADDRNOTAVAIL;
+   memcpy(netdev->dev_addr, addr->sa_data, netdev->addr_len);
+   return 0;
+}

 static const struct header_ops kni_net_header_ops = {
.create  = kni_net_header,
@@ -628,6 +643,7 @@ static const struct net_device_ops kni_net_netdev_ops = {
.ndo_do_ioctl = kni_net_ioctl,
.ndo_get_stats = kni_net_stats,
.ndo_tx_timeout = kni_net_tx_timeout,
+   .ndo_set_mac_address = kni_net_set_mac,
 };

 void
-- 
1.8.5.2 (Apple Git-48)