[dpdk-dev] driver initialization in DPDK 2.0 built into a shared library.

[Keunhong Lee]

Did you tried 'mk/rte_app.mk' to build you application?
Simply including -lrte_pmd_xxx may cause problem because
PMD_REGISTER_DRIVER uses 'init' attribute,
which is called before the 'main' function.
See the 'start-group' and 'whole-archive' linker options.

Keunhong.


2015-07-15 3:21 GMT+09:00 Polevoy, Igor :

> Hi,
> We are developing an application that uses DPDK PMD functionality .
> We are using a linux shared library which contains the network packets
> processing code and it is statically linked with all the necessary DPDK
> libs.
> The .so is loaded by the main program.
> For the DPDK compilation we have added the -fPIC to the GCC options.
>
> While it all worked fine with DPDK 1.6 where we had the rte_pmd_init_all
> method, in the 2.0 version the
> drivers registration methods (PMD_REGISTER_DRIVER) are not called when the
> shared library is loaded.
>
> Although, I can go along the lines of the rte_pmd_init all and manually
> call the driver registration, I'm concerned
> that DPDK has other drivers initialization calls, and I don't actually
> know which are needed or could be needed and when.
>
> Do you have any advice on that? What is the best way to resolve this issue?
>
> Thank you
> Igor.
>
>

[dpdk-dev] [PATCH 0/2] Native uio-based PMD for Mellanox ConnectX-3 devices

[Keunhong Lee]

We found that optimizing fragmentation configuration of mlx4 driver
performs as fast as native PMD.
I think we have to re-consider using native driver rather than ib driver.

Keunhong.

2015-07-07 1:14 GMT+09:00 Thomas Monjalon :

> 2015-07-07 00:57, Keunhong Lee:
> > Answer 1. UIO based driver is faster then ib based driver.
> > It can saturate 40G link with MTU sized packets using a single thread
> while
> > ib wrapper cannot.
>
> OK, interesting. Do you have numbers and details about your
> testbed/scenario?
>
> > Answer 2. Sorry, I missed that. I'll make a new patch email with my real
> > name.
> >
> > Question 1. Is it OK if I separate GPL-based and BSD-based codes into
> > separated patches?
> > mlx4 kernel driver itself is dual licenses, so I think they are
> considered
> > as BSD in my source code.
> > The only source code under GPL is bitmap, integer logarithm, and
> red-black
> > tree contained in mlnx_uio/kernel directory.
>
> These parts will be built in the user-space driver library, right?
> It would change the license, which is not desirable.
>
> Technically, your approach may be interesting.
> But from a maintenance point of view, this huge codebase may be a
> nightmare.
>
>
> > 2015-07-06 23:17 GMT+09:00 Thomas Monjalon :
> >
> > > 2015-07-06 22:28, leeopop:
> > > > This is a native UIO-based PMD for Mellanox ConnectX-3 devices.
> > > > It uses a persistent memory library in order to provide a persistent
> > > > scartch area for the mlx4 HCA driver.
> > >
> > > What is the benefit of this UIO approach compared to the OFED based
> driver?
> > >
> > > > We release the driver itself under BSD license, but to use it for
> > > > commercial products, you may have to re-implement the separated GPL
> > > > sources.
> > >
> > > The GPL sources are not really separated.
> > > The DPDK libraries must be BSD-licensed.
> > >
> > > > The GPL affected source codes reside in the mlnx_uio/kernel
> directory.
> > >
> > > It seems that a large part of the GPL driver was also copied in
> > > mlnx_uio/mlnx/.
> > >
> > > Given that you are dropping a huge GPL codebase (whose you don't own
> the
> > > copyright) in a BSD library, and that you didn't give your real name in
> > > the signed-off line, it is NACK.
>
>
>
>

[dpdk-dev] [PATCH 0/2] Native uio-based PMD for Mellanox ConnectX-3 devices

[Keunhong Lee]

Answer 1. UIO based driver is faster then ib based driver.
It can saturate 40G link with MTU sized packets using a single thread while
ib wrapper cannot.

Answer 2. Sorry, I missed that. I'll make a new patch email with my real
name.


Question 1. Is it OK if I separate GPL-based and BSD-based codes into
separated patches?
mlx4 kernel driver itself is dual licenses, so I think they are considered
as BSD
in my source code.
The only source code under GPL is bitmap, integer logarithm, and red-black
tree
contained in mlnx_uio/kernel directory.

Keunhong.



2015-07-06 23:17 GMT+09:00 Thomas Monjalon :

> 2015-07-06 22:28, leeopop:
> > This is a native UIO-based PMD for Mellanox ConnectX-3 devices.
> > It uses a persistent memory library in order to provide a persistent
> > scartch area for the mlx4 HCA driver.
>
> What is the benefit of this UIO approach compared to the OFED based driver?
>
> > We release the driver itself under BSD license, but to use it for
> > commercial products, you may have to re-implement the separated GPL
> sources.
>
> The GPL sources are not really separated.
> The DPDK libraries must be BSD-licensed.
>
> > The GPL affected source codes reside in the mlnx_uio/kernel directory.
>
> It seems that a large part of the GPL driver was also copied in
> mlnx_uio/mlnx/.
>
> Given that you are dropping a huge GPL codebase (whose you don't own the
> copyright)
> in a BSD library, and that you didn't give your real name in the
> signed-off line,
> it is NACK.
>

[dpdk-dev] Receiving packets on only one port

[Keunhong Lee]

Check that whether you are polling all RX queues of your program.
You said that your configuration works well with Wireshark.
So I don't think that this is the problem.

You can check your port statistics for the number of received packets.
If the statistics show 2 packets, but you received 1 packet, then your
program might have mistakes.

Keunhong.


2015-06-30 0:39 GMT+09:00 Daeyoung Kim :

> OK, I see. My program uses RSS. Is it related to my problem? If the
> testpmd application does not work, the switch configuration might be wrong.
> Am I right?
>
> Thank you for your answer.
>
> Regards,
> Daeyoung
>
> 2015-06-29 11:28 GMT-04:00 Keunhong Lee :
>
>> Actually I have no idea why this happens.
>> If your switch configuration is correct, your program should work just
>> like Wireshark does.
>> I wander whether your program uses RSS.
>> In my opinion, program bug is the most probable reason.
>>
>> Keunhong.
>>
>>
>> 2015-06-29 23:32 GMT+09:00 Daeyoung Kim :
>>
>>> Hi Keunhong,
>>>
>>> Thank you for your help. Here is the network topology.
>>>
>>> DNS Client - Switch 1 - Switch 2 - Switch 3 - DNS Server
>>>  |  |
>>>  port 0  port 1
>>>
>>> DPDK port 0 receives packets using from the Switch 1 port mirroring and
>>> port 1 receives packets from the Switch 2 using port mirroring as well. As
>>> I already said, when I send DNS packets, the wireshark simultaneously gets
>>> all the packets on two ports. I'm sorry what I told you is incorrect. Using
>>> my program with promiscuous mode, the port 0 receives only DNS queries from
>>> the DNS client, but the port 1 receives only DNS replies from the DNS
>>> server. I'd like to know why it happens.
>>>
>>> Thank you very much!
>>>
>>> Regards,
>>> Daeyoung
>>>
>>> 2015-06-28 20:22 GMT-04:00 Keunhong Lee :
>>>
>>>> I don't know your situation exactly, but here are possible problems.
>>>>
>>>> 1. Your switch learned MAC addresses of two ports.
>>>> 2. Your program bug.
>>>> 3. l3fwd itself contains some bug.
>>>> 4. You did not set all ports in promiscuous mode.
>>>>
>>>> You'd better try 'pktgen' application to test your environment.
>>>>
>>>> Keunhong.
>>>>
>>>>
>>>>
>>>> 2015-06-27 0:45 GMT+09:00 Daeyoung Kim :
>>>>
>>>>> Hello,
>>>>>
>>>>> I'm writing a packet capture program from the l3fwd. When I send DNS
>>>>> packets, the wireshark simultaneously gets all the packets on two
>>>>> ports.
>>>>> However, using my program with promiscuous mode, one port receives all
>>>>> the
>>>>> packets, but the other port gets nothing. Do you know why it happens?
>>>>> Could
>>>>> it be network topology problem using DPDK, a DPDK design like
>>>>> forwarding
>>>>> mechanism, or just my program bugs? Any comments would be appreciated.
>>>>>
>>>>> Thanks,
>>>>> Daeyoung
>>>>>
>>>>
>>>>
>>>
>>
>

[dpdk-dev] Receiving packets on only one port

[Keunhong Lee]

Actually I have no idea why this happens.
If your switch configuration is correct, your program should work just like
Wireshark does.
I wander whether your program uses RSS.
In my opinion, program bug is the most probable reason.

Keunhong.


2015-06-29 23:32 GMT+09:00 Daeyoung Kim :

> Hi Keunhong,
>
> Thank you for your help. Here is the network topology.
>
> DNS Client - Switch 1 - Switch 2 - Switch 3 - DNS Server
>  |  |
>  port 0  port 1
>
> DPDK port 0 receives packets using from the Switch 1 port mirroring and
> port 1 receives packets from the Switch 2 using port mirroring as well. As
> I already said, when I send DNS packets, the wireshark simultaneously gets
> all the packets on two ports. I'm sorry what I told you is incorrect. Using
> my program with promiscuous mode, the port 0 receives only DNS queries from
> the DNS client, but the port 1 receives only DNS replies from the DNS
> server. I'd like to know why it happens.
>
> Thank you very much!
>
> Regards,
> Daeyoung
>
> 2015-06-28 20:22 GMT-04:00 Keunhong Lee :
>
>> I don't know your situation exactly, but here are possible problems.
>>
>> 1. Your switch learned MAC addresses of two ports.
>> 2. Your program bug.
>> 3. l3fwd itself contains some bug.
>> 4. You did not set all ports in promiscuous mode.
>>
>> You'd better try 'pktgen' application to test your environment.
>>
>> Keunhong.
>>
>>
>>
>> 2015-06-27 0:45 GMT+09:00 Daeyoung Kim :
>>
>>> Hello,
>>>
>>> I'm writing a packet capture program from the l3fwd. When I send DNS
>>> packets, the wireshark simultaneously gets all the packets on two ports.
>>> However, using my program with promiscuous mode, one port receives all
>>> the
>>> packets, but the other port gets nothing. Do you know why it happens?
>>> Could
>>> it be network topology problem using DPDK, a DPDK design like forwarding
>>> mechanism, or just my program bugs? Any comments would be appreciated.
>>>
>>> Thanks,
>>> Daeyoung
>>>
>>
>>
>

[dpdk-dev] Could not achieve wire speed for 40GE with any DPDK version on XL710 NIC's

[Keunhong Lee]

I have not used XL710 or i40e.
I have no opinion for those NICs.

Keunhong.

2015-06-29 15:59 GMT+09:00 Pavel Odintsov :

> Hello!
>
> Lee, thank you so much for sharing your experience! What do you think
> about 40GE version of 82599?
>
> On Mon, Jun 29, 2015 at 2:35 AM, Keunhong Lee  wrote:
> > DISCLAIMER: This information is not verified. This is truly my personal
> > opinion.
> >
> > As I know, intel 82599 is the only 10G NIC which supports line rate with
> > minimum sized packets (64 byte).
> > According to our internal tests, Mellanox's 40G NICs even support less
> than
> > 30Mpps.
> > I think 40 Mpps is the hardware capacity.
> >
> > Keunhong.
> >
> >
> >
> > 2015-06-28 19:34 GMT+09:00 Pavel Odintsov :
> >>
> >> Hello, folks!
> >>
> >> We have execute bunch of tests for receive data with Intel XL710 40GE
> >> NIC. We want to achieve wire speed on this platform for traffic
> >> capture.
> >>
> >> But we definitely can't do it. We tried with different versions of
> >> DPDK: 1.4, 1.6, 1.8, 2.0. And have not success.
> >>
> >> We achieved only 40Mpps and could do more.
> >>
> >> Could anybody help us with this issue? Looks like this NIC's could not
> >> work on wire speed :(
> >>
> >> Platform: Intel Xeon E5 e5 2670 + XL 710.
> >>
> >> --
> >> Sincerely yours, Pavel Odintsov
> >
> >
>
>
>
> --
> Sincerely yours, Pavel Odintsov
>

[dpdk-dev] Receive errors on all packets with multi-queues

[Keunhong Lee]

Did you set RSS mode?
Multi-queue RX won't work unless you have set RSS(Receive Side Scaling).

Keunhong.


2015-06-22 14:34 GMT+09:00 Jakub Mazur :

> Hi
>
> I'm having problems with packets received when running 2 receive queues (on
> seperate lcores) on the same port but when I run 1 queue everything works
> correctly. The queues are using 32 bursts and are using the same mempool
> which is allocated on the same socket as the NIC, as are the queues. When
> checking the stats of the port using rte_eth_stats_get nearly all of the
> received packets are errors. I'm generating packets using Pktgen with 64
> byte frames and using ranges on the src ip for rss filtering.
>
> Heres the output:
>
> $ sudo -E ./build/app/newapp -c 0xb -n 3 -d
> ../../../dpdk-2.0.0/x86_64-native-linuxapp-gcc/lib/librte_pmd_ixgbe.so
> EAL: Detected lcore 0 as core 0 on socket 0
> EAL: Detected lcore 1 as core 0 on socket 1
> EAL: Detected lcore 2 as core 1 on socket 0
> EAL: Detected lcore 3 as core 1 on socket 1
> EAL: Detected lcore 4 as core 2 on socket 0
> EAL: Detected lcore 5 as core 2 on socket 1
> EAL: Detected lcore 6 as core 3 on socket 0
> EAL: Detected lcore 7 as core 3 on socket 1
> EAL: Detected lcore 8 as core 4 on socket 0
> EAL: Detected lcore 9 as core 4 on socket 1
> EAL: Detected lcore 10 as core 5 on socket 0
> EAL: Detected lcore 11 as core 5 on socket 1
> EAL: Detected lcore 12 as core 6 on socket 0
> EAL: Detected lcore 13 as core 6 on socket 1
> EAL: Detected lcore 14 as core 7 on socket 0
> EAL: Detected lcore 15 as core 7 on socket 1
> EAL: Detected lcore 16 as core 0 on socket 0
> EAL: Detected lcore 17 as core 0 on socket 1
> EAL: Detected lcore 18 as core 1 on socket 0
> EAL: Detected lcore 19 as core 1 on socket 1
> EAL: Detected lcore 20 as core 2 on socket 0
> EAL: Detected lcore 21 as core 2 on socket 1
> EAL: Detected lcore 22 as core 3 on socket 0
> EAL: Detected lcore 23 as core 3 on socket 1
> EAL: Detected lcore 24 as core 4 on socket 0
> EAL: Detected lcore 25 as core 4 on socket 1
> EAL: Detected lcore 26 as core 5 on socket 0
> EAL: Detected lcore 27 as core 5 on socket 1
> EAL: Detected lcore 28 as core 6 on socket 0
> EAL: Detected lcore 29 as core 6 on socket 1
> EAL: Detected lcore 30 as core 7 on socket 0
> EAL: Detected lcore 31 as core 7 on socket 1
> EAL: Support maximum 128 logical core(s) by configuration.
> EAL: Detected 32 lcore(s)
> EAL: Setting up memory...
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fc98fe0 (size = 0x20)
> EAL: Ask a virtual area of 0x3fc0 bytes
> EAL: Virtual area found at 0x7fc95000 (size = 0x3fc0)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fc94fc0 (size = 0x20)
> EAL: Ask a virtual area of 0x3f60 bytes
> EAL: Virtual area found at 0x7fc91040 (size = 0x3f60)
> EAL: Ask a virtual area of 0x40 bytes
> EAL: Virtual area found at 0x7fc90fe0 (size = 0x40)
> EAL: Ask a virtual area of 0x40 bytes
> EAL: Virtual area found at 0x7fc90f80 (size = 0x40)
> EAL: Ask a virtual area of 0x20 bytes
> EAL: Virtual area found at 0x7fc90f40 (size = 0x20)
> EAL: Requesting 512 pages of size 2MB from socket 0
> EAL: Requesting 512 pages of size 2MB from socket 1
> EAL: TSC frequency is ~290 KHz
> EAL: open shared lib
> ../../../dpdk-2.0.0/x86_64-native-linuxapp-gcc/lib/librte_pmd_ixgbe.so
> EAL: Master lcore 0 is ready (tid=94769900;cpuset=[0])
> EAL: lcore 3 is ready (tid=dff1700;cpuset=[3])
> EAL: lcore 1 is ready (tid=e7f2700;cpuset=[1])
> EAL: PCI device :05:00.1 on NUMA socket 1
> EAL:   probe driver: 8086:10fb rte_ixgbe_pmd
> EAL:   PCI memory mapped at 0x7fc99000
> EAL:   PCI memory mapped at 0x7fc99008
> # of eth ports = 1
> rte_eth_dev_configure() for port 0 done
> rte_eth_rx_queue_setup() for port 0 queue 0 done
> rte_eth_rx_queue_setup() for port 0 queue 1 done
> rte_eth_tx_queue_setup() for port 0 queue 0 done
> rte_eth_tx_queue_setup() for port 0 queue 1 done
> rte_eth_dev_start() for port 0 done
> Core 1 starting
> Core 3 starting
> Bytes: 2048
> Packets: 32
> Missed: 13255672
> Sent: 64
>
> Any help would be appreciated.
>

[dpdk-dev] Receiving packets on only one port

[Keunhong Lee]

I don't know your situation exactly, but here are possible problems.

1. Your switch learned MAC addresses of two ports.
2. Your program bug.
3. l3fwd itself contains some bug.
4. You did not set all ports in promiscuous mode.

You'd better try 'pktgen' application to test your environment.

Keunhong.



2015-06-27 0:45 GMT+09:00 Daeyoung Kim :

> Hello,
>
> I'm writing a packet capture program from the l3fwd. When I send DNS
> packets, the wireshark simultaneously gets all the packets on two ports.
> However, using my program with promiscuous mode, one port receives all the
> packets, but the other port gets nothing. Do you know why it happens? Could
> it be network topology problem using DPDK, a DPDK design like forwarding
> mechanism, or just my program bugs? Any comments would be appreciated.
>
> Thanks,
> Daeyoung
>

[dpdk-dev] Interface once added to DPDK goes missing from ifconfig

[Keunhong Lee]

DPDK's igb_uio driver does not generate 'netdev' interfaces,
so you cannot see any interface via ifconfig.
(However, they are registered as 'rte_eth_dev' in your DPDK application.)

You'd better to use KNI drivers to have 'netdev' intefaces.

Keunhong.


2015-06-29 8:51 GMT+09:00 Abhishek Verma :

> Hi,
>
> I have three ethernet interfaces in my VirtualBox as shown below:
>
> root at VirtualBox:/home/akabra/dpdk/dpdk-2.0.0/tools# ifconfig
> eth1  Link encap:Ethernet  HWaddr 08:00:27:3f:34:56
>   inet addr:192.168.0.123  Bcast:192.168.0.255  Mask:255.255.255.0
>   inet6 addr: fe80::a00:27ff:fe3f:3456/64 Scope:Link
>   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>   RX packets:132 errors:0 dropped:0 overruns:0 frame:0
>   TX packets:26 errors:0 dropped:0 overruns:0 carrier:0
>   collisions:0 txqueuelen:1000
>   RX bytes:20982 (20.9 KB)  TX bytes:3724 (3.7 KB)
>
> eth2  Link encap:Ethernet  HWaddr 08:00:27:f7:13:f5
>   inet addr:1.1.1.1  Bcast:1.1.1.255  Mask:255.255.255.0
>   inet6 addr: fe80::a00:27ff:fef7:13f5/64 Scope:Link
>   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>   RX packets:51932 errors:0 dropped:0 overruns:0 frame:0
>   TX packets:660805 errors:0 dropped:0 overruns:0 carrier:0
>   collisions:0 txqueuelen:1000
>   RX bytes:32407739 (32.4 MB)  TX bytes:1647410289 (1.6 GB)
>
> loLink encap:Local Loopback
>   inet addr:127.0.0.1  Mask:255.0.0.0
>   inet6 addr: ::1/128 Scope:Host
>   UP LOOPBACK RUNNING  MTU:65536  Metric:1
>   RX packets:1681 errors:0 dropped:0 overruns:0 frame:0
>   TX packets:1681 errors:0 dropped:0 overruns:0 carrier:0
>   collisions:0 txqueuelen:0
>   RX bytes:358901 (358.9 KB)  TX bytes:358901 (358.9 KB)
>
> root at manav-VirtualBox:/home/globus/dpdk/dpdk-2.0.0/tools#
> ./dpdk_nic_bind.py --status
>
>
>
> Network devices using DPDK-compatible driver
>
> 
>
> 
>
> Network devices using kernel driver
>
> ===
>
> :00:03.0 '82540EM Gigabit Ethernet Controller' if=eth1 drv=e1000
> unused=igb_uio,vfio-pci *Active*
>
> :00:08.0 '82540EM Gigabit Ethernet Controller' if=eth2 drv=e1000
> unused=igb_uio,vfio-pci *Active*
>
>
> I will now add eth2 to DPDK
>
> root at manav-VirtualBox:/home/globus/dpdk/dpdk-2.0.0/tools# sudo ifconfig
> eth2 down
>
> root at manav-VirtualBox:/home/globus/dpdk/dpdk-2.0.0/tools#
> ./dpdk_nic_bind.py -b igb_uio :00:08.0
>
>
> I now see that eth2 has been removed from my ifconfig output. I cant seem
> to understand why this has happened?
>
>
> root at manav-VirtualBox:/home/globus/dpdk/dpdk-2.0.0/tools# ifconfig
>
> eth1  Link encap:Ethernet  HWaddr 08:00:27:3f:34:56
>
>   inet addr:192.168.0.123  Bcast:192.168.0.255  Mask:255.255.255.0
>
>   inet6 addr: fe80::a00:27ff:fe3f:3456/64 Scope:Link
>
>   UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>
>   RX packets:182 errors:0 dropped:0 overruns:0 frame:0
>
>   TX packets:86 errors:0 dropped:0 overruns:0 carrier:0
>
>   collisions:0 txqueuelen:1000
>
>   RX bytes:27757 (27.7 KB)  TX bytes:12789 (12.7 KB)
>
>
>
> loLink encap:Local Loopback
>
>   inet addr:127.0.0.1  Mask:255.0.0.0
>
>   inet6 addr: ::1/128 Scope:Host
>
>   UP LOOPBACK RUNNING  MTU:65536  Metric:1
>
>   RX packets:1705 errors:0 dropped:0 overruns:0 frame:0
>
>   TX packets:1705 errors:0 dropped:0 overruns:0 carrier:0
>
>   collisions:0 txqueuelen:0
>
>   RX bytes:360948 (360.9 KB)  TX bytes:360948 (360.9 KB)
>
>
> I see that it has been successfully recognized as an interface using DPDK
> driver, but why dont i see it in my ifconfig. Any ideas here?
>
>
> root at manav-VirtualBox:/home/globus/dpdk/dpdk-2.0.0/tools#
> ./dpdk_nic_bind.py --status
>
>
>
> Network devices using DPDK-compatible driver
>
> 
>
> :00:08.0 '82540EM Gigabit Ethernet Controller' drv=igb_uio
> unused=vfio-pci
>
>
>
> Network devices using kernel driver
>
> ===
>
> :00:03.0 '82540EM Gigabit Ethernet Controller' if=eth1 drv=e1000
> unused=igb_uio,vfio-pci *Active*
>
>
>
> Other network devices
>
>
> Would appreciate some help here.
>
>
> Thanks, Abhishek
>

[dpdk-dev] Number of memory channels per processor socket?

[Keunhong Lee]

Currently, I have no idea to find out memory mapping of virtual machines.
I think -n 4 will work on both dual and quad channel memory.
This optimizes entries of mempool or ring structure to interleave memory
access into n-ways.
I think interleaving in 4 ways will also include 2 way interleaving.

Keunhong.

2015-06-29 8:46 GMT+09:00 Abhishek Verma :

> Thanks Keunhong.
>
> How do i get the motherboard spec on a virtual machine thats spawned on
> the cloud, for example Amazon EC2?
>
> Cheers, Abhishek
>
>
> On Mon, Jun 29, 2015 at 4:53 AM, Keunhong Lee  wrote:
>
>> See your motherboard spec.
>> For example, I'm using Intel 5930K with ASUS X99 Delux Motherboard (x99
>> chipset).
>> This site https://www.asus.com/us/Motherboards/X99DELUXE/specifications/
>> tells that
>> "Quad Channel Memory Architecture "
>> which means that "-n 4" is the correct configuration for me.
>>
>> To fully use quad channel memory, I have to use 4 separated DRAM,
>> installed on different memory slots of my motherboard.
>>
>> -n option optimizes memory bank access pattern for datastructures.
>> Since most motherboards support dual or quad memory channel,
>> -n 4 will work for most systems. (Only few motherboards support
>> tri-channel memory)
>>
>> I hope that this information is useful to you.
>> Keunhong.
>>
>>
>>
>> 2015-06-28 18:54 GMT+09:00 Abhishek Verma :
>>
>>> Hi,
>>>
>>> I am new to DPDK and i tried searching the archives in case this had been
>>> discussed but couldnt find any references and hence this email:
>>>
>>> What is the significance of " -n NUM: Number of memory channels per
>>> processor socket" which is passed as an EAL option? I have a virtual
>>> machine (VM) spawned using VirtualBox and i am trying to use DPDK to get
>>> faster access to packets there. I did lscpu, but that didnt give me
>>> anything interesting that i could use here:
>>>
>>> abhishekV at VirtualBox:~/dpdk/dpdk-2.0.0/x86_64-native-linuxapp-gcc/app$
>>> sudo
>>> lscpu
>>> [sudo] password for abhishekV:
>>>
>>> Architecture:  x86_64
>>> CPU op-mode(s):32-bit, 64-bit
>>> Byte Order:Little Endian
>>> CPU(s):2
>>> On-line CPU(s) list:   0,1
>>> Thread(s) per core:1
>>> Core(s) per socket:2
>>> Socket(s): 1
>>> NUMA node(s):  1
>>> Vendor ID: GenuineIntel
>>> CPU family:6
>>> Model: 58
>>> Stepping:  9
>>> CPU MHz:   2594.017
>>> BogoMIPS:  5188.03
>>> L1d cache: 32K
>>> L1d cache: 32K
>>> L2d cache: 6144K
>>> NUMA node0 CPU(s): 0,1
>>> abhishekV at VirtualBox:~/dpdk/dpdk-2.0.0/x86_64-native-linuxapp-gcc/app$
>>>
>>> I understand that i have two cores and hence should use -c 1 (since i
>>> have
>>> cores 0 and 1). However, i dont understand what to give as the "-n"
>>> option.
>>> Is it always 4?
>>>
>>> Thanks, Abhishek
>>>
>>
>>
>

[dpdk-dev] Could not achieve wire speed for 40GE with any DPDK version on XL710 NIC's

[Keunhong Lee]

DISCLAIMER: This information is not verified. This is truly my personal
opinion.

As I know, intel 82599 is the only 10G NIC which supports line rate with
minimum sized packets (64 byte).
According to our internal tests, Mellanox's 40G NICs even support less than
30Mpps.
I think 40 Mpps is the hardware capacity.

Keunhong.



2015-06-28 19:34 GMT+09:00 Pavel Odintsov :

> Hello, folks!
>
> We have execute bunch of tests for receive data with Intel XL710 40GE
> NIC. We want to achieve wire speed on this platform for traffic
> capture.
>
> But we definitely can't do it. We tried with different versions of
> DPDK: 1.4, 1.6, 1.8, 2.0. And have not success.
>
> We achieved only 40Mpps and could do more.
>
> Could anybody help us with this issue? Looks like this NIC's could not
> work on wire speed :(
>
> Platform: Intel Xeon E5 e5 2670 + XL 710.
>
> --
> Sincerely yours, Pavel Odintsov
>

[dpdk-dev] Number of memory channels per processor socket?

[Keunhong Lee]

...and -c option gives 'masking' for CPUs.
-c 1 will only activate single core (especially second core) of your system.
If you want to activate both core, you have to give the mask '11'b ('3' in
decimal).


2015-06-28 18:54 GMT+09:00 Abhishek Verma :

> Hi,
>
> I am new to DPDK and i tried searching the archives in case this had been
> discussed but couldnt find any references and hence this email:
>
> What is the significance of " -n NUM: Number of memory channels per
> processor socket" which is passed as an EAL option? I have a virtual
> machine (VM) spawned using VirtualBox and i am trying to use DPDK to get
> faster access to packets there. I did lscpu, but that didnt give me
> anything interesting that i could use here:
>
> abhishekV at VirtualBox:~/dpdk/dpdk-2.0.0/x86_64-native-linuxapp-gcc/app$
> sudo
> lscpu
> [sudo] password for abhishekV:
>
> Architecture:  x86_64
> CPU op-mode(s):32-bit, 64-bit
> Byte Order:Little Endian
> CPU(s):2
> On-line CPU(s) list:   0,1
> Thread(s) per core:1
> Core(s) per socket:2
> Socket(s): 1
> NUMA node(s):  1
> Vendor ID: GenuineIntel
> CPU family:6
> Model: 58
> Stepping:  9
> CPU MHz:   2594.017
> BogoMIPS:  5188.03
> L1d cache: 32K
> L1d cache: 32K
> L2d cache: 6144K
> NUMA node0 CPU(s): 0,1
> abhishekV at VirtualBox:~/dpdk/dpdk-2.0.0/x86_64-native-linuxapp-gcc/app$
>
> I understand that i have two cores and hence should use -c 1 (since i have
> cores 0 and 1). However, i dont understand what to give as the "-n" option.
> Is it always 4?
>
> Thanks, Abhishek
>

[dpdk-dev] Number of memory channels per processor socket?

[Keunhong Lee]

See your motherboard spec.
For example, I'm using Intel 5930K with ASUS X99 Delux Motherboard (x99
chipset).
This site https://www.asus.com/us/Motherboards/X99DELUXE/specifications/
tells that
"Quad Channel Memory Architecture "
which means that "-n 4" is the correct configuration for me.

To fully use quad channel memory, I have to use 4 separated DRAM,
installed on different memory slots of my motherboard.

-n option optimizes memory bank access pattern for datastructures.
Since most motherboards support dual or quad memory channel,
-n 4 will work for most systems. (Only few motherboards support tri-channel
memory)

I hope that this information is useful to you.
Keunhong.



2015-06-28 18:54 GMT+09:00 Abhishek Verma :

> Hi,
>
> I am new to DPDK and i tried searching the archives in case this had been
> discussed but couldnt find any references and hence this email:
>
> What is the significance of " -n NUM: Number of memory channels per
> processor socket" which is passed as an EAL option? I have a virtual
> machine (VM) spawned using VirtualBox and i am trying to use DPDK to get
> faster access to packets there. I did lscpu, but that didnt give me
> anything interesting that i could use here:
>
> abhishekV at VirtualBox:~/dpdk/dpdk-2.0.0/x86_64-native-linuxapp-gcc/app$
> sudo
> lscpu
> [sudo] password for abhishekV:
>
> Architecture:  x86_64
> CPU op-mode(s):32-bit, 64-bit
> Byte Order:Little Endian
> CPU(s):2
> On-line CPU(s) list:   0,1
> Thread(s) per core:1
> Core(s) per socket:2
> Socket(s): 1
> NUMA node(s):  1
> Vendor ID: GenuineIntel
> CPU family:6
> Model: 58
> Stepping:  9
> CPU MHz:   2594.017
> BogoMIPS:  5188.03
> L1d cache: 32K
> L1d cache: 32K
> L2d cache: 6144K
> NUMA node0 CPU(s): 0,1
> abhishekV at VirtualBox:~/dpdk/dpdk-2.0.0/x86_64-native-linuxapp-gcc/app$
>
> I understand that i have two cores and hence should use -c 1 (since i have
> cores 0 and 1). However, i dont understand what to give as the "-n" option.
> Is it always 4?
>
> Thanks, Abhishek
>

[dpdk-dev] How to set timestamp in 82599 NICs in DPDK?

[Keunhong Lee]

http://dpdk.org/browse/dpdk/tree/app/test-pmd/ieee1588fwd.c

This code example contains enabling PTP with intel NICs.

Keunhong.



2015-06-01 12:48 GMT+09:00 Keunhong Lee :

> 82599 supports hw timestamping for PTP packets.
> I don't know whether it supports timestamping for general packets.
>
>
>
> http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L640
> says that
>
> 640 
> <http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L640>*
>  default:
> *641 
> <http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L641>*
>  /**642 
> <http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L642>*
>   * register RXMTRL must be set in order to do V1 
> packets,*643 
> <http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L643>*
>   * therefore it is not possible to time stamp both V1 Sync 
> and*644 
> <http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L644>
>  * * Delay_Req messages and hardware does not support**
> 645 
> <http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L645>
>   * timestamping all packets => return error
> *646 
> <http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L646>*
>   */*647 
> <http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L647>
>  config 
> <http://lxr.free-electrons.com/ident?i=config>*->rx_filter = 
> HWTSTAMP_FILTER_NONE;
> *648 
> <http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L648>*
>  return -*ERANGE 
> <http://lxr.free-electrons.com/ident?i=ERANGE>*;
> *649 
> <http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L649>*
>  }*
>
>
> Keunhong.
>
> 2015-06-01 11:38 GMT+09:00 ?? ??? :
>
>> Hi
>>
>>
>>
>> I noticed that there is a patch which can set hardware timestamp for the
>> received
>> packets(
>> http://www.wand.net.nz/trac/libtrace/browser/Intel%20DPDK%20Patches/
>> hardware_timestamp.patch?rev=ce7153dbc6a13c18bf8033af08c1249527754168
>> <http://www.wand.net.nz/trac/libtrace/browser/Intel%20DPDK%20Patches/hardware_timestamp.patch?rev=ce7153dbc6a13c18bf8033af08c1249527754168>),
>> but
>> it only works in e1000 and igb NICs. I want to capture packets with
>> timestamp info in 82599 NICs, what should I do? Could you give me some
>> help?
>>
>>
>>
>> Thanks.
>>
>>
>>
>> Chunhai Tan
>>
>>
>>
>>
>>
>>
>

[dpdk-dev] How to set timestamp in 82599 NICs in DPDK?

[Keunhong Lee]

82599 supports hw timestamping for PTP packets.
I don't know whether it supports timestamping for general packets.


http://lxr.free-electrons.com/source/drivers/net/ethernet/intel/ixgbe/ixgbe_ptp.c#L640
says that

640 
*
default:
*641 
*
/**642
*
 * register RXMTRL must be set in order to do V1
packets,*643 
*
 * therefore it is not possible to time stamp both V1
Sync and*644 

* * Delay_Req messages and hardware does not support**
645 

 * timestamping all packets => return error
*646 
*
 */*647

config
*->rx_filter =
HWTSTAMP_FILTER_NONE;
*648 
*
return -*ERANGE
*;
*649 
*
}*


Keunhong.

2015-06-01 11:38 GMT+09:00 ?? ??? :

> Hi
>
>
>
> I noticed that there is a patch which can set hardware timestamp for the
> received
> packets(
> http://www.wand.net.nz/trac/libtrace/browser/Intel%20DPDK%20Patches/
> hardware_timestamp.patch?rev=ce7153dbc6a13c18bf8033af08c1249527754168),
> but
> it only works in e1000 and igb NICs. I want to capture packets with
> timestamp info in 82599 NICs, what should I do? Could you give me some
> help?
>
>
>
> Thanks.
>
>
>
> Chunhai Tan
>
>
>
>
>
>

[dpdk-dev] [PATCH 0/2] Mellanox ConnectX-3 PMD

[Keunhong Lee]

How fast does this driver perform?
Is it capable for sending/receiving 64B packets at 40G line rate?
I'm using another version of user driver and it is not scalable for line
rate, regardless of the number of TX cores.

Keunhong.
2015. 1. 30. ?? 12:22? "Adrien Mazarguil" ?? ??:

> This PMD adds support for Mellanox ConnectX-3-based adapters through the
> verbs framework. It relies on external libraries (libibverbs and user space
> driver libmlx4) and kernel support to do so.
>
> While these libraries and kernel modules are available on OpenFabrics
> Alliance's website [1] and provided by package managers on most
> distributions, this PMD requires Ethernet extensions that may not be
> supported at the moment (this is a work in progress).
>
> Mellanox OFED [2] includes the necessary support and should be used in the
> meantime. For DPDK, only libibverbs, libmlx4 and mlnx-ofed-kernel packages
> are required from that distribution.
>
> The following kernel modules must be loaded before using this PMD:
>
> - mlx4_core (hardware driver, does global initialization)
> - mlx4_en (Ethernet device driver)
> - mlx4_ib (InfiniBand device driver)
> - ib_uverbs (user space driver for verbs)
>
> Actual documentation will be added in V2.
>
> [1] https://www.openfabrics.org/
> [2]
> http://www.mellanox.com/page/products_dyn?product_family=26=linux_sw_drivers
>
> Adrien Mazarguil (2):
>   scripts: add auto-config-h.sh
>   mlx4: new poll mode driver
>
>  config/common_bsdapp |   11 +
>  config/common_linuxapp   |   11 +
>  lib/Makefile |1 +
>  lib/librte_pmd_mlx4/Makefile |  119 ++
>  lib/librte_pmd_mlx4/mlx4.c   | 4739
> ++
>  lib/librte_pmd_mlx4/mlx4.h   |  166 ++
>  mk/rte.app.mk|8 +
>  scripts/auto-config-h.sh |  137 ++
>  8 files changed, 5192 insertions(+)
>  create mode 100755 lib/librte_pmd_mlx4/Makefile
>  create mode 100755 lib/librte_pmd_mlx4/mlx4.c
>  create mode 100644 lib/librte_pmd_mlx4/mlx4.h
>  create mode 100755 scripts/auto-config-h.sh
>
> --
> 2.1.0
>
>

[dpdk-dev] Bugs in newest patches

[Keunhong Lee]

I did not included the generic header directly.
I thought that this nested "extern C" is intended,
however, according to your reply, this might be removed at all.

There is a remaining
"""
#ifdef _cplusplus
}
#endif
"""
at the end of "generic/rte_cycles.h".
I think this should be removed. This causes compile error on C++.

Keunhong.


2014-11-07 6:05 GMT+09:00 Thomas Monjalon :

> Hi,
>
> 2014-11-07 05:21, Keunhong Lee:
> > I just pulled new patches from the master branch, and found that it
> doesn't
> > work with C++.
> >
> > in  lib/librte_eal/common/include/generic/rte_cycles.h
> > +#ifdef __cplusplus
> > +extern "C" {
> > +#endif
>
> It's already included in
> lib/librte_eal/common/include/arch/x86/rte_cycles.h
> which includes lib/librte_eal/common/include/generic/rte_cycles.h.
>
> You shouldn't include the generic header directly.
>
> --
> Thomas
>

[dpdk-dev] Bugs in newest patches

[Keunhong Lee]

Hi.
I just pulled new patches from the master branch, and found that it doesn't
work with C++.

in  lib/librte_eal/common/include/generic/rte_cycles.h
missing opening extern "C" in
commit fa4001c30ee9d9ecfb3ca5d93d68ef0394e9950a

here is the fix.

-

  * Simple Time Reference Functions (Cycles and HPET).
  */

+#ifdef __cplusplus
+extern "C" {
+#endif
+
 #include 
 #include 
 #include 



Keunhong.

[dpdk-dev] Can not init NIC after merge to DPDK 1.7 problem

[Keunhong Lee]

In DPDK1.7, PMDs are loaded by __constructor__ functions.
As there is no direct reference to each PMD driver structures, linker may
not link the PMD library.
You should use -Wl,--whole-archive and -Wl,--no-whole-archive for linking
DPDK applications.
See http://dpdk.org/browse/dpdk/tree/mk/rte.app.mk for further information.

Keunhong.



2014-09-24 4:12 GMT+09:00 Neil Horman :

> On Tue, Sep 23, 2014 at 06:53:57PM +, Wang, Shawn wrote:
> > Hi:
> >
> > We are using our own Makefile in building dpdk program. Recently we are
> working on upgrading from DPDK 1.3 to DPDK 1.7. I found the
> rte_ixgbe_pmd_init has been replaced by PMD_REGISTER_DRIVER. So I delete
> rte_ixgbe_pmd_init calls. But after that, our dpdk program could not
> correctly find the NIC anymore. After digging into it a little more, I
> found the code dose not correctly register the driver type we are using,
> which is ixgbe.
> > To isolate the problem, I hacked a smal example l3fwd, and only have the
> main.c file like this for my testing purpose.
> >
> > #include 
> > #include 
> >
> > #include "main.h"
> >
> > int
> > MAIN(int argc, char **argv)
> > {
> > /* init EAL */
> > int ret = rte_eal_init(argc, argv);
> > printf("ret %d\n", ret);
> > return 0;
> > }
> >
> > I found if I use the Makefile provided in the example, the program will
> find the ixgbe NIC. But if I just use these 2 commands to compile and link
> it. It will not find the ixgbe NIC.
> >
> > gcc -I../../x86_64-native-linuxapp-gcc/include
> -L../../x86_64-native-linuxapp-gcc/lib -lrte_eal -c main.c
> > gcc -o l3fwd main.o -L../../x86_64-native-linuxapp-gcc/lib -lrte_eal
> -lrte_distributor -lrte_pipeline -lrte_port -lrte_timer -lrte_hash
> -lrte_acl -lm -lrt -lrte_mbuf -lethdev -lrte_malloc -lrte_mempool
> -lrte_ring -lc -lm -lrte_cmdline -lrte_cfgfile -lrte_pmd_bond
> -lrte_pmd_ixgbe -lrte_pmd_e1000 -lrte_pmd_ring -lpthread -ldl -lrt
> >
> > Can someone share some light on what is magic of the dpdk Makefile to
> correctly register the NIC type?
> >
> > Thank you so much.
> > Xingbo Wang
> >
>
> I'm not really sure why you would strip out the Makefiles to dpdk, but I
> suppose
> thats not the germaine question.
>
> First, how are you building the DPDK?  As a set of shared libraries, or as
> a set
> of static archives?  If you're building shared libraries, you need to pass
> -shared to gcc, or the constructors will get stripped out using your
> command
> line above.  There might be some other options that escape me, but you can
> find
> out for sure by using the packaged makefiles and running make V=1 to see
> all the
> passed options in the link stage
>
> Secondly, when you say register the NIC type, do you mean that you don't
> see the
> NIC get registered with dpdk, or you don't see an instance of the NIC
> created?
> If its the former, you need to confirm that by running a debugger and
> looking at
> what elements are on the device_list after your applications starts.  If
> its the
> latter, that may well be a config error, as you may need to pass the
> --whitelist
> option on the command line to trigger a device probe.
>
> Neil
>
>