[dpdk-dev] tcpdump support in DPDK 2.3

[Gray, Mark D]

> Bruce,
> 
> Please reconsider your interpretation of the word "debuggability".
> Debugging is not only something that R staff does in a lab. Debuggability
> can also be interpreted as a network engineer's ability to debug what is
> happening in a production network.

Is tcpdump used in large production cloud environments? I would have 
thought other less intrusive (and less manual) tools would be used? Isn't
that one of the benefits of SDN.

[dpdk-dev] tcpdump support in DPDK 2.3

[Gray, Mark D]

> This is something also being looked for by folks such as those
> working on OVS e.g. called out at
> http://openvswitch.org/pipermail/dev/2015-August/058814.html
> 
>   "- Insight into the system and debuggability: nothing beats tcpdump for the
> kernel datapath.  Can something similar be done for the userspace
> datapath?
> 
>   - Consistency of the tools: some commands are slightly different for the
> userspace/kernel datapath.  Ideally there shouldn't be any difference."
> 

I had a painful experience with OVS-DPDK recently which may be representative
of a typical usability issue encountered. 

I was trying to connect two Openstack compute nodes together.  I had done
the configuration without DPDK first. It was easy to debug as I could use
tcpdump to look at the eth ports and see what type of traffic
was entering the compute node. I also needed to check if the traffic
was actually VxLAN traffic and what the VNI was in order to be able to
follow the traffic around the bridges in OVS. This all went quite well and
I was able to bring up my set up quite easily. 

Then I tried to set up the same thing with DPDK. I couldn't get traffic between
the compute nodes but I had no easy way to just dump the traffic coming into
(or out of) the compute node. Of course, there were some things I could do but,
for me, DPDK would be far more usable if I could just use tcpdump. As I know
DPDK to some extent, I can usually get around these problems but I suspect
that a new user to DPDK  would get very discouraged and frustrated by an 
experience like that. 

I'm not sure how often tcpdump is used in production environments but it is
very useful when debugging a live system without having to modify code. It 
would be
good if it could work at high rates and be really flexible but it probably makes
sense to focus on the basics first.

[dpdk-dev] [ovs-discuss] [announce] driverctl: utility for persistent alternative driver binding

[Gray, Mark D]

> -Original Message-
> From: Panu Matilainen [mailto:pmatilai at redhat.com]
> Sent: Tuesday, December 8, 2015 12:05 PM
> To: Gray, Mark D; dev at dpdk.org; users at dpdk.org; dev at openvswitch.org;
> discuss at openvswitch.org
> Subject: Re: [ovs-discuss] [announce] driverctl: utility for persistent
> alternative driver binding
> 
> On 12/04/2015 05:44 PM, Gray, Mark D wrote:
> > I welcome this initiative, one question below:
> >
> >> -Original Message-
> >> From: discuss [mailto:discuss-bounces at openvswitch.org] On Behalf Of
> >> Panu Matilainen
> >> Sent: Friday, December 4, 2015 10:54 AM
> >> To: dev at dpdk.org; users at dpdk.org; dev at openvswitch.org;
> >> discuss at openvswitch.org
> >> Subject: [ovs-discuss] [announce] driverctl: utility for persistent
> >> alternative driver binding
> >>
> >> Hi all,
> >>
> >> While this is not directly related to DPDK or OVS, it is potentially
> >> useful for users of both, so excuse me for cross-posting.
> >>
> >> Quoting from the project README (for the full text see
> >> http://laiskiainen.org/git/?p=driverctl.git;a=blob_plain;f=README)
> >>
> >>   > driverctl is a tool for manipulating and inspecting the system
> >>   > device driver choices.
> >>   >
> >>   > Devices are normally assigned to their sole designated kernel driver
> >>   > by default. However in some situations it may be desireable to
> >>   > override that default, for example to try an older driver to
> >>   > work around a regression in a driver or to try an experimental
> >>   > alternative driver. Another common use-case is pass-through
> >>   > drivers and driver stubs to allow userspace to drive the device,
> >>   > such as in case of virtualization.
> >>   >
> >>   > driverctl integrates with udev to support overriding
> >>   > driver selection for both cold- and hotplugged devices from the
> >>   > moment of discovery, but can also change already assigned drivers,
> >>   > assuming they are not in use by the system. The driver overrides
> >>   > created by driverctl are persistent across system reboots
> >>   > by default.
> >>   >
> >>   > Usage
> >>   > -
> >>   >
> >>   > Find devices currently driven by ixgbe driver:
> >>   >
> >>   > # driverctl -v list-devices | grep ixgbe
> >>   > :01:00.0 ixgbe (Ethernet 10G 4P X520/I350 rNDC)
> >>   > :01:00.1 ixgbe (Ethernet 10G 4P X520/I350 rNDC)
> >>   >
> >>   > Change them to use the vfio-pci driver:
> >>   > # driverctl set-override :01:00.0 vfio-pci
> >>   > # driverctl set-override :01:00.1 vfio-pci
> >>   >
> >>   > Find devices with driver overrides:
> >>   > # driverctl -v list-devices|grep \\*
> >>   > :01:00.0 vfio-pci [*] (Ethernet 10G 4P X520/I350 rNDC)
> >>   > :01:00.1 vfio-pci [*] (Ethernet 10G 4P X520/I350 rNDC)
> >>   >
> >>   > Remove the override from slot :01:00.1:
> >>   > # driverctl unset-override :01:00.1
> >>
> >> DPDK of course has its own dpdk_nic_bind(.py) tool for this purpose,
> >> the main differences to driverctl are:
> >> - driverctl bindings are persistent across system boots
> >
> >   [Gray, Mark D] This is great!
> >
> > Will this integrate with, for example in Red Hat-based systems,
> > /etc/sysconfig/network-scripts/ifcfg-X? In DPDK, could we then potentially
> reference devices by that (arbitrary) name?
> 
> driverctl is not specific to NICs so network-scripts integration is out of 
> scope.
> 

I forgot that point. It makes sense.

> That aside, maybe I'm missing something but I'm not sure what there is to
> integrate with since DPDK ports are ultimately application specific.
> For OVS I've sent a patch to support managing OVS DPDK ports via
> network-scripts:
> http://openvswitch.org/pipermail/dev/2015-December/062850.html

Thanks, I wasn?t familiar with this work but this is a good approach.

> 
>   . Panu -
> 
What I am thinking would be an ideal end state for OVS would be the possibility 
to add a port using ovs-vsctl add-port referencing the device name

ovs-vsctl add-port p1p2

If dpdk was enabled, then OVS could query something(?) to determine which 
physical interface this name actually referred to and then from that figure out 
what dpdk port number it referred to. 

[dpdk-dev] [ovs-discuss] [announce] driverctl: utility for persistent alternative driver binding

[Gray, Mark D]

I welcome this initiative, one question below:

> -Original Message-
> From: discuss [mailto:discuss-bounces at openvswitch.org] On Behalf Of Panu
> Matilainen
> Sent: Friday, December 4, 2015 10:54 AM
> To: dev at dpdk.org; users at dpdk.org; dev at openvswitch.org;
> discuss at openvswitch.org
> Subject: [ovs-discuss] [announce] driverctl: utility for persistent 
> alternative
> driver binding
> 
> Hi all,
> 
> While this is not directly related to DPDK or OVS, it is potentially
> useful for users of both, so excuse me for cross-posting.
> 
> Quoting from the project README (for the full text see
> http://laiskiainen.org/git/?p=driverctl.git;a=blob_plain;f=README)
> 
>  > driverctl is a tool for manipulating and inspecting the system
>  > device driver choices.
>  >
>  > Devices are normally assigned to their sole designated kernel driver
>  > by default. However in some situations it may be desireable to
>  > override that default, for example to try an older driver to
>  > work around a regression in a driver or to try an experimental
>  > alternative driver. Another common use-case is pass-through
>  > drivers and driver stubs to allow userspace to drive the device,
>  > such as in case of virtualization.
>  >
>  > driverctl integrates with udev to support overriding
>  > driver selection for both cold- and hotplugged devices from the
>  > moment of discovery, but can also change already assigned drivers,
>  > assuming they are not in use by the system. The driver overrides
>  > created by driverctl are persistent across system reboots
>  > by default.
>  >
>  > Usage
>  > -
>  >
>  > Find devices currently driven by ixgbe driver:
>  >
>  > # driverctl -v list-devices | grep ixgbe
>  > :01:00.0 ixgbe (Ethernet 10G 4P X520/I350 rNDC)
>  > :01:00.1 ixgbe (Ethernet 10G 4P X520/I350 rNDC)
>  >
>  > Change them to use the vfio-pci driver:
>  > # driverctl set-override :01:00.0 vfio-pci
>  > # driverctl set-override :01:00.1 vfio-pci
>  >
>  > Find devices with driver overrides:
>  > # driverctl -v list-devices|grep \\*
>  > :01:00.0 vfio-pci [*] (Ethernet 10G 4P X520/I350 rNDC)
>  > :01:00.1 vfio-pci [*] (Ethernet 10G 4P X520/I350 rNDC)
>  >
>  > Remove the override from slot :01:00.1:
>  > # driverctl unset-override :01:00.1
> 
> DPDK of course has its own dpdk_nic_bind(.py) tool for this purpose, the
> main differences to driverctl are:
> - driverctl bindings are persistent across system boots

 [Gray, Mark D] This is great!

Will this integrate with, for example in Red Hat-based systems, 
/etc/sysconfig/network-scripts/ifcfg-X? In DPDK, could we then
potentially reference devices by that (arbitrary) name?

> - driverctl bindings take place immediately on cold- and hotplug
> - driverctl is a generic tool not limited to network adapters
> - dpdk_nic_bind being a special purpose tool has many more
>sanity checks for its supported use-cases
> - dpdk_nic_bind supports binding multiple NICs at once
> 
> The project currently lives at
>  http://laiskiainen.org/git/?p=driverctl.git
> 
> Feedback, patches etc are welcome.
> 
>   - Panu -
> ___
> discuss mailing list
> discuss at openvswitch.org
> http://openvswitch.org/mailman/listinfo/discuss

[dpdk-dev] DPDK OVS on Ubuntu 14.04

[Gray, Mark D]

+ discuss at openvswitch.org

one comment below: 

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Polehn, Mike A
> Sent: Tuesday, December 1, 2015 2:46 PM
> To: Abhijeet Karve; dev at dpdk.org
> Cc: bhavya.addep at gmail.com
> Subject: Re: [dpdk-dev] DPDK OVS on Ubuntu 14.04
> 
> May need to setup huge pages on kernel boot line (this is example, you may
> need to adjust):
> 
> The huge page configuration can be added to the default configuration file
> /etc/default/grub by adding to the GRUB_CMDLINE_LINUX and the grub
> configuration file regenerated to get an updated configuration file for Linux
> boot.
> # vim /etc/default/grub// edit file
> 
> . . .
> GRUB_CMDLINE_LINUX_DEFAULT="... default_hugepagesz=1GB
> hugepagesz=1GB hugepages=4 hugepagesize=2m hugepages=2048 ..."
> . . .
> 
> 
> This example sets up huge pages for both 1 GB pages for 4 GB of 1 GB
> hugepage memory and 2 MB pages for 4 GB of 2 MB hugepage memory.
> After boot the number of 1 GB pages cannot be changed, but the number of
> 2 MB pages can be changed.
> 
> After editing configuration file /etc/default/grub , the new grub.cfg boot 
> file
> needs to be regenerated:
> # update-grub
> 
> And reboot. After reboot memory managers need to be setup:
> 
> If /dev/hugepages does not exist:# mkdir /dev/hugepages
> 
> # mount -t hugetlbfs nodev   /dev/hugepages
> 
> # mkdir /dev/hugepages_2mb
> # mount -t hugetlbfs nodev /dev/hugepages_2mb -o pagesize=2MB
> 
> Mike
> 
> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Abhijeet Karve
> Sent: Monday, November 30, 2015 10:14 PM
> To: dev at dpdk.org
> Cc: bhavya.addep at gmail.com
> Subject: [dpdk-dev] DPDK OVS on Ubuntu 14.04
> 
> Dear All,
> 
> 
> We are trying to install DPDK OVS on top of the openstack juno in Ubuntu
> 14.04 single server. We are referring following steps for executing the same.
> 
> https://software.intel.com/en-us/blogs/2015/06/09/building-vhost-user-for-
> ovs-today-using-dpdk-200
> 
> During execution we are getting some issues with ovs-vswitchd service as its
> getting hang during starting.
> __
> ___
> 
> nfv-dpdk at nfv-dpdk:~$ tail -f /var/log/openvswitch/ovs-vswitchd.log
> 2015-11-
> 24T10:54:34.036Z|6|reconnect|INFO|unix:/var/run/openvswitch/db.so
> ck:
> connecting...
> 2015-11-
> 24T10:54:34.036Z|7|reconnect|INFO|unix:/var/run/openvswitch/db.so
> ck:
> connected
> 2015-11-24T10:54:34.064Z|8|bridge|INFO|ovs-vswitchd (Open vSwitch)
> 2.4.90
> 2015-11-24T11:03:42.957Z|2|vlog|INFO|opened log file
> /var/log/openvswitch/ov
>  
> s-vswitchd.log 2015-11-
> 24T11:03:42.958Z|3|ovs_numa|INFO|Discovered 24 CPU cores on
> NUMA
> nod
> e 0
> 2015-11-24T11:03:42.958Z|4|ovs_numa|INFO|Discovered 24 CPU cores
> on NUMA
> nod
> e 1
> 2015-11-24T11:03:42.958Z|5|ovs_numa|INFO|Discovered 2 NUMA
> nodes and
> 48 CPU
>  cores
> 2015-11-
> 24T11:03:42.958Z|6|reconnect|INFO|unix:/var/run/openvswitch/db.so
> ck:
> connecting...
> 2015-11-
> 24T11:03:42.958Z|7|reconnect|INFO|unix:/var/run/openvswitch/db.so
> ck:
> connected
> 2015-11-24T11:03:42.961Z|8|bridge|INFO|ovs-vswitchd (Open vSwitch)
> 2.4.90
> __
> ___
> 
> Also, attaching output(Hugepage.txt) of  ? ./ovs-vswitchd --dpdk -c 0x0FF8 -n
> 4 --socket-mem 1024,0 -- --log-file=/var/log/openvswitch/ovs-vswitchd.log
> --pidfile=/var/run/oppenvswitch/ovs-vswitchd.pid?
> 
> -  We tried seting up echo 0 >
> /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages, but couldn?t
> succeeded.
>   Can anyone please help us in getting the things if we are missing any 
> and
> causing ovs-vswitchd to stuck while starting?
> 
> Also, when we create vm in openstack with DPDK OVS, dpdkvhost-user type
> interfaces are getting created automatically. If this interfaces are getting
> mapped with regular br-int bridge rather than DPDK bridge br0 then is this

You can still have a bridge named br-int  that is backed with a userspace 
datapath. You can't add
a dpdkvhostuser port to a kernel space datapath. So in this case, I think you 
are ok and are
using DPDK.

> mean that we have successfully enabled DPDK with netdev datapath?
> 
> 
> 
> We really appreciate for all the advice if you have.
> 
> Thanks,
> Abhijeet
> Thanks & Regards
> Abhijeet Karve
> 
> =-=-=
> Notice: The information contained in this e-mail message and/or
> attachments to it may contain confidential or privileged information. If you
> are not the intended recipient, any dissemination, use, review, distribution,
> printing or copying of the information contained in this e-mail message
> and/or attachments to it are strictly prohibited. If you have received this
> communication in error, please 

[dpdk-dev] OVS-DPDK performance problem on ixgbe vector PMD

[Gray, Mark D]

> 
> Hi,
> 
> I've set up a simple packet forwarding perf test on a dual-port 10G
> 82599ES: one port receives 64 byte UDP packets, the other sends it out,
> one core used. I've used latest OVS with DPDK 2.1, and the first result
> was only 13.2 Mpps, which was a bit far from the 13.9 I've seen last
> year with the same test. The first thing I've changed was to revert back
> to the old behaviour about this issue:
> 
> http://permalink.gmane.org/gmane.comp.networking.dpdk.devel/22731
> 
> So instead of the new default I've passed 2048 +
> RTE_PKTMBUF_HEADROOM.

We'll post a patch this week that should resolve this (if it is the same issue).

> That increased the performance to 13.5, but to figure out what's wrong
> started to play with the receive functions. First I've disabled vector
> PMD, but ixgbe_recv_pkts_bulk_alloc() was even worse, only 12.5 Mpps. So
> then I've enabled scattered RX, and with
> ixgbe_recv_pkts_lro_bulk_alloc() I could manage to get 13.98 Mpps, which
> is I guess as close as possible to the 14.2 line rate (on my HW at
> least, with one core)
> Does anyone has a good explanation about why the vector PMD performs so
> significantly worse? I would expect that on a 3.2 GHz i5-4570 one core
> should be able to reach ~14 Mpps, SG and vector PMD shouldn't make a
> difference.
> I've tried to look into it with oprofile, but the results were quite
> strange: 35% of the samples were from miniflow_extract, the part where
> parse_vlan calls data_pull to jump after the MAC addresses. The oprofile
> snippet (1M samples):
> 
>511454 190.0037  flow.c:511
>511458 149   0.0292  dp-packet.h:266
>51145f 4264  0.8357  dp-packet.h:267
>511466 180.0035  dp-packet.h:268
>51146d 430.0084  dp-packet.h:269
>511474 172   0.0337  flow.c:511
>51147a 4320  0.8467  string3.h:51
>51147e 358763   70.3176  flow.c:99
>511482 23.9e-04  string3.h:51
>511485 3060  0.5998  string3.h:51
>511488 1693  0.3318  string3.h:51
>51148c 2933  0.5749  flow.c:326
>511491 470.0092  flow.c:326
> 
> And the corresponding disassembled code:
> 
>511454:   49 83 f9 0d cmpr9,0xd
>511458:   c6 83 81 00 00 00 00movBYTE PTR [rbx+0x81],0x0
>51145f:   66 89 83 82 00 00 00movWORD PTR [rbx+0x82],ax
>511466:   66 89 93 84 00 00 00movWORD PTR [rbx+0x84],dx
>51146d:   66 89 8b 86 00 00 00movWORD PTR [rbx+0x86],cx
>511474:   0f 86 af 01 00 00   jbe511629
> 
>51147a:   48 8b 45 00 movrax,QWORD PTR [rbp+0x0]
>51147e:   4c 8d 5d 0c lear11,[rbp+0xc]
>511482:   49 89 00movQWORD PTR [r8],rax
>511485:   8b 45 08moveax,DWORD PTR [rbp+0x8]
>511488:   41 89 40 08 movDWORD PTR [r8+0x8],eax
>51148c:   44 0f b7 55 0c  movzx  r10d,WORD PTR [rbp+0xc]
>511491:   66 41 81 fa 81 00   cmpr10w,0x81
> 
> My only explanation to this so far is that I misunderstand something
> about the oprofile results.
> 
> Regards,
> 
> Zoltan

[dpdk-dev] Status of Open vSwitch with DPDK

[Gray, Mark D]

Hi Daniele,

Thanks for starting this conversation. It is a good list :) I have 
crossed-posted this
to dpdk.org as I feel that some of the points could be interesting to that 
community
as they are related to how DPDK is used.

How do "users" of OVS with DPDK feel about this list? Does anyone disagree or
does anyone have any additions? What are your experiences?

> 
> There has been some discussion lately about the status of the Open vSwitch
> port to DPDK.  While part of the code has been tested for quite some time,
> I think we can agree that there are a few rough spots that prevent it from
> being easily deployed and used.
> 
> I was hoping to get some feedback from the community about those rough
> spots,
> i.e. areas where OVS+DPDK can/needs to improve to become more
> "production
> ready" and user-friendly.
> 
> - PMD threads and queues management: the code has shown several bugs
> and
> the
>   netdev interfaces don't seem up to the job anymore.

You had a few ideas about how to refactor this before but I was concerned 
about the effect it would have on throughput. I can't find the thread. 

Do you have some further ideas about how to achieve this?

> 
>   There's a lot of margin of improvement: we could factor out the code from
>   dpif-netdev, add configuration parameters for advanced users, and figure
> out
>   a way to add unit tests.
> 

I think this is a general issue with both the kernel datapath (and netdevs)
and the userspace datapath. There isn't much unit testing (or testing) outside
of the slow path. 

>   Related to this, the system should be as fast as possible out-of-the-box,
>   without requiring too much tuning.

This is a good point. I think the kernel datapath has a similar issue. You can
get a certain level of performance without compiling with -Ofast or
pinning threads but you will (even with the kernel datapath) get better
performance if you pin threads (and possibly compile differently). I guess
it is more visible with the dpdk datapath as performance is one of the key
values. It is also more detrimental to the performance if you don't set it
up correctly.

Perhaps we could provide scripts to help do this?

I think this is also interesting to the DPDK community. There is 
knowledge required when running DPDK enabled apps to
get good performance: core pinning is one thing that comes to mind.

> 
> - Userspace tunneling: while the code has been there for quite some time it
>   hasn't received the level of testing that the Linux kernel datapath
> tunneling
>   has.
> 

Again, there is a lack of test infrastructure in general for OVS. vsperf is a 
good
start, and it would be great to see more people use and contribute to it!

> - Documentation: other than a step by step tutorial,  it cannot be said
> that
>   DPDK is a first class citizen in the OVS documentation.  Manpages could
> be
>   improved.

Easily done. The INSTALL guide is pretty good but the structure could be 
better. 
There is also a lack of manpages. Good point.

> 
> - Vhost: the code has not received the level of testing of the kernel
> vhost.
>   Another doubt shared by some developers is whether we should keep
>   vhost-cuse, given its relatively low ease of use and the overlapping with
>   the far more standard vhost-user.

vhost-cuse is required for older versions of qemu. I'm aware of some companies
using it as they are restricted to an older version of qemu. I think it is 
deprecated
at the moment? Is there a notice to that effect? We just need a plan for when to
remove it and make sure that plan is clear?

> 
> - Interface management and naming: interfaces must be manually removed
> from
>   the kernel drivers.
> 
>   We still don't have an easy way to identify them. Ideas are welcome: how
> can
>   we make this user friendly?  Is there a better solution on the DPDK side?

This is a tough one and is interesting to the DPDK community.  The basic issue
here is that users are more familiar with linux interfaces and linux naming
conventions.

"ovs-vsctl add-port bro eth0" makes a lot more sense than

"dpdk_nic_bind -b igb_uio ", then check the order that the ports
are enumerated and then run "ovs-vsctl add-port br0 dpdkN".

I can think of ways to do this with physical NICs. For example,
you could reference the port by the linux name and when you try to add it, OVS
could unbind from the kernel module and bind it to igb_uio?

However, I am not sure how you would do it with virtual nics as there is not
even a real device.

I think a general solution from the dpdk community would be really helpful here.
> 
>   How are DPDK interfaces handled by linux distributions? I've heard about
>   ongoing work for RHEL and Ubuntu, it would be interesting to coordinate.
> 
> 
> - Insight into the system and debuggability: nothing beats tcpdump for the
>   kernel datapath.  Can something similar be done for the userspace
> datapath?

Yeah, this would be useful. I have my own way of dealing with this. For example,
you could dump 

[dpdk-dev] vswitches performance comparison

[Gray, Mark D]

> "Jun Xiao"  wrote:
> 
> > After CloudNetEngine vswitch technical preview is launched, we
> > received quite a few queries on vswitches performance comparison, but
> > we cannot simply give a  test result on our platform because
> > performance varies on different H/Ws and different workloads, and
> > that's why we encourage you to try the evaluation package to get real data
> on your setup.
> >
> > Anyway, we share a little more performance data on our H/W which is a
> > comparison  among native kernel OVS/OVS-DPDK/CNE vswitch under the
> most common workload:
> > concurrent bi-directional TCP traffics cross hosts, and hope you can have a
> rough idea.
> > http://www.cloudnetengine.com/weblog/2015/07/22/vswitches-
> performance-
> > comparison/
> >
> > Thanks,
> > Jun
> 
> Since the real bottleneck in most vswitches is per-packet overhead.
> I would recommend running RFC-2544 tests for better data.
> 
> You probably need to use something like pktgen to get enough packets-per-
> second.

Yeah this is the methodology that we use aswell.

[dpdk-dev] [ovs-discuss] vswitches performance comparison

[Gray, Mark D]

> >>
> >> I'd like to hope that's my methodology problem, but I just follow the
> >> installation guide without any customization.
> >>
> >> Hi Mark, do you have any performance data share with us? Maybe we are
> >> using different type of workloads, like I mentioned I am using
> >> typical data center workload, I guess you are talking about NFV type of
> workload?
> >
> > The number getting floated around on the mailing list recently is
> > 16.5Mpps for phy-phy. However, I don't think we have any iperf data
> > off-hand for your usecase. When we test throughput into the vm we
> > usually generate the traffic externally and send NIC->OVS->VM->OVS-
> >NIC. This is a little different to your setup.
> >
> 
> I guess pmd driver is used inside VM in that case, right?

Yes, but even when we use virtio-net we see the same if not *slightly* better
performance.

> > I do know, however, that ovs-dpdk typically has a much larger
> > throughput than the kernel space datapath.
> >
> 
> I'd like to say it depends on workloads, for small/medium packet size
> workload, that's definitely true, while for TSO size workload, it's not that
> obvious (or worse) as data path overheads are amortized and H/W can be
> leveraged.

For large packets the switch will eventually saturate the NIC at line rate but 
the
total aggregate throughput of the switch should be faster (you could
add more interfaces for example to take advantage of that). 

TSO is missing from the DPDK ports at the moment but it is something
we plan to look at. We are currently enabling Jumbo frames (which don't
work at the moment).

> > Have you seen this?
> > https://wiki.opnfv.org/characterize_vswitch_performance_for_telco_nfv_
> > use_cases
> >
> 
> Thanks for the pointer, I'll try later.
> >>
> >> Thanks,
> >> Jun

[dpdk-dev] [ovs-discuss] vswitches performance comparison

[Gray, Mark D]

> 
> I'd like to hope that's my methodology problem, but I just follow the
> installation guide without any customization.
> 
> Hi Mark, do you have any performance data share with us? Maybe we are
> using different type of workloads, like I mentioned I am using typical data
> center workload, I guess you are talking about NFV type of workload?

The number getting floated around on the mailing list recently is 16.5Mpps
for phy-phy. However, I don't think we have any iperf data off-hand for your
usecase. When we test throughput into the vm we usually generate the traffic 
externally
and send NIC->OVS->VM->OVS->NIC. This is a little different to your setup.

I do know, however, that ovs-dpdk typically has a much larger throughput than
the kernel space datapath.

Have you seen this? 
https://wiki.opnfv.org/characterize_vswitch_performance_for_telco_nfv_use_cases

> 
> Thanks,
> Jun

[dpdk-dev] [ovs-discuss] vswitches performance comparison

[Gray, Mark D]

> -Original Message-
> From: discuss [mailto:discuss-bounces at openvswitch.org] On Behalf Of Jun
> Xiao
> Sent: Tuesday, July 21, 2015 7:01 PM
> To: discuss; dev
> Subject: [ovs-discuss] vswitches performance comparison
> 
> After CloudNetEngine vswitch technical preview is launched, we received
> quite a few queries on vswitches performance comparison, but we cannot
> simply give a  test result on our platform because performance varies on
> different H/Ws and different workloads, and that's why we encourage you to
> try the evaluation package to get real data on your setup.
> 
> Anyway, we share a little more performance data on our H/W which is a
> comparison  among native kernel OVS/OVS-DPDK/CNE vswitch under the
> most common workload:
> concurrent bi-directional TCP traffics cross hosts, and hope you can have a
> rough idea.
> http://www.cloudnetengine.com/weblog/2015/07/22/vswitches-
> performance-comparison/

I think there is an issue with you methodology. ovs-dpdk performance should be
significantly higher than kernel ovs.

> 
> Thanks,
> Jun
> ___
> discuss mailing list
> discuss at openvswitch.org
> http://openvswitch.org/mailman/listinfo/discuss

[dpdk-dev] OVDK userspace vhost : Issue with VIRTIO_RING_F_INDIRECT_DESC capability

[Gray, Mark D]

Cross posting to OVDK list.

On a side point, Have you looked at the DPDK enabled build in upstream OVS?

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of sai kiran
> Sent: Wednesday, July 1, 2015 1:02 PM
> To: dev at dpdk.org
> Subject: [dpdk-dev] OVDK userspace vhost : Issue with
> VIRTIO_RING_F_INDIRECT_DESC capability
> 
> Hi,
> 
> I am using OVDK userspace-vhost interfaces for VM-to-external
> communication and facing an issue with them.
> 
> I am using the topology mentioned in
> https://github.com/01org/dpdk-
> ovs/blob/development/docs/04_Sample_Configurations/02_Userspace-
> vHost.md
> But the Guest is a Freebsd VM and our own custom userspace virtio drivers.
> Guest does not have DPDK.
> 
> 
>1. I could start OVDK application and vswitchd, and provision a FreeBSD
>Guest VM with two userspace-vhost interfaces.
>2. These two userspace-vhost interfaces do not have
> VIRTIO_RING_F_INDIRECT_DESC
>capability negotiated from backend OVDK-Qemu
>3. Inside the freebsd VM, i have my own userspace drivers running, which
>make use of indirect descriptors of virtio.
>4. *Question 1* : By default, without INDIRECT descriptor capability, my
>drivers fail to run.Is there any way to increase ring size of
>virtio? This can help us  avoid using indirect descriptors because of
>more space in the ring
>5. *Question 2*: When I try to set the capability from backend QEMU,
>guest drivers do not see any packets reaching the guest. Is there
>any way to resolve this ??
> 
> Any help/suggestion would be of great help.
> 
> *Thanks & Regards,*
> *Saikiran V*

[dpdk-dev] How to use dpdk ovs

[Gray, Mark D]

> Subject: [dpdk-dev] How to use dpdk ovs
[...]
> Who can tell me, thanks a lot.
> 
you should post to discuss at openvswitch.org

[dpdk-dev] [PATCH v4 1/4] compat: Add infrastructure to support symbol versioning

[Gray, Mark D]

I agree in principle with this patchset. OVS links with DPDK and providing 
stability in the ABI/API (and the Policy to manage this) makes deployment 
easier for OVS when linking with shared dpdk libs. It should also be easy for 
us to track changes in the API through the deprecation notices making 
development easier! 

Good job.

Acked-by: Mark D. Gray 

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman
> Sent: Thursday, January 15, 2015 7:35 PM
> To: dev at dpdk.org
> Subject: [dpdk-dev] [PATCH v4 1/4] compat: Add infrastructure to support
> symbol versioning
> 
> Add initial pass header files to support symbol versioning.
> 
> Signed-off-by: Neil Horman 
> CC: Thomas Monjalon 
> CC: "Richardson, Bruce" 
> CC: "Gonzalez Monroy, Sergio" 
> 
> ---
> Change Notes:
> V2)
>   Moved ifeq to _INSTALL target
> 
> V3)
>   Undo V2 changes and make librte_compat use the rte.install.mk file
> instead
> 
> v4)
>   changed --version-script to accept SRCDIR in this patch at per request
>   documented versioning macros
>   cleaned up macro parameter consistency
>   converted SA macro to RTE_STR macro
>   fixed copyright
> ---
>  lib/Makefile   |   1 +
>  lib/librte_compat/Makefile |  38 +
>  lib/librte_compat/rte_compat.h | 117
> +
>  mk/rte.lib.mk  |   4 ++
>  4 files changed, 160 insertions(+)
>  create mode 100644 lib/librte_compat/Makefile  create mode 100644
> lib/librte_compat/rte_compat.h
> 
> diff --git a/lib/Makefile b/lib/Makefile index 0ffc982..d617d81 100644
> --- a/lib/Makefile
> +++ b/lib/Makefile
> @@ -31,6 +31,7 @@
> 
>  include $(RTE_SDK)/mk/rte.vars.mk
> 
> +DIRS-y += librte_compat
>  DIRS-$(CONFIG_RTE_LIBRTE_EAL) += librte_eal
>  DIRS-$(CONFIG_RTE_LIBRTE_MALLOC) += librte_malloc
>  DIRS-$(CONFIG_RTE_LIBRTE_RING) += librte_ring diff --git
> a/lib/librte_compat/Makefile b/lib/librte_compat/Makefile new file mode
> 100644 index 000..0bab870
> --- /dev/null
> +++ b/lib/librte_compat/Makefile
> @@ -0,0 +1,38 @@
> +#   BSD LICENSE
> +#
> +#   Copyright(c) 2013 Neil Horman 
> +#   All rights reserved.
> +#
> +#   Redistribution and use in source and binary forms, with or without
> +#   modification, are permitted provided that the following conditions
> +#   are met:
> +#
> +# * Redistributions of source code must retain the above copyright
> +#   notice, this list of conditions and the following disclaimer.
> +# * Redistributions in binary form must reproduce the above copyright
> +#   notice, this list of conditions and the following disclaimer in
> +#   the documentation and/or other materials provided with the
> +#   distribution.
> +# * Neither the name of Intel Corporation nor the names of its
> +#   contributors may be used to endorse or promote products derived
> +#   from this software without specific prior written permission.
> +#
> +#   THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND
> CONTRIBUTORS
> +#   "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT
> NOT
> +#   LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND
> FITNESS FOR
> +#   A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE
> COPYRIGHT
> +#   OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
> INCIDENTAL,
> +#   SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
> NOT
> +#   LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS
> OF USE,
> +#   DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED
> AND ON ANY
> +#   THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR
> TORT
> +#   (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF
> THE USE
> +#   OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH
> DAMAGE.
> +
> +include $(RTE_SDK)/mk/rte.vars.mk
> +
> +
> +# install includes
> +SYMLINK-y-include := rte_compat.h
> +
> +include $(RTE_SDK)/mk/rte.install.mk
> diff --git a/lib/librte_compat/rte_compat.h
> b/lib/librte_compat/rte_compat.h new file mode 100644 index
> 000..d7cc176
> --- /dev/null
> +++ b/lib/librte_compat/rte_compat.h
> @@ -0,0 +1,117 @@
> +/*-
> + *   BSD LICENSE
> + *
> + *   Copyright(c) 2010 Neil Horman .
> + *   All rights reserved.
> + *
> + *   Redistribution and use in source and binary forms, with or without
> + *   modification, are permitted provided that the following conditions
> + *   are met:
> + *
> + * * Redistributions of source code must retain the above copyright
> + *   notice, this list of conditions and the following disclaimer.
> + * * Redistributions in binary form must reproduce the above copyright
> + *   notice, this list of conditions and the following disclaimer in
> + *   the documentation and/or other materials provided with the
> + *   distribution.
> + * * Neither the name of Intel Corporation nor the names of its
> + *   contributors may be used to 

[dpdk-dev] vfio bind failing

[Gray, Mark D]

Hi,

I am trying to enable vfio with a DPDK app. However, I cannot bind a NIC to the 
vfio-pci driver

* I have enabled it in the BIOS and in the kernel.
[root at hostname tools]# cat /proc/cmdline
BOOT_IMAGE=/boot/vmlinuz-3.16.7-200.fc20.x86_64  rhgb quiet 
default_hugepagesz=1G hugepagesz=1G hugepages=16 iommu=on

[root at hostname tools]# dmesg | grep -e DMAR -e IOMMU
[0.00] ACPI: DMAR 0xBDFA9618 000120 (v01 INTEL  S2600CP  
06222004 INTL 20090903)
[0.118209] dmar: IOMMU 0: reg_base_addr fbffe000 ver 1:0 cap d2078c106f0466 
ecap f020de
[0.118215] dmar: IOMMU 1: reg_base_addr ebffc000 ver 1:0 cap d2078c106f0466 
ecap f020de
[0.118341] IOAPIC id 2 under DRHD base  0xfbffe000 IOMMU 0
[0.118342] IOAPIC id 0 under DRHD base  0xebffc000 IOMMU 1
[0.118343] IOAPIC id 1 under DRHD base  0xebffc000 IOMMU 1

* I have inserted the kernel module

[root at hostname tools]# modprobe vfio-pci
[root at hostname tools]# lsmod | grep vfio
vfio_iommu_type1   17632  0
vfio_pci   36425  0
vfio   20426  2 vfio_iommu_type1,vfio_pci

* I ran ulimit -l to change to memlock limit to 64MB. I have run 'setup.sh' to 
Setup VFIO permissions

Option: 10

chmod /dev/vfio
OK
chmod /dev/vfio/*
OK

Current user memlock limit: 64 MB

This is the maximum amount of memory you will be
able to use with DPDK and VFIO if run as current user.
To change this, please adjust limits.conf memlock limit for current user.

* I have run 'setup.sh' to try and bind some Niantics to the module but it is 
failing with the following
message (and the following dmesg error)

Enter PCI address of device to bind to VFIO driver: :05:00.0
Error: bind failed for :05:00.0 - Cannot bind to driver vfio-pci
Error: unbind failed for :05:00.0 - Cannot open /sys/bus/pci/drivers//unbind

Press enter to continue ...^C
[root at hostname tools]# dmesg | tail

[18020.508045] ixgbe :05:00.0: removed PHC on p3p1
[18020.603477] ixgbe :05:00.0: complete
[18020.603731] vfio-pci: probe of :05:00.0 failed with error -22
[18020.603839] vfio-pci: probe of :05:00.0 failed with error -22

Any ideas?

Thanks,

Mark

[dpdk-dev] IVSHMEM documentation

[Gray, Mark D]

> 
> Hi,
> 
> I want to run dpdk l2fwd-ivshmem application. I could not find any
> documentation on it. While digging the forum I found that I need to have
> memnic-pmd installed(http://dpdk.org/doc/memnic-pmd) . But I am not
> sure where do I download memnic-pmd. I am using dpdk-1.7.1
> 
> Can anyone please tell me what are steps for running ivshmem application.
> 
Hi Rahul, I don't think memnic works with this ivshmem. To see examples
of how to use ivshmem, you could look at DPDK vSwitch 

https://github.com/01org/dpdk-ovs

Is there no documentation for l2fwd-ivshmem in the DPDK documentation?

> Rahul Shah
> Software Engineer - CID/CSIG/DCG
> Intel Corporation
> Email: rahul.r.shah at intel.com

[dpdk-dev] Safe practices for adding to rte_table_hash_lru

[Gray, Mark D]

> 
> Hey guys
>Is it safe to add an entry to the rte_table_hash while the pipeline is 
> being
> run - for instance if I were to try and add an entry on a port reader action
> when the packet enters the pipeline?

If the table is part of the pipeline, it is not safe. i.e. the table is not 
threadsafe.

You can use a control message ring to add entries to the table in an 
asynchronous manner. Your main dpdk loop would then periodically handle
these messages. There are examples of this in Intel DPDK vSwitch 
(https://github.com/01org/dpdk-ovs)
and also, I think, in the DPDK ip_pipeline sample application.

> 
> Thanks
> Avik

[dpdk-dev] reg:porting intel dpdk kit as LINC switch

[Gray, Mark D]

> 
> Hi All,
> I wanted to port my intel DPDK kit as LINC switch, could you help me how can
> i do it.

* There are two open source software switches using DPDK. You could use them
as a reference.

https://github.com/openvswitch/ovs (look at lib/netdev-dpdk.c)
https://github.com/01org/dpdk-ovs

* Packet Framework in dpdk may also be a good starting point.

* At the very least you will probably want to modify the Linc dataplane to use
dpdk PMDs

One thing I am not sure about is how you would call DPDK functions from Linc as 
it is written in Erlang. However, I am not familiar with Erlang.

Regards,
> 
> --
> Regards,
> ANAND

[dpdk-dev] Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)

[Gray, Mark D]

> 
> Hi,
> 
> On 09/03/14 13:13, Xie, Huawei wrote:
> > Looping in the dpdk-ovs list.
> >
> > * Does the new vhost API allow a user to know if all the relevant
> > offloads have been turned on/off for that interface? It seems that
> > this is possible through the virtio_net structure but it would be good
> > to get some feedback from the relevant person working on DPDK
> > (Huawei?).
> >
> > * If this is the case, then it is probably in the realm of the vswitch
> > do the actual checksum (for VM-VM) or correctly configure the NIC when
> > sending out through the physical interface.
> >
> > Comments?
> >
> > Mark:
> > So far not supported. This is important as well in VxLan case. For the
> > packet flow Guest A->  virtio -> ..->OVDK->.. -> Guest B.
> > 1) If guest A and B are on different host machines, say A and B
> > respectively,  and if the nic on A supports vxlan checksum offload,
> > then both guest and host needn't generate checksum, the nic will generate
> checksum for both inner and outer packet.
> > 2) In VM2VM case, as it is trusted communication channel, could we
> > negotiate with the guest tcp stack not to verify checksum for received
> packet?
> The problem is that any TCP packet send by a vanilla Linux guest through
> vhost is incorrect (VM to anything, including other colocalied VMs). In other
> words, the VM cannot use TCP. QEMU options and ethtool -K csum off tso
> off ("TCP stack negociation") have no effect, maybe because the vhost
> backend is misbehaving.

This sounds like a DPDK issue. However, we did "clone and own" the vhost
code as it is a sample app. It may have changed since we integrated it.

We plan to use the vhost library interface when it is available on dpdk.org. So
that would pick up the latest version of the vhost code.

> 
> Franck

[dpdk-dev] Wrong TCP checksum of packets sent by Linux guest (virtIO/vhost)

[Gray, Mark D]

> 
> Hi Franck,
> 
> 2014-09-02 13:20, Franck BAUDIN:
> > I am using dpdk-ovs 1.1.0 (latest release) as follow :
> >
> > linux-guest (no DPKD) <---  virtIO ---> OVDK 1.1.0 (with latest DPDK [*]) < 
> > --
> - Niantic --- > linux-native
> >
> [...]
> > UDP/ICMP connectivity is fine, but TCP checksum of packet sent by the
> guest are incorrect, as showed with tcpdump on linux-native.
> 
> Thanks for reporting.
> Could you try virtio without vhost?
> 
> 

Looping in the dpdk-ovs list. 

* Does the new vhost API allow a user to know if all the relevant offloads have 
been
turned on/off for that interface? It seems that this is possible through the 
virtio_net
structure but it would be good to get some feedback from the relevant person
working on DPDK (Huawei?).

* If this is the case, then it is probably in the realm of the vswitch do the 
actual 
checksum (for VM-VM) or correctly configure the NIC when sending out through
the physical interface.

Comments?

Mark

[dpdk-dev] [PATCH] eal/linuxapp: Add parameter to specify master lcore id

[Gray, Mark D]

> Hi all,
> 
> does anyone have interest in this functionality?

Yes, I think this is useful for DPDK vSwitch.
> 
> I think this is important and useful.
> Since we should care about core assignment to get high performance and the
> master lcore thread is special in DPDK, we will want to assign the master to
> the target core.
> For example, with hyperthreading I'd like to make a pair of packet processing
> threads into one physical core and separate the master thread which does
> some management.
> 
> thanks,
> Hiroshi
> 

[dpdk-dev] Multiprocess /dev/uio mmap Bug/possible fix

[Gray, Mark D]

> All,
> 
> If this was fixed in 1.7 and I missed it I apologize (but it looks from 
> source to
> still be broken). I am using DPDK 1.6.0r2 (will be upgrading to
> 1.7.0 soon) on RHEL 6.4. I've converted the functions below to 1.7.0
> names/locations since it looks to still be an issue there.
> 
> tl;dr -- to get examples/symmetric_mp to work at all I had to manually force
> the address used to map PCI UIO devices in the primary process which I
> achieved by:
> In eal_memory.c :: get_virtual_area(...) -- Use directly then increment
> internal_config.base_virtaddr instead of incrementing/adding
> baseaddr_offset In eal_pci_uio.c in 1.7.0 :: pci_uio_map_resource -- In the
> primary process, use and increment internal_config.base_virtaddr as the first
> argument when calling pci_uio_map_resource (instead of NULL).
> 
> 
> Background/reason:
> Recently I was trying to get multiprocessing with each process reading from
> one queue working and ran into issues with the call to mmap the memory for
> /dev/uioX. Even the example 'symmetric_mp' application was failing. The
> issue is that there are two memory regions which are mmap'd for each
> /dev/uioX device and the second one, in the secondary process, is being
> mapped to an address other than the requested address. This causes the
> check for pci_map_resource(...) != uio_res->maps[i].addr to fail. I tracked
> the reason for not using the requested address to the mmap call to create
> the stack for a thread created during one of (I forget which - I think eal)
> eal/pci/pmd_init calls which is used to handle interrupts.
> 
> With DPDK compiled as a single shared library on our system the memory
> address handed for the second memory reach for each /dev/uioX in the
> primary process is right in the middle of the stack for that thread.
> 
> My possible fix: use --base-virtaddr to populate the requested address for
> mmaping the memory for /dev/uioX I am not sure if this is safe but if it is I
> can submit a patch. My approach was in eal_memory.c to, rather than keep
> an 'offset' variable locally just increment the global base_virtaddr value 
> each
> time through that loop and then use the resulting final value when doing the
> /dev/uioX mappings.
> 
> Of course in 1.6.0r2 I also ran into the errno bug with parsing 
> --base-virtaddr
> and strtoull but it looks like that was fixed in 1.7.0.

We also see this. There is a patch in a patchset that resolves this

http://dpdk.org/ml/archives/dev/2014-June/003689.html

It hasn?t been pushed yet.

> 
> 
> Thoughts? Thanks!
> Stefan

[dpdk-dev] No "pci_unbind.py" in tools subdirectory

[Gray, Mark D]

> Thank you for the reply.
> 
> By the way, I got an error message while compiling Open vSwitch. Is it
> essential to bind a port to igb_uio module in order to compile Open vSwitch
> without error? I just wonder whether the errors attached below just
> happened because I did not bind the port to the module or not...
> 
> make RTE_TARGET=x86_64-ivshmem-linuxapp-gcc
> ...
> /home/stack/dpdk/dpdk-1.6.0r2/x86_64-ivshmem-linuxapp-
> gcc/lib/librte_eal.a(eal.o):
> In function `rte_eal_init':
> eal.c:(.text+0xedf): undefined reference to `dlopen'
> eal.c:(.text+0xefe): undefined reference to `dlerror'
> collect2: error: ld returned 1 exit status
> make[2]: *** [utilities/ovs-controller] Error 1
> make[2]: Leaving directory `/home/stack/dpdk-ovs/openvswitch'
> make[1]: *** [all-recursive] Error 1
> make[1]: Leaving directory `/home/stack/dpdk-ovs/openvswitch'
> make: *** [all] Error 2
> 


Try this, 

https://lists.01.org/pipermail/dpdk-ovs/2014-June/001120.html

Or you could  the latest 'development' code

> Best regards
> 
> Byeong-Gi KIM
> 
> 
> 2014-07-14 17:10 GMT+09:00 De Lara Guarch, Pablo <
> pablo.de.lara.guarch at intel.com>:
> 
> > > -Original Message-
> > > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Aravind
> > > Sent: Monday, July 14, 2014 8:52 AM
> > > To: BYEONG-GI KIM
> > > Cc: dev at dpdk.org; Dpdk-ovs at lists.01.org
> > > Subject: Re: [dpdk-dev] No "pci_unbind.py" in tools subdirectory
> > >
> > > Hi,
> > >
> > > You should be using igb_uio_bind.py , have a look at the quick start
> > guide
> > > : http://dpdk.org/doc/quick-start
> > >
> >
> > Actually, now it is called dpdk_nic_bind.py :)
> >
> > >
> > >
> > > On Mon, Jul 14, 2014 at 1:09 PM, BYEONG-GI KIM
> > > 
> > > wrote:
> > >
> > > > Hello.
> > > >
> > > > I'm sorry for the repeated question.
> > > >
> > > > I finally compiled DPDK successfully (there was a problem in the
> > > > source code what I downloaded), and I'm moving to compile dpdk-ovs.
> > > >
> > > > Before compiling it, I'm trying to binding a network port to
> > > > igb_uio module. In the dpdk-getting-started-guide, "pci_unbind.py"
> > > > utility
> > script
> > > > is used to provide a view of the current state of the network
> > > > ports on
> > the
> > > > system, and to bind/unbind those ports from the different kernel
> > modules.
> > > >
> > > > The script, however, was not there in tools directory; There were
> > > > only "igb_uio_bind.py". Is it fine as it is, or do I miss something?
> > > >
> > > > I downloaded the DPDK source from
> > > > http://dpdk.org/browse/dpdk/refs/,
> > and
> > > > the file was dpdk-1.6.0r2.zip. I compiled both
> > x86-64-defaultlinuxapp-gcc
> > > > and x86-64-ivshmem-linuxapp-gcc now.
> > > >
> > > > Thanks in advance.
> > > >
> > > > Best regards
> > > >
> > > > Byeong-Gi KIM
> > > >
> >
> >

[dpdk-dev] Make DPDK tailqs fully local

[Gray, Mark D]

Hi,

What are the plans to resolve this issue? Will this patch get upstreamed?

http://www.dpdk.org/ml/archives/dev/2014-June/003591.html

Thanks,

Mark

[dpdk-dev] Random mbuf corruption

[Gray, Mark D]

> 
> Paul,
> 
> Thanks for the advice; we ran memtest as well as the Dell complete system
> diagnostic and neither found an issue. The plot thickens, though!
> 
> Our admins messed up our kickstart labels and what I *thought* was CentOS
> 6.4 was actually RHEL 6.4 and the problem seems to be following the CentOS
> 6.4 installations -- the current configuration of success/failure is:
>   1 server - Westmere - RHEL 6.4 -- works
>   1 server - Sandy Bridge - RHEL 6.4 -- works
>   2 servers - Sandy Bridge - CentOS 6.4 -- fails
> 
> Given that the hardware seems otherwise stable/checks out I'm trying to
> figure out how to determine if this is:
>   a) our software has a bug
>   b) a kernel/hugetlbfs bug
>   c) a  DPDK 1.6.0r2 bug
> 
> I have seen similar issues where calling rte_eal_init too late in a process 
> also
> causes similar issues (things like calling 'free' on memory that was allocated
> with 'malloc' before 'rte_eal_init' is called fails/results in segfault in 
> libc)
> which seems odd to me but in this case we are calling rte_eal_init as the 
> first
> thing we do in main().

I have seen the following issues causing mbuf corruption of this type

1. Calling an rte_pktmbuf_free() on an mbuf and then still using a reference
to that mbuf.
2. Using rte_pktmbuf_free() and rte_pktmbuf_alloc() in a pthread (i.e. not
a "dpdk" thread). This corrupted the per-lcore mbuf cache.

Not pleasant to debug, especially if you are sharing the mempool between 
primary and secondary processes. I have no tips for debug other than careful
code review everywhere an mbuf is freed or allocated. 

Mark

[dpdk-dev] Can anyone help me to run the l2fwd-ivshmem example?

[Gray, Mark D]

> 
> Hi,
> 
> 2014-06-18 15:56, GongJinrong:
> >I want find a zero copy framework from host to vm without any
> > physical NIC device,
> 
> I think memnic is what you want:
>   http://dpdk.org/doc/memnic-pmd
> 
> > it seems l2fwd-ivshmem can be used, but I have problems to run this
> > example:
> >1. there is no document about this example, even a simple readme :-(
> >2. does this example need ovdk?
> 
> No
> 
> >3. can I use standard qemu to run this example? Does the standard
> > qemu support ivshmem?
> 
> You should be able to use standard Qemu.

Standard QEMU will work for Memnic but not when using DPDK ivshmem. It uses
the standard QEMU ivshem and doesn't use DPDK in the way you would like it to

You should look at  the DPDK vSwitch code to see how DPDK ivshmem is used. 
Basically, in the host you need to identify what objects that you want to share 
with the 
virtual machine

e.g. rings, memzones

>From this, you can generate a command line to pass to QEMU (with a modified 
>ivshmem.c file
- we haven't tried to upstream this yet)

Then when you start a DPDK application in the guest, each of the objects that
you shared from the host are also available in the guest.

I presume the l2fwd-ivshmem does the same

> 
> --
> Thomas
--
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole 
use of the intended recipient(s). Any review or distribution by others is 
strictly prohibited. If you are not the intended recipient, please contact the 
sender and delete all copies.

[dpdk-dev] About ivshmem

[Gray, Mark D]

> 
> Thanks, I want use DPDK, not OVDK, if I run l2fwd-ivshmem, do I use a
> patched qemu or just a normal qemu?
> 

Patched.
--
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole 
use of the intended recipient(s). Any review or distribution by others is 
strictly prohibited. If you are not the intended recipient, please contact the 
sender and delete all copies.

[dpdk-dev] About ivshmem

[Gray, Mark D]

> Hi,
> 
> 2014-06-11 16:53, GongJinrong:
> > I found that the ivshmem are supported in DPDK 1.6 but has no any
> > document about it, how do I use ivshmem? I remember that we must use a
> > patched qemu binary for enabling ivshmem support in OVDK, does DPDK
> > 1.6 also need do this? Also, where can I find some ivshmem docs?
> 
> If you're looking for information about OVDK, you should ask on the
> appropriate mailing list.

ovdk mailing List: https://lists.01.org/pipermail/dpdk-ovs/

ivshmem instructions: 
https://github.com/01org/dpdk-ovs/blob/development/docs/04_Sample_Configurations/01_IVSHM.md

> 
> If you are looking for a PMD using ivshmem, you should look at memnic:
>   http://dpdk.org/doc/memnic-pmd
> 
> --
> Thomas
--
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole 
use of the intended recipient(s). Any review or distribution by others is 
strictly prohibited. If you are not the intended recipient, please contact the 
sender and delete all copies.