[dpdk-dev] [PATCH] llib/ibrte_net: workaround to avoid macro conflict

2014-10-08 Thread Matthew Hall
If the DPDK wants to conflict with all those system headers it means they also 
have to provide working replacement for inet_pton, inet_ntop, and every other 
important socket function which depends upon in.h or depends upon code 
depending upon in.h. Clearly this doesn't represent a sustainable path for 
software development... I had to reimplement a number of these in my app to get 
around the header conflict and I'm sure I'm not the only one who ran into this 
issue either.

Matthew.
-- 
Sent from my mobile device.

On October 8, 2014 10:20:31 PM PDT, "Wu, Jingjing"  
wrote:
>Hi, Neil
>
>To have rte_ip.h include netinet/in.h directly is also a choice.
>
>But netinet/in.h contains a lot of extra stuff, and these may be
>useless some DPDK applications, such as classification.
>rte_ip.h provides a more simplify way for the IP protocol layer.
>
>-Original Message-
>From: Neil Horman [mailto:nhorman at tuxdriver.com] 
>Sent: Tuesday, September 30, 2014 9:10 PM
>To: Wu, Jingjing
>Cc: dev at dpdk.org
>Subject: Re: [dpdk-dev] [PATCH] llib/ibrte_net: workaround to avoid
>macro conflict
>
>On Tue, Sep 30, 2014 at 10:49:08AM +0800, Jingjing Wu wrote:
>> Macros such as IPPROTO_TCP, IPPROTO_UDP are already defined in
>.
>> If user's application includes  and rte_ip.h at the
>same 
>> time, there will be conflict error.
>> 
>> This patch uses the way "#ifndef #endif" to avoid the conflict. 
>> 
>> Signed-off-by: Jingjing Wu 
>> ---
>>  lib/librte_net/rte_ip.h | 5 +
>>  1 file changed, 5 insertions(+)
>> 
>> diff --git a/lib/librte_net/rte_ip.h b/lib/librte_net/rte_ip.h index 
>> e3f65c1..2bcb479 100644
>> --- a/lib/librte_net/rte_ip.h
>> +++ b/lib/librte_net/rte_ip.h
>> @@ -116,6 +116,8 @@ struct ipv4_hdr {
>>  
>>  #define IPV4_HDR_OFFSET_UNITS   8
>>  
>> +#ifndef _NETINET_IN_H
>> +#ifndef _NETINET_IN_H_
>>  /* IPv4 protocols */
>>  #define IPPROTO_IP 0  /**< dummy for IP */
>>  #define IPPROTO_HOPOPTS0  /**< IP6 hop-by-hop options */
>> @@ -227,6 +229,9 @@ struct ipv4_hdr {
>>  #define IPPROTO_RAW  255  /**< raw IP packet */
>>  #define IPPROTO_MAX  256  /**< maximum protocol number */
>>  
>> +#endif /*_NETINET_IN_H_*/
>> +#endif /*_NETINET_IN_H*/
>> +
>>  /*
>>   * IPv4 address types
>>   */
>> --
>> 1.8.1.4
>> 
>> 
>Why define them at all?  Why not just have rte_ip.h include
>netinet/in.h directly?  Its a standard include file in a standard
>location for both bsd and linux IIRC.
>Neil



[dpdk-dev] section mismatch warnings

2014-10-08 Thread Michael Hu (NSBU)
Thanks Neil and Sergio for your info.

The mismatch warning did not show up with normal kernel but in the kernel
with PAX grsecurity patches as PaX team enhanced writeable function
pointer detection in patches out in this link.
http://en.wikibooks.org/wiki/Grsecurity/Configuring_and_Installing_grsecuri
ty#Compilation_Differences
---
the extra section mismatches are due to my Pax changes, i explicitly
added detection for writeable function pointers which are potential
exploit targets, just to know how many of them there are. we've been
eliminating some of them already but this work will never finish.

as for what they are in general, a mismatch means an unwanted reference
from one section to another. say, accessing init code or data from
normal code/data is not good since init sections are freed up on boot,
so any reference to them must not exist from permanent sections.

---

To reproduce, you just need to patch any supported kernel with grsecurity
patches then compile dpdk with it.
https://grsecurity.net/download.php



Thanks,
Michael 






On 10/7/14 9:46 AM, "Neil Horman"  wrote:

>On Mon, Oct 06, 2014 at 06:42:04PM +, Michael Hu (NSBU) wrote:
>> Hi Everyone,
>> 
>> When we enable kernel config CONFIG_DEBUG_SECTION_MISMATCH=y and
>>compile DPDK 1.7 with 3.14.17 kernel + gcc 4.8.2, we got lots of
>>warnings like these.
>> Do we have plan to fix them? If so, please advise on which version.
>>Thanks.
>> 
>Seems erroneous.  These errors come from referencing symbols between
>incompatible sections (e.g. A static function reading a variable that is
>in an
>__initdata section).  But theres nothing like that going on in any of
>these
>symbols referenced below (and the kernel version shouldn't matter, as
>this is
>all dpdk internal).  It could be old objects in your build from a DPDK
>version
>where the symbols were in different sections.  Do a make clean; git clean
>-f -d
>Then reconfigure and rebuild.  I expect your issue will go away.
>
>Neil
>
>> 
>> 
>> == Build lib/librte_eal/linuxapp/kni
>> cut: /proc/version_signature: No such file or directory
>>   LD  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/igb_uio/built-i
>>n.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/igb_uio/igb_uio
>>.o
>>   Building modules, stage 2.
>>   MODPOST 1 modules
>> WARNING: 
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/igb_uio/igb_uio
>>.o(.data+0x20): Section mismatch in reference from the variable
>>igbuio_pci_driver to the function .text:igbuio_pci_probe()
>> 
>> WARNING: 
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/igb_uio/igb_uio
>>.o(.data+0x28): Section mismatch in reference from the variable
>>igbuio_pci_driver to the function .text:igbuio_pci_remove()
>> 
>> WARNING: 
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/igb_uio/igb_uio
>>.o(.data+0x130): Section mismatch in reference from the variable
>>dev_attr_max_vfs to the function .text:show_max_vfs()
>> 
>> WARNING: 
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/igb_uio/igb_uio
>>.o(.data+0x138): Section mismatch in reference from the variable
>>dev_attr_max_vfs to the function .text:store_max_vfs()
>> 
>>   CC  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/igb_uio/igb_uio
>>.mod.o
>>   LD [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/igb_uio/igb_uio
>>.ko
>> INSTALL-MODULE igb_uio.ko
>> 
>>   LD  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/built-in.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/ixgbe_main.
>>o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/ixgbe_api.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/ixgbe_commo
>>n.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/ixgbe_ethto
>>ol.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/ixgbe_82599
>>.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/ixgbe_82598
>>.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/ixgbe_x540.
>>o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/ixgbe_phy.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/kcompat.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/e1000_82575
>>.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/e1000_i210.
>>o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/e1000_api.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/e1000_mac.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/e1000_manag
>>e.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/e1000_mbx.o
>>   CC [M]  
>>/git/Pktgen-DPDK/dpdk/build/build/lib/librte_eal/linuxapp/kni/e

[dpdk-dev] vmxnet3 pmd dev restart

2014-10-08 Thread Patel, Rashmin N
Yes I had a local copy working with couple of lines fix. But someone else, I 
think Stephen added a fix patch for the same, and I assume if it's been merged, 
should be working, so did not follow up later. 

I don't have a VMware setup handy at moment but I think Waterman would have 
more information about testing that patch if he has found any issue with it.

Thanks,
Rashmin

-Original Message-
From: Navakanth M [mailto:navakanth...@gmail.com] 
Sent: Wednesday, October 08, 2014 4:14 AM
To: dev at dpdk.org; Patel, Rashmin N
Subject: Re: vmxnet3 pmd dev restart

Hi Rashmin

I have come across your reply in following post that you have worked on this 
problem and would submit the patch for it.
Can you please share information on the changes you worked on or patch log if 
you had submitted any for it?
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/4683

Thanks
Navakanth

On Tue, Sep 30, 2014 at 1:44 PM, Navakanth M  wrote:
> Hi
>
> I am using DPDKv1.7.0 running on Vmware Esxi 5.1 and am trying to 
> reset the port which uses pmd_vmnet3 library functions from below 
> function calls.
> rte_eth_dev_stop
> rte_eth_dev_start
>
> Doing this, i face panic while rte_free(ring->buf_info) in 
> Vmxnet3_cmd_ring_release().
> I have gone through following thread but the patch mentioned didn't 
> help rather it crashed in start function while accessing buf_info in 
> vmxnet3_post_rx_bufs. I see this buf_info is allocated in queue setup 
> functions which are called at initialization.
> http://thread.gmane.org/gmane.comp.networking.dpdk.devel/4683
>
> I tried not freeing it and then rx packets are not received due to 
> mismatch in while (rcd->gen == rxq->comp_ring.gen) in 
> vmxnet3_recv_pkts()
>
> To reset the device port, is this the right way what i am doing?
> Or do I have to call vmxnet3_dev_tx_queue_setup()
> vmxnet3_dev_rx_queue_setup() once stop is called? I have checked 
> recent patches and threads but did not get much information on this.
>
> Thanks
> Navakanth


[dpdk-dev] [PATCH] Pass CC option when building kernel modules

2014-10-08 Thread Thomas Monjalon
2014-10-06 16:57, Sergio Gonzalez Monroy:
> At least on kernels 3.15 or newer, DPDK build is broken for CLANG target.
> The issue is that the kernel build system sets the flags before including
> DPDK makefile and therefore assumes the incorrect compiler.
> 
> Signed-off-by: Sergio Gonzalez Monroy 

> --- a/mk/rte.module.mk
> +++ b/mk/rte.module.mk
> @@ -78,7 +78,7 @@ build: _postbuild
>  $(MODULE).ko: $(SRCS_LINKS)
>   @if [ ! -f $(notdir Makefile) ]; then ln -nfs $(SRCDIR)/Makefile . ; fi
>   @$(MAKE) -C $(RTE_KERNELDIR) M=$(CURDIR) O=$(RTE_KERNELDIR) \
> - CROSS_COMPILE=$(CROSS)
> + CC=$(KERNELCC) CROSS_COMPILE=$(CROSS)

> --- a/mk/target/generic/rte.vars.mk
> +++ b/mk/target/generic/rte.vars.mk
> @@ -149,4 +149,6 @@ endif
>  export CFLAGS
>  export LDFLAGS
>  
> +else # ! ifeq ($(KERNELRELEASE),)
> +CC = $(KERNELCC)
>  endif

Why is it needed here? Setting CC in rte.module.mk is not sufficient?

Thanks
-- 
Thomas


[dpdk-dev] [PATCH] Pass verbose flag to kernel module

2014-10-08 Thread Thomas Monjalon
Hi Sergio,

2014-10-06 17:09, Sergio Gonzalez Monroy:
> --- a/mk/rte.module.mk
> +++ b/mk/rte.module.mk
> @@ -78,7 +78,7 @@ build: _postbuild
>  $(MODULE).ko: $(SRCS_LINKS)
>   @if [ ! -f $(notdir Makefile) ]; then ln -nfs $(SRCDIR)/Makefile . ; fi
>   @$(MAKE) -C $(RTE_KERNELDIR) M=$(CURDIR) O=$(RTE_KERNELDIR) \
> - CROSS_COMPILE=$(CROSS)
> + V=$(if $(V),1,0) CROSS_COMPILE=$(CROSS)

Please could you explain why it is needed?
The variable V should be inherited by the recursive make.
It's working without your patch in my test.

-- 
Thomas


[dpdk-dev] [PATCH RFC] librte_reorder: new reorder library

2014-10-08 Thread Neil Horman
On Wed, Oct 08, 2014 at 03:41:11PM -0700, Matthew Hall wrote:
> On Tue, Oct 07, 2014 at 09:33:06AM +, Pattan, Reshma wrote:
> > To be used along with distributor library. Next version of distributor are 
> > planned to distribute incoming packets to all worker cores irrespective of 
> > the flow type. In this case to ensure in order delivery of the packets at 
> > output side reorder library will used by the tx end.
> 
> Hello,
> 
> I am trying to understand, why is usage of the RSS options on the RX Queues 
> insufficient to keep the packets ordered per-flow on the TX Queues?
> 
 I think because there is a possibility that multiple workers may be used for a
single tx queue.

Neil

> Thanks,
> Matthew.
> 


[dpdk-dev] [PATCH 0/4] Add DSO symbol versioning to support backwards compatibility

2014-10-08 Thread Butler, Siobhan A

From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Thomas Monjalon
Sent: Wednesday, October 8, 2014 4:57 PM
To: Neil Horman
Cc: dev at dpdk.org
Subject: Re: [dpdk-dev] [PATCH 0/4] Add DSO symbol versioning to support 
backwards compatibility

Hi Neil,

2014-10-07 17:01, Neil Horman:
> On Wed, Oct 01, 2014 at 02:59:40PM -0400, Neil Horman wrote:
> > On Fri, Sep 26, 2014 at 10:45:49AM -0400, Neil Horman wrote:
> > > On Fri, Sep 26, 2014 at 12:41:33PM +0200, Thomas Monjalon wrote:
> > > > Hi Neil,
> > > > 
> > > > 2014-09-24 14:19, Neil Horman:
> > > > > Ping Thomas. I know you're busy, but I would like this to not 
> > > > > fall off anyones radar.  You alluded to concerns regarding 
> > > > > what, for lack of a better term, ABI/API lockin.  I had asked 
> > > > > you to enuumerate/elaborate on specifics, but never heard 
> > > > > back.  Are there further specifics you wish to discuss, or are you 
> > > > > satisfied with the above answers?
> > > > 
> > > > Sorry for not being very reactive on this thread.
> > > > All this discussion is very interesting but it's really not the 
> > > > proper time to apply it. As you said, it requires an extra 
> > > > effort. I'm not saying it will never be integrated. I'm just 
> > > > saying that we cannot change everything at the same time.
> > > > 
> > > > Let me sum up the situation. This community project has been 
> > > > very active for few months now. First, we learnt how to make 
> > > > some releases together and we are improving the process to be 
> > > > able to deliver a new major release every 4 months while having some 
> > > > good quality process.
> > > > But these releases are still not complete because documentation 
> > > > is not integrated yet. Then developers should have a role in 
> > > > documentation updates.
> > > > We also need to integrate and learn how to use more tools to be 
> > > > more efficient and improve quality.
> > > > 
> > > > So the question is "when should we care about API compatibility"?
> > > > And the answer is: ASAP, but not now. I feel next year is a better 
> > > > target.
> > > > Because the most important priority is to move together at a 
> > > > pace which allow most of us to stay in the race.
> > > 
> > > I'm sorry Thomas, I don't accept this.  I asked you for details as 
> > > to your concerns regarding this patch series, and you've provided more 
> > > vague comments.
> > > I need details to address
> > > 
> > > You say it requires extra effort, you're right it does.  Any 
> > > feature that you integreate requires some additional effort.  How 
> > > is this patch any different from adding the acl library or any 
> > > other new API?  Everything requires maintenence, thats how 
> > > software works.  What specfically about this patch series makes the 
> > > effort insurmountable to you?
> > > 
> > > You say you're improving your process.  Great, this patch aids in 
> > > that process by ensuring backwards compatibility for a period of 
> > > time.  Given that the API and ABI can still evolve within this 
> > > framework, as I've described, how is this patch series not a significant 
> > > step forward toward your goal of quality process.
> > > 
> > > You say documentation isn't integrated.  So, what does getting 
> > > documentation integrated have to do with this patch set, or any 
> > > other?  I don't see you holding any other patches based on 
> > > documentation.  Again, nothing in this series prevents evolution 
> > > of the API or ABI.  If you're hope is to wait until everything is 
> > > perfect, then apply some control to the public facing API, and get it all 
> > > documented, none of thosse things will ever happen, I promise you.
> > > 
> > > You say you also need to learn to use more tools to be more 
> > > efficient and improve quality.  Great!  Thats exactly what this 
> > > is. If we mandate even a short term commitment to ABI stability (1 
> > > single relese worth of time), we will quickly identify what API's 
> > > change quickly and where we need to be cautious with our API 
> > > design.  If you just assume that developers will get better of their own 
> > > volition, it will never happen.
> > > 
> > > You say this should go in next year, but not now.  When exactly?  
> > > What event do you forsee occuring in the next 12-18 months that 
> > > will change everything such that we can start supporing an ABI for 
> > > more than just a few weeks at the head of the tree?
> > > 
> > > To this end, I just did a quick search through the git history for 
> > > dpdk to look at the histories of all the header files that are 
> > > exposed via the makefile SYMLINK command (given that that provides 
> > > a list of header files that applications can include, and embodies 
> > > all the function symbols and data types applications have access to.
> > > 
> > > There are 179 total commits in that list Of those, a bit of spot 
> > > checking suggests that about 10-15% of them actually change ABI, 
> > > and m

[dpdk-dev] [PATCH v2] librte_pmd_packet: add PMD for AF_PACKET-based virtual devices

2014-10-08 Thread Thomas Monjalon
2014-09-29 11:05, Bruce Richardson:
> On Fri, Sep 26, 2014 at 10:08:55AM -0400, Neil Horman wrote:
> > On Fri, Sep 26, 2014 at 11:28:05AM +0200, Thomas Monjalon wrote:
> > > 2014-09-16 16:16, Neil Horman:
> > > > On Fri, Sep 12, 2014 at 02:05:23PM -0400, John W. Linville wrote:
> > > > > Ping?  Are there objections to this patch from mid-July?
> > > > 
> > > > Thomas, Where are you on this?  It seems like if you don't have any 
> > > > objections
> > > > to this patch, it should go in, in ilght of the lack of further 
> > > > commentary.
> > > 
> > > 1) It doesn't appear as a top priority.
> > Thats your responsibility.  Patches can't languish and rot on a list forever
> > just because others aren't willing to test it.  If theres further testing 
> > that
> > you feel it needs, ask. But from my read, its been tested for functionality 
> > and
> > performance (though high performance is never expected from a AF_PACKET 
> > PMD).
> > Given that any one PMD will not affect the performance of another in 
> > isolation,
> > I'm not sure what more you're waiting for here.

Yes, integration of new PMD must be accelerated.

> > > 2) It's competing with pcap PMD and bifurcated PMD to come
> > >(http://dpdk.org/ml/archives/dev/2014-September/005379.html)
> > Regarding the pcap PMD, so?  Its an alternate implementation that provides
> > different features with different limitations.  The fact that they are 
> > simmilar
> > is irrelevant.  If simmilarity was the test, then we wouldn't bother with 
> > the
> > bifurcated driver either, because the pcap pmd already exists.
> > 
> > Regarding the bifurcated driver, you can't hold existing patches on the 
> > promise
> > of another pmd thats comming at an indeterminate time in the future.  
> > Theres no
> > reason not to take this now and deprecate it in the future if there is
> > sufficient overlap with the bifurcated driver, though to my point above, 
> > they
> > still address different needs with different limitations, so I don't see 
> > doing
> > so as necessecary.

Yes, we'll discuss it when bifurcated driver will be released.

> > > 3) There is no test associated with this PMD.
> > That would have been a great comment to make a few months back, though whats
> > wrong with testpmd here?  That seems to be the same test that every other 
> > pmd
> > uses. What exactly are you looking for?

I was thinking of testing behaviour with different kernel configurations and
unit tests for --vdev options. But it's not a major blocker.

> > > If one of this item becomes wrong, it should go in.
> > 
> > > Currently, 2 projects are being initiated for validation (dcts) and
> > > documentation. Keeping new things outside of the DPDK core makes it
> > > clear that they have not to be supported by dcts and doc yet.
> > > So, it is better to have an external PMD, like memnic, acting as a
> > > staging area.
> > > 
> > So, this brings up an excellent point - Validation and support.  Commonly 
> > open
> > source projects don't provide support at the upstream HEAD. Those items are
> > applied and inforced by distributors.  Theres no need to ensure that the
> > upstream head is always the most performance and stable point of the tree.  
> > Its
> > that need that keeps the development pace slow, and creates frustrations 
> > like
> > this one, where a patch sits unaddressed for long periods of time.  
> > Commonly the
> > workflow for most open source projects is for there to be a window of time 
> > where
> > visual review and basic functional testing are sufficient for acceptance 
> > into
> > the head of the tree.  After the development window closes there is a
> > stabilization period where testing/validation is done to ensure that no
> > regressions have been encountered, optionally with a -next branch 
> > temporarily
> > being created to accept patches for upcomming future releases.  If 
> > regressions
> > are found, its a simple matter in git to bisect back to the offending patch,
> > allow the contributing developer an opportunity to fix the issue, or to 
> > drop the
> > patch.  Using a workflow like this we can have a reasonable balance of needs
> > (good patch turn around time, as well as reasonable testing).  We've 
> > discussed
> > this when I posted the PMD_REGISTER_DRIVER patch months ago, and I thought 
> > you
> > were going to move in the direction of this workflow.  What happened?

Yes, we are moving to a "merge window" workflow.

> > > During this time, keeping this PMD separately will allow you to update it
> > > with a maintainer account in dpdk.org. I just need your SSH public key.
> > > 
> > We've discussed this too, keeping PMDs maintained separately is a very bad 
> > idea.
> > Doing so means developers have to constantly be aware of changes to the core
> > tree and try to keep up individually.  Integrating them all means that API
> > changes can be easily propogated to all PMD's when needed without making 
> > work
> > for many people.  Its exactly th

[dpdk-dev] [PATCH 0/4] Add DSO symbol versioning to support backwards compatibility

2014-10-08 Thread Thomas Monjalon
Hi Neil,

2014-10-07 17:01, Neil Horman:
> On Wed, Oct 01, 2014 at 02:59:40PM -0400, Neil Horman wrote:
> > On Fri, Sep 26, 2014 at 10:45:49AM -0400, Neil Horman wrote:
> > > On Fri, Sep 26, 2014 at 12:41:33PM +0200, Thomas Monjalon wrote:
> > > > Hi Neil,
> > > > 
> > > > 2014-09-24 14:19, Neil Horman:
> > > > > Ping Thomas. I know you're busy, but I would like this to not fall 
> > > > > off anyones
> > > > > radar.  You alluded to concerns regarding what, for lack of a better 
> > > > > term,
> > > > > ABI/API lockin.  I had asked you to enuumerate/elaborate on 
> > > > > specifics, but never
> > > > > heard back.  Are there further specifics you wish to discuss, or are 
> > > > > you
> > > > > satisfied with the above answers?
> > > > 
> > > > Sorry for not being very reactive on this thread.
> > > > All this discussion is very interesting but it's really not the proper
> > > > time to apply it. As you said, it requires an extra effort. I'm not 
> > > > saying
> > > > it will never be integrated. I'm just saying that we cannot change
> > > > everything at the same time.
> > > > 
> > > > Let me sum up the situation. This community project has been very active
> > > > for few months now. First, we learnt how to make some releases together
> > > > and we are improving the process to be able to deliver a new major 
> > > > release
> > > > every 4 months while having some good quality process.
> > > > But these releases are still not complete because documentation is not
> > > > integrated yet. Then developers should have a role in documentation 
> > > > updates.
> > > > We also need to integrate and learn how to use more tools to be more
> > > > efficient and improve quality.
> > > > 
> > > > So the question is "when should we care about API compatibility"?
> > > > And the answer is: ASAP, but not now. I feel next year is a better 
> > > > target.
> > > > Because the most important priority is to move together at a pace which
> > > > allow most of us to stay in the race.
> > > 
> > > I'm sorry Thomas, I don't accept this.  I asked you for details as to your
> > > concerns regarding this patch series, and you've provided more vague 
> > > comments.
> > > I need details to address
> > > 
> > > You say it requires extra effort, you're right it does.  Any feature that 
> > > you
> > > integreate requires some additional effort.  How is this patch any 
> > > different
> > > from adding the acl library or any other new API?  Everything requires
> > > maintenence, thats how software works.  What specfically about this patch 
> > > series
> > > makes the effort insurmountable to you?
> > > 
> > > You say you're improving your process.  Great, this patch aids in that 
> > > process
> > > by ensuring backwards compatibility for a period of time.  Given that the 
> > > API
> > > and ABI can still evolve within this framework, as I've described, how is 
> > > this
> > > patch series not a significant step forward toward your goal of quality 
> > > process.
> > > 
> > > You say documentation isn't integrated.  So, what does getting 
> > > documentation
> > > integrated have to do with this patch set, or any other?  I don't see you
> > > holding any other patches based on documentation.  Again, nothing in this 
> > > series
> > > prevents evolution of the API or ABI.  If you're hope is to wait until
> > > everything is perfect, then apply some control to the public facing API, 
> > > and get
> > > it all documented, none of thosse things will ever happen, I promise you.
> > > 
> > > You say you also need to learn to use more tools to be more efficient and
> > > improve quality.  Great!  Thats exactly what this is. If we mandate even 
> > > a short
> > > term commitment to ABI stability (1 single relese worth of time), we will
> > > quickly identify what API's change quickly and where we need to be 
> > > cautious with
> > > our API design.  If you just assume that developers will get better of 
> > > their own
> > > volition, it will never happen.
> > > 
> > > You say this should go in next year, but not now.  When exactly?  What 
> > > event do
> > > you forsee occuring in the next 12-18 months that will change everything 
> > > such
> > > that we can start supporing an ABI for more than just a few weeks at the 
> > > head of
> > > the tree?  
> > > 
> > > To this end, I just did a quick search through the git history for dpdk 
> > > to look
> > > at the histories of all the header files that are exposed via the makefile
> > > SYMLINK command (given that that provides a list of header files that
> > > applications can include, and embodies all the function symbols and data 
> > > types
> > > applications have access to.
> > > 
> > > There are 179 total commits in that list
> > > Of those, a bit of spot checking suggests that about 10-15% of them 
> > > actually
> > > change ABI, and many of those came from Bruce's rework of the mbuf 
> > > structure.
> > > That about 17-20 instances over the last 2 years where an ABI u

[dpdk-dev] [PATCH] mk: link combined shared library with compiler to enable elf ctors

2014-10-08 Thread Thomas Monjalon
Hi,

Sergio's patchset provides a fixed version of this patch:
http://dpdk.org/dev/patchwork/patch/715/

2014-10-06 22:56, michal:
> From: Michal Bella 
> 
> Signed-off-by: michal 
> ---
>  mk/rte.sharelib.mk | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mk/rte.sharelib.mk b/mk/rte.sharelib.mk
> index c0a811a..a315d98 100644
> --- a/mk/rte.sharelib.mk
> +++ b/mk/rte.sharelib.mk
> @@ -45,7 +45,7 @@ sharelib: $(LIB_ONE) FORCE
>  
>  OBJS = $(wildcard $(RTE_OUTPUT)/build/lib/*.o)
>  
> -O_TO_S = $(LD) $(CPU_LDFLAGS) -shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
> +O_TO_S = $(CC) $(CPU_LDFLAGS) -shared $(OBJS) -o $(RTE_OUTPUT)/lib/$(LIB_ONE)
>  O_TO_S_STR = $(subst ','\'',$(O_TO_S)) #'# fix syntax highlight
>  O_TO_S_DISP = $(if $(V),"$(O_TO_S_STR)","  LD $(@)")
>  O_TO_S_CMD = "cmd_$@ = $(O_TO_S_STR)"



[dpdk-dev] [PATCH v2 4/4] Link apps/DSOs against EXECENV_LDLIBS with --as-needed

2014-10-08 Thread Thomas Monjalon
Please could you explain why patch is needed?

-- 
Thomas


[dpdk-dev] [PATCH v2 3/4] Update library build process

2014-10-08 Thread Thomas Monjalon
Hi Sergio,

2014-10-06 11:52, Sergio Gonzalez Monroy:
> Remove COMBINE_LIBS option and by default build:
> - CONFIG_RTE_BUILD_SHARED_LIB=y : both individual and combined libraries
> - CONFIG_RTE_BUILD_SHARED_LIB=n : single combined library
> 
> Signed-off-by: Sergio Gonzalez Monroy 

I'd appreciate this hairy patch to be splitted and commented if possible.
It's not easy to review as is.

Thanks
-- 
Thomas


[dpdk-dev] vmxnet3 pmd dev restart

2014-10-08 Thread Navakanth M
Hi Rashmin

I have come across your reply in following post that you have worked
on this problem and would submit the patch for it.
Can you please share information on the changes you worked on or patch
log if you had submitted any for it?
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/4683

Thanks
Navakanth

On Tue, Sep 30, 2014 at 1:44 PM, Navakanth M  wrote:
> Hi
>
> I am using DPDKv1.7.0 running on Vmware Esxi 5.1 and am trying to
> reset the port which uses pmd_vmnet3 library functions from below
> function calls.
> rte_eth_dev_stop
> rte_eth_dev_start
>
> Doing this, i face panic while rte_free(ring->buf_info) in
> Vmxnet3_cmd_ring_release().
> I have gone through following thread but the patch mentioned didn't
> help rather it crashed in start function while accessing buf_info in
> vmxnet3_post_rx_bufs. I see this buf_info is allocated in queue setup
> functions which are called at initialization.
> http://thread.gmane.org/gmane.comp.networking.dpdk.devel/4683
>
> I tried not freeing it and then rx packets are not received due to mismatch in
> while (rcd->gen == rxq->comp_ring.gen) in vmxnet3_recv_pkts()
>
> To reset the device port, is this the right way what i am doing?
> Or do I have to call vmxnet3_dev_tx_queue_setup()
> vmxnet3_dev_rx_queue_setup() once stop is called? I have checked
> recent patches and threads but did not get much information on this.
>
> Thanks
> Navakanth


[dpdk-dev] [PULL REQUEST] doc: generate DPDK_Linux_GSG with Sphinx.

2014-10-08 Thread Bernard Iremonger
These changes are a conversion of the DPDK_Linux_GSG from an MSWord file to 
Sphinx rst files,
and makefile changes.

The following changes since commit a5235ae1cd83360d4a7b156b8d983126c327b154:

  delete DPDK_Linux_GSG (2014-10-08 14:41:14 +0100)

are available in the git repository at:
  ssh://bernard at dpdk.org/home/bernard/dpdk-doc-next.git/ master

Bernard Iremonger (2):
  makefiles for doxygen and sphinx
  config and rst files for DPDK_Linux_GSG

 .../Compiling_and_Running_Sample_Applications.rst  |  201 +
 ...Compiling_the_Intel_DPDK_Target_from_Source.rst |  226 +++
 .../Enabling_Additional_Functionality.rst  |  167 +++
 doc/guides/DPDK_Linux_GSG/Introduction.rst |   42 +++
 .../DPDK_Linux_GSG/Quick_Start_Setup_Script.rst|  294 
 doc/guides/DPDK_Linux_GSG/System_Requirements.rst  |  240 
 doc/guides/DPDK_Linux_GSG/index.rst|   92 ++
 doc/guides/conf.py |  101 +++
 doc/guides/index.rst   |   14 +
 mk/rte.sdkdoc.mk   |   70 +
 mk/rte.sdkguides.mk|  184 
 mk/rte.sdkroot.mk  |  136 +
 12 files changed, 1767 insertions(+), 0 deletions(-)
 create mode 100755 
doc/guides/DPDK_Linux_GSG/Compiling_and_Running_Sample_Applications.rst
 create mode 100755 
doc/guides/DPDK_Linux_GSG/Compiling_the_Intel_DPDK_Target_from_Source.rst
 create mode 100755 
doc/guides/DPDK_Linux_GSG/Enabling_Additional_Functionality.rst
 create mode 100755 doc/guides/DPDK_Linux_GSG/Introduction.rst
 create mode 100755 doc/guides/DPDK_Linux_GSG/Quick_Start_Setup_Script.rst
 create mode 100755 doc/guides/DPDK_Linux_GSG/System_Requirements.rst
 create mode 100755 doc/guides/DPDK_Linux_GSG/index.rst
 create mode 100755 doc/guides/conf.py
 create mode 100755 doc/guides/index.rst
 create mode 100644 mk/rte.sdkdoc.mk
 create mode 100644 mk/rte.sdkguides.mk
 create mode 100644 mk/rte.sdkroot.mk


[dpdk-dev] [PATCH RFC] librte_reorder: new reorder library

2014-10-08 Thread Matthew Hall
On Wed, Oct 08, 2014 at 06:55:41PM -0400, Neil Horman wrote:
>  I think because there is a possibility that multiple workers may be used for 
> a
> single tx queue.
> 
> Neil

OK, so, in my application packets are RX'ed to a predictable RX queue and core 
using RSS.

Then you put them into a predictable TX queue for the same core, in the same 
order they came in from the RX queue with RSS enabled.

So you've got a consistent-hashed subset of packets as input, being converted 
to output in the same order.

Will it work, or not work? I'm just curious if my app is doing it wrong and I 
need to fix it, or how this case should be handled in general...

Matthew.


[dpdk-dev] [PATCH RFC] librte_reorder: new reorder library

2014-10-08 Thread Matthew Hall
On Tue, Oct 07, 2014 at 09:33:06AM +, Pattan, Reshma wrote:
> To be used along with distributor library. Next version of distributor are 
> planned to distribute incoming packets to all worker cores irrespective of 
> the flow type. In this case to ensure in order delivery of the packets at 
> output side reorder library will used by the tx end.

Hello,

I am trying to understand, why is usage of the RSS options on the RX Queues 
insufficient to keep the packets ordered per-flow on the TX Queues?

Thanks,
Matthew.


[dpdk-dev] [PATCH v2 3/4] Update library build process

2014-10-08 Thread Matthew Hall
On Tue, Oct 07, 2014 at 10:55:11AM +0100, Sergio Gonzalez Monroy wrote:
> I don't see how this particular patch would force people to change their code,
> though in all likelihood they will have to as a result of ABI changes in the
> following release.
>
> The only difference now would be when they link their applications against a
> single library instead of multiple separated libraries.
> 
> Thanks,
> Sergio

Hi Sergio,

Changing all of your Makefiles does sound like changing code to me.

Matthew.


[dpdk-dev] [PATCH] eal/bsd: fix core detection

2014-10-08 Thread Neil Horman
On Wed, Oct 08, 2014 at 01:23:49PM +0200, David Marchand wrote:
> Hello Neil,
> 
> On Wed, Oct 8, 2014 at 12:53 PM, Neil Horman  wrote:
> >
> > Though, that said, why are these files diverged in the first place?  At
> > least as
> > far as eal_lcore.c and eal.c are concerned the only differences appear to
> > be
> > cases of one file being updated and not the other.  It seems that, rather
> > than
> > doing a patch like this to bring bsd up to date with linux, we should just
> > de-dup the files, put them in a common location and handle any real
> > behavioral
> > differences with macros/ifdefs.  Is there a reason for separating them?
> >
> 
> I agree that there should only be one file for those things (and a lot of
> other stuff like eal_debug, eal_log etc...).
> This is not that easy: I tried once and gave up because the resulting
> patchset was huge.
> Personally, I don't have time to dig into this.
> 
No one ever does, thats how messes like this happen :)

I've got a bsd guest installed, I'll see if I can clean it up.
Neil

> 
> -- 
> David Marchand


[dpdk-dev] [PATCH RFC] librte_reorder: new reorder library

2014-10-08 Thread Neil Horman
On Wed, Oct 08, 2014 at 02:11:34PM +, Pattan, Reshma wrote:
> 
> 
> > -Original Message-
> > From: Neil Horman [mailto:nhorman at tuxdriver.com]
> > Sent: Tuesday, October 7, 2014 12:22 PM
> > To: Pattan, Reshma
> > Cc: dev at dpdk.org
> > Subject: Re: [dpdk-dev] [PATCH RFC] librte_reorder: new reorder library
> > 
> > On Tue, Oct 07, 2014 at 09:33:06AM +, Pattan, Reshma wrote:
> > > Hi All,
> > >
> > > I am planning  to implement packet reorder library. Details are as below,
> > please go through them and provide the comments.
> > >
> > > Requirement:
> > >To reorder out of ordered packets that are received from 
> > > different
> > cores.
> > >
> > > Usage:
> > > To be used along with distributor library. Next version of distributor are
> > planned to distribute incoming packets to all worker cores irrespective of 
> > the
> > flow type.
> > > In this case to ensure in order delivery of the packets at output side 
> > > reorder
> > library will used by the tx end.
> > >
> > > Assumption:
> > > All input packets will be marked with sequence number in seqn field of 
> > > mbuf,
> > this will be the reference for reordering at the tx end.
> > > Sequence number will be of type uint32_t. New sequence number field seqn
> > will be added to mbuf structure.
> > >
> > > Design:
> > > a)There will be reorder buffer(circular buffer) structure maintained in 
> > > reorder
> > library to store reordered packets and other details of buffer like head to 
> > drain
> > the packet from, min sequence number and other details.
> > >b)Library will provide insert and drain functions to 
> > > reorder and fetch
> > out the reordered packets respectively.
> > > c)Users of library should pass the packets to insert functions for 
> > > reordering.
> > >
> > > Insertion logic:
> > > Sequence number of current packet will be used to calculate offset in 
> > > reorder
> > buffer and write packet to the location  in the reorder buffer 
> > corresponding to
> > offset.
> > >  Offset is calculated as difference of 
> > > current packet  sequence
> > number and sequence number associated with  reorder buffer.
> > >
> > > During sequence number wrapping or wrapping over of reorder buffer size,
> > before inserting the new packet we should move offset number of packets to
> > other buffer called overflow buffer and advance the head of reorder buffer 
> > by
> > "offset-reorder buffer size" and insert the new packet.
> > >
> > > Insert function:
> > > int rte_reorder_insert(struct rte_reorder_buffer *buffer, struct
> > > rte_mbuf *mbuf);
> > > Note: Other insert function is also under plan to insert burst of packets.
> > >
> > >Reorder buffer:
> > > struct rte_reorder_buffer {
> > > unsigned int size;  /* The size (number of entries) of the 
> > > buffer. */
> > > unsigned int mask;  /* Mask (size - 1) of the buffer */
> > > unsigned int head;  /* Current head of buffer */
> > > uint32_t min_seqn;  /* latest sequence number associated with 
> > > buffer
> > */
> > > struct rte_mbuf *entries[MAX_REORDER_BUFFER_SIZE]; /* buffer
> > > to hold reordered mbufs */ };
> > >
> > > d)Users can fetch out the reordered packets by drain function provided by
> > library. Users must pass the mbuf array , drain function should fill  
> > passed mbuff
> > array  with the reordered buffer packets.
> > > During drain operation, overflow buffer  packets will be fetched out 
> > > first and
> > then reorder buffer.
> > >
> > > Drain function:
> > >int rte_reorder_drain(struct rte_reorder_buffer
> > > *buffer, struct rte_mbuf **mbufs)
> > >
> > > Thanks,
> > > Reshma
> > >
> > 
> > This seems reasonable, but why not integrate it with the distributor 
> > library rather
> > than create a separate library for it?  It seems as though the distributor 
> > library is
> > a pre-requisite for this libraries use anyway, as otherwise there will not 
> > be
> > anything to reorder Neil
> > 
> 
> Hi  Neil,
> 
> Reorder library should be standalone , as there are many ways that can cause 
> out of ordering of packets, I just mentioned future packet distributor 
> enhancements 
>  as one of the example for out of ordering. 
> Other ways like, users can directly distribute the packets to different cores 
> via rings and that causes packet out of ordering as well. 
> So, keeping reorder library standalone would be good to work with all packet 
> distribution ways.
> 


Hmm, ok, that seems reasonable.

Just out of curiosity, where do you intend to inject the packet sequence number
for this library?

Neil



[dpdk-dev] [PATCH v2] librte_pmd_packet: add PMD for AF_PACKET-based virtual devices

2014-10-08 Thread Neil Horman
On Wed, Oct 08, 2014 at 05:57:46PM +0200, Thomas Monjalon wrote:
> 2014-09-29 11:05, Bruce Richardson:
> > On Fri, Sep 26, 2014 at 10:08:55AM -0400, Neil Horman wrote:
> > > On Fri, Sep 26, 2014 at 11:28:05AM +0200, Thomas Monjalon wrote:
> > > > 2014-09-16 16:16, Neil Horman:
> > > > > On Fri, Sep 12, 2014 at 02:05:23PM -0400, John W. Linville wrote:
> > > > > > Ping?  Are there objections to this patch from mid-July?
> > > > > 
> > > > > Thomas, Where are you on this?  It seems like if you don't have any 
> > > > > objections
> > > > > to this patch, it should go in, in ilght of the lack of further 
> > > > > commentary.
> > > > 
> > > > 1) It doesn't appear as a top priority.
> > > Thats your responsibility.  Patches can't languish and rot on a list 
> > > forever
> > > just because others aren't willing to test it.  If theres further testing 
> > > that
> > > you feel it needs, ask. But from my read, its been tested for 
> > > functionality and
> > > performance (though high performance is never expected from a AF_PACKET 
> > > PMD).
> > > Given that any one PMD will not affect the performance of another in 
> > > isolation,
> > > I'm not sure what more you're waiting for here.
> 
> Yes, integration of new PMD must be accelerated.
> 
> > > > 2) It's competing with pcap PMD and bifurcated PMD to come
> > > >(http://dpdk.org/ml/archives/dev/2014-September/005379.html)
> > > Regarding the pcap PMD, so?  Its an alternate implementation that provides
> > > different features with different limitations.  The fact that they are 
> > > simmilar
> > > is irrelevant.  If simmilarity was the test, then we wouldn't bother with 
> > > the
> > > bifurcated driver either, because the pcap pmd already exists.
> > > 
> > > Regarding the bifurcated driver, you can't hold existing patches on the 
> > > promise
> > > of another pmd thats comming at an indeterminate time in the future.  
> > > Theres no
> > > reason not to take this now and deprecate it in the future if there is
> > > sufficient overlap with the bifurcated driver, though to my point above, 
> > > they
> > > still address different needs with different limitations, so I don't see 
> > > doing
> > > so as necessecary.
> 
> Yes, we'll discuss it when bifurcated driver will be released.
> 
john Fastabend posted it to netdev just a few days ago. There have been some
concerns raised, which he is trying to address.  I'm watching how that goes.

> > > > 3) There is no test associated with this PMD.
> > > That would have been a great comment to make a few months back, though 
> > > whats
> > > wrong with testpmd here?  That seems to be the same test that every other 
> > > pmd
> > > uses. What exactly are you looking for?
> 
> I was thinking of testing behaviour with different kernel configurations and
> unit tests for --vdev options. But it's not a major blocker.
> 
Thats fine with me.  If theres a set of unit tests that you have documentation
for, I'm sure we would be happy to run them.  I presume you just want all the
pmd vdev option exercised?  Any specific sets of kernel configurations?


> > > > If one of this item becomes wrong, it should go in.
> > > 
> > > > Currently, 2 projects are being initiated for validation (dcts) and
> > > > documentation. Keeping new things outside of the DPDK core makes it
> > > > clear that they have not to be supported by dcts and doc yet.
> > > > So, it is better to have an external PMD, like memnic, acting as a
> > > > staging area.
> > > > 
> > > So, this brings up an excellent point - Validation and support.  Commonly 
> > > open
> > > source projects don't provide support at the upstream HEAD. Those items 
> > > are
> > > applied and inforced by distributors.  Theres no need to ensure that the
> > > upstream head is always the most performance and stable point of the 
> > > tree.  Its
> > > that need that keeps the development pace slow, and creates frustrations 
> > > like
> > > this one, where a patch sits unaddressed for long periods of time.  
> > > Commonly the
> > > workflow for most open source projects is for there to be a window of 
> > > time where
> > > visual review and basic functional testing are sufficient for acceptance 
> > > into
> > > the head of the tree.  After the development window closes there is a
> > > stabilization period where testing/validation is done to ensure that no
> > > regressions have been encountered, optionally with a -next branch 
> > > temporarily
> > > being created to accept patches for upcomming future releases.  If 
> > > regressions
> > > are found, its a simple matter in git to bisect back to the offending 
> > > patch,
> > > allow the contributing developer an opportunity to fix the issue, or to 
> > > drop the
> > > patch.  Using a workflow like this we can have a reasonable balance of 
> > > needs
> > > (good patch turn around time, as well as reasonable testing).  We've 
> > > discussed
> > > this when I posted the PMD_REGISTER_DRIVER patch months ago, and I 
> > > tho

[dpdk-dev] [PATCH 0/4] Add DSO symbol versioning to support backwards compatibility

2014-10-08 Thread Neil Horman
On Wed, Oct 08, 2014 at 05:57:29PM +0200, Thomas Monjalon wrote:
> Hi Neil,
> 
> 2014-10-07 17:01, Neil Horman:
> > On Wed, Oct 01, 2014 at 02:59:40PM -0400, Neil Horman wrote:
> > > On Fri, Sep 26, 2014 at 10:45:49AM -0400, Neil Horman wrote:
> > > > On Fri, Sep 26, 2014 at 12:41:33PM +0200, Thomas Monjalon wrote:
> > > > > Hi Neil,
> > > > > 
> > > > > 2014-09-24 14:19, Neil Horman:
> > > > > > Ping Thomas. I know you're busy, but I would like this to not fall 
> > > > > > off anyones
> > > > > > radar.  You alluded to concerns regarding what, for lack of a 
> > > > > > better term,
> > > > > > ABI/API lockin.  I had asked you to enuumerate/elaborate on 
> > > > > > specifics, but never
> > > > > > heard back.  Are there further specifics you wish to discuss, or 
> > > > > > are you
> > > > > > satisfied with the above answers?
> > > > > 
> > > > > Sorry for not being very reactive on this thread.
> > > > > All this discussion is very interesting but it's really not the proper
> > > > > time to apply it. As you said, it requires an extra effort. I'm not 
> > > > > saying
> > > > > it will never be integrated. I'm just saying that we cannot change
> > > > > everything at the same time.
> > > > > 
> > > > > Let me sum up the situation. This community project has been very 
> > > > > active
> > > > > for few months now. First, we learnt how to make some releases 
> > > > > together
> > > > > and we are improving the process to be able to deliver a new major 
> > > > > release
> > > > > every 4 months while having some good quality process.
> > > > > But these releases are still not complete because documentation is not
> > > > > integrated yet. Then developers should have a role in documentation 
> > > > > updates.
> > > > > We also need to integrate and learn how to use more tools to be more
> > > > > efficient and improve quality.
> > > > > 
> > > > > So the question is "when should we care about API compatibility"?
> > > > > And the answer is: ASAP, but not now. I feel next year is a better 
> > > > > target.
> > > > > Because the most important priority is to move together at a pace 
> > > > > which
> > > > > allow most of us to stay in the race.
> > > > 
> > > > I'm sorry Thomas, I don't accept this.  I asked you for details as to 
> > > > your
> > > > concerns regarding this patch series, and you've provided more vague 
> > > > comments.
> > > > I need details to address
> > > > 
> > > > You say it requires extra effort, you're right it does.  Any feature 
> > > > that you
> > > > integreate requires some additional effort.  How is this patch any 
> > > > different
> > > > from adding the acl library or any other new API?  Everything requires
> > > > maintenence, thats how software works.  What specfically about this 
> > > > patch series
> > > > makes the effort insurmountable to you?
> > > > 
> > > > You say you're improving your process.  Great, this patch aids in that 
> > > > process
> > > > by ensuring backwards compatibility for a period of time.  Given that 
> > > > the API
> > > > and ABI can still evolve within this framework, as I've described, how 
> > > > is this
> > > > patch series not a significant step forward toward your goal of quality 
> > > > process.
> > > > 
> > > > You say documentation isn't integrated.  So, what does getting 
> > > > documentation
> > > > integrated have to do with this patch set, or any other?  I don't see 
> > > > you
> > > > holding any other patches based on documentation.  Again, nothing in 
> > > > this series
> > > > prevents evolution of the API or ABI.  If you're hope is to wait until
> > > > everything is perfect, then apply some control to the public facing 
> > > > API, and get
> > > > it all documented, none of thosse things will ever happen, I promise 
> > > > you.
> > > > 
> > > > You say you also need to learn to use more tools to be more efficient 
> > > > and
> > > > improve quality.  Great!  Thats exactly what this is. If we mandate 
> > > > even a short
> > > > term commitment to ABI stability (1 single relese worth of time), we 
> > > > will
> > > > quickly identify what API's change quickly and where we need to be 
> > > > cautious with
> > > > our API design.  If you just assume that developers will get better of 
> > > > their own
> > > > volition, it will never happen.
> > > > 
> > > > You say this should go in next year, but not now.  When exactly?  What 
> > > > event do
> > > > you forsee occuring in the next 12-18 months that will change 
> > > > everything such
> > > > that we can start supporing an ABI for more than just a few weeks at 
> > > > the head of
> > > > the tree?  
> > > > 
> > > > To this end, I just did a quick search through the git history for dpdk 
> > > > to look
> > > > at the histories of all the header files that are exposed via the 
> > > > makefile
> > > > SYMLINK command (given that that provides a list of header files that
> > > > applications can include, and embodies all the function symbols and 
> > > 

[dpdk-dev] [PATCH 0/3] Adjust mbuf flag definitions

2014-10-08 Thread Thomas Monjalon
2014-10-03 16:36, Bruce Richardson:
> This patchset does some cleanup work on the mbuf flag definitions. It is 
> based off the ideas discussed in the previous RFC patch. 
> [Ref: http://thread.gmane.org/gmane.comp.networking.dpdk.devel/6155]
> 
> Summary of changes:
> * Adjust TX flags to start at bit 55 in flags field and work downwards,
>   leaving bits 56 upward reserved for generic mbuf flags, i.e. non-offload
>   flags.
> * Update the existing RX flags to be in a similar format to the newly
>   redefined TX flags for clarity.
> * Finally we add a comment for the ctrl mbuf flag to ensure all flags are
>   properly documented.
> 
> Bruce Richardson (3):
>   mbuf: move TX flags to group them near end of field
>   mbuf: RX flag format update
>   mbuf: add comment for ctrl mbuf flag

Acked and applied

Thanks
-- 
Thomas


[dpdk-dev] [PATCH v2 0/5] Mbuf Structure Rework, part 3

2014-10-08 Thread Thomas Monjalon
2014-09-29 15:58, De Lara Guarch, Pablo:
> > -Original Message-
> > This is the final planned set of patches to make changes to the mbuf
> > data structure and associated files. This patch set makes more changes to
> > help improve performance following the mbuf changes and adds in two new
> > fields into the mbuf structure.
> > 
> > It is planned to add other fields other than the two provided here, but
> > patches for adding those fields will be included in the patch sets for the
> > changes making use of those fields, since adding them does not affect, or
> > move, any other mbuf fields.
> > 
> > Changes in V2:
> > * Updated userdata pointer in mbuf to always be 8 bytes big
> > * Updated a number of commit messages to have more details about the
> > performance benefits of the changes proposed in the patches
> > * Removed old patch 5 which added the second vlan tag, and replaced it with
> > a new, smaller patch which just moves the existing vlan_tci field above the
> > 16-bit reserved space.
> > 
> > Bruce Richardson (5):
> >   mbuf: ensure next pointer is set to null on free
> >   ixgbe: add prefetch to improve slow-path tx perf
> >   testpmd: Change rxfreet default to 32
> >   mbuf: add userdata pointer field
> >   mbuf: switch vlan_tci and reserved2 fields
> 
> Acked-by: Pablo de Lara 

Applied

Thanks
-- 
Thomas


[dpdk-dev] [PATCH RFC] librte_reorder: new reorder library

2014-10-08 Thread Pattan, Reshma


> -Original Message-
> From: Neil Horman [mailto:nhorman at tuxdriver.com]
> Sent: Tuesday, October 7, 2014 12:22 PM
> To: Pattan, Reshma
> Cc: dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH RFC] librte_reorder: new reorder library
> 
> On Tue, Oct 07, 2014 at 09:33:06AM +, Pattan, Reshma wrote:
> > Hi All,
> >
> > I am planning  to implement packet reorder library. Details are as below,
> please go through them and provide the comments.
> >
> > Requirement:
> >To reorder out of ordered packets that are received from 
> > different
> cores.
> >
> > Usage:
> > To be used along with distributor library. Next version of distributor are
> planned to distribute incoming packets to all worker cores irrespective of the
> flow type.
> > In this case to ensure in order delivery of the packets at output side 
> > reorder
> library will used by the tx end.
> >
> > Assumption:
> > All input packets will be marked with sequence number in seqn field of mbuf,
> this will be the reference for reordering at the tx end.
> > Sequence number will be of type uint32_t. New sequence number field seqn
> will be added to mbuf structure.
> >
> > Design:
> > a)There will be reorder buffer(circular buffer) structure maintained in 
> > reorder
> library to store reordered packets and other details of buffer like head to 
> drain
> the packet from, min sequence number and other details.
> >b)Library will provide insert and drain functions to reorder 
> > and fetch
> out the reordered packets respectively.
> > c)Users of library should pass the packets to insert functions for 
> > reordering.
> >
> > Insertion logic:
> > Sequence number of current packet will be used to calculate offset in 
> > reorder
> buffer and write packet to the location  in the reorder buffer corresponding 
> to
> offset.
> >  Offset is calculated as difference of current 
> > packet  sequence
> number and sequence number associated with  reorder buffer.
> >
> > During sequence number wrapping or wrapping over of reorder buffer size,
> before inserting the new packet we should move offset number of packets to
> other buffer called overflow buffer and advance the head of reorder buffer by
> "offset-reorder buffer size" and insert the new packet.
> >
> > Insert function:
> > int rte_reorder_insert(struct rte_reorder_buffer *buffer, struct
> > rte_mbuf *mbuf);
> > Note: Other insert function is also under plan to insert burst of packets.
> >
> >Reorder buffer:
> > struct rte_reorder_buffer {
> > unsigned int size;  /* The size (number of entries) of the 
> > buffer. */
> > unsigned int mask;  /* Mask (size - 1) of the buffer */
> > unsigned int head;  /* Current head of buffer */
> > uint32_t min_seqn;  /* latest sequence number associated with 
> > buffer
> */
> > struct rte_mbuf *entries[MAX_REORDER_BUFFER_SIZE]; /* buffer
> > to hold reordered mbufs */ };
> >
> > d)Users can fetch out the reordered packets by drain function provided by
> library. Users must pass the mbuf array , drain function should fill  passed 
> mbuff
> array  with the reordered buffer packets.
> > During drain operation, overflow buffer  packets will be fetched out first 
> > and
> then reorder buffer.
> >
> > Drain function:
> >int rte_reorder_drain(struct rte_reorder_buffer
> > *buffer, struct rte_mbuf **mbufs)
> >
> > Thanks,
> > Reshma
> >
> 
> This seems reasonable, but why not integrate it with the distributor library 
> rather
> than create a separate library for it?  It seems as though the distributor 
> library is
> a pre-requisite for this libraries use anyway, as otherwise there will not be
> anything to reorder Neil
> 

Hi  Neil,

Reorder library should be standalone , as there are many ways that can cause 
out of ordering of packets, I just mentioned future packet distributor 
enhancements 
 as one of the example for out of ordering. 
Other ways like, users can directly distribute the packets to different cores 
via rings and that causes packet out of ordering as well. 
So, keeping reorder library standalone would be good to work with all packet 
distribution ways.

Thanks,
Reshma 

> > --
> > Intel Shannon Limited
> > Registered in Ireland
> > Registered Office: Collinstown Industrial Park, Leixlip, County
> > Kildare Registered Number: 308263 Business address: Dromore House,
> > East Park, Shannon, Co. Clare
> >
> > This e-mail and any attachments may contain confidential material for the 
> > sole
> use of the intended recipient(s). Any review or distribution by others is 
> strictly
> prohibited. If you are not the intended recipient, please contact the sender 
> and
> delete all copies.
> >
> >


[dpdk-dev] [PATCH] Modify tools/setup.sh to be compatible with fedora 21

2014-10-08 Thread Daniel Mrzyglod
From: Daniel Mrzyglod 

script was expecting /lib/modules/$(uname -r)/kernel/drivers/uio/uio.ko but in 
fedora 21
there are Compressed kernel modules - xz (LZMA)

Signed-off-by: Daniel Mrzyglod 
---
 tools/setup.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/setup.sh b/tools/setup.sh
index 369e09e..6a9d23c 100755
--- a/tools/setup.sh
+++ b/tools/setup.sh
@@ -169,7 +169,7 @@ load_igb_uio_module()

/sbin/lsmod | grep -s uio > /dev/null
if [ $? -ne 0 ] ; then
-   if [ -f /lib/modules/$(uname -r)/kernel/drivers/uio/uio.ko ] ; 
then
+   if ls /lib/modules/$(uname -r)/kernel/drivers/uio/uio.ko* &> 
/dev/null; then
echo "Loading uio module"
sudo /sbin/modprobe uio
fi
-- 
2.1.0



[dpdk-dev] [PATCH] Modify tools/setup.sh to be compatible with fedora 21

2014-10-08 Thread Mrzyglod, DanielX T
Section From: should be 
From: Daniel Mrzyglod . 

error due to setting fedora 21 alpha on the new machine
> -Original Message-
> From: Mrzyglod, DanielX T
> Sent: Wednesday, October 08, 2014 3:12 PM
> To: dev at dpdk.org
> Cc: Daniel Mrzyglod; Mrzyglod, DanielX T
> Subject: [PATCH] Modify tools/setup.sh to be compatible with fedora 21
> 
> From: Daniel Mrzyglod 
> 
> script was expecting /lib/modules/$(uname -r)/kernel/drivers/uio/uio.ko but in
> fedora 21
> there are Compressed kernel modules - xz (LZMA)
> 
> Signed-off-by: Daniel Mrzyglod 
> ---
>  tools/setup.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/setup.sh b/tools/setup.sh
> index 369e09e..6a9d23c 100755
> --- a/tools/setup.sh
> +++ b/tools/setup.sh
> @@ -169,7 +169,7 @@ load_igb_uio_module()
> 
>   /sbin/lsmod | grep -s uio > /dev/null
>   if [ $? -ne 0 ] ; then
> - if [ -f /lib/modules/$(uname -r)/kernel/drivers/uio/uio.ko ] ;
> then
> + if ls /lib/modules/$(uname -r)/kernel/drivers/uio/uio.ko* &>
> /dev/null; then
>   echo "Loading uio module"
>   sudo /sbin/modprobe uio
>   fi
> --
> 2.1.0





[dpdk-dev] [README]:Running DPDK in a LXC-based Container

2014-10-08 Thread Murthy, Krishna J
 Running DPDK in a LXC-based Container 

Author: Krishnamurthy Jambur

Note: This README assumes you have the a LXC based container running on 
  the IA platform. This was tested on Windriver OVP Linux 6.0 

In this README we are will go through some of the unique steps involved 
in attaching a NIC to the LXC-based Container and the running a sample 
DPDP application in the container. 

1. Attaching NIC device to the container

Let's assume for this example that you would like to have a management interface
and then separate dedicated 10G NIC to run DPDK. With this requirement your LXC 
config file should have one networking sections one for veth. For the 10G NIC we
need to use uio device file system 

veth uses a paired set of network interfaces, one inside the container 
specified 
by the option lxc.network.name, and one in the host specified by the option 
lxc.network.pair. The latter is attached automatically to a bridge in the host 
if the name of the bridge is specified by the option lxc.network.link.

# networking-veth
lxc.network.type = veth
lxc.network.link = virbr0
lxc.network.veth.pair = vc0
lxc.network.name = eth0
lxc.network.hwaddr = ac:de:48:34:01:01
lxc.network.flags = up
lxc.network.ipv4 = 192.168.122.12/24
lxc.network.ipv4.gateway = 192.168.122.1

once you have this your LXC container will have an IP using bridge. Next step 
is to install DPDK igb_uio on the host and then bind it to the the 10G Niantic 
NIC ports as usual. Once installed do ls -l /dev/uio* on the host. These are the
uioX interface corresponding to the the NIC ports to which DPDK igb_uio is 
binded. DPDK application will use these device files. In order to provide access
to these device files inside the LXC container not down the major number [e.g. 
in this case 249] and add this config to the LXC config file 

lxc.cgroup.devices.allow = c 249:* rwm

once you start the container then you need to create these uio0, uio1,... 
devices nodes inside the container file system using mknod command as usual 
with the same major and minor numbers. 

mknod /dev/uio3 c 249 0
mknod /dev/uio3 c 249 3
mknod /dev/uio2 c 249 2
mknod /dev/uio1 c 249 1

2. Host allocated Huge page access inside container

For this set it is assumes that you would have pre-allocated hugepages and 
mounted on the on the host. Once you have this set-up then you need to provide 
access to hugetlbfs file system to the container using the LXC config file. 

lxc.mount.entry = /mnt/huge mnt/huge none bind,create=dir 0 0

once you have this DPDK application will be able to use the host mounted huge 
pages in the container. 

Next step is run the DPDK application as usual in the LXC Container

Regards,
Krishna 

--
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole 
use of the intended recipient(s). Any review or distribution by others is 
strictly prohibited. If you are not the intended recipient, please contact the 
sender and delete all copies.




[dpdk-dev] [PATCH] eal/bsd: fix core detection

2014-10-08 Thread David Marchand
Hello Neil,

On Wed, Oct 8, 2014 at 12:53 PM, Neil Horman  wrote:
>
> Though, that said, why are these files diverged in the first place?  At
> least as
> far as eal_lcore.c and eal.c are concerned the only differences appear to
> be
> cases of one file being updated and not the other.  It seems that, rather
> than
> doing a patch like this to bring bsd up to date with linux, we should just
> de-dup the files, put them in a common location and handle any real
> behavioral
> differences with macros/ifdefs.  Is there a reason for separating them?
>

I agree that there should only be one file for those things (and a lot of
other stuff like eal_debug, eal_log etc...).
This is not that easy: I tried once and gave up because the resulting
patchset was huge.
Personally, I don't have time to dig into this.


-- 
David Marchand


[dpdk-dev] [PATCH] eal/bsd: fix core detection

2014-10-08 Thread David Marchand
Following "options parsing" patchset (commit d7cb626f and 489a9d6c), core
detection is not working correctly on bsd.

./x86_64-native-bsdapp-gcc/app/test -c f -n 4 -- -i
[...]
EAL: lcore 0 unavailable
EAL: invalid coremask

Align bsd to linux:
- commit f563a372 "eal: fix recording of detected/enabled logical cores"
- commit 4f04db8b "eal: check coremask against detected lcores"

Reported-by: Zhan, Zhaochen 
Signed-off-by: David Marchand 
---
 lib/librte_eal/bsdapp/eal/eal.c   |6 +++---
 lib/librte_eal/bsdapp/eal/eal_lcore.c |   15 +--
 2 files changed, 12 insertions(+), 9 deletions(-)

diff --git a/lib/librte_eal/bsdapp/eal/eal.c b/lib/librte_eal/bsdapp/eal/eal.c
index c40a59a..ca99cb9 100644
--- a/lib/librte_eal/bsdapp/eal/eal.c
+++ b/lib/librte_eal/bsdapp/eal/eal.c
@@ -517,6 +517,9 @@ rte_eal_init(int argc, char **argv)
if (rte_eal_log_early_init() < 0)
rte_panic("Cannot init early logs\n");

+   if (rte_eal_cpu_init() < 0)
+   rte_panic("Cannot detect lcores\n");
+
fctret = eal_parse_args(argc, argv);
if (fctret < 0)
exit(1);
@@ -551,9 +554,6 @@ rte_eal_init(int argc, char **argv)

rte_config_init();

-   if (rte_eal_cpu_init() < 0)
-   rte_panic("Cannot detect lcores\n");
-
if (rte_eal_memory_init() < 0)
rte_panic("Cannot init memory\n");

diff --git a/lib/librte_eal/bsdapp/eal/eal_lcore.c 
b/lib/librte_eal/bsdapp/eal/eal_lcore.c
index 43a5c01..5b52eba 100644
--- a/lib/librte_eal/bsdapp/eal/eal_lcore.c
+++ b/lib/librte_eal/bsdapp/eal/eal_lcore.c
@@ -71,16 +71,18 @@ rte_eal_cpu_init(void)
unsigned count = 0;

const unsigned ncpus = get_ncpus();
-
-   /* disable lcores that were not detected */
-   RTE_LCORE_FOREACH(lcore_id) {
-
+   /*
+* Parse the maximum set of logical cores, detect the subset of running
+* ones and enable them by default.
+*/
+   for (lcore_id = 0; lcore_id < RTE_MAX_LCORE; lcore_id++) {
lcore_config[lcore_id].detected = (lcore_id < ncpus);
if (lcore_config[lcore_id].detected == 0) {
config->lcore_role[lcore_id] = ROLE_OFF;
continue;
}
-   count++;
+   /* By default, each detected core is enabled */
+   config->lcore_role[lcore_id] = ROLE_RTE;
lcore_config[lcore_id].core_id = cpu_core_id(lcore_id);
lcore_config[lcore_id].socket_id = cpu_socket_id(lcore_id);
if (lcore_config[lcore_id].socket_id >= RTE_MAX_NUMA_NODES)
@@ -93,8 +95,9 @@ rte_eal_cpu_init(void)
 #endif
RTE_LOG(DEBUG, EAL, "Detected lcore %u\n",
lcore_id);
+   count ++;
}
-
+   /* Set the count of enabled logical cores of the EAL configuration */
config->lcore_count = count;
RTE_LOG(DEBUG, EAL, "Support maximum %u logical core(s) by 
configuration.\n",
RTE_MAX_LCORE);
-- 
1.7.10.4



[dpdk-dev] [PATCH] Modify tools/setup.sh to be compatible with fedora 21

2014-10-08 Thread Neil Horman
On Wed, Oct 08, 2014 at 02:11:31PM +0100, Daniel Mrzyglod wrote:
> From: Daniel Mrzyglod 
> 
> script was expecting /lib/modules/$(uname -r)/kernel/drivers/uio/uio.ko but 
> in fedora 21
> there are Compressed kernel modules - xz (LZMA)
> 
> Signed-off-by: Daniel Mrzyglod 
> ---
>  tools/setup.sh | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/setup.sh b/tools/setup.sh
> index 369e09e..6a9d23c 100755
> --- a/tools/setup.sh
> +++ b/tools/setup.sh
> @@ -169,7 +169,7 @@ load_igb_uio_module()
>  
>   /sbin/lsmod | grep -s uio > /dev/null
>   if [ $? -ne 0 ] ; then
> - if [ -f /lib/modules/$(uname -r)/kernel/drivers/uio/uio.ko ] ; 
> then
> + if ls /lib/modules/$(uname -r)/kernel/drivers/uio/uio.ko* &> 
> /dev/null; then
>   echo "Loading uio module"
>   sudo /sbin/modprobe uio
>   fi
> -- 
> 2.1.0
> 
> 

Just use modinfo uio and test the exit code.  That will make the setup script
agnostic to both filename extension and to kernel version, as modinfo looks in
the running kernel path automatically

Neil



[dpdk-dev] [PATCH] eal/bsd: fix core detection

2014-10-08 Thread Bruce Richardson
On Wed, Oct 08, 2014 at 10:43:31AM +0200, David Marchand wrote:
> Following "options parsing" patchset (commit d7cb626f and 489a9d6c), core
> detection is not working correctly on bsd.
> 
> ./x86_64-native-bsdapp-gcc/app/test -c f -n 4 -- -i
> [...]
> EAL: lcore 0 unavailable
> EAL: invalid coremask
> 
> Align bsd to linux:
> - commit f563a372 "eal: fix recording of detected/enabled logical cores"
> - commit 4f04db8b "eal: check coremask against detected lcores"
> 
> Reported-by: Zhan, Zhaochen 
> Signed-off-by: David Marchand 

Acked-by: Bruce Richardson 




[dpdk-dev] [PATCH] eal/bsd: fix core detection

2014-10-08 Thread Zhan, Zhaochen
> > Following "options parsing" patchset (commit d7cb626f and 489a9d6c), core
> > detection is not working correctly on bsd.
> >
> > ./x86_64-native-bsdapp-gcc/app/test -c f -n 4 -- -i
> > [...]
> > EAL: lcore 0 unavailable
> > EAL: invalid coremask
> >
> > Align bsd to linux:
> > - commit f563a372 "eal: fix recording of detected/enabled logical cores"
> > - commit 4f04db8b "eal: check coremask against detected lcores"
> >
> > Reported-by: Zhan, Zhaochen 
> > Signed-off-by: David Marchand 
> 
> Acked-by: Bruce Richardson 

Tested-by: Zhaochen Zhan 

This patch has been verified on FreeBSD 10.0.
DPDK base commit: 3043ce5011aa7075b32c80c79b9db96199938602
CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
GCC: gcc48 (FreeBSD Ports Collection) 4.8.3



[dpdk-dev] [PATCH v4 0/8] link bonding

2014-10-08 Thread Jiajia, SunX
Tested-by: Jiajia, SunX 

This patch has been verified on FC20 with Niantic.
The test environment detail information as the following:

CPU: Intel(R) Xeon(R) CPU E5-2680 v2 @ 2.80GHz
OS: Linux  3.11.10-301.fc20.x86_64
GCC: 4.8.2
NIC: Niantic.

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Declan Doherty
> Sent: Tuesday, September 30, 2014 5:58 PM
> To: dev at dpdk.org
> Subject: [dpdk-dev] [PATCH v4 0/8] link bonding
> 
> v4:
> - Rebased to account for changes in master.
> - Fix for rte_eth_bond_slaves_get() introduced in v3 patch set
> - Addressed issue around disabling/enabling link status polling around
> adding/
>   removing slaves devices.
> 
> v3 :
> - Typo fix for the bond free mbufs patch.
> - Rebased to account for changes in the mbuf patches.
> - Add support for slave devices which don't support link status
> interrupts
> - Tidy up the link bonding unit test so that all tests use the new test
> macros.
> 
> v2 :
> Addresses issues with the logic around the handling of fail
> transmissions.
> In this version all modes behave in a manner similar to a standard PMD,
> returning the number of successfully transmitted mbufs and with the
> failing
> mbufs at the end of bufs array for freeing / retransmission by the
> application software
> 
> v1:
> 
> This patch set adds support for link status interrupt in the link
> bonding
> pmd. It also contains some patches to tidy up the code structure and to
> of the link bonding code and to fix bugs relating to transmission
> failures in the under lying slave pmd which could lead to leaked mbufs.
> 
> Declan Doherty (8):
>   bond: link status interrupt support
>   bond: removing switch statement from rx burst method
>   bond: fix naming inconsistency in tx_burst_round_robin
>   bond: free mbufs if transmission fails in bonding tx_burst functions
>   test app: adding support for generating variable sized packet
>   testpmd: adding parameter to reconfig method to set socket_id when
> adding new port to portlist
>   bond: lsc polling support
>   bond: unit test test macro refactor
> 
>  app/test-pmd/cmdline.c |   65 +-
>  app/test-pmd/testpmd.c |3 +-
>  app/test-pmd/testpmd.h |2 +-
>  app/test/packet_burst_generator.c  |   25 +-
>  app/test/packet_burst_generator.h  |6 +-
>  app/test/test.h|7 +-
>  app/test/test_link_bonding.c   | 3342 ++--
> 
>  app/test/virtual_pmd.c |   97 +-
>  app/test/virtual_pmd.h |   53 +-
>  lib/librte_pmd_bond/rte_eth_bond.h |   80 +
>  lib/librte_pmd_bond/rte_eth_bond_api.c |  319 ++-
>  lib/librte_pmd_bond/rte_eth_bond_args.c|   30 +-
>  lib/librte_pmd_bond/rte_eth_bond_pmd.c |  550 +++--
>  lib/librte_pmd_bond/rte_eth_bond_private.h |   71 +-
>  14 files changed, 2692 insertions(+), 1958 deletions(-)
> 
> --
> 1.7.12.2



[dpdk-dev] [PATCH] eal/bsd: fix core detection

2014-10-08 Thread Neil Horman
On Wed, Oct 08, 2014 at 10:43:31AM +0200, David Marchand wrote:
> Following "options parsing" patchset (commit d7cb626f and 489a9d6c), core
> detection is not working correctly on bsd.
> 
> ./x86_64-native-bsdapp-gcc/app/test -c f -n 4 -- -i
> [...]
> EAL: lcore 0 unavailable
> EAL: invalid coremask
> 
> Align bsd to linux:
> - commit f563a372 "eal: fix recording of detected/enabled logical cores"
> - commit 4f04db8b "eal: check coremask against detected lcores"
> 
> Reported-by: Zhan, Zhaochen 
> Signed-off-by: David Marchand 

Acked-by: Neil Horman 

Though, that said, why are these files diverged in the first place?  At least as
far as eal_lcore.c and eal.c are concerned the only differences appear to be
cases of one file being updated and not the other.  It seems that, rather than
doing a patch like this to bring bsd up to date with linux, we should just
de-dup the files, put them in a common location and handle any real behavioral
differences with macros/ifdefs.  Is there a reason for separating them?

Neil



[dpdk-dev] Queries related to Hyperthreading

2014-10-08 Thread Ramia, Kannan Babu
Please read this document for more detail information.

http://www.intel.com/content/www/us/en/architecture-and-technology/64-ia-32-architectures-optimization-manual.html?wapkw=248966

Also look into the chapter-8 in this document.

Regards
Kannan Babu

-Original Message-
From: Ravi Kumar Iyer [mailto:ravi.i...@aricent.com] 
Sent: Tuesday, October 7, 2014 11:35 AM
To: Ramia, Kannan Babu; dev at dpdk.org
Subject: RE: Queries related to Hyperthreading

Thanks Kannan,

Do we have any links/docs where in we could read more about the behavior of 
Hyperthreads.
We are executing a code which is similar to the following example:


So let us we have very tight loop of code which does basic arithmetic 
operations[addition, subtraction]. Now the cost to complete these operations is 
say C clock cycles on a single core[ with HT on]. Now if we run the same code 
with HT off, shouldn't the instruction pipeline of this code get executed 
faster [ theoretical at double speed].

Thanks
ravi




-Original Message-
From: Ramia, Kannan Babu [mailto:kannan.babu.ra...@intel.com]
Sent: Tuesday, October 07, 2014 9:48 AM
To: Ravi Kumar Iyer; dev at dpdk.org
Subject: RE: Queries related to Hyperthreading

Hi Ravi

The Intel HT actually has its own resources except few places like execution 
units and other resources in the pipeline, hence there are many factors play 
here for the performance of the application with and without HT feature. In 
simple words, even when you turned on the HT, and use only one of the HT, then 
the performance will be more or less be similar to the core with HT turned off. 
Also assume that if you are running same piece of code in both the HTs, but it 
doesn't have lots have contention on the common resource (you can figure out 
this using vTune profiling), then there will be very marginal performance 
difference between HT turned ON Vs OFF.

Regards
Kannan Babu

-Original Message-
From: dev [mailto:dev-boun...@dpdk.org] On Behalf Of Ravi Kumar Iyer
Sent: Tuesday, October 7, 2014 8:59 AM
To: dev at dpdk.org
Subject: [dpdk-dev] Queries related to Hyperthreading

hello Intel Gurus,

I have some queries related to intel HT technology. I am running an intel dpdk 
application and  trying to benchmark as well as optimize the same. As of now it 
is a single threaded application and would like to keep it like that for now. I 
am running it on an intel sandy bridge arch processor with HT on. Now to 
increase the performance I have disabled HT, but am still getting the same 
performance[ just a marginal improvement]. Now as per our understanding so far 
wrt HT, two Hardware threads share the same physical core. Now if we switch off 
HT[from bios] shouldn't just one hardware Thread get hold of the complete CPU 
core.  The application is heavily compute oriented and as such minimal memory 
access or any IO operations. Appreciate any insight on this.


thanks
ravi


"DISCLAIMER: This message is proprietary to Aricent and is intended solely for 
the use of the individual to whom it is addressed. It may contain privileged or 
confidential information and should not be circulated or used for any purpose 
other than for what it is intended. If you have received this message in error, 
please notify the originator immediately. If you are not the intended 
recipient, you are notified that you are strictly prohibited from using, 
copying, altering, or disclosing the contents of this message. Aricent accepts 
no responsibility for loss or damage arising from the use of the information 
transmitted by this email including damage from virus."


"DISCLAIMER: This message is proprietary to Aricent and is intended solely for 
the use of the individual to whom it is addressed. It may contain privileged or 
confidential information and should not be circulated or used for any purpose 
other than for what it is intended. If you have received this message in error, 
please notify the originator immediately. If you are not the intended 
recipient, you are notified that you are strictly prohibited from using, 
copying, altering, or disclosing the contents of this message. Aricent accepts 
no responsibility for loss or damage arising from the use of the information 
transmitted by this email including damage from virus."


[dpdk-dev] [PATCH v4 2/8]i40e:support VxLAN packet identification in librte_pmd_i40e

2014-10-08 Thread Liu, Jijiang
Hi Bruce,

> -Original Message-
> From: Richardson, Bruce
> Sent: Monday, September 29, 2014 6:48 PM
> To: Liu, Jijiang
> Cc: dev at dpdk.org
> Subject: Re: [dpdk-dev] [PATCH v4 2/8]i40e:support VxLAN packet
> identification in librte_pmd_i40e
> 
> On Fri, Sep 26, 2014 at 10:02:03AM +0800, Jijiang Liu wrote:
> > Support tunneling UDP port configuration on i40e in librte_pmd_i40e.
> > Currently, only VxLAN is implemented, which include
> >  -  VxLAN UDP port initialization
> >  -  Implement the APIs to configure VxLAN UDP port in librte_pmd_i40e.
> >
> > Signed-off-by: Jijiang Liu 
> > Acked-by: Helin Zhang 
> > Acked-by: Jingjing Wu 
> > Acked-by: Jing Chen 
> >
> > ---
> >  config/common_linuxapp|5 +
> >  lib/librte_mbuf/rte_mbuf.h|2 +
> >  lib/librte_pmd_i40e/i40e_ethdev.c |  200
> -
> >  lib/librte_pmd_i40e/i40e_ethdev.h |5 +
> >  lib/librte_pmd_i40e/i40e_rxtx.c   |   10 ++
> >  5 files changed, 221 insertions(+), 1 deletions(-)
> >
> > diff --git a/config/common_linuxapp b/config/common_linuxapp index
> > 5bee910..75a4cd7 100644
> > --- a/config/common_linuxapp
> > +++ b/config/common_linuxapp
> > @@ -212,6 +212,11 @@
> CONFIG_RTE_LIBRTE_I40E_QUEUE_NUM_PER_VF=4
> >  CONFIG_RTE_LIBRTE_I40E_ITR_INTERVAL=-1
> >
> >  #
> > +# Compile tunneling UDP port support
> > +#
> > +CONFIG_RTE_LIBRTE_TUNNEL_UDP_PORT=4789
> > +
> > +#
> >  # Compile burst-oriented VIRTIO PMD driver  #
> > CONFIG_RTE_LIBRTE_VIRTIO_PMD=y diff --git a/lib/librte_mbuf/rte_mbuf.h
> > b/lib/librte_mbuf/rte_mbuf.h index 1c6e115..4955684 100644
> > --- a/lib/librte_mbuf/rte_mbuf.h
> > +++ b/lib/librte_mbuf/rte_mbuf.h
> > @@ -538,6 +538,7 @@ static inline void rte_pktmbuf_reset(struct rte_mbuf
> *m)
> > m->port = 0xff;
> >
> > m->ol_flags = 0;
> > +   m->reserved = 0;
> > m->data_off = (RTE_PKTMBUF_HEADROOM <= m->buf_len) ?
> > RTE_PKTMBUF_HEADROOM : m->buf_len;
> >
> > @@ -607,6 +608,7 @@ static inline void rte_pktmbuf_attach(struct
> rte_mbuf *mi, struct rte_mbuf *md)
> > mi->pkt_len = mi->data_len;
> > mi->nb_segs = 1;
> > mi->ol_flags = md->ol_flags;
> > +   mi->reserved = md->reserved;
> >
> > __rte_mbuf_sanity_check(mi, 1);
> > __rte_mbuf_sanity_check(md, 0);
> 
> If the "reserved" field in the mbuf is now being used, it should be renamed to
> what its actually being used for. If it is still not being used, why this 
> change?
> 
> /Bruce

As I said in the cover letter, when your Mbuf Structure Rework(part 3) is 
applied, there will be minor changes later.
I will not keep the current changes in mbuf part, and rename the "reserved2" 
field to the "packet_type" in rte_mbuf structure instead.
Tunneling application need to know what packet type of incoming packet is. 

/Jijiang Liu