[dpdk-dev] [PATCH] app/test/test_table_acl: fix incorrect IP

[Betts, Ian]

Date: Mon, 14 Mar 2016 12:22:01 +
From: Fan Zhang <roy.fan.zh...@intel.com>
To: dev at dpdk.org
Subject: [dpdk-dev] [PATCH] app/test/test_table_acl: fix incorrect IP
header
Message-ID:
<1457958122-20136-1-git-send-email-roy.fan.zhang at intel.com>

This patch fixes the incorrect IP header in ACL table test.

Signed-off-by: Fan Zhang 

I have reviewed this, it looks fine to me, but test_acl maintainer should also 
Ack this.
Reviewed-by: Ian Betts 
Acked-by: Ian Betts  

Ian Betts
Intel Shannon Limited
Registered in Ireland
Registered Office Collinstown Industrial Park, Leixlip, County Kildare
Registered Number:308263
Business address: Dromore House, East Park, Shannon, Co. Clare

[dpdk-dev] [PATCH v10 0/4] examples: add performance-thread

[Betts, Ian]

-Original Message-
From: Thomas Monjalon [mailto:thomas.monja...@6wind.com] 
Sent: Friday, December 11, 2015 1:41 AM
To: Betts, Ian
Cc: dev at dpdk.org; Kulasek, TomaszX
Subject: Re: [dpdk-dev] [PATCH v10 0/4] examples: add performance-thread

>Applied, thanks
>The patch to mark it experimental and disable it has also been applied.
>The huge codebase has been integrated to give visibility.
>We need to discuss it more and decide how to continue (as a lib or drop).

Thanks Thomas,
When you analyze the "huge" codebase it breaks into 3 parts: 

1. A common l-thread subsystem ( ~2Kloc ) 
2. An l3fwd derived example ( ~2Kloc)
3. An hello world example pthread shim ( ~700loc )

If there is interest in lightweight threads the hope/expectation is that the 
l-thread
subsystem would become an rte library. On the scale of things I would describe 
it as an average sized lib in DPDK terms.

As for the examples, obviously there needs to be some example, but perhaps 
it is not so critical if they are retained in the present form, at least this 
is the only
place I can see to save lines of code.

Ian

[dpdk-dev] dev Digest, Vol 68, Issue 68

[Betts, Ian]

Date: Fri, 27 Nov 2015 15:09:24 +0200
From: Panu Matilainen 
To: "Doherty, Declan" , Thomas Monjalon

Cc: "dev at dpdk.org" 
Subject: Re: [dpdk-dev] [PATCH] cryptodev: mark experimental state
Message-ID: <56585604.9030909 at redhat.com>
Content-Type: text/plain; charset=utf-8; format=flowed

On 11/26/2015 03:51 PM, Doherty, Declan wrote:
>> -Original Message-
>> From: Thomas Monjalon [mailto:thomas.monjalon at 6wind.com]
>> Sent: Thursday, November 26, 2015 10:09 AM
>> To: Panu Matilainen; Doherty, Declan
>> Cc: dev at dpdk.org
>> Subject: Re: [dpdk-dev] [PATCH] cryptodev: mark experimental state
>>
>> 2015-11-26 10:00, Panu Matilainen:
>>> On 11/26/2015 09:39 AM, Panu Matilainen wrote:
 On 11/25/2015 07:38 PM, Thomas Monjalon wrote:
> --- a/config/common_linuxapp
> +++ b/config/common_linuxapp
> @@ -319,6 +319,7 @@ CONFIG_RTE_PMD_PACKET_PREFETCH=y
>
>#
># Compile generic crypto device library
> +# EXPERIMENTAL: API may change without prior notice
>#
>CONFIG_RTE_LIBRTE_CRYPTODEV=y
>CONFIG_RTE_LIBRTE_CRYPTODEV_DEBUG=n
 [...]

 I think an experimental library which declares itself exempt from the
 ABI policy should not be compiled by default. That way anybody wanting
 to try it out will be forced to notice the experimental status.

 More generally / longer term, perhaps there should be a
 CONFIG_RTE_EXPERIMENTAL which wraps all experimental features and
 defaults to off.
>>>
>>> On a related note, librte_mbuf_offload cannot be built if
>>> CONFIG_RTE_LIBRTE_CRYPTODEV is disabled. Which seems to suggest its (at
>>> least currently) so tightly couple to cryptodev that perhaps it too
>>> should be marked experimental and default to off.
>>
>> I think you are right.
>> Declan, what is your opinion?
>
>
> Hey Thomas, yes librte_mbuf_offload should also be set as experimental, it's
> probably one of the areas which will most likely change in the future.
>
> On the issue of turning off experimental libraries in the build by default, my
> preference would be not to turn them off unless the library has external
> dependencies, otherwise the possibility of patches being submitted which
> could break an experimental library will be much higher. In my opinion the
> fewer build configurations developers have to test against the better.

>>What I'm more worried about is users and developers starting to rely on 
>>it while still in experimental state, a single comment in the header is 
>>really easy to miss.

>>So I'd like to see *some* mechanism which forces users and developers to 
>>acknowledge the fact that they're dealing with experimental work. 
>>Defaulting to off is one possibility, another one would be wrapping 
>>experimental APIs behind a define which you have to set to be able to 
>>use the API, eg:

>>#if defined(I_KNOW_THIS_IS_EXPERIMENTAL_AND_MAY_EAT_BABIES)
>>[...]
>>#endif

>>  - Panu -

I can see alternative/complementary  that is to introduce a new "experimental" 
patch state in patchwork. 


This approach might be useful to get wider exposure and feedback on  
experimental work earlier in its lifecycle.
Experimental patches may be constrained to a limited subset of target and host 
environments.
Experimental patches should be based off of a stable dpdk release, but would 
not be applied in the release.

Whilst the objective would be that any such patch would mature to  become the 
kind of "experimental component" 
as proposed above in this thread, and/or eventually be adopted as a mainstream 
component, 
there would be no guarantee of that.

For the user it should very clearly be "Buyer beware" on such patches,  and for 
the
developer community the support burden should be solely  the responsibility of 
the
patch maintainer. 

The only reason to have a new patchwork state is to make it easier to filter 
them in patchwork.
Some way of publishing the list of experimental patches available for a release 
would be
helpful, maybe as an addendum to the release note ?

[dpdk-dev] [PATCH v9 3/4] examples: add l3fwd-thread example in performance-thread

[Betts, Ian]

-Original Message-
From: Thomas Monjalon [mailto:thomas.monja...@6wind.com] 
Sent: Tuesday, December 8, 2015 1:36 AM
To: Betts, Ian
Cc: dev at dpdk.org; stephen at networkplumber.org; Richardson, Bruce
Subject: Re: [PATCH v9 3/4] examples: add l3fwd-thread example in 
performance-thread

> +M: Ian Betts 
> +M: John McNamara 
> +F: doc/guides/sample_app_ug/performance_thread.rst

> Why doing 2 sections?
> John is already the doc maintainer. You don't need to add him here.

I just copy most of the other examples which also do this.
I also thought it was odd.
I will remove him.

> --- a/examples/Makefile
> +++ b/examples/Makefile
> @@ -77,5 +77,9 @@ DIRS-y += vmdq
>  DIRS-y += vmdq_dcb
>  DIRS-$(CONFIG_RTE_LIBRTE_POWER) += vm_power_manager
>  DIRS-$(CONFIG_RTE_LIBRTE_CRYPTODEV) += l2fwd-crypto
> -
> +ifneq ($(CONFIG_RTE_EXEC_ENV_LINUXAPP),)
> +ifneq ($(CONFIG_RTE_ARCH_X86_64),)
> +DIRS-y += performance-thread
> +endif
> +endif

> Matter of taste, I would prefer DIRS-$(CONFIG_RTE_ARCH_X86_64)

I need to combine CONFIG_RTE_EXEC_ENV_LINUXAPP ( the two clauses )

[dpdk-dev] [PATCH v8 3/4] examples: add l3fwd-thread example in performance-thread

[Betts, Ian]

-Original Message-
From: Thomas Monjalon [mailto:thomas.monja...@6wind.com] 
Sent: Monday, December 7, 2015 2:36 AM
To: Betts, Ian
Cc: dev at dpdk.org
Subject: Re: [dpdk-dev] [PATCH v8 3/4] examples: add l3fwd-thread example in 
performance-thread

> Hi,

>>I had no time to check how it works.
>> I'm just surprised that you introduce a new config option.
>> +DIRS-$(CONFIG_RTE_PERFORMANCE_THREAD) += performance-thread
> I think this option should be removed.

It is only to ensure that performance-thread example is built, along with all 
other examples,
if you run  "make"  top level of examples folder.
Also to ensure it is only enabled for x86_64 linux.

Is there is another  way ?

[dpdk-dev] [PATCH v8 0/4] examples: add performance-thread

[Betts, Ian]

-Original Message-
From: Thomas Monjalon [mailto:thomas.monja...@6wind.com] 
Sent: Saturday, December 5, 2015 9:21 PM
To: Stephen Hemminger; Glynn, Michael J; Betts, Ian
Cc: dev at dpdk.org; O'Driscoll, Tim; Richardson, Bruce
Subject: Re: [dpdk-dev] [PATCH v8 0/4] examples: add performance-thread

> > > Intel may have some milestone to get it into DPDK 2.2 but really 
> > > this seems too late...
> > 
> > >>>Yes, sure it is too late to have enough discussions in 2.2 
> > >>>timeframe
> > >Just to understand what we mean by too late...
> > >The original RFC was issued on 2nd September.
> > >Thus there have been some three months available for discussion, and for 
> > >people to raise any questions or concerns.
> > >The first patch was available on 30th September, and a number of 
> > >subsequent patch versions have been issued, meaning the code has been 
> > >available for review for two month As mentioned in the reply to Stephen, 
> > >there has been no adverse feedback during this period.
> > >/Ian
> > 
> > Hi Thomas/Stephen
> > 
> > I agree with Ian, how much time is expected for a discussion to happen? 
> > 
> > As Ian stated, the feature was stated in our 2.2 planned feature list, we 
> > created a RFC over 3 months ago, and there's been code available for review 
> > for over 2 months now! (not to mention several version updates, docs, 
> > etc.). 
> > Given this, I believe that there has been ample time for the community to 
> > review and provide feedback rather than waiting until the eve of RC3 and 
> > then requesting more time. 
> > 
> > In addition, by making it a sample application first people can test 
> > it, see if it's useful, and further enhance it. Based on usefulness 
> > and feedback, we can then decide whether to make it a DPDK library 
> > in a future release, make it a separate library somewhere else, or 
> > do nothing further on it
> > 
> > For these reasons, I believe it should be merged into RC3

I am not against this idea.
I just say that we have no time anymore for discussions.
There was no review probably because it is "just" an example.
Given that the code is huge, it would be better to have some good feedback for 
its integration. But it does not hurt to integrate anyway.
The only problem is the maintenance overload. When we change something in a 
library, every examples must be updated to keep them working.
That's why we must avoid keeping some useless examples.
So this one must be discussed during the 2.3 cycle and will see if we keep it, 
shrink it, revert it or move to a library.

> Since it is an example and well documented that is fine.
> Is there an explicit statement that ABI is not binding for examples?

>What do you mean by "binding"?

I think these are fair comments.

I do take the point about maintenance overhead, there is no value in dragging 
along
something that is not being used. After making it available and allowing time 
for
users to explore we can gauge the answer best.

Feedback should inform the direction, and whilst we do continue pathfinding 
activity internally, 
in fact we had not planned any updates during the 2.3 cycle, precisely to allow 
time
to understand if/how the community would like to see it evolve.

So personally I see 2.4 as an appropriate target to implement whatever we 
decide.

[dpdk-dev] dev Digest, Vol 69, Issue 101

[Betts, Ian]

Message: 1
Date: Fri, 04 Dec 2015 19:33:47 +0100
From: Thomas Monjalon 
To: Stephen Hemminger 
Cc: dev at dpdk.org, Ian Betts 
Subject: Re: [dpdk-dev] [PATCH v8 0/4] examples: add
performance-thread
Message-ID: <1850494.rdblc2vbpm at xps13>
Content-Type: text/plain; charset="us-ascii"

> Yes, sure it is too late to have enough discussions in 2.2 timeframe.
Please see my response to Stephens questions/concerns


--

Message: 2
Date: Fri, 4 Dec 2015 19:36:09 +0100
From: Andriy Berestovskyy 
To: Stephen Hemminger 
Cc: dev at dpdk.org, Eric Kinzie 
Subject: Re: [dpdk-dev] [PATCH 6/8] bond: handle slaves with fewer
queues than bonding device
Message-ID:

[dpdk-dev] [PATCH v5 2/4] examples: add lthread subsystem for performance-thread

[Betts, Ian]

On Thu, Dec 03, 2015 at 08:31:39AM -0800, Stephen Hemminger wrote:
> On Thu,  3 Dec 2015 09:28:23 +
> ibetts  wrote:
> 
> > +/*
> > + * Atomically set a value and return the old value  */ static 
> > +inline uint64_t atomic64_xchg(uint64_t *ptr, uint64_t val) 
> > +__attribute__ ((always_inline)); static inline uint64_t 
> > +atomic64_xchg(uint64_t *ptr, uint64_t val)
> 
> You don't need a forward declaration for this.
> Instead do:
> 
> static inline uint64_t __attribute__((always_inline)) 
> atomic_xchg64(uint64_t *ptr, uint64_t val)
> 
> Really should be in rte_atomic.h as a primitive and the assembly macro 
> is missing change to ptr so Gcc might optmize it away.
> 
> Something like this mayb?
> 
> static inline uint64_t __attribute__ ((always_inline)); 
> rte_atomic64_xchg(uint64_t *ptr, uint64_t val) {
>   asm volatile (
>   MPLOCKED
>   "xchgq %[ptr],%[val];"
>   : [val] "=r" (val)
>   [ptr] "=m" (*ptr)
>   : [ptr] "m" (*ptr),
> "a" (val)
>   : "memory");
> 
>   return val;
> }

>-Original Message-
>From: Richardson, Bruce 
>Sent: Thursday, December 3, 2015 4:46 PM
>To: Stephen Hemminger
>Cc: Betts, Ian; dev at dpdk.org
>Subject: Re: [dpdk-dev] [PATCH v5 2/4] examples: add lthread subsystem for 
>performance-thread

>Rather than using assembly, I believe the gcc builtin __sync_lock_test_and_set 
>is actually an xchg op, so can be used here.

Thanks for the suggestion Bruce, I had looked for a builin I must have missed 
it.
I just tested it and its fine.



/Bruce

[dpdk-dev] [PATCH v3 2/2] examples: add pthread-shim in performance-thread sample app

[Betts, Ian]

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of ibetts
> Sent: Tuesday, November 17, 2015 12:11 PM
> To: dev at dpdk.org
> Cc: Betts, Ian
> Subject: [dpdk-dev] [PATCH v3 2/2] examples: add pthread-shim in 
> performance-thread sample app

Hi Ian,

For some reason it looks like [PATCH v3 1/2] didn't make it to the list.

John.
-- 


It is blocked due to exceeding 300kb limit ( it is actually 400kb )
I requested admin to unblock
--
Ian

Ian Betts
Intel Shannon Limited
Registered in Ireland
Registered Office Collinstown Industrial Park, Leixlip, County Kildare
Registered Number:308263
Business address: Dromore House, East Park, Shannon, Co. Clare

[dpdk-dev] [ RFC ] performance thread example

[Betts, Ian]

Hi Folks,

Would welcome feedback on this proposal for an example app to explore 
performance
with different threading models.

Thanks,
Ian

Performance thread example application

This example will comprise a simple layer 3 forwarding application distributed 
across multiple threads.
It will be possible to configure the application for, and contrast forwarding 
performance of different threading models,
More specifically when the different threads are:-


1.   EAL threads running on different physical cores

2.   EAL threads running on the same physical core ( multiple lcore per 
physical core)

3.   Lightweight threads running in an EAL thread on one more physical 
cores.


Purpose and justification

Since dpdk 2.0 it has been possible to assign multiple EAL threads to a 
physical core ( case 2 above ).
Currently no example application has focused on demonstrating the performance 
constraints of differing threading models.

Whilst purpose built applications that fully comprehend the DPDK single 
threaded programming model will always yield superior performance,
the desire to preserve ROI in legacy code written for multithreaded operating 
environments  makes lightweight threads ( case 3 above )
an option worthy of consideration.

As well as aiding with legacy code reuse, it is anticipated that lightweight 
threads will make it possible to scale a multithreaded
application with fine granularity allowing an application  to more easily take 
advantage of headroom on EAL cores,
or conversely occupy more cores, as dictated by system load.

To explore performance with lightweight threads a simple cooperative scheduler 
subsystem is being included in this example application.
If the expected benefits and use cases prove to be of value, it is anticipated 
that this lightweight thread subsystem would become a library
in some future DPDK release.

Proposed High-Level Solution(s)
A layer 3 forwarding application will comprise a number of threads from rx 
through  tx stages, and will accept command line parameters enabling
the assignment of queues to threads, threads to lcores, and lcores to physical 
cores.

In the case that the threads are to be lightweight threads then a lightweight 
thread scheduler will be run as the main loop of one or more EAL threads.
The lightweight thread subsystem will have an API comparable in purpose to a 
subset of the equivalent POSIX P-thread functions.

Main APIs

* Thread control APIs

o   Lthread_create

o   Lthread_cancel

o   Lthread_join

o   Lthread_detach

o   Lthread_exit

o   Lthread_sleep

o   Lthread_yield

o   Lthread_set_affinity

o   Lthread_current



* Thread local storage APIs

o   Lthread_set_data

o   Lthread_get_data

o   Lthread_key_create

o   Lthread_key_delete

o   Lthread_get_specific

o   Lthread_set_specific

o   PER_LTHREAD macros



* Sychronization APIs

o   Lthread_mutex_init

o   Lthread_mutex_destroy

o   Lthread_mutex_lock

o   Lthread_mutex_trylock

o   Lthread_mutex_unlock

o   Lthread_cond_init

o   Lthread_cond_destroy

o   Lthread_cond_wait

o   Lthread_cond_signal

o   Lthread_cond_broadcast



* Scheduler control APIs

o   Lthread_num_schedulers_set

o   Lthread_active_schedulers

o   Lthread_run


Proposed example format
The L-thread packet forwarding application, (and test code) , will be example 
applications under RTE_SDK/examples/. The L-thread subsystem will be a common 
source folder included by these applications.
RTE_SDK/examples/lthread
RTE_SDK/examples/lthread/common/
RTE_SDK/examples/lthread/l3fwd-lthread/
RTE_SDK/examples/lthread/test/



Ian Betts
Intel Shannon Limited
Registered in Ireland
Registered Office Collinstown Industrial Park, Leixlip, County Kildare
Registered Number:308263
Business address: Dromore House, East Park, Shannon, Co. Clare

--
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole 
use of the intended recipient(s). Any review or distribution by others is 
strictly prohibited. If you are not the intended recipient, please contact the 
sender and delete all copies.