[dpdk-dev] Beyond DPDK 2.0

[Wiles, Keith]

Hi Luke,

On 5/7/15, 10:29 PM, "Luke Gorrie"  wrote:

>On 8 May 2015 at 06:16, Wiles, Keith  wrote:
>
>The PMDs or drivers would not be useful without DPDK MBUFS IMO
>
>
>
>
>
>Surprisingly perhaps, I would find them very useful.
>
>
>To me there are two parts to a driver: the hardware setup and the
>transmit/receive.
>
>
>The hardware setup is complex and generic. You have to read a
>thousand-page data sheet and then write code to initialize the hardware,
>setup queues, enable promisc/multicast, enable features you want like
>vmdq or flow director, and so on. You need to accumulate
> workarounds for hard-to-test problems like cards being discovered with
>unsuitable values in their EEPROM. There is not much intellectual value
>in this code being written more than once.
>
>
>I would like to see this hardware setup code shared between many
>projects. That code does not depend on a specific mbuf struct. Sharing
>could be done with an embeddable PMD library, with a bifurcated driver in
>the kernel, with the SR-IOV PF/VF model, or
> surely other ways too. These all have limited applicability today.
>
>
>The transmit/receive part, on the other hand, seems very
>application-dependent. This part depends on the specific mbuf struct and
>the way you are developing your application around it. You will need to
>write code to suit your design for using scatter/gather,
> allowed sizes of individual buffers, the granularity at which you are
>keeping track of checksum validity, how you use TSO/LRO, how you use
>interrupts, how you batch work together, and so on. This is easy or hard
>depending on how simple or complex the application
> is.
>
>
>I am not so interested in sharing this code. I think that different
>applications will legitimately have different designs - including mbuf
>structs - and they all need code that suits their own design. I think
>there is a lot of value in people being creative
> in these areas and trying different things.
>
>
>So while Avi might only mean that he wants to allocate the bytes for his
>mbufs himself, on our side we want to design our own mbuf struct. The
>cost of that today is to write our own device drivers from scratch but
>for now that seems justified. Going forward
> if there were a simpler mechanism that reduced our workload and gave us
>access to more hardware - libixgbe, libi40e, etc - that would be
>extremely interesting to us.

I think I see your point about hardware setup and handling packets from
the rings as it would be nice to allow others to utilize those parts of
the code. The drivers (I believe) are mostly from FreeBSD and changed to
be our PMDs, which to me they are fairly generic in some cases. I will
have a look at the drivers when I get back home. In the past I have
written drivers using the your suggestion around we have a upper and lower
layer the lower layer is all hardware specific and the upper layer is all
around the network stack interface. My point is we should be able to split
the two and possible provide you the lower layer APIs in a cleaner way.

>
>
>
>I suppose that another background question is whether the DPDK community
>are chiefly concerned with advancing DPDK as a platform and a brand or
>are broadly keen to develop and share code that is useful in diverse
>networking projects. (Is this whole discussion
> off-topic for dpdk-devel?)

I would suggested you are correct DPDK as platform is more how it started
and is going, but it does not mean we can not move is a slightly different
direction to help other access the parts which are more generic.

Regards,
++Keith
>
>
>This is one of the many reasons why I would love to use parts of DPDK but
>do not want to use all of it. (We also allocate our HugeTLBs differently,
>etc, because we have different priorities.)
>
>
>
>
>
>
>

[dpdk-dev] Beyond DPDK 2.0

[Hobywan Kenoby]

> Sounds like you want something like libc, but DPDK is a system like a user
> space OS more then it is a collection of functions that are independent
> like strlen, strcpy, memcpy, printf or ... Some parts of DPDK are
> independent and can be used as you suggest, but the real performance
> sections are tied together.
> 
> >> Regards,
> >> ++Keith

This is indeed quite a statement. DPDK is not just a
bunch of NIC drivers, but "a user space OS" (DPDK 1.0 had a baremetal
boot: why did it disappeared?).



Why Linux or Windows do not integrate DPDK concepts to
catch up performance wise? Is it something so deep like the "Big
Kernel Lock" that took so many years to get rid of?



My assumption is that all current kernels have been
built with one implicit hypothesis: the memory is much faster than cpu. This is
the opposite today. DPDK internal structure has been adapted to the new
paradigm where the TLBs, the memory bandwidth are the scarce resources to 
manage. So I
guess Linux and Windows will not be able to integrate DPDK concepts for
performance anytime soon, if ever...



Reading the list carefully, I expect disk block PMDs
(and block framework?) to come next.



Beyond DPDK 2.0: is it time to accept the fact that
DPDK community is actually paving the way to the next generation lightweight,
high performance, para-virtualized OS? Is it a DPDK task? Another project ?
Should we rename DPDK to PVDK?



- HK

[dpdk-dev] Beyond DPDK 2.0

[Luke Gorrie]

On 8 May 2015 at 11:42, Bruce Richardson  wrote:

> The code in those directories is "common" code that is maintained by Intel
> -
> which is why you see repeated comments about not modifying it for DPDK. It
> is
> just contained in it's own subfolder in each DPDK driver for easier
> updating
> off the internal Intel baseline.
>

Thanks for pointing this out to me, Bruce. Food for thought.

Cheers,
-Luke

[dpdk-dev] Beyond DPDK 2.0

[Luke Gorrie]

Hi Bruce,

On 8 May 2015 at 11:06, Bruce Richardson  wrote:

> For the Intel NIC drivers, the hardware setup part used in DPDK is based
> off
> the other Intel drivers for other OS's. The code you are interested in
> should
> therefore be contained within the subfolders off each individual PMD. As
> you point
> out below, the mbuf specific part is only present in the files in the
> top-level
> PMD folder with the DPDK-specific RX/TX and queue setup routines.


Interesting!

How could one embed these Intel drivers (igb, ixgbe, i40e, ...) into new
programs?

If there is documentation, a platform-agnostic master repository, etc, that
would be really interesting.

I have the impression as an outsider that the various incarnations of these
drivers (Linux, FreeBSD, DPDK) are loosely synchronized forks maintained at
considerable effort by each project. If there is actually a common core
that is easy to adopt, I am interested!

(If dpdk-devel is the wrong mailing list for this discussion then perhaps
you could reply with Cc: to a more suitable one and I will subscribe there.)

Cheers,
-Luke

[dpdk-dev] Beyond DPDK 2.0

[Bruce Richardson]

On Fri, May 08, 2015 at 11:32:04AM +0200, Luke Gorrie wrote:
> Hi Bruce,
> 
> On 8 May 2015 at 11:06, Bruce Richardson  
> wrote:
> 
> > For the Intel NIC drivers, the hardware setup part used in DPDK is based
> > off
> > the other Intel drivers for other OS's. The code you are interested in
> > should
> > therefore be contained within the subfolders off each individual PMD. As
> > you point
> > out below, the mbuf specific part is only present in the files in the
> > top-level
> > PMD folder with the DPDK-specific RX/TX and queue setup routines.
> 
> 
> Interesting!
> 
> How could one embed these Intel drivers (igb, ixgbe, i40e, ...) into new
> programs?
> 
> If there is documentation, a platform-agnostic master repository, etc, that
> would be really interesting.
> 
> I have the impression as an outsider that the various incarnations of these
> drivers (Linux, FreeBSD, DPDK) are loosely synchronized forks maintained at
> considerable effort by each project. If there is actually a common core
> that is easy to adopt, I am interested!
> 
> (If dpdk-devel is the wrong mailing list for this discussion then perhaps
> you could reply with Cc: to a more suitable one and I will subscribe there.)
> 
> Cheers,
> -Luke

The code in those directories is "common" code that is maintained by Intel -
which is why you see repeated comments about not modifying it for DPDK. It is
just contained in it's own subfolder in each DPDK driver for easier updating 
off the internal Intel baseline.

/Bruce

[dpdk-dev] Beyond DPDK 2.0

[Bruce Richardson]

On Fri, May 08, 2015 at 07:29:39AM +0200, Luke Gorrie wrote:
> On 8 May 2015 at 06:16, Wiles, Keith  wrote:
> 
> > The PMDs or drivers would not be useful without DPDK MBUFS IMO
> >
> 
> Surprisingly perhaps, I would find them very useful.
> 
> To me there are two parts to a driver: the hardware setup and the
> transmit/receive.
> 
> The hardware setup is complex and generic. You have to read a thousand-page
> data sheet and then write code to initialize the hardware, setup queues,
> enable promisc/multicast, enable features you want like vmdq or flow
> director, and so on. You need to accumulate workarounds for hard-to-test
> problems like cards being discovered with unsuitable values in their
> EEPROM. There is not much intellectual value in this code being written
> more than once.

For the Intel NIC drivers, the hardware setup part used in DPDK is based off
the other Intel drivers for other OS's. The code you are interested in should
therefore be contained within the subfolders off each individual PMD. As you 
point
out below, the mbuf specific part is only present in the files in the top-level
PMD folder with the DPDK-specific RX/TX and queue setup routines.

Regards,
/Bruce

> 
> I would like to see this hardware setup code shared between many projects.
> That code does not depend on a specific mbuf struct. Sharing could be done
> with an embeddable PMD library, with a bifurcated driver in the kernel,
> with the SR-IOV PF/VF model, or surely other ways too. These all have
> limited applicability today.
> 
> The transmit/receive part, on the other hand, seems very
> application-dependent. This part depends on the specific mbuf struct and
> the way you are developing your application around it. You will need to
> write code to suit your design for using scatter/gather, allowed sizes of
> individual buffers, the granularity at which you are keeping track of
> checksum validity, how you use TSO/LRO, how you use interrupts, how you
> batch work together, and so on. This is easy or hard depending on how
> simple or complex the application is.
> 
> I am not so interested in sharing this code. I think that different
> applications will legitimately have different designs - including mbuf
> structs - and they all need code that suits their own design. I think there
> is a lot of value in people being creative in these areas and trying
> different things.
> 
> So while Avi might only mean that he wants to allocate the bytes for his
> mbufs himself, on our side we want to design our own mbuf struct. The cost
> of that today is to write our own device drivers from scratch but for now
> that seems justified. Going forward if there were a simpler mechanism that
> reduced our workload and gave us access to more hardware - libixgbe,
> libi40e, etc - that would be extremely interesting to us.
> 
> I suppose that another background question is whether the DPDK community
> are chiefly concerned with advancing DPDK as a platform and a brand or are
> broadly keen to develop and share code that is useful in diverse networking
> projects. (Is this whole discussion off-topic for dpdk-devel?)
> 
> This is one of the many reasons why I would love to use parts of DPDK but
> do not want to use all of it. (We also allocate our HugeTLBs differently,
> etc, because we have different priorities.)

[dpdk-dev] Beyond DPDK 2.0

[Neil Horman]

On Fri, May 08, 2015 at 12:26:39PM +0200, Hobywan Kenoby wrote:
> 
> > Sounds like you want something like libc, but DPDK is a system like a user
> > space OS more then it is a collection of functions that are independent
> > like strlen, strcpy, memcpy, printf or ... Some parts of DPDK are
> > independent and can be used as you suggest, but the real performance
> > sections are tied together.
> > 
> > >> Regards,
> > >> ++Keith
> 
> This is indeed quite a statement. DPDK is not just a
> bunch of NIC drivers, but "a user space OS" (DPDK 1.0 had a baremetal
> boot: why did it disappeared?).
> 
>  
> 
> Why Linux or Windows do not integrate DPDK concepts to
> catch up performance wise? Is it something so deep like the "Big
> Kernel Lock" that took so many years to get rid of?
> 
Some optimizations are being looked at in the kernel (more deeply ingrained use
of accelerators/offloads like cam management/flow steering, checksum & encap
offloads, tx batching, etc)  Those are features which the kernel can
opportunistically take advantage of in a hardware agnostic fashion.

Some optimizations simply aren't worth the effort to take into a general purpose
OS that seeks to support multiple arches.  Many of the DPDK optimizations
utilize instruction families like AVX or SSE, which, while potentially useful in
some situations have equal potential to be catastrophic to non network-i/o bound
workloads. 
>  
> 
> My assumption is that all current kernels have been
> built with one implicit hypothesis: the memory is much faster than cpu. This 
> is
Thats not entirely true.  Or more to the point, its not true in any way thats
relevant to a comparison between DPDK and the Linux network stack.  Linux is as
careful with its cache management as DPDK is (arguably more so, as it has to
juggle multiple workloads instead of the single purpose workload that DPDK is
designed for).  The difference is that Linux often has to ignore some
performance improvements because it has the additional responsibiilty of
providing secruity and isolation to multiple processes on multiple
architectures.

> the opposite today. DPDK internal structure has been adapted to the new
> paradigm where the TLBs, the memory bandwidth are the scarce resources to 
> manage. So I
> guess Linux and Windows will not be able to integrate DPDK concepts for
> performance anytime soon, if ever...
> 
This is the case with every bit of software.  Memory bandwidth is always a
scarce resoruce to manage.  The difference is that general purpose operating
systems consider protection/layering/isolation to be of equal or greater
importance than performance.  Tradeoffs have to be made.  Linux in general
strives to isolate hardware from applications both functionally and physically
so as to ensure that there is minimal risk in one process adversely affecting
the other.  The tradeoff is that the Linux device model can't just do anything
it wants to improve performance.

Converserly, DPDK is all about performance.  Up until recently (and likely still
somewhat in the future), you have to rebuild your application every time you
move to a new version of DPDK, because the API fluctuated with every release to
eek out additional performance.  The DPDK can optimize using vectorized x86
instructions and other cpu specific features througout because it is in the
position to only worry about a very narrow field of architectures.

>  
> 
> Reading the list carefully, I expect disk block PMDs
> (and block framework?) to come next.
> 
>  
> 
> Beyond DPDK 2.0: is it time to accept the fact that
> DPDK community is actually paving the way to the next generation lightweight,
> high performance, para-virtualized OS? Is it a DPDK task? Another project ?
> Should we rename DPDK to PVDK?
> 
>  
> 
> - HK
> 
> 

[dpdk-dev] Beyond DPDK 2.0

[Stephen Hemminger]

On Fri, 8 May 2015 09:31:34 -0400
Neil Horman  wrote:

> On Fri, May 08, 2015 at 12:26:39PM +0200, Hobywan Kenoby wrote:
> > 
> > > Sounds like you want something like libc, but DPDK is a system like a user
> > > space OS more then it is a collection of functions that are independent
> > > like strlen, strcpy, memcpy, printf or ... Some parts of DPDK are
> > > independent and can be used as you suggest, but the real performance
> > > sections are tied together.
> > > 
> > > >> Regards,
> > > >> ++Keith
> > 
> > This is indeed quite a statement. DPDK is not just a
> > bunch of NIC drivers, but "a user space OS" (DPDK 1.0 had a baremetal
> > boot: why did it disappeared?).
> > 
> >  
> > 
> > Why Linux or Windows do not integrate DPDK concepts to
> > catch up performance wise? Is it something so deep like the "Big
> > Kernel Lock" that took so many years to get rid of?
> > 
> Some optimizations are being looked at in the kernel (more deeply ingrained 
> use
> of accelerators/offloads like cam management/flow steering, checksum & encap
> offloads, tx batching, etc)  Those are features which the kernel can
> opportunistically take advantage of in a hardware agnostic fashion.
> 
> Some optimizations simply aren't worth the effort to take into a general 
> purpose
> OS that seeks to support multiple arches.  Many of the DPDK optimizations
> utilize instruction families like AVX or SSE, which, while potentially useful 
> in
> some situations have equal potential to be catastrophic to non network-i/o 
> bound
> workloads. 
> >  
> > 
> > My assumption is that all current kernels have been
> > built with one implicit hypothesis: the memory is much faster than cpu. 
> > This is
> Thats not entirely true.  Or more to the point, its not true in any way thats
> relevant to a comparison between DPDK and the Linux network stack.  Linux is 
> as
> careful with its cache management as DPDK is (arguably more so, as it has to
> juggle multiple workloads instead of the single purpose workload that DPDK is
> designed for).  The difference is that Linux often has to ignore some
> performance improvements because it has the additional responsibiilty of
> providing secruity and isolation to multiple processes on multiple
> architectures.
> 
> > the opposite today. DPDK internal structure has been adapted to the new
> > paradigm where the TLBs, the memory bandwidth are the scarce resources to 
> > manage. So I
> > guess Linux and Windows will not be able to integrate DPDK concepts for
> > performance anytime soon, if ever...
> > 
> This is the case with every bit of software.  Memory bandwidth is always a
> scarce resoruce to manage.  The difference is that general purpose operating
> systems consider protection/layering/isolation to be of equal or greater
> importance than performance.  Tradeoffs have to be made.  Linux in general
> strives to isolate hardware from applications both functionally and physically
> so as to ensure that there is minimal risk in one process adversely affecting
> the other.  The tradeoff is that the Linux device model can't just do anything
> it wants to improve performance.
> 
> Converserly, DPDK is all about performance.  Up until recently (and likely 
> still
> somewhat in the future), you have to rebuild your application every time you
> move to a new version of DPDK, because the API fluctuated with every release 
> to
> eek out additional performance.  The DPDK can optimize using vectorized x86
> instructions and other cpu specific features througout because it is in the
> position to only worry about a very narrow field of architectures.
> 
> >  
> > 
> > Reading the list carefully, I expect disk block PMDs
> > (and block framework?) to come next.
> > 
> >  
> > 
> > Beyond DPDK 2.0: is it time to accept the fact that
> > DPDK community is actually paving the way to the next generation 
> > lightweight,
> > high performance, para-virtualized OS? Is it a DPDK task? Another project ?
> > Should we rename DPDK to PVDK?

The difference is DPDK doesn't care about being general purpose:
  - scheduler, that is the applications problem
  - locking, the application must be bound to cpus or do its own locking
  - buffer management, up to the application.
  - memory protection (haha)

Any operating system provides an abstraction that makes programming easier.
If you strip away those abstractions, then sure things go faster but it is
less safe and harder.

Linux is about providing safe abstraction. If you want an OS that doesn't
do that, look to Cloudius or the other DIY environments like DPDK.

This is not a new concept. Oracle and other DBMS vendors have been asking
for the OS to get out of the way for years, but then customers find that
things like filesystems are convenient necessities.

[dpdk-dev] Beyond DPDK 2.0

[Stephen Hemminger]

On Fri, 8 May 2015 14:44:17 +
"Wiles, Keith"  wrote:

> Hi Luke,
> 
> On 5/7/15, 10:29 PM, "Luke Gorrie"  wrote:
> 
> >On 8 May 2015 at 06:16, Wiles, Keith  wrote:
> >
> >The PMDs or drivers would not be useful without DPDK MBUFS IMO
> >
> >
> >
> >
> >
> >Surprisingly perhaps, I would find them very useful.
> >
> >
> >To me there are two parts to a driver: the hardware setup and the
> >transmit/receive.
> >
> >
> >The hardware setup is complex and generic. You have to read a
> >thousand-page data sheet and then write code to initialize the hardware,
> >setup queues, enable promisc/multicast, enable features you want like
> >vmdq or flow director, and so on. You need to accumulate
> > workarounds for hard-to-test problems like cards being discovered with
> >unsuitable values in their EEPROM. There is not much intellectual value
> >in this code being written more than once.
> >
> >
> >I would like to see this hardware setup code shared between many
> >projects. That code does not depend on a specific mbuf struct. Sharing
> >could be done with an embeddable PMD library, with a bifurcated driver in
> >the kernel, with the SR-IOV PF/VF model, or
> > surely other ways too. These all have limited applicability today.
> >
> >
> >The transmit/receive part, on the other hand, seems very
> >application-dependent. This part depends on the specific mbuf struct and
> >the way you are developing your application around it. You will need to
> >write code to suit your design for using scatter/gather,
> > allowed sizes of individual buffers, the granularity at which you are
> >keeping track of checksum validity, how you use TSO/LRO, how you use
> >interrupts, how you batch work together, and so on. This is easy or hard
> >depending on how simple or complex the application
> > is.
> >
> >
> >I am not so interested in sharing this code. I think that different
> >applications will legitimately have different designs - including mbuf
> >structs - and they all need code that suits their own design. I think
> >there is a lot of value in people being creative
> > in these areas and trying different things.
> >
> >
> >So while Avi might only mean that he wants to allocate the bytes for his
> >mbufs himself, on our side we want to design our own mbuf struct. The
> >cost of that today is to write our own device drivers from scratch but
> >for now that seems justified. Going forward
> > if there were a simpler mechanism that reduced our workload and gave us
> >access to more hardware - libixgbe, libi40e, etc - that would be
> >extremely interesting to us.  
> 
> I think I see your point about hardware setup and handling packets from
> the rings as it would be nice to allow others to utilize those parts of
> the code. The drivers (I believe) are mostly from FreeBSD and changed to
> be our PMDs, which to me they are fairly generic in some cases. I will
> have a look at the drivers when I get back home. In the past I have
> written drivers using the your suggestion around we have a upper and lower
> layer the lower layer is all hardware specific and the upper layer is all
> around the network stack interface. My point is we should be able to split
> the two and possible provide you the lower layer APIs in a cleaner way.

The point is this is BSD code, you can do with it what you will.
But the DPDK community doesn't have to care about changes breaking your
proprietary application.

That is the problem with the whole concept of making DPDK drivers
a separate component. It makes them immutable and unmaintainable.
Developers don't want to be responsible for code that is used outside
its original scope.

[dpdk-dev] Beyond DPDK 2.0

[Luke Gorrie]

On 8 May 2015 at 06:16, Wiles, Keith  wrote:

> The PMDs or drivers would not be useful without DPDK MBUFS IMO
>

Surprisingly perhaps, I would find them very useful.

To me there are two parts to a driver: the hardware setup and the
transmit/receive.

The hardware setup is complex and generic. You have to read a thousand-page
data sheet and then write code to initialize the hardware, setup queues,
enable promisc/multicast, enable features you want like vmdq or flow
director, and so on. You need to accumulate workarounds for hard-to-test
problems like cards being discovered with unsuitable values in their
EEPROM. There is not much intellectual value in this code being written
more than once.

I would like to see this hardware setup code shared between many projects.
That code does not depend on a specific mbuf struct. Sharing could be done
with an embeddable PMD library, with a bifurcated driver in the kernel,
with the SR-IOV PF/VF model, or surely other ways too. These all have
limited applicability today.

The transmit/receive part, on the other hand, seems very
application-dependent. This part depends on the specific mbuf struct and
the way you are developing your application around it. You will need to
write code to suit your design for using scatter/gather, allowed sizes of
individual buffers, the granularity at which you are keeping track of
checksum validity, how you use TSO/LRO, how you use interrupts, how you
batch work together, and so on. This is easy or hard depending on how
simple or complex the application is.

I am not so interested in sharing this code. I think that different
applications will legitimately have different designs - including mbuf
structs - and they all need code that suits their own design. I think there
is a lot of value in people being creative in these areas and trying
different things.

So while Avi might only mean that he wants to allocate the bytes for his
mbufs himself, on our side we want to design our own mbuf struct. The cost
of that today is to write our own device drivers from scratch but for now
that seems justified. Going forward if there were a simpler mechanism that
reduced our workload and gave us access to more hardware - libixgbe,
libi40e, etc - that would be extremely interesting to us.

I suppose that another background question is whether the DPDK community
are chiefly concerned with advancing DPDK as a platform and a brand or are
broadly keen to develop and share code that is useful in diverse networking
projects. (Is this whole discussion off-topic for dpdk-devel?)

This is one of the many reasons why I would love to use parts of DPDK but
do not want to use all of it. (We also allocate our HugeTLBs differently,
etc, because we have different priorities.)

[dpdk-dev] Beyond DPDK 2.0

[Wiles, Keith]

Hi Luke

On 5/7/15, 8:34 AM, "Luke Gorrie"  wrote:

>On 7 May 2015 at 16:02, Avi Kivity  wrote:
>
>> One problem we've seen with dpdk is that it is a framework, not a
>>library:
>> it wants to create threads, manage memory, and generally take over.
>>This
>> is a problem for us, as we are writing a framework (seastar, [1]) and
>>need
>> to create threads, manage memory, and generally take over ourselves.
>>
>
>That is also broadly why we don't currently use DPDK in Snabb Switch [1].
>
>There is a bunch of functionality in DPDK that would be tempting for us to
>use and contribute back to: device drivers, SIMD routines, data
>structures,
>and so on. I think that we would do this if they were available piecemeal
>as stand-alone libi40e, libsimd, liblpn, etc.
>
>The whole DPDK platform/framework is too much for us to adopt though. Some
>aspects of it are in conflict with our goals and it is an all-or-nothing
>proposition. So for now we are staying self-sufficient even when it means
>writing our own ixgbe replacement, etc.
>
>Having said that we are able to share code that doesn't require linking
>into our address space e.g. vhost-user and potentially the bifurcated
>drivers in the future. That seems like a nice direction for things to be
>going in and a way to collaborate even without our directly linking with
>DPDK.

Would the shared library support in DPDK be useful here? I know it still
links in a dynamic way.

I believe DPDK is much like your snabbswitch as it provides a basic system
to run networking applications, in your case a vSwitch like design. The
design has some parts that are standalone, but to be effective they
require other parts of DPDK to work correctly. If you have some suggestion
as to how DPDK could be split up and maintain its features and performance
I would like to understand how.

Regards,
++Keith

>
>[1] https://github.com/lukego/snabbswitch/blob/README/README.md

[dpdk-dev] Beyond DPDK 2.0

[Wiles, Keith]

On 5/7/15, 9:05 AM, "Avi Kivity"  wrote:

>On 05/07/2015 06:49 PM, Wiles, Keith wrote:
>>
>> On 5/7/15, 8:33 AM, "Avi Kivity"  wrote:
>>
>>> On 05/07/2015 06:27 PM, Wiles, Keith wrote:
 On 5/7/15, 7:02 AM, "Avi Kivity"  wrote:

> On Wed, Apr 22, 2015 at 6:11 PM, O'Driscoll, Tim
> 
> wrote:
>
>> Does anybody have any input or comments on this?
>>
>>
>>> -Original Message-
>>> From: O'Driscoll, Tim
>>> Sent: Thursday, April 16, 2015 11:39 AM
>>> To: dev at dpdk.org
>>> Subject: Beyond DPDK 2.0
>>>
>>> Following the launch of DPDK by Intel as an internal development
>>> project, the launch of dpdk.org by 6WIND in 2013, and the first
>>>DPDK
>> RPM
>>> packages for Fedora in 2014, 6WIND, Red Hat and Intel would like to
>>> prepare for future releases after DPDK 2.0 by starting a discussion
>>> on
>>> its evolution. Anyone is welcome to join this initiative.
>>>
>>> Since then, the project has grown significantly:
>>> -The number of commits and mailing list posts has increased
>>> steadily.
>>> -Support has been added for a wide range of new NICs (Mellanox
>>> support submitted by 6WIND, Cisco VIC, Intel i40e and fm10k etc.).
>>> -DPDK is now supported on multiple architectures (IBM Power
>> support
>>> in DPDK 1.8, Tile support submitted by EZchip but not yet reviewed
>>>or
>>> applied).
>>>
>>> While this is great progress, we need to make sure that the project
>>> is
>>> structured in a way that enables it to continue to grow. To achieve
>>> this, 6WIND, Red Hat and Intel would like to start a discussion
>>>about
>>> the future of the project, so that we can agree and establish
>> processes
>>> that satisfy the needs of the current and future DPDK community.
>>>
>>> We're very interested in hearing the views of everybody in the
>>> community. In addition to debate on the mailing list, we'll also
>>> schedule community calls to discuss this.
>>>
>>>
>>> Project Goals
>>> -
>>>
>>> Some topics to be considered for the DPDK project include:
>>> -Project Charter: The charter of the DPDK project should be
>> clearly
>>> defined, and should explain the limits of DPDK (what it does and
>>>does
>>> not cover). This does not mean that we would be stuck with a
>>>singular
>>> charter for all time, but the direction and intent of the project
>> should
>>> be well understood.
> One problem we've seen with dpdk is that it is a framework, not a
> library:
> it wants to create threads, manage memory, and generally take over.
> This
> is a problem for us, as we are writing a framework (seastar, [1]) and
> need
> to create threads, manage memory, and generally take over ourselves.
>
> Perhaps dpdk can be split into two layers, a library layer that only
> provides mechanisms, and a framework layer that glues together those
> mechanisms and applies a policy, trading in generality for ease of
>use.
 The DPDK system is somewhat divided now between the EAL, PMDS and
 utility
 functions like malloc/rings/?

 The problem I see is the PMDs need a framework to be usable and the
EAL
 plus the ethdev layers provide that support today. Setting up and
 initializing the DPDK system is pretty clean just call the EAL init
 routines along with the pool creates and the basic configs for the
 PMDs/hardware. Once the system is inited one can create new threads
and
 not requiring anyone to use DPDK launch routines. Maybe I am not
 understanding your needs can you explain more?
>>> An initialization routine that accepts argc/argv can hardly be called
>>> clean.
>> You want a config file or structure initialization design? If that is
>>the
>> case you can contribute that support as another way to initialize DPDK.
>
>A config file would be even worse.  But we are discussing why
>dpdk-as-a-framework is detrimental, not new ways for me to contribute.

In a way you stated argc/argv was not a clean, I was only suggesting (more
I was asking) what you would like to see?

The contribute part was just an example of how you or anyone can help make
DPDK better. I wanted to understand why argc/argv was not a clan way for
your needs.
>
>>> In seastar, we have our own malloc() (since seastar is sharded we can
>>> provide a faster thread-unsafe malloc implementation).  We also have
>>>our
>>> own threading, and since dpdk is an optional component in seastar, dpdk
>>> support requires code duplication.
>> DPDK replies one the huge page support for allocation to get the
>> performance, do you also not require huge page support.
>
>Sorry, is this a question?  Please rephrase.

Sorry, auto correct got me and trying to answer quickly before a meeting.

DPDK 

[dpdk-dev] Beyond DPDK 2.0

[Avi Kivity]

On 05/07/2015 06:49 PM, Wiles, Keith wrote:
>
> On 5/7/15, 8:33 AM, "Avi Kivity"  wrote:
>
>> On 05/07/2015 06:27 PM, Wiles, Keith wrote:
>>> On 5/7/15, 7:02 AM, "Avi Kivity"  wrote:
>>>
 On Wed, Apr 22, 2015 at 6:11 PM, O'Driscoll, Tim
 
 wrote:

> Does anybody have any input or comments on this?
>
>
>> -Original Message-
>> From: O'Driscoll, Tim
>> Sent: Thursday, April 16, 2015 11:39 AM
>> To: dev at dpdk.org
>> Subject: Beyond DPDK 2.0
>>
>> Following the launch of DPDK by Intel as an internal development
>> project, the launch of dpdk.org by 6WIND in 2013, and the first DPDK
> RPM
>> packages for Fedora in 2014, 6WIND, Red Hat and Intel would like to
>> prepare for future releases after DPDK 2.0 by starting a discussion
>> on
>> its evolution. Anyone is welcome to join this initiative.
>>
>> Since then, the project has grown significantly:
>> -The number of commits and mailing list posts has increased
>> steadily.
>> -Support has been added for a wide range of new NICs (Mellanox
>> support submitted by 6WIND, Cisco VIC, Intel i40e and fm10k etc.).
>> -DPDK is now supported on multiple architectures (IBM Power
> support
>> in DPDK 1.8, Tile support submitted by EZchip but not yet reviewed or
>> applied).
>>
>> While this is great progress, we need to make sure that the project
>> is
>> structured in a way that enables it to continue to grow. To achieve
>> this, 6WIND, Red Hat and Intel would like to start a discussion about
>> the future of the project, so that we can agree and establish
> processes
>> that satisfy the needs of the current and future DPDK community.
>>
>> We're very interested in hearing the views of everybody in the
>> community. In addition to debate on the mailing list, we'll also
>> schedule community calls to discuss this.
>>
>>
>> Project Goals
>> -
>>
>> Some topics to be considered for the DPDK project include:
>> -Project Charter: The charter of the DPDK project should be
> clearly
>> defined, and should explain the limits of DPDK (what it does and does
>> not cover). This does not mean that we would be stuck with a singular
>> charter for all time, but the direction and intent of the project
> should
>> be well understood.
 One problem we've seen with dpdk is that it is a framework, not a
 library:
 it wants to create threads, manage memory, and generally take over.
 This
 is a problem for us, as we are writing a framework (seastar, [1]) and
 need
 to create threads, manage memory, and generally take over ourselves.

 Perhaps dpdk can be split into two layers, a library layer that only
 provides mechanisms, and a framework layer that glues together those
 mechanisms and applies a policy, trading in generality for ease of use.
>>> The DPDK system is somewhat divided now between the EAL, PMDS and
>>> utility
>>> functions like malloc/rings/?
>>>
>>> The problem I see is the PMDs need a framework to be usable and the EAL
>>> plus the ethdev layers provide that support today. Setting up and
>>> initializing the DPDK system is pretty clean just call the EAL init
>>> routines along with the pool creates and the basic configs for the
>>> PMDs/hardware. Once the system is inited one can create new threads and
>>> not requiring anyone to use DPDK launch routines. Maybe I am not
>>> understanding your needs can you explain more?
>> An initialization routine that accepts argc/argv can hardly be called
>> clean.
> You want a config file or structure initialization design? If that is the
> case you can contribute that support as another way to initialize DPDK.

A config file would be even worse.  But we are discussing why 
dpdk-as-a-framework is detrimental, not new ways for me to contribute.

>> In seastar, we have our own malloc() (since seastar is sharded we can
>> provide a faster thread-unsafe malloc implementation).  We also have our
>> own threading, and since dpdk is an optional component in seastar, dpdk
>> support requires code duplication.
> DPDK replies one the huge page support for allocation to get the
> performance, do you also not require huge page support.

Sorry, is this a question?  Please rephrase.

>   The malloc system
> in DPDK can be used as a replacement for the standard malloc if that works
> for your needs. Also after DPDK inits you can use your own malloc and any
> other tools you want to use.

How is memory partitioned between dpdk and my application?  If I 
underallocate dpdk memory, something bad will happen.  If I overallocate 
dpdk memory, then I am depriving my application of this memory.  A 
common pool means I do not overallocate or underallocate, but since dpdk 
insists on managing its own pools, I can't do this.

>   I do not 

[dpdk-dev] Beyond DPDK 2.0

[Avi Kivity]

On 05/07/2015 06:27 PM, Wiles, Keith wrote:
>
> On 5/7/15, 7:02 AM, "Avi Kivity"  wrote:
>
>> On Wed, Apr 22, 2015 at 6:11 PM, O'Driscoll, Tim
>> 
>> wrote:
>>
>>> Does anybody have any input or comments on this?
>>>
>>>
 -Original Message-
 From: O'Driscoll, Tim
 Sent: Thursday, April 16, 2015 11:39 AM
 To: dev at dpdk.org
 Subject: Beyond DPDK 2.0

 Following the launch of DPDK by Intel as an internal development
 project, the launch of dpdk.org by 6WIND in 2013, and the first DPDK
>>> RPM
 packages for Fedora in 2014, 6WIND, Red Hat and Intel would like to
 prepare for future releases after DPDK 2.0 by starting a discussion on
 its evolution. Anyone is welcome to join this initiative.

 Since then, the project has grown significantly:
 -The number of commits and mailing list posts has increased
 steadily.
 -Support has been added for a wide range of new NICs (Mellanox
 support submitted by 6WIND, Cisco VIC, Intel i40e and fm10k etc.).
 -DPDK is now supported on multiple architectures (IBM Power
>>> support
 in DPDK 1.8, Tile support submitted by EZchip but not yet reviewed or
 applied).

 While this is great progress, we need to make sure that the project is
 structured in a way that enables it to continue to grow. To achieve
 this, 6WIND, Red Hat and Intel would like to start a discussion about
 the future of the project, so that we can agree and establish
>>> processes
 that satisfy the needs of the current and future DPDK community.

 We're very interested in hearing the views of everybody in the
 community. In addition to debate on the mailing list, we'll also
 schedule community calls to discuss this.


 Project Goals
 -

 Some topics to be considered for the DPDK project include:
 -Project Charter: The charter of the DPDK project should be
>>> clearly
 defined, and should explain the limits of DPDK (what it does and does
 not cover). This does not mean that we would be stuck with a singular
 charter for all time, but the direction and intent of the project
>>> should
 be well understood.
>>
>> One problem we've seen with dpdk is that it is a framework, not a library:
>> it wants to create threads, manage memory, and generally take over.  This
>> is a problem for us, as we are writing a framework (seastar, [1]) and need
>> to create threads, manage memory, and generally take over ourselves.
>>
>> Perhaps dpdk can be split into two layers, a library layer that only
>> provides mechanisms, and a framework layer that glues together those
>> mechanisms and applies a policy, trading in generality for ease of use.
> The DPDK system is somewhat divided now between the EAL, PMDS and utility
> functions like malloc/rings/?
>
> The problem I see is the PMDs need a framework to be usable and the EAL
> plus the ethdev layers provide that support today. Setting up and
> initializing the DPDK system is pretty clean just call the EAL init
> routines along with the pool creates and the basic configs for the
> PMDs/hardware. Once the system is inited one can create new threads and
> not requiring anyone to use DPDK launch routines. Maybe I am not
> understanding your needs can you explain more?

An initialization routine that accepts argc/argv can hardly be called clean.

In seastar, we have our own malloc() (since seastar is sharded we can 
provide a faster thread-unsafe malloc implementation).  We also have our 
own threading, and since dpdk is an optional component in seastar, dpdk 
support requires code duplication.

I would like to launch my own threads, pin them where I like, and call 
PMD drivers to send and receive packets.  Practically everything else 
that dpdk does gets in my way, including mbuf pools.  I'd much prefer to 
allocate mbufs myself.


>> [1] http://seastar-project.org

[dpdk-dev] Beyond DPDK 2.0

[Avi Kivity]

On 05/07/2015 06:27 PM, Wiles, Keith wrote:
>
> On 5/7/15, 7:02 AM, "Avi Kivity"  wrote:
>
>> On Wed, Apr 22, 2015 at 6:11 PM, O'Driscoll, Tim
>> 
>> wrote:
>>
>>> Does anybody have any input or comments on this?
>>>
>>>
 -Original Message-
 From: O'Driscoll, Tim
 Sent: Thursday, April 16, 2015 11:39 AM
 To: dev at dpdk.org
 Subject: Beyond DPDK 2.0

 Following the launch of DPDK by Intel as an internal development
 project, the launch of dpdk.org by 6WIND in 2013, and the first DPDK
>>> RPM
 packages for Fedora in 2014, 6WIND, Red Hat and Intel would like to
 prepare for future releases after DPDK 2.0 by starting a discussion on
 its evolution. Anyone is welcome to join this initiative.

 Since then, the project has grown significantly:
 -The number of commits and mailing list posts has increased
 steadily.
 -Support has been added for a wide range of new NICs (Mellanox
 support submitted by 6WIND, Cisco VIC, Intel i40e and fm10k etc.).
 -DPDK is now supported on multiple architectures (IBM Power
>>> support
 in DPDK 1.8, Tile support submitted by EZchip but not yet reviewed or
 applied).

 While this is great progress, we need to make sure that the project is
 structured in a way that enables it to continue to grow. To achieve
 this, 6WIND, Red Hat and Intel would like to start a discussion about
 the future of the project, so that we can agree and establish
>>> processes
 that satisfy the needs of the current and future DPDK community.

 We're very interested in hearing the views of everybody in the
 community. In addition to debate on the mailing list, we'll also
 schedule community calls to discuss this.


 Project Goals
 -

 Some topics to be considered for the DPDK project include:
 -Project Charter: The charter of the DPDK project should be
>>> clearly
 defined, and should explain the limits of DPDK (what it does and does
 not cover). This does not mean that we would be stuck with a singular
 charter for all time, but the direction and intent of the project
>>> should
 be well understood.
>>
>> One problem we've seen with dpdk is that it is a framework, not a library:
>> it wants to create threads, manage memory, and generally take over.  This
>> is a problem for us, as we are writing a framework (seastar, [1]) and need
>> to create threads, manage memory, and generally take over ourselves.
>>
>> Perhaps dpdk can be split into two layers, a library layer that only
>> provides mechanisms, and a framework layer that glues together those
>> mechanisms and applies a policy, trading in generality for ease of use.
> The DPDK system is somewhat divided now between the EAL, PMDS and utility
> functions like malloc/rings/?
>
> The problem I see is the PMDs need a framework to be usable and the EAL
> plus the ethdev layers provide that support today. Setting up and
> initializing the DPDK system is pretty clean just call the EAL init
> routines along with the pool creates and the basic configs for the
> PMDs/hardware. Once the system is inited one can create new threads and
> not requiring anyone to use DPDK launch routines. Maybe I am not
> understanding your needs can you explain more?

An initialization routine that accepts argc/argv can hardly be called clean.

In seastar, we have our own malloc() (since seastar is sharded we can 
provide a faster thread-unsafe malloc implementation).  We also have our 
own threading, and since dpdk is an optional component in seastar, dpdk 
support requires code duplication.

I would like to launch my own threads, pin them where I like, and call 
PMD drivers to send and receive packets.  Practically everything else 
that dpdk does gets in my way, including mbuf pools.  I'd much prefer to 
allocate mbufs myself.


>> [1] http://seastar-project.org

[dpdk-dev] Beyond DPDK 2.0

[Luke Gorrie]

On 7 May 2015 at 16:02, Avi Kivity  wrote:

> One problem we've seen with dpdk is that it is a framework, not a library:
> it wants to create threads, manage memory, and generally take over.  This
> is a problem for us, as we are writing a framework (seastar, [1]) and need
> to create threads, manage memory, and generally take over ourselves.
>

That is also broadly why we don't currently use DPDK in Snabb Switch [1].

There is a bunch of functionality in DPDK that would be tempting for us to
use and contribute back to: device drivers, SIMD routines, data structures,
and so on. I think that we would do this if they were available piecemeal
as stand-alone libi40e, libsimd, liblpn, etc.

The whole DPDK platform/framework is too much for us to adopt though. Some
aspects of it are in conflict with our goals and it is an all-or-nothing
proposition. So for now we are staying self-sufficient even when it means
writing our own ixgbe replacement, etc.

Having said that we are able to share code that doesn't require linking
into our address space e.g. vhost-user and potentially the bifurcated
drivers in the future. That seems like a nice direction for things to be
going in and a way to collaborate even without our directly linking with
DPDK.

[1] https://github.com/lukego/snabbswitch/blob/README/README.md

[dpdk-dev] Beyond DPDK 2.0

[Avi Kivity]

On Wed, Apr 22, 2015 at 6:11 PM, O'Driscoll, Tim 
wrote:

> Does anybody have any input or comments on this?
>
>
> > -Original Message-
> > From: O'Driscoll, Tim
> > Sent: Thursday, April 16, 2015 11:39 AM
> > To: dev at dpdk.org
> > Subject: Beyond DPDK 2.0
> >
> > Following the launch of DPDK by Intel as an internal development
> > project, the launch of dpdk.org by 6WIND in 2013, and the first DPDK RPM
> > packages for Fedora in 2014, 6WIND, Red Hat and Intel would like to
> > prepare for future releases after DPDK 2.0 by starting a discussion on
> > its evolution. Anyone is welcome to join this initiative.
> >
> > Since then, the project has grown significantly:
> > -The number of commits and mailing list posts has increased
> > steadily.
> > -Support has been added for a wide range of new NICs (Mellanox
> > support submitted by 6WIND, Cisco VIC, Intel i40e and fm10k etc.).
> > -DPDK is now supported on multiple architectures (IBM Power support
> > in DPDK 1.8, Tile support submitted by EZchip but not yet reviewed or
> > applied).
> >
> > While this is great progress, we need to make sure that the project is
> > structured in a way that enables it to continue to grow. To achieve
> > this, 6WIND, Red Hat and Intel would like to start a discussion about
> > the future of the project, so that we can agree and establish processes
> > that satisfy the needs of the current and future DPDK community.
> >
> > We're very interested in hearing the views of everybody in the
> > community. In addition to debate on the mailing list, we'll also
> > schedule community calls to discuss this.
> >
> >
> > Project Goals
> > -
> >
> > Some topics to be considered for the DPDK project include:
> > -Project Charter: The charter of the DPDK project should be clearly
> > defined, and should explain the limits of DPDK (what it does and does
> > not cover). This does not mean that we would be stuck with a singular
> > charter for all time, but the direction and intent of the project should
> > be well understood.
>


One problem we've seen with dpdk is that it is a framework, not a library:
it wants to create threads, manage memory, and generally take over.  This
is a problem for us, as we are writing a framework (seastar, [1]) and need
to create threads, manage memory, and generally take over ourselves.

Perhaps dpdk can be split into two layers, a library layer that only
provides mechanisms, and a framework layer that glues together those
mechanisms and applies a policy, trading in generality for ease of use.

[1] http://seastar-project.org

[dpdk-dev] Beyond DPDK 2.0

[Ivan Boule]

Hi Avi,

On 05/07/2015 04:02 PM, Avi Kivity wrote:
> On Wed, Apr 22, 2015 at 6:11 PM, O'Driscoll, Tim 
> wrote:
>
>> Does anybody have any input or comments on this?
>>
>>
>>> -Original Message-
>>> From: O'Driscoll, Tim
>>> Sent: Thursday, April 16, 2015 11:39 AM
>>> To: dev at dpdk.org
>>> Subject: Beyond DPDK 2.0
>>>
>>> Following the launch of DPDK by Intel as an internal development
>>> project, the launch of dpdk.org by 6WIND in 2013, and the first DPDK RPM
>>> packages for Fedora in 2014, 6WIND, Red Hat and Intel would like to
>>> prepare for future releases after DPDK 2.0 by starting a discussion on
>>> its evolution. Anyone is welcome to join this initiative.
>>>
>>> Since then, the project has grown significantly:
>>> -The number of commits and mailing list posts has increased
>>> steadily.
>>> -Support has been added for a wide range of new NICs (Mellanox
>>> support submitted by 6WIND, Cisco VIC, Intel i40e and fm10k etc.).
>>> -DPDK is now supported on multiple architectures (IBM Power support
>>> in DPDK 1.8, Tile support submitted by EZchip but not yet reviewed or
>>> applied).
>>>
>>> While this is great progress, we need to make sure that the project is
>>> structured in a way that enables it to continue to grow. To achieve
>>> this, 6WIND, Red Hat and Intel would like to start a discussion about
>>> the future of the project, so that we can agree and establish processes
>>> that satisfy the needs of the current and future DPDK community.
>>>
>>> We're very interested in hearing the views of everybody in the
>>> community. In addition to debate on the mailing list, we'll also
>>> schedule community calls to discuss this.
>>>
>>>
>>> Project Goals
>>> -
>>>
>>> Some topics to be considered for the DPDK project include:
>>> -Project Charter: The charter of the DPDK project should be clearly
>>> defined, and should explain the limits of DPDK (what it does and does
>>> not cover). This does not mean that we would be stuck with a singular
>>> charter for all time, but the direction and intent of the project should
>>> be well understood.
>>
>
>
> One problem we've seen with dpdk is that it is a framework, not a library:
> it wants to create threads, manage memory, and generally take over.  This
> is a problem for us, as we are writing a framework (seastar, [1]) and need
> to create threads, manage memory, and generally take over ourselves.
>
> Perhaps dpdk can be split into two layers, a library layer that only
> provides mechanisms, and a framework layer that glues together those
> mechanisms and applies a policy, trading in generality for ease of use.
>
> [1] http://seastar-project.org
>
I fully agree with this analysis/proposal.
And I think that:
- the associated modifications should be done ASAP,
- the underlying design rules that this proposal refers to should drive 
the design and review of new DPDK features.

Regards,
Ivan

[dpdk-dev] Beyond DPDK 2.0

[Wiles, Keith]

On 5/7/15, 8:33 AM, "Avi Kivity"  wrote:

>On 05/07/2015 06:27 PM, Wiles, Keith wrote:
>>
>> On 5/7/15, 7:02 AM, "Avi Kivity"  wrote:
>>
>>> On Wed, Apr 22, 2015 at 6:11 PM, O'Driscoll, Tim
>>> 
>>> wrote:
>>>
 Does anybody have any input or comments on this?


> -Original Message-
> From: O'Driscoll, Tim
> Sent: Thursday, April 16, 2015 11:39 AM
> To: dev at dpdk.org
> Subject: Beyond DPDK 2.0
>
> Following the launch of DPDK by Intel as an internal development
> project, the launch of dpdk.org by 6WIND in 2013, and the first DPDK
 RPM
> packages for Fedora in 2014, 6WIND, Red Hat and Intel would like to
> prepare for future releases after DPDK 2.0 by starting a discussion
>on
> its evolution. Anyone is welcome to join this initiative.
>
> Since then, the project has grown significantly:
> -The number of commits and mailing list posts has increased
> steadily.
> -Support has been added for a wide range of new NICs (Mellanox
> support submitted by 6WIND, Cisco VIC, Intel i40e and fm10k etc.).
> -DPDK is now supported on multiple architectures (IBM Power
 support
> in DPDK 1.8, Tile support submitted by EZchip but not yet reviewed or
> applied).
>
> While this is great progress, we need to make sure that the project
>is
> structured in a way that enables it to continue to grow. To achieve
> this, 6WIND, Red Hat and Intel would like to start a discussion about
> the future of the project, so that we can agree and establish
 processes
> that satisfy the needs of the current and future DPDK community.
>
> We're very interested in hearing the views of everybody in the
> community. In addition to debate on the mailing list, we'll also
> schedule community calls to discuss this.
>
>
> Project Goals
> -
>
> Some topics to be considered for the DPDK project include:
> -Project Charter: The charter of the DPDK project should be
 clearly
> defined, and should explain the limits of DPDK (what it does and does
> not cover). This does not mean that we would be stuck with a singular
> charter for all time, but the direction and intent of the project
 should
> be well understood.
>>>
>>> One problem we've seen with dpdk is that it is a framework, not a
>>>library:
>>> it wants to create threads, manage memory, and generally take over.
>>>This
>>> is a problem for us, as we are writing a framework (seastar, [1]) and
>>>need
>>> to create threads, manage memory, and generally take over ourselves.
>>>
>>> Perhaps dpdk can be split into two layers, a library layer that only
>>> provides mechanisms, and a framework layer that glues together those
>>> mechanisms and applies a policy, trading in generality for ease of use.
>> The DPDK system is somewhat divided now between the EAL, PMDS and
>>utility
>> functions like malloc/rings/?
>>
>> The problem I see is the PMDs need a framework to be usable and the EAL
>> plus the ethdev layers provide that support today. Setting up and
>> initializing the DPDK system is pretty clean just call the EAL init
>> routines along with the pool creates and the basic configs for the
>> PMDs/hardware. Once the system is inited one can create new threads and
>> not requiring anyone to use DPDK launch routines. Maybe I am not
>> understanding your needs can you explain more?
>
>An initialization routine that accepts argc/argv can hardly be called
>clean.

You want a config file or structure initialization design? If that is the
case you can contribute that support as another way to initialize DPDK.
>
>In seastar, we have our own malloc() (since seastar is sharded we can
>provide a faster thread-unsafe malloc implementation).  We also have our
>own threading, and since dpdk is an optional component in seastar, dpdk
>support requires code duplication.

DPDK replies one the huge page support for allocation to get the
performance, do you also not require huge page support. The malloc system
in DPDK can be used as a replacement for the standard malloc if that works
for your needs. Also after DPDK inits you can use your own malloc and any
other tools you want to use. I do not see a lot of duplicate code here
IMO. I guess if you are installing into a very small memory system then
yes it could be a problem, but DPDK is was not designed to run in a system
with limited memory.

>
>I would like to launch my own threads, pin them where I like, and call
>PMD drivers to send and receive packets.  Practically everything else
>that dpdk does gets in my way, including mbuf pools.  I'd much prefer to
>allocate mbufs myself.

You do not need to use the lauching of threads in the EAL and can supply
your own, right?

Regards,
++Keith
>
>
>>> [1] http://seastar-project.org
>

[dpdk-dev] Beyond DPDK 2.0

[Wiles, Keith]

On 5/7/15, 7:02 AM, "Avi Kivity"  wrote:

>On Wed, Apr 22, 2015 at 6:11 PM, O'Driscoll, Tim
>
>wrote:
>
>> Does anybody have any input or comments on this?
>>
>>
>> > -Original Message-
>> > From: O'Driscoll, Tim
>> > Sent: Thursday, April 16, 2015 11:39 AM
>> > To: dev at dpdk.org
>> > Subject: Beyond DPDK 2.0
>> >
>> > Following the launch of DPDK by Intel as an internal development
>> > project, the launch of dpdk.org by 6WIND in 2013, and the first DPDK
>>RPM
>> > packages for Fedora in 2014, 6WIND, Red Hat and Intel would like to
>> > prepare for future releases after DPDK 2.0 by starting a discussion on
>> > its evolution. Anyone is welcome to join this initiative.
>> >
>> > Since then, the project has grown significantly:
>> > -The number of commits and mailing list posts has increased
>> > steadily.
>> > -Support has been added for a wide range of new NICs (Mellanox
>> > support submitted by 6WIND, Cisco VIC, Intel i40e and fm10k etc.).
>> > -DPDK is now supported on multiple architectures (IBM Power
>>support
>> > in DPDK 1.8, Tile support submitted by EZchip but not yet reviewed or
>> > applied).
>> >
>> > While this is great progress, we need to make sure that the project is
>> > structured in a way that enables it to continue to grow. To achieve
>> > this, 6WIND, Red Hat and Intel would like to start a discussion about
>> > the future of the project, so that we can agree and establish
>>processes
>> > that satisfy the needs of the current and future DPDK community.
>> >
>> > We're very interested in hearing the views of everybody in the
>> > community. In addition to debate on the mailing list, we'll also
>> > schedule community calls to discuss this.
>> >
>> >
>> > Project Goals
>> > -
>> >
>> > Some topics to be considered for the DPDK project include:
>> > -Project Charter: The charter of the DPDK project should be
>>clearly
>> > defined, and should explain the limits of DPDK (what it does and does
>> > not cover). This does not mean that we would be stuck with a singular
>> > charter for all time, but the direction and intent of the project
>>should
>> > be well understood.
>>
>
>
>One problem we've seen with dpdk is that it is a framework, not a library:
>it wants to create threads, manage memory, and generally take over.  This
>is a problem for us, as we are writing a framework (seastar, [1]) and need
>to create threads, manage memory, and generally take over ourselves.
>
>Perhaps dpdk can be split into two layers, a library layer that only
>provides mechanisms, and a framework layer that glues together those
>mechanisms and applies a policy, trading in generality for ease of use.

The DPDK system is somewhat divided now between the EAL, PMDS and utility
functions like malloc/rings/?

The problem I see is the PMDs need a framework to be usable and the EAL
plus the ethdev layers provide that support today. Setting up and
initializing the DPDK system is pretty clean just call the EAL init
routines along with the pool creates and the basic configs for the
PMDs/hardware. Once the system is inited one can create new threads and
not requiring anyone to use DPDK launch routines. Maybe I am not
understanding your needs can you explain more?
>
>[1] http://seastar-project.org

[dpdk-dev] Beyond DPDK 2.0

[Wiles, Keith]

Darn forgot the site link, below.

On 4/30/15, 4:31 PM, "Wiles, Keith"  wrote:

>
>(I snipped out the content here only because it had been snipped a lot
>already)
>
>Sorry, if I am highjacking the thread.
>
>I believe the DPDK community would benefit from moving to GitHub as the
>primary DPDK site. http://github.com
>
>I believe the DPDK community can benefit from being at a very well know
>world wide site. GitHub seems to have the most eyes of any of the open
>source Git repos today and it appears they have more then twice as many
>developers. GitHub has a number of features I see as some good additions
>to
>our community using the GitHub organization account type.
>
>The cost for an organization account is $0 as long as we do not need more
>then 5 private repos. 10 private repos is $25/month and has other plans
>for more. I do not see us needing more then 5 private repos today and the
>only reason I can see having a private repo is to do some prep work on the
>repo before making public. Every contributor would need to create a GitHub
>personal account, which is at no cost unless you need more then 5 private
>repos. In both accounts you can have unlimited public repos.
>
>https://help.github.com/articles/where-can-i-find-open-source-projects-to-
>w
>ork-on/
>
>http://www.sitepoint.com/using-git-open-source-projects/
>
>- Adding more committers can lead to a security problems for 6Wind (I
>assume).
>- 6Wind appearing to own DPDK.org is not a good message to the community.
>  - Not assuming 6Wind?s dpdk.org site will disappear only where the
>community stores the master repos and how the community interacts with the
>master.
>- Permission and access levels in dpdk.org is only one level and we can
>benefit from having 4 levels and teams as well.
>- The patch process today suffers from timely reviews, which will not be
>fixed by moving.
>  - GitHub has a per pull request discussions area, which gives a clean
>way to
>review all discussions on a specific change.
>- The current patch model is clone dpdk.org/modify/commit/send patch
>set
>- The model with GitHub is fork on GitHub/modify/commit/send pull
>request
>- The patchwork web site is reasonable, but has some draw backs in
>maintaining the site.
>  - GitHub manages the patches via pull requests and can be easily seen
>via a web browser.
>  - The down side is you do have to use a web browser to do some work, but
>the everyday work would be done as it is today.
>- I think we all have a web browser now :-)
>- GitHub has team support and gives a group better control plus
>collaboration is much easier as we have a external location to work.
>  - Most companies have some pretty high security level and being to
>collaborate between two or more companies is very difficult if one company
>is hosting the repo behind a firewall.
>  - Using GitHub and teams would make collaboration a lot easier or
>collaboration between two or more user accounts as well.
>- GitHub has a Web Page system, which can be customized for the community
>needs via a public or private repo.
>- We still need a dpdk.org email list I believe as I did not find one at
>GitHub.
>  - We can also forward GitHub emails to the list.
>  - I believe you can reply to an email from GitHub and the email will get
>appended to the discussion thread.
>
>As most do not like to read long emails :-) I will stop here and add one
>more thing.
>
>I have create a sandbox on GitHub for anyone to play with using GitHub.
>You will need to create a GitHub account and an email me your account name
>to add you to the organization site as a contributor.
>
>The GitHub site is not a fork of dpdk.org only a sandbox to play with how
>GitHub can help the community to gain more developers in a clean manner.

https://github.com/dpdk-org


>Regards
>++Keith
>
> 
>
>
>

[dpdk-dev] Beyond DPDK 2.0

[Wiles, Keith]

(I snipped out the content here only because it had been snipped a lot
already)

Sorry, if I am highjacking the thread.

I believe the DPDK community would benefit from moving to GitHub as the
primary DPDK site. http://github.com

I believe the DPDK community can benefit from being at a very well know
world wide site. GitHub seems to have the most eyes of any of the open
source Git repos today and it appears they have more then twice as many
developers. GitHub has a number of features I see as some good additions to
our community using the GitHub organization account type.

The cost for an organization account is $0 as long as we do not need more
then 5 private repos. 10 private repos is $25/month and had other plans
for more. I do not see us needing more then 5 private repos today and the
only reason I can see having a private repo is to do some prep work on the
repo before making public. Every contributor would need to create a GitHub
personal account, which is at no cost unless you need more then 5 private
repos. In both accounts you can have unlimited public repos.

https://help.github.com/articles/where-can-i-find-open-source-projects-to-w
ork-on/

http://www.sitepoint.com/using-git-open-source-projects/

- Adding more committers can lead to a security problems for 6Wind (I
assume).
- 6Wind appearing to own DPDK.org is not a good message to the community.
  - Not assuming 6Wind?s dpdk.org site will disappear only where the
community stores the master repos and how the community interacts with the
master.
- Permission and access levels in dpdk.org is only one level and we can
benefit from having 4 levels and teams as well.
- The patch process today suffers from timely reviews, which will not be
fixed by moving.
  - GitHub has a per pull request discussions area, which gives a clean
way to
review all discussions on a specific change.
- The current patch model is clone dpdk.org/modify/commit/send patch
set
- The model with GitHub is fork on GitHub/modify/commit/send pull
request
- The patchwork web site is reasonable, but has some draw backs in
maintaining the site.
  - GitHub manages the patches via pull requests and can be easily seen
via a web browser.
  - The down side is you do have to use a web browser to do some work, but
the everyday work would be done as it is today.
- I think we all have a web browser now :-)
- GitHub has team support and gives a group better control plus
collaboration is much easier as we have a external location to work.
  - Most companies have some pretty high security level and being to
collaborate between two or more companies is very difficult if one company
is hosting the repo behind a firewall.
  - Using GitHub and teams would make collaboration a lot easier or
collaboration between two or more user accounts as well.
- GitHub has a Web Page system, which can be customized for the community
needs via a public or private repo.
- We still need a dpdk.org email list I believe as I did not find one at
GitHub.
  - We can also forward GitHub emails to the list.
  - I believe you can reply to an email from GitHub and the email will get
appended to the discussion thread.

As most do not like to read long emails :-) I will stop here and add one
more thing.

I have create a sandbox on GitHub for anyone to play with using GitHub.
You will need to create a GitHub account and an email me your account name
to add you to the organization site as a contributor.

The GitHub site is not a fork of dpdk.org only a sandbox to play with how
GitHub can help the community to gain more developers in a clean manner.

Regards
++Keith

[dpdk-dev] Beyond DPDK 2.0

[Jay Rolette]

On Tue, Apr 28, 2015 at 12:26 PM, Neil Horman  wrote:

> On Mon, Apr 27, 2015 at 08:46:01AM -0500, Jay Rolette wrote:
> > On Sat, Apr 25, 2015 at 7:10 AM, Neil Horman 
> wrote:
> >
> > > On Fri, Apr 24, 2015 at 02:55:33PM -0500, Jay Rolette wrote:
> > > > On Fri, Apr 24, 2015 at 1:51 PM, Neil Horman 
> > > wrote:
> > > >
> > > > > So, I hear your arguments, and its understandable that you might
> not
> > > want
> > > > > a GPL
> > > > > licensed product, given that the DPDK is a library (though I'm not
> sure
> > > > > what the
> > > > > aversion to LGPL would be).  Regardless, I think this conversation
> is a
> > > > > bit more
> > > > > about participation than license choice.  While you are correct, in
> > > that
> > > > > the
> > > > > first step to support (by which I presume you mean participation
> in the
> > > > > community) is use, the goal here is to get people contributing
> patches
> > > and
> > > > > helping increase the usefulness of DPDK.
> > > >
> > > >
> > > > > Given that DPDK is primarily licensed as BSD now, whats preventing
> > > you, or
> > > > > what
> > > > > would encourage you to participate in the community?  I see emails
> from
> > > > > infiniteio addresss in the archives asking questions and making
> > > > > suggestions on
> > > > > occasion, but no patches.  What would get you (or others in a
> simmilar
> > > > > situation) to submit those?
> > > > >
> > > >
> > > > 36 hours in the day? :)
> > > >
> > > > It's not a lot, but we've submitted a couple of small patches. It's
> > > mostly
> > > > a matter of opportunity. We submit patches as we come across DPDK
> bugs or
> > > > find useful optos.
> > > >
> > >
> > 
> >
> > >
> > > Understand, I'm not trying to single you out here.  I see that you, and
> > > others
> > > from infiniteio have had some participation in the project, and thats
> > > great, and
> > > appreciated. I'm more focused on why that level of participation is not
> > > higher
> > > (not only from you, but from the larger presumed user base in
> general).  As
> > > noted at the start of this thread, one of the goals of this discussion
> is
> > > to
> > > find ways to maximize participation in the project, and from you I'm
> > > hearing
> > > that:
> > >
> > > 1) you use the dpdk because it lowers maintenence overhead
> > > 2) you don't participate more in the project because your product work
> > > schedule
> > > doesn't allow you to do so.
> > >
> > > Are those both accurate statements?
> > >
> >
> > (1) was just my response to Luke about what would motivate a company to
> > submit patches if the license didn't require it (GPL vs BSD discussion).
> > Maint overhead had little to do with why we decided to use DPDK.
> >
> > (2) is certainly true enough, but not really all that big of a factor in
> > the reasons why our participation isn't at some higher level. I'd say
> there
> > are two primary factors:
> >
> > *New Contributors*
> > Dropping a major patch on a project where we have no history would be
> > foolish and frustrating. Every open source project has a vibe to it and
> its
> > own way of operating. It normally takes some time to figure that out, but
> > for major contributions, it generally involves a level of trust.
> >
> > Major code drops or patches aren't generally well received unless it is
> > from someone that is known and trusted by the community. Building up that
> > trust ("street cred") normally involves participating in smaller, less
> > risky ways. Answering questions where you can, small patches to fix bugs,
> > possibly reviewing code (although this can be iffy as well), etc.
> >
> > This facilitates understanding the process, who the players are and what
> > sort of toes you are likely to step on.
> >
> > It also gives you time to ramp up on the internals of the code and
> > philosophy/goals of the project better. With DPDK, performance is
> obviously
> > more important than portability. Until recently, very little care was
> given
> > to API stability or the impact that has on DPDK apps. Both of those are
> > very different approaches than typical and it affects what you might do
> > when approaching submitting a patch.
> >
> > If you want to build up the number of folks contributing, expect them to
> > start small and make sure it actually GOES somewhere.
> >
> > The first patch I submitted in mid-December had some quick responses and
> > questions back-and-forth, but then stalled on a couple of undocumented
> > minor stylistic things (comment style and use of #ifdefs). I never heard
> > anything back and 4.5 months later, a simple startup opto is still
> sitting
> > there.
> >
> > The second patch I submitted (also mid-December) got no response at all
> for
> > 4 months. I've replied to the feedback that came eventually, but once
> > again, no subsequent answers.
> >
> > Neither of the patches were important, but the process doesn't exactly
> > inspire a strong desire to contribute more.
> >
> > *Different Goals*
> > I see at least 

[dpdk-dev] Beyond DPDK 2.0

[Neil Horman]

On Mon, Apr 27, 2015 at 08:46:01AM -0500, Jay Rolette wrote:
> On Sat, Apr 25, 2015 at 7:10 AM, Neil Horman  wrote:
> 
> > On Fri, Apr 24, 2015 at 02:55:33PM -0500, Jay Rolette wrote:
> > > On Fri, Apr 24, 2015 at 1:51 PM, Neil Horman 
> > wrote:
> > >
> > > > So, I hear your arguments, and its understandable that you might not
> > want
> > > > a GPL
> > > > licensed product, given that the DPDK is a library (though I'm not sure
> > > > what the
> > > > aversion to LGPL would be).  Regardless, I think this conversation is a
> > > > bit more
> > > > about participation than license choice.  While you are correct, in
> > that
> > > > the
> > > > first step to support (by which I presume you mean participation in the
> > > > community) is use, the goal here is to get people contributing patches
> > and
> > > > helping increase the usefulness of DPDK.
> > >
> > >
> > > > Given that DPDK is primarily licensed as BSD now, whats preventing
> > you, or
> > > > what
> > > > would encourage you to participate in the community?  I see emails from
> > > > infiniteio addresss in the archives asking questions and making
> > > > suggestions on
> > > > occasion, but no patches.  What would get you (or others in a simmilar
> > > > situation) to submit those?
> > > >
> > >
> > > 36 hours in the day? :)
> > >
> > > It's not a lot, but we've submitted a couple of small patches. It's
> > mostly
> > > a matter of opportunity. We submit patches as we come across DPDK bugs or
> > > find useful optos.
> > >
> >
> 
> 
> >
> > Understand, I'm not trying to single you out here.  I see that you, and
> > others
> > from infiniteio have had some participation in the project, and thats
> > great, and
> > appreciated. I'm more focused on why that level of participation is not
> > higher
> > (not only from you, but from the larger presumed user base in general).  As
> > noted at the start of this thread, one of the goals of this discussion is
> > to
> > find ways to maximize participation in the project, and from you I'm
> > hearing
> > that:
> >
> > 1) you use the dpdk because it lowers maintenence overhead
> > 2) you don't participate more in the project because your product work
> > schedule
> > doesn't allow you to do so.
> >
> > Are those both accurate statements?
> >
> 
> (1) was just my response to Luke about what would motivate a company to
> submit patches if the license didn't require it (GPL vs BSD discussion).
> Maint overhead had little to do with why we decided to use DPDK.
> 
> (2) is certainly true enough, but not really all that big of a factor in
> the reasons why our participation isn't at some higher level. I'd say there
> are two primary factors:
> 
> *New Contributors*
> Dropping a major patch on a project where we have no history would be
> foolish and frustrating. Every open source project has a vibe to it and its
> own way of operating. It normally takes some time to figure that out, but
> for major contributions, it generally involves a level of trust.
> 
> Major code drops or patches aren't generally well received unless it is
> from someone that is known and trusted by the community. Building up that
> trust ("street cred") normally involves participating in smaller, less
> risky ways. Answering questions where you can, small patches to fix bugs,
> possibly reviewing code (although this can be iffy as well), etc.
> 
> This facilitates understanding the process, who the players are and what
> sort of toes you are likely to step on.
> 
> It also gives you time to ramp up on the internals of the code and
> philosophy/goals of the project better. With DPDK, performance is obviously
> more important than portability. Until recently, very little care was given
> to API stability or the impact that has on DPDK apps. Both of those are
> very different approaches than typical and it affects what you might do
> when approaching submitting a patch.
> 
> If you want to build up the number of folks contributing, expect them to
> start small and make sure it actually GOES somewhere.
> 
> The first patch I submitted in mid-December had some quick responses and
> questions back-and-forth, but then stalled on a couple of undocumented
> minor stylistic things (comment style and use of #ifdefs). I never heard
> anything back and 4.5 months later, a simple startup opto is still sitting
> there.
> 
> The second patch I submitted (also mid-December) got no response at all for
> 4 months. I've replied to the feedback that came eventually, but once
> again, no subsequent answers.
> 
> Neither of the patches were important, but the process doesn't exactly
> inspire a strong desire to contribute more.
> 
> *Different Goals*
> I see at least two different sets of people on the mailing list. The heavy
> hitters generally have a product reason for their high level of involvement
> with DPDK. For Intel and 6Wind, the reasons are pretty obvious. Same deal
> with NIC vendors. For large companies, sometimes the reasons are less
> obvious, but 

[dpdk-dev] Beyond DPDK 2.0

[Stephen Hemminger]

On Fri, 24 Apr 2015 09:47:58 +0200
Luke Gorrie  wrote:

> Hi Tim,
> 
> On 16 April 2015 at 12:38, O'Driscoll, Tim  
> wrote:
> 
> > Following the launch of DPDK by Intel as an internal development project,
> > the launch of dpdk.org by 6WIND in 2013, and the first DPDK RPM packages
> > for Fedora in 2014, 6WIND, Red Hat and Intel would like to prepare for
> > future releases after DPDK 2.0 by starting a discussion on its evolution.
> > Anyone is welcome to join this initiative.
> >
> 
> Thank you for the open invitation.
> 
> I have a couple of questions about the long term of DPDK:
> 
> 1. How will DPDK manage overlap with other project over time?
> 

In general DPDK will be successful only if it stick to differentiating
technology and avoids NIH and reinvention. 
I.e if DPDK tries to redo things in libc or have special
needs, then DPDK becomes harder to use for many things and makes DPDK 
applications
hard to integrate with other libraries. It is bad enough now that each
library has its own view of threads.

> DPDK into the kernel than the rest of the good bits of the kernel into DPDK?

If DPDK tries to become too general it will lose the performance
advantage. The kernel has to serve all types of applications,
and have many layers of services therefore it is slow. For example,
if every DPDK facility had its own locking and was thread safe
the performance would end up being about the same as just using
kernel.


> 2. How will DPDK users justify contributing to DPDK upstream?
> 
> Engineers in network equipment vendors want to contribute to open source,
> but what is the incentive for the companies to support this? This would be
> easy if DPDK were GPL'd (they are compelled) or if everybody were
> dynamically linking with the upstream libdpdk (can't have private patches).
> However, in a world where DPDK is BSD-licensed and statically linked, is it
> not both cheaper and competitively advantageous to keep fixes and
> optimizations in house?

There are several incentives.
a. Brocade views open source as a differentiator from competitors and wants
to contribute as much as possible to open source, this includes DPDK, Open 
Daylight
and Openstack. Marketing benefit.

b. By contributing what we do back we get benefits of more testing and review.
Several bugs have been spotted in areas that were not covered because the 
current
product usage and testing will not cover all possibilities.

c. By contributing back, the contributor gets to set the agenda and make the 
API's.
If you go first, you set the API and you can make life hard for competitors or
other users who do the same thing but haven't contributed. In fact, the worst 
pain
for us was cases where there were two or more parallel implementations of 
something
to deal with (ie vmxnet3).  "Lead, follow, or get of the way"

[dpdk-dev] Beyond DPDK 2.0

[Dor Laor]

On Mon, Apr 27, 2015 at 5:29 AM, Jim Thompson  wrote:

>
> > On Apr 26, 2015, at 4:56 PM, Neil Horman  wrote:
> >
> > On Sat, Apr 25, 2015 at 04:08:23PM +, Wiles, Keith wrote:
> >> +1 and besides the GPL or LGPL ship has sailed IMHO and we can not go
> back.
> > Actually, IANAL, but I think we can.  The BSD license allows us to fork
> and
> > relicense the code I think, under GPL or any other license.  I'm not
> advocating
> > for that mind you, just suggesting that its possible should it ever
> become
> > needed.
>
> I, on the other hand, am fairly certain that you can not ?relicense BSD
> licensed code under the GPL (or any other license).
>
> Were this true at law, then the opposite would also be possible.  (?Don?t
> like the license?  Just fork!?)
>
>
+1

While BSD carries many benefits for DPDK (similar to any other library),
GPL doesn't
carry any benefit in this case. It's not wise not to contribute back to
DPDK regardless
of any license of choice. Those who do not wish to contribute will always
manage to do..


Going back to the list of open source projects on top of DPDK I like to
mention
the SeaStar project:

   http://www.seastar-project.org

For those who aren't familiar, SeaStar is a high speed messaging framework
with a unique shared-nothing approach with a per-core granularity.
For our knowledge it offers the first open source TCP/IP implementation on
top
of DPDK along with many other advantages.

DPDK is one of the main building blocks that allow us to reach millions of
iops and we're
pretty pleased with the project - both the source code as well as the
community!

Dor

[dpdk-dev] Beyond DPDK 2.0

[Matthew Hall]

On Apr 25, 2015, at 5:10 AM, Neil Horman  wrote:
> I'm more focused on why that level of participation is not higher

Hi Neal,

This mail is probably way too long, but here is what I saw about participation, 
in my case I used DPDK on two projects so far:

1) proprietary project for a L4-L7 stateful replay DPI firewall performance 
tester at a large network test and measurement corporation using the old 
original 6WIND DPDK 1.X under NDA before it became available to a wider channel

2) open-source self-created threat intelligence sensor project using DPDK 
1.7.X, under development in my spare time

I think the things that making more DPDK contributions are ironically more 
technical and social in nature than legal or bureaucratic in nature as one 
would normally suspect, and as you theorized in your original mail. Let me go 
through some things and see what people think.

At the social level, there are not very many people in the world who are 
familiar and comfortable with the LKML style embedded coding workflow (heavy 
mailing list usage, sending and reviewing patches in emails, putting specific 
people in To or Cc to get patches approved and ACKed in subtrees, etc.) I 
happen to know some if not most of this tribally, because I used Linux since 
1997, but very few developers have any clue about this stuff. However I never 
participated in the actual LKML flow any further than tester / bug reporter, 
and I actually use DPDK very deliberately, to avoid fighting with the headaches 
of the linux-net code and flamewars.

This is why I was proposing that we try to find a way to allow contributions 
via Github or Bitbucket... their fork-and-pull model is much simpler for 
outsiders to comprehend and make quick patches when they find little bugs or 
issues as they integrate with the library... given we are BSD licensed, the low 
barrier to entry is the ultimate way to keep the patch velocity as high as 
possible, and keep the community going.

At the technical level, I see two or three difficulties:

1) A lot of the various performance enhancements one can use are kind of 
"magical" or "jigsaw puzzle" and not presented in a unified way, where I can 
methodically go through a checklist and enable everything my app should use 
even though I have no clue what they all are. For example, 1) let's talk about 
hashing... there is RSS hashing (symmetric or asymmetric), JHASH, CRC hash, ... 
not sure how many different ones. 2) Let's talk about CPU models... SSE, SSE4, 
SSE4.1, SSE4.2, etc. I don't know what I have myself, much less what my users 
will have, much less what I actually need or should use, without guidance from 
some processor people. 3) Let's talk about PCIe bus... there is DCA, some other 
non-DCA acceleration that's faster if you are on the same NUMA node as the PCIe 
slot, but fails to work if you aren't... etc.

A lot of the people from Intel, 6WIND, and the kernel people are just thinking 
"this stuff is obvious... we've used it since 200X and it's 2015!" That's true 
if you are a processor / kernel hacker... but if you spent your whole career on 
packet processing or network security like a lot of us app developers might 
have done, that's very orthogonal to Intel-specific and compiler-specific and 
hardware-specific performance hacks... so a lot of us have no real clue how to 
configure and test them all, much less enhance them and make some patches to 
it. We just blindly trust 6WIND and Intel to get it right, because as far as we 
can see, all the DPDK code is pretty clean and readable, and we're pretty sure 
we don't know anything better than what's already coded and put into the 
repository, if we don't have some checklists to follow to enable and test every 
combination, and find any more bugfixes to suggest.

2) A lot of the network adapters DPDK uses, especially when you begin using the 
more crazy accelerations, are either hard to obtain or expensive... for example 
from what I saw in my jobs, the 10 gigabit boards were a minimum ~$250 USD in 
manufacturing quantities. The 2-port gigabit latest-gen Intel board I got was 
$100 USD... I think a lot of higher-ed students and overseas people from less 
developed nations might have a hard time paying for some of this stuff to start 
hacking... some kind of program to get these sort of people some sample gear 
might help.

I also hit difficulties with the virtio-net driver... it doesn't work with the 
virtio-net adapter in Virtualbox... which makes it harder for me to use cool, 
convenient environments configured by the tool Vagrant, to make very simple dev 
VM environments to quickly get new hackers up to speed on my open source 
running just a command or two, and by extension harder for me to show them why 
they should use DPDK for all their cool new network ${GADGET}s.

The same difficulty comes into place if I wanted to do some performance 
patches... I don't have the money to buy the VTune profiler for my spare time 
project, that you 

[dpdk-dev] Beyond DPDK 2.0

[Marc Sune]

On 27/04/15 15:39, Wiles, Keith wrote:
>
> On 4/27/15, 4:52 AM, "Marc Sune"  wrote:
>
>>
>> On 27/04/15 03:41, Wiles, Keith wrote:
>>> On 4/26/15, 4:56 PM, "Neil Horman"  wrote:
>>>
 On Sat, Apr 25, 2015 at 04:08:23PM +, Wiles, Keith wrote:
> On 4/25/15, 8:30 AM, "Marc Sune"  wrote:
>
>> On 24/04/15 19:51, Matthew Hall wrote:
>>> On Fri, Apr 24, 2015 at 12:39:47PM -0500, Jay Rolette wrote:
 I can tell you that if DPDK were GPL-based, my company wouldn't be
 using
 it. I suspect we wouldn't be the only ones...

 Jay
>>> I could second this, from the past employer where I used it. Right
> now
>>> I am
>>> using it in an open source app, I have a bit of GPL here and there
> but
>>> I'm
>>> trying to get rid of it or confine it to separate address spaces,
> where
>>> it
>>> won't impact the core code written around DPDK, as I don't want to
> cause
>>> headaches for any downstream users I attract someday.
>>>
>>> Hard-core GPL would not be possible for most. LGPL could be
>>> possible,
>>> but I
>>> don't think it could be worth the relicensing headache for that
>>> small
>>> change.
>>>
>>> Instead we should make the patch process as easy as humanly possible
> so
>>> people
>>> are encouraged to send us the fixes and not cart them around their
>>> companies
>>> constantly.
> +1 and besides the GPL or LGPL ship has sailed IMHO and we can not go
> back.
 Actually, IANAL, but I think we can.  The BSD license allows us to fork
 and
 relicense the code I think, under GPL or any other license.  I'm not
 advocating
 for that mind you, just suggesting that its possible should it ever
 become
 needed.

>> I agree. My feeling is that as the number of patches in the mailing
> list
>> grows, keeping track of them gets more and more complicated.
>> Patchwork
>> website was a way to try to address this issue. I think it was an
>> improvement, but to be honest, patchwork lacks a lot of
>> functionality,
>> such as properly tracking multiple versions of the patch (superseding
>> them automatically), and it lacks some filtering capabilities e.g.
>> per
>> user, per tag/label or library, automatically track if it has been
>> merged, give an overall status of the pending vs merged patches, set
>> milestones... Is there any alternative tool or improved version for
> that?
>
 Agreed, this has come up before, off list unfortunately.  The volume of
 patches
 seems to be increasing at such a rate that a single maintainer has
 difficulty
 keeping up.  I proposed that the workload be split out to multiple
 subtrees,
 with prefixes being added to patch subjects on the list for local
 filtering to
 stem the tide.  Specifically I had proposed that the PMD's be split
 into a
 separate subtree, but that received pushback in favor of having each
 library
 having its own separate subtree, with a pilot program being made out of
 the I40e
 driver (which you might note sends pull requests to the list now).  I'd
 still
 like to see all PMD's come under a single subtree, but thats likely an
 argument
 for later.

 That said, Do you think that this patch latency is really a contributor
 to low
 project participation?  It definately a problem, but it seems to me
 that
 this
 sort of issue would lead to people trying to parcitipate, then giving
 up
 (i.e.
 we would see 1-2 emails from an individual, then not see them again).
 I'd need
 to look through the mailing list for such a pattern, but anecdotally
 I've
 not
 seen that happen.  The problem you describe above is definately a
 problem, but
 its one for those individuals who are participating, not for those who
 are
 simply choosing not to.  And I think we need to address both.

> I agree patchwork has some limitation, but I think the biggest issue
> is
> keeping up with the patches. Getting patches introduced into the main
> line
> is very slow. A patch submitted today may not get applied for weeks or
> months, then when another person submits a patch he is starting to
> run a
> very high risk of having to redo that patch, because a pervious patch
> makes his fail weeks/months later. I would love to see a better tool
> then
> patchwork, but the biggest issue is we have a huge backlog of patches.
> Personally I am not sure how Thomas or any is able to keep up with the
> patches.
>
 This is absolutely a problem.  I'd like to think, more than a tool like
 patchwork, a subtree organization to allow some modicum of parallel
 review and
 integration would really be a benefit here.
>>> Subtrees could work, but the 

[dpdk-dev] Beyond DPDK 2.0

[Wiles, Keith]

On 4/27/15, 5:29 AM, "Neil Horman"  wrote:

>On Mon, Apr 27, 2015 at 01:41:11AM +, Wiles, Keith wrote:
>> 
>> 
>> On 4/26/15, 4:56 PM, "Neil Horman"  wrote:
>> 
>> >On Sat, Apr 25, 2015 at 04:08:23PM +, Wiles, Keith wrote:
>> >> 
>> >> 
>> >> On 4/25/15, 8:30 AM, "Marc Sune"  wrote:
>> >> 
>> >> >
>> >> >
>> >> >On 24/04/15 19:51, Matthew Hall wrote:
>> >> >> On Fri, Apr 24, 2015 at 12:39:47PM -0500, Jay Rolette wrote:
>> >> >>> I can tell you that if DPDK were GPL-based, my company wouldn't
>>be
>> >> >>>using
>> >> >>> it. I suspect we wouldn't be the only ones...
>> >> >>>
>> >> >>> Jay
>> >> >> I could second this, from the past employer where I used it. Right
>> >>now
>> >> >>I am
>> >> >> using it in an open source app, I have a bit of GPL here and there
>> >>but
>> >> >>I'm
>> >> >> trying to get rid of it or confine it to separate address spaces,
>> >>where
>> >> >>it
>> >> >> won't impact the core code written around DPDK, as I don't want to
>> >>cause
>> >> >> headaches for any downstream users I attract someday.
>> >> >>
>> >> >> Hard-core GPL would not be possible for most. LGPL could be
>>possible,
>> >> >>but I
>> >> >> don't think it could be worth the relicensing headache for that
>>small
>> >> >>change.
>> >> >>
>> >> >> Instead we should make the patch process as easy as humanly
>>possible
>> >>so
>> >> >>people
>> >> >> are encouraged to send us the fixes and not cart them around their
>> >> >>companies
>> >> >> constantly.
>> >> 
>> >> +1 and besides the GPL or LGPL ship has sailed IMHO and we can not go
>> >>back.
>> >Actually, IANAL, but I think we can.  The BSD license allows us to fork
>> >and
>> >relicense the code I think, under GPL or any other license.  I'm not
>> >advocating
>> >for that mind you, just suggesting that its possible should it ever
>>become
>> >needed.
>> >
>> >> >
>> >> >I agree. My feeling is that as the number of patches in the mailing
>> >>list
>> >> >grows, keeping track of them gets more and more complicated.
>>Patchwork
>> >> >website was a way to try to address this issue. I think it was an
>> >> >improvement, but to be honest, patchwork lacks a lot of
>>functionality,
>> >> >such as properly tracking multiple versions of the patch
>>(superseding
>> >> >them automatically), and it lacks some filtering capabilities e.g.
>>per
>> >> >user, per tag/label or library, automatically track if it has been
>> >> >merged, give an overall status of the pending vs merged patches, set
>> >> >milestones... Is there any alternative tool or improved version for
>> >>that?
>> >> 
>> >Agreed, this has come up before, off list unfortunately.  The volume of
>> >patches
>> >seems to be increasing at such a rate that a single maintainer has
>> >difficulty
>> >keeping up.  I proposed that the workload be split out to multiple
>> >subtrees,
>> >with prefixes being added to patch subjects on the list for local
>> >filtering to
>> >stem the tide.  Specifically I had proposed that the PMD's be split
>>into a
>> >separate subtree, but that received pushback in favor of having each
>> >library
>> >having its own separate subtree, with a pilot program being made out of
>> >the I40e
>> >driver (which you might note sends pull requests to the list now).  I'd
>> >still
>> >like to see all PMD's come under a single subtree, but thats likely an
>> >argument
>> >for later.
>> >
>> >That said, Do you think that this patch latency is really a contributor
>> >to low
>> >project participation?  It definately a problem, but it seems to me
>>that
>> >this
>> >sort of issue would lead to people trying to parcitipate, then giving
>>up
>> >(i.e.
>> >we would see 1-2 emails from an individual, then not see them again).
>> >I'd need
>> >to look through the mailing list for such a pattern, but anecdotally
>>I've
>> >not
>> >seen that happen.  The problem you describe above is definately a
>> >problem, but
>> >its one for those individuals who are participating, not for those who
>>are
>> >simply choosing not to.  And I think we need to address both.
>> >
>> >> I agree patchwork has some limitation, but I think the biggest issue
>>is
>> >> keeping up with the patches. Getting patches introduced into the main
>> >>line
>> >> is very slow. A patch submitted today may not get applied for weeks
>>or
>> >> months, then when another person submits a patch he is starting to
>>run a
>> >> very high risk of having to redo that patch, because a pervious patch
>> >> makes his fail weeks/months later. I would love to see a better tool
>> >>then
>> >> patchwork, but the biggest issue is we have a huge backlog of
>>patches.
>> >> Personally I am not sure how Thomas or any is able to keep up with
>>the
>> >> patches.
>> >> 
>> >This is absolutely a problem.  I'd like to think, more than a tool like
>> >patchwork, a subtree organization to allow some modicum of parallel
>> >review and
>> >integration would really be a benefit here.
>> Subtrees could work, but the real problem I think is the number of
>> 

[dpdk-dev] Beyond DPDK 2.0

[Wiles, Keith]

On 4/27/15, 4:52 AM, "Marc Sune"  wrote:

>
>
>On 27/04/15 03:41, Wiles, Keith wrote:
>>
>> On 4/26/15, 4:56 PM, "Neil Horman"  wrote:
>>
>>> On Sat, Apr 25, 2015 at 04:08:23PM +, Wiles, Keith wrote:

 On 4/25/15, 8:30 AM, "Marc Sune"  wrote:

>
> On 24/04/15 19:51, Matthew Hall wrote:
>> On Fri, Apr 24, 2015 at 12:39:47PM -0500, Jay Rolette wrote:
>>> I can tell you that if DPDK were GPL-based, my company wouldn't be
>>> using
>>> it. I suspect we wouldn't be the only ones...
>>>
>>> Jay
>> I could second this, from the past employer where I used it. Right
 now
>> I am
>> using it in an open source app, I have a bit of GPL here and there
 but
>> I'm
>> trying to get rid of it or confine it to separate address spaces,
 where
>> it
>> won't impact the core code written around DPDK, as I don't want to
 cause
>> headaches for any downstream users I attract someday.
>>
>> Hard-core GPL would not be possible for most. LGPL could be
>>possible,
>> but I
>> don't think it could be worth the relicensing headache for that
>>small
>> change.
>>
>> Instead we should make the patch process as easy as humanly possible
 so
>> people
>> are encouraged to send us the fixes and not cart them around their
>> companies
>> constantly.
 +1 and besides the GPL or LGPL ship has sailed IMHO and we can not go
 back.
>>> Actually, IANAL, but I think we can.  The BSD license allows us to fork
>>> and
>>> relicense the code I think, under GPL or any other license.  I'm not
>>> advocating
>>> for that mind you, just suggesting that its possible should it ever
>>>become
>>> needed.
>>>
> I agree. My feeling is that as the number of patches in the mailing
 list
> grows, keeping track of them gets more and more complicated.
>Patchwork
> website was a way to try to address this issue. I think it was an
> improvement, but to be honest, patchwork lacks a lot of
>functionality,
> such as properly tracking multiple versions of the patch (superseding
> them automatically), and it lacks some filtering capabilities e.g.
>per
> user, per tag/label or library, automatically track if it has been
> merged, give an overall status of the pending vs merged patches, set
> milestones... Is there any alternative tool or improved version for
 that?

>>> Agreed, this has come up before, off list unfortunately.  The volume of
>>> patches
>>> seems to be increasing at such a rate that a single maintainer has
>>> difficulty
>>> keeping up.  I proposed that the workload be split out to multiple
>>> subtrees,
>>> with prefixes being added to patch subjects on the list for local
>>> filtering to
>>> stem the tide.  Specifically I had proposed that the PMD's be split
>>>into a
>>> separate subtree, but that received pushback in favor of having each
>>> library
>>> having its own separate subtree, with a pilot program being made out of
>>> the I40e
>>> driver (which you might note sends pull requests to the list now).  I'd
>>> still
>>> like to see all PMD's come under a single subtree, but thats likely an
>>> argument
>>> for later.
>>>
>>> That said, Do you think that this patch latency is really a contributor
>>> to low
>>> project participation?  It definately a problem, but it seems to me
>>>that
>>> this
>>> sort of issue would lead to people trying to parcitipate, then giving
>>>up
>>> (i.e.
>>> we would see 1-2 emails from an individual, then not see them again).
>>> I'd need
>>> to look through the mailing list for such a pattern, but anecdotally
>>>I've
>>> not
>>> seen that happen.  The problem you describe above is definately a
>>> problem, but
>>> its one for those individuals who are participating, not for those who
>>>are
>>> simply choosing not to.  And I think we need to address both.
>>>
 I agree patchwork has some limitation, but I think the biggest issue
is
 keeping up with the patches. Getting patches introduced into the main
 line
 is very slow. A patch submitted today may not get applied for weeks or
 months, then when another person submits a patch he is starting to
run a
 very high risk of having to redo that patch, because a pervious patch
 makes his fail weeks/months later. I would love to see a better tool
 then
 patchwork, but the biggest issue is we have a huge backlog of patches.
 Personally I am not sure how Thomas or any is able to keep up with the
 patches.

>>> This is absolutely a problem.  I'd like to think, more than a tool like
>>> patchwork, a subtree organization to allow some modicum of parallel
>>> review and
>>> integration would really be a benefit here.
>> Subtrees could work, but the real problem I think is the number of
>> committers must be higher then one. Something like GitHub (and I assume
>> Linux Foundation) have a method to add 

[dpdk-dev] Beyond DPDK 2.0

[Neil Horman]

On Mon, Apr 27, 2015 at 01:50:17PM +, Wiles, Keith wrote:
 >> >> 
> >> >This is absolutely a problem.  I'd like to think, more than a tool like
> >> >patchwork, a subtree organization to allow some modicum of parallel
> >> >review and
> >> >integration would really be a benefit here.
> >> Subtrees could work, but the real problem I think is the number of
> >> committers must be higher then one. Something like GitHub (and I assume
> >> Linux Foundation) have a method to add committers to a project. In the
> >> case of GitHub they just have to have a free GitHub account and they can
> >> become committers of the project buying the owner of the project enables
> >> them.
> >> 
> >> On GitHub they have personal accounts and organization accounts I know
> >> only about the personal accounts, but they allow for 5 private repos and
> >> any number of public repos. The organization account has a lot of extra
> >> features that seem better for a DPDK community IMO and should be the one
> >> we use if we decide it is the right direction. We can always give it a
> >> shot for while and keep the dpdk.org and use dev at dpdk.org and its repo
> >> mirrored from GitHub as a transition phase. This way we can fall back to
> >> dpdk.org or move one to something else if we like.
> >> 
> >> https://help.github.com/categories/organizations/
> >> 
> >> The developers could still send patches via email list, but creating a
> >> repo and forking dpdk is easy, then send a pull request.
> >> 
> >I'm not opposed to github per-se, but nothing described above is unique to
> >github. Theres no reason we can't allow multiple comitters to the current
> >tree
> >as hosted on the current server, we just have to configure it as such.
> >
> >And FWIW, the assumption is that, with multiple subtrees, you implicitly
> >have
> >multiple comitters, assuming that pull requests from those subtree
> >maintainers
> >are trusted by the top level tree maintainer.
> >
> >In fact I feel somewhat better about that model as it provides a nice
> >stairstep
> >integration path for new features.
> >
> >Not explicitly opposed to a movement to github, I just feel like it may
> >not
> >address the problem at hand.
> 
> As I see your concerns is creating multiple repos or splitting up the
> current repo, which can be done in a single GitHub org account and they
> all appear on the page. This way we can move the current other repos like
> Pktgen to this location and everyone sees all of the repos in a much
> easier way IMO. The org account at GitHub gives you the multiple
> committers and even teams. I see we only need one team today for DPDK repo
> and then we have something like Pktgen as a different team and so on.
> >
But again, we have, and do this now:
http://dpdk.org/browse/

There are seveal git repositories there, all related to the dpdk, hosted on the
current site.  All we need to do is expand our use of subtrees.  We don't have
to move the product to a new location to make that happen.

Neil

[dpdk-dev] Beyond DPDK 2.0

[Neil Horman]

On Mon, Apr 27, 2015 at 08:38:48AM -0400, Dave Neary wrote:
> Hi,
> 
> On 04/26/2015 05:56 PM, Neil Horman wrote:
> > On Sat, Apr 25, 2015 at 04:08:23PM +, Wiles, Keith wrote:
> >> I would like to see some type of layering process to allow patches to be
> >> applied in a timely manner a few weeks not months or completely ignored.
> >> Maybe some type of voting is reasonable, but we need to do something to
> >> turn around the patches in clean reasonable manner.
> >>
> >> Think we need some type of group meeting every week to look at the patches
> >> and determining which ones get applied, this gives quick feedback to the
> >> submitter as to the status of the patch.
> >>
> > I think a group meeting is going to be way too much overhead to manage 
> > properly.
> > You'll get different people every week with agenda that may not line up with
> > code quality, which is really what the review is meant to provide.  I think
> > perhaps a better approach would be to require that that code owners from the
> > maintainer file provide and ACK/NAK on their patches within 3-4 days, and
> > require a corresponding tree maintainer to apply the patch within 7 or so.  
> > That
> > would cap our patch latency.  Likewise, if a patch slips in creating a
> > regression, the author needs to be alerted and given a time window in which 
> > to
> > fix the problem before the offending patch is reverted during the QE cycle.
> 
> What Keith is describing is very similar to a change management/change
> control board you might find for production/IT processes:
> http://en.wikipedia.org/wiki/Change_control_board
> 
> An efficient change management board approves "low overhead" changes
> automatically/very quickly, and focusses on the 10% of changes which
> could be disruptive (and what disruptive means changes from one
> environment to another) - for code it would be any patches that
> potentially conflict, anything that could cause regressions, add
> instability or uncertainty, and any feature which can be implemented
> multiple ways.
> 
> Not saying this would work - I have never seen an open source project
> implement a change management process for handling patches, and
> instinctively I agree with you Neil that it would be a lot of overhead,
> but it's an interesting thought exercise to think how it might work.
> 

I take you're meaning Dave, and it is an interesting thought experiment.  how
would such a change control board mesh with a public review list however?  That
is to say, would such a voting board not insulate decision makers from community
participation?  Perhaps I'm mistaken there, but it seems like allowing a small
group of maintainers make acceptance decisions in a private meeting would
insulate them from individual accountability on a list.

Neil

> Thanks,
> Dave.
> 
> -- 
> Dave Neary - NFV/SDN Community Strategy
> Open Source and Standards, Red Hat - http://community.redhat.com
> Ph: +1-978-399-2182 / Cell: +1-978-799-3338
> 

[dpdk-dev] Beyond DPDK 2.0

[Stephen Hemminger]

On Mon, 27 Apr 2015 08:38:48 -0400
Dave Neary  wrote:

> What Keith is describing is very similar to a change management/change
> control board you might find for production/IT processes:
> http://en.wikipedia.org/wiki/Change_control_board
> 
> An efficient change management board approves "low overhead" changes
> automatically/very quickly, and focusses on the 10% of changes which
> could be disruptive (and what disruptive means changes from one
> environment to another) - for code it would be any patches that
> potentially conflict, anything that could cause regressions, add
> instability or uncertainty, and any feature which can be implemented
> multiple ways.
> 
> Not saying this would work - I have never seen an open source project
> implement a change management process for handling patches, and
> instinctively I agree with you Neil that it would be a lot of overhead,
> but it's an interesting thought exercise to think how it might work

ZMQ has a community process with a simple review process
and a "default YES" policy.
 http://rfc.zeromq.org/spec:22

[dpdk-dev] Beyond DPDK 2.0

[Neil Horman]

On Sun, Apr 26, 2015 at 09:29:13PM -0500, Jim Thompson wrote:
> 
> > On Apr 26, 2015, at 4:56 PM, Neil Horman  wrote:
> > 
> > On Sat, Apr 25, 2015 at 04:08:23PM +, Wiles, Keith wrote:
> >> 
> >> 
> >> On 4/25/15, 8:30 AM, "Marc Sune"  wrote:
> >> 
> >>> 
> >>> 
> >>> On 24/04/15 19:51, Matthew Hall wrote:
>  On Fri, Apr 24, 2015 at 12:39:47PM -0500, Jay Rolette wrote:
> > I can tell you that if DPDK were GPL-based, my company wouldn't be
> > using
> > it. I suspect we wouldn't be the only ones...
> > 
> > Jay
>  I could second this, from the past employer where I used it. Right now
>  I am
>  using it in an open source app, I have a bit of GPL here and there but
>  I'm
>  trying to get rid of it or confine it to separate address spaces, where
>  it
>  won't impact the core code written around DPDK, as I don't want to cause
>  headaches for any downstream users I attract someday.
>  
>  Hard-core GPL would not be possible for most. LGPL could be possible,
>  but I
>  don't think it could be worth the relicensing headache for that small
>  change.
>  
>  Instead we should make the patch process as easy as humanly possible so
>  people
>  are encouraged to send us the fixes and not cart them around their
>  companies
>  constantly.
> >> 
> >> +1 and besides the GPL or LGPL ship has sailed IMHO and we can not go back.
> > Actually, IANAL, but I think we can.  The BSD license allows us to fork and
> > relicense the code I think, under GPL or any other license.  I'm not 
> > advocating
> > for that mind you, just suggesting that its possible should it ever become
> > needed.
> 
> I, on the other hand, am fairly certain that you can not ?relicense BSD 
> licensed code under the GPL (or any other license).
> 
> Were this true at law, then the opposite would also be possible.  (?Don?t 
> like the license?  Just fork!?)
> 
Isn't that in effect, exactly what most of the end users of the DPDK do however?

The 3 clause BSD license states that:
1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation and/or
other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its contributors
may be used to endorse or promote products derived from this software without
specific prior written permission.

If you build a product based on the DPDK, and statically link it, I presume
there are at least some vendors that are redistributing DPDK binary code using a
non-BSD license?  It may be a de-facto rather than a de-jure relicensing, but
the end result is the same.  The only thing that the BSD license says is that
you have to reproduce this copyright notice if you distribute BSD code.  Theres
nothing that says you can't add further copyright to derivations on that code
that you make.  IANAL, but it seems like this is done quite often.

Regardless, the canonical way to relicense code of course is have all the
copyright holder agree to relicense it under some other license.  Currently as
it stands a quick git scan indicates that 118 individuals are responsible for
the most recent change to every line of code in the dpdk (using git blame on
every file).  Of those 118 fully half belong to intel, or 6wind.  It wouldn't be
hard to hold a meeting and generate an agreement to relicense.

Of course, that won't happen.  BSD licensing is the desired solution here.
Neil

[dpdk-dev] Beyond DPDK 2.0

[Stephen Hemminger]

On Sun, 26 Apr 2015 21:29:13 -0500
Jim Thompson  wrote:

> I, on the other hand, am fairly certain that you can not ?relicense BSD 
> licensed code under the GPL (or any other license).
> 
> Were this true at law, then the opposite would also be possible.  (?Don?t 
> like the license?  Just fork!?)


IANAL but it is possible to go from BSD to GPL and it is done all the time.
The other way is not possible. You only have to see the OpenOffice -> 
LibreOffice
as an example.  LibreOffice can freely take anything out of OpenOffice but
OpenOffice can not take from LibrOffice.

[dpdk-dev] Beyond DPDK 2.0

[Jay Rolette]

On Sat, Apr 25, 2015 at 7:10 AM, Neil Horman  wrote:

> On Fri, Apr 24, 2015 at 02:55:33PM -0500, Jay Rolette wrote:
> > On Fri, Apr 24, 2015 at 1:51 PM, Neil Horman 
> wrote:
> >
> > > So, I hear your arguments, and its understandable that you might not
> want
> > > a GPL
> > > licensed product, given that the DPDK is a library (though I'm not sure
> > > what the
> > > aversion to LGPL would be).  Regardless, I think this conversation is a
> > > bit more
> > > about participation than license choice.  While you are correct, in
> that
> > > the
> > > first step to support (by which I presume you mean participation in the
> > > community) is use, the goal here is to get people contributing patches
> and
> > > helping increase the usefulness of DPDK.
> >
> >
> > > Given that DPDK is primarily licensed as BSD now, whats preventing
> you, or
> > > what
> > > would encourage you to participate in the community?  I see emails from
> > > infiniteio addresss in the archives asking questions and making
> > > suggestions on
> > > occasion, but no patches.  What would get you (or others in a simmilar
> > > situation) to submit those?
> > >
> >
> > 36 hours in the day? :)
> >
> > It's not a lot, but we've submitted a couple of small patches. It's
> mostly
> > a matter of opportunity. We submit patches as we come across DPDK bugs or
> > find useful optos.
> >
>


>
> Understand, I'm not trying to single you out here.  I see that you, and
> others
> from infiniteio have had some participation in the project, and thats
> great, and
> appreciated. I'm more focused on why that level of participation is not
> higher
> (not only from you, but from the larger presumed user base in general).  As
> noted at the start of this thread, one of the goals of this discussion is
> to
> find ways to maximize participation in the project, and from you I'm
> hearing
> that:
>
> 1) you use the dpdk because it lowers maintenence overhead
> 2) you don't participate more in the project because your product work
> schedule
> doesn't allow you to do so.
>
> Are those both accurate statements?
>

(1) was just my response to Luke about what would motivate a company to
submit patches if the license didn't require it (GPL vs BSD discussion).
Maint overhead had little to do with why we decided to use DPDK.

(2) is certainly true enough, but not really all that big of a factor in
the reasons why our participation isn't at some higher level. I'd say there
are two primary factors:

*New Contributors*
Dropping a major patch on a project where we have no history would be
foolish and frustrating. Every open source project has a vibe to it and its
own way of operating. It normally takes some time to figure that out, but
for major contributions, it generally involves a level of trust.

Major code drops or patches aren't generally well received unless it is
from someone that is known and trusted by the community. Building up that
trust ("street cred") normally involves participating in smaller, less
risky ways. Answering questions where you can, small patches to fix bugs,
possibly reviewing code (although this can be iffy as well), etc.

This facilitates understanding the process, who the players are and what
sort of toes you are likely to step on.

It also gives you time to ramp up on the internals of the code and
philosophy/goals of the project better. With DPDK, performance is obviously
more important than portability. Until recently, very little care was given
to API stability or the impact that has on DPDK apps. Both of those are
very different approaches than typical and it affects what you might do
when approaching submitting a patch.

If you want to build up the number of folks contributing, expect them to
start small and make sure it actually GOES somewhere.

The first patch I submitted in mid-December had some quick responses and
questions back-and-forth, but then stalled on a couple of undocumented
minor stylistic things (comment style and use of #ifdefs). I never heard
anything back and 4.5 months later, a simple startup opto is still sitting
there.

The second patch I submitted (also mid-December) got no response at all for
4 months. I've replied to the feedback that came eventually, but once
again, no subsequent answers.

Neither of the patches were important, but the process doesn't exactly
inspire a strong desire to contribute more.

*Different Goals*
I see at least two different sets of people on the mailing list. The heavy
hitters generally have a product reason for their high level of involvement
with DPDK. For Intel and 6Wind, the reasons are pretty obvious. Same deal
with NIC vendors. For large companies, sometimes the reasons are less
obvious, but supporting certain open source projects can be strategic for
them for several reasons.

For this group, improving DPDK itself is important enough to dedicate
resources to. It's a goal in and of itself, even if it isn't the main goal.

The other group are what I'd call "DPDK users". They 

[dpdk-dev] Beyond DPDK 2.0

[Wiles, Keith]

On 4/26/15, 4:56 PM, "Neil Horman"  wrote:

>On Sat, Apr 25, 2015 at 04:08:23PM +, Wiles, Keith wrote:
>> 
>> 
>> On 4/25/15, 8:30 AM, "Marc Sune"  wrote:
>> 
>> >
>> >
>> >On 24/04/15 19:51, Matthew Hall wrote:
>> >> On Fri, Apr 24, 2015 at 12:39:47PM -0500, Jay Rolette wrote:
>> >>> I can tell you that if DPDK were GPL-based, my company wouldn't be
>> >>>using
>> >>> it. I suspect we wouldn't be the only ones...
>> >>>
>> >>> Jay
>> >> I could second this, from the past employer where I used it. Right
>>now
>> >>I am
>> >> using it in an open source app, I have a bit of GPL here and there
>>but
>> >>I'm
>> >> trying to get rid of it or confine it to separate address spaces,
>>where
>> >>it
>> >> won't impact the core code written around DPDK, as I don't want to
>>cause
>> >> headaches for any downstream users I attract someday.
>> >>
>> >> Hard-core GPL would not be possible for most. LGPL could be possible,
>> >>but I
>> >> don't think it could be worth the relicensing headache for that small
>> >>change.
>> >>
>> >> Instead we should make the patch process as easy as humanly possible
>>so
>> >>people
>> >> are encouraged to send us the fixes and not cart them around their
>> >>companies
>> >> constantly.
>> 
>> +1 and besides the GPL or LGPL ship has sailed IMHO and we can not go
>>back.
>Actually, IANAL, but I think we can.  The BSD license allows us to fork
>and
>relicense the code I think, under GPL or any other license.  I'm not
>advocating
>for that mind you, just suggesting that its possible should it ever become
>needed.
>
>> >
>> >I agree. My feeling is that as the number of patches in the mailing
>>list
>> >grows, keeping track of them gets more and more complicated. Patchwork
>> >website was a way to try to address this issue. I think it was an
>> >improvement, but to be honest, patchwork lacks a lot of functionality,
>> >such as properly tracking multiple versions of the patch (superseding
>> >them automatically), and it lacks some filtering capabilities e.g. per
>> >user, per tag/label or library, automatically track if it has been
>> >merged, give an overall status of the pending vs merged patches, set
>> >milestones... Is there any alternative tool or improved version for
>>that?
>> 
>Agreed, this has come up before, off list unfortunately.  The volume of
>patches
>seems to be increasing at such a rate that a single maintainer has
>difficulty
>keeping up.  I proposed that the workload be split out to multiple
>subtrees,
>with prefixes being added to patch subjects on the list for local
>filtering to
>stem the tide.  Specifically I had proposed that the PMD's be split into a
>separate subtree, but that received pushback in favor of having each
>library
>having its own separate subtree, with a pilot program being made out of
>the I40e
>driver (which you might note sends pull requests to the list now).  I'd
>still
>like to see all PMD's come under a single subtree, but thats likely an
>argument
>for later.
>
>That said, Do you think that this patch latency is really a contributor
>to low
>project participation?  It definately a problem, but it seems to me that
>this
>sort of issue would lead to people trying to parcitipate, then giving up
>(i.e.
>we would see 1-2 emails from an individual, then not see them again).
>I'd need
>to look through the mailing list for such a pattern, but anecdotally I've
>not
>seen that happen.  The problem you describe above is definately a
>problem, but
>its one for those individuals who are participating, not for those who are
>simply choosing not to.  And I think we need to address both.
>
>> I agree patchwork has some limitation, but I think the biggest issue is
>> keeping up with the patches. Getting patches introduced into the main
>>line
>> is very slow. A patch submitted today may not get applied for weeks or
>> months, then when another person submits a patch he is starting to run a
>> very high risk of having to redo that patch, because a pervious patch
>> makes his fail weeks/months later. I would love to see a better tool
>>then
>> patchwork, but the biggest issue is we have a huge backlog of patches.
>> Personally I am not sure how Thomas or any is able to keep up with the
>> patches.
>> 
>This is absolutely a problem.  I'd like to think, more than a tool like
>patchwork, a subtree organization to allow some modicum of parallel
>review and
>integration would really be a benefit here.
Subtrees could work, but the real problem I think is the number of
committers must be higher then one. Something like GitHub (and I assume
Linux Foundation) have a method to add committers to a project. In the
case of GitHub they just have to have a free GitHub account and they can
become committers of the project buying the owner of the project enables
them.

On GitHub they have personal accounts and organization accounts I know
only about the personal accounts, but they allow for 5 private repos and
any number of public repos. The organization account has a lot 

[dpdk-dev] Beyond DPDK 2.0

[Neil Horman]

On Sat, Apr 25, 2015 at 04:08:23PM +, Wiles, Keith wrote:
> 
> 
> On 4/25/15, 8:30 AM, "Marc Sune"  wrote:
> 
> >
> >
> >On 24/04/15 19:51, Matthew Hall wrote:
> >> On Fri, Apr 24, 2015 at 12:39:47PM -0500, Jay Rolette wrote:
> >>> I can tell you that if DPDK were GPL-based, my company wouldn't be
> >>>using
> >>> it. I suspect we wouldn't be the only ones...
> >>>
> >>> Jay
> >> I could second this, from the past employer where I used it. Right now
> >>I am
> >> using it in an open source app, I have a bit of GPL here and there but
> >>I'm
> >> trying to get rid of it or confine it to separate address spaces, where
> >>it
> >> won't impact the core code written around DPDK, as I don't want to cause
> >> headaches for any downstream users I attract someday.
> >>
> >> Hard-core GPL would not be possible for most. LGPL could be possible,
> >>but I
> >> don't think it could be worth the relicensing headache for that small
> >>change.
> >>
> >> Instead we should make the patch process as easy as humanly possible so
> >>people
> >> are encouraged to send us the fixes and not cart them around their
> >>companies
> >> constantly.
> 
> +1 and besides the GPL or LGPL ship has sailed IMHO and we can not go back.
Actually, IANAL, but I think we can.  The BSD license allows us to fork and
relicense the code I think, under GPL or any other license.  I'm not advocating
for that mind you, just suggesting that its possible should it ever become
needed.

> >
> >I agree. My feeling is that as the number of patches in the mailing list
> >grows, keeping track of them gets more and more complicated. Patchwork
> >website was a way to try to address this issue. I think it was an
> >improvement, but to be honest, patchwork lacks a lot of functionality,
> >such as properly tracking multiple versions of the patch (superseding
> >them automatically), and it lacks some filtering capabilities e.g. per
> >user, per tag/label or library, automatically track if it has been
> >merged, give an overall status of the pending vs merged patches, set
> >milestones... Is there any alternative tool or improved version for that?
> 
Agreed, this has come up before, off list unfortunately.  The volume of patches
seems to be increasing at such a rate that a single maintainer has difficulty
keeping up.  I proposed that the workload be split out to multiple subtrees,
with prefixes being added to patch subjects on the list for local filtering to
stem the tide.  Specifically I had proposed that the PMD's be split into a
separate subtree, but that received pushback in favor of having each library
having its own separate subtree, with a pilot program being made out of the I40e
driver (which you might note sends pull requests to the list now).  I'd still
like to see all PMD's come under a single subtree, but thats likely an argument
for later.

That said, Do you think that this patch latency is really a contributor to low
project participation?  It definately a problem, but it seems to me that this
sort of issue would lead to people trying to parcitipate, then giving up (i.e.
we would see 1-2 emails from an individual, then not see them again).  I'd need
to look through the mailing list for such a pattern, but anecdotally I've not
seen that happen.  The problem you describe above is definately a problem, but
its one for those individuals who are participating, not for those who are
simply choosing not to.  And I think we need to address both.

> I agree patchwork has some limitation, but I think the biggest issue is
> keeping up with the patches. Getting patches introduced into the main line
> is very slow. A patch submitted today may not get applied for weeks or
> months, then when another person submits a patch he is starting to run a
> very high risk of having to redo that patch, because a pervious patch
> makes his fail weeks/months later. I would love to see a better tool then
> patchwork, but the biggest issue is we have a huge backlog of patches.
> Personally I am not sure how Thomas or any is able to keep up with the
> patches.
> 
This is absolutely a problem.  I'd like to think, more than a tool like
patchwork, a subtree organization to allow some modicum of parallel review and
integration would really be a benefit here.

> The other problem I see is how patches are agreed on to be included in the
> mainline. Today it is just an ACK or a NAK on the mailing list. Then I see
> what I think to be only a few people ACKing or NAKing patches. This
> process has a lot of problems from a patch being ignore for some reason or
> someone having negative feed back on very minor detail or no way to push a
> patch forward a single NAK or comment.
> 

So, this is an interesting issue in ideal meritocracies.  Currently is/should be
looking for ACKs/NAK/s from the individuals listed in the MAINTAINER files, and
those people should be the definitive subject matter experts on the code they
cover.  As such, I would agrue that they should be entitled to a modicum 

[dpdk-dev] Beyond DPDK 2.0

[Luke Gorrie]

Hi Neil,

Thanks for taking the time to reflect on my ideas.

On 24 April 2015 at 19:00, Neil Horman  wrote:

> DPDK will always be
> something of a niche market for user to whoom every last ounce of
> performance is
> the primary requirement


This does seem like an excellent position. It is succinct, it sets
expectations for users, and it tells developers how to resolve trade-offs
(performance takes priority over FOO, for all values of FOO). I agree that
this niche will always be there and so it seems like there is a permanent
place in the world for DPDK.

This focus on performance also makes DPDK useful as a reference for other
projects. People making trade-offs between performance and other factors
(portability, compatibility, simplicity, etc) can use DPDK as a yardstick
to estimate what this costs. This benefits everybody doing networking on
x86.

I suppose that a separate discussion would be how to increase participation
from people who are using DPDK as a reference but not as a software
dependency. That is perhaps a less pressing topic for the future.

OVS is a great example here.  If we can make it easy for them to use DPDK
> to get
> better performance, I think we'll see a larger uptake in adoption.
>

I will be interested to see how this plays out.

I agree it is a great opportunity for DPDK and a chance to take it
mainstream.

I also think it is fundamentally a missed opportunity of the kernel. OVS
would be just fine with a kernel data plane that performs adequately. OVS
users don't seem to be in the "maximum performance at any cost" niche
defined above. Many of them benefit a lot from the kernel integration.
However, if the kernel can't promise the meet their performance
requirements then DPDK does seem like a knight in shining armour.

It's an exciting time in open source networking :-)

Cheers,
-Luke

[dpdk-dev] Beyond DPDK 2.0

[Wiles, Keith]

On 4/25/15, 8:30 AM, "Marc Sune"  wrote:

>
>
>On 24/04/15 19:51, Matthew Hall wrote:
>> On Fri, Apr 24, 2015 at 12:39:47PM -0500, Jay Rolette wrote:
>>> I can tell you that if DPDK were GPL-based, my company wouldn't be
>>>using
>>> it. I suspect we wouldn't be the only ones...
>>>
>>> Jay
>> I could second this, from the past employer where I used it. Right now
>>I am
>> using it in an open source app, I have a bit of GPL here and there but
>>I'm
>> trying to get rid of it or confine it to separate address spaces, where
>>it
>> won't impact the core code written around DPDK, as I don't want to cause
>> headaches for any downstream users I attract someday.
>>
>> Hard-core GPL would not be possible for most. LGPL could be possible,
>>but I
>> don't think it could be worth the relicensing headache for that small
>>change.
>>
>> Instead we should make the patch process as easy as humanly possible so
>>people
>> are encouraged to send us the fixes and not cart them around their
>>companies
>> constantly.

+1 and besides the GPL or LGPL ship has sailed IMHO and we can not go back.
>
>I agree. My feeling is that as the number of patches in the mailing list
>grows, keeping track of them gets more and more complicated. Patchwork
>website was a way to try to address this issue. I think it was an
>improvement, but to be honest, patchwork lacks a lot of functionality,
>such as properly tracking multiple versions of the patch (superseding
>them automatically), and it lacks some filtering capabilities e.g. per
>user, per tag/label or library, automatically track if it has been
>merged, give an overall status of the pending vs merged patches, set
>milestones... Is there any alternative tool or improved version for that?

I agree patchwork has some limitation, but I think the biggest issue is
keeping up with the patches. Getting patches introduced into the main line
is very slow. A patch submitted today may not get applied for weeks or
months, then when another person submits a patch he is starting to run a
very high risk of having to redo that patch, because a pervious patch
makes his fail weeks/months later. I would love to see a better tool then
patchwork, but the biggest issue is we have a huge backlog of patches.
Personally I am not sure how Thomas or any is able to keep up with the
patches.

The other problem I see is how patches are agreed on to be included in the
mainline. Today it is just an ACK or a NAK on the mailing list. Then I see
what I think to be only a few people ACKing or NAKing patches. This
process has a lot of problems from a patch being ignore for some reason or
someone having negative feed back on very minor detail or no way to push a
patch forward a single NAK or comment.

I would like to see some type of layering process to allow patches to be
applied in a timely manner a few weeks not months or completely ignored.
Maybe some type of voting is reasonable, but we need to do something to
turn around the patches in clean reasonable manner.

Think we need some type of group meeting every week to look at the patches
and determining which ones get applied, this gives quick feedback to the
submitter as to the status of the patch.

>
>On the other side, since user questions, community discussions and
>development happens in the same mailing list, things get really
>complicated, specially for users seeking for help. Even though I think
>the average skills of the users of DPDK is generally higher than in
>other software projects, if DPDK wants to attract more users, having a
>better user support is key, IMHO.
>
>So I would see with good eyes a separation between, at least, dpdk-user
>and dpdk-dev.

I do not remember seeing too many users on the list and making a list just
for then is OK if everyone is fine with a list that has very few emails.
>
>If the number of patches keeps growing, splitting the "dev" mailing
>lists into different categories (eal and common, pmds, higher level
>abstractions...) could be an option. However, this last point opens a
>lot of questions on how to minimize interference between the different
>parts and API/ABI compatibility during the development.

I believe if we just make sure we use tags in the subject line then we can
have our email clients do the splitting of the emails instead of adding
more emails lists.

>
>>
>> Perhaps it means having some ReviewBoard type of tools, a clone in
>>Github or
>> Bitbucket where the less hardcore kernel-workflow types could send back
>>their
>> small bug fixes a bit more easily, this kind of stuff. Google has been
>>getting
>> good uptake since they moved most of their open source across to Github,
>> because the contribution workflow was more convenient than Google Code
>>was.

I like GitHub it is a much better designed tool then patchwork, plus it
could get more eyes as it is very well know to the developer community in
general. I feel GitHub has many advantages over the current systems in
place but, it does not solve the all patch 

[dpdk-dev] Beyond DPDK 2.0

[Marc Sune]

On 24/04/15 19:51, Matthew Hall wrote:
> On Fri, Apr 24, 2015 at 12:39:47PM -0500, Jay Rolette wrote:
>> I can tell you that if DPDK were GPL-based, my company wouldn't be using
>> it. I suspect we wouldn't be the only ones...
>>
>> Jay
> I could second this, from the past employer where I used it. Right now I am
> using it in an open source app, I have a bit of GPL here and there but I'm
> trying to get rid of it or confine it to separate address spaces, where it
> won't impact the core code written around DPDK, as I don't want to cause
> headaches for any downstream users I attract someday.
>
> Hard-core GPL would not be possible for most. LGPL could be possible, but I
> don't think it could be worth the relicensing headache for that small change.
>
> Instead we should make the patch process as easy as humanly possible so people
> are encouraged to send us the fixes and not cart them around their companies
> constantly.

I agree. My feeling is that as the number of patches in the mailing list 
grows, keeping track of them gets more and more complicated. Patchwork 
website was a way to try to address this issue. I think it was an 
improvement, but to be honest, patchwork lacks a lot of functionality, 
such as properly tracking multiple versions of the patch (superseding 
them automatically), and it lacks some filtering capabilities e.g. per 
user, per tag/label or library, automatically track if it has been 
merged, give an overall status of the pending vs merged patches, set 
milestones... Is there any alternative tool or improved version for that?

On the other side, since user questions, community discussions and 
development happens in the same mailing list, things get really 
complicated, specially for users seeking for help. Even though I think 
the average skills of the users of DPDK is generally higher than in 
other software projects, if DPDK wants to attract more users, having a 
better user support is key, IMHO.

So I would see with good eyes a separation between, at least, dpdk-user 
and dpdk-dev.

If the number of patches keeps growing, splitting the "dev" mailing 
lists into different categories (eal and common, pmds, higher level 
abstractions...) could be an option. However, this last point opens a 
lot of questions on how to minimize interference between the different 
parts and API/ABI compatibility during the development.

>
> Perhaps it means having some ReviewBoard type of tools, a clone in Github or
> Bitbucket where the less hardcore kernel-workflow types could send back their
> small bug fixes a bit more easily, this kind of stuff. Google has been getting
> good uptake since they moved most of their open source across to Github,
> because the contribution workflow was more convenient than Google Code was.

Although I agree, we have to be careful on how github or bitbucket is 
used. Having issues or even (e.g. github) pull requests *in addition* to 
the normal contribution workflow can be a nightmare to deal with, in 
terms of synchronization and preventing double work. So I guess setting 
up an official github or bitbucket mirror would be fine, via some simple 
cronjob, but I guess it would end-up not using PRs or issues in github 
like the Linux kernel does.

Btw, is this github organization already registered by Intel or some 
other company of the community?

https://github.com/dpdk

Marc

>
> Matthew.

[dpdk-dev] Beyond DPDK 2.0

[Neil Horman]

On Fri, Apr 24, 2015 at 02:55:33PM -0500, Jay Rolette wrote:
> On Fri, Apr 24, 2015 at 1:51 PM, Neil Horman  wrote:
> 
> > So, I hear your arguments, and its understandable that you might not want
> > a GPL
> > licensed product, given that the DPDK is a library (though I'm not sure
> > what the
> > aversion to LGPL would be).  Regardless, I think this conversation is a
> > bit more
> > about participation than license choice.  While you are correct, in that
> > the
> > first step to support (by which I presume you mean participation in the
> > community) is use, the goal here is to get people contributing patches and
> > helping increase the usefulness of DPDK.
> 
> 
> > Given that DPDK is primarily licensed as BSD now, whats preventing you, or
> > what
> > would encourage you to participate in the community?  I see emails from
> > infiniteio addresss in the archives asking questions and making
> > suggestions on
> > occasion, but no patches.  What would get you (or others in a simmilar
> > situation) to submit those?
> >
> 
> 36 hours in the day? :)
> 
> It's not a lot, but we've submitted a couple of small patches. It's mostly
> a matter of opportunity. We submit patches as we come across DPDK bugs or
> find useful optos.
> 
> *Patches*
> 
>- replaced O(n^2) sort in sort_by_physaddr() with qsort() from standard
>library 
>- Fixed spam from kni_allocate_mbufs() when no mbufs are free. If mbufs
>exhausted, 'out of memory' message logged at EXTREMELY high rates. Now logs
>no more than once per 10 mins 
> 
> *Reviews*
> 
>- kni: optimizing the rte_kni_rx_burst
>
>- [PATCH RFC] librte_reorder: new reorder library
>
>- [PATCH v2 09/17] i40e: clean log messages
> (several in
>that series, but I figure 1 link is plenty)
> 
> *Other*
> Not really patches or reviews, but trying to participate in the community:
> 
>- VMware Fusion + DPDK and KNI
>
>- Appropriate DPDK data structures for TCP sockets
>
>- kernel: BUG: soft lockup - CPU#1 stuck for 22s! [kni_single:1782]
>
>- segmented recv ixgbevf
>
> 
> Jay

Understand, I'm not trying to single you out here.  I see that you, and others
from infiniteio have had some participation in the project, and thats great, and
appreciated. I'm more focused on why that level of participation is not higher
(not only from you, but from the larger presumed user base in general).  As
noted at the start of this thread, one of the goals of this discussion is to
find ways to maximize participation in the project, and from you I'm hearing
that:

1) you use the dpdk because it lowers maintenence overhead
2) you don't participate more in the project because your product work schedule
doesn't allow you to do so.

Are those both accurate statements?

Can we also assume, for the sake of discussion that you have encountered
problems, or desired additions to the DPDK, for which you have implemented your
own code in the library that is not contributed back to the DPDK? 

If that is true, perhaps part of this conversation needs to revolve around the
tangible benefits of contributing that code back to the upstream project (the
aforementioned reduction of overhead) as well as the intangible (thought
leadership as the project develops).  Its been my experience, that these
situations often arise because management at a company often places a strong
bias on development of their product over participation in the open source
portion of it, treating the latter as if they are a customer of it, rather than
a participant in it, and it would be my desire to see that bias adjusted so as
to allow you greater freedom to participate in the project.

Would you agree to that assessment?  If so, how would you suggest that we, as a
community address this, and illustrate the appeal of contribution and
participation to your company and the benefits thereof?

Best
Neil

[dpdk-dev] Beyond DPDK 2.0

[Dave Neary]

Hi Tim,

On 04/23/2015 07:36 AM, O'Driscoll, Tim wrote:
>> Alternatively, propose some options and vote, but I don't think we have 
>> things defined
>> enough for that yet.
> 
> We tried to keep the initial communication neutral and avoid suggesting 
> solutions to give others a chance to comment. At a very high level, there 
> seem to be 3 possible approaches though:
> 
> 1. Do nothing. The project is increasing in size, and the releases are 
> getting delivered according to the roadmap, so one option is to continue as 
> we are.
> 
> 2. Add a more formal governance structure to dpdk.org. This might involve 
> putting in place a Technical Steering Committee to give long-term technical 
> direction to the project, and to resolve any differences of opinion that 
> don't reach a conclusion on the mailing list.
> 
> 3. Transition the project to an organization such as the Linux Foundation, 
> and use their help to implement a more formal governance structure. This 
> would probably involve a TSC, and possibly also a governing board and the 
> creation of some form of centralized marketing/branding/promotional budget 
> for the project.

I see at least one other option, which is to document the process for
becoming a maintainer/core reviewer (whatever terminology we choose),
and move from a single project lead to multiple committers. This would
allow the project to scale, reducing average review time and removing
bottlenecks, but would avoid the potential for "design by committee"
which a TSC would bring, and also avoid the operational and cost
overhead of a formal foundation.

This will not address the issue of how the project's scope, priorities
and roadmap are defined, but will definitely help with both the scaling
of project contributions and the diversity of the project.

The MAINTAINERS file: http://dpdk.org/browse/dpdk/tree/MAINTAINERS is an
awesome step in the right direction by clearly defining where people
maintain a module.

Regards,
Dave.

-- 
Dave Neary - NFV/SDN Community Strategy
Open Source and Standards, Red Hat - http://community.redhat.com
Ph: +1-978-399-2182 / Cell: +1-978-799-3338

[dpdk-dev] Beyond DPDK 2.0

[O'Driscoll, Tim]

> From: lukego at gmail.com [mailto:lukego at gmail.com] On Behalf Of Luke 
> Gorrie
>
> > On 16 April 2015 at 12:38, O'Driscoll, Tim  
> > wrote:
> > Following the launch of DPDK by Intel as an internal development project, 
> > the launch of dpdk.org by
> > 6WIND in 2013, and the first DPDK RPM packages for Fedora in 2014, 6WIND, 
> > Red Hat and Intel would
> > like to prepare for future releases after DPDK 2.0 by starting a discussion 
> > on its evolution. Anyone
> > is welcome to join this initiative.
>
> Thank you for the open invitation.
> 
> I have a couple of questions about the long term of DPDK:
> 
> 1. How will DPDK manage overlap with other project over time?
> 
> In some ways DPDK is growing more overlap with other projects e.g. 
> forking/rewriting functionality from
> Linux (e.g. ixgbe), FreeBSD (e.g. Broadcom PMD), GLIBC (e.g. memcpy).
> 
> In other ways DPDK is delegating functionality to external systems instead 
> e.g. the bifurcated driver
> (delegate to kernel) and Mellanox PMD (delegate to vendor shared library).
> 
> How is this going to play out over the long term? And is there an existential 
> risk that it will end up
> being easier to port the good bits of DPDK into the kernel than the rest of 
> the good bits of the kernel
> into DPDK?

Good question. I don't have a good answer to this, but it is something we will 
need to consider. Perhaps others have opinions?

> 2. How will DPDK users justify contributing to DPDK upstream?
> 
> Engineers in network equipment vendors want to contribute to open source, but 
> what is the incentive for
> the companies to support this? This would be easy if DPDK were GPL'd (they 
> are compelled) or if everybody
> were dynamically linking with the upstream libdpdk (can't have private 
> patches). However, in a world where
> DPDK is BSD-licensed and statically linked, is it not both cheaper and 
> competitively advantageous to keep
> fixes and optimizations in house?
> 
> Today the community is benefiting immensely from the contributions of 
> companies like 6WIND and Brocade,
> but I wonder if this going to be the exception or the rule.

That's another good question. Expanding the community and soliciting more 
contributions is one of the reasons for initiating this discussion.

At first glance, it can seem cheaper and competitively advantageous for people 
to keep DPDK enhancements/optimisations in house. However, that's not 
necessarily the case. There is an advantage in upstreaming these changes 
because firstly others in the community may contribute further enhancements, 
and also because it makes upgrading to new DPDK versions easier because those 
enhancements will be a core part of DPDK rather than having to be ported 
separately to new DPDK versions.

> That's all from me. Thanks for listening :-).

Thanks for contributing your views.

[dpdk-dev] Beyond DPDK 2.0

[Jay Rolette]

On Fri, Apr 24, 2015 at 1:51 PM, Neil Horman  wrote:

> So, I hear your arguments, and its understandable that you might not want
> a GPL
> licensed product, given that the DPDK is a library (though I'm not sure
> what the
> aversion to LGPL would be).  Regardless, I think this conversation is a
> bit more
> about participation than license choice.  While you are correct, in that
> the
> first step to support (by which I presume you mean participation in the
> community) is use, the goal here is to get people contributing patches and
> helping increase the usefulness of DPDK.


> Given that DPDK is primarily licensed as BSD now, whats preventing you, or
> what
> would encourage you to participate in the community?  I see emails from
> infiniteio addresss in the archives asking questions and making
> suggestions on
> occasion, but no patches.  What would get you (or others in a simmilar
> situation) to submit those?
>

36 hours in the day? :)

It's not a lot, but we've submitted a couple of small patches. It's mostly
a matter of opportunity. We submit patches as we come across DPDK bugs or
find useful optos.

*Patches*

   - replaced O(n^2) sort in sort_by_physaddr() with qsort() from standard
   library 
   - Fixed spam from kni_allocate_mbufs() when no mbufs are free. If mbufs
   exhausted, 'out of memory' message logged at EXTREMELY high rates. Now logs
   no more than once per 10 mins 

*Reviews*

   - kni: optimizing the rte_kni_rx_burst
   
   - [PATCH RFC] librte_reorder: new reorder library
   
   - [PATCH v2 09/17] i40e: clean log messages
    (several in
   that series, but I figure 1 link is plenty)

*Other*
Not really patches or reviews, but trying to participate in the community:

   - VMware Fusion + DPDK and KNI
   
   - Appropriate DPDK data structures for TCP sockets
   
   - kernel: BUG: soft lockup - CPU#1 stuck for 22s! [kni_single:1782]
   
   - segmented recv ixgbevf
   

Jay

[dpdk-dev] Beyond DPDK 2.0

[Neil Horman]

On Fri, Apr 24, 2015 at 01:12:13PM -0500, Matt Laswell wrote:
> On Fri, Apr 24, 2015 at 12:39 PM, Jay Rolette 
> wrote:
> >
> > I can tell you that if DPDK were GPL-based, my company wouldn't be using
> > it. I suspect we wouldn't be the only ones...
> >
> 
> I want to emphasize this point.  It's unsurprising that Jay and I agree,
> since we work together.  But I can say with quite a bit of confidence that
> my last employer also would stop using DPDK if it were GPL licensed.   Or,
> if they didn't jettison it entirely, they would never move beyond the last
> BSD-licensed version.  If you want to incentivize companies to support
> DPDK, the first step is to ensure they're using it.  For that reason, GPL
> seems like a step in the wrong direction to me.
> 
> - Matt
> 

So, I hear your arguments, and its understandable that you might not want a GPL
licensed product, given that the DPDK is a library (though I'm not sure what the
aversion to LGPL would be).  Regardless, I think this conversation is a bit more
about participation than license choice.  While you are correct, in that the
first step to support (by which I presume you mean participation in the
community) is use, the goal here is to get people contributing patches and
helping increase the usefulness of DPDK.

Given that DPDK is primarily licensed as BSD now, whats preventing you, or what
would encourage you to participate in the community?  I see emails from
infiniteio addresss in the archives asking questions and making suggestions on
occasion, but no patches.  What would get you (or others in a simmilar
situation) to submit those?


Neil

[dpdk-dev] Beyond DPDK 2.0

[Matt Laswell]

On Fri, Apr 24, 2015 at 12:39 PM, Jay Rolette 
wrote:
>
> I can tell you that if DPDK were GPL-based, my company wouldn't be using
> it. I suspect we wouldn't be the only ones...
>

I want to emphasize this point.  It's unsurprising that Jay and I agree,
since we work together.  But I can say with quite a bit of confidence that
my last employer also would stop using DPDK if it were GPL licensed.   Or,
if they didn't jettison it entirely, they would never move beyond the last
BSD-licensed version.  If you want to incentivize companies to support
DPDK, the first step is to ensure they're using it.  For that reason, GPL
seems like a step in the wrong direction to me.

- Matt

[dpdk-dev] Beyond DPDK 2.0

[Neil Horman]

On Fri, Apr 24, 2015 at 03:29:01PM +, O'Driscoll, Tim wrote:
> 
> > From: lukego at gmail.com [mailto:lukego at gmail.com] On Behalf Of Luke 
> > Gorrie
> >
> > > On 16 April 2015 at 12:38, O'Driscoll, Tim  
> > > wrote:
> > > Following the launch of DPDK by Intel as an internal development project, 
> > > the launch of dpdk.org by
> > > 6WIND in 2013, and the first DPDK RPM packages for Fedora in 2014, 6WIND, 
> > > Red Hat and Intel would
> > > like to prepare for future releases after DPDK 2.0 by starting a 
> > > discussion on its evolution. Anyone
> > > is welcome to join this initiative.
> >
> > Thank you for the open invitation.
> > 
> > I have a couple of questions about the long term of DPDK:
> > 
> > 1. How will DPDK manage overlap with other project over time?
> > 
> > In some ways DPDK is growing more overlap with other projects e.g. 
> > forking/rewriting functionality from
> > Linux (e.g. ixgbe), FreeBSD (e.g. Broadcom PMD), GLIBC (e.g. memcpy).
> > 
> > In other ways DPDK is delegating functionality to external systems instead 
> > e.g. the bifurcated driver
> > (delegate to kernel) and Mellanox PMD (delegate to vendor shared library).
> > 
> > How is this going to play out over the long term? And is there an 
> > existential risk that it will end up
> > being easier to port the good bits of DPDK into the kernel than the rest of 
> > the good bits of the kernel
> > into DPDK?
> 
> Good question. I don't have a good answer to this, but it is something we 
> will need to consider. Perhaps others have opinions?
> 

This is a good question, and I know efforts are underway to that end.  I can
tell you for certain that:

1) Many DPDK enhancement (tx batching for instance) are being investigated for
kernel integration

2) The DPDK will never be backported in its entirety to the kernel (too many x86
specific bits/optimizations), and api requirements.

As such, I think you'll find that the kernel will approach, but never quite
reach the performance goals of the DPDK. As such I think the DPDK will always be
something of a niche market for user to whoom every last ounce of performance is
the primary requirement (even above and beyond compatibility with standard OS
interfaces), but it will continue to exist independently as a project, as that
user demographic is out there.

> > 2. How will DPDK users justify contributing to DPDK upstream?
> > 
> > Engineers in network equipment vendors want to contribute to open source, 
> > but what is the incentive for
> > the companies to support this? This would be easy if DPDK were GPL'd (they 
> > are compelled) or if everybody
> > were dynamically linking with the upstream libdpdk (can't have private 
> > patches). However, in a world where
> > DPDK is BSD-licensed and statically linked, is it not both cheaper and 
> > competitively advantageous to keep
> > fixes and optimizations in house?
> > 
> > Today the community is benefiting immensely from the contributions of 
> > companies like 6WIND and Brocade,
> > but I wonder if this going to be the exception or the rule.
> 
> That's another good question. Expanding the community and soliciting more 
> contributions is one of the reasons for initiating this discussion.
> 

Well, BSD licensing is a guarantee that users won't contribute back to the
project (look at openssh for an example).  That said, the space that DPDK
operates in does tend to make a it a building block for larger applications that
are not of general use, by implementors that are not tradionally inclined to
contrubute back to the community.  I think what we need here is a 'killer-app'
that makes the DPDK relevant to developers who are biased toward an open source
mindset.  Rather than a telco app that sits on one vendors hardware, never to be
used by other telcos, we should support and encourage the participation of open
projects who might benefit from using DPDK to accelerate network functionality.
OVS is a great example here.  If we can make it easy for them to use DPDK to get
better performance, I think we'll see a larger uptake in adoption.

Neil

> At first glance, it can seem cheaper and competitively advantageous for 
> people to keep DPDK enhancements/optimisations in house. However, that's not 
> necessarily the case. There is an advantage in upstreaming these changes 
> because firstly others in the community may contribute further enhancements, 
> and also because it makes upgrading to new DPDK versions easier because those 
> enhancements will be a core part of DPDK rather than having to be ported 
> separately to new DPDK versions.
> 
> > That's all from me. Thanks for listening :-).
> 
> Thanks for contributing your views.
> 

[dpdk-dev] Beyond DPDK 2.0

[Jay Rolette]

On Fri, Apr 24, 2015 at 2:47 AM, Luke Gorrie  wrote:

> 2. How will DPDK users justify contributing to DPDK upstream?
>
> Engineers in network equipment vendors want to contribute to open source,
> but what is the incentive for the companies to support this? This would be
> easy if DPDK were GPL'd (they are compelled) or if everybody were
> dynamically linking with the upstream libdpdk (can't have private patches).
> However, in a world where DPDK is BSD-licensed and statically linked, is it
> not both cheaper and competitively advantageous to keep fixes and
> optimizations in house?
>

The main incentive for most companies to support it is that it reduces
their maintenance load. It makes it easier to not get "stuck" on a
particular version of DPDK and they don't have to waste time constantly
back-porting improvements and bug fixes.

I can tell you that if DPDK were GPL-based, my company wouldn't be using
it. I suspect we wouldn't be the only ones...

Jay

[dpdk-dev] Beyond DPDK 2.0

[Matthew Hall]

On Fri, Apr 24, 2015 at 12:39:47PM -0500, Jay Rolette wrote:
> I can tell you that if DPDK were GPL-based, my company wouldn't be using
> it. I suspect we wouldn't be the only ones...
> 
> Jay

I could second this, from the past employer where I used it. Right now I am 
using it in an open source app, I have a bit of GPL here and there but I'm 
trying to get rid of it or confine it to separate address spaces, where it 
won't impact the core code written around DPDK, as I don't want to cause 
headaches for any downstream users I attract someday.

Hard-core GPL would not be possible for most. LGPL could be possible, but I 
don't think it could be worth the relicensing headache for that small change.

Instead we should make the patch process as easy as humanly possible so people 
are encouraged to send us the fixes and not cart them around their companies 
constantly.

Perhaps it means having some ReviewBoard type of tools, a clone in Github or 
Bitbucket where the less hardcore kernel-workflow types could send back their 
small bug fixes a bit more easily, this kind of stuff. Google has been getting 
good uptake since they moved most of their open source across to Github, 
because the contribution workflow was more convenient than Google Code was.

Matthew.

[dpdk-dev] Beyond DPDK 2.0

[Luke Gorrie]

Hi Tim,

On 16 April 2015 at 12:38, O'Driscoll, Tim  wrote:

> Following the launch of DPDK by Intel as an internal development project,
> the launch of dpdk.org by 6WIND in 2013, and the first DPDK RPM packages
> for Fedora in 2014, 6WIND, Red Hat and Intel would like to prepare for
> future releases after DPDK 2.0 by starting a discussion on its evolution.
> Anyone is welcome to join this initiative.
>

Thank you for the open invitation.

I have a couple of questions about the long term of DPDK:

1. How will DPDK manage overlap with other project over time?

In some ways DPDK is growing more overlap with other projects e.g.
forking/rewriting functionality from Linux (e.g. ixgbe), FreeBSD (e.g.
Broadcom PMD), GLIBC (e.g. memcpy).

In other ways DPDK is delegating functionality to external systems instead
e.g. the bifurcated driver (delegate to kernel) and Mellanox PMD (delegate
to vendor shared library).

How is this going to play out over the long term? And is there an
existential risk that it will end up being easier to port the good bits of
DPDK into the kernel than the rest of the good bits of the kernel into DPDK?

2. How will DPDK users justify contributing to DPDK upstream?

Engineers in network equipment vendors want to contribute to open source,
but what is the incentive for the companies to support this? This would be
easy if DPDK were GPL'd (they are compelled) or if everybody were
dynamically linking with the upstream libdpdk (can't have private patches).
However, in a world where DPDK is BSD-licensed and statically linked, is it
not both cheaper and competitively advantageous to keep fixes and
optimizations in house?

Today the community is benefiting immensely from the contributions of
companies like 6WIND and Brocade, but I wonder if this going to be the
exception or the rule.

That's all from me. Thanks for listening :-).

Cheers,
-Luke

[dpdk-dev] Beyond DPDK 2.0

[Stephen Hemminger]

This does a good job of stating the need for action without getting into
the details.
Perhaps this would be better resolved by some more interactive discussion.
I know it is hard to all get together, but there needs to be more some more
creative and focused
thought on this. A phone conference is just not enough.

Alternatively, propose some options and vote, but I don't think we have
things defined
enough for that yet.