[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

2015-06-19 12:13, Thomas F Herbert:
> 
> On 6/19/15 9:16 AM, Thomas Monjalon wrote:
> > 2015-06-19 09:02, Neil Horman:
> >> On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> >>> 2015-06-19 06:26, Neil Horman:
>  On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote:
> > For the 2.1 release, I think we should agree to make patches that change
> > the ABI controllable via a compile-time option. I like Olivier's 
> > proposal
> > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> > changes instead of a separate option per patch set (see
> > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> > should rework the affected patch sets to use that approach for 2.1.
> 
>  This is a bad idea.  Making ABI dependent on compile time options isn't a
>  maintainable solution.  It breaks the notion of how LIBABIVER is 
>  supposed to
>  work (that is to say you make it impossible to really tell what ABI 
>  version you
>  are building).
> >>>
> >>> The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
> >>> So one ABI version number refers always to the same ABI.
> >>>
>  If you have two compile time options that modify the ABI, you
>  have to burn through 4 possible LIBABIVER version values to accomodate 
>  all
>  possible combinations, and then you need to remember that when you make 
>  them
>  statically applicable.
> >>>
> >>> The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
> >>>
> >>> Your intent when introducing ABI policy was to allow smooth porting of
> >>> applications from a DPDK version to another. Right?
> >>> The adopted solution was to provide backward compatibility during 1 
> >>> release.
> >>> But there are cases where it's not possible. So the policy was to notice
> >>> the future change and wait one release cycle to break the ABI (failing
> >>> compatibility goals).
> >>> The compile-time option may provide an alternative DPDK packaging when the
> >>> ABI backward compatibility cannot be provided (case of mbuf changes).
> >>> In such case, it's still possible to upgrade DPDK by providing 2 versions 
> >>> of
> >>> DPDK libs. So the existing apps continue to link with the previous ABI and
> >>> have the possibility of migrating to the new one.
> >>> Another advantage of this approach is that we don't have to wait 1 release
> >>> to integrate the changes.
> >>> The last advantage is to benefit early of these changes with static 
> >>> libraries.
> >>
> >> Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
> >> Implementing an ABI preview option like this implies the notion that, 
> >> after a
> >> release, you have to remove all the ifdefs that you inserted to create the 
> >> new
> >> ABI.  That seems like an easy task, but it becomes a pain when the ABI 
> >> delta is
> >> large, and is predicated on the centralization of work effort (that is to 
> >> say
> >> you need to identify someone to submit the 'remove the NEXT_ABI config 
> >> ifdefs
> >> from the build' patch every release.
> >
> > It won't be so huge if we reserve the NEXT_ABI solution to changes which 
> > cannot
> > have easy backward compatibility with the compat macros you introduced.
> > I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
> > At the same time, the deprecated API, using the compat macros, will be 
> > removed.
> >
> >> What might be better would be a dpdk-next branch (or even a dpdk-next 
> >> tree, of
> >> the sort that Thomas Herbert proposed a few weeks ago).
> >
> > This tree was created after Thomas' request:
> > http://dpdk.org/browse/next/dpdk-next/
> 
> Thomas, I am sorry if I went quiet for awhile but I was on personal 
> travel with inconsistent access so I almost missed most of this 
> discussion about ABI changes.
> 
> My understanding of the purpose of the dpdk-next tree is to validate 
> patches by applying and compiling against a "pull" from the main dpdk 
> tree. I think a good way to handle ABI change while effectively using 
> the dpdk-next might be to do as follows:
> 
> Create a specific branch for the new ABI such as 2.X in the main dpdk 
> tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X 
> branch along with master.
> 
> Since, dpdk-next would also have the 2.X branch that is in the main dpdk 
> tree, submitted patches could be applied to either the main branch or 
> the new-ABI 2.X branch. Providing that patch submitters make it clear 
> whether a submitted patch is for the new ABI or the old ABI, dpdk-next 
> could continue to validate the patches for either the main branch or the 
> new ABI 2.X branch.

What is the benefit of a new-ABI branch in the -next tree?

The goal of this discussion is to find a consensus on ABI policy to
smoothly integrate new features without forcing users of shared libraries
to re-build their application when up

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

2015-06-19 11:27, Neil Horman:
> On Fri, Jun 19, 2015 at 03:16:53PM +0200, Thomas Monjalon wrote:
> > 2015-06-19 09:02, Neil Horman:
> > > On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> > > > 2015-06-19 06:26, Neil Horman:
> > > > > On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote:
> > > > > > For the 2.1 release, I think we should agree to make patches that 
> > > > > > change
> > > > > > the ABI controllable via a compile-time option. I like Olivier's 
> > > > > > proposal
> > > > > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of 
> > > > > > these
> > > > > > changes instead of a separate option per patch set (see
> > > > > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think 
> > > > > > we
> > > > > > should rework the affected patch sets to use that approach for 2.1.
> > > > > 
> > > > > This is a bad idea.  Making ABI dependent on compile time options 
> > > > > isn't a
> > > > > maintainable solution.  It breaks the notion of how LIBABIVER is 
> > > > > supposed to
> > > > > work (that is to say you make it impossible to really tell what ABI 
> > > > > version you
> > > > > are building).
> > > > 
> > > > The idea was to make LIBABIVER increment dependent of 
> > > > CONFIG_RTE_NEXT_ABI.
> > > > So one ABI version number refers always to the same ABI.
> > > > 
> > > > > If you have two compile time options that modify the ABI, you
> > > > > have to burn through 4 possible LIBABIVER version values to 
> > > > > accomodate all
> > > > > possible combinations, and then you need to remember that when you 
> > > > > make them
> > > > > statically applicable.
> > > > 
> > > > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
> > > > 
> > > > Your intent when introducing ABI policy was to allow smooth porting of
> > > > applications from a DPDK version to another. Right?
> > > > The adopted solution was to provide backward compatibility during 1 
> > > > release.
> > > > But there are cases where it's not possible. So the policy was to notice
> > > > the future change and wait one release cycle to break the ABI (failing
> > > > compatibility goals).
> > > > The compile-time option may provide an alternative DPDK packaging when 
> > > > the
> > > > ABI backward compatibility cannot be provided (case of mbuf changes).
> > > > In such case, it's still possible to upgrade DPDK by providing 2 
> > > > versions of
> > > > DPDK libs. So the existing apps continue to link with the previous ABI 
> > > > and
> > > > have the possibility of migrating to the new one.
> > > > Another advantage of this approach is that we don't have to wait 1 
> > > > release
> > > > to integrate the changes.
> > > > The last advantage is to benefit early of these changes with static 
> > > > libraries.
> > > 
> > > Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
> > > Implementing an ABI preview option like this implies the notion that, 
> > > after a
> > > release, you have to remove all the ifdefs that you inserted to create 
> > > the new
> > > ABI.  That seems like an easy task, but it becomes a pain when the ABI 
> > > delta is
> > > large, and is predicated on the centralization of work effort (that is to 
> > > say
> > > you need to identify someone to submit the 'remove the NEXT_ABI config 
> > > ifdefs
> > > from the build' patch every release.
> > 
> > It won't be so huge if we reserve the NEXT_ABI solution to changes which 
> > cannot
> > have easy backward compatibility with the compat macros you introduced.
> > I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
> > At the same time, the deprecated API, using the compat macros, will be 
> > removed.
> > 
> I think that is something you can't really predict, as its not an issue of how
> stringent we are with its use, but rather a function of how much change
> developers want in a given release.  That is to say, if you only reserve it 
> for
> the most important/urgently needed changes, thats fine, but if you have a
> release in which 50 developers want to make urgent and important changes that
> breaks ABI, you still have quite a job on your hands to back out the config
> changes.
> 
> Not to mention the fact that backing those changes out is a manual process.
> 
> > > What might be better would be a dpdk-next branch (or even a dpdk-next 
> > > tree, of
> > > the sort that Thomas Herbert proposed a few weeks ago).
> > 
> > This tree was created after Thomas' request:
> > http://dpdk.org/browse/next/dpdk-next/
> > 
> Awesome, Though I'm not sure thats entirely the right place either.  IIRC that
> location was intended to be an early integration site that took unreviewed
> patches.  I think this really calls for a branch from the mainline tree that
> exclusively accepts reviewed ABI changing patches, that can then be merged 
> after
> the next release
> 
> > > Patches that aren't ABI stable can be put on the next-branch/tree in thier
> > >

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

2015-06-19 09:02, Neil Horman:
> On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> > 2015-06-19 06:26, Neil Horman:
> > > On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote:
> > > > For the 2.1 release, I think we should agree to make patches that change
> > > > the ABI controllable via a compile-time option. I like Olivier's 
> > > > proposal
> > > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> > > > changes instead of a separate option per patch set (see
> > > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> > > > should rework the affected patch sets to use that approach for 2.1.
> > > 
> > > This is a bad idea.  Making ABI dependent on compile time options isn't a
> > > maintainable solution.  It breaks the notion of how LIBABIVER is supposed 
> > > to
> > > work (that is to say you make it impossible to really tell what ABI 
> > > version you
> > > are building).
> > 
> > The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
> > So one ABI version number refers always to the same ABI.
> > 
> > > If you have two compile time options that modify the ABI, you
> > > have to burn through 4 possible LIBABIVER version values to accomodate all
> > > possible combinations, and then you need to remember that when you make 
> > > them
> > > statically applicable.
> > 
> > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
> > 
> > Your intent when introducing ABI policy was to allow smooth porting of
> > applications from a DPDK version to another. Right?
> > The adopted solution was to provide backward compatibility during 1 release.
> > But there are cases where it's not possible. So the policy was to notice
> > the future change and wait one release cycle to break the ABI (failing
> > compatibility goals).
> > The compile-time option may provide an alternative DPDK packaging when the
> > ABI backward compatibility cannot be provided (case of mbuf changes).
> > In such case, it's still possible to upgrade DPDK by providing 2 versions of
> > DPDK libs. So the existing apps continue to link with the previous ABI and
> > have the possibility of migrating to the new one.
> > Another advantage of this approach is that we don't have to wait 1 release
> > to integrate the changes.
> > The last advantage is to benefit early of these changes with static 
> > libraries.
> 
> Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
> Implementing an ABI preview option like this implies the notion that, after a
> release, you have to remove all the ifdefs that you inserted to create the new
> ABI.  That seems like an easy task, but it becomes a pain when the ABI delta 
> is
> large, and is predicated on the centralization of work effort (that is to say
> you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
> from the build' patch every release.

It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot
have easy backward compatibility with the compat macros you introduced.
I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
At the same time, the deprecated API, using the compat macros, will be removed.

> What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
> the sort that Thomas Herbert proposed a few weeks ago).

This tree was created after Thomas' request:
http://dpdk.org/browse/next/dpdk-next/

> Patches that aren't ABI stable can be put on the next-branch/tree in thier
> final format.  You can delcare the branch unstable (thereby reserving your
> right to rebase it).  People can use that to preview the next ABI version
> (complete with the update LIBABIVER bump), and when you release dpdk-X,
> the new ABI for dpdk-X+1 is achieved by simply merging.

Having this tree living would be a nice improvement but it won't provide any
stable (and enough validated) releases to rely on.

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

2015-06-19 06:26, Neil Horman:
> On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote:
> > For the 2.1 release, I think we should agree to make patches that change
> > the ABI controllable via a compile-time option. I like Olivier's proposal
> > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> > changes instead of a separate option per patch set (see
> > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> > should rework the affected patch sets to use that approach for 2.1.
> 
> This is a bad idea.  Making ABI dependent on compile time options isn't a
> maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
> work (that is to say you make it impossible to really tell what ABI version 
> you
> are building).

The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
So one ABI version number refers always to the same ABI.

> If you have two compile time options that modify the ABI, you
> have to burn through 4 possible LIBABIVER version values to accomodate all
> possible combinations, and then you need to remember that when you make them
> statically applicable.

The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.

Your intent when introducing ABI policy was to allow smooth porting of
applications from a DPDK version to another. Right?
The adopted solution was to provide backward compatibility during 1 release.
But there are cases where it's not possible. So the policy was to notice
the future change and wait one release cycle to break the ABI (failing
compatibility goals).
The compile-time option may provide an alternative DPDK packaging when the
ABI backward compatibility cannot be provided (case of mbuf changes).
In such case, it's still possible to upgrade DPDK by providing 2 versions of
DPDK libs. So the existing apps continue to link with the previous ABI and
have the possibility of migrating to the new one.
Another advantage of this approach is that we don't have to wait 1 release
to integrate the changes.
The last advantage is to benefit early of these changes with static libraries.

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas F Herbert]

On 6/19/15 1:02 PM, Thomas Monjalon wrote:
> 2015-06-19 12:13, Thomas F Herbert:
>>
>> On 6/19/15 9:16 AM, Thomas Monjalon wrote:
>>> 2015-06-19 09:02, Neil Horman:
 On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> 2015-06-19 06:26, Neil Horman:
>> On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote:
>>> For the 2.1 release, I think we should agree to make patches that change
>>> the ABI controllable via a compile-time option. I like Olivier's 
>>> proposal
>>> on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
>>> changes instead of a separate option per patch set (see
>>> http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
>>> should rework the affected patch sets to use that approach for 2.1.
>>
>> This is a bad idea.  Making ABI dependent on compile time options isn't a
>> maintainable solution.  It breaks the notion of how LIBABIVER is 
>> supposed to
>> work (that is to say you make it impossible to really tell what ABI 
>> version you
>> are building).
>
> The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
> So one ABI version number refers always to the same ABI.
>
>> If you have two compile time options that modify the ABI, you
>> have to burn through 4 possible LIBABIVER version values to accomodate 
>> all
>> possible combinations, and then you need to remember that when you make 
>> them
>> statically applicable.
>
> The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
>
> Your intent when introducing ABI policy was to allow smooth porting of
> applications from a DPDK version to another. Right?
> The adopted solution was to provide backward compatibility during 1 
> release.
> But there are cases where it's not possible. So the policy was to notice
> the future change and wait one release cycle to break the ABI (failing
> compatibility goals).
> The compile-time option may provide an alternative DPDK packaging when the
> ABI backward compatibility cannot be provided (case of mbuf changes).
> In such case, it's still possible to upgrade DPDK by providing 2 versions 
> of
> DPDK libs. So the existing apps continue to link with the previous ABI and
> have the possibility of migrating to the new one.
> Another advantage of this approach is that we don't have to wait 1 release
> to integrate the changes.
> The last advantage is to benefit early of these changes with static 
> libraries.

 Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
 Implementing an ABI preview option like this implies the notion that, 
 after a
 release, you have to remove all the ifdefs that you inserted to create the 
 new
 ABI.  That seems like an easy task, but it becomes a pain when the ABI 
 delta is
 large, and is predicated on the centralization of work effort (that is to 
 say
 you need to identify someone to submit the 'remove the NEXT_ABI config 
 ifdefs
 from the build' patch every release.
>>>
>>> It won't be so huge if we reserve the NEXT_ABI solution to changes which 
>>> cannot
>>> have easy backward compatibility with the compat macros you introduced.
>>> I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
>>> At the same time, the deprecated API, using the compat macros, will be 
>>> removed.
>>>
 What might be better would be a dpdk-next branch (or even a dpdk-next 
 tree, of
 the sort that Thomas Herbert proposed a few weeks ago).
>>>
>>> This tree was created after Thomas' request:
>>> http://dpdk.org/browse/next/dpdk-next/
>>
>> Thomas, I am sorry if I went quiet for awhile but I was on personal
>> travel with inconsistent access so I almost missed most of this
>> discussion about ABI changes.
>>
>> My understanding of the purpose of the dpdk-next tree is to validate
>> patches by applying and compiling against a "pull" from the main dpdk
>> tree. I think a good way to handle ABI change while effectively using
>> the dpdk-next might be to do as follows:
>>
>> Create a specific branch for the new ABI such as 2.X in the main dpdk
>> tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X
>> branch along with master.
>>
>> Since, dpdk-next would also have the 2.X branch that is in the main dpdk
>> tree, submitted patches could be applied to either the main branch or
>> the new-ABI 2.X branch. Providing that patch submitters make it clear
>> whether a submitted patch is for the new ABI or the old ABI, dpdk-next
>> could continue to validate the patches for either the main branch or the
>> new ABI 2.X branch.
>
> What is the benefit of a new-ABI branch in the -next tree?
I don't think that there is any specific benefit to an new-ABI branch in 
the dpdk-next tree. I was responding to the suggest

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas F Herbert]

On 6/19/15 9:16 AM, Thomas Monjalon wrote:
> 2015-06-19 09:02, Neil Horman:
>> On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
>>> 2015-06-19 06:26, Neil Horman:
 On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote:
> For the 2.1 release, I think we should agree to make patches that change
> the ABI controllable via a compile-time option. I like Olivier's proposal
> on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> changes instead of a separate option per patch set (see
> http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> should rework the affected patch sets to use that approach for 2.1.

 This is a bad idea.  Making ABI dependent on compile time options isn't a
 maintainable solution.  It breaks the notion of how LIBABIVER is supposed 
 to
 work (that is to say you make it impossible to really tell what ABI 
 version you
 are building).
>>>
>>> The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
>>> So one ABI version number refers always to the same ABI.
>>>
 If you have two compile time options that modify the ABI, you
 have to burn through 4 possible LIBABIVER version values to accomodate all
 possible combinations, and then you need to remember that when you make 
 them
 statically applicable.
>>>
>>> The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
>>>
>>> Your intent when introducing ABI policy was to allow smooth porting of
>>> applications from a DPDK version to another. Right?
>>> The adopted solution was to provide backward compatibility during 1 release.
>>> But there are cases where it's not possible. So the policy was to notice
>>> the future change and wait one release cycle to break the ABI (failing
>>> compatibility goals).
>>> The compile-time option may provide an alternative DPDK packaging when the
>>> ABI backward compatibility cannot be provided (case of mbuf changes).
>>> In such case, it's still possible to upgrade DPDK by providing 2 versions of
>>> DPDK libs. So the existing apps continue to link with the previous ABI and
>>> have the possibility of migrating to the new one.
>>> Another advantage of this approach is that we don't have to wait 1 release
>>> to integrate the changes.
>>> The last advantage is to benefit early of these changes with static 
>>> libraries.
>>
>> Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
>> Implementing an ABI preview option like this implies the notion that, after a
>> release, you have to remove all the ifdefs that you inserted to create the 
>> new
>> ABI.  That seems like an easy task, but it becomes a pain when the ABI delta 
>> is
>> large, and is predicated on the centralization of work effort (that is to say
>> you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
>> from the build' patch every release.
>
> It won't be so huge if we reserve the NEXT_ABI solution to changes which 
> cannot
> have easy backward compatibility with the compat macros you introduced.
> I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
> At the same time, the deprecated API, using the compat macros, will be 
> removed.
>
>> What might be better would be a dpdk-next branch (or even a dpdk-next tree, 
>> of
>> the sort that Thomas Herbert proposed a few weeks ago).
>
> This tree was created after Thomas' request:
>   http://dpdk.org/browse/next/dpdk-next/

Thomas, I am sorry if I went quiet for awhile but I was on personal 
travel with inconsistent access so I almost missed most of this 
discussion about ABI changes.

My understanding of the purpose of the dpdk-next tree is to validate 
patches by applying and compiling against a "pull" from the main dpdk 
tree. I think a good way to handle ABI change while effectively using 
the dpdk-next might be to do as follows:

Create a specific branch for the new ABI such as 2.X in the main dpdk 
tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X 
branch along with master.

Since, dpdk-next would also have the 2.X branch that is in the main dpdk 
tree, submitted patches could be applied to either the main branch or 
the new-ABI 2.X branch. Providing that patch submitters make it clear 
whether a submitted patch is for the new ABI or the old ABI, dpdk-next 
could continue to validate the patches for either the main branch or the 
new ABI 2.X branch.

>
>> Patches that aren't ABI stable can be put on the next-branch/tree in thier
>> final format.  You can delcare the branch unstable (thereby reserving your
>> right to rebase it).  People can use that to preview the next ABI version
>> (complete with the update LIBABIVER bump), and when you release dpdk-X,
>> the new ABI for dpdk-X+1 is achieved by simply merging.
>
> Having this tree living would be a nice improvement but it won't provide any
> stable (and enough validated) releases to rely on

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Neil Horman]

On Fri, Jun 19, 2015 at 03:16:53PM +0200, Thomas Monjalon wrote:
> 2015-06-19 09:02, Neil Horman:
> > On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> > > 2015-06-19 06:26, Neil Horman:
> > > > On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote:
> > > > > For the 2.1 release, I think we should agree to make patches that 
> > > > > change
> > > > > the ABI controllable via a compile-time option. I like Olivier's 
> > > > > proposal
> > > > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> > > > > changes instead of a separate option per patch set (see
> > > > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> > > > > should rework the affected patch sets to use that approach for 2.1.
> > > > 
> > > > This is a bad idea.  Making ABI dependent on compile time options isn't 
> > > > a
> > > > maintainable solution.  It breaks the notion of how LIBABIVER is 
> > > > supposed to
> > > > work (that is to say you make it impossible to really tell what ABI 
> > > > version you
> > > > are building).
> > > 
> > > The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
> > > So one ABI version number refers always to the same ABI.
> > > 
> > > > If you have two compile time options that modify the ABI, you
> > > > have to burn through 4 possible LIBABIVER version values to accomodate 
> > > > all
> > > > possible combinations, and then you need to remember that when you make 
> > > > them
> > > > statically applicable.
> > > 
> > > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
> > > 
> > > Your intent when introducing ABI policy was to allow smooth porting of
> > > applications from a DPDK version to another. Right?
> > > The adopted solution was to provide backward compatibility during 1 
> > > release.
> > > But there are cases where it's not possible. So the policy was to notice
> > > the future change and wait one release cycle to break the ABI (failing
> > > compatibility goals).
> > > The compile-time option may provide an alternative DPDK packaging when the
> > > ABI backward compatibility cannot be provided (case of mbuf changes).
> > > In such case, it's still possible to upgrade DPDK by providing 2 versions 
> > > of
> > > DPDK libs. So the existing apps continue to link with the previous ABI and
> > > have the possibility of migrating to the new one.
> > > Another advantage of this approach is that we don't have to wait 1 release
> > > to integrate the changes.
> > > The last advantage is to benefit early of these changes with static 
> > > libraries.
> > 
> > Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
> > Implementing an ABI preview option like this implies the notion that, after 
> > a
> > release, you have to remove all the ifdefs that you inserted to create the 
> > new
> > ABI.  That seems like an easy task, but it becomes a pain when the ABI 
> > delta is
> > large, and is predicated on the centralization of work effort (that is to 
> > say
> > you need to identify someone to submit the 'remove the NEXT_ABI config 
> > ifdefs
> > from the build' patch every release.
> 
> It won't be so huge if we reserve the NEXT_ABI solution to changes which 
> cannot
> have easy backward compatibility with the compat macros you introduced.
> I feel I can do the job of removing the ifdefs NEXT_ABI after each release.
> At the same time, the deprecated API, using the compat macros, will be 
> removed.
> 
I think that is something you can't really predict, as its not an issue of how
stringent we are with its use, but rather a function of how much change
developers want in a given release.  That is to say, if you only reserve it for
the most important/urgently needed changes, thats fine, but if you have a
release in which 50 developers want to make urgent and important changes that
breaks ABI, you still have quite a job on your hands to back out the config
changes.

Not to mention the fact that backing those changes out is a manual process.

> > What might be better would be a dpdk-next branch (or even a dpdk-next tree, 
> > of
> > the sort that Thomas Herbert proposed a few weeks ago).
> 
> This tree was created after Thomas' request:
>   http://dpdk.org/browse/next/dpdk-next/
> 
Awesome, Though I'm not sure thats entirely the right place either.  IIRC that
location was intended to be an early integration site that took unreviewed
patches.  I think this really calls for a branch from the mainline tree that
exclusively accepts reviewed ABI changing patches, that can then be merged after
the next release

> > Patches that aren't ABI stable can be put on the next-branch/tree in thier
> > final format.  You can delcare the branch unstable (thereby reserving your
> > right to rebase it).  People can use that to preview the next ABI version
> > (complete with the update LIBABIVER bump), and when you release dpdk-X,
> > the new ABI for dpdk-X+1 is achieved by simply merging.
>

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Mcnamara, John]

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Richardson, Bruce
> Sent: Wednesday, June 17, 2015 12:07 PM
> To: Neil Horman; Thomas Monjalon
> Cc: dev at dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
> statistics - ABI
> Hi Neil,
> 
> on my end, some suggestions:
> 
> 1. the documentation on changing an API function provided in rte_compat.h
> is really good, but I don't think this is present in our documentation in
> the docs folder or on website is it (apologies if it is and I've missed
> it)? This needs to go into programmers guide or some other doc (perhaps
> the new doc that the coding style went into).
> 
> 2. The documentation also needs an example of: this is how you add a new
> function and update the map file, and this is how you a) mark a function
> as deprecated and b) remove it completely. That way we could have one
> guide covering API versioning, how to add, modify and remove functions.
> 
> 3. This doc should also cover how to use the API checker tool, something I
> haven't had the chance to look at yet, but should do in the near future!
> :-)


+1 on all three. We need better documentation on how to work with the ABI in 
DPDK. A new document in doc/guides/guidelines/ would be good.

John

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Neil Horman]

On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote:
> 2015-06-19 06:26, Neil Horman:
> > On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote:
> > > For the 2.1 release, I think we should agree to make patches that change
> > > the ABI controllable via a compile-time option. I like Olivier's proposal
> > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these
> > > changes instead of a separate option per patch set (see
> > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we
> > > should rework the affected patch sets to use that approach for 2.1.
> > 
> > This is a bad idea.  Making ABI dependent on compile time options isn't a
> > maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
> > work (that is to say you make it impossible to really tell what ABI version 
> > you
> > are building).
> 
> The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI.
> So one ABI version number refers always to the same ABI.
> 
> > If you have two compile time options that modify the ABI, you
> > have to burn through 4 possible LIBABIVER version values to accomodate all
> > possible combinations, and then you need to remember that when you make them
> > statically applicable.
> 
> The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI.
> 
> Your intent when introducing ABI policy was to allow smooth porting of
> applications from a DPDK version to another. Right?
> The adopted solution was to provide backward compatibility during 1 release.
> But there are cases where it's not possible. So the policy was to notice
> the future change and wait one release cycle to break the ABI (failing
> compatibility goals).
> The compile-time option may provide an alternative DPDK packaging when the
> ABI backward compatibility cannot be provided (case of mbuf changes).
> In such case, it's still possible to upgrade DPDK by providing 2 versions of
> DPDK libs. So the existing apps continue to link with the previous ABI and
> have the possibility of migrating to the new one.
> Another advantage of this approach is that we don't have to wait 1 release
> to integrate the changes.
> The last advantage is to benefit early of these changes with static libraries.
> 


Hm, ok, thats a bit more reasonable, but it still seems shaky to me.
Implementing an ABI preview option like this implies the notion that, after a
release, you have to remove all the ifdefs that you inserted to create the new
ABI.  That seems like an easy task, but it becomes a pain when the ABI delta is
large, and is predicated on the centralization of work effort (that is to say
you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs
from the build' patch every release.

What might be better would be a dpdk-next branch (or even a dpdk-next tree, of
the sort that Thomas Herbert proposed a few weeks ago).  Patches that aren't ABI
stable can be put on the next-branch/tree in thier final format.  You can
delcare the branch unstable (thereby reserving your right to rebase it).  People
can use that to preview the next ABI version (complete with the update LIBABIVER
bump), and when you release dpdk-X, the new ABI for dpdk-X+1 is achieved by
simply merging.

Neil

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Neil Horman]

On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote:
> > -Original Message-
> > From: announce [mailto:announce-bounces at dpdk.org] On Behalf Of Thomas
> > Monjalon
> > Sent: Wednesday, June 17, 2015 12:30 AM
> > To: announce at dpdk.org
> > Subject: [dpdk-announce] important design choices - statistics - ABI
> > 
> > Hi all,
> > 
> 
> > During the development of the release 2.0, there was an agreement to
> > keep
> > ABI compatibility or to bring new ABI while keeping old one during one
> > release.
> > In case it's not possible to have this transition, the (exceptional)
> > break
> > should be acknowledged by several developers.
> > http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> > There were some interesting discussions but not a lot of participants:
> > http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus
> > =8461
> > 
> > During the current development cycle for the release 2.1, the ABI
> > question
> > arises many times in different threads.
> > To add the hash key size field, it is proposed to use a struct padding
> > gap:
> > http://dpdk.org/ml/archives/dev/2015-June/019386.html
> > To support the flow director for VF, there is no proposal yet:
> > http://dpdk.org/ml/archives/dev/2015-June/019343.html
> > To add the speed capability, it is proposed to break ABI in the release
> > 2.2:
> > http://dpdk.org/ml/archives/dev/2015-June/019225.html
> > To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
> > http://dpdk.org/ml/archives/dev/2015-June/019443.html
> > To add the interrupt mode, it is proposed to add a build-time option
> > CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
> > binary:
> > http://dpdk.org/ml/archives/dev/2015-June/018947.html
> > To add the packet type, there is a proposal to add a build-time option
> > CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
> > http://dpdk.org/ml/archives/dev/2015-June/019172.html
> > We must also better document how to remove a deprecated ABI:
> > http://dpdk.org/ml/archives/dev/2015-June/019465.html
> > The ABI compatibility is a new constraint and we need to better
> > understand
> > what it means and how to proceed. Even the macros are not yet well
> > documented:
> > http://dpdk.org/ml/archives/dev/2015-June/019357.html
> > 
> > Thanks for your attention and your participation in these important
> > choices.
> 
> There's been some good discussion on the ABI policy in various responses to 
> this email. I think we now need to reach a conclusion on how we're going to 
> proceed for the 2.1 release. Then, we can have further discussion on the use 
> of versioning or other methods for avoiding the problem in future.
> 
> For the 2.1 release, I think we should agree to make patches that change the 
> ABI controllable via a compile-time option. I like Olivier's proposal on 
> using a single option (CONFIG_RTE_NEXT_ABI) to control all of these changes 
> instead of a separate option per patch set (see 
> http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we should 
> rework the affected patch sets to use that approach for 2.1.
> 
This is a bad idea.  Making ABI dependent on compile time options isn't a
maintainable solution.  It breaks the notion of how LIBABIVER is supposed to
work (that is to say you make it impossible to really tell what ABI version you
are building).  If you have two compile time options that modify the ABI, you
have to burn through 4 possible LIBABIVER version values to accomodate all
possible combinations, and then you need to remember that when you make them
statically applicable.

Neil

> 
> Tim
> 

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Vincent JARDIN]

On 18/06/2015 18:55, O'Driscoll, Tim wrote:
> I like Olivier's proposal on using a single option (CONFIG_RTE_NEXT_ABI) to 
> control all of these changes instead of a separate option per patch set 
> (seehttp://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we 
> should rework the affected patch sets to use that approach for 2.1.

Do we have any other options to meet the short deadlines of 2.1?

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[O'Driscoll, Tim]

> -Original Message-
> From: announce [mailto:announce-bounces at dpdk.org] On Behalf Of Thomas
> Monjalon
> Sent: Wednesday, June 17, 2015 12:30 AM
> To: announce at dpdk.org
> Subject: [dpdk-announce] important design choices - statistics - ABI
> 
> Hi all,
> 

> During the development of the release 2.0, there was an agreement to
> keep
> ABI compatibility or to bring new ABI while keeping old one during one
> release.
> In case it's not possible to have this transition, the (exceptional)
> break
> should be acknowledged by several developers.
>   http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> There were some interesting discussions but not a lot of participants:
>   http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus
> =8461
> 
> During the current development cycle for the release 2.1, the ABI
> question
> arises many times in different threads.
> To add the hash key size field, it is proposed to use a struct padding
> gap:
>   http://dpdk.org/ml/archives/dev/2015-June/019386.html
> To support the flow director for VF, there is no proposal yet:
>   http://dpdk.org/ml/archives/dev/2015-June/019343.html
> To add the speed capability, it is proposed to break ABI in the release
> 2.2:
>   http://dpdk.org/ml/archives/dev/2015-June/019225.html
> To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
>   http://dpdk.org/ml/archives/dev/2015-June/019443.html
> To add the interrupt mode, it is proposed to add a build-time option
> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
> binary:
>   http://dpdk.org/ml/archives/dev/2015-June/018947.html
> To add the packet type, there is a proposal to add a build-time option
> CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
>   http://dpdk.org/ml/archives/dev/2015-June/019172.html
> We must also better document how to remove a deprecated ABI:
>   http://dpdk.org/ml/archives/dev/2015-June/019465.html
> The ABI compatibility is a new constraint and we need to better
> understand
> what it means and how to proceed. Even the macros are not yet well
> documented:
>   http://dpdk.org/ml/archives/dev/2015-June/019357.html
> 
> Thanks for your attention and your participation in these important
> choices.

There's been some good discussion on the ABI policy in various responses to 
this email. I think we now need to reach a conclusion on how we're going to 
proceed for the 2.1 release. Then, we can have further discussion on the use of 
versioning or other methods for avoiding the problem in future.

For the 2.1 release, I think we should agree to make patches that change the 
ABI controllable via a compile-time option. I like Olivier's proposal on using 
a single option (CONFIG_RTE_NEXT_ABI) to control all of these changes instead 
of a separate option per patch set (see 
http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we should 
rework the affected patch sets to use that approach for 2.1.


Tim

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Dumitrescu, Cristian]

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Bruce Richardson
> Sent: Wednesday, June 17, 2015 12:17 PM
> To: Matthew Hall
> Cc: dev at dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
> statistics - ABI
> 
> On Tue, Jun 16, 2015 at 09:36:54PM -0700, Matthew Hall wrote:
> > On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > > There were some debates about software statistics disabling.
> > > Should they be always on or possibly disabled when compiled?
> > > We need to take a decision shortly and discuss (or agree) this proposal:
> > >   http://dpdk.org/ml/archives/dev/2015-June/019461.html
> >
> > This goes against the idea I have seen before that we should be moving
> toward
> > a distro-friendly approach where one copy of DPDK can be used by multiple
> apps
> > without having to rebuild it. It seems like it is also a bit ABI hostile
> > according to the below goals / discussions.
> >
> > Jemalloc is also very high-performance code and still manages to allow
> > enabling and disabling statistics at runtime. Are we sure it's impossible 
> > for
> > DPDK or just theorizing?
> >
> 
> +1 to this. I think that any compile-time option to disable stats should only
> be used when we have a proven performance issue with just disabling them
> at runtime.
> I would assume that apps do not switch on or off stats multiple times per
> second,
> so any code branches to track stats or not would be entirely predictable in
> the
> code - since they always go one way. Therefore, when disabledi, we should
> be looking
> at a very minimal overhead per stat. If there are lots of checks for the same
> value in the one path, i.e. lots of stats in a hot path, hopefully the 
> compiler
> will be smart enough to make the check just once. If not, we can always do
> that in the C code by duplicating the hotpath code for with or without stats
> cases -
> again selectable at runtime.
> 

I see where you're coming from, but reality is you cannot guarantee that the 
few conditional branches in the library are going to be predicted correctly 
simply because the application and the other libraries used by the app have an 
unknown number of conditional branches themselves. For complex apps, the total 
number of conditional branches is large and the more there are, the lesser 
probability of such a branch being predicted correctly is. I agree that for 
test apps like l3fwd with just a few branches the probability to have library 
branches being predicted correctly is high, but for me is an incorrect proof 
point. The cost of ~14 cycles per branch misprediction is important for DPDK 
packet budgets.

Since we don't control the application, I am feeling very uncomfortable with 
generic statements about how application is likely to execute and the impact of 
library conditional branches over the application should be. To me, they sound 
like we are willing to take chances, and to me this is not the right decision. 
In my opinion, the right decision for a significant framework like DPDK is to 
keep all options open for the apps: keep counters always enabled, keep counters 
always disabled, keep counters enabled first and disabled later. I suggest we 
should move the focus from WHY arguments like: "I don't think anybody will ever 
need this this" to the HOW part of making sure that technical solution we pick 
is correct and keeps all options open.

I think stats are not always equivalent to incrementing a counter. In the 
guideline proposal, I am providing several examples of more complex statistics 
logic that is more than just inc one counter. As an extreme case, think about 
the case where the metric to compute requires complex math like prediction of 
next packet arrival time based on recent history, etc. When defining a policy, 
we should consider a broad spectrum of metrics, not just n_pkts_in.


> Also, there is also the case where the stats tracking itself is such low
> overhead
> that its not worth disabling. In that case, neither runtime nor compile-time
> disabling
> should need to be provided. For example, any library that is keeping a track
> of
> bursts of packets should not need to have that stat disable option - one
> increment
> or addition per burst of (32) packets is not going to seriously affect any 
> app. :-
> )
> 
> Regards,
> /Bruce

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Dumitrescu, Cristian]

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Matthew Hall
> Sent: Wednesday, June 17, 2015 5:37 AM
> To: dev at dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
> statistics - ABI
> 
> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > There were some debates about software statistics disabling.
> > Should they be always on or possibly disabled when compiled?
> > We need to take a decision shortly and discuss (or agree) this proposal:
> > http://dpdk.org/ml/archives/dev/2015-June/019461.html
> 
> This goes against the idea I have seen before that we should be moving
> toward
> a distro-friendly approach where one copy of DPDK can be used by multiple
> apps
> without having to rebuild it. It seems like it is also a bit ABI hostile
> according to the below goals / discussions.

Matthew, thanks for your input. As Thomas also mentions, it would be good to 
first read the proposal on the guideline 
(http://dpdk.org/ml/archives/dev/2015-June/019461.html).

In the guideline proposal, we are addressing the topic of preventing the ABI 
changes due to library statistics support, it would be good to get your opinion 
on that, I am not sure you saw it. Given DPDK paramount focus on performance, I 
think probably provides the best solution for ABI compatibility in this case.

> 
> Jemalloc is also very high-performance code and still manages to allow
> enabling and disabling statistics at runtime. Are we sure it's impossible for
> DPDK or just theorizing?

A performance improvement for one environment might still be an overkill for 
DPDK run-time environment.

Stats cannot be enabled/disabled at run-time without performance impact, as it 
typically requires testing a persistent flag for incrementing the stats 
counters, which requires a conditional branch instruction. Sometimes the 
branches are predicted correctly, sometimes not, and in this case the CPU 
pipeline needs to be flushed; typical branch misprediction cost is ~14 cycles, 
which is important given a packet budget of ~200 cycles.

When only a small number of branches are present in the (application + library) 
code, they are predicted correctly, so I am sure that for a simple test 
application like l3fw the cost of run-time enabled stats is not visible. The 
problem is that real applications are much more complex and they typically have 
a lot of conditional branches, out of which the library stats test is just one 
of them, so they cannot be all predicted correctly, hence the cost of stats 
run-time enablement becomes very much visible.

> 
> > During the development of the release 2.0, there was an agreement to
> keep
> > ABI compatibility or to bring new ABI while keeping old one during one
> release.
> > In case it's not possible to have this transition, the (exceptional) break
> > should be acknowledged by several developers.
> 
> Personally to me it seems more important to preserve the ABI on patch
> releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?
> 
> > During the current development cycle for the release 2.1, the ABI question
> > arises many times in different threads.
> 
> Most but not all of these examples point to a different issue which
> sometimes
> happens in libraries... often seen as "old-style" versus "new-style" C library
> interface. For example, in old-style like libpcap there are a lot of structs,
> both opaque and non-opaque, which the caller must allocate in order to run
> libpcap.
> 
> However new-style libraries such as libcurl usually just have init functions
> which initialize all the secret structs based on some defaults and some user
> parameters and hide the actual structs from the user. If you want to adjust
> some you call an adjuster function that modifies the actual secret struct
> contents, with some enum saying what field to adjust, and the new value
> you
> want it to have.
> 
> If you want to keep a stable ABI for a non-stable library like DPDK, there's a
> good chance you must begin hiding all these weird device specific structs all
> over the DPDK from the user needing to directly allocate and modify them.
> Otherwise the ABI breaks everytime you have to add adjustments,
> extensions,
> modifications to all these obscure special features.
> 
> Matthew.

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Dumitrescu, Cristian]

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Morten Br?rup
> Sent: Wednesday, June 17, 2015 10:54 AM
> To: Thomas Monjalon
> Cc: dev at dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
> statistics - ABI
> 
> Dear Thomas,
> 
> I don't have time to follow the DPDK Developers mailing list, but since you 
> call
> for feedback, I would like to share my thoughts regarding these design
> choices.
> 
> 
> Regarding the statistics discussion:
> 
> 1. The suggested solution assumes that, when statistics is disabled, the cost
> of allocating and maintaining zero-value statistics is negligible. If 
> statistics
> counters are only available through accessor functions, this is probably true.
> 
> However, if statistics counters are directly accessible, e.g. as elements in 
> the
> fast path data structures of a library, maintaining zero-value statistics may 
> a
> have memory and/or performance impact.
> 

Counters are only accessible through API functions.

> Since the compile time flag
> CONFIG_RTE__STATS_COLLECT already tells the
> application if the statistics are present or not, the application should 
> simply
> use this flag to determine if statistics are accessible or not.
> 
> 2. The suggested solution with only one single flag per library prevents
> implementing statistics with varying granularity for different purposes. E.g. 
> a
> library could have one set of statistics counters for ordinary SNMP purposes,
> and another set of statistics counters for debugging/optimization purposes.
> 
> Multiple flags per library should be possible. A hierarchy of flags per 
> library is
> probably not required.
> 

Morten, thank you for your input. It would be good if you could add your 
contribution to the of the guidelines documentation patch by replying to the 
thread that Thomas indicated: 
http://dpdk.org/ml/archives/dev/2015-June/019461.html.

Our initial stats patch submission had a much finer granularity of stats 
configuration: per object type instead of per library, but a lot of people on 
this mailing list are against this, so we are now looking for one configuration 
flag per library.

> 
> Regarding the PHY speed ABI:
> 
> 1. The Ethernet PHY ABI for speed, duplex, etc. should be common
> throughout the entire DPDK. It might be confusing if some
> structures/functions use a bitmask to indicate PHY
> speed/duplex/personality/etc. and other structures/functions use a
> combination of an unsigned integer, duplex flag, personality enumeration
> etc. (By personality enumeration, I am referring to PHYs with multiple
> electrical interfaces. E.g. a dual personality PHY might have both an RJ45
> copper interface and an SFP module interface, whereof only one can be
> active at any time.)
> 
> 2. The auto-negotiation standard allows the PHY to announce (to its link
> partner) any subset of its capabilities to its link partner. E.g. a standard
> 10/100/100 Ethernet PHY (which can handle both 10 and 100 Mbit/s in both
> half and full duplex and 1 Gbit/s full duplex) can be configured to announce
> 10 Mbit/s half duplex and 100 Mbit/s full duplex capabilities to its link 
> partner.
> (Of course, more useful combinations are normally announced, but the
> purpose of the example is to show that any combination is possible.)
> 
> The ABI for auto-negotiation should include options to select the list of
> capabilities to announce to the link partner. The Linux PHY ABI only allows
> forcing a selected speed and duplex (thereby disabling auto-negotiation) or
> enabling auto-negotiation (thereby announcing all possible speeds and
> duplex combinations the PHY is capable of). Don't make the same mistake in
> DPDK.
> 
> PS: While working for Vitesse Semiconductors (an Ethernet chip company) a
> long time ago, I actually wrote the API for their line of Ethernet PHYs. So I
> have hands on experience in this area.
> 
> 
> Regarding the discussion about backwards/forwards compatibility in the ABI:
> 
> 1. Sometimes, ABI breakage is required. That is the cost the users pay for
> getting the benefits from upgrading to the latest and greatest version of any
> library. The current solution of requiring acknowledgement from several
> qualified developers is fine - these developers will consider the cost/benefit
> on behalf of all the DPDK users and make a qualified decision.
> 
> 2. It is my general experience that documentation is not always updated to
> reflect the fine details of the source code, and this also applies to release
> notes. For open source software, the primary point of documentation is
> usually the source code itself.
> 
> 2a. It should be clearly visible dire

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Zhang, Helin]

Hi Neil

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman
> Sent: Wednesday, June 17, 2015 6:35 PM
> To: Thomas Monjalon
> Cc: announce at dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices - statistics 
> -
> ABI
> 
> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > Hi all,
> >
> > Sometimes there are some important discussions about architecture or
> > design which require opinions from several developers. Unfortunately,
> > we cannot read every threads. Maybe that using the announce mailing
> > list will help to bring more audience to these discussions.
> > Please note that
> > - the announce@ ML is moderated to keep a low traffic,
> > - every announce email is forwarded to dev@ ML.
> > In case you want to reply to this email, please use dev at dpdk.org address.
> >
> > There were some debates about software statistics disabling.
> > Should they be always on or possibly disabled when compiled?
> > We need to take a decision shortly and discuss (or agree) this proposal:
> > http://dpdk.org/ml/archives/dev/2015-June/019461.html
> >
> > During the development of the release 2.0, there was an agreement to
> > keep ABI compatibility or to bring new ABI while keeping old one during one
> release.
> > In case it's not possible to have this transition, the (exceptional)
> > break should be acknowledged by several developers.
> > http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> > There were some interesting discussions but not a lot of participants:
> >
> > http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=84
> > 61
> >
> > During the current development cycle for the release 2.1, the ABI
> > question arises many times in different threads.
> > To add the hash key size field, it is proposed to use a struct padding gap:
> > http://dpdk.org/ml/archives/dev/2015-June/019386.html
> > To support the flow director for VF, there is no proposal yet:
> > http://dpdk.org/ml/archives/dev/2015-June/019343.html
> > To add the speed capability, it is proposed to break ABI in the release 2.2:
> > http://dpdk.org/ml/archives/dev/2015-June/019225.html
> > To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
> > http://dpdk.org/ml/archives/dev/2015-June/019443.html
> > To add the interrupt mode, it is proposed to add a build-time option
> > CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
> binary:
> > http://dpdk.org/ml/archives/dev/2015-June/018947.html
> > To add the packet type, there is a proposal to add a build-time option
> > CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
> > http://dpdk.org/ml/archives/dev/2015-June/019172.html
> > We must also better document how to remove a deprecated ABI:
> > http://dpdk.org/ml/archives/dev/2015-June/019465.html
> > The ABI compatibility is a new constraint and we need to better
> > understand what it means and how to proceed. Even the macros are not yet
> well documented:
> > http://dpdk.org/ml/archives/dev/2015-June/019357.html
> >
> > Thanks for your attention and your participation in these important choices.
> >
> 
> Thomas-
>   Just to re-iterate what you said earlier, and what was discussed in the
> previous ABI discussions
> 
> 1) ABI stability was introduced to promote DPDK's ability to be included with
> various linux and BSD distributions.  Distributions, by and large, favor 
> building
> libraries as DSO's, favoring security and updatability in favor of all out
> performance.
> 
> 2) The desire was to put DPDK developers in a mindset whereby ABI stability 
> was
> something they needed to think about during development, as the DPDK
> exposes many data structures and instances that cannot be changed without
> breaking ABI
> 
> 3) The versioning mechanism was introduced to allow for backward compatibility
> during periods in which we needed to support both an old an new ABI
> 
> 4) As Stephan and others point out, its not expected that we will always be 
> able
> to maintain ABI, and as such an easy library versioning mechanism was
> introduced to prevent the loading of an incompatible library with an older
> application
> 
> 5) The ABI policy was introduced to create a method by which new ABI facets
> could be scheduled while allowing distros to prepare their downstream users 
> for
> the upcomming changes.
> 
> 
> It seems to me, looking back over these last few months, that we're falling 
> down

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Vincent JARDIN]

On 17/06/2015 14:14, Panu Matilainen wrote:
> (initially accidentally sent to announce, resending to dev)
>
> On 06/17/2015 01:35 PM, Neil Horman wrote:
>> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
>>> Hi all,
>>>
>>> Sometimes there are some important discussions about architecture or
>>> design
>>> which require opinions from several developers. Unfortunately, we cannot
>>> read every threads. Maybe that using the announce mailing list will help
>>> to bring more audience to these discussions.
>>> Please note that
>>> - the announce@ ML is moderated to keep a low traffic,
>>> - every announce email is forwarded to dev@ ML.
>>> In case you want to reply to this email, please use dev at dpdk.org
>>> address.
>>>
>>> There were some debates about software statistics disabling.
>>> Should they be always on or possibly disabled when compiled?
>>> We need to take a decision shortly and discuss (or agree) this proposal:
>>> http://dpdk.org/ml/archives/dev/2015-June/019461.html
>>>
>>> During the development of the release 2.0, there was an agreement to
>>> keep
>>> ABI compatibility or to bring new ABI while keeping old one during
>>> one release.
>>> In case it's not possible to have this transition, the (exceptional)
>>> break
>>> should be acknowledged by several developers.
>>> http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
>>> There were some interesting discussions but not a lot of participants:
>>> http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461
>>>
>>>
>>> During the current development cycle for the release 2.1, the ABI
>>> question
>>> arises many times in different threads.
>>> To add the hash key size field, it is proposed to use a struct
>>> padding gap:
>>> http://dpdk.org/ml/archives/dev/2015-June/019386.html
>>> To support the flow director for VF, there is no proposal yet:
>>> http://dpdk.org/ml/archives/dev/2015-June/019343.html
>>> To add the speed capability, it is proposed to break ABI in the
>>> release 2.2:
>>> http://dpdk.org/ml/archives/dev/2015-June/019225.html
>>> To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
>>> http://dpdk.org/ml/archives/dev/2015-June/019443.html
>>> To add the interrupt mode, it is proposed to add a build-time option
>>> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
>>> binary:
>>> http://dpdk.org/ml/archives/dev/2015-June/018947.html
>>> To add the packet type, there is a proposal to add a build-time option
>>> CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
>>> http://dpdk.org/ml/archives/dev/2015-June/019172.html
>>> We must also better document how to remove a deprecated ABI:
>>> http://dpdk.org/ml/archives/dev/2015-June/019465.html
>>> The ABI compatibility is a new constraint and we need to better
>>> understand
>>> what it means and how to proceed. Even the macros are not yet well
>>> documented:
>>> http://dpdk.org/ml/archives/dev/2015-June/019357.html
>>>
>>> Thanks for your attention and your participation in these important
>>> choices.
>>>
>>
>> Thomas-
>> Just to re-iterate what you said earlier, and what was discussed
>> in the
>> previous ABI discussions
>>
>> 1) ABI stability was introduced to promote DPDK's ability to be
>> included with
>> various linux and BSD distributions.  Distributions, by and large, favor
>> building libraries as DSO's, favoring security and updatability in
>> favor of all
>> out performance.
>>
>> 2) The desire was to put DPDK developers in a mindset whereby ABI
>> stability was
>> something they needed to think about during development, as the DPDK
>> exposes
>> many data structures and instances that cannot be changed without
>> breaking ABI
>>
>> 3) The versioning mechanism was introduced to allow for backward
>> compatibility
>> during periods in which we needed to support both an old an new ABI
>>
>> 4) As Stephan and others point out, its not expected that we will
>> always be able
>> to maintain ABI, and as such an easy library versioning mechanism was
>> introduced
>> to prevent the loading of an incompatible library with an older
>> application
>>
>> 5) The ABI policy was introduced to create a method by which new ABI
>> facets
>> could be scheduled while allowing distros to prepare their downstream
>> users for
>> the upcomming changes.
>>
>>
>> It seems to me, looking back over these last few months, that we're
>> falling down
>> a bit on our use of (3).  I've seen several people take advantage of
>> the ABI
>> scheduled updates, but no one has tried the versioning interface, and
>> as a
>> result patches are getting delayed, which was never my intent.  Not
>> sure whats
>> to be done about that, but we should probably address it.  Is use of the
>> versionnig interface just too hard or convoluted?
>
> To me it seems that by far the biggest problem with ABI stability in
> DPDK is features requiring changes to public structs (often directly
> allocate

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Panu Matilainen]

(initially accidentally sent to announce, resending to dev)

On 06/17/2015 01:35 PM, Neil Horman wrote:
> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
>> Hi all,
>>
>> Sometimes there are some important discussions about architecture or design
>> which require opinions from several developers. Unfortunately, we cannot
>> read every threads. Maybe that using the announce mailing list will help
>> to bring more audience to these discussions.
>> Please note that
>>  - the announce@ ML is moderated to keep a low traffic,
>>  - every announce email is forwarded to dev@ ML.
>> In case you want to reply to this email, please use dev at dpdk.org address.
>>
>> There were some debates about software statistics disabling.
>> Should they be always on or possibly disabled when compiled?
>> We need to take a decision shortly and discuss (or agree) this proposal:
>>  http://dpdk.org/ml/archives/dev/2015-June/019461.html
>>
>> During the development of the release 2.0, there was an agreement to keep
>> ABI compatibility or to bring new ABI while keeping old one during one 
>> release.
>> In case it's not possible to have this transition, the (exceptional) break
>> should be acknowledged by several developers.
>>  http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
>> There were some interesting discussions but not a lot of participants:
>>  http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461
>>
>> During the current development cycle for the release 2.1, the ABI question
>> arises many times in different threads.
>> To add the hash key size field, it is proposed to use a struct padding gap:
>>  http://dpdk.org/ml/archives/dev/2015-June/019386.html
>> To support the flow director for VF, there is no proposal yet:
>>  http://dpdk.org/ml/archives/dev/2015-June/019343.html
>> To add the speed capability, it is proposed to break ABI in the release 2.2:
>>  http://dpdk.org/ml/archives/dev/2015-June/019225.html
>> To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
>>  http://dpdk.org/ml/archives/dev/2015-June/019443.html
>> To add the interrupt mode, it is proposed to add a build-time option
>> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary:
>>  http://dpdk.org/ml/archives/dev/2015-June/018947.html
>> To add the packet type, there is a proposal to add a build-time option
>> CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
>>  http://dpdk.org/ml/archives/dev/2015-June/019172.html
>> We must also better document how to remove a deprecated ABI:
>>  http://dpdk.org/ml/archives/dev/2015-June/019465.html
>> The ABI compatibility is a new constraint and we need to better understand
>> what it means and how to proceed. Even the macros are not yet well 
>> documented:
>>  http://dpdk.org/ml/archives/dev/2015-June/019357.html
>>
>> Thanks for your attention and your participation in these important choices.
>>
>
> Thomas-
>   Just to re-iterate what you said earlier, and what was discussed in the
> previous ABI discussions
>
> 1) ABI stability was introduced to promote DPDK's ability to be included with
> various linux and BSD distributions.  Distributions, by and large, favor
> building libraries as DSO's, favoring security and updatability in favor of 
> all
> out performance.
>
> 2) The desire was to put DPDK developers in a mindset whereby ABI stability 
> was
> something they needed to think about during development, as the DPDK exposes
> many data structures and instances that cannot be changed without breaking ABI
>
> 3) The versioning mechanism was introduced to allow for backward compatibility
> during periods in which we needed to support both an old an new ABI
>
> 4) As Stephan and others point out, its not expected that we will always be 
> able
> to maintain ABI, and as such an easy library versioning mechanism was 
> introduced
> to prevent the loading of an incompatible library with an older application
>
> 5) The ABI policy was introduced to create a method by which new ABI facets
> could be scheduled while allowing distros to prepare their downstream users 
> for
> the upcomming changes.
>
>
> It seems to me, looking back over these last few months, that we're falling 
> down
> a bit on our use of (3).  I've seen several people take advantage of the ABI
> scheduled updates, but no one has tried the versioning interface, and as a
> result patches are getting delayed, which was never my intent.  Not sure whats
> to be done about that, but we should probably address it.  Is use of the
> versionnig interface just too hard or convoluted?

To me it seems that by far the biggest problem with ABI stability in 
DPDK is features requiring changes to public structs (often directly 
allocated and accessed by apps), which is something the symbol 
versioning doesn't directly help with, you'd need to version the structs 
too.

One only needs to glance at the glibc documentation on how

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Bruce Richardson]

On Tue, Jun 16, 2015 at 09:36:54PM -0700, Matthew Hall wrote:
> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > There were some debates about software statistics disabling.
> > Should they be always on or possibly disabled when compiled?
> > We need to take a decision shortly and discuss (or agree) this proposal:
> > http://dpdk.org/ml/archives/dev/2015-June/019461.html
> 
> This goes against the idea I have seen before that we should be moving toward 
> a distro-friendly approach where one copy of DPDK can be used by multiple 
> apps 
> without having to rebuild it. It seems like it is also a bit ABI hostile 
> according to the below goals / discussions.
> 
> Jemalloc is also very high-performance code and still manages to allow 
> enabling and disabling statistics at runtime. Are we sure it's impossible for 
> DPDK or just theorizing?
> 

+1 to this. I think that any compile-time option to disable stats should only
be used when we have a proven performance issue with just disabling them at 
runtime.
I would assume that apps do not switch on or off stats multiple times per 
second,
so any code branches to track stats or not would be entirely predictable in the
code - since they always go one way. Therefore, when disabledi, we should be 
looking
at a very minimal overhead per stat. If there are lots of checks for the same
value in the one path, i.e. lots of stats in a hot path, hopefully the compiler
will be smart enough to make the check just once. If not, we can always do
that in the C code by duplicating the hotpath code for with or without stats 
cases -
again selectable at runtime.

Also, there is also the case where the stats tracking itself is such low 
overhead
that its not worth disabling. In that case, neither runtime nor compile-time 
disabling
should need to be provided. For example, any library that is keeping a track of
bursts of packets should not need to have that stat disable option - one 
increment
or addition per burst of (32) packets is not going to seriously affect any app. 
:-)

Regards,
/Bruce

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Morten Brørup]

Dear Thomas,

I don't have time to follow the DPDK Developers mailing list, but since you 
call for feedback, I would like to share my thoughts regarding these design 
choices.


Regarding the statistics discussion:

1. The suggested solution assumes that, when statistics is disabled, the cost 
of allocating and maintaining zero-value statistics is negligible. If 
statistics counters are only available through accessor functions, this is 
probably true.

However, if statistics counters are directly accessible, e.g. as elements in 
the fast path data structures of a library, maintaining zero-value statistics 
may a have memory and/or performance impact.

Since the compile time flag CONFIG_RTE__STATS_COLLECT already 
tells the application if the statistics are present or not, the application 
should simply use this flag to determine if statistics are accessible or not.

2. The suggested solution with only one single flag per library prevents 
implementing statistics with varying granularity for different purposes. E.g. a 
library could have one set of statistics counters for ordinary SNMP purposes, 
and another set of statistics counters for debugging/optimization purposes.

Multiple flags per library should be possible. A hierarchy of flags per library 
is probably not required.


Regarding the PHY speed ABI:

1. The Ethernet PHY ABI for speed, duplex, etc. should be common throughout the 
entire DPDK. It might be confusing if some structures/functions use a bitmask 
to indicate PHY speed/duplex/personality/etc. and other structures/functions 
use a combination of an unsigned integer, duplex flag, personality enumeration 
etc. (By personality enumeration, I am referring to PHYs with multiple 
electrical interfaces. E.g. a dual personality PHY might have both an RJ45 
copper interface and an SFP module interface, whereof only one can be active at 
any time.)

2. The auto-negotiation standard allows the PHY to announce (to its link 
partner) any subset of its capabilities to its link partner. E.g. a standard 
10/100/100 Ethernet PHY (which can handle both 10 and 100 Mbit/s in both half 
and full duplex and 1 Gbit/s full duplex) can be configured to announce 10 
Mbit/s half duplex and 100 Mbit/s full duplex capabilities to its link partner. 
(Of course, more useful combinations are normally announced, but the purpose of 
the example is to show that any combination is possible.)

The ABI for auto-negotiation should include options to select the list of 
capabilities to announce to the link partner. The Linux PHY ABI only allows 
forcing a selected speed and duplex (thereby disabling auto-negotiation) or 
enabling auto-negotiation (thereby announcing all possible speeds and duplex 
combinations the PHY is capable of). Don't make the same mistake in DPDK.

PS: While working for Vitesse Semiconductors (an Ethernet chip company) a long 
time ago, I actually wrote the API for their line of Ethernet PHYs. So I have 
hands on experience in this area.


Regarding the discussion about backwards/forwards compatibility in the ABI:

1. Sometimes, ABI breakage is required. That is the cost the users pay for 
getting the benefits from upgrading to the latest and greatest version of any 
library. The current solution of requiring acknowledgement from several 
qualified developers is fine - these developers will consider the cost/benefit 
on behalf of all the DPDK users and make a qualified decision.

2. It is my general experience that documentation is not always updated to 
reflect the fine details of the source code, and this also applies to release 
notes. For open source software, the primary point of documentation is usually 
the source code itself.

2a. It should be clearly visible directly in the DPDK source code (including 
makefiles etc.) which ABI (i.e. functions, macros, type definitions etc.) is 
the current, the deprecated, and the future.

2b. When a developer migrates a project using DPDK from a previous version of 
the DPDK, it should be easy for the developer to identify all DPDK ABI 
modifications and variants, e.g. by using a common indicator in the DPDK source 
code, such as LIBAPIVER, that developer can simply search for.

3. Adding special feature flags, e.g. CONFIG_RTE_EAL_RX_INTR, to indicate a 
breakage of the ABI, should only be done if it is the intention to keep both 
the current and the new variants of the feature in the DPDK in the future. 
Otherwise, such a flag should be combined with the standard ABI version 
indication, so it is clear that this feature belongs to certain versions (i.e. 
deprecated, current or future).


Med venlig hilsen / kind regards

Morten Br?rup
CTO



SmartShare Systems A/S
Tonsbakken 16-18
DK-2740 Skovlunde
Denmark

Office??+45 70 20 00 93
Direct??+45 89 93 50 22
Mobile??+45 25 40 82 12

mb at smartsharesystems.com
www.smartsharesystems.com
-Original Message-
From: announce [mailto:announce-boun...@dpdk.org] On Behalf Of Thomas Monjalon
Sent: 17. ju

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Richardson, Bruce]

> -Original Message-
> From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman
> Sent: Wednesday, June 17, 2015 11:35 AM
> To: Thomas Monjalon
> Cc: announce at dpdk.org
> Subject: Re: [dpdk-dev] [dpdk-announce] important design choices -
> statistics - ABI
> 
> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > Hi all,
> >
> > Sometimes there are some important discussions about architecture or
> > design which require opinions from several developers. Unfortunately,
> > we cannot read every threads. Maybe that using the announce mailing
> > list will help to bring more audience to these discussions.
> > Please note that
> > - the announce@ ML is moderated to keep a low traffic,
> > - every announce email is forwarded to dev@ ML.
> > In case you want to reply to this email, please use dev at dpdk.org
> address.
> >
> > There were some debates about software statistics disabling.
> > Should they be always on or possibly disabled when compiled?
> > We need to take a decision shortly and discuss (or agree) this proposal:
> > http://dpdk.org/ml/archives/dev/2015-June/019461.html
> >
> > During the development of the release 2.0, there was an agreement to
> > keep ABI compatibility or to bring new ABI while keeping old one during
> one release.
> > In case it's not possible to have this transition, the (exceptional)
> > break should be acknowledged by several developers.
> > http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> > There were some interesting discussions but not a lot of participants:
> >
> > http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=84
> > 61
> >
> > During the current development cycle for the release 2.1, the ABI
> > question arises many times in different threads.
> > To add the hash key size field, it is proposed to use a struct padding
> gap:
> > http://dpdk.org/ml/archives/dev/2015-June/019386.html
> > To support the flow director for VF, there is no proposal yet:
> > http://dpdk.org/ml/archives/dev/2015-June/019343.html
> > To add the speed capability, it is proposed to break ABI in the release
> 2.2:
> > http://dpdk.org/ml/archives/dev/2015-June/019225.html
> > To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
> > http://dpdk.org/ml/archives/dev/2015-June/019443.html
> > To add the interrupt mode, it is proposed to add a build-time option
> > CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking
> binary:
> > http://dpdk.org/ml/archives/dev/2015-June/018947.html
> > To add the packet type, there is a proposal to add a build-time option
> > CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
> > http://dpdk.org/ml/archives/dev/2015-June/019172.html
> > We must also better document how to remove a deprecated ABI:
> > http://dpdk.org/ml/archives/dev/2015-June/019465.html
> > The ABI compatibility is a new constraint and we need to better
> > understand what it means and how to proceed. Even the macros are not yet
> well documented:
> > http://dpdk.org/ml/archives/dev/2015-June/019357.html
> >
> > Thanks for your attention and your participation in these important
> choices.
> >
> 
> Thomas-
>   Just to re-iterate what you said earlier, and what was discussed in
> the previous ABI discussions
> 
> 1) ABI stability was introduced to promote DPDK's ability to be included
> with various linux and BSD distributions.  Distributions, by and large,
> favor building libraries as DSO's, favoring security and updatability in
> favor of all out performance.
> 
> 2) The desire was to put DPDK developers in a mindset whereby ABI
> stability was something they needed to think about during development, as
> the DPDK exposes many data structures and instances that cannot be changed
> without breaking ABI
> 
> 3) The versioning mechanism was introduced to allow for backward
> compatibility during periods in which we needed to support both an old an
> new ABI
> 
> 4) As Stephan and others point out, its not expected that we will always
> be able to maintain ABI, and as such an easy library versioning mechanism
> was introduced to prevent the loading of an incompatible library with an
> older application
> 
> 5) The ABI policy was introduced to create a method by which new ABI
> facets could be scheduled while allowing distros to prepare their
> downstream users for the upcomming changes.
> 
> 
> It seems to me, looking back over these last few months, that we're
> falling down a bit on our us

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Marc Sune]

On 17/06/15 07:28, Stephen Hemminger wrote:
> On Tue, Jun 16, 2015 at 9:36 PM, Matthew Hall  
> wrote:
>
>> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
>>> There were some debates about software statistics disabling.
>>> Should they be always on or possibly disabled when compiled?
>>> We need to take a decision shortly and discuss (or agree) this proposal:
>>>http://dpdk.org/ml/archives/dev/2015-June/019461.html
>> This goes against the idea I have seen before that we should be moving
>> toward
>> a distro-friendly approach where one copy of DPDK can be used by multiple
>> apps
>> without having to rebuild it. It seems like it is also a bit ABI hostile
>> according to the below goals / discussions.
>>
>> Jemalloc is also very high-performance code and still manages to allow
>> enabling and disabling statistics at runtime. Are we sure it's impossible
>> for
>> DPDK or just theorizing?
>>
>>> During the development of the release 2.0, there was an agreement to keep
>>> ABI compatibility or to bring new ABI while keeping old one during one
>> release.
>>> In case it's not possible to have this transition, the (exceptional)
>> break
>>> should be acknowledged by several developers.
>> Personally to me it seems more important to preserve the ABI on patch
>> releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?
>>
>>> During the current development cycle for the release 2.1, the ABI
>> question
>>> arises many times in different threads.
>> Most but not all of these examples point to a different issue which
>> sometimes
>> happens in libraries... often seen as "old-style" versus "new-style" C
>> library
>> interface. For example, in old-style like libpcap there are a lot of
>> structs,
>> both opaque and non-opaque, which the caller must allocate in order to run
>> libpcap.
>>
>> However new-style libraries such as libcurl usually just have init
>> functions
>> which initialize all the secret structs based on some defaults and some
>> user
>> parameters and hide the actual structs from the user. If you want to adjust
>> some you call an adjuster function that modifies the actual secret struct
>> contents, with some enum saying what field to adjust, and the new value you
>> want it to have.
>>
>> If you want to keep a stable ABI for a non-stable library like DPDK,
>> there's a
>> good chance you must begin hiding all these weird device specific structs
>> all
>> over the DPDK from the user needing to directly allocate and modify them.
>> Otherwise the ABI breaks everytime you have to add adjustments, extensions,
>> modifications to all these obscure special features.
>>
>> Matthew.
>>
> The DPDK makes extensive use of inline functions which prevents data hiding
> necessary for ABI stablility. This a fundamental tradeoff, and since the
> whole
> reason for DPDK is performance; the ABI is going to be a moving target.
>
> It would make more sense to provide a higher level API which was abstracted,
> slower, but stable for applications. But in doing so it would mean giving
> up things
> like inline lockless rings. Just don't go as far as the Open (not)
> dataplane API;
> which is just an excuse for closed source.
+1 to what Stephen says. I don't think though a higher level API is 
really worth.

I unfortunately could not participate in the ABI discussion (there are 
times in which you cannot be following each and every single thread in 
the mailing list, and then it got lost in my inbox, my bad).

To me, ABI compatibility is a must if we'd have long term support 
releases (1.8.0, 1.8.1...), which we don't (I proposed to have them a 
couple of times).

On the other side, if the argument for ABI compatibility is shared 
libraries and to make it easy to package for distros, I don't see 
anything better than using the MAJOR.MIN version to identify the ABI 
(1.8, 1.9). Having LTS support would also help distros be able to stay 
in an ABI compatible version of DPDK and get recent bug fixes, and to 
better identify ABi changes (MAJ.MIN).

Moreover, I doubt that many people use shared libraries in DPDK. DPDK is 
built for performance and has a lot of code inlining which makes anyway 
your binary to be recompiled, as Stephen says.

DPDK is still growing, and the strict ABI policy is forcing patches to 
be delayed, to add hacky work-arounds to not break the ABI policy ( Re: 
[dpdk-dev] [PATCH v6 01/18] mbuf: redefine packet_type in rte_mbuf as an 
example)  that are announced that have to be removed afterwards. I see 
the entire ABI policy as a blocker and inducing to have botches all over 
the code, than anything else.

Marc

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

2015-06-16 22:28, Stephen Hemminger:
> On Tue, Jun 16, 2015 at 9:36 PM, Matthew Hall  
> wrote:
> > On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > > There were some debates about software statistics disabling.
> > > Should they be always on or possibly disabled when compiled?
> > > We need to take a decision shortly and discuss (or agree) this proposal:
> > >   http://dpdk.org/ml/archives/dev/2015-June/019461.html
> >
> > This goes against the idea I have seen before that we should be moving
> > toward a distro-friendly approach where one copy of DPDK can be used by
> > multiple apps without having to rebuild it. It seems like it is also a
> > bit ABI hostile according to the below goals / discussions.
> >
> > Jemalloc is also very high-performance code and still manages to allow
> > enabling and disabling statistics at runtime. Are we sure it's impossible
> > for DPDK or just theorizing?

Please Matthew, it is better to comment in the thread dedicated to statistics.

[...]
> > Personally to me it seems more important to preserve the ABI on patch
> > releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?

The goal of the ABI deprecation process was to provide a smooth integration
of the release 2.X+1.0. There are 4 months between releases 2.X.0 and 2.X+1.0.

[...]
> > However new-style libraries such as libcurl usually just have init
> > functions which initialize all the secret structs based on some defaults
> > and some user parameters and hide the actual structs from the user.
> > If you want to adjust some you call an adjuster function that modifies
> > the actual secret struct contents, with some enum saying what field to
> > adjust, and the new value you want it to have.
> >
> > If you want to keep a stable ABI for a non-stable library like DPDK,
> > there's a good chance you must begin hiding all these weird device
> > specific structs all over the DPDK from the user needing to directly
> > allocate and modify them.
> > Otherwise the ABI breaks everytime you have to add adjustments,
> > extensions, modifications to all these obscure special features.
> 
> The DPDK makes extensive use of inline functions which prevents data hiding
> necessary for ABI stablility. This a fundamental tradeoff, and since the
> whole reason for DPDK is performance; the ABI is going to be a moving target.
> 
> It would make more sense to provide a higher level API which was abstracted,
> slower, but stable for applications. But in doing so it would mean giving
> up things like inline lockless rings. Just don't go as far as the Open (not)
> dataplane API;
> which is just an excuse for closed source.

I don't understand what you mean.

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Neil Horman]

On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> Hi all,
> 
> Sometimes there are some important discussions about architecture or design
> which require opinions from several developers. Unfortunately, we cannot
> read every threads. Maybe that using the announce mailing list will help
> to bring more audience to these discussions.
> Please note that
>   - the announce@ ML is moderated to keep a low traffic,
>   - every announce email is forwarded to dev@ ML.
> In case you want to reply to this email, please use dev at dpdk.org address.
> 
> There were some debates about software statistics disabling.
> Should they be always on or possibly disabled when compiled?
> We need to take a decision shortly and discuss (or agree) this proposal:
>   http://dpdk.org/ml/archives/dev/2015-June/019461.html
> 
> During the development of the release 2.0, there was an agreement to keep
> ABI compatibility or to bring new ABI while keeping old one during one 
> release.
> In case it's not possible to have this transition, the (exceptional) break
> should be acknowledged by several developers.
>   http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
> There were some interesting discussions but not a lot of participants:
>   http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461
> 
> During the current development cycle for the release 2.1, the ABI question
> arises many times in different threads.
> To add the hash key size field, it is proposed to use a struct padding gap:
>   http://dpdk.org/ml/archives/dev/2015-June/019386.html
> To support the flow director for VF, there is no proposal yet:
>   http://dpdk.org/ml/archives/dev/2015-June/019343.html
> To add the speed capability, it is proposed to break ABI in the release 2.2:
>   http://dpdk.org/ml/archives/dev/2015-June/019225.html
> To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
>   http://dpdk.org/ml/archives/dev/2015-June/019443.html
> To add the interrupt mode, it is proposed to add a build-time option
> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary:
>   http://dpdk.org/ml/archives/dev/2015-June/018947.html
> To add the packet type, there is a proposal to add a build-time option
> CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
>   http://dpdk.org/ml/archives/dev/2015-June/019172.html
> We must also better document how to remove a deprecated ABI:
>   http://dpdk.org/ml/archives/dev/2015-June/019465.html
> The ABI compatibility is a new constraint and we need to better understand
> what it means and how to proceed. Even the macros are not yet well documented:
>   http://dpdk.org/ml/archives/dev/2015-June/019357.html
> 
> Thanks for your attention and your participation in these important choices.
> 

Thomas-
Just to re-iterate what you said earlier, and what was discussed in the
previous ABI discussions

1) ABI stability was introduced to promote DPDK's ability to be included with
various linux and BSD distributions.  Distributions, by and large, favor
building libraries as DSO's, favoring security and updatability in favor of all
out performance.

2) The desire was to put DPDK developers in a mindset whereby ABI stability was
something they needed to think about during development, as the DPDK exposes
many data structures and instances that cannot be changed without breaking ABI

3) The versioning mechanism was introduced to allow for backward compatibility
during periods in which we needed to support both an old an new ABI

4) As Stephan and others point out, its not expected that we will always be able
to maintain ABI, and as such an easy library versioning mechanism was introduced
to prevent the loading of an incompatible library with an older application

5) The ABI policy was introduced to create a method by which new ABI facets
could be scheduled while allowing distros to prepare their downstream users for
the upcomming changes.


It seems to me, looking back over these last few months, that we're falling down
a bit on our use of (3).  I've seen several people take advantage of the ABI
scheduled updates, but no one has tried the versioning interface, and as a
result patches are getting delayed, which was never my intent.  Not sure whats
to be done about that, but we should probably address it.  Is use of the
versionnig interface just too hard or convoluted?

Neil

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Thomas Monjalon]

Hi all,

Sometimes there are some important discussions about architecture or design
which require opinions from several developers. Unfortunately, we cannot
read every threads. Maybe that using the announce mailing list will help
to bring more audience to these discussions.
Please note that
- the announce@ ML is moderated to keep a low traffic,
- every announce email is forwarded to dev@ ML.
In case you want to reply to this email, please use dev at dpdk.org address.

There were some debates about software statistics disabling.
Should they be always on or possibly disabled when compiled?
We need to take a decision shortly and discuss (or agree) this proposal:
http://dpdk.org/ml/archives/dev/2015-June/019461.html

During the development of the release 2.0, there was an agreement to keep
ABI compatibility or to bring new ABI while keeping old one during one release.
In case it's not possible to have this transition, the (exceptional) break
should be acknowledged by several developers.
http://dpdk.org/doc/guides-2.0/rel_notes/abi.html
There were some interesting discussions but not a lot of participants:
http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461

During the current development cycle for the release 2.1, the ABI question
arises many times in different threads.
To add the hash key size field, it is proposed to use a struct padding gap:
http://dpdk.org/ml/archives/dev/2015-June/019386.html
To support the flow director for VF, there is no proposal yet:
http://dpdk.org/ml/archives/dev/2015-June/019343.html
To add the speed capability, it is proposed to break ABI in the release 2.2:
http://dpdk.org/ml/archives/dev/2015-June/019225.html
To support vhost-user multiqueues, it is proposed to break ABI in 2.2:
http://dpdk.org/ml/archives/dev/2015-June/019443.html
To add the interrupt mode, it is proposed to add a build-time option
CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary:
http://dpdk.org/ml/archives/dev/2015-June/018947.html
To add the packet type, there is a proposal to add a build-time option
CONFIG_RTE_NEXT_ABI common to every ABI breaking features:
http://dpdk.org/ml/archives/dev/2015-June/019172.html
We must also better document how to remove a deprecated ABI:
http://dpdk.org/ml/archives/dev/2015-June/019465.html
The ABI compatibility is a new constraint and we need to better understand
what it means and how to proceed. Even the macros are not yet well documented:
http://dpdk.org/ml/archives/dev/2015-June/019357.html

Thanks for your attention and your participation in these important choices.

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Stephen Hemminger]

On Tue, Jun 16, 2015 at 9:36 PM, Matthew Hall  wrote:

> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> > There were some debates about software statistics disabling.
> > Should they be always on or possibly disabled when compiled?
> > We need to take a decision shortly and discuss (or agree) this proposal:
> >   http://dpdk.org/ml/archives/dev/2015-June/019461.html
>
> This goes against the idea I have seen before that we should be moving
> toward
> a distro-friendly approach where one copy of DPDK can be used by multiple
> apps
> without having to rebuild it. It seems like it is also a bit ABI hostile
> according to the below goals / discussions.
>
> Jemalloc is also very high-performance code and still manages to allow
> enabling and disabling statistics at runtime. Are we sure it's impossible
> for
> DPDK or just theorizing?
>
> > During the development of the release 2.0, there was an agreement to keep
> > ABI compatibility or to bring new ABI while keeping old one during one
> release.
> > In case it's not possible to have this transition, the (exceptional)
> break
> > should be acknowledged by several developers.
>
> Personally to me it seems more important to preserve the ABI on patch
> releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?
>
> > During the current development cycle for the release 2.1, the ABI
> question
> > arises many times in different threads.
>
> Most but not all of these examples point to a different issue which
> sometimes
> happens in libraries... often seen as "old-style" versus "new-style" C
> library
> interface. For example, in old-style like libpcap there are a lot of
> structs,
> both opaque and non-opaque, which the caller must allocate in order to run
> libpcap.
>
> However new-style libraries such as libcurl usually just have init
> functions
> which initialize all the secret structs based on some defaults and some
> user
> parameters and hide the actual structs from the user. If you want to adjust
> some you call an adjuster function that modifies the actual secret struct
> contents, with some enum saying what field to adjust, and the new value you
> want it to have.
>
> If you want to keep a stable ABI for a non-stable library like DPDK,
> there's a
> good chance you must begin hiding all these weird device specific structs
> all
> over the DPDK from the user needing to directly allocate and modify them.
> Otherwise the ABI breaks everytime you have to add adjustments, extensions,
> modifications to all these obscure special features.
>
> Matthew.
>

The DPDK makes extensive use of inline functions which prevents data hiding
necessary for ABI stablility. This a fundamental tradeoff, and since the
whole
reason for DPDK is performance; the ABI is going to be a moving target.

It would make more sense to provide a higher level API which was abstracted,
slower, but stable for applications. But in doing so it would mean giving
up things
like inline lockless rings. Just don't go as far as the Open (not)
dataplane API;
which is just an excuse for closed source.

[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI

[Matthew Hall]

On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote:
> There were some debates about software statistics disabling.
> Should they be always on or possibly disabled when compiled?
> We need to take a decision shortly and discuss (or agree) this proposal:
>   http://dpdk.org/ml/archives/dev/2015-June/019461.html

This goes against the idea I have seen before that we should be moving toward 
a distro-friendly approach where one copy of DPDK can be used by multiple apps 
without having to rebuild it. It seems like it is also a bit ABI hostile 
according to the below goals / discussions.

Jemalloc is also very high-performance code and still manages to allow 
enabling and disabling statistics at runtime. Are we sure it's impossible for 
DPDK or just theorizing?

> During the development of the release 2.0, there was an agreement to keep
> ABI compatibility or to bring new ABI while keeping old one during one 
> release.
> In case it's not possible to have this transition, the (exceptional) break
> should be acknowledged by several developers.

Personally to me it seems more important to preserve the ABI on patch 
releases, like 2.X.Y going to 2.X.Z. But maybe I missed something?

> During the current development cycle for the release 2.1, the ABI question
> arises many times in different threads.

Most but not all of these examples point to a different issue which sometimes 
happens in libraries... often seen as "old-style" versus "new-style" C library 
interface. For example, in old-style like libpcap there are a lot of structs, 
both opaque and non-opaque, which the caller must allocate in order to run 
libpcap. 

However new-style libraries such as libcurl usually just have init functions 
which initialize all the secret structs based on some defaults and some user 
parameters and hide the actual structs from the user. If you want to adjust 
some you call an adjuster function that modifies the actual secret struct 
contents, with some enum saying what field to adjust, and the new value you 
want it to have.

If you want to keep a stable ABI for a non-stable library like DPDK, there's a 
good chance you must begin hiding all these weird device specific structs all 
over the DPDK from the user needing to directly allocate and modify them. 
Otherwise the ABI breaks everytime you have to add adjustments, extensions, 
modifications to all these obscure special features.

Matthew.