[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
2015-06-19 12:13, Thomas F Herbert: > > On 6/19/15 9:16 AM, Thomas Monjalon wrote: > > 2015-06-19 09:02, Neil Horman: > >> On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote: > >>> 2015-06-19 06:26, Neil Horman: > On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote: > > For the 2.1 release, I think we should agree to make patches that change > > the ABI controllable via a compile-time option. I like Olivier's > > proposal > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these > > changes instead of a separate option per patch set (see > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we > > should rework the affected patch sets to use that approach for 2.1. > > This is a bad idea. Making ABI dependent on compile time options isn't a > maintainable solution. It breaks the notion of how LIBABIVER is > supposed to > work (that is to say you make it impossible to really tell what ABI > version you > are building). > >>> > >>> The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI. > >>> So one ABI version number refers always to the same ABI. > >>> > If you have two compile time options that modify the ABI, you > have to burn through 4 possible LIBABIVER version values to accomodate > all > possible combinations, and then you need to remember that when you make > them > statically applicable. > >>> > >>> The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI. > >>> > >>> Your intent when introducing ABI policy was to allow smooth porting of > >>> applications from a DPDK version to another. Right? > >>> The adopted solution was to provide backward compatibility during 1 > >>> release. > >>> But there are cases where it's not possible. So the policy was to notice > >>> the future change and wait one release cycle to break the ABI (failing > >>> compatibility goals). > >>> The compile-time option may provide an alternative DPDK packaging when the > >>> ABI backward compatibility cannot be provided (case of mbuf changes). > >>> In such case, it's still possible to upgrade DPDK by providing 2 versions > >>> of > >>> DPDK libs. So the existing apps continue to link with the previous ABI and > >>> have the possibility of migrating to the new one. > >>> Another advantage of this approach is that we don't have to wait 1 release > >>> to integrate the changes. > >>> The last advantage is to benefit early of these changes with static > >>> libraries. > >> > >> Hm, ok, thats a bit more reasonable, but it still seems shaky to me. > >> Implementing an ABI preview option like this implies the notion that, > >> after a > >> release, you have to remove all the ifdefs that you inserted to create the > >> new > >> ABI. That seems like an easy task, but it becomes a pain when the ABI > >> delta is > >> large, and is predicated on the centralization of work effort (that is to > >> say > >> you need to identify someone to submit the 'remove the NEXT_ABI config > >> ifdefs > >> from the build' patch every release. > > > > It won't be so huge if we reserve the NEXT_ABI solution to changes which > > cannot > > have easy backward compatibility with the compat macros you introduced. > > I feel I can do the job of removing the ifdefs NEXT_ABI after each release. > > At the same time, the deprecated API, using the compat macros, will be > > removed. > > > >> What might be better would be a dpdk-next branch (or even a dpdk-next > >> tree, of > >> the sort that Thomas Herbert proposed a few weeks ago). > > > > This tree was created after Thomas' request: > > http://dpdk.org/browse/next/dpdk-next/ > > Thomas, I am sorry if I went quiet for awhile but I was on personal > travel with inconsistent access so I almost missed most of this > discussion about ABI changes. > > My understanding of the purpose of the dpdk-next tree is to validate > patches by applying and compiling against a "pull" from the main dpdk > tree. I think a good way to handle ABI change while effectively using > the dpdk-next might be to do as follows: > > Create a specific branch for the new ABI such as 2.X in the main dpdk > tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X > branch along with master. > > Since, dpdk-next would also have the 2.X branch that is in the main dpdk > tree, submitted patches could be applied to either the main branch or > the new-ABI 2.X branch. Providing that patch submitters make it clear > whether a submitted patch is for the new ABI or the old ABI, dpdk-next > could continue to validate the patches for either the main branch or the > new ABI 2.X branch. What is the benefit of a new-ABI branch in the -next tree? The goal of this discussion is to find a consensus on ABI policy to smoothly integrate new features without forcing users of shared libraries to re-build their application when
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
2015-06-19 11:27, Neil Horman: > On Fri, Jun 19, 2015 at 03:16:53PM +0200, Thomas Monjalon wrote: > > 2015-06-19 09:02, Neil Horman: > > > On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote: > > > > 2015-06-19 06:26, Neil Horman: > > > > > On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote: > > > > > > For the 2.1 release, I think we should agree to make patches that > > > > > > change > > > > > > the ABI controllable via a compile-time option. I like Olivier's > > > > > > proposal > > > > > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of > > > > > > these > > > > > > changes instead of a separate option per patch set (see > > > > > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think > > > > > > we > > > > > > should rework the affected patch sets to use that approach for 2.1. > > > > > > > > > > This is a bad idea. Making ABI dependent on compile time options > > > > > isn't a > > > > > maintainable solution. It breaks the notion of how LIBABIVER is > > > > > supposed to > > > > > work (that is to say you make it impossible to really tell what ABI > > > > > version you > > > > > are building). > > > > > > > > The idea was to make LIBABIVER increment dependent of > > > > CONFIG_RTE_NEXT_ABI. > > > > So one ABI version number refers always to the same ABI. > > > > > > > > > If you have two compile time options that modify the ABI, you > > > > > have to burn through 4 possible LIBABIVER version values to > > > > > accomodate all > > > > > possible combinations, and then you need to remember that when you > > > > > make them > > > > > statically applicable. > > > > > > > > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI. > > > > > > > > Your intent when introducing ABI policy was to allow smooth porting of > > > > applications from a DPDK version to another. Right? > > > > The adopted solution was to provide backward compatibility during 1 > > > > release. > > > > But there are cases where it's not possible. So the policy was to notice > > > > the future change and wait one release cycle to break the ABI (failing > > > > compatibility goals). > > > > The compile-time option may provide an alternative DPDK packaging when > > > > the > > > > ABI backward compatibility cannot be provided (case of mbuf changes). > > > > In such case, it's still possible to upgrade DPDK by providing 2 > > > > versions of > > > > DPDK libs. So the existing apps continue to link with the previous ABI > > > > and > > > > have the possibility of migrating to the new one. > > > > Another advantage of this approach is that we don't have to wait 1 > > > > release > > > > to integrate the changes. > > > > The last advantage is to benefit early of these changes with static > > > > libraries. > > > > > > Hm, ok, thats a bit more reasonable, but it still seems shaky to me. > > > Implementing an ABI preview option like this implies the notion that, > > > after a > > > release, you have to remove all the ifdefs that you inserted to create > > > the new > > > ABI. That seems like an easy task, but it becomes a pain when the ABI > > > delta is > > > large, and is predicated on the centralization of work effort (that is to > > > say > > > you need to identify someone to submit the 'remove the NEXT_ABI config > > > ifdefs > > > from the build' patch every release. > > > > It won't be so huge if we reserve the NEXT_ABI solution to changes which > > cannot > > have easy backward compatibility with the compat macros you introduced. > > I feel I can do the job of removing the ifdefs NEXT_ABI after each release. > > At the same time, the deprecated API, using the compat macros, will be > > removed. > > > I think that is something you can't really predict, as its not an issue of how > stringent we are with its use, but rather a function of how much change > developers want in a given release. That is to say, if you only reserve it > for > the most important/urgently needed changes, thats fine, but if you have a > release in which 50 developers want to make urgent and important changes that > breaks ABI, you still have quite a job on your hands to back out the config > changes. > > Not to mention the fact that backing those changes out is a manual process. > > > > What might be better would be a dpdk-next branch (or even a dpdk-next > > > tree, of > > > the sort that Thomas Herbert proposed a few weeks ago). > > > > This tree was created after Thomas' request: > > http://dpdk.org/browse/next/dpdk-next/ > > > Awesome, Though I'm not sure thats entirely the right place either. IIRC that > location was intended to be an early integration site that took unreviewed > patches. I think this really calls for a branch from the mainline tree that > exclusively accepts reviewed ABI changing patches, that can then be merged > after > the next release > > > > Patches that aren't ABI stable can be put on the next-branch/tree in thier > >
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
2015-06-19 09:02, Neil Horman: > On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote: > > 2015-06-19 06:26, Neil Horman: > > > On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote: > > > > For the 2.1 release, I think we should agree to make patches that change > > > > the ABI controllable via a compile-time option. I like Olivier's > > > > proposal > > > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these > > > > changes instead of a separate option per patch set (see > > > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we > > > > should rework the affected patch sets to use that approach for 2.1. > > > > > > This is a bad idea. Making ABI dependent on compile time options isn't a > > > maintainable solution. It breaks the notion of how LIBABIVER is supposed > > > to > > > work (that is to say you make it impossible to really tell what ABI > > > version you > > > are building). > > > > The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI. > > So one ABI version number refers always to the same ABI. > > > > > If you have two compile time options that modify the ABI, you > > > have to burn through 4 possible LIBABIVER version values to accomodate all > > > possible combinations, and then you need to remember that when you make > > > them > > > statically applicable. > > > > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI. > > > > Your intent when introducing ABI policy was to allow smooth porting of > > applications from a DPDK version to another. Right? > > The adopted solution was to provide backward compatibility during 1 release. > > But there are cases where it's not possible. So the policy was to notice > > the future change and wait one release cycle to break the ABI (failing > > compatibility goals). > > The compile-time option may provide an alternative DPDK packaging when the > > ABI backward compatibility cannot be provided (case of mbuf changes). > > In such case, it's still possible to upgrade DPDK by providing 2 versions of > > DPDK libs. So the existing apps continue to link with the previous ABI and > > have the possibility of migrating to the new one. > > Another advantage of this approach is that we don't have to wait 1 release > > to integrate the changes. > > The last advantage is to benefit early of these changes with static > > libraries. > > Hm, ok, thats a bit more reasonable, but it still seems shaky to me. > Implementing an ABI preview option like this implies the notion that, after a > release, you have to remove all the ifdefs that you inserted to create the new > ABI. That seems like an easy task, but it becomes a pain when the ABI delta > is > large, and is predicated on the centralization of work effort (that is to say > you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs > from the build' patch every release. It won't be so huge if we reserve the NEXT_ABI solution to changes which cannot have easy backward compatibility with the compat macros you introduced. I feel I can do the job of removing the ifdefs NEXT_ABI after each release. At the same time, the deprecated API, using the compat macros, will be removed. > What might be better would be a dpdk-next branch (or even a dpdk-next tree, of > the sort that Thomas Herbert proposed a few weeks ago). This tree was created after Thomas' request: http://dpdk.org/browse/next/dpdk-next/ > Patches that aren't ABI stable can be put on the next-branch/tree in thier > final format. You can delcare the branch unstable (thereby reserving your > right to rebase it). People can use that to preview the next ABI version > (complete with the update LIBABIVER bump), and when you release dpdk-X, > the new ABI for dpdk-X+1 is achieved by simply merging. Having this tree living would be a nice improvement but it won't provide any stable (and enough validated) releases to rely on.
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
2015-06-19 06:26, Neil Horman: > On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote: > > For the 2.1 release, I think we should agree to make patches that change > > the ABI controllable via a compile-time option. I like Olivier's proposal > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these > > changes instead of a separate option per patch set (see > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we > > should rework the affected patch sets to use that approach for 2.1. > > This is a bad idea. Making ABI dependent on compile time options isn't a > maintainable solution. It breaks the notion of how LIBABIVER is supposed to > work (that is to say you make it impossible to really tell what ABI version > you > are building). The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI. So one ABI version number refers always to the same ABI. > If you have two compile time options that modify the ABI, you > have to burn through 4 possible LIBABIVER version values to accomodate all > possible combinations, and then you need to remember that when you make them > statically applicable. The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI. Your intent when introducing ABI policy was to allow smooth porting of applications from a DPDK version to another. Right? The adopted solution was to provide backward compatibility during 1 release. But there are cases where it's not possible. So the policy was to notice the future change and wait one release cycle to break the ABI (failing compatibility goals). The compile-time option may provide an alternative DPDK packaging when the ABI backward compatibility cannot be provided (case of mbuf changes). In such case, it's still possible to upgrade DPDK by providing 2 versions of DPDK libs. So the existing apps continue to link with the previous ABI and have the possibility of migrating to the new one. Another advantage of this approach is that we don't have to wait 1 release to integrate the changes. The last advantage is to benefit early of these changes with static libraries.
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
On 6/19/15 1:02 PM, Thomas Monjalon wrote: > 2015-06-19 12:13, Thomas F Herbert: >> >> On 6/19/15 9:16 AM, Thomas Monjalon wrote: >>> 2015-06-19 09:02, Neil Horman: On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote: > 2015-06-19 06:26, Neil Horman: >> On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote: >>> For the 2.1 release, I think we should agree to make patches that change >>> the ABI controllable via a compile-time option. I like Olivier's >>> proposal >>> on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these >>> changes instead of a separate option per patch set (see >>> http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we >>> should rework the affected patch sets to use that approach for 2.1. >> >> This is a bad idea. Making ABI dependent on compile time options isn't a >> maintainable solution. It breaks the notion of how LIBABIVER is >> supposed to >> work (that is to say you make it impossible to really tell what ABI >> version you >> are building). > > The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI. > So one ABI version number refers always to the same ABI. > >> If you have two compile time options that modify the ABI, you >> have to burn through 4 possible LIBABIVER version values to accomodate >> all >> possible combinations, and then you need to remember that when you make >> them >> statically applicable. > > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI. > > Your intent when introducing ABI policy was to allow smooth porting of > applications from a DPDK version to another. Right? > The adopted solution was to provide backward compatibility during 1 > release. > But there are cases where it's not possible. So the policy was to notice > the future change and wait one release cycle to break the ABI (failing > compatibility goals). > The compile-time option may provide an alternative DPDK packaging when the > ABI backward compatibility cannot be provided (case of mbuf changes). > In such case, it's still possible to upgrade DPDK by providing 2 versions > of > DPDK libs. So the existing apps continue to link with the previous ABI and > have the possibility of migrating to the new one. > Another advantage of this approach is that we don't have to wait 1 release > to integrate the changes. > The last advantage is to benefit early of these changes with static > libraries. Hm, ok, thats a bit more reasonable, but it still seems shaky to me. Implementing an ABI preview option like this implies the notion that, after a release, you have to remove all the ifdefs that you inserted to create the new ABI. That seems like an easy task, but it becomes a pain when the ABI delta is large, and is predicated on the centralization of work effort (that is to say you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs from the build' patch every release. >>> >>> It won't be so huge if we reserve the NEXT_ABI solution to changes which >>> cannot >>> have easy backward compatibility with the compat macros you introduced. >>> I feel I can do the job of removing the ifdefs NEXT_ABI after each release. >>> At the same time, the deprecated API, using the compat macros, will be >>> removed. >>> What might be better would be a dpdk-next branch (or even a dpdk-next tree, of the sort that Thomas Herbert proposed a few weeks ago). >>> >>> This tree was created after Thomas' request: >>> http://dpdk.org/browse/next/dpdk-next/ >> >> Thomas, I am sorry if I went quiet for awhile but I was on personal >> travel with inconsistent access so I almost missed most of this >> discussion about ABI changes. >> >> My understanding of the purpose of the dpdk-next tree is to validate >> patches by applying and compiling against a "pull" from the main dpdk >> tree. I think a good way to handle ABI change while effectively using >> the dpdk-next might be to do as follows: >> >> Create a specific branch for the new ABI such as 2.X in the main dpdk >> tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X >> branch along with master. >> >> Since, dpdk-next would also have the 2.X branch that is in the main dpdk >> tree, submitted patches could be applied to either the main branch or >> the new-ABI 2.X branch. Providing that patch submitters make it clear >> whether a submitted patch is for the new ABI or the old ABI, dpdk-next >> could continue to validate the patches for either the main branch or the >> new ABI 2.X branch. > > What is the benefit of a new-ABI branch in the -next tree? I don't think that there is any specific benefit to an new-ABI branch in the dpdk-next tree. I was responding to the
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
On 6/19/15 9:16 AM, Thomas Monjalon wrote: > 2015-06-19 09:02, Neil Horman: >> On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote: >>> 2015-06-19 06:26, Neil Horman: On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote: > For the 2.1 release, I think we should agree to make patches that change > the ABI controllable via a compile-time option. I like Olivier's proposal > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these > changes instead of a separate option per patch set (see > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we > should rework the affected patch sets to use that approach for 2.1. This is a bad idea. Making ABI dependent on compile time options isn't a maintainable solution. It breaks the notion of how LIBABIVER is supposed to work (that is to say you make it impossible to really tell what ABI version you are building). >>> >>> The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI. >>> So one ABI version number refers always to the same ABI. >>> If you have two compile time options that modify the ABI, you have to burn through 4 possible LIBABIVER version values to accomodate all possible combinations, and then you need to remember that when you make them statically applicable. >>> >>> The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI. >>> >>> Your intent when introducing ABI policy was to allow smooth porting of >>> applications from a DPDK version to another. Right? >>> The adopted solution was to provide backward compatibility during 1 release. >>> But there are cases where it's not possible. So the policy was to notice >>> the future change and wait one release cycle to break the ABI (failing >>> compatibility goals). >>> The compile-time option may provide an alternative DPDK packaging when the >>> ABI backward compatibility cannot be provided (case of mbuf changes). >>> In such case, it's still possible to upgrade DPDK by providing 2 versions of >>> DPDK libs. So the existing apps continue to link with the previous ABI and >>> have the possibility of migrating to the new one. >>> Another advantage of this approach is that we don't have to wait 1 release >>> to integrate the changes. >>> The last advantage is to benefit early of these changes with static >>> libraries. >> >> Hm, ok, thats a bit more reasonable, but it still seems shaky to me. >> Implementing an ABI preview option like this implies the notion that, after a >> release, you have to remove all the ifdefs that you inserted to create the >> new >> ABI. That seems like an easy task, but it becomes a pain when the ABI delta >> is >> large, and is predicated on the centralization of work effort (that is to say >> you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs >> from the build' patch every release. > > It won't be so huge if we reserve the NEXT_ABI solution to changes which > cannot > have easy backward compatibility with the compat macros you introduced. > I feel I can do the job of removing the ifdefs NEXT_ABI after each release. > At the same time, the deprecated API, using the compat macros, will be > removed. > >> What might be better would be a dpdk-next branch (or even a dpdk-next tree, >> of >> the sort that Thomas Herbert proposed a few weeks ago). > > This tree was created after Thomas' request: > http://dpdk.org/browse/next/dpdk-next/ Thomas, I am sorry if I went quiet for awhile but I was on personal travel with inconsistent access so I almost missed most of this discussion about ABI changes. My understanding of the purpose of the dpdk-next tree is to validate patches by applying and compiling against a "pull" from the main dpdk tree. I think a good way to handle ABI change while effectively using the dpdk-next might be to do as follows: Create a specific branch for the new ABI such as 2.X in the main dpdk tree. Once that 2.X branch is created, dpdk-next would mirror the 2.X branch along with master. Since, dpdk-next would also have the 2.X branch that is in the main dpdk tree, submitted patches could be applied to either the main branch or the new-ABI 2.X branch. Providing that patch submitters make it clear whether a submitted patch is for the new ABI or the old ABI, dpdk-next could continue to validate the patches for either the main branch or the new ABI 2.X branch. > >> Patches that aren't ABI stable can be put on the next-branch/tree in thier >> final format. You can delcare the branch unstable (thereby reserving your >> right to rebase it). People can use that to preview the next ABI version >> (complete with the update LIBABIVER bump), and when you release dpdk-X, >> the new ABI for dpdk-X+1 is achieved by simply merging. > > Having this tree living would be a nice improvement but it won't provide any > stable (and enough validated) releases to rely
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
On Fri, Jun 19, 2015 at 03:16:53PM +0200, Thomas Monjalon wrote: > 2015-06-19 09:02, Neil Horman: > > On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote: > > > 2015-06-19 06:26, Neil Horman: > > > > On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote: > > > > > For the 2.1 release, I think we should agree to make patches that > > > > > change > > > > > the ABI controllable via a compile-time option. I like Olivier's > > > > > proposal > > > > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these > > > > > changes instead of a separate option per patch set (see > > > > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we > > > > > should rework the affected patch sets to use that approach for 2.1. > > > > > > > > This is a bad idea. Making ABI dependent on compile time options isn't > > > > a > > > > maintainable solution. It breaks the notion of how LIBABIVER is > > > > supposed to > > > > work (that is to say you make it impossible to really tell what ABI > > > > version you > > > > are building). > > > > > > The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI. > > > So one ABI version number refers always to the same ABI. > > > > > > > If you have two compile time options that modify the ABI, you > > > > have to burn through 4 possible LIBABIVER version values to accomodate > > > > all > > > > possible combinations, and then you need to remember that when you make > > > > them > > > > statically applicable. > > > > > > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI. > > > > > > Your intent when introducing ABI policy was to allow smooth porting of > > > applications from a DPDK version to another. Right? > > > The adopted solution was to provide backward compatibility during 1 > > > release. > > > But there are cases where it's not possible. So the policy was to notice > > > the future change and wait one release cycle to break the ABI (failing > > > compatibility goals). > > > The compile-time option may provide an alternative DPDK packaging when the > > > ABI backward compatibility cannot be provided (case of mbuf changes). > > > In such case, it's still possible to upgrade DPDK by providing 2 versions > > > of > > > DPDK libs. So the existing apps continue to link with the previous ABI and > > > have the possibility of migrating to the new one. > > > Another advantage of this approach is that we don't have to wait 1 release > > > to integrate the changes. > > > The last advantage is to benefit early of these changes with static > > > libraries. > > > > Hm, ok, thats a bit more reasonable, but it still seems shaky to me. > > Implementing an ABI preview option like this implies the notion that, after > > a > > release, you have to remove all the ifdefs that you inserted to create the > > new > > ABI. That seems like an easy task, but it becomes a pain when the ABI > > delta is > > large, and is predicated on the centralization of work effort (that is to > > say > > you need to identify someone to submit the 'remove the NEXT_ABI config > > ifdefs > > from the build' patch every release. > > It won't be so huge if we reserve the NEXT_ABI solution to changes which > cannot > have easy backward compatibility with the compat macros you introduced. > I feel I can do the job of removing the ifdefs NEXT_ABI after each release. > At the same time, the deprecated API, using the compat macros, will be > removed. > I think that is something you can't really predict, as its not an issue of how stringent we are with its use, but rather a function of how much change developers want in a given release. That is to say, if you only reserve it for the most important/urgently needed changes, thats fine, but if you have a release in which 50 developers want to make urgent and important changes that breaks ABI, you still have quite a job on your hands to back out the config changes. Not to mention the fact that backing those changes out is a manual process. > > What might be better would be a dpdk-next branch (or even a dpdk-next tree, > > of > > the sort that Thomas Herbert proposed a few weeks ago). > > This tree was created after Thomas' request: > http://dpdk.org/browse/next/dpdk-next/ > Awesome, Though I'm not sure thats entirely the right place either. IIRC that location was intended to be an early integration site that took unreviewed patches. I think this really calls for a branch from the mainline tree that exclusively accepts reviewed ABI changing patches, that can then be merged after the next release > > Patches that aren't ABI stable can be put on the next-branch/tree in thier > > final format. You can delcare the branch unstable (thereby reserving your > > right to rebase it). People can use that to preview the next ABI version > > (complete with the update LIBABIVER bump), and when you release dpdk-X, > > the new ABI for dpdk-X+1 is achieved by simply merging.
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
> -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Richardson, Bruce > Sent: Wednesday, June 17, 2015 12:07 PM > To: Neil Horman; Thomas Monjalon > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] [dpdk-announce] important design choices - > statistics - ABI > Hi Neil, > > on my end, some suggestions: > > 1. the documentation on changing an API function provided in rte_compat.h > is really good, but I don't think this is present in our documentation in > the docs folder or on website is it (apologies if it is and I've missed > it)? This needs to go into programmers guide or some other doc (perhaps > the new doc that the coding style went into). > > 2. The documentation also needs an example of: this is how you add a new > function and update the map file, and this is how you a) mark a function > as deprecated and b) remove it completely. That way we could have one > guide covering API versioning, how to add, modify and remove functions. > > 3. This doc should also cover how to use the API checker tool, something I > haven't had the chance to look at yet, but should do in the near future! > :-) +1 on all three. We need better documentation on how to work with the ABI in DPDK. A new document in doc/guides/guidelines/ would be good. John
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
On Fri, Jun 19, 2015 at 02:32:33PM +0200, Thomas Monjalon wrote: > 2015-06-19 06:26, Neil Horman: > > On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote: > > > For the 2.1 release, I think we should agree to make patches that change > > > the ABI controllable via a compile-time option. I like Olivier's proposal > > > on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these > > > changes instead of a separate option per patch set (see > > > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we > > > should rework the affected patch sets to use that approach for 2.1. > > > > This is a bad idea. Making ABI dependent on compile time options isn't a > > maintainable solution. It breaks the notion of how LIBABIVER is supposed to > > work (that is to say you make it impossible to really tell what ABI version > > you > > are building). > > The idea was to make LIBABIVER increment dependent of CONFIG_RTE_NEXT_ABI. > So one ABI version number refers always to the same ABI. > > > If you have two compile time options that modify the ABI, you > > have to burn through 4 possible LIBABIVER version values to accomodate all > > possible combinations, and then you need to remember that when you make them > > statically applicable. > > The idea is to have only 1 compile-time option: CONFIG_RTE_NEXT_ABI. > > Your intent when introducing ABI policy was to allow smooth porting of > applications from a DPDK version to another. Right? > The adopted solution was to provide backward compatibility during 1 release. > But there are cases where it's not possible. So the policy was to notice > the future change and wait one release cycle to break the ABI (failing > compatibility goals). > The compile-time option may provide an alternative DPDK packaging when the > ABI backward compatibility cannot be provided (case of mbuf changes). > In such case, it's still possible to upgrade DPDK by providing 2 versions of > DPDK libs. So the existing apps continue to link with the previous ABI and > have the possibility of migrating to the new one. > Another advantage of this approach is that we don't have to wait 1 release > to integrate the changes. > The last advantage is to benefit early of these changes with static libraries. > Hm, ok, thats a bit more reasonable, but it still seems shaky to me. Implementing an ABI preview option like this implies the notion that, after a release, you have to remove all the ifdefs that you inserted to create the new ABI. That seems like an easy task, but it becomes a pain when the ABI delta is large, and is predicated on the centralization of work effort (that is to say you need to identify someone to submit the 'remove the NEXT_ABI config ifdefs from the build' patch every release. What might be better would be a dpdk-next branch (or even a dpdk-next tree, of the sort that Thomas Herbert proposed a few weeks ago). Patches that aren't ABI stable can be put on the next-branch/tree in thier final format. You can delcare the branch unstable (thereby reserving your right to rebase it). People can use that to preview the next ABI version (complete with the update LIBABIVER bump), and when you release dpdk-X, the new ABI for dpdk-X+1 is achieved by simply merging. Neil
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
On Thu, Jun 18, 2015 at 04:55:45PM +, O'Driscoll, Tim wrote: > > -Original Message- > > From: announce [mailto:announce-bounces at dpdk.org] On Behalf Of Thomas > > Monjalon > > Sent: Wednesday, June 17, 2015 12:30 AM > > To: announce at dpdk.org > > Subject: [dpdk-announce] important design choices - statistics - ABI > > > > Hi all, > > > > > During the development of the release 2.0, there was an agreement to > > keep > > ABI compatibility or to bring new ABI while keeping old one during one > > release. > > In case it's not possible to have this transition, the (exceptional) > > break > > should be acknowledged by several developers. > > http://dpdk.org/doc/guides-2.0/rel_notes/abi.html > > There were some interesting discussions but not a lot of participants: > > http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus > > =8461 > > > > During the current development cycle for the release 2.1, the ABI > > question > > arises many times in different threads. > > To add the hash key size field, it is proposed to use a struct padding > > gap: > > http://dpdk.org/ml/archives/dev/2015-June/019386.html > > To support the flow director for VF, there is no proposal yet: > > http://dpdk.org/ml/archives/dev/2015-June/019343.html > > To add the speed capability, it is proposed to break ABI in the release > > 2.2: > > http://dpdk.org/ml/archives/dev/2015-June/019225.html > > To support vhost-user multiqueues, it is proposed to break ABI in 2.2: > > http://dpdk.org/ml/archives/dev/2015-June/019443.html > > To add the interrupt mode, it is proposed to add a build-time option > > CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking > > binary: > > http://dpdk.org/ml/archives/dev/2015-June/018947.html > > To add the packet type, there is a proposal to add a build-time option > > CONFIG_RTE_NEXT_ABI common to every ABI breaking features: > > http://dpdk.org/ml/archives/dev/2015-June/019172.html > > We must also better document how to remove a deprecated ABI: > > http://dpdk.org/ml/archives/dev/2015-June/019465.html > > The ABI compatibility is a new constraint and we need to better > > understand > > what it means and how to proceed. Even the macros are not yet well > > documented: > > http://dpdk.org/ml/archives/dev/2015-June/019357.html > > > > Thanks for your attention and your participation in these important > > choices. > > There's been some good discussion on the ABI policy in various responses to > this email. I think we now need to reach a conclusion on how we're going to > proceed for the 2.1 release. Then, we can have further discussion on the use > of versioning or other methods for avoiding the problem in future. > > For the 2.1 release, I think we should agree to make patches that change the > ABI controllable via a compile-time option. I like Olivier's proposal on > using a single option (CONFIG_RTE_NEXT_ABI) to control all of these changes > instead of a separate option per patch set (see > http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we should > rework the affected patch sets to use that approach for 2.1. > This is a bad idea. Making ABI dependent on compile time options isn't a maintainable solution. It breaks the notion of how LIBABIVER is supposed to work (that is to say you make it impossible to really tell what ABI version you are building). If you have two compile time options that modify the ABI, you have to burn through 4 possible LIBABIVER version values to accomodate all possible combinations, and then you need to remember that when you make them statically applicable. Neil > > Tim >
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
On 18/06/2015 18:55, O'Driscoll, Tim wrote: > I like Olivier's proposal on using a single option (CONFIG_RTE_NEXT_ABI) to > control all of these changes instead of a separate option per patch set > (seehttp://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we > should rework the affected patch sets to use that approach for 2.1. Do we have any other options to meet the short deadlines of 2.1?
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
> -Original Message- > From: announce [mailto:announce-bounces at dpdk.org] On Behalf Of Thomas > Monjalon > Sent: Wednesday, June 17, 2015 12:30 AM > To: announce at dpdk.org > Subject: [dpdk-announce] important design choices - statistics - ABI > > Hi all, > > During the development of the release 2.0, there was an agreement to > keep > ABI compatibility or to bring new ABI while keeping old one during one > release. > In case it's not possible to have this transition, the (exceptional) > break > should be acknowledged by several developers. > http://dpdk.org/doc/guides-2.0/rel_notes/abi.html > There were some interesting discussions but not a lot of participants: > http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus > =8461 > > During the current development cycle for the release 2.1, the ABI > question > arises many times in different threads. > To add the hash key size field, it is proposed to use a struct padding > gap: > http://dpdk.org/ml/archives/dev/2015-June/019386.html > To support the flow director for VF, there is no proposal yet: > http://dpdk.org/ml/archives/dev/2015-June/019343.html > To add the speed capability, it is proposed to break ABI in the release > 2.2: > http://dpdk.org/ml/archives/dev/2015-June/019225.html > To support vhost-user multiqueues, it is proposed to break ABI in 2.2: > http://dpdk.org/ml/archives/dev/2015-June/019443.html > To add the interrupt mode, it is proposed to add a build-time option > CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking > binary: > http://dpdk.org/ml/archives/dev/2015-June/018947.html > To add the packet type, there is a proposal to add a build-time option > CONFIG_RTE_NEXT_ABI common to every ABI breaking features: > http://dpdk.org/ml/archives/dev/2015-June/019172.html > We must also better document how to remove a deprecated ABI: > http://dpdk.org/ml/archives/dev/2015-June/019465.html > The ABI compatibility is a new constraint and we need to better > understand > what it means and how to proceed. Even the macros are not yet well > documented: > http://dpdk.org/ml/archives/dev/2015-June/019357.html > > Thanks for your attention and your participation in these important > choices. There's been some good discussion on the ABI policy in various responses to this email. I think we now need to reach a conclusion on how we're going to proceed for the 2.1 release. Then, we can have further discussion on the use of versioning or other methods for avoiding the problem in future. For the 2.1 release, I think we should agree to make patches that change the ABI controllable via a compile-time option. I like Olivier's proposal on using a single option (CONFIG_RTE_NEXT_ABI) to control all of these changes instead of a separate option per patch set (see http://dpdk.org/ml/archives/dev/2015-June/019147.html), so I think we should rework the affected patch sets to use that approach for 2.1. Tim
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
> -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Bruce Richardson > Sent: Wednesday, June 17, 2015 12:17 PM > To: Matthew Hall > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] [dpdk-announce] important design choices - > statistics - ABI > > On Tue, Jun 16, 2015 at 09:36:54PM -0700, Matthew Hall wrote: > > On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote: > > > There were some debates about software statistics disabling. > > > Should they be always on or possibly disabled when compiled? > > > We need to take a decision shortly and discuss (or agree) this proposal: > > > http://dpdk.org/ml/archives/dev/2015-June/019461.html > > > > This goes against the idea I have seen before that we should be moving > toward > > a distro-friendly approach where one copy of DPDK can be used by multiple > apps > > without having to rebuild it. It seems like it is also a bit ABI hostile > > according to the below goals / discussions. > > > > Jemalloc is also very high-performance code and still manages to allow > > enabling and disabling statistics at runtime. Are we sure it's impossible > > for > > DPDK or just theorizing? > > > > +1 to this. I think that any compile-time option to disable stats should only > be used when we have a proven performance issue with just disabling them > at runtime. > I would assume that apps do not switch on or off stats multiple times per > second, > so any code branches to track stats or not would be entirely predictable in > the > code - since they always go one way. Therefore, when disabledi, we should > be looking > at a very minimal overhead per stat. If there are lots of checks for the same > value in the one path, i.e. lots of stats in a hot path, hopefully the > compiler > will be smart enough to make the check just once. If not, we can always do > that in the C code by duplicating the hotpath code for with or without stats > cases - > again selectable at runtime. > I see where you're coming from, but reality is you cannot guarantee that the few conditional branches in the library are going to be predicted correctly simply because the application and the other libraries used by the app have an unknown number of conditional branches themselves. For complex apps, the total number of conditional branches is large and the more there are, the lesser probability of such a branch being predicted correctly is. I agree that for test apps like l3fwd with just a few branches the probability to have library branches being predicted correctly is high, but for me is an incorrect proof point. The cost of ~14 cycles per branch misprediction is important for DPDK packet budgets. Since we don't control the application, I am feeling very uncomfortable with generic statements about how application is likely to execute and the impact of library conditional branches over the application should be. To me, they sound like we are willing to take chances, and to me this is not the right decision. In my opinion, the right decision for a significant framework like DPDK is to keep all options open for the apps: keep counters always enabled, keep counters always disabled, keep counters enabled first and disabled later. I suggest we should move the focus from WHY arguments like: "I don't think anybody will ever need this this" to the HOW part of making sure that technical solution we pick is correct and keeps all options open. I think stats are not always equivalent to incrementing a counter. In the guideline proposal, I am providing several examples of more complex statistics logic that is more than just inc one counter. As an extreme case, think about the case where the metric to compute requires complex math like prediction of next packet arrival time based on recent history, etc. When defining a policy, we should consider a broad spectrum of metrics, not just n_pkts_in. > Also, there is also the case where the stats tracking itself is such low > overhead > that its not worth disabling. In that case, neither runtime nor compile-time > disabling > should need to be provided. For example, any library that is keeping a track > of > bursts of packets should not need to have that stat disable option - one > increment > or addition per burst of (32) packets is not going to seriously affect any > app. :- > ) > > Regards, > /Bruce
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
> -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Matthew Hall > Sent: Wednesday, June 17, 2015 5:37 AM > To: dev at dpdk.org > Subject: Re: [dpdk-dev] [dpdk-announce] important design choices - > statistics - ABI > > On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote: > > There were some debates about software statistics disabling. > > Should they be always on or possibly disabled when compiled? > > We need to take a decision shortly and discuss (or agree) this proposal: > > http://dpdk.org/ml/archives/dev/2015-June/019461.html > > This goes against the idea I have seen before that we should be moving > toward > a distro-friendly approach where one copy of DPDK can be used by multiple > apps > without having to rebuild it. It seems like it is also a bit ABI hostile > according to the below goals / discussions. Matthew, thanks for your input. As Thomas also mentions, it would be good to first read the proposal on the guideline (http://dpdk.org/ml/archives/dev/2015-June/019461.html). In the guideline proposal, we are addressing the topic of preventing the ABI changes due to library statistics support, it would be good to get your opinion on that, I am not sure you saw it. Given DPDK paramount focus on performance, I think probably provides the best solution for ABI compatibility in this case. > > Jemalloc is also very high-performance code and still manages to allow > enabling and disabling statistics at runtime. Are we sure it's impossible for > DPDK or just theorizing? A performance improvement for one environment might still be an overkill for DPDK run-time environment. Stats cannot be enabled/disabled at run-time without performance impact, as it typically requires testing a persistent flag for incrementing the stats counters, which requires a conditional branch instruction. Sometimes the branches are predicted correctly, sometimes not, and in this case the CPU pipeline needs to be flushed; typical branch misprediction cost is ~14 cycles, which is important given a packet budget of ~200 cycles. When only a small number of branches are present in the (application + library) code, they are predicted correctly, so I am sure that for a simple test application like l3fw the cost of run-time enabled stats is not visible. The problem is that real applications are much more complex and they typically have a lot of conditional branches, out of which the library stats test is just one of them, so they cannot be all predicted correctly, hence the cost of stats run-time enablement becomes very much visible. > > > During the development of the release 2.0, there was an agreement to > keep > > ABI compatibility or to bring new ABI while keeping old one during one > release. > > In case it's not possible to have this transition, the (exceptional) break > > should be acknowledged by several developers. > > Personally to me it seems more important to preserve the ABI on patch > releases, like 2.X.Y going to 2.X.Z. But maybe I missed something? > > > During the current development cycle for the release 2.1, the ABI question > > arises many times in different threads. > > Most but not all of these examples point to a different issue which > sometimes > happens in libraries... often seen as "old-style" versus "new-style" C library > interface. For example, in old-style like libpcap there are a lot of structs, > both opaque and non-opaque, which the caller must allocate in order to run > libpcap. > > However new-style libraries such as libcurl usually just have init functions > which initialize all the secret structs based on some defaults and some user > parameters and hide the actual structs from the user. If you want to adjust > some you call an adjuster function that modifies the actual secret struct > contents, with some enum saying what field to adjust, and the new value > you > want it to have. > > If you want to keep a stable ABI for a non-stable library like DPDK, there's a > good chance you must begin hiding all these weird device specific structs all > over the DPDK from the user needing to directly allocate and modify them. > Otherwise the ABI breaks everytime you have to add adjustments, > extensions, > modifications to all these obscure special features. > > Matthew.
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
> -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Morten Br?rup > Sent: Wednesday, June 17, 2015 10:54 AM > To: Thomas Monjalon > Cc: dev at dpdk.org > Subject: Re: [dpdk-dev] [dpdk-announce] important design choices - > statistics - ABI > > Dear Thomas, > > I don't have time to follow the DPDK Developers mailing list, but since you > call > for feedback, I would like to share my thoughts regarding these design > choices. > > > Regarding the statistics discussion: > > 1. The suggested solution assumes that, when statistics is disabled, the cost > of allocating and maintaining zero-value statistics is negligible. If > statistics > counters are only available through accessor functions, this is probably true. > > However, if statistics counters are directly accessible, e.g. as elements in > the > fast path data structures of a library, maintaining zero-value statistics may > a > have memory and/or performance impact. > Counters are only accessible through API functions. > Since the compile time flag > CONFIG_RTE__STATS_COLLECT already tells the > application if the statistics are present or not, the application should > simply > use this flag to determine if statistics are accessible or not. > > 2. The suggested solution with only one single flag per library prevents > implementing statistics with varying granularity for different purposes. E.g. > a > library could have one set of statistics counters for ordinary SNMP purposes, > and another set of statistics counters for debugging/optimization purposes. > > Multiple flags per library should be possible. A hierarchy of flags per > library is > probably not required. > Morten, thank you for your input. It would be good if you could add your contribution to the of the guidelines documentation patch by replying to the thread that Thomas indicated: http://dpdk.org/ml/archives/dev/2015-June/019461.html. Our initial stats patch submission had a much finer granularity of stats configuration: per object type instead of per library, but a lot of people on this mailing list are against this, so we are now looking for one configuration flag per library. > > Regarding the PHY speed ABI: > > 1. The Ethernet PHY ABI for speed, duplex, etc. should be common > throughout the entire DPDK. It might be confusing if some > structures/functions use a bitmask to indicate PHY > speed/duplex/personality/etc. and other structures/functions use a > combination of an unsigned integer, duplex flag, personality enumeration > etc. (By personality enumeration, I am referring to PHYs with multiple > electrical interfaces. E.g. a dual personality PHY might have both an RJ45 > copper interface and an SFP module interface, whereof only one can be > active at any time.) > > 2. The auto-negotiation standard allows the PHY to announce (to its link > partner) any subset of its capabilities to its link partner. E.g. a standard > 10/100/100 Ethernet PHY (which can handle both 10 and 100 Mbit/s in both > half and full duplex and 1 Gbit/s full duplex) can be configured to announce > 10 Mbit/s half duplex and 100 Mbit/s full duplex capabilities to its link > partner. > (Of course, more useful combinations are normally announced, but the > purpose of the example is to show that any combination is possible.) > > The ABI for auto-negotiation should include options to select the list of > capabilities to announce to the link partner. The Linux PHY ABI only allows > forcing a selected speed and duplex (thereby disabling auto-negotiation) or > enabling auto-negotiation (thereby announcing all possible speeds and > duplex combinations the PHY is capable of). Don't make the same mistake in > DPDK. > > PS: While working for Vitesse Semiconductors (an Ethernet chip company) a > long time ago, I actually wrote the API for their line of Ethernet PHYs. So I > have hands on experience in this area. > > > Regarding the discussion about backwards/forwards compatibility in the ABI: > > 1. Sometimes, ABI breakage is required. That is the cost the users pay for > getting the benefits from upgrading to the latest and greatest version of any > library. The current solution of requiring acknowledgement from several > qualified developers is fine - these developers will consider the cost/benefit > on behalf of all the DPDK users and make a qualified decision. > > 2. It is my general experience that documentation is not always updated to > reflect the fine details of the source code, and this also applies to release > notes. For open source software, the primary point of documentation is > usually the source code itself. > > 2a. It should be clearly visible direct
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
On 17/06/2015 14:14, Panu Matilainen wrote: > (initially accidentally sent to announce, resending to dev) > > On 06/17/2015 01:35 PM, Neil Horman wrote: >> On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote: >>> Hi all, >>> >>> Sometimes there are some important discussions about architecture or >>> design >>> which require opinions from several developers. Unfortunately, we cannot >>> read every threads. Maybe that using the announce mailing list will help >>> to bring more audience to these discussions. >>> Please note that >>> - the announce@ ML is moderated to keep a low traffic, >>> - every announce email is forwarded to dev@ ML. >>> In case you want to reply to this email, please use dev at dpdk.org >>> address. >>> >>> There were some debates about software statistics disabling. >>> Should they be always on or possibly disabled when compiled? >>> We need to take a decision shortly and discuss (or agree) this proposal: >>> http://dpdk.org/ml/archives/dev/2015-June/019461.html >>> >>> During the development of the release 2.0, there was an agreement to >>> keep >>> ABI compatibility or to bring new ABI while keeping old one during >>> one release. >>> In case it's not possible to have this transition, the (exceptional) >>> break >>> should be acknowledged by several developers. >>> http://dpdk.org/doc/guides-2.0/rel_notes/abi.html >>> There were some interesting discussions but not a lot of participants: >>> http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461 >>> >>> >>> During the current development cycle for the release 2.1, the ABI >>> question >>> arises many times in different threads. >>> To add the hash key size field, it is proposed to use a struct >>> padding gap: >>> http://dpdk.org/ml/archives/dev/2015-June/019386.html >>> To support the flow director for VF, there is no proposal yet: >>> http://dpdk.org/ml/archives/dev/2015-June/019343.html >>> To add the speed capability, it is proposed to break ABI in the >>> release 2.2: >>> http://dpdk.org/ml/archives/dev/2015-June/019225.html >>> To support vhost-user multiqueues, it is proposed to break ABI in 2.2: >>> http://dpdk.org/ml/archives/dev/2015-June/019443.html >>> To add the interrupt mode, it is proposed to add a build-time option >>> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking >>> binary: >>> http://dpdk.org/ml/archives/dev/2015-June/018947.html >>> To add the packet type, there is a proposal to add a build-time option >>> CONFIG_RTE_NEXT_ABI common to every ABI breaking features: >>> http://dpdk.org/ml/archives/dev/2015-June/019172.html >>> We must also better document how to remove a deprecated ABI: >>> http://dpdk.org/ml/archives/dev/2015-June/019465.html >>> The ABI compatibility is a new constraint and we need to better >>> understand >>> what it means and how to proceed. Even the macros are not yet well >>> documented: >>> http://dpdk.org/ml/archives/dev/2015-June/019357.html >>> >>> Thanks for your attention and your participation in these important >>> choices. >>> >> >> Thomas- >> Just to re-iterate what you said earlier, and what was discussed >> in the >> previous ABI discussions >> >> 1) ABI stability was introduced to promote DPDK's ability to be >> included with >> various linux and BSD distributions. Distributions, by and large, favor >> building libraries as DSO's, favoring security and updatability in >> favor of all >> out performance. >> >> 2) The desire was to put DPDK developers in a mindset whereby ABI >> stability was >> something they needed to think about during development, as the DPDK >> exposes >> many data structures and instances that cannot be changed without >> breaking ABI >> >> 3) The versioning mechanism was introduced to allow for backward >> compatibility >> during periods in which we needed to support both an old an new ABI >> >> 4) As Stephan and others point out, its not expected that we will >> always be able >> to maintain ABI, and as such an easy library versioning mechanism was >> introduced >> to prevent the loading of an incompatible library with an older >> application >> >> 5) The ABI policy was introduced to create a method by which new ABI >> facets >> could be scheduled while allowing distros to prepare their downstream >> users for >> the upcomming changes. >> >> >> It seems to me, looking back over these last few months, that we're >> falling down >> a bit on our use of (3). I've seen several people take advantage of >> the ABI >> scheduled updates, but no one has tried the versioning interface, and >> as a >> result patches are getting delayed, which was never my intent. Not >> sure whats >> to be done about that, but we should probably address it. Is use of the >> versionnig interface just too hard or convoluted? > > To me it seems that by far the biggest problem with ABI stability in > DPDK is features requiring changes to public structs (often directly >
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
(initially accidentally sent to announce, resending to dev) On 06/17/2015 01:35 PM, Neil Horman wrote: > On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote: >> Hi all, >> >> Sometimes there are some important discussions about architecture or design >> which require opinions from several developers. Unfortunately, we cannot >> read every threads. Maybe that using the announce mailing list will help >> to bring more audience to these discussions. >> Please note that >> - the announce@ ML is moderated to keep a low traffic, >> - every announce email is forwarded to dev@ ML. >> In case you want to reply to this email, please use dev at dpdk.org address. >> >> There were some debates about software statistics disabling. >> Should they be always on or possibly disabled when compiled? >> We need to take a decision shortly and discuss (or agree) this proposal: >> http://dpdk.org/ml/archives/dev/2015-June/019461.html >> >> During the development of the release 2.0, there was an agreement to keep >> ABI compatibility or to bring new ABI while keeping old one during one >> release. >> In case it's not possible to have this transition, the (exceptional) break >> should be acknowledged by several developers. >> http://dpdk.org/doc/guides-2.0/rel_notes/abi.html >> There were some interesting discussions but not a lot of participants: >> http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461 >> >> During the current development cycle for the release 2.1, the ABI question >> arises many times in different threads. >> To add the hash key size field, it is proposed to use a struct padding gap: >> http://dpdk.org/ml/archives/dev/2015-June/019386.html >> To support the flow director for VF, there is no proposal yet: >> http://dpdk.org/ml/archives/dev/2015-June/019343.html >> To add the speed capability, it is proposed to break ABI in the release 2.2: >> http://dpdk.org/ml/archives/dev/2015-June/019225.html >> To support vhost-user multiqueues, it is proposed to break ABI in 2.2: >> http://dpdk.org/ml/archives/dev/2015-June/019443.html >> To add the interrupt mode, it is proposed to add a build-time option >> CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary: >> http://dpdk.org/ml/archives/dev/2015-June/018947.html >> To add the packet type, there is a proposal to add a build-time option >> CONFIG_RTE_NEXT_ABI common to every ABI breaking features: >> http://dpdk.org/ml/archives/dev/2015-June/019172.html >> We must also better document how to remove a deprecated ABI: >> http://dpdk.org/ml/archives/dev/2015-June/019465.html >> The ABI compatibility is a new constraint and we need to better understand >> what it means and how to proceed. Even the macros are not yet well >> documented: >> http://dpdk.org/ml/archives/dev/2015-June/019357.html >> >> Thanks for your attention and your participation in these important choices. >> > > Thomas- > Just to re-iterate what you said earlier, and what was discussed in the > previous ABI discussions > > 1) ABI stability was introduced to promote DPDK's ability to be included with > various linux and BSD distributions. Distributions, by and large, favor > building libraries as DSO's, favoring security and updatability in favor of > all > out performance. > > 2) The desire was to put DPDK developers in a mindset whereby ABI stability > was > something they needed to think about during development, as the DPDK exposes > many data structures and instances that cannot be changed without breaking ABI > > 3) The versioning mechanism was introduced to allow for backward compatibility > during periods in which we needed to support both an old an new ABI > > 4) As Stephan and others point out, its not expected that we will always be > able > to maintain ABI, and as such an easy library versioning mechanism was > introduced > to prevent the loading of an incompatible library with an older application > > 5) The ABI policy was introduced to create a method by which new ABI facets > could be scheduled while allowing distros to prepare their downstream users > for > the upcomming changes. > > > It seems to me, looking back over these last few months, that we're falling > down > a bit on our use of (3). I've seen several people take advantage of the ABI > scheduled updates, but no one has tried the versioning interface, and as a > result patches are getting delayed, which was never my intent. Not sure whats > to be done about that, but we should probably address it. Is use of the > versionnig interface just too hard or convoluted? To me it seems that by far the biggest problem with ABI stability in DPDK is features requiring changes to public structs (often directly allocated and accessed by apps), which is something the symbol versioning doesn't directly help with, you'd need to version the structs too. One only needs to glance at the glibc documentation on
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
On Tue, Jun 16, 2015 at 09:36:54PM -0700, Matthew Hall wrote: > On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote: > > There were some debates about software statistics disabling. > > Should they be always on or possibly disabled when compiled? > > We need to take a decision shortly and discuss (or agree) this proposal: > > http://dpdk.org/ml/archives/dev/2015-June/019461.html > > This goes against the idea I have seen before that we should be moving toward > a distro-friendly approach where one copy of DPDK can be used by multiple > apps > without having to rebuild it. It seems like it is also a bit ABI hostile > according to the below goals / discussions. > > Jemalloc is also very high-performance code and still manages to allow > enabling and disabling statistics at runtime. Are we sure it's impossible for > DPDK or just theorizing? > +1 to this. I think that any compile-time option to disable stats should only be used when we have a proven performance issue with just disabling them at runtime. I would assume that apps do not switch on or off stats multiple times per second, so any code branches to track stats or not would be entirely predictable in the code - since they always go one way. Therefore, when disabledi, we should be looking at a very minimal overhead per stat. If there are lots of checks for the same value in the one path, i.e. lots of stats in a hot path, hopefully the compiler will be smart enough to make the check just once. If not, we can always do that in the C code by duplicating the hotpath code for with or without stats cases - again selectable at runtime. Also, there is also the case where the stats tracking itself is such low overhead that its not worth disabling. In that case, neither runtime nor compile-time disabling should need to be provided. For example, any library that is keeping a track of bursts of packets should not need to have that stat disable option - one increment or addition per burst of (32) packets is not going to seriously affect any app. :-) Regards, /Bruce
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
Dear Thomas, I don't have time to follow the DPDK Developers mailing list, but since you call for feedback, I would like to share my thoughts regarding these design choices. Regarding the statistics discussion: 1. The suggested solution assumes that, when statistics is disabled, the cost of allocating and maintaining zero-value statistics is negligible. If statistics counters are only available through accessor functions, this is probably true. However, if statistics counters are directly accessible, e.g. as elements in the fast path data structures of a library, maintaining zero-value statistics may a have memory and/or performance impact. Since the compile time flag CONFIG_RTE__STATS_COLLECT already tells the application if the statistics are present or not, the application should simply use this flag to determine if statistics are accessible or not. 2. The suggested solution with only one single flag per library prevents implementing statistics with varying granularity for different purposes. E.g. a library could have one set of statistics counters for ordinary SNMP purposes, and another set of statistics counters for debugging/optimization purposes. Multiple flags per library should be possible. A hierarchy of flags per library is probably not required. Regarding the PHY speed ABI: 1. The Ethernet PHY ABI for speed, duplex, etc. should be common throughout the entire DPDK. It might be confusing if some structures/functions use a bitmask to indicate PHY speed/duplex/personality/etc. and other structures/functions use a combination of an unsigned integer, duplex flag, personality enumeration etc. (By personality enumeration, I am referring to PHYs with multiple electrical interfaces. E.g. a dual personality PHY might have both an RJ45 copper interface and an SFP module interface, whereof only one can be active at any time.) 2. The auto-negotiation standard allows the PHY to announce (to its link partner) any subset of its capabilities to its link partner. E.g. a standard 10/100/100 Ethernet PHY (which can handle both 10 and 100 Mbit/s in both half and full duplex and 1 Gbit/s full duplex) can be configured to announce 10 Mbit/s half duplex and 100 Mbit/s full duplex capabilities to its link partner. (Of course, more useful combinations are normally announced, but the purpose of the example is to show that any combination is possible.) The ABI for auto-negotiation should include options to select the list of capabilities to announce to the link partner. The Linux PHY ABI only allows forcing a selected speed and duplex (thereby disabling auto-negotiation) or enabling auto-negotiation (thereby announcing all possible speeds and duplex combinations the PHY is capable of). Don't make the same mistake in DPDK. PS: While working for Vitesse Semiconductors (an Ethernet chip company) a long time ago, I actually wrote the API for their line of Ethernet PHYs. So I have hands on experience in this area. Regarding the discussion about backwards/forwards compatibility in the ABI: 1. Sometimes, ABI breakage is required. That is the cost the users pay for getting the benefits from upgrading to the latest and greatest version of any library. The current solution of requiring acknowledgement from several qualified developers is fine - these developers will consider the cost/benefit on behalf of all the DPDK users and make a qualified decision. 2. It is my general experience that documentation is not always updated to reflect the fine details of the source code, and this also applies to release notes. For open source software, the primary point of documentation is usually the source code itself. 2a. It should be clearly visible directly in the DPDK source code (including makefiles etc.) which ABI (i.e. functions, macros, type definitions etc.) is the current, the deprecated, and the future. 2b. When a developer migrates a project using DPDK from a previous version of the DPDK, it should be easy for the developer to identify all DPDK ABI modifications and variants, e.g. by using a common indicator in the DPDK source code, such as LIBAPIVER, that developer can simply search for. 3. Adding special feature flags, e.g. CONFIG_RTE_EAL_RX_INTR, to indicate a breakage of the ABI, should only be done if it is the intention to keep both the current and the new variants of the feature in the DPDK in the future. Otherwise, such a flag should be combined with the standard ABI version indication, so it is clear that this feature belongs to certain versions (i.e. deprecated, current or future). Med venlig hilsen / kind regards Morten Br?rup CTO SmartShare Systems A/S Tonsbakken 16-18 DK-2740 Skovlunde Denmark Office??+45 70 20 00 93 Direct??+45 89 93 50 22 Mobile??+45 25 40 82 12 mb at smartsharesystems.com www.smartsharesystems.com -Original Message- From: announce [mailto:announce-boun...@dpdk.org] On Behalf Of Thomas Monjalon Sent: 17.
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
> -Original Message- > From: dev [mailto:dev-bounces at dpdk.org] On Behalf Of Neil Horman > Sent: Wednesday, June 17, 2015 11:35 AM > To: Thomas Monjalon > Cc: announce at dpdk.org > Subject: Re: [dpdk-dev] [dpdk-announce] important design choices - > statistics - ABI > > On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote: > > Hi all, > > > > Sometimes there are some important discussions about architecture or > > design which require opinions from several developers. Unfortunately, > > we cannot read every threads. Maybe that using the announce mailing > > list will help to bring more audience to these discussions. > > Please note that > > - the announce@ ML is moderated to keep a low traffic, > > - every announce email is forwarded to dev@ ML. > > In case you want to reply to this email, please use dev at dpdk.org > address. > > > > There were some debates about software statistics disabling. > > Should they be always on or possibly disabled when compiled? > > We need to take a decision shortly and discuss (or agree) this proposal: > > http://dpdk.org/ml/archives/dev/2015-June/019461.html > > > > During the development of the release 2.0, there was an agreement to > > keep ABI compatibility or to bring new ABI while keeping old one during > one release. > > In case it's not possible to have this transition, the (exceptional) > > break should be acknowledged by several developers. > > http://dpdk.org/doc/guides-2.0/rel_notes/abi.html > > There were some interesting discussions but not a lot of participants: > > > > http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=84 > > 61 > > > > During the current development cycle for the release 2.1, the ABI > > question arises many times in different threads. > > To add the hash key size field, it is proposed to use a struct padding > gap: > > http://dpdk.org/ml/archives/dev/2015-June/019386.html > > To support the flow director for VF, there is no proposal yet: > > http://dpdk.org/ml/archives/dev/2015-June/019343.html > > To add the speed capability, it is proposed to break ABI in the release > 2.2: > > http://dpdk.org/ml/archives/dev/2015-June/019225.html > > To support vhost-user multiqueues, it is proposed to break ABI in 2.2: > > http://dpdk.org/ml/archives/dev/2015-June/019443.html > > To add the interrupt mode, it is proposed to add a build-time option > > CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking > binary: > > http://dpdk.org/ml/archives/dev/2015-June/018947.html > > To add the packet type, there is a proposal to add a build-time option > > CONFIG_RTE_NEXT_ABI common to every ABI breaking features: > > http://dpdk.org/ml/archives/dev/2015-June/019172.html > > We must also better document how to remove a deprecated ABI: > > http://dpdk.org/ml/archives/dev/2015-June/019465.html > > The ABI compatibility is a new constraint and we need to better > > understand what it means and how to proceed. Even the macros are not yet > well documented: > > http://dpdk.org/ml/archives/dev/2015-June/019357.html > > > > Thanks for your attention and your participation in these important > choices. > > > > Thomas- > Just to re-iterate what you said earlier, and what was discussed in > the previous ABI discussions > > 1) ABI stability was introduced to promote DPDK's ability to be included > with various linux and BSD distributions. Distributions, by and large, > favor building libraries as DSO's, favoring security and updatability in > favor of all out performance. > > 2) The desire was to put DPDK developers in a mindset whereby ABI > stability was something they needed to think about during development, as > the DPDK exposes many data structures and instances that cannot be changed > without breaking ABI > > 3) The versioning mechanism was introduced to allow for backward > compatibility during periods in which we needed to support both an old an > new ABI > > 4) As Stephan and others point out, its not expected that we will always > be able to maintain ABI, and as such an easy library versioning mechanism > was introduced to prevent the loading of an incompatible library with an > older application > > 5) The ABI policy was introduced to create a method by which new ABI > facets could be scheduled while allowing distros to prepare their > downstream users for the upcomming changes. > > > It seems to me, looking back over these last few months, that we're > falling down a bit on our use of (3). I've s
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
2015-06-16 22:28, Stephen Hemminger: > On Tue, Jun 16, 2015 at 9:36 PM, Matthew Hall > wrote: > > On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote: > > > There were some debates about software statistics disabling. > > > Should they be always on or possibly disabled when compiled? > > > We need to take a decision shortly and discuss (or agree) this proposal: > > > http://dpdk.org/ml/archives/dev/2015-June/019461.html > > > > This goes against the idea I have seen before that we should be moving > > toward a distro-friendly approach where one copy of DPDK can be used by > > multiple apps without having to rebuild it. It seems like it is also a > > bit ABI hostile according to the below goals / discussions. > > > > Jemalloc is also very high-performance code and still manages to allow > > enabling and disabling statistics at runtime. Are we sure it's impossible > > for DPDK or just theorizing? Please Matthew, it is better to comment in the thread dedicated to statistics. [...] > > Personally to me it seems more important to preserve the ABI on patch > > releases, like 2.X.Y going to 2.X.Z. But maybe I missed something? The goal of the ABI deprecation process was to provide a smooth integration of the release 2.X+1.0. There are 4 months between releases 2.X.0 and 2.X+1.0. [...] > > However new-style libraries such as libcurl usually just have init > > functions which initialize all the secret structs based on some defaults > > and some user parameters and hide the actual structs from the user. > > If you want to adjust some you call an adjuster function that modifies > > the actual secret struct contents, with some enum saying what field to > > adjust, and the new value you want it to have. > > > > If you want to keep a stable ABI for a non-stable library like DPDK, > > there's a good chance you must begin hiding all these weird device > > specific structs all over the DPDK from the user needing to directly > > allocate and modify them. > > Otherwise the ABI breaks everytime you have to add adjustments, > > extensions, modifications to all these obscure special features. > > The DPDK makes extensive use of inline functions which prevents data hiding > necessary for ABI stablility. This a fundamental tradeoff, and since the > whole reason for DPDK is performance; the ABI is going to be a moving target. > > It would make more sense to provide a higher level API which was abstracted, > slower, but stable for applications. But in doing so it would mean giving > up things like inline lockless rings. Just don't go as far as the Open (not) > dataplane API; > which is just an excuse for closed source. I don't understand what you mean.
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote: > Hi all, > > Sometimes there are some important discussions about architecture or design > which require opinions from several developers. Unfortunately, we cannot > read every threads. Maybe that using the announce mailing list will help > to bring more audience to these discussions. > Please note that > - the announce@ ML is moderated to keep a low traffic, > - every announce email is forwarded to dev@ ML. > In case you want to reply to this email, please use dev at dpdk.org address. > > There were some debates about software statistics disabling. > Should they be always on or possibly disabled when compiled? > We need to take a decision shortly and discuss (or agree) this proposal: > http://dpdk.org/ml/archives/dev/2015-June/019461.html > > During the development of the release 2.0, there was an agreement to keep > ABI compatibility or to bring new ABI while keeping old one during one > release. > In case it's not possible to have this transition, the (exceptional) break > should be acknowledged by several developers. > http://dpdk.org/doc/guides-2.0/rel_notes/abi.html > There were some interesting discussions but not a lot of participants: > http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461 > > During the current development cycle for the release 2.1, the ABI question > arises many times in different threads. > To add the hash key size field, it is proposed to use a struct padding gap: > http://dpdk.org/ml/archives/dev/2015-June/019386.html > To support the flow director for VF, there is no proposal yet: > http://dpdk.org/ml/archives/dev/2015-June/019343.html > To add the speed capability, it is proposed to break ABI in the release 2.2: > http://dpdk.org/ml/archives/dev/2015-June/019225.html > To support vhost-user multiqueues, it is proposed to break ABI in 2.2: > http://dpdk.org/ml/archives/dev/2015-June/019443.html > To add the interrupt mode, it is proposed to add a build-time option > CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary: > http://dpdk.org/ml/archives/dev/2015-June/018947.html > To add the packet type, there is a proposal to add a build-time option > CONFIG_RTE_NEXT_ABI common to every ABI breaking features: > http://dpdk.org/ml/archives/dev/2015-June/019172.html > We must also better document how to remove a deprecated ABI: > http://dpdk.org/ml/archives/dev/2015-June/019465.html > The ABI compatibility is a new constraint and we need to better understand > what it means and how to proceed. Even the macros are not yet well documented: > http://dpdk.org/ml/archives/dev/2015-June/019357.html > > Thanks for your attention and your participation in these important choices. > Thomas- Just to re-iterate what you said earlier, and what was discussed in the previous ABI discussions 1) ABI stability was introduced to promote DPDK's ability to be included with various linux and BSD distributions. Distributions, by and large, favor building libraries as DSO's, favoring security and updatability in favor of all out performance. 2) The desire was to put DPDK developers in a mindset whereby ABI stability was something they needed to think about during development, as the DPDK exposes many data structures and instances that cannot be changed without breaking ABI 3) The versioning mechanism was introduced to allow for backward compatibility during periods in which we needed to support both an old an new ABI 4) As Stephan and others point out, its not expected that we will always be able to maintain ABI, and as such an easy library versioning mechanism was introduced to prevent the loading of an incompatible library with an older application 5) The ABI policy was introduced to create a method by which new ABI facets could be scheduled while allowing distros to prepare their downstream users for the upcomming changes. It seems to me, looking back over these last few months, that we're falling down a bit on our use of (3). I've seen several people take advantage of the ABI scheduled updates, but no one has tried the versioning interface, and as a result patches are getting delayed, which was never my intent. Not sure whats to be done about that, but we should probably address it. Is use of the versionnig interface just too hard or convoluted? Neil
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
Hi all, Sometimes there are some important discussions about architecture or design which require opinions from several developers. Unfortunately, we cannot read every threads. Maybe that using the announce mailing list will help to bring more audience to these discussions. Please note that - the announce@ ML is moderated to keep a low traffic, - every announce email is forwarded to dev@ ML. In case you want to reply to this email, please use dev at dpdk.org address. There were some debates about software statistics disabling. Should they be always on or possibly disabled when compiled? We need to take a decision shortly and discuss (or agree) this proposal: http://dpdk.org/ml/archives/dev/2015-June/019461.html During the development of the release 2.0, there was an agreement to keep ABI compatibility or to bring new ABI while keeping old one during one release. In case it's not possible to have this transition, the (exceptional) break should be acknowledged by several developers. http://dpdk.org/doc/guides-2.0/rel_notes/abi.html There were some interesting discussions but not a lot of participants: http://thread.gmane.org/gmane.comp.networking.dpdk.devel/8367/focus=8461 During the current development cycle for the release 2.1, the ABI question arises many times in different threads. To add the hash key size field, it is proposed to use a struct padding gap: http://dpdk.org/ml/archives/dev/2015-June/019386.html To support the flow director for VF, there is no proposal yet: http://dpdk.org/ml/archives/dev/2015-June/019343.html To add the speed capability, it is proposed to break ABI in the release 2.2: http://dpdk.org/ml/archives/dev/2015-June/019225.html To support vhost-user multiqueues, it is proposed to break ABI in 2.2: http://dpdk.org/ml/archives/dev/2015-June/019443.html To add the interrupt mode, it is proposed to add a build-time option CONFIG_RTE_EAL_RX_INTR to switch between compatible and ABI breaking binary: http://dpdk.org/ml/archives/dev/2015-June/018947.html To add the packet type, there is a proposal to add a build-time option CONFIG_RTE_NEXT_ABI common to every ABI breaking features: http://dpdk.org/ml/archives/dev/2015-June/019172.html We must also better document how to remove a deprecated ABI: http://dpdk.org/ml/archives/dev/2015-June/019465.html The ABI compatibility is a new constraint and we need to better understand what it means and how to proceed. Even the macros are not yet well documented: http://dpdk.org/ml/archives/dev/2015-June/019357.html Thanks for your attention and your participation in these important choices.
[dpdk-dev] [dpdk-announce] important design choices - statistics - ABI
On Wed, Jun 17, 2015 at 01:29:47AM +0200, Thomas Monjalon wrote: > There were some debates about software statistics disabling. > Should they be always on or possibly disabled when compiled? > We need to take a decision shortly and discuss (or agree) this proposal: > http://dpdk.org/ml/archives/dev/2015-June/019461.html This goes against the idea I have seen before that we should be moving toward a distro-friendly approach where one copy of DPDK can be used by multiple apps without having to rebuild it. It seems like it is also a bit ABI hostile according to the below goals / discussions. Jemalloc is also very high-performance code and still manages to allow enabling and disabling statistics at runtime. Are we sure it's impossible for DPDK or just theorizing? > During the development of the release 2.0, there was an agreement to keep > ABI compatibility or to bring new ABI while keeping old one during one > release. > In case it's not possible to have this transition, the (exceptional) break > should be acknowledged by several developers. Personally to me it seems more important to preserve the ABI on patch releases, like 2.X.Y going to 2.X.Z. But maybe I missed something? > During the current development cycle for the release 2.1, the ABI question > arises many times in different threads. Most but not all of these examples point to a different issue which sometimes happens in libraries... often seen as "old-style" versus "new-style" C library interface. For example, in old-style like libpcap there are a lot of structs, both opaque and non-opaque, which the caller must allocate in order to run libpcap. However new-style libraries such as libcurl usually just have init functions which initialize all the secret structs based on some defaults and some user parameters and hide the actual structs from the user. If you want to adjust some you call an adjuster function that modifies the actual secret struct contents, with some enum saying what field to adjust, and the new value you want it to have. If you want to keep a stable ABI for a non-stable library like DPDK, there's a good chance you must begin hiding all these weird device specific structs all over the DPDK from the user needing to directly allocate and modify them. Otherwise the ABI breaks everytime you have to add adjustments, extensions, modifications to all these obscure special features. Matthew.
