On Thu, Sep 18, 2014 at 03:14:01PM -0400, Neil Horman wrote: > On Thu, Sep 18, 2014 at 08:23:36PM +0200, Thomas Monjalon wrote: > > Hi Neil, > > > > 2014-09-15 15:23, Neil Horman: > > > The DPDK ABI develops and changes quickly, which makes it difficult for > > > applications to keep up with the latest version of the library, > > > especially when > > > it (the DPDK) is built as a set of shared objects, as applications may be > > > built > > > against an older version of the library. > > > > > > To mitigate this, this patch series introduces support for library and > > > symbol > > > versioning when the DPDK is built as a DSO. Specifically, it does 4 > > > things: > > > > > > 1) Adds initial support for library versioning. Each library now has a > > > version > > > map that explicitly calls out what symbols are exported to using > > > applications, > > > and assigns version(s) to them > > > > > > 2) Adds support macros so that when libraries create incompatible ABI's, > > > multiple versions may be supported so that applications linked against > > > older > > > DPDK releases can continue to function > > > > > > 3) Adds library soname versioning suffixes so that when ABI's must be > > > broken in > > > a fashion that requires a rebuild of older applications, they will break > > > at load > > > time, rather than cause unexpected issues at run time. > > > > > > 4) Adds documentation for ABI policy, and provides space to document > > > deprecated > > > ABI versions, so that applications might be warned of impending changes. > > > > > > With these elements in place the DPDK has some support to allow for the > > > extended > > > maintenence of older API's while still allowing the freedom to develop > > > new and > > > improved API's. > > > > > > Implementing this feature will require some additional effort on the part > > > of > > > developers and reviewers. When reviewing patches, must be checked against > > > existing exports to ensure that the function prototypes are not changing. > > > If > > > they are, the versioning macros must be used, and the library export map > > > should > > > be updated to reflect the new version of the function. > > > > > > When data structures change, if those structures are application > > > accessible, > > > apis that accept or return instances of those data structures should have > > > new > > > versions created so that users of the old data structure version might > > > co-exist > > > at the same time. > > > > Thanks for your efforts. > > But I feel this change has too many constraints for the current status of > > the DPDK. It's probably too early to adopt such policy. > > > I think you may be misunderstanding something. What constraints do you > beleive > that this patch imposes? Note it doesn't in any way prevent changes to the > ABI > of the DPDK, but rather gives us infrastructure to support multiple ABI > revisions at the same time, so that applications built against DPDK shared > libraries can continue to function properly at least for some time until we > decide to deprecate that ABI level. > > This is all based on the versioning strategy outlined here: > http://www.akkadia.org/drepper/dsohowto.pdf > > That may help clarify things for you. > > > By the way, this versioning doesn't cover structure changes. > No, it doesn't. No link-time mechanism does so. > > > How could it be managed? > Thats a subject that is open to discussion, but my initial thinking is that we > need to handle it on a case by case basis: > > * For minor updates, where allocation of a structure is done on the heap and > new > fields need to be added, appending them to the end of a structure and > providing > an initial value is sufficient. > > * For major changes, where fields need to be removed, or re-arranged, mostly > likely the API surfaces which accept or return those structures as > inputs/outputs will need to have new versions written to accept the new > version > of the structure, and internally we will have to support both formats for a > time > (according to the policy I documented, that is currently a single major > release). I.e. if you want to change struct foo, which is accepted as a > parameter for the function bar(struct foo *ptr), then for a release we would > need to create struct foo_v2 with the new format, map a new function foo_v2 to > the exported foo@@DPDK_1.(X+1), and internally make the foo functions > understand > both the origional and v2 versions of the structure. Then in DPDK release > 1.X+2, we can remove the old version after posting a deprecation notice with > version 1.(X+1) > > > Don't you think it would be more reliable if managed by packaging? > Solving this with packaging defeats the purpose of having shared libraries at > all. While packaging each version of the dpdk separately is possible stopgap > solution, in that it allows applications to link to differing versions of the > library independently, but that negates any expectation of timely bugfixes for > any given version of the DPDK. That is to say, if you package things this > way, > and wind up with several parallel versions of the same package, and for any > bugfix that comes out upstream, the packager then has the responsibility to > adapt that fix to each package. Thats an unscalable solution, and not > something > any packager is going to undertake willingly. I did a hybrid version of this > in > fedora for exactly that reason. I packaged the dpdk into its own directory, > but > have every intention of changing that directory every major release, so that > application writers can clearly see when they need to stop updating the dpdk, > lest their applications stop linking. I'm not going to have multiple dpdk > packages to maintain in parallel, thats just too much work. > > > > > Thank you for opening this discussion with a constructive proposal. > > Let's check it later on once structures will be more stable since > > performance is the most critical target. > If I'm being honest, I have to say thats a cop out answer. We all know that > structure stability isn't a priority for the DPDK, nor will it ever be in all > likelyhood. It will continue to evolve and grow as the hardware does. And > this > patch set doesn't prevent that from happening. All it does is provide some > level of stability in the API for a period of time to let 3rd party > application > writers write and package applications with some allowance of time to keep up > with upstream changes on their own schedule. > > I grant you that writing a good API for a shared library is difficult, but > (and feel free to disagree with this), if we don't start enforcing policies > that > require good API design, its not going to happen on its own. This patch set > will highlight those API points which are difficult to maintain accross major > releases, and force us to address and improve them. To that end I've already > begun talking to some of the individual library maintainers off list to > address > some of the API aspects that I have concerns about (exporting variable rather > than accessor functions, structures that don't need to be visible to users, > etc), and they've started reviewing them. We can make this better, but we > can't > just say later, because theres no roadmap that lists structure stability as a > line item. As hardware improves, structures will change to operate more > efficiently or support more features. Without a hard plan, the initial goals > of > the DPDK (high performance networking) will relegate ABI to such a low > priority > that it will never be addressed. > > To that end, can we discuss specifics? Can you ennumerate direct points that > you feel make this patch unworkable at this time? I know you mentioned some > above, and I think I addressed them (though please ask follow up questions if > I've been unclear). What other concerns do you have? > > Neil > > Ping Thomas. I know you're busy, but I would like this to not fall off anyones radar. You alluded to concerns regarding what, for lack of a better term, ABI/API lockin. I had asked you to enuumerate/elaborate on specifics, but never heard back. Are there further specifics you wish to discuss, or are you satisfied with the above answers?
Best Neil
