John Sonnenschein wrote:
> I may regret throwing my hat in to this ring in the future, but with
> respect to the example about user downloading $APP and having it not
> work because it uses $FOO ( linked to libCstd ) and $BAR ( linked to
> stdcxx ), that would mean that the developer of $APP explicitly linked
> to both, which is just silly, that $FOO or $BAR changed under it's
> feet, which would be a different ARC case altogether, or that
> $FOO/$BAR had the STL ripped out from under /it's/ feet, and removal
> of old libraries would be yet another ARC case altogether.
No, the problem is that because they were linked with different
implementations of the C++ standard library, $FOO and $BAR cannot
coexist in the same address space.
-- Garrett
>
> Take care
> -John
>
> On 5-Sep-08, at 4:46 PM, Garrett D'Amore wrote:
>
>> Thank you for your instructional description of the C++ language with
>> respect to templates.
>>
>> There are a few significant points, that I think you keep missing, or
>> are glossing over:
>>
>> When I said C++ doesn't separate *binary* interface from implementation,
>> I was *not* talking about the nice separation done for the benefit of
>> the programmer. I was talking about separation done across the boundary
>> created by the linker ... AFAIK, C++ does not recognize that there is
>> such an interface boundary.
>>
>> In C++, if an inline function is expanded by the compiler, but makes use
>> of *private* members of the class, then the *binary implementation* of
>> the resulting code is *tied* to the implementation, and if the
>> implementation changes (e.g. by changing the dynamically linked library)
>> then there can be unfortunate consequences.
>>
>> I'm not a template expert, so I'll demonstrate with a simple inline
>> function using C++ circa 1990.
>>
>> class box {
>> int width;
>> int height;
>> /* put more private details here... */
>>
>> public:
>> void draw(); // private implementation elsewhere
>>
>> public inline int getwidth() { return width; }
>> };
>>
>> If this is in header, then the implementation parts (the width, the
>> height and their offsets within the class) wind up getting encoded into
>> the binary that includes this header. If the width or the height ever
>> move (e..g by moving another member in front of them in the class), then
>> the binary is busted. Likewise, if the private members change type
>> (e.g. from a uint16_t to a uint32_t), breakage ensues.
>>
>> It is possible to have implementations that don't suffer from this
>> problem, but it requires that the implementation refrain from inline
>> functions, and take care that the headers *only* expose portions of the
>> classes that will never ever change (or change in a way that breaks the
>> binary implementation of the class.) I won't endeavor to describe how
>> this problem may or may not affect templates, as such is out of my area
>> of expertise.
>>
>> I've not kept up with the C++ standards admittedly, but I believe that
>> this aspect of binary compatibility is not one that has been addressed
>> in the standards, and likely not in the specification of the libraries.
>> It is possible that the Apache product has taken great care to avoid
>> this kind of potential breakage, but my first reaction is doubt.
>>
>> Now, that's only *one* aspect of the "compatibility problem."
>>
>> There is a grave and other serious problem, which relates to the fact
>> that this proposal effectively creates a new baseline (i.e. a new ABI)
>> for C++ programs, where programs linked (directly or indirectly) against
>> this library are incompatible with our one and only bless (at this time)
>> ABI, based on the libC that ships with Solaris today.
>>
>> And, the project recognizes that future breakage is not just possible,
>> but *likely*, when either the compiler team ships another project, or
>> when the library needs an update due to changes in the standard.
>>
>> Imagine the plight of the poor user who downloads a program
>> (WhizzyDownloadTracker) which uses two different class libraries. On
>> the one hand, it uses the NiftyNetworking library for networking
>> functionality, and it uses the KDE libraries for user interface work.
>> Well, good thing, the IPS repo contains both NiftyNetworking and KDE.
>> Download both, install, and then download, compile (via ./configure) and
>> install the WhizzyDownloadTracker. All's good, right?
>>
>> Well, a few hours later, when the compile for WhizzyDownloadTracker
>> finally completes, we find out life isn't so good.
>>
>> Because you see, the NiftyNetworking set was not built with the Apache
>> libstdc++, but instead with the "default" libraries that we have been
>> recommending people use for years. The user tries to run the program,
>> and if he's lucky, gets a meaningful link error message. In reality,
>> the error message, if any, that results is probably completely cryptic
>> to the end user, who was just trying to install software. In a worse
>> scenario, the program just dumps core mysteriously.
>>
>> The worst part of the above scenario, apart from the confusion that the
>> poor non-C++-developing victim gets to experience, is that he's
>> downloaded the foundation libraries from a single source (Sun's IPS
>> repo), so they *should* work together, shouldn't they?!?
>>
>> Too bad.
>>
>> So now the user complains to the FOSS authors for WhizzyDownloadTracker,
>> who are unfamiliar with Solaris (they did their development on Linux,
>> see?), and don't know that Solaris has multiple incompatible C++ ABIs,
>> nor how to tell which components were built against which ABI.
>>
>> So, the FOSS author probably gives up, and tells the poor user that he
>> needs to recompile the base libraries (either KDE or NiftyNetworking),
>> so that they are built with compatible libraries.
>>
>> At this point, the FOSS author is laughing at Sun and Solaris, and
>> probably also cheerfully writes a blog somewhere admonishing folks
>> against running Solaris.
>>
>> And, the original user? He's probably given up. Most likely he either
>> he chooses not to use WhizzyDownloadTracker, or winds up giving up on
>> Solaris and switches back to Linux or Windows. Either way he's probably
>> pretty ticked that "Sun" is delivering crapware that just doesn't work
>> together.
>>
>> I don't know if the above illustration clearly enough paints the picture
>> of my features, but I think it certainly demonstrates that we cannot
>> just blithely ignore the issue and pretend it won't affect anyone.
>>
>> -- Garrett
>>
>> Stefan Teleman wrote:
>>>
>>>
>>> Garrett D'Amore wrote:
>>>> John Plocher wrote:
>>>>> Garrett D'Amore wrote:
>>>>>> One of the implications of such a binding (Volatile), is that
>>>>>> projects which build other C++ shared libraries upon this one
>>>>>> cannot have a commitment level higher than Volatile either.
>>>>>
>>>>> Braap. Bad Architecture Alert. The whole reason we provide
>>>>> abstractions
>>>>> like consolidations and components is precisely so that we can
>>>>> provide
>>>>> "higher than Volatile" expectations for things that theselves may
>>>>> exhibit
>>>>> "less than Volatile" stability.
>>>>>
>>>>> There is no reason this couldn't be made a part of the KDE
>>>>> consolidation,
>>>>> and maintained by them as Committed interfaces for use by any KDE
>>>>> consumers
>>>>> who need it. Volatile means "can change", not "will change", and
>>>>> both the
>>>>> Apache C++ Lib and the KDE projects certainly seem to meet the basic
>>>>> ARC
>>>>> expectations of managing the compatible evolution of their component.
>>>>>
>>>>> If the C++ basis that KDE builds upon were to change incompatibly,
>>>>> I'd
>>>>> expect KDE to react by producing a major release - again, just
>>>>> like the
>>>>> ARC would expect.
>>>>>
>>>>> Nothing here requires KDE to be Volatile.
>>>>
>>>> We're not talking about something that is delivered with
>>>> Consolidation Private binding... we're talking about (assuming
>>>> Volatile binding) something that could change underneath KDE. Such a
>>>> change would be devastating to binary compatibility for applications
>>>> linking against KDE C++ libraries.
>>>>
>>>> If KDE has a way to shield applications underneath from such a binary
>>>> breakage, then its a different story altogether. However, my
>>>> understanding is that in this case (unlike more simple cases
>>>> involving only C), there isn't a way for KDE to do that.
>>>
>>> This ARC Case is not about KDE, but about the Standard C++ Library.
>>> What KDE may or may not expose in its header files, and how it handles
>>> implementation delegation design patterns is for a different ARC Case.
>>>
>>>> I.e. with C++ code, if application #include's a KDE header, which
>>>> itself #include's a libstdc++ header, the binary bits of the
>>>> application *very* likely contain details of the underlying libstdc++
>>>> implementation encoded in them.
>>>
>>> It is the responsibility of the implementation to shield any private
>>> and potentially incompatible implementation details from the publicly
>>> exported interfaces. For the purposes of this statement, "interfaces"
>>> refers to both source and binary.
>>>
>>> The principle of separating interface from implementation is one of
>>> the Design Principles of the C++ Programming Language, and has been a
>>> C++ software design principle ever since the creation of the Language.
>>> It has been widely discussed and documented in relevant literature,
>>> and it has also been put in practice by many C++ software systems,
>>> including, but not limited to, the Apache Standard C++ Library, and/or
>>> the existing libCstd.so.1.
>>>
>>>> C++ is reasonably good at providing good programmatic boundaries
>>>> between interface and implementation at the *source* code level.
>>>> However, it falls down completely at the *binary* level. (Which
>>>> isn't to say that libraries simply *can't* prevent this sort of
>>>> problem -- merely that the expectation should *not* be that they do,
>>>> because it will probably require some rather grotesque contortions on
>>>> the part of the library to do so.)
>>>
>>> The private implementation details of any particular C++ software
>>> system are just that: private. The blanket statement that C++ falls
>>> down completely at the binary compatibility level is false, and it is
>>> invalidated by existing software practice. It is indeed possible to
>>> maintain binary compatibility for a C++ software system, and no
>>> grotesque contortions are required to achieve this goal.
>>>
>>>> Now, if the library (KDE) never #include's "standard" C++ headers
>>>> (provided by this library) in its own headers (that it exports to
>>>> applications), but only uses them in .cxx (or .cpp or .C or whatever)
>>>> implementation files, then I agree that there is no problem.
>>>> (Although the consuming application may itself still wind up needing
>>>> to have its own dependency upon the libstdc++, but that issue is
>>>> largely orthogonal as far as something like KDE is concerned.)
>>>
>>> The dependency constraint has already been clearly stated in the ARC
>>> Case.
>>>
>>> Although it is generally considered a poor software implementation
>>> choice to #include Standard C++ Library header files in application
>>> header files exporting public interfaces, the implementation of the
>>> Apache Standard C++ Library allows for this inclusion, without
>>> breaking ABI [ pursuant to the compatibility constraints described in
>>> the ARC Case Materials ]. However, the Language allows for the
>>> implicit inclusion of interfaces from the Standard C++ Library [ or
>>> for that matter, any other library ], in header files, without the
>>> need for explicit #include directives.
>>>
>>> The Library incompatibility constraint is still in effect: the Apache
>>> Standard C++ Library is not compatible with:
>>>
>>> - any implementation of the Apache Standard C++ Library, which is not
>>> at Major Release 4 level
>>> - any _other_ implementation of the Standard C++ Library, including,
>>> but not limited to: libCstd.so, the GNU Standard C++ Library, STLport,
>>> etc.
>>>
>>>> To put this in comparison, imagine if almost all of the standard C
>>>> functions were simply *macros* rather than functions, and the macros
>>>> made references to volatile innards of the C library. While the
>>>> *API* might be "clean" and safe, the *ABI* would most certainly not
>>>> be. This is the situation that we're in with C++, I think.
>>>
>>> Intentionally, and by Language Design, C++ inline functions, or
>>> templates (which is what you are referring to) are *NOT* C macros. C++
>>> inline functions are functions. C++ templates are templates. Their
>>> symbols are mangled, they obey all the language overloading rules,
>>> they are assigned the implicit "this" pointer by the compiler (if they
>>> are class members), and they behave exactly in the same way as any
>>> other C++ function. They [ classes ] also implement a distinct type.
>>>
>>> The complexity of the implementation of C++ functions increases in the
>>> case where the class, or function in question, is a template (which is
>>> the case with the majority of the classes in the Standard C++
>>> Library). If the class is a template, the compiler instantiates a
>>> distinct type, based on the type defined by the class template itself,
>>> and on the underlying type upon which the template class is
>>> instantiated. This instantiation mechanism alone is fundamentally
>>> different than that of C macros.
>>>
>>> The type instantiation mechanism applies to non-template functions and
>>> classes as well.
>>>
>>> The compiler may or may not decide to eliminate the function call
>>> altogether, by inlining in the resulting object file, and this is a
>>> private decision of the compiler. The compiler may or may not decide
>>> to eliminate the instantiation of the template [ class or function ]
>>> altogether, based on internal compiler heuristic rules, and/or based
>>> on whether or not the actual template function or template class is
>>> actually referenced in the translation unit [ the default compiler
>>> decision can be overridden with specific compiler flags, but that
>>> facility is besides the point for this discussion ]. This decision,
>>> again, is a private decision of the compiler.
>>>
>>> At this point, your comparison with C macros has fundamentally broken
>>> down: there is no guarantee whatsoever that any of the template
>>> classes or functions, whose interfaces have been imported by the
>>> translation unit via header files, would have actually created the
>>> required type and its corresponding instance, and/or that the compiler
>>> has, in fact, generated the corresponding symbol(s) in the binary
>>> object.
>>>
>>> The means by which the implementation achieves the asserted ABI
>>> compatibility goal is private to the implementation itself. In this
>>> particular case, under discussion, this compatibility is enforced by
>>> function calls to private, internal implementations of the facilities
>>> required by the Language Standard (hence the presence of the shared
>>> library object).
>>>
>>> Attempting to invalidate the ABI compatibility assertion of the
>>> implementation by drawing comparisons with C Language macros, or with
>>> the C Programming Language in general, is based on incorrect
>>> assumptions about the C++ Programming Language, and is bound to fail
>>> scrutiny.
>>>
>>> I am hereby requesting that the PSARC member who has derailed this
>>> case provides concrete proof of ABI breakage in the Apache Standard
>>> C++ Library, to the PSARC Committee, for review. Concrete proof means:
>>> source code, accompanied by an explanation of the breakage.
>>>
>>> For The Record: KDE has no intention whatsoever to modify the Standard
>>> C++ Library in an incompatible way.
>>>
>>> Thank you.
>>>
>>> --Stefan
>>>
>>
>> _______________________________________________
>> opensolaris-arc mailing list
>> opensolaris-arc at opensolaris.org
>
