A fascinating thread formed today on a large but private CG
mailing list that branched from a discussion on c++11
to STL ABI issues that I found quite interesting, as well
as the problems of exposing STL in a lib's API and why that
should be avoided.
It's a private list, so I don't think it's right to paste verbatim..
guess I'll have to paraphrase as a reporter would. So this is all
paraphrased by me, sorry for any errors. Complain to my editor ;)
The thread is still in progress, but I think a bit of what
has transpired so far seems generally agreed on as good info,
and therefore relevant to us.
It gets interesting wrt STL/ABI at (5) and (6):
* * *
1) J. writes:
VFX/Animation studios writing their own plug-ins for 3rd party tools
generally have to compile their code with the same compiler as the
app hosing their plugin. (In our chase, Autodesk Maya)
2) S. writes:
Maya 2013 now uses VS2010, but we were able to have our internal
plug-ins built with VS2008. We only recently noticed the docs said
the ABIs shouldn't be compatible for C++ (but OK with C), so we're
surprised it works.
3) M. writes:
We build our plug-ins for Maya/3DSMax and Softimage with one compiler,
regardless of what compiler they used. We use VS2008 with Maya8 thru Maya 2013
without major issues. There were cases where the linker tried to mix CRTs
with the ones we used vs. the ones the main app used causing a crash,
but by massaging the order of libs in the command line and avoiding
troublesome functions, we've gotten things to work.
ABI compatibility is a problem when STL is involved, but I think STL
is bad to use in an API anyway. Maya does this only in one case (MPxData,
and for no apparent good reason) but fortunately MStreamUtils can be used to
avoid touching the iostream objects. There's other issues when a static lib
is using STL internally that's different to the code we write, but we recompile
the lib with VS2008 when source is available, and use the DLL version if its
an option, or avoid the lib altogether. Aside from STL, other parts of the
ABI not normally guaranteed (name mangling, type sizes..) seem in practice
to be compatible between VS versions.
VS2010 has a multiple toolset feature allowing one to use the IDE with
previous versions of the compiler, but I dislike VS2010 because the IDE
is very slow, and one gets no new features for that penalty wrt VS2008.
4) J2 writes:
I agree; ignore the docs, always compile VS projects static (MT)
instead of dynamic (MD) to save headaches.
5) C. writes:
I'm surprised M. thinks STL in an API is bad.
While there are surely ABI compat issues, I'm surprised a standardized
feature like STL, stable for a decade, can't be relied on? The implication
is a template lib can't be used in other lib's APIs.
Is this solvable? Is it windows only?
6) J2 writes:
Templates aren't really standardized. No one actually follows the spec,
though having trouble recalling details. All compilers have their own quirks
when it comes to templates. Not an MS thing. It's a problem with the standards
committee making a standard impossible to actually implement, so everyone
made their own variation.
7) A. writes:
There are some common things for standard templates, such as the boost
docs show a nice OS-specific syntax followed by one not so nice that is
cross compatible.
I too know STL shouldn't get into public APIs, but did get lax about allowing
std::vectors to be exposed in my API, and would then replace them later if
issues arose. Not saying its OK to do this, but this is my solution.
8) M. writes:
The standard defines the interface + complexity constraints for STL,
but doesn't specify an implementation. Each vendor can implement STL
as it sees fit. You can't exchange STL objects between STL implementations.
For instance, one implementation of std::vector might contain 3 members:
array start ptr, end of allocated mem ptr, and a ptr to the last used element.
(The last two often being different, b/c std::vector allocs more space than
needed to amortize cost of successive push_back() calls).
But it's valid to have a ptr to the start of the array and two size_t variables
specifying the alloc'ed space and current total elements. So if an app tries
to mix one STL implementation with another, it will crash due to the different
member contents. Doesn't have to be different types.. just a difference in data
ordering can be an issue.
This isn't just theory. In the Dinkumware STL (used by MS), STL objects can
change based on compiler settings. std::vector (and iterators) have different
members depending on various macros (_HAS_ITERATOR_DEBUGGING and _SECURE_SCL)
which are in turn managed by the _DEBUG macro. Appealing to some if you try
to erase() an iterator belonging to a different vector, or if you run off an
end during an increment.. so even the *same compiler as the lib isn't enough.*
Same build settings are needed.
GCC's STL doesn't have these debug features, so less an issue on Linux/OSX,
but there's no guarantee, as they can change their minds at any time.
Unsure about what 'llvm' uses, but it's probably the same code, or there'd
be issues with 'interop'. But then again, most likely the Intel compiler
has its own version, so you can't mix/match unless you indicate GCC code
is desired.
So distributing a binary lib on Windows with STL in the interface, you
must either provide a build for each compiler with build flag combos you
support, a big pain, or tell people what compiler/flags to use, which doesn't
work.
The issue isn't due to templates. It's OK to expose template containers
because the user of your lib sees the same code the lib saw when it was
built. (Unless you screw up your code with macros the way STL does)
The problem occurs when you call something "vector" (or "istream", "map"..)
and provide different definitions for it, which STL will do without your
knowing it.
It's silly, but that's the state the language is in, and unlikely to get
fixed. Since the committee can't address this stuff, they spend 8 years
designing junk like rvalue references so bad code can sometimes run
a little faster if the compiler deems it worthy, and if you spend more
time implementing how to move data than it takes to write the code
properly in the first place.
9) V. writes:
The only thing standardized in STL is the interface, not the implementation.
The issue not being templates so much as the varying implementations of STL
across compilers and debug/release modes for the same compiler.
For example, debug modes can change the members within the STL class
for tracking issues. And newer versions can reimplement the class for
performance reasons.
Not a windows issue; even linux can involve 3rd party STL implementations
that are not ABI compatible with the lib you build against. Including STL
in your interface forces users to use a particular STL implementation.
The problem isn't solvable.. for STL anyway.
10) C. writes:
Thanks for the info V and M.
Sobering and frustrating, seems I just shouldn't use it.
But what should we do instead, make custom string/vector classes?
This makes designing APIs problematic and increases maintenance cost.
Seems like every lib would have its own string/vectors, all incompatible
yet solving the same problem.
11) H. writes;
Regarding J2's comment "Templates aren't really standardized/no one follows
spec",
they are standardized to a deep level of detail. The issue is compilers are
slow to catch up with the standard. It's a big issue, templates being
Turing-complete
and so on.
The current big difference for non-C++11 template implementations lay in
whether the compiler implements single vs. two-phase template lookups.
The standard has recommended the latter for ~20 years, and is used by
gcc/icc/clang, but MS still uses the former. This makes for broken behavior
when dodgy template code gets past the MS compiler, but not gcc/clang.
And also, valid template code can be rejected by MS.
MS seems to have the worst template support, even the latest, and is
frustrating. For instance this example won't build in MS VS:
------------- 8< --------------
class A
{
template<typename C, template<typename> class B >
typename B<C>::type func() const;
};
template< class C, template<typename> class B>
typename B<C>::type A::func() const
{
return 1;
}
------------- 8< --------------
Output:
1>test.cpp(11): error C2244: 'A::func' : unable to match function
definition to an existing declaration
1> definition
1> 'B<C>::type A::func(void) const'
1> existing declarations
1> 'B<C>::type A::func(void) const'
------------- 8< --------------
What the heck, this should be standard. Builds fine on other compilers.
Just one example of problems encountered during a linux -> windows port.
Their iterator debugging breaking ABI is also a big problem.
I wasted many hours on false positives, and saved none.
12) M. writes:
Regarding C's comments on it all being frustrating, and what to do
about it, maybe use custom classes?
Well, yes, either that or plain old pointers.
To make code run fast, one usually finds STL containers useless anyway,
most making one allocation per element.
If all libraries had their own string class, the user would be converting
between them constantly, often in convoluted ways. Using plain ptrs, methods
get uglier because ptrs and counts must always be passed.
*Choice depends on library scope.*
It is sensible that Maya uses MString, MPoint, MIntArray, etc because it's
a platform, not a lib. *OTOH, it's overkill to roll your own string for an
image lib which just needs a file pathname in a few places.*
API design is further complicated by memory ownership (can't free a ptr
allocated in another DLL), backwards bin compat, etc. Try to solve this
generically reinvents COM, which should be avoided. These issues are
caused by the fact C++ is built on the C linker, which is a very simple
system (makes sense for C). Other languages can hide these issues, but
introduce overhead not acceptable to some apps. C++ seems the lesser evil
compared to eg. python, java, etc.
So when all's said and done, the best API for C++ is to use C,
or at least design in C and add some C++ features to make it pretty.
13) C writes:
But using C prevents us from implementing high level APIs that can
make use of other APIs, since we would keep having to revert to C
ptrs.
The memory ownership issue is important to abstract when creating an API
that provides ways to make complex high-level objects, and C++11 smart
ptrs seem 'right' for doing this. That they can't be relied on due to ABI
is maddening.
This in-depth discussion has really been very educational.. thanks.
14) J2 writes:
Regarding H's comments on mine regarding template being standardized,
I was talking about issues of having template c++ functions being made
available to all source files compiled with them. In other words,
not putting function delcarations into cpp files. And the 'export' keyword
isn't supported by all compilers. This part of the spec no one seems to follow.
Well, except maybe "comeau", which I haven't used. URL:
http://www.comeaucomputing.com/techtalk/templates/#export
* * *
_______________________________________________
fltk-dev mailing list
fltk-dev@easysw.com
http://lists.easysw.com/mailman/listinfo/fltk-dev
