On Sun, Jul 1, 2012 at 8:32 PM, Nathaniel Smith <n...@pobox.com> wrote:
> On Sun, Jul 1, 2012 at 7:36 PM, David Cournapeau <courn...@gmail.com> wrote:
>> On Sun, Jul 1, 2012 at 6:36 PM, Nathaniel Smith <n...@pobox.com> wrote:
>>> On Wed, Jun 27, 2012 at 9:05 PM, David Cournapeau <courn...@gmail.com>
>>> wrote:
>>>> On Wed, Jun 27, 2012 at 8:53 PM, Nathaniel Smith <n...@pobox.com> wrote:
>>>>> But seriously, what compilers do we support that don't have
>>>>> -fvisibility=hidden? ...Is there even a list of compilers we support
>>>>> available anywhere?
>>>>
>>>> Well, I am not sure how all this is handled on the big guys (bluegen
>>>> and co), for once.
>>>>
>>>> There is also the issue of the consequence on statically linking numpy
>>>> to python: I don't what they are (I would actually like to make
>>>> statically linked numpy into python easier, not harder).
>>>
>>> All the docs I can find in a quick google seem to say that bluegene
>>> doesn't do shared libraries at all, though those may be out of date.
>>>
>>> Also, it looks like our current approach is not doing a great job of
>>> avoiding symbol table pollution... despite all the NPY_NO_EXPORTS all
>>> over the source, I still count ~170 exported symbols on Linux with
>>> numpy 1.6, many of them with non-namespaced names
>>> ("_n_to_n_data_copy", "_next", "npy_tan", etc.) Of course this is
>>> fixable, but it's interesting that no-one has noticed. (Current master
>>> brings this up to ~300 exported symbols.)
>>>
>>> It sounds like as far as our "officially supported" platforms go
>>> (linux/windows/osx with gcc/msvc), then the ideal approach would be to
>>> use -fvisibility=hidden or --retain-symbols-file to convince gcc to
>>> hide symbols by default, like msvc does. That would let us remove
>>> cruft from the source code, produce a more reliable result, and let us
>>> use the more convenient separate build, with no real downsides.
>>
>> What cruft would it allow us to remove ? Whatever method we use, we
>> need a whitelist of symbols to export.
>
> No, right now we don't have a whitelist, we have a blacklist -- every
> time we add a new function or global variable, we have to remember to
> add a NPY_NO_EXPORT tag to its definition. Except the evidence says
> that we don't do that reliably. (Everyone always sucks at maintaining
> blacklists, that's the nature of blacklists.) I'm saying that we'd
> better off if we did have a whitelist. Especially since CPython API
> makes maintaining this whitelist so very trivial -- each module
> exports exactly one symbol!
There may be some confusion on what NPY_NP_EXPORT does: it marks a
function that can be used between compilation units but is not
exported. The choice is between static and NPY_NO_EXPORT, not between
NPY_NO_EXPORT and nothing. In that sense, marking something
NPY_NO_EXPORT is a whitelist.
If we were to use -fvisibility=hidden, we would still need to mark
those functions static (as it would otherwise publish functions in the
single file build).
>
> Yes, of course, or I wouldn't have bothered researching it. But this
> research would have been easier if there were enough of a user base
> that the tools makers actually paid any attention to supporting this
> use case, is all I was saying :-).
>
>>> Of course there are presumably other platforms that we don't support
>>> or test on, but where we have users anyway. Building on such a
>>> platform sort of intrinsically requires build system hacks, and some
>>> equivalent to the above may well be available (e.g. I know icc
>>> supports -fvisibility). So I while I'm not going to do anything about
>>> this myself in the near future, I'd argue that it would be a good idea
>>> to:
>>> - Switch the build-system to export nothing by default when using
>>> gcc, using -fvisibility=hidden
>>> - Switch the default build to "separate"
>>> - Leave in the single-file build, but not "officially supported",
>>> i.e., we're happy to get patches but it's not used on any systems that
>>> we can actually test ourselves. (I suspect it's less fragile than the
>>> separate build anyway, since name clashes are less common than
>>> forgotten include files.)
>>
>> I am fine with making the separate build the default (I have a patch
>> somewhere that does that on supported platforms), but not with using
>> -fvisibility=hidden. When I implemented the initial support around
>> this, fvisibility was buggy on some platforms, including mingw 3.x
>
> It's true that mingw doesn't support -fvisibility=hidden, but that's
> because it would be a no-op; windows already works that way by
> default...
That's not my understanding: gcc behaves on windows as on linux (it
would break too many softwares that are the usual target of mingw
otherwise), but the -fvisibility flag is broken on gcc 3.x. The more
recent mingw supposedly handle this better, but we can't use gcc 4.x
because of another issue regarding private dll sharing :)
David
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
