On Mon, Oct 5, 2015 at 11:26 PM, Nathaniel Smith <n...@pobox.com> wrote:
> Hi all, > > For a long time, NumPy has supported two different ways of being compiled: > > "Separate compilation" mode: like most C projects, each .c file gets > compiled to a .o file, and then the .o files get linked together to > make a shared library. (This has been the default since 1.8.0.) > > "One file" mode: first concatenate all the .c files together to make > one monster .c file, and then compile that .c file to make a shared > library. (This was the default before 1.8.0.) > > Supporting these two different build modes creates a drag on > development progress; in particular Stefan recently ran into this in > this experiments with porting parts of the NumPy internals to Cython: > https://github.com/numpy/numpy/pull/6408 > (I suspect the particular problem he's running into can be fixed b/c > so far he only has one .pyx file, but I also suspect that it will be > impossible to support "one file" mode once we have multiple .pyx > files.) > > There are some rumors that "one file" mode might be needed on some > obscure platform somewhere, or that it might be necessary for > statically linking numpy into the CPython executable, but we can't > continue supporting things forever based only on rumors. If all we can > get are rumors, then eventually we have to risk breaking things just > to force anyone who cares to actually show up and explain what they > need so we can support it properly :-). > Assuming one of the rumour is related to some comments I made some time (years ?) earlier, the context was the ability to hide exported symbols. As you know, the issue is not to build extensions w/ multiple compilation units, but sharing functionalities between them without sharing them outside the extension. I am just reiterating that point so that we all discuss under the right context :) I also agree the current situation is not sustainable -- as we discussed privately before, cythonizing numpy.core is made quite more complicated by this. I have myself quite a few issues w/ cythonizing the other parts of umath. I would also like to support the static link better than we do now (do we know some static link users we can contact to validate our approach ?) Currently, what we have in numpy core is the following: numpy.core.multiarray -> compilation units in numpy/core/src/multiarray/ + statically link npymath numpy.core.umath -> compilation units in numpy/core/src/umath + statically link npymath/npysort + some shenanigans to use things in numpy.core.multiarray I would suggest to have a more layered approach, to enable both 'normal' build and static build, without polluting the public namespace too much. This is an approach followed by most large libraries (e.g. MKL), and is fairly flexible. Concretely, we could start by putting more common functionalities (aka the 'core' library) into its own static library. The API would be considered private to numpy (no stability guaranteed outside numpy), and every exported symbol from that library would be decorated appropriately to avoid potential clashes (e.g. '_npy_internal_'). FWIW, that has always been my intention to go toward this when I split up multiarray/umath into multiple .c files and extracted out npymath. cheers, David
_______________________________________________ NumPy-Discussion mailing list NumPy-Discussion@scipy.org https://mail.scipy.org/mailman/listinfo/numpy-discussion
