On 2024-02-14 11:06, Magnus Ihse Bursie wrote:
I am currently pursuing improved build functionality for static
libraries. One of the issues with static libraries are name
collisions, which led me back to an old discussion about which symbols
are exported from Hotspot, and how this is achieved. A long discussion
is available in JDK-8017234 [1], which was created in 2013 and has
been on the back burner ever since, coming back to life every now and
then.
There are basically two different problems here. They are both
distinct and interrelated, and we likely need to solve both in tandem.
The first problem is how Hotspot should say which symbols (functions)
that should be exported from libjvm.so. The historical, and still
used, system is to have a mapfile. In the "new" (not so new anymore)
build system, this was simplified to a list of function names in
make/data/hotspot-symbols, from which a mapfile is built.
This is in contrast with how all other libraries in the JDK are built,
and also with modern best practices. A better approach would be to
annotate the functions that should be exported in the source code. In
fact, such annotation already exists, even in Hotspot, as JNIEXPORT,
but this is currently not used in the compilation of Hotspot.
The second problem is that in addition to these explicitly exported
functions, we also export basically all other functions as well in
Hotspot. (So currently we could basically just forgo this complicated
machinery and just export everything by default...)
The reason for this seem somewhat unclear. The specifics are probably
forever lost in the mists of time past, but the essential point is
that these symbols are needed for SA to do it's job.
So what we do is that we list all symbols in hotspot after compiling
(but before linking), and selects some (most, I think) of them using
regexp patterns, and add these to the mapfile.
Actually, we are not doing what I thought we're doing, and what I assume
whoever wrote the original code thought we were doing. Seems no-one has
ever bothered to check the effect of this massive generated mapfile. :-/
At least not on linux/gcc (which is the only platform I've looked at so
far).
It turns out that the gcc linker never removes the "hidden" visibility
from symbols. Even if you list them in a mapfile under "global:", if
they were compiled with visibility "hidden", they will still be hidden.
The only thing you can ever hope to achieve with a mapfile is to make
symbols with "default" visibility be "hidden", by placing them under the
"local:" header. In hotspot, only functions marked JNIEXPORT have the
"default" visibility.
The upshot of all this is that the mapfile has very little impact on the
resulting symbols in Hotspot, and a lot of the difference it makes seems
to be incorrect. :-/
In other words, all the mapfile can ever do is hide stuff that would
otherwise have been visible. So what symbols are then made hidden by the
mapfile? In effect, symbols that are marked JNIEXPORT but are not listed
in make/data/hotspot-symbols, and symbols that arise from outside
hotspot source code. I have divided these symbols into four categories:
1) All symbols from debug.cpp that are marked JNIEXPORT (most of them).
2) All symbols from jvmciCompilerToVM.cpp and that are marked JNIEXPORT
(most of them), and data from vmStructs_jvmci.cpp that are marked JNIEXPORT.
3) Assembly functions such as _Copy_arrayof_conjoint_bytes that are
marked .globl
4) Symbols not present in hotspot source code, but which originates with
the compiler or standard libraries, e.g. __bss_start, __cxa_begin_catch,
_ZTIN10__cxxabiv117__class_type_infoE,
_ZTSN9__gnu_cxx20recursive_init_errorE, etc.
My interpretation of this is:
a) It is actually an error not to export functions marked JNIEXPORT, so
1 and 2 are actually more correct this way.
b) The assembly functions are incorrectly marked .globl. This should
just be removed.
c) As for the symbols from compiler and standard libraries; I don't
know. I assume these are exported by default by normal libs, so it
should probably be fine to do the same with Hotspot. Possibly there are
other ways to make these local, if we really want that (linker flags, or
something like that?).
The most important observation, however, is that generating a long list
of symbols from the object file to make them "available" to SA seems to
be a complete misunderstanding of what is happening. No hidden symbols
can ever be made visible in this manner.
/Magnus
