On Mon, Nov 26, 2018 at 2:16 PM Adam Farley8 <adam.far...@uk.ibm.com> wrote:
>
> Hi Volker,
>
> Apologies for the delay.
>
> I ran the contents of the file as requested (neat tip, thanks!) and I
> discovered something:
>
> If jdk_internal_jimage_NativeImageBuffer.h contains this:
>
> -----
> extern "C" {
> __attribute__((visibility("default"))) jobject JNICALL
> Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap
> (JNIEnv *, jclass, jstring);
> }
> -----
>
> It results in this error:
>
> -----
> blah blah "visibility" may be undeclared or ambiguous.
> -----
>
> But replacing that bit with this code:
>
> -----
> extern "C" __attribute__((visibility("default"))) jobject JNICALL
> Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap
> (JNIEnv *, jclass, jstring);
> -----
>
> Results in no error.
>
> So it seems the difference between an "extern "C"" block and an inline
> "extern"C"" is what's causing the issue.
>
Thanks for finding this out. Now at least we know the root cause of the problem.
Without having read through the C++ specification, I'd assume this is
a problem of XLC 13.
As long as XLC has problems to parse such kinds of constructs, I thing
we should just remove "-qvisibility" from the AIX build as proposed by
Magnus (and called "plan B" in you other mail) because changing the
javah generator for AIX would be a much larger task.
Thank you and best regards,
Volker
> I don't understand why, but there we have it.
>
> A shame the header file is generated. An ideal fix would either be to:
>
> 1) Change xlC to make this work inside an extern "C" block.
> or
> 2) Change the way the header file is generated to add that extern "C" bit
> onto the line itself.
>
> However both of those have the potential to break other stuff.
>
> I'm reaching out to the xlC developers now to figure out what's going on, but
> in the
> meantime it seems to me like the proposed fix is still the right way to go
> for a short-term
> resolution.
>
> What are your thoughts?
>
> Best Regards
>
> Adam Farley
> IBM Runtimes
>
>
> Volker Simonis <volker.simo...@gmail.com> wrote on 22/11/2018 14:25:04:
>
> > From: Volker Simonis <volker.simo...@gmail.com>
> > To: adam.far...@uk.ibm.com
> > Cc: Java Core Libs <core-libs-dev@openjdk.java.net>, "Stuefe,
> > Thomas" <thomas.stu...@gmail.com>
> > Date: 22/11/2018 14:25
> > Subject: Re: RFR: JDK-8214063: OpenJDK will not build on AIX while
> > using the xlc 13.1 compiler
> >
> > On Thu, Nov 22, 2018 at 3:00 PM Adam Farley8 <adam.far...@uk.ibm.com> wrote:
> > >
> > > Hi Volker,
> > >
> > > 1) Here is the "reasonable" code in the generated
> > jdk_internal_jimage_NativeImageBuffer.h
> > >
> > > ------------------------------------------------------
> > > /* DO NOT EDIT THIS FILE - it is machine generated */
> > > #include <jni.h>
> > > /* Header for class jdk_internal_jimage_NativeImageBuffer */
> > >
> > > #ifndef _Included_jdk_internal_jimage_NativeImageBuffer
> > > #define _Included_jdk_internal_jimage_NativeImageBuffer
> > > #ifdef __cplusplus
> > > extern "C" {
> > > #endif
> > > /*
> > > * Class: jdk_internal_jimage_NativeImageBuffer
> > > * Method: getNativeMap
> > > * Signature: (Ljava/lang/String;)Ljava/nio/ByteBuffer;
> > > */
> > > JNIEXPORT jobject JNICALL
> > Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap
> > > (JNIEnv *, jclass, jstring);
> > >
> > > #ifdef __cplusplus
> > > }
> > > #endif
> > > #endif
> > > ------------------------------------------------------
> > >
> > >
> > > 2) I have not yet reported this as a bug to the xlc developers. I
> > will contact
> > > them now.
> > >
> > > 3) I did some experimenting, and it seems that the
> > NativeImageBuffer.cpp change
> > > is the only thing standing between us and a successful compilation
> > on aix using
> > > xlc 13.1 (assuming you're using source that compiles on aix with xlc
> > > 12.1).
> > >
> > > With that change (plus the jni_md change), the compilation completes.
> > >
> > > Without that change (after you've added the jni_md change though),the
> > > build
> > > will fail with this error message:
> > >
> > > ------------------------------------------------------
> > > 12:19:58 "/workspace/build/aix-ppc64-normal-server-release/
> > support/headers/java.base/jdk_internal_jimage_NativeImageBuffer.h",
> > line 15.27: 1540-0040 (S) The text
> > "Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap" is
> > unexpected. "visibility" may be undeclared or ambiguous.
> > > 12:19:59 CoreLibraries.gmk:192: recipe for target '/workspace/
> > build/aix-ppc64-normal-server-release/support/native/java.base/
> > libjimage/NativeImageBuffer.o' failed
> > > ------------------------------------------------------
> > >
> >
> > Can you please do the following:
> > - take the command line from
> > /workspace/build/aix-ppc64-normal-server-release/support/native/
> > java.base/libjimage/NativeImageBuffer.o.cmdline
> > - replace '-c' with '-E' to get the preprocessor output
> > - have a look at the offending line (e.g. have JNIEXPORT / JNICALL
> > been correctly expanded ?)
> >
> > Unfortunately I don't have a version of XLC 13 to test this.
> >
> > > Best Regards
> > >
> > > Adam Farley
> > > IBM Runtimes
> > >
> > > P.S. Tried making a small, stand-alone example and it failed to
> > reproduce the problem.
> > > Will keep trying, and I'll supply a further update in the event of
> > a) results,
> > > or b) a response from the xlc guys.
> > >
> > >
> > > Volker Simonis <volker.simo...@gmail.com> wrote on 21/11/2018 14:07:07:
> > >
> > > > From: Volker Simonis <volker.simo...@gmail.com>
> > > > To: adam.far...@uk.ibm.com
> > > > Cc: Java Core Libs <core-libs-dev@openjdk.java.net>, "Stuefe,
> > > > Thomas" <thomas.stu...@gmail.com>
> > > > Date: 21/11/2018 14:07
> > > > Subject: Re: RFR: JDK-8214063: OpenJDK will not build on AIX while
> > > > using the xlc 13.1 compiler
> > > >
> > > > On Wed, Nov 21, 2018 at 1:46 PM Adam Farley8
> > <adam.far...@uk.ibm.com> wrote:
> > > > >
> > > > > Hi Volker,
> > > > >
> > > > > The NativeImageBuffer.cpp changes are best explained by the full text
> > > > > of
> > > > > the referenced GitHub Pull Request, copied here for simplicity:
> > > > >
> > > > > -----------------------------------------
> > > > > Define JNIEXPORT and JNIIMPORT for xlc version 13.1 or newer.
> > Without this,
> > > > > almost no symbols are exported from shared libraries due to use of
> > > > > -qvisibility=hidden as specified in make/lib/LibCommon.gmk. The
> > > > > symptoms
> > > > > are reported in eclipse/openj9#2468.
> > > > >
> > > > > Unfortunately, this encounters a bug in xlc: it fails to parsewhat
> > > > > seems
> > > > > to be reasonable code.
> > > >
> > > > Sorry, but I don't see how this answers my question.
> > > >
> > > > 1. Which "reasonable code" does xlc fails to parse. A stand-alone
> > > > example would be nice.
> > > >
> > > > 2. Have you reported this as bug to the xlc developers? What didthey
> > > > say?
> > > >
> > > > 3. "jdk_internal_jimage_NativeImageBuffer.h" doesn't seem to be
> > > > special. It's a plain, generated JNI header file as generated by
> > > > 'javah' or 'javac -h'. If XLC 13 has problems parsing it, there should
> > > > be much more places which need fixing. So what's special about
> > > > "jdk_internal_jimage_NativeImageBuffer.h".
> > > >
> > > > In the referenced pull request
> > > > (https://urldefense.proofpoint.com/v2/url?
> > > >
> > u=https-3A__github.com_eclipse_openj9_issues_2468&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > > > siA1ZOg&r=P5m8KWUXJf-
> > > >
> > CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=sgfFd6mB1EYM4nOM89rgFFzUyX7B21XbckIY7L0kUNU&s=TJ-4nr8ikZKImwDygirRTxLybsnQWBN71nEZCwZ59NQ&e=
> > > > ) I can only see linker
> > > > errors (and no compiler errors). The linker errors are for both
> > > > libjsig and libjava. They are related to the symbol ".sigaction" in
> > > > jsig.o and I don't see how this should be related to
> > > > NativeImageBuffer.cpp or "jdk_internal_jimage_NativeImageBuffer.h".
> > > > NativeImageBuffer.cpp is only used to create libjimage and not related
> > > > in any way to libjsig or libjava.
> > > >
> > > > It seems wired to do the change to NativeImageBuffer.cpp which you've
> > > > proposed without understanding the real cause of the problem.
> > > >
> > > > Regards,
> > > > Volker
> > > >
> > > > > A workaround is required in just one place:
> > > > > src/java.base/share/native/libjimage/NativeImageBuffer.cpp.
> > > > > -----------------------------------------
> > > > >
> > > > > Best Regards
> > > > >
> > > > > Adam Farley
> > > > > IBM Runtimes
> > > > >
> > > > >
> > > > > Volker Simonis <volker.simo...@gmail.com> wrote on 20/11/2018
> > > > > 17:50:41:
> > > > >
> > > > > > From: Volker Simonis <volker.simo...@gmail.com>
> > > > > > To: "Stuefe, Thomas" <thomas.stu...@gmail.com>
> > > > > > Cc: adam.far...@uk.ibm.com, Java Core Libs <core-libs-
> > > > d...@openjdk.java.net>
> > > > > > Date: 20/11/2018 17:59
> > > > > > Subject: Re: RFR: JDK-8214063: OpenJDK will not build on AIX while
> > > > > > using the xlc 13.1 compiler
> > > > > >
> > > > > > On Tue, Nov 20, 2018 at 6:15 PM Thomas Stüfe
> > > > <thomas.stu...@gmail.com> wrote:
> > > > > > >
> > > > > > > On Tue, Nov 20, 2018 at 6:12 PM Adam Farley8
> > > > <adam.far...@uk.ibm.com> wrote:
> > > > > > > >
> > > > > > > > Heya Tom,
> > > > > > > >
> > > > > > > > "In JDK11 and JDK12, source files are compiled with -
> > > > qvisibility=hidden
> > > > > > > > when using xlc version other than 12.1. That doesn't
> > seem toplay well
> > > > > > > > with link option -bexpall. "
> > > > > > > >
> > > > > > > > Found that buried in one of the associated Git issues.
> > It appears that
> > > > > > > > it's OpenJDK's use of that option that's causing the
> > problem, though
> > > > > > > > I couldn't speculate as to why it was added in the first place.
> > > > > > > >
> > > > > > > > I see this has also been noted in https://
> > > > > > urldefense.proofpoint.com/v2/url?
> > > > > >
> > > >
> > u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8204541&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > > > > > siA1ZOg&r=P5m8KWUXJf-
> > > > > >
> > > >
> > CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=SD6UdjysISJRBlWUm8pEzF5lRZ5opfbrKzEh_jrOras&s=5qDEdIfg8qZ-
> > > > > > vCglsZ9qNDTEPMnCkj-mVPVah6eEDLE&e=
> > > > > > > >
> > > > > > > > Does that answer your question?
> > > > > > > >
> > > > > > >
> > > > > > > Yes, Thank you. Odd. Will have to do archeology on that one.
> > > > > > >
> > > > > >
> > > > > > No I begin to understand the problem as well :)
> > > > > >
> > > > > > It was actually change "8202322: AIX: symbol visibility flags not
> > > > > > support on xlc 12.1" [1] which introduced "-qvisibility=hidden" for
> > > > > > XLC version not equal to 12.1. That's kind of a weak check and I
> > > > > > suppose nobody has ever tested this change with an XLC version other
> > > > > > than 12.1 (until you came along :). Maybe that check should be a
> > > > > > more
> > > > > > precisly check for >= 13.1 (but I know such version checks are hard
> > > > > > to
> > > > > > do in Makefile syntax)?
> > > > > >
> > > > > > The thing I don't understand about your patch (the changes in
> > > > > > "jni_md.h" look good although I haven't tested them) is why you need
> > > > > > the extra changes in NativeImageBuffer.cpp?
> > > > > > "jdk_internal_jimage_NativeImageBuffer.h" is a plain, generated JNI
> > > > > > header file. If XLC 13 has problems to parse it, there should be
> > > > > > much
> > > > > > more places which need fixing. I think that part of your change
> > > > > > needs
> > > > > > a closer evaluation.
> > > > > >
> > > > > > Thank you and best regards,
> > > > > > Volker
> > > > > >
> > > > > > [1] https://urldefense.proofpoint.com/v2/url?
> > > > > >
> > > >
> > u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8202322&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > > > > > siA1ZOg&r=P5m8KWUXJf-
> > > > > >
> > > >
> > CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=SD6UdjysISJRBlWUm8pEzF5lRZ5opfbrKzEh_jrOras&s=JAEK6rePGMPinZzOquHBzj5oc7vA3kaFt9x0WIIUzvk&e=
> > > > > >
> > > > > > > ..Thomas
> > > > > > >
> > > > > > > > Best Regards
> > > > > > > >
> > > > > > > > Adam Farley
> > > > > > > > IBM Runtimes
> > > > > > > >
> > > > > > > >
> > > > > > > > "Thomas Stüfe" <thomas.stu...@gmail.com> wrote on 20/11/
> > 201816:44:07:
> > > > > > > >
> > > > > > > > > From: "Thomas Stüfe" <thomas.stu...@gmail.com>
> > > > > > > > > To: Adam Farley8 <adam.far...@uk.ibm.com>
> > > > > > > > > Cc: Java Core Libs <core-libs-dev@openjdk.java.net>
> > > > > > > > > Date: 20/11/2018 16:48
> > > > > > > > > Subject: Re: RFR: JDK-8214063: OpenJDK will not build
> > on AIX while
> > > > > > > > > using the xlc 13.1 compiler
> > > > > > > > >
> > > > > > > > > Hi Adam,
> > > > > > > > >
> > > > > > > > > On Tue, Nov 20, 2018 at 5:12 PM Adam Farley8
> > > > > > <adam.far...@uk.ibm.com> wrote:
> > > > > > > > > >
> > > > > > > > > > Hi Tom,
> > > > > > > > > >
> > > > > > > > > > Sounds reasonable. I've added a webex to the bug, and
> > > > > > > > > > here's a
> > > > > > > > > link to the bug.
> > > > > > > > > >
> > > > > > > > > > https://urldefense.proofpoint.com/v2/url?
> > > > > > > > >
> > > > > >
> > > >
> > u=https-3A__bugs.openjdk.java.net_browse_JDK-2D8214063&d=DwIFaQ&c=jf_iaSHvJObTbx-
> > > > > > > > > siA1ZOg&r=P5m8KWUXJf-
> > > > > > > > >
> > > > > >
> > > >
> > CeVJc0hDGD9AQ2LkcXDC0PMV9ntVw5Ho&m=z8YYwBXEfN7UtX1suPjpp9CZSHf8v0GrIMK3XGIC9VY&s=81TP9mIjhYD2Hmt8g7p2EHWRZXgiep21hxKLYRU7zIQ&e=
> > > > > > > > > >
> > > > > > > > > > This patch is required because otherwise, when building on
> > > > > > > > > > AIX
> > > > > > > > > using xlc 3.1,
> > > > > > > > > > the build fails with this error:
> > > > > > > > > >
> > > > > > > > > > "Visibility is not allowed on a reference to an
> > imported symbol."
> > > > > > > > > >
> > > > > > > > > > We believe this is caused by JNIEXPORT and JNIIMPORTnot
> > > > > > > > > > being
> > > > > > > > > defined. Without
> > > > > > > > > > this, almost no symbols are exported from shared libraries
> > > > > > due to use of
> > > > > > > > > > -qvisibility=hidden as specified in make/lib/LibCommon.gmk.
> > > > > > > > >
> > > > > > > > > Yes but what I try to understand is why does this
> > happen now with
> > > > > > > > > xlc13? Did xlc change the rules for -qvisibility from
> > v12 to v13 ?
> > > > > > > > > That would be quite a break in backward compatibility.
> > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > For convenience, here's a summary of the diffs:
> > > > > > > > > >
> > > > > > > > > > --------------------------------------
> > > > > > > > > > File 1 of 2) src/java.base/share/native/libjimage/
> > > > > > NativeImageBuffer.cpp
> > > > > > > > > >
> > > > > > > > > > #include "osSupport.hpp"
> > > > > > > > > >
> > > > > > > > > > +#if defined(__xlC__) && (__xlC__ >= 0x0d01)
> > > > > > > > > > +/*
> > > > > > > > > > + * Version 13.1.3 of xlc seems to have trouble parsing the
> > > > > > `__attribute__`
> > > > > > > > > > + * annotation in the generated header file we're about to
> > > > > > > > > include. Repeating
> > > > > > > > > > + * the forward declaration (without the braces) here avoids
> > > > > > the diagnostic:
> > > > > > > > > > + * 1540-0040 (S) The text "void" is unexpected.
> > "visibility"
> > > > > > > > > may be undeclared or ambiguous.
> > > > > > > > > > + */
> > > > > > > > > > +extern "C" JNIEXPORT jobject JNICALL
> > > > > > > > >
> > Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap(JNIEnv *,
> > > > > > > > > jclass, jstring);
> > > > > > > > > > +#endif
> > > > > > > > > > +
> > > > > > > > > > #include "jdk_internal_jimage_NativeImageBuffer.h"
> > > > > > > > > > --------------------------------------
> > > > > > > > > > File 2 of 2) src/java.base/unix/native/include/jni_md.h
> > > > > > > > > >
> > > > > > > > > > #define JNIIMPORT __attribute__((visibility
> > ("default")))
> > > > > > > > > > #endif
> > > > > > > > > > +#elif defined(__xlC__) && (__xlC__ >= 0x0d01) /*
> > xlc version 13.1
> > > > > > > > > or better required */
> > > > > > > > > > + #define JNIEXPORT __attribute__((visibility
> > ("default")))
> > > > > > > > > > + #define JNIIMPORT __attribute__((visibility
> > ("default")))
> > > > > > > > > > #else
> > > > > > > > > > #define JNIEXPORT
> > > > > > > > > > --------------------------------------
> > > > > > > > > >
> > > > > > > > >
> > > > > > > > > Thank you.
> > > > > > > > >
> > > > > > > > > Cheers, Thomas
> > > > > > > > >
> > > > > > > > > > Best Regards
> > > > > > > > > >
> > > > > > > > > > Adam Farley
> > > > > > > > > > IBM Runtimes
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > "Thomas Stüfe" <thomas.stu...@gmail.com> wrote on 19/11/
> > > > 201818:11:34:
> > > > > > > > > >
> > > > > > > > > > > From: "Thomas Stüfe" <thomas.stu...@gmail.com>
> > > > > > > > > > > To: Adam Farley8 <adam.far...@uk.ibm.com>
> > > > > > > > > > > Cc: Java Core Libs <core-libs-dev@openjdk.java.net>
> > > > > > > > > > > Date: 19/11/2018 18:12
> > > > > > > > > > > Subject: Re: RFR: JDK-8214063: OpenJDK will not build
> > > > on AIX while
> > > > > > > > > > > using the xlc 13.1 compiler
> > > > > > > > > > >
> > > > > > > > > > > Hi Adam,
> > > > > > > > > > >
> > > > > > > > > > > could you please include link to the JBS issue and either
> > > > > > link to the
> > > > > > > > > > > patch/webrev or link to the webrev, or at the very
> > > > least the patch
> > > > > > > > > > > verbatim?
> > > > > > > > > > >
> > > > > > > > > > > As for the issue itself: could you please elaborate why
> > > > > > > > > > > this
> > > > > > > > > fails with xlc13?
> > > > > > > > > > >
> > > > > > > > > > > Also, a real patch would be helpful instead here of
> > > > yet another link
> > > > > > > > > > > to some J9 issue. We are really strapped for manpower and
> > > > > > the AIX port
> > > > > > > > > > > eats up enough time as it is.
> > > > > > > > > > >
> > > > > > > > > > > Thanks, Thomas
> > > > > > > > > > >
> > > > > > > > > > > On Mon, Nov 19, 2018 at 6:28 PM Adam Farley8
> > > > > > > > > <adam.far...@uk.ibm.com> wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > Hi All
> > > > > > > > > > > >
> > > > > > > > > > > > Both the problem and the solution appear straight-
> > > > forward enough.
> > > > > > > > > > > >
> > > > > > > > > > > > Details included in the bug description.
> > > > > > > > > > > >
> > > > > > > > > > > > Thoughts and opinions welcome.
> > > > > > > > > > > >
> > > > > > > > > > > > Best Regards
> > > > > > > > > > > >
> > > > > > > > > > > > Adam Farley
> > > > > > > > > > > > IBM Runtimes
> > > > > > > > > > > >
> > > > > > > > > > > > Unless stated otherwise above:
> > > > > > > > > > > > IBM United Kingdom Limited - Registered in England and
> > > > > > Wales with number
> > > > > > > > > > > > 741598.
> > > > > > > > > > > > Registered office: PO Box 41, North Harbour, Portsmouth,
> > > > > > > > > Hampshire PO6 3AU
> > > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Unless stated otherwise above:
> > > > > > > > > > IBM United Kingdom Limited - Registered in England
> > and Wales with
> > > > > > > > > number 741598.
> > > > > > > > > > Registered office: PO Box 41, North Harbour, Portsmouth,
> > > > > > Hampshire PO6 3AU
> > > > > > > > >
> > > > > > > >
> > > > > > > > Unless stated otherwise above:
> > > > > > > > IBM United Kingdom Limited - Registered in England and Wales
> > > > > > with number 741598.
> > > > > > > > Registered office: PO Box 41, North Harbour, Portsmouth,
> > > > Hampshire PO6 3AU
> > > > > >
> > > > >
> > > > > Unless stated otherwise above:
> > > > > IBM United Kingdom Limited - Registered in England and Wales with
> > > > number 741598.
> > > > > Registered office: PO Box 41, North Harbour, Portsmouth,
> > Hampshire PO6 3AU
> > > >
> > >
> > > Unless stated otherwise above:
> > > IBM United Kingdom Limited - Registered in England and Wales with
> > number 741598.
> > > Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
> >
>
> Unless stated otherwise above:
> IBM United Kingdom Limited - Registered in England and Wales with number
> 741598.
> Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU
