RFR: JDK-8204127: Change bundle format on Windows to zip
The compressed archive bundles we create are all tar.gz format. On Windows that's not a very common format, requiring third party applications to handle, while the zip format can be natively extracted. This patch changes the bundle format for the JDK on Windows to zip. This only applies to the actual product distributions. I think we prefer tar.gz for all the internal bundles, like symbols and tests. Bug: https://bugs.openjdk.java.net/browse/JDK-8204127 Webrev: http://cr.openjdk.java.net/~erikj/8204127/webrev.01 /Erik
Re: RFR: JDK-8202384: Introduce altserver jvm variant with speculative execution disabled
Doh, thanks, updated in place. /Erik On 2018-06-08 14:53, Daniel D. Daugherty wrote: On 6/8/18 5:50 PM, Erik Joelsson wrote: On 2018-06-07 17:30, David Holmes wrote: On 8/06/2018 6:11 AM, Erik Joelsson wrote: I just don't think the extra work is warranted or should be prioritized at this point. I also cannot think of a combination of options required for what you are suggesting that wouldn't be confusing to the user. If someone truly feels like these flags are forced on them and can't live with them, we or preferably that person can fix it then. I don't think that's dictatorship. OpenJDK is still open source and anyone can contribute. I don't see why --enable-hardened-jdk and --enable-hardened-hotspot to add to the right flags would be either complicated or confusing. For me the confusion surrounds the difference between --enable-hardened-hotspot and --with-jvm-variants=server, hardened and making the user understand it. But sure, it is doable. Here is a new webrev with those two options as I interpret them. Here is the help text: --enable-hardened-jdk enable hardenening compiler flags for all jdk Typo: 'hardenening' -> 'hardening' libraries (except the JVM), typically disabling speculative cti. [disabled] --enable-hardened-hotspot enable hardenening compiler flags for hotspot (all Typo: 'hardenening' -> 'hardening' jvm variants), typically disabling speculative cti. To make hardening of hotspot a runtime choice, consider the "hardened" jvm variant instead of this option. [disabled] Note that this changes the default for jdk libraries to not enable hardening unless the user requests it. Webrev: http://cr.openjdk.java.net/~erikj/8202384/webrev.04/ /Erik Dan
Re: RFR: JDK-8202384: Introduce altserver jvm variant with speculative execution disabled
On 6/8/18 5:50 PM, Erik Joelsson wrote: On 2018-06-07 17:30, David Holmes wrote: On 8/06/2018 6:11 AM, Erik Joelsson wrote: I just don't think the extra work is warranted or should be prioritized at this point. I also cannot think of a combination of options required for what you are suggesting that wouldn't be confusing to the user. If someone truly feels like these flags are forced on them and can't live with them, we or preferably that person can fix it then. I don't think that's dictatorship. OpenJDK is still open source and anyone can contribute. I don't see why --enable-hardened-jdk and --enable-hardened-hotspot to add to the right flags would be either complicated or confusing. For me the confusion surrounds the difference between --enable-hardened-hotspot and --with-jvm-variants=server, hardened and making the user understand it. But sure, it is doable. Here is a new webrev with those two options as I interpret them. Here is the help text: --enable-hardened-jdk enable hardenening compiler flags for all jdk Typo: 'hardenening' -> 'hardening' libraries (except the JVM), typically disabling speculative cti. [disabled] --enable-hardened-hotspot enable hardenening compiler flags for hotspot (all Typo: 'hardenening' -> 'hardening' jvm variants), typically disabling speculative cti. To make hardening of hotspot a runtime choice, consider the "hardened" jvm variant instead of this option. [disabled] Note that this changes the default for jdk libraries to not enable hardening unless the user requests it. Webrev: http://cr.openjdk.java.net/~erikj/8202384/webrev.04/ /Erik Dan
Re: RFR: JDK-8202384: Introduce altserver jvm variant with speculative execution disabled
On 2018-06-07 17:30, David Holmes wrote: On 8/06/2018 6:11 AM, Erik Joelsson wrote: I just don't think the extra work is warranted or should be prioritized at this point. I also cannot think of a combination of options required for what you are suggesting that wouldn't be confusing to the user. If someone truly feels like these flags are forced on them and can't live with them, we or preferably that person can fix it then. I don't think that's dictatorship. OpenJDK is still open source and anyone can contribute. I don't see why --enable-hardened-jdk and --enable-hardened-hotspot to add to the right flags would be either complicated or confusing. For me the confusion surrounds the difference between --enable-hardened-hotspot and --with-jvm-variants=server, hardened and making the user understand it. But sure, it is doable. Here is a new webrev with those two options as I interpret them. Here is the help text: --enable-hardened-jdk enable hardenening compiler flags for all jdk libraries (except the JVM), typically disabling speculative cti. [disabled] --enable-hardened-hotspot enable hardenening compiler flags for hotspot (all jvm variants), typically disabling speculative cti. To make hardening of hotspot a runtime choice, consider the "hardened" jvm variant instead of this option. [disabled] Note that this changes the default for jdk libraries to not enable hardening unless the user requests it. Webrev: http://cr.openjdk.java.net/~erikj/8202384/webrev.04/ /Erik
Re: RFR (XL): JDK-8204572 SetupJdkLibrary should setup SRC and -I flags automatically
Looks good. /Erik On 2018-06-08 01:50, Magnus Ihse Bursie wrote: On 2018-06-07 23:20, Erik Joelsson wrote: Hello Magnus, Very nice refactoring! Thanks! JdkNativeCompilation.gmk line 126-127 looks a bit long. There is an extra space on 126. Also, why not addprefix for adding -I instead of clunky foreach? Not that I care greatly, but I usually prefer that construct. Yeah, I agree, addprefix is better. I just forgot about it; foreach is a nice allround tool. Updated webrev: http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.02/ (Only changes is in JdkNativeCompilation.gmk, as per above comments). /Magnus Otherwise looks good. /Erik On 2018-06-07 13:22, Magnus Ihse Bursie wrote: This change needs some background information. I've been working on simplifying and streamlining the compilation of native libraries of the JDK. Previously, I introduced the SetupJdkLibrary function, and started to get a better control of compiler flags. This patch continues on both paths. The original intent of this change, which I naively thought was going to be much simpler than it turned out :-) was to separate the -I flags (location of header files) from other flags, and to generate these automatically, wherever possible. Since we have a standard way of locating native code, and most libraries just want to have an -I path to their own source base and the generated Java header, we should be able to provide this in SetupJdkLibrary. This turned out to be closely related to SetupJdkLibrary being able to discover the location of the SRC directories itself, using "convention over configuration" and assuming that the library "libfoo" for "java.module" would be located in java.module/*/native/libfoo. While this sounds simple in theory, when actually trying to implement this I of course ran into all the places where some special handling was indeed needed. So even if like 90% of all libraries were simple to get to build using automated discovery of source and header directories, the 10% that did not caused me much more headaches than I had anticipated. On the other hand, now that I've sorted out all those places, the few remaining odd solutions is clearly documented and not just something that "just happens" due to strange configurations. One file deserves mentioning specifically: Awt2dLibraries.gmk. The java.desktop libraries are unfortunately quite entangled with each other, and do not readily follow the patterns that are used elsewhere in the code base. So it might just look like the file has just gone from one state of messiness, to another, which would hardly be an improvement. :-( I would still argue that the new messiness is better: It is now much clearer in what ways the libraries diverge from our standard assumption, and what course of action needs to be taken to minimize these differences. (Which is something I believe should be done -- these issues are not just cosmetic but is the root of most of the issues we always see for these libraries, when upgrading compilers, etc.) During this change, I noticed that not all native libraries include the proper generated header file. This is a dangerous coding practice, since a change in the Java part of the interface might not get picked up properly in the native part. I've added the missing includes that I've detected, and due to these changes, I'm also including the component teams in what is really only a build change. As can be seen for jdk.crypto.mscapi; there had indeed been changes that needed correcting. Since this is (basically) a pure build change, my gold standard here has been the build compare script. In essence, the build output prior to my change and with this change are 100% identical. In truth, this is a bit too strong a claim. A few changes has occurred, but none of them should matter. Here's a breakdown of the compare.sh results: * Windows-x64: No differences at all. * Solaris: Two libraries are reported to differ: libsaproc.so and libfontmanager.so, both with a disass diff on ~700 bytes. Analyzing this, I found that the object files used to link these two libraries has no disass differences. They have a slight binary difference and a difference in size, due to the include paths being different (and this is stored in the .o file, which makes it different). Somehow this apparently triggers the linker to generate a slightly different code in a few places, using a different register or so. (Weird...) * MacOS: Two libraries are reported to differ: libjava.dylib and libmlib_image.dylib, both of them just reported as a binary diff (no symbol, disass or fulldump differences). This is not really unsuspected, but I analyzed it anyway. I found that for libjava.dylib, a single .o file was different. This one was actually picked up from closed sources, and are not really relevant for OpenJDK. Anyway, the reason for the difference was the same
Re: RFR: JDK-8204602 Add devkit for linux-arm32
Looks ok to me. /Erik On 2018-06-08 01:38, Magnus Ihse Bursie wrote: With some simple changes, our devkit scripts can be made to generate devkits for cross-compiling on linux-x64 to linux-arm (32-bit). Also add a jib profile to use this new linux-arm devkit, as opposed to the legacy devkits used by the traditional linux-arm profiles. Note that this change does not imply any change in Oracle's support of the linux-arm platform. Bug: https://bugs.openjdk.java.net/browse/JDK-8204602 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8204602-add-linux-arm-32-devkit/webrev.01 /Magnus
Re: [8u] RFR: 8196108: Add build support for VS 2015/2017
On 2018-06-08 11:27, Kevin Walls wrote: Hi, I'd like to get a review of a backport to jdk8u of: 8196108: Add build support for VS 2015/2017 https://bugs.openjdk.java.net/browse/JDK-8196108 JDK 11 changeset: http://hg.openjdk.java.net/jdk/jdk/rev/bcce1fa183e7 Original review thread: http://mail.openjdk.java.net/pipermail/build-dev/2018-January/020719.html jdk8u webrev: http://cr.openjdk.java.net/~kevinw/8196108/webrev.00/ Looks good to me. /Magnus Imports partly automatically, partly needs manual intervention. I haven't included the devkit creation scripts. They aren't in 8 already, we haven't backported: 8087208: Add devkit creation script for windows The scripts are available in the later repos, and the devkits created are the same... So it appears unnecessary. I can start a build using VS2017 installed, or from a devkit. Testing locally and with JPRT, using the existing default VS2010 compiler, so the standard build is unaffected. This change is not in 10 or 9, but this is a backport to 8. This of course does not change anything about what compilers are known to work or be supported to build jdk8u, and in toolchain_windows.m4, for the ordering of compiler versions I kept VALID_VS_VERSIONS starting with 2010, so no change in the default compiler we use. Will be further changes to get the VS2017 build actually working... Many thanks Kevin
[8u] RFR: 8196108: Add build support for VS 2015/2017
Hi, I'd like to get a review of a backport to jdk8u of: 8196108: Add build support for VS 2015/2017 https://bugs.openjdk.java.net/browse/JDK-8196108 JDK 11 changeset: http://hg.openjdk.java.net/jdk/jdk/rev/bcce1fa183e7 Original review thread: http://mail.openjdk.java.net/pipermail/build-dev/2018-January/020719.html jdk8u webrev: http://cr.openjdk.java.net/~kevinw/8196108/webrev.00/ Imports partly automatically, partly needs manual intervention. I haven't included the devkit creation scripts. They aren't in 8 already, we haven't backported: 8087208: Add devkit creation script for windows The scripts are available in the later repos, and the devkits created are the same... So it appears unnecessary. I can start a build using VS2017 installed, or from a devkit. Testing locally and with JPRT, using the existing default VS2010 compiler, so the standard build is unaffected. This change is not in 10 or 9, but this is a backport to 8. This of course does not change anything about what compilers are known to work or be supported to build jdk8u, and in toolchain_windows.m4, for the ordering of compiler versions I kept VALID_VS_VERSIONS starting with 2010, so no change in the default compiler we use. Will be further changes to get the VS2017 build actually working... Many thanks Kevin
Re: RFR (XL): JDK-8204572 SetupJdkLibrary should setup SRC and -I flags automatically
On 2018-06-07 23:20, Erik Joelsson wrote: Hello Magnus, Very nice refactoring! Thanks! JdkNativeCompilation.gmk line 126-127 looks a bit long. There is an extra space on 126. Also, why not addprefix for adding -I instead of clunky foreach? Not that I care greatly, but I usually prefer that construct. Yeah, I agree, addprefix is better. I just forgot about it; foreach is a nice allround tool. Updated webrev: http://cr.openjdk.java.net/~ihse/JDK-8204572-autodetect-SRC-and-headers-dirs/webrev.02/ (Only changes is in JdkNativeCompilation.gmk, as per above comments). /Magnus Otherwise looks good. /Erik On 2018-06-07 13:22, Magnus Ihse Bursie wrote: This change needs some background information. I've been working on simplifying and streamlining the compilation of native libraries of the JDK. Previously, I introduced the SetupJdkLibrary function, and started to get a better control of compiler flags. This patch continues on both paths. The original intent of this change, which I naively thought was going to be much simpler than it turned out :-) was to separate the -I flags (location of header files) from other flags, and to generate these automatically, wherever possible. Since we have a standard way of locating native code, and most libraries just want to have an -I path to their own source base and the generated Java header, we should be able to provide this in SetupJdkLibrary. This turned out to be closely related to SetupJdkLibrary being able to discover the location of the SRC directories itself, using "convention over configuration" and assuming that the library "libfoo" for "java.module" would be located in java.module/*/native/libfoo. While this sounds simple in theory, when actually trying to implement this I of course ran into all the places where some special handling was indeed needed. So even if like 90% of all libraries were simple to get to build using automated discovery of source and header directories, the 10% that did not caused me much more headaches than I had anticipated. On the other hand, now that I've sorted out all those places, the few remaining odd solutions is clearly documented and not just something that "just happens" due to strange configurations. One file deserves mentioning specifically: Awt2dLibraries.gmk. The java.desktop libraries are unfortunately quite entangled with each other, and do not readily follow the patterns that are used elsewhere in the code base. So it might just look like the file has just gone from one state of messiness, to another, which would hardly be an improvement. :-( I would still argue that the new messiness is better: It is now much clearer in what ways the libraries diverge from our standard assumption, and what course of action needs to be taken to minimize these differences. (Which is something I believe should be done -- these issues are not just cosmetic but is the root of most of the issues we always see for these libraries, when upgrading compilers, etc.) During this change, I noticed that not all native libraries include the proper generated header file. This is a dangerous coding practice, since a change in the Java part of the interface might not get picked up properly in the native part. I've added the missing includes that I've detected, and due to these changes, I'm also including the component teams in what is really only a build change. As can be seen for jdk.crypto.mscapi; there had indeed been changes that needed correcting. Since this is (basically) a pure build change, my gold standard here has been the build compare script. In essence, the build output prior to my change and with this change are 100% identical. In truth, this is a bit too strong a claim. A few changes has occurred, but none of them should matter. Here's a breakdown of the compare.sh results: * Windows-x64: No differences at all. * Solaris: Two libraries are reported to differ: libsaproc.so and libfontmanager.so, both with a disass diff on ~700 bytes. Analyzing this, I found that the object files used to link these two libraries has no disass differences. They have a slight binary difference and a difference in size, due to the include paths being different (and this is stored in the .o file, which makes it different). Somehow this apparently triggers the linker to generate a slightly different code in a few places, using a different register or so. (Weird...) * MacOS: Two libraries are reported to differ: libjava.dylib and libmlib_image.dylib, both of them just reported as a binary diff (no symbol, disass or fulldump differences). This is not really unsuspected, but I analyzed it anyway. I found that for libjava.dylib, a single .o file was different. This one was actually picked up from closed sources, and are not really relevant for OpenJDK. Anyway, the reason for the difference was the same as for the Solaris libs; the include paths had changes, which caused
RFR: JDK-8204602 Add devkit for linux-arm32
With some simple changes, our devkit scripts can be made to generate devkits for cross-compiling on linux-x64 to linux-arm (32-bit). Also add a jib profile to use this new linux-arm devkit, as opposed to the legacy devkits used by the traditional linux-arm profiles. Note that this change does not imply any change in Oracle's support of the linux-arm platform. Bug: https://bugs.openjdk.java.net/browse/JDK-8204602 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8204602-add-linux-arm-32-devkit/webrev.01 /Magnus
RE: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility flags
Hi Ichiroh, Ok, so as per the output, via the include of osSupport.hpp, something must happen which undeclares "visibility" or makes it ambiguous. Looking at osSupport.hpp, I can't see anything special. It would just include pthread.h and declare some c++ classes. You could try to get and analyze the preprocessed output of xlC by specifying the option -P or -E to the compile call. You will get the original xlC command line by calling 'cat support/native/java.base/libjimage/NativeImageBuffer.o.cmdline' in your build directory. I think we should really understand what's happening there and fix it correctly instead of just excluding osSupport.hpp. Best regards Christoph > -Original Message- > From: Ichiroh Takiguchi [mailto:taki...@linux.vnet.ibm.com] > Sent: Donnerstag, 7. Juni 2018 18:29 > To: Langer, Christoph > Cc: build-dev@openjdk.java.net; ppc-aix-port-...@openjdk.java.net; core- > libs-...@openjdk.java.net; Lindenmaier, Goetz > ; Baesken, Matthias > > Subject: RE: RFR : 8204541 Correctly support AIX xlC 13.1 symbol visibility > flags > > Hello Christoph > > According to build log, if <#include "osSupport.hpp"> was there: > "/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server- > release/support/headers/java.base/jdk_internal_jimage_NativeImageBuffe > r.h", > line 15.27: 1540-0040 (S) The text > "Java_jdk_internal_jimage_NativeImageBuffer_getNativeMap" is > unexpected. "visibility" may be undeclared or ambiguous. > make[3]: *** > [/home/jdktest/sandbox/jdk/build/aix-ppc64-normal-server- > release/support/native/java.base/libjimage/NativeImageBuffer.o] > Error 1 > make[3]: Leaving directory `/home/jdktest/sandbox/jdk/make' > make[2]: *** [java.base-libs] Error 2 > make[2]: *** Waiting for unfinished jobs > > On 2018-06-07 22:06, Langer, Christoph wrote: > > Hi Ichiroh, > > > > what's the exact error message if you #include "osSupport.hpp"? (I > > have no xlC 13 at hand to try myself...) > > > > Best regards > > Christoph > > > >> -Original Message- > >> From: Ichiroh Takiguchi [mailto:taki...@linux.vnet.ibm.com] > >> Sent: Donnerstag, 7. Juni 2018 14:53 > >> To: build-dev@openjdk.java.net; ppc-aix-port-...@openjdk.java.net; > >> core- > >> libs-...@openjdk.java.net > >> Cc: Lindenmaier, Goetz ; Baesken, > Matthias > >> ; Langer, Christoph > >> > >> Subject: RFR : 8204541 Correctly support AIX xlC 13.1 symbol > >> visibility flags > >> > >> Hello. > >> > >> Could you review it ? > >> Bug:https://bugs.openjdk.java.net/browse/JDK-8204541 > >> Change: http://cr.openjdk.java.net/~clanger/webrevs/8204541.0/ > >> > >> Thanks, > >> Ichiroh Takiguchi > >> IBM Japan, Ltd. > >> > >> Original Message > >> Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support > >> on > >> xlc 12.1 > >> Date: 2018-06-07 20:43 > >> From: "Langer, Christoph" > >> To: Ichiroh Takiguchi > >> Cc: "'build-dev@openjdk.java.net'" , > >> "ppc-aix-port-...@openjdk.java.net" >> d...@openjdk.java.net>, > >> "core-libs-...@openjdk.java.net" , > >> "Lindenmaier, Goetz" , "Baesken, > Matthias" > >> > >> > >> Hi Ichiroh, > >> > >> your proposal seems to make sense. I have created a bug for this: > >> https://bugs.openjdk.java.net/browse/JDK-8204541 > >> > >> Can you please generate a webrev (referencing this bug, -c option of > >> webrev.ksh) and mail it over to me. Then I'll upload it and you can > >> post > >> an official RFR mail. > >> > >> Best regards > >> Christoph > >> > >> > -Original Message- > >> > From: Ichiroh Takiguchi [mailto:taki...@linux.vnet.ibm.com] > >> > Sent: Dienstag, 5. Juni 2018 08:59 > >> > To: Baesken, Matthias > >> > Cc: Langer, Christoph ; 'build- > >> > d...@openjdk.java.net' ; ppc-aix-port- > >> > d...@openjdk.java.net; core-libs-...@openjdk.java.net; Lindenmaier, > >> > Goetz > >> > Subject: RE: RFR : 8202322: AIX: symbol visibility flags not support on > >> > xlc 12.1 > >> > > >> > Hello Matthias and Christoph. > >> > Thank you for your explanations. > >> > > >> > I did not have enough knowledge about "visibility". > >> > > >> > I created following patches. > >> > > >> > > >> > diff -r 02934b0d661b > >> > src/java.base/share/native/libjimage/NativeImageBuffer.cpp > >> > --- a/src/java.base/share/native/libjimage/NativeImageBuffer.cpp > Wed > >> > May > >> > 30 14:46:28 2018 +0200 > >> > +++ b/src/java.base/share/native/libjimage/NativeImageBuffer.cpp > >>Tue > >> > Jun > >> > 05 12:10:41 2018 +0900 > >> > @@ -39,7 +39,9 @@ > >> > #include "imageFile.hpp" > >> > #include "inttypes.hpp" > >> > #include "jimage.hpp" > >> > +#if !defined(_AIX) > >> > #include "osSupport.hpp" > >> > +#endif > >> > > >> > #include "jdk_internal_jimage_NativeImageBuffer.h" > >> > > >> > diff -r 02934b0d661b src/java.base/unix/native/include/jni_md.h > >> > --- a/src/java.base/unix/native/include/jni_md.h Wed May 30 > 14:46:28 > >> > 2018 +0200 > >> > +++ b/src/java.base/unix/native/include/jn
