Re: [Newbie question] Strange errors trying to build the JDK
Hi, On 08/21/2018 08:00 PM, David Holmes wrote: Hi, You need to search further up the build log to try and find the actual error that occurred when building hotspot. Run "make hotspot" and it should be easier to see. In addition to David's suggestion you can also add before the command LOG=trace, like: $ LOG=trace make hotspot to spot the executed commands in the build process. Regards, Gustavo David On 20/08/2018 11:53 AM, Zambonifofex wrote: Hello, everyone. I am new to collaborating to the JDK, so sorry if this question is too newbie‐ish. I have recently been affected by a minor bug in the ‘java.desktop’ module on Linux. I figured I’d try to fix it by myself, so I followed the steps in the “How to Contribute” page, the “Building” page, as well as some more that I could find online. The bug number is ‘7162479’. The bug is that it calling ‘setLocationByPlatform(true)’ on a ‘JFrame’ will not work if ‘setResizable(false)’ has been called on the same ‘JFrame’. When I tried to build the JDK with ‘make images’, after getting a couple errors related to the changes I made and fixing them, I started to face a strange error that was not related to the changes I made at all. Here is the log I get when trying to run ‘make images’ currently, as well as the output of ‘hg id’, if that’s of any help (where ‘/home/zambonifofex/jdk $’ is the prompt): /home/zambonifofex/jdk/ $ hg id 2e91d927e00c+ tip /home/zambonifofex/jdk/ $ make images Building target 'images' in configuration 'linux-x86_64-normal-server-release' lib/JvmMapfile.gmk:141: recipe for target '/home/zambonifofex/jdk/build/linux-x86_64-normal-server-release/hotspot/variant-server/libjvm/symbols-objects' failed make[3]: *** [/home/zambonifofex/jdk/build/linux-x86_64-normal-server-release/hotspot/variant-server/libjvm/symbols-objects] Error 1 make[3]: *** Deleting file '/home/zambonifofex/jdk/build/linux-x86_64-normal-server-release/hotspot/variant-server/libjvm/symbols-objects' make/Main.gmk:257: recipe for target 'hotspot-server-libs' failed make[2]: *** [hotspot-server-libs] Error 1 make[2]: *** Waiting for unfinished jobs ERROR: Build failed for target 'images' in configuration 'linux-x86_64-normal-server-release' (exit code 2) === Make failed targets repeated here === lib/JvmMapfile.gmk:141: recipe for target '/home/zambonifofex/jdk/build/linux-x86_64-normal-server-release/hotspot/variant-server/libjvm/symbols-objects' failed make/Main.gmk:257: recipe for target 'hotspot-server-libs' failed === End of repeated output === Hint: Try searching the build log for the name of the first failed target. Hint: See doc/building.html#troubleshooting for assistance. /home/zambonifofex/jdk/make/Init.gmk:300: recipe for target 'main' failed make[1]: *** [main] Error 1 /home/zambonifofex/jdk/make/Init.gmk:186: recipe for target 'images' failed make: *** [images] Error 2 /home/zambonifofex/jdk/ $ Any help would be appreciated. Thanks in advance.
Re: RFR: 8210416: [linux] Poor StrictMath performance due to non-optimized compilation
Hello, On 09/05/2018 04:15 PM, joe darcy wrote: Hello, On 9/5/2018 6:12 AM, Severin Gehwolf wrote: Hi, Cross-posting this review-thread on core-libs-dev and build-dev as this is a build change, but affects fdlibm which is core-libs. With JDK-8170153 optimization for fdlibm code has been turned on for ppc64 s390 and aarch64. This patch intends to turn it on on all arches on Linux. I've not observed any precision issues. Is there a good reason to not use -O3 -ffp-contract=off everywhere? Bug: https://bugs.openjdk.java.net/browse/JDK-8210416 webrev: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8210416/webrev.01/ Testing: - java/lang/Math, java/lang/StrictMath tests (all pass). - Currently running through submit repo. A simple micro benchmark from JDK-8170153[1] shows these numbers for StrictMath: Function | before | after -- sin | 0m33.382s | 0m18.731s cos | 0m31.562s | 0m18.796s tan | 0m33.657s | 0m21.093s atan | 0m5.714s | 0m4.902s log | 0m6.212s | 0m4.439s log10 | 0m7.946s | 0m5.543s sqrt | 0m0.481s | 0m0.449s cbrt | 0m5.295s | 0m5.214s tanh | 0m1.404s | 0m1.307s log1p | 0m6.457s | 0m5.131s IEEEremainder | 0m10.629s | 0m6.048s atan2 | 0m8.037s | 0m5.668s hypot | 0m2.171s | 0m2.147s Note that pow (JDK-8134795), hypot (JDK-7130085), cbrt (JDK-8136799), and exp (JDK-8139688), have been ported to Java as of JDK 9. The sqrt method is commonly handled as an intrinsic. Testing that was not geared toward finding precision/rounding issues would be unlikely to find them. I don't see the sources of the microbenchmark in JDK-8170153. The microbench can be found on the first email I sent discussing that issue on Power. Check it at the end, but there is nothing special in there: http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-November/002738.html There is also some discussing on the formal RFR: http://mail.openjdk.java.net/pipermail/ppc-aix-port-dev/2016-November/002751.html Severin, the precision issues were quite conspicuous on Power when -O3 was turned on. There are some discussions about it in the threads above IIRC. SAP helped to test the change against additional tests, like TCK. HTH. Best regards, Gustavo Cheers, -Joe
Re: RFR: 8210416: [linux] Poor StrictMath performance due to non-optimized compilation
Hi Severin, On 09/10/2018 06:27 AM, Severin Gehwolf wrote: On Mon, 2018-09-10 at 10:05 +0100, Andrew Haley wrote: On 09/05/2018 02:12 PM, Severin Gehwolf wrote: Is there a good reason to not use -O3 -ffp-contract=off everywhere? Is there a good reason to use -O3 rather than -O2? Not sure. I was following what JDK-8170153 did, which was using OPTIMIZATION := HIGH corresponding to -O3. cc'ing Gustavo. Gustavo, would you know why HIGH was chosen over, LOW? I don't remember exactly, but at least for ppc64 I discussed that a bit with the toolchain folks (also regarding the precision issue, etc) and they never said anything against using -O3. Unfortunately it was long time ago so I don't remember exactly the numbers on ppc64 for -O2 to check if it was worse and so I selected -O3 instead. -O3 can bloat the code which can increase cache pressure, which is not always noticeable in benchmarks but hurts real-world programs. Unless benchmarks are significantly better at -O3, -O2 is a good default choice. OK, thanks! I'll re-test and change to LOW (-O2) if it gives similar results. That's interesting. Andrew, do you mean bloat in the sense of final code size (for instance, due to unrolling), right? BTW (I just remembered that), on RISC the lack of optimization hurts way more than the lack of optimization on CISC, so I recall that it puzzled me the fact that turning on the optimization on x86_64 did not change much the scenario, contrary to the conspicuous gains on on ppc64 when turning on the optimization. I took me some time so to understand that the optimization flag was the culprit (a much simpler case lucky), because I tried first to profile and optimize the fdlibm code (after extracting it from JVM for detailed analysis) and only after getting to a dead end I turned to look at simpler causes. Are you checking the difference between -O2 and -O3 only on x86_64? Best regards, Gustavo
Re: RFR: 8210416: [linux] Poor StrictMath performance due to non-optimized compilation
Hi Severin, On 09/11/2018 09:02 AM, Severin Gehwolf wrote: Micro-benchmark results from running [1] for x86_64 and ppc64le are here (-O2 is sufficient it seems): http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8210416/microbenchmarks_results/ More thoughts? Thanks a lot for checking it on PPC64. I did additional tests on a POWER8 using the same microbench. I share the results below. The numbers are the arithmetic mean of 10 runs. Unit is second. 1 O3 | O2 2 MathStrictMath | MathStrictMath 3 | 4 acos:3.0728 3.0728 | acos:3.07325 3.07375 5 asin:3.0933 3.0916 | asin:3.09337 3.0915 6 atan2: 10.9743 10.9759 | atan2: 11.2994 11.2977 7 atan:7.653 7.6602 | atan:7.48463 7.48225 8 cbrt:13.444 13.4433 | cbrt:13.4535 13.4527 9 cos: 30.1561 33.9689 | cos: 30.157 35.1162 10 cosh:3.0911 3.1074 | cosh:3.09975 3.09088 11 exp: 1.413 3.0572 | exp: 1.41313 3.09737 12 expm1: 3.0281 3.0349 | expm1: 2.98813 2.98913 13 hypot: 33.7668 33.7471 | hypot: 27.9664 27.9675 14 log10: 4.2213 10.0832 | log10: 4.221 10.1109 15 log1p: 9.2886 9.2888 | log1p: 9.08713 9.08075 16 log: 3.1362 7.6327 | log: 3.136 7.633 17 pow: 13.7787 17.4739 | pow: 13.7829 17.483 18 sin: 30.216 33.9733 | sin: 30.2163 35.1684 19 sinh:3.1478 3.152 | sinh:3.18638 3.18375 20 sqrt:0.665 0.6714 | sqrt:0.665 0.671 21 tan: 31.6409 36.461 | tan: 31.6385 37.7166 22 tanh:3.0753 3.0925 | tanh:3.11088 3.10675 Based upon that, it looks like that O3 is better for most of the functions, specially for sin, cos, and tan. On the other hand, it looks like that O3 hurts hypot. Thus it seems -O2 is not quite the same as -O3, but it's not straightforward to decide which one is better too. Maybe Andrew can enlight us. I also CC:ed ppc-aix-port in case somebody there wishes to comment on that. Thank you. Best regards, Gustavo Thanks, Severin [1] https://github.com/gromero/strictmath/
Re: RFR: 8210416: [linux] Poor StrictMath performance due to non-optimized compilation
Hi Andrew, On 09/12/2018 12:51 AM, Gustavo Romero wrote: Maybe Andrew can enlight us. I was not sure if 'enlight us' was the correct form here, so I did some search and I found with horror today that 'enlighten us' can also be considered an insult (!?). That's really not the case here. So if it's really possible to interpret it in a non-positive / non-cordial way I sincerely apologize for the wrong use. I was really only asking for an advice. :) Regards, Gustavo
Re: RFR: 8210416: [linux] Poor StrictMath performance due to non-optimized compilation
Hello Severin, On 09/12/2018 04:48 AM, Severin Gehwolf wrote: Hi David, On Wed, 2018-09-12 at 13:57 +1000, David Holmes wrote: Hi Severin, Sorry I'm a bit confused now. Originally for ppc/s390x/aarch64 the optimization level for fdlibm was HIGH. But now it will be LOW due to: 45 ifneq ($(FDLIBM_CFLAGS), ) 46 BUILD_LIBFDLIBM_OPTIMIZATION := LOW as those platforms will use gcc/clang with support for -ffp-contract and so FDLIBM_CFLAGS will not be empty. ?? Correct. That was done based on Andrew Haley's comment here: http://mail.openjdk.java.net/pipermail/build-dev/2018-September/023160.html I've done some measurements with -O2 and -O3. -O2 is sufficient for a 2.75 speedup for sin/cos on ppc64le as compared to the -O0 baseline. On the other hand the speedup of -O3 as compared to -O2 is only 1.05 for sin/cos on ppc64le. Since the difference between them were not huge, I've used -O2, a.k.a LOW. See also: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8210416/microbenchmarks_results/ To me it seems -O2 is sufficiently good. Note that HIGH == -O3 and LOW == -O2 on gcc arches. Does that clear things up? FWIW, I'm happy to change it back to HIGH if there are concerns. See also Gustavo's reply here for some more data points: http://mail.openjdk.java.net/pipermail/build-dev/2018-September/023193.html I'm wondering why the possible issue of using -O3 on fdlibm would not affect other parts of the JVM code (like the Opto code in libjvm.so) that afaics are also built using -O3. Also, the gap of 1.05 for sin/cos, for instance, still sounds like a regression to me. Maybe a better approach to this would be to figure out a way to test -O3 and -O2 with the real world case in which -O3 can have a bad impact. I'm not sure if SPECjbb would qualify for that. Best regards, Gustavo Thanks, Severin David On 12/09/2018 2:14 AM, Severin Gehwolf wrote: Hi Erik, Thanks for the review! On Tue, 2018-09-11 at 08:57 -0700, Erik Joelsson wrote: Hello Severin, Even if using the macro, I still think you need to add a condition on the compiler types where the switch can be reasonably expected to exist. How about this? http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8210416/webrev.05/ Thanks, Severin On 2018-09-11 05:02, Severin Gehwolf wrote: On Mon, 2018-09-10 at 09:29 -0700, Erik Joelsson wrote: I see. I was not aware of that issue, but we clearly need to file a bug for it and fix it. In this case I think it's fine to us the macro however. OK. Update webrev, which now uses FLAGS_COMPILER_CHECK_ARGUMENTS. http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8210416/webrev.04/ Micro-benchmark results from running [1] for x86_64 and ppc64le are here (-O2 is sufficient it seems): http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK-8210416/microbenchmarks_results/ More thoughts? Thanks, Severin [1] https://github.com/gromero/strictmath/
Re: RFR: 8210416: [linux] Poor StrictMath performance due to non-optimized compilation
Hi Goetz and Severin, On 09/14/2018 07:12 AM, Lindenmaier, Goetz wrote: Hi, We are only talking about reducing O3 to O2 for the compilation step of fdlibm, right? Yes. As I know, this is being replaced by Java coding, anyways. Therefore I don't have a strong opinion for any of the choices. I totally agree with Goetz considering OpenJDK 12 and 11u. I was actually more concerned about a possible future backport of that change, for instance: it gets into 8u, but Java implementations never get. Maybe I forgot to mention my concern about a 8u backport earlier. Severin, anyway, as Goetz is also fine with either option, I'm fine with what you and Andrew decide as the best option. Thanks bearing with the discussion on ppc64. Best regards, Gustavo Best regards, Goetz. -Original Message- From: core-libs-dev On Behalf Of Gustavo Romero Sent: Freitag, 14. September 2018 01:30 To: Severin Gehwolf ; David Holmes ; Erik Joelsson ; build-dev ; core-libs-dev Subject: Re: RFR: 8210416: [linux] Poor StrictMath performance due to non- optimized compilation Hello Severin, On 09/12/2018 04:48 AM, Severin Gehwolf wrote: Hi David, On Wed, 2018-09-12 at 13:57 +1000, David Holmes wrote: Hi Severin, Sorry I'm a bit confused now. Originally for ppc/s390x/aarch64 the optimization level for fdlibm was HIGH. But now it will be LOW due to: 45 ifneq ($(FDLIBM_CFLAGS), ) 46 BUILD_LIBFDLIBM_OPTIMIZATION := LOW as those platforms will use gcc/clang with support for -ffp-contract and so FDLIBM_CFLAGS will not be empty. ?? Correct. That was done based on Andrew Haley's comment here: http://mail.openjdk.java.net/pipermail/build-dev/2018- September/023160.html I've done some measurements with -O2 and -O3. -O2 is sufficient for a 2.75 speedup for sin/cos on ppc64le as compared to the -O0 baseline. On the other hand the speedup of -O3 as compared to -O2 is only 1.05 for sin/cos on ppc64le. Since the difference between them were not huge, I've used -O2, a.k.a LOW. See also: http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- 8210416/microbenchmarks_results/ To me it seems -O2 is sufficiently good. Note that HIGH == -O3 and LOW == -O2 on gcc arches. Does that clear things up? FWIW, I'm happy to change it back to HIGH if there are concerns. See also Gustavo's reply here for some more data points: http://mail.openjdk.java.net/pipermail/build-dev/2018- September/023193.html I'm wondering why the possible issue of using -O3 on fdlibm would not affect other parts of the JVM code (like the Opto code in libjvm.so) that afaics are also built using -O3. Also, the gap of 1.05 for sin/cos, for instance, still sounds like a regression to me. Maybe a better approach to this would be to figure out a way to test -O3 and -O2 with the real world case in which -O3 can have a bad impact. I'm not sure if SPECjbb would qualify for that. Best regards, Gustavo Thanks, Severin David On 12/09/2018 2:14 AM, Severin Gehwolf wrote: Hi Erik, Thanks for the review! On Tue, 2018-09-11 at 08:57 -0700, Erik Joelsson wrote: Hello Severin, Even if using the macro, I still think you need to add a condition on the compiler types where the switch can be reasonably expected to exist. How about this? http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- 8210416/webrev.05/ Thanks, Severin On 2018-09-11 05:02, Severin Gehwolf wrote: On Mon, 2018-09-10 at 09:29 -0700, Erik Joelsson wrote: I see. I was not aware of that issue, but we clearly need to file a bug for it and fix it. In this case I think it's fine to us the macro however. OK. Update webrev, which now uses FLAGS_COMPILER_CHECK_ARGUMENTS. http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- 8210416/webrev.04/ Micro-benchmark results from running [1] for x86_64 and ppc64le are here (-O2 is sufficient it seems): http://cr.openjdk.java.net/~sgehwolf/webrevs/JDK- 8210416/microbenchmarks_results/ More thoughts? Thanks, Severin [1] https://github.com/gromero/strictmath/
RFR(T): 8221434: Fix typo in lib-x11 autoconf error message about missing headers
Hi, Could the following trivial change be reviewed, please? bug: https://bugs.openjdk.java.net/browse/JDK-8221434 webrev : http://cr.openjdk.java.net/~gromero/8221434/v1/ Thank you and best regards, Gustavo
Re: RFR(T): 8221434: Fix typo in lib-x11 autoconf error message about missing headers
Hi David, On 03/25/2019 09:11 PM, David Holmes wrote: Looks good Gustavo. Thanks a lot for the quick review. Pushed to jdk/jdk since it's a trivial change. Best regards, Gustavo
Re: strange error when running jtreg tests
Hi, I observed the same issue on Ubuntu 16.04 but not on RHEL 7.2. RHEL 7.2: https://paste.fedoraproject.org/409005/raw/ Ubuntu 16.04: https://paste.fedoraproject.org/409004/raw/ I checked on PPC64 LE arch (Ubuntu 16.04 and RHEL 7.2, logs above). It fails also on Ubuntu 16.04 x64 (but I could not check if it succeeds on RHEL 7.2 x64). On Ubuntu 16.04 PPC64 LE replacing -lpthread by -pthread in ./hotspot/make/test/JtregNative.gmk:82 solves the issue on pthread_* symbols and placing -ljvm (for instance) in front of ".../exeinvoke.o" solves the issue on JNI_CreateJavaVM reference, however I'm really not sure what's the root cause. Maurizio, what distro/arch are you using? Regards, Gustavo
Re: strange error when running jtreg tests
Hi David, On 17-08-2016 00:38, David Holmes wrote: > On 16/08/2016 8:41 AM, Gustavo Romero wrote: >> On Ubuntu 16.04 PPC64 LE replacing -lpthread by -pthread in >> ./hotspot/make/test/JtregNative.gmk:82 solves the issue on pthread_* symbols >> and >> placing -ljvm (for instance) in front of ".../exeinvoke.o" solves the issue >> on >> JNI_CreateJavaVM reference, however I'm really not sure what's the root >> cause. > > I checked the link command being used (it is generated into a file when the > test-image is built) and the -ljvm is already in front of the exeinvoke.o > > /var/tmp/jib-daholme/install/jpg/infra/builddeps/devkit-linux_x64/gcc4.9.2-OEL6.4+1.0/devkit-linux_x64-gcc4.9.2-OEL6.4+1.0.tar.gz/bin/gcc > -Wl,-z,defs -Wl,-O1 -Wl,--allow-shlib-undefined > -L/scratch/dh198349/jdk9-hs/build/linux-x64/support/modules_libs/java.base/amd64 > > -L/scratch/dh198349/jdk9-hs/build/linux-x64/support/modules_libs/java.base/amd64/server > -ljvm -lpthread > --sysroot=/var/tmp/jib-daholme/install/jpg/infra/builddeps/devkit-linux_x64/gcc4.9.2-OEL6.4+1.0/devkit-linux_x64-gcc4.9.2-OEL6.4+1.0.tar.gz/x86_64-unknown-linux-gnu/sysroot > -o > /scratch/dh198349/jdk9-hs/build/linux-x64/support/test/hotspot/jtreg/native/bin/invoke > > /scratch/dh198349/jdk9-hs/build/linux-x64/support/test/hotspot/jtreg/native/support/exeinvoke/exeinvoke.o Sorry, "in front of" was a misleading expression. I meant placed *after* exeinvoke.o (reading from right to left the link command). So on Ubuntu 16.04 x64 this fails (taken from the original build, the order is like in your link command): /usr/lib/gcc/x86_64-linux-gnu/5/collect2 -plugin /usr/lib/gcc/x86_64-linux-gnu/5/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper -plugin-opt=-fresolution=/tmp/cccQFwXy.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o /home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/bin/invoke /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/5/crtbegin.o -L/home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/modules_libs/java.base/amd64 -L/home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/modules_libs/java.base/amd64/server -L/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/5/../../.. --hash-style=both -z defs -O1 --allow-shlib-undefined -ljvm -lpthread /home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/support/exeinvoke/exeinvoke.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/5/crtend.o /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o but this succeeds (where -ljvm is after exeinvoke.o): /usr/lib/gcc/x86_64-linux-gnu/5/collect2 -plugin /usr/lib/gcc/x86_64-linux-gnu/5/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper -plugin-opt=-fresolution=/tmp/ccIRdp31.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lpthread -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o /home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/bin/invoke /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/5/crtbegin.o -L/home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/modules_libs/java.base/amd64 -L/home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/modules_libs/java.base/amd64/server -L/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/5/../../.. --hash-style=both -z defs -O1 /home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/support/exeinvoke/exeinvoke.o -ljvm -lgcc --as-needed -lgcc_s --no-
Re: strange error when running jtreg tests
Sorry, again: (reading from left to right the link command). Regards, Gustavo On 17-08-2016 11:15, Gustavo Romero wrote: > Hi David, > > On 17-08-2016 00:38, David Holmes wrote: >> On 16/08/2016 8:41 AM, Gustavo Romero wrote: >>> On Ubuntu 16.04 PPC64 LE replacing -lpthread by -pthread in >>> ./hotspot/make/test/JtregNative.gmk:82 solves the issue on pthread_* >>> symbols and >>> placing -ljvm (for instance) in front of ".../exeinvoke.o" solves the issue >>> on >>> JNI_CreateJavaVM reference, however I'm really not sure what's the root >>> cause. >> >> I checked the link command being used (it is generated into a file when the >> test-image is built) and the -ljvm is already in front of the exeinvoke.o >> >> /var/tmp/jib-daholme/install/jpg/infra/builddeps/devkit-linux_x64/gcc4.9.2-OEL6.4+1.0/devkit-linux_x64-gcc4.9.2-OEL6.4+1.0.tar.gz/bin/gcc >> -Wl,-z,defs -Wl,-O1 -Wl,--allow-shlib-undefined >> -L/scratch/dh198349/jdk9-hs/build/linux-x64/support/modules_libs/java.base/amd64 >> >> -L/scratch/dh198349/jdk9-hs/build/linux-x64/support/modules_libs/java.base/amd64/server >> -ljvm -lpthread >> --sysroot=/var/tmp/jib-daholme/install/jpg/infra/builddeps/devkit-linux_x64/gcc4.9.2-OEL6.4+1.0/devkit-linux_x64-gcc4.9.2-OEL6.4+1.0.tar.gz/x86_64-unknown-linux-gnu/sysroot >> -o >> /scratch/dh198349/jdk9-hs/build/linux-x64/support/test/hotspot/jtreg/native/bin/invoke >> >> /scratch/dh198349/jdk9-hs/build/linux-x64/support/test/hotspot/jtreg/native/support/exeinvoke/exeinvoke.o > > Sorry, "in front of" was a misleading expression. I meant placed *after* > exeinvoke.o (reading from right to left the link command). > > So on Ubuntu 16.04 x64 this fails (taken from the original build, the order > is like in your link command): > > /usr/lib/gcc/x86_64-linux-gnu/5/collect2 -plugin > /usr/lib/gcc/x86_64-linux-gnu/5/liblto_plugin.so > -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper > -plugin-opt=-fresolution=/tmp/cccQFwXy.res > -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s > -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc > -plugin-opt=-pass-through=-lgcc_s --sysroot=/ --build-id --eh-frame-hdr > -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker > /lib64/ld-linux-x86-64.so.2 -z relro -o > /home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/bin/invoke > /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o > /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o > /usr/lib/gcc/x86_64-linux-gnu/5/crtbegin.o > -L/home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/modules_libs/java.base/amd64 > -L/home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/modules_libs/java.base/amd64/server > -L/usr/lib/gcc/x86_64-linux-gnu/5 > -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu > -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib -L/lib/x86_64-linux-gnu > -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib > -L/usr/lib/gcc/x86_64-linux-gnu/5/../../.. --hash-style=both -z defs -O1 > --allow-shlib-undefined -ljvm -lpthread > /home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/support/exeinvoke/exeinvoke.o > -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed > -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/5/crtend.o > /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o > > but this succeeds (where -ljvm is after exeinvoke.o): > > /usr/lib/gcc/x86_64-linux-gnu/5/collect2 -plugin > /usr/lib/gcc/x86_64-linux-gnu/5/liblto_plugin.so > -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper > -plugin-opt=-fresolution=/tmp/ccIRdp31.res > -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s > -plugin-opt=-pass-through=-lpthread -plugin-opt=-pass-through=-lc > -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s > --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu > --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o > /home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/bin/invoke > /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o > /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o > /usr/lib/gcc/x86_64-linux-gnu/5/crtbegin.o > -L/home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/modules_libs/java.base/amd64 > -L/home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/modules_libs/java.base/amd64/serv
Re: strange error when running jtreg tests
Hi Erik, I applied your change: diff -r 397565766eb4 make/test/JtregNative.gmk --- a/make/test/JtregNative.gmk Thu Aug 11 16:22:08 2016 -0700 +++ b/make/test/JtregNative.gmk Wed Aug 17 09:54:20 2016 -0500 @@ -79,7 +79,7 @@ ifeq ($(OPENJDK_TARGET_OS), linux) BUILD_HOTSPOT_JTREG_LIBRARIES_LDFLAGS_libtest-rw := -z noexecstack BUILD_HOTSPOT_JTREG_LIBRARIES_LDFLAGS_libtest-rwx := -z execstack -BUILD_HOTSPOT_JTREG_EXECUTABLES_LDFLAGS_exeinvoke := -ljvm -lpthread +BUILD_HOTSPOT_JTREG_EXECUTABLES_LIBS_exeinvoke := -ljvm -lpthread BUILD_TEST_invoke_exeinvoke.c_OPTIMIZATION := NONE endif but it seems the new param is not passed to the linker now. So it failed. No "pthread" nor "ljvm" in the link command: /usr/lib/gcc/x86_64-linux-gnu/5/collect2 -plugin /usr/lib/gcc/x86_64-linux-gnu/5/liblto_plugin.so -plugin-opt=/usr/lib/gcc/x86_64-linux-gnu/5/lto-wrapper -plugin-opt=-fresolution=/tmp/ccAdCwgA.res -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s -plugin-opt=-pass-through=-lc -plugin-opt=-pass-through=-lgcc -plugin-opt=-pass-through=-lgcc_s --sysroot=/ --build-id --eh-frame-hdr -m elf_x86_64 --hash-style=gnu --as-needed -dynamic-linker /lib64/ld-linux-x86-64.so.2 -z relro -o /home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/bin/invoke /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crt1.o /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crti.o /usr/lib/gcc/x86_64-linux-gnu/5/crtbegin.o -L/home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/modules_libs/java.base/amd64 -L/home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/modules_libs/java.base/amd64/server -L/usr/lib/gcc/x86_64-linux-gnu/5 -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/5/../../.. --hash-style=both -z defs -O1 --allow-shlib-undefined /home/gromero/hg/jdk9/hs-comp/build/linux-x86_64-normal-server-release/support/test/hotspot/jtreg/native/support/exeinvoke/exeinvoke.o -lgcc --as-needed -lgcc_s --no-as-needed -lc -lgcc --as-needed -lgcc_s --no-as-needed /usr/lib/gcc/x86_64-linux-gnu/5/crtend.o /usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/crtn.o Regards, Gustavo On 17-08-2016 11:32, Erik Joelsson wrote: > I finally took a look at this and yes, there is a bug in the makefile. This > line in hotspot/make/test/JtregNative.gmk: > > BUILD_HOTSPOT_JTREG_EXECUTABLES_LDFLAGS_exeinvoke := -ljvm -lpthread > > should be: > > BUILD_HOTSPOT_JTREG_EXECUTABLES_LIBS_exeinvoke := -ljvm -lpthread > > Could someone who is able to repro try this? > > /Erik > > On 2016-08-17 16:15, Gustavo Romero wrote: >> Hi David, >> >> On 17-08-2016 00:38, David Holmes wrote: >>> On 16/08/2016 8:41 AM, Gustavo Romero wrote: >>>> On Ubuntu 16.04 PPC64 LE replacing -lpthread by -pthread in >>>> ./hotspot/make/test/JtregNative.gmk:82 solves the issue on pthread_* >>>> symbols and >>>> placing -ljvm (for instance) in front of ".../exeinvoke.o" solves the >>>> issue on >>>> JNI_CreateJavaVM reference, however I'm really not sure what's the root >>>> cause. >>> I checked the link command being used (it is generated into a file when the >>> test-image is built) and the -ljvm is already in front of the exeinvoke.o >>> >>> /var/tmp/jib-daholme/install/jpg/infra/builddeps/devkit-linux_x64/gcc4.9.2-OEL6.4+1.0/devkit-linux_x64-gcc4.9.2-OEL6.4+1.0.tar.gz/bin/gcc >>> -Wl,-z,defs -Wl,-O1 -Wl,--allow-shlib-undefined >>> -L/scratch/dh198349/jdk9-hs/build/linux-x64/support/modules_libs/java.base/amd64 >>> >>> -L/scratch/dh198349/jdk9-hs/build/linux-x64/support/modules_libs/java.base/amd64/server >>> -ljvm -lpthread >>> --sysroot=/var/tmp/jib-daholme/install/jpg/infra/builddeps/devkit-linux_x64/gcc4.9.2-OEL6.4+1.0/devkit-linux_x64-gcc4.9.2-OEL6.4+1.0.tar.gz/x86_64-unknown-linux-gnu/sysroot >>> -o >>> /scratch/dh198349/jdk9-hs/build/linux-x64/support/test/hotspot/jtreg/native/bin/invoke >>> >>> /scratch/dh198349/jdk9-hs/build/linux-x64/support/test/hotspot/jtreg/native/support/exeinvoke/exeinvoke.o >> Sorry, "in front of" was a misleading expression. I meant placed *after* >> exeinvoke.o (reading from right to left the link command). >> >> So on Ubuntu 16.04 x64 this fails (taken from the original build, the order >> is like in your link command): >> >> /usr/lib/gcc/x86_64-linux-gnu/5/collect2 -plugin >> /usr/lib/gcc/x86_64-linux-gnu/5/lib
Re: strange error when running jtreg tests
Hi Erik, On 17-08-2016 12:23, Erik Joelsson wrote: > I'm sorry, you also need this patch, which I for some reason I can't remember > had in my local forest. > > diff -r a24702d4d5ab make/common/TestFilesCompilation.gmk > --- a/make/common/TestFilesCompilation.gmk > +++ b/make/common/TestFilesCompilation.gmk > @@ -86,6 +86,7 @@ > LANG := C, \ > CFLAGS := $$($1_CFLAGS) $$($1_CFLAGS_$$($1_PREFIX)$$(name)), \ > LDFLAGS := $$($1_LDFLAGS) $$($1_LDFLAGS_$$($1_PREFIX)$$(name)), \ > +LIBS := $$($1_LIBS_$$($1_PREFIX)$$(name)), \ > OPTIMIZATION := LOW, \ > )) \ > $$(eval $1 += $$(BUILD_TEST_$$(name)) ) \ > > /Erik > > On 2016-08-17 16:56, Gustavo Romero wrote: >> Hi Erik, >> >> I applied your change: >> >> diff -r 397565766eb4 make/test/JtregNative.gmk >> --- a/make/test/JtregNative.gmk Thu Aug 11 16:22:08 2016 -0700 >> +++ b/make/test/JtregNative.gmk Wed Aug 17 09:54:20 2016 -0500 >> @@ -79,7 +79,7 @@ >> ifeq ($(OPENJDK_TARGET_OS), linux) >> BUILD_HOTSPOT_JTREG_LIBRARIES_LDFLAGS_libtest-rw := -z noexecstack >> BUILD_HOTSPOT_JTREG_LIBRARIES_LDFLAGS_libtest-rwx := -z execstack >> -BUILD_HOTSPOT_JTREG_EXECUTABLES_LDFLAGS_exeinvoke := -ljvm -lpthread >> +BUILD_HOTSPOT_JTREG_EXECUTABLES_LIBS_exeinvoke := -ljvm -lpthread >> BUILD_TEST_invoke_exeinvoke.c_OPTIMIZATION := NONE >> endif >> >> >> but it seems the new param is not passed to the linker now. So it failed. >> I confirm that your patch fixes the issue on Ubuntu 16.04 PPC64 && x64, applying cleanly to: http://hg.openjdk.java.net/jdk9/hs-comp, 37de4195dd18+ tip: diff -r 37de4195dd18 make/common/TestFilesCompilation.gmk --- a/make/common/TestFilesCompilation.gmk Fri Aug 05 09:50:23 2016 -0700 +++ b/make/common/TestFilesCompilation.gmk Wed Aug 17 13:54:43 2016 -0400 @@ -86,6 +86,7 @@ LANG := C, \ CFLAGS := $$($1_CFLAGS) $$($1_CFLAGS_$$($1_PREFIX)$$(name)), \ LDFLAGS := $$($1_LDFLAGS) $$($1_LDFLAGS_$$($1_PREFIX)$$(name)), \ +LIBS := $$($1_LIBS_$$($1_PREFIX)$$(name)), \ OPTIMIZATION := LOW, \ )) \ $$(eval $1 += $$(BUILD_TEST_$$(name)) ) \ http://hg.openjdk.java.net/jdk9/hs-comp/hotspot, 031f53ef620a+ tip: diff -r 031f53ef620a make/test/JtregNative.gmk --- a/make/test/JtregNative.gmk Wed Aug 17 06:23:04 2016 + +++ b/make/test/JtregNative.gmk Wed Aug 17 13:54:58 2016 -0400 @@ -79,7 +79,7 @@ ifeq ($(OPENJDK_TARGET_OS), linux) BUILD_HOTSPOT_JTREG_LIBRARIES_LDFLAGS_libtest-rw := -z noexecstack BUILD_HOTSPOT_JTREG_LIBRARIES_LDFLAGS_libtest-rwx := -z execstack -BUILD_HOTSPOT_JTREG_EXECUTABLES_LDFLAGS_exeinvoke := -ljvm -lpthread +BUILD_HOTSPOT_JTREG_EXECUTABLES_LIBS_exeinvoke := -ljvm -lpthread BUILD_TEST_invoke_exeinvoke.c_OPTIMIZATION := NONE endif Thank you very much for having a look. Kind regards, Gustavo
Re: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi David, On 17-11-2016 00:31, David Holmes wrote: > Adding in build-dev as they need to scrutinize all build changes. Thanks a lot. Regards, Gustavo
Re: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi Erik, On 17-11-2016 07:17, Erik Joelsson wrote: > Overall this looks reasonable to me. However, if we want to introduce a new > possible tuple for specifying compilation flags to SetupNativeCompilation, we > (the build team) would prefer if we used > OPENJDK_TARGET_CPU instead of OPENJDK_TARGET_CPU_ARCH. Got it. Thanks a lot for that info. I'll take that into account. Regards, Gustavo
Re: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi Joe, Thanks a lot for your valuable comments. On 17-11-2016 15:35, joe darcy wrote: >> Currently, optimization for building fdlibm is disabled, except for the >> "solaris" OS target [1]. > > The reason for that is because historically the Solaris compilers have had > sufficient discipline and control regarding floating-point semantics and > compiler optimizations to still implement the > Java-mandated results when optimization was enabled. The gcc family of > compilers, for example, has lacked such discipline. oh, I see. Thanks for clarifying that. I was exactly wondering why fdlibm optimization is off even for x86_x64 as it, AFAICS regarding gcc 5 only, does not affect the precision, even if setting -O3 does not improve the performance as much as on PPC64. >> As a consequence on PPC64 (Linux) StrictMath methods like, but not limited >> to, >> sin(), cos(), and tan() perform verify poor in comparison to the same methods >> in Math class [2]: > > If you are doing your work against JDK 9, note that the pow, hypot, and cbrt > fdlibm methods required by StrictMath have been ported to Java (JDK-8134780: > Port fdlibm to Java). I have intentions to > port the remaining methods to Java, but it is unclear whether or not this > will occur for JDK 9. Yes, I'm doing my work against 9. So is there any problem if I proceed with my change? I understand that there is no conflict as JDK-8134780 progresses and replaces the StrictMath methods by their counterparts in Java. Please, advice. Is it intended to downport JDK-8134780 to 8? > Methods in the Math class, such as pow, are often intrinsified and use a > different algorithm so a straight performance comparison may not be as fair > or meaningful in those cases. I agree. It's just that the issue on StrictMath methods was first noted due to that huge gap (Math vs StrictMath) on PPC64, which is not prominent on x64. > Accumulating the the results of the functions and comparisons the sums is not > a sufficiently robust way of checking to see if the optimized versions are > indeed equivalent to the non-optimized ones. > The specification of StrictMath requires a particular result for each set of > floating-point arguments and sums get round-away low-order bits that differ. That's really good point, thanks for letting me know about that. I'll re-test my change under that perspective. > Running the JDK math library regression tests and corresponding JCK tests is > recommended for work in this area. Got it. By "the JDK math library regression tests" you mean exactly which test suite? the jtreg tests? For testing against JCK/TCK I'll need some help on that. Thank you very much. Regards, Gustavo
Re: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi Joe,
On 17-11-2016 19:33, joe darcy wrote:
Currently, optimization for building fdlibm is disabled, except for the
"solaris" OS target [1].
>>> The reason for that is because historically the Solaris compilers have had
>>> sufficient discipline and control regarding floating-point semantics and
>>> compiler optimizations to still implement the
>>> Java-mandated results when optimization was enabled. The gcc family of
>>> compilers, for example, has lacked such discipline.
>> oh, I see. Thanks for clarifying that. I was exactly wondering why fdlibm
>> optimization is off even for x86_x64 as it, AFAICS regarding gcc 5 only, does
>> not affect the precision, even if setting -O3 does not improve the
>> performance
>> as much as on PPC64.
>
> The fdlibm code relies on aliasing a two-element array of int with a double
> to do bit-level reads and writes of floating-point values. As I understand
> it, the C spec allows compilers to assume values
> of different types don't overlap in memory. The compilation environment has
> to be configured in such a way that the C compiler disables code generation
> and optimization techniques that would run afoul
> of these fdlibm coding practices.
On discussing with the Power toolchain folks we narrowed down the issue on PPC64
to the FMA. -fno-strict-aliasing has no effect and when used with an aggressive
optimization does not solve the issue on precision. Thus -ffp-contract=off is
the best options we have by now to optimize the fdlibm on PPC64.
>>> Methods in the Math class, such as pow, are often intrinsified and use a
>>> different algorithm so a straight performance comparison may not be as fair
>>> or meaningful in those cases.
>> I agree. It's just that the issue on StrictMath methods was first noted due
>> to
>> that huge gap (Math vs StrictMath) on PPC64, which is not prominent on x64.
>
> Depending on how Math.{sin, cos} is implemented on PPC64, compiling the
> fdlibm sin/cos with more aggressive optimizations should not be expected to
> close the performance gap. In particular, if
> Math.{sin, cos} is an intrinsic on PPC64 (I haven't checked the sources) that
> used platform-specific feature (say fused multiply add instructions) then
> just compiling fdlibm more aggressively wouldn't
> necessarily make up that gap.
In our case (PPC64) it does close the gap. Non-optimized code will suffer a lot,
for instance, from load-hit-store issues. Contrary to what happens on PPC64, the
gap on x64 seems to be quite small as you said.
>
> To allow cross-platform and cross-release reproducibility, StrictMath is
> specified to use the particular fdlibm algorithms, which precludes using
> better algorithms developed more recently. If we were
> to start with a clean slate today, to get such reproducibility we would
> specify correctly-rounded behavior of all those methods, but such an approach
> was much less tractable technical 20+ years ago
> without benefit of the research that was been done in the interim, such as
> the work of Prof. Muller and associates:
> https://lipforge.ens-lyon.fr/projects/crlibm/.
>
>>
>>
>>> Accumulating the the results of the functions and comparisons the sums is
>>> not a sufficiently robust way of checking to see if the optimized versions
>>> are indeed equivalent to the non-optimized ones.
>>> The specification of StrictMath requires a particular result for each set
>>> of floating-point arguments and sums get round-away low-order bits that
>>> differ.
>> That's really good point, thanks for letting me know about that. I'll
>> re-test my
>> change under that perspective.
>>
>>
>>> Running the JDK math library regression tests and corresponding JCK tests
>>> is recommended for work in this area.
>> Got it. By "the JDK math library regression tests" you mean exactly which
>> test
>> suite? the jtreg tests?
>
> Specifically, the regression tests under test/java/lang/Math and
> test/java/lang/StrictMath in the jdk repository. There are some other math
> library tests in the hotspot repo, but I don't know where
> they are offhand.
>
> A note on methodologies, when I've been writing test for my port I've tried
> to include test cases that exercise all the branches point in the code. Due
> to the large input space (~2^64 for a
> single-argument method), random sampling alone is an inefficient way to try
> to find differences in behavior.
>> For testing against JCK/TCK I'll need some help on that.
>>
>
> I believe the JCK/TCK does have additional testcases relevant here.
>
> HTH; thanks,
>
> -Joe
>
Thank you very much for the valuable comments.
I'll send a webrev accordingly for review.
I filed a bug: https://bugs.openjdk.java.net/browse/JDK-8170153
Best regards,
Gustavo
Re: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi Chris, On 17-11-2016 19:48, Chris Plummer wrote: >> The fdlibm code relies on aliasing a two-element array of int with a double >> to do bit-level reads and writes of floating-point values. As I understand >> it, the C spec allows compilers to assume >> values of different types don't overlap in memory. The compilation >> environment has to be configured in such a way that the C compiler disables >> code generation and optimization techniques that would >> run afoul of these fdlibm coding practices. > This is the strict aliasing issue right? It's a long standing problem with > fdlibm that kept getting worse as gcc got smarter. IIRC, compiling with > -fno-strict-aliasing fixes it, but it's been more > than 12 years since I last dealt with fdlibm and compiler aliasing issues. I've tested with -O3 and -fno-strict-aliasing as you suggested but it did not fix the fp precision issue on PPC64. After finding that -fno-expensive-optimizations solved the problem, we narrowed down the problem to the FMA: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78386 Thank you. Regards, Gustavo
Re: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi Derek, On 17-11-2016 20:47, White, Derek wrote: > Hi Joe, > > Although neither a floating point expert (as I think I've proven to you over > the years), or a gcc expert, I checked with our in-house gcc expert and got > this following answer: > > "Yes using -fno-strict-aliasing fixes the issues. Also there are many > forks of fdlibm which has this fixed including the code inside glibc. " I've tried on PPC64 -O3 and -fno-strict-aliasing but it didn't work. Disabling the FMA fixed the issue although. Do you know if the gap between Math and StrictMath is also huge on aarch64? Thank you. Regards, Gustavo:
RFR(s) 8170153: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi, Could the following change be reviewed, please? webrev 1/2: http://cr.openjdk.java.net/~gromero/8170153/ webrev 2/2: http://cr.openjdk.java.net/~gromero/8170153/jdk/ bug:https://bugs.openjdk.java.net/browse/JDK-8170153 It enables fdlibm optimization on Linux PPC64 LE & BE and hence speeds up the StrictMath methods (in some cases up to 3x) on that platform. On PPC64 fdlibm optimization can be done without precision issues if floating-point expression contraction is disable, i.e. if the compiler does not use floating-point multiply-add (FMA). For further details please refer to gcc bug: https://gcc.gnu.org/bugzilla/show_bug.cgi?id=78386 No regression was observed on Math and StrictMath tests: Passed: java/lang/Math/AbsPositiveZero.java Passed: java/lang/Math/Atan2Tests.java Passed: java/lang/Math/CeilAndFloorTests.java Passed: java/lang/Math/CubeRootTests.java Passed: java/lang/Math/DivModTests.java Passed: java/lang/Math/ExactArithTests.java Passed: java/lang/Math/Expm1Tests.java Passed: java/lang/Math/FusedMultiplyAddTests.java Passed: java/lang/Math/HyperbolicTests.java Passed: java/lang/Math/HypotTests.java Passed: java/lang/Math/IeeeRecommendedTests.java Passed: java/lang/Math/Log10Tests.java Passed: java/lang/Math/Log1pTests.java Passed: java/lang/Math/MinMax.java Passed: java/lang/Math/MultiplicationTests.java Passed: java/lang/Math/PowTests.java Passed: java/lang/Math/Rint.java Passed: java/lang/Math/RoundTests.java Passed: java/lang/Math/SinCosCornerCasesTests.java Passed: java/lang/Math/TanTests.java Passed: java/lang/Math/WorstCaseTests.java Test results: passed: 21 Passed: java/lang/StrictMath/CubeRootTests.java Passed: java/lang/StrictMath/ExactArithTests.java Passed: java/lang/StrictMath/Expm1Tests.java Passed: java/lang/StrictMath/HyperbolicTests.java Passed: java/lang/StrictMath/HypotTests.java Passed: java/lang/StrictMath/Log10Tests.java Passed: java/lang/StrictMath/Log1pTests.java Passed: java/lang/StrictMath/PowTests.java Test results: passed: 8 and also on the following hotspot tests: Passed: compiler/intrinsics/mathexact/sanity/AddExactIntTest.java Passed: compiler/intrinsics/mathexact/sanity/AddExactLongTest.java Passed: compiler/intrinsics/mathexact/sanity/DecrementExactIntTest.java Passed: compiler/intrinsics/mathexact/sanity/DecrementExactLongTest.java Passed: compiler/intrinsics/mathexact/sanity/IncrementExactIntTest.java Passed: compiler/intrinsics/mathexact/sanity/IncrementExactLongTest.java Passed: compiler/intrinsics/mathexact/sanity/MultiplyExactIntTest.java Passed: compiler/intrinsics/mathexact/sanity/MultiplyExactLongTest.java Passed: compiler/intrinsics/mathexact/sanity/NegateExactIntTest.java Passed: compiler/intrinsics/mathexact/sanity/NegateExactLongTest.java Passed: compiler/intrinsics/mathexact/sanity/SubtractExactIntTest.java Passed: compiler/intrinsics/mathexact/sanity/SubtractExactLongTest.java Passed: compiler/intrinsics/mathexact/AddExactICondTest.java Passed: compiler/intrinsics/mathexact/AddExactIConstantTest.java Passed: compiler/intrinsics/mathexact/AddExactILoadTest.java Passed: compiler/intrinsics/mathexact/AddExactILoopDependentTest.java Passed: compiler/intrinsics/mathexact/AddExactINonConstantTest.java Passed: compiler/intrinsics/mathexact/AddExactIRepeatTest.java Passed: compiler/intrinsics/mathexact/AddExactLConstantTest.java Passed: compiler/intrinsics/mathexact/AddExactLNonConstantTest.java Passed: compiler/intrinsics/mathexact/CompareTest.java Passed: compiler/intrinsics/mathexact/DecExactITest.java Passed: compiler/intrinsics/mathexact/DecExactLTest.java Passed: compiler/intrinsics/mathexact/GVNTest.java Passed: compiler/intrinsics/mathexact/IncExactITest.java Passed: compiler/intrinsics/mathexact/IncExactLTest.java Passed: compiler/intrinsics/mathexact/MulExactICondTest.java Passed: compiler/intrinsics/mathexact/MulExactIConstantTest.java Passed: compiler/intrinsics/mathexact/MulExactILoadTest.java Passed: compiler/intrinsics/mathexact/MulExactILoopDependentTest.java Passed: compiler/intrinsics/mathexact/MulExactINonConstantTest.java Passed: compiler/intrinsics/mathexact/MulExactIRepeatTest.java Passed: compiler/intrinsics/mathexact/MulExactLConstantTest.java Passed: compiler/intrinsics/mathexact/MulExactLNonConstantTest.java Passed: compiler/intrinsics/mathexact/NegExactIConstantTest.java Passed: compiler/intrinsics/mathexact/NegExactILoadTest.java Passed: compiler/intrinsics/mathexact/NegExactILoopDependentTest.java Passed: compiler/intrinsics/mathexact/NegExactINonConstantTest.java Passed: compiler/intrinsics/mathexact/NegExactLConstantTest.java Passed: compiler/intrinsics/mathexact/NegExactLNonConstantTest.java Passed: compiler/intrinsics/mathexact/NestedMathExactTest.java Passed: compiler/intrinsics/mathexact/SplitThruPhiTest.java Passed: compiler/intrinsics/mathexact/SubExactICondTest.java Passed: compiler/intrinsics/mathexact/SubExactIConstantTest.java Passed: compiler/intrinsics/mathexact/SubExactILoadTest.java Passed: c
Re: RFR(s) 8170153: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi Volker, On 23-11-2016 12:05, Volker Simonis wrote: > thanks a lot for tracking this down! Happy to contribute :) > The change looks good and I a can sponsor it once you get another > review from the build group and the FC Extension Request was approved. Thanks a lot for sponsoring it! > In general I'd advise to sign the OCTLA [1] to get access to the Java > SE TCK [2] as this contains quite a lot of additional conformance > tests which can be quite valuable for changes like this. > > Regards, > Volker > > [1] http://openjdk.java.net/legal/octla-java-se-8.pdf > [2] http://openjdk.java.net/groups/conformance/JckAccess/ Right. I'll check the documentation and find a way to get access to the TCK. Best regards, Gustavo
Re: RFR(s) 8170153: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi Martin, On 23-11-2016 12:38, Doerr, Martin wrote: > Hi Gustavo, > > thanks for providing the webrevs. > > I have ran the StrictMath jck tests which fail when building with -O3 and > without -ffp-contract=off: > FailedTests: > api/java_lang/StrictMath/desc.html#acos > javasoft.sqe.tests.api.java.lang.StrictMath.acos_test > api/java_lang/StrictMath/desc.html#asin > javasoft.sqe.tests.api.java.lang.StrictMath.asin_test > api/java_lang/StrictMath/desc.html#atan > javasoft.sqe.tests.api.java.lang.StrictMath.atan_test > api/java_lang/StrictMath/desc.html#atan2 > javasoft.sqe.tests.api.java.lang.StrictMath.atan2_test > api/java_lang/StrictMath/desc.html#cos > javasoft.sqe.tests.api.java.lang.StrictMath.cos_test > api/java_lang/StrictMath/desc.html#exp > javasoft.sqe.tests.api.java.lang.StrictMath.exp_test > api/java_lang/StrictMath/desc.html#log > javasoft.sqe.tests.api.java.lang.StrictMath.log_test > api/java_lang/StrictMath/desc.html#sin > javasoft.sqe.tests.api.java.lang.StrictMath.sin_test > api/java_lang/StrictMath/desc.html#tan > javasoft.sqe.tests.api.java.lang.StrictMath.tan_test > api/java_lang/StrictMath/index.html#expm1 > javasoft.sqe.tests.api.java.lang.StrictMath.expm1Tests -TestCaseID ALL > api/java_lang/StrictMath/index.html#log10 > javasoft.sqe.tests.api.java.lang.StrictMath.log10Tests -TestCaseID ALL > api/java_lang/StrictMath/index.html#log1p > javasoft.sqe.tests.api.java.lang.StrictMath.log1pTests -TestCaseID ALL > > All of them have passed when building with -O3 and -ffp-contract=off (on > linuxppc64le). Thank you very much for running the additional StrictMath jck tests against the change! Best regards, Gustavo
Re: RFR(s) 8170153: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi Erik, On 23-11-2016 12:29, Erik Joelsson wrote: > Build changes look ok. > > In CoreLibraries.gmk, I think it would have been ok to keep the conditional > checking (OPENJDK_TARGET_CPU_ARCH, ppc), but this certainly works too. Thanks a lot for reviewing the change. Regards, Gustavo
Re: RFR(s) 8170153: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi Volker, Sorry for not replying earlier, it was day-off on Friday here... On 25-11-2016 11:32, Volker Simonis wrote: > Hi Gustavo, > > we've realized that we have exactly the same problem on Linux/s390 so > I hope you don't mind that I've updated the bug and the webrev to also > include the fix for Linux/s390: > > http://cr.openjdk.java.net/~simonis/webrevs/2016/8170153.top/ > http://cr.openjdk.java.net/~simonis/webrevs/2016/8170153.jdk/ > https://bugs.openjdk.java.net/browse/JDK-8170153 > > The top-level change stays the same (I've only added the current > reviewers) and for the jdk change I've just added Linux/s390 as > another platform which can compile fdlibm with HIGH optimization. Actually, it's really cool to know that an analysis on PPC64 contributed also to the s390 arch! :) Thanks for providing the updated webrevs. Regards, Gustavo
Re: PPC64: Poor StrictMath performance due to non-optimized compilation
Hi Volker, On 25-11-2016 14:32, Volker Simonis wrote: > On Fri, Nov 25, 2016 at 3:51 PM, Andrew Haley wrote: >> On 22/11/16 09:57, Andrew Haley wrote: >>> On 22/11/16 00:41, Gustavo Romero wrote: >>>> Do you know if the gap between Math and StrictMath is also huge on aarch64? >>> >>> I'll try to have a look. >> >> The gap is just the same as PPC. >> > > I've already extended the bug description and webrev for Linux/s390x. > Please see my mail on the review thread for this issue: > > http://mail.openjdk.java.net/pipermail/hotspot-dev/2016-November/025328.html > > If you want you're welcome to further extend it for aarch64 as well so > we can push it all in one change. > > Regards, > Volker As Andrew extended the JBS to include aarch64, I just talked to him and he will check if the solution also applies to aarch64. If so and if you don't mind I'll update your webrev to include aarch64 as well. Should I also update the JBS ticket title to include aarch64 and resent a RFR on the same thread? Finally, once the review process is done, should I update FC Extenstion Request "Remaining work" from "to be reviewed" to "none" in order to proceed with the approval request? Thank you! Regards, Gustavo
RFR(s) PPC64/s390x/aarch64: Poor StrictMath performance due to non-optimized compilation
Hi all, I'm re-sending due to JDK title update to include s390x and aarch64 archs. Could the following webrev be reviewed, please? webrev 1/2: http://cr.openjdk.java.net/~gromero/8170153/v2/ webrev 2/2: http://cr.openjdk.java.net/~gromero/8170153/v2/jdk/ bug:https://bugs.openjdk.java.net/browse/JDK-8170153 Thank you. Regards, Gustavo
Re: [aarch64-port-dev ] RFR(s) PPC64/s390x/aarch64: Poor StrictMath performance due to non-optimized compilation
Hi Andrew, On 29-11-2016 13:35, Andrew Haley wrote: > On 29/11/16 09:41, Volker Simonis wrote: >> Thanks Gustavo, >> >> the change looks good. >> >> So now we're just waiting for another review from somebody of the aarch64 >> folks. >> Once we have that and the fc-request is approved I'll push the changes. > > One thing I don't understand: > > cos 0.17098435541865692 1m7.433s 0.1709843554185943 0m56.678s > sin 1.7136493465700289 1m10.654s 1.7136493465700542 0m57.114s > > Do you know what causes the lower digits to be different? Is > it that Math and StrictMath use different algorithms, not just > different optimization levels? I don't know exactly what's the root cause for that difference (in the result). The difference is not present on x64, however on PPC64 even with -O0 (as it is by now) that difference exists. Math methods are intrisified, but StricMath are not. But I understand that Math and StrictMath share the fdlibm code since I already changed some code in fdlibm that reflected both on Math and StrictMath, so it's not clear to me where the Math relaxation occurs on PPC64 (given that such a relaxation is allowed [1]). For sure others much more experienced than I can comment about difference. Regards, Gustavo [1] https://docs.oracle.com/javase/8/docs/api/java/lang/Math.html
Re: [aarch64-port-dev ] RFR(s) PPC64/s390x/aarch64: Poor StrictMath performance due to non-optimized compilation
Hi Erik, Volker, Andrew On 29-11-2016 16:15, Andrew Haley wrote: > On 29/11/16 18:06, Volker Simonis wrote: >> @Andrew: are you fine with Gustavos latest version of the change? > > Sure. The StrictMath versions all seem to give the same results. > > Andrew. > Thanks for reviewing the change! I changed the "FC Extension Request" status to "reviewed". Regards, Gustavo
