Re: [13] RFR (S): 8217404: --with-jvm-features doesn't work when multiple features are explicitly disabled
Vladimir, if you're okay with it I'd like to propose this as a patch to
the problem instead:
http://cr.openjdk.java.net/~ihse/JDK-8217404-fix-multiple-disabled-jvm-features/webrev.01
Looks good! I verified that it fixes the bug.
Best regards,
Vladimir Ivanov
On Jan 18, 2019, at 3:33 PM, Vladimir Ivanov
wrote:
http://cr.openjdk.java.net/~vlivanov/8217404/webrev.00/
https://bugs.openjdk.java.net/browse/JDK-8217404
--with-jvm-features doesn't work properly when multiple features are
explicitly disabled:
$ bash configure --with-jvm-features="-aot -jvmci -graal"
...
checking if jvmci module jdk.internal.vm.ci should be built... yes
checking if graal module jdk.internal.vm.compiler should be built...
yes
checking if aot should be enabled... yes
...
The problem in the following code:
DISABLE_AOT=`$ECHO $DISABLED_JVM_FEATURES | $GREP aot`
if test "x$DISABLE_AOT" = "xaot"; then
ENABLE_AOT="false"
fi
Since DISABLED_JVM_FEATURES ("aot jvmci graal") contains the list of
explicitly disabled features, grep over it returns the whole list
when there's a match. The subsequent check fails because there's no
exact match, though DISABLE_AOT contains "aot" .
Proposed fix is to check there's no match instead.
After the fix it works as expected:
$ bash configure --with-jvm-features="-aot -jvmci -graal"
...
checking if jvmci module jdk.internal.vm.ci should be built... no,
forced
checking if graal module jdk.internal.vm.compiler should be built...
no, forced
checking if aot should be enabled... no, forced
...
(The fix doesn't address the case when one feature has a name which
is a proper substring of another feature, but there are no such
cases at the moment.)
Best regards,
Vladimir Ivanov
Re: RFR: JDK-8217448 Check for pandoc availability using ENABLE_PANDOC and not PANDOC
Magnus: Bug: https://bugs.openjdk.java.net/browse/JDK-8217448 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8217448-fix-pandoc-on-windows/webrev.01 Looks good. Tim
RFR: JDK-8217269: jpackage Makefile improvments
re-sending this to build-dev: Please review the jpackage fix for bug [1] at [2]. This is a fix for the JDK-8200758-branch branch of the open sandbox repository (jpackage). The webrev includes all the jpackage Makefile Improvements listed in [1], other than what is covered by [4], and also includes fix for white space errors requested in [3]. [1] https://bugs.openjdk.java.net/browse/JDK-8217269 [2] http://cr.openjdk.java.net/~herrick/8217269/webrev.01/ [3] https://bugs.openjdk.java.net/browse/JDK-8216521 [4] https://bugs.openjdk.java.net/browse/JDK-8217317 /Andy
RFR: JDK-8217448 Check for pandoc availability using ENABLE_PANDOC and not PANDOC
In some places, we have checked if we have pandoc available by comparing $(PANDOC) with the empty string, assuming that non-empty means that we've found pandoc. However, on Windows, PANDOC will always contain $(FIXPATH), so this is not a valid test. Instead we should use ENABLE_PANDOC. Bug: https://bugs.openjdk.java.net/browse/JDK-8217448 WebRev: http://cr.openjdk.java.net/~ihse/JDK-8217448-fix-pandoc-on-windows/webrev.01 /Magnus
Re: 'make update-build-docs' fails with 'fixpath Unknown argument: --toc' on Windows
On 2019-01-21 12:31, Jorn Vernee wrote: I'm downstream from JDK-8215635, I can see it in the revision history. I've also checked the files that were updated, and the changes are still there as well. Thank you. Then we need to provide additional checks to handle pandoc on Windows. I've filed https://bugs.openjdk.java.net/browse/JDK-8217448. /Magnus Jorn Magnus Ihse Bursie schreef op 2019-01-21 11:58: On 2019-01-17 15:42, Jorn Vernee wrote: Hello, I'm updating some documentation for the panama repo foreign branch [1]. When running `make update-build-docs` I get the following output: ``` $ make update-build-docs Building target 'update-build-docs' in configuration 'windows-x86_64-server-release' fixpath Unknown argument: --toc fixpath Unknown argument: --toc fixpath Unknown argument: --toc make[3]: *** [UpdateBuildDocs.gmk:50: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/building.html] Error 127 make[3]: *** Waiting for unfinished jobs make[3]: *** [UpdateBuildDocs.gmk:58: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/testing.html] Error 127 make[3]: *** [UpdateBuildDocs.gmk:66: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/panama_foreign.html] Error 127 make[2]: *** [make/Main.gmk:417: update-build-docs] Error 2 Can you please verify if the fix for JDK-8215635 is included in the source you're trying to build? I thought that this should not happen after that fix, but instead no markdown files should be processed if pandoc is missing from the configuration. However, if you have that fix and still see the above error without a proper pandoc, then the fix for JDK-8215635 is not complete. /Magnus ERROR: Build failed for target 'update-build-docs' in configuration 'windows-x86_64-server-release' (exit code 2) === Output from failing command(s) repeated here === * For target support_markdown_building_building.md: fixpath Unknown argument: --toc * For target support_markdown_panama_foreign_panama_foreign.md: fixpath Unknown argument: --toc * For target support_markdown_testing_testing.md: fixpath Unknown argument: --toc * All command lines available in /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/build/windows-x86_64-server-release/make-support/failure-logs. === End of repeated output === No indication of failed target found. Hint: Try searching the build log for '] Error'. Hint: See doc/building.html#troubleshooting for assistance. make[1]: *** [/home/Jorn/cygwin-projects-new/panama/make/Init.gmk:310: main] Error 2 make: *** [/home/Jorn/cygwin-projects-new/panama/make/Init.gmk:186: update-build-docs] Error 2 ``` Is this a know issue? Is this supported on Windows? Thanks, Jorn [1] : http://hg.openjdk.java.net/panama/dev/shortlog/b981c23cb71e
Re: 'make update-build-docs' fails with 'fixpath Unknown argument: --toc' on Windows
I'm downstream from JDK-8215635, I can see it in the revision history. I've also checked the files that were updated, and the changes are still there as well. Jorn Magnus Ihse Bursie schreef op 2019-01-21 11:58: On 2019-01-17 15:42, Jorn Vernee wrote: Hello, I'm updating some documentation for the panama repo foreign branch [1]. When running `make update-build-docs` I get the following output: ``` $ make update-build-docs Building target 'update-build-docs' in configuration 'windows-x86_64-server-release' fixpath Unknown argument: --toc fixpath Unknown argument: --toc fixpath Unknown argument: --toc make[3]: *** [UpdateBuildDocs.gmk:50: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/building.html] Error 127 make[3]: *** Waiting for unfinished jobs make[3]: *** [UpdateBuildDocs.gmk:58: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/testing.html] Error 127 make[3]: *** [UpdateBuildDocs.gmk:66: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/panama_foreign.html] Error 127 make[2]: *** [make/Main.gmk:417: update-build-docs] Error 2 Can you please verify if the fix for JDK-8215635 is included in the source you're trying to build? I thought that this should not happen after that fix, but instead no markdown files should be processed if pandoc is missing from the configuration. However, if you have that fix and still see the above error without a proper pandoc, then the fix for JDK-8215635 is not complete. /Magnus ERROR: Build failed for target 'update-build-docs' in configuration 'windows-x86_64-server-release' (exit code 2) === Output from failing command(s) repeated here === * For target support_markdown_building_building.md: fixpath Unknown argument: --toc * For target support_markdown_panama_foreign_panama_foreign.md: fixpath Unknown argument: --toc * For target support_markdown_testing_testing.md: fixpath Unknown argument: --toc * All command lines available in /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/build/windows-x86_64-server-release/make-support/failure-logs. === End of repeated output === No indication of failed target found. Hint: Try searching the build log for '] Error'. Hint: See doc/building.html#troubleshooting for assistance. make[1]: *** [/home/Jorn/cygwin-projects-new/panama/make/Init.gmk:310: main] Error 2 make: *** [/home/Jorn/cygwin-projects-new/panama/make/Init.gmk:186: update-build-docs] Error 2 ``` Is this a know issue? Is this supported on Windows? Thanks, Jorn [1] : http://hg.openjdk.java.net/panama/dev/shortlog/b981c23cb71e
Re: 'make update-build-docs' fails with 'fixpath Unknown argument: --toc' on Windows
On 2019-01-17 15:42, Jorn Vernee wrote: Hello, I'm updating some documentation for the panama repo foreign branch [1]. When running `make update-build-docs` I get the following output: ``` $ make update-build-docs Building target 'update-build-docs' in configuration 'windows-x86_64-server-release' fixpath Unknown argument: --toc fixpath Unknown argument: --toc fixpath Unknown argument: --toc make[3]: *** [UpdateBuildDocs.gmk:50: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/building.html] Error 127 make[3]: *** Waiting for unfinished jobs make[3]: *** [UpdateBuildDocs.gmk:58: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/testing.html] Error 127 make[3]: *** [UpdateBuildDocs.gmk:66: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/panama_foreign.html] Error 127 make[2]: *** [make/Main.gmk:417: update-build-docs] Error 2 Can you please verify if the fix for JDK-8215635 is included in the source you're trying to build? I thought that this should not happen after that fix, but instead no markdown files should be processed if pandoc is missing from the configuration. However, if you have that fix and still see the above error without a proper pandoc, then the fix for JDK-8215635 is not complete. /Magnus ERROR: Build failed for target 'update-build-docs' in configuration 'windows-x86_64-server-release' (exit code 2) === Output from failing command(s) repeated here === * For target support_markdown_building_building.md: fixpath Unknown argument: --toc * For target support_markdown_panama_foreign_panama_foreign.md: fixpath Unknown argument: --toc * For target support_markdown_testing_testing.md: fixpath Unknown argument: --toc * All command lines available in /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/build/windows-x86_64-server-release/make-support/failure-logs. === End of repeated output === No indication of failed target found. Hint: Try searching the build log for '] Error'. Hint: See doc/building.html#troubleshooting for assistance. make[1]: *** [/home/Jorn/cygwin-projects-new/panama/make/Init.gmk:310: main] Error 2 make: *** [/home/Jorn/cygwin-projects-new/panama/make/Init.gmk:186: update-build-docs] Error 2 ``` Is this a know issue? Is this supported on Windows? Thanks, Jorn [1] : http://hg.openjdk.java.net/panama/dev/shortlog/b981c23cb71e
Re: 'make update-build-docs' fails with 'fixpath Unknown argument: --toc' on Windows
On 2019-01-17 18:25, Jorn Vernee wrote: Never mind, this works: bash configure PANDOC=/cygdrive/j/ChocolateyInstall/bin/pandoc.exe I should have tried that first. Yes, that is the recommended way to override individual tools. "make reconfigure" is just re-running the last "bash configure" invocation, with the same command line. At that point, no changes can be made to the configuration. And while "PANDOC=foo bash configure" works, we recommend against it (and warn), since we are unable to catch an error such as "PANDCO=foo bash configure" -- but in contrast "bash configure PANDCO=foo" will result in a warning (but it really should be an error). /Magnus Sorry, Jorn Jorn Vernee schreef op 2019-01-17 18:07: Hi Erik, Thanks for the insights. The make/devkit/createPandocBundle.sh seems to try to install the linux version of pandoc. I have installed pandoc 2.5 through chocolatey instead. I'm trying to pas the exe path through the PANDOC environment variable, but this produces a configure warning: configure: WARNING: Ignoring value of PANDOC from the environment. Use command line variables instead. I'm not sure what is meant here by 'command line variable' there. I've tried the following: $ PANDOC=/cygdrive/j/ChocolateyInstall/bin/pandoc.exe $ make reconfigure But this gives the same warning. I've also tried: $ make reconfigure PANDOC=/cygdrive/j/ChocolateyInstall/bin/pandoc.exe But this gives a warning that I'm using a non-control variable. How should I pass the value to configure? Thanks, Jorn Erik Joelsson schreef op 2019-01-17 17:46: You need pandoc to generate the html files from md. We also use pandoc to generate files for the actual product documentation. It seems the build only checks that pandoc is available for the product documentation targets and for update-build-docs, it just tries and fails with this rather uninformative error message. I'm pretty sure the target works if you provide pandoc to configure on Windows. There is a script in make/devkit/createPandocBundle.sh that can help you get the pandoc you need. /Erik On 2019-01-17 06:42, Jorn Vernee wrote: Hello, I'm updating some documentation for the panama repo foreign branch [1]. When running `make update-build-docs` I get the following output: ``` $ make update-build-docs Building target 'update-build-docs' in configuration 'windows-x86_64-server-release' fixpath Unknown argument: --toc fixpath Unknown argument: --toc fixpath Unknown argument: --toc make[3]: *** [UpdateBuildDocs.gmk:50: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/building.html] Error 127 make[3]: *** Waiting for unfinished jobs make[3]: *** [UpdateBuildDocs.gmk:58: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/testing.html] Error 127 make[3]: *** [UpdateBuildDocs.gmk:66: /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/doc/panama_foreign.html] Error 127 make[2]: *** [make/Main.gmk:417: update-build-docs] Error 2 ERROR: Build failed for target 'update-build-docs' in configuration 'windows-x86_64-server-release' (exit code 2) === Output from failing command(s) repeated here === * For target support_markdown_building_building.md: fixpath Unknown argument: --toc * For target support_markdown_panama_foreign_panama_foreign.md: fixpath Unknown argument: --toc * For target support_markdown_testing_testing.md: fixpath Unknown argument: --toc * All command lines available in /cygdrive/h/cygwin64/home/Jorn/cygwin-projects-new/panama/build/windows-x86_64-server-release/make-support/failure-logs. === End of repeated output === No indication of failed target found. Hint: Try searching the build log for '] Error'. Hint: See doc/building.html#troubleshooting for assistance. make[1]: *** [/home/Jorn/cygwin-projects-new/panama/make/Init.gmk:310: main] Error 2 make: *** [/home/Jorn/cygwin-projects-new/panama/make/Init.gmk:186: update-build-docs] Error 2 ``` Is this a know issue? Is this supported on Windows? Thanks, Jorn [1] : http://hg.openjdk.java.net/panama/dev/shortlog/b981c23cb71e
Re: RFR: JDK-8217335: Add a script to generate --release data
On 2019-01-18 18:05, Jan Lahoda wrote: Hi Magnus, Thanks for the comments, an updated webrev: http://cr.openjdk.java.net/~jlahoda/8217335/webrev.01/ (Includes changes suggested by Joe was well). Looks good to me! /Magnus Jan On 18.1.2019 13:59, Magnus Ihse Bursie wrote: On 2019-01-17 19:51, Jan Lahoda wrote: Hi, As suggested here: https://mail.openjdk.java.net/pipermail/compiler-dev/2019-January/012876.html This is a separate patch that adds a script to generate --release N data. Webrev: http://cr.openjdk.java.net/~jlahoda/8217335/webrev.00/ JBS: https://bugs.openjdk.java.net/browse/JDK-8217335 Jan Hi Jan, I would recommend that you place the generate-data script in make/scripts; it's the default location for scripts -- make/data is reserved for, eh, data. :) But a bit more descriptive name would be suitable when moving it away from make/data/symbols -- perhaps generate-symbol-data? Also, the update to the README file does not seem correct. I assume you want to point to the new generate-data script, but the path is wrong (even for the location in your current patch). Finally, we in the build team always appreciates if you cc build-dev for changes in the make directory, even if it's not strictly part of the build system. /Magnus
Re: [13] RFR (S): 8217404: --with-jvm-features doesn't work when multiple features are explicitly disabled
On 2019-01-19 01:39, Vladimir Ivanov wrote:
Thanks, Igor.
overall your fix looks reasonable, but w/ it we can get
unintentionally disabled features (b/c grep doesn't do full word
match). although this problem wasn't really introduced by your fix, I
think it's be better to fix it as a part of your patch. I see two
possible solutions:
I was aware of such drawback, but decided to leave it as is, since it
doesn't affect existing features.
- add "-w" to grep, but I am not sure if "-w" is supported by all
grep implementations
- use $XARGS instead of $ECHO when we get DISABLE_X. in this case
you will need to revert your changes in 'if test ...' lines
I'm in favor of using "-w" and I see different grep flags being used
already, but would like somebody from Build team confirm they are OK
with such solution.
I think an even better solution is to use the pattern of
HOTSPOT_CHECK_JVM_FEATURE. This should solve all potential problems, and
is moving the abstraction level up slightly.
I've been working for some time now on better structure for handling the
JVM feature testing. While we are using the feature testing as I
intended it, the underlying support for doing this in a good way has
never been put into place. Unfortunately, this fix has been on low
priority and been mostly idling on my disk, half done, for several
months now. So we need to have an interim solution to this problem. But
I'd like to see that the fix takes at least a step towards a better
abstraction.
Vladimir, if you're okay with it I'd like to propose this as a patch to
the problem instead:
http://cr.openjdk.java.net/~ihse/JDK-8217404-fix-multiple-disabled-jvm-features/webrev.01
/Magnus
Best regards,
Vladimir Ivanov
On Jan 18, 2019, at 3:33 PM, Vladimir Ivanov
wrote:
http://cr.openjdk.java.net/~vlivanov/8217404/webrev.00/
https://bugs.openjdk.java.net/browse/JDK-8217404
--with-jvm-features doesn't work properly when multiple features are
explicitly disabled:
$ bash configure --with-jvm-features="-aot -jvmci -graal"
...
checking if jvmci module jdk.internal.vm.ci should be built... yes
checking if graal module jdk.internal.vm.compiler should be built...
yes
checking if aot should be enabled... yes
...
The problem in the following code:
DISABLE_AOT=`$ECHO $DISABLED_JVM_FEATURES | $GREP aot`
if test "x$DISABLE_AOT" = "xaot"; then
ENABLE_AOT="false"
fi
Since DISABLED_JVM_FEATURES ("aot jvmci graal") contains the list of
explicitly disabled features, grep over it returns the whole list
when there's a match. The subsequent check fails because there's no
exact match, though DISABLE_AOT contains "aot" .
Proposed fix is to check there's no match instead.
After the fix it works as expected:
$ bash configure --with-jvm-features="-aot -jvmci -graal"
...
checking if jvmci module jdk.internal.vm.ci should be built... no,
forced
checking if graal module jdk.internal.vm.compiler should be built...
no, forced
checking if aot should be enabled... no, forced
...
(The fix doesn't address the case when one feature has a name which
is a proper substring of another feature, but there are no such
cases at the moment.)
Best regards,
Vladimir Ivanov
Re: [13] RFR (S): 8217404: --with-jvm-features doesn't work when multiple features are explicitly disabled
On 2019-01-20 09:45, Kim Barrett wrote: On Jan 18, 2019, at 7:31 PM, Vladimir Ivanov wrote: Thanks, Vladimir. I usually used --with-jvm-features=-aot,-jvmci,-graal Did not work in this case too? I didn't know it supports comma-separated list, but it doesn't work as well: $ bash configure --with-jvm-features="-aot,-jvmci,-graal" checking if jvmci module jdk.internal.vm.ci should be built... yes checking if graal module jdk.internal.vm.compiler should be built... yes checking if aot should be enabled… yes Isn’t the problem here simply incorrect syntax in that command line? Drop the quotes around the —with-jvm-features argument and I think it should work fine. Let me bring some clarity in the syntax here. For --with-jvm-features, if you want to list multiple features, you can separate them by either space or comma. Both are valid and officially supported. My recommendation, though, is to use comma, to avoid the problem with spaces in command line options. If you want to use spaces, you *must* use quotes. A command line like: bash configure --with-jvm-features=-aot -jvmci would be interpreted as "-jvmci" was a flag for configure, which it is not. There are multiple ways of quoting, you could use ' or ", and include the flag name like "--with-jvm-features=aot graal", or just the argument list. My preference, if I need to use quotes, is to use the style Vladimir uses in his example; I believe that maximizes readability. But, as I said, for --with-jvm-features, I recommend using comma instead, to avoid the quoting issue completely. Internally, the list of enabled/disabled features are treated as lists of space-separated words; but that is an implementation detail and not part of the command-line interface. /Magnus
Fwd: Enhancing jaotc to automatically find VS2017 linker
I've completely missed vswhere. It's years and years overdue, but it's nice that Microsoft is finally shipping something like this. :-) We should support using vswhere as the primary way to detect a Visual Studio installation, if it is present, and the user do not override the default selection on the command line. /Magnus Forwarded Message Subject:Enhancing jaotc to automatically find VS2017 linker Date: Fri, 18 Jan 2019 22:16:51 + From: Andrew Luo To: hotspot-compiler-...@openjdk.java.net Hi, Has there been any plans to enhance jaotc to support automatically finding the link.exe in VS2017? If not, I am interested in contributing some work to support this. I see that in Linker.java (src/jdk.aot/share/classes/jdk.tools.jaotc/src/jdk/tools/jaotc/Linker.java) we find link.exe using the environment variables VS…COMNTOOLS, but since in VS2017 and forward, this is not defined, it seems another approach is necessary. Microsoft suggests that you use vswhere (https://github.com/Microsoft/vswhere, BSD licensed, included with Visual Studio 2017 15.2 and forward) or their COM API to find the latest VS2017 toolset. Anyways, if everyone agrees we should add VS2017 support, there are a few ways to do this (in order of simplest/easiest to most complex): 1.Check that vswhere exists on the system, if it does, call vswhere (out of process – not sure this is acceptable…) and use that to find the VS2017 link.exe 2.Ship vswhere with the JDK and call it out of process 3.Statically link a copy of vswhere (BSD licensed – is this okay?) into our code and add a JNI stub to call it 4.Call the COM API in a JNI function to get the latest version of VS2017 Personally I prefer (1), but if out-of-process isn’t acceptable I’m fine with doing (4) or (3). Let me know if you have any comments/feedback on this proposal. Thanks, -Andrew
