Re: Review Request : JDK-8213362 : Could not find libjava.dylib error when initializing JVM via JNI_CreateJavaVM

[Priyanka Mangal]

The Eclipse launcher is capable of loading the Java VM in the eclipse 
process using the Java Native Interface Invocation API.  The launcher is 
still capable of starting the Java VM in a separate process the same as 
previous version of Eclipse did.  Which method is used depends on how 
the VM was found.



   No -vm specified

When no -vm is specified, the launcher looks for a virtual machine first 
in a jre directory in the root of eclipse and then on the search path. 
If java is found in either location, then the launcher looks for a jvm 
shared library (jvm.dll on Windows, libjvm.so on *nix platforms) 
relative to that java executable.


 * If a jvm shared library is found the launcher loads it and uses the
   JNI invocation API to start the vm.
 * If no jvm shared library is found, the launcher executes the java
   launcher to start the vm in a new process.


   -vm specified on the command line or in eclipse.ini

Eclipse can be started with "-vm " to indicate a virtual 
machine to use. There are several possibilities for the value of :


1. directory:   is a directory. We look in that directory
   for: (1) a java launcher or (2) the jvm shared library. If we find
   the jvm shared library, we use JNI invocation. If we find a
   launcher, we attempt to find a jvm library in known locations
   relative to the launcher. If we find one, we use JNI invocation. If
   no jvm library is found, we exec java in a new process.
2. java.exe/javaw.exe:  is a path to a java launcher. We exec
   that java launcher to start the vm in a new process.
3. jvm dll or so:  is a path to a jvm shared library. We
   attempt to load that library and use the JNI Invocation API to start
   the vm in the current process.

Found from 
https://help.eclipse.org/luna/index.jsp?topic=%2Forg.eclipse.platform.doc.isv%2Freference%2Fmisc%2Flauncher.html

Regards
Priyanka
On 11/5/2018 8:24 PM, Alan Bateman wrote:



On 05/11/2018 14:21, Magnus Ihse Bursie wrote:

Hi,

Fix looks good, but maybe we should have a regression test of 
GetJREPath()?


/Magnus
The fix looks okay but I'm puzzled as to how Eclipse is running into 
this. Do they locate/call libjli/GetJREPath directly or is this 
happening then it is called indirectly?


-Alan

Re: RFR: JDK-8213227: Update jib src excludes to filter webrev and Jreg directories

[David Holmes]

Erik,

I think this has broken the creation of the "release" file in the build. 
It no longer contains the "hg" tag, presumably because all the .hg 
directories that would have indicated hg was used have been stripped.


CI tests are failing for test/jdk/build/releaseFile/CheckSource.java

David

On 1/11/2018 10:26 PM, Magnus Ihse Bursie wrote:

On 2018-11-01 01:22, Erik Joelsson wrote:
This patch adds some useful excludes for jib source bundle creation, 
which will help avoid sending in too much junk in Mach 5 jobs.


The strings are rewritten as arrays, which are now supported, as well 
as using ** globs to match arbitrary directory depths. I've also 
removed the no longer used .build dir from excludes.


http://cr.openjdk.java.net/~erikj/8213227/webrev.01/

Looks good to me.

/Magnus



Bug: https://bugs.openjdk.java.net/browse/JDK-8213227

/Erik

Re: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency

[Erik Joelsson]

On 2018-11-05 01:41, Magnus Ihse Bursie wrote:
If I were to spend more time on PCH, I'd rather spend it on either 
Windows (which is the slowest build platform in our CI), or Solaris, 
which does not have precompiled headers at all. (I actually looked 
into solstudio PCH as part of this fix to see if the new set of 
headers would allow us to enable it. Unfortunately, it seems like they 
need a single PCH per directory of source code files (e.g. all files 
in share/logging can share PCH with each other, but they cannot share 
PCH with the files in share/utilities. As I understood the problem, at 
least; the error messages were unclear and undocumented.)


I looked into this before. There were several limitations that made it 
very tricky. The compiler was extremely picky with command lines 
matching, including something with directories, which is probably what 
you describe (I don't remember well at this point). I managed to hack 
around it to get some C++ files compiled with a PCH so I could get a 
feel for the potential speedup. It seemed like the best gain I would get 
was about 10% per compiled file. I did not think that was enough gain to 
motivate the amount of hackery required to get it to actually work.


Looking inside the PCH from Solaris Studio, they were basically just 
preprocessed header files, no binaries.


/Erik

/Magnus



Cheers,
David


Thanks, Thomas


That would also remove the problem of things not compiling with PCH
disabled as a single set of includes would be used for both PCH and
non-PCH building.

I don't know if the other compilers support PCH in a similar manner 
to gcc.


Cheers,
David


/Magnus



David


   25 // Precompiled headers are turned off for Sun Studio,
May as well change to Solaris Studio if you're going to fix the 
typo :)

Thanks.
David

On 3/11/2018 7:06 PM, Magnus Ihse Bursie wrote:
The reasons for the current set of files included in 
precompiled.hpp is somewhat lost in the mists of history. 
However, it is clear that it is not optimal.


This patch replaces the current set with a new set, based on 
how often a header file is included in a C++ file. This 
selection contains all header files that are included by at 
least 130 C++ files. Testing has shown that this is around the 
optimal value -- include many more, and too many "innocent" 
files get hurt by unneeded work, leave out many more, and we 
miss out on optimization possibilities.


The same set turned out to work well for both clang and gcc. 
However, files named "*.inline.hpp" did hurt rather than help 
performance, so those have been left out. For visual studio, 
the same set was also optimal, as long as the inline files were 
included. Presumably, visual studio is better than gcc/clang on 
handling precompiled headers containing inlined code.


Here are some rough comparisons from our internal CI system, 
for building the target "hotspot" from scratch.


macosx-x64:
old: 00:05:00
new: 00:03:47

linux-x64:
old: 00:05:43
new: 00:04:51

windows-x64:
old: 00:05:18
new: 00:04:33

linux-aarch64:
old: 00:07:57
new: 00:03:48

Bug: https://bugs.openjdk.java.net/browse/JDK-8213339
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01


/Magnus

Re: Stop using precompiled headers for Linux?

[Aleksey Shipilev]

On 11/05/2018 05:46 PM, Erik Joelsson wrote:
>> If we decide to keep precompiled headers on by default, maybe we should add 
>> a simple no-PCH
>> verification task in tier1? It only needs to build hotspot, so it should be 
>> quick.
>>
> That is a good point, so sure we can do that. Which debug level would be most 
> appropriate for this
> test, debug or slowdebug? I very much doubt it's relevant to run release 
> builds without PCH.

Choosing between fastdebug or slowdebug, I'd choose fastdebug. slowdebug has, 
or at least used to
have its own warts (oopDesc verification?) that can mask build issues.

-Aleksey

Re: Stop using precompiled headers for Linux?

[Erik Joelsson]

On 2018-11-03 01:51, Magnus Ihse Bursie wrote:


On 2018-10-30 20:21, Erik Joelsson wrote:
Last I checked, it did provide significant build speed improvements 
when building just hotspot, but that could need revisiting.


We do have verification of --disable-precompiled-headers (in 
slowdebug) in builds-tier2 so we normally get notified if this fails. 
However, Mach5 has not been running since Friday so this particular 
bug wasn't detected automatically. Looking at the bug, it also failed 
on Solaris, which would have been caught by tier1 builds.


If we decide to keep precompiled headers on by default, maybe we 
should add a simple no-PCH verification task in tier1? It only needs 
to build hotspot, so it should be quick.


That is a good point, so sure we can do that. Which debug level would be 
most appropriate for this test, debug or slowdebug? I very much doubt 
it's relevant to run release builds without PCH.


/Erik

/Magnus



/Erik


On 2018-10-30 10:26, Ioi Lam wrote:
Is there any advantage of using precompiled headers on Linux? It's 
on by default and we keep having breakage where someone would forget 
to add #include. The latest instance is JDK-8213148.


I just turn on precompiled headers explicitly in all my builds. I 
don't see any difference in build time (at least not significant 
enough for me to bother).


Should we disable it by default on Linux?

Thanks

- Ioi

Re: Review Request : JDK-8213362 : Could not find libjava.dylib error when initializing JVM via JNI_CreateJavaVM

[Alan Bateman]

On 05/11/2018 14:21, Magnus Ihse Bursie wrote:

Hi,

Fix looks good, but maybe we should have a regression test of GetJREPath()?

/Magnus
The fix looks okay but I'm puzzled as to how Eclipse is running into 
this. Do they locate/call libjli/GetJREPath directly or is this 
happening then it is called indirectly?


-Alan

Re: Review Request : JDK-8213362 : Could not find libjava.dylib error when initializing JVM via JNI_CreateJavaVM

[Magnus Ihse Bursie]

Hi,

Fix looks good, but maybe we should have a regression test of GetJREPath()?

/Magnus

> 5 nov. 2018 kl. 12:09 skrev Priyanka Mangal :
> 
> Hi,
> 
> Please review the minor fix for :
> https://bugs.openjdk.java.net/browse/JDK-8213362
> http://cr.openjdk.java.net/~pmangal/8213362/webrev.00/
> 
> With the JDK-8210931 <https://bugs.openjdk.java.net/browse/JDK-8210931> 
> libjli has moved from "jli" subdirectory into standard lib directory.
> However `GetJREPath` method in 
> `src/java.base/macosx/native/libjli/java_md_macosx.m` still assumes that 
> `libjli.dylib` should be on `lib/jli/libjli.dylib` path.
> So corrected the same.
> 
> Testing:
> mach5 : tier1-3
> http://java.se.oracle.com:10065/mdash/jobs/fmatte-jdk-20181105-0941-8721?search=result.status:*
> 
> Thanks
> Regards
> Priyanka
> 
> 

Review Request : JDK-8213362 : Could not find libjava.dylib error when initializing JVM via JNI_CreateJavaVM

[Priyanka Mangal]

Hi,

Please review the minor fix for :
https://bugs.openjdk.java.net/browse/JDK-8213362
http://cr.openjdk.java.net/~pmangal/8213362/webrev.00/

With the JDK-8210931 <https://bugs.openjdk.java.net/browse/JDK-8210931> 
libjli has moved from "jli" subdirectory into standard lib directory.
However `GetJREPath` method in 
`src/java.base/macosx/native/libjli/java_md_macosx.m` still assumes that 
`libjli.dylib` should be on `lib/jli/libjli.dylib` path.

So corrected the same.

Testing:
mach5 : tier1-3
http://java.se.oracle.com:10065/mdash/jobs/fmatte-jdk-20181105-0941-8721?search=result.status:*

Thanks
Regards
Priyanka

Re: compiling openJdk 11 on windows 7 32bits fail

[Magnus Ihse Bursie]

On 2018-11-04 06:16, Franco Gastón Pellegrini wrote:

I used this command:
/bash ./configure --enable-debug --with-target-bits=32 
--with-toolchain-version=2017 --with-jvm-variants=client 
--with-boot-jdk="/home/Franco/Java/jdk10/build/windows-x86-normal-client-fastdebug/jdk/";/

I tried Visual studio 2015 and 2017 (with-toolchain-version).
I attached config.log as you asked.
Thanks.

El sáb., 3 de nov. de 2018 a la(s) 04:43, Magnus Ihse Bursie 
(magnus.ihse.bur...@oracle.com ) 
escribió:


Have you tried running

/cygdrive/c/progra~1/micros~1/2017/commun~1/vc/tools/msvc/1415~1.267/bin/hostx86/x86/cl

from the same prompt you ran configure to see what happens? It
seems to
have located a compiler but fails to run it.



Let me re-iterate this question: Have you tried running 
/cygdrive/c/progra~1/micros~1/2017/commun~1/vc/tools/msvc/1415~1.267/bin/hostx86/x86/cl 
from the same prompt you ran configure to see what happens?


Here is the relevant part of the config.log, it basically says it cannot 
compile a simple C program. You can try recreating conftest.c as shown 
in the log below, or try just any other simple C program.


configure:35681: checking whether the C compiler works
configure:35703: 
/cygdrive/c/progra~1/micros~1/2017/commun~1/vc/tools/msvc/1415~1.267/bin/hostx86/x86/cl 
conftest.c  >&5

Microsoft (R) C/C++ Optimizing Compiler Version 19.15.26732.1 for x86
Copyright (C) Microsoft Corporation.  All rights reserved.

conftest.c
Microsoft (R) Incremental Linker Version 14.15.26732.1
Copyright (C) Microsoft Corporation.  All rights reserved.

/out:conftest.exe
conftest.obj
LINK : fatal error LNK1104: cannot open file 'conftest.exe'
configure:35707: $? = 2
configure:35745: result: no
configure: failed program was:
| /* confdefs.h */
| #define PACKAGE_NAME "OpenJDK"
| #define PACKAGE_TARNAME "openjdk"
| #define PACKAGE_VERSION "openjdk"
| #define PACKAGE_STRING "OpenJDK openjdk"
| #define PACKAGE_BUGREPORT "build-dev@openjdk.java.net"
| #define PACKAGE_URL "http://openjdk.java.net";
| /* end confdefs.h.  */
|
| int
| main ()
| {
|
|   ;
|   return 0;
| }
configure:35750: error: in `/home/Franco/Java/jdk11':
configure:35752: error: C compiler cannot create executables

/Magnus



There is also a second log file, config.log, generated by
autoconf. It
will hopefully contain output from the failed run of CL, and the
complete command line it used when it concluded that the C compiler
cannot create executables.

/Magnus

On 2018-11-03 02:50, Franco Gastón Pellegrini wrote:
> Hello,
> When I try to compìle OpenJDK 11, I get an error. I tried using
visual
> studio 2015 and 2017 with the same error. *Note that compiling
OpenJDK 9
> and 10 work just fine.*
>
> I attached the log, but the error is:
>
>
**
> ** Visual Studio 2017 Developer Command Prompt v15.8.9
> ** Copyright (c) 2017 Microsoft Corporation
>
**
> [vcvarsall.bat] Environment initialized for: 'x86'
> configure: Setting extracted environment variables
> checking for Visual Studio variables... ok
> configure: Rewriting ipath to
>

"/cygdrive/c/progra~1/micros~1/2017/commun~1/vc/tools/msvc/1415~1.267/include"
> configure: Rewriting ipath to
> "/cygdrive/c/progra~1/wi3cf2~1/netfxsdk/46d346~1.1/include/um"
> configure: Rewriting ipath to
> "/cygdrive/c/progra~1/wi3cf2~1/10/include/100171~1.0/ucrt"
> configure: Rewriting ipath to
> "/cygdrive/c/progra~1/wi3cf2~1/10/include/100171~1.0/shared"
> configure: Rewriting ipath to
> "/cygdrive/c/progra~1/wi3cf2~1/10/include/100171~1.0/um"
> configure: Rewriting ipath to
> "/cygdrive/c/progra~1/wi3cf2~1/10/include/100171~1.0/winrt"
> configure: Rewriting ipath to
> "/cygdrive/c/progra~1/wi3cf2~1/10/include/100171~1.0/cppwinrt"
> configure: Rewriting libpath to
>

"/cygdrive/c/progra~1/micros~1/2017/commun~1/vc/tools/msvc/1415~1.267/lib/x86"
> configure: Rewriting libpath to
> "/cygdrive/c/progra~1/wi3cf2~1/netfxsdk/46d346~1.1/lib/um/x86"
> configure: Rewriting libpath to
> "/cygdrive/c/progra~1/wi3cf2~1/10/lib/100171~1.0/ucrt/x86"
> configure: Rewriting libpath to
> "/cygdrive/c/progra~1/wi3cf2~1/10/lib/100171~1.0/um/x86"
> checking for cl...
>

/cygdrive/c/PROGRA~1/MICROS~1/2017/COMMUN~1/VC/Tools/MSVC/14.15.26726/bin/HostX86/x86/cl
> configure: Rewriting CC to
>

"/cygdrive/c/progra~1/micros~1/2017/commun~1/vc/tools/msvc/1415~1.267/bin/hostx86/x86/cl"
> checking resolved symbolic links for CC... no symlink
> configure: Using microsoft C compiler version 19.15.26732.1
[Microsoft (R)
> C/C++ Optimizing Compiler Version 19.15.26732.1 for x86]
> checking whether the C compiler works... 

Re: RFR: JDK-8213339 Update precompiled.hpp with headers based on current frequency

[Magnus Ihse Bursie]

On 2018-11-05 08:28, David Holmes wrote:

Hi Thomas,

On 5/11/2018 5:17 PM, Thomas Stüfe wrote:

Hi David,

On Sun, Nov 4, 2018 at 11:09 PM David Holmes 
 wrote:


On 4/11/2018 5:27 PM, Magnus Ihse Bursie wrote:



3 nov. 2018 kl. 23:24 skrev David Holmes :


On 3/11/2018 10:09 PM, David Holmes wrote:
Looks okay - thanks for doing all the experiments! (Though I'm 
still curious what happens if you recompile individual header 
files :) ).


s/recompile/precompile/  :)


What do you mean? To have more than one set of PCH? As far as I 
know, all compilers we use only support a single PCH per source 
file to compile. So while you could have a PCH tailored per source 
file, that would only make sense in a scenario were you recompile 
all source files (but no header files) often.


The GCC description for PCH describes it as a per-file header file
optimization. You can precompile each .hpp file and if it is located in
the same directory as the .hpp (or is on an include directory ahead of
the .hpp) then the PCH file will be used instead. So rather than
precompiling precompiled.hpp and having source files include it, you
would precompile each individual (chosen) header file and have that in
the special PCH include directory, and they would then be included 
where

a source file asked for the corresponding .hpp file.



I assume precompiling a header means precompiling it itself and any
header it includes, no? In that case, would precompiling each header
individually not cause us to implicitly precompile included headers
many times (e.g. globalDefinitions.hpp)?


I presume you'd have to figure out the order in which to do the 
precompilation to avoid that. The makefile aspects of this seems 
non-trivial compared to precompiled.hpp.
I see. I think this is something we might have in mind for perhaps some 
kind of future improvement. The gcc behavior you describe is not shared 
by any other compiler, as far as I know, which also makes this a bit of 
a special solution. Since additional speed gains on gcc seems not like 
our most pressing issue, I can't say I'm interested right now in making 
a special solution.


If I were to spend more time on PCH, I'd rather spend it on either 
Windows (which is the slowest build platform in our CI), or Solaris, 
which does not have precompiled headers at all. (I actually looked into 
solstudio PCH as part of this fix to see if the new set of headers would 
allow us to enable it. Unfortunately, it seems like they need a single 
PCH per directory of source code files (e.g. all files in share/logging 
can share PCH with each other, but they cannot share PCH with the files 
in share/utilities. As I understood the problem, at least; the error 
messages were unclear and undocumented.)


/Magnus



Cheers,
David


Thanks, Thomas


That would also remove the problem of things not compiling with PCH
disabled as a single set of includes would be used for both PCH and
non-PCH building.

I don't know if the other compilers support PCH in a similar manner 
to gcc.


Cheers,
David


/Magnus



David


   25 // Precompiled headers are turned off for Sun Studio,
May as well change to Solaris Studio if you're going to fix the 
typo :)

Thanks.
David

On 3/11/2018 7:06 PM, Magnus Ihse Bursie wrote:
The reasons for the current set of files included in 
precompiled.hpp is somewhat lost in the mists of history. 
However, it is clear that it is not optimal.


This patch replaces the current set with a new set, based on how 
often a header file is included in a C++ file. This selection 
contains all header files that are included by at least 130 C++ 
files. Testing has shown that this is around the optimal value 
-- include many more, and too many "innocent" files get hurt by 
unneeded work, leave out many more, and we miss out on 
optimization possibilities.


The same set turned out to work well for both clang and gcc. 
However, files named "*.inline.hpp" did hurt rather than help 
performance, so those have been left out. For visual studio, the 
same set was also optimal, as long as the inline files were 
included. Presumably, visual studio is better than gcc/clang on 
handling precompiled headers containing inlined code.


Here are some rough comparisons from our internal CI system, for 
building the target "hotspot" from scratch.


macosx-x64:
old: 00:05:00
new: 00:03:47

linux-x64:
old: 00:05:43
new: 00:04:51

windows-x64:
old: 00:05:18
new: 00:04:33

linux-aarch64:
old: 00:07:57
new: 00:03:48

Bug: https://bugs.openjdk.java.net/browse/JDK-8213339
WebRev: 
http://cr.openjdk.java.net/~ihse/JDK-8213339-update-precompiled-headers/webrev.01


/Magnus