Re: CFV: New Jigsaw Committer: George Triantafillou

[Alan Bateman]

On 06/10/2015 19:14, Christian Tornqvist wrote:

I hereby nominate George Triantafillou to Jigsaw Committer.

Would it be better to wait until George has 8 or more significant 
contributions (as per the guidelines)?


-Alan

CFV: New Jigsaw Committer: George Triantafillou

[Christian Tornqvist]

I hereby nominate George Triantafillou to Jigsaw Committer.

 

George has contributed new tests and made changes to the existing hotspot
tests for jigsaw in the

jigsaw/m3 forest, including access control tests.

 

Votes are due by: October 20, 2015, 7:00 PST. Only current Jigsaw

Committers [1] are eligible to vote on this nomination.  Votes must be cast

in the open by replying to this mailing list.

 

For Lazy Consensus voting instructions, see [2].

 

Thanks,

Christian

 

[1] http://openjdk.java.net/census

[2] http://openjdk.java.net/projects/#committer-vote

 

George's jigsaw changesets:

 

http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/153611345cb7

http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/f7263fff0dcb

http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/6d159264d564

http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/28f1ee7176fa

http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/57a3057e9129

http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/10663d8aae2e

http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/11e1f3a521b9

http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/58eb7d6b0ab5

http://hg.openjdk.java.net/jigsaw/jake/hotspot/rev/e37c6f616b16

Re: Fixes in the jlink tool

[Andrei Eremeev]

 Resending. I am covered by the OCA; so I can contribute to OpenJDK.

Best regards,
Andrei Eremeev

>Tuesday, October  6, 2015 12:40 AM +03:00 from Andrei Eremeev 
>:
>
>Hi Jigsaw team,
>
>Please review fixes in jlink for the following issues:
>https://bugs.openjdk.java.net/browse/JDK-8135170
>https://bugs.openjdk.java.net/browse/JDK-8135079
>https://bugs.openjdk.java.net/browse/JDK-8135079
>
>Webrev:  http://cr.openjdk.java.net/~aeremeev/8135170.00/
>
>Best regards,
>Andrei Eremeev

RE: RFR: JDK-8138960 - Module version is checked incorrectly in libjimage

[Iris Clark]

Hi, Jim.

>> http://cr.openjdk.java.net/~jlaskey/8138960/webrev/index.html

These changes look good to me too.  I assume that you'll be pushing this to 
verona/stage?

Thanks,
iris

-Original Message-
From: Mandy Chung 
Sent: Tuesday, October 06, 2015 7:26 AM
To: Alan Bateman
Cc: jigsaw-dev
Subject: Re: RFR: JDK-8138960 - Module version is checked incorrectly in 
libjimage


> On Oct 6, 2015, at 6:28 AM, Alan Bateman  wrote:
> 
> 
> On 06/10/2015 14:22, Jim Laskey (Oracle) wrote:
>> Verona pit failing do to NEXT_VERSION +1 .  libjimage code assumed that the 
>> module version would be constant until some later date.
>> 
>> http://cr.openjdk.java.net/~jlaskey/8138960/webrev/index.html 
>> 
>> 
>> JDK-8138960 
>> 
>> Follow on changes at some later date coming with
>>  JDK-8138961  Include 
>> version information in jimage locations
>> JDK-8138962  Use 
>> correct module version from hotspot when using jimage find resource
> (Moving this to jigsaw-dev).
> 
> This change looks okay, it means the version is ignored in JDK 9 main line, 
> which make sense.

+1

We should check the version checking in jake and do the right checking (a 
separate issue).

Mandy

Re: jlink plugin experiment to improve jake startup time

[Mandy Chung]

> On Oct 6, 2015, at 4:38 AM, Claes Redestad  wrote:
> 
> Hi Mandy,
> 
> On 2015-10-05 23:13, Mandy Chung wrote:
>> I have been experimenting a jlink plugin to improve the module system 
>> startup time.   On a Quad-Core Intel Xeon E5 @3.7 GHz, 16G memory (Mac Pro) 
>> machine, the module system startup takes 110 ms and 75% of it is to 
>> reconstitute 64 module descriptors with 2388 packages altogether.
>> 
>> This high overhead includes initializing lambda forms (class loading, 
>> linking, initialization), parsing of module-info.class and compilation of 
>> many methods during startup.
>> 
>> The attached charts shows the summary of module system startup
>> 1) jake-b83 (as of Sep 29)
>> module bootstrap time110 ms
>> module descriptor reconstitution 82 ms
>> VM startup time is 3x of jdk9-b83 vm startup time (148ms vs 47 ms)
>> 
>> 2) No lambda at startup
>> module bootstrap time66 ms  (saved 44 ms compared to #1)
>> module descriptor reconstitution 33.6 ms (saved 48.4ms)
>> VM startup time is 2.2x of jdk9-b83 (105ms vs 47ms)
>> 
>> 
>> 3) jlink plugin to generate a .class file to build module descriptors at 
>> link time
>> module bootstrap time50.7 ms  (saved 15.3ms compared to #2)
>> module descriptor reconstitution 16.8 ms (saved 16.8ms)
>> VM startup time is 1.9x of jdk9-b83 (90ms vs 47ms)
> 
> Is it safe to assume that #3 include the changes in #2?

#2 and #3 are different changes.  ModuleDescriptor.Requires::modsValue is one 
use of lambda that is changed in both #2 and #3.  Other change of 
Stream::forEach in ModuleDescriptor in #2 are not in #3.

#3 has its own builder to create ModuleDescriptor.  Stream::forEach is used in 
3 places in ModuleDescriptor in the loop validating the strings passing in the 
Exports and Provides constructor for example that is skipped in this installed 
module descriptor reconstitution.   

#2 is mainly to measure the overhead due to lambda at startup to help 
understand what attributes to 82ms. 

> 
>> 
>> For #3, it parses module-info.class at link time and validates all names.  
>> It generates a .class file to call a custom Builder to create 
>> ModuleDescriptor for the installed modules.  At runtime, the generated class 
>> will construct the Builder and ModuleDescriptor objects will skip name 
>> validation, no defensive copy of the input set/map and skip reading and 
>> parsing of module-info.class (this is done by a special module descriptor 
>> builder class that doesn’t use lambda.  This special builder is only used by 
>> installed modules).
> 
> Since you're generating and loading classes that would otherwise not be 
> loaded, have you done any footprint measurements?
> 

The charts show the number of loaded classes and heap usage for these 3 runs:
   http://cr.openjdk.java.net/~mchung/jigsaw/jake-startup.20151005.pdf

692 classes are loaded in #1 and 503 classes are loaded in #3.  Heap usage 
after GC in #1 and #3 is 1443 KB 1380 KB respectively.
  
> Alternatively, wouldn't it be possible for the plugin to modify the 
> module-info.class directly?
> 

Yes it’s possible but I don’t see how this can improve the startup time.

>> 
>> It saves 15.3 ms (23% of the module system bootstrap time in #2).   The 
>> downside of this optimization is a little harder to make change and diagnose 
>> (this plugin implementation is straight forward though).   There may be 
>> other small optimization to explore that could be done at jlink time (e.g. 
>> BuiltinClassLoader maintains a package to module map that can be constructed 
>> with a specific size to avoid map resizing).
>> 
>> What’s your thought/opinion to integrate this jlink plugin into jake?
> 
> In general I think improving startup by staving off the first use of lambda 
> isn't very helpful except for trivial applications (which we often 
> over-emphasize when testing startup), while moving module validation to link 
> time seems more like a real gain. It's a bit hard to tell if it's worth it 
> without having seen a patch for the prototype, though.

I hope that LambdaForm initialization can be speed up so that it won’t incur 
significant overhead at startup time.  As I explain above,  
ModuleDescriptor.Requires::modsValue is the only method changed from 
Stream::forEach to foreach loop.

Mandy

Re: jlink plugin experiment to improve jake startup time

[Mandy Chung]

See http://cr.openjdk.java.net/~mchung/jigsaw/jake-startup.20151005.pdf

Mandy
p.s. In fact the attachment got trimmed.

> On Oct 6, 2015, at 7:31 AM, Mandy Chung  wrote:
> 
> I realize I missed the attachment after seeing Claes’ question.  This 
> includes the heap usage after GC.
> 
> 
> 
> 
> 
> 
>> On Oct 5, 2015, at 2:13 PM, Mandy Chung  wrote:
>> 
>> I have been experimenting a jlink plugin to improve the module system 
>> startup time.   On a Quad-Core Intel Xeon E5 @3.7 GHz, 16G memory (Mac Pro) 
>> machine, the module system startup takes 110 ms and 75% of it is to 
>> reconstitute 64 module descriptors with 2388 packages altogether.
>> 
>> This high overhead includes initializing lambda forms (class loading, 
>> linking, initialization), parsing of module-info.class and compilation of 
>> many methods during startup.
>> 
>> The attached charts shows the summary of module system startup
>> 1) jake-b83 (as of Sep 29)
>>   module bootstrap time  110 ms
>>   module descriptor reconstitution 82 ms
>>   VM startup time is 3x of jdk9-b83 vm startup time (148ms vs 47 ms)
>> 
>> 2) No lambda at startup
>>   module bootstrap time66 ms  (saved 44 ms compared to #1)
>>   module descriptor reconstitution 33.6 ms (saved 48.4ms)
>>   VM startup time is 2.2x of jdk9-b83 (105ms vs 47ms)
>> 
>> 
>> 3) jlink plugin to generate a .class file to build module descriptors at 
>> link time
>>   module bootstrap time50.7 ms  (saved 15.3ms compared to #2)
>>   module descriptor reconstitution 16.8 ms (saved 16.8ms)
>>   VM startup time is 1.9x of jdk9-b83 (90ms vs 47ms)
>> 
>> For #3, it parses module-info.class at link time and validates all names.  
>> It generates a .class file to call a custom Builder to create 
>> ModuleDescriptor for the installed modules.  At runtime, the generated class 
>> will construct the Builder and ModuleDescriptor objects will skip name 
>> validation, no defensive copy of the input set/map and skip reading and 
>> parsing of module-info.class (this is done by a special module descriptor 
>> builder class that doesn’t use lambda.  This special builder is only used by 
>> installed modules).
>> 
>> It saves 15.3 ms (23% of the module system bootstrap time in #2).   The 
>> downside of this optimization is a little harder to make change and diagnose 
>> (this plugin implementation is straight forward though).   There may be 
>> other small optimization to explore that could be done at jlink time (e.g. 
>> BuiltinClassLoader maintains a package to module map that can be constructed 
>> with a specific size to avoid map resizing).
>> 
>> What’s your thought/opinion to integrate this jlink plugin into jake?
>> 
>> Mandy
>> 
> 

Re: RFR: JDK-8138960 - Module version is checked incorrectly in libjimage

[Mandy Chung]

> On Oct 6, 2015, at 6:28 AM, Alan Bateman  wrote:
> 
> 
> On 06/10/2015 14:22, Jim Laskey (Oracle) wrote:
>> Verona pit failing do to NEXT_VERSION +1 .  libjimage code assumed that the 
>> module version would be constant until some later date.
>> 
>> http://cr.openjdk.java.net/~jlaskey/8138960/webrev/index.html 
>> 
>> 
>> JDK-8138960 
>> 
>> Follow on changes at some later date coming with
>>  JDK-8138961  Include 
>> version information in jimage locations
>> JDK-8138962  Use 
>> correct module version from hotspot when using jimage find resource
> (Moving this to jigsaw-dev).
> 
> This change looks okay, it means the version is ignored in JDK 9 main line, 
> which make sense.

+1

We should check the version checking in jake and do the right checking (a 
separate issue).

Mandy

Re: jlink plugin experiment to improve jake startup time

[Mandy Chung]

I realize I missed the attachment after seeing Claes’ question.  This includes 
the heap usage after GC.






> On Oct 5, 2015, at 2:13 PM, Mandy Chung  wrote:
> 
> I have been experimenting a jlink plugin to improve the module system startup 
> time.   On a Quad-Core Intel Xeon E5 @3.7 GHz, 16G memory (Mac Pro) machine, 
> the module system startup takes 110 ms and 75% of it is to reconstitute 64 
> module descriptors with 2388 packages altogether.
> 
> This high overhead includes initializing lambda forms (class loading, 
> linking, initialization), parsing of module-info.class and compilation of 
> many methods during startup.
> 
> The attached charts shows the summary of module system startup
> 1) jake-b83 (as of Sep 29)
>module bootstrap time  110 ms
>module descriptor reconstitution 82 ms
>VM startup time is 3x of jdk9-b83 vm startup time (148ms vs 47 ms)
> 
> 2) No lambda at startup
>module bootstrap time66 ms  (saved 44 ms compared to #1)
>module descriptor reconstitution 33.6 ms (saved 48.4ms)
>VM startup time is 2.2x of jdk9-b83 (105ms vs 47ms)
> 
> 
> 3) jlink plugin to generate a .class file to build module descriptors at link 
> time
>module bootstrap time50.7 ms  (saved 15.3ms compared to #2)
>module descriptor reconstitution 16.8 ms (saved 16.8ms)
>VM startup time is 1.9x of jdk9-b83 (90ms vs 47ms)
> 
> For #3, it parses module-info.class at link time and validates all names.  It 
> generates a .class file to call a custom Builder to create ModuleDescriptor 
> for the installed modules.  At runtime, the generated class will construct 
> the Builder and ModuleDescriptor objects will skip name validation, no 
> defensive copy of the input set/map and skip reading and parsing of 
> module-info.class (this is done by a special module descriptor builder class 
> that doesn’t use lambda.  This special builder is only used by installed 
> modules).
> 
> It saves 15.3 ms (23% of the module system bootstrap time in #2).   The 
> downside of this optimization is a little harder to make change and diagnose 
> (this plugin implementation is straight forward though).   There may be other 
> small optimization to explore that could be done at jlink time (e.g. 
> BuiltinClassLoader maintains a package to module map that can be constructed 
> with a specific size to avoid map resizing).
> 
> What’s your thought/opinion to integrate this jlink plugin into jake?
> 
> Mandy
> 

Re: RFR: JDK-8138960 - Module version is checked incorrectly in libjimage

[Alan Bateman]

On 06/10/2015 14:22, Jim Laskey (Oracle) wrote:

Verona pit failing do to NEXT_VERSION +1 .  libjimage code assumed that the 
module version would be constant until some later date.

http://cr.openjdk.java.net/~jlaskey/8138960/webrev/index.html 


JDK-8138960 

Follow on changes at some later date coming with
JDK-8138961  Include 
version information in jimage locations
 JDK-8138962  Use 
correct module version from hotspot when using jimage find resource

(Moving this to jigsaw-dev).

This change looks okay, it means the version is ignored in JDK 9 main 
line, which make sense.


-Alan.

Re: jlink plugin experiment to improve jake startup time

[Alan Bateman]

On 06/10/2015 12:38, Claes Redestad wrote:


Alternatively, wouldn't it be possible for the plugin to modify the 
module-info.class directly?
Generating the classes at link time is about eliminating the need to 
parse the module-info and reconstitute the module descriptors via the 
builder API. This is the 82ms that is dominating the initialization, a 
lot of it lambda related, some of it because we are starting out in the 
interpreter and of course triggering class loading and compilation.



:

In general I think improving startup by staving off the first use of 
lambda isn't very helpful except for trivial applications (which we 
often over-emphasize when testing startup), while moving module 
validation to link time seems more like a real gain.
I would hope that over time that the LF initialization can be improved 
as the cost of first usage is currently very significant. With a link 
step then we have the opportunity to generate lambda proxy classes at 
link time so that could potentially help too.


That said, it is also important to not regress startup. If it's 50 or 
100ms more than JDK 8 then someone running a short lived tool will 
notice and complain.


-Alan.

Re: jlink plugin experiment to improve jake startup time

[Claes Redestad]

Hi Mandy,

On 2015-10-05 23:13, Mandy Chung wrote:

I have been experimenting a jlink plugin to improve the module system startup 
time.   On a Quad-Core Intel Xeon E5 @3.7 GHz, 16G memory (Mac Pro) machine, 
the module system startup takes 110 ms and 75% of it is to reconstitute 64 
module descriptors with 2388 packages altogether.

This high overhead includes initializing lambda forms (class loading, linking, 
initialization), parsing of module-info.class and compilation of many methods 
during startup.

The attached charts shows the summary of module system startup
1) jake-b83 (as of Sep 29)
 module bootstrap time  110 ms
 module descriptor reconstitution 82 ms
 VM startup time is 3x of jdk9-b83 vm startup time (148ms vs 47 ms)

2) No lambda at startup
 module bootstrap time66 ms  (saved 44 ms compared to #1)
 module descriptor reconstitution 33.6 ms (saved 48.4ms)
 VM startup time is 2.2x of jdk9-b83 (105ms vs 47ms)


3) jlink plugin to generate a .class file to build module descriptors at link 
time
 module bootstrap time50.7 ms  (saved 15.3ms compared to #2)
 module descriptor reconstitution 16.8 ms (saved 16.8ms)
 VM startup time is 1.9x of jdk9-b83 (90ms vs 47ms)


Is it safe to assume that #3 include the changes in #2?



For #3, it parses module-info.class at link time and validates all names.  It 
generates a .class file to call a custom Builder to create ModuleDescriptor for 
the installed modules.  At runtime, the generated class will construct the 
Builder and ModuleDescriptor objects will skip name validation, no defensive 
copy of the input set/map and skip reading and parsing of module-info.class 
(this is done by a special module descriptor builder class that doesn’t use 
lambda.  This special builder is only used by installed modules).


Since you're generating and loading classes that would otherwise not be 
loaded, have you done any footprint measurements?


Alternatively, wouldn't it be possible for the plugin to modify the 
module-info.class directly?




It saves 15.3 ms (23% of the module system bootstrap time in #2).   The 
downside of this optimization is a little harder to make change and diagnose 
(this plugin implementation is straight forward though).   There may be other 
small optimization to explore that could be done at jlink time (e.g. 
BuiltinClassLoader maintains a package to module map that can be constructed 
with a specific size to avoid map resizing).

What’s your thought/opinion to integrate this jlink plugin into jake?


In general I think improving startup by staving off the first use of 
lambda isn't very helpful except for trivial applications (which we 
often over-emphasize when testing startup), while moving module 
validation to link time seems more like a real gain. It's a bit hard to 
tell if it's worth it without having seen a patch for the prototype, 
though.


/Claes



Mandy

Re: Review request 8132297: JDK9 build.tools.module.ImageBuilder does not filter out .class.bc files

[Chris Hegarty]

On 5 Oct 2015, at 23:32, Mandy Chung  wrote:

> JDK 9 currently use a build tool (ImageBuilder) to create the JDK modular 
> image.  A in-house tool doing static analysis generates additional files that 
> should be excluded in the image.
> 
> jlink and jmod tool in jake build has the option to exclude specific files.
> 
> This patch proposes to fix the interim ImageBuilder to exclude files with .bc 
> suffix as a temporary workaround.
> 
> Mandy
> 
> diff --git a/make/src/classes/build/tools/module/ModuleArchive.java 
> b/make/src/classes/build/tools/module/ModuleArchive.java
> --- a/make/src/classes/build/tools/module/ModuleArchive.java
> +++ b/make/src/classes/build/tools/module/ModuleArchive.java
> @@ -105,6 +105,7 @@
> entries.addAll(stream
> .filter(p -> !Files.isDirectory(p)
> && 
> !classes.relativize(p).toString().startsWith("_the.")
> +&& 
> !classes.relativize(p).toString().endsWith(".bc")
> && 
> !classes.relativize(p).toString().equals("javac_state"))
> .sorted()
> .map(p -> toEntry(p, classes, 
> EntryType.CLASS_OR_RESOURCE))

Thanks Mandy, this looks good to me.

-Chris.