Re: Current Project Status

[Remi Forax]

- Original Message -
> From: "Alan Bateman" 
> To: "Julian Waters" , "jigsaw-dev" 
> 
> Sent: Sunday, April 3, 2022 4:13:53 PM
> Subject: Re: Current Project Status

> On 03/04/2022 09:26, Julian Waters wrote:
>> Out of curiosity, is the Jigsaw project still active in the background
>> today, seeing that this mailing list is still up?
>>
> The features developed by this project were integrated in JDK 9. The
> mailing list lives on, it is used periodically for questions, bug
> reports, and related discussion.

The migration Java 8 -> 9, now Java 8 -> 11 has been slow,
even if major frameworks/libraries are now Java 11 compatible, there is still a 
long tail of applications in that process,
so i think it's a good idea to keep this list open.

> 
> -Alan

Rémi

Re: Annotation Dependencies and Requires Static Transitive

[Remi Forax]

- Mail original -
> De: "Anand Beh" 
> À: "jigsaw-dev" 
> Envoyé: Jeudi 3 Juin 2021 22:10:11
> Objet: Annotation Dependencies and Requires Static Transitive

> Hello,
> 
> The cache library Caffeine recently added a full module descriptor. It
> has no runtime dependencies, but it depends on metadata annotations
> from checker-qual and errorprone, for example @NotNull and
> @CanIgnoreReturnValue. The module looks like this:
> module com.github.benmanes.caffeine {
>  exports com.github.benmanes.caffeine.cache;
>  exports com.github.benmanes.caffeine.cache.stats;
> 
>  requires static transitive com.google.errorprone.annotations;
>  requires static transitive org.checkerframework.checker.qual;
> }
> 
> The annotations are not required at runtime, hence static. They're
> visibly placed on public methods and return types, so that API clients
> can benefit from them for the purposes of annotation-based null
> analysis, kotlin interop, etc. As the annotations are part of the API,
> they're marked transitive.
> 
> However, the "transitive" aspect imposes some requirements on users. I
> am wondering if there is a more correct way to declare these
> annotation dependencies than static transitive.
> 
> One user would like to avoid the presence of these annotations at
> compile-time. For reference, here's the relevant discussion:
> https://github.com/ben-manes/caffeine/issues/535
> 
> I'm not a maintainer of caffeine, though I was involved in its modularization.

Hi,
i've used just "requires static" without "transitive" for non null/nullable 
annotations in the past.

Thos annotations are consumed by tools, i don't think transitive is necessary.

> 
> Regards,
> Anand

regards,
Rémi

Re: Add reads to open module

[Remi Forax]

- Mail original -
> De: "Christian Beikov" 
> À: "jigsaw-dev" 
> Envoyé: Lundi 17 Mai 2021 17:43:15
> Objet: Fwd: Add reads to open module

> Hi,
> 
> I have a use case where I generate some code and that code refers to
> classes of some module B.
> 
> When the module A (an open module), into which I define that class, has
> no read-edges to the module B, this fails saying that A does not read B.
> 
> That's fine and totally understandable, so I tried to add the following
> code:
> 
> moduleA.addReads("moduleB.pkg", moduleB)
> 
> but this is not allowed as `addReads` only works if `this == callerModule`.
> 
> My workaround is to define a class into module A that can then call this
> method.
> 
> Is there a reason to this limitation? I mean the module is open anyway,
> so anyone can just define a class into it, so why not also allow calling
> addReads?

It's basic security [1], you can not see something you do not ask for.
By example, if your module as a security issue, you don't want an attacker to 
be able to see all other open modules from your module.

> 
> 
> Regards,
> 
> Christian

regards,
Rémi

[1] https://en.wikipedia.org/wiki/Defense_in_depth_(computing)

Re: Why service provider method is called "provider", but not "provide"?

[Remi Forax]

- Mail original -
> De: "Alex Sviridov" 
> À: "jigsaw-dev" 
> Envoyé: Vendredi 20 Novembre 2020 12:48:21
> Objet: Why service provider method is called "provider", but not "provide"?

> Hello all,

Hi Alex,

> 
> According to this tutorial
> https://www.logicbig.com/tutorials/core-java-tutorial/modules/service-provider-method.html
> we can declare in module-info provider class with "provider" method. For
> example,
> 
> ..
> public class TheServiceProvider {
>   public static AService provider() {
>   return new AServiceImpl();
>   }
> }
> 
> The only thing what I don’t understand is why this method is called 
> "provider".
> The method doesn’t return
> provider, method returns an instance of the service, so, as I understand, the
> method must be named
> "provide". And, as I understand, provider is usually the class, that has
> "provide" method.

The spec is in ServiceLoader
https://docs.oracle.com/en/java/javase/15/docs/api/java.base/java/util/ServiceLoader.html

yes, the name is not very logical.

> 
> Of course that’s not very important, but it is JDK, so it was developed by 
> very
> experienced java developer
> who should know java naming convention and I want to understand why this did 
> so.
> Could anyone explain?

I believe it's due to the fact that most services used by the JDK are 
themselves factories.
And naming is hard.

> 
> 
> --
> Best regards, Alex Orlov

regards,
Rémi

Re: Illegal reflection access denial in which future release

[Remi Forax]

- Mail original -
> De: "Claes Redestad" 
> À: "S A" , "jigsaw-dev" 
> 
> Cc: "Andrey Loskutov" 
> Envoyé: Lundi 8 Juin 2020 16:09:28
> Objet: Re: Illegal reflection access denial in which future release

> Hi Simeon,
> 
> it's not documented or discussed much anywhere because it's as-of-yet
> undecided.
> 
> *Expectation* is that there'll be a JEP proposed to make
> --illegal-access=deny the *default* in JDK 16. I expect the ability to
> explicitly permit illegal accesses with --illegal-access=permit will
> stick around for a good while longer, though.

As a member of the expert group on JPMS, i think we should move to deny by 
default for 16.
A lot of people are now migrating their applications to 11, i think it's 
cheaper to do the migration to a world with --illegal-access=deny than to do 
first a migration to 11 and then a migration to 17.

> 
> Best regards
> /Claes

regards,
Rémi

> 
> On 2020-06-08 15:40, S A wrote:
>> Hi all,
>> 
>> TL;DR is a future JDK release known, when illegal reflection access is no
>> longer possible? If not, are there estimations / discussions / FAQs /
>> documentation?
>> 
>> The product I work on (Eclipse+xtext IDE + backend) has nearly completed
>> the move to OpenJDK 11 (from OpenJDK 8) and we are working on supposedly
>> not pressing issues. One of those issues is illegal access via reflection.
>> 
>> We would obviously like to be prepared and so are estimating effort and
>> time constraints of fixing illegal reflection access. As this includes
>> multiple libraries, knowing *when* we need to finish our efforts is quite
>> important.
>> 
>> Sorry if this has been asked before (I imagine yes), but searching for the
>> topic yields lots of how to work around or fix the issue. Thank you in
>> advance.
>> 
>> Best regards,
>> Simeon

Re: Is it possible to add classes to JPMS module dynamically?

[Remi Forax]

- Mail original -
> De: "Alex Sviridov" 
> À: "jigsaw-dev" 
> Envoyé: Jeudi 4 Juin 2020 14:34:08
> Objet: Is it possible to add classes to JPMS module dynamically?

> Hi all,

Hi Alex,

> 
> Let's suppose we have a JPMS module  moduleA with the following  module-info :
> module moduleA {
>exports modulea.generated;
> }
> Now, using javassist/byte-buddy we generate new  modulea.generated.Foo class.
> Could anyone say if we can add this class to  moduleA dynamically (at runtime)
> so as JPMS work with it as it works with module existing classes ?

It will be added automatically, a module only contains packages (exported or 
not) so any classes in those packages (generated at runtime or not) are part of 
the module automatically.

> 
> 
> --
> Alex Orlov

cheers,
Rémi

Re: ModuleMainClass attribute via module-info.java

[Remi Forax]

- Mail original -
> De: "Florian Weimer" 
> À: "jigsaw-dev" 
> Envoyé: Dimanche 31 Mai 2020 19:31:48
> Objet: ModuleMainClass attribute via module-info.java

> Is it possible to instruct javac to generate a ModuleMainClass
> attribute via a module declaration?  I can't find the appropriate
> syntax in the JLS.  If there is no way to get javac to generate the
> attribute, what is the reason for that?

Hi Florian,
the design of JPMS separates what is useful for Java described by the 
module-info.java (required modules, exported or open packages, etc) and what is 
metadata for distribution (module version, module main, modules hashes, etc) 
stored as supplementary attributes (or optional fields of the Module attribute) 
in the module-info.class.

Changing the main class or the version of a module do not require a 
recompilation, so those information are not handled by javac but injected into 
the module-info.class by jar, because those information are useful only when 
you want to create a jar in order to distribute it.

We came to that design to be able to support already existing ways of packaging 
the metadata associated with a jar file so a jar can be at the same time a 
module for JPMS and also a module for Jboss module or an OSGI bundle, etc. As 
an example of metadata fusion, you can use a tool like bnd [1], a tool 
developed originally to package OSGI bundles, if you want to use a Java 
annotation to specify the main class. 

regards,
Rémi

[1] https://bnd.bndtools.org/chapters/330-jpms.html

Re: jlink 14.0.1 with 100 modules fail

[Remi Forax]

- Mail original -
> De: "Christian Stein" 
> À: "jigsaw-dev" 
> Envoyé: Samedi 30 Mai 2020 18:40:53
> Objet: jlink 14.0.1 with 100 modules fail

> Hi,
> 
> the modular [bach-demo-99] I generated to check my build tool
> against, failed to create a custom runtime image using jlink
> via itsToolProvider-based service entry-point.
> 
> The project consists of 100 module descriptors. No other Java
> source files (for classes and interfaces) are part of that demo.
> Each descriptor ball${i}n declares a "requires ball${j}n;" with j
> set to a number from 0 to i-1 ... in words, all lower modules.
> 
> javac compiles this project with problems.
> 
> Arguments passed to jlink (as an array of string) reads:
> 
> jlink
>  --add-modules ball27n,ball43n,ball88n,[...],ball77n,ball93n
>  --module-path .bach\workspace\modules
>  --output .bach\workspace\image
>  --compress 2
>  --no-header-files
>  --no-man-pages
> 
> The error message reads:
> 
> jdk.internal.org.objectweb.asm.MethodTooLargeException: Method too large:
> jdk/internal/module/SystemModules$all.moduleDescriptors
> ()[Ljava/lang/module/ModuleDescriptor;
> 
> Is there a limitation of supported modules that can be linked?

It's a bug in the implementation,
jlink pre-resolve the module graph and encodes it in a generated classfile.
The fact that that classfile can have a method too big was overlooked. 

> Is the module graph of this conceived project too far out of scope?
> 
> Shall I open an issue at JBS?


yes !

> 
> Cheers,
> Christian

regards,
Rémi

> 
> [ bach-demo-99]:
> https://github.com/sormuras/bach/blob/master/src/bach/bach-demo-99.jsh

Re: replacements for Unsafe

[Remi Forax]

Hi Jochen,
Per the Java spec, calling an empty static method of the inner class works.

Rémi

- Mail original -
> De: "Jochen Theodorou" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 29 Avril 2020 13:23:47
> Objet: replacements for Unsafe

> Hi,
> 
> when jigsaw started there was a lot of talk about providing alternatives
> to what Unsafe offers. Today I am facing a problem I cannot solve
> anymore it seems with this.
> 
> My problem is that I need to ensure the static block of an inner class
> has been executed before I do something, because the static block
> contains setup data the class will need and influence what I do with the
> class. Of course this might be not the best choice, I am just trying to
> keep certain functionality.
> 
> In the past I would have used Unsafe#ensureClassInitialized, but this
> method has not found a replacement yet if I read JDK-8235521 :
> Replacement API for Unsafe::ensureClassInitialized correctly.
> 
> Also because of Unsafe#getUnsafe, checking for the platform loader for
> the caller (which is not the case for me) I cannot get the unsafe object
> itself "properly".
> 
> Now I am wondering what alternatives exist. Are there any?
> 
> bye Jochen

Re: excluding transitive module

[Remi Forax]

- Mail original -
> De: "Jochen Theodorou" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 15 Avril 2020 08:55:00
> Objet: Re: excluding transitive module

> On 14.04.20 20:20, Remi Forax wrote:
>> Hi Jochen,
>> JPMS has no notion of of API and implementation of the same jar. It's a 
>> concept
>> of your build tool and not something JPMS knows.
>>
>> The notion of compilation dependencies and runtime dependencies is not a 
>> concept
>> of JPMS but a concept of your build tools.
>>
>> In term of JPMS, if you want to be able to use a jar at compile time and 
>> another
>> one at runtime,
>> both should have the same module name, in fact they should have exactly the 
>> same
>> module descriptor (the same module-info).
>>
>> So in term of dependency,
>> - for,Maven/Gradle, the compile dependency will use SharedAPI and the runtime
>> dependency SharedApiImpl
>> - for JPMS, both have the same module-info and if you want to depend on that
>> module, just require it.
>>
>>  From the Maven Central POV, obviously, you have two jars so they can not 
>> have
>>  the same arfifact ID (coordinate), but they contains the same
>>  module-info.class.
> 
> If the case is api and implementation and the same module name, then
> using the build tool to exclude the api dependency works.
> 
> If the module name is not the same I have not found a solution. It is a
> problem here because of shared packages.

as Alex said, this should be fixed by the maintainers of the jars, not you.

> 
> I found this being a problem btw with a lot of jee libraries.
> 
> Of course I can make my project require whatever, but if it is libraries
> I do not have under control things get a bit difficult without changing
> them. But changing them is not an action really well supported by any
> build tool I know.

For the Jakarta EE libraries, the main issue is that there are not many jars 
that has a module-info, so the default strategy to name the automatic module 
with the jar name fail here because an api an its implementations should have 
the same name.
Injecting the right module-info until your pull request is accepted is an 
escape hatch.

> 
> bye Jochen

Rémi

Re: excluding transitive module

[Remi Forax]

Hi Jochen,
JPMS has no notion of of API and implementation of the same jar. It's a concept 
of your build tool and not something JPMS knows.

The notion of compilation dependencies and runtime dependencies is not a 
concept of JPMS but a concept of your build tools.

In term of JPMS, if you want to be able to use a jar at compile time and 
another one at runtime,
both should have the same module name, in fact they should have exactly the 
same module descriptor (the same module-info). 

So in term of dependency,
- for,Maven/Gradle, the compile dependency will use SharedAPI and the runtime 
dependency SharedApiImpl
- for JPMS, both have the same module-info and if you want to depend on that 
module, just require it.

>From the Maven Central POV, obviously, you have two jars so they can not have 
>the same arfifact ID (coordinate), but they contains the same 
>module-info.class.

regards,
Rémi

- Mail original -
> De: "Jochen Theodorou" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 14 Avril 2020 10:24:34
> Objet: excluding transitive module

> Hi all,
> 
> I am wondering if there is a solution purely in the module-info for this:
> 
> * Project requires Library1 and Library2
> * SomeLibrary requires SharedApi
> * OtherLibrary requires SharedApiImpl
> 
> The problem is, that it will not compile because SharedApi and
> SharedApiImpl implement the same packages but not the same logic. One is
> just an API, while the other is an implementation of the API.
> 
> Scenario 1:
> 
> What would seem to be logic to me is to exclude SharedApi, but does
> SomeLibrary then need its requires to be changed to SharedApiImpl? How
> would I do that if I cannot change SomeLibrary?
> 
> Scenario 2:
> 
> Let us assume I could change the module-info of SomeLibrary. How would I
> express that Shared API is compilation only? Is that what "requires
> SomeLibrary for compilation" would be for?
> 
> bye Jochen

Re: RFE simplify usage of patched module [by Robert Scholte, from jdk-dev]

[Remi Forax]

- Mail original -
> De: "Alex Buckley" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 18 Février 2020 18:42:10
> Objet: Re: RFE simplify usage of patched module [by Robert Scholte, from 
> jdk-dev]

> On 2/14/2020 9:34 PM, Christian Stein wrote:
>> Assuming `--patch-module-descriptor` was already available, the
>> org.astro test story would read:
>>
>> - org.astro/main/module-info.java:
>>
>> module org.astro {
>>exports org.astro;
>>requires java.logging;
>> }
>>
>> - org.astro/test/module-info.java:
>>
>> open /*test*/ module org.astro /*extends main module org.astro*/ {
>>requires java.sql; // system-provided module needed by test code
>>requires org.junit.jupiter; // framework reflecting entry-points
>>requires org.assertj.core; // ...and more test-related modules
>> }
>>
>> A build tool now easily may detect that `--patch-module-descriptor`
>> is required at test compile and run time. It could infer it from the
>> same module name or simply by seeing a main and a test
>> `module-info.java` compilation unit within the sources of the
>> org.astro Maven-/Gradle-/pro-/Bach-/any-build-tool-"module".
>>
>> Build tools could pass the following argument to javac/java
>> while being in their test-scope tasks. Of course, implying that
>> org.astro/test/module-info.java is the current "primary" module
>> descriptor:
>>
>> --patch-module-descriptor org.astro=org.astro/main/module-info.java
>>
>> That's it.
>>
>> The primary test-relevant modular directives are enriched by
>> those of the main module as declared in org.astro/main/module-info.java
>> resulting effectively in a synthetic in-module test module descriptor that
>> resembles the inter-module test descriptor presented above:
>>
>> - SYNTHETIC test module for org.astro
>>
>> open /*test*/ module org.astro /*extends main module org.astro*/ {
>>requires java.sql; // system-provided module needed by test code
>>requires org.junit.jupiter; // framework reflecting entry-points
>>requires org.assertj.core; // ...and more test-related modules
>>//
>>exports org.astro; // patched from org.astro/main/module-info.java
>>requires java.logging; // patched from org.astro/main/module-info.java
>> }
>
> Good description of how the main module
> (org.astro/main/module-info.java) and the test module
> (org.astro/test/module-info.java) interact to form a test-scoped module.
> If a build tool has detected the presence of the test module, then the
> build tool can use javax.lang.model.element.ModuleElement to inspect the
> test module's declaration and infer what it requires, whether it's open,
> etc. Then, the build tool can enrich the main module by passing
> --add-modules, --add-reads, and --add-opens to javac/java while in a
> test scope.

Hi Alex,
no you can't because you can not reconstruct a module-info from the flags 
--add-modules, --add-reads, etc.

Those flags has been created to patch (mostly the JDK) module metadata so an 
application using the classpath can open/patch the JDK modules metadata enough 
to be able to run.
The semantics of a module-info is richer than the semantics of the flags, by 
example, there is no way to define a transitive dependency or to define a 
service.
I believe that roughly half of the semantics of a module-info has no equivalent 
in term of flags (and for good reason, those flags have not been created to 
describe a module-info).

> No need for --patch-module-descriptor in the JDK.

We need a way to express that the module graph when testing is different from 
the one when running in production.
--path-module-descriptor is a way to get that.

>
> Alex

Rémi

Re: Lookup objects and setAccessible

[Remi Forax]

see 
https://docs.oracle.com/en/java/javase/13/docs/api/java.base/java/lang/invoke/MethodHandles.html#privateLookupIn(java.lang.Class,java.lang.invoke.MethodHandles.Lookup)

Rémi

- Mail original -
> De: "Kasper Nielsen" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 8 Janvier 2020 12:14:30
> Objet: Lookup objects and setAccessible

> Hi,
> 
> I'm trying to bridge some old code using java.lang.reflect and some new code
> that uses Lookup objects. And I'm wondering if there is some way to make
> a member accessible using a Lookup object? Or if open packages/modules via
> module-info.java + member.setAccessible(true) is the only way?
> 
> Thanks
>   Kasper

Re: Could we get module-private visibility?

[Remi Forax]

- Mail original -
> De: "Mike Hearn" 
> À: "Michael Rasmussen" 
> Cc: "jigsaw-dev" 
> Envoyé: Mercredi 25 Septembre 2019 11:30:00
> Objet: Re: Could we get module-private visibility?

> Yes, I know. I'm talking about module-private inside exported packages.
> Otherwise it can be quite difficult to allow exported and non-exported
> packages to work closely together without weakening visibility, and at any
> rate, whilst the compatibility considerations are obvious, having a
> visibility level of "public" that's actually private to a scope is not a
> design anyone would pick if they had a blank slate I imagine!

at the same time, a public method is not really public if the class is not 
public too,
so it's like russian dolls all the way down.

Rémi

Re: Qualified exports to an unknown module

[Remi Forax]

- Mail original -
> De: "Alan Bateman" 
> À: "jigsaw-dev" 
> Envoyé: Samedi 8 Décembre 2018 09:44:05
> Objet: Re: Qualified exports to an unknown module

> On 07/12/2018 18:59, Alex Buckley wrote:
>>
>> The decision was: Qualified exports/opens to a non-observable module
>> is NOT an error (see
>> https://docs.oracle.com/javase/specs/jls/se11/html/jls-7.html#jls-7.7.2-310).
>> javac will give an on-by-default lint warning instead.
>>
>> The StackOverflow question says "Because I configured the compiler to
>> treat warnings as errors (-Werror), the build fails." so the
>> questioner should look at disabling the lint warning (see
>> https://docs.oracle.com/en/java/javase/11/tools/javac.html#GUID-AEEC9F07-CB49-4E96-8BC7-BCC2C7F725C9).
> The scenario in the post seems to be a Maven multi-module project so an
> alternative to -Xlint:-module is to compile with --module-source-path so
> that module B is observable when compiling A. This should work:
>     javac --module-source-path project/"*"/src/main/java -d  -m A
> 
> although I don't know if the Maven compiler plugin can be configured to
> use this option/syntax.

Maven still can not do that, which is a bummer because compiling in 
multi-module mode is also faster when you starts to have quite a lot of classes.

> 
> -Alan

Rémi

Re: Separate logging for JPMS module/layer

[Remi Forax]

You can use the StackWalker
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/lang/StackWalker.html

regards,
Rémi

- Mail original -
> De: "Alex Sviridov" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 2 Octobre 2018 23:54:48
> Objet: Separate logging for JPMS module/layer

> Hi all,
> 
> Could anyone say how the following problem can be solved. I want to create
> separate
> log file for every JPMS module/layer. The problem is that many
> libraries/programs
> use LoggerFactory.getLogger(String className) so in getLogger I have only
> the name of the class as String, so I can't get module and layer.
> 
> If I had not String className, but Class klass then the problem would be 
> easily
> solved.
> As I understand I can't load class by name because it would require all 
> modules
> export
> their packages to logging framework that has no sense.
> 
> Are there any solutions for such problem?
> 
> 
> --
> Alex Sviridov

Re: Add posibility to add custom ModuleReaderFactory to ModuleFinder

[Remi Forax]

- Mail original -
> De: "Alan Bateman" 
> À: "Alex Sviridov" , "jigsaw-dev" 
> 
> Envoyé: Vendredi 28 Septembre 2018 15:51:56
> Objet: Re: Add posibility to add custom ModuleReaderFactory to ModuleFinder

> On 28/09/2018 13:10, Alex Sviridov wrote:
>> Hi Alan
>>
>> Thank you for your answer. But my main problem is not jars inside .war
>> - this is a so far from my current problem. Now I need to 1) add .war
>> file to layer 2). to map file location, for example instead of
>> "module-info.java" I must find "WEB-INF/classes/module-info.java" etc.
>> That is all I need. How can I do it without implementing ModuleFinder?
> You'll need a ModuleFinder because the packaging formats that
> ModuleFinder.of(Path) is required to support doesn't know anything about
> WAR files. It's not super difficult to develop your own. I attach a
> simple implementation that may get you started. It's really basic but
> would need a few iterations to be robust. Invoke
> WarModuleFinder.of(Path) with the file path to the WAR file and it will
> create a ModuleFinder that can find the application module in the WAR
> file. A more complete implementation would be a lot more robust and
> polished that this sample, it would also find the modules WEB-INF/lib.
> 
> Once you have a ModuleFinder then you specify it to Conguration::resolve
> method when resolving the application as the root module. You'll
> probably start with something like:
> 
>     Path war = Path.of("app.war");
>     ModuleFinder finder = WarModuleFinder.of(war);
> 
>     String appModuleName = finder.findAll().stream()
>     .findFirst()
>     .map(ModuleReference::descriptor)
>     .map(ModuleDescriptor::name)
>     .orElseThrow();
> 
>     ModuleLayer boot = ModuleLayer.boot();
>     Configuration cf = boot.configuration().resolve(finder,
> ModuleFinder.of(), Set.of(appModuleName));
>     ModuleLayer layer = boot.defineModulesWithOneLoader(cf,
> ClassLoader.getSystemClassLoader());
> 
> and now you have a module layer with the application module loaded from
> the WEB-INF/classes part of the WAR file.
> 
> -Alan
> 
> 
>     static class WarModuleFinder implements ModuleFinder {
>     private final FileSystem warfs;
>     private final Path classes;
>     private final ModuleReference mref;
> 
>     private WarModuleFinder(Path warfile) throws IOException {
>     ClassLoader scl = ClassLoader.getSystemClassLoader();
>     FileSystem fs = FileSystems.newFileSystem(warfile, scl);
>     Path classes = fs.getPath("/WEB-INF/classes");
> 
>     ModuleDescriptor descriptor;
>     try (InputStream in =
> Files.newInputStream(classes.resolve("module-info.class"))) {
>     descriptor = ModuleDescriptor.read(in, () ->
> packages(classes));
>     }
> 
>     this.warfs = fs;
>     this.classes = classes;
>     this.mref = new ModuleReference(descriptor, classes.toUri()) {
>     @Override
>     public ModuleReader open() {
>     return new WarModuleReader();
>     }
>     public String toString() {
>     StringBuilder sb = new StringBuilder();
>     sb.append("[module ");
>     sb.append(descriptor().name());
>     sb.append(", location=");
>     sb.append(location());
>     sb.append("]");
>     return sb.toString();
>     }
>     };
>     }
> 
>     static WarModuleFinder of(Path war) throws IOException {
>     return new WarModuleFinder(war);
>     }
> 
>     @Override
>     public Optional find(String name) {
>     if (name.equals(mref.descriptor().name())) {
>     return Optional.of(mref);
>     } else {
>     return Optional.empty();
>     }
>     }
> 
>     @Override
>     public Set findAll() {
>     return Set.of(mref);
>     }
> 
>     private Set packages(Path classes) {
>     try {
>     return Files.find(classes, Integer.MAX_VALUE,
>   (path, attrs) -> !attrs.isDirectory())
>     .map(entry -> classes.relativize(entry).toString())
>     .map(this::toPackageName)
>     .flatMap(Optional::stream)
>     .collect(Collectors.toSet());
>     } catch (IOException ioe) {
>     throw new UncheckedIOException(ioe);
>     }
>     }
> 
>     private Optional toPackageName(String name) {
>     int index = name.lastIndexOf("/");
>     if (index > 0) {
>     return Optional.of(name.substring(0,
> index).replace('/', '.'));
>     } else {
>     return Optional.empty();
>     }
>     }
> 
>   

Re: method handles and open packages

[Remi Forax]

Hi,

On July 21, 2018 3:43:51 PM UTC, "Michał Zegan"  
wrote:
>Hello,
>If I use reflection, I can access every member of every package in a
>module if a module opens such package, and that includes private
>members.
>What, however, about using method handles for that purpose?

https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.html#privateLookupIn-java.lang.Class-java.lang.invoke.MethodHandles.Lookup-

>Also, if you have a class in module a that wants to deserialize data to
>objects of classes in module b, then should module b open a package
>with
>those classes to a, or maybe, give the deserializer it's lookup object,
>in order for it to be able to access members of non exported non opened
>packages? It is for the case where the deserializer is called by the
>module b. Also I assume that I do not need access to non public members
>of classes, like I deserialize using public setters always.

Rémi



-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: ClassLoader.getResources vs Module.getResourceAsStream

[Remi Forax]

- Mail original -
> De: "Sander Mak" 
> À: "jigsaw-dev" 
> Envoyé: Lundi 16 Juillet 2018 08:52:41
> Objet: Re: ClassLoader.getResources vs Module.getResourceAsStream

> In that case you'd expose the factories through the services mechanism
> (`provides com.acme.api.MyWidgetFactory with
> com.acme.factories.XmlBasedWidgetFactory` in the module descriptor). Or, if 
> you
> must expose the XML itself to the outside world rather than the factories, you
> can create a service that offers the XML as String or InputStream. Each module
> then has an implementation reading its own encapsulated XML. The former
> approach is preferable IMO.
> 
> Note that you can also use the services mechanism without module descriptors 
> by
> placing text files in META-INF/services
> (https://docs.oracle.com/javase/tutorial/ext/basics/spi.html). However, if 
> your
> code isn't a reusable library itself, you can also choose to modularize your
> own code and depend on non-modularized dependencies through their automatic
> module name
> (http://branchandbound.net/blog/java/2017/12/automatic-module-name/).
> 

You have to do both, each service declared in a module-info MUST be declared in 
META-INF/services too because you can use a modular jar in the classpath as a 
plain old jar. 

Alan, we should patch jar to warn when there is a module-info.class that 
declare services with no corresponding META-INF/services.

> Sander
> 

Rémi

> On 14 Jul 2018, at 17:58, Michał Zegan
> mailto:webczat_...@poczta.onet.pl>> wrote:
> 
> It is a completely new code. It is not modularized for now because some
> dependencies are not modularized, but I want it to be compatible to ease
> later modularization.
> My actual goal is to load xml files defining and configuring some
> factories, from all modules that contain them. Not sure how would you do
> that with services?
> 
> W dniu 14.07.2018 o 17:53, Alan Bateman pisze:
> On 14/07/2018 16:38, Michał Zegan wrote:
> What is then a recommendation for searching for all resources with name
> x that i can access? Something like load all configuration including
> some kind of extensions. I cannot list resources easily.
> 
> Services is the cleaner way to do this kind of thing, esp. if you have
> flexibility to change the code and move away from legacy ClassLoader
> getResources.
> 
> -Alan

The baby and the bathwater (the return)

[Remi Forax]

Hi all,
There were discussions on that list [1] about the fact that beginning with Java 
9, there were 2 ways to deploy modules, classpath vs module-path.

I've discovered last Friday there that are not 2 configurations but 3 
configurations.
You can also use jlink [2] and in that case, the modules are not loaded though 
the module-path but are considered as system modules, so a library should also 
be tested with that configuration.

In my case, Spring Boot annotations scanning works with the classpath, works 
with the module-path but fails if deployed as system modules [3].

regards,
Rémi


[1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2018-March/013689.html
[2] http://openjdk.java.net/jeps/282
[3] https://github.com/forax/pro-demo/tree/master/spring-demo

Re: Compiling module descriptors is not reproducible?

[Remi Forax]

- Mail original -
> De: "Alan Bateman" 
> À: "org openjdk" , "jigsaw-dev" 
> 
> Envoyé: Samedi 19 Mai 2018 13:26:55
> Objet: Re: Compiling module descriptors is not reproducible?

> On 19/05/2018 10:18, Mark Raynsford wrote:
>> Hello!
>>
>> I've been interested for a while in making all of my software builds
>> reproducible [0]. I don't think there's ever been any kind of
>> written guarantee that the output of javac will be completely
>> deterministic, but to date it seems like it actually has been. However,
>> I was a bit disappointed today to find that the output of one of the
>> builds at AdoptOpenJDK [1] seems to be inserting compiler version
>> information into compiled module descriptors. It doesn't seem to be
>> doing this for any other class files.
> The requires table in the Module attribute can be used by compilers to
> record version information about dependences. javac does record the
> version information, I don't know if there is a way to disable that. 

It seems there is no way:
http://hg.openjdk.java.net/jdk/jdk/file/5ec7380f671d/src/jdk.compiler/share/classes/com/sun/tools/javac/jvm/ClassWriter.java#l986

> In any case, I assume your question will be partly answered by looking at
> the output of `java --list-modules` on the different builds. If you
> download OpenJDK builds from jdk.java.net/10 then you should see that
> the standard and JDK-specific modules all report their version string as
> "10" or "10.0.1" as they have been built with configure options that
> make it so. At a guess, your mail may have included the output from an
> exploded (as opposed to images) build.
> 
> -Alan

Rémi

Re: Compiling module descriptors is not reproducible?

[Remi Forax]

- Mail original -
> De: "org openjdk" 
> À: "jigsaw-dev" 
> Envoyé: Samedi 19 Mai 2018 11:18:09
> Objet: Compiling module descriptors is not reproducible?

> Hello!

Hi Mark,

> 
> I've been interested for a while in making all of my software builds
> reproducible [0]. I don't think there's ever been any kind of
> written guarantee that the output of javac will be completely
> deterministic, but to date it seems like it actually has been. However,
> I was a bit disappointed today to find that the output of one of the
> builds at AdoptOpenJDK [1] seems to be inserting compiler version
> information into compiled module descriptors. It doesn't seem to be
> doing this for any other class files.
> 
> Here's the output of "openjdk version 10.0.1 2018-04-17" on Linux
> (not an AdoptOpenJDK build) when compiling a module descriptor:
> 
> ---
> module com.io7m.jtensors.core
>  minor version: 0
>  major version: 53
>  flags: (0x8000) ACC_MODULE
>  this_class: #1  // "module-info"
>  super_class: #0
>  interfaces: 0, fields: 0, methods: 0, attributes: 2
> Constant pool:
>   #1 = Class  #17// "module-info"
>   #2 = Utf8   SourceFile
>   #3 = Utf8   module-info.java
>   #4 = Utf8   Module
>   #5 = Module #18// "com.io7m.jtensors.core"
>   #6 = Module #19// "java.base"
>   #7 = Module #20// "org.immutables.value"
>   #8 = Module #21// "com.io7m.junreachable.core"
>   #9 = Package#22// 
> com/io7m/jtensors/core/determinants
>  #10 = Package#23// com/io7m/jtensors/core/dotproducts
>  #11 = Package#24//
>  com/io7m/jtensors/core/parameterized/matrices
>  #12 = Package#25//
>  com/io7m/jtensors/core/parameterized/vectors
>  #13 = Package#26// com/io7m/jtensors/core/quaternions
>  #14 = Package#27//
>  com/io7m/jtensors/core/unparameterized/matrices
>  #15 = Package#28//
>  com/io7m/jtensors/core/unparameterized/vectors
>  #16 = Package#29// com/io7m/jtensors/core
>  #17 = Utf8   module-info
>  #18 = Utf8   com.io7m.jtensors.core
>  #19 = Utf8   java.base
>  #20 = Utf8   org.immutables.value
>  #21 = Utf8   com.io7m.junreachable.core
>  #22 = Utf8   com/io7m/jtensors/core/determinants
>  #23 = Utf8   com/io7m/jtensors/core/dotproducts
>  #24 = Utf8   com/io7m/jtensors/core/parameterized/matrices
>  #25 = Utf8   com/io7m/jtensors/core/parameterized/vectors
>  #26 = Utf8   com/io7m/jtensors/core/quaternions
>  #27 = Utf8   com/io7m/jtensors/core/unparameterized/matrices
>  #28 = Utf8   com/io7m/jtensors/core/unparameterized/vectors
>  #29 = Utf8   com/io7m/jtensors/core
> {
> }
> SourceFile: "module-info.java"
> Module:
>  #5,0// "com.io7m.jtensors.core"
>  #0
>  3   // requires
>#6,8000 // "java.base" ACC_MANDATED
>#0
>#7,40   // "org.immutables.value"
>ACC_STATIC_PHASE
>#0
>#8,0// "com.io7m.junreachable.core"
>#0
>  8   // exports
>#9,0// 
> com/io7m/jtensors/core/determinants
>#10,0   // 
> com/io7m/jtensors/core/dotproducts
>#11,0   //
>com/io7m/jtensors/core/parameterized/matrices
>#12,0   //
>com/io7m/jtensors/core/parameterized/vectors
>#13,0   // 
> com/io7m/jtensors/core/quaternions
>#14,0   //
>com/io7m/jtensors/core/unparameterized/matrices
>#15,0   //
>com/io7m/jtensors/core/unparameterized/vectors
>#16,0   // com/io7m/jtensors/core
>  0   // opens
>  0   // uses
>  0   // provides
> ---
> 
> Here's the same module descriptor compiled on "openjdk version
> 10-internal" on MacOS (this is actually OpenJDK 10+23):
> 
> ---
> module com.io7m.jtensors.core
>  minor version: 0
>  major version: 53
>  flags: (0x8000) ACC_MODULE
>  this_class: #1  // "module-info"
>  super_class: #0
>  interfaces: 0, fields: 0, methods: 0, attributes: 2
> Constant pool:
>   #1 = Class  #18// "module-info"
>   #2 = Utf8   SourceFile
>   #3 = Utf8   module-info.java
>   #4 = Utf8   Module
>   

Re: The baby and the bathwater

[Remi Forax]

yes, 
!Whatever.class.getModule().isNamed()

Rémi

- Mail original -
> De: "Peter Levart" 
> À: "mark reinhold" , "jigsaw-dev" 
> 
> Envoyé: Mercredi 28 Mars 2018 09:28:23
> Objet: Re: The baby and the bathwater

> On 03/26/18 20:08, mark.reinh...@oracle.com wrote:
>> Stephen closes with a specific suggestion:
>>
>>"There needs to be a way for a library author to insist that the
>> modular jar file they are producing can only be run on the module-path
>> (with any attempt to use it on the class-path preventing application
>> startup).  This would eliminate the need for testing both class-path
>> and module-path."
> 
> That's easy to enforce in runtime. Just take a "victim" class from your
> library that is most often needed when your library is being used (or
> take a couple of them) and add a class initialization block like the
> following to them:
> 
> public class Whatever {
> 
>     static {
>         if (Whatever.class.getModule().getName() == null) {
>             throw new Error("Can only use this library as a module");
>         }
>     }
> 
> 
> Regards, Peter

Re: The baby and the bathwater

[Remi Forax]

- Mail original -
> De: "Alan Bateman" 
> À: "Cédric Champeau" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 27 Mars 2018 10:29:36
> Objet: Re: The baby and the bathwater

> On 27/03/2018 08:15, Cédric Champeau wrote:
>> Dual testing is a minimum. In practice, it depends on the kind of tests.
>> Typically, before JDK 9 for unit tests you never needed a jar to execute
>> unit tests. Maven happens to built it, but in practice a class directory +
>> resources is enough
> This hasn't changed. You can put directories containing the test classes
> + resources on the class path as before. When testing modules you can
> patch a module to add the test classes (and resources) that are compiled
> into a directory, no need for either the module or the tests to be
> packaged as JAR files.

with the limitation that you can not patch a module-info so if you have 
testing-only dependencies like JUnit and you want to run them in module-mode, 
you have to generate a jar.

> 
> Maybe your comment is about testing libraries that are Multi-Release JARs?
> 
> -Alan.

Rémi

Re: Some points on JPMS/Jigsaw after he modularised some code (from Stephen Colebourne)

[Remi Forax]

- Mail original -
> De: "Stephen Colebourne" 
> À: "jigsaw-dev" 
> Envoyé: Vendredi 23 Mars 2018 13:51:06
> Objet: Re: Some points on JPMS/Jigsaw after he modularised some code (from 
> Stephen Colebourne)

> On 23 March 2018 at 10:19, Martijn Verburg  wrote:
>> Stephen's comments are in his blog post:
>> http://blog.joda.org/2018/03/jpms-negative-benefits.html
> 
> Firstly, I want to emphasise that my goal is effectively that of a
> retrospective, to examine what hasn't worked so well and to improve
> things from where we are. The jigsaw team did a great job in getting
> the feature out at all - I want to make sure it is used rather than
> ignored or used as a blocker to progressing the JDK.
> 
> 
>> @Stephen are you able to share the source code?
> 
> The source code is all public:
> https://github.com/ThreeTen/threeten-extra
> https://github.com/JodaOrg/joda-parent
> https://github.com/JodaOrg/joda-convert
> https://github.com/JodaOrg/joda-beans
> 
> While some of the problems are tool-based, the most fundamental issues
> are about JPMS itself.
> 
> At the heart of the problem is the split between class-path and
> module-path. Since this split has happened now, and can't "unhappen"
> what is needed is a way to make it easier to manage (fix the problems
> the split has created).
> 
> As it stands, a library developer cannot control whether they run on
> the class-path or module-path. This increases the bug-surface of the
> library, and requires testing in two different environments (which is
> not widely known). Without being able to insist that a library is on
> the module-path it is also clear that the benefits of strong
> encapsulation and reliable configuration don't apply to library
> consumers.
> 
> My belief is that a way needs to be found for a library author to
> insist that their library is run as a named module. There are probably
> a number of ways this could be achieved - I'm interested in whether
> the change makes things better, not what the specific change is.
> 
> One approach is to say that modular jar files are always treated as
> named modules:
> 
> - a modular jar on the class-path is treated as being named, not part
> of the unnamed module It is therefore encapsulated, but depends on the
> unnamed module (class-path), so does not have full reliable
> configuration. There is the potential for some incompatibility with
> this change where code that uses the modular jar now can't access the
> encapsulated packages, but this is a Good thing (as the library author
> specifically coded for that encapsulation). Any incompatibilities are
> smaller than JPMS has already caused, and could be managed with the
> existing command line flags. I would hope this does not require a JVM
> spec change to achieve.
> 
> A second approach is to say that the module author must mark modules strict:
> 
> - a module author could mark a module as "strict" in module-info.java
> so that it is not permitted to be on the class-path. This does not
> force the whole application to move to the module-path - only strict
> modules and their dependencies would need to move. At runtime, if a
> strict module is found on the class-path an error occurs. I suspect
> this requires a JVM spec change, so may be harder.
> 
> There may well be more approaches. My gut feeling is that the first
> approach is the better one, being relatively simple and implementable,
> but of course I may be wrong.
> 
> Ultimately, the requirement is that library authors who go to the
> effort of adding module-info.java should see some benefits in doing
> so, where today they only have increased cost through running as both
> an unnamed and named module.
> 
> thanks
> Stephen

You can check dynamically if a class is in the unamed module (classpath) or not 
and emit an error message.

This is exactly what the VM does when the class version is wrong.

Rémi

Re: p example (Re: Reflection: how can one access public fields (members) in a superclass ?

[Remi Forax]

- Mail original -
> De: "Rony G. Flatscher" 
> À: "Alan Bateman" , "jigsaw-dev" 
> 
> Envoyé: Mardi 13 Mars 2018 13:16:49
> Objet: Re: p example (Re: Reflection: how can one access public fields 
> (members) in a superclass ?

> Just to conclude this posting: first many thanks to Alan!
> 
> Alan's little nutshell example about how to use MethodHandles.Lookup and 
> employ
> MethodHandle s to
> invoke helped me a lot to get afloat with rewriting the reflection part for 
> the
> bindings also using
> MH. The new reflection mechanism has been implemented such that on Java 1.6/6
> and 1.7/7 it uses core
> reflection (on Java 1.7/7 MethodHandle only reflection runs in some errors 
> that
> cannot be
> circumvented, hence using core reflection) reflection with the need to do
> setAccessible(true) in
> quite a few places. The support for Java 1.8/8 and 9 can use both, core
> reflection (without a need
> for setAccessible() at all!) and MethodHandle reflection (with one case found 
> so
> far, that does not
> work on MH, but with core reflection).
> 
> The type of reflection that is being carried out is in all my use cases (my
> bridge intentionally has
> been only supporting public classes and public members) not really
> specific/relevant to jigsaw. Just
> wanted to point that out explicitly.
> 
> ---
> 
> The only grief I have at the moment is with the meaning of "public": "public"
> has become a homonym
> in jigsaw, introducing a *lot* of confusion into Java (one time "public" is
> public, another time
> "public" is not public).
> 
> This could be probably eased considerably IMHO, if there was a class Modifier
> introduced named
> "module" that would be set for reflective access to classes from non exported
> modules.

It already exists, but not at class level, 
you can open a module, or each package individually, in that case, 
setAccessible is allowed.

> 
> This would have at least the following benefits:
> 
> - communicating (and teaching!) about the publicness of a class can 
> immediately
> be clarified ("is
> the class truly public or does it reside in a non-exported module?"),

Teaching is not a big deal because you can teach package and module at the same 
time and say that it works like onions or russian dolls,
you have module access, class access and member access, by example, a public 
field is not really public if the class is not public and the package not 
exported.
If you take a look from the point of view of accessing a member (and not a 
class), things are regular at module level and class level.  

> 
> - also, programmatically it would be simple to learn whether a class object
> fetched reflectively is
> truly public or not by testing against the presence of such a "Module" 
> modifier
> bit at runtime in
> Java 9 or higher.

you can use Lookup.accessClass() [1].

> 
> ---rony

Rémi
[1] 
https://docs.oracle.com/javase/9/docs/api/java/lang/invoke/MethodHandles.Lookup.html#accessClass-java.lang.Class-

> 
> 
> On 24.01.2018 20:48, Alan Bateman wrote:
>> On 24/01/2018 15:42, Rony G. Flatscher wrote:
>>>
>>> OK, now add to this the following class that uses p.C2 objects to access 
>>> e.g.
>>> m() via it:
>>>
>>> G:\xfer\java9modules\03-20180124-AlanBatmanP\p>type UseC2.java
>>>
>>> public class UseC2
>>> {
>>>     public static void main (String args[]) {
>>>     p.C2 o=new p.C2();
>>>     System.out.println("o="+o);
>>>     System.out.println("o.m()="+o.m());
>>>     }
>>> }
>>>
>>> Compiling all three classes works.
>>>
>>> Running "UseC2" works and outputs:
>>>
>>> G:\xfer\java9modules\03-20180124-AlanBatmanP\p>java -cp ".;.." UseC2
>>> o=p.C2@66048bfd
>>> o.m()=-1
>>>
>>> So it is possible to access m() via the p.C2 object from UseC2.
>> That's right and this is the reason for moving to the simpler example.
>>
>> To be absolutely sure then you should decompile C2.class to make sure that 
>> there
>> isn't a bridge
>> method calling C1's m2(). If you change m() to be final then that will keep 
>> the
>> bridge method from
>> complicating the picture.
>>
>> If you change UseC2 to use core reflection and you hit the issue because the
>> Method object you get
>> is p.C1.m(). Attempting to invoke this will fail with 
>> IllegalAccessException. In
>> your other mail
>> you show a code fragment where it catches exceptions and calls setAccessible 
>> -
>> I'll guess that
>> this may have been masking the issue in the Rexx bridge.
>>
>> For completeness then you may want to try out the new reflection API. I 
>> realize
>> you have to
>> compile to JDK 6 but I think you'll find it will work the same way as the
>> invokevirtual that o.m()
>> compiles to.
>>
>>     MethodHandles.Lookup lookup = MethodHandles.lookup();
>>     MethodType mt = MethodType.methodType(int.class);
>>     MethodHandle mh = lookup.findVirtual(p.C2.class, "m", mt);
>>     Object res = mh.invoke(o);
>>
> > -Alan

Re: java.beans package in java.desktop module

[Remi Forax]

- Mail original -
> De: "Stephen Colebourne" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 7 Mars 2018 13:23:10
> Objet: Re: java.beans package in java.desktop module

> What is needed is an abstraction that can work for all sorts of
> data-like classes
> 
> - classic JavaBeans
> - records
> - value types
> - HashMaps
> - JSON objects
> - etc
> 
> Its not rocket science as an API, but has been needed for many years
> as so many projects have this code duplicated (leading to lots of
> subtle differences).
> 
> The API cannot be based on method handles, as in the HashMap case
> there is no property-specific method to call.

you can bind [1] (do partial evaluation of) Map::get with the property name,

>  But there is no reason why the implementation of the interface for records 
> could not use
> method handles internally.

apart if valhalla generics are around, such interface will require to box and 
unbox values.

> 
> Stephen

Rémi

> 
> 
> On 7 March 2018 at 11:21, Remi Forax  wrote:
>> As Stephen said, with the introduction of the Pattern Matching in the near
>> future, an API to extract the values from an object (the de-constructor API) 
>> or
>> at least from a record object will have to be created, but it may be based on
>> method handles, so perhaps not using a direct interface.
>>
>> Rémi
>>
>> - Mail original -
>>> De: "Guillaume Smet" 
>>> À: "Stephen Colebourne" 
>>> Cc: "jigsaw-dev" 
>>> Envoyé: Mardi 6 Mars 2018 18:45:21
>>> Objet: Re: java.beans package in java.desktop module
>>
>>> Hi Stephen,
>>>
>>> On Tue, Mar 6, 2018 at 3:29 PM, Stephen Colebourne 
>>> wrote:
>>>
>>>> It had been my hope that we might see a replacement for the java.beans
>>>> package. I drew up a rough prototype here:
>>>>  https://github.com/jodastephen/property-alliance
>>>> based on previous work in Joda-Beans. More info here:
>>>>  http://jodastephen.github.io/property-alliance/
>>>>
>>>> If an interface-based design were adopted, it could be implemented by
>>>> the existing JavaBeans code and also by future language changes such
>>>> as records (data classes), while the API could be used far more
>>>> broadly, such as in Hibernate or EL.
>>>>
>>>> Sadly, I haven't had the time or energy to progress this, but the need is
>>>> there.
>>>
>>>
>>> Thanks for sharing. It indeed sound like something worth pursuing.
>>>
>>> Having a new API in the JDK indeed seems the only way out for the
>>> java.beans issue.
>>>
>>> --
> >> Guillaume

Re: java.beans package in java.desktop module

[Remi Forax]

As Stephen said, with the introduction of the Pattern Matching in the near 
future, an API to extract the values from an object (the de-constructor API) or 
at least from a record object will have to be created, but it may be based on 
method handles, so perhaps not using a direct interface.

Rémi

- Mail original -
> De: "Guillaume Smet" 
> À: "Stephen Colebourne" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 6 Mars 2018 18:45:21
> Objet: Re: java.beans package in java.desktop module

> Hi Stephen,
> 
> On Tue, Mar 6, 2018 at 3:29 PM, Stephen Colebourne 
> wrote:
> 
>> It had been my hope that we might see a replacement for the java.beans
>> package. I drew up a rough prototype here:
>>  https://github.com/jodastephen/property-alliance
>> based on previous work in Joda-Beans. More info here:
>>  http://jodastephen.github.io/property-alliance/
>>
>> If an interface-based design were adopted, it could be implemented by
>> the existing JavaBeans code and also by future language changes such
>> as records (data classes), while the API could be used far more
>> broadly, such as in Hibernate or EL.
>>
>> Sadly, I haven't had the time or energy to progress this, but the need is
>> there.
> 
> 
> Thanks for sharing. It indeed sound like something worth pursuing.
> 
> Having a new API in the JDK indeed seems the only way out for the
> java.beans issue.
> 
> --
> Guillaume

Re: Finding module-info.class without loading a jar

[Remi Forax]

On February 24, 2018 12:33:25 PM UTC, Mark Raynsford  
wrote:
>On 2018-02-24T07:53:16 +
>Alan Bateman  wrote:
>>
>> The JarFile API does this for you. Are you sure you've used the 
>> constructor that specifies the runtime version? Once you do that then
>
>> getJarEntry("module-info.class") will locate the module-info.class in
>
>> the versioned section. Use JarEntry::getRealName to satisfy yourself 
>> that it always locates the right one (say where you have a 
>> module-info.class in the top-level directory and one each in 
>> META-INF/versions/9 and META-INF/versions/10).
>
>Ah, thank you. I had completely overlooked this constructor.
>
>> ModuleDescriptor.read is an easy way to parse the module-info.class
>in 
>> case you need it.
>
>Yes, I'm planning to move to this from ASM.

The main difference between the ModuleDescriptor and the ModuleVisitor of ASM 
is that you can read the annotations with the ModuleVisitor.
So it may not worth having ASM as dependency.

Remi


-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

OpenJDK 10 and Oracle JDK10 doesn't have the same default modules

[Remi Forax]

Hi all,
it seems that the OpenJDK 10 and OracleJDK 10 doest not declare the same set of 
default modules, so java --add-modules ALL-DEFAULT do not behave the same way :(

With Oracle JDK 10 b42, module java.scripting is part of the default modules
https://travis-ci.org/sormuras/beautiful_logger/jobs/337153634

With Oracle JDK 10 b42, module java.scripting is NOT part of the default 
modules,
so javax/script/ScriptException is not found
https://travis-ci.org/sormuras/beautiful_logger/jobs/337153635

Should not ALL-DEFAULT mean the same set of default modules for a JDK release ?

cheers,
Rémi

Re: Where do modules go in jlink images?

[Remi Forax]

All classes are merged in the file named 'modules' and the native libraries are 
extracted from the jmods and put in the lib folder, because jlink create an 
image for a specific architecture/OS.

cheers,
Rémi

- Mail original -
> De: "Nicolai Parlog" 
> À: "jigsaw-dev" 
> Envoyé: Dimanche 7 Janvier 2018 13:57:51
> Objet: Where do modules go in jlink images?

> Hi!
> 
> Another question about jlink (guess what chapter I'm working on ;) ).
> I've tried to find out where jlink puts the content of the jmods
> folder, but hit a wall.
> 
> I created a full runtime image with jlink and diffed the file trees
> (including disk usage), but all that got me was that my image doesn't
> contain bin/jmc, jmods, lib/misssioncontrol, lib/src.zip, and
> lib/plugin-legacy.jar - all of that is expected, but where are the
> class files from the jmod archives?
> 
> Now I wonder, whether these are just ornamental - does lib/modules
> contain all the bytecode? I'm confused.
> 
> so long ... Nicolai
> 
> 
> 
> --
> 
> PGP Key:
>http://keys.gnupg.net/pks/lookup?op=vindex&search=0xCA3BAD2E9CCCD509
> 
> Web:
>http://codefx.org
>a blog about software development
>https://www.sitepoint.com/java
>high-quality Java/JVM content
>http://do-foss.de
>Free and Open Source Software for the City of Dortmund
> 
> Twitter:
> https://twitter.com/nipafx

Re: Update a runtime image

[Remi Forax]

jlink is not a distribution tool,
you need to use a tool on top of jlink, in the past, i've used the rpm format 
for that.

We have stopped to distribute deltas when we have moved to docker that as far 
as i know as no concept of incremental update.

Rémi 

- Mail original -
> De: "Alan Bateman" 
> À: "Thomas Brand" , "jigsaw-dev" 
> 
> Envoyé: Dimanche 31 Décembre 2017 16:55:37
> Objet: Re: Update a runtime image

> On 31/12/2017 15:42, Thomas Brand wrote:
>> Hello, I have a question related to jlink.
>>
>> In a scenario where a runnable image created with jlink is installed, and
>> it should be updated, what are good strategies for doing this?
>>
>> I think if an update will introduce new dependencies this will be
>> different to updates that are known to be satisfied with what is already
>> in the image.
>>
>> The idea is to 'ship' just a small sort of diff / replacement part that
>> can be applied to the 'old' image to become 'new'.
>>
> The jlink tool always generates a new run-time image, there is no
> support at this time to install additional modules into a run-time image.
> 
> -Alan.

Re: Module system and services directory in META-INF

[Remi Forax]

I believe that the doc has been moved to the doc of the ServiceLoader
  https://docs.oracle.com/javase/9/docs/api/java/util/ServiceLoader.html

Rémi

- Mail original -
> De: "David Holmes" 
> À: "Jochen Theodorou" , "jigsaw-dev" 
> 
> Envoyé: Mercredi 27 Décembre 2017 07:27:21
> Objet: Re: Module system and services directory in META-INF

> On 27/12/2017 5:43 AM, Jochen Theodorou wrote:
>> On 26.12.2017 19:30, Jochen Theodorou wrote:
>>> hi all,
>>>
>>> Do all files in META-INF/services now have to be well formed according
>>> to the SPI strucutres in Java, or is it still valid to have other
>>> files in there as well? Before the module system it was no problem to
>>> have in there service decriptors, that are services, but do not use
>>> the ServiceProvider structure. Nothing was disturbed as long as nobody
>>> tried to load it that way. Now this seems to be different and I am
>>> wondering where this got specified.
>>>
>>> Because the spec does not try to define file layouts so much I have
>>> the feeling that this is not really specified by the java module
>>> system. Was it then a decision by some maven plugin writers or maybe
>>> even the javac guys?
>> 
>> I found
>> https://docs.oracle.com/javase/8/docs/technotes/guides/jar/jar.html#Service_Provider
>> 
>> Is there no Java 9 version? Also the spec is not specific enough for
>> this as it sounds more like implicit assumption than specification
> 
> https://docs.oracle.com/javase/9/docs/specs/jar/jar.html
> 
> but the Service provider section is gone. Also the document seems
> mis-formatted.
> 
> David
> 
>> bye Jochen

Re: Resolution exception when service interface gets exposed via --add-exports

[Remi Forax]

Hi Alan,
I use jlink plugins for 2 things,
- code generation, i.e. find patterns in the bytecode and optimize them because 
at link time you are in a closed world, so something that was dynamic is not 
anymore.
- just bytecode crawling to find specific annotations to improve startup.

I believe implementation like Weld or Spring should have access to an open API 
to be able to do thing kind of optimizations, imagine, you could create all 
proxies upfront or better use invokedynamic to even avoid proxy creation at all.

The plugin API doesn't have to be complex, it's a read/replace/append of 
bytecode files.

Rémi

- Mail original -
> De: "Alan Bateman" 
> À: "Gunnar Morling" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 21 Novembre 2017 09:33:33
> Objet: Re: Resolution exception when service interface gets exposed via 
> --add-exports

> On 20/11/2017 21:56, Gunnar Morling wrote:
>> I see, thanks for the clarification.
>>
>> It's a pity, though, I hoped to employ that approach for providing a
>> custom jlink plug-in (implementation of
>> jdk.tools.jlink.plugin.Plugin). So it seems that's not possible until
>> jdk.jlink exports j.t.j.plugin? Are there any plans to do so some time
>> soon?
>>
>> Even if it was marked as experimental/incubating, being able to
>> provide custom jlink plug-ins would enable interesting use cases.
>>
> There was interest in creating an incubating module in JDK 9 but it was
> put aside due to the issues of tool modules depending on an incubating
> module. The other issue is that most of the interesting plugins to date
> do code generation at link time and are deeply tied to specific areas of
> java.base and other core modules (they can't really live outside of the
> jdk repo). So while it might be desirable to expose the interface for
> plugins, I think it will need further exploration to see if it makes
> sense or not.
> 
> -Alan

Re: Resolution exception when service interface gets exposed via --add-exports

[Remi Forax]

Hi Gunnar,
you can use --patch-module to add your plugin inside jdk.jlink if you want to 
experiment.

Rémi

- Mail original -
> De: "Gunnar Morling" 
> À: "Alan Bateman" 
> Cc: "jigsaw-dev" 
> Envoyé: Lundi 20 Novembre 2017 22:56:35
> Objet: Re: Resolution exception when service interface gets exposed via 
> --add-exports

> I see, thanks for the clarification.
> 
> It's a pity, though, I hoped to employ that approach for providing a custom
> jlink plug-in (implementation of jdk.tools.jlink.plugin.Plugin). So it
> seems that's not possible until jdk.jlink exports j.t.j.plugin? Are there
> any plans to do so some time soon?
> 
> Even if it was marked as experimental/incubating, being able to provide
> custom jlink plug-ins would enable interesting use cases.
> 
> 2017-11-20 22:24 GMT+01:00 Alan Bateman :
> 
>> On 20/11/2017 21:04, Gunnar Morling wrote:
>>
>>> Hi,
>>>
>>> I'm having one module which defines a service interface but does not
>>> export
>>> that interface's package. In another module I wish to create an
>>> implementation of that service, which should be possible via
>>> --add-exports.
>>>
>>> I can successfully compile that second module, but running it fails:
>>>
>>>  Error occurred during initialization of boot layer
>>>  j.l.m.ResolutionException: Module com.example.b does not read a
>>> module
>>> that exports com.example.a
>>>
>>> It seems as if the service binding happens before the --add-exports of the
>>> java invocation is processed. Please see below for the steps to reproduce.
>>>
>>> Is it a bug or am I trying to do something unsupported here?
>>>
>>> At run-time, the --add-reads/--add-exports options are the equivalent of
>> the module invoking addReads or addExports. So yes, they come into effect
>> after resolution and service binding (and so after the resolution
>> consistency check that detects that a module uses a service type in a
>> package that isn't exported by anyone to the module).
>>
>> -Alan

Re: JDK-8189251

[Remi Forax]

Hi Gili,
the main issue is that an automatic module can see classes from the classpath, 
but it also exports all its package so there is no encapsulation, and once you 
require one automatic module all automatic modules from the module path are 
visible.

So an automatic module is a great tool when you transitioned to the module 
world, but in fine, you do not want any automatic modules in you dependency 
graph.

regards,
Rémi

- Mail original -
> De: "cowwoc" 
> À: "jigsaw-dev" 
> Envoyé: Samedi 14 Octobre 2017 04:13:31
> Objet: JDK-8189251

> Hi,
> 
> I'm trying to understand the reasoning behind
> https://bugs.openjdk.java.net/browse/JDK-8189251 being closed as Won't Fix.
> 
> Can someone please clarify the downside of named automatic-modules vs normal
> modules? I assume that there is some sort of downside; otherwise, we
> wouldn't be warning about them.
> 
> Thank you,
> Gili
> 
> 
> 
> --
> Sent from: http://jigsaw-dev.1059479.n5.nabble.com/

Re: javax.inject module?

[Remi Forax]

Hi Lance,
i'm not sure you have the right to use java or jdk as prefix for your module, 
or do you mean "javax.inject" ?

Rémi

- Mail original -
> De: "Lance Andersen" 
> À: "Tom De Wolf" 
> Cc: "jigsaw-dev" 
> Envoyé: Mercredi 4 Octobre 2017 18:37:53
> Objet: Re: javax.inject module?

> You can grab the jar from maven central
> https://mvnrepository.com/artifact/javax.inject/javax.inject/1
> 
> The JSR  can be found here https://jcp.org/en/jsr/detail?id=330.
> 
> I am hoping to try and get the JSR 330 spec leads to support updating the
> implementation jar with an automatic module name if at all possible. The
> proposed module name is java.inject
> 
> Best
> Lance
> On Oct 4, 2017, at 11:17 AM, Tom De Wolf  wrote:
> 
>> Hi,
>> 
>> Does someone know why the javax.inject annotations contained in for example
>> the following maven dependency are not part of the jdk?
>> 
>> 
>> 
>>javax.inject
>>javax.inject
>>1
>> 
>> 
>> Who or which dependency/jar can be used to depend on this set of injection
>> annotations as an explicit module or a module with a reserved automatic
>> module name? Who is responsible to tackle this for javax.inject?
>> 
>> Thanks,
>> 
>> Tom De Wolf
>> Architect
>> ACA-IT Solutions
>> Belgium
> 
> 
> 
> 
> Lance Andersen| Principal Member of Technical Staff | +1.781.442.2037
> Oracle Java Engineering
> 1 Network Drive
> Burlington, MA 01803
> lance.ander...@oracle.com

Re: issue deriving automatic module name

[Remi Forax]

Hi Paul,
the whole idea of jigsaw is that if you pass the startup check of the module 
graph, there will be not surprise anymore,
so preventing the VM to boot is by design.

Here, you will not be able to reference the jar in another module-info so there 
is no point to start with a wrong configuration.

Rémi

- Mail original -
> De: "Paul Bakker" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 4 Octobre 2017 02:44:01
> Objet: issue deriving automatic module name

> Hi,
> 
> I ran into an issue testing some migration scenarios. When using
> jboss-transaction-api_1.2_spec-1.0.1.Final.jar as an automatic module, startup
> fails with:
> 
> Caused by: java.lang.IllegalArgumentException: jboss.transaction.api.1.2.spec:
> Invalid module name: '1' is not a Java identifier
> 
> This is clearly a strange JAR name, and can be fixed by providing a module 
> name,
> but it should probably not prevent startup.
> 
> Best,
> 
> Paul

Re: Compile Jigsaw project with Intellij IDEA supported structure

[Remi Forax]

And you can use the same trick with Eclipse too that said the Eclipse compiler 
does not support the multi-module layout (only one module by project).

Rémi


On September 27, 2017 2:09:06 PM GMT+02:00, Trisha Gee  
wrote:
>Hi Alexandru,
>
>The structure in IntelliJ IDEA isn't as strict as that, it doesn't
>require
>you to have a "src" folder.  For example you could have:
>
>com.myModule.m1
>/module-info.java
>/com.myModule.m2
>/Main.java
>
>com.myModule.m2
>/module-info.java
>/com.myModule.m2
>/Main.java
>
>You just need to mark com.myModule.m1 and com.myModule.m2 as your
>sources
>root instead of having a src folder as the sources root.
>
>https://www.jetbrains.com/help/idea/configuring-content-roots.html#d85322e277
>
>Trisha
>
>-- Forwarded message -
>Date: Tue, 26 Sep 2017 15:37:18 +0200
>From: "Alexandru Jecan" 
>To: 
>Subject: Compile Jigsaw project with Intellij IDEA supported structure
>Message-ID: <000e01d336cc$93a52ba0$baef82e0$@yahoo.com>
>Content-Type: text/plain;   charset="utf-8"
>
>Hello,
>
>
>
>In order to make my Jigsaw project be supported by Intellij IDEA
>2017.2, I
>need a structure like this, according to IDEA Video:
>https://www.youtube.com/watch?v=WL48zkLvK3I (00:10):
>
>
>
>com.myModule.m1
>
>/src
>
>  /module-info.java
>
> /com.myModule.m2
>
> /Main.java
>
>
>
>com.myModule.m2
>
>/src
>
>  /module-info.java
>
> /com.myModule.m2
>
> /Main.java
>
>
>
>
>
>How do I perform multi-module compilation in this case using the
>--module-source-path option?
>
>
>
>As far as I know, to perform multi-module compilation, the source code
>directory should have the same name like the name of the module.
>
>In this case, I have the ?src? folder there which does not comply to
>this
>rule.
>
>
>
>
>
>Best regards
>
>
>
>Alexandru Jecan

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Scanning multi version jars?

[Remi Forax]

Hi Greg,
the notion of inner classes do not exist at runtime (apart if you explicitly 
ask by reflection).

The compiler desugar inner classes to several classes, so the compiler needs 
attributes (InnerClasses and EnclosingMethod) in the classfile to be able to 
reconstruct the java source file view of the world when it sees already 
compiled classes.

But the VM/runtime doesn't need those attributes, the VM doesn't care about 
inner classes. There is a comment at the end of section 4.7.6 of the JVMS that 
explain that the VM does not check the InnerClasses attribute at runtime 
"Oracle's Java Virtual Machine implementation does not check the consistency of 
an InnerClasses attribute against a class file representing a class or 
interface referenced by the attribute." 

So to answer to your question, the classloader do not care about inner classes.

Note that in a close future with the introduction of nestmates [1], it will be 
another story, but we have at least to wait 18.9 for that.

Rémi
[1] http://openjdk.java.net/jeps/181

- Mail original -
> De: "Greg Wilkins" 
> À: "Stephen Felts" 
> Cc: "jigsaw-dev" , "core-libs-dev" 
> 
> Envoyé: Mardi 19 Septembre 2017 06:37:37
> Objet: Re: Scanning multi version jars?

> Stephen,
> 
> I think the use-case can be pretty well defined.
> 
> There should be an enumeration/iterator/stream available that provides the
> contents of a jar file as it would be seen/interpreted by the JVMs
> classloader.So if the classloader is doing any processing to handle
> versioned classes/resources, then we need an iterator that implements the
> exact same logic.
> 
> Which raises an interesting point   with the multi versioned jar I have
> used as an example, which contains:
> 
>   - org/example/Foo.class
>   - org/example/Foo$Bar.class
>   - META-INF/versions/9/org/example/Foo.class
> 
> What does the classloader do when asked to load "org.example.Foo$Bar" ?
> If it loads it OK, then the JarFile enumerator/iterator/stream should also
> return it.   If it throws a ClassNotFoundException, then the
> JarFile enumerator/iterator/stream should skip it.
> 
> Currently the classloader will happily return a resource for a base inner
> class even if its outerclass does not refer to it, so that suggests that
> the iteration should also not process out the inappropriate inner classes.
> However I think it could be argued that the loader should not load it.
> 
> Eitherway, there should be an iteration available that is entirely
> consistent with what the classloader does.
> 
> regards
> 
> 
> 
> 
> On 19 September 2017 at 13:41, Stephen Felts 
> wrote:
> 
>> Thanks for the clarification – I overstated the “any JarEntry”.
>>
>> I didn’t look at VersionedStream so I now understand the limitations you
>> mention.
>>
>>
>>
>> In my case, it’s necessary to look at all files in the jar file to do the
>> elimination of unneeded ordinary/inner classes so JarInputStream
>> getNextJarEntry()
>> can be used.
>>
>> By using the versioned JarFile constructor, getting the JarEntry returns
>> the right one for processing.  If I needed further filtering on the file
>> names, I’d need to return the real file names.
>>
>>
>>
>> Maybe the use case isn’t so universal or well defined.
>>
>>
>>
>>
>>
>>
>>
>>
>> *From:* Greg Wilkins [mailto:gr...@webtide.com]
>> *Sent:* Monday, September 18, 2017 9:33 PM
>> *To:* Stephen Felts 
>> *Cc:* Paul Sandoz ; jigsaw-dev <
>> jigsaw-dev@openjdk.java.net>; core-libs-...@openjdk.java.net
>>
>> *Subject:* Re: Scanning multi version jars?
>>
>>
>>
>> Stephen,
>>
>>
>>
>> It is not the case that the getName() always returns the path starting
>> with "META-INF/versions/". Specifically, if the entry is obtained from
>> getJarEntry() API (and not from the enumerator), then the name is that of
>> the unversioned file, but the metadata and contents obtained using the
>> jarEntry are for the versioned entry.
>>
>>
>>
>> For example the following code:
>>
>>
>>
>> JarFile jarFile = new JarFile(new File("/tmp/example.jar"),
>>   false,
>>   JarFile.OPEN_READ,
>>   Runtime.version());
>> JarEntry entry = jarFile.getJarEntry("org/example/OnlyIn9.class");
>> System.err.printf("%s -> %s%n",entry.getName(),IO.toString(jarFile.
>> getInputStream(entry)));
>>
>> when run against a jar where the class files contain just the text of
>> their full path produces the following output:
>>
>>
>>
>> org/example/OnlyIn9.class -> META-INF/versions/9/org/example/OnlyIn9.class
>>
>>
>>
>> There is nothing in the public API of the JarEntry so obtained that
>> indicates that it the versioned entry, nor can I distinguish it from an
>> entry obtained by iteration that may report the same name (if the entry was
>> also in the base), although at least equals does return false.
>>
>>
>>
>> Moreover, the proposed stream API as represented by the current
>> implementation of jdk.internal.util.jar.VersionedStream, a

Re: Moving to Java 9 - module-info.class not found for module

[Remi Forax]

As Alex said,
jackson.databind is not an explicit module.

You should report that issue to their bugtracker.

cheers,
Rémi

- Mail original -
> De: "Alex Buckley" 
> À: "jigsaw-dev" 
> Envoyé: Vendredi 1 Septembre 2017 23:06:16
> Objet: Re: Moving to Java 9 - module-info.class not found for  
> module

> On 9/1/2017 1:21 PM, Rahman USTA wrote:
>> java --module-path
>> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
>> terminalfx -m terminalfx/com.terminalfx.AppStarter
> 
> (You shouldn't need the --add-modules, since terminalfx is already the
> main module.)
> 
>> It works normally. Then, I want to generate a jlink image with the
>> following script
>>
>> jlink --module-path
>> %JAVA_HOME%/jmods;target\terminalfx.jar;target\dependency --add-modules
>> terminalfx --launcher terminalfx=terminalfx/com.terminalfx.AppStarter
>> --output target/release
>>
>> However it gives me the following error;
>>
>> Error: module-info.class not found for jackson.databind module
> 
> I suspect jackson.databind is an automatic module. jlink does not
> support linking of automatic modules because they can rely on the
> arbitrary content of the classpath, which goes against the idea of a
> self-contained Java runtime.
> 
> Alex

Re: Jigsaw and containers

[Remi Forax]

Hi Daniel,
several points first,
the JRE doesn't exist anymore apart for compatibility (the jre is just a bunch 
of modules) and a jar can be a modular jar, just have a module-info.class in 
the jar, so applications that uses jars can be modular, they do not have to use 
jlink.

So you can have a jlink image as small as just java.base and all your 
applications and their dependencies as modular jars, the modules will be shared 
by the different applications, or a jlink all applications and have no sharing 
at all and all the states in between.

Rémi

- Mail original -
> De: "Daniel Latrémolière" 
> À: "Alan Bateman" , "jigsaw-dev" 
> 
> Envoyé: Jeudi 31 Août 2017 20:47:35
> Objet: Re: Jigsaw and containers

>> Are you using `jlink` to create a run-time image for each application?
>> (just trying to establish if you are actually running into an issue or
>> not).
> With jlink per application, all will be working without problems, but
> each application will have his own copy of JRE and common libraries (big
> size).
> 
> With jlink only for jre and common libraries, these parts can be shared
> between applications. But specific parts of each application will be
> only in JAR files (and not be modularized).
> 
> These two choices works as reasonable targets, but no one is really
> completely good (big size or no full modularization).
> 
> Daniel.

Re: Jigsaw questions

[Remi Forax]

- Mail original -
> De: "Oleg Tsal-Tsalko" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Jeudi 13 Juillet 2017 09:34:53
> Objet: Jigsaw questions

> Dear experts,
> 
> I'm from JUG UA and currently playing with new Java 9 Module System using
> Early Access Jigsaw build #174. As an exercise I'm modularising JUnit 5. In
> process I faced couple of issues and have couple of questions to you:
> 
>   1. *How to compile test classes that are packaged in same packages as
>   production code? *When I'm trying to compile test sources separately
>   from application sources that have been already modularised (packaged in
>   modules) I'm getting errors saying that particular packages already found
>   in certain modules on module path. What are recommendations (best practice)
>   here?

--patch-module is your friend.

The other solution is to put the test classes in another module and to be able 
to merge module (the tested module and the test module) but it requires support 
either by the build tool you use or by JUnit itself.


>   2. When I'm trying to run unit tests placed on classpath using
>   modularised JUnit5 library put on module path I'm getting errors
> like "*Could
>   not load class with
>   name: org.junit.platform.commons.util.CollectionUtilsTests*". I have
>   deduced that it is because CollectionUtilsTests class lives in same
>   package as already exists in junit.platform.commons module placed on
>   module path. If I run org.junit.mytests.SimpleTest from custom unique
>   package instead everything works fine. My question is more general here
>   though: *how to deal (access via reflection for example) with classes on
>   classpath that use same packages as certain modules on module path?* We
>   can't (it will be very difficult and inconvenient) ensure that no library
>   on classpath uses same package as some of the modules on module path...

Christian Stein (sormuras on Github) has already played with running JUnit5 on 
Java 9,
see https://github.com/sormuras/application-junit5-jdk9-demo

and for your second question, you can't, a package should be in only one module 
at a time (a package in the classpath is part of the unamed module), that's why 
you have to use --path-module to put all the test classes with the tested 
classes in the same package inside the same module.

> 
> Could you please clarify these aspects to me please or point me where I can
> read about it?
> 
> Thank you,
> Oleg

cheers,
Rémi

Re: javap cannot read module-info.class

[Remi Forax]

Hi Stephan,
there is something wrong from your side, it works for me :)

/usr/jdk/jdk-9-b175/bin/javap --module-path target/main/exploded/ --module 
fr.umlv.asm.test module-info
Compiled from "module-info.java"
module fr.umlv.asm.test {
  requires java.base;
  requires org.objectweb.asm.all.debug;
  exports fr.umlv.asm.test;
}

/usr/jdk/jdk-9-b175/bin/javap -m java.base module-info
Compiled from "module-info.java"
module java.base@9 {
  exports sun.reflect.generics.reflectiveObjects to
java.desktop;
  exports java.util;
  exports java.lang.ref;
  exports jdk.internal.misc to
java.rmi,
java.sql,
jdk.jshell,
...

cheers,
Rémi

- Mail original -
> De: "Stephan Herrmann" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Samedi 1 Juillet 2017 17:08:20
> Objet: javap cannot read module-info.class

> compile an arbitrary module-info.java and then:
> 
> $ javap module-info.class
>  Error: error while reading constant pool for /tmp/bin/module-info.class:
>  unexpected tag at #5: 19
> 
> From tweaking the Eclipse compiler it seems that javap is expecting a
> CONSTANT_Utf8_info,
> where according to JVMS 4.7.25 a CONSTANT_Module_info is required.
> 
> JVMS and javac agree, so javap is the odd man out.
> 
> Stephan
> 
> PS: build is 9+175

Non Standadrd Module Attribute

[Remi Forax]

Hi Alan, hi all,
i've implemented the non standard attributes ModuleTarget, ModuleHashes and 
ModuleResolution in ASM.

First, given that i was not able to find a document that describe them, i've 
used the javadoc of the classes inside jdk.internal.module.ClassFileAttributes.
I've discovered that the jdk modules have an attribute ModuleTarget that can 
have a null platform, why not removing the ModuleTarget attribute from the 
module-info instead ?
And for the module jdk.incubator.httpclient, the value of the resolution is 9, 
so it means DO_NOT_RESOLVE_BY_DEFAULT | WARN_INCUBATING;
What is the exact meaning of DO_NOT_RESOLVE_BY_DEFAULT ?

regards,
Rémi

Re: 8182482: Module System spec updates

[Remi Forax]

- Mail original -
> De: "Alex Buckley" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 20 Juin 2017 20:09:14
> Objet: Re: 8182482: Module System spec updates

> Hi Remi,
> 
> On 6/20/2017 6:29 AM, fo...@univ-mlv.fr wrote:
>> ok, let's focus on abstract class defining a service.
> 
> I would be happy for the "Designing services" section to give more
> advice about the tradeoffs between an interface and an abstract class.
> Two sentences, written in a style that leads a junior developer but does
> not judge them if they don't follow the advice. Can you write it? :-

sure let's try:
if you are wondering when to use an interface and when to use an abstract 
class, you can use this advice: most of the time you should favor interfaces, 
unless you want to execute your own code as part of any service implementations 
like by example when you want to add a security check to validate something 
about the implementation.

I'm sure you will be able to transform it in proper English.

> 
> -
> A service is a single type, usually an interface or abstract class.
> ***REMI'S TEXT HERE*** A concrete class can be used, but this is not
> recommended. The type may have any accessibility.
> 
> The methods of a service are highly ...
> -
> 
> Alex

Rémi

Re: 8182482: Module System spec updates

[Remi Forax]

Hi Alan,
the doc is clearly better.

Can you change
  A service is a single type, usually an interface or abstract class. A 
concrete class can be used, but this is not recommended.
to 
  A service is a single type, usually an interface. An abstract class or a 
concrete class can be used, but this is not recommended.

An abstract class has a constructor so you can observe when a service 
implementation is instantiated by the ServiceLoader, which may make the code 
dependent of the loading order of the service implementations. So using an 
abstract class for a service implementation is also a bad idea.

regards,
Rémi   

- Mail original -
> De: "Alan Bateman" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 20 Juin 2017 12:20:16
> Objet: 8182482: Module System spec updates

> We have two javadoc/spec updates that I'd like to get into the JDK 9
> Initial Release Candidate that is scheduled for this week.
> 
> The spec updates are for two issues:
> 
> 1. ServiceLoader: The API spec has been updated significantly to support
> modules but it needs another round of update to do clean-up to get it
> more readable and consistent, and also to align it with the JLS.  Most
> of reorganization and re-wording has been proposed by Alex. Joe Darcy
> has also proposed a few adjustments.
> 
> 2. Upgradable modules aren't specified anywhere. Java SE will designate
> a number of standard modules as upgradeable but we don't have anywhere
> in the docs to link to that or describe how the upgraded versions are
> used in preference to the modules built into the environment.
> 
> The webrev with the proposed (docs only, no implementation) changes is here:
>   http://cr.openjdk.java.net/~alanb/8182482/webrev/index.html
> 
> The ServiceLoader diffs are hard to read. It might be easier to read the
> generated javadoc:
> http://cr.openjdk.java.net/~alanb/8182482/docs/java/util/ServiceLoader.html
> 
> -Alan

Re: Review Request JDK-8182032: Make java.compiler upgradeable

[Remi Forax]

My concern is that by example javac and javap need to be aligned and by making 
a module upgradeable you can not make only some packages upgradeable.
So it seems logical that if java.compiler is upgradeable the other modules of 
langtools are upgradable too.

Rémi 


On June 13, 2017 10:26:18 AM GMT+02:00, Alan Bateman  
wrote:
>On 13/06/2017 08:42, Remi Forax wrote:
>> Hi Mandy,
>> why only java.compiler is upgradable and not all modules defined in
>langtools ?
>>
>java.compiler needs to be upgradeable because JSR 199 and JSR 269 are 
>standalone technologies.
>
>The other modules in the langtools repository are JDK-specific. They 
>aren't meant to be upgraded via the upgrade module path. If someone is 
>forking the code to create their own version of jdk.compiler (for 
>example) then they can re-package or load it into a child layer.
>
>-Alan

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Review Request JDK-8182032: Make java.compiler upgradeable

[Remi Forax]

Hi Mandy,
why only java.compiler is upgradable and not all modules defined in langtools ?

cheers,
Rémi

- Mail original -
> De: "Mandy Chung" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 13 Juin 2017 07:35:04
> Objet: Review Request JDK-8182032: Make java.compiler upgradeable

> Webrev at:
>   http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8182032/webrev.00/
> 
> java.compiler is a standalone technology that allows to be running on older 
> JDK,
> in particular for IDE to support new language features.
> 
> This patch takes out the make logic to find the modules that directly and
> indirectly require any upgradeable modules and include them as upgradeable.
> Instead it lists all upgradeable modules.  The list of upgradeable modules is
> small whereas the exclude list is not (which is also error-prone to find 
> them).
> I have added a new test to verify what modules are hashed in java.base.
> 
> Mandy

Re: Annotation-based syntax for module-info.java

[Remi Forax]

Hi Markus,

- Mail original -
> De: "Markus Keller" 
> À: jigsaw-dev@openjdk.java.net
> Cc: "Daniel Megert" 
> Envoyé: Lundi 12 Juin 2017 15:26:10
> Objet: Annotation-based syntax for module-info.java

> [I know this is coming way too late, and my recent accident that
> incapacitated me for several weeks didn't help either. Nevertheless, I
> think the current module-info.java syntax was a snap decision that should
> not end up in the Java 9 spec.]

i hope you're well now,
and as you said, it's too late, and it doesn't solve the issue we had (see 
below ...)

> 
> The discussions about restricted keywords
> http://openjdk.java.net/projects/jigsaw/spec/issues/#RestrictedKeywords
> and the difficulties with defining a proper grammar for module-info.java
> indicate that something could be fundamentally wrong with the chosen
> approach that tries to come up with a new way of specifying a language.

The problem is not to define the grammar, the module-info grammar is very 
straight forward.
The problem is how to introduce new keywords when the language evolves without 
breaking the source compatibility as we have done with assert and enum in the 
past.

Even if we can use annotations (that's ugly but it works) to solve the 
particular case of the module-info, we will have the same issue when we will 
want to introduce new keywords like 'var', 'data', 'closed', 'value'*, etc in 
the future.
Local keyword (named restricted keyword in the spec) is a good answer for 
specifying how a keyword and an identifier can have the same name and 
disambiguate them.

cheers,
Rémi

* those keywords are among the ones proposed by projects Amber and Valhalla for 
the next releases of Java.

RE: Will split-packages ever be supported?

[Remi Forax]

On May 30, 2017 4:28:00 PM GMT+02:00, Stephen Felts  
wrote:

Hi Stephen,

>Wouldn't it be possible to add an enhancement to allow for a module to
>add a package to an existing module? 

You can already use --patch-module at compile time and runtime and obviously 
inject packages before creating the module DAG if you create a ModuleLayer.

> Sort of like OSGI fragment
>bundles, which are very popular.

popular among OSGI users, and  sometimes for the wrong reason IMO like a 
practical workaround of the one artifact == one classloader limitation of OSGI

>I don't understand what is so fundamental about this.

OSGI requires packages, not modules, so fragments make a lot of more sense.

In JPMS, the identity of a module is its containing packages (even the non 
exported ones at least for 9) so fragment makes even less sense because they 
will change the ABI of a module.

Also note that the configuration adaptability use cases of fragments is covered 
by different mechanisms, services (using an interface instead of injecting 
property files is cleaner IMO), the require static (to react if a class is 
present or not by reflection), the multi version jars (for features depending 
on the JDK version) and jmod file (for including native libraries).

cheers,
Rémi 

>
>
>-Original Message-
>From: Alan Bateman 
>Sent: Tuesday, May 30, 2017 8:17 AM
>To: wzberger; jigsaw-dev@openjdk.java.net
>Subject: Re: Will split-packages ever be supported?
>
>On 30/05/2017 11:52, wzberger wrote:
>
>> Our project has around 45 jar libraries and we currently using 
>> split-packages as simplest modularization concept. It's a desktop 
>> project with different kind of controls - all have in common that
>they 
>> are located in the same package e.g. 'com.swing'. So users can add 
>> only required libraries to their project. However, in Jigsaw the 
>> split-package concept is not supported - so we have to completely 
>> rework our package structure. This means:
>>
>> - the new package structure will become more complicated because we 
>> have to add new packages
>> - our API isn't backward compatible
>> - our users have to rework their applications
>> - our users have to learn the new API (package structure)
>>
>> So how likely is it that split packages will be supported in the near
>
>> future?
>It's fundamental to reliable configuration that two or more modules do
>not export a package with the same name to a module that reads both
>(continuing from the spec "This includes the case where a module M
>containing package p reads another module that exports p to M"). It
>seems very unlikely to me that something as fundamental and core as
>this will ever be dropped.
>
>To your example, then if the project can't be restructured to address
>the split packages issues then it will need to stay on the class path. 
>There's nothing wrong with that, it should continue to work as before.
>
>-Alan.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Will split-packages ever be supported?

[Remi Forax]

- Mail original -
> De: "Jochen Theodorou" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mardi 30 Mai 2017 15:16:46
> Objet: Re: Will split-packages ever be supported?

> On 30.05.2017 14:16, Alan Bateman wrote:
>> On 30/05/2017 11:52, wzberger wrote:
>>
>>> Our project has around 45 jar libraries and we currently using
>>> split-packages as simplest modularization concept. It's a desktop
>>> project with different kind of controls - all have in common that they
>>> are located in the same package e.g. 'com.swing'. So users can add
>>> only required libraries to their project
> [...]
>> To your example, then if the project can't be restructured to address
>> the split packages issues then it will need to stay on the class path.
>> There's nothing wrong with that, it should continue to work as before.
> 
> Of course staying on the classpath means to get replaced in the future.
> So I don't agree with you here: there is a lot wrong with that imho.
> Nobody really cares in the end if you are used as module or not, but if
> you cannot do it, you are being replaced with something that can,
> because you are not future proof. Breaking the code of about everyone
> depending of you can have of course the same effect.
> 
> I mean... assume you want to write a named module... how do you depend
> on a class in an unnamed module? You do not. You can for example not
> extend such a class. javac would not allow that. I can of course still
> compile in classpath mode and then use a yet to be written tool, that
> generates the module information independent of javac... It means your
> module would have an undeclared requirement. I think that is a
> complication you normally would not want to have. It is probably more
> easy to drop the framework and use something else then.
> 
> Oh, you forgot the other alternative... make it a big monolith and
> squash the 45 jars into one big module. Then it can move away from the
> classpath too. Sure, making a monolithic module really defies the
> purpose of the module system, but it is still better than breaking code
> or requiring the users to do module system acrobatics.

the big monolithic module is a good transition solution,
i've done that with Aether to get access to Maven Central inside a fully 
modularized project,
but it's like with automatic modules, it's just a temporary solution that you 
can use in your application and not something you can do if you want to publish 
a library.

> 
> bye Jochen


Rémi

Re: JPMS Access Checks, Verification and the Security Manager

[Remi Forax]

Hi Volker,
this is the behavior of the verifier since 6 (when the split verifier becomes 
the default one).

Let say you have a code like this,

public class TestVerifier {
  static class B extends A { }
  static class A { }

  public static void main(String[] args) {
A a;
if (args.length == 0) {
  a = new A();
} else {
  a = new B();
}
System.out.println(a.toString());
  }
}

if you compile and then remove TestVerifier$B.class, you will get
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.NoClassDefFoundError: TestVerifier$B
at java.lang.Class.getDeclaredMethods0(Native Method)
at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
at java.lang.Class.getMethod0(Class.java:3018)
at java.lang.Class.getMethod(Class.java:1784)
at 
sun.launcher.LauncherHelper.validateMainClass(LauncherHelper.java:544)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:526)
Caused by: java.lang.ClassNotFoundException: TestVerifier$B
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:335)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 7 more

whatever the comand line arguments because it's the verifier that throws an 
error. 

There is a workaround for you, if you use an interface instead of a class (in 
my example, make A an interface), it will work !
The verifier does not verify interface at verification time, the checks in done 
once by the VM at runtime.

so if there is a bug, it's in the way the verifier actually works i.e. if it 
can not find a class or do not see a class because of the module encapsulation, 
it should postpone the check at runtime instead of reporting an error at 
compile time. Thats said, i'm not sure it's a good idea.

cheers,
Rémi

- Mail original -
> De: "Volker Simonis" 
> À: "jigsaw-dev" , "security-dev" 
> 
> Envoyé: Mardi 23 Mai 2017 09:27:22
> Objet: JPMS Access Checks, Verification and the Security Manager

> Hi,
> 
> my question is if it is OK for a program which otherwise runs fine to
> fail during class verification if running with a security manager
> because of module access restrictions?
> 
> As the following write-up got a little lengthy, I've also uploaded a
> html version for a nicer reading experience :)
> 
> http://cr.openjdk.java.net/~simonis/webrevs/2017/veri_with_secman/sm.html
> 
> Consider the following small, self-contained program which statically
> references com.sun.crypto.provider.SunJCE:
> 
> import java.security.Provider;
> import com.sun.crypto.provider.SunJCE;
> 
> public class Tricky {
> 
>  public static void main(String args[]) throws Exception {
>try {
>  System.out.println(Tricky.class + " (" +
> Tricky.class.getClassLoader() + ")" + args[0]);
>}
>catch (Exception e) {
>  Provider p = new SunJCE();
>  System.out.println(p + " (" + p.toString() + ")");
>}
>  }
> }
> 
> This program easily compiles and runs with Oracle/OpenJDK 8:
> 
> $ javac Tricky
> $ java Tricky ""
> class Tricky (sun.misc.Launcher$AppClassLoader@4e0e2f2a)
> $ java Tricky
> SunJCE version 1.8 (SunJCE version 1.8)
> 
> The second invocation (without argument) will cause an exception (when
> trying to access the first element of the zero length argument array
> at 'args[0]') which will be caught in the exception handler where we
> create a new 'SunJCE' object and print its string representation to
> stdout. The first invocation (with an empty argument) doesn't provoke
> an exception and will just print the class itself together with its
> class loader.
> 
> When we run the same (jdk8 compiled) program with jdk9, we'll see the
> following result:
> 
> $ jdk9/java Tricky ""
> class Tricky (jdk.internal.loader.ClassLoaders$AppClassLoader@ba8a1dc)
> $ jdk9/java Tricky
> Exception in thread "main" java.lang.IllegalAccessError: class Tricky
> (in unnamed module @0x3b192d32) cannot access class
> com.sun.crypto.provider.SunJCE (in module java.base) because module
> java.base does not export com.sun.crypto.provider to unnamed module
> @0x3b192d32
>  at Tricky.main(Tricky.java:11)
> $ jdk9/java --add-exports java.base/com.sun.crypto.provider=ALL-UNNAMED Tricky
> SunJCE version 9 (SunJCE version 9)
> 
> The first invocation (with an empty argument) still works because we
> have lazy class loading and 'SunJCE' provider is actually never used.
> The second invocation (without argument) obviously fails with jdk9
> because 'java.base' doesn't export the package com.sun.crypto.provider
> anymore. This can be fixed by adding '--add-exports
> java.base/com.sun.crypto.provider=ALL-UNNAMED' to the command line as
> demonstrated in the third invocation.
> 
> So far so good -

Re: An alternative to "restricted keywords" + helping automatic modules

[Remi Forax]

Hi Stephen,

- Mail original -
> De: "Stephen Colebourne" 
> À: "jigsaw-dev" 
> Envoyé: Vendredi 19 Mai 2017 10:58:01
> Objet: Re: An alternative to "restricted keywords" + helping automatic modules

> I don't support the ^ element or escaping like that either.
> 
> However, would adding a "module" keyword help?
> 
> module com.foo.lib {
>  requires module com.foo.bar;
> }
> 
> thus:
> 
> module com.foo.lib {
>  requires static module blah;
>  requires transitive module transitive;
> }
> 
> ie. the module name is always prefixed by "module" in a "requires"
> statement. But does this help?

I will only answer on this question from the technical point of view, the EG as 
already rule out this syntax and i do not think it's wise to reopen an issue on 
the module-info syntax now.
So yes it helps but we don't care.

Adding any mandatory keyword between the modifiers and the module name in the 
require directive as the effect to allow to require a module with module name 
that starts with transitive.
But this is solve a corner case of a corner case.

The corner case we are trying to solve here is to allow package name and module 
name that contains any keywords defined in module-info that are not defined as 
keywords in JLS8, (open, module, requires, transitive, exports, opens, to, 
uses, provides, and with). For that we need, a way to activate keywords locally 
when the grammar wait for them and do not activate them where we don't care (in 
the middle of a package name or a module name).

The fact that activating keywords do not solve the problem of module that 
starts with transitive in the require directive should be seen in my point of 
view a corner case.
And as i already said, because module name should use reverse DNS convention, 
i'm not even sure this problem will occur in practice,
thus here we can just say that we will not solve that issue.

> 
> Stephen
> 
> 

regards,
Rémi


> On 18 May 2017 at 09:59, Stephan Herrmann  wrote:
>> Remi,
>>
>> I see your proposal as a minimal compromise, avoiding the worst
>> of difficulties, but I think we can do better.
>>
>> Trade-off:
>> In all posts I could not find a real reason against escaping,
>> aside from aesthetics. I don't see this as sufficient motivation
>> for a less-then-perfect solution.
>>
>>
>> Clarity:
>> I'm still not completely following your explanations, partly because
>> of the jargon you are using. I'll leave it to Alex to decide if he
>> likes the idea that JLS would have to explain terms like dotted
>> production.
>>
>> Compare this to just adding a few more rules to the grammar,
>> where no hand-waving is needed for an explanation.
>> No, I did not say that escaping is a pervasive change.
>> I never said that the grammar for ordinary compilation units
>> should be changed.
>> If you like we only need to extend one rule for the scope of
>> modular compilation units: Identifier. It can't get simpler.
>>
>>
>> Completeness:
>> I understand you as saying, module names cannot start with
>> "transitive". Mind you, that every modifier that will be added
>> to the grammar for modules in the future will cause conflicts for
>> names that are now legal, and you won't have a means to resolve this.
>>
>> By contrast, we can use the escaping approach even to solve one
>> more problem that has been briefly touched on this list before:
>>
>> Automatic modules suffer from the fact that some artifact names may
>> have Java keywords in their name, which means that these artifacts
>> simply cannot be used as automatic modules, right?
>> Why not apply escaping also here? *Any* dot-separated sequence
>> of words could be used as module name, as long as module references
>> have a means to escape any keywords in that sequence.
>>
>>
>> Suitability for implementation:
>> As said, your proposal resolves one problem, but still IDE
>> functionality suffers from restricted keywords, because scanning
>> and parsing need more context information than normal.
>> - Recovery after a syntax error will regress.
>> - Scanning arbitrary regions of code is not possible.
>> Remember:
>> In an IDE code with syntax errors is the norm, not an exception,
>> as the IDE provides functionality to work on incomplete code.
>>
>>
>> Stephan
>>
>>
>> On 18.05.2017 00:34, Remi Forax wrote:
>>>
>>> I want to answer this before we start the meetings because i really think
>>> that restricted keyword as i propose solve t

Re: Attaching to a JVM image that does not include java.instrument

[Remi Forax]

- Mail original -
> De: "Alan Bateman" 
> À: "Rafael Winterhalter" 
> Cc: "jigsaw-dev" 
> Envoyé: Vendredi 19 Mai 2017 10:22:04
> Objet: Re: Attaching to a JVM image that does not include java.instrument

> On 19/05/2017 08:08, Rafael Winterhalter wrote:
> 
>> Hi Alan,
>>
>> I just retested with the most recent Java 9 snapshot and this is what
>> I get:
>>
>> rafael@rafc:~/jdk9-test/greetingsapp$ ./bin/java
>> -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar
>> -m com.greetings/com.greetings.Main
>> Error occurred during initialization of VM
>> Could not find agent library instrument on the library path, with
>> error: libinstrument.so: cannot open shared object file: No such file
>> or directory
>> rafael@rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules
>> java.instrument
>> -javaagent:/home/rafael/.m2/repository/sample-agent/sample-agent/1.0-SNAPSHOT/sample-agent-1.0-SNAPSHOT.jar
>> -m com.greetings/com.greetings.Main
>> Error occurred during initialization of VM
>> Could not find agent library instrument on the library path, with
>> error: libinstrument.so: cannot open shared object file: No such file
>> or directory
>> rafael@rafc:~/jdk9-test/greetingsapp$ ./bin/java --add-modules
>> java.instrument -m com.greetings/com.greetings.MainError occurred
>> during initialization of boot layer
>> java.lang.module.FindException: Module java.instrument not found
>>
>> Do I understand you correctly that you expect the last outcome also as
>> a result for the first command?
> If the run-time image does not contain java.instrument then the error
> for each of these cases should be like the last one ("Module
> java.instrument not found"). We have it right for -Dcom.sun.management.*
> and -XX:+EnableJVMCI but not -javaagent. The reason for the strange
> error with -javaagent is because java agents are implemented as a JVM TI
> agent that is loaded early in VM startup, long before the root modules
> are resolved. I will create a bug for this.
>>
>> I argue that this outcome would still be rather suprising for most
>> Java users. Many monitoring tools use Java agents to hook into Java
>> processes and those tools have become omnipresent in most production
>> environments. Also, it is very unlikely that a main application module
>> would require the java.instrument module as it is only useful to Java
>> agents. As a result, Java agents and the tooling that relies on them
>> would stop working for the majority of jlink modules. Furthermore, the
>> agent attachment also fails if a Javaagent does not require the
>> Instrumentation interface what makes this problem even less intuitive.
>> (The reason being the lack of libinstrument.)
> The jlink user is a power user and so decides the modules and features
> to include in the run-time image.
> 
> If the jlink user doesn't select the VM type when creating the run-time
> image then the attach mechanism will work (because it is compiled into
> libjvm.so) and the troubleshooting tools should work fine. However some
> features that are exposed via the attach mechanism may not be present,
> specifically VirtualMachine.startManagementAgent and
> VirtualMachine.startLocalManagementAgent will fail if module
> jdk.management.agent is not included, and VirtualMachine.loadAgent will
> fail if module java.instrument is not included.
> 
> If the jlink user is targeting an embedded system or wants to create a
> minimal image to put in a docker container then maybe they select the
> minimal VM, in which case the serviceability features (including the VM
> side of the attach mechanism) are not included. It gets harder again for
> troubleshooting when additional jlink power options are used to strip
> debug attributes. That's the trade-off.
> 
> As I said, the goal has always been to allow someone create a run-time
> image that only contains java.base. I'm not sure that subsuming
> java.instrument into java.base is right. Introducing options to ask
> jlink to exclude modules creates the potential for conflict on the
> command line. One thing that jlink could do is emit a warning that the
> resulting run-time image doesn't have the management and instrumentation
> features, might that be the right balance.

yes, that seems the right way to solve this issue.

> 
>>
>> On the above machine, a jlink image is incapable of starting despite
>> the _JAVA_OPTIONS only specifying standard features that are
>> guaranteed by the JVM independently of a specific VM vendor.
> There isn't any guarantee, a point that was clarified in the
> java.instrument spec in Java SE 8 (JDK-8006565). The motivation for the
> clarification at the time was embedded deployments and the Compact
> Profiles defined for Java SE 8 (compact1 and compact2 didn't include the
> java.lang.instrument package).
> 
> -Alan

Rémi

Re: An alternative to "restricted keywords"

[Remi Forax]

I want to answer this before we start the meetings because i really think that 
restricted keyword as i propose solve the issues Stephan raised.


- Mail original -
> De: "Stephan Herrmann" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mardi 16 Mai 2017 11:49:45
> Objet: Re: An alternative to "restricted keywords"

> Thanks, Remi, for taking this to the EG list.
> 
> Some collected responses:
> 
> 
> Remi: "from the user point of view, '^' looks like a hack"
> 
> This is, of course, a subjective statement. I don't share this view
> and in years of experience with Xtext-languages (where this concept
> is used by default) I never heard any user complain about this.
> 
> More importantly, I hold that such aesthetic considerations are of
> much lesser significance than the question, whether we can explain
> - unambiguously explain - the concept in a few simple sentences.
> Explaining must be possible at two levels: in a rigorous specification
> and in simple words for users of the language.

I'm not against ^, or ` as it has already asked to escape an identifier, but as 
you said it's a pervasive change that applies on the whole grammar while i 
think that with restricted keyword (that really should be called local 
keywords) the changes only impact the grammar that specifies a module-info.java

> 
> Remi: "a keyword which is activated if you are at a position in the
>  grammar where it can be recognized".
> 
> I don't think 'being at a position in the grammar' is a good way of
> explaining. Parsing doesn't generally have one position in a grammar,
> multiple productions can be active in the same parser state.
> Also speaking of a "loop" for modifiers seems to complicate matters
> more than necessary.
> 
> Under these considerations I still see '^' as the clearest of all
> solutions. Clear as a specification, simple to explain to users.

Eclipse uses a LR parser, for a LR parser, position == dotted production as i 
have written earlier, so no problem because it corresponds to only one parser 
state.  Note that even if one do not use an LR or a LL parser, most hand 
written parser i've seen, javac is one of them, also refers to dotted 
production in the comments of the corresponding methods.

> 
> 
> 
> Peter spoke about module names vs. package names.
> 
> I think we agree, that module names cannot use "module words",
> whereas package names should be expected to contain them.

yes, that the main issue, package names may contains unqualified name like 
'transitive, ''with' or 'to'.
but i think people will also want to use existing package or more exactly 
prefix of existing package as module name, so we should also support having 
restricted keyword name as part of a module name.

The grammar is:

  open? module module_name {
requires (transitive | static)* module_name;
exports package_name;
exports package_name to module_name1, module_name2;
opens package_name;
opens package_name to module_name1, module_name2;
uses xxx;
provides xxx with xxx, yyy;
  }

If we just consider package name, only 'opens' and 'exports' are followed by a 
package name and a package name can only been followed by ';' or 'to', so once 
'opens' is parsed, you know that you can have only an identifier so if it's not 
an identifier by one of the restricted keywords, it should be considered as an 
identifier.

As i said earlier, the scanner can see the restricted keyword as keyword and 
before feeding the token to the parser, you can check the parser state to see 
if the keyword as to be lowered to an identifier or not.

For module name, there is the supplementary problem of transitive, because if a 
module starts with transitive, you can have a conflict. As i said earlier, 
instead of using the next token to know if transitive is the keyword or part of 
the module name, i think we should consider it as a keyword, as the JLS said a 
restricted keyword is activated when it can appear, so "requires transitive" is 
not a valid directive.

> 
> Remi: "you should use reverse DNS naming for package so no problem :)"
> 
> "to" is a "module word" and a TLD.
> I think we should be very careful in judging that a existing conflict
> is not a real problem. Better to clearly and rigorously avoid the
> conflict in the first place.

to as the first part of a package/module and to as in exports ... to can not be 
present on the same dotted production, because exports as to be followed by a 
package_name so 'to' here means the start of a package name and then because a 
package name can not ends with '.' you always know if you are inside the 
production recognizing the package_name or outside matching the to of the 
directive exports. 

> 
> 
> 
> Some additional notes from my side:
> 
> In the escape-approach, it may be prudent to technically allow
> escaping even words that are identifiers in Java 9, but could become
> keywords in a future version. This ensures that modules which need
> more escaping in Java 9+X can stil

Re: An alternative to "restricted keywords"

[Remi Forax]

Hi Peter,

On May 12, 2017 6:08:58 PM GMT+02:00, Peter Levart  
wrote:
>Hi Remi,
>
>On 05/12/2017 08:17 AM, Remi Forax wrote:
>> [CC JPMS expert mailing list because, it's an important issue IMO]
>>
>> I've a counter proposition.
>>
>> I do not like your proposal because from the user point of view, '^'
>looks like a hack, it's not used anywhere else in the grammar.
>> I agree that restricted keywords are not properly specified in JLS.
>Reading your mail, i've discovered that what i was calling restricted
>keywords is not what javac implements :(
>> I agree that restricted keywords should be only enabled when parsing
>module-info.java
>> I agree that doing error recovery on the way the grammar for
>module-info is currently implemented in javac leads to less than ideal
>error messages.
>>
>> In my opinion, both
>> module m { requires transitive transitive; }
>> module m { requires transitive; }
>> should be rejected because what javac implements something more close
>to the javascript ASI rules than restricted keywords as currently
>specified by Alex.
>>
>> For me, a restricted keyword is a keyword which is activated if you
>are at a position in the grammar where it can be recognized and because
>it's a keyword, it tooks over an identifier.
>> by example for
>>module m {
>> if the next token is 'requires', it should be recognized as a keyword
>because you can parse a directive 'required ...' so there is a
>production that will starts with the 'required' keyword.
>>
>> so
>>module m { requires transitive; }
>> should be rejected because transitive should be recognized as a
>keyword after requires and the compiler should report a missing module
>name.
>>   
>> and
>>module m { requires transitive transitive; }
>> should be rejected because the grammar that parse the modifiers is
>defined as "a loop" so from the grammar point of view it's like
>>module m { requires Modifier Modifier; }
>> so the the front end of the compiler should report a missing module
>name and a later phase should report that there is twice the same
>modifier 'transitive'.
>>
>> I believe that with this definition of 'restricted keyword', compiler
>can recover error more easily and offers meaningful error message and
>the module-info part of the grammar is LR(1).
>
>This will make "requires", "uses", "provides", "with", "to", "static", 
>"transitive", "exports", etc  all illegal module names. Ok, no big 
>deal, because there are no module names yet (apart from JDK modules and
>
>those are named differently). But...

you should use reverse DNS naming for module name, so no problem. 

>
>What about:
>
>module m { exports transitive; }
>
>Here 'transitive' is an existing package name for example. Who 
>guarantees that there are no packages out there with names matching 
>restricted keywords? Current restriction for modules is that they can 
>not have an unnamed package. Do we want to restrict package names a 
>module can export too?

you should use reverse DNS naming for package so no problem :)

>
>Stephan's solution does not have this problem.
>
>Regards, Peter

I think those issues are not real problem. 

Rémi 

>
>>
>> regards,
>> Rémi
>>
>> - Mail original -
>>> De: "Stephan Herrmann" 
>>> À: jigsaw-dev@openjdk.java.net
>>> Envoyé: Mardi 9 Mai 2017 16:56:11
>>> Objet: An alternative to "restricted keywords"
>>> (1) I understand the need for avoiding that new module-related
>>> keywords conflict with existing code, where these words may be used
>>> as identifiers. Moreover, it must be possible for a module
>declaration
>>> to refer to packages or types thusly named.
>>>
>>> However,
>>>
>>> (2) The currently proposed "restricted keywords" are not
>appropriately
>>> specified in JLS.
>>>
>>> (3) The currently proposed "restricted keywords" pose difficulties
>to
>>> the implementation of all tools that need to parse a module
>declaration.
>>>
>>> (4) A simple alternative to "restricted keywords" exists, which has
>not
>>> received the attention it deserves.
>>>
>>> Details:
>>>
>>> (2) The current specification implicitly violates the assumption
>that
>>> parsing can be performed on the basis of a tok

Re: Why not multiple modules of the same name in a class loader?

[Remi Forax]

On May 12, 2017 8:08:38 PM GMT+02:00, "David M. Lloyd"  
wrote:
>This has come up a couple times now and I'm not sure why the rule 
>exists: why not allow multiple modules with the same name in the same 
>class loader?

module names appear in the stack traces,  it will make life of people miserable 
if they have to open several artifacts to find the good one.

and if later the VM is able to isolate un-exported packages, it will be a real 
mess because at worst, you will have to follow all stack elements to find the 
class you want.


>
>As far as I can tell, there is no way to locate a module by class 
>loader, and module namespaces are generally already well-governed by 
>their layer, so from an API perspective at least, it seems like this is
>
>a harmless scenario (as long as there are no package conflicts, 
>obviously, but a container generally is able to manage this concern).
>
>Preventing this does block one potential workaround for the lack of a 
>ModuleLayer.Controller.addPackage() method, which would be to
>supplement 
>a module's content at run time by defining a new Layer that contains a 
>module of the same name as the original content within the same class 
>loader, where the container would then take care of such details as 
>mutual read/opening and cyclic access.
>
>So that made me curious: is there some hotspot-internal dictionary that
>
>I missed, which manages the set of modules within a class loader?

take a look to defineModule :)

Remi

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: An alternative to "restricted keywords"

[Remi Forax]

[CC JPMS expert mailing list because, it's an important issue IMO]

I've a counter proposition.

I do not like your proposal because from the user point of view, '^' looks like 
a hack, it's not used anywhere else in the grammar. 
I agree that restricted keywords are not properly specified in JLS. Reading 
your mail, i've discovered that what i was calling restricted keywords is not 
what javac implements :(
I agree that restricted keywords should be only enabled when parsing 
module-info.java
I agree that doing error recovery on the way the grammar for module-info is 
currently implemented in javac leads to less than ideal error messages.

In my opinion, both
   module m { requires transitive transitive; }
   module m { requires transitive; }
should be rejected because what javac implements something more close to the 
javascript ASI rules than restricted keywords as currently specified by Alex.

For me, a restricted keyword is a keyword which is activated if you are at a 
position in the grammar where it can be recognized and because it's a keyword, 
it tooks over an identifier.
by example for 
  module m { 
if the next token is 'requires', it should be recognized as a keyword because 
you can parse a directive 'required ...' so there is a production that will 
starts with the 'required' keyword.

so 
  module m { requires transitive; }
should be rejected because transitive should be recognized as a keyword after 
requires and the compiler should report a missing module name.
 
and
  module m { requires transitive transitive; }
should be rejected because the grammar that parse the modifiers is defined as 
"a loop" so from the grammar point of view it's like
  module m { requires Modifier Modifier; }
so the the front end of the compiler should report a missing module name and a 
later phase should report that there is twice the same modifier 'transitive'.

I believe that with this definition of 'restricted keyword', compiler can 
recover error more easily and offers meaningful error message and the 
module-info part of the grammar is LR(1).

regards,
Rémi

- Mail original -
> De: "Stephan Herrmann" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mardi 9 Mai 2017 16:56:11
> Objet: An alternative to "restricted keywords"

> (1) I understand the need for avoiding that new module-related
> keywords conflict with existing code, where these words may be used
> as identifiers. Moreover, it must be possible for a module declaration
> to refer to packages or types thusly named.
> 
> However,
> 
> (2) The currently proposed "restricted keywords" are not appropriately
> specified in JLS.
> 
> (3) The currently proposed "restricted keywords" pose difficulties to
> the implementation of all tools that need to parse a module declaration.
> 
> (4) A simple alternative to "restricted keywords" exists, which has not
> received the attention it deserves.
> 
> Details:
> 
> (2) The current specification implicitly violates the assumption that
> parsing can be performed on the basis of a token stream produced by
> a scanner (aka lexer). From discussion on this list we learned that
> the following examples are intended to be syntactically legal:
>module m { requires transitive transitive; }
>module m { requires transitive; }
> (Please for the moment disregard heuristic solutions, while we are
>  investigating whether generally "restricted keywords" is a well-defined
>  concept, or not.)
> Of the three occurrences of "transitive", #1 is a keyword, the others
> are identifiers. At the point when the parser has consumed "requires"
> and now asks about classification of the word "transitive", the scanner
> cannot possible answer this classification. It can only answer for sure,
> after the *parser* has accepted the full declaration. Put differently,
> the parser must consume more tokens than have been classified by the
> Scanner. Put differently, to faithfully parse arbitrary grammars using
> a concept of "restricted keywords", scanners must provide speculative
> answers, which may later need to be revised by backtracking or similar
> exhaustive exploration of the space of possible interpretations.
> 
> The specification is totally silent about this fundamental change.
> 
> 
> (3) "restricted keywords" pose three problems to tool implementations:
> 
> (3.a) Any known practical approach to implement a parser with
> "restricted keywords" requires to leverage heuristics, which are based
> on the exact set of rules defined in the grammar. Such heuristics
> reduce the look-ahead that needs to be performed by the scanner,
> in order to avoid the full exhaustive exploration mentioned above.
> A set of such heuristic is extremely fragile and can easily break when
> later more rules are added to the grammar. This means small future
> language changes can easily break any chosen strategy.
> 
> (3.b) If parsing works for error-free input, this doesn't imply that
> a parser will be able to give any useful answer for input with sy

Re: Need help implementing Java modules

[Remi Forax]

On May 10, 2017 2:20:31 AM GMT+02:00, Ralph Goers  wrote:
>
>> On May 9, 2017, at 3:39 PM, Alex Buckley 
>wrote:
>> 
>> On 5/9/2017 3:04 PM, Ralph Goers wrote:
>>> Pardon me for being dense, but my reading said that Java modules
>>> disallowed runtime cycles as well as compile time. Once LoggerFinder
>>> binds with the module that provides that service does that not
>create
>>> a runtime dependency?  I don’t recall seeing anything describing
>what
>>> the behavior of that is.
>> 
>> The module system disallows cycles in the 'requires' directives of
>module declarations. The module system allows cycles in the "reads"
>relation of run-time modules.
>> 
>> When java.base 'uses LoggerFinder;' and invokes ServiceLoader to find
>providers, there is indeed a "reads" cycle created between the provider
>module and java.base. ServiceLoader is not special in creating this
>cycle -- you can create them yourself with the two addReads methods in
>the API, and all automatic modules have cyclic readability. But there
>is no 'requires' cycle.
>> 
>> Alex
>> 
>
>Thanks Alex, that would indeed be enough to break the cycle as
>log4j-api can easily use serviceLoader to bind with log4j-core as that
>is a very minor change. However your previous response leads me to
>another question.
>
>Log4j already has a robust plugin approach for users to implement their
>own Appenders, Filters, Layouts, and other Log4j components. We are not
>going to modify that as it would severely impact our users who have
>already implemented custom components and what we have works very well.
>Although the FlumeAppender I mentioned previously is provided by Log4j
>it, and all other Log4j components, are located via an annotation
>processor provided by Log4j. The processor runs at compile time and
>generates a file named
>META-INF/org/apache/logging/log4j/core/config/plugins/Log4j2Plugins.dat.
>All of these files are then located at runtime using
>ClassLoader.getResources(). As we parse the configuration provided by
>the user we instantiate the plugins using the class names provided in
>those files and the static builder or static factory method provided in
>the plugin class. From what I am reading in the javadoc we should not
>have any trouble with this because META-INF is not a valid package name
>so the resource file will not be hidden. Log4j uses reflection to call
>the static builder or static method to create the plugin.  
>
>With all this in mind, if users create modules will they be required to
>declare the packages where they have created plugins as “open” to log4j
>for this to work?

It depends if when you call the static method you need to bypass the 
encapsulation or not i.e. if your current code uses setAccessible, yes, the 
plugin's module has to be opened.

In any case, you need to add a read edge at runtime from log4j to the plugin, 
otherwise you will not find the plugins class.

> I am assuming that Log4j will be able to access this
>module without having to declare a dependency on it? 

yes, if you add a read edge at runtime.

>
>Thanks,
>Ralph

cheers,
Rémi 

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

--ignore-signing-information do not suppress all errors

[Remi Forax]

Hi all,
I wonder if it's an issue or if there is a rational for not suppressing this 
error when using --ignore-signing-information.

I'm patching the compiler related modules of the jdk9 with the one from 
amber/langtools, so i know that i'm doing something borderline but i gently ask 
jlink to do not bother me with the hash signature.

...
jlink --module-path ./target/main/artifact:./deps:/usr/jdk/jdk-9-b167/jmods
  --add-modules 
jdk.policytool,jdk.charsets,jdk.pack,jdk.xml.dom,java.xml,jdk.deploy,java.datatransfer,java.jnlp,java.desktop,com.github.forax.pro.uberbooter,java.se,jdk.snmp,java.security.sasl,jdk.deploy.controlpanel,jdk.zipfs,java.base,jdk.crypto.ec,com.github.forax.pro.main,jdk.management.agent,jdk.jshell,jdk.editpad,oracle.desktop,jdk.jsobject,jdk.unsupported,java.smartcardio,jdk.scripting.nashorn,java.security.jgss,com.github.forax.pro.plugin.convention,jdk.dynalink,java.activation,java.sql,java.logging,jdk.jfr,jdk.internal.vm.ci,com.github.forax.pro.plugin.packager,jdk.packager.services,jdk.net,javafx.controls,jdk.internal.ed,jdk.compiler,jdk.naming.rmi,jdk.jconsole,jdk.internal.le,jdk.packager,jdk.jdwp.agent,jdk.internal.vm.compiler,com.github.forax.pro.daemon.imp,java.scripting,jdk.jartool,java.rmi,jdk.jdi,jdk.javaws,jdk.rmic,com.github.forax.pro.plugin.resolver,jdk.jstatd,jdk.httpserver,java.se.ee,jdk.jcmd,javafx.base,com.github.forax.pro.plugin.linker,com.github.forax.pro.plugin.compiler,jdk.plugin.dom,com.github.forax.pro.plugin.modulefixer,jdk.javadoc,jdk.xml.ws,java.sql.rowset,jdk.sctp,javafx.swing,jdk.jlink,jdk.scripting.nashorn.shell,oracle.net,java.xml.bind,java.compiler,javafx.graphics,javafx.fxml,jdk.plugin,javafx.media,jdk.accessibility,jdk.security.jgss,javafx.web,com.github.forax.pro.plugin.uberpackager,jdk.hotspot.agent,javafx.deploy,java.xml.crypto,jdk.incubator.httpclient,jdk.plugin.server,jdk.crypto.cryptoki,java.naming,java.prefs,jdk.internal.opt,jdk.attach,java.transaction,java.xml.ws,java.xml.ws.annotation,java.management,jdk.xml.bind,jdk.internal.jvmstat,java.instrument,jdk.management,jdk.security.auth,jdk.jdeps,jdk.aot,java.corba,java.management.rmi,jdk.naming.dns,jdk.localedata
  --launcher pro=com.github.forax.pro.main/com.github.forax.pro.main.Main
  --compress 0
  --ignore-signing-information
  --output ./target/image
Error: Hash of jdk.javadoc 
(5b3a628bce2aad807392ea8a3f2ce69a652e885e33db586694b47fcf6f5ebf52) differs to 
expected hash 
(6bdf7f26f72c9f102c5052f8ddf68a80a4b18f997a16e56805e3acc7347469b5) recorded in 
java.base

cheers,
Rémi

Re: Revised proposal for #AutomaticModuleNames

[Remi Forax]

Hi Robert,

- Mail original -
> De: "Robert Scholte" 
> À: "mark reinhold" 
> Cc: jigsaw-dev@openjdk.java.net
> Envoyé: Vendredi 5 Mai 2017 12:04:15
> Objet: Re: Revised proposal for #AutomaticModuleNames

> Hi Mark,
> 
> thanks for these adjustments. In general they look good, but it all
> depends on the details:
> 
> Define a JAR-file manifest attribute, `Automatic-Module-Name`, whose
> value is used as the name of the automatic module defined by that JAR
> file when it is placed on the module path, as previously suggested
> [2][3].  If a JAR file on the module path does not have such a
> manifest attribute then its automatic-module name is computed using
> the existing filename-based algorithm.
> 
> Just to be clear: does this make it an auto module? In that case we're
> still a bit stuck, because I still insist that jars published to any
> public artifact repository should never refer to an automodule. Published
> jars with this attribute should have received this name by their developer
> and this also implies that the jar is Jigsaw-ready, e.g. no more split
> packages. In the future its users should be able to switch to the fully
> modular jar with the same name without any problem. So I need an extra
> state, e.g. isAutomatic() + isNamed(), in which case I have to change my
> insisting line: "Never publish jars to any public artifact repository that
> refers to *unnamed* automatic modules".

yes, knowing if the name is auto-calculated or not may be interesting,
that's said, you can already get the manifest file from the ModuleReader and 
check if there is an entry named Automatic-Module-Name so i'm not sure 
surfacing isNamed() in the API is a good idea.

> 
> A user of a library can suggest a stable name to the library's
> maintainer, and easily submit a patch that implements it.  This will
> enable the uncoordinated modularization of libraries across the
> entire ecosystem.
> 
> I don't think this is realistic. Jars live in the local repository and
> will stay there even during compilation. Is a user patches such jars,
> it'll effect all his local Java projects. Only when Maven will do both
> compiling and generation the distributable/deployable one could isolate
> jars. 

[...]

I think there is a misunderstanding here, 
i read 'easily submit a patch' as 'easily submit a pull request' that will 
change the source so the next time the jar is published on central, it will 
have a module name in its manifest.


> 
> thanks,
> Robert

Rémi

> 
> On Thu, 04 May 2017 19:39:06 +0200,  wrote:
> 
>> Thanks to everyone, and especially Stephen Colebourne, Brian Fox, and
>> Robert Scholte, for the extensive feedback.
>>
>>   
>> http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000687.html
>>
>> TL;DR: Keep automatic modules, bring back the module-name JAR-file
>> manifest attribute, and strongly recommend reverse-DNS module names.
>>
>> Comments?
>>
> > - Mark

Re: Revised proposal for #AutomaticModuleNames

[Remi Forax]

On May 4, 2017 11:21:38 PM GMT+02:00, Stephen Colebourne  
wrote:
>On 4 May 2017 at 18:38,   wrote:

...

>>   - To increase awareness of when automatic modules are used, and of
>the
>> consequences of their use, suggest that Java language compilers
>issue
>> two new types of warnings, and implement these warnings in the
>RI:
>
>Perhaps these should be reconsidered? If the module depends on an
>automatic module with a module name in the manifest, it can be
>considered reasonably stable. Whereas anything depending on a filename
>derived module name is unstable. That isn't being captured by the
>warnings.

an automatic module is still not stable because it exports all packages by 
default while a real module may only export some of them.

 
>
>Stephen

Rémi 

>
>
>On 4 May 2017 at 18:39,   wrote:
>> Thanks to everyone, and especially Stephen Colebourne, Brian Fox, and
>> Robert Scholte, for the extensive feedback.
>>
>>  
>http://mail.openjdk.java.net/pipermail/jpms-spec-experts/2017-May/000687.html
>>
>> TL;DR: Keep automatic modules, bring back the module-name JAR-file
>> manifest attribute, and strongly recommend reverse-DNS module names.
>>
>> Comments?
>>
>> - Mark

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Java Platform Module System

[Remi Forax]

It's context-free because a context free grammar defined its input in term of 
terminals and the theory do not say how to map a token to a terminal.

Jay is right that it requires to use either some specific parser generator like 
Tatoo [1] the one i've written 10 years ago (because i wanted the tool to help 
me to extend a grammar easily) or to modify an existing parser generator so the 
parser can send the production state to the lexer which will enable/disable the 
automata that recognize the associated keywords .

Rémi

[1] http://dl.acm.org/citation.cfm?id=1168057

- Mail original -
> De: "Alex Buckley" 
> À: "Jayaprakash Arthanareeswaran" , "Dan Smith" 
> , "Brian Goetz"
> 
> Cc: jigsaw-dev@openjdk.java.net
> Envoyé: Mercredi 3 Mai 2017 19:46:54
> Objet: Re: Java Platform Module System

> On 5/2/2017 3:39 PM, Alex Buckley wrote:
>> On 5/2/2017 7:07 AM, Jayaprakash Arthanareeswaran wrote:
>>> Chapter 2 in [1] describes context-free grammars. The addition to "3.9
>>> Keywords" defines "restricted keywords", which prevent the grammar for
>>> ModuleDeclaration from being context-free. This prevents compilers from
>>> using common parser generators, since those typically only support
>>> context-free grammars. The lexical/syntactic grammar split defined in
>>> chapter 2 is not of much use for actual implementations of
>>> module-info.java parsers.
>>> The spec at least needs to point out that the given grammar for
>>> ModuleDeclaration is not actually context-free.
>>
>> The syntactic grammar in JLS8 was not context-free either; the opening
>> line of Chapter 2 has been false for years. For JLS9, I will remove the
>> claim that the lexical and syntactic grammars are context-free, and
>> perhaps a future JLS can discuss the difficulties in parsing the
> 
> Jan Lahoda pointed out privately that the syntactic grammar in JLS8 and
> JLS9 is in fact context-free -- it's just not LL(1). Not being LL(1) is
> what I should have said the grammar hasn't been for a long time.
> 
> Alex

Re: Accessing module internals from bytecode rewriting agent

[Remi Forax]

Hi Kirk,
--patch-module both at compile time and at runtime is what your are looking for.

cheers,
Rémi

- Mail original -
> De: "Kirk Pepperdine" 
> À: "Alan Bateman" 
> Cc: "jigsaw-dev" , core-libs-...@openjdk.java.net
> Envoyé: Mardi 2 Mai 2017 19:50:50
> Objet: Re: Accessing module internals from bytecode rewriting agent

> Hi Alan,
> 
> One of the techniques that I’ve used in the past for troubleshooting is to
> prepend modified classes to the bootstrap class path. I fear all this mucking
> about with modularization is going to leave the road to diagnostics littered
> with a ton of tools that will not longer work. As it is, 8 broke all the
> techniques and tooling used to diagnose class loader leaks… without fixing the
> fundamental problems that caused them to leak in the first place..and added a
> new sources for performance regressions for which there is little in the way 
> of
> tooling and certainly much less of an understanding of the fundamental issues
> out in the general developer population. The availability of high quality
> tooling is a huge bonus in the Java eco structure…. Please, don’t send us back
> to the Stone Age.
> 
> Kind regards,
> Kirk Pepperdine
> 
>> On May 2, 2017, at 9:48 AM, Alan Bateman  wrote:
>> 
>> On 02/05/2017 07:50, Jeremy Manson wrote:
>>> :
>>> 
>>> If we follow this path, before we migrate to Java 9, we would need to make 
>>> sure
>>> all of our code builds and the tests pass with Java 9 and Java 8.  We can't
>>> make all of the code build and the tests pass with Java 9 as-is, because 
>>> many
>>> of them use options like Xbootclasspath, which have been dropped.  We can't
>>> migrate those tests to use the new command line flags before we migrate to 
>>> Java
>>> 9, because if we do, they will stop working with Java 8.  Due to the size of
>>> our code base, it's pretty much impossible to migrate all of the users to a 
>>> new
>>> set of command line flags at the same time as we migrate to Java 9.
>> So I'm curious about -Xbootclasspath as that completely overrides the runtime
>> classes. Is this static instrumentation or combining the VM/launcher from one
>> build with the classes from a different JDK build? In any case, and as you 
>> have
>> found, -Xbootclasspath  and -Xbootclasspath/p are meaningless now, a
>> consequence of moving all the core APIs to modules. The -Xbootclasspath/a
>> option works as before with the caveat that not all platform classes are
>> visible to the boot class loader now (a consequence of ongoing security work 
>> to
>> move non-core components away from the fully privileged boot loader).
>> 
>> It might be that you mean the `javac -bootclasspath,  this works as before 
>> when
>> you target 8 or older.
>> 
>> -Alan

Re: Alternatives for naming automatic modules, and a proposal (#AutomaticModuleNames)

[Remi Forax]

- Mail original -
> De: "mark reinhold" 
> À: "Brian Fox" 
> Cc: jigsaw-dev@openjdk.java.net
> Envoyé: Lundi 24 Avril 2017 20:54:18
> Objet: Re: Alternatives for naming automatic modules, and a proposal 
> (#AutomaticModuleNames)

> 2017/4/24 10:39:25 -0700, Brian Fox :
>> On Mon, Apr 24, 2017 at 12:14 PM, mark.reinh...@oracle.com wrote:
>>> The problem with any approach that allows you to give a stable name to
>>> what is otherwise an automatic module is that the API of that module is
>>> inherently unstable.  If and when the module is completely modularized
>>> then its API will almost certainly change, which in turn will require
>>> incompatible changes to any previously-published fully-modularized
>>> modules that depend upon it.
>> 
>> I don't quite follow, why does modularizing a module imply its public api
>> will change?
> 
> The effective API of a module is determined by the packages that it
> exports and, also, by the packages exported by the modules it depends
> upon, directly and indirectly, via `requires transitive` clauses.  The
> effective API of an existing library that's used as an automatic module
> is inherently unstable in two ways:
> 
>  - When you modularize the library explicitly, with a `module-info.java`
>file, you'll have to decide which packages to export.  Packages that
>are meant only for internal use should not be exported, but when your
>library was used as an automatic module those packages were exported.
>If you choose not to export those packages when you modularize then
>code in other modules that references types in those packages will
>break.
> 
>  - When you modularize the library explicitly you'll also have to decide
>which modules it requires transitively.  This will almost certainly
>further reduce the number of packages accessible to modules that use
>your module, and perhaps dramatically so: When your library was an
>automatic module it transitively required every other automatic
>module observable at compile time or run time, thus making every
>package of every automatic module accessible to modules that use your
>module.  Even worse, this set of packages depended upon the context
>in which the library was used; you can't compute it just by examining
>the library and its known dependencies.
> 
> An explicit module that depends upon one or more modules that are
> automatic today is, itself, no more stable than those automatic modules.
> It could be broken when those automatic modules are modularized
> explicitly, and if it `requires transitive` an automatic module then any
> modules that depend upon it could be broken when that automatic module
> is modularized explicitly.
> 
> If we define a way to give stable names to automatic modules, as the
> `Module-Name` manifest attribute would do, then that would tend to
> encourage the publication of named automatic modules and, further, the
> publication of explicit modules that depend upon them.  All of these
> modules would rest on shaky ground, if not actual quicksand.
> 
> If we encourage developers at large to take this approach then I worry
> that many will give up in frustration when they try to modularize their
> named automatic modules, later on, with explicit `module-info.java`
> files.  We'd wind up, in the long run, with public repositories that
> contain lots of named automatic modules but few actual explicit modules,
> when what we want to see is lots of explicit modules.  The best way to
> attain that long-term goal is to minimize the chances for pain and
> confusion along the way, and if that means a slower pace of migration
> then so be it.

It will be worst than that.

People want module resolution to be magic, it's what Maven or Gradle provide,
specifies your direct dependencies, we will care about the rest.

If build tools are not able to see existing plain old jars as modules, people 
will write plugins to work aroud that,
after all, Maven at its core is a tool that patch the deficiency of the 
classpath by linearizing the graph of dependencies,
so plugins will patch the non modular jars to make them modular and nobody will 
care how this is done.

In my opinion, having a tool that automatically modularize a bunch of jars is 
far worst that having authors of modules to delay the time to go full module.


Now about the stability of an automatic module. I don't think we have to care 
about that.
When migrating to the module world, either it's easy to wrap an existing code 
into a module and everybody is happy,
or it's hard and people will make a breaking change, create a new name (like 
common-collection4), and wrap the updated API into a new module.
Unlike with the JDK, as Brian said, people are used to upgrade (not update) 
their dependencies.


I believe that we should provide a way to be able to require a non modular jar 
with a stable name, so new project library or application will be able to use 
old jars as module rapidly,
oth

Re: Alternatives for naming automatic modules, and a proposal (#AutomaticModuleNames)

[Remi Forax]

- Mail original -
> De: "David M. Lloyd" 
> À: "Brian Fox" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 25 Avril 2017 16:38:54
> Objet: Re: Alternatives for naming automatic modules, and a proposal 
> (#AutomaticModuleNames)


[...]

> 
> OSGi, by the way, avoids this situation somewhat more effectively than
> Jigsaw can, mainly because of three factors: resolving by package
> instead of by bundle name, performing the bulk of resolution work at run
> time, and by the simple fact that it is less invasive towards
> reflection, thus there is more "wiggle room".
> 
> This isn't to say that OSGi is an ideal solution of course, but I just
> wanted to point out why the OSGi ecosystem can generally get away with
> publishing artifacts containing descriptor information, while Jigsaw
> probably won't be able to (or at least, not as cleanly).

yes, 
you need a way to merge an artifact descriptor and its corresponding 
non-modular jar before the VM checks the modules configuration.

as you said, adding an assembly step after the modules being resolved and 
downloaded locally and before the VM is started is a solution.
At that point, you can either rename the automatic modules and put the internal 
dependencies of that module in the classpath or you can rewrite the module 
infos by doing a static analysis (+ some manual info because of the reflection).

> 
> --
> - DML

Rémi

Re: Alternatives for naming automatic modules, and a proposal (#AutomaticModuleNames)

[Remi Forax]

About multiple versions or true hiding of non exported packages, I do not 
believe that someone is against those features.

But they both requires to change the VM in an extensive way and to my knowledge 
nobody has ever done a prototype of those features.

The good news is that both these features can be introduced later in a 
compatible way.

I would love to see Oracle or RedHat to put engineering ressources on these 
features, but until that, the reality is that those features are not blockers 
to the release of JPMS*

Rémi 
* given the tumultuous past of the modules in Java, being able to release 
something even if not perfect, is already a victory.







On April 25, 2017 3:16:54 PM GMT+02:00, Michael Nascimento  
wrote:
>On Tue, Apr 25, 2017 at 10:10 AM, David M. Lloyd
>
>wrote:
>
>> Just the fact that there is the *very idea* of a "fit"/"non-fit" for
>JPMS
>> is sad though.  It should have been the ubiquitous thing that
>everyone was
>> expecting.  But denying multiple versions?  Blowing up on run time
>cycles?
>> Reneging on the idea of being the basis of Java EE 9? These are
>things that
>> people will not be expecting.
>
>
>Some extra feedback: I delivered a talk yesterday at QCon SP and people
>were very much disappointed with these limitations, and also the fact
>packages must be defined by a single module even if they are not
>exported,
>because it was considered nearly impossible to track in real world
>scenarios.
>
>Regards,
>Michael

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Module naming restrictions

[Remi Forax]

I was thinking of (a), an automatic module should not allow Java keywords, 
apart all methods ModuleDescriptor.new*Module should checks that each component 
of the qualified name is not a Java keyword.

The idea is that all variants of ModuleDescriptor.new*Module implements the JLS 
semantics and ModuleDescriptor.read() implements the JVMS semantics.

Rémi

- Mail original -
> De: "Stephan Herrmann" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mardi 25 Avril 2017 13:49:20
> Objet: Re: Module naming restrictions

> When you say "bug" do you mean
> (a) the name of that automatic module should avoid any Java keyword, or
> (b) Java keywords should be accepted as (part of) a module name?
> 
> Option (a) would require a change in the specification of ModuleFinder
> whereas (b) currently conflicts with JLS 7.7 which clearly states
>   A module name consists of one or more Java identifiers (§3.8) separated by 
> "."
>   tokens.
> 
> While neither (a) nor (b) is fulfilled, libraries with a Java keyword in their
> name
> simply cannot be used as automatic modules, right?
> 
> Stephan
> 
> 
> On 25.04.2017 09:50, Remi Forax wrote:
>> Seems to be a bug to me.
>>
>> Rémi
>>
>> - Mail original -
>>> De: "Rafael Winterhalter" 
>>> À: "jigsaw-dev" 
>>> Envoyé: Mardi 25 Avril 2017 09:12:04
>>> Objet: Module naming restrictions
>>
>>> Hello,
>>>
>>> I did another update of my libraries to support Java 9 and found out that
>>> Byte Buddy cannot easily be used as an automatic module. It is published as
>>> byte-buddy.jar to Maven Central where its automatic module name would be
>>> byte.buddy. This does however not compile due to "byte" being a keyword in
>>> Java. Here is what javac gives me:
>>>
>>> src/module-info.java:2: error:  expected
>>>requires byte.buddy;
>>>^
>>> 1 error
>>>
>>> I expect that this is not a common problem but not an uncommon one either.
>>> Looking for artifacts named after Java keywords in Maven Central, you can
>>> find quite a few.
>>>
>>> I do not know if this is technically feasible but I suggest that module
>>> names should not come with any restrictions due to the historic freedom in
>>> naming which would just cause confusion if some projects had to be renamed
>>> just to satisfy the module descriptors restrictions.
>>>
> >> Best regards, Rafael

Re: Accessing module internals from bytecode rewriting agent

[Remi Forax]

[...]

> 
> Meanwhile, Mandy Chung has kindly offered to look into the security
> considerations that are at play and come up with a less restrictive
> policy which enforces the security needs more accurately. At that point
> I will probably remove the fallback -- in part because I hope that by
> then both reflection and method handles will employ a common blacklist
> of inaccessible methods. I think it would make sense for them both to
> offer equivalent access and for that access to take security seriously.

I've seen this kind of fallback in different language runtimes, mine included,
having a "less blunt" checks would be great.

> 
> regards,
> 
> 
> Andrew Dinn

Rémi

Re: Getting the automatic module name of non-modular JAR

[Remi Forax]

If you want an Optional, you can use findFirst() on a stream,
  Optional ref = ModuleFinder.of( jar 
).findAll().stream().findFirst();

Rémi

- Mail original -
> De: "Gunnar Morling" 
> À: "Alan Bateman" 
> Cc: "jigsaw-dev" 
> Envoyé: Mardi 25 Avril 2017 09:10:45
> Objet: Re: Getting the automatic module name of non-modular JAR

> I see; thanks, Alan.
> 
> I wanted to avoid using a regex or similar, in order to make sure the
> JDK's own automatic naming rules are applied instead of
> "re-implementing" them. I was kinda hoping for a method like
> 
>Path jar = ...;
>Optional ref = ModuleReference.of( jar );
> 
> 
> 2017-04-25 8:49 GMT+02:00 Alan Bateman :
>> On 24/04/2017 21:23, Gunnar Morling wrote:
>>
>>> Hi,
>>>
>>> Given a non-modular JAR (e.g. represented as Path), what's the easiest
>>> way to obtain the automatic module name derived for this JAR?
>>
>> If you just want the name then it might be more efficient to do it with a
>> regular expression.
>>
>>>
>>> I found the following:
>>>
>>>  Path nonModularJar = ...;
>>>  String automaticModuleName = ModuleFinder.of( nonModularJar )
>>>  .findAll()
>>>  .iterator()
>>>  .next()
>>>  .descriptor()
>>>  .name();
>>>
>>> Is this the best I can do?
>>>
>>> More generally speaking, is using ModuleFinder with a single path the
>>> only way to obtain a ModuleReference/ModuleDescriptor for a specific
>>> JAR?
>>
>> Yes, ModuleFinder is the only way (it might be more succulent to use stream
>> + findFirst but that is just detail). If you are only interested in the name
>> then you could of course open the JAR file. If it contains module-info.class
>> then read it with ModuleDescriptor.read, otherwise use a regex to derive the
>> name.
>>
> > -Alan

Re: Module naming restrictions

[Remi Forax]

Seems to be a bug to me.

Rémi

- Mail original -
> De: "Rafael Winterhalter" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 25 Avril 2017 09:12:04
> Objet: Module naming restrictions

> Hello,
> 
> I did another update of my libraries to support Java 9 and found out that
> Byte Buddy cannot easily be used as an automatic module. It is published as
> byte-buddy.jar to Maven Central where its automatic module name would be
> byte.buddy. This does however not compile due to "byte" being a keyword in
> Java. Here is what javac gives me:
> 
> src/module-info.java:2: error:  expected
>requires byte.buddy;
>^
> 1 error
> 
> I expect that this is not a common problem but not an uncommon one either.
> Looking for artifacts named after Java keywords in Maven Central, you can
> find quite a few.
> 
> I do not know if this is technically feasible but I suggest that module
> names should not come with any restrictions due to the historic freedom in
> naming which would just cause confusion if some projects had to be renamed
> just to satisfy the module descriptors restrictions.
> 
> Best regards, Rafael

Re: setAccessible() alternative with Jigsaw - feedback on Lookup

[Remi Forax]

- Mail original -
> De: "Matej Novotny" 

> Hi,
> 
> thanks for your time. Comments inline.
> 

[...]

>> 
>> If you are looking to define a class in a new runtime package and the
>> class loader that is not your implementation then you are out of luck.
>> Lookup is designed for doing computations in the context of an existing
>> class. There is a note in the archives here from John Rose about
>> extending the lookup API for cases like this but it would come with a
>> lot of complexity. Also there isn't any notion of adding a package to a
>> named module at this time. When java.lang.reflect.Proxy spins proxy
>> classes then it doesn't add packages to named modules, it instead spins
>> the proxy classes into its own module (something you could do too,
>> assuming the classes that you are proxying are accessible).
> 
> I think it's more or less what we did/are doing - we spin the proxies either
> into the same package the came from or to our own package/module.
> But in order to avoid proxy name clash in the same package, we just took the
> original package and replaced the prefix 'java' with something like
> 'org.jboss.weld.proxies'.
> This effectively means that 'java.util.Map' proxy will end up as something 
> like
> 'org.jboss.weld.proxies.util.Map$Proxy'.
> Obviously, 'org.jboss.weld.proxies.util' did not exist beforehand and putting
> everything into one existing package using Lookup is bound to blow up very
> quickly.
> So there is basically no way to achieve this with JDK 9?

You can spin one module per package, by creating one ModuleLayer per package, 
not unlike j.l.r.Proxy does.

Rémi

> 
>> 
>> >
>> > 3) All in all the changes carried along a lot of complexity compared to
>> > plain old ClassLoader.defineClass
>> > We need to create A LOT of Lookups (might present performance overhead,
>> > haven't tested yet).
>> > Extra care needs to be given to the packages for Lookup not to blow up
>> > while we in fact don't really care where the proxy lands as long as it is
>> > usable.
>> Can you expand on the "don't really care" comment? Do you mean that you
>> don't care about either the defining loader or runtime package?
> 
> I was referring to the runtime package.
> What we need is to ensure that there won't be a clash - there can be two 
> classes
> with same name and different packages, which, for some reason, will have to
> land in a package created by us.
> If this happened and we were using a fixed existing package to place them in, 
> it
> would blow up.
> So what I meant was that with JDK 8 I don't need to care for this, I just use
> prefix and I am done with it. In JDK 9, this seems like quite an obstacle.
> 
>> 
>> >
>> >
>> > Another nasty thing is that the code of course needs to work with both, JDK
>> > 9 and 8.
>> > While it isn't impossible, it will add a not-so-nice reflection magic layer
>> > to the mix.
>> >
>> Multi-release JARs (JEP 238 [3]) and the ability to compile to older
>> releases (JEP 247 [4]) might be useful here (you might know about these
>> features already).
> 
> Haven't really explored this yet, it might be a way.
> Gotta see how Maven deals with this feature to allow us to create MRJARs in a
> reasonable manner.
> 
> Matej
> 
>> 
>> -Alan.
>> 
>> [1] https://bugs.openjdk.java.net/browse/JDK-8171335
>> [2] http://openjdk.java.net/jeps/181
>> [3] http://openjdk.java.net/jeps/238
>> [4] http://openjdk.java.net/jeps/247

Re: Changing the default file system provider is broken :(

[Remi Forax]

- Mail original -
> De: "Alan Bateman" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Dimanche 23 Avril 2017 12:38:21
> Objet: Re: Changing the default file system provider is broken :(

> On 22/04/2017 21:48, Peter Levart wrote:
> 
>>
>> Well, it works at least when
>> -Djava.nio.file.spi.DefaultFileSystemProvider=... is specified on the
>> command line. But when the system property is set as 1st thing in
>> main() method, it is already too late. It seems that after boot
>> sequence is finished (VM.isBooted() == true)and before main() method
>> is called, some code needs the FileSystem again, so default file
>> system is already initialized before the main() is executed.
>>
>> Perhaps there needs to be another initLevel which would be set just
>> before the invocation of the main() method?
> I don't think we go there as there are just too many scenarios where
> "user code" can execute before the application main method. Instead, I
> think this is a case where the default file system provider can only be
> overridden by setting the property on the command line.

This should be added to the compatibility list because mocking the FileSystem 
for testing purpose is common, at least in my own bubble.

> 
> -Alan

Rémi

Changing the default file system provider is broken :(

[Remi Forax]

Ooops !
if you use the module path, you can not change the default system provider 
mechanism anymore,
mostly because the module implementation internally uses java.nio.file.Path at 
different places.

First, trying to initialize the default system provider in Java code do not 
work at all,
calling
  System.setProperty("java.nio.file.spi.DefaultFileSystemProvider", "...");
in a main does nothing because the jigsaw implementation has already loaded the 
default implementation to be able to inspect the modules.

Then, if you try to use the command line
 -Djava.nio.file.spi.DefaultFileSystemProvider=...
you have various bootstrapping issues.

With exploded modules, when trying to initialize the default file system, the 
part of jigsaw that reads exploded modules uses path.toFile() that fails it 
checks that the path belongs to the default file system which is not already 
initialized
(see below for the whole stacktrace)

With modular jars, when trying to initialize the default file system, the 
JarModuleReader also uses path.toFile(), leading to the same error

One way to solve that is to change the code in FileSystems.getDefault() to add 
a nullcheck,
FileSystem defaultFileSystem = 
DefaultFileSystemHolder.defaultFileSystem;
if (jdk.internal.misc.VM.isBooted() && defaultFileSystem != null) {
return defaultFileSystem;
} else {
return BuiltinFileSystemHolder.builtinFileSystem;
}
Another solution is to change the code of jigsaw to only use the 
builtinFileSystem, delaying the initialization of the default system which will 
also solve the case where an application uses System.setProperty.

cheers,
Rémi


Stacktrace with exploded modules:
Error: A JNI error has occurred, please check your installation and try again
Exception in thread "main" java.lang.Error: 
java.lang.UnsupportedOperationException: Path not associated with default file 
system.
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:139)
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.access$100(FileSystems.java:100)
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:109)
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:107)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:107)
at 
java.base/java.nio.file.FileSystems$DefaultFileSystemHolder.(FileSystems.java:101)
at java.base/java.nio.file.FileSystems.getDefault(FileSystems.java:188)
at java.base/java.nio.file.Path.toFile(Path.java:655)
at 
java.base/jdk.internal.module.ModuleReferences$JarModuleReader.newJarFile(ModuleReferences.java:229)
at 
java.base/jdk.internal.module.ModuleReferences$JarModuleReader.(ModuleReferences.java:239)
at 
java.base/jdk.internal.module.ModuleReferences.lambda$newJarModule$0(ModuleReferences.java:96)
at 
java.base/jdk.internal.module.ModuleReferenceImpl.open(ModuleReferenceImpl.java:88)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.createModuleReader(BuiltinClassLoader.java:953)
at 
java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1719)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.moduleReaderFor(BuiltinClassLoader.java:945)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.defineClass(BuiltinClassLoader.java:702)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.findClassInModuleOrNull(BuiltinClassLoader.java:651)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.findClass(BuiltinClassLoader.java:532)
at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:585)
at java.base/java.lang.Class.forName(Class.java:447)
at 
java.base/sun.launcher.LauncherHelper.loadModuleMainClass(LauncherHelper.java:585)
at 
java.base/sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java:523)
Caused by: java.lang.UnsupportedOperationException: Path not associated with 
default file system.
at java.base/java.nio.file.Path.toFile(Path.java:658)
at 
java.base/jdk.internal.module.ModuleReferences$JarModuleReader.newJarFile(ModuleReferences.java:229)
at 
java.base/jdk.internal.module.ModuleReferences$JarModuleReader.(ModuleReferences.java:239)
at 
java.base/jdk.internal.module.ModuleReferences.lambda$newJarModule$0(ModuleReferences.java:96)
at 
java.base/jdk.internal.module.ModuleReferenceImpl.open(ModuleReferenceImpl.java:88)
at 
java.base/jdk.internal.loader.BuiltinClassLoader.createModuleReader(BuiltinClassLoader.java:953)
at 
java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1719)
at 
java.base/jdk.internal.loader.BuiltinC

Re: Review Request JDK-8177844: Ensure non-upgradeable modules cannot be upgraded even with --patch-module

[Remi Forax]

Ok,
thanks.

Rémi 

On April 8, 2017 4:49:13 PM GMT+02:00, Mandy Chung  
wrote:
>No change in that regards.  The updated test [1] in the webrev shows it
>patches java.base/jdk.internal.modules.SystemModules
>
>Before this fix, you can do —patch-module jdk.internal.vm.compiler=.jar
>—upgrade-module-path graal.jar. The reason is that —patch-module
>disables the hash checking that makes non-upgradeable module
>upgradeable.  After this fix, it fails if you attempt upgrade a patched
>non-upgradeable module.
>
>[1]
>http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8177844/webrev.01/test/tools/launcher/modules/patch/systemmodules/PatchSystemModules.java.frames.html
>
>> On Apr 8, 2017, at 7:22 AM, Remi Forax  wrote:
>> 
>> Hi Mandy,
>> how can I test a change in java.lang after that patch ?
>> 
>> Rémi
>> 
>> 
>> On April 8, 2017 7:50:50 AM GMT+02:00, Mandy Chung
> wrote:
>> http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8177844/webrev.01
><http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8177844/webrev.01>/
>> 
>> This fixes -—patch-module to do hash checking on the module being
>patched
>> so that it will ensure that a non-upgradeable module remains not
>upgradeable.
>> 
>> As Graal has been using —-patch-module option to disable the hash
>checking
>> to load a different version of jdk.internal.vm.compiler, it needs a 
>> mechanism to load Graal, as tracked by JDK-8177845.
>> 
>> Mandy
>> 
>> -- 
>> Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Review Request JDK-8177844: Ensure non-upgradeable modules cannot be upgraded even with --patch-module

[Remi Forax]

Hi Mandy,
how can I test a change in java.lang after that patch ?

Rémi


On April 8, 2017 7:50:50 AM GMT+02:00, Mandy Chung  
wrote:
>http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8177844/webrev.01/
>
>This fixes -—patch-module to do hash checking on the module being
>patched
>so that it will ensure that a non-upgradeable module remains not
>upgradeable.
>
>As Graal has been using —-patch-module option to disable the hash
>checking
>to load a different version of jdk.internal.vm.compiler, it needs a 
>mechanism to load Graal, as tracked by JDK-8177845.
>
>Mandy

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Disallowing the dynamic loading of agents by default (revised)

[Remi Forax]

- Mail original -
> De: "mark reinhold" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mercredi 5 Avril 2017 18:15:20
> Objet: Disallowing the dynamic loading of agents by default (revised)

> Thanks to everyone for the quick feedback on this topic, and especially
> to Andrew for the constructive dialogue.
> 
> Here's a revised proposal:
> 
>  - Define a new VM option, `-XX:+EnableDynamicAgentLoading`, that's
>on by default in JDK 9 but off by default in JDK 10.
> 
>This will allow launch scripts that use this option on JDK 10 to
>work on JDK 9 without change, and will allow early testing of the
>JDK 10 behavior on JDK 9.

yes


> 
>  - Revise the `com.sun.tools.attach` API to forbid attachment to the
>current process or to an ancestor of the current process, and
>define a read-only system property that allows such attachment to
>be enabled via the command line.
> 
>This will discourage the inadvertent use of libraries that, for
>better or for worse, intentionally violate strong encapsulation.

don't get this one, as David said, if you span a new VM with an exec, you have 
more right ??


> 
>  - Enhance the `-jar` launcher option so that if the JAR file being
>launched contains a `Premain-Class` attribute then it's launched
>as both an application and as an agent for that application.
> 
>This will allow `java -jar foo.jar` to be used in place of the
>more verbose `java -javaagent:foo.jar -jar foo.jar` [1].


Can be very useful indeed.
(with another name that "Premain-Class" for backward compatibility).

> 
> Taken together, these changes are intended to enable the continued use
> of legitimate dynamically-loaded agents without change on JDK 9 and with
> a small change on JDK 10.  That later change will align the treatment of
> such agents with the other means of breaking encapsulation (`--add-opens`,
> etc.) in order to ensure integrity by default for all code.
> 
> This proposal does not attempt to lock down platform classes as distinct
> from user classes.  Many agents have legitimate reasons to transform
> platform classes, so an additional mechanism to protect those classes
> does not appear to be worthwhile.
> 
> Comments?
> 
> - Mark
> 
> 
> [1] http://mail.openjdk.java.net/pipermail/jigsaw-dev/2017-April/012000.html

Rémi

Re: Progress report on SLF4J project modularization

[Remi Forax]

Robert,
they are two ways to compile a project with a version which is not java 9, let 
say 8 and a module-info with java 9.

Either, like the Maven compiler plugin, you compile twice the source, once with 
java 9 and once with java 8 excluding the module-info.java,
or you compile once with java 8 excluding the module-info and you then compile 
only the module-info.java with java 9 (it's what Ceki has used).

The later solution is usually faster and avoid subtle bugs like if between java 
x and java 9, the code of javac changes the way a synthetic class is named (you 
will have the two versions with the compiler plugin).

cheers,
Rémi
 
- Mail original -
> De: "Robert Scholte" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Samedi 25 Mars 2017 18:56:50
> Objet: Re: Progress report on SLF4J project modularization

> On Sat, 25 Mar 2017 15:38:27 +0100, Ceki Gulcu  wrote:
> 
>> Following the recipe at [2], it all seems to work.
> 
> Good to read that the recipe works. If we need to add things to this
> recipe to improve it, just let me know.
> Although in general I don't think we'll be helping developers if we show
> the javac with its commandline arguments, especially since most of it is
> hidden or calculated inside the maven-compiler-plugin.
> 
> thanks,
> Robert

Re: Better tools for adjusting to strong encapsulation

[Remi Forax]

good to me.

Rémi


On March 23, 2017 4:30:00 AM GMT+01:00, mark.reinh...@oracle.com wrote:
>Thanks to everyone for all the feedback on this topic.
>
>It appears that issuing warning messages for illegal-access operations
>enabled by the precise `--add-opens` and `--add-exports` options is a
>bit too aggressive, at least for JDK 9.  Perhaps we can enable that in
>JDK 10 after there's been more time for libraries, frameworks, and even
>the JDK itself to adjust to the realities of strong encapsulation.
>
>For now I suggest that we revert to the previous behavior of these two
>options, so that they do not cause warning messages to be issued.  The
>new `--permit-illegal-access` option will continue to generate warning
>messages, as proposed.  If those messages are a problem in a particular
>scenario then they can be eliminated by switching to an appropriate set
>of `--add-opens` options, which can be constructed from the information
>contained in those messages.
>
>Comments?
>
>- Mark

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: 8174823: Module system implementation refresh (3/2017)

[Remi Forax]

So ModuleTarget.osVersion is removed, i will have to update ASM accordingly.

In jdk/internal/module/ModuleBootstrap.java,
  addExtraExportsOrOpens(bootLayer, extraOpens, true, builder);
should be
  addExtraExportsOrOpens(bootLayer, extraOpens, ADD_OPENS, builder);
there is no point to introduce a boolean here.

In java.lang.reflect.Method,
the StringJoiner is created even if there are no parameters.

AccessibleObject.toShortString() should have a javadoc.

Rémi

- Mail original -
> De: "Alan Bateman" 
> À: "jigsaw-dev" , 
> "hotspot-runtime-...@openjdk.java.net runtime"
> 
> Envoyé: Mardi 21 Mars 2017 22:42:02
> Objet: 8174823: Module system implementation refresh (3/2017)

> We have been accumulating changes in the jake forest for the last few
> weeks to align with the proposals in JSR 376 and also to pick up API
> changes and bug fixes. It's time to bring these changes into jdk9/dev.
> The issue to bring these changes into jdk9/dev in bulk has been approved
> via the FC extension process (still in use during JDK 9 RDP2).
> 
> A summary of the main changes is listed in JDK-8174823 [1], and the
> webrevs with the changes to bring to jdk9/dev are here:
>http://cr.openjdk.java.net/~alanb/8174823/1/
> 
> The changes are currently based on jdk-9+161 and will be rebased before
> pushing.
> 
> -Alan
> 
> [1] https://bugs.openjdk.java.net/browse/JDK-8174823

Re: Better tools for adjusting to strong encapsulation

[Remi Forax]

I do not understand why the uses of --add-exports and --add-opens has to 
generate a warning,
they currently don't, this is a regression from my POV.

I do not think that introducing --permit-illegal-access worth the deal ion that 
case.

regards,
Rémi

- Mail original -
> De: "mark reinhold" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Mardi 21 Mars 2017 15:48:13
> Objet: Better tools for adjusting to strong encapsulation

> By now we've heard quite a bit of feedback, both publicly and privately,
> from developers who've migrated existing applications to JDK 9.  This can
> be a non-trivial task, involving the iterative construction of a set of
> command-line workarounds, since some popular libraries and frameworks use
> the core reflection API to access internal APIs of the JDK.  On JDK 9
> such components no longer work out-of-the-box because most internal APIs
> are now strongly encapsulated in order to improve the integrity of the
> platform.
> 
> In order to make it easier to adjust to the strong encapsulation of
> internal APIs in JDK 9, in both the short term and over the long term,
> we've introduced a temporary "big kill switch", described below.  This
> switch will only be supported in JDK 9.  It's already available in the
> current Jigsaw EA builds [1], and will shortly be merged into JDK 9.
> 
> Background
> --
> 
> Diagnosing the failures of libraries and frameworks that access internal
> APIs of the JDK and then working around those failures is possible, but
> tedious.  The initial symptom is usually an `IllegalAccessException` or
> an `InaccessibleObjectException`.  The workaround is to use an
> `--add-opens` command-line option whose arguments are based upon the
> information shown in the exception message.  A large application that
> uses several reflective components may suffer many such failures, so this
> process may have to be repeated many times, resulting in a long list of
> command-line options that will typically be recorded in a launch script
> or an @-file.
> 
> These options are tedious to construct but, additionally, there's a
> significant risk that once they're baked into a file they'll never be
> removed, even long after the problems that they work around have been
> resolved.  An application that needs some workarounds today might not
> benefit as well as it could from strong encapsulation in the long term,
> after all of the reflective components that it uses have been fixed to
> replace their use of JDK-internal APIs with proper exported APIs.
> 
> To help the entire ecosystem migrate to the modular Java platform we want
> to motivate such fixes as strongly as we possibly can, especially in the
> near term.  We can do that by making it easier for you to get an existing
> application to work while also ensuring that the run-time system reports
> illegal reflective-access operations in a way that allows you to identify
> which libraries and frameworks on your class path require illegal access
> in order to work.  If a component's maintainers have already released a
> new, fixed version that no longer requires illegal access then you can
> consider upgrading to that version.  If, however, a component still needs
> to be fixed then we encourage you to contact its maintainers and ask them
> to stop using JDK-internal APIs [2].
> 
> With all this in mind we've introduced a "big kill switch", described
> below, together with some new warning messages.  This switch will be
> supported in JDK 9, in order to enable application migration and motivate
> fixes to libraries and frameworks.  It will not be supported in JDK 10 or
> later releases, in order to motivate fixes to libraries and frameworks
> sooner rather than later.  If this switch were to be supported long-term
> then that would tend to reduce the pressure on maintainers to fix their
> components.
> 
> New command-line option: `--permit-illegal-access`
> --
> 
> If this option is present at run time then it will permit illegal access
> operations by code in any unnamed module (i.e., code on the class path)
> to members of types in any named module via the standard reflective APIs
> (`java.lang.reflect` and `java.lang.invoke`), including in particular
> those that would otherwise result in an `IllegalAccessException` or an
> `InaccessibleObjectException`.
> 
>+--+
>| This option will only be supported in JDK 9. |
>| It will be removed in JDK 10.|
>+--+
> 
> This option does not permit illegal access operations by code in named
> modules to members of types in other named modules; for that you must
> use the `--add-opens` or `--add-exports` options.  Those options can be
> combined with `--permit-illegal-access`.
> 
> Warnings of illegal reflective-access operations
> ---

Re: Better tools for adjusting to strong encapsulation

[Remi Forax]

Hi Andrew,
it's not that relevant, we are beyond that point.

Introducing SystemLogger is an implicit way to recognize that java.logging is a 
bad API and that the JDK should cut its ties to that API (when it's possible, 
i.e. when Logger is not part of the public API).

I see the SystemLogger API is a way to get logging info from the JDK and to 
redirect them to whatever logger i want, so one can have the JDK log info and 
the application log info at the same place.

cheers,
Rémi

- Mail original -
> De: "Andrew Dinn" 
> À: "Alan Bateman" 
> Cc: jigsaw-dev@openjdk.java.net
> Envoyé: Mercredi 22 Mars 2017 12:57:56
> Objet: Re: Better tools for adjusting to strong encapsulation

> On 22/03/17 11:42, Alan Bateman wrote:
>> On your comment about neglect then you might want to look at JEP 264 [1].
>> ...
>> [1] http://openjdk.java.net/jeps/264
> 
> You might also want to look at this xkcd
> 
>  https://xkcd.com/927/
> 
> which is, perhaps, just as relevant.
> 
> regards,
> 
> 
> Andrew Dinn
> ---
> Senior Principal Software Engineer
> Red Hat UK Ltd
> Registered in England and Wales under Company Registration No. 03798903
> Directors: Michael Cunningham, Michael ("Mike") O'Neill, Eric Shander

Re: Lookup.defineClass

[Remi Forax]

yes :)

Remi


On March 8, 2017 9:41:13 PM GMT+01:00, John Rose  wrote:
>On Mar 8, 2017, at 10:47 AM, Remi Forax  wrote:
>> 
>> The exception if the lookup is PRIVATE should be
>IllegalStateException and not UnuspportedOperationException.
>> then, given that the loaded class will have the same protection
>domain as the lookup class, i do not fully understand why having a
>PRIVATE lookup is a problem.
>
>Since lookups are stateless, so there is never a reason for them to
>throw ISE; thus UOE is better (loose analogy with immutable lists).
>
>We are disallowing PRIVATE in order to reserve it for future use.  If
>you define a class into a private context, the resulting class *should*
>have private access to the lookup-class, but we currently cannot
>express this in the JVM (except by hacks).  Defining a new class with
>private access to a lookup-class is the replacement we are planning for
>the unsafe, undocumented notion of "host class".  But first we need a
>better treatment of what is "private access", which will be provided by
>a future concept of "nestmate" (at the JVM level) which mirrors the
>language-level permissions of nestmates.
>
>Make sense?
>
>— John

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Instrumentation.redefineModule -- extraPackages argument?

[Remi Forax]

Hi Mickael,
i do not think you can use addPackage() directly because 
Module.getResourceAsStream() will not work on the resources associated with a 
package added by addPackage.

The code of Module.getResourceAsStream() currently supposes that only proxies 
can add private packages, and proxies can not add new resources.

Rémi

- Mail original -
> De: "Michael Rasmussen" 
> À: "jigsaw-dev" 
> Envoyé: Samedi 11 Mars 2017 14:31:08
> Objet: Re: Instrumentation.redefineModule -- extraPackages argument?

> Looking a bit closer in the source, and not just the public API, I did find
> addPackage in the private API part of Module (and the corresponding
> native JVM_AddModulePackage).
> So the capability for dynamically evolving a named module is present,
> supported, and utilized by Proxy!
> 
> Adding a new package isn't just for use-cases of development, modules that
> dynamically expand have other use cases as well, for instance classes
> dynamically created at runtime based on factors that cannot be determined
> at compile time (well, proxy is a good example of that, and as mentioned
> already does exactly this!).
> 
> Also, as Jochen pointed out, setting up a new classloader with new layers
> and modules etc, is a non-trivial task, just for adding a new package --
> especially considering the functionality is present, just not public.
> 
> sure, from JRebel's point of view, a method being non-public isn't exactly
> an obstacle for us; but "normal" frameworks don't have that luxury.
> 
> /Michael

Re: Lookup.defineClass

[Remi Forax]

Hi Alan,
i've already updated the only code i had that was using ClassLoader.defineClass 
after a Method.setAccessible() to use Unsafe.defineClass
(no way to remove it, i need to be able to inject code a la Mockito)
so Lookup.defineClass is a safe replacement of Unsafe.defineClass , i suppose 
that Unsafe.defineClass should be deprecated and will be removed in 10 ?

The exception if the lookup is PRIVATE should be IllegalStateException and not 
UnuspportedOperationException.
then, given that the loaded class will have the same protection domain as the 
lookup class, i do not fully understand why having a PRIVATE lookup is a 
problem.

Rémi

- Mail original -
> De: "Alan Bateman" 
> À: "jigsaw-dev" 
> Envoyé: Mardi 7 Mars 2017 15:15:15
> Objet: Lookup.defineClass

> One of the issues that strong encapsulation exposes is the number of
> libraries and frameworks that hack into the ClassLoader's defineClass
> methods to define proxy and other classes. We've been looking to get
> some of these use-cases onto a "supported footing" for some time. To
> that end, a promising proposal is to add a defineClass method to
> MethodHandles.Lookup so that a class can be defined to the same class
> loader and in the same runtime package as the Lookup's lookup class.
> 
> An initial implementation of Lookup.defineClass is in the current Jigsaw
> EA builds [1], the javadoc is also online [2]. We are currently looking
> for help in trying out this method to see how it might be used to
> replace existing hacks. The proposed method is not a drop-in replacement
> for the non-public ClassLoader.defineClass, it instead relies on being
> able to obtain a Lookup with PACKAGE access on a target class in the
> package. The Lookup is ideally obtained cooperatively but it could be
> obtained non-cooperatively too (using the new privateLookupIn and
> dropping PRIVATE access for example). Using a Lookup is likely a
> significant change for frameworks using bytecode libraries today and it
> might be that a number of libraries will need to be updated to try this
> out. If you do try it out then please reply here and let us know how you
> got on.
> 
> -Alan
> 
> [1] https://jdk9.java.net/jigsaw/
> [2]
> http://download.java.net/java/jigsaw/docs/api/java/lang/invoke/MethodHandles.Lookup.html#defineClass-byte:A-

Re: Review Request: JDK-8175193: jlink and `requires static`

[Remi Forax]

Hi Mandy,
I prefer that semantics.

In the code, you can remove the ".stream()".

regards,
Rémi

- Mail original -
> De: "Mandy Chung" 
> À: "jigsaw-dev" 
> Envoyé: Vendredi 24 Février 2017 00:47:35
> Objet: Review Request: JDK-8175193: jlink and `requires static`

> This patch fixes jlink to link modules that has `requires static M`.
> jlink does not modify module-info.class. Hence, at runtime if
> module M is on the module path and added in the resolved graph.
> `requires static` dependency will be satisfied.
> 
> Webrev:
>  http://cr.openjdk.java.net/~mchung/jdk9/webrevs/8175193/webrev.00/index.html
> 
> Mandy

Re: sun.nio.ch.DirectBuffer and jdk9/jigsaw

[Remi Forax]

Hi Vitaly,
I believe that what you are looking for is 
MethodHandles.byteBufferViewVarHandle [1] that creates a VarHandle that allows 
to do aligned/unaligned access on a ByteBuffer.
It should be a little slower that using Unsafe because of the bound check (and 
the nullcheck but it is usually removed by the VM).
That's said, unaligned access are faster with a VarHandle because 
sun.misc.Unsafe (which is stuck in the Java 8 world unlinke 
jdk.internal.misc.Unsafe) does not provide a way to do an unaligned access so 
you have do multiple read which is slower.

cheers,
Rémi

http://download.java.net/java/jdk9/docs/api/java/lang/invoke/MethodHandles.html#byteBufferViewVarHandle-java.lang.Class-java.nio.ByteOrder-

- Mail original -
> De: "Vitaly Davidovich" 
> À: "Uwe Schindler" 
> Cc: "jigsaw-dev" 
> Envoyé: Jeudi 23 Février 2017 18:05:36
> Objet: Re: sun.nio.ch.DirectBuffer and jdk9/jigsaw

> On Thu, Feb 23, 2017 at 11:54 AM, Uwe Schindler 
> wrote:
> 
>> Hi,
>>
>> Why do you need the address at all in the Java code? Java code can use the
>> official ByteBuffer methods to access the memory you are wrapping. In Java
>> 9 that’s optimized very good by Hotspot and should be almost as fast as
>> array accesses (we proved that in Apache Lucene - congrats to the Hotspot
>> committers). If you need special access modes like volatile access, then
>> you can use Java 9's VarHandles. You can get a VarHandle to the backing
>> direct buffer using the MethodHandles API.
>>
> I mentioned this upthread - the base address is used for index calculations
> to read/write data using Unsafe directly.  I don't know about Java 9 as
> I've not tried it yet, but the generated assembly for using BB methods vs
> Unsafe did not favor BB.  There are also various safety checks in
> DBB/Buffer internals that won't get optimized away.
> 
> Also, my general experience with looking at C2 optimizations has led me to
> the conclusion that the optimizations are "unstable" - innocent code
> changes, differences in order of how classes are loaded, differences in
> what callsites trigger compilation first, and a bunch of other otherwise
> benign things can interfere with inlining decisions, which is typically the
> reason things go sideways in terms of optimization.
> 
> As for MethodHandles and VarHandles, that's one possibility I'm considering
> as a way to migrate off using DirectBuffer (although I'd still like to know
> if there's any plan to standardize/formalize some notion of a direct
> buffer).  However, my understanding is that using MH will still require me
> to crack into jdk code (to get access to the DBB and friends) and thus
> require addExports.  DirectBuffer is still accessible if using addExports,
> but it's a wrinkle I was hoping to iron out, hence what started this email
> chain.
> 
>>
>> Uwe
>>
>> -
>> Uwe Schindler
>> uschind...@apache.org
>> ASF Member, Apache Lucene PMC / Committer
>> Bremen, Germany
>> http://lucene.apache.org/
>>
>> > -Original Message-
>> > From: jigsaw-dev [mailto:jigsaw-dev-boun...@openjdk.java.net] On Behalf
>> > Of Vitaly Davidovich
>> > Sent: Thursday, February 23, 2017 5:30 PM
>> > To: Chris Hegarty 
>> > Cc: jigsaw-dev 
>> > Subject: Re: sun.nio.ch.DirectBuffer and jdk9/jigsaw
>> >
>> > On Thu, Feb 23, 2017 at 11:10 AM, Chris Hegarty
>> > 
>> > wrote:
>> >
>> > >
>> > > > On 23 Feb 2017, at 11:30, Vitaly Davidovich 
>> wrote:
>> > > >> ...
>> > > > The buffers are reused by having them point to different native
>> memory
>> > > > block addresses; those blocks are managed by native code.  As
>> > mentioned,
>> > > > the ByteBuffer (DirectByteBuffer concretely) is used as the Java
>> level
>> > > > interface/view of native memory, allowing Java and native code to
>> > > > communicate.
>> > >
>> > > So a DBB, under your code, may report a different address at some time
>> > > in the future, to that of what it currently reports?
>> >
>> > Correct.
>> >
>> > > I was not aware of this
>> > > usecase. Is any similar code available on the web, or elsewhere, so we
>> > > could try to determine why this is being done?
>> > >
>> > Unfortunately it's not open source code, and I don't immediately know of
>> > anything similar on the web (or otherwise).  However, the gist is the
>> > following:
>> > 1) Allocate a 0-size DBB (i.e. ByteBuffer.allocateDirect(0)).  This gives
>> > you a Java "handle", if you will, to some native memory.  But, since this
>> > DBB will be attached/reattached to different memory dynamically, there's
>> no
>> > need for an actual allocation.
>> > 2) Native code wants to expose a segment of memory to Java.  In JNI, it
>> > sets the address and capacity of this DBB to the pointer where the native
>> > memory segment starts, and to the capacity (it knows how big the native
>> > segment is).  Java code asks for this DBB to be "attached" to, say, some
>> > sort of message, and the JNI/native code perform these functions.
>> > 3) Java gets the attached DBB back, and can t

Re: Why can't main class be given in module-info.java?

[Remi Forax]

Hi Gunnar,
all module metadata, the main-class but also the version, the os etc, are 
injected by the jar command.

In 8, the MainClass (or all of metadata that are in the manifest) only exist 
from the runtime point of view after being put inside of a jar, so in order to 
be the less disruptive possible for the existing build processes, we have 
chosen to keep the same strategy. This also allow us to support application 
that are built by compiling the code once and split the generated classes in 
several jars afterwards.

cheers,
Rémi

- Mail original -
> De: "Gunnar Morling" 
> À: "jigsaw-dev" 
> Envoyé: Mercredi 22 Février 2017 23:44:59
> Objet: Why can't main class be given in module-info.java?

> Hi,
> 
> Why is it that the binary form of module declarations can carry
> information on a main class ("ModuleMainClass" attribute), but no main
> class can be specified in module-info.java?
> 
> It seems setting ModuleMainClass is left to the jar tool, but I'd like
> to understand the rationale behind it.
> 
> Thanks,
> 
> --Gunnar

Re: Remove sun.misc.Unsafe.defineAnonymousClass

[Remi Forax]

- Mail original -
> De: "Paul Sandoz" 
> À: "jigsaw-dev" 
> Cc: "Brian Goetz" 
> Envoyé: Jeudi 23 Février 2017 02:22:27
> Objet: Remove sun.misc.Unsafe.defineAnonymousClass

> Hi,

Hi Paul,
my first reaction was 'over my dead body',
(you have sent this email to jigsaw-dev that i hope, it's just a mistake)

> 
> I think we should remove the method sun.misc.Unsafe.defineAnonymousClass.

so yes for 10 when public replacement APIs will be introduced, but no for 9.

> 
> This is one leaked through in the move to the unsupported module. I believe
> under the remit of unsupported module we can still do this.

It does not leaked, we discuss about removing it for 9 during the last JVM 
Summit,
and the consensus (i believe) was to add several public methods to cover all 
the 'features' of defineAnonymousClass in 10 before removing it.   

> 
> There are no usages on grepcode.com , and we really 
> don’t
> want anyone to start depending on this, especially for constant pool patching,
> where such a public API might require some careful thought, compared to the
> anon class support itself which is somewhat simpler.

disentangling the different 'features' of defineAnonymousClass also requires 
some thought, defineNestmate [1] is one step in the right direction (even if it 
still mix the fact that the class is not loaded by a classloader and the 
NestMate access), but by example for my proxy library [2], i still also need to 
be able to do the constant pool patching because i want to inject method 
handles that are not direct (i.e. live objects from the VM POV).

> 
> If developers really want constant pool patching they can crack open the
> jdk.internal.misc package and then they really know this is not supported.

I see this has being more evil that using sun.misc.Unsafe.
Nobody should use a non exported package like jdk.internal.misc. It's fine if 
you are in the middle of the migration process to 9 but jdk.internal.misc is a 
package introduced in 9, so nobody outside of the modules listed in the 
module-info should ever using it. 

> 
> For the anon class support we are considering retconning to nest mates and 
> with
> a public method on MethodHandles.lookup.
> 
> Paul.

Rémi

[1] https://bugs.openjdk.java.net/browse/JDK-8171335
[2] https://github.com/forax/proxy2

Re: How to name modules, automatic and otherwise

[Remi Forax]

No,
both are two unrelated projects,
  https://spark.apache.org/
  http://sparkjava.com/

Rémi

- Mail original -
> De: "Alan Bateman" 
> À: "Michael Rasmussen" 
> Cc: "jigsaw-dev" 
> Envoyé: Vendredi 17 Février 2017 10:49:13
> Objet: Re: How to name modules, automatic and otherwise

> On 17/02/2017 09:07, Michael Rasmussen wrote:
> 
>> :
>> I can only agree with this. s/willow/spark/ and we have an example of such
>> a situation in maven already today, of two different libraries with the
>> same name.
>> With the options to specify group, no problem, then I could just write:
>> requires spark-core from com.sparkjava or requires spark-core from
>> org.apache.spark in order to clarify which was intended.
>>
> Are these two libraries exporting the same API?
> 
> -Alan

Re: Running jaotc with an external Graal

[Remi Forax]

Can i say again that when using --patch-module, module-info.class should be 
merged instead of having a warning ?

Rémi

- Mail original -
> De: "Mandy Chung" 
> À: "Christian Thalinger" 
> Cc: jigsaw-dev@openjdk.java.net, "hotspot compiler" 
> , graal-...@openjdk.java.net
> Envoyé: Mercredi 15 Février 2017 23:44:19
> Objet: Re: Running jaotc with an external Graal

>> On Feb 15, 2017, at 1:56 PM, Christian Thalinger  
>> wrote:
>> 
>> 
>>> On Feb 14, 2017, at 4:38 AM, Andrew Haley  wrote:
>>> 
>>> On 14/02/17 14:37, Alan Bateman wrote:
 --patch-module can be used to patch any module in the boot layer. So if
 you are looking to override or add classes then --patch-module should work.
>>> 
>>> Aha!  Thanks,
>> 
>> Does it?
> 
> Yes it does except that module-info.class can’t be patched.  You will get a
> warning if the patched path contains module-info.class.  Are you seeing the
> otherwise?
> 
> Mandy

Re: Java SE JSR 250 annotations module renamed to java.xml.ws.annotation?

[Remi Forax]

I agree.

Remi


On February 15, 2017 12:52:53 AM GMT+01:00, mark.reinh...@oracle.com wrote:
>2017/2/13 9:17:47 -0800, Guillaume Smet :
>> On Mon, Feb 13, 2017 at 6:10 PM, alan.bate...@oracle.com wrote:
>>> On 13/02/2017 16:58, Guillaume Smet wrote:
 On Mon, Feb 13, 2017 at 5:12 PM, alan.bate...@oracle.com wrote:
> 
> I agree that @Generated is awkward but I haven't suggested
>removing it.
 
 What do you suggest then? As far as I understood you, you were
>suggesting
 removing the module in Java 10 so the @Generated annotation would
>also be
 gone? Or did I misunderstand?
>>> 
>>> That is the proposal. If it goes ahead then it means that tools that
>rely
>>> on these annotations in the JDK would need to deploy the standalone
>version
>>> on the class path or as a module on the module path.
>> 
>> Yeah, so basically, it would end up with
>> http://hg.openjdk.java.net/code-tools/jmh/rev/d74b2861222c .
>> 
>> I don't think it's the best possible outcome for this useful
>annotation.
>
>I agree.
>
>The `@Generated` annotation falls outside the original charter of the
>`java.lang.annotation` package, which was meant for annotations that
>directly support the language's annotation facility, but we already
>added `@Native` in SE 8, so let's add `@Generated` in SE 9 as David
>suggests and encourage people to use it when running on this and later
>releases.
>
>The fact that `@Generated` is so widely used is new information to some
>of us, so thanks for bringing it up.
>
>- Mark

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Java SE JSR 250 annotations module renamed to java.xml.ws.annotation?

[Remi Forax]

ACC_SYNTHETIC is a binary annotation/attribute/value, not a source one and 
synthetic members are not visible from the compiler point of view (think 
bridges by example).

ACC_MANDATED can be used (again only in classfiles) but you have a good chance 
to be tracked and killed by Alex Buckley :)

Rémi

- Mail original -
> De: "Alan Bateman" 
> À: "Guillaume Smet" 
> Cc: jigsaw-dev@openjdk.java.net
> Envoyé: Lundi 13 Février 2017 17:12:43
> Objet: Re: Java SE JSR 250 annotations module renamed to 
> java.xml.ws.annotation?

> On 13/02/2017 15:56, Guillaume Smet wrote:
> 
>>
>> So basically, your position is that we should update all the Java
>> tooling out there to remove the usage of the @Generated annotation?
>> Because I don't think introducing a dependency to the JSR 250 API jar
>> everywhere is a good thing.
>>
>> If so, as mentioned by David, it would be nice to provide something
>> else as it's considered a useful marker and is used by a lot of tools.
>> And even if we do so, the transition will be chaotic to say the least.
>>
> I agree that @Generated is awkward but I haven't suggested removing it.
> It would be useful to go through a few examples as it may be that
> ACC_SYNTHETIC is more appropriate for some of these cases.
> 
> -Alan

Re: Java SE JSR 250 annotations module renamed to java.xml.ws.annotation?

[Remi Forax]

Perhaps, it's a problem of communication, what about
"As an effort to make Java on diet, Java 10 will not be shipped with modules 
corba, jaxb or common annnotations, for Java 9, you can use a command line 
option to have access to those deprecated modules."

Rémi

- Mail original -
> De: "Andrew Haley" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Lundi 13 Février 2017 16:11:48
> Objet: Re: Java SE JSR 250 annotations module renamed to 
> java.xml.ws.annotation?

> On 13/02/17 14:34, Alan Bateman wrote:
>> The small subset of the JSR-250 defined "Common Annotations" that you
>> see in Java SE is part of the web services stack that was added in Java
>> SE 6. If it weren't for JAX-WS then these annotations would have no
>> business being in Java SE or the JDK.
> 
> Well, okay, but they have been advertised as "Common Annotations" for
> ten years now.  It's a bit unconvincing to say "Yes, but we didn't
> really mean it."
> 
> Andrew.

Re: Java SE JSR 250 annotations module renamed to java.xml.ws.annotation?

[Remi Forax]

TLDR; we can't.

In Java 6, we made the mistake to create what you can call twin packages, a 
package with few classes in the JDK and the same package with more classes 
including the classes already present in the JDK. The idea was that you can run 
an application using the JDK, in that case you have access the the small subset 
and you can also use the full packages in a container env.

This approach fails once you have to update such kind of packages because the 
classes that can from the JDK (from the bootclasspath) can have a different 
versions from the classes that comes from the container (from the classpath). 
Note that this problem is just a variation of the problem of having two 
different versions of the same jars in the classpath.

In the java module world, a package can comes from only one module (i.e. one 
modular jar). So with java 9, if you have different versions of the same jars, 
from the VM point of view different modular jars containing the same package, 
the VM fails when trying to resolve the module graph at startup. So with Java 9 
you can only have one jar containing a package in the module graph.

With Java 9, the JDK is modularized otherwise it will not be able to run 
modules from the module path, it also has to provide a way for classes on the 
classpath to see modules of the JDK otherwise your application will not be able 
to find java.lang.Object. For that, classes from classpath are put in a fake 
module named the unamed module.

Now if the module containing the JSR 250 annotations is included by default in 
the module graph of the JDK, every projects using a container that will also 
use the JSR 250 (so the JSR 250 jars is in the classpath) will fail because the 
module of the JDK and unamed  module will both have the same package. To solve 
that issue, the idea is that the JDK is released with a module containing the 
JSR 250 annotations but this module is not included in the root modules and you 
have to use --add-modules to add it in the root modules.

so there are 3 cases:
- your application is modular, in that case one module defines a dependency 
(with requires) on a module* containing the JSR 250 annotation and you have 
nothing to do.
- your application uses a container that provides a jar containing the JSR 250 
annotations in the classpath, you still have nothing to do.
- your application uses the classpath and do not provide any jars containing 
the JSR 250 annotations, you can use --add-modules but be aware after some 
point the JDK will stop to provide this jar (the module is marked deprecated).

perhaps another way to see this is to think that we made a mistake to include 
parts of the Java EE modules inside the JDK when releasing Java 6, and we are 
trying to fix that mistake now, which means removing those packages but because 
we are Java instead of removing them now, we made them accessible under a flag 
and we will remove them in Java 10.

cheers,
Rémi

- Mail original -
> De: "Guillaume Smet" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Lundi 13 Février 2017 14:34:28
> Objet: Java SE JSR 250 annotations module renamed to java.xml.ws.annotation?

> Hi,
> 
> (Posting this on the list at the request of Rory O'Donnell)
> 
> This morning I tried to upgrade our Hibernate Validator build and came
> across this issue:
> https://bugs.openjdk.java.net/browse/JDK-8173604 .
> 
> As a lot of projects experimenting with the Early access builds of the JDK
> 9, we added --add-modules java.annotations.common to our build options.
> 
> This doesn't work anymore as the Java SE JSR 250 annotations (@Generated,
> @PreDestroy, @PostConstruct, @Resource...) that were previously in a module
> called java.annotations.common were moved to a module called
> java.xml.ws.annotation as part of the aforementioned issue.
> 
> The fix is easy, I just have to change my --add-modules directive but I
> started to wonder if there's some sort of misunderstanding here.
> 
> As far as I understand it, these annotations are supposed to be common
> annotations available in the JDK (see JSR 250: Common Annotations for the
> JavaTM Platform). Some are used by a lot of projects out there: nearly
> every project based on generated code requires the @Generated annotation to
> be present as most tools generating code use it.
> 
> I don't know why these annotations were hosted in the jaxws part of the
> source code to begin with but I don't think we can limit them to this usage.
> 
> At the end, I'm wondering if these classes shouldn't be part of the default
> set of classes exposed without requiring an --add-modules option.
> 
> Thanks for your feedback.
> 
> --
> Guillaume

Re: Automatic module names

[Remi Forax]

I've coded such kind of tool [1] :)

It uses Maven (Aether) to download the dependencies then do a simple static 
analysis on the downloaded jars to calculate the dependencies and inject the 
module-info.class.
Because of the runtime dependencies, annotations, Class.forName etc, you also 
have to provide additional dependencies by hand so it's not the silver bullet.

And you still have the naming problem, which I've solved by asking the user to 
provide a name for each non modular jars.

So in order to automatically generate the module-info.class, you end up writing 
an additional configuration file which is not very different of a 
module-info.java :(

Rémi 

[1] 
https://github.com/forax/pro/blob/master/src/main/java/com.github.forax.pro.plugin.modulefixer/com/github/forax/pro/plugin/modulefixer/ModuleFixerPlugin.java



On February 7, 2017 3:03:38 PM GMT+01:00, "David M. Lloyd" 
 wrote:
>On 02/06/2017 12:51 PM, Alan Bateman wrote:
>> On 06/02/2017 18:25, David M. Lloyd wrote:
>>
>>>
>>> I don't think there is really a substantial increase in risk between
>>> manually modularizing a library and relying on automatic modules.
>> If several projects take on the responsibility to modularize a
>library
>> that they don't maintain then there is no guarantee that they will
>end
>> up with equivalent modules. Static analysis (`jdeps
>> --generate-module-info ...`) gets you so far but it often needs a
>deeper
>> understanding of the library to be confident that the module
>declaration
>> that they write for the library is complete. Automatic modules take
>on
>> much of the behavior of a library on the class path so that it's not
>> necessary to synthesize everything that an explicit module would
>declare.
>
>As could a tool, which has the added advantage that the module 
>information could easily be incrementally narrowed instead of going 
>fully from automatic to fully specified.  To do incremental change from
>
>automatic to fully specified, you'd need a tool to generate an 
>automatic-equivalent descriptor anyway.
>
>-- 
>- DML

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

Re: Current list of new (contextual) keywords

[Remi Forax]

Hi Barry,
from http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html#jigsaw-1.1,

open, module, requires, exports, to, opens, uses, provides, with, transitive

cheers,
Rémi

- Mail original -
> De: "Barry Burd" 
> À: jigsaw-dev@openjdk.java.net
> Envoyé: Samedi 4 Février 2017 22:57:36
> Objet: Current list of new (contextual) keywords

> Is there an up-to-date list of keywords (contextual or otherwise) being
> added to Java for Project Jigsaw (or for Java 9 in general)?

Re: The module implementation uses the wrong values for ACC_TRANSITIVE and ACC_STATIC_PHASE

[Remi Forax]

If the spec is updated, i really want to get ride of ACC_STATIC_PHASE and uses 
ACC_STATIC instead.
Renaming public to transitive was the right move but I think we go a little 
over the top with static.

In the module-info, the Java syntax already uses static, i do not see a lot of 
value to use a different flag.

As a example, we did not introduce a new flag for static methods in an 
interface even if they do not have the same behavior as static methods in a 
class.

Remi 


On February 1, 2017 9:22:28 AM GMT+01:00, Alan Bateman 
 wrote:
>On 30/01/2017 22:15, Remi Forax wrote:
>
>> Hi all,
>> chasing a bug on ASM, i've found that the module implementation in
>the JDK uses the wrong values for ACC_TRANSITIVE and ACC_STATIC_PHASE,
>> in the spec [1],
>>0x0020 (ACC_TRANSITIVE)
>>0x0040 (ACC_STATIC_PHASE)
>> but in the implementation [2],
>>public static final int ACC_TRANSITIVE= 0x0010;
>>public static final int ACC_STATIC_PHASE  = 0x0020;
>>
>Sigh, we do have a mismatch, I think a consequence of the churn last 
>year on the design proposals. Note that javac and the run time are in 
>sync so this doesn't impact anyone except when they start using the 
>updated ASM to read/write module-info.class. I would like to get the 
>spec updates that we are accumulating in jake into JDK 9 before JDK 9 
>his ZBB and and right time to sort out the requires_flags too.
>
>-Alan

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.

The module implementation uses the wrong values for ACC_TRANSITIVE and ACC_STATIC_PHASE

[Remi Forax]

Hi all,
chasing a bug on ASM, i've found that the module implementation in the JDK uses 
the wrong values for ACC_TRANSITIVE and ACC_STATIC_PHASE,
in the spec [1],
  0x0020 (ACC_TRANSITIVE)
  0x0040 (ACC_STATIC_PHASE)
but in the implementation [2],
  public static final int ACC_TRANSITIVE= 0x0010;
  public static final int ACC_STATIC_PHASE  = 0x0020;  

The output of javap is wrong too (java.xml) is tagged as ACC_STATIC_PHASE with 
the value 20,
but 20 means ACC_TRANSITIVE !
open module org.objectweb.asm.all@6.0_ALPHA2
  minor version: 0
  major version: 53
  flags: (0x8000) ACC_MODULE
  this_class: #2  // "module-info"
  super_class: #0
  interfaces: 0, fields: 0, methods: 0, attributes: 1
Constant pool:
   ...
{
}
Module:
  #4,20   // "org.objectweb.asm.all" ACC_OPEN
  #5  // 6.0_ALPHA2
  2   // requires
#7,8000 // "java.base" ACC_MANDATED
#0
#9,20   // "java.xml" ACC_STATIC_PHASE
#0
  7   // exports
#11,0   // org/objectweb/asm
#13,0   // org/objectweb/asm/commons
#15,0   // org/objectweb/asm/signature
#17,0   // org/objectweb/asm/tree
#19,0   // org/objectweb/asm/tree/analysis
#21,0   // org/objectweb/asm/util
#23,0   // org/objectweb/asm/xml
  0   // opens
  0   // uses
  0   // provides


This means that currently ASM is the only correct implementation of the 
classfile format :)

Rémi

[1] http://cr.openjdk.java.net/~mr/jigsaw/spec/lang-vm.html
[2] 
http://hg.openjdk.java.net/jdk9/jdk9/jdk/file/e4b19b8d4bbf/src/java.base/share/classes/jdk/internal/module/ClassFileConstants.java#l49