Re: module-info.java just causes problems

[Paul Benedict]

I think the interception would also solve the use case of Redhat's Byteman.
If the Layer can be given access to the Module Configuration and before any
classes in the Module are loaded (chicken-and-egg for module-info.class?),
then class transformation can be accomplished at the same time.

Cheers,
Paul

On Thu, May 12, 2016 at 11:37 AM, David M. Lloyd 
wrote:

> This is along the lines of what I've requested.  Still waiting for any
> kind of feedback though.
>
> On 05/12/2016 11:19 AM, Paul Benedict wrote:
>
>> Why can't the layer assist specifying the Module Configuration to each
>> module in the Module system? By that I mean that the module-info.class
>> shouldn't be read and accepted without mediation. The Layer should get
>> the chance to augment (add, change, remove) anything it wants. Now
>> perhaps the default behavior is to blindly accept module-info.class, but
>> in an EE world, I say this is insufficient. The container should be able
>> to do whatever it wants to the module information before it gets applied
>> by the JDK.
>>
>> Cheers,
>> Paul
>>
>> On Thu, May 12, 2016 at 6:46 AM, David M. Lloyd > > wrote:
>>
>> On 05/12/2016 02:28 AM, Alan Bateman wrote:
>>
>> On 11/05/2016 15:14, David M. Lloyd wrote:
>>
>>
>> We package several hundred JARs in our modular environment
>> today, only
>> some of which originate in-house.  The dependency
>> information for
>> these modules is established not by the author of these
>> JARs, but by
>> us.  The Maven artifact for such a JAR might stipulate
>> certain things,
>> like Log4j or a certain version of ASM.  When we distribute
>> that JAR
>> though, we don't package the exact artifacts and versions of
>> the
>> dependencies that were stipulated in the Maven POM; instead
>> we package
>> single consistent versions which are ABI-compatible with all
>> of its
>> dependents, or maybe even a completely different artifact
>> that meets
>> the same ABI but performs its functions in a different manner.
>>
>> Over time the environment we distribute evolves, and we
>> split or join
>> modules, or we rename old modules to introduce a new major
>> version of
>> the same module in parallel, or we replace one
>> implementation with
>> another.  When we do this we may chose to deprecate or
>> eliminate a
>> module from our environment.  Thus we update all the module
>> descriptors that reference the deprecated module, and set new
>> dependencies on them, and after a certain amount of time, we
>> delete
>> the old module name.
>>
>> None of the artifacts that we package are impacted by this
>> process,
>> and generally no recompilation is necessary: after all, many
>> of these
>> artifacts come directly from Maven or are otherwise built
>> independently at an earlier time outside of the context of
>> our target
>> environment. It's the ABI that matters; as long as that
>> doesn't change
>> (in an incompatible way), recompilation should never be
>> necessary.  I
>> think "recompile just to be safe" puts us squarely into
>> "turn it off
>> and on again" territory, logically speaking.
>>
>> Thanks for the context. So I'm curious what will happen when you
>> download JAR that is a modular JAR where the author have put in
>> the
>> effort to declare their dependences and exports, maybe services
>> too. Are
>> you going to override that too?
>>
>>
>> Almost certainly.  At the very least we're going to review it to see
>> if the module names match with our environment.  If we retain our
>> current modularity system then we'd likely have to externalize the
>> descriptor as well, probably manually, and the internal one would be
>> stripped out or ignored.  But, that is as yet undecided and may
>> depend on whether and how many of our issues get resolved in Jigsaw.
>>
>> --
>> - DML
>>
>>
>>
> --
> - DML
>

Re: module-info.java just causes problems

[David M. Lloyd]

This is along the lines of what I've requested.  Still waiting for any 
kind of feedback though.


On 05/12/2016 11:19 AM, Paul Benedict wrote:

Why can't the layer assist specifying the Module Configuration to each
module in the Module system? By that I mean that the module-info.class
shouldn't be read and accepted without mediation. The Layer should get
the chance to augment (add, change, remove) anything it wants. Now
perhaps the default behavior is to blindly accept module-info.class, but
in an EE world, I say this is insufficient. The container should be able
to do whatever it wants to the module information before it gets applied
by the JDK.

Cheers,
Paul

On Thu, May 12, 2016 at 6:46 AM, David M. Lloyd > wrote:

On 05/12/2016 02:28 AM, Alan Bateman wrote:

On 11/05/2016 15:14, David M. Lloyd wrote:


We package several hundred JARs in our modular environment
today, only
some of which originate in-house.  The dependency
information for
these modules is established not by the author of these
JARs, but by
us.  The Maven artifact for such a JAR might stipulate
certain things,
like Log4j or a certain version of ASM.  When we distribute
that JAR
though, we don't package the exact artifacts and versions of the
dependencies that were stipulated in the Maven POM; instead
we package
single consistent versions which are ABI-compatible with all
of its
dependents, or maybe even a completely different artifact
that meets
the same ABI but performs its functions in a different manner.

Over time the environment we distribute evolves, and we
split or join
modules, or we rename old modules to introduce a new major
version of
the same module in parallel, or we replace one
implementation with
another.  When we do this we may chose to deprecate or
eliminate a
module from our environment.  Thus we update all the module
descriptors that reference the deprecated module, and set new
dependencies on them, and after a certain amount of time, we
delete
the old module name.

None of the artifacts that we package are impacted by this
process,
and generally no recompilation is necessary: after all, many
of these
artifacts come directly from Maven or are otherwise built
independently at an earlier time outside of the context of
our target
environment. It's the ABI that matters; as long as that
doesn't change
(in an incompatible way), recompilation should never be
necessary.  I
think "recompile just to be safe" puts us squarely into
"turn it off
and on again" territory, logically speaking.

Thanks for the context. So I'm curious what will happen when you
download JAR that is a modular JAR where the author have put in the
effort to declare their dependences and exports, maybe services
too. Are
you going to override that too?


Almost certainly.  At the very least we're going to review it to see
if the module names match with our environment.  If we retain our
current modularity system then we'd likely have to externalize the
descriptor as well, probably manually, and the internal one would be
stripped out or ignored.  But, that is as yet undecided and may
depend on whether and how many of our issues get resolved in Jigsaw.

--
- DML




--
- DML

Re: module-info.java just causes problems

[Paul Benedict]

Why can't the layer assist specifying the Module Configuration to each
module in the Module system? By that I mean that the module-info.class
shouldn't be read and accepted without mediation. The Layer should get the
chance to augment (add, change, remove) anything it wants. Now perhaps the
default behavior is to blindly accept module-info.class, but in an EE
world, I say this is insufficient. The container should be able to do
whatever it wants to the module information before it gets applied by the
JDK.

Cheers,
Paul

On Thu, May 12, 2016 at 6:46 AM, David M. Lloyd 
wrote:

> On 05/12/2016 02:28 AM, Alan Bateman wrote:
>
>> On 11/05/2016 15:14, David M. Lloyd wrote:
>>
>>>
>>> We package several hundred JARs in our modular environment today, only
>>> some of which originate in-house.  The dependency information for
>>> these modules is established not by the author of these JARs, but by
>>> us.  The Maven artifact for such a JAR might stipulate certain things,
>>> like Log4j or a certain version of ASM.  When we distribute that JAR
>>> though, we don't package the exact artifacts and versions of the
>>> dependencies that were stipulated in the Maven POM; instead we package
>>> single consistent versions which are ABI-compatible with all of its
>>> dependents, or maybe even a completely different artifact that meets
>>> the same ABI but performs its functions in a different manner.
>>>
>>> Over time the environment we distribute evolves, and we split or join
>>> modules, or we rename old modules to introduce a new major version of
>>> the same module in parallel, or we replace one implementation with
>>> another.  When we do this we may chose to deprecate or eliminate a
>>> module from our environment.  Thus we update all the module
>>> descriptors that reference the deprecated module, and set new
>>> dependencies on them, and after a certain amount of time, we delete
>>> the old module name.
>>>
>>> None of the artifacts that we package are impacted by this process,
>>> and generally no recompilation is necessary: after all, many of these
>>> artifacts come directly from Maven or are otherwise built
>>> independently at an earlier time outside of the context of our target
>>> environment. It's the ABI that matters; as long as that doesn't change
>>> (in an incompatible way), recompilation should never be necessary.  I
>>> think "recompile just to be safe" puts us squarely into "turn it off
>>> and on again" territory, logically speaking.
>>>
>>> Thanks for the context. So I'm curious what will happen when you
>> download JAR that is a modular JAR where the author have put in the
>> effort to declare their dependences and exports, maybe services too. Are
>> you going to override that too?
>>
>
> Almost certainly.  At the very least we're going to review it to see if
> the module names match with our environment.  If we retain our current
> modularity system then we'd likely have to externalize the descriptor as
> well, probably manually, and the internal one would be stripped out or
> ignored.  But, that is as yet undecided and may depend on whether and how
> many of our issues get resolved in Jigsaw.
>
> --
> - DML
>

Re: module-info.java just causes problems

[David M. Lloyd]

On 05/12/2016 02:28 AM, Alan Bateman wrote:

On 11/05/2016 15:14, David M. Lloyd wrote:


We package several hundred JARs in our modular environment today, only
some of which originate in-house.  The dependency information for
these modules is established not by the author of these JARs, but by
us.  The Maven artifact for such a JAR might stipulate certain things,
like Log4j or a certain version of ASM.  When we distribute that JAR
though, we don't package the exact artifacts and versions of the
dependencies that were stipulated in the Maven POM; instead we package
single consistent versions which are ABI-compatible with all of its
dependents, or maybe even a completely different artifact that meets
the same ABI but performs its functions in a different manner.

Over time the environment we distribute evolves, and we split or join
modules, or we rename old modules to introduce a new major version of
the same module in parallel, or we replace one implementation with
another.  When we do this we may chose to deprecate or eliminate a
module from our environment.  Thus we update all the module
descriptors that reference the deprecated module, and set new
dependencies on them, and after a certain amount of time, we delete
the old module name.

None of the artifacts that we package are impacted by this process,
and generally no recompilation is necessary: after all, many of these
artifacts come directly from Maven or are otherwise built
independently at an earlier time outside of the context of our target
environment. It's the ABI that matters; as long as that doesn't change
(in an incompatible way), recompilation should never be necessary.  I
think "recompile just to be safe" puts us squarely into "turn it off
and on again" territory, logically speaking.


Thanks for the context. So I'm curious what will happen when you
download JAR that is a modular JAR where the author have put in the
effort to declare their dependences and exports, maybe services too. Are
you going to override that too?


Almost certainly.  At the very least we're going to review it to see if 
the module names match with our environment.  If we retain our current 
modularity system then we'd likely have to externalize the descriptor as 
well, probably manually, and the internal one would be stripped out or 
ignored.  But, that is as yet undecided and may depend on whether and 
how many of our issues get resolved in Jigsaw.


--
- DML

Name of the module -info in the classfile format Was: module-info.java just causes problems

[forax]

- Mail original -
> De: "Jonathan Gibbons" <jonathan.gibb...@oracle.com>
> À: fo...@univ-mlv.fr, "jigsaw-dev" <jigsaw-dev@openjdk.java.net>
> Envoyé: Mercredi 11 Mai 2016 22:03:09
> Objet: Re: module-info.java just causes problems

Hi Jon,

> 
> 
> 
> On 05/11/2016 11:53 AM, fo...@univ-mlv.fr wrote:
> > I also want to gather experience on the bytecode representation,
> > BTW, the current format has a bug, encoding the name of a module as a part
> > of the class name fell like a hack. It is a bad idea because the JVM Spec
> > doesn't allow some characters in a class name, i think it should be
> > encoded as a Utf8 (at the starts of the Module attribute),
> > it's also weird because the name of the module-info.class is not
> > "module-info" unlike any other classes, this will require to change more
> > code than necessary in a lot of bytecode tools i know.
> 
> package-info.class is a precedent.

No, package-info.java is not stored at the root of a module but in the 
directory containing the class of the package,
so if the source file is java/util/package-info.java, the class file is 
java/util/package-info.class and the name of the class inside the class file is 
java/util/package-info.

for the module-info,
the source file is module-info.java, the class file is module-info.class and 
the class inside the class file is 
whatever/is/the/name/of/my/module/module-info.
 

> 
> -- Jon
> 

Rémi

Re: module-info.java just causes problems

[Alan Bateman]

On 11/05/2016 15:14, David M. Lloyd wrote:


We package several hundred JARs in our modular environment today, only 
some of which originate in-house.  The dependency information for 
these modules is established not by the author of these JARs, but by 
us.  The Maven artifact for such a JAR might stipulate certain things, 
like Log4j or a certain version of ASM.  When we distribute that JAR 
though, we don't package the exact artifacts and versions of the 
dependencies that were stipulated in the Maven POM; instead we package 
single consistent versions which are ABI-compatible with all of its 
dependents, or maybe even a completely different artifact that meets 
the same ABI but performs its functions in a different manner.


Over time the environment we distribute evolves, and we split or join 
modules, or we rename old modules to introduce a new major version of 
the same module in parallel, or we replace one implementation with 
another.  When we do this we may chose to deprecate or eliminate a 
module from our environment.  Thus we update all the module 
descriptors that reference the deprecated module, and set new 
dependencies on them, and after a certain amount of time, we delete 
the old module name.


None of the artifacts that we package are impacted by this process, 
and generally no recompilation is necessary: after all, many of these 
artifacts come directly from Maven or are otherwise built 
independently at an earlier time outside of the context of our target 
environment. It's the ABI that matters; as long as that doesn't change 
(in an incompatible way), recompilation should never be necessary.  I 
think "recompile just to be safe" puts us squarely into "turn it off 
and on again" territory, logically speaking.


Thanks for the context. So I'm curious what will happen when you 
download JAR that is a modular JAR where the author have put in the 
effort to declare their dependences and exports, maybe services too. Are 
you going to override that too?


-Alan

Re: module-info.java just causes problems

[Jonathan Gibbons]

On 05/11/2016 11:53 AM, fo...@univ-mlv.fr wrote:

I also want to gather experience on the bytecode representation,
BTW, the current format has a bug, encoding the name of a module as a part of 
the class name fell like a hack. It is a bad idea because the JVM Spec doesn't 
allow some characters in a class name, i think it should be encoded as a Utf8 
(at the starts of the Module attribute),
it's also weird because the name of the module-info.class is not "module-info" 
unlike any other classes, this will require to change more code than necessary in a lot 
of bytecode tools i know.


package-info.class is a precedent.

-- Jon

Re: module-info.java just causes problems

[forax]

- Mail original -

> De: "Alan Bateman" <alan.bate...@oracle.com>
> À: "Rémi Forax" <fo...@univ-mlv.fr>, jigsaw-dev@openjdk.java.net
> Envoyé: Mercredi 11 Mai 2016 18:56:19
> Objet: Re: module-info.java just causes problems

> On 11/05/2016 16:37, Rémi Forax wrote:

> > :
> 

> > Note that i will update ASM soon to be able to read/write a module-info, so
> > at least there will be a way to update a module-info.class without elying
> > on
> > javac.
> 

> Good to hear. As you probably know, we have asm.Attribute implementation of
> the Module and other attributes and these are already used by the `jar` and
> `jmod` tools when they read or add attributes to module-info.class. All very
> straight-forward.
yes, 
you means the attributes defined in jdk.internal.module.ClassFileAttributes. 
I want to add a visitor API (to allow incremental changes) and re-work the 
Remapper, the tool that help to rename a class or a package, because the 
module-info also need to be updated (exports, uses/provides, conceal attribute, 
etc). 

I also want to gather experience on the bytecode representation, 
BTW, the current format has a bug, encoding the name of a module as a part of 
the class name fell like a hack. It is a bad idea because the JVM Spec doesn't 
allow some characters in a class name, i think it should be encoded as a Utf8 
(at the starts of the Module attribute), 
it's also weird because the name of the module-info.class is not "module-info" 
unlike any other classes, this will require to change more code than necessary 
in a lot of bytecode tools i know. 

> -Alan.

Rémi 

Re: module-info.java just causes problems

[Alan Bateman]

On 11/05/2016 16:37, Rémi Forax wrote:

:

Note that i will update ASM soon to be able to read/write a module-info, so at 
least there will be a way to update a module-info.class without elying on javac.
Good to hear.  As you probably know, we have asm.Attribute 
implementation of the Module and other attributes and these are already 
used by the `jar` and `jmod` tools when they read or add attributes to 
module-info.class. All very straight-forward.


-Alan.

Re: module-info.java just causes problems

[Rémi Forax]

Le 11 mai 2016 17:05:07 CEST, Stephane Epardaud  a écrit :
>Agreed, and this is the conclusion we came to in Ceylon wrt modules
>too. 
>But that's orthogonal to the format of the module descriptor.
>
>I think the Java Modules already have a lot of overriding capabilities 
>in the form of command-line flags (-addExports and friends), but 
>ultimately an external overrides descriptor to encompass all those 
>options will help. 

Will this descriptor different from module-info.java in the end ?

Note that i will update ASM soon to be able to read/write a module-info, so at 
least there will be a way to update a module-info.class without elying on javac.

> Of course, tools could define that and generate the 
>options based on it, but then you will end up with different forms for 
>different build tools which is less than ideal.
>

Maven is the dominant player here, if it comes with a tool, every body will use 
it.

Rémi 

>On 11/05/16 16:57, Gunnar Morling wrote:
>> Having an (external) mechanism allowing to override module
>requirements
>> appears very desirable to me.
>>

Re: module-info.java just causes problems

[Stephane Epardaud]

Agreed, and this is the conclusion we came to in Ceylon wrt modules too. 
But that's orthogonal to the format of the module descriptor.


I think the Java Modules already have a lot of overriding capabilities 
in the form of command-line flags (-addExports and friends), but 
ultimately an external overrides descriptor to encompass all those 
options will help. Of course, tools could define that and generate the 
options based on it, but then you will end up with different forms for 
different build tools which is less than ideal.


On 11/05/16 16:57, Gunnar Morling wrote:

Having an (external) mechanism allowing to override module requirements
appears very desirable to me.

Re: module-info.java just causes problems

[Gunnar Morling]

Recompiling entire modules might not be an option in cases where you don't
have the sources of your dependencies. It might even be that you are not
allowed to javap a module's module-info.java, depending on its licensing
terms.

But also leaving such cases aside, having to re-compile module-info
descriptors for the sake of overriding required modules seems very tedious.

E.g. imagine you hadn't the means of dependency exclusions in Maven (in
your application's pom.xml) but were required to patch the pom.xml files of
your dependencies in order to converge to a consistent set of logging
dependencies.

Having an (external) mechanism allowing to override module requirements
appears very desirable to me.



2016-05-11 16:14 GMT+02:00 David M. Lloyd :

> On 05/11/2016 08:15 AM, Alan Bateman wrote:
>
>>
>> On 11/05/2016 13:52, David M. Lloyd wrote:
>>
>>>
>>> In practice this happens a lot.  A module's dependency graph depends
>>> just as much on the environment as it depends on the classes in the
>>> module (if not more so).  Modules are merged and split, replaced with
>>> compatibility stubs, renamed, etc.  If you have to recompile every
>>> module for every environment, a lot of the benefit of modularity and
>>> compatibility-by-ABI is lost; changes to a module in an environment
>>> would lead to a massive cascade of recompilation.
>>>
>> The module system can support many refactoring scenarios, including
>> merging and splitting. I can see myself refactoring modules that I
>> maintain, I'm less sure that I want to refactor modules coming from
>> other projects. If I found myself refactoring modules from elsewhere
>> then I would expect to have to build and test those modules, it would be
>> strange not to do and I don't understand how you can get away with this
>> when changing code in those modules. It might be useful if you could lay
>> out a specific scenario as it may be that we are talking completely
>> different things.
>>
>
> We package several hundred JARs in our modular environment today, only
> some of which originate in-house.  The dependency information for these
> modules is established not by the author of these JARs, but by us.  The
> Maven artifact for such a JAR might stipulate certain things, like Log4j or
> a certain version of ASM.  When we distribute that JAR though, we don't
> package the exact artifacts and versions of the dependencies that were
> stipulated in the Maven POM; instead we package single consistent versions
> which are ABI-compatible with all of its dependents, or maybe even a
> completely different artifact that meets the same ABI but performs its
> functions in a different manner.
>
> Over time the environment we distribute evolves, and we split or join
> modules, or we rename old modules to introduce a new major version of the
> same module in parallel, or we replace one implementation with another.
> When we do this we may chose to deprecate or eliminate a module from our
> environment.  Thus we update all the module descriptors that reference the
> deprecated module, and set new dependencies on them, and after a certain
> amount of time, we delete the old module name.
>
> None of the artifacts that we package are impacted by this process, and
> generally no recompilation is necessary: after all, many of these artifacts
> come directly from Maven or are otherwise built independently at an earlier
> time outside of the context of our target environment. It's the ABI that
> matters; as long as that doesn't change (in an incompatible way),
> recompilation should never be necessary.  I think "recompile just to be
> safe" puts us squarely into "turn it off and on again" territory, logically
> speaking.
>
> --
> - DML
>

Re: module-info.java just causes problems

[Stephane Epardaud]

I expect such a tool to generate module-info.class to materialise. I 
personally use ClassWriter from the JDK tools for that. Anyone with a 
JDK has that module and can write that tool trivially.


The JBoss Modules use-case (and the Ceylon distrib one) is rather 
uncommon, though, IMO. Most people deal with modules as Maven modules 
and they can't really rewrite the descriptors as in most cases they're 
not in the jar and Maven will only use descriptors from the repos and 
not in any flat folder where you have Maven jars (I think). In 
Maven-land you deal with this via external (to the modules being 
overridden) overrides, which Java Modules don't support ATM.


OSGi is already a pain to override because you have to extract the 
manifest, edit it and put it back. Here a tool would help. Overriding 
Ceylon modules is done via external overrides too, otherwise you'd have 
to extract the manifest (which is also a .class), edit it and put it 
back, for which we'd have generated tooling if there was ever demand for 
that (but there hasn't been yet).


All that to say, in general after-packaging editing of module 
descriptors is already either rare or hard. I don't think 
module-info.class changes that much.


On 11/05/16 16:44, David M. Lloyd wrote:

On 05/11/2016 09:21 AM, Stephane Epardaud wrote:

I don't understand the problem with the module-info.java class. We also
repackage jars to add module descriptors (for OSGi, Maven and JBoss
Modules) and for Jigsaw we generate the `module-info.class` file when we
need to repackage them. Between this binary format and XML or
MANIFEST.MF, I don't see why it makes a big difference?

I wouldn't recompile the module, though, just as I don't recompile them
when we change the other manifests.


If you have a .class file, you have to generate it with something.  I 
think Alan is arguing that it should be done with javac, but we'd 
likely end up using something else.  In our case, just editing an XML 
file and doing "git commit" is easier than running a tool to read and 
update a binary descriptor inside an artifact though, particularly as 
we don't package the actual artifacts.  Thus we'd have to package a 
textual descriptor of some sort, to which we'd then have to apply a 
tool in order to update our final packaged artifacts, making it that 
much more difficult to use artifacts that already exist (e.g. from 
Maven).


In addition it is common for our users to examine or modify module 
descriptors; having them be external is very useful for this purpose.


We also allow features like specification of properties for modules 
which can be read, specification of module permission sets, and 
filtering of artifact resources and module dependencies in various ways.

Re: module-info.java just causes problems

[David M. Lloyd]

On 05/11/2016 09:21 AM, Stephane Epardaud wrote:

I don't understand the problem with the module-info.java class. We also
repackage jars to add module descriptors (for OSGi, Maven and JBoss
Modules) and for Jigsaw we generate the `module-info.class` file when we
need to repackage them. Between this binary format and XML or
MANIFEST.MF, I don't see why it makes a big difference?

I wouldn't recompile the module, though, just as I don't recompile them
when we change the other manifests.


If you have a .class file, you have to generate it with something.  I 
think Alan is arguing that it should be done with javac, but we'd likely 
end up using something else.  In our case, just editing an XML file and 
doing "git commit" is easier than running a tool to read and update a 
binary descriptor inside an artifact though, particularly as we don't 
package the actual artifacts.  Thus we'd have to package a textual 
descriptor of some sort, to which we'd then have to apply a tool in 
order to update our final packaged artifacts, making it that much more 
difficult to use artifacts that already exist (e.g. from Maven).


In addition it is common for our users to examine or modify module 
descriptors; having them be external is very useful for this purpose.


We also allow features like specification of properties for modules 
which can be read, specification of module permission sets, and 
filtering of artifact resources and module dependencies in various ways.

--
- DML

Re: module-info.java just causes problems

[Stephane Epardaud]

I don't understand the problem with the module-info.java class. We also 
repackage jars to add module descriptors (for OSGi, Maven and JBoss 
Modules) and for Jigsaw we generate the `module-info.class` file when we 
need to repackage them. Between this binary format and XML or 
MANIFEST.MF, I don't see why it makes a big difference?


I wouldn't recompile the module, though, just as I don't recompile them 
when we change the other manifests.

Re: module-info.java just causes problems

[David M. Lloyd]

On 05/11/2016 08:15 AM, Alan Bateman wrote:


On 11/05/2016 13:52, David M. Lloyd wrote:


In practice this happens a lot.  A module's dependency graph depends
just as much on the environment as it depends on the classes in the
module (if not more so).  Modules are merged and split, replaced with
compatibility stubs, renamed, etc.  If you have to recompile every
module for every environment, a lot of the benefit of modularity and
compatibility-by-ABI is lost; changes to a module in an environment
would lead to a massive cascade of recompilation.

The module system can support many refactoring scenarios, including
merging and splitting. I can see myself refactoring modules that I
maintain, I'm less sure that I want to refactor modules coming from
other projects. If I found myself refactoring modules from elsewhere
then I would expect to have to build and test those modules, it would be
strange not to do and I don't understand how you can get away with this
when changing code in those modules. It might be useful if you could lay
out a specific scenario as it may be that we are talking completely
different things.


We package several hundred JARs in our modular environment today, only 
some of which originate in-house.  The dependency information for these 
modules is established not by the author of these JARs, but by us.  The 
Maven artifact for such a JAR might stipulate certain things, like Log4j 
or a certain version of ASM.  When we distribute that JAR though, we 
don't package the exact artifacts and versions of the dependencies that 
were stipulated in the Maven POM; instead we package single consistent 
versions which are ABI-compatible with all of its dependents, or maybe 
even a completely different artifact that meets the same ABI but 
performs its functions in a different manner.


Over time the environment we distribute evolves, and we split or join 
modules, or we rename old modules to introduce a new major version of 
the same module in parallel, or we replace one implementation with 
another.  When we do this we may chose to deprecate or eliminate a 
module from our environment.  Thus we update all the module descriptors 
that reference the deprecated module, and set new dependencies on them, 
and after a certain amount of time, we delete the old module name.


None of the artifacts that we package are impacted by this process, and 
generally no recompilation is necessary: after all, many of these 
artifacts come directly from Maven or are otherwise built independently 
at an earlier time outside of the context of our target environment. 
It's the ABI that matters; as long as that doesn't change (in an 
incompatible way), recompilation should never be necessary.  I think 
"recompile just to be safe" puts us squarely into "turn it off and on 
again" territory, logically speaking.


--
- DML

Re: module-info.java just causes problems

[Alan Bateman]

On 11/05/2016 13:52, David M. Lloyd wrote:


In practice this happens a lot.  A module's dependency graph depends 
just as much on the environment as it depends on the classes in the 
module (if not more so).  Modules are merged and split, replaced with 
compatibility stubs, renamed, etc.  If you have to recompile every 
module for every environment, a lot of the benefit of modularity and 
compatibility-by-ABI is lost; changes to a module in an environment 
would lead to a massive cascade of recompilation.
The module system can support many refactoring scenarios, including 
merging and splitting. I can see myself refactoring modules that I 
maintain, I'm less sure that I want to refactor modules coming from 
other projects. If I found myself refactoring modules from elsewhere 
then I would expect to have to build and test those modules, it would be 
strange not to do and I don't understand how you can get away with this 
when changing code in those modules. It might be useful if you could lay 
out a specific scenario as it may be that we are talking completely 
different things.


-Alan.

Re: module-info.java just causes problems

[David M. Lloyd]

On 05/11/2016 07:39 AM, Alan Bateman wrote:

On 11/05/2016 08:45, Gunnar Morling wrote:

:

Assuming a developer needs to change the module requirements of one of
the
modules they use, this means - as far as I understand the current
design -
they need to get the source of the module's module-info.java, adapt it as
needed and re-compile it.

Assuming there are actually good reasons for changing it then I would
hope you would recompile the module completely, not just module
declaration as otherwise it may not be in sync with the code in the module.


In practice this happens a lot.  A module's dependency graph depends 
just as much on the environment as it depends on the classes in the 
module (if not more so).  Modules are merged and split, replaced with 
compatibility stubs, renamed, etc.  If you have to recompile every 
module for every environment, a lot of the benefit of modularity and 
compatibility-by-ABI is lost; changes to a module in an environment 
would lead to a massive cascade of recompilation.

--
- DML

Re: module-info.java just causes problems

[Remi Forax]

Like Alan,
i hope you will re-compile the module, it's more safe.

Otherwise, you can use javap to dump the actual module-info.java from the 
module-info.class,
patch it, recompile it with javac and then do a jar --update to replace the 
module-info.class in the jar

cheers,
Rémi

- Mail original -
> De: "Alan Bateman" <alan.bate...@oracle.com>
> À: "Gunnar Morling" <gun...@hibernate.org>
> Cc: "jigsaw-dev" <jigsaw-dev@openjdk.java.net>
> Envoyé: Mercredi 11 Mai 2016 14:39:53
> Objet: Re: module-info.java just causes problems
> 
> On 11/05/2016 08:45, Gunnar Morling wrote:
> > :
> >
> > Assuming a developer needs to change the module requirements of one of the
> > modules they use, this means - as far as I understand the current design -
> > they need to get the source of the module's module-info.java, adapt it as
> > needed and re-compile it.
> Assuming there are actually good reasons for changing it then I would
> hope you would recompile the module completely, not just module
> declaration as otherwise it may not be in sync with the code in the module.
> 
> -Alan
> 

Re: module-info.java just causes problems

[Alan Bateman]

On 11/05/2016 08:45, Gunnar Morling wrote:

:

Assuming a developer needs to change the module requirements of one of the
modules they use, this means - as far as I understand the current design -
they need to get the source of the module's module-info.java, adapt it as
needed and re-compile it.
Assuming there are actually good reasons for changing it then I would 
hope you would recompile the module completely, not just module 
declaration as otherwise it may not be in sync with the code in the module.


-Alan

Re: module-info.java just causes problems

[Paul Benedict]

It's tough to rehash issues. I don't want to re-open old discussions that
go nowhere, yet, I still share David's concerns. Personally, I was very
jealous :-) of the proprietary module.xml the initial jigsaw system had. I
liked the XML way better and still do.

Cheers,
Paul

On Wed, May 4, 2016 at 1:33 PM, David M. Lloyd 
wrote:

> Tools like Maven and JUnit are yet still having a difficult time coping
> with the oddball module-info.java file.  Can we just drop this thing and
> let the layer provide metadata for the module?  This way of defining module
> metadata is clearly causing trouble.  Reference: all previous arguments
> from non-Oracle people over the last 5 years or so.
>
> So far none of our software actually functions with Jigsaw, and the
> prospect is not improving.  In the meantime the EG is alternately
> completely dead or in a state of
> we'll-register-your-issue-but-cannot-help-resolve-them mode.  I'm highly
> concerned.
> --
> - DML
>

module-info.java just causes problems

[David M. Lloyd]

Tools like Maven and JUnit are yet still having a difficult time coping 
with the oddball module-info.java file.  Can we just drop this thing and 
let the layer provide metadata for the module?  This way of defining 
module metadata is clearly causing trouble.  Reference: all previous 
arguments from non-Oracle people over the last 5 years or so.


So far none of our software actually functions with Jigsaw, and the 
prospect is not improving.  In the meantime the EG is alternately 
completely dead or in a state of 
we'll-register-your-issue-but-cannot-help-resolve-them mode.  I'm highly 
concerned.

--
- DML