The baby and the bathwater

2018-03-26 Thread mark . reinhold
Stephen Colebourne's recent blog entry [1] contains many true statements,
along with some reasonable advice for library maintainers.  To summarize:

  - As of Java 9, with Jigsaw, there are two ways in which a library can
be used: Either on the traditional class path, or on the newfangled
module path.  If you maintain a library but don't modularize it then
it can still -- unbeknownst to you -- be used on the module path as
an automatic module.

  - When code runs on the module path there are some differences in the
behavior of some pre-9 APIs, in particular those related to resource
lookup.

  - As a consequence, if you maintain a library that's intended to work
on Java 9 or later then you should test it on both the class path
and the module path, even if you do nothing to convert your library
to a module.  If your library doesn't work on the module path then
you should either fix it or document that limitation.

  - If you don't modularize your library, or at least claim an automatic
module name for it via the `Automatic-Module-Name` manifest entry,
then you potentially block the maintainers of libraries that depend
upon yours from modularizing their own libraries.

  - The tools that we use, and in particular Maven, could be improved.
It's difficult to compile the classes for a modular JAR file that's
intended to work on the class path of pre-9 releases, it's difficult
to test a library on both the class path and the module path, and
various Maven plugins still need to be upgraded to handle (or else
ignore) `module-info.java` files.  (Stephen has helpfully filed
issues in the appropriate bug trackers for some of these problems.)

  - Some old tools, bytecode libraries, and other systems fail when they
encounter `module-info.class` files or multi-release JAR files.

>From these points Stephen concludes that the module system, "as currently
designed, has 'negative benefits' for open source libraries," saying that
this is primarily because "the split (bifurcation) of the module-path
from the class-path is an absolute nightmare."

Hyperbole aside, Stephen's main complaint here is only about the need
to test a library on both the class path and the module path if it's
intended to work on Java 9 or later.  With automated testing this
shouldn't, in principle, be a huge burden, but still it's worth asking
the question: Could we have avoided the need for such dual testing if
we hadn't introduced the module path as separate from the class path?

Consider, as a thought experiment, an alternative Jigsaw design that
didn't have a separate module path, and instead treated modular JARs
on the class path as modules rather than traditional JAR files.  You
wouldn't have to dual-test if your baseline is Java 9 or later, but
if you want to support earlier releases with the same artifact then
you'd still have to test on the class path.

With the actual Jigsaw design you do need to dual-test your library
when your baseline is Java 9 or later.  There is, however, a benefit
to this: If someone uses your library in an application that works on
the Java 8 class path today then they can migrate it to the Java 9 (or
later) class path and then, when they're ready, move your library (and
perhaps some others) over to the module path.  (There were many other
reasons to define the module path as separate from the class path, but
those aren't directly relevant here.)

The tradeoff, then, is a little bit more dual testing on the part of
library maintainers in exchange for greater flexibility for those who
will migrate existing applications to Java 9 or later releases.  Many
library maintainers will be reluctant to baseline to Java 9 (or later)
for a while yet, so they'll be dual-testing anyway, so I think this
was the right tradeoff.

Stephen closes with a specific suggestion:

  "There needs to be a way for a library author to insist that the
   modular jar file they are producing can only be run on the module-path
   (with any attempt to use it on the class-path preventing application
   startup).  This would eliminate the need for testing both class-path
   and module-path."

Yes, this would eliminate the need for dual testing, but only if you're
willing to baseline to Java 11.  As with a unified class/module path,
however, if you want your library to work on earlier releases then you'd
also, still, have to test on the class path, and you'd make it harder for
application maintainers to migrate old class-path applications.  I don't
think this idea is worth pursuing.

What ideas are worth pursuing?  We should, by all means, continue to
improve our tools, Jigsaw itself, and the rest of the JDK.  Several of
us here collaborated on the initial support for modular development in
Maven, but clearly there's more to do.  If nothing else, the Surefire
plugin should be able to test a library on both the class path and the
module path.  There's also at least one improv

Re: The baby and the bathwater

2018-03-26 Thread Stephen Colebourne
On 26 March 2018 at 19:08,   wrote:
> Stephen Colebourne's recent blog entry

Thanks for the thoughtful reply, of which I agree with much of it.

> Stephen's main complaint here is only about the need
> to test a library on both the class path and the module path if it's
> intended to work on Java 9 or later.  With automated testing this
> shouldn't, in principle, be a huge burden,

To a degree this depends on the size of your test suite. Some suites
are large, and running the entire suite twice in continuous
integration could be onerous.

I think the main complaint however is more subtle than a need to test
twice. It is the need to test twice _forevermore_.

ie. if this were just a transition phase, which would pass when Java
11 is the baseline, the situation would be painful but manageable. But
as things stand, there is no future time when my module will be
guaranteed to be treated as a module.

> Stephen closes with a specific suggestion:
> [snip]
> Yes, this would eliminate the need for dual testing, but only if you're
> willing to baseline to Java 11.

And that is exactly the point. At some point, Java 11 will be the new
baseline. The trade-offs jigsaw chose for the Java 8 to 9 transition
will at some point not be the right ones for the long term. There has
to be a time when a library developer can rely on strong encapsulation
and when the class-path can't just be used to completely nullify
module-info.java. Otherwise, whats the point in modularisation?

I'm arguing that it should be the module authors choice to apply the
tougher rules, but I'm very happy to hear other alternatives in the
problem-space.

> There's also at least one improvement in the JDK worth
> considering, which a few of us have already discussed, namely a small
> enhancement to javac that would allow a single invocation to compile a
> module, in module mode [2], but target class files other than
> `module-info.class` to an earlier release [3].

+1

Stephen