Re: m-javadoc-plugin with JPMS

[Daniel B. Widdis]

It's amusing when you come upon an error, google the error message, and
find an email thread you participated in and forgot about.

Resurrecting this thread from last July and hoping for some insight.

I last released my project using maven-release-plugin on July 20, 2022 with
no issues.  My project setup includes a parent pom and modules
 - one module builds with compatibility for Java 8 and has
automatic-module-name
 - another module builds
 - another module contains only the src/main/java/module-info.java file,
and uses the maven-resources-plugin to add source and resource paths to the
other module's source tree
 - I have made no significant changes to my POM since the previous release,
and it works just fine for test, verify, deploy goals, but breaks on
maven release:perform with the message cited earlier in this thread:

[ERROR] Creating an aggregated report for both named and unnamed modules is
not possible.
[ERROR] Ensure that every module has a module descriptor or is a jar with a
MANIFEST.MF containing an Automatic-Module-Name.
[ERROR] Fix the following projects:
[ERROR]  - com.github.oshi:oshi-core-java11

The error message is not helpful, as that module descriptor IS in that
module.

I can reproduce the error on the commandline by running mvn javadoc:jar.
 I can work around it by running mvn package javadoc:jar.  However,
attempting to add the package goal to the maven release plugin does not fix
the problem.

I have tried adding an automatic module name in addition to the module
descriptor.  Same problem.

I have tried changing the lifecycle phase during which these javadoc goals
are executed.  Same problem.

I have tried adding "dummy" class files in the problematic module's source
tree just in case the solo file and lack of support for the added source
files is an issue.  Same problem.

I have traced the error down to the following code:

The log error message is generated in AbstractJavadocMogo line 5085 when
the unnamedProjects list is not empty.
https://github.com/apache/maven-javadoc-plugin/blob/8b589325762280cef9ff8df2bc3431d4fccafbc7/src/main/java/org/apache/maven/plugins/javadoc/AbstractJavadocMojo.java#L5087-L5094

That occurs when the conditional on line 5046 (result not null and
getModuleDescriptor not null) fails.
https://github.com/apache/maven-javadoc-plugin/blob/8b589325762280cef9ff8df2bc3431d4fccafbc7/src/main/java/org/apache/maven/plugins/javadoc/AbstractJavadocMojo.java#L5046

The result variable checks two sources.  First, it looks for a jar with
getClassesFile() if it exists, and selects the automatic module name from
it.  Even if result fails here, it does not bother to check for a
descriptor elsewhere due to the if/else.
https://github.com/apache/maven-javadoc-plugin/blob/8b589325762280cef9ff8df2bc3431d4fccafbc7/src/main/java/org/apache/maven/plugins/javadoc/AbstractJavadocMojo.java#L5003

Tracing through getClasses() it eventually returns the output directory if
it's not null, so in the case there's no jar in the output directory, I
think this fails and never gets to the "else" case (findMainDescriptor())
where it actually looks for module-info.java
https://github.com/apache/maven-javadoc-plugin/blob/8b589325762280cef9ff8df2bc3431d4fccafbc7/src/main/java/org/apache/maven/plugins/javadoc/AbstractJavadocMojo.java#L5031

I believe this may be the bug reported earlier in this thread by Maxim
Solodovnik, and may be the reason performing "package" first works as a
workaround.

However, I attempted to add an automatic module name (in addition to adding
the package goal for the release) and still got this problem so this may
not be the only issue.

Hopefully these hints can help someone try to get to the source of this
bug...


On Sun, Aug 28, 2022 at 10:25 PM Maxim Solodovnik 
wrote:

> maven-javadoc-plugin:3.4.1 - same error :(((
>
> On Wed, 20 Jul 2022 at 22:19, Maxim Solodovnik 
> wrote:
> >
> > done,
> >
> > same error :(
> >
> > On Wed, 20 Jul 2022 at 22:13, Maxim Solodovnik 
> wrote:
> > >
> > > well,
> > >
> > > m-javadoc-plugin is being called at `*db` module only which is only
> > > depends on `*util` module
> > > both has `module-info.java`
> > >
> > > let me add `module-info.java` to all modules (not sure why) so we can
> > > ensure this is not a problem :)
> > >
> > > On Wed, 20 Jul 2022 at 21:51, Daniel Widdis  wrote:
> > > >
> > > > Some modules have module-info.java and some don’t.  That’s exactly
> what the error says.
> > > >
> > > > > On Jul 20, 2022, at 7:24 AM, Maxim Solodovnik <
> solomax...@gmail.com> wrote:
> > > > >
> > > > > On Tue, 19 Jul 2022 at 17:20, Olivier Lamy 
> wrote:
> > > > >>
> > > > >>> On Tue, 19 Jul 2022 at 16:42, Maxim Solodovnik <
> solomax...@gmail.com> wrote:
> > > > >>>
> > > > >>> On Tue, 19 Jul 2022 at 13:16, Daniel Widdis 
> wrote:
> > > > 
> > > > > The only problem of having both is redundancy :)
> > > > 
> > > >  Technically speaking, you always have an automatic module name
> even if
> > > > >>> you don't expl

Re: Shading, relocation and modules?

[Daniel B. Widdis]

> My question is what is the correct/preferred/recommended way to do this?

Not combine shading and JPMS! :-)

But seriously.  Have a look at Moditect.  It has the capability of adding a
module-descriptor to a JAR after the fact, and may suit your needs.  I
honestly can't think of any other sane way to go about this.

moditect/moditect: Tooling for the Java Module System (github.com)





On Mon, Apr 11, 2022 at 1:29 PM Niels Basjes  wrote:

> Hi,
>
> I have a project ( https://yauaa.basjes.nl/ ) which is a library that is
> used by others as a dependency.
> My project uses (among other things) Antlr4 as a building block.
> Antlr4 is a code generator that is (rightfully) very strict about the
> Antlr4-runtime version available at runtime: they must be identical
> versions.
>
> I do not want to impose "my" Antlr4 version onto the users of my library so
> I have shaded and relocated the correct Antlr4 version into my final jar.
> Now anyone can use it next to a different Antrl4 version in the same
> application without any conflicts.
>
> Entering JPMS...
>
> I'm trying to make my library as "JPMS" as possible.
>
> From what I understand at this time: when I shade some dependencies into my
> final jar the modules-info is removed by the maven-shade-plugin as this
> would break everything.
>
> In general I agree with this; yet in my case all of the shaded
> dependencies:
> 1) Have been relocated to a new package that does not exist anywhere else (
> nl.basjes.shaded. ).
> 2) None of these should be visible outside the module.
>
> So in my mind these classes are modified to become "my" classes as they are
> moved to my namespace and I expect them to fall under similar rules as the
> classes I have created.
>
> Yet I have not yet been successful in making all of this work.
>
> My question is what is the correct/preferred/recommended way to do this?
>
> --
> Best regards / Met vriendelijke groeten,
>
> Niels Basjes
>


-- 
Dan Widdis

Re: Possible file separator bug in 3.10.0 using bnd-maven-plugin

[Daniel B. Widdis]

OK, so I didn't reach the end of my investigation.  I did just look at the
Java specifications for the class file format [1].  They specify the
internal representation of packages always replace periods with forward
slashes (for historical reasons).   So the binary class file generated by
the ClassWriter on Windows is incorrect.

Looking through its code, it faithfully stores the name given it as the
third argument to visit().

So the bug is this argument (line 1356) which simply passes the parsed
path, which on Windows contains backslashes.

javaFile.substring( 0, javaFile.length() - ".java".length() )

I think I have enough info to file a Jira now.

[1] -
https://docs.oracle.com/javase/specs/jvms/se7/html/jvms-4.html#jvms-4.2.1

On Mon, Feb 14, 2022 at 2:52 PM Daniel B. Widdis  wrote:

> I believe I've traced the problem to the binary inside the generated
> package-info.class files.  The path stored internally there uses the
> backslash, while all other class files have forward slashes on Windows.
>
> That appears to be written by the Classwriter (lines 1352-1358).   Debug
> lines show the first call to visitSource have a null/empty file name.
>
> Documentation here [1] states "The methods of this class must be called in
> the following order: visit [ visitSource ]" but in the PR the visitSource
> is called first.
>
> I've now reached the end of my uninformed debugging/speculation.
>
> [1] - ClassVisitor (ASM 9.2) (ow2.io)
> <https://asm.ow2.io/javadoc/org/objectweb/asm/ClassVisitor.html>
>
> On Mon, Feb 14, 2022 at 1:40 PM Daniel B. Widdis  wrote:
>
>> I've confirmed this PR is the relevant problem as the symptoms can be
>> suppressed by using
>>
>> mvn clean test "-Dmaven.compiler.createMissingPackageInfoClass=false"
>>
>>
>> On Mon, Feb 14, 2022 at 9:50 AM Greg Chabala 
>> wrote:
>>
>>> Just my opinion, https://github.com/apache/maven-compiler-plugin/pull/88
>>> feels like a hack solution.
>>>
>>> Generating classes for package-info.java files, just to appease some
>>> issue
>>> with the incremental compiler? That seems like it should have been
>>> filtered
>>> in the compile plugin, not litter the build with classes that have no
>>> function.
>>>
>>> On Mon, Feb 14, 2022 at 10:56 AM Daniel B. Widdis 
>>> wrote:
>>>
>>> > I just bumped the maven-compiler-plugin version on my project from
>>> 3.9.0 to
>>> > 3.10.0 and it caused Windows-based builds to fail maven-bnd-plugin
>>> tests
>>> > for package-info files in the "wrong" directory.  The symptom looks to
>>> be
>>> > associated with the windows file separator.
>>> >
>>> > Sample failure looks like:
>>> > [INFO] Compiling 386 source files to
>>> > D:\a\oshi\oshi\oshi-core\target\classes
>>> > [INFO]
>>> > [INFO] --- bnd-maven-plugin:6.1.0:bnd-process (default) @ oshi-core ---
>>> > Error:  D:\a\oshi\oshi\oshi-core\pom.xml [0:0]: Classes found in the
>>> wrong
>>> > directory:
>>> >
>>> >
>>> {oshi/jna/platform/unix/package-info.class=oshi\jna\platform\unix\package-info,
>>> >
>>> >
>>> oshi/driver/unix/solaris/disk/package-info.class=oshi\driver\unix\solaris\disk\package-info,
>>> >
>>> >
>>> oshi/hardware/platform/mac/package-info.class=oshi\hardware\platform\mac\package-info,
>>> > oshi/software/os/package-info.class=oshi\software\os\package-info,
>>> >
>>> >
>>> oshi/software/os/linux/package-info.class=oshi\software\os\linux\package-info,
>>> > ... (many more)
>>> >
>>> > While the failure is in another plugin, is it possible something in
>>> this PR
>>> > didn't handle the change?
>>> >  [MCOMPILER-205] Add a boolean to generate missing package-info
>>> classes by
>>> > default by gnodet · Pull Request #88 · apache/maven-compiler-plugin
>>> > (github.com) <
>>> > https://github.com/apache/maven-compiler-plugin/pull/88/files>
>>> >
>>> >
>>> > I've looked over the code and don't see anything obvious there.
>>> >
>>> > I do have package-info files in all my packages so new ones should not
>>> have
>>> > been generated, but it seems new ones may have.
>>> >
>>> > Full test run here:  Bump maven-compiler-plugin from 3.9.0 to 3.10.0 ·
>>> > oshi/oshi@471641b (github.com)
>>> > <https://github.com/oshi/oshi/runs/5184078693?check_suite_focus=true>
>>> >
>>> > --
>>> > Dan Widdis
>>> >
>>>
>>
>>
>> --
>> Dan Widdis
>>
>
>
> --
> Dan Widdis
>


-- 
Dan Widdis

Re: Possible file separator bug in 3.10.0 using bnd-maven-plugin

[Daniel B. Widdis]

I believe I've traced the problem to the binary inside the generated
package-info.class files.  The path stored internally there uses the
backslash, while all other class files have forward slashes on Windows.

That appears to be written by the Classwriter (lines 1352-1358).   Debug
lines show the first call to visitSource have a null/empty file name.

Documentation here [1] states "The methods of this class must be called in
the following order: visit [ visitSource ]" but in the PR the visitSource
is called first.

I've now reached the end of my uninformed debugging/speculation.

[1] - ClassVisitor (ASM 9.2) (ow2.io)
<https://asm.ow2.io/javadoc/org/objectweb/asm/ClassVisitor.html>

On Mon, Feb 14, 2022 at 1:40 PM Daniel B. Widdis  wrote:

> I've confirmed this PR is the relevant problem as the symptoms can be
> suppressed by using
>
> mvn clean test "-Dmaven.compiler.createMissingPackageInfoClass=false"
>
>
> On Mon, Feb 14, 2022 at 9:50 AM Greg Chabala 
> wrote:
>
>> Just my opinion, https://github.com/apache/maven-compiler-plugin/pull/88
>> feels like a hack solution.
>>
>> Generating classes for package-info.java files, just to appease some issue
>> with the incremental compiler? That seems like it should have been
>> filtered
>> in the compile plugin, not litter the build with classes that have no
>> function.
>>
>> On Mon, Feb 14, 2022 at 10:56 AM Daniel B. Widdis 
>> wrote:
>>
>> > I just bumped the maven-compiler-plugin version on my project from
>> 3.9.0 to
>> > 3.10.0 and it caused Windows-based builds to fail maven-bnd-plugin tests
>> > for package-info files in the "wrong" directory.  The symptom looks to
>> be
>> > associated with the windows file separator.
>> >
>> > Sample failure looks like:
>> > [INFO] Compiling 386 source files to
>> > D:\a\oshi\oshi\oshi-core\target\classes
>> > [INFO]
>> > [INFO] --- bnd-maven-plugin:6.1.0:bnd-process (default) @ oshi-core ---
>> > Error:  D:\a\oshi\oshi\oshi-core\pom.xml [0:0]: Classes found in the
>> wrong
>> > directory:
>> >
>> >
>> {oshi/jna/platform/unix/package-info.class=oshi\jna\platform\unix\package-info,
>> >
>> >
>> oshi/driver/unix/solaris/disk/package-info.class=oshi\driver\unix\solaris\disk\package-info,
>> >
>> >
>> oshi/hardware/platform/mac/package-info.class=oshi\hardware\platform\mac\package-info,
>> > oshi/software/os/package-info.class=oshi\software\os\package-info,
>> >
>> >
>> oshi/software/os/linux/package-info.class=oshi\software\os\linux\package-info,
>> > ... (many more)
>> >
>> > While the failure is in another plugin, is it possible something in
>> this PR
>> > didn't handle the change?
>> >  [MCOMPILER-205] Add a boolean to generate missing package-info classes
>> by
>> > default by gnodet · Pull Request #88 · apache/maven-compiler-plugin
>> > (github.com) <
>> > https://github.com/apache/maven-compiler-plugin/pull/88/files>
>> >
>> >
>> > I've looked over the code and don't see anything obvious there.
>> >
>> > I do have package-info files in all my packages so new ones should not
>> have
>> > been generated, but it seems new ones may have.
>> >
>> > Full test run here:  Bump maven-compiler-plugin from 3.9.0 to 3.10.0 ·
>> > oshi/oshi@471641b (github.com)
>> > <https://github.com/oshi/oshi/runs/5184078693?check_suite_focus=true>
>> >
>> > --
>> > Dan Widdis
>> >
>>
>
>
> --
> Dan Widdis
>


-- 
Dan Widdis

Re: Possible file separator bug in 3.10.0 using bnd-maven-plugin

[Daniel B. Widdis]

I've confirmed this PR is the relevant problem as the symptoms can be
suppressed by using

mvn clean test "-Dmaven.compiler.createMissingPackageInfoClass=false"


On Mon, Feb 14, 2022 at 9:50 AM Greg Chabala  wrote:

> Just my opinion, https://github.com/apache/maven-compiler-plugin/pull/88
> feels like a hack solution.
>
> Generating classes for package-info.java files, just to appease some issue
> with the incremental compiler? That seems like it should have been filtered
> in the compile plugin, not litter the build with classes that have no
> function.
>
> On Mon, Feb 14, 2022 at 10:56 AM Daniel B. Widdis 
> wrote:
>
> > I just bumped the maven-compiler-plugin version on my project from 3.9.0
> to
> > 3.10.0 and it caused Windows-based builds to fail maven-bnd-plugin tests
> > for package-info files in the "wrong" directory.  The symptom looks to be
> > associated with the windows file separator.
> >
> > Sample failure looks like:
> > [INFO] Compiling 386 source files to
> > D:\a\oshi\oshi\oshi-core\target\classes
> > [INFO]
> > [INFO] --- bnd-maven-plugin:6.1.0:bnd-process (default) @ oshi-core ---
> > Error:  D:\a\oshi\oshi\oshi-core\pom.xml [0:0]: Classes found in the
> wrong
> > directory:
> >
> >
> {oshi/jna/platform/unix/package-info.class=oshi\jna\platform\unix\package-info,
> >
> >
> oshi/driver/unix/solaris/disk/package-info.class=oshi\driver\unix\solaris\disk\package-info,
> >
> >
> oshi/hardware/platform/mac/package-info.class=oshi\hardware\platform\mac\package-info,
> > oshi/software/os/package-info.class=oshi\software\os\package-info,
> >
> >
> oshi/software/os/linux/package-info.class=oshi\software\os\linux\package-info,
> > ... (many more)
> >
> > While the failure is in another plugin, is it possible something in this
> PR
> > didn't handle the change?
> >  [MCOMPILER-205] Add a boolean to generate missing package-info classes
> by
> > default by gnodet · Pull Request #88 · apache/maven-compiler-plugin
> > (github.com) <
> > https://github.com/apache/maven-compiler-plugin/pull/88/files>
> >
> >
> > I've looked over the code and don't see anything obvious there.
> >
> > I do have package-info files in all my packages so new ones should not
> have
> > been generated, but it seems new ones may have.
> >
> > Full test run here:  Bump maven-compiler-plugin from 3.9.0 to 3.10.0 ·
> > oshi/oshi@471641b (github.com)
> > <https://github.com/oshi/oshi/runs/5184078693?check_suite_focus=true>
> >
> > --
> > Dan Widdis
> >
>


-- 
Dan Widdis

Possible file separator bug in 3.10.0 using bnd-maven-plugin

[Daniel B. Widdis]

I just bumped the maven-compiler-plugin version on my project from 3.9.0 to
3.10.0 and it caused Windows-based builds to fail maven-bnd-plugin tests
for package-info files in the "wrong" directory.  The symptom looks to be
associated with the windows file separator.

Sample failure looks like:
[INFO] Compiling 386 source files to D:\a\oshi\oshi\oshi-core\target\classes
[INFO]
[INFO] --- bnd-maven-plugin:6.1.0:bnd-process (default) @ oshi-core ---
Error:  D:\a\oshi\oshi\oshi-core\pom.xml [0:0]: Classes found in the wrong
directory:
{oshi/jna/platform/unix/package-info.class=oshi\jna\platform\unix\package-info,
oshi/driver/unix/solaris/disk/package-info.class=oshi\driver\unix\solaris\disk\package-info,
oshi/hardware/platform/mac/package-info.class=oshi\hardware\platform\mac\package-info,
oshi/software/os/package-info.class=oshi\software\os\package-info,
oshi/software/os/linux/package-info.class=oshi\software\os\linux\package-info,
... (many more)

While the failure is in another plugin, is it possible something in this PR
didn't handle the change?
 [MCOMPILER-205] Add a boolean to generate missing package-info classes by
default by gnodet · Pull Request #88 · apache/maven-compiler-plugin
(github.com) 


I've looked over the code and don't see anything obvious there.

I do have package-info files in all my packages so new ones should not have
been generated, but it seems new ones may have.

Full test run here:  Bump maven-compiler-plugin from 3.9.0 to 3.10.0 ·
oshi/oshi@471641b (github.com)


-- 
Dan Widdis