[jira] [Commented] (MASSEMBLY-940) moduleSet - incorrect binary dependencies added to assembly
[ https://issues.apache.org/jira/browse/MASSEMBLY-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17524578#comment-17524578 ] Scott Shealy commented on MASSEMBLY-940: Can confirm I am hitting this bug. Does anyone have a workaround for this? > moduleSet - incorrect binary dependencies added to assembly > > > Key: MASSEMBLY-940 > URL: https://issues.apache.org/jira/browse/MASSEMBLY-940 > Project: Maven Assembly Plugin > Issue Type: Bug >Affects Versions: 3.3.0 >Reporter: Nigel Jones >Priority: Major > > In moduleSet, dependencies are not based on specific modules included: > I have a large multimodule project (https://github.com/odpi/egeria) which has > over 300 modules, mostly java (->jar). > One of our submodules is responsible for creating the composite assembly. > Previously we were doing it 'the wrong way' (IMO) with lots of relative paths > (../../.. etc) and I've recently been trying to switch to using moduleSets, > and follow best practices for assembly. We were also previously doing lots of > building 'jar with dependencies' but this doesn't scale as every possibly > useful for ends up far too big... > I have hit a problem with dependencies and moduleSets > We have a series of 'client' artifacts. These have some dependencies (for > example we use restclient from spring, jackson etc). One section of my > assembly attempts to package the clients into a folder, and include any > dependencies they may need: > > > true > > org.odpi.egeria:asset-catalog-client:* > org.odpi.egeria:asset-consumer-client:* > org.odpi.egeria:asset-owner-client:* > org.odpi.egeria:community-profile-client:* > org.odpi.egeria:data-engine-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:data-manager-client:* > org.odpi.egeria:data-privacy-client:* > org.odpi.egeria:data-science-client:* > org.odpi.egeria:dev-ops-client:* > org.odpi.egeria:digital-architecture-client:* > org.odpi.egeria:discovery-engine-client:* > org.odpi.egeria:governance-engine-client:* > org.odpi.egeria:governance-program-client:* > org.odpi.egeria:information-view-client:* > org.odpi.egeria:project-management-client:* > org.odpi.egeria:security-officer-client:* > org.odpi.egeria:software-developer-client:* > org.odpi.egeria:stewardship-action-client:* > org.odpi.egeria:subject-area-client:* > org.odpi.egeria:admin-services-client:* > org.odpi.egeria:project-management-client:* > > > ./clients > false > true > > We have a similar pattern for utilities, samples, connectors etc. > I had thought 'includeDependencies' would JUST pull in dependencies for the > modules listed in the included. However I seem to get every dependency for > the entire project (reactor) > I then tried: > false > > > instead, with the same result. > Then switched to this -- since the pom for the module doing the assembly DOES > have a lot of dependencies (in fact it needs to be everything the assembly > refers to as a module, so that we ensure those modules are built first - and > exist) - so thinking (but unsure) if useProjectArtifact was relevant: > false > > true > false > false > > but yet again this still pulls in all dependencies. > The results are inconsistent with 'mvn dependency:tree' - and in fact > whatever is built I just always seem to get the same. It's not transitively > based off every module in the reactor, but seems to be based of the pom of > the project containing the assembly. > Shouldn't I just be pulling in dependencies of the binaries I specify in the > moduleSet? > I've read the definitive maven book chapter, and the docs, but I can't quite > figure out what I'm missing here. > To clarify as well, here's another example > > > true > > org.odpi.egeria:open-metadata-conformance-suite-client:* > > > ./conformance-suite > false > false > > > true > false > false > > > > > Again, I get the same dependencies - this time into ./conformance-suite yet > that is just a single module I am putting into that dir - and I can simply > show > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @ > open-metadata-conformance-suite-client --- > [INFO] org.odpi.egeria:open-metadata-conformance-suite-client:jar:2.1-SNAPSHOT > [INFO] +- > org.odpi.egeria:open-metadata-conformance-suite-api:jar:2.1-SNAPSHOT:compile > [INFO] | +- org.slf4j:slf4j-api:jar:1.7.30:compile > [INFO] | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.11.1:compile > [INFO] +- org.odpi.egeria:rest-client-connectors-api:jar:2.1-SNAPSHOT:compile > [INFO] | \- org.odpi.egeria:open-connector-framework:jar:2.1-SNAPSHOT:compile > [INFO] | \- org.odpi.egeria:audit-log-framework:jar:2.1-SNAPSHOT:compile
[jira] [Commented] (MASSEMBLY-940) moduleSet - incorrect binary dependencies added to assembly
[
https://issues.apache.org/jira/browse/MASSEMBLY-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17375930#comment-17375930
]
Jody Garnett commented on MASSEMBLY-940:
I can verify *In moduleSet, dependencies are not based on specific modules
included* while experimenting with a large multi-module project
([https://github.com/geotools/geotools]).
* modules/library/...
* modules/plugins/...
* release/pom.xml
When using moduleSet assembly and release pom.xml includes a single dependency
(to force execution at the end of the build) the resulting jar has *101* jars.
When using the same moduleSet assembly and updating the release pom.xml to
explicitly include all the required modules as dependencies the resulting jar
has *272* jars (including all the required transitive dependencies).
Conclusion is that the dependencySet is based on the release pom.xml
dependencies, and not based on the dependencies of the specific modules
included. Based on this experience I will be using dependencySets until this
issue is resolved.
binaryDist.xml as follows:
{code:java}
true
true
org.geotools:gt-arcgrid
...
false
true
lib
true
{code}
> moduleSet - incorrect binary dependencies added to assembly
>
>
> Key: MASSEMBLY-940
> URL: https://issues.apache.org/jira/browse/MASSEMBLY-940
> Project: Maven Assembly Plugin
> Issue Type: Bug
>Affects Versions: 3.3.0
>Reporter: Nigel Jones
>Priority: Major
>
> In moduleSet, dependencies are not based on specific modules included:
> I have a large multimodule project (https://github.com/odpi/egeria) which has
> over 300 modules, mostly java (->jar).
> One of our submodules is responsible for creating the composite assembly.
> Previously we were doing it 'the wrong way' (IMO) with lots of relative paths
> (../../.. etc) and I've recently been trying to switch to using moduleSets,
> and follow best practices for assembly. We were also previously doing lots of
> building 'jar with dependencies' but this doesn't scale as every possibly
> useful for ends up far too big...
> I have hit a problem with dependencies and moduleSets
> We have a series of 'client' artifacts. These have some dependencies (for
> example we use restclient from spring, jackson etc). One section of my
> assembly attempts to package the clients into a folder, and include any
> dependencies they may need:
>
>
> true
>
> org.odpi.egeria:asset-catalog-client:*
> org.odpi.egeria:asset-consumer-client:*
> org.odpi.egeria:asset-owner-client:*
> org.odpi.egeria:community-profile-client:*
> org.odpi.egeria:data-engine-client:*
> org.odpi.egeria:it-infrastructure-client:*
> org.odpi.egeria:it-infrastructure-client:*
> org.odpi.egeria:data-manager-client:*
> org.odpi.egeria:data-privacy-client:*
> org.odpi.egeria:data-science-client:*
> org.odpi.egeria:dev-ops-client:*
> org.odpi.egeria:digital-architecture-client:*
> org.odpi.egeria:discovery-engine-client:*
> org.odpi.egeria:governance-engine-client:*
> org.odpi.egeria:governance-program-client:*
> org.odpi.egeria:information-view-client:*
> org.odpi.egeria:project-management-client:*
> org.odpi.egeria:security-officer-client:*
> org.odpi.egeria:software-developer-client:*
> org.odpi.egeria:stewardship-action-client:*
> org.odpi.egeria:subject-area-client:*
> org.odpi.egeria:admin-services-client:*
> org.odpi.egeria:project-management-client:*
>
>
> ./clients
> false
> true
>
> We have a similar pattern for utilities, samples, connectors etc.
> I had thought 'includeDependencies' would JUST pull in dependencies for the
> modules listed in the included. However I seem to get every dependency for
> the entire project (reactor)
> I then tried:
> false
>
>
> instead, with the same result.
> Then switched to this -- since the pom for the module doing the assembly DOES
> have a lot of dependencies (in fact it needs to be everything the assembly
> refers to as a module, so that we ensure those modules are built first - and
> exist) - so thinking (but unsure) if useProjectArtifact was relevant:
> false
>
> true
> false
> false
>
> but yet again this still pulls in all dependencies.
> The results are inconsistent with 'mvn dependency:tree' - and in fact
> whatever is built I just always seem to get the same. It's not transitively
> based off every module in the reactor, but seems to be based of the pom of
> the project containing the assembly.
> Shouldn't I just be pulling in dependencies of the binaries I specify in the
> moduleSet?
> I've read the definitive maven book chapter, and the docs, but I can't quite
> figure out what I
[jira] [Commented] (MASSEMBLY-940) moduleSet - incorrect binary dependencies added to assembly
[ https://issues.apache.org/jira/browse/MASSEMBLY-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17173666#comment-17173666 ] Karl Heinz Marbaise commented on MASSEMBLY-940: --- Hi [~jonesn] maybe I mistaken something or simply missed can you please summarize what you like to achieve with the setup? > moduleSet - incorrect binary dependencies added to assembly > > > Key: MASSEMBLY-940 > URL: https://issues.apache.org/jira/browse/MASSEMBLY-940 > Project: Maven Assembly Plugin > Issue Type: Bug >Affects Versions: 3.3.0 >Reporter: Nigel Jones >Priority: Major > > In moduleSet, dependencies are not based on specific modules included: > I have a large multimodule project (https://github.com/odpi/egeria) which has > over 300 modules, mostly java (->jar). > One of our submodules is responsible for creating the composite assembly. > Previously we were doing it 'the wrong way' (IMO) with lots of relative paths > (../../.. etc) and I've recently been trying to switch to using moduleSets, > and follow best practices for assembly. We were also previously doing lots of > building 'jar with dependencies' but this doesn't scale as every possibly > useful for ends up far too big... > I have hit a problem with dependencies and moduleSets > We have a series of 'client' artifacts. These have some dependencies (for > example we use restclient from spring, jackson etc). One section of my > assembly attempts to package the clients into a folder, and include any > dependencies they may need: > > > true > > org.odpi.egeria:asset-catalog-client:* > org.odpi.egeria:asset-consumer-client:* > org.odpi.egeria:asset-owner-client:* > org.odpi.egeria:community-profile-client:* > org.odpi.egeria:data-engine-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:data-manager-client:* > org.odpi.egeria:data-privacy-client:* > org.odpi.egeria:data-science-client:* > org.odpi.egeria:dev-ops-client:* > org.odpi.egeria:digital-architecture-client:* > org.odpi.egeria:discovery-engine-client:* > org.odpi.egeria:governance-engine-client:* > org.odpi.egeria:governance-program-client:* > org.odpi.egeria:information-view-client:* > org.odpi.egeria:project-management-client:* > org.odpi.egeria:security-officer-client:* > org.odpi.egeria:software-developer-client:* > org.odpi.egeria:stewardship-action-client:* > org.odpi.egeria:subject-area-client:* > org.odpi.egeria:admin-services-client:* > org.odpi.egeria:project-management-client:* > > > ./clients > false > true > > We have a similar pattern for utilities, samples, connectors etc. > I had thought 'includeDependencies' would JUST pull in dependencies for the > modules listed in the included. However I seem to get every dependency for > the entire project (reactor) > I then tried: > false > > > instead, with the same result. > Then switched to this -- since the pom for the module doing the assembly DOES > have a lot of dependencies (in fact it needs to be everything the assembly > refers to as a module, so that we ensure those modules are built first - and > exist) - so thinking (but unsure) if useProjectArtifact was relevant: > false > > true > false > false > > but yet again this still pulls in all dependencies. > The results are inconsistent with 'mvn dependency:tree' - and in fact > whatever is built I just always seem to get the same. It's not transitively > based off every module in the reactor, but seems to be based of the pom of > the project containing the assembly. > Shouldn't I just be pulling in dependencies of the binaries I specify in the > moduleSet? > I've read the definitive maven book chapter, and the docs, but I can't quite > figure out what I'm missing here. > To clarify as well, here's another example > > > true > > org.odpi.egeria:open-metadata-conformance-suite-client:* > > > ./conformance-suite > false > false > > > true > false > false > > > > > Again, I get the same dependencies - this time into ./conformance-suite yet > that is just a single module I am putting into that dir - and I can simply > show > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @ > open-metadata-conformance-suite-client --- > [INFO] org.odpi.egeria:open-metadata-conformance-suite-client:jar:2.1-SNAPSHOT > [INFO] +- > org.odpi.egeria:open-metadata-conformance-suite-api:jar:2.1-SNAPSHOT:compile > [INFO] | +- org.slf4j:slf4j-api:jar:1.7.30:compile > [INFO] | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.11.1:compile > [INFO] +- org.odpi.egeria:rest-client-connectors-api:jar:2.1-SNAPSHOT:compile > [INFO] | \- org.odpi.egeria:open-connector-framework:jar:2.1-SNAPSHOT:compile > [INFO] |
[jira] [Commented] (MASSEMBLY-940) moduleSet - incorrect binary dependencies added to assembly
[ https://issues.apache.org/jira/browse/MASSEMBLY-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17164382#comment-17164382 ] Nigel Jones commented on MASSEMBLY-940: --- As a further test * I split up the assemblies into two. one for logging (which has some dependencies), one for print (which doesn't). I used two executions in the assembly pom. the result was the same. I still got all dependent modules included * I then switched to using 'scope=provided' for the dependency in the pom itself - the result was no dependencies in either I can only conclude from this that the modules matched by the moduleset do not influence dependencies in either way. All we have is the pom level dependencies as specified on the assembly pom. > moduleSet - incorrect binary dependencies added to assembly > > > Key: MASSEMBLY-940 > URL: https://issues.apache.org/jira/browse/MASSEMBLY-940 > Project: Maven Assembly Plugin > Issue Type: Bug >Affects Versions: 3.3.0 >Reporter: Nigel Jones >Priority: Major > > In moduleSet, dependencies are not based on specific modules included: > I have a large multimodule project (https://github.com/odpi/egeria) which has > over 300 modules, mostly java (->jar). > One of our submodules is responsible for creating the composite assembly. > Previously we were doing it 'the wrong way' (IMO) with lots of relative paths > (../../.. etc) and I've recently been trying to switch to using moduleSets, > and follow best practices for assembly. We were also previously doing lots of > building 'jar with dependencies' but this doesn't scale as every possibly > useful for ends up far too big... > I have hit a problem with dependencies and moduleSets > We have a series of 'client' artifacts. These have some dependencies (for > example we use restclient from spring, jackson etc). One section of my > assembly attempts to package the clients into a folder, and include any > dependencies they may need: > > > true > > org.odpi.egeria:asset-catalog-client:* > org.odpi.egeria:asset-consumer-client:* > org.odpi.egeria:asset-owner-client:* > org.odpi.egeria:community-profile-client:* > org.odpi.egeria:data-engine-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:data-manager-client:* > org.odpi.egeria:data-privacy-client:* > org.odpi.egeria:data-science-client:* > org.odpi.egeria:dev-ops-client:* > org.odpi.egeria:digital-architecture-client:* > org.odpi.egeria:discovery-engine-client:* > org.odpi.egeria:governance-engine-client:* > org.odpi.egeria:governance-program-client:* > org.odpi.egeria:information-view-client:* > org.odpi.egeria:project-management-client:* > org.odpi.egeria:security-officer-client:* > org.odpi.egeria:software-developer-client:* > org.odpi.egeria:stewardship-action-client:* > org.odpi.egeria:subject-area-client:* > org.odpi.egeria:admin-services-client:* > org.odpi.egeria:project-management-client:* > > > ./clients > false > true > > We have a similar pattern for utilities, samples, connectors etc. > I had thought 'includeDependencies' would JUST pull in dependencies for the > modules listed in the included. However I seem to get every dependency for > the entire project (reactor) > I then tried: > false > > > instead, with the same result. > Then switched to this -- since the pom for the module doing the assembly DOES > have a lot of dependencies (in fact it needs to be everything the assembly > refers to as a module, so that we ensure those modules are built first - and > exist) - so thinking (but unsure) if useProjectArtifact was relevant: > false > > true > false > false > > but yet again this still pulls in all dependencies. > The results are inconsistent with 'mvn dependency:tree' - and in fact > whatever is built I just always seem to get the same. It's not transitively > based off every module in the reactor, but seems to be based of the pom of > the project containing the assembly. > Shouldn't I just be pulling in dependencies of the binaries I specify in the > moduleSet? > I've read the definitive maven book chapter, and the docs, but I can't quite > figure out what I'm missing here. > To clarify as well, here's another example > > > true > > org.odpi.egeria:open-metadata-conformance-suite-client:* > > > ./conformance-suite > false > false > > > true > false > false > > > > > Again, I get the same dependencies - this time into ./conformance-suite yet > that is just a single module I am putting into that dir - and I can simply > show > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @ > open-metadata-conformance-suite-client --- > [INFO] org.odpi.egeria:open-metadata-conforma
[jira] [Commented] (MASSEMBLY-940) moduleSet - incorrect binary dependencies added to assembly
[ https://issues.apache.org/jira/browse/MASSEMBLY-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161066#comment-17161066 ] Nigel Jones commented on MASSEMBLY-940: --- I also tried setting scope=provided in the dependency to see how that might affect behaviour – in this case the assembly failed with: [WARNING] The following patterns were never triggered in this artifact inclusion filter: o 'org.example:logging' even though the build order looked correct. > moduleSet - incorrect binary dependencies added to assembly > > > Key: MASSEMBLY-940 > URL: https://issues.apache.org/jira/browse/MASSEMBLY-940 > Project: Maven Assembly Plugin > Issue Type: Bug >Affects Versions: 3.3.0 >Reporter: Nigel Jones >Priority: Major > > In moduleSet, dependencies are not based on specific modules included: > I have a large multimodule project (https://github.com/odpi/egeria) which has > over 300 modules, mostly java (->jar). > One of our submodules is responsible for creating the composite assembly. > Previously we were doing it 'the wrong way' (IMO) with lots of relative paths > (../../.. etc) and I've recently been trying to switch to using moduleSets, > and follow best practices for assembly. We were also previously doing lots of > building 'jar with dependencies' but this doesn't scale as every possibly > useful for ends up far too big... > I have hit a problem with dependencies and moduleSets > We have a series of 'client' artifacts. These have some dependencies (for > example we use restclient from spring, jackson etc). One section of my > assembly attempts to package the clients into a folder, and include any > dependencies they may need: > > > true > > org.odpi.egeria:asset-catalog-client:* > org.odpi.egeria:asset-consumer-client:* > org.odpi.egeria:asset-owner-client:* > org.odpi.egeria:community-profile-client:* > org.odpi.egeria:data-engine-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:data-manager-client:* > org.odpi.egeria:data-privacy-client:* > org.odpi.egeria:data-science-client:* > org.odpi.egeria:dev-ops-client:* > org.odpi.egeria:digital-architecture-client:* > org.odpi.egeria:discovery-engine-client:* > org.odpi.egeria:governance-engine-client:* > org.odpi.egeria:governance-program-client:* > org.odpi.egeria:information-view-client:* > org.odpi.egeria:project-management-client:* > org.odpi.egeria:security-officer-client:* > org.odpi.egeria:software-developer-client:* > org.odpi.egeria:stewardship-action-client:* > org.odpi.egeria:subject-area-client:* > org.odpi.egeria:admin-services-client:* > org.odpi.egeria:project-management-client:* > > > ./clients > false > true > > We have a similar pattern for utilities, samples, connectors etc. > I had thought 'includeDependencies' would JUST pull in dependencies for the > modules listed in the included. However I seem to get every dependency for > the entire project (reactor) > I then tried: > false > > > instead, with the same result. > Then switched to this -- since the pom for the module doing the assembly DOES > have a lot of dependencies (in fact it needs to be everything the assembly > refers to as a module, so that we ensure those modules are built first - and > exist) - so thinking (but unsure) if useProjectArtifact was relevant: > false > > true > false > false > > but yet again this still pulls in all dependencies. > The results are inconsistent with 'mvn dependency:tree' - and in fact > whatever is built I just always seem to get the same. It's not transitively > based off every module in the reactor, but seems to be based of the pom of > the project containing the assembly. > Shouldn't I just be pulling in dependencies of the binaries I specify in the > moduleSet? > I've read the definitive maven book chapter, and the docs, but I can't quite > figure out what I'm missing here. > To clarify as well, here's another example > > > true > > org.odpi.egeria:open-metadata-conformance-suite-client:* > > > ./conformance-suite > false > false > > > true > false > false > > > > > Again, I get the same dependencies - this time into ./conformance-suite yet > that is just a single module I am putting into that dir - and I can simply > show > [INFO] --- maven-dependency-plugin:3.1.2:tree (default-cli) @ > open-metadata-conformance-suite-client --- > [INFO] org.odpi.egeria:open-metadata-conformance-suite-client:jar:2.1-SNAPSHOT > [INFO] +- > org.odpi.egeria:open-metadata-conformance-suite-api:jar:2.1-SNAPSHOT:compile > [INFO] | +- org.slf4j:slf4j-api:jar:1.7.30:compile > [INFO] | \- com.fasterxml.jackson.core:jackson-annotations:jar:2.11.1:compile > [INFO] +- or
[jira] [Commented] (MASSEMBLY-940) moduleSet - incorrect binary dependencies added to assembly
[ https://issues.apache.org/jira/browse/MASSEMBLY-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161063#comment-17161063 ] Nigel Jones commented on MASSEMBLY-940: --- I then removed the dependency from the assembly pom - but of course this then fails with an error like: ``` [INFO] --- maven-assembly-plugin:3.3.0:single (default) @ assembly --- [INFO] Reading assembly descriptor: src/main/assembly/assembly.xml [INFO] [INFO] Reactor Summary for coolstuff 1.0-SNAPSHOT: [INFO] [INFO] coolstuff .. SUCCESS [ 0.262 s] [INFO] logging SUCCESS [ 1.244 s] [INFO] assembly ... FAILURE [ 0.493 s] [INFO] time ... SKIPPED [INFO] print .. SKIPPED [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 2.069 s [INFO] Finished at: 2020-07-20T10:19:35+01:00 [INFO] [ERROR] Failed to execute goal org.apache.maven.plugins:maven-assembly-plugin:3.3.0:single (default) on project assembly: Failed to create assembly: Artifact: org.example:print:jar:1.0-SNAPSHOT (included by module) does not have an artifact with a file. Please ensure the package phase is run before the assembly is generated. -> [Help 1]``` Quite sensibly.. since we try and create the assembly before the modules we need to use > moduleSet - incorrect binary dependencies added to assembly > > > Key: MASSEMBLY-940 > URL: https://issues.apache.org/jira/browse/MASSEMBLY-940 > Project: Maven Assembly Plugin > Issue Type: Bug >Affects Versions: 3.3.0 >Reporter: Nigel Jones >Priority: Major > > In moduleSet, dependencies are not based on specific modules included: > I have a large multimodule project (https://github.com/odpi/egeria) which has > over 300 modules, mostly java (->jar). > One of our submodules is responsible for creating the composite assembly. > Previously we were doing it 'the wrong way' (IMO) with lots of relative paths > (../../.. etc) and I've recently been trying to switch to using moduleSets, > and follow best practices for assembly. We were also previously doing lots of > building 'jar with dependencies' but this doesn't scale as every possibly > useful for ends up far too big... > I have hit a problem with dependencies and moduleSets > We have a series of 'client' artifacts. These have some dependencies (for > example we use restclient from spring, jackson etc). One section of my > assembly attempts to package the clients into a folder, and include any > dependencies they may need: > > > true > > org.odpi.egeria:asset-catalog-client:* > org.odpi.egeria:asset-consumer-client:* > org.odpi.egeria:asset-owner-client:* > org.odpi.egeria:community-profile-client:* > org.odpi.egeria:data-engine-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:data-manager-client:* > org.odpi.egeria:data-privacy-client:* > org.odpi.egeria:data-science-client:* > org.odpi.egeria:dev-ops-client:* > org.odpi.egeria:digital-architecture-client:* > org.odpi.egeria:discovery-engine-client:* > org.odpi.egeria:governance-engine-client:* > org.odpi.egeria:governance-program-client:* > org.odpi.egeria:information-view-client:* > org.odpi.egeria:project-management-client:* > org.odpi.egeria:security-officer-client:* > org.odpi.egeria:software-developer-client:* > org.odpi.egeria:stewardship-action-client:* > org.odpi.egeria:subject-area-client:* > org.odpi.egeria:admin-services-client:* > org.odpi.egeria:project-management-client:* > > > ./clients > false > true > > We have a similar pattern for utilities, samples, connectors etc. > I had thought 'includeDependencies' would JUST pull in dependencies for the > modules listed in the included. However I seem to get every dependency for > the entire project (reactor) > I then tried: > false > > > instead, with the same result. > Then switched to this -- since the pom for the module doing the assembly DOES > have a lot of dependencies (in fact it needs to be everything the assembly > refers to as a module, so that we ensure those modules are built first - and > exist) - so thinking (but unsure) if useProjectArtifact was relevant: > false > > true > false > false > > but yet again this still pulls in all dependencies. > The results are inconsistent with 'mvn dependency:tree' - an
[jira] [Commented] (MASSEMBLY-940) moduleSet - incorrect binary dependencies added to assembly
[ https://issues.apache.org/jira/browse/MASSEMBLY-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161058#comment-17161058 ] Nigel Jones commented on MASSEMBLY-940: --- I have created a much simpler test project to demonstrate the problem [https://github.com/planetf1/mvntest] This has a parent & 3 child projects that have java classes to a) log b) just print c) use joda tome – random examples There is also a simple assembly. What I note is that I started off creating an assembly JUST for the 'print' module. This has no additional dependencies, and the assembly output directory, with dependencies is correct. I then added the 'logging' module and noted I now get logging dependencies included both in the 'print' module based moduleSet as well as the new one for logging. I then commented out the moduleSet and realised what we get with the dependencySet (within moduleSet) is ALL the dependencies declared in the POM - which doesn't seem correct. Dependencies in the pom are needed to ensure the build order is correct but should not be added as a moduleSet dependency unless, well, they really are a dependency FOR THAT MODULE. At least that is my understanding of the documentation on moduleSets and what seems to be 'useful' but I may have misunderstood! > moduleSet - incorrect binary dependencies added to assembly > > > Key: MASSEMBLY-940 > URL: https://issues.apache.org/jira/browse/MASSEMBLY-940 > Project: Maven Assembly Plugin > Issue Type: Bug >Affects Versions: 3.3.0 >Reporter: Nigel Jones >Priority: Major > > In moduleSet, dependencies are not based on specific modules included: > I have a large multimodule project (https://github.com/odpi/egeria) which has > over 300 modules, mostly java (->jar). > One of our submodules is responsible for creating the composite assembly. > Previously we were doing it 'the wrong way' (IMO) with lots of relative paths > (../../.. etc) and I've recently been trying to switch to using moduleSets, > and follow best practices for assembly. We were also previously doing lots of > building 'jar with dependencies' but this doesn't scale as every possibly > useful for ends up far too big... > I have hit a problem with dependencies and moduleSets > We have a series of 'client' artifacts. These have some dependencies (for > example we use restclient from spring, jackson etc). One section of my > assembly attempts to package the clients into a folder, and include any > dependencies they may need: > > > true > > org.odpi.egeria:asset-catalog-client:* > org.odpi.egeria:asset-consumer-client:* > org.odpi.egeria:asset-owner-client:* > org.odpi.egeria:community-profile-client:* > org.odpi.egeria:data-engine-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:data-manager-client:* > org.odpi.egeria:data-privacy-client:* > org.odpi.egeria:data-science-client:* > org.odpi.egeria:dev-ops-client:* > org.odpi.egeria:digital-architecture-client:* > org.odpi.egeria:discovery-engine-client:* > org.odpi.egeria:governance-engine-client:* > org.odpi.egeria:governance-program-client:* > org.odpi.egeria:information-view-client:* > org.odpi.egeria:project-management-client:* > org.odpi.egeria:security-officer-client:* > org.odpi.egeria:software-developer-client:* > org.odpi.egeria:stewardship-action-client:* > org.odpi.egeria:subject-area-client:* > org.odpi.egeria:admin-services-client:* > org.odpi.egeria:project-management-client:* > > > ./clients > false > true > > We have a similar pattern for utilities, samples, connectors etc. > I had thought 'includeDependencies' would JUST pull in dependencies for the > modules listed in the included. However I seem to get every dependency for > the entire project (reactor) > I then tried: > false > > > instead, with the same result. > Then switched to this -- since the pom for the module doing the assembly DOES > have a lot of dependencies (in fact it needs to be everything the assembly > refers to as a module, so that we ensure those modules are built first - and > exist) - so thinking (but unsure) if useProjectArtifact was relevant: > false > > true > false > false > > but yet again this still pulls in all dependencies. > The results are inconsistent with 'mvn dependency:tree' - and in fact > whatever is built I just always seem to get the same. It's not transitively > based off every module in the reactor, but seems to be based of the pom of > the project containing the assembly. > Shouldn't I just be pulling in dependencies of the binaries I specify in the > moduleSet? > I've read the definitive maven book chapter, and the docs, but I can't quite > figure out what I'm missing here. >
[jira] [Commented] (MASSEMBLY-940) moduleSet - incorrect binary dependencies added to assembly
[ https://issues.apache.org/jira/browse/MASSEMBLY-940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17161008#comment-17161008 ] Nigel Jones commented on MASSEMBLY-940: --- Full assembly : [https://gist.github.com/planetf1/4bef80e8454b3e57a3a8d948775abaf4] >From the log of a build: For that last example which has a handful of dependencies, what I actually see the assembly plugin reporting is: 17:11:33,629 [DEBUG] Dependencies for project: org.odpi.egeria:open-metadata-conformance-suite-client:jar:2.1-SNAPSHOT are: 17:11:33,629 [DEBUG] First version:2.1-SNAPSHOT 17:11:33,629 [DEBUG] -> checking org.odpi.egeria:open-metadata-conformance-suite-client:jar:2.1-SNAPSHOT 17:11:33,629 [DEBUG] Processing binary dependencies for module project: org.odpi.egeria:open-metadata-conformance-suite-client:jar:2.1-SNAPSHOT 17:11:33,630 [DEBUG] Processing DependencySet (output=./conformance-suite) 17:11:33,630 [DEBUG] Filtering dependency artifacts WITHOUT transitive dependency path information. 17:11:33,630 [DEBUG] org.slf4j:slf4j-simple:jar:1.7.30 was removed by one or more filters. 17:11:33,630 [DEBUG] Adding 470 dependency artifacts. There's the 470 but why. That definately looks more like the overall project -- though if I do a pstree on the dependencies of this module, I actually resolve to 213 dependencies. Each time 470 yet the actual number (as above) should be a lot smaller and indeed should differ from moduleSet to moduleSet whereas in this case they do not > moduleSet - incorrect binary dependencies added to assembly > > > Key: MASSEMBLY-940 > URL: https://issues.apache.org/jira/browse/MASSEMBLY-940 > Project: Maven Assembly Plugin > Issue Type: Bug >Affects Versions: 3.3.0 >Reporter: Nigel Jones >Priority: Major > > In moduleSet, dependencies are not based on specific modules included: > I have a large multimodule project (https://github.com/odpi/egeria) which has > over 300 modules, mostly java (->jar). > One of our submodules is responsible for creating the composite assembly. > Previously we were doing it 'the wrong way' (IMO) with lots of relative paths > (../../.. etc) and I've recently been trying to switch to using moduleSets, > and follow best practices for assembly. We were also previously doing lots of > building 'jar with dependencies' but this doesn't scale as every possibly > useful for ends up far too big... > I have hit a problem with dependencies and moduleSets > We have a series of 'client' artifacts. These have some dependencies (for > example we use restclient from spring, jackson etc). One section of my > assembly attempts to package the clients into a folder, and include any > dependencies they may need: > > > true > > org.odpi.egeria:asset-catalog-client:* > org.odpi.egeria:asset-consumer-client:* > org.odpi.egeria:asset-owner-client:* > org.odpi.egeria:community-profile-client:* > org.odpi.egeria:data-engine-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:it-infrastructure-client:* > org.odpi.egeria:data-manager-client:* > org.odpi.egeria:data-privacy-client:* > org.odpi.egeria:data-science-client:* > org.odpi.egeria:dev-ops-client:* > org.odpi.egeria:digital-architecture-client:* > org.odpi.egeria:discovery-engine-client:* > org.odpi.egeria:governance-engine-client:* > org.odpi.egeria:governance-program-client:* > org.odpi.egeria:information-view-client:* > org.odpi.egeria:project-management-client:* > org.odpi.egeria:security-officer-client:* > org.odpi.egeria:software-developer-client:* > org.odpi.egeria:stewardship-action-client:* > org.odpi.egeria:subject-area-client:* > org.odpi.egeria:admin-services-client:* > org.odpi.egeria:project-management-client:* > > > ./clients > false > true > > We have a similar pattern for utilities, samples, connectors etc. > I had thought 'includeDependencies' would JUST pull in dependencies for the > modules listed in the included. However I seem to get every dependency for > the entire project (reactor) > I then tried: > false > > > instead, with the same result. > Then switched to this -- since the pom for the module doing the assembly DOES > have a lot of dependencies (in fact it needs to be everything the assembly > refers to as a module, so that we ensure those modules are built first - and > exist) - so thinking (but unsure) if useProjectArtifact was relevant: > false > > true > false > false > > but yet again this still pulls in all dependencies. > The results are inconsistent with 'mvn dependency:tree' - and in fact > whatever is built I just always seem to get the same. It's not transitively > based off every module in the reactor, but seems to be based of the pom of > the project containing the
