[jira] [Commented] (BEAM-1984) Enable dependency analysis of non-compile dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075882#comment-16075882
 ] 

ASF GitHub Bot commented on BEAM-1984:
--

Github user kennknowles closed the pull request at:

https://github.com/apache/beam/pull/3488


> Enable dependency analysis of non-compile dependencies
> --
>
> Key: BEAM-1984
> URL: https://issues.apache.org/jira/browse/BEAM-1984
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
>
> In the Beam parent pom at this moment we have a configuration on the maven 
> dependency plugin to ignore the verification of non-compile dependencies so 
> unused or badly configured dependencies that are used only for test or 
> runtime purposes are not reported.
> {code:xml}
>   
>   true
> {code}
> I discovered this case when I executed:
> {code}
>   mvn clean install -Dmaven.test.skip=true
> {code}
> Notice that this is really particular because this does not compile the tests 
> classes, so it can detect when the ‘main’ classes are not using some code. 
> This is different from the behavior of:
> {code}
>   mvn clean install -DskipTests=true
> {code}
> That does not complain because skipTests compiles the tests.
> For the first case (maven.test.skip) I got a warning ‘Unused declared 
> dependencies found’ so with this I discovered the test/runtime only 
> dependencies that are not correctly scoped. I will do a PR to fix this, 
> because we can get rid of compile dependencies that are test or runtime only.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-1984) Enable dependency analysis of non-compile dependencies

[Kenneth Knowles (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16075695#comment-16075695
 ] 

Kenneth Knowles commented on BEAM-1984:
---

I think the reason we added this was specifically for {{runtime}} dependencies, 
which actually the plugin should ignore already since sort of by definition you 
cannot find a reference for. It could be that the plugin actually works and we 
(probably me) just set it up wrong.

> Enable dependency analysis of non-compile dependencies
> --
>
> Key: BEAM-1984
> URL: https://issues.apache.org/jira/browse/BEAM-1984
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
>
> In the Beam parent pom at this moment we have a configuration on the maven 
> dependency plugin to ignore the verification of non-compile dependencies so 
> unused or badly configured dependencies that are used only for test or 
> runtime purposes are not reported.
> {code:xml}
>   
>   true
> {code}
> I discovered this case when I executed:
> {code}
>   mvn clean install -Dmaven.test.skip=true
> {code}
> Notice that this is really particular because this does not compile the tests 
> classes, so it can detect when the ‘main’ classes are not using some code. 
> This is different from the behavior of:
> {code}
>   mvn clean install -DskipTests=true
> {code}
> That does not complain because skipTests compiles the tests.
> For the first case (maven.test.skip) I got a warning ‘Unused declared 
> dependencies found’ so with this I discovered the test/runtime only 
> dependencies that are not correctly scoped. I will do a PR to fix this, 
> because we can get rid of compile dependencies that are test or runtime only.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (BEAM-1984) Enable dependency analysis of non-compile dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992667#comment-15992667
 ] 

ASF GitHub Bot commented on BEAM-1984:
--

GitHub user iemejia opened a pull request:

https://github.com/apache/beam/pull/2821

[BEAM-1984] Fix scope for dependencies needed only for test/runtime purposes

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [ ] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [ ] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/iemejia/beam 
BEAM-1984-enable-dep-analysis-for-tests

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2821.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2821


commit 85b4b03675a3f23a4258f4156450e21ffc03621c
Author: Ismaël Mejía 
Date:   2017-05-02T09:58:54Z

[BEAM-1984] Fix scope for dependencies needed only for test purposes




> Enable dependency analysis of non-compile dependencies
> --
>
> Key: BEAM-1984
> URL: https://issues.apache.org/jira/browse/BEAM-1984
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
>
> In the Beam parent pom at this moment we have a configuration on the maven 
> dependency plugin to ignore the verification of non-compile dependencies so 
> unused or badly configured dependencies that are used only for test or 
> runtime purposes are not reported.
> {code:xml}
>   
>   true
> {code}
> I discovered this case when I executed:
> {code}
>   mvn clean install -Dmaven.test.skip=true
> {code}
> Notice that this is really particular because this does not compile the tests 
> classes, so it can detect when the ‘main’ classes are not using some code. 
> This is different from the behavior of:
> {code}
>   mvn clean install -DskipTests=true
> {code}
> That does not complain because skipTests compiles the tests.
> For the first case (maven.test.skip) I got a warning ‘Unused declared 
> dependencies found’ so with this I discovered the test/runtime only 
> dependencies that are not correctly scoped. I will do a PR to fix this, 
> because we can get rid of compile dependencies that are test or runtime only.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1984) Enable dependency analysis of non-compile dependencies

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15992660#comment-15992660
 ] 

Ismaël Mejía commented on BEAM-1984:


I just updated the ticket description for more context on the issue.

The real question and a deeper problem is that we can enable this validation so 
the dependency plugin complains about all the test/runtime dependencies, this 
is a good idea because we will be more precise about dependencies, but it has a 
not so good tradeoff, we need to build exclusion lists, the problem is that 
once a jar is moved, it should be also removed from the exclusion list, but 
this is up to the person who moves the jar out, and I have not found an 
automatic way to detect this, so I am afraid that the solution is worse than 
the sickness. I will do a subsequent PR to show you this in detail so you tell 
me what you think.

{code}

  

com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:jar
  

{code}

> Enable dependency analysis of non-compile dependencies
> --
>
> Key: BEAM-1984
> URL: https://issues.apache.org/jira/browse/BEAM-1984
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
>
> In the Beam parent pom at this moment we have a configuration on the maven 
> dependency plugin to ignore the verification of non-compile dependencies so 
> unused or badly configured dependencies that are used only for test or 
> runtime purposes are not reported.
> {code:xml}
>   
>   true
> {code}
> I discovered this case when I executed:
> {code}
>   mvn clean install -Dmaven.test.skip=true
> {code}
> Notice that this is really particular because this does not compile the tests 
> classes, so it can detect when the ‘main’ classes are not using some code. 
> This is different from the behavior of:
> {code}
>   mvn clean install -DskipTests=true
> {code}
> That does not complain because skipTests compiles the tests.
> For the first case (maven.test.skip) I got a warning ‘Unused declared 
> dependencies found’ so with this I discovered the test/runtime only 
> dependencies that are not correctly scoped. I will do a PR to fix this, 
> because we can get rid of compile dependencies that are test or runtime only.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1984) Enable dependency analysis of non-compile dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984833#comment-15984833
 ] 

ASF GitHub Bot commented on BEAM-1984:
--

Github user iemejia closed the pull request at:

https://github.com/apache/beam/pull/2591


> Enable dependency analysis of non-compile dependencies
> --
>
> Key: BEAM-1984
> URL: https://issues.apache.org/jira/browse/BEAM-1984
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
>
> Currently the dependency analysis is ignoring test dependencies, however if 
> we run:
> mvn install -Dmaven.test.skip=true
> It complains on multiple modules on dependencies that should be scoped 
> properly into the test mode but aren’t currently.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1984) Enable dependency analysis of non-compile dependencies

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15974960#comment-15974960
 ] 

ASF GitHub Bot commented on BEAM-1984:
--

GitHub user iemejia opened a pull request:

https://github.com/apache/beam/pull/2591

[BEAM-1984] Enable dependency analysis of non-compile dependencies

Be sure to do all of the following to help us incorporate your contribution
quickly and easily:

 - [x] Make sure the PR title is formatted like:
   `[BEAM-] Description of pull request`
 - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
   Travis-CI on your fork and ensure the whole test matrix passes).
 - [x] Replace `` in the title with the actual Jira issue
   number, if there is one.
 - [ ] If this contribution is large, please file an Apache
   [Individual Contributor License 
Agreement](https://www.apache.org/licenses/icla.pdf).

---


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/iemejia/beam 
BEAM-1984-enable-dep-analysis-for-tests

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/beam/pull/2591.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2591


commit 852d8aea354714a6d4b2ca6121fba454a9939de3
Author: Ismaël Mejía 
Date:   2017-04-13T16:52:58Z

[BEAM-1984] Update maven surefire and failsafe plugins to version 2.20

commit 14af68d717e87162bbda81c7f64d456f82182674
Author: Ismaël Mejía 
Date:   2017-04-16T12:41:44Z

[BEAM-1984] Fix scope for dependencies needed only for test purposes

commit 28bbca63b474cd622a99a67bc30b2e6370368138
Author: Ismaël Mejía 
Date:   2017-04-17T21:34:12Z

[BEAM-1984] Enable runtime dependency analysis




> Enable dependency analysis of non-compile dependencies
> --
>
> Key: BEAM-1984
> URL: https://issues.apache.org/jira/browse/BEAM-1984
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
>
> Currently the dependency analysis is ignoring test dependencies, however if 
> we run:
> mvn install -Dmaven.test.skip=true
> It complains on multiple modules on dependencies that should be scoped 
> properly into the test mode but aren’t currently.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1984) Enable dependency analysis of non-compile dependencies

[Aviem Zur (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15970425#comment-15970425
 ] 

Aviem Zur commented on BEAM-1984:
-

Another issue with the blanket {{ignoreNonCompile}} is we could have actually 
unused dependencies silently hidden by this. A more accurate way is to remove 
this and exclude specific dependencies in the relevant modules which need them 
at certain scopes even though they are not used in compile time. This can be 
achieved using 
https://maven.apache.org/plugins/maven-dependency-plugin/analyze-mojo.html#ignoredUnusedDeclaredDependencies

> Enable dependency analysis of non-compile dependencies
> --
>
> Key: BEAM-1984
> URL: https://issues.apache.org/jira/browse/BEAM-1984
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
>
> Currently the dependency analysis is ignoring test dependencies, however if 
> we run:
> mvn install -Dmaven.test.skip=true
> It complains on multiple modules on dependencies that should be scoped 
> properly into the test mode but aren’t currently.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1984) Enable dependency analysis of non-compile dependencies

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15970420#comment-15970420
 ] 

Ismaël Mejía commented on BEAM-1984:


One possible caveat is that a particular case can happen, a user can add a 
dependency for tests but forget to put it in the test scope and get that code 
merged, then other user takes the new code with the dependency and runs but 
skip the tests locally, in this case it will fail for them, this is not bad 
because at least we will find the issue, but can be annoying for the person who 
detects it.

> Enable dependency analysis of non-compile dependencies
> --
>
> Key: BEAM-1984
> URL: https://issues.apache.org/jira/browse/BEAM-1984
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
>
> Currently the dependency analysis is ignoring test dependencies, however if 
> we run:
> mvn install -Dmaven.test.skip=true
> It complains on multiple modules on dependencies that should be scoped 
> properly into the test mode but aren’t currently.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (BEAM-1984) Enable dependency analysis of non-compile dependencies

[JIRA]

[ 
https://issues.apache.org/jira/browse/BEAM-1984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15970419#comment-15970419
 ] 

Ismaël Mejía commented on BEAM-1984:


This issue fix should be in two parts, first just put the dependencies in the 
correct scope, and then the fix for the future is to stop ignoring the analysis 
of non compile dependencies. From now on unused declared test dependencies 
should opt-out of this validation in a per-module basis. This helps us by been 
explicit now in every dependency.

> Enable dependency analysis of non-compile dependencies
> --
>
> Key: BEAM-1984
> URL: https://issues.apache.org/jira/browse/BEAM-1984
> Project: Beam
>  Issue Type: Improvement
>  Components: build-system
>Affects Versions: Not applicable
>Reporter: Ismaël Mejía
>Assignee: Ismaël Mejía
>Priority: Minor
>
> Currently the dependency analysis is ignoring test dependencies, however if 
> we run:
> mvn install -Dmaven.test.skip=true
> It complains on multiple modules on dependencies that should be scoped 
> properly into the test mode but aren’t currently.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)