Hi Yifan, I found resolutionStrategy is interfering gradle-versions-plugin (detail in BEAM-8654). Would you check this PR? https://github.com/apache/beam/pull/10127
On Thu, Nov 14, 2019 at 1:42 PM Kenneth Knowles <k...@apache.org> wrote: > On Thu, Nov 14, 2019 at 8:04 AM Alexey Romanenko <aromanenko....@gmail.com> > wrote: > >> Good example about Guava deps, let me go a bit deeper. >> >> $ find . -name build.gradle | xargs grep library.java.guava >> ./sdks/java/core/build.gradle: shadowTest library.java.guava_testlib >> >> ./sdks/java/io/kinesis/build.gradle: testCompile >> library.java.guava_testlib >> >> >> Regarding using non-vendored Guava in KinesisIO (and "java/core" as >> well), it’s all about *“library.java.guava_testlib” *and >> *“com.google.common.testing.EqualsTester”* only in particular, which is >> used for tests. >> Do we need to vendor “*com.google.guava:guava-testlib*” for this in this >> case? >> > > I didn't worry about test scopes because they don't ship to users so much. > It could be useful if there is a runner with a conflict and they want to > run the integration tests. > > >> - KinesisIO does depend on Guava at compile scope but has incorrect >> dependencies (Kinesis libs have Guava on API surface so it is OK here, but >> should be correctly declared) >> >> >> Sorry, but I didn’t understand what do you mean by “*but should be >> correctly declared*”. >> Since Kinesis client libs have own Guava deps and we shade our own guava, >> so it should be fine, no? >> > > I mean that any module with an "import X" should have a dependency on X in > its build.gradle. When you leave it off, the dep analysis (such as Maven's) > calls it out as "used undeclared dependency". > > Kenn > > >> >> On 11 Nov 2019, at 22:29, Kenneth Knowles <k...@apache.org> wrote: >> >> BeamModulePlugin just contains lists of versions to ease coordination >> across Beam modules, but mostly does not create dependencies. Most of >> Beam's modules only depend on a few things there. For example Guava is not >> a core dependency, but here is where it is actually depended upon: >> >> $ find . -name build.gradle | xargs grep library.java.guava >> ./sdks/java/core/build.gradle: shadowTest library.java.guava_testlib >> ./sdks/java/extensions/sql/jdbc/build.gradle: compile library.java.guava >> ./sdks/java/io/google-cloud-platform/build.gradle: compile >> library.java.guava >> ./sdks/java/io/kinesis/build.gradle: testCompile >> library.java.guava_testlib >> >> These results appear to be misleading. Grepping for 'import >> com.google.common', I see this as the actual state of things: >> >> - GCP connector does not appear to actually depend on Guava in compile >> scope >> - The Beam SQL JDBC driver does not appear to actually depend on Guava >> in compile scope >> - The Dataflow Java worker does depend on Guava at compile scope but has >> incorrect dependencies (and it probably shouldn't) >> - KinesisIO does depend on Guava at compile scope but has incorrect >> dependencies (Kinesis libs have Guava on API surface so it is OK here, but >> should be correctly declared) >> - ZetaSQL translator does depend on Guava at compile scope but has >> incorrect dependencies (ZetaSQL has it on API surface so it is OK here, but >> should be correctly declared) >> >> We used to have an analysis that prevented this class of error. >> >> Once the errors are fixed, the guava_version is simply a version that we >> have discovered that seems to work for both Kinesis and ZetaSQL, libraries >> we do not control. Kinesis producer is built against 18.0. Kinesis client >> against 26.0-jre. ZetaSQL against 26.0-android. >> >> (or maybe I messed up in my analysis) >> >> Kenn >> >> On Mon, Nov 11, 2019 at 12:07 PM Tomo Suzuki <suzt...@google.com> wrote: >> >>> >>> Chamikara and Yifan, >>> Thank you for the responses! Looking forward to hearing the >>> investigation result. >>> In the meantime, I'll explore .test-infra/jenkins/dependency_check >>> directory. >>> >>> >> -- Regards, Tomo
