[
https://issues.apache.org/jira/browse/BEAM-9444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17061320#comment-17061320
]
Tomo Suzuki edited comment on BEAM-9444 at 3/18/20, 2:15 AM:
-------------------------------------------------------------
{{BEAM-9444-gcp-bom(fec5361f)}} still has problem in missing class such as
{{org.apache.beam.sdk.options.PipelineOptions}} and
{{com.google.protobuf.ByteString}}.
{noformat}
Linkage Check difference on beam-sdks-java-io-google-cloud-platform between
master(b00885a1) and BEAM-9444-gcp-bom(fec5361f):
Lines starting with '<' mean the branch remedies the errors (good)
Lines starting with '>' mean the branch introduces new errors (bad)
7c7,436
< Class com.github.luben.zstd.ZstdInputStream is not found;
---
> Class org.apache.beam.sdk.options.PipelineOptions is not found;
> referenced by 27 class files
> org.apache.beam.sdk.io.gcp.bigtable.BigtableIO
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigtable.BigtableConfig
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigquery.WriteRename
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteTables
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryQuerySource
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
...
> Class com.google.protobuf.ByteString is not found;
> referenced by 13 class files
> org.apache.beam.sdk.io.gcp.bigtable.BigtableIO
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigtable.BigtableServiceImpl
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageStreamSource
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
{noformat}
was (Author: suztomo):
BEAM-9444-gcp-bom(fec5361f) still has problem in missing class such as
{{org.apache.beam.sdk.options.PipelineOptions}} and
{{com.google.protobuf.ByteString}}.
{noformat}
Linkage Check difference on beam-sdks-java-io-google-cloud-platform between
master(b00885a1) and BEAM-9444-gcp-bom(fec5361f):
Lines starting with '<' mean the branch remedies the errors (good)
Lines starting with '>' mean the branch introduces new errors (bad)
7c7,436
< Class com.github.luben.zstd.ZstdInputStream is not found;
---
> Class org.apache.beam.sdk.options.PipelineOptions is not found;
> referenced by 27 class files
> org.apache.beam.sdk.io.gcp.bigtable.BigtableIO
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigtable.BigtableConfig
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigquery.WriteRename
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteTables
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryQuerySource
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
...
> Class com.google.protobuf.ByteString is not found;
> referenced by 13 class files
> org.apache.beam.sdk.io.gcp.bigtable.BigtableIO
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigtable.BigtableServiceImpl
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
> org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageStreamSource
> (beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.jar)
{noformat}
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --------------------------------------------------------------------------
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
> Reporter: Tomo Suzuki
> Assignee: Tomo Suzuki
> Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
> <dependencies>
> <dependency>
> <groupId>com.google.cloud</groupId>
> <artifactId>libraries-bom</artifactId>
> <version>4.2.0</version>
> <scope>compile</scope>
> <exclusions>
> <exclusion>
> <groupId>com.google.guava</groupId>
> <artifactId>guava-jdk5</artifactId>
> </exclusion>...
> </exclusions>
> </dependency>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
> <dependency>
> <groupId>com.google.api</groupId>
> <artifactId>gax</artifactId>
> <version/>
> <scope>compile</scope>
> ...
> </dependency>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
