[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=116867=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-116867 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 28/Jun/18 16:23 Start Date: 28/Jun/18 16:23 Worklog Time Spent: 10m Work Description: kennknowles commented on issue #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#issuecomment-401092738 We have turned on autoformatting of the codebase, which causes small conflicts across the board. You can probably safely rebase and just keep your changes. Like this: ``` $ git rebase ... see some conflicts $ git diff ... confirmed that the conflicts are just autoformatting ... so we can just keep our changes are do our own autoformat $ git checkout --theirs -- $ git add -u $ git rebase --continue $ ./gradlew spotlessJavaApply ``` Please ping me if you run into any difficulty. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 116867) Time Spent: 6h 50m (was: 6h 40m) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 6h 50m > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111601=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111601 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 13/Jun/18 18:32 Start Date: 13/Jun/18 18:32 Worklog Time Spent: 10m Work Description: swegner commented on issue #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#issuecomment-397040445 Above comments are mostly about documentation. No other major concerns from me (LGTM) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 111601) Time Spent: 6h 40m (was: 6.5h) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 6h 40m > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111595=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111595
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 13/Jun/18 18:31
Start Date: 13/Jun/18 18:31
Worklog Time Spent: 10m
Work Description: swegner commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r195184085
##
File path: build_rules.gradle
##
@@ -1260,6 +1261,146 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(failOnWarning: true, enableFindbugs: false, shadowClosure: {
+// The relocation paths below specifically use the major version number of
the dependency
+// since we assume that packages following semantic versioning rules. For
packages
+// with a major version number that is 0 (implying that the dependency is
unstable),
+// we use the full version identifier in the relocated path (e.g.
v0_minor_patch).
+// For stable dependencies, if two or more packages use the same major
version number,
Review comment:
Thanks for writing this extensive documentation. This will be a useful
reference for anyone that needs to maintain dependencies.
Could you define what you mean by "stable dependencies"; in contrast to
those which change more frequently?
It may also be useful to talk about the implications of having multiple
versions of a dependency. IMO shading is one of the more
difficult-to-understand areas of the Beam codebase, so I welcome any
documentation that helps explain how we use it. This may even deserve separate
markdown doc or wiki page.
Just my 2cents. Feel free to disagree or defer to later.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111595)
Time Spent: 6h 10m (was: 6h)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111598=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111598
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 13/Jun/18 18:31
Start Date: 13/Jun/18 18:31
Worklog Time Spent: 10m
Work Description: swegner commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r195187056
##
File path: build_rules.gradle
##
@@ -1260,6 +1261,146 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(failOnWarning: true, enableFindbugs: false, shadowClosure: {
+// The relocation paths below specifically use the major version number of
the dependency
+// since we assume that packages following semantic versioning rules. For
packages
+// with a major version number that is 0 (implying that the dependency is
unstable),
+// we use the full version identifier in the relocated path (e.g.
v0_minor_patch).
+// For stable dependencies, if two or more packages use the same major
version number,
+// we should bias to vendoring the highest version to minimize jar files
instead of
+// vendoring using their minor or patch version numbers. Some packages may
be incompatible
+// across minor or patch versions and in those instances we should either
attempt to update our
+// usage of the dependency so we don't need to vendor both or choose to
vendor
+// both specifying the minor version in the relocation path.
+
+// To produce the list of necessary relocations, one needs to start with a
set of target
Review comment:
This also seems like an area to expand into its own document somewhere ("How
to update Beam Dependencies"). I wouldn't normally expect to go through this
workflow when updating a dependency version. /cc @yifanzou and @chamikaramj,
who are adding policy checks to make dependency updates more regular.
Can you mention any tools or commands you're familiar with to capture the
set of transitive dependency versions which need to be updated?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111598)
Time Spent: 6.5h (was: 6h 20m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111597=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111597
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 13/Jun/18 18:31
Start Date: 13/Jun/18 18:31
Worklog Time Spent: 10m
Work Description: swegner commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r195190138
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
+relocate "com.google.gson",
"org.apache.beam.vendor.gson.v2.com.google.gson"
+relocate "io.grpc", "org.apache.beam.vendor.grpc.v1.io.grpc"
+relocate "com.google.auth",
"org.apache.beam.vendor.google_auth_library_credentials.v0_9_1.com.google.auth"
+relocate "com.google.api",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.api"
+relocate "com.google.cloud",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.cloud"
+relocate "com.google.logging",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.logging"
+relocate "com.google.longrunning",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.longrunning"
+relocate "com.google.rpc",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.rpc"
+relocate "com.google.type",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.type"
+relocate "io.opencensus",
"org.apache.beam.vendor.opencensus.v0_11.io.opencensus"
+
+// Adapted from
https://github.com/grpc/grpc-java/blob/e283f70ad91f99c7fee8b31b605ef12a4f9b1690/netty/shaded/build.gradle#L41
+relocate "io.netty", "org.apache.beam.vendor.netty.v4.io.netty"
+// We have to be careful with these replacements as they must not match any
+// string in NativeLibraryLoader, else they cause corruption. Note that
+// this includes concatenation of string literals and constants.
+relocate 'META-INF/native/libnetty',
'META-INF/native/liborg_apache_beam_vendor_netty_v4_netty'
+relocate 'META-INF/native/netty',
'META-INF/native/org_apache_beam_vendor_netty_v4_netty'
+
+// Don't include errorprone, JDK8 annotations, objenesis, junit, and
mockito in the bundled jar
+exclude "com/google/errorprone/**"
+exclude "com/google/instrumentation/**"
+exclude "javax/annotation/**"
+exclude "junit/**"
+exclude "org/hamcrest/**"
+exclude "org/junit/**"
+exclude "org/mockito/**"
+exclude "org/objenesis/**"
+ })
+
+ // Don't force modules here because we don't want to take the shared
declarations in build_rules.gradle
+ // because we would like to have the freedom to choose which versions of
dependencies we
+ // are using for the portability APIs separate from what is being used
inside other modules such as GCP.
+ configurations.all { config ->
+config.resolutionStrategy {
+ forcedModules = []
+}
+ }
+
+ apply plugin: "com.google.protobuf"
+ protobuf {
+protoc {
+ // The artifact spec for the Protobuf Compiler
+ artifact = "com.google.protobuf:protoc:3.5.1"
+}
+
+// Configure the codegen plugins
+plugins {
+ // An artifact spec for a protoc plugin, with "grpc" as
+ // the identifier, which can be referred to in the "plugins"
+ // container of the "generateProtoTasks" closure.
+ grpc {
+artifact = "io.grpc:protoc-gen-grpc-java:1.12.0"
+ }
+}
+
+generateProtoTasks {
+ ofSourceSet("main")*.plugins {
+// Apply the "grpc" plugin whose spec is defined above, without
+// options. Note the braces cannot be omitted, otherwise the
+// plugin will not be added. This is because of the implicit way
+// NamedDomainObjectContainer binds the methods.
+grpc { }
+ }
+}
+ }
+
+ dependencies {
+compile 'com.google.guava:guava:20.0'
Review comment:
Hmm, I still worry about the two version lists drifting. It seems like in
the common scenario we want the versions to match, right? Would
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111596=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111596 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 13/Jun/18 18:31 Start Date: 13/Jun/18 18:31 Worklog Time Spent: 10m Work Description: swegner commented on a change in pull request #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#discussion_r195190537 ## File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/ArtifactServiceStager.java ## @@ -57,6 +54,9 @@ import org.apache.beam.model.jobmanagement.v1.ArtifactStagingServiceGrpc.ArtifactStagingServiceStub; import org.apache.beam.sdk.util.MoreFutures; import org.apache.beam.sdk.util.ThrowingSupplier; +import org.apache.beam.vendor.grpc.v1.io.grpc.Channel; Review comment: Makes sense to me, thanks This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 111596) Time Spent: 6h 20m (was: 6h 10m) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 6h 20m > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111592=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111592 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 13/Jun/18 18:16 Start Date: 13/Jun/18 18:16 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#issuecomment-397035671 This change is blocked on deletion of the POMs as this will break the Maven build. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 111592) Time Spent: 6h (was: 5h 50m) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 6h > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111583=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111583
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 13/Jun/18 17:57
Start Date: 13/Jun/18 17:57
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r195180327
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
Review comment:
Done
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111583)
Time Spent: 5h 50m (was: 5h 40m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111581=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111581
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 13/Jun/18 17:54
Start Date: 13/Jun/18 17:54
Worklog Time Spent: 10m
Work Description: swegner commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r195179217
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
Review comment:
I agree that the value of static analysis on a project of all generated code
is dubious. But there is value in getting the entire codebase opted-in to
static analysis, and our build logic will be simpler if there's just one set of
configuration.
Since ErrorProne passes, let's enable that. My understanding is that there
is effort to remove FindBugs, so don't worry about that for now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111581)
Time Spent: 5h 40m (was: 5.5h)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111579=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111579
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 13/Jun/18 17:48
Start Date: 13/Jun/18 17:48
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r195176159
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
Review comment:
We have no control over the generated code so it seems silly to force it to
a standard such as error prone/findbugs
I tested this out right now and findbugs fails but error prone doesn't cause
any warnings so if you really want the check, I'll swap.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111579)
Time Spent: 5.5h (was: 5h 20m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111578=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111578
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 13/Jun/18 17:45
Start Date: 13/Jun/18 17:45
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r195176159
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
Review comment:
We have no control over the generated code so it seems silly to force it to
a standard such as error prone/findbugs
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111578)
Time Spent: 5h 20m (was: 5h 10m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111571=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111571
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 13/Jun/18 17:32
Start Date: 13/Jun/18 17:32
Worklog Time Spent: 10m
Work Description: swegner commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r195172189
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
Review comment:
If that's the case, can we make `failOnWarning: true`? Eventually we want to
remove the switch altogether and make failOnWarning default; if these projects
are only generated code then I wouldn't expect to get any violations anyway.
ErrorProne is configured to ignore generated code.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111571)
Time Spent: 5h 10m (was: 5h)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111251=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111251
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 20:15
Start Date: 12/Jun/18 20:15
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194861219
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
+relocate "com.google.gson",
"org.apache.beam.vendor.gson.v2.com.google.gson"
+relocate "io.grpc", "org.apache.beam.vendor.grpc.v1.io.grpc"
+relocate "com.google.auth",
"org.apache.beam.vendor.google_auth_library_credentials.v0_9_1.com.google.auth"
+relocate "com.google.api",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.api"
+relocate "com.google.cloud",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.cloud"
+relocate "com.google.logging",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.logging"
+relocate "com.google.longrunning",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.longrunning"
+relocate "com.google.rpc",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.rpc"
+relocate "com.google.type",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.type"
+relocate "io.opencensus",
"org.apache.beam.vendor.opencensus.v0_11.io.opencensus"
+
+// Adapted from
https://github.com/grpc/grpc-java/blob/e283f70ad91f99c7fee8b31b605ef12a4f9b1690/netty/shaded/build.gradle#L41
+relocate "io.netty", "org.apache.beam.vendor.netty.v4.io.netty"
+// We have to be careful with these replacements as they must not match any
+// string in NativeLibraryLoader, else they cause corruption. Note that
+// this includes concatenation of string literals and constants.
+relocate 'META-INF/native/libnetty',
'META-INF/native/liborg_apache_beam_vendor_netty_v4_netty'
+relocate 'META-INF/native/netty',
'META-INF/native/org_apache_beam_vendor_netty_v4_netty'
+
+// Don't include errorprone, JDK8 annotations, objenesis, junit, and
mockito in the bundled jar
+exclude "com/google/errorprone/**"
+exclude "com/google/instrumentation/**"
+exclude "javax/annotation/**"
+exclude "junit/**"
+exclude "org/hamcrest/**"
+exclude "org/junit/**"
+exclude "org/mockito/**"
+exclude "org/objenesis/**"
+ })
+
+ // Don't force modules here because we don't want to take the shared
declarations in build_rules.gradle
+ // because we would like to have the freedom to choose which versions of
dependencies we
+ // are using for the portability APIs separate from what is being used
inside other modules such as GCP.
+ configurations.all { config ->
+config.resolutionStrategy {
+ forcedModules = []
+}
+ }
+
+ apply plugin: "com.google.protobuf"
+ protobuf {
+protoc {
+ // The artifact spec for the Protobuf Compiler
+ artifact = "com.google.protobuf:protoc:3.5.1"
+}
+
+// Configure the codegen plugins
+plugins {
+ // An artifact spec for a protoc plugin, with "grpc" as
+ // the identifier, which can be referred to in the "plugins"
+ // container of the "generateProtoTasks" closure.
+ grpc {
+artifact = "io.grpc:protoc-gen-grpc-java:1.12.0"
+ }
+}
+
+generateProtoTasks {
+ ofSourceSet("main")*.plugins {
+// Apply the "grpc" plugin whose spec is defined above, without
+// options. Note the braces cannot be omitted, otherwise the
+// plugin will not be added. This is because of the implicit way
+// NamedDomainObjectContainer binds the methods.
+grpc { }
+ }
+}
+ }
+
+ dependencies {
+compile 'com.google.guava:guava:20.0'
Review comment:
With vendoring, version drift only leads to larger jars since we will
possibly be packaging multiple versions of the same dependency,
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111241=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111241
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 20:00
Start Date: 12/Jun/18 20:00
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194860296
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
Review comment:
Added comment explaining the process.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111241)
Time Spent: 4.5h (was: 4h 20m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111243=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111243
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 20:00
Start Date: 12/Jun/18 20:00
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194861219
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
+relocate "com.google.gson",
"org.apache.beam.vendor.gson.v2.com.google.gson"
+relocate "io.grpc", "org.apache.beam.vendor.grpc.v1.io.grpc"
+relocate "com.google.auth",
"org.apache.beam.vendor.google_auth_library_credentials.v0_9_1.com.google.auth"
+relocate "com.google.api",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.api"
+relocate "com.google.cloud",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.cloud"
+relocate "com.google.logging",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.logging"
+relocate "com.google.longrunning",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.longrunning"
+relocate "com.google.rpc",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.rpc"
+relocate "com.google.type",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.type"
+relocate "io.opencensus",
"org.apache.beam.vendor.opencensus.v0_11.io.opencensus"
+
+// Adapted from
https://github.com/grpc/grpc-java/blob/e283f70ad91f99c7fee8b31b605ef12a4f9b1690/netty/shaded/build.gradle#L41
+relocate "io.netty", "org.apache.beam.vendor.netty.v4.io.netty"
+// We have to be careful with these replacements as they must not match any
+// string in NativeLibraryLoader, else they cause corruption. Note that
+// this includes concatenation of string literals and constants.
+relocate 'META-INF/native/libnetty',
'META-INF/native/liborg_apache_beam_vendor_netty_v4_netty'
+relocate 'META-INF/native/netty',
'META-INF/native/org_apache_beam_vendor_netty_v4_netty'
+
+// Don't include errorprone, JDK8 annotations, objenesis, junit, and
mockito in the bundled jar
+exclude "com/google/errorprone/**"
+exclude "com/google/instrumentation/**"
+exclude "javax/annotation/**"
+exclude "junit/**"
+exclude "org/hamcrest/**"
+exclude "org/junit/**"
+exclude "org/mockito/**"
+exclude "org/objenesis/**"
+ })
+
+ // Don't force modules here because we don't want to take the shared
declarations in build_rules.gradle
+ // because we would like to have the freedom to choose which versions of
dependencies we
+ // are using for the portability APIs separate from what is being used
inside other modules such as GCP.
+ configurations.all { config ->
+config.resolutionStrategy {
+ forcedModules = []
+}
+ }
+
+ apply plugin: "com.google.protobuf"
+ protobuf {
+protoc {
+ // The artifact spec for the Protobuf Compiler
+ artifact = "com.google.protobuf:protoc:3.5.1"
+}
+
+// Configure the codegen plugins
+plugins {
+ // An artifact spec for a protoc plugin, with "grpc" as
+ // the identifier, which can be referred to in the "plugins"
+ // container of the "generateProtoTasks" closure.
+ grpc {
+artifact = "io.grpc:protoc-gen-grpc-java:1.12.0"
+ }
+}
+
+generateProtoTasks {
+ ofSourceSet("main")*.plugins {
+// Apply the "grpc" plugin whose spec is defined above, without
+// options. Note the braces cannot be omitted, otherwise the
+// plugin will not be added. This is because of the implicit way
+// NamedDomainObjectContainer binds the methods.
+grpc { }
+ }
+}
+ }
+
+ dependencies {
+compile 'com.google.guava:guava:20.0'
Review comment:
With vendoring, version drift only leads to larger jars since we will
possibly be packaging multiple versions of the same dependency,
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111244=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111244
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 20:00
Start Date: 12/Jun/18 20:00
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194863255
##
File path: model/fn-execution/build.gradle
##
@@ -17,17 +17,14 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature(enableFindbugs: false, enableErrorProne: false)
-applyGrpcNature()
+applyPortabilityNature()
description = "Apache Beam :: Model :: Fn Execution"
ext.summary = "Portable definitions for execution user-defined functions."
dependencies {
- compile library.java.guava
- shadow project(path: ":beam-model-pipeline", configuration: "shadow")
- shadow library.java.protobuf_java
- shadow library.java.grpc_core
- shadow library.java.grpc_protobuf
- shadow library.java.grpc_stub
+ // We purposely depend on the unshaded classes for compilation and
Review comment:
I couldn't figure out a way to have the generated source use a vendored
version of grpc/protobuf. So the generated code uses imports like
`com.google.protobuf.Message` and expects classes from `beam-model-pipeline` to
extend non-vendored gRPC/protobuf classes.
The flow is:
1) generate code using non-vendored imports
2) compile code using non-vendored dependencies
3) produce shaded classes using the same shading rules as
`beam-model-pipeline`
This works because `beam-model-pipeline`, `beam-model-job-management`, and
`beam-model-fn-execution` all use the same relocation rules so when the jar is
loaded by java, java will see that they all extend the same vendored code
locations.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111244)
Time Spent: 4h 50m (was: 4h 40m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111242=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111242
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 20:00
Start Date: 12/Jun/18 20:00
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194861371
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
+relocate "com.google.gson",
"org.apache.beam.vendor.gson.v2.com.google.gson"
+relocate "io.grpc", "org.apache.beam.vendor.grpc.v1.io.grpc"
+relocate "com.google.auth",
"org.apache.beam.vendor.google_auth_library_credentials.v0_9_1.com.google.auth"
+relocate "com.google.api",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.api"
+relocate "com.google.cloud",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.cloud"
+relocate "com.google.logging",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.logging"
+relocate "com.google.longrunning",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.longrunning"
+relocate "com.google.rpc",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.rpc"
+relocate "com.google.type",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.type"
+relocate "io.opencensus",
"org.apache.beam.vendor.opencensus.v0_11.io.opencensus"
+
+// Adapted from
https://github.com/grpc/grpc-java/blob/e283f70ad91f99c7fee8b31b605ef12a4f9b1690/netty/shaded/build.gradle#L41
+relocate "io.netty", "org.apache.beam.vendor.netty.v4.io.netty"
+// We have to be careful with these replacements as they must not match any
+// string in NativeLibraryLoader, else they cause corruption. Note that
+// this includes concatenation of string literals and constants.
+relocate 'META-INF/native/libnetty',
'META-INF/native/liborg_apache_beam_vendor_netty_v4_netty'
+relocate 'META-INF/native/netty',
'META-INF/native/org_apache_beam_vendor_netty_v4_netty'
+
+// Don't include errorprone, JDK8 annotations, objenesis, junit, and
mockito in the bundled jar
+exclude "com/google/errorprone/**"
+exclude "com/google/instrumentation/**"
+exclude "javax/annotation/**"
+exclude "junit/**"
+exclude "org/hamcrest/**"
+exclude "org/junit/**"
+exclude "org/mockito/**"
+exclude "org/objenesis/**"
+ })
+
+ // Don't force modules here because we don't want to take the shared
declarations in build_rules.gradle
+ // because we would like to have the freedom to choose which versions of
dependencies we
+ // are using for the portability APIs separate from what is being used
inside other modules such as GCP.
+ configurations.all { config ->
+config.resolutionStrategy {
+ forcedModules = []
+}
+ }
+
+ apply plugin: "com.google.protobuf"
+ protobuf {
+protoc {
+ // The artifact spec for the Protobuf Compiler
+ artifact = "com.google.protobuf:protoc:3.5.1"
+}
+
+// Configure the codegen plugins
+plugins {
+ // An artifact spec for a protoc plugin, with "grpc" as
+ // the identifier, which can be referred to in the "plugins"
+ // container of the "generateProtoTasks" closure.
+ grpc {
+artifact = "io.grpc:protoc-gen-grpc-java:1.12.0"
+ }
+}
+
+generateProtoTasks {
+ ofSourceSet("main")*.plugins {
+// Apply the "grpc" plugin whose spec is defined above, without
+// options. Note the braces cannot be omitted, otherwise the
+// plugin will not be added. This is because of the implicit way
+// NamedDomainObjectContainer binds the methods.
+grpc { }
+ }
+}
+ }
+
+ dependencies {
+compile 'com.google.guava:guava:20.0'
+compile 'com.google.protobuf:protobuf-java:3.5.1'
+compile 'com.google.protobuf:protobuf-java-util:3.5.1'
+compile 'com.google.code.gson:gson:2.7'
+
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111233=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111233
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 19:24
Start Date: 12/Jun/18 19:24
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r19486
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
Review comment:
That is already being done by the
`validateShadedJarDoesntLeakNonOrgApacheBeamClasses` task.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111233)
Time Spent: 4h 20m (was: 4h 10m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111232=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111232
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 19:23
Start Date: 12/Jun/18 19:23
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194859792
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
Review comment:
Eventually all projects will not do any shading and instead use normal
dependencies (no more `shadow`, `shadowTest`). The only reason why I place this
into a function right now is that I want to repeat this same logic across
`model/pipeline`, `model/job-management`, `model/fn-execution`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111232)
Time Spent: 4h 10m (was: 4h)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 4h 10m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=111229=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-111229
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 19:20
Start Date: 12/Jun/18 19:20
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194858852
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
+relocate "com.google.gson",
"org.apache.beam.vendor.gson.v2.com.google.gson"
+relocate "io.grpc", "org.apache.beam.vendor.grpc.v1.io.grpc"
+relocate "com.google.auth",
"org.apache.beam.vendor.google_auth_library_credentials.v0_9_1.com.google.auth"
+relocate "com.google.api",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.api"
+relocate "com.google.cloud",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.cloud"
+relocate "com.google.logging",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.logging"
+relocate "com.google.longrunning",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.longrunning"
+relocate "com.google.rpc",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.rpc"
+relocate "com.google.type",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.type"
+relocate "io.opencensus",
"org.apache.beam.vendor.opencensus.v0_11.io.opencensus"
+
+// Adapted from
https://github.com/grpc/grpc-java/blob/e283f70ad91f99c7fee8b31b605ef12a4f9b1690/netty/shaded/build.gradle#L41
+relocate "io.netty", "org.apache.beam.vendor.netty.v4.io.netty"
Review comment:
Because I wanted to include support for domain sockets and other epoll
alternatives which is not part of the gRPC netty-shaded equivalent.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 111229)
Time Spent: 3h 50m (was: 3h 40m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110919=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110919
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 00:41
Start Date: 12/Jun/18 00:41
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not
have dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194586156
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
Review comment:
Is there a way to error out if a jar contains a package that is not
relocated here ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 110919)
Time Spent: 3h 40m (was: 3.5h)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110917=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110917
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 00:41
Start Date: 12/Jun/18 00:41
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not
have dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194587134
##
File path: model/fn-execution/build.gradle
##
@@ -17,17 +17,14 @@
*/
apply from: project(":").file("build_rules.gradle")
-applyJavaNature(enableFindbugs: false, enableErrorProne: false)
-applyGrpcNature()
+applyPortabilityNature()
description = "Apache Beam :: Model :: Fn Execution"
ext.summary = "Portable definitions for execution user-defined functions."
dependencies {
- compile library.java.guava
- shadow project(path: ":beam-model-pipeline", configuration: "shadow")
- shadow library.java.protobuf_java
- shadow library.java.grpc_core
- shadow library.java.grpc_protobuf
- shadow library.java.grpc_stub
+ // We purposely depend on the unshaded classes for compilation and
Review comment:
Can you explain how this works ? Won't compiled classes refer to unshaded
classes ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 110917)
Time Spent: 3.5h (was: 3h 20m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110918=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110918
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 00:41
Start Date: 12/Jun/18 00:41
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not
have dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194586401
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
+relocate "com.google.gson",
"org.apache.beam.vendor.gson.v2.com.google.gson"
+relocate "io.grpc", "org.apache.beam.vendor.grpc.v1.io.grpc"
+relocate "com.google.auth",
"org.apache.beam.vendor.google_auth_library_credentials.v0_9_1.com.google.auth"
Review comment:
Add a function for formatting relocation path from various parts
("org.apache.beam", library name, package, etc) ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 110918)
Time Spent: 3.5h (was: 3h 20m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110920=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110920
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 00:41
Start Date: 12/Jun/18 00:41
Worklog Time Spent: 10m
Work Description: chamikaramj commented on a change in pull request
#5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not
have dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194586642
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
+relocate "com.google.gson",
"org.apache.beam.vendor.gson.v2.com.google.gson"
+relocate "io.grpc", "org.apache.beam.vendor.grpc.v1.io.grpc"
+relocate "com.google.auth",
"org.apache.beam.vendor.google_auth_library_credentials.v0_9_1.com.google.auth"
+relocate "com.google.api",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.api"
+relocate "com.google.cloud",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.cloud"
+relocate "com.google.logging",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.logging"
+relocate "com.google.longrunning",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.longrunning"
+relocate "com.google.rpc",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.rpc"
+relocate "com.google.type",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.type"
+relocate "io.opencensus",
"org.apache.beam.vendor.opencensus.v0_11.io.opencensus"
+
+// Adapted from
https://github.com/grpc/grpc-java/blob/e283f70ad91f99c7fee8b31b605ef12a4f9b1690/netty/shaded/build.gradle#L41
+relocate "io.netty", "org.apache.beam.vendor.netty.v4.io.netty"
Review comment:
Why not upgrade to gRPC netty shaded instead ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 110920)
Time Spent: 3h 40m (was: 3.5h)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110914=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110914 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 12/Jun/18 00:29 Start Date: 12/Jun/18 00:29 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#issuecomment-396427930 @kennknowles It isn't a separate package yet, its just being shaded into the model jars since one of the issues that I didn't figure out was how to get the generated code to use vendored classes. As a side note, I was hoping that you would use this as an example in the PR that you created that vendored guava and I would be able to re-use what you created. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110914) Time Spent: 3h 20m (was: 3h 10m) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 3h 20m > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110908=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110908 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 12/Jun/18 00:25 Start Date: 12/Jun/18 00:25 Worklog Time Spent: 10m Work Description: lukecwik commented on a change in pull request #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#discussion_r194585437 ## File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/ArtifactServiceStager.java ## @@ -57,6 +54,9 @@ import org.apache.beam.model.jobmanagement.v1.ArtifactStagingServiceGrpc.ArtifactStagingServiceStub; import org.apache.beam.sdk.util.MoreFutures; import org.apache.beam.sdk.util.ThrowingSupplier; +import org.apache.beam.vendor.grpc.v1.io.grpc.Channel; Review comment: Having sorted order is easiest and supported by the most amount of tooling so deviating makes those peoples lives worse. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110908) Time Spent: 3h 10m (was: 3h) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 3h 10m > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110907=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110907
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 00:23
Start Date: 12/Jun/18 00:23
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194585279
##
File path: build_rules.gradle
##
@@ -560,6 +560,7 @@ ext.applyJavaNature = {
shadowJar ({
classifier = "shaded"
mergeServiceFiles()
+zip64 true
Review comment:
It allows jar files to contain 2^16 or more files.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 110907)
Time Spent: 3h (was: 2h 50m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 3h
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110906=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110906
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 12/Jun/18 00:22
Start Date: 12/Jun/18 00:22
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194585154
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
Review comment:
These modules that this applies to only have generated code so I don't
believe error prone will be of use which is why I don't expose it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 110906)
Time Spent: 2h 50m (was: 2h 40m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110849=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110849 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 11/Jun/18 21:26 Start Date: 11/Jun/18 21:26 Worklog Time Spent: 10m Work Description: kennknowles commented on issue #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#issuecomment-396392713 Where is the bit where the vendored jar is published? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110849) Time Spent: 2h 40m (was: 2.5h) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 2h 40m > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110804=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110804
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 11/Jun/18 20:29
Start Date: 11/Jun/18 20:29
Worklog Time Spent: 10m
Work Description: swegner commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194534119
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
+relocate "com.google.gson",
"org.apache.beam.vendor.gson.v2.com.google.gson"
+relocate "io.grpc", "org.apache.beam.vendor.grpc.v1.io.grpc"
+relocate "com.google.auth",
"org.apache.beam.vendor.google_auth_library_credentials.v0_9_1.com.google.auth"
+relocate "com.google.api",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.api"
+relocate "com.google.cloud",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.cloud"
+relocate "com.google.logging",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.logging"
+relocate "com.google.longrunning",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.longrunning"
+relocate "com.google.rpc",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.rpc"
+relocate "com.google.type",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.type"
+relocate "io.opencensus",
"org.apache.beam.vendor.opencensus.v0_11.io.opencensus"
+
+// Adapted from
https://github.com/grpc/grpc-java/blob/e283f70ad91f99c7fee8b31b605ef12a4f9b1690/netty/shaded/build.gradle#L41
+relocate "io.netty", "org.apache.beam.vendor.netty.v4.io.netty"
+// We have to be careful with these replacements as they must not match any
+// string in NativeLibraryLoader, else they cause corruption. Note that
+// this includes concatenation of string literals and constants.
+relocate 'META-INF/native/libnetty',
'META-INF/native/liborg_apache_beam_vendor_netty_v4_netty'
+relocate 'META-INF/native/netty',
'META-INF/native/org_apache_beam_vendor_netty_v4_netty'
+
+// Don't include errorprone, JDK8 annotations, objenesis, junit, and
mockito in the bundled jar
+exclude "com/google/errorprone/**"
+exclude "com/google/instrumentation/**"
+exclude "javax/annotation/**"
+exclude "junit/**"
+exclude "org/hamcrest/**"
+exclude "org/junit/**"
+exclude "org/mockito/**"
+exclude "org/objenesis/**"
+ })
+
+ // Don't force modules here because we don't want to take the shared
declarations in build_rules.gradle
+ // because we would like to have the freedom to choose which versions of
dependencies we
+ // are using for the portability APIs separate from what is being used
inside other modules such as GCP.
+ configurations.all { config ->
+config.resolutionStrategy {
+ forcedModules = []
+}
+ }
+
+ apply plugin: "com.google.protobuf"
+ protobuf {
+protoc {
+ // The artifact spec for the Protobuf Compiler
+ artifact = "com.google.protobuf:protoc:3.5.1"
+}
+
+// Configure the codegen plugins
+plugins {
+ // An artifact spec for a protoc plugin, with "grpc" as
+ // the identifier, which can be referred to in the "plugins"
+ // container of the "generateProtoTasks" closure.
+ grpc {
+artifact = "io.grpc:protoc-gen-grpc-java:1.12.0"
+ }
+}
+
+generateProtoTasks {
+ ofSourceSet("main")*.plugins {
+// Apply the "grpc" plugin whose spec is defined above, without
+// options. Note the braces cannot be omitted, otherwise the
+// plugin will not be added. This is because of the implicit way
+// NamedDomainObjectContainer binds the methods.
+grpc { }
+ }
+}
+ }
+
+ dependencies {
+compile 'com.google.guava:guava:20.0'
+compile 'com.google.protobuf:protobuf-java:3.5.1'
+compile 'com.google.protobuf:protobuf-java-util:3.5.1'
+compile 'com.google.code.gson:gson:2.7'
+
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110808=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110808
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 11/Jun/18 20:29
Start Date: 11/Jun/18 20:29
Worklog Time Spent: 10m
Work Description: swegner commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194532725
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
Review comment:
Where do these version strings come from? If they need to be maintained, can
you add some notes/validation?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 110808)
Time Spent: 2h 20m (was: 2h 10m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110806=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110806
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 11/Jun/18 20:29
Start Date: 11/Jun/18 20:29
Worklog Time Spent: 10m
Work Description: swegner commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194531533
##
File path: build_rules.gradle
##
@@ -560,6 +560,7 @@ ext.applyJavaNature = {
shadowJar ({
classifier = "shaded"
mergeServiceFiles()
+zip64 true
Review comment:
What does this do?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 110806)
Time Spent: 2h (was: 1h 50m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110809=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110809 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 11/Jun/18 20:29 Start Date: 11/Jun/18 20:29 Worklog Time Spent: 10m Work Description: swegner commented on a change in pull request #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#discussion_r194536787 ## File path: runners/core-construction-java/src/main/java/org/apache/beam/runners/core/construction/ArtifactServiceStager.java ## @@ -57,6 +54,9 @@ import org.apache.beam.model.jobmanagement.v1.ArtifactStagingServiceGrpc.ArtifactStagingServiceStub; import org.apache.beam.sdk.util.MoreFutures; import org.apache.beam.sdk.util.ThrowingSupplier; +import org.apache.beam.vendor.grpc.v1.io.grpc.Channel; Review comment: Code style bikeshed: Would it be useful to separate out vendored imports? For checkstyle we could declare a separate import group to sort them at the end: http://checkstyle.sourceforge.net/config_imports.html#ImportOrder This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110809) Time Spent: 2.5h (was: 2h 20m) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 2.5h > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110807=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110807
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 11/Jun/18 20:29
Start Date: 11/Jun/18 20:29
Worklog Time Spent: 10m
Work Description: swegner commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194533859
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
+// guava uses the com.google.common and com.google.thirdparty package
namespaces
+relocate "com.google.common",
"org.apache.beam.vendor.guava.v20.com.google.common"
+relocate "com.google.thirdparty",
"org.apache.beam.vendor.guava.v20.com.google.thirdparty"
+
+relocate "com.google.protobuf",
"org.apache.beam.vendor.protobuf.v3.com.google.protobuf"
+relocate "com.google.gson",
"org.apache.beam.vendor.gson.v2.com.google.gson"
+relocate "io.grpc", "org.apache.beam.vendor.grpc.v1.io.grpc"
+relocate "com.google.auth",
"org.apache.beam.vendor.google_auth_library_credentials.v0_9_1.com.google.auth"
+relocate "com.google.api",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.api"
+relocate "com.google.cloud",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.cloud"
+relocate "com.google.logging",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.logging"
+relocate "com.google.longrunning",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.longrunning"
+relocate "com.google.rpc",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.rpc"
+relocate "com.google.type",
"org.apache.beam.vendor.proto_google_common_protos.v1.com.google.type"
+relocate "io.opencensus",
"org.apache.beam.vendor.opencensus.v0_11.io.opencensus"
+
+// Adapted from
https://github.com/grpc/grpc-java/blob/e283f70ad91f99c7fee8b31b605ef12a4f9b1690/netty/shaded/build.gradle#L41
+relocate "io.netty", "org.apache.beam.vendor.netty.v4.io.netty"
+// We have to be careful with these replacements as they must not match any
+// string in NativeLibraryLoader, else they cause corruption. Note that
+// this includes concatenation of string literals and constants.
+relocate 'META-INF/native/libnetty',
'META-INF/native/liborg_apache_beam_vendor_netty_v4_netty'
+relocate 'META-INF/native/netty',
'META-INF/native/org_apache_beam_vendor_netty_v4_netty'
+
+// Don't include errorprone, JDK8 annotations, objenesis, junit, and
mockito in the bundled jar
+exclude "com/google/errorprone/**"
+exclude "com/google/instrumentation/**"
+exclude "javax/annotation/**"
+exclude "junit/**"
+exclude "org/hamcrest/**"
+exclude "org/junit/**"
+exclude "org/mockito/**"
+exclude "org/objenesis/**"
+ })
+
+ // Don't force modules here because we don't want to take the shared
declarations in build_rules.gradle
+ // because we would like to have the freedom to choose which versions of
dependencies we
+ // are using for the portability APIs separate from what is being used
inside other modules such as GCP.
+ configurations.all { config ->
+config.resolutionStrategy {
+ forcedModules = []
+}
+ }
+
+ apply plugin: "com.google.protobuf"
+ protobuf {
+protoc {
+ // The artifact spec for the Protobuf Compiler
+ artifact = "com.google.protobuf:protoc:3.5.1"
+}
+
+// Configure the codegen plugins
+plugins {
+ // An artifact spec for a protoc plugin, with "grpc" as
+ // the identifier, which can be referred to in the "plugins"
+ // container of the "generateProtoTasks" closure.
+ grpc {
+artifact = "io.grpc:protoc-gen-grpc-java:1.12.0"
+ }
+}
+
+generateProtoTasks {
+ ofSourceSet("main")*.plugins {
+// Apply the "grpc" plugin whose spec is defined above, without
+// options. Note the braces cannot be omitted, otherwise the
+// plugin will not be added. This is because of the implicit way
+// NamedDomainObjectContainer binds the methods.
+grpc { }
+ }
+}
+ }
+
+ dependencies {
+compile 'com.google.guava:guava:20.0'
Review comment:
Is it an explicit choice to maintain these dependency versions separate from
the `ext.library` map? Any reason why? It seems like we
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110805=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110805
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 11/Jun/18 20:29
Start Date: 11/Jun/18 20:29
Worklog Time Spent: 10m
Work Description: swegner commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194537105
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
Review comment:
Will this shading configuration eventually be common to all java projects
(not just portability)?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 110805)
Time Spent: 1h 50m (was: 1h 40m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110803=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110803
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 11/Jun/18 20:29
Start Date: 11/Jun/18 20:29
Worklog Time Spent: 10m
Work Description: swegner commented on a change in pull request #5594:
[BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have
dependency conflicts
URL: https://github.com/apache/beam/pull/5594#discussion_r194532318
##
File path: build_rules.gradle
##
@@ -1253,6 +1254,125 @@ ext.applyGrpcNature = {
/*/
+ext.applyPortabilityNature = {
+ println "applyPortabilityNature with " + (it ? "$it" : "default
configuration") + " for project $project.name"
+ applyJavaNature(enableFindbugs: false, enableErrorProne: false,
shadowClosure: {
Review comment:
We've been working hard to fix ErrorProne warnings per-module with the
eventual goal of making ErrorProne violations produce errors by default. How
about hoisting the `enableErrorProne`configuration as a parameter to
`applyPortabilityNature()`. Then make sure to retain the configured state for
migrated projects.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 110803)
Time Spent: 1.5h (was: 1h 20m)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110759=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110759 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 11/Jun/18 20:05 Start Date: 11/Jun/18 20:05 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#issuecomment-396369306 R: @swegner This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110759) Time Spent: 1h 20m (was: 1h 10m) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110755=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110755 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 11/Jun/18 20:04 Start Date: 11/Jun/18 20:04 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#issuecomment-396368946 Run Go PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110755) Time Spent: 1h 10m (was: 1h) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110689=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110689 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 11/Jun/18 17:06 Start Date: 11/Jun/18 17:06 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#issuecomment-396289440 Run Go PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110689) Time Spent: 1h (was: 50m) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=110654=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-110654 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 11/Jun/18 15:43 Start Date: 11/Jun/18 15:43 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5594: [BEAM-4481, BEAM-4484] Start vendoring portability dependencies to not have dependency conflicts URL: https://github.com/apache/beam/pull/5594#issuecomment-396289440 Run Go PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 110654) Time Spent: 50m (was: 40m) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[
https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=109157=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109157
]
ASF GitHub Bot logged work on BEAM-4481:
Author: ASF GitHub Bot
Created on: 05/Jun/18 18:25
Start Date: 05/Jun/18 18:25
Worklog Time Spent: 10m
Work Description: lukecwik closed pull request #5560: [BEAM-4481] Remove
duplicate definitions of dependencies in runners/direct-java
URL: https://github.com/apache/beam/pull/5560
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/runners/direct-java/build.gradle b/runners/direct-java/build.gradle
index 1cb6f49709f..d095ae116f6 100644
--- a/runners/direct-java/build.gradle
+++ b/runners/direct-java/build.gradle
@@ -63,13 +63,7 @@ dependencies {
dependOnProjects.each {
compile project(path: it, configuration: "shadow")
}
- compile project(path: ":beam-model-pipeline", configuration: "shadow")
- compile project(path: ":beam-runners-core-construction-java", configuration:
"shadow")
- compile project(path: ":beam-runners-core-java", configuration: "shadow")
- compile project(path: ":beam-runners-java-fn-execution", configuration:
"shadow")
- compile project(path: ":beam-runners-local-java-core", configuration:
"shadow")
shadow project(path: ":beam-sdks-java-core", configuration: "shadow")
- shadow project(path: ":beam-sdks-java-fn-execution", configuration: "shadow")
shadow library.java.joda_time
shadow library.java.findbugs_jsr305
shadow library.java.slf4j_api
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 109157)
Time Spent: 40m (was: 0.5h)
> Remove duplicate dependency declarations from runners/direct-java
> -
>
> Key: BEAM-4481
> URL: https://issues.apache.org/jira/browse/BEAM-4481
> Project: Beam
> Issue Type: Bug
> Components: runner-direct
>Reporter: Luke Cwik
>Assignee: Luke Cwik
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> beam-model-pipeline and others are duplicated in the dependency list
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=109133=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109133 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 05/Jun/18 17:12 Start Date: 05/Jun/18 17:12 Worklog Time Spent: 10m Work Description: boyuanzz commented on issue #5560: [BEAM-4481] Remove duplicate definitions of dependencies in runners/direct-java URL: https://github.com/apache/beam/pull/5560#issuecomment-394789026 So sorry forgetting remove these duplications last time. Thanks Luke! LGTM. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 109133) Time Spent: 0.5h (was: 20m) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=109131=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109131 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 05/Jun/18 17:07 Start Date: 05/Jun/18 17:07 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #5560: [BEAM-4481] Remove duplicate definitions of dependencies in runners/direct-java URL: https://github.com/apache/beam/pull/5560#issuecomment-394787548 R: @boyuanzz This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 109131) Time Spent: 20m (was: 10m) > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Work logged] (BEAM-4481) Remove duplicate dependency declarations from runners/direct-java
[ https://issues.apache.org/jira/browse/BEAM-4481?focusedWorklogId=109130=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109130 ] ASF GitHub Bot logged work on BEAM-4481: Author: ASF GitHub Bot Created on: 05/Jun/18 17:07 Start Date: 05/Jun/18 17:07 Worklog Time Spent: 10m Work Description: lukecwik opened a new pull request #5560: [BEAM-4481] Remove duplicate definitions of dependencies in runners/direct-java URL: https://github.com/apache/beam/pull/5560 Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). It will help us expedite review of your Pull Request if you tag someone (e.g. `@username`) to look at it. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 109130) Time Spent: 10m Remaining Estimate: 0h > Remove duplicate dependency declarations from runners/direct-java > - > > Key: BEAM-4481 > URL: https://issues.apache.org/jira/browse/BEAM-4481 > Project: Beam > Issue Type: Bug > Components: runner-direct >Reporter: Luke Cwik >Assignee: Luke Cwik >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > beam-model-pipeline and others are duplicated in the dependency list -- This message was sent by Atlassian JIRA (v7.6.3#76005)
