[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=541981&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-541981
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 26/Jan/21 04:43
Start Date: 26/Jan/21 04:43
Worklog Time Spent: 10m
Work Description: kennknowles merged pull request #13732:
URL: https://github.com/apache/beam/pull/13732
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 541981)
Time Spent: 20.5h (was: 20h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Kiley Sok
>Priority: P3
> Fix For: 2.26.0
>
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 20.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=541367&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-541367
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 25/Jan/21 23:35
Start Date: 25/Jan/21 23:35
Worklog Time Spent: 10m
Work Description: kennknowles merged pull request #13732:
URL: https://github.com/apache/beam/pull/13732
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 541367)
Time Spent: 20h 20m (was: 20h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Kiley Sok
>Priority: P3
> Fix For: 2.26.0
>
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 20h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=534865&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534865
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Jan/21 14:11
Start Date: 12/Jan/21 14:11
Worklog Time Spent: 10m
Work Description: codecov[bot] edited a comment on pull request #13732:
URL: https://github.com/apache/beam/pull/13732#issuecomment-758664639
# [Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=h1) Report
> Merging
[#13732](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=desc) (2a4caf6)
into
[master](https://codecov.io/gh/apache/beam/commit/6c9da02d553cc0d39d29b0293c9b37ac0ceec73e?el=desc)
(6c9da02) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=tree)
```diff
@@Coverage Diff @@
## master #13732 +/- ##
==
- Coverage 82.75% 82.74% -0.01%
==
Files 466 466
Lines 5752057520
==
- Hits4759847597 -1
- Misses 9922 9923 +1
```
| [Impacted
Files](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=tree) | Coverage
Δ | |
|---|---|---|
|
[sdks/python/apache\_beam/runners/direct/executor.py](https://codecov.io/gh/apache/beam/pull/13732/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZXhlY3V0b3IucHk=)
| `96.29% <0.00%> (-0.53%)` | :arrow_down: |
|
[...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13732/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==)
| `93.83% <0.00%> (+0.12%)` | :arrow_up: |
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=footer). Last
update
[6c9da02...2a4caf6](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 534865)
Time Spent: 20h 10m (was: 20h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Kiley Sok
>Priority: P3
> Fix For: 2.26.0
>
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 20h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=534857&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534857
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Jan/21 14:02
Start Date: 12/Jan/21 14:02
Worklog Time Spent: 10m
Work Description: codecov[bot] edited a comment on pull request #13732:
URL: https://github.com/apache/beam/pull/13732#issuecomment-758664639
# [Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=h1) Report
> Merging
[#13732](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=desc) (2a4caf6)
into
[master](https://codecov.io/gh/apache/beam/commit/6c9da02d553cc0d39d29b0293c9b37ac0ceec73e?el=desc)
(6c9da02) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=tree)
```diff
@@Coverage Diff @@
## master #13732 +/- ##
==
- Coverage 82.75% 82.74% -0.01%
==
Files 466 466
Lines 5752057520
==
- Hits4759847597 -1
- Misses 9922 9923 +1
```
| [Impacted
Files](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=tree) | Coverage
Δ | |
|---|---|---|
|
[sdks/python/apache\_beam/runners/direct/executor.py](https://codecov.io/gh/apache/beam/pull/13732/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZXhlY3V0b3IucHk=)
| `96.29% <0.00%> (-0.53%)` | :arrow_down: |
|
[...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13732/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==)
| `93.83% <0.00%> (+0.12%)` | :arrow_up: |
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=footer). Last
update
[6c9da02...2a4caf6](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 534857)
Time Spent: 20h (was: 19h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Kiley Sok
>Priority: P3
> Fix For: 2.26.0
>
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 20h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=534848&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534848
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Jan/21 13:50
Start Date: 12/Jan/21 13:50
Worklog Time Spent: 10m
Work Description: codecov[bot] edited a comment on pull request #13732:
URL: https://github.com/apache/beam/pull/13732#issuecomment-758664639
# [Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=h1) Report
> Merging
[#13732](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=desc) (2a4caf6)
into
[master](https://codecov.io/gh/apache/beam/commit/6c9da02d553cc0d39d29b0293c9b37ac0ceec73e?el=desc)
(6c9da02) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=tree)
```diff
@@Coverage Diff @@
## master #13732 +/- ##
==
- Coverage 82.75% 82.74% -0.01%
==
Files 466 466
Lines 5752057520
==
- Hits4759847597 -1
- Misses 9922 9923 +1
```
| [Impacted
Files](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=tree) | Coverage
Δ | |
|---|---|---|
|
[sdks/python/apache\_beam/runners/direct/executor.py](https://codecov.io/gh/apache/beam/pull/13732/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZXhlY3V0b3IucHk=)
| `96.29% <0.00%> (-0.53%)` | :arrow_down: |
|
[...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13732/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==)
| `93.83% <0.00%> (+0.12%)` | :arrow_up: |
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=footer). Last
update
[6c9da02...2a4caf6](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 534848)
Time Spent: 19h 50m (was: 19h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Kiley Sok
>Priority: P3
> Fix For: 2.26.0
>
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 19h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependen
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=534846&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534846
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Jan/21 13:46
Start Date: 12/Jan/21 13:46
Worklog Time Spent: 10m
Work Description: codecov[bot] commented on pull request #13732:
URL: https://github.com/apache/beam/pull/13732#issuecomment-758664639
# [Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=h1) Report
> Merging
[#13732](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=desc) (2a4caf6)
into
[master](https://codecov.io/gh/apache/beam/commit/6c9da02d553cc0d39d29b0293c9b37ac0ceec73e?el=desc)
(6c9da02) will **decrease** coverage by `0.00%`.
> The diff coverage is `n/a`.
[](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=tree)
```diff
@@Coverage Diff @@
## master #13732 +/- ##
==
- Coverage 82.75% 82.74% -0.01%
==
Files 466 466
Lines 5752057520
==
- Hits4759847597 -1
- Misses 9922 9923 +1
```
| [Impacted
Files](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=tree) | Coverage
Δ | |
|---|---|---|
|
[sdks/python/apache\_beam/runners/direct/executor.py](https://codecov.io/gh/apache/beam/pull/13732/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvZXhlY3V0b3IucHk=)
| `96.29% <0.00%> (-0.53%)` | :arrow_down: |
|
[...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13732/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==)
| `93.83% <0.00%> (+0.12%)` | :arrow_up: |
--
[Continue to review full report at
Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=continue).
> **Legend** - [Click here to learn
more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute (impact)`, `ø = not affected`, `? = missing data`
> Powered by
[Codecov](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=footer). Last
update
[6c9da02...2a4caf6](https://codecov.io/gh/apache/beam/pull/13732?src=pr&el=lastupdated).
Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 534846)
Time Spent: 19h 40m (was: 19.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Kiley Sok
>Priority: P3
> Fix For: 2.26.0
>
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 19h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[ https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=534836&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534836 ] ASF GitHub Bot logged work on BEAM-9444: Author: ASF GitHub Bot Created on: 12/Jan/21 13:30 Start Date: 12/Jan/21 13:30 Worklog Time Spent: 10m Work Description: elharo opened a new pull request #13732: URL: https://github.com/apache/beam/pull/13732 @lukecwik Post-Commit Tests Status (on master branch) Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2 --- | --- | --- | --- | --- | --- | --- Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | --- Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/lastCompletedBuild/) Python | [](https://ci-beam.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Python37/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Python38/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastComplete
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=512672&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-512672
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 17/Nov/20 00:00
Start Date: 17/Nov/20 00:00
Worklog Time Spent: 10m
Work Description: kileys closed pull request #13289:
URL: https://github.com/apache/beam/pull/13289
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 512672)
Time Spent: 19h 20m (was: 19h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 19h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=512637&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-512637
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 16/Nov/20 22:44
Start Date: 16/Nov/20 22:44
Worklog Time Spent: 10m
Work Description: kennknowles commented on pull request #13289:
URL: https://github.com/apache/beam/pull/13289#issuecomment-728374842
We chatted about that. I think the risk of forgetting to add the BOM to a
module is less and the risk of messing up the deps of a module is higher.
Eventually we will want dependency convergence but without some big picture
testing and analysis strategy I would say let's skip this change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 512637)
Time Spent: 19h 10m (was: 19h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 19h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=509474&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-509474
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 10/Nov/20 03:03
Start Date: 10/Nov/20 03:03
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13289:
URL: https://github.com/apache/beam/pull/13289#issuecomment-724418928
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 509474)
Time Spent: 19h (was: 18h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 19h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[ https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=509420&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-509420 ] ASF GitHub Bot logged work on BEAM-9444: Author: ASF GitHub Bot Created on: 09/Nov/20 23:47 Start Date: 09/Nov/20 23:47 Worklog Time Spent: 10m Work Description: kileys opened a new pull request #13289: URL: https://github.com/apache/beam/pull/13289 Include GCP libraries BOM to control versions for Java projects Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2 --- | --- | --- | --- | --- | --- | --- Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | --- Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Twister2/la
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=507037&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-507037
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 03/Nov/20 14:14
Start Date: 03/Nov/20 14:14
Worklog Time Spent: 10m
Work Description: kennknowles merged pull request #13075:
URL: https://github.com/apache/beam/pull/13075
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 507037)
Time Spent: 18h 40m (was: 18.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 18h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=506784&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-506784
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 03/Nov/20 13:45
Start Date: 03/Nov/20 13:45
Worklog Time Spent: 10m
Work Description: kennknowles commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-720711343
It does seem that if a module had any of these deps but forgot the
`enforcedPlatform` line then it would go back to getting the arbitrary latest
version. I understand fixing that is in a follow-up.
Here's what I did to view the dep changes:
```
# gcp is GNU cp here
mkdir ~/tmp/deps-before
git checkout github/master
./gradlew dependencyReport
find . -name dependencies.txt -exec gcp --parents \{\} ~/tmp/deps-before/ \;
mkdir ~/tmp/deps-after
git checkout github/pr/13075
./gradlew dependencyReport
find . -name dependencies.txt -exec gcp --parents \{\} ~/tmp/deps-after/ \;
diff --brief --recursive ~/tmp/deps-before ~/tmp/deps-after
diff --unified=0 --recursive ~/tmp/deps-before ~/tmp/deps-after
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 506784)
Time Spent: 18.5h (was: 18h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 18.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
>
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=503443&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-503443
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 21/Oct/20 23:30
Start Date: 21/Oct/20 23:30
Worklog Time Spent: 10m
Work Description: kileys commented on a change in pull request #13075:
URL: https://github.com/apache/beam/pull/13075#discussion_r509793166
##
File path: examples/java/build.gradle
##
@@ -48,6 +48,7 @@ configurations.sparkRunnerPreCommit {
}
dependencies {
+ compile platform(library.java.google_cloud_platform_libraries_bom)
Review comment:
Updated to enforcedPlatform and going to update the rest of the
dependencies in a follow up
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 503443)
Time Spent: 18h 20m (was: 18h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 18h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=502421&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502421
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 19/Oct/20 22:07
Start Date: 19/Oct/20 22:07
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-712468763
Also, it seems as though existing modules that don't add a platform level
dep to the GCP libraries bom will get some version of the dep since we have
cleared the version.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502421)
Time Spent: 18h 10m (was: 18h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 18h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=502419&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-502419
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 19/Oct/20 22:05
Start Date: 19/Oct/20 22:05
Worklog Time Spent: 10m
Work Description: lukecwik commented on a change in pull request #13075:
URL: https://github.com/apache/beam/pull/13075#discussion_r508082875
##
File path: examples/java/build.gradle
##
@@ -48,6 +48,7 @@ configurations.sparkRunnerPreCommit {
}
dependencies {
+ compile platform(library.java.google_cloud_platform_libraries_bom)
Review comment:
Should we be using enforcedPlatform instead of platform everywhere?
Before this change, we get enforcedPlatform like behavior because in
BeamModulePlugin.groovy we are explicitly forcing version overrides.
Migrating to enforcedPlatform would force the remainder of versions within
the BOM to be used though which you haven't done as part of this PR (e.g. guava
being 29.0-android).
I don't see any gradle documentation that explains how an enforcedPlatform
and a `force` `resolutionStrategy` behave together.
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -21,6 +21,8 @@ package org.apache.beam.gradle
import com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar
import groovy.json.JsonOutput
import groovy.json.JsonSlurper
+import java.util.concurrent.atomic.AtomicInteger
Review comment:
I don't think this is necessary as well.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 502419)
Time Spent: 18h (was: 17h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 18h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies s
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=501629&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501629
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 16/Oct/20 17:04
Start Date: 16/Oct/20 17:04
Worklog Time Spent: 10m
Work Description: suztomo commented on a change in pull request #13075:
URL: https://github.com/apache/beam/pull/13075#discussion_r506605232
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -21,6 +21,8 @@ package org.apache.beam.gradle
import com.github.jengelman.gradle.plugins.shadow.tasks.ShadowJar
import groovy.json.JsonOutput
import groovy.json.JsonSlurper
+import java.util.concurrent.atomic.AtomicInteger
Review comment:
I didn't expect to use AtomicInteger. Do you really need this?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 501629)
Time Spent: 17h 50m (was: 17h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 17h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=501300&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501300
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 15/Oct/20 21:09
Start Date: 15/Oct/20 21:09
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-709591550
The Dataflow validates runner tests are also failing on master
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 501300)
Time Spent: 17h 40m (was: 17.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 17h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=501297&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501297
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 15/Oct/20 21:08
Start Date: 15/Oct/20 21:08
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-709591028
R: @kennknowles, @suztomo, @lukecwik
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 501297)
Time Spent: 17.5h (was: 17h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 17.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=501295&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501295
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 15/Oct/20 21:04
Start Date: 15/Oct/20 21:04
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-709589412
@suztomo are there other tests I should run to verify this? Linkage checker
runs without any new errors
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 501295)
Time Spent: 17h 20m (was: 17h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 17h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=501215&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501215
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 15/Oct/20 18:08
Start Date: 15/Oct/20 18:08
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-709499714
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 501215)
Time Spent: 17h 10m (was: 17h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 17h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=501213&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-501213
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 15/Oct/20 18:07
Start Date: 15/Oct/20 18:07
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-709499008
Run Spark ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 501213)
Time Spent: 17h (was: 16h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 17h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=500317&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-500317
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 13/Oct/20 22:24
Start Date: 13/Oct/20 22:24
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-708042134
Run Python_PVR_Flink PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 500317)
Time Spent: 16h 50m (was: 16h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 16h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=499744&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499744
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 13/Oct/20 00:01
Start Date: 13/Oct/20 00:01
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-707405231
Run Python PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 499744)
Time Spent: 16h 40m (was: 16.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 16h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=499733&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499733
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Oct/20 23:39
Start Date: 12/Oct/20 23:39
Worklog Time Spent: 10m
Work Description: kennknowles commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-707398139
I don't see any reason for that strange error in spotless.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 499733)
Time Spent: 16.5h (was: 16h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 16.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=499732&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499732
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Oct/20 23:38
Start Date: 12/Oct/20 23:38
Worklog Time Spent: 10m
Work Description: kennknowles commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-707398052
Run Spotless PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 499732)
Time Spent: 16h 20m (was: 16h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 16h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=499581&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499581
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Oct/20 18:48
Start Date: 12/Oct/20 18:48
Worklog Time Spent: 10m
Work Description: github-actions[bot] commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-707287334
[The Build Workflow
run](https://github.com/apache/beam/actions/runs/302756925) is cancelling this
PR. It in earlier duplicate of 2173354 run.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 499581)
Time Spent: 16h 10m (was: 16h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 16h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=499580&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499580
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Oct/20 18:46
Start Date: 12/Oct/20 18:46
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-707286492
Run Java PostCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 499580)
Time Spent: 16h (was: 15h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 16h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=499578&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499578
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Oct/20 18:45
Start Date: 12/Oct/20 18:45
Worklog Time Spent: 10m
Work Description: kileys closed pull request #13075:
URL: https://github.com/apache/beam/pull/13075
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 499578)
Time Spent: 15h 40m (was: 15.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 15h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=499577&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499577
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Oct/20 18:45
Start Date: 12/Oct/20 18:45
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-707285931
Run Dataflow ValidatesRunner Java 11
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 499577)
Time Spent: 15.5h (was: 15h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 15.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=499579&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499579
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Oct/20 18:45
Start Date: 12/Oct/20 18:45
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-707286120
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 499579)
Time Spent: 15h 50m (was: 15h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 15h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=499576&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499576
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 12/Oct/20 18:44
Start Date: 12/Oct/20 18:44
Worklog Time Spent: 10m
Work Description: kileys commented on pull request #13075:
URL: https://github.com/apache/beam/pull/13075#issuecomment-707285770
Run Java examples on Dataflow Java 11
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 499576)
Time Spent: 15h 20m (was: 15h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 15h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[ https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=499575&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-499575 ] ASF GitHub Bot logged work on BEAM-9444: Author: ASF GitHub Bot Created on: 12/Oct/20 18:41 Start Date: 12/Oct/20 18:41 Worklog Time Spent: 10m Work Description: kileys opened a new pull request #13075: URL: https://github.com/apache/beam/pull/13075 Use GCP libraries bom to set versions for GCP related libraries Updated version of this PR: https://github.com/apache/beam/pull/11156 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2 --- | --- | --- | --- | --- | --- | --- Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | --- Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [](https:
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=498095&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-498095
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 09/Oct/20 13:39
Start Date: 09/Oct/20 13:39
Worklog Time Spent: 10m
Work Description: suztomo commented on a change in pull request #11156:
URL: https://github.com/apache/beam/pull/11156#discussion_r502063556
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -1210,6 +1222,18 @@ class BeamModulePlugin implements Plugin {
configuration: (configuration.shadowClosure ? 'shadow'
: 'compile'), scope: 'compile')
generateDependenciesFromConfiguration(configuration:
'provided', scope: 'provided')
+if (!boms.isEmpty()) {
Review comment:
When this Beam module uses a BOM, we need to set dependencyManagent
element
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 498095)
Time Spent: 15h (was: 14h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 15h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent b
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=497647&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-497647
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 08/Oct/20 23:06
Start Date: 08/Oct/20 23:06
Worklog Time Spent: 10m
Work Description: suztomo commented on a change in pull request #11156:
URL: https://github.com/apache/beam/pull/11156#discussion_r502063556
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -1210,6 +1222,18 @@ class BeamModulePlugin implements Plugin {
configuration: (configuration.shadowClosure ? 'shadow'
: 'compile'), scope: 'compile')
generateDependenciesFromConfiguration(configuration:
'provided', scope: 'provided')
+if (!boms.isEmpty()) {
Review comment:
When this Beam module uses a BOM, we need to set dependencyManagent
element
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 497647)
Time Spent: 14h 50m (was: 14h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Priority: P3
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 14h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message wa
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=441656&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441656
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 05/Jun/20 03:14
Start Date: 05/Jun/20 03:14
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11586:
URL: https://github.com/apache/beam/pull/11586#issuecomment-639232697
This idea was a bad idea; a hack on top of another hack.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441656)
Time Spent: 14.5h (was: 14h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: P2
> Labels: stale-assigned
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 14.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=441657&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-441657
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 05/Jun/20 03:14
Start Date: 05/Jun/20 03:14
Worklog Time Spent: 10m
Work Description: suztomo closed pull request #11586:
URL: https://github.com/apache/beam/pull/11586
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 441657)
Time Spent: 14h 40m (was: 14.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: P2
> Labels: stale-assigned
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 14h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=429537&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429537
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 01/May/20 16:02
Start Date: 01/May/20 16:02
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11586:
URL: https://github.com/apache/beam/pull/11586#issuecomment-622447857
@iemejia Thank you.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429537)
Time Spent: 14h 20m (was: 14h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 14h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=429487&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429487
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 01/May/20 13:55
Start Date: 01/May/20 13:55
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11586:
URL: https://github.com/apache/beam/pull/11586#issuecomment-622398262
Java precommit check failed on
org.apache.beam.runners.flink.FlinkSavepointTest.testSavepointRestorePortable
“test timed out after 60 seconds“. This seems Transient failure.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429487)
Time Spent: 14h 10m (was: 14h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 14h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=429454&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429454
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 01/May/20 12:10
Start Date: 01/May/20 12:10
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11586:
URL: https://github.com/apache/beam/pull/11586#issuecomment-622362982
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429454)
Time Spent: 14h (was: 13h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 14h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=429452&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429452
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 01/May/20 12:08
Start Date: 01/May/20 12:08
Worklog Time Spent: 10m
Work Description: iemejia commented on pull request #11586:
URL: https://github.com/apache/beam/pull/11586#issuecomment-622362502
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429452)
Time Spent: 13h 50m (was: 13h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 13h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[ https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=429349&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429349 ] ASF GitHub Bot logged work on BEAM-9444: Author: ASF GitHub Bot Created on: 01/May/20 03:28 Start Date: 01/May/20 03:28 Worklog Time Spent: 10m Work Description: suztomo opened a new pull request #11586: URL: https://github.com/apache/beam/pull/11586 Using GCP Libraries BOM to set GCP related libraries. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[ Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=427879&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-427879
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 27/Apr/20 22:05
Start Date: 27/Apr/20 22:05
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11156:
URL: https://github.com/apache/beam/pull/11156#issuecomment-620260310
Gradle's updateOfflineRepository task (via
https://github.com/mdietrichstein/gradle-offline-dependencies-plugin) does not
work with Maven BOMs. The plugin is marked as not maintained. Closing this for
now.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 427879)
Time Spent: 13.5h (was: 13h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 13.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=423891&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-423891
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 17/Apr/20 03:11
Start Date: 17/Apr/20 03:11
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11391: [BEAM-9444]
Rebasing old version of PR 11156 (no need to review)
URL: https://github.com/apache/beam/pull/11391
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 423891)
Time Spent: 13h 20m (was: 13h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 13h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[ https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=420529&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-420529 ] ASF GitHub Bot logged work on BEAM-9444: Author: ASF GitHub Bot Created on: 10/Apr/20 21:36 Start Date: 10/Apr/20 21:36 Worklog Time Spent: 10m Work Description: suztomo commented on pull request #11391: [BEAM-9444] Rebasing old version of PR 11156 (no need to review) URL: https://github.com/apache/beam/pull/11391 ``` suztomo-macbookpro44% git log -3 commit b47744e6c69f5646a2c91026952aff65decc20b2 (HEAD -> BEAM-9444-rebase-c00df443, suztomo/BEAM-9444-new, BEAM-9444-new) Author: Tomo Suzuki Date: Fri Apr 10 17:03:21 2020 -0400 Applying difference commit c17e7c9614fe181f33c93143ac760c0b7696622e (origin/master, origin/HEAD) Merge: d5929ba273 81092b7134 Author: Pablo Estrada Date: Fri Apr 10 12:45:29 2020 -0700 Merge pull request #11383 from [BEAM-9736] Fix from_container_image call [BEAM-9736] Fix from_container_image call commit d5929ba2738b38e82f91038a541089a303161e34 Merge: 75433f0322 c26af84a24 Author: Robert Bradshaw Date: Fri Apr 10 12:44:38 2020 -0700 Merge pull request #11311 [BEAM-9577] Attempt to stage resources via new API in portable runner. suztomo-macbookpro44% git rebase c00df443f68efc643779fe13821c2cfdfea1f100 First, rewinding head to replay your work on top of it... Applying: Update go protos. Applying: [BEAM-9618] Pull bundle descriptors for Go. Applying: [BEAM-9577] Plumb resources through Python job service and runner. Applying: [BEAM-9529] Remove datastore.v1, googledatastore (#11175) Applying: [BEAM-5422] Document DynamicDestinations.getTable uniqueness requirement Applying: [BEAM-9147] Add a VideoIntelligence transform to Java SDK (#11261) Applying: [BEAM-9580] Allow Flink 1.10 processing timers to finish on pipeline shutdown Applying: Revert "[BEAM-9580] Downgrade Flink version to 1.9 for Nexmark and ValidatesRunner" Applying: [BEAM-9557] Fix strings used to verify test output Applying: [BEAM-9714] [Go SDK] Require --region flag in Dataflow runner. Applying: Update session.go Applying: Update stage.go Applying: Update server_test.go Applying: Update materialize.go Applying: Update materialize_test.go Applying: Update stage_test.go Applying: Update artifact.go Applying: Update provision.go Applying: Update retrieval.go Applying: Update staging.go Applying: Update translate.go Applying: Update datamgr.go Applying: Update datamgr_test.go Applying: Update logging.go Applying: Update logging_test.go Applying: Update monitoring.go Applying: Update session.go Applying: Update statemgr.go Applying: Update statemgr_test.go Applying: Update replace.go Applying: Update replace_test.go Applying: Update provision.go Applying: Update execute.go Applying: Update job.go Applying: Update translate.go Applying: Update translate.go Applying: Update job.go Applying: Update materialize.go Applying: Update translate.go Applying: Update session.go Applying: Update materialize_test.go Applying: [BEAM-9596] Ensure metrics are available in PipelineResult when the pipeline fails Applying: Updates documentation for WINDOWED_VALUE coder. Applying: Uses iterable coder for windows and copies all of timestamp encoding specification. Applying: Update session.go (#11352) Applying: [BEAM-9322] [BEAM-1833] Better naming for composite transform output tags. Applying: [BEAM-9618] Java SDK worker support for pulling bundle descriptors. (#11328) Applying: [BEAM-9716] Alias zone to worker_zone and warn user. Applying: [Beam-9063]update documentation (#10952) Applying: Adds nose back under packages needed for testing. Applying: [BEAM-9618] Mark push registration as deprecated. Applying: Ensure that empty messages are not flushed to handler. Applying: [BEAM-9721] Add --region to Dataflow-based load tests Applying: [BEAM-9721] LoadTestConfig: handle --region parameter and put default value if absent Applying: [BEAM-9550] Increase JVM Metaspace size for the TaskExecutors. (#11193) Applying: [BEAM-9721]Conditionally add Dataflow region to Dataflow-based performance tests (#11344) Applying: [BEAM-9647] fixes MQTT clientId to long Applying: [BEAM-8280] Document Python 3 annotations support (#11232) Applying: [BEAM-9731] Include more detail in passert.Equals errors. (#11359) Applying: [BEAM-9085] Fix performance regression in SyntheticSource on Python 3 (#11092) Applying: [BEAM-9726] [py] Make region optional for non-service Dataflow. Applying: [BEAM-9726] [java] Make region optional for non-service runner. Applying: [BEAM-4374] Fix missing deletion of metrics.
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[ https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=420503&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-420503 ] ASF GitHub Bot logged work on BEAM-9444: Author: ASF GitHub Bot Created on: 10/Apr/20 21:05 Start Date: 10/Apr/20 21:05 Worklog Time Spent: 10m Work Description: suztomo commented on pull request #11388: [BEAM-9444] Clean version of PR 11156 (no need to review) URL: https://github.com/apache/beam/pull/11388 Checking pull request validation for #11156 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStruc
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=420492&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-420492
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 10/Apr/20 20:59
Start Date: 10/Apr/20 20:59
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11387: [BEAM-9444]
rebased version of PR 11156
URL: https://github.com/apache/beam/pull/11387
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 420492)
Time Spent: 12h 50m (was: 12h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 12h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[ https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=420489&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-420489 ] ASF GitHub Bot logged work on BEAM-9444: Author: ASF GitHub Bot Created on: 10/Apr/20 20:58 Start Date: 10/Apr/20 20:58 Worklog Time Spent: 10m Work Description: suztomo commented on pull request #11387: [BEAM-9444] rebased version of PR 11156 URL: https://github.com/apache/beam/pull/11387 Rebased version of https://github.com/apache/beam/pull/11156. Experimenting pull request validation. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/b
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=415303&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415303
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 03/Apr/20 03:27
Start Date: 03/Apr/20 03:27
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402715747
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
-force project.library.java.values()
+// Filtering versionless coordinates that depend on BOM
+def librariesWithVersion = project.library.java.values().findAll {
it.split(':').size() > 2 }
+force librariesWithVersion
Review comment:
Added comment with ticket ID BEAM-9542.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 415303)
Time Spent: 12h 20m (was: 12h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 12h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
>
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=415304&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415304
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 03/Apr/20 03:27
Start Date: 03/Apr/20 03:27
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402716462
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin {
commons_lang3 :
"org.apache.commons:commons-lang3:3.9",
commons_math3 :
"org.apache.commons:commons-math3:3.6.1",
error_prone_annotations :
"com.google.errorprone:error_prone_annotations:2.0.15",
-gax :
"com.google.api:gax:$gax_version",
-gax_grpc:
"com.google.api:gax-grpc:$gax_version",
+gax : "com.google.api:gax",
+gax_grpc:
"com.google.api:gax-grpc",
google_api_client :
"com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 :
"com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 :
"com.google.api-client:google-api-client-java6:$google_clients_version",
-google_api_common :
"com.google.api:api-common:1.8.1",
+google_api_common :
"com.google.api:api-common",
google_api_services_bigquery:
"com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger :
"com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager:
"com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow:
"com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub :
"com.google.apis:google-api-services-pubsub:v1-rev2019-$google_clients_version",
google_api_services_storage :
"com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
-google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials:$google_auth_version",
-google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
-google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery:1.108.0",
-google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials",
+google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http",
+google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery",
+google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core :
"com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
Yes, Linkage Checker detects the discrepancy if there's any.
> Note that our existing integration tests are not that useful here since
they mostly check one GCP service at a time while users may use a combination.
This change is not intended to fix compatibility with users' libraries. It's
about how Beam maintains the versions of these Google libraries with ease.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 415304)
Time Spent: 12.5h (was: 12h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Projec
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=415198&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415198
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 03/Apr/20 00:14
Start Date: 03/Apr/20 00:14
Worklog Time Spent: 10m
Work Description: chamikaramj commented on pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402666322
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin {
commons_lang3 :
"org.apache.commons:commons-lang3:3.9",
commons_math3 :
"org.apache.commons:commons-math3:3.6.1",
error_prone_annotations :
"com.google.errorprone:error_prone_annotations:2.0.15",
-gax :
"com.google.api:gax:$gax_version",
-gax_grpc:
"com.google.api:gax-grpc:$gax_version",
+gax : "com.google.api:gax",
+gax_grpc:
"com.google.api:gax-grpc",
google_api_client :
"com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 :
"com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 :
"com.google.api-client:google-api-client-java6:$google_clients_version",
-google_api_common :
"com.google.api:api-common:1.8.1",
+google_api_common :
"com.google.api:api-common",
google_api_services_bigquery:
"com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger :
"com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager:
"com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow:
"com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub :
"com.google.apis:google-api-services-pubsub:v1-rev2019-$google_clients_version",
google_api_services_storage :
"com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
-google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials:$google_auth_version",
-google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
-google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery:1.108.0",
-google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials",
+google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http",
+google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery",
+google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core :
"com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
GCP libraries you already commented about that are not included in the BOM.
For example " google_cloud_bigtable_client_core". I guess it should be fine if
linkage checker passes. Note that our existing integration tests are not that
useful here since they mostly check one GCP service at a time while users may
use a combination.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 415198)
Time Spent: 12h 10m (was: 12h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
>
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=415194&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415194
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 03/Apr/20 00:11
Start Date: 03/Apr/20 00:11
Worklog Time Spent: 10m
Work Description: chamikaramj commented on pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402666374
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
-force project.library.java.values()
+// Filtering versionless coordinates that depend on BOM
+def librariesWithVersion = project.library.java.values().findAll {
it.split(':').size() > 2 }
+force librariesWithVersion
Review comment:
Yes, please.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 415194)
Time Spent: 12h (was: 11h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 12h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=415195&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415195
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 03/Apr/20 00:11
Start Date: 03/Apr/20 00:11
Worklog Time Spent: 10m
Work Description: chamikaramj commented on pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402666322
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin {
commons_lang3 :
"org.apache.commons:commons-lang3:3.9",
commons_math3 :
"org.apache.commons:commons-math3:3.6.1",
error_prone_annotations :
"com.google.errorprone:error_prone_annotations:2.0.15",
-gax :
"com.google.api:gax:$gax_version",
-gax_grpc:
"com.google.api:gax-grpc:$gax_version",
+gax : "com.google.api:gax",
+gax_grpc:
"com.google.api:gax-grpc",
google_api_client :
"com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 :
"com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 :
"com.google.api-client:google-api-client-java6:$google_clients_version",
-google_api_common :
"com.google.api:api-common:1.8.1",
+google_api_common :
"com.google.api:api-common",
google_api_services_bigquery:
"com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger :
"com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager:
"com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow:
"com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub :
"com.google.apis:google-api-services-pubsub:v1-rev2019-$google_clients_version",
google_api_services_storage :
"com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
-google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials:$google_auth_version",
-google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
-google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery:1.108.0",
-google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials",
+google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http",
+google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery",
+google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core :
"com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
GCP libraries you already commented about that are not included in the BOM.
For example " google_cloud_bigtable_client_core". I guess it should be fine if
linkage checker passes. Note that our existing integration tests are not that
useful here since they mostly check one GCS service at a time while users may
use a combination.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 415195)
Time Spent: 12h (was: 11h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
>
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=415152&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415152
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 02/Apr/20 23:58
Start Date: 02/Apr/20 23:58
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402662447
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
-force project.library.java.values()
+// Filtering versionless coordinates that depend on BOM
+def librariesWithVersion = project.library.java.values().findAll {
it.split(':').size() > 2 }
+force librariesWithVersion
Review comment:
Do you mean comment?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 415152)
Time Spent: 11h 50m (was: 11h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 11h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {n
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=415151&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415151
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 02/Apr/20 23:58
Start Date: 02/Apr/20 23:58
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402662384
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin {
commons_lang3 :
"org.apache.commons:commons-lang3:3.9",
commons_math3 :
"org.apache.commons:commons-math3:3.6.1",
error_prone_annotations :
"com.google.errorprone:error_prone_annotations:2.0.15",
-gax :
"com.google.api:gax:$gax_version",
-gax_grpc:
"com.google.api:gax-grpc:$gax_version",
+gax : "com.google.api:gax",
+gax_grpc:
"com.google.api:gax-grpc",
google_api_client :
"com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 :
"com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 :
"com.google.api-client:google-api-client-java6:$google_clients_version",
-google_api_common :
"com.google.api:api-common:1.8.1",
+google_api_common :
"com.google.api:api-common",
google_api_services_bigquery:
"com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger :
"com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager:
"com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow:
"com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub :
"com.google.apis:google-api-services-pubsub:v1-rev2019-$google_clients_version",
google_api_services_storage :
"com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
-google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials:$google_auth_version",
-google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
-google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery:1.108.0",
-google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials",
+google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http",
+google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery",
+google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core :
"com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
Which library are you referring to?
There’s no additional conflicts. Linkage Checker verified that.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 415151)
Time Spent: 11h 40m (was: 11.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=415147&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415147
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 02/Apr/20 23:51
Start Date: 02/Apr/20 23:51
Worklog Time Spent: 10m
Work Description: chamikaramj commented on pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402660255
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
-force project.library.java.values()
+// Filtering versionless coordinates that depend on BOM
+def librariesWithVersion = project.library.java.values().findAll {
it.split(':').size() > 2 }
+force librariesWithVersion
Review comment:
Please force findings here just for the record.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 415147)
Time Spent: 11.5h (was: 11h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 11.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> comp
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=415148&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415148
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 02/Apr/20 23:51
Start Date: 02/Apr/20 23:51
Worklog Time Spent: 10m
Work Description: chamikaramj commented on pull request #11156:
[BEAM-9444] Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r402658146
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin {
commons_lang3 :
"org.apache.commons:commons-lang3:3.9",
commons_math3 :
"org.apache.commons:commons-math3:3.6.1",
error_prone_annotations :
"com.google.errorprone:error_prone_annotations:2.0.15",
-gax :
"com.google.api:gax:$gax_version",
-gax_grpc:
"com.google.api:gax-grpc:$gax_version",
+gax : "com.google.api:gax",
+gax_grpc:
"com.google.api:gax-grpc",
google_api_client :
"com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 :
"com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 :
"com.google.api-client:google-api-client-java6:$google_clients_version",
-google_api_common :
"com.google.api:api-common:1.8.1",
+google_api_common :
"com.google.api:api-common",
google_api_services_bigquery:
"com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger :
"com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager:
"com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow:
"com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub :
"com.google.apis:google-api-services-pubsub:v1-rev2019-$google_clients_version",
google_api_services_storage :
"com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
-google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials:$google_auth_version",
-google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
-google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery:1.108.0",
-google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials",
+google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http",
+google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery",
+google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core :
"com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
Have we confirmed that GCS artifacts not included in this BOM do not
conflict with artifacts included in the BOM ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 415148)
Time Spent: 11.5h (was: 11h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=412489&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-412489
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 30/Mar/20 18:15
Start Date: 30/Mar/20 18:15
Worklog Time Spent: 10m
Work Description: suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606154369
`:runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest`
failed:
https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_PR/252/
```
11:12:20 > Task
:runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest
11:13:55
11:13:55 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests >
testTimerWithMultipleTimerFamilyUnbounded FAILED
11:13:55 java.lang.RuntimeException at ParDoTest.java:4770
11:13:55 Caused by: java.lang.IllegalArgumentException at
ParDoTest.java:4770
11:29:11
11:29:11 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests >
testTimerFamilyEventTimeUnbounded FAILED
11:29:11 java.lang.RuntimeException at ParDoTest.java:4708
11:29:11 Caused by: java.lang.IllegalArgumentException at
ParDoTest.java:4708
```
It seems other post commit jobs are also failing:
https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/4738/console
```
08:01:43 > Task
:runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest
08:03:14
08:03:14 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests >
testTimerWithMultipleTimerFamilyUnbounded FAILED
08:03:14 java.lang.RuntimeException at ParDoTest.java:4770
08:03:14 Caused by: java.lang.IllegalArgumentException at
ParDoTest.java:4770
08:16:38
08:16:38 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests >
testTimerFamilyEventTimeUnbounded FAILED
08:16:38 java.lang.RuntimeException at ParDoTest.java:4708
08:16:38 Caused by: java.lang.IllegalArgumentException at
ParDoTest.java:4708
08:33:31
08:33:31 org.apache.beam.sdk.transforms.FlattenTest >
testFlattenPCollectionsThenParDo FAILED
08:33:31 java.lang.RuntimeException at FlattenTest.java:118
```
Checking which PR started this failure. -> Found
https://github.com/apache/beam/pull/11216#issuecomment-606158291
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 412489)
Time Spent: 11h 20m (was: 11h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 11h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=412483&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-412483
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 30/Mar/20 18:05
Start Date: 30/Mar/20 18:05
Worklog Time Spent: 10m
Work Description: suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606154369
`:runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest`
failed:
https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_PR/252/
```
11:12:20 > Task
:runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest
11:13:55
11:13:55 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests >
testTimerWithMultipleTimerFamilyUnbounded FAILED
11:13:55 java.lang.RuntimeException at ParDoTest.java:4770
11:13:55 Caused by: java.lang.IllegalArgumentException at
ParDoTest.java:4770
11:29:11
11:29:11 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests >
testTimerFamilyEventTimeUnbounded FAILED
11:29:11 java.lang.RuntimeException at ParDoTest.java:4708
11:29:11 Caused by: java.lang.IllegalArgumentException at
ParDoTest.java:4708
```
It seems other post commit jobs are also failing:
https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/4738/console
```
08:01:43 > Task
:runners:google-cloud-dataflow-java:validatesRunnerLegacyWorkerTest
08:03:14
08:03:14 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests >
testTimerWithMultipleTimerFamilyUnbounded FAILED
08:03:14 java.lang.RuntimeException at ParDoTest.java:4770
08:03:14 Caused by: java.lang.IllegalArgumentException at
ParDoTest.java:4770
08:16:38
08:16:38 org.apache.beam.sdk.transforms.ParDoTest$TimerFamilyTests >
testTimerFamilyEventTimeUnbounded FAILED
08:16:38 java.lang.RuntimeException at ParDoTest.java:4708
08:16:38 Caused by: java.lang.IllegalArgumentException at
ParDoTest.java:4708
08:33:31
08:33:31 org.apache.beam.sdk.transforms.FlattenTest >
testFlattenPCollectionsThenParDo FAILED
08:33:31 java.lang.RuntimeException at FlattenTest.java:118
```
Checking which PR started this failure.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 412483)
Time Spent: 11h 10m (was: 11h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 11h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examp
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=412384&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-412384
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 30/Mar/20 15:09
Start Date: 30/Mar/20 15:09
Worklog Time Spent: 10m
Work Description: kamilwu commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606057311
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 412384)
Time Spent: 11h (was: 10h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 11h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=412383&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-412383
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 30/Mar/20 15:08
Start Date: 30/Mar/20 15:08
Worklog Time Spent: 10m
Work Description: kamilwu commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606057763
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 412383)
Time Spent: 10h 50m (was: 10h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 10h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=412382&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-412382
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 30/Mar/20 15:07
Start Date: 30/Mar/20 15:07
Worklog Time Spent: 10m
Work Description: kamilwu commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606057311
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 412382)
Time Spent: 10h 40m (was: 10.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 10h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=412381&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-412381
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 30/Mar/20 15:07
Start Date: 30/Mar/20 15:07
Worklog Time Spent: 10m
Work Description: kamilwu commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-606057218
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 412381)
Time Spent: 10.5h (was: 10h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 10.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=410759&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410759
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 27/Mar/20 01:44
Start Date: 27/Mar/20 01:44
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604772021
Run SQL Postcommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 410759)
Time Spent: 10h 20m (was: 10h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 10h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=410754&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410754
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 27/Mar/20 01:43
Start Date: 27/Mar/20 01:43
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604771821
Run Java PostCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 410754)
Time Spent: 9.5h (was: 9h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 9.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=410755&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410755
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 27/Mar/20 01:43
Start Date: 27/Mar/20 01:43
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604771862
Run Java HadoopFormatIO Performance Test
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 410755)
Time Spent: 9h 40m (was: 9.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 9h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=410756&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410756
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 27/Mar/20 01:43
Start Date: 27/Mar/20 01:43
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604771894
Run BigQueryIO Streaming Performance Test Java
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 410756)
Time Spent: 9h 50m (was: 9h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 9h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=410757&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410757
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 27/Mar/20 01:43
Start Date: 27/Mar/20 01:43
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604771955
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 410757)
Time Spent: 10h (was: 9h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 10h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=410758&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-410758
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 27/Mar/20 01:43
Start Date: 27/Mar/20 01:43
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-604771990
Run Spark ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 410758)
Time Spent: 10h 10m (was: 10h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 10h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=409142&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409142
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 24/Mar/20 22:35
Start Date: 24/Mar/20 22:35
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603539923
Run Spark ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 409142)
Time Spent: 9h (was: 8h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 9h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=409145&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409145
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 24/Mar/20 22:35
Start Date: 24/Mar/20 22:35
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603540099
Run Java PostCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 409145)
Time Spent: 9h 20m (was: 9h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 9h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=409144&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409144
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 24/Mar/20 22:35
Start Date: 24/Mar/20 22:35
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603540005
Run SQL Postcommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 409144)
Time Spent: 9h 10m (was: 9h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 9h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=409140&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409140
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 24/Mar/20 22:34
Start Date: 24/Mar/20 22:34
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603539817
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 409140)
Time Spent: 8h 50m (was: 8h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 8h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=409136&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409136
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 24/Mar/20 22:34
Start Date: 24/Mar/20 22:34
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603539614
Run Java HadoopFormatIO Performance Test
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 409136)
Time Spent: 8.5h (was: 8h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 8.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=409138&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409138
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 24/Mar/20 22:34
Start Date: 24/Mar/20 22:34
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603539728
Run BigQueryIO Streaming Performance Test Java
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 409138)
Time Spent: 8h 40m (was: 8.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 8h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=409134&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409134
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 24/Mar/20 22:33
Start Date: 24/Mar/20 22:33
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-603539526
Run Java PostCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 409134)
Time Spent: 8h 20m (was: 8h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 8h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=408099&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-408099
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 23/Mar/20 17:58
Start Date: 23/Mar/20 17:58
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r396647354
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin {
commons_lang3 :
"org.apache.commons:commons-lang3:3.9",
commons_math3 :
"org.apache.commons:commons-math3:3.6.1",
error_prone_annotations :
"com.google.errorprone:error_prone_annotations:2.0.15",
-gax :
"com.google.api:gax:$gax_version",
-gax_grpc:
"com.google.api:gax-grpc:$gax_version",
+gax : "com.google.api:gax",
+gax_grpc:
"com.google.api:gax-grpc",
google_api_client :
"com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 :
"com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 :
"com.google.api-client:google-api-client-java6:$google_clients_version",
-google_api_common :
"com.google.api:api-common:1.8.1",
+google_api_common :
"com.google.api:api-common",
Review comment:
Added the link and comments ` // google_cloud_platform_libraries_bom sets
version`. They may look too verbose, but they'll help future question "_where
is the version of this artifact?_", especially when we start using other BOMs
than GCP Libraries BOM in future.
> Beam will depend on that exact list of dependencies shown in that
dashboard.
Not exact list. Beam will pick up the version from the BOM for Maven
artifacts without a version.
For example,
- Beam will depend on the BOM's gax version "1.54.0", because Beam's
dependency `com.google.api:gax` does not specify a version.
- Beam **will not** depend on the BOM's Guava version "28.2-android",
because Beam explicitly specify Guava's version "25.1-jre" as in
`com.google.guava:guava:$guava_version`.
- Beam **will not** depend on the BOM's
`com.google.http-client:google-http-client-apache-v2:1.34.2`, because Beam does
not depend on `com.google.http-client:google-http-client-apache-v2`.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 408099)
Time Spent: 8h 10m (was: 8h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 8h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via vers
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=408045&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-408045
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 23/Mar/20 16:41
Start Date: 23/Mar/20 16:41
Worklog Time Spent: 10m
Work Description: aaltay commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r396593515
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin {
commons_lang3 :
"org.apache.commons:commons-lang3:3.9",
commons_math3 :
"org.apache.commons:commons-math3:3.6.1",
error_prone_annotations :
"com.google.errorprone:error_prone_annotations:2.0.15",
-gax :
"com.google.api:gax:$gax_version",
-gax_grpc:
"com.google.api:gax-grpc:$gax_version",
+gax : "com.google.api:gax",
+gax_grpc:
"com.google.api:gax-grpc",
google_api_client :
"com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 :
"com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 :
"com.google.api-client:google-api-client-java6:$google_clients_version",
-google_api_common :
"com.google.api:api-common:1.8.1",
+google_api_common :
"com.google.api:api-common",
Review comment:
OK, this is good.
- For easy access could we link to that dashboard from the place we are
including the bom in BeamModulePlugin.groovy
One clarification question. Beam will depend on that exact list of
dependencies shown in that dashboard. Is that correct?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 408045)
Time Spent: 8h (was: 7h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 8h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=408041&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-408041
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 23/Mar/20 16:38
Start Date: 23/Mar/20 16:38
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-602716275
> @aaltay Added "Artifact Version Changes" section in the PR description.
Thanks. @chamikaramj - could you review this section, especially the IOs in
downgrade/upgrade sections?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 408041)
Time Spent: 7h 50m (was: 7h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 7h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=408012&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-408012
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 23/Mar/20 16:05
Start Date: 23/Mar/20 16:05
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r396544685
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin {
commons_lang3 :
"org.apache.commons:commons-lang3:3.9",
commons_math3 :
"org.apache.commons:commons-math3:3.6.1",
error_prone_annotations :
"com.google.errorprone:error_prone_annotations:2.0.15",
-gax :
"com.google.api:gax:$gax_version",
-gax_grpc:
"com.google.api:gax-grpc:$gax_version",
+gax : "com.google.api:gax",
+gax_grpc:
"com.google.api:gax-grpc",
google_api_client :
"com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 :
"com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 :
"com.google.api-client:google-api-client-java6:$google_clients_version",
-google_api_common :
"com.google.api:api-common:1.8.1",
+google_api_common :
"com.google.api:api-common",
Review comment:
The BOM has dashboard. For 4.2.0, the dashboard URL is
https://storage.googleapis.com/cloud-opensource-java-dashboard/com.google.cloud/libraries-bom/4.2.0/artifact_details.html
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 408012)
Time Spent: 7h 40m (was: 7.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 7h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
>
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=407417&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-407417
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 21/Mar/20 13:02
Start Date: 21/Mar/20 13:02
Worklog Time Spent: 10m
Work Description: suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-602040903
Yes, I will summarize the version change on Monday.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 407417)
Time Spent: 7.5h (was: 7h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 7.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=407297&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-407297
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 20/Mar/20 22:09
Start Date: 20/Mar/20 22:09
Worklog Time Spent: 10m
Work Description: aaltay commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r395911140
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin {
commons_lang3 :
"org.apache.commons:commons-lang3:3.9",
commons_math3 :
"org.apache.commons:commons-math3:3.6.1",
error_prone_annotations :
"com.google.errorprone:error_prone_annotations:2.0.15",
-gax :
"com.google.api:gax:$gax_version",
-gax_grpc:
"com.google.api:gax-grpc:$gax_version",
+gax : "com.google.api:gax",
+gax_grpc:
"com.google.api:gax-grpc",
google_api_client :
"com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 :
"com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 :
"com.google.api-client:google-api-client-java6:$google_clients_version",
-google_api_common :
"com.google.api:api-common:1.8.1",
+google_api_common :
"com.google.api:api-common",
Review comment:
A question for my learning. Previously this file had specific dependency
versions. For any released Beam version we could check the file in the release
branch and get a list of dependencies and their versions.
After this change, how can we do the same thing? Would it happen through a
generated BOM file in the release branch?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 407297)
Time Spent: 7h 20m (was: 7h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
>
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405887&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405887
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 19/Mar/20 01:07
Start Date: 19/Mar/20 01:07
Worklog Time Spent: 10m
Work Description: suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600933706
@aaltay Thank you.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405887)
Time Spent: 7h 10m (was: 7h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405885&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405885
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 19/Mar/20 01:07
Start Date: 19/Mar/20 01:07
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600933591
Run Python PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405885)
Time Spent: 7h (was: 6h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 7h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405884&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405884
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 19/Mar/20 01:07
Start Date: 19/Mar/20 01:07
Worklog Time Spent: 10m
Work Description: aaltay commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600933550
Run Java PreCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405884)
Time Spent: 6h 50m (was: 6h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405878&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405878
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 19/Mar/20 00:53
Start Date: 19/Mar/20 00:53
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394727429
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
-force project.library.java.values()
+// Filtering versionless coordinates that depend on BOM
+def librariesWithVersion = project.library.java.values().findAll {
it.split(':').size() > 2 }
+force librariesWithVersion
Review comment:
Created PR for my suggestion https://github.com/apache/beam/pull/11168
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405878)
Time Spent: 6h 40m (was: 6.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 6h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405754&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405754
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 21:31
Start Date: 18/Mar/20 21:31
Worklog Time Spent: 10m
Work Description: suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600868999
## Java Precommit failed
`org.apache.beam.runners.flink.translation.wrappers.streaming.io.UnboundedSourceWrapperTest`
is BEAM-9470.
Samza error:
https://builds.apache.org/job/beam_PreCommit_Java_Commit/10414/testReport/junit/org.apache.beam.runners.samza.runtime/SamzaTimerInternalsFactoryTest/testRestore/
```
Caused by: org.rocksdb.RocksDBException: Can't access /000370.sst: IO error:
while stat a file for size: /tmp/store2/000370.sst: No such file or directory
```
## Python PreCommit failed
https://builds.apache.org/job/beam_PreCommit_Python_Commit/11782/testReport/junit/apache_beam.runners.portability.fn_api_runner_test/FnApiRunnerTestWithGrpc/test_large_elements/
complains `IndexError: index out of range`.
I find another irrelevant build
([11780](https://builds.apache.org/job/beam_PreCommit_Python_Commit/11780/testReport/junit/apache_beam.runners.portability.fn_api_runner_test/FnApiRunnerTestWithGrpc/test_large_elements/))
also show this error.
They all seem irrelevant to this PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405754)
Time Spent: 6.5h (was: 6h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 6.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {nofor
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405631&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405631
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 17:53
Start Date: 18/Mar/20 17:53
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394535755
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
-force project.library.java.values()
+// Filtering versionless coordinates that depend on BOM
+def librariesWithVersion = project.library.java.values().findAll {
it.split(':').size() > 2 }
+force librariesWithVersion
Review comment:
Yes, I'll try that.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405631)
Time Spent: 6h 20m (was: 6h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 6h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {nofor
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405625&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405625
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 17:52
Start Date: 18/Mar/20 17:52
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394535077
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
-force project.library.java.values()
+// Filtering versionless coordinates that depend on BOM
+def librariesWithVersion = project.library.java.values().findAll {
it.split(':').size() > 2 }
+force librariesWithVersion
Review comment:
Try to remove it everywhere and see what happens for the tests but that
should be decoupled from this PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405625)
Time Spent: 6h 10m (was: 6h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 6h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
>
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405618&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405618
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 17:40
Start Date: 18/Mar/20 17:40
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394527172
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
-force project.library.java.values()
+// Filtering versionless coordinates that depend on BOM
+def librariesWithVersion = project.library.java.values().findAll {
it.split(':').size() > 2 }
+force librariesWithVersion
Review comment:
> Would it be wise to enumerate all the artifacts in the BOM and force their
resolution?
I couldn't find a way to do that. Even if there's a way, I feel forcing the
all library versions for builds is unnecessary.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405618)
Time Spent: 5h 50m (was: 5h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {no
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405621&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405621
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 17:42
Start Date: 18/Mar/20 17:42
Worklog Time Spent: 10m
Work Description: suztomo commented on issue #11156: [BEAM-9444] Use GCP
Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600770827
@aromanenko-dev Thank you.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405621)
Time Spent: 6h (was: 5h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 6h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405594&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405594
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 17:07
Start Date: 18/Mar/20 17:07
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600752230
Run Spark ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405594)
Time Spent: 5.5h (was: 5h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405593&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405593
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 17:07
Start Date: 18/Mar/20 17:07
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600752160
Run Dataflow ValidatesRunner
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405593)
Time Spent: 5h 20m (was: 5h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405595&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405595
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 17:07
Start Date: 18/Mar/20 17:07
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600752336
Run SQL Postcommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405595)
Time Spent: 5h 40m (was: 5.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405591&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405591
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 17:06
Start Date: 18/Mar/20 17:06
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600751766
Run BigQueryIO Streaming Performance Test Java
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405591)
Time Spent: 5h (was: 4h 50m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405592&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405592
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 17:06
Start Date: 18/Mar/20 17:06
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600752021
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405592)
Time Spent: 5h 10m (was: 5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 5h 10m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405590&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405590
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 17:05
Start Date: 18/Mar/20 17:05
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600751588
Run Java HadoopFormatIO Performance Test
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405590)
Time Spent: 4h 50m (was: 4h 40m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405589&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405589
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 17:05
Start Date: 18/Mar/20 17:05
Worklog Time Spent: 10m
Work Description: aromanenko-dev commented on issue #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#issuecomment-600751410
Run Java PostCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405589)
Time Spent: 4h 40m (was: 4.5h)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/beam-sdks-java-io-google-cloud-platform/2.21.0-SNAPSHOT/beam-sdks-java-io-google-cloud-platform-2.21.0-SNAPSHOT.pom}}
> had the following problems.
> h2. The GCP Libraries BOM showing up in dependencies section:
> {noformat}
>
>
> com.google.cloud
> libraries-bom
> 4.2.0
> compile
>
>
> com.google.guava
> guava-jdk5
> ...
>
>
> {noformat}
> h2. The artifact that use the BOM in Gradle is missing version in the
> dependency.
> {noformat}
>
> com.google.api
> gax
>
> compile
> ...
>
> {noformat}
> h1. DependencyManagement section in generated pom.xml
> How can I check whether a entry in dependencies is "platform"?
> !Screen Shot 2020-03-17 at 16.01.16.png|width=504,height=344!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405566&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405566
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 16:35
Start Date: 18/Mar/20 16:35
Worklog Time Spent: 10m
Work Description: lukecwik commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394484244
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -1274,7 +1296,9 @@ class BeamModulePlugin implements Plugin {
// has different dependencies than our project.
if (config.getName() != "errorprone" && !inDependencyUpdates) {
config.resolutionStrategy {
-force project.library.java.values()
+// Filtering versionless coordinates that depend on BOM
+def librariesWithVersion = project.library.java.values().findAll {
it.split(':').size() > 2 }
+force librariesWithVersion
Review comment:
> > to increase the compatibility across multiple Beam modules
>
> That increased compatibility only affects Beam's build time. Not for Beam
users who consume Beam's Maven artifacts. (Correct me if I'm wrong)
Yes, that is my understanding as well.
> > How would the BOM interact with the force if it stayed or if it was
removed?
>
> The question whether force is not directly relevant to the BOM (although
artifacts that rely on the BOM do not have versions in this file, therefore
cannot be used with `force`)
Would it be wise to enumerate all the artifacts in the BOM and `force` their
resolution?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405566)
Time Spent: 4.5h (was: 4h 20m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
> Attachments: Screen Shot 2020-03-13 at 13.33.01.png, Screen Shot
> 2020-03-17 at 16.01.16.png
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Shall we use GCP Libraries BOM to specify Google-related library versions?
>
> I've been working on Beam's dependency upgrades in the past few months. I
> think it's time to consider a long-term solution to keep the libraries
> up-to-date with small maintenance effort. To achieve that, I propose Beam to
> use GCP Libraries BOM to set the Google-related library versions, rather than
> trying to make changes in each of ~30 Google libraries.
>
> h1. Background
> A BOM is pom.xml that provides dependencyManagement to importing projects.
>
> GCP Libraries BOM is a BOM that includes many Google Cloud related libraries
> + gRPC + protobuf. We (Google Cloud Java Diamond Dependency team) maintain
> the BOM so that the set of the libraries are compatible with each other.
>
> h1. Implementation
> Notes for obstacles.
> h2. BeamModulePlugin's "force" does not take BOM into account (thus fails)
> {{forcedModules}} via version resolution strategy is playing bad. This causes
> {noformat}
> A problem occurred evaluating project ':sdks:java:extensions:sql'.
> Could not resolve all dependencies for configuration
> ':sdks:java:extensions:sql:fmppTemplates'.
> Invalid format: 'com.google.cloud:google-cloud-core'. Group, name and version
> cannot be empty. Correct example: 'org.gradle:gradle-core:1.0'{noformat}
> !Screen Shot 2020-03-13 at 13.33.01.png|width=489,height=287!
>
> h2. :sdks:java:maven-archetypes:examples needs the version of
> google-http-client
> The task requires the version for the library:
> {code:java}
> 'google-http-client.version':
> dependencies.create(project.library.java.google_http_client).getVersion(),
> {code}
> This would generate NullPointerException. Running gradlew without the
> subproject:
>
> {code:java}
> ./gradlew -p sdks/java check -x :sdks:java:maven-archetypes:examples:check
> {code}
> h1. Problem in Gradle-generated pom files
> The generated Maven artifact POM has invalid data due to the BOM change. For
> example my locally installed
> {{~/.m2/repository/org/apache/beam/bea
[jira] [Work logged] (BEAM-9444) Shall we use GCP Libraries BOM to specify Google-related library versions?
[
https://issues.apache.org/jira/browse/BEAM-9444?focusedWorklogId=405544&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-405544
]
ASF GitHub Bot logged work on BEAM-9444:
Author: ASF GitHub Bot
Created on: 18/Mar/20 16:10
Start Date: 18/Mar/20 16:10
Worklog Time Spent: 10m
Work Description: suztomo commented on pull request #11156: [BEAM-9444]
Use GCP Libraries BOM for Google Cloud Dependencies
URL: https://github.com/apache/beam/pull/11156#discussion_r394466904
##
File path:
buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy
##
@@ -444,45 +439,46 @@ class BeamModulePlugin implements Plugin {
commons_lang3 :
"org.apache.commons:commons-lang3:3.9",
commons_math3 :
"org.apache.commons:commons-math3:3.6.1",
error_prone_annotations :
"com.google.errorprone:error_prone_annotations:2.0.15",
-gax :
"com.google.api:gax:$gax_version",
-gax_grpc:
"com.google.api:gax-grpc:$gax_version",
+gax : "com.google.api:gax",
+gax_grpc:
"com.google.api:gax-grpc",
google_api_client :
"com.google.api-client:google-api-client:$google_clients_version",
google_api_client_jackson2 :
"com.google.api-client:google-api-client-jackson2:$google_clients_version",
google_api_client_java6 :
"com.google.api-client:google-api-client-java6:$google_clients_version",
-google_api_common :
"com.google.api:api-common:1.8.1",
+google_api_common :
"com.google.api:api-common",
google_api_services_bigquery:
"com.google.apis:google-api-services-bigquery:v2-rev20191211-$google_clients_version",
google_api_services_clouddebugger :
"com.google.apis:google-api-services-clouddebugger:v2-rev20191003-$google_clients_version",
google_api_services_cloudresourcemanager:
"com.google.apis:google-api-services-cloudresourcemanager:v1-rev20191206-$google_clients_version",
google_api_services_dataflow:
"com.google.apis:google-api-services-dataflow:v1b3-rev20190927-$google_clients_version",
google_api_services_pubsub :
"com.google.apis:google-api-services-pubsub:v1-rev2019-$google_clients_version",
google_api_services_storage :
"com.google.apis:google-api-services-storage:v1-rev20191011-$google_clients_version",
-google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials:$google_auth_version",
-google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http:$google_auth_version",
-google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery:1.108.0",
-google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage:0.125.0-beta",
+google_auth_library_credentials :
"com.google.auth:google-auth-library-credentials",
+google_auth_library_oauth2_http :
"com.google.auth:google-auth-library-oauth2-http",
+google_cloud_bigquery :
"com.google.cloud:google-cloud-bigquery",
+google_cloud_bigquery_storage :
"com.google.cloud:google-cloud-bigquerystorage",
google_cloud_bigtable_client_core :
"com.google.cloud.bigtable:bigtable-client-core:1.13.0",
Review comment:
As of now we don't have plan to increase the BOM coverage to keep
maintenance cost low. Low maintenance burden leads frequent release cycle. When
we find the burden is still low, we may add them.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 405544)
Time Spent: 4h 20m (was: 4h 10m)
> Shall we use GCP Libraries BOM to specify Google-related library versions?
> --
>
> Key: BEAM-9444
> URL: https://issues.apache.org/jira/browse/BEAM-9444
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Tomo Suzuki
>Assignee: Tomo Suzuki
>Priority: Major
>
