[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=504362&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504362 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 23/Oct/20 23:11 Start Date: 23/Oct/20 23:11 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on a change in pull request #11243: URL: https://github.com/apache/beam/pull/11243#discussion_r511192385 ## File path: sdks/java/container/license_scripts/license_script.sh ## @@ -0,0 +1,38 @@ + # Licensed to the Apache Software Foundation (ASF) under one + # or more contributor license agreements. See the NOTICE file + # distributed with this work for additional information + # regarding copyright ownership. The ASF licenses this file + # to you under the Apache License, Version 2.0 (the + # License); you may not use this file except in compliance + # with the License. You may obtain a copy of the License at + # + # http://www.apache.org/licenses/LICENSE-2.0 + # + # Unless required by applicable law or agreed to in writing, software + # distributed under the License is distributed on an AS IS BASIS, + # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + # See the License for the specific language governing permissions and + # limitations under the License. + +set -e +# reports are generated at ~/beam/java_third_party_licenses +./gradlew generateLicenseReport --rerun-tasks Review comment: If I recall correctly, cached tasks don't rerun and no licenses are pulled from this run. But the temp folder was deleted, so no licenses are available. It is somewhat related to the above question. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 504362) Time Spent: 33.5h (was: 33h 20m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: P2 > Fix For: 2.21.0 > > Time Spent: 33.5h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=504361&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504361 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 23/Oct/20 23:08 Start Date: 23/Oct/20 23:08 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on a change in pull request #11243: URL: https://github.com/apache/beam/pull/11243#discussion_r511191738 ## File path: sdks/java/container/license_scripts/license_script.sh ## @@ -0,0 +1,38 @@ + # Licensed to the Apache Software Foundation (ASF) under one + # or more contributor license agreements. See the NOTICE file + # distributed with this work for additional information + # regarding copyright ownership. The ASF licenses this file + # to you under the Apache License, Version 2.0 (the + # License); you may not use this file except in compliance + # with the License. You may obtain a copy of the License at + # + # http://www.apache.org/licenses/LICENSE-2.0 + # + # Unless required by applicable law or agreed to in writing, software + # distributed under the License is distributed on an AS IS BASIS, + # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + # See the License for the specific language governing permissions and + # limitations under the License. + +set -e +# reports are generated at ~/beam/java_third_party_licenses +./gradlew generateLicenseReport --rerun-tasks + +# install packages needed for pull_licenses_java.py +pip install beautifulsoup4==4.9.0 +pip install future==0.17.1 +pip install PyYAML==5.3 +pip install tenacity==5.0.4 + +# pull licenses, notices and source code +python sdks/java/container/license_scripts/pull_licenses_java.py + +pip uninstall -y beautifulsoup4 +pip uninstall -y future +pip uninstall -y PyYAML +pip uninstall -y tenacity + +mkdir sdks/java/container/third_party_licenses +cp -r java_third_party_licenses/*.jar sdks/java/container/third_party_licenses/ +cp -r java_third_party_licenses/*.csv sdks/java/container/third_party_licenses/ +rm -rf java_third_party_licenses Review comment: Here is what happens. 1. Create `~/beam/java_third_party_licenses` folder, because the plugin generates correct dependency list only when run from root and the folder should be under root. 2. Copy the files from `~/beam/java_third_party_licenses` to `sdks/java/container/third_party_licenses/`. 3. Add the files to the containers. 4. Remove `~/beam/java_third_party_licenses` in order to remove temp files and write to a clean new folder for next run. If the problem with 1 is solved with new version of the plugin, 1 can be skipped. I think it's a good practice to remove temp files from local. But I agree it might miss leading, better to add a comment. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 504361) Time Spent: 33h 20m (was: 33h 10m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: P2 > Fix For: 2.21.0 > > Time Spent: 33h 20m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=504349&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504349 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 23/Oct/20 22:28 Start Date: 23/Oct/20 22:28 Worklog Time Spent: 10m Work Description: kennknowles commented on pull request #11243: URL: https://github.com/apache/beam/pull/11243#issuecomment-715616864 Trying to fix https://issues.apache.org/jira/browse/BEAM-6 (or maybe that bug is wrong, but I am trying to simplify and figure it out) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 504349) Time Spent: 33h 10m (was: 33h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: P2 > Fix For: 2.21.0 > > Time Spent: 33h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=504348&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-504348 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 23/Oct/20 22:27 Start Date: 23/Oct/20 22:27 Worklog Time Spent: 10m Work Description: kennknowles commented on a change in pull request #11243: URL: https://github.com/apache/beam/pull/11243#discussion_r511181455 ## File path: sdks/java/container/license_scripts/license_script.sh ## @@ -0,0 +1,38 @@ + # Licensed to the Apache Software Foundation (ASF) under one + # or more contributor license agreements. See the NOTICE file + # distributed with this work for additional information + # regarding copyright ownership. The ASF licenses this file + # to you under the Apache License, Version 2.0 (the + # License); you may not use this file except in compliance + # with the License. You may obtain a copy of the License at + # + # http://www.apache.org/licenses/LICENSE-2.0 + # + # Unless required by applicable law or agreed to in writing, software + # distributed under the License is distributed on an AS IS BASIS, + # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + # See the License for the specific language governing permissions and + # limitations under the License. + +set -e +# reports are generated at ~/beam/java_third_party_licenses +./gradlew generateLicenseReport --rerun-tasks Review comment: Why is it necessary to `--rerun-tasks`? This should not be necessary. ## File path: sdks/java/container/license_scripts/license_script.sh ## @@ -0,0 +1,38 @@ + # Licensed to the Apache Software Foundation (ASF) under one + # or more contributor license agreements. See the NOTICE file + # distributed with this work for additional information + # regarding copyright ownership. The ASF licenses this file + # to you under the Apache License, Version 2.0 (the + # License); you may not use this file except in compliance + # with the License. You may obtain a copy of the License at + # + # http://www.apache.org/licenses/LICENSE-2.0 + # + # Unless required by applicable law or agreed to in writing, software + # distributed under the License is distributed on an AS IS BASIS, + # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + # See the License for the specific language governing permissions and + # limitations under the License. + +set -e +# reports are generated at ~/beam/java_third_party_licenses +./gradlew generateLicenseReport --rerun-tasks + +# install packages needed for pull_licenses_java.py +pip install beautifulsoup4==4.9.0 +pip install future==0.17.1 +pip install PyYAML==5.3 +pip install tenacity==5.0.4 + +# pull licenses, notices and source code +python sdks/java/container/license_scripts/pull_licenses_java.py + +pip uninstall -y beautifulsoup4 +pip uninstall -y future +pip uninstall -y PyYAML +pip uninstall -y tenacity + +mkdir sdks/java/container/third_party_licenses +cp -r java_third_party_licenses/*.jar sdks/java/container/third_party_licenses/ +cp -r java_third_party_licenses/*.csv sdks/java/container/third_party_licenses/ +rm -rf java_third_party_licenses Review comment: Why remove this directory? It is an output directory for a task so it confuses things a bit. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 504348) Time Spent: 33h (was: 32h 50m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: P2 > Fix For: 2.21.0 > > Time Spent: 33h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=487695&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-487695 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 22/Sep/20 03:28 Start Date: 22/Sep/20 03:28 Worklog Time Spent: 10m Work Description: alanmyrvold merged pull request #12879: URL: https://github.com/apache/beam/pull/12879 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 487695) Time Spent: 32h 50m (was: 32h 40m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: P2 > Fix For: 2.21.0 > > Time Spent: 32h 50m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=487145&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-487145 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 21/Sep/20 18:11 Start Date: 21/Sep/20 18:11 Worklog Time Spent: 10m Work Description: alanmyrvold merged pull request #12879: URL: https://github.com/apache/beam/pull/12879 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 487145) Time Spent: 32h 40m (was: 32.5h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: P2 > Fix For: 2.21.0 > > Time Spent: 32h 40m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=486449&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-486449 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 19/Sep/20 00:44 Start Date: 19/Sep/20 00:44 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #12879: URL: https://github.com/apache/beam/pull/12879#issuecomment-695137542 # [Codecov](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=h1) Report > Merging [#12879](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/1914976c6f9e2e756761f576e13c6e5f681d532a?el=desc) will **decrease** coverage by `0.01%`. > The diff coverage is `n/a`. [](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master #12879 +/- ## == - Coverage 82.33% 82.31% -0.02% == Files 452 452 Lines 5398553985 == - Hits944439 -10 - Misses 9536 9546 +10 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `88.63% <0.00%> (-4.55%)` | :arrow_down: | | [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `90.90% <0.00%> (-1.82%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `88.68% <0.00%> (-1.23%)` | :arrow_down: | | [sdks/python/apache\_beam/io/localfilesystem.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vbG9jYWxmaWxlc3lzdGVtLnB5) | `90.90% <0.00%> (-0.76%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.98% <0.00%> (-0.36%)` | :arrow_down: | | [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.44% <0.00%> (-0.19%)` | :arrow_down: | | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `80.57% <0.00%> (-0.18%)` | :arrow_down: | | [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `89.20% <0.00%> (+0.44%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=footer). Last update [1914976...8fe5805](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 486449) Time Spent: 32.5h (was: 32h 20m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-syst
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=486447&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-486447 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 19/Sep/20 00:30 Start Date: 19/Sep/20 00:30 Worklog Time Spent: 10m Work Description: codecov[bot] commented on pull request #12879: URL: https://github.com/apache/beam/pull/12879#issuecomment-695137542 # [Codecov](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=h1) Report > Merging [#12879](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=desc) into [master](https://codecov.io/gh/apache/beam/commit/bbda2208a62341c63b9fa75b744b39962b344afd?el=desc) will **increase** coverage by `0.00%`. > The diff coverage is `90.09%`. [](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=tree) ```diff @@ Coverage Diff@@ ## master #12879+/- ## Coverage 82.31% 82.31% Files 451 452 +1 Lines 5387453985 +111 + Hits4434444439+95 - Misses 9530 9546+16 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/io.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2lvLnB5) | `89.62% <89.62%> (ø)` | | | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `90.02% <100.00%> (+0.11%)` | :arrow_up: | | [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `88.63% <0.00%> (-4.55%)` | :arrow_down: | | [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `90.90% <0.00%> (-1.82%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `88.68% <0.00%> (-1.23%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.98% <0.00%> (-0.36%)` | :arrow_down: | | [...nners/portability/fn\_api\_runner/worker\_handlers.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9wb3J0YWJpbGl0eS9mbl9hcGlfcnVubmVyL3dvcmtlcl9oYW5kbGVycy5weQ==) | `80.57% <0.00%> (-0.18%)` | :arrow_down: | | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `94.58% <0.00%> (+0.13%)` | :arrow_up: | | [sdks/python/apache\_beam/io/fileio.py](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vZmlsZWlvLnB5) | `94.07% <0.00%> (+0.34%)` | :arrow_up: | | ... and [2 more](https://codecov.io/gh/apache/beam/pull/12879/diff?src=pr&el=tree-more) | | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=footer). Last update [1914976...8fe5805](https://codecov.io/gh/apache/beam/pull/12879?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 486447) Time Spent: 32h 20m (w
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=486441&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-486441 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 19/Sep/20 00:06 Start Date: 19/Sep/20 00:06 Worklog Time Spent: 10m Work Description: alanmyrvold opened a new pull request #12879: URL: https://github.com/apache/beam/pull/12879 To help check the python dependency licenses, add python-licenses.csv to /opt/apache/beam/third_party_licenses/ that lists the name, url, and license type for the licenses found by pip-licenses in the pull_licenses_py.py script. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2 --- | --- | --- | --- | --- | --- | --- Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | --- Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) | [ Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=457176&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-457176
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 10/Jul/20 14:05
Start Date: 10/Jul/20 14:05
Worklog Time Spent: 10m
Work Description: kennknowles commented on a change in pull request
#11243:
URL: https://github.com/apache/beam/pull/11243#discussion_r452865162
##
File path: sdks/java/container/build.gradle
##
@@ -68,6 +73,28 @@ golang {
}
}
+// this is a workaround to call generateLicenseReport task from project root
directory.
+// generateLicenseReport does not return correct dependency list when not
called from the root.
+task generateThirdPartyLicenses(type: Exec) {
+ workingDir project.rootProject.projectDir
+ commandLine './sdks/java/container/license_scripts/license_script.sh'
+}
+
+//command: ./gradlew generateLicenseReport
+import com.github.jk1.license.render.*
+licenseReport {
+ outputDir = 'java_third_party_licenses'
+ projects = [project.rootProject] + project.rootProject.subprojects
+ excludeOwnGroup = true
+ configurations = ALL
Review comment:
The issue I hit was
https://github.com/kelloggm/checkerframework-gradle-plugin/issues/109 where the
plugin creates a broken configuration that is actually not needed.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 457176)
Time Spent: 32h (was: 31h 50m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: P2
> Fix For: 2.21.0
>
> Time Spent: 32h
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=455262&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-455262
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 07/Jul/20 07:14
Start Date: 07/Jul/20 07:14
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11243:
URL: https://github.com/apache/beam/pull/11243#discussion_r450656496
##
File path: sdks/java/container/build.gradle
##
@@ -68,6 +73,28 @@ golang {
}
}
+// this is a workaround to call generateLicenseReport task from project root
directory.
+// generateLicenseReport does not return correct dependency list when not
called from the root.
+task generateThirdPartyLicenses(type: Exec) {
+ workingDir project.rootProject.projectDir
+ commandLine './sdks/java/container/license_scripts/license_script.sh'
+}
+
+//command: ./gradlew generateLicenseReport
+import com.github.jk1.license.render.*
+licenseReport {
+ outputDir = 'java_third_party_licenses'
+ projects = [project.rootProject] + project.rootProject.subprojects
+ excludeOwnGroup = true
+ configurations = ALL
Review comment:
I tried with `runtime` at first, and it detected way less dependencies
than `ALL`. To be safe (with legal issues), I changed the setting to `ALL`.
I noticed that multiple different versions are detected for some
dependencies. I checked several of them and they all looked correct. Does the
issue go away if you change the setting to `runtime`? How did you notice the
issue? (the licenses were not pulled automatically so you have to add links
manually?) I am asking this to see if there is an easy way to check unrelated
dependencies/versions are detected.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 455262)
Time Spent: 31h 50m (was: 31h 40m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: P2
> Fix For: 2.21.0
>
> Time Spent: 31h 50m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=455075&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-455075
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 06/Jul/20 20:41
Start Date: 06/Jul/20 20:41
Worklog Time Spent: 10m
Work Description: kennknowles commented on a change in pull request
#11243:
URL: https://github.com/apache/beam/pull/11243#discussion_r450468077
##
File path: sdks/java/container/build.gradle
##
@@ -68,6 +73,28 @@ golang {
}
}
+// this is a workaround to call generateLicenseReport task from project root
directory.
+// generateLicenseReport does not return correct dependency list when not
called from the root.
+task generateThirdPartyLicenses(type: Exec) {
+ workingDir project.rootProject.projectDir
+ commandLine './sdks/java/container/license_scripts/license_script.sh'
+}
+
+//command: ./gradlew generateLicenseReport
+import com.github.jk1.license.render.*
+licenseReport {
+ outputDir = 'java_third_party_licenses'
+ projects = [project.rootProject] + project.rootProject.subprojects
+ excludeOwnGroup = true
+ configurations = ALL
Review comment:
I recently hit a problem where dependency resolution here is incorrect
for one of these configurations. It is pulling versions that don't exist and
nothing depends on.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 455075)
Time Spent: 31h 40m (was: 31.5h)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: P2
> Fix For: 2.21.0
>
> Time Spent: 31h 40m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=455074&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-455074
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 06/Jul/20 20:40
Start Date: 06/Jul/20 20:40
Worklog Time Spent: 10m
Work Description: kennknowles commented on a change in pull request
#11243:
URL: https://github.com/apache/beam/pull/11243#discussion_r450467677
##
File path: sdks/java/container/build.gradle
##
@@ -68,6 +73,28 @@ golang {
}
}
+// this is a workaround to call generateLicenseReport task from project root
directory.
+// generateLicenseReport does not return correct dependency list when not
called from the root.
+task generateThirdPartyLicenses(type: Exec) {
+ workingDir project.rootProject.projectDir
+ commandLine './sdks/java/container/license_scripts/license_script.sh'
+}
+
+//command: ./gradlew generateLicenseReport
+import com.github.jk1.license.render.*
+licenseReport {
+ outputDir = 'java_third_party_licenses'
+ projects = [project.rootProject] + project.rootProject.subprojects
+ excludeOwnGroup = true
+ configurations = ALL
Review comment:
Why ALL? Should this just be `runtime`?
This pulls in lots of configurations that only are used to run static
analysis at build time, etc, and do not affect the license of the container.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 455074)
Time Spent: 31.5h (was: 31h 20m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: P2
> Fix For: 2.21.0
>
> Time Spent: 31.5h
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433971&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433971 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 23:48 Start Date: 15/May/20 23:48 Worklog Time Spent: 10m Work Description: ibzib merged pull request #11717: URL: https://github.com/apache/beam/pull/11717 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 433971) Time Spent: 31h 20m (was: 31h 10m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 31h 20m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433829&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433829 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 18:26 Start Date: 15/May/20 18:26 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11717: URL: https://github.com/apache/beam/pull/11717#issuecomment-629411176 > R: @ibzib Oops, thank you. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 433829) Time Spent: 31h 10m (was: 31h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 31h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433827&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433827 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 18:25 Start Date: 15/May/20 18:25 Worklog Time Spent: 10m Work Description: Hannah-Jiang merged pull request #11584: URL: https://github.com/apache/beam/pull/11584 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 433827) Time Spent: 31h (was: 30h 50m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 31h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433776&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433776 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 16:55 Start Date: 15/May/20 16:55 Worklog Time Spent: 10m Work Description: aaltay commented on pull request #11717: URL: https://github.com/apache/beam/pull/11717#issuecomment-629369332 R: @ibzib This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 433776) Time Spent: 30h 40m (was: 30.5h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 30h 40m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433777&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433777 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 16:55 Start Date: 15/May/20 16:55 Worklog Time Spent: 10m Work Description: aaltay commented on pull request #11584: URL: https://github.com/apache/beam/pull/11584#issuecomment-629369467 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 433777) Time Spent: 30h 50m (was: 30h 40m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 30h 50m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433560&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433560 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 05:14 Start Date: 15/May/20 05:14 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11717: URL: https://github.com/apache/beam/pull/11717#issuecomment-629031663 Please merge it if it looks good. The PR was reviewed at https://github.com/apache/beam/pull/11549. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 433560) Time Spent: 30.5h (was: 30h 20m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 30.5h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433558&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433558 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 05:12 Start Date: 15/May/20 05:12 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11584: URL: https://github.com/apache/beam/pull/11584#issuecomment-629030902 It is rebased. Please take a look when you have time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 433558) Time Spent: 30h 20m (was: 30h 10m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 30h 20m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433541&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433541 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 04:50 Start Date: 15/May/20 04:50 Worklog Time Spent: 10m Work Description: Hannah-Jiang closed pull request #11549: URL: https://github.com/apache/beam/pull/11549 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 433541) Time Spent: 30h 10m (was: 30h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 30h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433540&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433540 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 04:50 Start Date: 15/May/20 04:50 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11549: URL: https://github.com/apache/beam/pull/11549#issuecomment-629024197 > @Hannah-Jiang - you can continue with this change now. However you will need to rebase. Thanks for letting me know. I created a new PR https://github.com/apache/beam/pull/11549 and will close this one. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 433540) Time Spent: 30h (was: 29h 50m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 30h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433538&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433538 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 04:48 Start Date: 15/May/20 04:48 Worklog Time Spent: 10m Work Description: Hannah-Jiang opened a new pull request #11717: URL: https://github.com/apache/beam/pull/11717 @Kyle Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[ Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433460&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433460 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 01:07 Start Date: 15/May/20 01:07 Worklog Time Spent: 10m Work Description: aaltay commented on pull request #11584: URL: https://github.com/apache/beam/pull/11584#issuecomment-628965737 @Hannah-Jiang - you can continue with this change now. However you will need to rebase. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 433460) Time Spent: 29h 40m (was: 29.5h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 29h 40m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=433459&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-433459 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 15/May/20 01:07 Start Date: 15/May/20 01:07 Worklog Time Spent: 10m Work Description: aaltay commented on pull request #11549: URL: https://github.com/apache/beam/pull/11549#issuecomment-628965670 @Hannah-Jiang - you can continue with this change now. However you will need to rebase. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 433459) Time Spent: 29.5h (was: 29h 20m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 29.5h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=431525&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-431525 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 07/May/20 00:50 Start Date: 07/May/20 00:50 Worklog Time Spent: 10m Work Description: chamikaramj commented on pull request #11548: URL: https://github.com/apache/beam/pull/11548#issuecomment-624966273 Seems like this broke cross-language tests. https://builds.apache.org/view/A-D/view/Beam/view/PostCommit/job/beam_PostCommit_XVR_Flink/ https://scans.gradle.com/s/x24iwckafknka/console-log?task=:sdks:java:container:pullLicenses This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 431525) Time Spent: 29h 20m (was: 29h 10m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 29h 20m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=430314&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-430314 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 04/May/20 17:24 Start Date: 04/May/20 17:24 Worklog Time Spent: 10m Work Description: ibzib commented on pull request #11549: URL: https://github.com/apache/beam/pull/11549#issuecomment-623596078 The folks working on the website migration have requested that we pause website changes until they are finished with #11554 ([email thread](https://lists.apache.org/thread.html/r6b999b6d7d1f6cbb94e16bb2deed2b65098a6b14c4ac98707fe0c36a%40%3Cdev.beam.apache.org%3E)). @Hannah-Jiang can you please split this PR into 2 separate PRs for release notes and website changes? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 430314) Time Spent: 29h 10m (was: 29h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 29h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=429705&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429705
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 01/May/20 22:42
Start Date: 01/May/20 22:42
Worklog Time Spent: 10m
Work Description: ibzib commented on a change in pull request #11584:
URL: https://github.com/apache/beam/pull/11584#discussion_r418767547
##
File path: website/src/contribute/release-guide.md
##
@@ -688,8 +688,20 @@ Verify that files are
[present](https://dist.apache.org/repos/dist/dev/beam).
* Build Python images and push to DockerHub.
```
-./gradlew :sdks:python:container:buildAll -Pdocker-tag=${RELEASE}_rc{RC_NUM}
+./gradlew :sdks:python:container:buildAll -Pdocker-pull-licenses
-Pdocker-tag=${RELEASE}_rc{RC_NUM}
+```
+
+Verify that third party licenses are included by logging in to the images. For
Python SDK images, there should be around 80 ~ 100 dependencies.
+Please note that dependencies for the SDKs with different Python versions
vary.
+Need to verify all Python images by replacing `${ver}` in the following
command to `python2.7, python3.5, python3.6, python3.7`.
Review comment:
Ack, thanks for the heads up
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429705)
Time Spent: 29h (was: 28h 50m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 29h
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=429700&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429700
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 01/May/20 22:39
Start Date: 01/May/20 22:39
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11584:
URL: https://github.com/apache/beam/pull/11584#discussion_r418766264
##
File path: website/src/contribute/release-guide.md
##
@@ -688,8 +688,20 @@ Verify that files are
[present](https://dist.apache.org/repos/dist/dev/beam).
* Build Python images and push to DockerHub.
```
-./gradlew :sdks:python:container:buildAll -Pdocker-tag=${RELEASE}_rc{RC_NUM}
+./gradlew :sdks:python:container:buildAll -Pdocker-pull-licenses
-Pdocker-tag=${RELEASE}_rc{RC_NUM}
+```
+
+Verify that third party licenses are included by logging in to the images. For
Python SDK images, there should be around 80 ~ 100 dependencies.
+Please note that dependencies for the SDKs with different Python versions
vary.
+Need to verify all Python images by replacing `${ver}` in the following
command to `python2.7, python3.5, python3.6, python3.7`.
Review comment:
@ibzib , this part should be verified BEFORE pushing Python images. I'm
not sure when this PR can be merged because of website issues, so explicitly
pinging you here.
##
File path: website/src/contribute/release-guide.md
##
@@ -699,7 +711,18 @@ done
* Build Java images and push to DockerHub.
```
-./gradlew :sdks:java:container:dockerPush -Pdocker-pull-licenses
-Pdocker-tag=${RELEASE}_rc{RC_NUM}
+./gradlew :sdks:java:container:docker -Pdocker-pull-licenses
-Pdocker-tag=${RELEASE}_rc{RC_NUM}
+```
+
+Verify that third party licenses are included by logging in to the images. For
Java SDK images, there should be around 1400 dependencies.
+```
+docker run -it --entrypoint=/bin/bash
apache/beam_java_sdk:${RELEASE}_rc{RC_NUM}
+ls -al /opt/apache/beam/third_party_licenses/ | wc -l
+```
+
+After verifying the third party licenses are included correctly, push the
images to DockerHub.
+```
Review comment:
@ibzib , this part changed slightly. Please note the change from
`dockerPush` to `docker` at L714.
After creating the Java image, we should verify it BEFORE pushing to
DockerHub. I'm not sure when this PR can be merged because of website issues,
so explicitly pinging you here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429700)
Time Spent: 28h 50m (was: 28h 40m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 28h 50m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=429301&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429301
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 30/Apr/20 23:14
Start Date: 30/Apr/20 23:14
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #11584:
URL: https://github.com/apache/beam/pull/11584#discussion_r418340633
##
File path: sdks/python/container/py2/build.gradle
##
@@ -66,7 +66,8 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
files "../Dockerfile", "./build"
buildArgs(['py_version': "2.7",
- 'pull_licenses':
project.rootProject.hasProperty(["docker-pull-licenses"])])
+ 'pull_licenses':
project.rootProject.hasProperty(["docker-pull-licenses"]) ||
+ project.rootProject.hasProperty(["isRelease"])])
Review comment:
I see, thanks. I did not know that we need to set isRelease manually.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429301)
Time Spent: 28h 40m (was: 28.5h)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 28h 40m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=429255&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429255
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 30/Apr/20 20:49
Start Date: 30/Apr/20 20:49
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11584:
URL: https://github.com/apache/beam/pull/11584#discussion_r418278914
##
File path: sdks/python/container/py2/build.gradle
##
@@ -66,7 +66,8 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
files "../Dockerfile", "./build"
buildArgs(['py_version': "2.7",
- 'pull_licenses':
project.rootProject.hasProperty(["docker-pull-licenses"])])
+ 'pull_licenses':
project.rootProject.hasProperty(["docker-pull-licenses"]) ||
+ project.rootProject.hasProperty(["isRelease"])])
Review comment:
Currently, the `isRelease` tag needs to be passed with commands.
Ideally, this tag should be set by checking branch name.
Adding this support to docker images are not that helpful for now, but could
be used when we support auto setting `isRelease` tag.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429255)
Time Spent: 28h 20m (was: 28h 10m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 28h 20m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=429256&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429256
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 30/Apr/20 20:49
Start Date: 30/Apr/20 20:49
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11584:
URL: https://github.com/apache/beam/pull/11584#discussion_r418278914
##
File path: sdks/python/container/py2/build.gradle
##
@@ -66,7 +66,8 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
files "../Dockerfile", "./build"
buildArgs(['py_version': "2.7",
- 'pull_licenses':
project.rootProject.hasProperty(["docker-pull-licenses"])])
+ 'pull_licenses':
project.rootProject.hasProperty(["docker-pull-licenses"]) ||
+ project.rootProject.hasProperty(["isRelease"])])
Review comment:
Currently, the `isRelease` tag needs to be passed with commands.
Ideally, this tag should be set by checking branch name.
Detecting `isRelease` for docker image build is not that helpful for now,
but could be used when we support auto setting `isRelease` tag.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429256)
Time Spent: 28.5h (was: 28h 20m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 28.5h
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=429250&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429250 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 30/Apr/20 20:41 Start Date: 30/Apr/20 20:41 Worklog Time Spent: 10m Work Description: Hannah-Jiang opened a new pull request #11584: URL: https://github.com/apache/beam/pull/11584 R: @tvalentyn Cc: @ibzib Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[ Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=429217&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429217 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 30/Apr/20 19:32 Start Date: 30/Apr/20 19:32 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11548: URL: https://github.com/apache/beam/pull/11548#issuecomment-622059726 Thank you Valentyn. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 429217) Time Spent: 28h (was: 27h 50m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 28h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=429215&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429215
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 30/Apr/20 19:32
Start Date: 30/Apr/20 19:32
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r418241059
##
File path: .test-infra/jenkins/CommonJobProperties.groovy
##
@@ -163,6 +163,10 @@ class CommonJobProperties {
// For [BEAM-4847], hardcode Xms and Xmx to reasonable values (2g/4g).
context.switches("-Dorg.gradle.jvmargs=-Xms2g")
context.switches("-Dorg.gradle.jvmargs=-Xmx4g")
+
+// Add docker-pull-licenses option to all Jenkins test to add licenses to
when build docker images.
Review comment:
yep, rewording and `isRelease` tag detection for docker image build with
be in another PR.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429215)
Time Spent: 27h 50m (was: 27h 40m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 27h 50m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=429154&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429154
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 30/Apr/20 17:27
Start Date: 30/Apr/20 17:27
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r418171888
##
File path: .test-infra/jenkins/CommonJobProperties.groovy
##
@@ -163,6 +163,10 @@ class CommonJobProperties {
// For [BEAM-4847], hardcode Xms and Xmx to reasonable values (2g/4g).
context.switches("-Dorg.gradle.jvmargs=-Xms2g")
context.switches("-Dorg.gradle.jvmargs=-Xmx4g")
+
+// Add docker-pull-licenses option to all Jenkins test to add licenses to
when build docker images.
Review comment:
I saw on the mailing list that you have updated release guide to include
the config flags to build images, and will be updating them. You can do those
changes in a follow-up pr.
You can conisider looking at the value of
https://github.com/apache/beam/blob/74a6565c8b64d9fadf256370df47a4c5dadafb55/buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy#L296
to determine whether the task is running for release branch, instead of
asking release manager to pass a config flag.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429154)
Time Spent: 27h 40m (was: 27.5h)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 27h 40m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=429142&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429142
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 30/Apr/20 17:09
Start Date: 30/Apr/20 17:09
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r418160747
##
File path: .test-infra/jenkins/CommonJobProperties.groovy
##
@@ -163,6 +163,10 @@ class CommonJobProperties {
// For [BEAM-4847], hardcode Xms and Xmx to reasonable values (2g/4g).
context.switches("-Dorg.gradle.jvmargs=-Xms2g")
context.switches("-Dorg.gradle.jvmargs=-Xmx4g")
+
+// Add docker-pull-licenses option to all Jenkins test to add licenses to
when build docker images.
Review comment:
This is my other comment(copying from above, in case you missed it): I
suggested on mailing list that we can also pull licences whenever we run build
containers on the release branch. If community does not like that option for
some reason, then we should also update the release guide in this PR to make
sure release manager is aware that they need to pass a non-default option, and
check that licenses were added successfully.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429142)
Time Spent: 27.5h (was: 27h 20m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 27.5h
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=429141&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-429141
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 30/Apr/20 17:08
Start Date: 30/Apr/20 17:08
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r418160747
##
File path: .test-infra/jenkins/CommonJobProperties.groovy
##
@@ -163,6 +163,10 @@ class CommonJobProperties {
// For [BEAM-4847], hardcode Xms and Xmx to reasonable values (2g/4g).
context.switches("-Dorg.gradle.jvmargs=-Xms2g")
context.switches("-Dorg.gradle.jvmargs=-Xmx4g")
+
+// Add docker-pull-licenses option to all Jenkins test to add licenses to
when build docker images.
Review comment:
This is my other comment: I suggested on mailing list that we can also
pull licences whenever we run build containers on the release branch. If
community does not like that option for some reason, then we should also update
the release guide in this PR to make sure release manager is aware that they
need to pass a non-default option, and check that licenses were added
successfully.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 429141)
Time Spent: 27h 20m (was: 27h 10m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 27h 20m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428878&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428878 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 30/Apr/20 04:23 Start Date: 30/Apr/20 04:23 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on a change in pull request #11549: URL: https://github.com/apache/beam/pull/11549#discussion_r417745330 ## File path: CHANGES.md ## @@ -122,13 +122,22 @@ conversion to beam schema options. *Remark: Schema aware is still experimental.* values as strings) into Python native types that are written to Avro (Python's date, datetime types, decimal, etc). For more information see https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#avro_conversions. +* Licenses/notices of third party dependencies are added to Java and Python SDK docker images. ([BEAM-9136](https://issues.apache.org/jira/browse/BEAM-9136)) + +Java SDK images: By default, it checks if the licenses/notices are pullable, instead of pulling the licenses/notices. In order to pull licenses, `docker-pull-licenses` variable should ba passed. + +Python SDK images: By default, it pulls licenses/notices of the third party licenses. + +If you want to create lightweight images or do not want to deal with licenses, you can pass `no-licenses` variable when build docker images to skip it. Review comment: It is updated with the recent consensus. PTAL. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 428878) Time Spent: 27h 10m (was: 27h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 27h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428870&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428870
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 30/Apr/20 03:56
Start Date: 30/Apr/20 03:56
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r417739219
##
File path: .test-infra/jenkins/CommonJobProperties.groovy
##
@@ -163,6 +163,10 @@ class CommonJobProperties {
// For [BEAM-4847], hardcode Xms and Xmx to reasonable values (2g/4g).
context.switches("-Dorg.gradle.jvmargs=-Xms2g")
context.switches("-Dorg.gradle.jvmargs=-Xmx4g")
+
+// Add docker-pull-licenses option to all Jenkins test to add licenses to
when build docker images.
Review comment:
Do you have other comments? If not, I am going to merge it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428870)
Time Spent: 27h (was: 26h 50m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 27h
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428868&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428868
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 30/Apr/20 03:55
Start Date: 30/Apr/20 03:55
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r417735845
##
File path: .test-infra/jenkins/CommonJobProperties.groovy
##
@@ -163,6 +163,10 @@ class CommonJobProperties {
// For [BEAM-4847], hardcode Xms and Xmx to reasonable values (2g/4g).
context.switches("-Dorg.gradle.jvmargs=-Xms2g")
context.switches("-Dorg.gradle.jvmargs=-Xmx4g")
+
+// Add docker-pull-licenses option to all Jenkins test to add licenses to
when build docker images.
Review comment:
I will change the comment with different PR, because this PR triggers 10
tests and takes time to run and some tests are flaky.
For the comment about detecting isRelease tag, it will also be part of
another PR about release.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428868)
Time Spent: 26h 50m (was: 26h 40m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 26h 50m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428867&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428867 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 30/Apr/20 03:54 Start Date: 30/Apr/20 03:54 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11552: URL: https://github.com/apache/beam/pull/11552#issuecomment-621597254 This was added to PR11548, so close it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 428867) Time Spent: 26h 40m (was: 26.5h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 26h 40m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428866&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428866
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 30/Apr/20 03:43
Start Date: 30/Apr/20 03:43
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r417735845
##
File path: .test-infra/jenkins/CommonJobProperties.groovy
##
@@ -163,6 +163,10 @@ class CommonJobProperties {
// For [BEAM-4847], hardcode Xms and Xmx to reasonable values (2g/4g).
context.switches("-Dorg.gradle.jvmargs=-Xms2g")
context.switches("-Dorg.gradle.jvmargs=-Xmx4g")
+
+// Add docker-pull-licenses option to all Jenkins test to add licenses to
when build docker images.
Review comment:
I will change the comment with different PR, because this PR triggers 10
tests and takes time to run and some tests are flaky.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428866)
Time Spent: 26.5h (was: 26h 20m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 26.5h
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428865&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428865
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 30/Apr/20 03:39
Start Date: 30/Apr/20 03:39
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r417735845
##
File path: .test-infra/jenkins/CommonJobProperties.groovy
##
@@ -163,6 +163,10 @@ class CommonJobProperties {
// For [BEAM-4847], hardcode Xms and Xmx to reasonable values (2g/4g).
context.switches("-Dorg.gradle.jvmargs=-Xms2g")
context.switches("-Dorg.gradle.jvmargs=-Xmx4g")
+
+// Add docker-pull-licenses option to all Jenkins test to add licenses to
when build docker images.
Review comment:
I will change the comment with different PR, because this PR triggers 10
tests takes time to run and some tests are flaky.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428865)
Time Spent: 26h 20m (was: 26h 10m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 26h 20m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428775&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428775
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 29/Apr/20 22:51
Start Date: 29/Apr/20 22:51
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r417654992
##
File path: .test-infra/jenkins/CommonJobProperties.groovy
##
@@ -163,6 +163,10 @@ class CommonJobProperties {
// For [BEAM-4847], hardcode Xms and Xmx to reasonable values (2g/4g).
context.switches("-Dorg.gradle.jvmargs=-Xms2g")
context.switches("-Dorg.gradle.jvmargs=-Xmx4g")
+
+// Add docker-pull-licenses option to all Jenkins test to add licenses to
when build docker images.
Review comment:
Wording suggestion:
// Include dependency licences when we build docker images on Jenkins, see:
https://s.apache.org/zt68q.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428775)
Time Spent: 26h 10m (was: 26h)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 26h 10m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428516&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428516 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 29/Apr/20 05:07 Start Date: 29/Apr/20 05:07 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11548: URL: https://github.com/apache/beam/pull/11548#issuecomment-620990823 @tvalentyn, I changed it to pull licenses only with Jenkins test. License pulling is skipped by default. Can you please take a look? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 428516) Time Spent: 26h (was: 25h 50m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 26h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428505&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428505 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 29/Apr/20 04:03 Start Date: 29/Apr/20 04:03 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11548: URL: https://github.com/apache/beam/pull/11548#issuecomment-620977492 Run PythonDocker PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 428505) Time Spent: 25h 50m (was: 25h 40m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 25h 50m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428504&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428504 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 29/Apr/20 04:02 Start Date: 29/Apr/20 04:02 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11548: URL: https://github.com/apache/beam/pull/11548#issuecomment-620977305 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 428504) Time Spent: 25h 40m (was: 25.5h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 25h 40m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428501&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428501 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 29/Apr/20 03:48 Start Date: 29/Apr/20 03:48 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11548: URL: https://github.com/apache/beam/pull/11548#issuecomment-620974622 Run seed job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 428501) Time Spent: 25.5h (was: 25h 20m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 25.5h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428492&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428492 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 29/Apr/20 03:25 Start Date: 29/Apr/20 03:25 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11552: URL: https://github.com/apache/beam/pull/11552#issuecomment-620970095 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 428492) Time Spent: 25h 20m (was: 25h 10m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 25h 20m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428426&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428426
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 28/Apr/20 23:05
Start Date: 28/Apr/20 23:05
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r416975896
##
File path: sdks/java/container/build.gradle
##
@@ -101,16 +84,44 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
dockerfile project.file("./${dockerfileName}")
files "./build/"
+ buildArgs(['pull_licenses':
!project.rootProject.hasProperty(["no-licenses"])])
Review comment:
The purpose of checking urls was to make sure files can be pulled from
the urls when create images. If we set it to skip, no url checking process
happened for the PRs and we may see many license issues when we create release
images. Then release managers need to fix the issues. If we add the checking
urls for each PR, contributors should fix the issues if any and will not have
to fix them during release.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428426)
Time Spent: 25h (was: 24h 50m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 25h
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428427&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428427
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 28/Apr/20 23:05
Start Date: 28/Apr/20 23:05
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r416976074
##
File path: sdks/java/container/build.gradle
##
@@ -101,16 +84,44 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
dockerfile project.file("./${dockerfileName}")
files "./build/"
+ buildArgs(['pull_licenses':
!project.rootProject.hasProperty(["no-licenses"])])
Review comment:
There are some more discussion at the thread today, in case you haven't
seen it yet.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428427)
Time Spent: 25h 10m (was: 25h)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 25h 10m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428415&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428415
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 28/Apr/20 22:30
Start Date: 28/Apr/20 22:30
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r416962452
##
File path: sdks/java/container/build.gradle
##
@@ -101,16 +84,44 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
dockerfile project.file("./${dockerfileName}")
files "./build/"
+ buildArgs(['pull_licenses':
!project.rootProject.hasProperty(["no-licenses"])])
Review comment:
Right, so to produce lightweight images we can skip license download.
Why would we ever want to 'check' urls but not downloading the licenses?
Quoting your comment on the mailing list:
> I tried to check if urls are valid instead of pulling the files and it
> reduced only 1 min of running time. So, it's not an option here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428415)
Time Spent: 24h 50m (was: 24h 40m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 24h 50m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428414&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428414
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 28/Apr/20 22:29
Start Date: 28/Apr/20 22:29
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r416962452
##
File path: sdks/java/container/build.gradle
##
@@ -101,16 +84,44 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
dockerfile project.file("./${dockerfileName}")
files "./build/"
+ buildArgs(['pull_licenses':
!project.rootProject.hasProperty(["no-licenses"])])
Review comment:
Right, so to produce lightweight images we can skip license download.
Why would we ever want to 'check' urls but not downloading the licenses? From
your comment:
```
I tried to check if urls are valid instead of pulling the files and it
reduced only 1 min of running time. So, it's not an option here.
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428414)
Time Spent: 24h 40m (was: 24.5h)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 24h 40m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428338&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428338
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 28/Apr/20 19:40
Start Date: 28/Apr/20 19:40
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r416874101
##
File path: sdks/java/container/build.gradle
##
@@ -101,16 +84,44 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
dockerfile project.file("./${dockerfileName}")
files "./build/"
+ buildArgs(['pull_licenses':
!project.rootProject.hasProperty(["no-licenses"])])
Review comment:
In short, users want to create a lightweight images, without adding
licenses.
Lightweight images are welcomed for Jenkins test as well, it reduces image
size by 85MB for Java image. More discussion can be found at
[here](https://lists.apache.org/thread.html/rff9f05e08de6adf7c39c0f4c59f97ae1a2f3602768480fe9e31e0428%40%3Cdev.beam.apache.org%3E)
Naming suggestion sounds good to me, thank you.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428338)
Time Spent: 24.5h (was: 24h 20m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 24.5h
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428332&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428332
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 28/Apr/20 19:08
Start Date: 28/Apr/20 19:08
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r416855737
##
File path: sdks/java/container/build.gradle
##
@@ -101,16 +84,44 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
dockerfile project.file("./${dockerfileName}")
files "./build/"
+ buildArgs(['pull_licenses':
!project.rootProject.hasProperty(["no-licenses"])])
Review comment:
Sounds good. Sorry if this was covered in the discussion that I didn't
read, but why would we want to check urls but not pull? Feel free to point me
to the discussion.
Naming suggestion:
licenses-in-docker-images=add
licenses-in-docker-images=skip
licenses-in-docker-images=check_urls
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428332)
Time Spent: 24h 20m (was: 24h 10m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 24h 20m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428302&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428302 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 28/Apr/20 18:12 Start Date: 28/Apr/20 18:12 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on a change in pull request #11549: URL: https://github.com/apache/beam/pull/11549#discussion_r416821826 ## File path: CHANGES.md ## @@ -122,13 +122,22 @@ conversion to beam schema options. *Remark: Schema aware is still experimental.* values as strings) into Python native types that are written to Avro (Python's date, datetime types, decimal, etc). For more information see https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#avro_conversions. +* Licenses/notices of third party dependencies are added to Java and Python SDK docker images. ([BEAM-9136](https://issues.apache.org/jira/browse/BEAM-9136)) + +Java SDK images: By default, it checks if the licenses/notices are pullable, instead of pulling the licenses/notices. In order to pull licenses, `docker-pull-licenses` variable should ba passed. + +Python SDK images: By default, it pulls licenses/notices of the third party licenses. + +If you want to create lightweight images or do not want to deal with licenses, you can pass `no-licenses` variable when build docker images to skip it. Review comment: yep, correct. The sentence is very likely to be changed. I will add WIP to the title for now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 428302) Time Spent: 24h 10m (was: 24h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 24h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428301&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428301
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 28/Apr/20 18:10
Start Date: 28/Apr/20 18:10
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r416735375
##
File path: sdks/java/container/build.gradle
##
@@ -101,16 +84,44 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
dockerfile project.file("./${dockerfileName}")
files "./build/"
+ buildArgs(['pull_licenses':
!project.rootProject.hasProperty(["no-licenses"])])
Review comment:
Currently, there are three pull licenses options for Java.
when docker-pull-licenses is set, licenses are pulled.
when no-licenses is set, all license pulling related tasks are skipped.
when no tag is passed, which is default mode, license urls are checked.
With Python, there are two options available.
with no-licenses tag, license pulling is skipped.
when no tag is passed, which is default, licenses are pulled.
Python doesn't support checking urls at the moment.
I agree above settings are confuse, and would like to simplify it and make
it consistent between docker images.
So, we will have only one tag, which is `docker-pull-licenses`.
when docker-pull-licenses = pull, licenses are pulled.
when docker-pull-licenses = skip, license related tasks are skipped.
when docker-pull-licenses = check, license urls are checked.
Default mode is docker-pull-licenses = check both for Java and Python.
We need to update the script that pull licenses for Python to support
checking urls.
Does this sound good?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428301)
Time Spent: 24h (was: 23h 50m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 24h
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428300&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428300 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 28/Apr/20 18:07 Start Date: 28/Apr/20 18:07 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11552: URL: https://github.com/apache/beam/pull/11552#issuecomment-620768056 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 428300) Time Spent: 23h 50m (was: 23h 40m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 23h 50m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428270&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428270 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 28/Apr/20 17:15 Start Date: 28/Apr/20 17:15 Worklog Time Spent: 10m Work Description: ibzib commented on a change in pull request #11549: URL: https://github.com/apache/beam/pull/11549#discussion_r416781568 ## File path: CHANGES.md ## @@ -122,13 +122,22 @@ conversion to beam schema options. *Remark: Schema aware is still experimental.* values as strings) into Python native types that are written to Avro (Python's date, datetime types, decimal, etc). For more information see https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#avro_conversions. +* Licenses/notices of third party dependencies are added to Java and Python SDK docker images. ([BEAM-9136](https://issues.apache.org/jira/browse/BEAM-9136)) + +Java SDK images: By default, it checks if the licenses/notices are pullable, instead of pulling the licenses/notices. In order to pull licenses, `docker-pull-licenses` variable should ba passed. Review comment: ```suggestion Java SDK images: By default, it checks if the licenses/notices are pullable, instead of pulling the licenses/notices. In order to pull licenses, `docker-pull-licenses` variable should be passed. ``` ## File path: CHANGES.md ## @@ -122,13 +122,22 @@ conversion to beam schema options. *Remark: Schema aware is still experimental.* values as strings) into Python native types that are written to Avro (Python's date, datetime types, decimal, etc). For more information see https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#avro_conversions. +* Licenses/notices of third party dependencies are added to Java and Python SDK docker images. ([BEAM-9136](https://issues.apache.org/jira/browse/BEAM-9136)) + +Java SDK images: By default, it checks if the licenses/notices are pullable, instead of pulling the licenses/notices. In order to pull licenses, `docker-pull-licenses` variable should ba passed. + +Python SDK images: By default, it pulls licenses/notices of the third party licenses. + +If you want to create lightweight images or do not want to deal with licenses, you can pass `no-licenses` variable when build docker images to skip it. Review comment: This depends on #11548, right? ## File path: CHANGES.md ## @@ -122,13 +122,22 @@ conversion to beam schema options. *Remark: Schema aware is still experimental.* values as strings) into Python native types that are written to Avro (Python's date, datetime types, decimal, etc). For more information see https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#avro_conversions. +* Licenses/notices of third party dependencies are added to Java and Python SDK docker images. ([BEAM-9136](https://issues.apache.org/jira/browse/BEAM-9136)) + +Java SDK images: By default, it checks if the licenses/notices are pullable, instead of pulling the licenses/notices. In order to pull licenses, `docker-pull-licenses` variable should ba passed. Review comment: Please say what "it" refers to here (ie the Docker gradle task) ## File path: CHANGES.md ## @@ -122,13 +122,22 @@ conversion to beam schema options. *Remark: Schema aware is still experimental.* values as strings) into Python native types that are written to Avro (Python's date, datetime types, decimal, etc). For more information see https://cloud.google.com/bigquery/docs/loading-data-cloud-storage-avro#avro_conversions. +* Licenses/notices of third party dependencies are added to Java and Python SDK docker images. ([BEAM-9136](https://issues.apache.org/jira/browse/BEAM-9136)) + +Java SDK images: By default, it checks if the licenses/notices are pullable, instead of pulling the licenses/notices. In order to pull licenses, `docker-pull-licenses` variable should ba passed. + +Python SDK images: By default, it pulls licenses/notices of the third party licenses. Review comment: ```suggestion Python SDK images: By default, it pulls licenses/notices of the third party dependencies. ``` ## File path: website/src/documentation/runtime/environments.md ## @@ -119,6 +119,10 @@ To build Beam SDK container images: ./gradlew [--file=path/to/new/Dockerfile] :sdks:python:container buildAll ``` +From 2.21.0, licenses of third party dependencies were added to docker images by default. +If you want to creat lightweight images or do not like to deal with licenses, `no-licenses` variable can be used to skip adding licenses. Review comment: ```suggestion
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428243&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428243 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 28/Apr/20 16:44 Start Date: 28/Apr/20 16:44 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11552: URL: https://github.com/apache/beam/pull/11552#issuecomment-620724611 Thanks Udi! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 428243) Time Spent: 23.5h (was: 23h 20m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 23.5h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428242&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428242 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 28/Apr/20 16:42 Start Date: 28/Apr/20 16:42 Worklog Time Spent: 10m Work Description: udim commented on pull request #11552: URL: https://github.com/apache/beam/pull/11552#issuecomment-620723553 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 428242) Time Spent: 23h 20m (was: 23h 10m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 23h 20m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428221&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428221
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 28/Apr/20 16:04
Start Date: 28/Apr/20 16:04
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r416735375
##
File path: sdks/java/container/build.gradle
##
@@ -101,16 +84,44 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
dockerfile project.file("./${dockerfileName}")
files "./build/"
+ buildArgs(['pull_licenses':
!project.rootProject.hasProperty(["no-licenses"])])
Review comment:
Currently, there are three pull licenses options for Java.
when docker-pull-licenses is set, licenses are pulled.
when no-licenses is set, all license pulling related tasks are skipped.
when no tag is passed, which is default mode, license urls are checked.
With Python, there are two options available.
with no-licenses tag, license pulling is skipped.
when no tag is passed, which is default, licenses are pulled.
Python doesn't support checking urls at the moment.
I agree above settings are confuse, and would like to simplify it and make
it consistent between docker images.
So, we will have only one tag, which is `docker-pull-licenses`.
when docker-pull-licenses = 1 or true, licenses are pulled.
when docker-pull-licenses = 0 or false, license related tasks are skipped.
when docker-pull-licenses = check, license urls are checked.
Default mode is docker-pull-licenses = check both for Java and Python.
We need to update the script that pull licenses for Python to support
checking urls.
Does this sound good?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428221)
Time Spent: 23h 10m (was: 23h)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 23h 10m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428220&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428220
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 28/Apr/20 16:02
Start Date: 28/Apr/20 16:02
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on a change in pull request
#11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r416735375
##
File path: sdks/java/container/build.gradle
##
@@ -101,16 +84,44 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
dockerfile project.file("./${dockerfileName}")
files "./build/"
+ buildArgs(['pull_licenses':
!project.rootProject.hasProperty(["no-licenses"])])
Review comment:
Currently, there are three pull licenses options for Java.
when docker-pull-licenses is set, licenses are pulled.
when no-licenses is set, all license pulling related tasks are skipped.
when no tag is passed, which is default mode, license urls are checked.
With Python, there are two options available.
with no-licenses tag, license pulling is skipped.
when no tag is passed, which is default, licenses are pulled.
I agree above settings are confuse, and would like to simplify it and make
it consistent between docker images.
So, we will have only one tag, which is docker-pull-licenses.
when docker-pull-licenses = 1 or true, licenses are pulled.
when docker-pull-licenses = 0 or false, license related tasks are skipped.
when docker-pull-licenses = check, license urls are checked.
Default mode is docker-pull-licenses = check both for Java and Python.
We need to update the script that pull licenses for Python to support
checking urls.
Does this sound good?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428220)
Time Spent: 23h (was: 22h 50m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 23h
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=428149&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-428149
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 28/Apr/20 13:39
Start Date: 28/Apr/20 13:39
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #11548:
URL: https://github.com/apache/beam/pull/11548#discussion_r416621001
##
File path: sdks/java/container/build.gradle
##
@@ -101,16 +84,44 @@ docker {
project.rootProject["docker-tag"] : project.sdk_version)
dockerfile project.file("./${dockerfileName}")
files "./build/"
+ buildArgs(['pull_licenses':
!project.rootProject.hasProperty(["no-licenses"])])
Review comment:
How about we call both flags 'pull_licenses' and add
"pull_licenses=true" to project root, with a comment what this flag does, and
how to disable it?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 428149)
Time Spent: 22h 50m (was: 22h 40m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 22h 50m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=427964&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-427964 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 28/Apr/20 04:56 Start Date: 28/Apr/20 04:56 Worklog Time Spent: 10m Work Description: Hannah-Jiang opened a new pull request #11552: URL: https://github.com/apache/beam/pull/11552 R: @udim Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[ Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=427955&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-427955 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 28/Apr/20 04:15 Start Date: 28/Apr/20 04:15 Worklog Time Spent: 10m Work Description: Hannah-Jiang opened a new pull request #11549: URL: https://github.com/apache/beam/pull/11549 R: @ibzib Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[ Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=427952&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-427952 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 28/Apr/20 03:42 Start Date: 28/Apr/20 03:42 Worklog Time Spent: 10m Work Description: Hannah-Jiang opened a new pull request #11548: URL: https://github.com/apache/beam/pull/11548 `no-licenses` tag is used to skip license related tasks when create docker images with Java and Python. R: @tvalentyn Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_P
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=422393&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-422393 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 14/Apr/20 22:30 Start Date: 14/Apr/20 22:30 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11416: [BEAM-9136] reduce third_party_dependencies size URL: https://github.com/apache/beam/pull/11416 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 422393) Time Spent: 22h 10m (was: 22h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 22h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=422343&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-422343 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 14/Apr/20 21:33 Start Date: 14/Apr/20 21:33 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #11416: [BEAM-9136] reduce third_party_dependencies size URL: https://github.com/apache/beam/pull/11416#issuecomment-613692624 Run Java PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 422343) Time Spent: 22h (was: 21h 50m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 22h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=422215&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-422215 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 14/Apr/20 18:24 Start Date: 14/Apr/20 18:24 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #11416: [BEAM-9136] reduce third_party_dependencies size URL: https://github.com/apache/beam/pull/11416#issuecomment-613604582 R: @aaltay This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 422215) Time Spent: 21h 50m (was: 21h 40m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 21h 50m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=422214&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-422214 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 14/Apr/20 18:22 Start Date: 14/Apr/20 18:22 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11416: [BEAM-9136] reduce third_party_dependencies size URL: https://github.com/apache/beam/pull/11416 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunn
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=421479&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-421479 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 13/Apr/20 17:54 Start Date: 13/Apr/20 17:54 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #11243: [BEAM-9136]Add licenses for dependencies for Java URL: https://github.com/apache/beam/pull/11243#issuecomment-613013312 Thanks @alanmyrvold. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 421479) Time Spent: 21h 20m (was: 21h 10m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 21h 20m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=421480&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-421480 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 13/Apr/20 17:54 Start Date: 13/Apr/20 17:54 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11243: [BEAM-9136]Add licenses for dependencies for Java URL: https://github.com/apache/beam/pull/11243 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 421480) Time Spent: 21.5h (was: 21h 20m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 21.5h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=420434&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-420434 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 10/Apr/20 19:20 Start Date: 10/Apr/20 19:20 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #11243: [BEAM-9136]Add licenses for dependencies for Java URL: https://github.com/apache/beam/pull/11243#issuecomment-612180179 Thanks @alanmyrvold for reviewing it. I added csv part and replied to your comments. Are these all resolved now or are there something I need to improve/fix? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 420434) Time Spent: 21h 10m (was: 21h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 21h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=419867&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-419867 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 09/Apr/20 23:03 Start Date: 09/Apr/20 23:03 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11243: [BEAM-9136]Add licenses for dependencies for Java URL: https://github.com/apache/beam/pull/11243#discussion_r406523425 ## File path: sdks/java/container/license_scripts/pull_licenses_java.py ## @@ -0,0 +1,186 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +""" +A script to pull licenses/notices/source code for Java dependencies. Review comment: This is done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 419867) Time Spent: 21h (was: 20h 50m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 21h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=419814&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-419814 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 09/Apr/20 22:01 Start Date: 09/Apr/20 22:01 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11243: [BEAM-9136]Add licenses for dependencies for Java URL: https://github.com/apache/beam/pull/11243#discussion_r406502531 ## File path: sdks/java/container/license_scripts/license_script.sh ## @@ -0,0 +1,37 @@ + # Licensed to the Apache Software Foundation (ASF) under one + # or more contributor license agreements. See the NOTICE file + # distributed with this work for additional information + # regarding copyright ownership. The ASF licenses this file + # to you under the Apache License, Version 2.0 (the + # License); you may not use this file except in compliance + # with the License. You may obtain a copy of the License at + # + # http://www.apache.org/licenses/LICENSE-2.0 + # + # Unless required by applicable law or agreed to in writing, software + # distributed under the License is distributed on an AS IS BASIS, + # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + # See the License for the specific language governing permissions and + # limitations under the License. + +set -e +# reports are generated at ~/beam/java_third_party_licenses +./gradlew generateLicenseReport --rerun-tasks Review comment: This is same as answered below. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 419814) Time Spent: 20h 50m (was: 20h 40m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 20h 50m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=419813&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-419813
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 09/Apr/20 22:01
Start Date: 09/Apr/20 22:01
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on pull request #11243:
[BEAM-9136]Add licenses for dependencies for Java
URL: https://github.com/apache/beam/pull/11243#discussion_r406502326
##
File path: sdks/java/container/build.gradle
##
@@ -16,7 +16,11 @@
* limitations under the License.
*/
-plugins { id 'org.apache.beam.module' }
+plugins {
+ id 'org.apache.beam.module'
+ id 'com.github.jk1.dependency-license-report' version '1.13'
Review comment:
If I understand it correctly, dependencies without licenses/notice included
in jar are *considered* as the missing dependencies at first. However, the
reality is dependency scan is correct, it only missing licenses.
`com.github.jk1.dependency-license-report` generates `index.json` file as well,
which provides a list of all dependencies. I used this file to go through all
dependencies and pull licenses/notices if they are not pulled automatically.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 419813)
Time Spent: 20h 40m (was: 20.5h)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 20h 40m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=419808&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-419808 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 09/Apr/20 21:45 Start Date: 09/Apr/20 21:45 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11243: [BEAM-9136]Add licenses for dependencies for Java URL: https://github.com/apache/beam/pull/11243#discussion_r406495748 ## File path: sdks/java/container/license_scripts/pull_licenses_java.py ## @@ -0,0 +1,186 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +""" +A script to pull licenses/notices/source code for Java dependencies. Review comment: We can do that by adding some more code to the scripts. I will add it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 419808) Time Spent: 20.5h (was: 20h 20m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 20.5h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=419806&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-419806
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 09/Apr/20 21:43
Start Date: 09/Apr/20 21:43
Worklog Time Spent: 10m
Work Description: Hannah-Jiang commented on pull request #11243:
[BEAM-9136]Add licenses for dependencies for Java
URL: https://github.com/apache/beam/pull/11243#discussion_r406494846
##
File path: sdks/java/container/build.gradle
##
@@ -68,6 +73,28 @@ golang {
}
}
+// this is a workaround to call generateLicenseReport task from project root
directory.
+// generateLicenseReport does not return correct dependency list when not
called from the root.
+task generateThirdPartyLicenses(type: Exec) {
+ workingDir project.rootProject.projectDir
+ commandLine './sdks/java/container/license_scripts/license_script.sh'
Review comment:
When the `generateLicenseReport` task is triggered by `dependsOn`, the task
is not running at the project root directory regardless how to set project
scope, and dependency list is not correct. This is a workaround to execute the
`generateLicenseReport` from root directory and get correct dependency list.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 419806)
Time Spent: 20h 20m (was: 20h 10m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 20h 20m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=419801&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-419801
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 09/Apr/20 21:37
Start Date: 09/Apr/20 21:37
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on pull request #11243:
[BEAM-9136]Add licenses for dependencies for Java
URL: https://github.com/apache/beam/pull/11243#discussion_r406487569
##
File path: sdks/java/container/build.gradle
##
@@ -16,7 +16,11 @@
* limitations under the License.
*/
-plugins { id 'org.apache.beam.module' }
+plugins {
+ id 'org.apache.beam.module'
+ id 'com.github.jk1.dependency-license-report' version '1.13'
Review comment:
Kenn mentioned hierynomus/license-gradle-plugin found more accurate license
dependencies; does that need to be added?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 419801)
Time Spent: 19h 50m (was: 19h 40m)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 19h 50m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[
https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=419804&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-419804
]
ASF GitHub Bot logged work on BEAM-9136:
Author: ASF GitHub Bot
Created on: 09/Apr/20 21:37
Start Date: 09/Apr/20 21:37
Worklog Time Spent: 10m
Work Description: alanmyrvold commented on pull request #11243:
[BEAM-9136]Add licenses for dependencies for Java
URL: https://github.com/apache/beam/pull/11243#discussion_r40641
##
File path: sdks/java/container/build.gradle
##
@@ -68,6 +73,28 @@ golang {
}
}
+// this is a workaround to call generateLicenseReport task from project root
directory.
+// generateLicenseReport does not return correct dependency list when not
called from the root.
+task generateThirdPartyLicenses(type: Exec) {
+ workingDir project.rootProject.projectDir
+ commandLine './sdks/java/container/license_scripts/license_script.sh'
Review comment:
Why does the gradle task need to start a shell script that runs gradle? Can
the gradle task be a dependency instead?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 419804)
Time Spent: 20h 10m (was: 20h)
> Add LICENSES and NOTICES to docker images
> -
>
> Key: BEAM-9136
> URL: https://issues.apache.org/jira/browse/BEAM-9136
> Project: Beam
> Issue Type: Task
> Components: build-system
>Reporter: Hannah Jiang
>Assignee: Hannah Jiang
>Priority: Major
> Fix For: 2.21.0
>
> Time Spent: 20h 10m
> Remaining Estimate: 0h
>
> Scan dependencies and add licenses and notices of the dependencies to SDK
> docker images.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=419803&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-419803 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 09/Apr/20 21:37 Start Date: 09/Apr/20 21:37 Worklog Time Spent: 10m Work Description: alanmyrvold commented on pull request #11243: [BEAM-9136]Add licenses for dependencies for Java URL: https://github.com/apache/beam/pull/11243#discussion_r406488322 ## File path: sdks/java/container/license_scripts/license_script.sh ## @@ -0,0 +1,37 @@ + # Licensed to the Apache Software Foundation (ASF) under one + # or more contributor license agreements. See the NOTICE file + # distributed with this work for additional information + # regarding copyright ownership. The ASF licenses this file + # to you under the Apache License, Version 2.0 (the + # License); you may not use this file except in compliance + # with the License. You may obtain a copy of the License at + # + # http://www.apache.org/licenses/LICENSE-2.0 + # + # Unless required by applicable law or agreed to in writing, software + # distributed under the License is distributed on an AS IS BASIS, + # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + # See the License for the specific language governing permissions and + # limitations under the License. + +set -e +# reports are generated at ~/beam/java_third_party_licenses +./gradlew generateLicenseReport --rerun-tasks Review comment: Why does the gradle task need to run gradle? Can it be a dependency above? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 419803) Time Spent: 20h 10m (was: 20h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 20h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=419802&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-419802 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 09/Apr/20 21:37 Start Date: 09/Apr/20 21:37 Worklog Time Spent: 10m Work Description: alanmyrvold commented on pull request #11243: [BEAM-9136]Add licenses for dependencies for Java URL: https://github.com/apache/beam/pull/11243#discussion_r406491805 ## File path: sdks/java/container/license_scripts/pull_licenses_java.py ## @@ -0,0 +1,186 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +#http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +""" +A script to pull licenses/notices/source code for Java dependencies. Review comment: In addition to pulling the licenses, can this script generate a CSV of Library name, link to license, license name (like Apache, LGPL, etc), and whether source is downloaded/included? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 419802) Time Spent: 20h (was: 19h 50m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 20h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=418860&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-418860 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 08/Apr/20 20:25 Start Date: 08/Apr/20 20:25 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #11243: [BEAM-9136]Add licenses for dependencies for Java URL: https://github.com/apache/beam/pull/11243#issuecomment-611175578 @alanmyrvold, the PR is updated with 1. Pull missing licenses/notices from URL 2. Pull source code of packages if the license is one of ['CDDL', 'LGPL', 'GLP', 'MPL' ]. Could you please take a look? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 418860) Time Spent: 19h 40m (was: 19.5h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 19h 40m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=418061&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-418061 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 07/Apr/20 23:04 Start Date: 07/Apr/20 23:04 Worklog Time Spent: 10m Work Description: chamikaramj commented on issue #11307: [BEAM-9136] (test remove behavior) remove nose URL: https://github.com/apache/beam/pull/11307#issuecomment-610662424 Created https://issues.apache.org/jira/browse/BEAM-9719 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 418061) Time Spent: 19.5h (was: 19h 20m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 19.5h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=418004&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-418004 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 07/Apr/20 21:53 Start Date: 07/Apr/20 21:53 Worklog Time Spent: 10m Work Description: chamikaramj commented on issue #11307: [BEAM-9136] (test remove behavior) remove nose URL: https://github.com/apache/beam/pull/11307#issuecomment-610639233 cc: @ihji should we update these tests to include the dependency ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 418004) Time Spent: 19h 20m (was: 19h 10m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 19h 20m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=418003&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-418003 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 07/Apr/20 21:50 Start Date: 07/Apr/20 21:50 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #11307: [BEAM-9136] (test remove behavior) remove nose URL: https://github.com/apache/beam/pull/11307#issuecomment-610638131 We can revert it to unblock the tests, but this is strange - nose is a test-only dependency; shouldn't the tests install test dependencies if they need them? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 418003) Time Spent: 19h 10m (was: 19h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 19h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=417983&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417983 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 07/Apr/20 21:14 Start Date: 07/Apr/20 21:14 Worklog Time Spent: 10m Work Description: chamikaramj commented on issue #11307: [BEAM-9136] (test remove behavior) remove nose URL: https://github.com/apache/beam/pull/11307#issuecomment-610624192 Looks like cross-language post-commit tests started failing due to this. https://builds.apache.org/view/A-D/view/Beam/view/PostCommit/job/beam_PostCommit_XVR_Spark/ https://builds.apache.org/view/A-D/view/Beam/view/PostCommit/job/beam_PostCommit_XVR_Spark/549/ 13:01:16 ImportError: No module named nose.plugins.attrib 13:01:16 13:01:16 at org.apache.beam.runners.fnexecution.control.FnApiControlClient$ResponseStreamObserver.onNext(FnApiControlClient.java:160) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 417983) Time Spent: 19h (was: 18h 50m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 19h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=417054&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-417054 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 06/Apr/20 19:26 Start Date: 06/Apr/20 19:26 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11307: [BEAM-9136] (test remove behavior) remove nose URL: https://github.com/apache/beam/pull/11307 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 417054) Time Spent: 18h 50m (was: 18h 40m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 18h 50m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=416783&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-416783 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 06/Apr/20 17:25 Start Date: 06/Apr/20 17:25 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #11307: [BEAM-9136] (test remove behavior) remove nose URL: https://github.com/apache/beam/pull/11307#issuecomment-609929129 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 416783) Time Spent: 18h 40m (was: 18.5h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 18h 40m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=415817&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415817 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 03/Apr/20 23:10 Start Date: 03/Apr/20 23:10 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #11307: [BEAM-9136] (test remove behavior) remove nose URL: https://github.com/apache/beam/pull/11307#issuecomment-608785737 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 415817) Time Spent: 18.5h (was: 18h 20m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 18.5h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=415677&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415677 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 03/Apr/20 19:13 Start Date: 03/Apr/20 19:13 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11307: [BEAM-9136] (test remove behavior) remove nose URL: https://github.com/apache/beam/pull/11307 **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=415309&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415309 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 03/Apr/20 03:47 Start Date: 03/Apr/20 03:47 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on pull request #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 415309) Time Spent: 18h 10m (was: 18h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 18h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=415307&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415307 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 03/Apr/20 03:35 Start Date: 03/Apr/20 03:35 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#issuecomment-608211064 > LGTM for last commit as well. Please squash commits before or during the merge. Thanks! Thank you Valentyn, your review helped a lot! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 415307) Time Spent: 18h (was: 17h 50m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 18h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=415300&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415300 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 03/Apr/20 03:15 Start Date: 03/Apr/20 03:15 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#issuecomment-608206356 > CommunityMetrics job fails with dockerpycreds.errors.InitializationError: docker-credential-gcloud not installed or not available in PATH, which doesn't seem related to this PR. cc: @yifanzou This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 415300) Time Spent: 17h 50m (was: 17h 40m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 17h 50m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=415299&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415299 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 03/Apr/20 03:14 Start Date: 03/Apr/20 03:14 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#issuecomment-608206132 CommunityMetrics job fails with `dockerpycreds.errors.InitializationError: docker-credential-gcloud not installed or not available in PATH`, which doesn't seem related to this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 415299) Time Spent: 17h 40m (was: 17.5h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 17h 40m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=415292&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415292 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 03/Apr/20 02:50 Start Date: 03/Apr/20 02:50 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#issuecomment-608200645 Run Python DockerBuild PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 415292) Time Spent: 17.5h (was: 17h 20m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 17.5h > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=415290&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415290 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 03/Apr/20 02:33 Start Date: 03/Apr/20 02:33 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#issuecomment-608196084 LGTM for last commit as well. Please squash commits before or during the merge. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 415290) Time Spent: 17h 20m (was: 17h 10m) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 17h 20m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9136) Add LICENSES and NOTICES to docker images
[ https://issues.apache.org/jira/browse/BEAM-9136?focusedWorklogId=415289&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-415289 ] ASF GitHub Bot logged work on BEAM-9136: Author: ASF GitHub Bot Created on: 03/Apr/20 02:28 Start Date: 03/Apr/20 02:28 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #11067: [BEAM-9136]Add licenses for dependencies for Python URL: https://github.com/apache/beam/pull/11067#issuecomment-608195001 Run Seed Job This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 415289) Time Spent: 17h 10m (was: 17h) > Add LICENSES and NOTICES to docker images > - > > Key: BEAM-9136 > URL: https://issues.apache.org/jira/browse/BEAM-9136 > Project: Beam > Issue Type: Task > Components: build-system >Reporter: Hannah Jiang >Assignee: Hannah Jiang >Priority: Major > Fix For: 2.21.0 > > Time Spent: 17h 10m > Remaining Estimate: 0h > > Scan dependencies and add licenses and notices of the dependencies to SDK > docker images. -- This message was sent by Atlassian Jira (v8.3.4#803005)
