[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=634559&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634559 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 05/Aug/21 13:21 Start Date: 05/Aug/21 13:21 Worklog Time Spent: 10m Work Description: tysonjh closed pull request #15285: URL: https://github.com/apache/beam/pull/15285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 634559) Time Spent: 10h 50m (was: 10h 40m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 10h 50m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=634558&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634558 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 05/Aug/21 13:21 Start Date: 05/Aug/21 13:21 Worklog Time Spent: 10m Work Description: tysonjh commented on pull request #15285: URL: https://github.com/apache/beam/pull/15285#issuecomment-893455604 Unrelated, see https://issues.apache.org/jira/browse/BEAM-12676 . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 634558) Time Spent: 10h 40m (was: 10.5h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 10h 40m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=634557&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634557 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 05/Aug/21 13:19 Start Date: 05/Aug/21 13:19 Worklog Time Spent: 10m Work Description: codecov[bot] commented on pull request #15285: URL: https://github.com/apache/beam/pull/15285#issuecomment-893454067 # [Codecov](https://codecov.io/gh/apache/beam/pull/15285?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15285](https://codecov.io/gh/apache/beam/pull/15285?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (8088e3b) into [master](https://codecov.io/gh/apache/beam/commit/291aa6c98004d3157658205dfbacd6352bf909a4?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (291aa6c) will **decrease** coverage by `0.00%`. > The diff coverage is `96.96%`. [](https://codecov.io/gh/apache/beam/pull/15285?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@Coverage Diff @@ ## master #15285 +/- ## == - Coverage 83.84% 83.83% -0.01% == Files 441 441 Lines 5972659725 -1 == - Hits5007550072 -3 - Misses 9651 9653 +2 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15285?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15285/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15285/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.60% <92.85%> (-0.07%)` | :arrow_down: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15285/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15285/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.60% <100.00%> (+0.05%)` | :arrow_up: | | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/15285/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `90.42% <0.00%> (-1.07%)` | :arrow_down: | | [...pache\_beam/runners/interactive/interactive\_beam.py](https://codecov.io/gh/apache/beam/pull/15285/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9iZWFtLnB5) | `75.53% <0.00%> (-1.07%)` | :arrow_down: | | [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15285/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Z
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=634552&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-634552 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 05/Aug/21 13:02 Start Date: 05/Aug/21 13:02 Worklog Time Spent: 10m Work Description: tysonjh opened a new pull request #15285: URL: https://github.com/apache/beam/pull/15285 Reverts apache/beam#15202 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 634552) Time Spent: 10h 20m (was: 10h 10m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 10h 20m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=631792&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631792 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 30/Jul/21 17:17 Start Date: 30/Jul/21 17:17 Worklog Time Spent: 10m Work Description: emilymye closed pull request #15251: URL: https://github.com/apache/beam/pull/15251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 631792) Time Spent: 10h 10m (was: 10h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 10h 10m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=631450&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631450 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 30/Jul/21 02:43 Start Date: 30/Jul/21 02:43 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #15251: URL: https://github.com/apache/beam/pull/15251#issuecomment-889555287 # [Codecov](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15251](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (636cac9) into [master](https://codecov.io/gh/apache/beam/commit/37d71e2529e2511f5f028632423e3297d50661d0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (37d71e2) will **increase** coverage by `0.01%`. > The diff coverage is `96.96%`. [](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@Coverage Diff @@ ## master #15251 +/- ## == + Coverage 83.83% 83.85% +0.01% == Files 441 441 Lines 5972459723 -1 == + Hits5006950078 +9 + Misses 9655 9645 -10 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.60% <92.85%> (-0.07%)` | :arrow_down: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.60% <100.00%> (+0.05%)` | :arrow_up: | | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.39% <0.00%> (+0.12%)` | :arrow_up: | | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `92.42% <0.00%> (+1.81%)` | :arrow_up: | | [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=631430&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631430 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 30/Jul/21 01:39 Start Date: 30/Jul/21 01:39 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #15251: URL: https://github.com/apache/beam/pull/15251#issuecomment-889555287 # [Codecov](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15251](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (636cac9) into [master](https://codecov.io/gh/apache/beam/commit/37d71e2529e2511f5f028632423e3297d50661d0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (37d71e2) will **increase** coverage by `0.01%`. > The diff coverage is `96.96%`. [](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@Coverage Diff @@ ## master #15251 +/- ## == + Coverage 83.83% 83.85% +0.01% == Files 441 441 Lines 5972459723 -1 == + Hits5006950078 +9 + Misses 9655 9645 -10 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.60% <92.85%> (-0.07%)` | :arrow_down: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.60% <100.00%> (+0.05%)` | :arrow_up: | | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.39% <0.00%> (+0.12%)` | :arrow_up: | | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `92.42% <0.00%> (+1.81%)` | :arrow_up: | | [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=631426&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631426 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 30/Jul/21 01:06 Start Date: 30/Jul/21 01:06 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #15251: URL: https://github.com/apache/beam/pull/15251#issuecomment-889555287 # [Codecov](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15251](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (636cac9) into [master](https://codecov.io/gh/apache/beam/commit/37d71e2529e2511f5f028632423e3297d50661d0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (37d71e2) will **increase** coverage by `0.01%`. > The diff coverage is `96.96%`. [](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@Coverage Diff @@ ## master #15251 +/- ## == + Coverage 83.83% 83.85% +0.01% == Files 441 441 Lines 5972459723 -1 == + Hits5006950078 +9 + Misses 9655 9645 -10 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.60% <92.85%> (-0.07%)` | :arrow_down: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.60% <100.00%> (+0.05%)` | :arrow_up: | | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.39% <0.00%> (+0.12%)` | :arrow_up: | | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `92.42% <0.00%> (+1.81%)` | :arrow_up: | | [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=631423&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631423 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 30/Jul/21 01:05 Start Date: 30/Jul/21 01:05 Worklog Time Spent: 10m Work Description: codecov[bot] commented on pull request #15251: URL: https://github.com/apache/beam/pull/15251#issuecomment-889555287 # [Codecov](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15251](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (636cac9) into [master](https://codecov.io/gh/apache/beam/commit/37d71e2529e2511f5f028632423e3297d50661d0?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (37d71e2) will **increase** coverage by `0.01%`. > The diff coverage is `96.96%`. [](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@Coverage Diff @@ ## master #15251 +/- ## == + Coverage 83.83% 83.85% +0.01% == Files 441 441 Lines 5972459723 -1 == + Hits5006950078 +9 + Misses 9655 9645 -10 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15251?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.60% <92.85%> (-0.07%)` | :arrow_down: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.60% <100.00%> (+0.05%)` | :arrow_up: | | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.39% <0.00%> (+0.12%)` | :arrow_up: | | [...ks/python/apache\_beam/runners/worker/data\_plane.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvZGF0YV9wbGFuZS5weQ==) | `92.42% <0.00%> (+1.81%)` | :arrow_up: | | [...hon/apache\_beam/runners/direct/test\_stream\_impl.py](https://codecov.io/gh/apache/beam/pull/15251/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kaXJlY3QvdGVzdF9zdHJlYW1faW1wbC5we
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=631419&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631419 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 30/Jul/21 00:50 Start Date: 30/Jul/21 00:50 Worklog Time Spent: 10m Work Description: emilymye commented on pull request #15251: URL: https://github.com/apache/beam/pull/15251#issuecomment-889550713 Run Python 3.7 PostCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 631419) Time Spent: 9h 20m (was: 9h 10m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 9h 20m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=631417&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-631417 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 30/Jul/21 00:43 Start Date: 30/Jul/21 00:43 Worklog Time Spent: 10m Work Description: emilymye opened a new pull request #15251: URL: https://github.com/apache/beam/pull/15251 …hrough proto transaltion. (#15202)" This reverts commit 939fa99ce943a30da46cb3d67c924d524fbf1be4. **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). `ValidatesRunner` compliance status (on master branch) Lang ULR Dataflow Flink Samza Spark Twister2 Go --- https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/badge/icon";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon";> --- Java https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/badge/icon";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon?subject=V1";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/badge/icon?subject=V1+Streaming";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon?subject=V1+Java+11";> https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2";> https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/badge/icon?subject=V2+Streaming";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon?subject=Java+8";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon?subject=Java+11";>
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=630855&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630855 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 29/Jul/21 00:18 Start Date: 29/Jul/21 00:18 Worklog Time Spent: 10m Work Description: robertwb merged pull request #15202: URL: https://github.com/apache/beam/pull/15202 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 630855) Time Spent: 9h (was: 8h 50m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 9h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=630719&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630719 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 28/Jul/21 19:13 Start Date: 28/Jul/21 19:13 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #15202: URL: https://github.com/apache/beam/pull/15202#issuecomment-887115752 # [Codecov](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15202](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (96d82a4) into [master](https://codecov.io/gh/apache/beam/commit/61a884eafd66edae7df461226d3110235f88affd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a884e) will **increase** coverage by `0.00%`. > The diff coverage is `97.05%`. [](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #15202 +/- ## === Coverage 83.83% 83.83% === Files 441 441 Lines 5970659707+1 === + Hits5005350055+2 + Misses 9653 9652-1 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.66% <94.73%> (+0.06%)` | :arrow_up: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.54% <100.00%> (-0.06%)` | :arrow_down: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.85% <0.00%> (-0.16%)` | :arrow_down: | | [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.78%
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=630715&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630715 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 28/Jul/21 19:03 Start Date: 28/Jul/21 19:03 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #15202: URL: https://github.com/apache/beam/pull/15202#issuecomment-887115752 # [Codecov](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15202](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (96d82a4) into [master](https://codecov.io/gh/apache/beam/commit/61a884eafd66edae7df461226d3110235f88affd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a884e) will **increase** coverage by `0.00%`. > The diff coverage is `97.05%`. [](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #15202 +/- ## === Coverage 83.83% 83.83% === Files 441 441 Lines 5970659707+1 === + Hits5005350055+2 + Misses 9653 9652-1 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.66% <94.73%> (+0.06%)` | :arrow_up: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.54% <100.00%> (-0.06%)` | :arrow_down: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.85% <0.00%> (-0.16%)` | :arrow_down: | | [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.78%
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=630709&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630709 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 28/Jul/21 18:43 Start Date: 28/Jul/21 18:43 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #15202: URL: https://github.com/apache/beam/pull/15202#issuecomment-888535843 Run Portable_Python PreCommit -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 630709) Time Spent: 8.5h (was: 8h 20m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 8.5h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=630699&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630699 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 28/Jul/21 18:22 Start Date: 28/Jul/21 18:22 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #15202: URL: https://github.com/apache/beam/pull/15202#issuecomment-887115752 # [Codecov](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15202](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (96d82a4) into [master](https://codecov.io/gh/apache/beam/commit/61a884eafd66edae7df461226d3110235f88affd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a884e) will **increase** coverage by `0.00%`. > The diff coverage is `97.05%`. [](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #15202 +/- ## === Coverage 83.83% 83.83% === Files 441 441 Lines 5970659707+1 === + Hits5005350055+2 + Misses 9653 9652-1 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.66% <94.73%> (+0.06%)` | :arrow_up: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.54% <100.00%> (-0.06%)` | :arrow_down: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `92.68% <0.00%> (-2.44%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.85% <0.00%> (-0.16%)` | :arrow_down: | | [...runners/interactive/display/pcoll\_visualization.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9kaXNwbGF5L3Bjb2xsX3Zpc3VhbGl6YXRpb24ucHk=) | `85.78%
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=630686&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-630686 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 28/Jul/21 17:45 Start Date: 28/Jul/21 17:45 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #15202: URL: https://github.com/apache/beam/pull/15202#issuecomment-887115752 # [Codecov](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15202](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c27cde3) into [master](https://codecov.io/gh/apache/beam/commit/61a884eafd66edae7df461226d3110235f88affd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a884e) will **increase** coverage by `0.00%`. > The diff coverage is `97.05%`. > :exclamation: Current head c27cde3 differs from pull request most recent head 96d82a4. Consider uploading reports for the commit 96d82a4 to get more accurate results [](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #15202 +/- ## === Coverage 83.83% 83.83% === Files 441 441 Lines 5970659707+1 === + Hits5005350055+2 + Misses 9653 9652-1 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.66% <94.73%> (+0.06%)` | :arrow_up: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.54% <100.00%> (-0.06%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.85% <0.00%> (-0.16%)` | :arrow_down: | | [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `88.87% <0.00%> (+0.14%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=628092&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-628092 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 27/Jul/21 00:45 Start Date: 27/Jul/21 00:45 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #15202: URL: https://github.com/apache/beam/pull/15202#issuecomment-887115752 # [Codecov](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15202](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c27cde3) into [master](https://codecov.io/gh/apache/beam/commit/61a884eafd66edae7df461226d3110235f88affd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a884e) will **increase** coverage by `0.00%`. > The diff coverage is `97.05%`. [](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #15202 +/- ## === Coverage 83.83% 83.83% === Files 441 441 Lines 5970659707+1 === + Hits5005350055+2 + Misses 9653 9652-1 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.66% <94.73%> (+0.06%)` | :arrow_up: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.54% <100.00%> (-0.06%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.85% <0.00%> (-0.16%)` | :arrow_down: | | [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `88.87% <0.00%> (+0.14%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.98% <0.00%> (+0.16%)` | :arrow_up: | -- [Continue to review full
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=628091&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-628091 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 27/Jul/21 00:36 Start Date: 27/Jul/21 00:36 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #15202: URL: https://github.com/apache/beam/pull/15202#issuecomment-887115752 # [Codecov](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15202](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c27cde3) into [master](https://codecov.io/gh/apache/beam/commit/61a884eafd66edae7df461226d3110235f88affd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a884e) will **increase** coverage by `0.00%`. > The diff coverage is `97.05%`. [](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #15202 +/- ## === Coverage 83.83% 83.83% === Files 441 441 Lines 5970659707+1 === + Hits5005350055+2 + Misses 9653 9652-1 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.66% <94.73%> (+0.06%)` | :arrow_up: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.54% <100.00%> (-0.06%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.85% <0.00%> (-0.16%)` | :arrow_down: | | [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `88.87% <0.00%> (+0.14%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.98% <0.00%> (+0.16%)` | :arrow_up: | -- [Continue to review full
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=628087&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-628087 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 27/Jul/21 00:24 Start Date: 27/Jul/21 00:24 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #15202: URL: https://github.com/apache/beam/pull/15202#issuecomment-887115752 # [Codecov](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15202](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c27cde3) into [master](https://codecov.io/gh/apache/beam/commit/61a884eafd66edae7df461226d3110235f88affd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a884e) will **increase** coverage by `0.00%`. > The diff coverage is `97.05%`. [](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #15202 +/- ## === Coverage 83.83% 83.83% === Files 441 441 Lines 5970659707+1 === + Hits5005350055+2 + Misses 9653 9652-1 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.66% <94.73%> (+0.06%)` | :arrow_up: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.54% <100.00%> (-0.06%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.85% <0.00%> (-0.16%)` | :arrow_down: | | [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `88.87% <0.00%> (+0.14%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.98% <0.00%> (+0.16%)` | :arrow_up: | -- [Continue to review full
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=628085&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-628085 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 27/Jul/21 00:19 Start Date: 27/Jul/21 00:19 Worklog Time Spent: 10m Work Description: codecov[bot] commented on pull request #15202: URL: https://github.com/apache/beam/pull/15202#issuecomment-887115752 # [Codecov](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) Report > Merging [#15202](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (c27cde3) into [master](https://codecov.io/gh/apache/beam/commit/61a884eafd66edae7df461226d3110235f88affd?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) (61a884e) will **increase** coverage by `0.00%`. > The diff coverage is `97.05%`. [](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) ```diff @@ Coverage Diff @@ ## master #15202 +/- ## === Coverage 83.83% 83.83% === Files 441 441 Lines 5970659707+1 === + Hits5005350055+2 + Misses 9653 9652-1 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/15202?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation) | Coverage Δ | | |---|---|---| | [...on/apache\_beam/runners/dataflow/dataflow\_runner.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9kYXRhZmxvdy9kYXRhZmxvd19ydW5uZXIucHk=) | `82.81% <ø> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.66% <94.73%> (+0.06%)` | :arrow_up: | | [...he\_beam/runners/interactive/pipeline\_instrument.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9waXBlbGluZV9pbnN0cnVtZW50LnB5) | `99.47% <100.00%> (ø)` | | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.54% <100.00%> (-0.06%)` | :arrow_down: | | [...ks/python/apache\_beam/runners/worker/sdk\_worker.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvc2RrX3dvcmtlci5weQ==) | `88.85% <0.00%> (-0.16%)` | :arrow_down: | | [sdks/python/apache\_beam/runners/common.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9jb21tb24ucHk=) | `88.87% <0.00%> (+0.14%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/15202/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=The+Apache+Software+Foundation#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.98% <0.00%> (+0.16%)` | :arrow_up: | -- [Continue to review full report
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[
https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=628075&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-628075
]
ASF GitHub Bot logged work on BEAM-1833:
Author: ASF GitHub Bot
Created on: 26/Jul/21 23:59
Start Date: 26/Jul/21 23:59
Worklog Time Spent: 10m
Work Description: robertwb commented on a change in pull request #15202:
URL: https://github.com/apache/beam/pull/15202#discussion_r677019161
##
File path: sdks/python/apache_beam/pipeline.py
##
@@ -670,15 +671,18 @@ def apply(
pvalueish, inputs = transform._extract_input_pvalues(pvalueish)
try:
- inputs = tuple(inputs)
- for leaf_input in inputs:
-if not isinstance(leaf_input, pvalue.PValue):
- raise TypeError
+ if not isinstance(inputs, dict):
+inputs = {str(ix): input for (ix, input) in enumerate(inputs)}
except TypeError:
Review comment:
This catches a possible failure in enumerate.
##
File path: sdks/python/apache_beam/pipeline.py
##
@@ -437,11 +438,11 @@ def visit_transform(self, transform_node):
output_replacements[transform_node].append((tag, replacement))
if replace_input:
- new_input = [
- input if not input in output_map else output_map[input]
- for input in transform_node.inputs
- ]
- input_replacements[transform_node] = new_input
+ new_inputs = {
+ tag: input if not input in output_map else output_map[input]
+ for (tag, input) in transform_node.main_inputs.items()
+ }
+ input_replacements[transform_node] = new_inputs
Review comment:
Good catch. Done.
##
File path: sdks/python/apache_beam/transforms/ptransform.py
##
@@ -262,16 +263,21 @@ def get_named_nested_pvalues(pvalueish):
else:
tagged_values = enumerate(pvalueish)
elif isinstance(pvalueish, list):
+if as_inputs:
+ yield None, pvalueish
Review comment:
This is for backwards compatibility with the "eager" mode (e.g. `[1, 2,
3] | beam.Map(lambda x: x*x)`)
##
File path: sdks/python/apache_beam/transforms/ptransform.py
##
@@ -253,7 +254,7 @@ def visit(self, node):
return self.visit_nested(node)
-def get_named_nested_pvalues(pvalueish):
+def get_named_nested_pvalues(pvalueish, as_inputs=False):
Review comment:
This is done elsewhere as needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@beam.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 628075)
Time Spent: 7h 20m (was: 7h 10m)
> Restructure Python pipeline construction to better follow the Runner API
>
>
> Key: BEAM-1833
> URL: https://issues.apache.org/jira/browse/BEAM-1833
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
>Reporter: Robert Bradshaw
>Assignee: Sam Rohde
>Priority: P2
> Fix For: 2.20.0
>
> Time Spent: 7h 20m
> Remaining Estimate: 0h
>
> The most important part is removing the runner.apply overrides, but there are
> also various other improvements (e.g. all inputs and outputs should be named).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[
https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=628061&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-628061
]
ASF GitHub Bot logged work on BEAM-1833:
Author: ASF GitHub Bot
Created on: 26/Jul/21 22:55
Start Date: 26/Jul/21 22:55
Worklog Time Spent: 10m
Work Description: yifanmai commented on a change in pull request #15202:
URL: https://github.com/apache/beam/pull/15202#discussion_r676974351
##
File path: sdks/python/apache_beam/pipeline.py
##
@@ -437,11 +438,11 @@ def visit_transform(self, transform_node):
output_replacements[transform_node].append((tag, replacement))
if replace_input:
- new_input = [
- input if not input in output_map else output_map[input]
- for input in transform_node.inputs
- ]
- input_replacements[transform_node] = new_input
+ new_inputs = {
+ tag: input if not input in output_map else output_map[input]
+ for (tag, input) in transform_node.main_inputs.items()
+ }
+ input_replacements[transform_node] = new_inputs
Review comment:
I can't leave a comment on Line 274 but the type annotation for
`input_replacements` there needs to change from `Dict[AppliedPTransform,
Sequence[Union[pvalue.PBegin, pvalue.PCollection]]]` to
`Dict[AppliedPTransform, Dict[str, Union[pvalue.PBegin, pvalue.PCollection]]]`
(or `Mapping`).
##
File path: sdks/python/apache_beam/pipeline.py
##
@@ -670,15 +671,18 @@ def apply(
pvalueish, inputs = transform._extract_input_pvalues(pvalueish)
try:
- inputs = tuple(inputs)
- for leaf_input in inputs:
-if not isinstance(leaf_input, pvalue.PValue):
- raise TypeError
+ if not isinstance(inputs, dict):
+inputs = {str(ix): input for (ix, input) in enumerate(inputs)}
except TypeError:
Review comment:
Delete the `except` branch; it is no longer needed because the `PValue`
check is now done below.
##
File path: sdks/python/apache_beam/transforms/ptransform.py
##
@@ -253,7 +254,7 @@ def visit(self, node):
return self.visit_nested(node)
-def get_named_nested_pvalues(pvalueish):
+def get_named_nested_pvalues(pvalueish, as_inputs=False):
Review comment:
In the invocation of `get_named_nested_pvalues()` in `pipeline.py`, do
we also need to stringify the tags (so that `None` becomes `'None'`)?
##
File path: sdks/python/apache_beam/pipeline_test.py
##
@@ -972,6 +972,24 @@ def expand(self, p):
for transform_id in runner_api_proto.components.transforms:
self.assertRegex(transform_id, r'[a-zA-Z0-9-_]+')
+ def test_input_bames(self):
Review comment:
typo: `test_input_names`
##
File path: sdks/python/apache_beam/transforms/ptransform.py
##
@@ -262,16 +263,21 @@ def get_named_nested_pvalues(pvalueish):
else:
tagged_values = enumerate(pvalueish)
elif isinstance(pvalueish, list):
+if as_inputs:
+ yield None, pvalueish
Review comment:
What's the rationale for this branch i.e. yielding the whole `pvalueish`
versus enumerating it?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: github-unsubscr...@beam.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 628061)
Time Spent: 7h 10m (was: 7h)
> Restructure Python pipeline construction to better follow the Runner API
>
>
> Key: BEAM-1833
> URL: https://issues.apache.org/jira/browse/BEAM-1833
> Project: Beam
> Issue Type: Improvement
> Components: sdk-py-core
>Reporter: Robert Bradshaw
>Assignee: Sam Rohde
>Priority: P2
> Fix For: 2.20.0
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> The most important part is removing the runner.apply overrides, but there are
> also various other improvements (e.g. all inputs and outputs should be named).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=626375&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626375 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 22/Jul/21 00:00 Start Date: 22/Jul/21 00:00 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #15202: URL: https://github.com/apache/beam/pull/15202#issuecomment-884572140 R: @yifanmai -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 626375) Time Spent: 7h (was: 6h 50m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 7h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=626374&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-626374 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 22/Jul/21 00:00 Start Date: 22/Jul/21 00:00 Worklog Time Spent: 10m Work Description: robertwb opened a new pull request #15202: URL: https://github.com/apache/beam/pull/15202 Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). `ValidatesRunner` compliance status (on master branch) Lang ULR Dataflow Flink Samza Spark Twister2 Go --- https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/badge/icon";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/badge/icon";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Samza/lastCompletedBuild/badge/icon";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/badge/icon";> --- Java https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_ULR/lastCompletedBuild/badge/icon";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/badge/icon?subject=V1";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Streaming/lastCompletedBuild/badge/icon?subject=V1+Streaming";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/badge/icon?subject=V1+Java+11";> https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/badge/icon?subject=V2";> https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2_Streaming/lastCompletedBuild/badge/icon?subject=V2+Streaming";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/badge/icon?subject=Java+8";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/badge/icon?subject=Java+11";> https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/";> https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Fli
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=532779&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532779 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 07/Jan/21 23:48 Start Date: 07/Jan/21 23:48 Worklog Time Spent: 10m Work Description: robertwb merged pull request #13672: URL: https://github.com/apache/beam/pull/13672 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532779) Time Spent: 6h 40m (was: 6.5h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 6h 40m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=532769&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532769 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 07/Jan/21 23:28 Start Date: 07/Jan/21 23:28 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755025462 # [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=h1) Report > Merging [#13672](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=desc) (7e8fa4e) into [master](https://codecov.io/gh/apache/beam/commit/e12a0d84b2cd0fc561e89796cedbce141b3b4359?el=desc) (e12a0d8) will **increase** coverage by `0.01%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master #13672 +/- ## == + Coverage 82.73% 82.75% +0.01% == Files 466 466 Lines 5751857520 +2 == + Hits4758947598 +9 + Misses 9929 9922 -7 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.98% <100.00%> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.11% <100.00%> (+0.78%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.06% <100.00%> (+0.10%)` | :arrow_up: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `88.09% <0.00%> (-2.39%)` | :arrow_down: | | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.81% <0.00%> (-0.27%)` | :arrow_down: | | [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.84% <0.00%> (+0.17%)` | :arrow_up: | | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.83% <0.00%> (+0.38%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=footer). Last update [9f9dcd6...7e8fa4e](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532769) Time Spent: 6.5h (was: 6h 20m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 6.5h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvement
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=532761&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532761 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 07/Jan/21 23:06 Start Date: 07/Jan/21 23:06 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755025462 # [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=h1) Report > Merging [#13672](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=desc) (7e8fa4e) into [master](https://codecov.io/gh/apache/beam/commit/e12a0d84b2cd0fc561e89796cedbce141b3b4359?el=desc) (e12a0d8) will **increase** coverage by `0.01%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master #13672 +/- ## == + Coverage 82.73% 82.75% +0.01% == Files 466 466 Lines 5751857520 +2 == + Hits4758947598 +9 + Misses 9929 9922 -7 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.98% <100.00%> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.11% <100.00%> (+0.78%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.06% <100.00%> (+0.10%)` | :arrow_up: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `88.09% <0.00%> (-2.39%)` | :arrow_down: | | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.81% <0.00%> (-0.27%)` | :arrow_down: | | [sdks/python/apache\_beam/transforms/util.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy91dGlsLnB5) | `95.84% <0.00%> (+0.17%)` | :arrow_up: | | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.83% <0.00%> (+0.38%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=footer). Last update [9f9dcd6...7e8fa4e](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532761) Time Spent: 6h 20m (was: 6h 10m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 6h 20m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improve
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=532745&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532745 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 07/Jan/21 22:36 Start Date: 07/Jan/21 22:36 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-756429687 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532745) Time Spent: 6h 10m (was: 6h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 6h 10m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=532710&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532710 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 07/Jan/21 20:45 Start Date: 07/Jan/21 20:45 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755025462 # [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=h1) Report > Merging [#13672](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=desc) (27ac727) into [master](https://codecov.io/gh/apache/beam/commit/e12a0d84b2cd0fc561e89796cedbce141b3b4359?el=desc) (e12a0d8) will **increase** coverage by `0.00%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #13672 +/- ## === Coverage 82.73% 82.74% === Files 466 466 Lines 5751857520+2 === + Hits4758947596+7 + Misses 9929 9924-5 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.98% <100.00%> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.11% <100.00%> (+0.78%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.06% <100.00%> (+0.10%)` | :arrow_up: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `88.09% <0.00%> (-2.39%)` | :arrow_down: | | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `86.45% <0.00%> (-1.05%)` | :arrow_down: | | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.81% <0.00%> (-0.27%)` | :arrow_down: | | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.70% <0.00%> (+0.25%)` | :arrow_up: | | [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `98.24% <0.00%> (+1.75%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=footer). Last update [9f9dcd6...7e8fa4e](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532710) Time Spent: 6h (was: 5h 50m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assign
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=532701&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532701 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 07/Jan/21 20:00 Start Date: 07/Jan/21 20:00 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755025462 # [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=h1) Report > Merging [#13672](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=desc) (27ac727) into [master](https://codecov.io/gh/apache/beam/commit/e12a0d84b2cd0fc561e89796cedbce141b3b4359?el=desc) (e12a0d8) will **increase** coverage by `0.00%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #13672 +/- ## === Coverage 82.73% 82.74% === Files 466 466 Lines 5751857520+2 === + Hits4758947596+7 + Misses 9929 9924-5 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.98% <100.00%> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.11% <100.00%> (+0.78%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.06% <100.00%> (+0.10%)` | :arrow_up: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `88.09% <0.00%> (-2.39%)` | :arrow_down: | | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `86.45% <0.00%> (-1.05%)` | :arrow_down: | | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.81% <0.00%> (-0.27%)` | :arrow_down: | | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.70% <0.00%> (+0.25%)` | :arrow_up: | | [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `98.24% <0.00%> (+1.75%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=footer). Last update [9f9dcd6...27ac727](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532701) Time Spent: 5h 50m (was: 5h 40m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >As
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=532690&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532690 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 07/Jan/21 19:46 Start Date: 07/Jan/21 19:46 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755025462 # [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=h1) Report > Merging [#13672](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=desc) (27ac727) into [master](https://codecov.io/gh/apache/beam/commit/e12a0d84b2cd0fc561e89796cedbce141b3b4359?el=desc) (e12a0d8) will **increase** coverage by `0.00%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #13672 +/- ## === Coverage 82.73% 82.74% === Files 466 466 Lines 5751857520+2 === + Hits4758947596+7 + Misses 9929 9924-5 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.98% <100.00%> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.11% <100.00%> (+0.78%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.06% <100.00%> (+0.10%)` | :arrow_up: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `88.09% <0.00%> (-2.39%)` | :arrow_down: | | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `86.45% <0.00%> (-1.05%)` | :arrow_down: | | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.81% <0.00%> (-0.27%)` | :arrow_down: | | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.70% <0.00%> (+0.25%)` | :arrow_up: | | [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `98.24% <0.00%> (+1.75%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=footer). Last update [9f9dcd6...27ac727](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532690) Time Spent: 5h 40m (was: 5.5h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assi
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=532688&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532688 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 07/Jan/21 19:38 Start Date: 07/Jan/21 19:38 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755025462 # [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=h1) Report > Merging [#13672](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=desc) (27ac727) into [master](https://codecov.io/gh/apache/beam/commit/e12a0d84b2cd0fc561e89796cedbce141b3b4359?el=desc) (e12a0d8) will **increase** coverage by `0.00%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) ```diff @@ Coverage Diff @@ ## master #13672 +/- ## === Coverage 82.73% 82.74% === Files 466 466 Lines 5751857520+2 === + Hits4758947596+7 + Misses 9929 9924-5 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.98% <100.00%> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.11% <100.00%> (+0.78%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.06% <100.00%> (+0.10%)` | :arrow_up: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `88.09% <0.00%> (-2.39%)` | :arrow_down: | | [sdks/python/apache\_beam/internal/metrics/metric.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW50ZXJuYWwvbWV0cmljcy9tZXRyaWMucHk=) | `86.45% <0.00%> (-1.05%)` | :arrow_down: | | [sdks/python/apache\_beam/io/iobase.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vaW8vaW9iYXNlLnB5) | `84.81% <0.00%> (-0.27%)` | :arrow_down: | | [...hon/apache\_beam/runners/worker/bundle\_processor.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy93b3JrZXIvYnVuZGxlX3Byb2Nlc3Nvci5weQ==) | `93.70% <0.00%> (+0.25%)` | :arrow_up: | | [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `98.24% <0.00%> (+1.75%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=footer). Last update [9f9dcd6...27ac727](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532688) Time Spent: 5.5h (was: 5h 20m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assi
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=532681&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532681 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 07/Jan/21 19:23 Start Date: 07/Jan/21 19:23 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755025462 # [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=h1) Report > Merging [#13672](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=desc) (9aefb8a) into [master](https://codecov.io/gh/apache/beam/commit/e12a0d84b2cd0fc561e89796cedbce141b3b4359?el=desc) (e12a0d8) will **increase** coverage by `0.01%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master #13672 +/- ## == + Coverage 82.73% 82.75% +0.01% == Files 466 466 Lines 5751857521 +3 == + Hits4758947601 +12 + Misses 9929 9920 -9 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.98% <100.00%> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.13% <100.00%> (+0.80%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.06% <100.00%> (+0.10%)` | :arrow_up: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `88.09% <0.00%> (-2.39%)` | :arrow_down: | | [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `98.24% <0.00%> (+1.75%)` | :arrow_up: | | [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `92.72% <0.00%> (+1.81%)` | :arrow_up: | | [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `93.18% <0.00%> (+4.54%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=footer). Last update [9f9dcd6...27ac727](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532681) Time Spent: 5h 20m (was: 5h 10m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 5h 20m > Remaining Estimate: 0h > > The most important part is remo
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=532007&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532007 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 06/Jan/21 17:31 Start Date: 06/Jan/21 17:31 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755025462 # [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=h1) Report > Merging [#13672](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=desc) (9aefb8a) into [master](https://codecov.io/gh/apache/beam/commit/e12a0d84b2cd0fc561e89796cedbce141b3b4359?el=desc) (e12a0d8) will **increase** coverage by `0.01%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master #13672 +/- ## == + Coverage 82.73% 82.75% +0.01% == Files 466 466 Lines 5751857521 +3 == + Hits4758947601 +12 + Misses 9929 9920 -9 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.98% <100.00%> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.13% <100.00%> (+0.80%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.06% <100.00%> (+0.10%)` | :arrow_up: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `88.09% <0.00%> (-2.39%)` | :arrow_down: | | [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `98.24% <0.00%> (+1.75%)` | :arrow_up: | | [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `92.72% <0.00%> (+1.81%)` | :arrow_up: | | [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `93.18% <0.00%> (+4.54%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=footer). Last update [9f9dcd6...93c2bd8](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532007) Time Spent: 5h 10m (was: 5h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 5h 10m > Remaining Estimate: 0h > > The most important part is removing
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=532005&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-532005 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 06/Jan/21 17:30 Start Date: 06/Jan/21 17:30 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755444700 R: @boyuanzz This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 532005) Time Spent: 5h (was: 4h 50m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 5h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=531632&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531632 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 06/Jan/21 02:20 Start Date: 06/Jan/21 02:20 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755025462 # [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=h1) Report > Merging [#13672](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=desc) (9aefb8a) into [master](https://codecov.io/gh/apache/beam/commit/e12a0d84b2cd0fc561e89796cedbce141b3b4359?el=desc) (e12a0d8) will **increase** coverage by `0.01%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master #13672 +/- ## == + Coverage 82.73% 82.75% +0.01% == Files 466 466 Lines 5751857521 +3 == + Hits4758947601 +12 + Misses 9929 9920 -9 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.98% <100.00%> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.13% <100.00%> (+0.80%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.06% <100.00%> (+0.10%)` | :arrow_up: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `88.09% <0.00%> (-2.39%)` | :arrow_down: | | [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `98.24% <0.00%> (+1.75%)` | :arrow_up: | | [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `92.72% <0.00%> (+1.81%)` | :arrow_up: | | [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `93.18% <0.00%> (+4.54%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=footer). Last update [9f9dcd6...9aefb8a](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 531632) Time Spent: 4h 50m (was: 4h 40m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 4h 50m > Remaining Estimate: 0h > > The most important part is remo
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=531626&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531626 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 06/Jan/21 02:02 Start Date: 06/Jan/21 02:02 Worklog Time Spent: 10m Work Description: codecov[bot] edited a comment on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755025462 # [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=h1) Report > Merging [#13672](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=desc) (9aefb8a) into [master](https://codecov.io/gh/apache/beam/commit/e12a0d84b2cd0fc561e89796cedbce141b3b4359?el=desc) (e12a0d8) will **increase** coverage by `0.01%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master #13672 +/- ## == + Coverage 82.73% 82.75% +0.01% == Files 466 466 Lines 5751857521 +3 == + Hits4758947601 +12 + Misses 9929 9920 -9 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.98% <100.00%> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.13% <100.00%> (+0.80%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.06% <100.00%> (+0.10%)` | :arrow_up: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `88.09% <0.00%> (-2.39%)` | :arrow_down: | | [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `98.24% <0.00%> (+1.75%)` | :arrow_up: | | [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `92.72% <0.00%> (+1.81%)` | :arrow_up: | | [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `93.18% <0.00%> (+4.54%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=footer). Last update [9f9dcd6...9aefb8a](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 531626) Time Spent: 4h 40m (was: 4.5h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > The most important part is removi
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=531624&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531624 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 06/Jan/21 01:57 Start Date: 06/Jan/21 01:57 Worklog Time Spent: 10m Work Description: codecov[bot] commented on pull request #13672: URL: https://github.com/apache/beam/pull/13672#issuecomment-755025462 # [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=h1) Report > Merging [#13672](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=desc) (9aefb8a) into [master](https://codecov.io/gh/apache/beam/commit/e12a0d84b2cd0fc561e89796cedbce141b3b4359?el=desc) (e12a0d8) will **increase** coverage by `0.01%`. > The diff coverage is `100.00%`. [](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) ```diff @@Coverage Diff @@ ## master #13672 +/- ## == + Coverage 82.73% 82.75% +0.01% == Files 466 466 Lines 5751857521 +3 == + Hits4758947601 +12 + Misses 9929 9920 -9 ``` | [Impacted Files](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=tree) | Coverage Δ | | |---|---|---| | [sdks/python/apache\_beam/dataframe/frames.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vZGF0YWZyYW1lL2ZyYW1lcy5weQ==) | `91.98% <100.00%> (ø)` | | | [sdks/python/apache\_beam/pipeline.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcGlwZWxpbmUucHk=) | `91.13% <100.00%> (+0.80%)` | :arrow_up: | | [sdks/python/apache\_beam/transforms/ptransform.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wdHJhbnNmb3JtLnB5) | `93.06% <100.00%> (+0.10%)` | :arrow_up: | | [sdks/python/apache\_beam/utils/interactive\_utils.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdXRpbHMvaW50ZXJhY3RpdmVfdXRpbHMucHk=) | `88.09% <0.00%> (-2.39%)` | :arrow_down: | | [.../python/apache\_beam/transforms/periodicsequence.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdHJhbnNmb3Jtcy9wZXJpb2RpY3NlcXVlbmNlLnB5) | `98.24% <0.00%> (+1.75%)` | :arrow_up: | | [...che\_beam/runners/interactive/interactive\_runner.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vcnVubmVycy9pbnRlcmFjdGl2ZS9pbnRlcmFjdGl2ZV9ydW5uZXIucHk=) | `92.72% <0.00%> (+1.81%)` | :arrow_up: | | [.../python/apache\_beam/testing/test\_stream\_service.py](https://codecov.io/gh/apache/beam/pull/13672/diff?src=pr&el=tree#diff-c2Rrcy9weXRob24vYXBhY2hlX2JlYW0vdGVzdGluZy90ZXN0X3N0cmVhbV9zZXJ2aWNlLnB5) | `93.18% <0.00%> (+4.54%)` | :arrow_up: | -- [Continue to review full report at Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=continue). > **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta) > `Δ = absolute (impact)`, `ø = not affected`, `? = missing data` > Powered by [Codecov](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=footer). Last update [9f9dcd6...9aefb8a](https://codecov.io/gh/apache/beam/pull/13672?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 531624) Time Spent: 4.5h (was: 4h 20m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: P2 > Fix For: 2.20.0 > > Time Spent: 4.5h > Remaining Estimate: 0h > > The most important part is removing the ru
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=531620&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-531620 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 06/Jan/21 01:41 Start Date: 06/Jan/21 01:41 Worklog Time Spent: 10m Work Description: robertwb opened a new pull request #13672: URL: https://github.com/apache/beam/pull/13672 This does not fully resolve BEAM-1833, but that cleanup will be best done once all runners are portable. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Dataflow | Flink | Samza | Spark | Twister2 --- | --- | --- | --- | --- | --- | --- Go | [](https://ci-beam.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | [](https://ci-beam.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) | --- Java | [](https://ci-beam.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_VR_Dataflow_V2/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Java11/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://ci-beam.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=392183&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392183 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 24/Feb/20 23:33 Start Date: 24/Feb/20 23:33 Worklog Time Spent: 10m Work Description: aaltay commented on issue #10934: [BEAM-1833] Broke some people, setting the default to have the experiment be disabled URL: https://github.com/apache/beam/pull/10934#issuecomment-590602666 LGTM. @lukecwik - Are you fine with this change? Or do you prefer a different fix? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 392183) Time Spent: 4h 10m (was: 4h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Fix For: 2.20.0 > > Time Spent: 4h 10m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=392119&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392119 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 24/Feb/20 21:59 Start Date: 24/Feb/20 21:59 Worklog Time Spent: 10m Work Description: rohdesamuel commented on issue #10934: [BEAM-1833] Broke some people, setting the default to have the experiment be disabled URL: https://github.com/apache/beam/pull/10934#issuecomment-590570107 > The intention is to fix Python SDK for 2.20 release (Wednesday), Sam do you have enough time to do that? Yep, I believe the PR is good-to-go. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 392119) Time Spent: 4h (was: 3h 50m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Fix For: 2.20.0 > > Time Spent: 4h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=392118&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392118 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 24/Feb/20 21:58 Start Date: 24/Feb/20 21:58 Worklog Time Spent: 10m Work Description: rohdesamuel commented on issue #10934: [BEAM-1833] Broke some people, setting the default to have the experiment be disabled URL: https://github.com/apache/beam/pull/10934#issuecomment-590569784 > Could you also update the CHANGES.md? (There is a breaking change note that can be removed now.) Done, removed the breaking changes line. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 392118) Time Spent: 3h 50m (was: 3h 40m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Fix For: 2.20.0 > > Time Spent: 3h 50m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=392083&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392083 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 24/Feb/20 20:50 Start Date: 24/Feb/20 20:50 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #10934: [BEAM-1833] Broke some people, setting the default to have the experiment be disabled URL: https://github.com/apache/beam/pull/10934#issuecomment-590541865 The intention is to fix Python SDK for 2.20 release (Wednesday), Sam do you have enough time to do that? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 392083) Time Spent: 3h 40m (was: 3.5h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Fix For: 2.20.0 > > Time Spent: 3h 40m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=391876&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391876 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 24/Feb/20 18:06 Start Date: 24/Feb/20 18:06 Worklog Time Spent: 10m Work Description: aaltay commented on issue #10934: [BEAM-1833] Broke some people, setting the default to have the experiment be disabled URL: https://github.com/apache/beam/pull/10934#issuecomment-590469751 Could you also update the CHANGES.md? (There is a breaking change note that can be removed now.) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 391876) Time Spent: 3.5h (was: 3h 20m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Fix For: 2.20.0 > > Time Spent: 3.5h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=391874&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391874 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 24/Feb/20 18:02 Start Date: 24/Feb/20 18:02 Worklog Time Spent: 10m Work Description: rohdesamuel commented on issue #10934: [BEAM-1833] Broke some people, setting the default to have the experiment be disabled URL: https://github.com/apache/beam/pull/10934#issuecomment-590468146 > I agree with this as a short-term fix, but we should still keep it on our radar to fix this for real and eventually remove this option. Definitely. BEAM-9322 is on my plate when I have more time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 391874) Time Spent: 3h 20m (was: 3h 10m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Fix For: 2.20.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=391863&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-391863 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 24/Feb/20 17:54 Start Date: 24/Feb/20 17:54 Worklog Time Spent: 10m Work Description: robertwb commented on issue #10934: [BEAM-1833] Broke some people, setting the default to have the experiment be disabled URL: https://github.com/apache/beam/pull/10934#issuecomment-590464792 I agree with this as a short-term fix, but we should still keep it on our radar to fix this for real and eventually remove this option. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 391863) Time Spent: 3h 10m (was: 3h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Fix For: 2.20.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=390976&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-390976 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 21/Feb/20 23:45 Start Date: 21/Feb/20 23:45 Worklog Time Spent: 10m Work Description: rohdesamuel commented on issue #10934: [BEAM-1833] Broke some people, setting the default to have the experiment be disabled URL: https://github.com/apache/beam/pull/10934#issuecomment-589883541 > I was under the impression that you would update the pipelines to pass in the experiment and not change this default. I didn't want to burden others with modifying the PipelineOptions. And we don't have a good story on how to fix code if they are relying on the old implementation. I decided that the best would be to be disabled by default and then we enable it for InteractiveBeam. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 390976) Time Spent: 3h (was: 2h 50m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Fix For: 2.20.0 > > Time Spent: 3h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=390975&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-390975 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 21/Feb/20 23:41 Start Date: 21/Feb/20 23:41 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #10934: [BEAM-1833] Broke some people, setting the default to have the experiment be disabled URL: https://github.com/apache/beam/pull/10934#issuecomment-589882650 I was under the impression that you would update the pipelines to pass in the experiment and not change this default. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 390975) Time Spent: 2h 50m (was: 2h 40m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Fix For: 2.20.0 > > Time Spent: 2h 50m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=390972&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-390972 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 21/Feb/20 23:40 Start Date: 21/Feb/20 23:40 Worklog Time Spent: 10m Work Description: rohdesamuel commented on pull request #10934: [BEAM-1833] Broke some people, setting the default to have the experiment be disabled URL: https://github.com/apache/beam/pull/10934 Change-Id: Ie83fefc63ee03c917028b994dd2f2487f06e8109 The fix for BEAM-1833 broke some people, so I am setting the experiment to be disabled by default. For the InteractiveRunner we enable it by default. Since the runner ordinarily doesn't have access to the PipelineOptions we set the experiment through the apply() method. This is so that users won't have to set the experiment manually. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[ Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=390974&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-390974 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 21/Feb/20 23:40 Start Date: 21/Feb/20 23:40 Worklog Time Spent: 10m Work Description: rohdesamuel commented on issue #10934: [BEAM-1833] Broke some people, setting the default to have the experiment be disabled URL: https://github.com/apache/beam/pull/10934#issuecomment-589882465 R: @davidyan74 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 390974) Time Spent: 2h 40m (was: 2.5h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Fix For: 2.20.0 > > Time Spent: 2h 40m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=389190&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389190 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 18/Feb/20 23:48 Start Date: 18/Feb/20 23:48 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389190) Time Spent: 2h 20m (was: 2h 10m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 2h 20m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=389158&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389158 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 18/Feb/20 22:55 Start Date: 18/Feb/20 22:55 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#issuecomment-587945517 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389158) Time Spent: 2h 10m (was: 2h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 2h 10m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=389144&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389144 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 18/Feb/20 22:33 Start Date: 18/Feb/20 22:33 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#issuecomment-587938021 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389144) Time Spent: 2h (was: 1h 50m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=389104&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389104 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 18/Feb/20 21:24 Start Date: 18/Feb/20 21:24 Worklog Time Spent: 10m Work Description: rohdesamuel commented on pull request #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#discussion_r380943475 ## File path: sdks/python/apache_beam/pipeline.py ## @@ -314,7 +314,15 @@ def _replace_if_needed(self, original_transform_node): new_output.element_type = None Review comment: Added the assert This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389104) Time Spent: 1h 50m (was: 1h 40m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=389031&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389031 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 18/Feb/20 19:47 Start Date: 18/Feb/20 19:47 Worklog Time Spent: 10m Work Description: aaltay commented on pull request #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#discussion_r380896125 ## File path: CHANGES.md ## @@ -36,6 +36,7 @@ * ReadFromPubSub(topic=) in Python previously created a subscription under the same project as the topic. Now it will create the subscription under the project specified in pipeline_options. If the project is not specified in pipeline_options, then it will create the subscription under the same project as the topic. ([BEAM-3453](https://issues.apache.org/jira/browse/BEAM-3453)). * SpannerAccessor in Java is now package-private to reduce API surface. `SpannerConfig.connectToSpanner` has been moved to `SpannerAccessor.create`. ([BEAM-9310](https://issues.apache.org/jira/browse/BEAM-9310)). +* PCollections will now have their tags correctly propagated through the Pipeline. Users may expect the old implementation which gave PCollection output ids a monotonically increasing id. To go back to the old implementation, use the "force_generated_pcollection_output_ids" experiment. The default is the new implementation (force_generated_pcollection_output_ids=False). Review comment: With this explanation, I think it is fine to change the default in this release. FWIW, dictionaries have deterministic iteration order for py >= 3.6. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389031) Time Spent: 1h 40m (was: 1.5h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=389025&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389025 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 18/Feb/20 19:39 Start Date: 18/Feb/20 19:39 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#discussion_r380885998 ## File path: sdks/python/apache_beam/pipeline.py ## @@ -314,7 +314,15 @@ def _replace_if_needed(self, original_transform_node): new_output.element_type = None Review comment: nit: right below new_output assignment, please assert that it is one of the supported types: ``` assert isinstance(result, (dict, pvalue.PValue, pvalue.DoOutputsTuple)) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389025) Time Spent: 1.5h (was: 1h 20m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=389024&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389024 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 18/Feb/20 19:37 Start Date: 18/Feb/20 19:37 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#discussion_r380885998 ## File path: sdks/python/apache_beam/pipeline.py ## @@ -314,7 +314,15 @@ def _replace_if_needed(self, original_transform_node): new_output.element_type = None Review comment: right below new_output assignment, please assert that it is one of the supported types: ``` assert isinstance(result, (dict, pvalue.PValue, pvalue.DoOutputsTuple)) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389024) Time Spent: 1h 20m (was: 1h 10m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=389010&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389010 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 18/Feb/20 19:13 Start Date: 18/Feb/20 19:13 Worklog Time Spent: 10m Work Description: lukecwik commented on pull request #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#discussion_r380878467 ## File path: CHANGES.md ## @@ -36,6 +36,7 @@ * ReadFromPubSub(topic=) in Python previously created a subscription under the same project as the topic. Now it will create the subscription under the project specified in pipeline_options. If the project is not specified in pipeline_options, then it will create the subscription under the same project as the topic. ([BEAM-3453](https://issues.apache.org/jira/browse/BEAM-3453)). * SpannerAccessor in Java is now package-private to reduce API surface. `SpannerConfig.connectToSpanner` has been moved to `SpannerAccessor.create`. ([BEAM-9310](https://issues.apache.org/jira/browse/BEAM-9310)). +* PCollections will now have their tags correctly propagated through the Pipeline. Users may expect the old implementation which gave PCollection output ids a monotonically increasing id. To go back to the old implementation, use the "force_generated_pcollection_output_ids" experiment. The default is the new implementation (force_generated_pcollection_output_ids=False). Review comment: Sam, I would say that the existing behavior was a bug for the points that you mentioned or that the user never really cared what the tag names are (e.g. for a partition transform). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389010) Time Spent: 1h (was: 50m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=389011&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389011 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 18/Feb/20 19:13 Start Date: 18/Feb/20 19:13 Worklog Time Spent: 10m Work Description: lukecwik commented on issue #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#issuecomment-587684109 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389011) Time Spent: 1h 10m (was: 1h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=388971&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-388971 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 18/Feb/20 18:21 Start Date: 18/Feb/20 18:21 Worklog Time Spent: 10m Work Description: rohdesamuel commented on pull request #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#discussion_r380851306 ## File path: CHANGES.md ## @@ -36,6 +36,7 @@ * ReadFromPubSub(topic=) in Python previously created a subscription under the same project as the topic. Now it will create the subscription under the project specified in pipeline_options. If the project is not specified in pipeline_options, then it will create the subscription under the same project as the topic. ([BEAM-3453](https://issues.apache.org/jira/browse/BEAM-3453)). * SpannerAccessor in Java is now package-private to reduce API surface. `SpannerConfig.connectToSpanner` has been moved to `SpannerAccessor.create`. ([BEAM-9310](https://issues.apache.org/jira/browse/BEAM-9310)). +* PCollections will now have their tags correctly propagated through the Pipeline. Users may expect the old implementation which gave PCollection output ids a monotonically increasing id. To go back to the old implementation, use the "force_generated_pcollection_output_ids" experiment. The default is the new implementation (force_generated_pcollection_output_ids=False). Review comment: I looked into your question a bit more about determinism in the generated ids, and I found that since it traverses dicts, the order is non-deterministic. DoOutputsTuple manually add themselves correctly to their producer. For Tuples, I'm keeping the old implementation. For PValues, I fixed the bug to now correctly propagate the tag. Unless I'm mistaken, for the user to want to use the old implementation they are then either: - Relying on a bug (PValue) - Relying on non-deterministic behavior for generated tags (dicts) - Or using Tuples, which I didn't change. Is this okay to leave the default as-is? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 388971) Time Spent: 40m (was: 0.5h) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=388973&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-388973 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 18/Feb/20 18:21 Start Date: 18/Feb/20 18:21 Worklog Time Spent: 10m Work Description: davidyan74 commented on pull request #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#discussion_r380851549 ## File path: CHANGES.md ## @@ -36,6 +36,7 @@ * ReadFromPubSub(topic=) in Python previously created a subscription under the same project as the topic. Now it will create the subscription under the project specified in pipeline_options. If the project is not specified in pipeline_options, then it will create the subscription under the same project as the topic. ([BEAM-3453](https://issues.apache.org/jira/browse/BEAM-3453)). * SpannerAccessor in Java is now package-private to reduce API surface. `SpannerConfig.connectToSpanner` has been moved to `SpannerAccessor.create`. ([BEAM-9310](https://issues.apache.org/jira/browse/BEAM-9310)). +* PCollections will now have their tags correctly propagated through the Pipeline. Users may expect the old implementation which gave PCollection output ids a monotonically increasing id. To go back to the old implementation, use the "force_generated_pcollection_output_ids" experiment. The default is the new implementation (force_generated_pcollection_output_ids=False). Review comment: What is the best practice for rolling out experiments in Beam? In any case, we should set it as the default at least within interactive runner. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 388973) Time Spent: 50m (was: 40m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=387794&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387794 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 15/Feb/20 02:20 Start Date: 15/Feb/20 02:20 Worklog Time Spent: 10m Work Description: aaltay commented on pull request #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#discussion_r379710737 ## File path: CHANGES.md ## @@ -36,6 +36,7 @@ * ReadFromPubSub(topic=) in Python previously created a subscription under the same project as the topic. Now it will create the subscription under the project specified in pipeline_options. If the project is not specified in pipeline_options, then it will create the subscription under the same project as the topic. ([BEAM-3453](https://issues.apache.org/jira/browse/BEAM-3453)). * SpannerAccessor in Java is now package-private to reduce API surface. `SpannerConfig.connectToSpanner` has been moved to `SpannerAccessor.create`. ([BEAM-9310](https://issues.apache.org/jira/browse/BEAM-9310)). +* PCollections will now have their tags correctly propagated through the Pipeline. Users may expect the old implementation which gave PCollection output ids a monotonically increasing id. To go back to the old implementation, use the "force_generated_pcollection_output_ids" experiment. The default is the new implementation (force_generated_pcollection_output_ids=False). Review comment: Are you sure, that we would like to change the default in the first release of the change? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387794) Time Spent: 0.5h (was: 20m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=387789&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387789 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 15/Feb/20 01:49 Start Date: 15/Feb/20 01:49 Worklog Time Spent: 10m Work Description: rohdesamuel commented on issue #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860#issuecomment-586541258 R: @lukecwik R: @aaltay R: @davidyan74 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387789) Time Spent: 20m (was: 10m) > Restructure Python pipeline construction to better follow the Runner API > > > Key: BEAM-1833 > URL: https://issues.apache.org/jira/browse/BEAM-1833 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Robert Bradshaw >Assignee: Sam Rohde >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > The most important part is removing the runner.apply overrides, but there are > also various other improvements (e.g. all inputs and outputs should be named). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-1833) Restructure Python pipeline construction to better follow the Runner API
[ https://issues.apache.org/jira/browse/BEAM-1833?focusedWorklogId=386997&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386997 ] ASF GitHub Bot logged work on BEAM-1833: Author: ASF GitHub Bot Created on: 14/Feb/20 00:46 Start Date: 14/Feb/20 00:46 Worklog Time Spent: 10m Work Description: rohdesamuel commented on pull request #10860: [BEAM-1833] Fixes BEAM-1833 URL: https://github.com/apache/beam/pull/10860 * This changes the "add_output" interface to require a PCollection tag when adding an output to a PTransform. * This also changes the replacement algorithm's to propogate the PCollection tag when doing replacements. * This also moves the DirectRunner's TestStream implementation to a replacement transform. This is because the TestStream relies on getting the output_tags from the PTransform. Change-Id: Ibd80b0d25cd8cc5ff5c28e127f7313638e6664da **Please** add a meaningful description for your change here Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastComp
