[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=409953=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-409953 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 25/Mar/20 23:12 Start Date: 25/Mar/20 23:12 Worklog Time Spent: 10m Work Description: tvalentyn commented on pull request #10277: [BEAM-8575] Reenable passing VR tests. URL: https://github.com/apache/beam/pull/10277 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 409953) Time Spent: 59h 20m (was: 59h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 59h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=407646=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-407646 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 22/Mar/20 19:02 Start Date: 22/Mar/20 19:02 Worklog Time Spent: 10m Work Description: stale[bot] commented on issue #10277: [BEAM-8575] Reenable passing VR tests. URL: https://github.com/apache/beam/pull/10277#issuecomment-602256002 This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the d...@beam.apache.org list. Thank you for your contributions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 407646) Time Spent: 59h 10m (was: 59h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 59h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=397086=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-397086 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 03/Mar/20 22:32 Start Date: 03/Mar/20 22:32 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 397086) Time Spent: 59h (was: 58h 50m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 59h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=397041=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-397041 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 03/Mar/20 21:06 Start Date: 03/Mar/20 21:06 Worklog Time Spent: 10m Work Description: chamikaramj commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-594168462 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 397041) Time Spent: 58h 50m (was: 58h 40m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 58h 50m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=397039=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-397039 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 03/Mar/20 21:02 Start Date: 03/Mar/20 21:02 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#discussion_r387288561 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -482,9 +482,17 @@ def test_combining_with_accumulation_mode_and_fanout(self): trigger=AfterWatermark(early=AfterAll(AfterCount(1 | beam.CombineGlobally(sum).without_defaults().with_fanout(2)) - # The frings for DISCARDING mode is [1, 2, 3, 4, 5, 0, 0]. - firings = [1, 3, 6, 10, 15, 15, 15] - assert_that(result, equal_to(firings)) + def has_expected_values(actual): +from hamcrest.core import assert_that as hamcrest_assert +from hamcrest.library.collection import only_contains +ordered = sorted(actual) +# Early firings. +hamcrest_assert(ordered[:4], only_contains(1, 3, 6, 10)) Review comment: Done. Thank you! I forgot that `contains` here means exactly match the entire sequence. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 397039) Time Spent: 58h 40m (was: 58.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 58h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=397021=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-397021 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 03/Mar/20 20:57 Start Date: 03/Mar/20 20:57 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#discussion_r387288561 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -482,9 +482,17 @@ def test_combining_with_accumulation_mode_and_fanout(self): trigger=AfterWatermark(early=AfterAll(AfterCount(1 | beam.CombineGlobally(sum).without_defaults().with_fanout(2)) - # The frings for DISCARDING mode is [1, 2, 3, 4, 5, 0, 0]. - firings = [1, 3, 6, 10, 15, 15, 15] - assert_that(result, equal_to(firings)) + def has_expected_values(actual): +from hamcrest.core import assert_that as hamcrest_assert +from hamcrest.library.collection import only_contains +ordered = sorted(actual) +# Early firings. +hamcrest_assert(ordered[:4], only_contains(1, 3, 6, 10)) Review comment: Done. Thank you! I forgot that `contains` exactly match the entire sequence. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 397021) Time Spent: 58.5h (was: 58h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 58.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=397020=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-397020 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 03/Mar/20 20:56 Start Date: 03/Mar/20 20:56 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#discussion_r387288561 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -482,9 +482,17 @@ def test_combining_with_accumulation_mode_and_fanout(self): trigger=AfterWatermark(early=AfterAll(AfterCount(1 | beam.CombineGlobally(sum).without_defaults().with_fanout(2)) - # The frings for DISCARDING mode is [1, 2, 3, 4, 5, 0, 0]. - firings = [1, 3, 6, 10, 15, 15, 15] - assert_that(result, equal_to(firings)) + def has_expected_values(actual): +from hamcrest.core import assert_that as hamcrest_assert +from hamcrest.library.collection import only_contains +ordered = sorted(actual) +# Early firings. +hamcrest_assert(ordered[:4], only_contains(1, 3, 6, 10)) Review comment: Done. Thank you! I forgot that `contains` exactly match the entire sequence This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 397020) Time Spent: 58h 20m (was: 58h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 58h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=396959=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396959 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 03/Mar/20 19:39 Start Date: 03/Mar/20 19:39 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#discussion_r387249252 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -482,9 +482,17 @@ def test_combining_with_accumulation_mode_and_fanout(self): trigger=AfterWatermark(early=AfterAll(AfterCount(1 | beam.CombineGlobally(sum).without_defaults().with_fanout(2)) - # The frings for DISCARDING mode is [1, 2, 3, 4, 5, 0, 0]. - firings = [1, 3, 6, 10, 15, 15, 15] - assert_that(result, equal_to(firings)) + def has_expected_values(actual): +from hamcrest.core import assert_that as hamcrest_assert +from hamcrest.library.collection import only_contains +ordered = sorted(actual) +# Early firings. +hamcrest_assert(ordered[:4], only_contains(1, 3, 6, 10)) Review comment: `only_contains` allows for duplicates, which you don't want here. Use `contains` instead. (You could use it below, and drop the `set` in `set(ordered[4:])`.) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 396959) Time Spent: 58h 10m (was: 58h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 58h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=396916=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396916 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 03/Mar/20 18:28 Start Date: 03/Mar/20 18:28 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#discussion_r387210573 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -482,9 +492,20 @@ def test_combining_with_accumulation_mode_and_fanout(self): trigger=AfterWatermark(early=AfterAll(AfterCount(1 | beam.CombineGlobally(sum).without_defaults().with_fanout(2)) - # The frings for DISCARDING mode is [1, 2, 3, 4, 5, 0, 0]. - firings = [1, 3, 6, 10, 15, 15, 15] - assert_that(result, equal_to(firings)) + # Partition the result into early_firings and _. + # In ACCUMULATING mode, the early_frings is [1, 3, 6, 10], + # other_firings is [15, 15, ...]. Different runners have different + # number of 15s, but there should be at least one 15. + smaller_than_fifteen, fifteen, greater_than_fifteen = ( Review comment: Rather than do a partition, I would write something like ``` def has_expected_values(actual): ordered = sorted(actual) # Early firings. hamcrest.assert_that(ordered[:4], hamcrest.contains(1, 3, 6, 10)) # Different runners have different number of 15s, but there should be at least one 15. hamcrest.assert_that(set(ordered[4:]), hamcrest.contains(15)) assert_that(result, has_expected_values) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 396916) Time Spent: 58h (was: 57h 50m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 58h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=396567=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396567 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 03/Mar/20 02:19 Start Date: 03/Mar/20 02:19 Worklog Time Spent: 10m Work Description: chamikaramj commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-593731682 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 396567) Time Spent: 57h 50m (was: 57h 40m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 57h 50m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=396561=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-396561 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 03/Mar/20 02:10 Start Date: 03/Mar/20 02:10 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-593729091 I first added a new matcher to test there are any number of 15s. Later I realized I can partition the PCollection into three (<15, ==15, >15) and use the existing match is_empty and is_not_empty to verify that PCollections (==15, >15). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 396561) Time Spent: 57h 40m (was: 57.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 57h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=395326=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-395326 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 28/Feb/20 23:08 Start Date: 28/Feb/20 23:08 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#discussion_r385962866 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -470,6 +470,9 @@ def test_combining_with_accumulation_mode_and_fanout(self): ts.add_elements([i]) ts.advance_watermark_to_infinity() +def is_early_firing(element, num_partitions): + return 0 if element < 15 else 1 Review comment: I tried to do this, but it doesn't work. Can't get the timing by calling pane.timing(). AttributeError: '_DoFnParam' object has no attribute 'timing' def partition_firings(element, num_partitions, pane=beam.DoFn.PaneInfoParam): if pane.timing() == windowed_value.PaneInfoTiming.EARLY: return 0 elif pane.timing() == windowed_value.PaneInfoTiming.ON_TIME: return 1 else: return 2 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 395326) Time Spent: 57.5h (was: 57h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 57.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=393867=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-393867 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 27/Feb/20 00:26 Start Date: 27/Feb/20 00:26 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#discussion_r384848033 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -470,6 +470,9 @@ def test_combining_with_accumulation_mode_and_fanout(self): ts.add_elements([i]) ts.advance_watermark_to_infinity() +def is_early_firing(element, num_partitions): + return 0 if element < 15 else 1 Review comment: This was fixed last year. Looks like the bug was never closed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 393867) Time Spent: 57h 20m (was: 57h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 57h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=393865=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-393865 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 27/Feb/20 00:18 Start Date: 27/Feb/20 00:18 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#discussion_r384845501 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -470,6 +470,9 @@ def test_combining_with_accumulation_mode_and_fanout(self): ts.add_elements([i]) ts.advance_watermark_to_infinity() +def is_early_firing(element, num_partitions): + return 0 if element < 15 else 1 Review comment: PaneInfo is not supported yet in Python. https://issues.apache.org/jira/browse/BEAM-3759 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 393865) Time Spent: 57h 10m (was: 57h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 57h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=393770=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-393770 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 26/Feb/20 21:53 Start Date: 26/Feb/20 21:53 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#discussion_r384789304 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -482,9 +485,13 @@ def test_combining_with_accumulation_mode_and_fanout(self): trigger=AfterWatermark(early=AfterAll(AfterCount(1 | beam.CombineGlobally(sum).without_defaults().with_fanout(2)) - # The frings for DISCARDING mode is [1, 2, 3, 4, 5, 0, 0]. - firings = [1, 3, 6, 10, 15, 15, 15] - assert_that(result, equal_to(firings)) + # Partition the result into early_firings and _. + # In ACCUMULATING mode, the early_frings is [1, 3, 6, 10], other + # firings are [15, 15, ...]. Different runners have different number + # of 15s. + early_firings, _ = result | beam.Partition(is_early_firing, 2) + exepected_early_firings = [1, 3, 6, 10] + assert_that(early_firings, equal_to(exepected_early_firings)) Review comment: We still want to ensure there is at least one 15 firing. (If we're being pedantic, a single on-time firing, plus a number of late firings) The second argument to assert_that can be any predicate taking the set of actual elements. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 393770) Time Spent: 56h 50m (was: 56h 40m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 56h 50m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=393771=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-393771 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 26/Feb/20 21:53 Start Date: 26/Feb/20 21:53 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#discussion_r384789396 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -470,6 +470,9 @@ def test_combining_with_accumulation_mode_and_fanout(self): ts.add_elements([i]) ts.advance_watermark_to_infinity() +def is_early_firing(element, num_partitions): + return 0 if element < 15 else 1 Review comment: Wouldn't it be better to filter on the pane info itself? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 393771) Time Spent: 57h (was: 56h 50m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 57h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=393758=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-393758 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 26/Feb/20 21:24 Start Date: 26/Feb/20 21:24 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-591651589 R: @robertwb This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 393758) Time Spent: 56h 40m (was: 56.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 56h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=393686=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-393686 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 26/Feb/20 18:44 Start Date: 26/Feb/20 18:44 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-591579870 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 393686) Time Spent: 56.5h (was: 56h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 56.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=393090=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-393090 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 26/Feb/20 00:47 Start Date: 26/Feb/20 00:47 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-591172635 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 393090) Time Spent: 56h 20m (was: 56h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 56h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=393089=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-393089 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 26/Feb/20 00:46 Start Date: 26/Feb/20 00:46 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-591172499 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 393089) Time Spent: 56h 10m (was: 56h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 56h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=392946=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392946 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 25/Feb/20 21:24 Start Date: 25/Feb/20 21:24 Worklog Time Spent: 10m Work Description: Hannah-Jiang commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-591076342 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 392946) Time Spent: 56h (was: 55h 50m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 56h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=392116=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392116 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 24/Feb/20 21:46 Start Date: 24/Feb/20 21:46 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-590564815 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 392116) Time Spent: 55h 50m (was: 55h 40m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 55h 50m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=392068=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392068 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 24/Feb/20 20:23 Start Date: 24/Feb/20 20:23 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-590531243 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 392068) Time Spent: 55h 40m (was: 55.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 55h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=392057=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392057 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 24/Feb/20 20:08 Start Date: 24/Feb/20 20:08 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-590524813 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 392057) Time Spent: 55.5h (was: 55h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 55.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=392055=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392055 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 24/Feb/20 20:05 Start Date: 24/Feb/20 20:05 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-590523542 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 392055) Time Spent: 55h 20m (was: 55h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 55h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=392054=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392054 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 24/Feb/20 20:04 Start Date: 24/Feb/20 20:04 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951#issuecomment-590523346 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 392054) Time Spent: 55h 10m (was: 55h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 55h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=392051=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-392051 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 24/Feb/20 20:02 Start Date: 24/Feb/20 20:02 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on pull request #10951: [BEAM-8575] Modified the test to work for different runners. URL: https://github.com/apache/beam/pull/10951 On direct runner the firings are [1, 3, 6, 10, 15, 15, 15]; while on Unified Worker the firings are [1, 3, 6, 10, 15, 15]. The number of fanouts also influence the number of 15s at the end. Therefore we modified the test to allow any number of 15s at the end. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[![Build
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=389634=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389634 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 19/Feb/20 21:31 Start Date: 19/Feb/20 21:31 Worklog Time Spent: 10m Work Description: angoenka commented on pull request #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389634) Time Spent: 54h 50m (was: 54h 40m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 54h 50m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=389632=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389632 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 19/Feb/20 21:14 Start Date: 19/Feb/20 21:14 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-588473555 Run PythonLint PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389632) Time Spent: 54h 40m (was: 54.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 54h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=389119=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389119 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 21:47 Start Date: 18/Feb/20 21:47 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587907253 Will wait for the tests to pass and merge This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389119) Time Spent: 54.5h (was: 54h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 54.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=389118=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389118 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 21:46 Start Date: 18/Feb/20 21:46 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587906045 Run PythonLint PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389118) Time Spent: 54h 20m (was: 54h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 54h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=389116=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389116 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 21:45 Start Date: 18/Feb/20 21:45 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587904955 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389116) Time Spent: 54h (was: 53h 50m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 54h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=389117=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389117 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 21:45 Start Date: 18/Feb/20 21:45 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587905264 Run PythonLint PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389117) Time Spent: 54h 10m (was: 54h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 54h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=389107=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389107 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 21:25 Start Date: 18/Feb/20 21:25 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587875180 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389107) Time Spent: 53h 50m (was: 53h 40m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 53h 50m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=389106=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389106 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 21:25 Start Date: 18/Feb/20 21:25 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587875076 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389106) Time Spent: 53h 40m (was: 53.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 53h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=389105=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389105 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 21:24 Start Date: 18/Feb/20 21:24 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587874929 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389105) Time Spent: 53.5h (was: 53h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 53.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=389102=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389102 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 21:23 Start Date: 18/Feb/20 21:23 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587872503 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389102) Time Spent: 53h 20m (was: 53h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 53h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=389101=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-389101 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 21:19 Start Date: 18/Feb/20 21:19 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587866982 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 389101) Time Spent: 53h 10m (was: 53h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 53h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=388995=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-388995 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 18:49 Start Date: 18/Feb/20 18:49 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587648002 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 388995) Time Spent: 53h (was: 52h 50m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 53h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=388992=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-388992 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 18:41 Start Date: 18/Feb/20 18:41 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587635845 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 388992) Time Spent: 52h 50m (was: 52h 40m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 52h 50m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=388991=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-388991 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 18:40 Start Date: 18/Feb/20 18:40 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587635452 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 388991) Time Spent: 52h 40m (was: 52.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 52h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=388990=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-388990 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 18:38 Start Date: 18/Feb/20 18:38 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587631362 This PR fixed BEAM-9003, so I unsickbayed this test. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 388990) Time Spent: 52.5h (was: 52h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 52.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=388989=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-388989 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 18:36 Start Date: 18/Feb/20 18:36 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587629245 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 388989) Time Spent: 52h 20m (was: 52h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 52h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=388980=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-388980 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 18:30 Start Date: 18/Feb/20 18:30 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-587619329 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 388980) Time Spent: 52h 10m (was: 52h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 52h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=388976=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-388976 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 18/Feb/20 18:30 Start Date: 18/Feb/20 18:30 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on pull request #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835 Removed MAX_TIMESTAMP from testing data, because the semantics of these extremum elements are not decided yet. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) Python | [![Build
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=387686=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387686 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 14/Feb/20 21:21 Start Date: 14/Feb/20 21:21 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on pull request #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387686) Time Spent: 51h 50m (was: 51h 40m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 51h 50m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=387653=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387653 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 14/Feb/20 20:24 Start Date: 14/Feb/20 20:24 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-586458808 LGTM You can un-sickbay the tests separately if needed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387653) Time Spent: 51h 40m (was: 51.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 51h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=387002=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-387002 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 14/Feb/20 00:54 Start Date: 14/Feb/20 00:54 Worklog Time Spent: 10m Work Description: robertwb commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-586045114 This looks good to me. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 387002) Time Spent: 51.5h (was: 51h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 51.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=386830=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386830
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 13/Feb/20 19:19
Start Date: 13/Feb/20 19:19
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10835:
[BEAM-8575] Removed MAX_TIMESTAMP from testing data
URL: https://github.com/apache/beam/pull/10835#discussion_r379066562
##
File path: sdks/python/apache_beam/transforms/util_test.py
##
@@ -551,9 +551,6 @@ def test_reshuffle_preserves_timestamps(self):
{
'name': 'bar', 'timestamp': 33
},
Review comment:
I'll try to run the test on VR to see if it passes.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 386830)
Time Spent: 51h 20m (was: 51h 10m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 51h 20m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=386827=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386827
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 13/Feb/20 19:17
Start Date: 13/Feb/20 19:17
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10835:
[BEAM-8575] Removed MAX_TIMESTAMP from testing data
URL: https://github.com/apache/beam/pull/10835#discussion_r378614547
##
File path: sdks/python/apache_beam/transforms/util_test.py
##
@@ -551,9 +551,6 @@ def test_reshuffle_preserves_timestamps(self):
{
'name': 'bar', 'timestamp': 33
},
Review comment:
Valentyn helped reproduce BEAM-9003 bug on Dataflow. The pipeline runs
forever (timeout) on Dataflow streaming. So maybe BEAM-9003 is caused by a
different reason.
We'll concentrate on b/146457921 here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 386827)
Time Spent: 50h 50m (was: 50h 40m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 50h 50m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=386829=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386829
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 13/Feb/20 19:17
Start Date: 13/Feb/20 19:17
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10835:
[BEAM-8575] Removed MAX_TIMESTAMP from testing data
URL: https://github.com/apache/beam/pull/10835#discussion_r378019506
##
File path: sdks/python/apache_beam/transforms/util_test.py
##
@@ -551,9 +551,6 @@ def test_reshuffle_preserves_timestamps(self):
{
'name': 'bar', 'timestamp': 33
},
Review comment:
I'm afraid not.
It seems b/146457921 and BEAM-9003 are reported separately.
b/146457921 is for Unified Worker and BEAM-9003 is for Dataflow.
b/146457921 causes this test to be sickbayed in validates_runner_tests.bzl.
BEAM-9003 causes this test to be sickbayed on Jenkins streaming VR test
suite.
It's possible that BEAM-9003 could be solved at the same time.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 386829)
Time Spent: 51h 10m (was: 51h)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 51h 10m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=386828=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386828
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 13/Feb/20 19:17
Start Date: 13/Feb/20 19:17
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10835:
[BEAM-8575] Removed MAX_TIMESTAMP from testing data
URL: https://github.com/apache/beam/pull/10835#discussion_r378465184
##
File path: sdks/python/apache_beam/transforms/util_test.py
##
@@ -551,9 +551,6 @@ def test_reshuffle_preserves_timestamps(self):
{
'name': 'bar', 'timestamp': 33
},
Review comment:
I'm not a member, I can't run the Jenkins VR test or see the results, so I
can't say much about BEAM-9003.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 386828)
Time Spent: 51h (was: 50h 50m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 51h
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=386350=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386350
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 13/Feb/20 02:01
Start Date: 13/Feb/20 02:01
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10835:
[BEAM-8575] Removed MAX_TIMESTAMP from testing data
URL: https://github.com/apache/beam/pull/10835#discussion_r378614547
##
File path: sdks/python/apache_beam/transforms/util_test.py
##
@@ -551,9 +551,6 @@ def test_reshuffle_preserves_timestamps(self):
{
'name': 'bar', 'timestamp': 33
},
Review comment:
Valentyn helped reproduce BEAM-9003 bug on Dataflow. The pipeline runs
forever (timeout) on Dataflow streaming. So maybe BEAM-9003 is caused by a
different reason.
We'll concentrate on b/146457921 here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 386350)
Time Spent: 50h 40m (was: 50.5h)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 50h 40m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=386326=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386326 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 13/Feb/20 00:43 Start Date: 13/Feb/20 00:43 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-585491368 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 386326) Time Spent: 50.5h (was: 50h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 50.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=386170=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-386170
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 12/Feb/20 19:28
Start Date: 12/Feb/20 19:28
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10835:
[BEAM-8575] Removed MAX_TIMESTAMP from testing data
URL: https://github.com/apache/beam/pull/10835#discussion_r378465184
##
File path: sdks/python/apache_beam/transforms/util_test.py
##
@@ -551,9 +551,6 @@ def test_reshuffle_preserves_timestamps(self):
{
'name': 'bar', 'timestamp': 33
},
Review comment:
I'm not a member, I can't run the Jenkins VR test or see the results, so I
can't say much about BEAM-9003.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 386170)
Time Spent: 50h 20m (was: 50h 10m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 50h 20m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=385715=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385715 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 12/Feb/20 08:02 Start Date: 12/Feb/20 08:02 Worklog Time Spent: 10m Work Description: ananvay commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-585080387 R: @robertwb This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 385715) Time Spent: 50h 10m (was: 50h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 50h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=385656=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385656
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 12/Feb/20 03:33
Start Date: 12/Feb/20 03:33
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10835:
[BEAM-8575] Removed MAX_TIMESTAMP from testing data
URL: https://github.com/apache/beam/pull/10835#discussion_r378019506
##
File path: sdks/python/apache_beam/transforms/util_test.py
##
@@ -551,9 +551,6 @@ def test_reshuffle_preserves_timestamps(self):
{
'name': 'bar', 'timestamp': 33
},
Review comment:
I'm afraid not.
It seems b/146457921 and BEAM-9003 are reported separately.
b/146457921 is for Unified Worker and BEAM-9003 is for Dataflow.
b/146457921 causes this test to be sickbayed in validates_runner_tests.bzl.
BEAM-9003 causes this test to be sickbayed on Jenkins streaming VR test
suite.
It's possible that BEAM-9003 could be solved at the same time.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 385656)
Time Spent: 50h (was: 49h 50m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 50h
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=385648=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385648
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 12/Feb/20 03:09
Start Date: 12/Feb/20 03:09
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10835:
[BEAM-8575] Removed MAX_TIMESTAMP from testing data
URL: https://github.com/apache/beam/pull/10835#discussion_r378019506
##
File path: sdks/python/apache_beam/transforms/util_test.py
##
@@ -551,9 +551,6 @@ def test_reshuffle_preserves_timestamps(self):
{
'name': 'bar', 'timestamp': 33
},
Review comment:
I'm afraid not.
It seems b/146457921 and BEAM-9003 are two different issues.
b/146457921 is for Unified Worker and BEAM-9003 is for Dataflow.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 385648)
Time Spent: 49h 50m (was: 49h 40m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 49h 50m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=385556=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385556 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 12/Feb/20 00:05 Start Date: 12/Feb/20 00:05 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-584930246 R: @angoenka This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 385556) Time Spent: 49h 40m (was: 49.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 49h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=38=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-38
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 12/Feb/20 00:05
Start Date: 12/Feb/20 00:05
Worklog Time Spent: 10m
Work Description: angoenka commented on pull request #10835: [BEAM-8575]
Removed MAX_TIMESTAMP from testing data
URL: https://github.com/apache/beam/pull/10835#discussion_r377972727
##
File path: sdks/python/apache_beam/transforms/util_test.py
##
@@ -551,9 +551,6 @@ def test_reshuffle_preserves_timestamps(self):
{
'name': 'bar', 'timestamp': 33
},
Review comment:
Can we un-sickbay this test after this fix?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 38)
Time Spent: 49.5h (was: 49h 20m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 49.5h
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=385553=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385553 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 12/Feb/20 00:05 Start Date: 12/Feb/20 00:05 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835#issuecomment-584929984 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 385553) Time Spent: 49h 20m (was: 49h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 49h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=385528=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-385528 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 11/Feb/20 23:14 Start Date: 11/Feb/20 23:14 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on pull request #10835: [BEAM-8575] Removed MAX_TIMESTAMP from testing data URL: https://github.com/apache/beam/pull/10835 Removed MAX_TIMESTAMP from testing data, because the semantics of these extremum elements are not decided yet. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_SparkStructuredStreaming/lastCompletedBuild/) Python | [![Build
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=375785=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-375785 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 22/Jan/20 18:19 Start Date: 22/Jan/20 18:19 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10277: [BEAM-8575] Reenable passing VR tests. URL: https://github.com/apache/beam/pull/10277#issuecomment-577317690 Run Python Dataflow ValidatesRunner This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 375785) Time Spent: 49h (was: 48h 50m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 49h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=370159=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-370159 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 11/Jan/20 00:09 Start Date: 11/Jan/20 00:09 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #10159: [BEAM-8575] Added a unit test to CombineTest class to test that Combi… URL: https://github.com/apache/beam/pull/10159 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 370159) Time Spent: 48h 50m (was: 48h 40m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 48h 50m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=370098=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-370098 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 10/Jan/20 22:15 Start Date: 10/Jan/20 22:15 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 370098) Time Spent: 48h 40m (was: 48.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 48h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=370056=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-370056 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 10/Jan/20 20:44 Start Date: 10/Jan/20 20:44 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-573198095 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 370056) Time Spent: 48.5h (was: 48h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 48.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=370011=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-370011 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 10/Jan/20 19:10 Start Date: 10/Jan/20 19:10 Worklog Time Spent: 10m Work Description: chamikaramj commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-573167019 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 370011) Time Spent: 48h 10m (was: 48h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 48h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=370012=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-370012 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 10/Jan/20 19:10 Start Date: 10/Jan/20 19:10 Worklog Time Spent: 10m Work Description: chamikaramj commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-573167057 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 370012) Time Spent: 48h 20m (was: 48h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 48h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369992=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369992
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 18:57
Start Date: 10/Jan/20 18:57
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365050870
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
+
+ def merge_accumulators(self, accs):
+return ''.join(sorted(''.join(accs)))
Review comment:
sum(accs, ' ') doesn't work, because sum couldn't concatenate strings.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369992)
Time Spent: 48h (was: 47h 50m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 48h
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369988=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369988 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 10/Jan/20 18:55 Start Date: 10/Jan/20 18:55 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on pull request #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#discussion_r365049352 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -50,6 +52,39 @@ from apache_beam.utils.timestamp import Timestamp +class CounterIncrememtingCombineFn(beam.CombineFn): + """CombineFn for incrementing three different counters: + counter, distribution, gauge, + at the same time concatenating words.""" + + def __init__(self): +beam.CombineFn.__init__(self) +self.word_counter = Metrics.counter(self.__class__, 'word_counter') +self.word_lengths_counter = Metrics.counter( +self.__class__, 'word_lengths') +self.word_lengths_dist = Metrics.distribution( +self.__class__, 'word_len_dist') +self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len') + + def create_accumulator(self): +return '' + + def add_input(self, acc, element): +self.word_counter.inc(1) +self.word_lengths_counter.inc(len(element)) +self.word_lengths_dist.update(len(element)) +self.last_word_len.set(len(element)) + +# ''.join() converts the list to a string. +return ''.join(sorted(acc + element)) Review comment: Done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369988) Time Spent: 47h 50m (was: 47h 40m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 47h 50m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369596=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369596 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 10/Jan/20 03:36 Start Date: 10/Jan/20 03:36 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572858348 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369596) Time Spent: 47h 40m (was: 47.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 47h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369595=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369595 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 10/Jan/20 03:36 Start Date: 10/Jan/20 03:36 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572858605 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369595) Time Spent: 47.5h (was: 47h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 47.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369594=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369594 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 10/Jan/20 03:35 Start Date: 10/Jan/20 03:35 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572858348 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369594) Time Spent: 47h 20m (was: 47h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 47h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369593=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369593
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 03:27
Start Date: 10/Jan/20 03:27
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365059540
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -485,6 +520,113 @@ def test_fixed_windows_combine(self):
equal_to([('c', 3), ('c', 10), ('d', 5), ('d', 17)]),
label='sum per key')
+ # Test that three different kinds of metrics work with a customized
+ # CounterIncrememtingCombineFn.
+ def test_custormized_counters_in_combine_fn(self):
+p = TestPipeline()
+input = (p
+ | beam.Create([('c', 'i'),
+('c', 'go'),
+('c', 'run'),
+('d', 'beam'),
+('d', 'tests')]))
+
+# The result of concatenating all values regardless of key.
+global_concat = (input
+ | beam.Values()
+ | beam.CombineGlobally(CounterIncrememtingCombineFn())
+ | "sort global result" >> _SortLists)
Review comment:
Deleted.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369593)
Time Spent: 47h 10m (was: 47h)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 47h 10m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369592=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369592
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 03:20
Start Date: 10/Jan/20 03:20
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365055230
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
+
+ def merge_accumulators(self, accs):
+return ''.join(sorted(''.join(accs)))
+
+ def extract_output(self, acc):
+return acc
Review comment:
Once sorted, acc became a list again. So I use ''.join() to convert it back
to a string.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369592)
Time Spent: 47h (was: 46h 50m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 47h
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369585=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369585
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 03:06
Start Date: 10/Jan/20 03:06
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365049352
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
Review comment:
That doesn't work.
This is quite tricky.
The input strings are automatically taken as a list. For example, for the
input item ('key1', 'abc'), the string 'abc' is processed as ['a', 'b', 'c'].
Therefore, if we don't use ''.join() to convert the list to a string, we
will get this result:
[('key2', ['u', 'u', 'v', 'v', 'x', 'x', 'y', 'y', 'z']), ('key1', ['a',
'a', 'a', 'b', 'b', 'c'])],
with modified code:
def create_accumulator(self):
return ''
def add_input(self, acc, element):
return acc + element
def merge_accumulators(self, accs):
return [ele for acc in accs for ele in acc]
def extract_output(self, acc):
return sorted(acc)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369585)
Time Spent: 46h 50m (was: 46h 40m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 46h 50m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369584=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369584
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 03:05
Start Date: 10/Jan/20 03:05
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365049352
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
Review comment:
That doesn't work.
This is quite tricky.
The input strings are automatically taken as a list. For example, for the
input item ('key1', 'abc'), the string 'abc' is processed ['a', 'b', 'c'].
Therefore, if we don't use ''.join() to convert the list to a string, we
will get this result:
[('key2', ['u', 'u', 'v', 'v', 'x', 'x', 'y', 'y', 'z']), ('key1', ['a',
'a', 'a', 'b', 'b', 'c'])],
with modified code:
def create_accumulator(self):
return ''
def add_input(self, acc, element):
return acc + element
def merge_accumulators(self, accs):
return [ele for acc in accs for ele in acc]
def extract_output(self, acc):
return sorted(acc)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369584)
Time Spent: 46h 40m (was: 46.5h)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 46h 40m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369583=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369583
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 03:01
Start Date: 10/Jan/20 03:01
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365055230
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
+
+ def merge_accumulators(self, accs):
+return ''.join(sorted(''.join(accs)))
+
+ def extract_output(self, acc):
+return acc
Review comment:
Once sorted, acc became a list again. ''.join() is used to convert it back
to a string.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369583)
Time Spent: 46.5h (was: 46h 20m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 46.5h
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369582=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369582
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 03:00
Start Date: 10/Jan/20 03:00
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365055059
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
+
+ def merge_accumulators(self, accs):
+return ''.join(sorted(''.join(accs)))
+
+ def extract_output(self, acc):
+return acc
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369582)
Time Spent: 46h 20m (was: 46h 10m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 46h 20m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369572=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369572
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 02:37
Start Date: 10/Jan/20 02:37
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365049352
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
Review comment:
That doesn't work.
This is quick tricky.
The input strings are automatically taken as a list. For example, for the
input item ('key1', 'abc'), the string 'abc' is processed ['a', 'b', 'c'].
Therefore, if we don't use ''.join() to convert the list to a string, we
will get this result:
[('key2', ['u', 'u', 'v', 'v', 'x', 'x', 'y', 'y', 'z']), ('key1', ['a',
'a', 'a', 'b', 'b', 'c'])],
with modified code:
def create_accumulator(self):
return ''
def add_input(self, acc, element):
return acc + element
def merge_accumulators(self, accs):
return [ele for acc in accs for ele in acc]
def extract_output(self, acc):
return sorted(acc)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369572)
Time Spent: 46h 10m (was: 46h)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 46h 10m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369570=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369570
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 02:36
Start Date: 10/Jan/20 02:36
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365049352
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
Review comment:
That doesn't work.
The input strings are automatically taken as a list. For example, for the
input item ('key1', 'abc'), the string 'abc' is processed ['a', 'b', 'c'].
Therefore, if we don't use ''.join() to convert the list to a string, we
will get this result:
[('key2', ['u', 'u', 'v', 'v', 'x', 'x', 'y', 'y', 'z']), ('key1', ['a',
'a', 'a', 'b', 'b', 'c'])],
with modified code:
def create_accumulator(self):
return ''
def add_input(self, acc, element):
return acc + element
def merge_accumulators(self, accs):
return [ele for acc in accs for ele in acc]
def extract_output(self, acc):
return sorted(acc)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369570)
Time Spent: 45h 50m (was: 45h 40m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 45h 50m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369569=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369569
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 02:36
Start Date: 10/Jan/20 02:36
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365049352
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
Review comment:
That doesn't work.
The input strings are automatically taken as a list. For example, for the
input item ('key1', 'abc'), the string 'abc' is processed ['a', 'b', 'c'].
Therefore, if we don't use ''.join() to convert the list to a string, we
will get this result:
[('key2', ['u', 'u', 'v', 'v', 'x', 'x', 'y', 'y', 'z']), ('key1', ['a',
'a', 'a', 'b', 'b', 'c'])],
with modified code:
def create_accumulator(self):
return ''
def add_input(self, acc, element):
return acc + element
def merge_accumulators(self, accs):
return [ele for acc in accs for ele in acc]
def extract_output(self, acc):
return sorted(acc)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369569)
Time Spent: 45h 40m (was: 45.5h)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 45h 40m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369565=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369565
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 02:29
Start Date: 10/Jan/20 02:29
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365049352
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
Review comment:
That doesn't work.
The input strings are automatically taken as a list. For example, for the
input item ('key1', 'abc'), the string 'abc' is processed ['a', 'b', 'c'].
Therefore, if we don't use ''.join() to convert the list to a string, we
will get this result:
[('key2', ['u', 'u', 'v', 'v', 'x', 'x', 'y', 'y', 'z']), ('key1', ['a',
'a', 'a', 'b', 'b', 'c'])].
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369565)
Time Spent: 45.5h (was: 45h 20m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 45.5h
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369539=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369539 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 10/Jan/20 01:51 Start Date: 10/Jan/20 01:51 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on pull request #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#discussion_r365042199 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -50,6 +52,39 @@ from apache_beam.utils.timestamp import Timestamp +class CounterIncrememtingCombineFn(beam.CombineFn): Review comment: Done. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369539) Time Spent: 45h 20m (was: 45h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 45h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369538=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369538
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 01:48
Start Date: 10/Jan/20 01:48
Worklog Time Spent: 10m
Work Description: bumblebee-coming commented on pull request #10190:
[BEAM-8575] Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365041530
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -485,6 +520,113 @@ def test_fixed_windows_combine(self):
equal_to([('c', 3), ('c', 10), ('d', 5), ('d', 17)]),
label='sum per key')
+ # Test that three different kinds of metrics work with a customized
+ # CounterIncrememtingCombineFn.
+ def test_custormized_counters_in_combine_fn(self):
+p = TestPipeline()
+input = (p
+ | beam.Create([('c', 'i'),
+('c', 'go'),
+('c', 'run'),
+('d', 'beam'),
+('d', 'tests')]))
+
+# The result of concatenating all values regardless of key.
+global_concat = (input
+ | beam.Values()
+ | beam.CombineGlobally(CounterIncrememtingCombineFn())
+ | "sort global result" >> _SortLists)
+
+# The (key, concatenated_string) pairs for all keys.
+concat_per_key = (input
+ | beam.CombinePerKey(CounterIncrememtingCombineFn())
+ | "sort per key result" >> _SortLists)
+
+result = p.run()
+result.wait_until_finish()
+
+# Verify the concatenated strings are correct.
+expected_concat_per_key = [('c', 'ginoru'), ('d', 'abeemsstt')]
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369538)
Time Spent: 45h 10m (was: 45h)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 45h 10m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369517=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369517 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 10/Jan/20 01:24 Start Date: 10/Jan/20 01:24 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#discussion_r365036430 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -50,6 +52,39 @@ from apache_beam.utils.timestamp import Timestamp +class CounterIncrememtingCombineFn(beam.CombineFn): + """CombineFn for incrementing three different counters: + counter, distribution, gauge, + at the same time concatenating words.""" + + def __init__(self): +beam.CombineFn.__init__(self) +self.word_counter = Metrics.counter(self.__class__, 'word_counter') +self.word_lengths_counter = Metrics.counter( +self.__class__, 'word_lengths') +self.word_lengths_dist = Metrics.distribution( +self.__class__, 'word_len_dist') +self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len') + + def create_accumulator(self): +return '' + + def add_input(self, acc, element): +self.word_counter.inc(1) +self.word_lengths_counter.inc(len(element)) +self.word_lengths_dist.update(len(element)) +self.last_word_len.set(len(element)) + +# ''.join() converts the list to a string. +return ''.join(sorted(acc + element)) Review comment: No need to sort here, just return acc + element. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369517) Time Spent: 44.5h (was: 44h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 44.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369521=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369521
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 01:24
Start Date: 10/Jan/20 01:24
Worklog Time Spent: 10m
Work Description: robertwb commented on pull request #10190: [BEAM-8575]
Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365035848
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -485,6 +520,113 @@ def test_fixed_windows_combine(self):
equal_to([('c', 3), ('c', 10), ('d', 5), ('d', 17)]),
label='sum per key')
+ # Test that three different kinds of metrics work with a customized
+ # CounterIncrememtingCombineFn.
+ def test_custormized_counters_in_combine_fn(self):
+p = TestPipeline()
+input = (p
+ | beam.Create([('c', 'i'),
+('c', 'go'),
+('c', 'run'),
+('d', 'beam'),
+('d', 'tests')]))
+
+# The result of concatenating all values regardless of key.
+global_concat = (input
+ | beam.Values()
+ | beam.CombineGlobally(CounterIncrememtingCombineFn())
+ | "sort global result" >> _SortLists)
+
+# The (key, concatenated_string) pairs for all keys.
+concat_per_key = (input
+ | beam.CombinePerKey(CounterIncrememtingCombineFn())
+ | "sort per key result" >> _SortLists)
+
+result = p.run()
+result.wait_until_finish()
+
+# Verify the concatenated strings are correct.
+expected_concat_per_key = [('c', 'ginoru'), ('d', 'abeemsstt')]
Review comment:
Nit: it's really hard to tell from reading this whether it is correct. Maybe
make the input something like
('key1': 'a'), ('key1': 'ab'), .., ('key2', 'xyz'), ...
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369521)
Time Spent: 45h (was: 44h 50m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 45h
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369516=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369516 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 10/Jan/20 01:24 Start Date: 10/Jan/20 01:24 Worklog Time Spent: 10m Work Description: robertwb commented on pull request #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#discussion_r365036214 ## File path: sdks/python/apache_beam/transforms/combiners_test.py ## @@ -50,6 +52,39 @@ from apache_beam.utils.timestamp import Timestamp +class CounterIncrememtingCombineFn(beam.CombineFn): Review comment: Maybe call this SortedConcatWithCounters? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369516) Time Spent: 44.5h (was: 44h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 44.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369519=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369519
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 01:24
Start Date: 10/Jan/20 01:24
Worklog Time Spent: 10m
Work Description: robertwb commented on pull request #10190: [BEAM-8575]
Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365036559
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
+
+ def merge_accumulators(self, accs):
+return ''.join(sorted(''.join(accs)))
+
+ def extract_output(self, acc):
+return acc
Review comment:
I'd do the sorting here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369519)
Time Spent: 44h 40m (was: 44.5h)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 44h 40m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369520=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369520
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 01:24
Start Date: 10/Jan/20 01:24
Worklog Time Spent: 10m
Work Description: robertwb commented on pull request #10190: [BEAM-8575]
Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365036669
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -485,6 +520,113 @@ def test_fixed_windows_combine(self):
equal_to([('c', 3), ('c', 10), ('d', 5), ('d', 17)]),
label='sum per key')
+ # Test that three different kinds of metrics work with a customized
+ # CounterIncrememtingCombineFn.
+ def test_custormized_counters_in_combine_fn(self):
+p = TestPipeline()
+input = (p
+ | beam.Create([('c', 'i'),
+('c', 'go'),
+('c', 'run'),
+('d', 'beam'),
+('d', 'tests')]))
+
+# The result of concatenating all values regardless of key.
+global_concat = (input
+ | beam.Values()
+ | beam.CombineGlobally(CounterIncrememtingCombineFn())
+ | "sort global result" >> _SortLists)
Review comment:
There's no need for _SortLists anywhere here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369520)
Time Spent: 44h 50m (was: 44h 40m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 44h 50m
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[
https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369518=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369518
]
ASF GitHub Bot logged work on BEAM-8575:
Author: ASF GitHub Bot
Created on: 10/Jan/20 01:24
Start Date: 10/Jan/20 01:24
Worklog Time Spent: 10m
Work Description: robertwb commented on pull request #10190: [BEAM-8575]
Added two unit tests to CombineTest class to test that Co…
URL: https://github.com/apache/beam/pull/10190#discussion_r365036490
##
File path: sdks/python/apache_beam/transforms/combiners_test.py
##
@@ -50,6 +52,39 @@
from apache_beam.utils.timestamp import Timestamp
+class CounterIncrememtingCombineFn(beam.CombineFn):
+ """CombineFn for incrementing three different counters:
+ counter, distribution, gauge,
+ at the same time concatenating words."""
+
+ def __init__(self):
+beam.CombineFn.__init__(self)
+self.word_counter = Metrics.counter(self.__class__, 'word_counter')
+self.word_lengths_counter = Metrics.counter(
+self.__class__, 'word_lengths')
+self.word_lengths_dist = Metrics.distribution(
+self.__class__, 'word_len_dist')
+self.last_word_len = Metrics.gauge(self.__class__, 'last_word_len')
+
+ def create_accumulator(self):
+return ''
+
+ def add_input(self, acc, element):
+self.word_counter.inc(1)
+self.word_lengths_counter.inc(len(element))
+self.word_lengths_dist.update(len(element))
+self.last_word_len.set(len(element))
+
+# ''.join() converts the list to a string.
+return ''.join(sorted(acc + element))
+
+ def merge_accumulators(self, accs):
+return ''.join(sorted(''.join(accs)))
Review comment:
Likewise, `return sum(accs, '')`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 369518)
Time Spent: 44.5h (was: 44h 20m)
> Add more Python validates runner tests
> --
>
> Key: BEAM-8575
> URL: https://issues.apache.org/jira/browse/BEAM-8575
> Project: Beam
> Issue Type: Test
> Components: sdk-py-core, testing
>Reporter: wendy liu
>Assignee: wendy liu
>Priority: Major
> Time Spent: 44.5h
> Remaining Estimate: 0h
>
> This is the umbrella issue to track the work of adding more Python tests to
> improve test coverage.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369385=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369385 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 09/Jan/20 21:38 Start Date: 09/Jan/20 21:38 Worklog Time Spent: 10m Work Description: tvalentyn commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572768192 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369385) Time Spent: 44h 20m (was: 44h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 44h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369363=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369363 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 09/Jan/20 21:02 Start Date: 09/Jan/20 21:02 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572754219 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369363) Time Spent: 44h 10m (was: 44h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 44h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369346=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369346 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 09/Jan/20 20:41 Start Date: 09/Jan/20 20:41 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572745433 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369346) Time Spent: 43h 50m (was: 43h 40m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 43h 50m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369347=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369347 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 09/Jan/20 20:41 Start Date: 09/Jan/20 20:41 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572745538 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369347) Time Spent: 44h (was: 43h 50m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 44h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369248=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369248 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 09/Jan/20 18:45 Start Date: 09/Jan/20 18:45 Worklog Time Spent: 10m Work Description: chamikaramj commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572697673 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369248) Time Spent: 43.5h (was: 43h 20m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 43.5h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=369249=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-369249 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 09/Jan/20 18:45 Start Date: 09/Jan/20 18:45 Worklog Time Spent: 10m Work Description: chamikaramj commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572697694 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 369249) Time Spent: 43h 40m (was: 43.5h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 43h 40m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=368762=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-368762 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 09/Jan/20 02:07 Start Date: 09/Jan/20 02:07 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572347542 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 368762) Time Spent: 43h 20m (was: 43h 10m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 43h 20m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=368761=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-368761 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 09/Jan/20 02:06 Start Date: 09/Jan/20 02:06 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572347325 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 368761) Time Spent: 43h 10m (was: 43h) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 43h 10m > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-8575) Add more Python validates runner tests
[ https://issues.apache.org/jira/browse/BEAM-8575?focusedWorklogId=368747=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-368747 ] ASF GitHub Bot logged work on BEAM-8575: Author: ASF GitHub Bot Created on: 09/Jan/20 01:52 Start Date: 09/Jan/20 01:52 Worklog Time Spent: 10m Work Description: bumblebee-coming commented on issue #10190: [BEAM-8575] Added two unit tests to CombineTest class to test that Co… URL: https://github.com/apache/beam/pull/10190#issuecomment-572344103 Retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 368747) Time Spent: 43h (was: 42h 50m) > Add more Python validates runner tests > -- > > Key: BEAM-8575 > URL: https://issues.apache.org/jira/browse/BEAM-8575 > Project: Beam > Issue Type: Test > Components: sdk-py-core, testing >Reporter: wendy liu >Assignee: wendy liu >Priority: Major > Time Spent: 43h > Remaining Estimate: 0h > > This is the umbrella issue to track the work of adding more Python tests to > improve test coverage. -- This message was sent by Atlassian Jira (v8.3.4#803005)
