[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=374152=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-374152 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 19/Jan/20 00:35 Start Date: 19/Jan/20 00:35 Worklog Time Spent: 10m Work Description: stale[bot] commented on issue #9685: [BEAM-7765] - Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9685#issuecomment-575952382 This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 374152) Time Spent: 2h 50m (was: 2h 40m) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Assignee: John Patoch >Priority: Major > Labels: easy > Time Spent: 2h 50m > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=374153=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-374153 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 19/Jan/20 00:35 Start Date: 19/Jan/20 00:35 Worklog Time Spent: 10m Work Description: stale[bot] commented on pull request #9685: [BEAM-7765] - Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9685 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 374153) Time Spent: 3h (was: 2h 50m) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Assignee: John Patoch >Priority: Major > Labels: easy > Time Spent: 3h > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=370400=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-370400 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 11/Jan/20 21:53 Start Date: 11/Jan/20 21:53 Worklog Time Spent: 10m Work Description: stale[bot] commented on issue #9685: [BEAM-7765] - Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9685#issuecomment-573357790 This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the d...@beam.apache.org list. Thank you for your contributions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 370400) Time Spent: 2h 40m (was: 2.5h) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Assignee: John Patoch >Priority: Major > Labels: easy > Time Spent: 2h 40m > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=342145=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-342145 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 12/Nov/19 20:54 Start Date: 12/Nov/19 20:54 Worklog Time Spent: 10m Work Description: angulartist commented on issue #9685: [BEAM-7765] - Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9685#issuecomment-553109804 Hey yes I'm working on it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 342145) Time Spent: 2.5h (was: 2h 20m) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Assignee: John Patoch >Priority: Major > Labels: easy > Time Spent: 2.5h > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=340837=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-340837 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 09/Nov/19 00:33 Start Date: 09/Nov/19 00:33 Worklog Time Spent: 10m Work Description: aaltay commented on issue #9685: [BEAM-7765] - Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9685#issuecomment-552043480 @angulartist are you still working on this PR? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 340837) Time Spent: 2h 20m (was: 2h 10m) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Assignee: John Patoch >Priority: Major > Labels: easy > Time Spent: 2h 20m > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=332335=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-332335 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 23/Oct/19 00:29 Start Date: 23/Oct/19 00:29 Worklog Time Spent: 10m Work Description: pabloem commented on issue #9685: [BEAM-7765] - Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9685#issuecomment-545211504 Thanks for adding the snippets. There's only one more issue, with lints: ``` 01:02:12 > Task :sdks:python:test-suites:tox:py2:lintPy27 01:02:12 * Module apache_beam.examples.snippets.snippets_test 01:02:12 C:1281, 0: Line too long (97/80) (line-too-long) 01:02:12 C:1306, 0: Wrong hanging indentation (add 7 spaces). 01:02:12 LogValueProvidersFn(my_options.string_value))) 01:02:12 ^ | (bad-continuation) 01:02:12 C:1315, 0: Line too long (87/80) (line-too-long) 01:02:12 C:1317, 0: Line too long (82/80) (line-too-long) 01:02:12 C:1318, 0: Trailing whitespace (trailing-whitespace) 01:02:12 01:02:12 01:02:12 Your code has been rated at 10.00/10 (previous run: 10.00/10, -0.00) ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 332335) Time Spent: 2h 10m (was: 2h) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Assignee: John Patoch >Priority: Major > Labels: easy > Time Spent: 2h 10m > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[
https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=326165=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-326165
]
ASF GitHub Bot logged work on BEAM-7765:
Author: ASF GitHub Bot
Created on: 10/Oct/19 08:01
Start Date: 10/Oct/19 08:01
Worklog Time Spent: 10m
Work Description: angulartist commented on pull request #9685:
[BEAM-7765] - Add test for snippet accessing_valueprovider_info_after_run
URL: https://github.com/apache/beam/pull/9685#discussion_r78753
##
File path: sdks/python/apache_beam/examples/snippets/snippets.py
##
@@ -1394,28 +1397,24 @@ def __init__(self, string_vp):
# The DoFn is called when creating the pipeline branch.
# This example logs the ValueProvider value, but
# you could store it by pushing it to an external database.
-def process(self, an_int):
+def process(self, an_int, **kwargs):
logging.info('The string_value is %s' % self.string_vp.get())
- # Another option (where you don't need to pass the value at all) is:
- logging.info('The string value is %s' %
- RuntimeValueProvider.get_value('string_value', str, ''))
pipeline_options = PipelineOptions()
- # Create pipeline.
- p = beam.Pipeline(options=pipeline_options)
my_options = pipeline_options.view_as(MyOptions)
- # Add a branch for logging the ValueProvider value.
- _ = (p
- | beam.Create([None])
- | 'LogValueProvs' >> beam.ParDo(
- LogValueProvidersFn(my_options.string_value)))
-
- # The main pipeline.
- result_pc = (p
- | "main_pc" >> beam.Create([1, 2, 3])
- | beam.combiners.Sum.Globally())
- p.run().wait_until_finish()
-
- # [END AccessingValueProviderInfoAfterRunSnip1]
+ # Create pipeline.
+ with beam.Pipeline(options=my_options) as p:
+# Add a branch for logging the ValueProvider value.
+_ = (p
+ | beam.Create([None])
+ | 'LogValueProvider' >> beam.ParDo(
+ LogValueProvidersFn(my_options.string_value)))
+
+# The main pipeline.
Review comment:
Sure, I've moved it to snippets_test.py
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 326165)
Time Spent: 2h (was: 1h 50m)
> Add test for snippet accessing_valueprovider_info_after_run
> ---
>
> Key: BEAM-7765
> URL: https://issues.apache.org/jira/browse/BEAM-7765
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Udi Meiri
>Assignee: John Patoch
>Priority: Major
> Labels: easy
> Time Spent: 2h
> Remaining Estimate: 0h
>
> This snippet needs a unit test.
> It has bugs. For example:
> - apache_beam.utils.value_provider doesn't exist
> - beam.combiners.Sum doesn't exist
> - unused import of: WriteToText
> cc: [~pabloem]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[
https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=326163=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-326163
]
ASF GitHub Bot logged work on BEAM-7765:
Author: ASF GitHub Bot
Created on: 10/Oct/19 08:00
Start Date: 10/Oct/19 08:00
Worklog Time Spent: 10m
Work Description: angulartist commented on pull request #9685:
[BEAM-7765] - Add test for snippet accessing_valueprovider_info_after_run
URL: https://github.com/apache/beam/pull/9685#discussion_r78566
##
File path: sdks/python/apache_beam/examples/snippets/snippets_test.py
##
@@ -1266,6 +1266,59 @@ def expand(self, pcoll):
lengths = p | beam.Create(["a", "ab", "abc"]) | ComputeWordLengths()
assert_that(lengths, equal_to([1, 2, 3]))
+class AccessingValueProviderInfoAfterRunTest(unittest.TestCase):
+ """Tests for accessing value provider info after run."""
+
+ def test_accessing_valueprovider_info_after_run(self):
+# [START AccessingValueProviderInfoAfterRunSnip1]
+
+class MyOptions(PipelineOptions):
+ @classmethod
+ def _add_argparse_args(cls, parser):
+# Use add_value_provider_argument for arguments to be templatable
+# Use add_argument as usual for non-templatable arguments
+parser.add_value_provider_argument('--string_value', type=str,
+ default='the quick brown fox jumps
over the lazy dog')
+
+class LogValueProvidersFn(beam.DoFn):
+ def __init__(self, string_vp):
+self.string_vp = string_vp
+
+ # Define the DoFn that logs the ValueProvider value.
+ # The DoFn is called when creating the pipeline branch.
+ # This example logs the ValueProvider value, but
+ # you could store it by pushing it to an external database.
+ def process(self, an_int, **kwargs):
+logging.info('The string_value is %s' % self.string_vp.get())
+
+yield self.string_vp.get()
+
+pipeline_options = PipelineOptions()
+
+my_options = pipeline_options.view_as(MyOptions)
+
+# Create pipeline.
+with beam.Pipeline(options=my_options) as p:
+ # Add a branch for logging the ValueProvider value.
+ vp_output = (p
+ | beam.Create([None])
+ | 'LogValueProvider' >> beam.ParDo(
+LogValueProvidersFn(my_options.string_value)))
+
+ # Test value provider argument : is equal to given string 'the quick
brown fox jumps over the lazy dog'
+ assert_that(vp_output, equal_to(['the quick brown fox jumps over the
lazy dog']),
Review comment:
Thanks
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 326163)
Time Spent: 1h 50m (was: 1h 40m)
> Add test for snippet accessing_valueprovider_info_after_run
> ---
>
> Key: BEAM-7765
> URL: https://issues.apache.org/jira/browse/BEAM-7765
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Udi Meiri
>Assignee: John Patoch
>Priority: Major
> Labels: easy
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> This snippet needs a unit test.
> It has bugs. For example:
> - apache_beam.utils.value_provider doesn't exist
> - beam.combiners.Sum doesn't exist
> - unused import of: WriteToText
> cc: [~pabloem]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[
https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=323618=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323618
]
ASF GitHub Bot logged work on BEAM-7765:
Author: ASF GitHub Bot
Created on: 04/Oct/19 17:48
Start Date: 04/Oct/19 17:48
Worklog Time Spent: 10m
Work Description: pabloem commented on pull request #9685: [BEAM-7765] -
Add test for snippet accessing_valueprovider_info_after_run
URL: https://github.com/apache/beam/pull/9685#discussion_r331615477
##
File path: sdks/python/apache_beam/examples/snippets/snippets_test.py
##
@@ -1266,6 +1266,59 @@ def expand(self, pcoll):
lengths = p | beam.Create(["a", "ab", "abc"]) | ComputeWordLengths()
assert_that(lengths, equal_to([1, 2, 3]))
+class AccessingValueProviderInfoAfterRunTest(unittest.TestCase):
+ """Tests for accessing value provider info after run."""
+
+ def test_accessing_valueprovider_info_after_run(self):
+# [START AccessingValueProviderInfoAfterRunSnip1]
+
+class MyOptions(PipelineOptions):
+ @classmethod
+ def _add_argparse_args(cls, parser):
+# Use add_value_provider_argument for arguments to be templatable
+# Use add_argument as usual for non-templatable arguments
+parser.add_value_provider_argument('--string_value', type=str,
+ default='the quick brown fox jumps
over the lazy dog')
+
+class LogValueProvidersFn(beam.DoFn):
+ def __init__(self, string_vp):
+self.string_vp = string_vp
+
+ # Define the DoFn that logs the ValueProvider value.
+ # The DoFn is called when creating the pipeline branch.
+ # This example logs the ValueProvider value, but
+ # you could store it by pushing it to an external database.
+ def process(self, an_int, **kwargs):
+logging.info('The string_value is %s' % self.string_vp.get())
+
+yield self.string_vp.get()
+
+pipeline_options = PipelineOptions()
+
+my_options = pipeline_options.view_as(MyOptions)
+
+# Create pipeline.
+with beam.Pipeline(options=my_options) as p:
+ # Add a branch for logging the ValueProvider value.
+ vp_output = (p
+ | beam.Create([None])
+ | 'LogValueProvider' >> beam.ParDo(
+LogValueProvidersFn(my_options.string_value)))
+
+ # Test value provider argument : is equal to given string 'the quick
brown fox jumps over the lazy dog'
+ assert_that(vp_output, equal_to(['the quick brown fox jumps over the
lazy dog']),
Review comment:
If you move this to the end, you can end the snippet before the assertion
```
# [END AccessingValueProviderInfoAfterRunSnip1]
assert_that(main_pipeline_output, equal_to([6]), label='assert_main_output')
assert_that(vp_output, equal_to(['the quick brown fox jumps over the lazy
dog']),
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 323618)
Time Spent: 1h 40m (was: 1.5h)
> Add test for snippet accessing_valueprovider_info_after_run
> ---
>
> Key: BEAM-7765
> URL: https://issues.apache.org/jira/browse/BEAM-7765
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Udi Meiri
>Assignee: John Patoch
>Priority: Major
> Labels: easy
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> This snippet needs a unit test.
> It has bugs. For example:
> - apache_beam.utils.value_provider doesn't exist
> - beam.combiners.Sum doesn't exist
> - unused import of: WriteToText
> cc: [~pabloem]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[
https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=323619=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323619
]
ASF GitHub Bot logged work on BEAM-7765:
Author: ASF GitHub Bot
Created on: 04/Oct/19 17:48
Start Date: 04/Oct/19 17:48
Worklog Time Spent: 10m
Work Description: pabloem commented on pull request #9685: [BEAM-7765] -
Add test for snippet accessing_valueprovider_info_after_run
URL: https://github.com/apache/beam/pull/9685#discussion_r331614775
##
File path: sdks/python/apache_beam/examples/snippets/snippets.py
##
@@ -1394,28 +1397,24 @@ def __init__(self, string_vp):
# The DoFn is called when creating the pipeline branch.
# This example logs the ValueProvider value, but
# you could store it by pushing it to an external database.
-def process(self, an_int):
+def process(self, an_int, **kwargs):
logging.info('The string_value is %s' % self.string_vp.get())
- # Another option (where you don't need to pass the value at all) is:
- logging.info('The string value is %s' %
- RuntimeValueProvider.get_value('string_value', str, ''))
pipeline_options = PipelineOptions()
- # Create pipeline.
- p = beam.Pipeline(options=pipeline_options)
my_options = pipeline_options.view_as(MyOptions)
- # Add a branch for logging the ValueProvider value.
- _ = (p
- | beam.Create([None])
- | 'LogValueProvs' >> beam.ParDo(
- LogValueProvidersFn(my_options.string_value)))
-
- # The main pipeline.
- result_pc = (p
- | "main_pc" >> beam.Create([1, 2, 3])
- | beam.combiners.Sum.Globally())
- p.run().wait_until_finish()
-
- # [END AccessingValueProviderInfoAfterRunSnip1]
+ # Create pipeline.
+ with beam.Pipeline(options=my_options) as p:
+# Add a branch for logging the ValueProvider value.
+_ = (p
+ | beam.Create([None])
+ | 'LogValueProvider' >> beam.ParDo(
+ LogValueProvidersFn(my_options.string_value)))
+
+# The main pipeline.
Review comment:
Does it make sense to completely move the snippet to the `snippets_test.py`
file, and remove it from snippets.py?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 323619)
Time Spent: 1h 40m (was: 1.5h)
> Add test for snippet accessing_valueprovider_info_after_run
> ---
>
> Key: BEAM-7765
> URL: https://issues.apache.org/jira/browse/BEAM-7765
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Udi Meiri
>Assignee: John Patoch
>Priority: Major
> Labels: easy
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> This snippet needs a unit test.
> It has bugs. For example:
> - apache_beam.utils.value_provider doesn't exist
> - beam.combiners.Sum doesn't exist
> - unused import of: WriteToText
> cc: [~pabloem]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[
https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=323617=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323617
]
ASF GitHub Bot logged work on BEAM-7765:
Author: ASF GitHub Bot
Created on: 04/Oct/19 17:48
Start Date: 04/Oct/19 17:48
Worklog Time Spent: 10m
Work Description: pabloem commented on pull request #9685: [BEAM-7765] -
Add test for snippet accessing_valueprovider_info_after_run
URL: https://github.com/apache/beam/pull/9685#discussion_r331615148
##
File path: sdks/python/apache_beam/examples/snippets/snippets_test.py
##
@@ -1266,6 +1266,59 @@ def expand(self, pcoll):
lengths = p | beam.Create(["a", "ab", "abc"]) | ComputeWordLengths()
assert_that(lengths, equal_to([1, 2, 3]))
+class AccessingValueProviderInfoAfterRunTest(unittest.TestCase):
+ """Tests for accessing value provider info after run."""
+
+ def test_accessing_valueprovider_info_after_run(self):
+# [START AccessingValueProviderInfoAfterRunSnip1]
+
+class MyOptions(PipelineOptions):
+ @classmethod
+ def _add_argparse_args(cls, parser):
+# Use add_value_provider_argument for arguments to be templatable
+# Use add_argument as usual for non-templatable arguments
+parser.add_value_provider_argument('--string_value', type=str,
+ default='the quick brown fox jumps
over the lazy dog')
+
+class LogValueProvidersFn(beam.DoFn):
+ def __init__(self, string_vp):
+self.string_vp = string_vp
+
+ # Define the DoFn that logs the ValueProvider value.
+ # The DoFn is called when creating the pipeline branch.
+ # This example logs the ValueProvider value, but
+ # you could store it by pushing it to an external database.
+ def process(self, an_int, **kwargs):
Review comment:
You don't need the `kwargs`.
```suggestion
def process(self, an_int):
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 323617)
Time Spent: 1h 40m (was: 1.5h)
> Add test for snippet accessing_valueprovider_info_after_run
> ---
>
> Key: BEAM-7765
> URL: https://issues.apache.org/jira/browse/BEAM-7765
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
>Reporter: Udi Meiri
>Assignee: John Patoch
>Priority: Major
> Labels: easy
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> This snippet needs a unit test.
> It has bugs. For example:
> - apache_beam.utils.value_provider doesn't exist
> - beam.combiners.Sum doesn't exist
> - unused import of: WriteToText
> cc: [~pabloem]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=323606=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-323606 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 04/Oct/19 17:41 Start Date: 04/Oct/19 17:41 Worklog Time Spent: 10m Work Description: pabloem commented on issue #9685: [BEAM-7765] - Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9685#issuecomment-538494124 Run Portable_Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 323606) Time Spent: 1.5h (was: 1h 20m) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Assignee: John Patoch >Priority: Major > Labels: easy > Time Spent: 1.5h > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=322459=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-322459 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 03/Oct/19 08:01 Start Date: 03/Oct/19 08:01 Worklog Time Spent: 10m Work Description: angulartist commented on issue #9684: [BEAM-7765] [Closed] URL: https://github.com/apache/beam/pull/9684#issuecomment-537834214 > Canceling the change? Yup, I opened another PR [#9685](https://github.com/apache/beam/pull/9685) This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 322459) Time Spent: 1h 20m (was: 1h 10m) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Assignee: John Patoch >Priority: Major > Labels: easy > Time Spent: 1h 20m > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=321977=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-321977 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 02/Oct/19 15:17 Start Date: 02/Oct/19 15:17 Worklog Time Spent: 10m Work Description: pabloem commented on issue #9684: [BEAM-7765] [Closed] URL: https://github.com/apache/beam/pull/9684#issuecomment-537542371 Canceling the change? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 321977) Time Spent: 1h 10m (was: 1h) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Assignee: John Patoch >Priority: Major > Labels: easy > Time Spent: 1h 10m > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=320065=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-320065 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 28/Sep/19 21:36 Start Date: 28/Sep/19 21:36 Worklog Time Spent: 10m Work Description: angulartist commented on pull request #9685: [BEAM-7765] - Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9685 Added unit test for snippet accessing_valueprovider_info_after_run - Removed unused import (WriteToText) - Updated bad function call Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Python36/lastCompletedBuild/)[![Build
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=320066=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-320066 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 28/Sep/19 21:36 Start Date: 28/Sep/19 21:36 Worklog Time Spent: 10m Work Description: angulartist commented on issue #9685: [BEAM-7765] - Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9685#issuecomment-536227357 R: @pabloem This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 320066) Time Spent: 1h (was: 50m) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Priority: Major > Labels: easy > Time Spent: 1h > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=320064=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-320064 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 28/Sep/19 21:26 Start Date: 28/Sep/19 21:26 Worklog Time Spent: 10m Work Description: angulartist commented on pull request #9684: [BEAM-7765] Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9684 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 320064) Time Spent: 40m (was: 0.5h) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Priority: Major > Labels: easy > Time Spent: 40m > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=320063=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-320063 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 28/Sep/19 21:24 Start Date: 28/Sep/19 21:24 Worklog Time Spent: 10m Work Description: angulartist commented on pull request #9684: [BEAM-7765] Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9684 Added a unit test for snippet accessing_valueprovider_info_after_run - Removed unused import WriteToText - Changed unknown beam.combiners.Sum function to beam.CombineGlobally(sum) @pabloem Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [X ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)[![Build
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=320062=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-320062 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 28/Sep/19 21:20 Start Date: 28/Sep/19 21:20 Worklog Time Spent: 10m Work Description: angulartist commented on pull request #9684: [BEAM-7765] Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9684 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 320062) Time Spent: 20m (was: 10m) > Add test for snippet accessing_valueprovider_info_after_run > --- > > Key: BEAM-7765 > URL: https://issues.apache.org/jira/browse/BEAM-7765 > Project: Beam > Issue Type: Bug > Components: sdk-py-core >Reporter: Udi Meiri >Priority: Major > Labels: easy > Time Spent: 20m > Remaining Estimate: 0h > > This snippet needs a unit test. > It has bugs. For example: > - apache_beam.utils.value_provider doesn't exist > - beam.combiners.Sum doesn't exist > - unused import of: WriteToText > cc: [~pabloem] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-7765) Add test for snippet accessing_valueprovider_info_after_run
[ https://issues.apache.org/jira/browse/BEAM-7765?focusedWorklogId=320061=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-320061 ] ASF GitHub Bot logged work on BEAM-7765: Author: ASF GitHub Bot Created on: 28/Sep/19 21:20 Start Date: 28/Sep/19 21:20 Worklog Time Spent: 10m Work Description: angulartist commented on pull request #9684: [BEAM-7765] Add test for snippet accessing_valueprovider_info_after_run URL: https://github.com/apache/beam/pull/9684 Added a unit test for snippet accessing_valueprovider_info_after_run - Removed unused import WriteToText - Changed unknown beam.combiners.Sum function to beam.CombineGlobally(sum) @pabloem Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [X ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/) Python | [](https://builds.apache.org/job/beam_PostCommit_Python2/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Python35/lastCompletedBuild/)[