Hello
thanks for getting back.I tracked down the problem to one of the pipeline
option classes i have been using for years.
Somehow the argparse library is clashing, not sure if it is my IDE or
something else.....
I have replaced the option class at the bottom of this email with this
method and i will see how it goes.. but i think it's entirely my issue
Kind regards
def parse_known_args(argv):
"""Parses args for the workflow."""
parser = argparse.ArgumentParser()
parser.add_argument('--recipients')
class XyzOptions(PipelineOptions):
@classmethod
def _add_argparse_args(cls, parser):
parser.add_argument('--recipients', default='[email protected]')
parser.add_argument('--key')
parser.add_argument('--sendgridkey')
On Tue, Dec 3, 2024 at 10:05 PM XQ Hu via user <[email protected]> wrote:
> Can you share the full example to reproduce this?
>
> On Mon, Dec 2, 2024 at 4:30 PM Sofia’s World <[email protected]> wrote:
>
>> HI all
>> i am starting writing beam test in a new environment and suddenly i am
>> finding this error
>> while running a very simple test
>>
>> def test_word_count(self):
>> with TestPipeline() as p:
>> input_data = p | 'Start' >> beam.Create(['hello world', 'hello beam'])
>> counts = (
>> input_data
>> | 'One' >>beam.FlatMap(lambda x: x.split(' '))
>> | 'Two' >> beam.Map(lambda x: (x, 1))
>> | 'Three' >> beam.CombinePerKey(sum)
>> )
>> debugSink = beam.Map(print)
>> counts | debugSink
>>
>>
>> the bot says it's an issue with argparse, and perhaps i have some lib
>> clash... i fixed it long time ago but now i forgot
>> Could anyone advise how can i address this problem?
>>
>> Kind regards
>> Marco
>>
>>
>> test_word_count
>> (shareloader.test.test_stock_pick.TestEdgarUtils.test_word_count) ...
>> Missing pipeline option (runner). Executing pipeline using the default
>> runner: DirectRunner.
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> 'function' object has no attribute '__self__'
>> ERROR
>>
>> ======================================================================
>> ERROR: test_word_count
>> (shareloader.test.test_stock_pick.TestEdgarUtils.test_word_count)
>> ----------------------------------------------------------------------
>> Traceback (most recent call last):
>> File
>> "/workspaces/GCP_Experiments/dataflow/shareloader/test/test_stock_pick.py",
>> line 63, in test_word_count
>> with TestPipeline() as p:
>> File
>> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/pipeline.py",
>> line 620, in __exit__
>> self.result = self.run()
>> ^^^^^^^^^^
>> File
>> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/testing/test_pipeline.py",
>> line 115, in run
>> result = super().run(
>> ^^^^^^^^^^^^
>> File
>> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/pipeline.py",
>> line 570, in run
>> self._options).run(False)
>> ^^^^^^^^^^
>> File
>> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/pipeline.py",
>> line 594, in run
>> return self.runner.run_pipeline(self, self._options)
>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> File
>> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/runners/direct/direct_runner.py",
>> line 151, in run_pipeline
>> all_options = options.get_all_options()
>> ^^^^^^^^^^^^^^^^^^^^^^^^^
>> File
>> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/options/pipeline_options.py",
>> line 351, in get_all_options
>> cls._add_argparse_args(parser) # pylint: disable=protected-access
>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> File "/workspaces/GCP_Experiments/dataflow/samples/email_pipeline.py",
>> line 46, in _add_argparse_args
>> parser.add_value_provider_argument('--input_file', type=str,
>> File
>> "/home/codespace/.python/current/lib/python3.12/site-packages/apache_beam/options/pipeline_options.py",
>> line 129, in add_value_provider_argument
>> self.add_argument(*args, **kwargs)
>> File "/home/codespace/.python/current/lib/python3.12/argparse.py", line
>> 1472, in add_argument
>> action = action_class(**kwargs)
>> ^^^^^^^^^^^^^^^^^^^^^^
>> TypeError: _StoreAction.__init__() got an unexpected keyword argument
>> 'default_value'
>>
>
