Hello
indeed i found something quite interesting
Below is the full stack trace
in module *\sector_loader.py:33: , * which i am importing at the top of
my unittest, i am declaring this class
class XyzOptions(PipelineOptions):
@classmethod
def _add_argparse_args(cls, parser):
parser.add_argument('--recipients', default='mmistr...@gmail.com')
parser.add_argument('--key')
parser.add_argument('--sendgridkey')
and for some reason it seems argparser is clashing with pytest argparser
(which i am using to run my test)
this is really bizarre... i have sorted it out by doing a local import of
sector_loader... i'll debug further to find out why this is happening
PLs feel free to jump in if you found similar
rgds
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _
C:\Users\Marco\envs\dataflow\lib\site-packages\apache_beam\pipeline.py:585:
in __exit__
self.result = self.run()
C:\Users\Marco\envs\dataflow\lib\site-packages\apache_beam\testing\test_pipeline.py:108:
in run
result = super(TestPipeline, self).run(
C:\Users\Marco\envs\dataflow\lib\site-packages\apache_beam\pipeline.py:537:
in run
return Pipeline.from_runner_api(
C:\Users\Marco\envs\dataflow\lib\site-packages\apache_beam\pipeline.py:564:
in run
return self.runner.run_pipeline(self, self._options)
C:\Users\Marco\envs\dataflow\lib\site-packages\apache_beam\runners\direct\direct_runner.py:122:
in run_pipeline
all_options = options.get_all_options()
C:\Users\Marco\envs\dataflow\lib\site-packages\apache_beam\options\pipeline_options.py:290:
in get_all_options
cls._add_argparse_args(parser) # pylint: disable=protected-access
..\modules\sector_loader.py:33: in _add_argparse_args
parser.add_argument('--recipients', default='mmistr...@gmail.com')
On Thu, Dec 8, 2022 at 8:35 AM Sofia’s World <mmistr...@gmail.com> wrote:
> Hi
> Thanks for getting back to me. No it's not and I think it's pytest
> related ..something clashing with my data flow Options (which parses
> initial args) have overcome issue by renaming my options Params but will
> need to go to the bottom of this...
> If anyone has experience same pls advise
> I m using pytest 6 on py 3.8
> Regards
>
> On Thu, Dec 8, 2022, 8:28 AM Mátyás Manninger via user <
> user@beam.apache.org> wrote:
>
>> Hi,
>>
>> Is this the full stack trace? How do you start this pipeline?
>>
>>
>>
>> Med Vänlig Hälsning / Best Regards
>> <https://lunar.app/>
>> *Matyas Manninger*
>> *Data Engineer Consultant*
>> +46 76 050 6326
>> m...@lunar.app, lunar.app
>> CVR: DK36982837
>>
>>
>> On Wed, 7 Dec 2022 at 19:40, Sofia’s World <mmistr...@gmail.com> wrote:
>>
>>> Hello all
>>> i have been writing pipelines with beam , and suddenly my unit tests
>>> with a weird exception
>>> This is a sample pipeline i have that basically does nothing
>>>
>>> key = os.environ['FMPREPKEY']
>>>
>>>
>>> with TestPipeline() as p:
>>> (p | 'start run_mm' >> beam.Create(['20210101'])
>>> | 'prnt' >> beam.Map(print)
>>> )
>>>
>>>
>>> and still i am getting the exception below
>>> have i somehow messed up my imports?
>>>
>>> Kind regards
>>> Marco
>>>
>>>
>>> elf = <argparse._ArgumentGroup object at 0x0000014024923A60>
>>> action = _StoreAction(option_strings=['--key'], dest='key', nargs=None,
>>> const=None, default=None, type=None, choices=None, help=None, metavar=None)
>>> conflicting_actions = [('--key', _StoreAction(option_strings=['--key'],
>>> dest='key', nargs=None, const=None, default=None, type=None, choices=None,
>>> help=None, metavar=None))]
>>>
>>> def _handle_conflict_error(self, action, conflicting_actions):
>>> message = ngettext('conflicting option string: %s',
>>> 'conflicting option strings: %s',
>>> len(conflicting_actions))
>>> conflict_string = ', '.join([option_string
>>> for option_string, action
>>> in conflicting_actions])
>>> > raise ArgumentError(action, message % conflict_string)
>>> E argparse.ArgumentError: argument --key: conflicting option
>>> string: --key
>>>
>>
