Fixed. Can someone please take a look at the usage of the --beam_plugins flag in the Dataflow runner so that we can address the root cause?
We can probably do more to avoid Friday Python post commit excitement. In this case, extra checking was done pre-merge by running the Python VR tests for Flink, but the failure occurred with the Dataflow runner. The changes were pipeline options related, so (pre-existing) test coverage should have been better. But beyond that, we can probably make it easier for contributors and reviewers to know what extra checks are available and possibly appropriate to run pre-commit. Should we add some pointers to https://beam.apache.org/contribute/testing/#pre-commit or is there a better place? Thanks On Fri, Oct 5, 2018 at 10:38 AM Udi Meiri <eh...@google.com> wrote: > More details in https://issues.apache.org/jira/browse/BEAM-5442 > > On Fri, Oct 5, 2018 at 10:26 AM Udi Meiri <eh...@google.com> wrote: > >> I'm seeing these errors at least in one test: >> "Python sdk harness failed: >> Traceback (most recent call last): >> File >> "/usr/local/lib/python2.7/dist-packages/apache_beam/runners/worker/sdk_worker_main.py", >> line 133, in main >> sdk_pipeline_options.get_all_options(drop_default=True)) >> File >> "/usr/local/lib/python2.7/dist-packages/apache_beam/options/pipeline_options.py", >> line 224, in get_all_options >> parser.add_argument(arg.split('=', 1)[0], nargs='?') >> File "/usr/lib/python2.7/argparse.py", line 1308, in add_argument >> return self._add_action(action) >> File "/usr/lib/python2.7/argparse.py", line 1682, in _add_action >> self._optionals._add_action(action) >> File "/usr/lib/python2.7/argparse.py", line 1509, in _add_action >> action = super(_ArgumentGroup, self)._add_action(action) >> File "/usr/lib/python2.7/argparse.py", line 1322, in _add_action >> self._check_conflict(action) >> File "/usr/lib/python2.7/argparse.py", line 1460, in _check_conflict >> conflict_handler(action, confl_optionals) >> File "/usr/lib/python2.7/argparse.py", line 1467, in >> _handle_conflict_error >> raise ArgumentError(action, message % conflict_string) >> ArgumentError: argument --beam_plugins: conflicting option string(s): >> --beam_plugins" >> >> This looks like https://github.com/apache/beam/pull/6557 >> >> On Fri, Oct 5, 2018 at 9:41 AM Boyuan Zhang <boyu...@google.com> wrote: >> >>> Seems like tests failed: >>> test_leader_board_it >>> (apache_beam.examples.complete.game.leader_board_it_test.LeaderBoardIT) -> >>> Bigquery table not found >>> test_game_stats_it >>> (apache_beam.examples.complete.game.game_stats_it_test.GameStatsIT) -> >>> Bigquery table not found >>> streaming related tests -> Assertion errors >>> >>> On Fri, Oct 5, 2018 at 9:33 AM Udi Meiri <eh...@google.com> wrote: >>> >>>> I'm seeing post-commit failures in :beam-sdks-python:postCommitITTests: >>>> https://builds.apache.org/job/beam_PostCommit_Python_Verify/6181/console >>>> https://builds.apache.org/job/beam_PostCommit_Python_Verify/6182/console >>>> >>>> >>>>
