[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=401824=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-401824 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 11/Mar/20 22:13 Start Date: 11/Mar/20 22:13 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997#issuecomment-597903850 Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 401824) Time Spent: 2h (was: 1h 50m) > test_multi_triggered_gbk_side_input is always using DirectRunner > > > Key: BEAM-9402 > URL: https://issues.apache.org/jira/browse/BEAM-9402 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, testing >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 2h > Remaining Estimate: 0h > > The option provided in the test overwrite the argument provided during > validates runner test > > > - >> end captured logging << - > == > FAIL: Test a GBK sideinput, with multiple triggering. > -- > Traceback (most recent call last): > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/sideinputs_test.py", > line 401, in test_multi_triggered_gbk_side_input > p.run() > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_pipeline.py", > line 112, in run > False if self.not_use_test_runner_api else test_runner_api)) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 483, in run > self._options).run(False) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 809, in from_runner_api > p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1117, in from_runner_api > part = context.transforms.get_by_id(transform_id) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1104, in from_runner_api > transform = ptransform.PTransform.from_runner_api(proto, context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/ptransform.py", > line 684, in from_runner_api > context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 316, in from_runner_api_parameter > output_tags=output_tags) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 200, in __init__ > assert event_tags.issubset(self.output_tags) > AssertionError: > >> begin captured logging << > root: INFO: Missing pipeline option (runner). Executing pipeline using the > default runner: DirectRunner. > - >> end captured logging << - -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=401823=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-401823 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 11/Mar/20 22:13 Start Date: 11/Mar/20 22:13 Worklog Time Spent: 10m Work Description: angoenka commented on pull request #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 401823) Time Spent: 1h 50m (was: 1h 40m) > test_multi_triggered_gbk_side_input is always using DirectRunner > > > Key: BEAM-9402 > URL: https://issues.apache.org/jira/browse/BEAM-9402 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, testing >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 1h 50m > Remaining Estimate: 0h > > The option provided in the test overwrite the argument provided during > validates runner test > > > - >> end captured logging << - > == > FAIL: Test a GBK sideinput, with multiple triggering. > -- > Traceback (most recent call last): > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/sideinputs_test.py", > line 401, in test_multi_triggered_gbk_side_input > p.run() > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_pipeline.py", > line 112, in run > False if self.not_use_test_runner_api else test_runner_api)) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 483, in run > self._options).run(False) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 809, in from_runner_api > p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1117, in from_runner_api > part = context.transforms.get_by_id(transform_id) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1104, in from_runner_api > transform = ptransform.PTransform.from_runner_api(proto, context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/ptransform.py", > line 684, in from_runner_api > context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 316, in from_runner_api_parameter > output_tags=output_tags) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 200, in __init__ > assert event_tags.issubset(self.output_tags) > AssertionError: > >> begin captured logging << > root: INFO: Missing pipeline option (runner). Executing pipeline using the > default runner: DirectRunner. > - >> end captured logging << - -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=401806=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-401806 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 11/Mar/20 21:43 Start Date: 11/Mar/20 21:43 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997#issuecomment-597892970 Thanks for the fix. LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 401806) Time Spent: 1h 40m (was: 1.5h) > test_multi_triggered_gbk_side_input is always using DirectRunner > > > Key: BEAM-9402 > URL: https://issues.apache.org/jira/browse/BEAM-9402 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, testing >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 1h 40m > Remaining Estimate: 0h > > The option provided in the test overwrite the argument provided during > validates runner test > > > - >> end captured logging << - > == > FAIL: Test a GBK sideinput, with multiple triggering. > -- > Traceback (most recent call last): > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/sideinputs_test.py", > line 401, in test_multi_triggered_gbk_side_input > p.run() > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_pipeline.py", > line 112, in run > False if self.not_use_test_runner_api else test_runner_api)) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 483, in run > self._options).run(False) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 809, in from_runner_api > p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1117, in from_runner_api > part = context.transforms.get_by_id(transform_id) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1104, in from_runner_api > transform = ptransform.PTransform.from_runner_api(proto, context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/ptransform.py", > line 684, in from_runner_api > context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 316, in from_runner_api_parameter > output_tags=output_tags) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 200, in __init__ > assert event_tags.issubset(self.output_tags) > AssertionError: > >> begin captured logging << > root: INFO: Missing pipeline option (runner). Executing pipeline using the > default runner: DirectRunner. > - >> end captured logging << - -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=401719=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-401719 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 11/Mar/20 19:33 Start Date: 11/Mar/20 19:33 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997#issuecomment-597827074 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 401719) Time Spent: 1.5h (was: 1h 20m) > test_multi_triggered_gbk_side_input is always using DirectRunner > > > Key: BEAM-9402 > URL: https://issues.apache.org/jira/browse/BEAM-9402 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, testing >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 1.5h > Remaining Estimate: 0h > > The option provided in the test overwrite the argument provided during > validates runner test > > > - >> end captured logging << - > == > FAIL: Test a GBK sideinput, with multiple triggering. > -- > Traceback (most recent call last): > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/sideinputs_test.py", > line 401, in test_multi_triggered_gbk_side_input > p.run() > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_pipeline.py", > line 112, in run > False if self.not_use_test_runner_api else test_runner_api)) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 483, in run > self._options).run(False) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 809, in from_runner_api > p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1117, in from_runner_api > part = context.transforms.get_by_id(transform_id) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1104, in from_runner_api > transform = ptransform.PTransform.from_runner_api(proto, context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/ptransform.py", > line 684, in from_runner_api > context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 316, in from_runner_api_parameter > output_tags=output_tags) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 200, in __init__ > assert event_tags.issubset(self.output_tags) > AssertionError: > >> begin captured logging << > root: INFO: Missing pipeline option (runner). Executing pipeline using the > default runner: DirectRunner. > - >> end captured logging << - -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=401127=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-401127 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 11/Mar/20 01:43 Start Date: 11/Mar/20 01:43 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997#issuecomment-597401316 Run Python2_PVR_Flink PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 401127) Time Spent: 1h 10m (was: 1h) > test_multi_triggered_gbk_side_input is always using DirectRunner > > > Key: BEAM-9402 > URL: https://issues.apache.org/jira/browse/BEAM-9402 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, testing >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 1h 10m > Remaining Estimate: 0h > > The option provided in the test overwrite the argument provided during > validates runner test > > > - >> end captured logging << - > == > FAIL: Test a GBK sideinput, with multiple triggering. > -- > Traceback (most recent call last): > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/sideinputs_test.py", > line 401, in test_multi_triggered_gbk_side_input > p.run() > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_pipeline.py", > line 112, in run > False if self.not_use_test_runner_api else test_runner_api)) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 483, in run > self._options).run(False) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 809, in from_runner_api > p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1117, in from_runner_api > part = context.transforms.get_by_id(transform_id) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1104, in from_runner_api > transform = ptransform.PTransform.from_runner_api(proto, context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/ptransform.py", > line 684, in from_runner_api > context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 316, in from_runner_api_parameter > output_tags=output_tags) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 200, in __init__ > assert event_tags.issubset(self.output_tags) > AssertionError: > >> begin captured logging << > root: INFO: Missing pipeline option (runner). Executing pipeline using the > default runner: DirectRunner. > - >> end captured logging << - -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=401128=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-401128 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 11/Mar/20 01:43 Start Date: 11/Mar/20 01:43 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997#issuecomment-597401353 Run Python PreCommit This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 401128) Time Spent: 1h 20m (was: 1h 10m) > test_multi_triggered_gbk_side_input is always using DirectRunner > > > Key: BEAM-9402 > URL: https://issues.apache.org/jira/browse/BEAM-9402 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, testing >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 1h 20m > Remaining Estimate: 0h > > The option provided in the test overwrite the argument provided during > validates runner test > > > - >> end captured logging << - > == > FAIL: Test a GBK sideinput, with multiple triggering. > -- > Traceback (most recent call last): > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/sideinputs_test.py", > line 401, in test_multi_triggered_gbk_side_input > p.run() > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_pipeline.py", > line 112, in run > False if self.not_use_test_runner_api else test_runner_api)) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 483, in run > self._options).run(False) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 809, in from_runner_api > p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1117, in from_runner_api > part = context.transforms.get_by_id(transform_id) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1104, in from_runner_api > transform = ptransform.PTransform.from_runner_api(proto, context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/ptransform.py", > line 684, in from_runner_api > context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 316, in from_runner_api_parameter > output_tags=output_tags) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 200, in __init__ > assert event_tags.issubset(self.output_tags) > AssertionError: > >> begin captured logging << > root: INFO: Missing pipeline option (runner). Executing pipeline using the > default runner: DirectRunner. > - >> end captured logging << - -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=399407=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-399407 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 06/Mar/20 22:40 Start Date: 06/Mar/20 22:40 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997#issuecomment-595995023 The issue was that the options are either interpreted from the command line or from the test if passed explicitly. My marking the pipeline streaming in the test also means that the pipeline will run as streaming even when running batch validates runner so I think its better to remove the string flag from the test. Thanks! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 399407) Time Spent: 1h (was: 50m) > test_multi_triggered_gbk_side_input is always using DirectRunner > > > Key: BEAM-9402 > URL: https://issues.apache.org/jira/browse/BEAM-9402 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, testing >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 1h > Remaining Estimate: 0h > > The option provided in the test overwrite the argument provided during > validates runner test > > > - >> end captured logging << - > == > FAIL: Test a GBK sideinput, with multiple triggering. > -- > Traceback (most recent call last): > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/sideinputs_test.py", > line 401, in test_multi_triggered_gbk_side_input > p.run() > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_pipeline.py", > line 112, in run > False if self.not_use_test_runner_api else test_runner_api)) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 483, in run > self._options).run(False) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 809, in from_runner_api > p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1117, in from_runner_api > part = context.transforms.get_by_id(transform_id) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1104, in from_runner_api > transform = ptransform.PTransform.from_runner_api(proto, context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/ptransform.py", > line 684, in from_runner_api > context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 316, in from_runner_api_parameter > output_tags=output_tags) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 200, in __init__ > assert event_tags.issubset(self.output_tags) > AssertionError: > >> begin captured logging << > root: INFO: Missing pipeline option (runner). Executing pipeline using the > default runner: DirectRunner. > - >> end captured logging << - -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=399347=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-399347 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 06/Mar/20 20:15 Start Date: 06/Mar/20 20:15 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997#issuecomment-595947239 The option was there to make sure the test case runs through streaming mode. I don't understand why this causes running against direct runner always. otherwise LGTM though. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 399347) Time Spent: 40m (was: 0.5h) > test_multi_triggered_gbk_side_input is always using DirectRunner > > > Key: BEAM-9402 > URL: https://issues.apache.org/jira/browse/BEAM-9402 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, testing >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > The option provided in the test overwrite the argument provided during > validates runner test > > > - >> end captured logging << - > == > FAIL: Test a GBK sideinput, with multiple triggering. > -- > Traceback (most recent call last): > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/sideinputs_test.py", > line 401, in test_multi_triggered_gbk_side_input > p.run() > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_pipeline.py", > line 112, in run > False if self.not_use_test_runner_api else test_runner_api)) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 483, in run > self._options).run(False) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 809, in from_runner_api > p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1117, in from_runner_api > part = context.transforms.get_by_id(transform_id) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1104, in from_runner_api > transform = ptransform.PTransform.from_runner_api(proto, context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/ptransform.py", > line 684, in from_runner_api > context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 316, in from_runner_api_parameter > output_tags=output_tags) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 200, in __init__ > assert event_tags.issubset(self.output_tags) > AssertionError: > >> begin captured logging << > root: INFO: Missing pipeline option (runner). Executing pipeline using the > default runner: DirectRunner. > - >> end captured logging << - -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=399348=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-399348 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 06/Mar/20 20:15 Start Date: 06/Mar/20 20:15 Worklog Time Spent: 10m Work Description: HuangLED commented on issue #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997#issuecomment-595947239 The option was there to make sure the test case runs through streaming mode. I don't understand why this causes running against direct runner always. otherwise LGTM though. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 399348) Time Spent: 50m (was: 40m) > test_multi_triggered_gbk_side_input is always using DirectRunner > > > Key: BEAM-9402 > URL: https://issues.apache.org/jira/browse/BEAM-9402 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, testing >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > The option provided in the test overwrite the argument provided during > validates runner test > > > - >> end captured logging << - > == > FAIL: Test a GBK sideinput, with multiple triggering. > -- > Traceback (most recent call last): > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/sideinputs_test.py", > line 401, in test_multi_triggered_gbk_side_input > p.run() > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_pipeline.py", > line 112, in run > False if self.not_use_test_runner_api else test_runner_api)) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 483, in run > self._options).run(False) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 809, in from_runner_api > p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1117, in from_runner_api > part = context.transforms.get_by_id(transform_id) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1104, in from_runner_api > transform = ptransform.PTransform.from_runner_api(proto, context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/ptransform.py", > line 684, in from_runner_api > context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 316, in from_runner_api_parameter > output_tags=output_tags) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 200, in __init__ > assert event_tags.issubset(self.output_tags) > AssertionError: > >> begin captured logging << > root: INFO: Missing pipeline option (runner). Executing pipeline using the > default runner: DirectRunner. > - >> end captured logging << - -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=399346=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-399346 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 06/Mar/20 20:10 Start Date: 06/Mar/20 20:10 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997#issuecomment-595945588 Friendly ping @HuangLED @rohdesamuel This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 399346) Time Spent: 0.5h (was: 20m) > test_multi_triggered_gbk_side_input is always using DirectRunner > > > Key: BEAM-9402 > URL: https://issues.apache.org/jira/browse/BEAM-9402 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, testing >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > The option provided in the test overwrite the argument provided during > validates runner test > > > - >> end captured logging << - > == > FAIL: Test a GBK sideinput, with multiple triggering. > -- > Traceback (most recent call last): > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/sideinputs_test.py", > line 401, in test_multi_triggered_gbk_side_input > p.run() > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_pipeline.py", > line 112, in run > False if self.not_use_test_runner_api else test_runner_api)) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 483, in run > self._options).run(False) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 809, in from_runner_api > p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1117, in from_runner_api > part = context.transforms.get_by_id(transform_id) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1104, in from_runner_api > transform = ptransform.PTransform.from_runner_api(proto, context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/ptransform.py", > line 684, in from_runner_api > context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 316, in from_runner_api_parameter > output_tags=output_tags) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 200, in __init__ > assert event_tags.issubset(self.output_tags) > AssertionError: > >> begin captured logging << > root: INFO: Missing pipeline option (runner). Executing pipeline using the > default runner: DirectRunner. > - >> end captured logging << - -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=394605=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-394605 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 28/Feb/20 02:19 Start Date: 28/Feb/20 02:19 Worklog Time Spent: 10m Work Description: angoenka commented on pull request #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997 The option does not allow addition of user provided option like in case of validates runner tests. This means that the test always tries to run against DirectRunner. Further the streaming flag can be provided by the target which is executing this test. Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] [**Choose reviewer(s)**](https://beam.apache.org/contribute/#make-your-change) and mention them in a comment (`R: @username`). - [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue. - [ ] Update `CHANGES.md` with noteworthy changes. - [ ] If this contribution is large, please file an Apache [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). See the [Contributor Guide](https://beam.apache.org/contribute) for more tips on [how to make review process smoother](https://beam.apache.org/contribute/#make-reviewers-job-easier). Post-Commit Tests Status (on master branch) Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark --- | --- | --- | --- | --- | --- | --- | --- Go | [](https://builds.apache.org/job/beam_PostCommit_Go/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Flink/lastCompletedBuild/) | --- | --- | [](https://builds.apache.org/job/beam_PostCommit_Go_VR_Spark/lastCompletedBuild/) Java | [](https://builds.apache.org/job/beam_PostCommit_Java/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Java11/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Batch/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink_Streaming/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza/lastCompletedBuild/) | [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark/lastCompletedBuild/)[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Spark_Batch/lastCompletedBuild/)[![Build
[jira] [Work logged] (BEAM-9402) test_multi_triggered_gbk_side_input is always using DirectRunner
[ https://issues.apache.org/jira/browse/BEAM-9402?focusedWorklogId=394606=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-394606 ] ASF GitHub Bot logged work on BEAM-9402: Author: ASF GitHub Bot Created on: 28/Feb/20 02:19 Start Date: 28/Feb/20 02:19 Worklog Time Spent: 10m Work Description: angoenka commented on issue #10997: [BEAM-9402] Remove options overwrite URL: https://github.com/apache/beam/pull/10997#issuecomment-592277859 R: @HuangLED @rohdesamuel This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 394606) Time Spent: 20m (was: 10m) > test_multi_triggered_gbk_side_input is always using DirectRunner > > > Key: BEAM-9402 > URL: https://issues.apache.org/jira/browse/BEAM-9402 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, testing >Reporter: Ankur Goenka >Assignee: Ankur Goenka >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > The option provided in the test overwrite the argument provided during > validates runner test > > > - >> end captured logging << - > == > FAIL: Test a GBK sideinput, with multiple triggering. > -- > Traceback (most recent call last): > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/sideinputs_test.py", > line 401, in test_multi_triggered_gbk_side_input > p.run() > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_pipeline.py", > line 112, in run > False if self.not_use_test_runner_api else test_runner_api)) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 483, in run > self._options).run(False) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 809, in from_runner_api > p.transforms_stack = [context.transforms.get_by_id(root_transform_id)] > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1117, in from_runner_api > part = context.transforms.get_by_id(transform_id) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/runners/pipeline_context.py", > line 103, in get_by_id > self._id_to_proto[id], self._pipeline_context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/pipeline.py", > line 1104, in from_runner_api > transform = ptransform.PTransform.from_runner_api(proto, context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/transforms/ptransform.py", > line 684, in from_runner_api > context) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 316, in from_runner_api_parameter > output_tags=output_tags) > File > "/usr/local/google/home/goenka/d/work/beam/beam/sdks/python/apache_beam/testing/test_stream.py", > line 200, in __init__ > assert event_tags.issubset(self.output_tags) > AssertionError: > >> begin captured logging << > root: INFO: Missing pipeline option (runner). Executing pipeline using the > default runner: DirectRunner. > - >> end captured logging << - -- This message was sent by Atlassian Jira (v8.3.4#803005)
