robertwb commented on a change in pull request #12713:
URL: https://github.com/apache/beam/pull/12713#discussion_r483316482
##########
File path: sdks/python/apache_beam/runners/dataflow/dataflow_runner.py
##########
@@ -118,13 +118,15 @@ class DataflowRunner(PipelineRunner):
from apache_beam.runners.dataflow.ptransform_overrides import
CreatePTransformOverride
from apache_beam.runners.dataflow.ptransform_overrides import
JrhReadPTransformOverride
from apache_beam.runners.dataflow.ptransform_overrides import
ReadPTransformOverride
+ from apache_beam.runners.dataflow.ptransform_overrides import
ReadBigQuerySourcePTransformOverride
from apache_beam.runners.dataflow.ptransform_overrides import
NativeReadPTransformOverride
# These overrides should be applied before the proto representation of the
# graph is created.
_PTRANSFORM_OVERRIDES = [
CombineValuesPTransformOverride(),
NativeReadPTransformOverride(),
+ ReadBigQuerySourcePTransformOverride(),
Review comment:
You're right, the proto is created later. I still think it would be
cleaner to construct the right pipeline from the start than mutate it later
with overrides. Let's patch up Read to accept PTransforms just as we do for
Write
https://github.com/apache/beam/blob/release-2.18.0/sdks/python/apache_beam/io/iobase.py#L951
(I thought this was already done.)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
