Abacn commented on code in PR #29395:
URL: https://github.com/apache/beam/pull/29395#discussion_r1391653051
##########
sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/pubsub/PubSubPayloadTranslation.java:
##########
@@ -59,6 +60,10 @@ public String getUrn() {
@Override
public RunnerApi.FunctionSpec translate(
AppliedPTransform<?, ?, Unbounded<?>> transform, SdkComponents
components) {
+ if (ExperimentalOptions.hasExperiment(
Review Comment:
Would you mind explain a little bit why the current transform override
(referred above, override invoked when the experiments not provided) is not
sufficient runner v2? From DataflowRunner.java the stacktrack for execution to
this point is
DataflowRunner.getOverrides() L552
DataflowRunner.replaceV1Transforms() L1579
DataflowRunner.run() L1193
which means both Dataflow v1 and v2 job submission had the override by
default, while non-Dataflow runner always has custom PubsubIO (aka Beam
provided Pubsub IO)
So there are two questions here
- why the current override working for Dataflow v1 [override can be enabled
(default) or disabled (with flag)] but not v2 (persumably always enabled?)?
- is it possible to keep the change inside Dataflow runner, follow the
existing pattern there.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
