kennknowles commented on a change in pull request #14276:
URL: https://github.com/apache/beam/pull/14276#discussion_r613575491
##########
File path:
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
##########
@@ -2344,31 +2300,22 @@ static String
getContainerImageForJob(DataflowPipelineOptions options) {
}
static boolean useUnifiedWorker(DataflowPipelineOptions options) {
- return hasExperiment(options, "beam_fn_api")
- || hasExperiment(options, "use_runner_v2")
- || hasExperiment(options, "use_unified_worker");
+ return hasExperiment(options, "beam_fn_api");
Review comment:
Done. I suppose soon we simply delete this method.
##########
File path:
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
##########
@@ -2344,31 +2300,22 @@ static String
getContainerImageForJob(DataflowPipelineOptions options) {
}
static boolean useUnifiedWorker(DataflowPipelineOptions options) {
- return hasExperiment(options, "beam_fn_api")
- || hasExperiment(options, "use_runner_v2")
- || hasExperiment(options, "use_unified_worker");
+ return hasExperiment(options, "beam_fn_api");
}
static boolean useStreamingEngine(DataflowPipelineOptions options) {
return hasExperiment(options, GcpOptions.STREAMING_ENGINE_EXPERIMENT)
|| hasExperiment(options, GcpOptions.WINDMILL_SERVICE_EXPERIMENT);
}
- static void verifyDoFnSupported(
- DoFn<?, ?> fn, boolean streaming, boolean workerV2, boolean
streamingEngine) {
+ static void verifyDoFnSupported(DoFn<?, ?> fn, boolean streaming, boolean
streamingEngine) {
if (streaming && DoFnSignatures.requiresTimeSortedInput(fn)) {
throw new UnsupportedOperationException(
String.format(
"%s does not currently support @RequiresTimeSortedInput in
streaming mode.",
DataflowRunner.class.getSimpleName()));
}
if (DoFnSignatures.usesSetState(fn)) {
- if (workerV2) {
Review comment:
Done
##########
File path:
runners/google-cloud-dataflow-java/src/test/java/org/apache/beam/runners/dataflow/DataflowRunnerTest.java
##########
@@ -1334,57 +1329,6 @@ public void testTransformTranslator() throws IOException
{
assertTrue(transform.translated);
}
- @Test
- public void testSdkHarnessConfiguration() throws IOException {
Review comment:
When I wrote this, I thought this was testing an unused case. Is that
not true? I have restored the test, to investigate if needed...
##########
File path:
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowRunner.java
##########
@@ -1005,7 +961,7 @@ public DataflowPipelineJob run(Pipeline pipeline) {
options.getStager().stageToFile(serializedProtoPipeline,
PIPELINE_FILE_NAME);
dataflowOptions.setPipelineUrl(stagedPipeline.getLocation());
// Now rewrite things to be as needed for v1 (mutates the pipeline)
- replaceTransforms(pipeline);
+ replaceV1Transforms(pipeline);
Review comment:
Done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
