scwhittle commented on code in PR #37762:
URL: https://github.com/apache/beam/pull/37762#discussion_r2910769199
##########
runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/options/DataflowStreamingPipelineOptions.java:
##########
@@ -226,6 +227,12 @@ public interface DataflowStreamingPipelineOptions extends
PipelineOptions {
void setWindmillServiceStreamMaxBackoffMillis(int value);
+ @Description(
+ "If true, log and skip input elements that are unable to successfully
decode from the streaming backend.")
+ ValueProvider<Boolean> getSkipInputElementsWithDecodingExceptions();
Review Comment:
I was thinking that ValueProvider was needed for all templates but it is
just classic templates. In the classic template case, the graph and pipeline
serialization happens during template creation and the valueprovider is a
placeholder that allows injecting the value later when the template is
instantiated. Flex templates on the other hand rerun the graph and pipeline
serialization logic each invocation and don't require them. In particular the
source/reader is a serialized part of the graph. Given that this is more of an
emergency option and a customer using a classic template may wish to use it, it
seems worthwhile keeping it as a valueprovider.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
