C0urante commented on a change in pull request #11775:
URL: https://github.com/apache/kafka/pull/11775#discussion_r818841452
##########
File path:
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedConfig.java
##########
@@ -192,6 +197,40 @@
public static final String INTER_WORKER_VERIFICATION_ALGORITHMS_DOC = "A
list of permitted algorithms for verifying internal requests";
public static final List<String>
INTER_WORKER_VERIFICATION_ALGORITHMS_DEFAULT =
Collections.singletonList(INTER_WORKER_SIGNATURE_ALGORITHM_DEFAULT);
+ private enum ExactlyOnceSourceSupport {
+ DISABLED(false),
+ PREPARING(true),
+ ENABLED(true);
+
+ public final boolean usesTransactionalLeader;
+
+ ExactlyOnceSourceSupport(boolean usesTransactionalLeader) {
+ this.usesTransactionalLeader = usesTransactionalLeader;
+ }
+
+ public static ExactlyOnceSourceSupport fromProperty(String property) {
+ return
ExactlyOnceSourceSupport.valueOf(property.toUpperCase(Locale.ROOT));
+ }
+
+ @Override
+ public String toString() {
+ return name().toLowerCase(Locale.ROOT);
+ }
+ }
+
+ public static final String EXACTLY_ONCE_SOURCE_SUPPORT_CONFIG =
"exactly.once.source.support";
+ public static final String EXACTLY_ONCE_SOURCE_SUPPORT_DOC = "Whether to
enable exactly-once support for source connectors in the cluster "
+ + "by using transactions to write source records and their source
offsets, and by proactively fencing out old task generations before bringing up
new ones. "
+ + "If this feature is enabled, consumers of the topics to which
exactly-once source connectors write should be configured with the "
+ + ConsumerConfig.ISOLATION_LEVEL_CONFIG + " property set to '" +
IsolationLevel.READ_COMMITTED.name().toLowerCase(Locale.ROOT) + "'. "
+ + "Note that this must be enabled on every worker in a cluster in
order for exactly-once delivery to be guaranteed, "
+ + "and that some source connectors may still not be able to
provide exactly-once delivery guarantees even with this support enabled. "
+ + "Permitted values are \"" + ExactlyOnceSourceSupport.DISABLED +
"\", \"" + ExactlyOnceSourceSupport.PREPARING + "\", and \"" +
ExactlyOnceSourceSupport.ENABLED + "\". "
+ + "In order to safely enable exactly-once support for source
connectors, "
+ + "all workers in the cluster must first be updated to use the
\"preparing\" value for this property. "
+ + "Once this has been done, a second update of all of the workers
in the cluster should be performed to change the value of this property to
\"enabled\".";
Review comment:
That's a great point. The docstring here is already long enough and
you're absolutely right that it's misleading for the steps required to bring up
a new cluster.
I'll cut a bunch of this out and add a TODO on linking to the appropriate
section of the Connect docs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
