C0urante commented on a change in pull request #10907:
URL: https://github.com/apache/kafka/pull/10907#discussion_r730964164



##########
File path: 
connect/runtime/src/main/java/org/apache/kafka/connect/runtime/distributed/DistributedConfig.java
##########
@@ -192,6 +198,19 @@
     public static final String INTER_WORKER_VERIFICATION_ALGORITHMS_DOC = "A 
list of permitted algorithms for verifying internal requests";
     public static final List<String> 
INTER_WORKER_VERIFICATION_ALGORITHMS_DEFAULT = 
Collections.singletonList(INTER_WORKER_SIGNATURE_ALGORITHM_DEFAULT);
 
+    public static final String EXACTLY_ONCE_SOURCE_SUPPORT_CONFIG = 
"exactly.once.source.support";
+    public static final String EXACTLY_ONCE_SOURCE_SUPPORT_DOC = "Whether to 
enable exactly-once support for source connectors in the cluster "
+            + "by writing source records and their offsets in a Kafka 
transaction, and by proactively fencing out old task generations before 
bringing up new ones. "
+            + "Note that this must be enabled on every worker in a cluster in 
order for exactly-once delivery to be guaranteed, "
+            + "and that some source connectors may still not be able to 
provide exactly-once delivery guarantees even with this support enabled. "
+            + "Permitted values are \"disabled\", \"preparing\", and 
\"enabled\". In order to safely enable exactly-once support for source 
connectors, "
+            + "all workers in the cluster must first be updated to use the 
\"preparing\" value for this property. "
+            + "Once this has been done, a second update of all of the workers 
in the cluster should be performed to change the value of this property to 
\"enabled\".";

Review comment:
       > It's clear to me that we should mention this, but it's not clear where 
we should do so. The user documentation generated from this doc string might be 
one spot that users will see routinely, so maybe it's a candidate. Another 
would be in the Kafka Connect docs about EOS for source connectors.
   
   I think both are acceptable. I'll add this to the docstring now and include 
it in the high-level docs when I write those as well.
   
   > BTW, can you add to this PR changes to the Kafka docs that describe this 
feature?
   
   Given how massive this PR is already, I'd like to do this in a follow-up, 
with the understanding that Kafka docs changes are a requisite for including 
this feature in a release.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Reply via email to