[jira] [Commented] (KAFKA-13598) idempotence producer is not enabled by default if not set explicitly
[ https://issues.apache.org/jira/browse/KAFKA-13598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17547803#comment-17547803 ] Derek Troy-West commented on KAFKA-13598: - This represents a breaking change where: # Broker version is < 2.8.0 # Cluster has ACL configured, but no IDEMPOTENT_WRITE permission set # Producer has default configuration (or no config captured in KAFKA-13673) # Kafka Client library version is bumped to 3.2.0 The resulting production error has a ClusterAuthorizationException but it wasn't clear that a new ACL was required until we found KAFKA-13759. I think 1-3 might be quite a common scenario for Kafka users in the wild, and 4 is very easy to do on the application side. Perhaps it might be an idea to add a note to the 3.2.0 upgrade guide or release notes about this one. More info: https://kpow.io/articles/kafka-producer-breaking-change/ > idempotence producer is not enabled by default if not set explicitly > > > Key: KAFKA-13598 > URL: https://issues.apache.org/jira/browse/KAFKA-13598 > Project: Kafka > Issue Type: Bug > Components: clients, config >Affects Versions: 3.1.0, 3.0.0 >Reporter: Luke Chen >Assignee: Luke Chen >Priority: Major > Fix For: 3.0.1, 3.2.0, 3.1.1 > > > In KAFKA-10619, we intended to enable idempotence by default, but this was > not achieved due to a bug in the config validation logic. The change from > acks=1 to acks=all worked correctly, however. > This is the following up for KIP-679: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-679%3A+Producer+will+enable+the+strongest+delivery+guarantee+by+default] > > Note: In KAFKA-13673, we'll disable idempotent producer when > acks/retries/max.in.flight config conflicts, to avoid breaking existing > producers. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (KAFKA-13598) idempotence producer is not enabled by default if not set explicitly
[ https://issues.apache.org/jira/browse/KAFKA-13598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17547813#comment-17547813 ] Luke Chen commented on KAFKA-13598: --- [~d-t-w] , yes, adding a note to notable changes in 3.2.0 section is a good idea. Are you interested in submitting a PR for it? Thanks. > idempotence producer is not enabled by default if not set explicitly > > > Key: KAFKA-13598 > URL: https://issues.apache.org/jira/browse/KAFKA-13598 > Project: Kafka > Issue Type: Bug > Components: clients, config >Affects Versions: 3.1.0, 3.0.0 >Reporter: Luke Chen >Assignee: Luke Chen >Priority: Major > Fix For: 3.0.1, 3.2.0, 3.1.1 > > > In KAFKA-10619, we intended to enable idempotence by default, but this was > not achieved due to a bug in the config validation logic. The change from > acks=1 to acks=all worked correctly, however. > This is the following up for KIP-679: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-679%3A+Producer+will+enable+the+strongest+delivery+guarantee+by+default] > > Note: In KAFKA-13673, we'll disable idempotent producer when > acks/retries/max.in.flight config conflicts, to avoid breaking existing > producers. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (KAFKA-13598) idempotence producer is not enabled by default if not set explicitly
[ https://issues.apache.org/jira/browse/KAFKA-13598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17547818#comment-17547818 ] Derek Troy-West commented on KAFKA-13598: - Very happy to [~showuon] I will just copy points 1-4 into the notes if that's ok? I'll raise a PR later this afternoon. Should I raise that PR on the apache-site github repository? > idempotence producer is not enabled by default if not set explicitly > > > Key: KAFKA-13598 > URL: https://issues.apache.org/jira/browse/KAFKA-13598 > Project: Kafka > Issue Type: Bug > Components: clients, config >Affects Versions: 3.1.0, 3.0.0 >Reporter: Luke Chen >Assignee: Luke Chen >Priority: Major > Fix For: 3.0.1, 3.2.0, 3.1.1 > > > In KAFKA-10619, we intended to enable idempotence by default, but this was > not achieved due to a bug in the config validation logic. The change from > acks=1 to acks=all worked correctly, however. > This is the following up for KIP-679: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-679%3A+Producer+will+enable+the+strongest+delivery+guarantee+by+default] > > Note: In KAFKA-13673, we'll disable idempotent producer when > acks/retries/max.in.flight config conflicts, to avoid breaking existing > producers. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (KAFKA-13598) idempotence producer is not enabled by default if not set explicitly
[ https://issues.apache.org/jira/browse/KAFKA-13598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17547916#comment-17547916 ] Luke Chen commented on KAFKA-13598: --- [~d-t-w] , let's open a PR to Kafka repo first, we can discuss how to phrase it there. And then open a PR to kafka-site after merged. For the points 1-4, I'm thinking we should make it into a sentence. This is the current note for v3.2.0 related to this issue: _Notable changes in 3.2.0_ _- Idempotence for the producer is enabled by default if no conflicting configurations are set. In 3.0.0 and 3.1.0, a bug prevented this default from being applied, which meant that idempotence remained disabled unless the user had explicitly set enable.idempotence to true (See KAFKA-13598 for more details). This issue was fixed and the default is properly applied in 3.0.1, 3.1.1, and 3.2.0._ I'm thinking you can add a sub-bullet after it, ex: _Notable changes in 3.2.0_ _- Idempotence for the producer is enabled by default if no conflicting configurations are set. In 3.0.0 and 3.1.0, a bug prevented this default from being applied, which meant that idempotence remained disabled unless the user had explicitly set enable.idempotence to true (See KAFKA-13598 for more details). This issue was fixed and the default is properly applied in 3.0.1, 3.1.1, and 3.2.0._ _-- When the broker version is lower than 2.8.0, and the client version is_ _3.0.1, 3.1.1, and later, the IDEMPOTENT_WRITE permission is required to produce data._ Something like that. Again, we can discuss it in the PR. Thank you. > idempotence producer is not enabled by default if not set explicitly > > > Key: KAFKA-13598 > URL: https://issues.apache.org/jira/browse/KAFKA-13598 > Project: Kafka > Issue Type: Bug > Components: clients, config >Affects Versions: 3.1.0, 3.0.0 >Reporter: Luke Chen >Assignee: Luke Chen >Priority: Major > Fix For: 3.0.1, 3.2.0, 3.1.1 > > > In KAFKA-10619, we intended to enable idempotence by default, but this was > not achieved due to a bug in the config validation logic. The change from > acks=1 to acks=all worked correctly, however. > This is the following up for KIP-679: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-679%3A+Producer+will+enable+the+strongest+delivery+guarantee+by+default] > > Note: In KAFKA-13673, we'll disable idempotent producer when > acks/retries/max.in.flight config conflicts, to avoid breaking existing > producers. -- This message was sent by Atlassian Jira (v8.20.7#820007)
[jira] [Commented] (KAFKA-13598) idempotence producer is not enabled by default if not set explicitly
[ https://issues.apache.org/jira/browse/KAFKA-13598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17550736#comment-17550736 ] Derek Troy-West commented on KAFKA-13598: - Thanks Luke, I raised [https://github.com/apache/kafka/pull/12260] > idempotence producer is not enabled by default if not set explicitly > > > Key: KAFKA-13598 > URL: https://issues.apache.org/jira/browse/KAFKA-13598 > Project: Kafka > Issue Type: Bug > Components: clients, config >Affects Versions: 3.1.0, 3.0.0 >Reporter: Luke Chen >Assignee: Luke Chen >Priority: Major > Fix For: 3.0.1, 3.2.0, 3.1.1 > > > In KAFKA-10619, we intended to enable idempotence by default, but this was > not achieved due to a bug in the config validation logic. The change from > acks=1 to acks=all worked correctly, however. > This is the following up for KIP-679: > [https://cwiki.apache.org/confluence/display/KAFKA/KIP-679%3A+Producer+will+enable+the+strongest+delivery+guarantee+by+default] > > Note: In KAFKA-13673, we'll disable idempotent producer when > acks/retries/max.in.flight config conflicts, to avoid breaking existing > producers. -- This message was sent by Atlassian Jira (v8.20.7#820007)
