[ https://issues.apache.org/jira/browse/KAFKA-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312782#comment-17312782 ]
Guozhang Wang edited comment on KAFKA-12574 at 4/1/21, 11:22 PM: ----------------------------------------------------------------- Okay, if we are concerned about 4) to be too aggressive, I'm also happy with 3) as well. On the other hand I'm a bit reserved about adding the suffix as a workaround, since the config name is `processing.guarantee` and putting the value as `exactly_once_semantics` seems a bit misaligned. So let's say if we go with option 3), we can: * Deprecate `eos` in 3.0. * Remove `eos` in 4.0; and at the same time we can remove the alpha implementation. * Add `eos` back in 4.x, and deprecate `eos-beta`. * Remove `eos-beta` in 5.0. Regarding the eos-alpha to eos-beta upgrade path, I agree we should well document the upgrade path (we should at least copy-paste what's in https://cwiki.apache.org/confluence/display/KAFKA/KIP-447%3A+Producer+scalability+for+exactly+once+semantics#KIP447:Producerscalabilityforexactlyoncesemantics-Compatibility,Deprecation,andMigrationPlan). Besides, when checking on the upgrade test paths, I think our system tests do not have a case to cover the eos -> eos-beta upgrade path yet, but I only looked at https://issues.apache.org/jira/browse/KAFKA-9832 so maybe there're other tests that I forgot. [~mjsax] [~boyang] could you chime in and let us know if it is covered in system upgrade test or not? was (Author: guozhang): Okay, if we are concerned about 4) to be too aggressive, I'm also happy with 3) as well. On the other hand I'm a bit reserved about adding the suffix as a workaround, since the config name is `processing.guarantee` and putting the value as `exactly_once_semantics` seems a bit misaligned. Regarding the upgrade path, I agree we should well document the upgrade path (we should at least copy-paste what's in https://cwiki.apache.org/confluence/display/KAFKA/KIP-447%3A+Producer+scalability+for+exactly+once+semantics#KIP447:Producerscalabilityforexactlyoncesemantics-Compatibility,Deprecation,andMigrationPlan). Besides, when checking on the upgrade test paths, I think our system tests do not have a case to cover the eos -> eos-beta upgrade path yet, but I only looked at https://issues.apache.org/jira/browse/KAFKA-9832 so maybe there're other tests that I forgot. [~mjsax] [~boyang] could you chime in and let us know if it is covered in system upgrade test or not? > Deprecate eos-alpha > ------------------- > > Key: KAFKA-12574 > URL: https://issues.apache.org/jira/browse/KAFKA-12574 > Project: Kafka > Issue Type: Improvement > Components: streams > Reporter: A. Sophie Blee-Goldman > Priority: Blocker > Labels: needs-kip > Fix For: 3.0.0 > > > In KIP-447 we introduced a new thread-producer which is capable of > exactly-once semantics across multiple tasks. The new mode of EOS, called > eos-beta, is intended to eventually be the preferred processing mode for EOS > as it improves the performance and scaling of partitions/tasks. The only > downside is that it requires brokers to be on version 2.5+ in order to > understand the latest APIs that are necessary for this thread-producer. > We should consider deprecating the eos-alpha config, ie > StreamsConfig.EXACTLY_ONCE, to encourage new. & existing EOS users to migrate > to the new-and-improved processing mode, and upgrade their brokers if > necessary. > Eventually we would like to be able to remove the eos-alpha code paths from > Streams as this will help to simplify the logic and reduce the processing > mode branching. But since this will break client-broker compatibility, and > 2.5 is still a relatively recent version, we probably can't actually remove > eos-alpha in the near future -- This message was sent by Atlassian Jira (v8.3.4#803005)