[
https://issues.apache.org/jira/browse/KAFKA-12574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17312782#comment-17312782
]
Guozhang Wang edited comment on KAFKA-12574 at 4/1/21, 11:22 PM:
-----------------------------------------------------------------
Okay, if we are concerned about 4) to be too aggressive, I'm also happy with 3)
as well. On the other hand I'm a bit reserved about adding the suffix as a
workaround, since the config name is `processing.guarantee` and putting the
value as `exactly_once_semantics` seems a bit misaligned.
So let's say if we go with option 3), we can:
* Deprecate `eos` in 3.0.
* Remove `eos` in 4.0; and at the same time we can remove the alpha
implementation.
* Add `eos` back in 4.x, and deprecate `eos-beta`.
* Remove `eos-beta` in 5.0.
Regarding the eos-alpha to eos-beta upgrade path, I agree we should well
document the upgrade path (we should at least copy-paste what's in
https://cwiki.apache.org/confluence/display/KAFKA/KIP-447%3A+Producer+scalability+for+exactly+once+semantics#KIP447:Producerscalabilityforexactlyoncesemantics-Compatibility,Deprecation,andMigrationPlan).
Besides, when checking on the upgrade test paths, I think our system tests do
not have a case to cover the eos -> eos-beta upgrade path yet, but I only
looked at https://issues.apache.org/jira/browse/KAFKA-9832 so maybe there're
other tests that I forgot. [~mjsax] [~boyang] could you chime in and let us
know if it is covered in system upgrade test or not?
was (Author: guozhang):
Okay, if we are concerned about 4) to be too aggressive, I'm also happy with 3)
as well. On the other hand I'm a bit reserved about adding the suffix as a
workaround, since the config name is `processing.guarantee` and putting the
value as `exactly_once_semantics` seems a bit misaligned.
Regarding the upgrade path, I agree we should well document the upgrade path
(we should at least copy-paste what's in
https://cwiki.apache.org/confluence/display/KAFKA/KIP-447%3A+Producer+scalability+for+exactly+once+semantics#KIP447:Producerscalabilityforexactlyoncesemantics-Compatibility,Deprecation,andMigrationPlan).
Besides, when checking on the upgrade test paths, I think our system tests do
not have a case to cover the eos -> eos-beta upgrade path yet, but I only
looked at https://issues.apache.org/jira/browse/KAFKA-9832 so maybe there're
other tests that I forgot. [~mjsax] [~boyang] could you chime in and let us
know if it is covered in system upgrade test or not?
> Deprecate eos-alpha
> -------------------
>
> Key: KAFKA-12574
> URL: https://issues.apache.org/jira/browse/KAFKA-12574
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: A. Sophie Blee-Goldman
> Priority: Blocker
> Labels: needs-kip
> Fix For: 3.0.0
>
>
> In KIP-447 we introduced a new thread-producer which is capable of
> exactly-once semantics across multiple tasks. The new mode of EOS, called
> eos-beta, is intended to eventually be the preferred processing mode for EOS
> as it improves the performance and scaling of partitions/tasks. The only
> downside is that it requires brokers to be on version 2.5+ in order to
> understand the latest APIs that are necessary for this thread-producer.
> We should consider deprecating the eos-alpha config, ie
> StreamsConfig.EXACTLY_ONCE, to encourage new. & existing EOS users to migrate
> to the new-and-improved processing mode, and upgrade their brokers if
> necessary.
> Eventually we would like to be able to remove the eos-alpha code paths from
> Streams as this will help to simplify the logic and reduce the processing
> mode branching. But since this will break client-broker compatibility, and
> 2.5 is still a relatively recent version, we probably can't actually remove
> eos-alpha in the near future
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
