Justine Olshan created KAFKA-18227:
--------------------------------------
Summary: Ensure v2 partitions are not added to last transaction
during upgrade
Key: KAFKA-18227
URL: https://issues.apache.org/jira/browse/KAFKA-18227
Project: Kafka
Issue Type: Sub-task
Reporter: Justine Olshan
With https://issues.apache.org/jira/browse/KAFKA-14563 there are two cases:
either a transaction is already added to the transaction and we continue
producing/committing offsets, or if it is not we add it with an interbroker
call. With TV2, we can tell if a transaction was added in this or a previous
transaction by checking the epoch. However, with the upgrade path, the epoch
check doesn't work because the first TV2 transaction will have the same epoch
as the last TV0 transaction.
If a transaction is in PrepareAbort/PrepareComplete, the producer will get
EndTxn response and is free to start the next transaction. It is possible that
this new transaction will produce and commit offsets without informing the
coordinator if the partitions are the same as the last transaction. Then on
EndTxn, we can get fenced.
In other scenarios (like there are new partitions)we may also see
InvalidStateTransitions.
Upgrades should not require producer restarts or lose guarantees so we should
fix this.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
