Thanks Fabian,
That was the information I was missing. (Late reply ... same here, FlinkForward 😊 ) Thias -----Original Message----- From: Fabian Paul <fabianp...@ververica.com> Sent: Donnerstag, 28. Oktober 2021 08:38 To: Schwalbe Matthias <matthias.schwa...@viseca.ch> Cc: Mason Chen <mas.chen6...@gmail.com>; user <user@flink.apache.org> Subject: Re: FlinkKafkaConsumer -> KafkaSource State Migration Hi, Sorry for the late reply but most of use were involved in the Flink Forward conference. The upgrade strategies for the Kafka sink and source are pretty similar. Source and sink do not rely on state migration but leveraging Kafka as source of truth. When running with FlinkKafkaConsumer Mason pointed out correctly that you have stop the job with a save point and set `setCommittedOffsetsOnCheckpoint(true)` [1]. For the FlinkKafkaProducer it is similar on a final savepoint the producer will finalize all pending transactions and submit them to Kafka. The KafkaSink can start without the need of any state migration because there should not be any pending transactions anymore. I do not think you must use `allowNonRestoredState` because there shouldn’t be any state anymore after stopping with a savepoint the source or sink. Best, Fabian [1] https://ci.apache.org/projects/flink/flink-docs-release-1.14/docs/connectors/datastream/kafka/#upgrading-to-the-latest-connector-version Diese Nachricht ist ausschliesslich für den Adressaten bestimmt und beinhaltet unter Umständen vertrauliche Mitteilungen. Da die Vertraulichkeit von e-Mail-Nachrichten nicht gewährleistet werden kann, übernehmen wir keine Haftung für die Gewährung der Vertraulichkeit und Unversehrtheit dieser Mitteilung. Bei irrtümlicher Zustellung bitten wir Sie um Benachrichtigung per e-Mail und um Löschung dieser Nachricht sowie eventueller Anhänge. Jegliche unberechtigte Verwendung oder Verbreitung dieser Informationen ist streng verboten. This message is intended only for the named recipient and may contain confidential or privileged information. As the confidentiality of email communication cannot be guaranteed, we do not accept any responsibility for the confidentiality and the intactness of this message. If you have received it in error, please advise the sender by return e-mail and delete this message and any attachments. Any unauthorised use or dissemination of this information is strictly prohibited.
