Hi,
When you say that the `JobManager` goes down, you're referring to the
fact that the Flink job will finish in a failed state after too many
exceptions have occurred in the `FlinkKafkaConsumer. Is that correct?
I'm afraid right now there is no code path that would allow catching
those `TopicUnthaurizationException`. We basically treat most exceptions
coming from Kafka as errors that require recovery.
What behaviour would you have in mind as a reaction to those exceptions?
Best,
Aljoscha
On 2021/01/06 17:59, BELGHITH Amira (EXT) wrote:
Thank you for your answer.
I have been subscribed.
This is the previous topic I’m referring to
http://mail-archives.apache.org/mod_mbox/flink-user/202008.mbox/%3CCACzKVZQ093HixMewb_prtP41ceXgmxCv=cmpsbphw-9+h8b...@mail.gmail.com%3E
Our flink job manager fails after multiple restarting, when the Kafka Consumer
does not find a topic for example. We have a kafka exception
TopicUnthaurizationException. We listen to a list a topics and whenever one is
down , all our streaming system is down .. is there a way to handle those
exceptions in the FlinkKafkaConsumer so the job manager does not fail?
De : Amira Belghith <belghith.am...@gmail.com>
Envoyé : mercredi 6 janvier 2021 18:36
À : BELGHITH Amira (EXT) ResgGtsOpmOptVdf <amira.belghith-...@socgen.com>;
amira.belghith-...@soge.com
Objet : Fwd: Flink kafka exceptions handling
[EMETTEUR EXTERNE] / [EXTERNAL SENDER]
Soyez vigilant avant d'ouvrir les pièces jointes ou de cliquer sur les liens. En cas de
doute, signalez le message via le bouton "Message suspect" ou consultez go/secu.
Be cautious before opening attachments or clicking on any links. If in doubt, use
"Suspicious email" button or visit go/secu.
---------- Message transféré ---------
De : Piotr Nowojski <pnowoj...@apache.org<mailto:pnowoj...@apache.org>>
Date : mer. 6 janv. 2021 à 17:26
Objet : Re: Flink kafka exceptions handling
À : Amira Belghith <belghith.am...@gmail.com<mailto:belghith.am...@gmail.com>>
CC : buggi...@gmail.com<mailto:buggi...@gmail.com>
<buggi...@gmail.com<mailto:buggi...@gmail.com>>
I think you first need to be subscribed as it's explained here [1]. Could you
also link to which previous topic are you referring to?
Piotrek
[1] https://flink.apache.org/community.html#mailing-lists
śr., 6 sty 2021 o 17:09 Amira Belghith
<belghith.am...@gmail.com<mailto:belghith.am...@gmail.com>> napisał(a):
Hey,
Thanks for your fast reply.
The mail couldnt be delivered to the mailing list.
Le mer. 6 janv. 2021 à 16:59, Piotr Nowojski
<pnowoj...@apache.org<mailto:pnowoj...@apache.org>> a écrit :
Hey,
could you post the question on the user
<user@flink.apache.org<mailto:user@flink.apache.org>> mailing list?
Thanks,
Piotrek
śr., 6 sty 2021 o 15:11 Amira Belghith
<belghith.am...@gmail.com<mailto:belghith.am...@gmail.com>> napisał(a):
Hi Nick, Piotr,
Im a software engineer working for Societe Generale bank.
I saw your discussion about FlinkKafkaConsumer and exceptions handling.
I have the same problem for a week now, and I wanted to know if you have found
a solution.
Our flink job manager fails after multiple restarting, when the Kafka Consumer
does not find a topic for example. We have a kafka exception
TopicUnthaurizationException. We listen to a list a topics and whenever one is
down , all our streaming system is down .. is there a way to handle those
exceptions in the FlinkKafkaConsumer so the job manager does not fail?
Thanks a lot for your help,
Amira belghith
=========================================================
Ce message et toutes les pieces jointes (ci-apres le "message")
sont confidentiels et susceptibles de contenir des informations
couvertes par le secret professionnel. Ce message est etabli
a l'intention exclusive de ses destinataires. Toute utilisation
ou diffusion non autorisee interdite.
Tout message electronique est susceptible d'alteration. La SOCIETE GENERALE
et ses filiales declinent toute responsabilite au titre de ce message
s'il a ete altere, deforme falsifie.
=========================================================
This message and any attachments (the "message") are confidential,
intended solely for the addresses, and may contain legally privileged
information. Any unauthorized use or dissemination is prohibited.
E-mails are susceptible to alteration. Neither SOCIETE GENERALE nor any
of its subsidiaries or affiliates shall be liable for the message
if altered, changed or falsified.
=========================================================
