Thanks for your feedbacks.
Please find below my answers:
-Message d'origine-
De : Aljoscha Krettek
Envoyé : jeudi 7 janvier 2021 13:55
À : user@flink.apache.org
Objet : Re: Flink kafka exceptions handling
[EMETTEUR EXTERNE] / [EXTERNAL SENDER]
Soyez vigilant avant d'ouvrir les pièces jointes ou de cliquer sur les liens.
En cas de doute, signalez le message via le bouton "Message suspect" ou
consultez go/secu.
Be cautious before opening attachments or clicking on any links. If in doubt,
use "Suspicious email" button or visit go/secu.
Hi,
When you say that the `JobManager` goes down, you're referring to the fact that
the Flink job will finish in a failed state after too many exceptions have
occurred in the `FlinkKafkaConsumer. Is that correct?
--> Yes, this is exactly what I meant, the Flink job is in a failed state
I'm afraid right now there is no code path that would allow catching those
`TopicUnthaurizationException`. We basically treat most exceptions coming from
Kafka as errors that require recovery.
--> We can have other exceptions, like like TimeoutException when our Kafka
brokers are degraded.
What behaviour would you have in mind as a reaction to those exceptions?
--> Our processing System is supposed to continue streaming data even though
there is some Kafka errors, we are expecting that the KafkaConsumer fails but
not the Flink job, do you think it is possible?
Best,
Aljoscha
On 2021/01/06 17:59, BELGHITH Amira (EXT) wrote:
>
>Thank you for your answer.
>I have been subscribed.
>
>This is the previous topic I’m referring to
>http://mail-archives.apache.org/mod_mbox/flink-user/202008.mbox/%3CCACz
>KVZQ093HixMewb_prtP41ceXgmxCv=cmpsbphw-9+h8b...@mail.gmail.com%3E
>
>Our flink job manager fails after multiple restarting, when the Kafka Consumer
>does not find a topic for example. We have a kafka exception
>TopicUnthaurizationException. We listen to a list a topics and whenever one is
>down , all our streaming system is down .. is there a way to handle those
>exceptions in the FlinkKafkaConsumer so the job manager does not fail?
>
>
>De : Amira Belghith Envoyé : mercredi 6
>janvier 2021 18:36 À : BELGHITH Amira (EXT) ResgGtsOpmOptVdf
>; amira.belghith-...@soge.com Objet :
>Fwd: Flink kafka exceptions handling
>
>[EMETTEUR EXTERNE] / [EXTERNAL SENDER]
>Soyez vigilant avant d'ouvrir les pièces jointes ou de cliquer sur les liens.
>En cas de doute, signalez le message via le bouton "Message suspect" ou
>consultez go/secu.
>Be cautious before opening attachments or clicking on any links. If in doubt,
>use "Suspicious email" button or visit go/secu.
>
>
>
>-- Message transféré -
>De : Piotr Nowojski mailto:pnowoj...@apache.org>>
>Date : mer. 6 janv. 2021 à 17:26
>Objet : Re: Flink kafka exceptions handling À : Amira Belghith
>mailto:belghith.am...@gmail.com>>
>CC : buggi...@gmail.com<mailto:buggi...@gmail.com>
>mailto:buggi...@gmail.com>>
>
>I think you first need to be subscribed as it's explained here [1]. Could you
>also link to which previous topic are you referring to?
>
>Piotrek
>
>[1] https://flink.apache.org/community.html#mailing-lists
>
>śr., 6 sty 2021 o 17:09 Amira Belghith
>mailto:belghith.am...@gmail.com>> napisał(a):
>Hey,
>Thanks for your fast reply.
>The mail couldnt be delivered to the mailing list.
>
>Le mer. 6 janv. 2021 à 16:59, Piotr Nowojski
>mailto:pnowoj...@apache.org>> a écrit :
>Hey,
>
>could you post the question on the user
>mailto:user@flink.apache.org>> mailing list?
>
>Thanks,
>Piotrek
>
>śr., 6 sty 2021 o 15:11 Amira Belghith
>mailto:belghith.am...@gmail.com>> napisał(a):
>Hi Nick, Piotr,
>
>Im a software engineer working for Societe Generale bank.
>I saw your discussion about FlinkKafkaConsumer and exceptions handling.
>I have the same problem for a week now, and I wanted to know if you have found
>a solution.
>Our flink job manager fails after multiple restarting, when the Kafka Consumer
>does not find a topic for example. We have a kafka exception
>TopicUnthaurizationException. We listen to a list a topics and whenever one is
>down , all our streaming system is down .. is there a way to handle those
>exceptions in the FlinkKafkaConsumer so the job manager does not fail?
>
>Thanks a lot for your help,
>Amira belghith
>
>=
>
>Ce message et toutes les pieces jointes (ci-apres le "message") sont
>confidentiels et susceptibles de co
