Hello everyone,
I am doing performance tests for one of our streaming applications and, after
increasing the throughput a bit (~500 events per minute), it has started
failing because checkpoints cannot be completed within 10 minutes. The Flink
cluster is not exactly under my control and is runn
Hi!
I see you're using sliding event time windows. What's the exact value of
windowLengthMinutes and windowSlideTimeMinutes? If windowLengthMinutes is
large and windowSlideTimeMinutes is small then each record may be assigned
to a large number of windows as the pipeline proceeds, thus gradually sl
I had similar problem, where i have concurrent two checkpoints were
configured. Also, i used to save it in S3(using minio) on k8s 1.18 env.
Flink service were getting restarted and timeout was happening. It got
resolved:
1. As minio ran out of disk space, caused failure of checkpoints(this was
the
; Flink ML
Subject: Re: Troubleshooting checkpoint timeout
I had similar problem, where i have concurrent two checkpoints were configured.
Also, i used to save it in S3(using minio) on k8s 1.18 env.
Flink service were getting restarted and timeout was happening. It got resolved:
1. As minio ran
,
Alexis.
From: Alexis Sarda-Espinosa
Sent: Mittwoch, 20. Oktober 2021 09:43
To: Parag Somani ; Caizhi Weng
Cc: Flink ML
Subject: RE: Troubleshooting checkpoint timeout
Currently the windows are 10 minutes in size with a 1-minute slide time. The
approximate 500 event/minute throughput is already
Otherwise those metrics don’t really help me know in which areas to look
> for issues.
>
>
>
> Regards,
>
> Alexis.
>
>
>
> *From:* Alexis Sarda-Espinosa
> *Sent:* Mittwoch, 20. Oktober 2021 09:43
> *To:* Parag Somani ; Caizhi Weng <
> tsreape...@gmail.c
eam operator has lower parallelism?
Regards,
Alexis.
From: Piotr Nowojski
Sent: Montag, 25. Oktober 2021 09:59
To: Alexis Sarda-Espinosa
Cc: Parag Somani ; Caizhi Weng ;
Flink ML
Subject: Re: Troubleshooting checkpoint timeout
Hi Alexis,
You can read about those metrics in the documentation
t;happens to barriers if a downstream operator has lower parallelism?
>
>
>
> Regards,
>
> Alexis.
>
>
>
> *From:* Piotr Nowojski
> *Sent:* Montag, 25. Oktober 2021 09:59
> *To:* Alexis Sarda-Espinosa
> *Cc:* Parag Somani ; Caizhi Weng <
> tsreape...@gma
Nowojski mailto:pnowoj...@apache.org>>
Sent: Montag, 25. Oktober 2021 09:59
To: Alexis Sarda-Espinosa
mailto:alexis.sarda-espin...@microfocus.com>>
Cc: Parag Somani mailto:somanipa...@gmail.com>>; Caizhi
Weng mailto:tsreape...@gmail.com>>; Flink ML
mailto:user@flink.apache.org>>
he first restart are behind more data than before it
> restarted, no?
>
>
>
> Regards,
>
> Alexis.
>
>
>
> *From:* Piotr Nowojski
> *Sent:* Montag, 25. Oktober 2021 13:35
> *To:* Alexis Sarda-Espinosa
> *Cc:* Parag Somani ; Caizhi Weng <
> tsreape..
checkpoints. Thanks again for all the info.
Regards,
Alexis.
From: Piotr Nowojski
Sent: Montag, 25. Oktober 2021 15:51
To: Alexis Sarda-Espinosa
Cc: Parag Somani ; Caizhi Weng ;
Flink ML
Subject: Re: Troubleshooting checkpoint timeout
Hi Alexis,
> Should I understand these metrics as a prope
:* Piotr Nowojski
> *Sent:* Montag, 25. Oktober 2021 15:51
> *To:* Alexis Sarda-Espinosa
> *Cc:* Parag Somani ; Caizhi Weng <
> tsreape...@gmail.com>; Flink ML
> *Subject:* Re: Troubleshooting checkpoint timeout
>
>
>
> Hi Alexis,
>
>
>
> > Should I
12 matches
Mail list logo