Great! I’ll try it out – thank you Piotrek.
Michael
From: Piotr Nowojski on behalf of Piotr Nowojski
Date: Thursday, December 5, 2019 at 11:03 AM
To: Michael Nguyen
Cc: Khachatryan Roman , "user@flink.apache.org"
Subject: Re: How does Flink handle backpressure in EMR
[External]
> <mailto:user@flink.apache.org>" <mailto:user@flink.apache.org>>
> Subject: Re: How does Flink handle backpressure in EMR
>
> [External]
>
> Hi,
>
> If you are using event time and watermarks, you can monitor the delays using
> `currentInpu
Date: Thursday, December 5, 2019 at 10:27 AM
To: Michael Nguyen
Cc: Khachatryan Roman , "user@flink.apache.org"
Subject: Re: How does Flink handle backpressure in EMR
[External]
Hi,
If you are using event time and watermarks, you can monitor the delays using
`currentInputWaterma
;mailto:khachatryan.ro...@gmail.com>>
> Date: Thursday, December 5, 2019 at 9:47 AM
> To: Michael Nguyen <mailto:michael.nguye...@t-mobile.com>>
> Cc: Piotr Nowojski mailto:pi...@ververica.com>>,
> "user@flink.apache.org <mailto:user@flink.apach
, December 5, 2019 at 9:47 AM
To: Michael Nguyen
Cc: Piotr Nowojski , "user@flink.apache.org"
Subject: Re: How does Flink handle backpressure in EMR
[External]
@Michael,
Could you please describe your topology with which operators being slow,
back-pressured and probably skews in sources
@Michael,
Could you please describe your topology with which operators being slow,
back-pressured and probably skews in sources?
Regards,
Roman
On Thu, Dec 5, 2019 at 6:20 PM Nguyen, Michael <
michael.nguye...@t-mobile.com> wrote:
> Thank you for the response Roman and Piotrek!
>
> @Roman -
Thank you for the response Roman and Piotrek!
@Roman - can you clarify on what you mean when you mentioned Flink propagating
it back to the sources?
Also, if one of my Flink operators is processing records too slowly and is
getting further away from the latest record of my source data stream,
Hi Michael,
As Roman pointed out Flink currently doesn’t support the auto-scaling. It’s on
our roadmap but it requires quite a bit of preliminary work to happen before.
Piotrek
> On 5 Dec 2019, at 15:32, r_khachatryan wrote:
>
> Hi Michael
>
> Flink *does* detect backpressure but currently,
Hi Michael
Flink *does* detect backpressure but currently, it only propagates it back
to sources.
And so it doesn't support auto-scaling.
Regards,
Roman
Nguyen, Michael wrote
> How does Flink handle backpressure (caused by an increase in traffic) in a
> Flink job when it’s being hosted in an
Hello all,
How does Flink handle backpressure (caused by an increase in traffic) in a
Flink job when it’s being hosted in an EMR cluster? Does Flink detect the
backpressure and auto-scales the EMR cluster to handle the workload to relieve
the backpressure? Once the backpressure is gone, then
10 matches
Mail list logo
