Re: What happens to the channels when there is backpressure?

[Felipe Gutierrez]

Hi Yingjie,

I read this post and the next one as well (
https://flink.apache.org/2019/07/23/flink-network-stack-2.html).

I mean the bandwidth of the channels between two physical operators. When
they are in different hosts, so when the channels are a network channel.

Thanks
*--*
*-- Felipe Gutierrez*

*-- skype: felipe.o.gutierrez*
*--* *https://felipeogutierrez.blogspot.com
*


On Thu, Nov 28, 2019 at 5:07 AM yingjie cao  wrote:

> Hi Felipe,
>
> That depends on what do you mean by 'bandwidth'. If you mean the
> capability of the network stack, the answer would be no.
>
> Here is a post about Flink network stack which may help:
> https://flink.apache.org/2019/06/05/flink-network-stack.html.
>
> Thanks,
> Yingjie
>
> Felipe Gutierrez  于2019年11月27日周三 下午11:13写道：
>
>> Hi community,
>>
>> I have a question about backpressure. Suppose a scenario that I have a
>> map and a reducer, and the reducer is back pressuring the map operator. I
>> know that the reducer is processing tuples at a lower rate than it is
>> receiving.
>>
>> However, can I say that at least one channel between the map and the
>> reducer is totally using its available bandwidth?
>>
>> My guess is it is not, at least in the beginning. But as the time goes on
>> the tuples will be queued in the network buffer of the reducer and then the
>> bandwidth will be 100% of usage. Am I right?
>>
>> Thanks,
>> Felipe
>>
>>
>>

Re: What happens to the channels when there is backpressure?

[yingjie cao]

Hi Felipe,

That depends on what do you mean by 'bandwidth'. If you mean the capability
of the network stack, the answer would be no.

Here is a post about Flink network stack which may help:
https://flink.apache.org/2019/06/05/flink-network-stack.html.

Thanks,
Yingjie

Felipe Gutierrez  于2019年11月27日周三 下午11:13写道：

> Hi community,
>
> I have a question about backpressure. Suppose a scenario that I have a map
> and a reducer, and the reducer is back pressuring the map operator. I know
> that the reducer is processing tuples at a lower rate than it is receiving.
>
> However, can I say that at least one channel between the map and the
> reducer is totally using its available bandwidth?
>
> My guess is it is not, at least in the beginning. But as the time goes on
> the tuples will be queued in the network buffer of the reducer and then the
> bandwidth will be 100% of usage. Am I right?
>
> Thanks,
> Felipe
>
>
>

What happens to the channels when there is backpressure?

[Felipe Gutierrez]

Hi community,

I have a question about backpressure. Suppose a scenario that I have a map
and a reducer, and the reducer is back pressuring the map operator. I know
that the reducer is processing tuples at a lower rate than it is receiving.

However, can I say that at least one channel between the map and the
reducer is totally using its available bandwidth?

My guess is it is not, at least in the beginning. But as the time goes on
the tuples will be queued in the network buffer of the reducer and then the
bandwidth will be 100% of usage. Am I right?

Thanks,
Felipe