[jira] [Commented] (FLINK-16012) Reduce the default number of exclusive buffers from 2 to 1 on receiver side
[ https://issues.apache.org/jira/browse/FLINK-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379748#comment-17379748 ] Yingjie Cao commented on FLINK-16012: - Thanks [~pnowojski]. Looking forward to FLIP-183. > Reduce the default number of exclusive buffers from 2 to 1 on receiver side > --- > > Key: FLINK-16012 > URL: https://issues.apache.org/jira/browse/FLINK-16012 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Reporter: Zhijiang >Assignee: Yingjie Cao >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In order to reduce the inflight buffers for checkpoint in the case of back > pressure, we can reduce the number of exclusive buffers for remote input > channel from default 2 to 1 as the first step. Besides that, the total > required buffers are also reduced as a result. We can further verify the > performance effect via various of benchmarks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-16012) Reduce the default number of exclusive buffers from 2 to 1 on receiver side
[ https://issues.apache.org/jira/browse/FLINK-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17379549#comment-17379549 ] Yingjie Cao commented on FLINK-16012: - Hi [~pnowojski], do we still need this? I am asking because this has some influence on the micro benchmark performance. (The performance lost can be compensated by increasing the floating buffer size.) > Reduce the default number of exclusive buffers from 2 to 1 on receiver side > --- > > Key: FLINK-16012 > URL: https://issues.apache.org/jira/browse/FLINK-16012 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Reporter: Zhijiang >Assignee: Yingjie Cao >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In order to reduce the inflight buffers for checkpoint in the case of back > pressure, we can reduce the number of exclusive buffers for remote input > channel from default 2 to 1 as the first step. Besides that, the total > required buffers are also reduced as a result. We can further verify the > performance effect via various of benchmarks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-16012) Reduce the default number of exclusive buffers from 2 to 1 on receiver side
[ https://issues.apache.org/jira/browse/FLINK-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17324164#comment-17324164 ] Yingjie Cao commented on FLINK-16012: - I am still working on it. > Reduce the default number of exclusive buffers from 2 to 1 on receiver side > --- > > Key: FLINK-16012 > URL: https://issues.apache.org/jira/browse/FLINK-16012 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Reporter: Zhijiang >Assignee: Yingjie Cao >Priority: Major > Labels: pull-request-available > Fix For: 1.14.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In order to reduce the inflight buffers for checkpoint in the case of back > pressure, we can reduce the number of exclusive buffers for remote input > channel from default 2 to 1 as the first step. Besides that, the total > required buffers are also reduced as a result. We can further verify the > performance effect via various of benchmarks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-16012) Reduce the default number of exclusive buffers from 2 to 1 on receiver side
[ https://issues.apache.org/jira/browse/FLINK-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17323094#comment-17323094 ] Flink Jira Bot commented on FLINK-16012: This issue is assigned but has not received an update in 7 days so it has been labeled "stale-assigned". If you are still working on the issue, please give an update and remove the label. If you are no longer working on the issue, please unassign so someone else may work on it. In 7 days the issue will be automatically unassigned. > Reduce the default number of exclusive buffers from 2 to 1 on receiver side > --- > > Key: FLINK-16012 > URL: https://issues.apache.org/jira/browse/FLINK-16012 > Project: Flink > Issue Type: Sub-task > Components: Runtime / Network >Reporter: Zhijiang >Assignee: Yingjie Cao >Priority: Major > Labels: pull-request-available, stale-assigned > Fix For: 1.14.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In order to reduce the inflight buffers for checkpoint in the case of back > pressure, we can reduce the number of exclusive buffers for remote input > channel from default 2 to 1 as the first step. Besides that, the total > required buffers are also reduced as a result. We can further verify the > performance effect via various of benchmarks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-16012) Reduce the default number of exclusive buffers from 2 to 1 on receiver side
[ https://issues.apache.org/jira/browse/FLINK-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036715#comment-17036715 ] Yingjie Cao commented on FLINK-16012: - [~wind_ljy] I have updated the results and the previous results were remove. > Reduce the default number of exclusive buffers from 2 to 1 on receiver side > --- > > Key: FLINK-16012 > URL: https://issues.apache.org/jira/browse/FLINK-16012 > Project: Flink > Issue Type: Improvement >Reporter: Zhijiang >Assignee: Yingjie Cao >Priority: Major > Labels: pull-request-available > Fix For: 1.11.0 > > Time Spent: 10m > Remaining Estimate: 0h > > In order to reduce the inflight buffers for checkpoint in the case of back > pressure, we can reduce the number of exclusive buffers for remote input > channel from default 2 to 1 as the first step. Besides that, the total > required buffers are also reduced as a result. We can further verify the > performance effect via various of benchmarks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (FLINK-16012) Reduce the default number of exclusive buffers from 2 to 1 on receiver side
[
https://issues.apache.org/jira/browse/FLINK-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036714#comment-17036714
]
Yingjie Cao commented on FLINK-16012:
-
Theoretically, reducing the number of buffers may break the data processing
pipeline which can influence the performance. For verification, I hava tested
the change using the flink micro benchmark and a simple benchmark job.
Unfortunately, regressions are seen for both tests.
For micro benchmark, the following are some results with regression (Because of
the unstable result, I run each test three times.):
Using 2 buffer:
{code:java}
Benchmark (channelsFlushTimeout) (writers) Mode
Cnt Score Error Units
networkThroughput 1000,100ms 1 thrpt
30 15972.952 ± 752.985 ops/ms
networkThroughput 1000,100ms 4 thrpt
30 27650.498 ± 713.728 ops/ms
networkThroughput 1000,100ms 1 thrpt
30 15566.705 ± 2007.335 ops/ms
networkThroughput 1000,100ms 4 thrpt
30 27769.195 ± 1632.614 ops/ms
networkThroughput 1000,100ms 1 thrpt
30 15598.175 ± 1671.515 ops/ms
networkThroughput 1000,100ms 4 thrpt
30 27499.901 ± 1035.415 ops/ms{code}
Using 1 buffer：
{code:java}
Benchmark (channelsFlushTimeout) (writers) Mode
Cnt Score Error Units
networkThroughput 1000,100ms 1 thrpt
30 13116.610 ± 325.587 ops/ms
networkThroughput 1000,100ms 4 thrpt
30 22837.502 ± 1024.360 ops/ms
networkThroughput 1000,100ms 1 thrpt
30 11924.883 ± 1038.508 ops/ms
networkThroughput 1000,100ms 4 thrpt
30 22823.586 ± 892.918 ops/ms
networkThroughput 1000,100ms 1 thrpt
30 12960.345 ± 1596.465 ops/ms
networkThroughput 1000,100ms 4 thrpt
30 23028.803 ± 933.609 ops/ms{code}
>From the above results, we can see about 20% performance regression. For the
>benchmark job, there are also regressions (about 10% - 20%) in some cases
>where input channel numbers are small, for example 2 input channels, which
>means the number of buffer can be used is limited.
> Reduce the default number of exclusive buffers from 2 to 1 on receiver side
> ---
>
> Key: FLINK-16012
> URL: https://issues.apache.org/jira/browse/FLINK-16012
> Project: Flink
> Issue Type: Improvement
>Reporter: Zhijiang
>Assignee: Yingjie Cao
>Priority: Major
> Labels: pull-request-available
> Fix For: 1.11.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In order to reduce the inflight buffers for checkpoint in the case of back
> pressure, we can reduce the number of exclusive buffers for remote input
> channel from default 2 to 1 as the first step. Besides that, the total
> required buffers are also reduced as a result. We can further verify the
> performance effect via various of benchmarks.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (FLINK-16012) Reduce the default number of exclusive buffers from 2 to 1 on receiver side
[ https://issues.apache.org/jira/browse/FLINK-16012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17036647#comment-17036647 ] Jiayi Liao commented on FLINK-16012: [~kevin.cyj] Which result is tested before the change? And which is after? > Reduce the default number of exclusive buffers from 2 to 1 on receiver side > --- > > Key: FLINK-16012 > URL: https://issues.apache.org/jira/browse/FLINK-16012 > Project: Flink > Issue Type: Improvement >Reporter: Zhijiang >Assignee: Yingjie Cao >Priority: Major > Labels: pull-request-available > Fix For: 1.11.0 > > Attachments: image-2020-02-13-21-54-05-026.png, > image-2020-02-13-23-30-17-951.png, image-2020-02-14-07-23-16-171.png, > image-2020-02-14-09-13-07-967.png > > Time Spent: 10m > Remaining Estimate: 0h > > In order to reduce the inflight buffers for checkpoint in the case of back > pressure, we can reduce the number of exclusive buffers for remote input > channel from default 2 to 1 as the first step. Besides that, the total > required buffers are also reduced as a result. We can further verify the > performance effect via various of benchmarks. -- This message was sent by Atlassian Jira (v8.3.4#803005)
