The RFC-13 Flink writer has some bottlenecks that make it hard to adapter
to production:
- The InstantGeneratorOperator is parallelism 1, which is a limit for
high-throughput consumption; because all the split inputs drain to a single
thread, the network IO would gains pressure too
- The WriteProc
Overall +1 on the idea.
Danny, could we move this to the apache cwiki if you don't mind?
That's what we have been using for other RFC discussions.
On Mon, Jan 4, 2021 at 1:22 AM Danny Chan wrote:
> The RFC-13 Flink writer has some bottlenecks that make it hard to adapter
> to production:
>
> -
+1 on the idea as well, these changes could be super useful. Let’s collaborate
more on the cwiki.
-Nishith
> On Jan 4, 2021, at 10:58 AM, Vinoth Chandar wrote:
>
> Overall +1 on the idea.
>
> Danny, could we move this to the apache cwiki if you don't mind?
> That's what we have been using fo
Sure, i can update the RFC-13 cwiki if you agree with that.
Vinoth Chandar 于2021年1月5日周二 上午2:58写道：
> Overall +1 on the idea.
>
> Danny, could we move this to the apache cwiki if you don't mind?
> That's what we have been using for other RFC discussions.
>
> On Mon, Jan 4, 2021 at 1:22 AM Danny Ch
Hi Danny,
Thanks for the proposal. I'd recommend starting a new RFC. RFC-13 was done and
including some work about the refactoring so we should mark it as completed.
Looking forward to having further discussion on the RFC.
Best,
Gary Li
From: Danny Chan
Sent: T
Hi,
+1, thank you Danny for introducing this new feature
(OperatorCoordinator)[1] of Flink in the recently latest version.
This feature is very helpful for improving the implementation mechanism of
Flink write-client.
But this feature is only available after Flink 1.11. Before that, there was
no
Hi,
Sharing more details, the OperatorConrdinator is the part of the new Data
Source API(Beta) involved in the Flink 1.11's release note[1].
Flink 1.11 was released only about half a year ago. The design of RFC-13
began at the end of 2019, and most of the implementation was completed when
Flink 1
+ 1, Thanks Danny!
I believe this new feature OperatorConrdinator in flink-1.11 will help improve
the current implementation
Best,
XianghuWang
At 2021-01-05 14:17:37, "vino yang" wrote:
>Hi,
>
>Sharing more details, the OperatorConrdinator is the part of the new Data
>Source API(Beta) involved
