Hi devs,
I'd like to start a discussion about FLIP-408: [Umbrella] Introduce
DataStream API V2 [1].
The DataStream API is one of the two main APIs that Flink provides for
writing data processing programs. As an API that was introduced
practically since day-1 of the project and has been evolved
Hi Weijie,
Thank you for the effort you've put into the DataStream API ! By reorganizing
and
redesigning the DataStream API, as well as addressing some of the unreasonable
designs within it, we can enhance the efficiency of job development for
developers.
It also allows developers to design
Hi Weijie,
Thanks for driving the work! There are indeed many pain points in the
current DataStream API, which are challenging to resolve with its
existing design. It is a great opportunity to propose a new DataStream
API that tackles these issues. I like the way we've divided the FLIP
into multip
Hi Wencong:
> The Processing TimerService is currently
defined as one of the basic primitives, partly because it's understood that
you have to choose between processing time and event time.
The other part of the reason is that it needs to work based on the task's
mailbox thread model to avoid conc
Thanks for working on this, Weijie.
The design flaws of the current DataStream API (i.e., V1) have been a pain
for a long time. It's great to see efforts going on trying to resolve them.
Significant changes to such an important and comprehensive set of public
APIs deserves caution. From that pers
Hi All,
Thanks for all the feedback.
If there are no more comments, I would like to start the vote thread,
thanks again!
Best regards,
Weijie
Xintong Song 于2024年1月30日周二 11:04写道:
> Thanks for working on this, Weijie.
>
> The design flaws of the current DataStream API (i.e., V1) have been a p
Hi,weijie
Thank you very much to Weijie for proposing this series of improvements,
especially the complete decoupling of user interface and implementation.
This part is actually a very serious problem that disturbs downstream users
in the community. I hope this problem can be completely solved in
Hi Guowei,
thanks for your reply!
> Do connectors and SQL currently have similar problems?
- Connectors:
The APIs for FLIP-27 based source and Sink-V2 are currently in flink-core,
and we will gradually move them to flink-core-api. Anyway, connector should
be free of this problem.
- SQL/Table:
Hi,
Thanks for your reply!
I have no other comments!
Best,
Guowei
On Mon, Feb 26, 2024 at 3:43 PM weijie guo
wrote:
> Hi Guowei,
>
> thanks for your reply!
>
> > Do connectors and SQL currently have similar problems?
>
> - Connectors:
> The APIs for FLIP-27 based source and Sink-V2 are curren