Hi everyone,
Thanks for your discussion and feedback!
Our discussions have been going on for a while and there have been no new
concerns for several days. So I would like to start voting recently.
Best regards,
Weijie
Zakelly Lan 于2024年3月12日周二 17:40写道：
> Hi Weijie,
>
> Thanks for your
Hi Weijie,
Thanks for your reply!
Overall I'd be fine with the builder pattern, but it is a little bit long
when carrying explicit 'build()' and declaring the builder. Keeping the
StateDeclaration immutable is OK, but it is a little bit inconvenient for
overriding the undefined options by job
Hi Zakelly,
> But still, from a user's point of view, state can be characterized along
two relatively independent dimensions, how states redistribute and the data
structure. Thus I still suggest a chained-like configuration API that
configures one aspect on each call.
I think the chained-like
Hi Hangxiang,
> So these operators only define all states they may use which could be
explained by the caller, right ?
Yes, you're right.
Best regards,
Weijie
weijie guo 于2024年3月12日周二 13:59写道：
> Hi Max,
>
> > In this thread it looks like the plan is to remove the old state
> declaration
Hi Max,
> In this thread it looks like the plan is to remove the old state
declaration API. I think we should consider keeping the old APIs to
avoid breaking too many jobs.
We're not plan to remove any old apis, which means that changes made in V2
won't affect any V1 DataStream jobs. But V2 is
Hi, Weijie.
Thanks for your answer!
> No, Introducing and declaring new state
> at runtime is something we want to explicitly disallow.
I just thinked about how some operators define their useState() when their
real used states may be changed at runtime, e.g. different state types for
different
The FLIP mentions: "The contents described in this FLIP are all new
APIs and do not involve compatibility issues."
In this thread it looks like the plan is to remove the old state
declaration API. I think we should consider keeping the old APIs to
avoid breaking too many jobs. The new APIs will
Hi Weijie,
Thanks for your answer! Well I get your point. Since partitions are
first-class citizens, and redistribution means how states migrate when
partitions change, I'd be fine with deemphasizing the concept of
keyed/operator state if we highlight the definition of partition in the
document.
Hi Jinzhong,
Thanks for the reply!
> Overall, I think that the “Eager State Declaration” is a good proposal,
which can enhance Flink's state management capabilities and provide
possibilities for subsequent state optimizations.
It's nice to see that people who are familiar with the state stuff
Hi Weijie,
Thanks for driving this!
1. Overall, I think that the “Eager State Declaration” is a good proposal,
which can enhance Flink's state management capabilities and provide
possibilities for subsequent state optimizations.
2. When the user attempts to access an undeclared state at
Hi Hangxiang,
Thanks for your reply!
> We have also discussed in FLIP-359/FLINK-32658 about limiting the user
operation to avoid creating state when processElement. Could current
interfaces also help this?
I think so. It is illegal to create state at runtime in our proposal.
> Could you
Hi Zakelly,
Thanks for your reply!
> My advice would be to conceal RedistributionMode/Strategy from the
standard user interface, particularly within the helper class 'State'. But
I'm OK to keep it in `StateDeclaration` since its interfaces are basically
used by the framework.
I'm sorry, I
Hi Gyula,
Thanks for your reply!
Let me answer these questions:
> What is the semantics of the usesStates method? When is it called? Can
the used state change dynamically at runtime? Can the logic depend on
something computed in open(..) for example?
useStates is used to predefine all the
Hi, Weijie.
Thanks for your proposal.
I'd like to start the discussion with some questions:
1. We have also discussed in FLIP-359/FLINK-32658 about limiting the user
operation to avoid creating state when processElement. Could current
interfaces also help this?
2. Could you provide more examples
Hi Weijie,
Thanks for proposing this!
Unifying and optimizing state definitions is a very good thing. I like the
idea of 'definition goes before using', so overall +1 for this proposal.
However, I think the current definition is somewhat unclear. From a user's
point of view, I believe that
Hi Weijie!
Thank you for the proposal.
I have some initial questions to start the discussion:
1. What is the semantics of the usesStates method? When is it called? Can
the used state change dynamically at runtime? Can the logic depend on
something computed in open(..) for example?
Currently
Hi devs,
I'd like to start a discussion about FLIP-433: State Access on
DataStream API V2
[1]. This is the third sub-FLIP of DataStream API V2.
After FLIP-410 [2], we can already write a simple stateless job using the
DataStream V2 API. But as we all know, stateful computing is Flink's trump
17 matches
Mail list logo
