Thanks for the discussion. I will go ahead then and refactor the code
without touching the HighAvailabilityServices interface. We will keep the
per-component factory methods and return a single leader election instance
for all of them. Other HighAvailabilityServices implementations can then
come up
thanks to Matthias, I read the previous email here, and I will express my
own views on some issues
@Matthias
My opinion is that the scheme of high-availability splitting of each
component should be retained. As mentioned in David, when we need to split
each component, we need to use each LeaderEl
Hi David,
Thanks for the explanation. Yes, I understand we are figuring out whether
it is OK to remove the existing interfaces. I definitely agree that we
should preserve the capability to eventually split components out to
separate processes.
I guess my question is, suppose we remove the interfa
Hi Dong,
> Adding regarding the effort to add back the per-component election
capability: given that the implementation already follows per-process
election, and given that there will likely be a lot of extra
design/implementation/test effort needed to achieve the use-cases described
above, maybe
Hi Matthias,
Thanks for the explanation. I was trying to understand the concrete
user-facing benefits of preserving the flexibility of per-component leader
election. Now I get that maybe they want to scale those components
independently, and maybe run the UI in an environment that is more accessib
Hi Chesnay,
I like the use-cases (e.g. running multiple UIs for load-balancing
purposes) mentioned. On the other hand, these are probably not
high-priority features, and we don't know when the community will get to
implement these features. It seems a bit over-design to add implementation
complexi
I generally agree that the internals of the HA services are currently
too complex, but I'm wondering if the proposal doesn't go a bit too far
to resolve those.
Is there maybe some way we can refactor things internally to reduce
complexity while keeping the per-component semantics?
Ultimately,
Hi Matthias,
Thanks for the proposal! I am in favor of cleaning up this interface, and
It seems a bit cumbersome now. Especially, the implementation of
per-component leader election has been removed from our current code path.
To be honest, I don't like the per-component approach. I'm even often
Hi Dong,
see my answers below.
Regarding "Interface change might affect other projects that customize HA
> services", are you referring to those projects which hack into Flink's
> source code (as opposed to using Flink's public API) to customize HA
> services?
Yes, the proposed change might affe
Hi Matthias,
Thanks for the proposal! Overall I am in favor of making this interface
change to make Flink's codebase more maintainable.
Regarding "Interface change might affect other projects that customize HA
services", are you referring to those projects which hack into Flink's
source code (as
Hi everyone,
The Flink community introduced a new way how leader election works in Flink
1.15 with FLINK-24038 [1]. Instead of a per-component leader election, all
components (i.e. ResourceManager, Dispatcher, REST server, JobMaster) use a
single (per-JM-process) leader election instance. It was m
11 matches
Mail list logo
