Hi Tison, I'm not sure whether I fully understand your distinction between customizable and pluggable. Maybe you could clarify your ideas a bit because you seem to favour support for pluggable implementations.
Maybe let me try to answer some other questions you raised. With the HighAvailabilityServices interface and the functionality to load custom implementations specified via `high-availability: FQDN` it is indeed possible to provide a custom implementation of the HA services. This is, however, pretty much a power user feature where users have to implement against an internal API. As such, it can be subject to change and we don't give guarantees that these interfaces won't change. Of course, if possible, we should extend/change it in a way that we guarantee backwards compatibility. You are right that at the moment this interface is not stable enough for being public API. Once this changes and we are happy with it, then we can think about making it a public API and documenting it properly. Afaik, there is no documentation how to implement your own HA services at the moment. This underlines as well that this interface is an internal API. Cheers, Till On Fri, Oct 18, 2019 at 5:56 AM Zili Chen <wander4...@gmail.com> wrote: > Another perspective is that a stable, carefully-designed interface with > clear semantic > could be safer to customize. > > Following the discussion in FLINK-10333 our JobGraphStore is actually > required > performing write operation only with leadership, which is a basic > requirement > for coordination rather than an implementation detail. > Thus it depends on LeaderElectionService(in the design, we narrow the > specific interface > as LeaderStore). HighAvailabilityServices#getJobGraphStore() infers a > implicit field for > that which is hard to express the relationship between them. > > If the interface is unstable(also we introduce a ClientHAService for > separate concern and > have to keep b/w comp. for customized), we'd better keep it internal for > freely evolution. > And when we try to support customized, it would be helpful to start a > proposal to revisit > the interface to be well-designed and stable. ref[2]. > > In short, current high-availability services as well as > runtime/coordination is still under > development and active evolution. It is possibly not a good time for make > it public and > customizable. > > Best, > tison. > > [1] https://issues.apache.org/jira/browse/FLINK-10333 > [2] > > https://github.com/apache/pulsar/wiki/PIP-45%3A-Pluggable-metadata-interface > > > Zili Chen <wander4...@gmail.com> 于2019年10月17日周四 下午8:13写道： > > > A challenge is how we ensure the support for customized implementation. > > When > > we introduce JobGraphStore#releaseJobGraph we actually change quite a bit > > codepath > > in Dispatcher. While we are unable to test arbitrarily customized > > implementation our > > compatibility promise is actually no more than compilation compatible. > > > > Customer should still be required to be familiar with implementation > > details to figure > > out the fitment when they bump Flink version. This effort requires also > > and no extra > > when we support pluggable strategy. In another word, a customized support > > tends > > to hide the challenge when customer want to use their own implementation. > > >
