Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-16 Thread Yang Wang
> > > > > > > >> > > > > > > [1] https://cwiki.apache.org/confluence/x/QwqZE > > >> > > > > > > > > >> > > > > > > Best, > > >> > > > > > > Rui > > >>

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-16 Thread Yangze Guo
al > >> > > > debt > >> > > > > > > > and a side effect of the FLIP. In order to achieve a cleaner > >> > > > interface > >> > > > > > > > hierarchy for High Availability befo

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-15 Thread Zhu Zhu
ever, I have >> chosen >> > > > the >> > > > > > > > current separation for a clear separation of concerns. After >> > > > FLIP-285, >> > > > > > > > each process has a dedicated LeaderElectionService >> responsible >&

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-15 Thread Zhu Zhu
relevant classes. This > coupling > > > and > > > > > > > > implicit design assumption will increase the complexity and > > > testing > > > > > > > > difficulty of the system. WDYT? > > >

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-10 Thread Yangze Guo
; > need to abstract two services as `LeaderServices` and > > > > > > > > `PersistenceServices`? > > > > > > > > > > > > > > > > From the content, the purpose of this FLIP is to make job > > > failover &g

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-10 Thread Matthias Pohl
gt; leader > > > > > > > service and store for the job that meets the requirements based > > on > > > > the > > > > > > > configuration in the zk/k8s high availability service. > > > > > > > > >

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-09 Thread Zhu Zhu
; > > > > > > On Fri, Dec 29, 2023 at 8:10 PM xiangyu feng < > xiangyu...@gmail.com> > > > > > wrote: > > > > > > > > > > > > > Thanks Yangze for restart this discussion. > > > > > > >

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-08 Thread Yangze Guo
l problems in production where too much job > > > > metadata was > > > > > > being stored on ZK, causing system instability. > > > > > > > > > > > > > > > > > > Yangze Guo 于2023年12月29日周五

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-08 Thread Zhu Zhu
ty. > > > > > > > > > > > > > > > Yangze Guo 于2023年12月29日周五 10:21写道: > > > > > > > > > > > Thanks for the response, Zhanghao. > > > > > > > > > > > > PersistenceServices sounds good

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-08 Thread Yangze Guo
ed, Dec 27, 2023 at 11:30 AM Zhanghao Chen > > > > > wrote: > > > > > > > > > > > > Thanks for driving this effort, Yangze! The proposal overall LGTM. > > > > Other > > > > > from the throughput enhancement in the OL

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-07 Thread Rui Fan
> > > > > Thanks for driving this effort, Yangze! The proposal overall LGTM. > > > Other > > > > from the throughput enhancement in the OLAP scenario, the separation > of > > > > leader election/discovery services and the metadata persistenc

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-07 Thread Yangze Guo
l clearer and easier to maintain. Just a minor > > > comment on naming: would it better to rename PersistentServices to > > > PersistenceServices, as usually we put a noun before Services? > > > > > > > > Best, > > > > Zhanghao Chen > > > > _

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2024-01-07 Thread Yong Fang
aming: would it better to rename PersistentServices to > > PersistenceServices, as usually we put a noun before Services? > > > > > > Best, > > > Zhanghao Chen > > > ________ > > > From: Yangze Guo > > > Sent: Tuesday, Dec

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2023-12-29 Thread xiangyu feng
ust a minor > comment on naming: would it better to rename PersistentServices to > PersistenceServices, as usually we put a noun before Services? > > > > Best, > > Zhanghao Chen > > > > From: Yangze Guo > > Sent: Tuesda

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2023-12-28 Thread Yangze Guo
re Services? > > Best, > Zhanghao Chen > > From: Yangze Guo > Sent: Tuesday, December 19, 2023 17:33 > To: dev > Subject: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios > > Hi, there, > > We would like to st

Re: [DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2023-12-26 Thread Zhanghao Chen
comment on naming: would it better to rename PersistentServices to PersistenceServices, as usually we put a noun before Services? Best, Zhanghao Chen From: Yangze Guo Sent: Tuesday, December 19, 2023 17:33 To: dev Subject: [DISCUSS] FLIP-403: High Availability

[DISCUSS] FLIP-403: High Availability Services for OLAP Scenarios

2023-12-19 Thread Yangze Guo
Hi, there, We would like to start a discussion thread on "FLIP-403: High Availability Services for OLAP Scenarios"[1]. Currently, Flink's high availability service consists of two mechanisms: leader election/retrieval services for JobManager and persistent services for job metadata. However, thes