Re: FLIP-403: High Availability Services for OLAP Scenarios
I just found that I missed the "[DISCUSS]" prefix. Will start another thread for this FLIP. Best, Yangze Guo On Tue, Dec 19, 2023 at 5:31 PM Yangze Guo wrote: > > Thanks for the response, Xiangyu. > > As said in FLIP, this is pure refactoring and should not affect the > functionality of the current HA mechanism. The leader path should not > be changed as well. There will not be any compatibility issue for > existing HA. > > Best, > Yangze Guo > > On Tue, Dec 19, 2023 at 5:05 PM xiangyu feng wrote: > > > > Hi Yangze, > > > > Thanks for driving this. I like the idea to separate the current > > HighAvailabilityServices into LeaderServices and PersistentServices. AFAIK, > > there is no need to > > bind JobGraphStore/JobResultStore/BlobStore/CheckpointStore modules with > > LeaderElection/LeaderRetrieval services. The formers are mainly used to > > store the job related metadata while the latter is designed to distinguish > > the current working jobmanager and standby jobmanager. > > > > I have one question for this, will this FLIP change the leader path or job > > path stored in the HighAvailabilityServices? If user is upgrading a Flink > > job from using current HighAvailabilityServices, will there be any > > compatibility issues, do we need any data migration? > > > > Thx, > > Xiangyu
Re: FLIP-403: High Availability Services for OLAP Scenarios
Thanks for the response, Xiangyu. As said in FLIP, this is pure refactoring and should not affect the functionality of the current HA mechanism. The leader path should not be changed as well. There will not be any compatibility issue for existing HA. Best, Yangze Guo On Tue, Dec 19, 2023 at 5:05 PM xiangyu feng wrote: > > Hi Yangze, > > Thanks for driving this. I like the idea to separate the current > HighAvailabilityServices into LeaderServices and PersistentServices. AFAIK, > there is no need to > bind JobGraphStore/JobResultStore/BlobStore/CheckpointStore modules with > LeaderElection/LeaderRetrieval services. The formers are mainly used to > store the job related metadata while the latter is designed to distinguish > the current working jobmanager and standby jobmanager. > > I have one question for this, will this FLIP change the leader path or job > path stored in the HighAvailabilityServices? If user is upgrading a Flink > job from using current HighAvailabilityServices, will there be any > compatibility issues, do we need any data migration? > > Thx, > Xiangyu
Re: FLIP-403: High Availability Services for OLAP Scenarios
Hi Yangze, Thanks for driving this. I like the idea to separate the current HighAvailabilityServices into LeaderServices and PersistentServices. AFAIK, there is no need to bind JobGraphStore/JobResultStore/BlobStore/CheckpointStore modules with LeaderElection/LeaderRetrieval services. The formers are mainly used to store the job related metadata while the latter is designed to distinguish the current working jobmanager and standby jobmanager. I have one question for this, will this FLIP change the leader path or job path stored in the HighAvailabilityServices? If user is upgrading a Flink job from using current HighAvailabilityServices, will there be any compatibility issues, do we need any data migration? Thx, Xiangyu
