Hello Alexey, I think you need FileSystemBlobStore as you are implementing HA Services, and BLOBs should be highly available too. However, I'm a bit concerned about the direction in general: it essentially means re-implementing ZK functionality on top of FS. What are the motivation and the use case?
Regards, Roman On Fri, Aug 28, 2020 at 5:15 PM Alexey Trenikhun <yen...@msn.com> wrote: > Hello, > I'm thinking about implementing FileSystemHaServices - single leader, but > persistent RunningJobRegistry, CheckpointIDCounter, > CompletedCheckpointStore and JobGraphStore. I'm not sure do you need > FileSystemBlobStore or VoidBlobStore is enough. Can't figure out, should > BlobStore survive JobManager crash. I see that ZookeeperHaServices use > FileSystemBlobStore, > but not clear is to due to having multiple JobManagers (leader + follower) > or necessity to preserve BLOBs on restart. > > Thanks, > Alexey >
