Hi mates, got a question about different state backends. As I've properly understood, on every checkpoint, Flink flushes it’s current state into backend. In case of FsStateBackend we’ll have a separate file for each checkpoint, and during the job lifecycle we got a risk of a huge amount of state files in hdfs, that is not very cool for a hadoop name-node.
Does Flink have any clean-up strategies for it’s state in different implementation of backends ? If you could provide any links, where I could read about more details of this process, it’ll be awesome )) Thx a lot for your help. Sincerely yours, Rinat Sharipov Software Engineer at 1DMP CORE Team email: r.shari...@cleverdata.ru <mailto:a.totma...@cleverdata.ru> mobile: +7 (925) 416-37-26 CleverDATA make your data clever
