swuferhong opened a new issue, #2600: URL: https://github.com/apache/fluss/issues/2600
### Search before asking - [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar. ### Description Check whether kv snapshot exists is an VERY HEAVY operation, if there is 100K bucket, retain 10 snapshots, this will involve 1 million ZK IO operations! AND IN THE LOCK! This will soon become a bottleneck of the system and the Flink job. I think a better solution is to check the availability (and also pin snapshots) via CompletedSnapshotStoreManager. But I think this will involve a lot of code refactoring. I remembered we have discussed this, we have increased the retained snapshot number to 2 (this is the reason we increased it), so it's rarely to happen there is unavailable snapshots when we leasing it. And we can implement the unavailabe check in the future task. So could you remove the zk snapshot check here, and create an issue for this, and add a TODO comment about the missing check with the issue id. See comments: https://github.com/apache/fluss/pull/2179#discussion_r2777753842 ### Willingness to contribute - [ ] I'm willing to submit a PR! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
