ashishkumar50 commented on PR #11564: URL: https://github.com/apache/ozone/pull/11564#issuecomment-6341773083
> I'm happy for others to add an opinion here too. I guess this cache would have prevented SCM from having trouble during the recent bugs that sent duplicate block deletes, but then perhaps those bugs would have gone unnoticed, which is bad too. We could have more similar bugs in the future and this could prevent an outage, but it could also hide the new bugs. As I said earlier, adding 60MB of heap for this forever feels to me like a lot of resources to address a problem that has already been fixed. > > How does OM handle failed calls to SCM to delete a batch of blocks? Is another potential solution a kind of backoff approach, where SCM could reject a call from OM to delete a batch of blocks if its received too many block deletes recently, or has too many queued? I have thought before that its a good idea to prevent work piling up at the receiver if it cannot cope with it, by using back pressure to slow the sender down. This would also have saved SCM in this case, and metrics could highlight an issue (SCM dropping block delete requests) that may have caused the OM bugs to be noticed rather than hidden. If we do backoff approach then OM also has to do backoff and due to this client request will be rejected at some point. But there is no such restriction to client currently. @sumitagrawl what do you think? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
