kerneltime commented on PR #10503: URL: https://github.com/apache/ozone/pull/10503#issuecomment-4713937624
@spacemonkd Thank you for reviving this work. This is probably the largest OM write performance project that is in the pipeline of proposals. I took the time to use Opus and go over all the past spec work, review comments and my original design to generate a detailed a very formal spec here https://github.com/kerneltime/ozone/tree/9c59ae0d66227ce10849b4f914f9efdce9318103/hadoop-hdds/docs/content/design/leader-execution I am also experimenting using formal methods to validate the spec and the concurrency model. The docs above include a formal locking proposal that we can base the implementation on. @ivandika3 @spacemonkd has been working with @sumitagrawl and me for this. As for the reason for the performance improvement cause. I believe, that any consensus framework even with batching has a performance envelope in terms of number of concurrent requests in a time frame that it can drive. The size of each slot in the linear write order has some flexibility and adding more requests per slot in the linear history of consensus will bump up the reqs/sec after we reach performance for the consensus framework. This change has a few parts to it 1. Pre compute the DB write and send the DB delta through a consensus round. This simplifies the post Ratis concensus execution and simplifies the caching model. Both these have been a sustained source of bugs and performance challenges. 2. Adds batching when sending a request to Ratis which empirically improves performance more than 3x and reduces overall CPU profile. 3. Eliminates Object ID and other OM correctness from Ratis internals. If in the future if Ratis improves the batching such that batching does not lead to a performance improvement we can make the batch size to be one. Thus, this change overall is a good cleanup of the code that simplifies the processing, caching and improves performance. The use of AI tooling should help us eliminate a lot of the effort involved while keeping our focus on the correctness. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
