keith-turner opened a new pull request, #5375: URL: https://github.com/apache/accumulo/pull/5375
While profiling bulk load v2 noticed that the manager would send a message to a tablet that contained a lot of tablets to load. The tablet server would process each tablet one at time and do a metadata write for it. This caused a lot of serial metadata writes per tablet server which caused this part of bulk import to take longer. Attempted to parallelize these metadata writes by changing the manager to send a RPC per tablet. The hope was that the tablet server would process each RPC request in a separate thread and this would avoid the serial metadata writes. However this is not currently working and I am not sure why. The manager is getting a thrift client and then sending a lot of one way RPCs to load tablets. These one way messages all appear to be being processed by a single thread on the tablet servers. Still investigating why this happening, if anyone knows more about this please let me know. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
