keith-turner commented on PR #5375:
URL: https://github.com/apache/accumulo/pull/5375#issuecomment-2695690891
In a5f8b889cb659dd0a3aa4275a12e6599ba6861cc made the following changes
* Removed the oneway from the thrift call
* In the manager changed the code to call send_loadFiles RPC for all
connections w/o waiting on result
* In a second loop go through and call recv_loadFiles for all connections
This change has a nice advantage that is unrelated to initial goal of
parallelization. The current bulk code w/ one way messages only knows if
somethnig is done by scanning the metadata table. Because the changes in
a5f8b889cb659dd0a3aa4275a12e6599ba6861cc waits for the tablets servers it does
not keep scanning the metadata table and then sending more one way messages
queuing up more uneeded work for the tablet servers, causing more metadata
scans, and having to guess when things are done.
The following are some log message from running the new test w/ the changes
in a5f8b889cb659dd0a3aa4275a12e6599ba6861cc
```
$ grep sent Manager_200744172.out
2025-03-03T22:10:44,362 98 [bulkVer2.LoadFiles] DEBUG:
FATE[08ad9a5880022b01] sent 16 messages to 2 tablet servers, send time:8ms recv
time:872ms
2025-03-03T22:10:44,811 98 [bulkVer2.LoadFiles] DEBUG:
FATE[08ad9a5880022b01] sent 0 messages to 0 tablet servers, send time:0ms recv
time:0ms
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
