keith-turner commented on a change in pull request #2422:
URL: https://github.com/apache/accumulo/pull/2422#discussion_r790981594
##########
File path:
core/src/main/java/org/apache/accumulo/core/clientImpl/TabletServerBatchReaderIterator.java
##########
@@ -497,26 +499,44 @@ private void
doLookups(Map<String,Map<KeyExtent,List<Range>>> binnedRanges,
for (final String tsLocation : locations) {
final Map<KeyExtent,List<Range>> tabletsRanges =
binnedRanges.get(tsLocation);
- if (maxTabletsPerRequest == Integer.MAX_VALUE || tabletsRanges.size() ==
1) {
- QueryTask queryTask = new QueryTask(tsLocation, tabletsRanges,
failures, receiver, columns);
- queryTasks.add(queryTask);
+ if (options.isUseScanServer()) {
+ // Ignore the tablets location and find a scan server to use
+ ScanServerLocator ssl = context.getScanServerLocator();
+ tabletsRanges.forEach((k, v) -> {
+ try {
+ String location = ssl.reserveScanServer(new TabletIdImpl(k));
Review comment:
This seems really expensive doing an RPC to ZK before a scan and an RPC
to ZK after the scan to unreserve. I was thinking we could possibly just
select a scan server and try to use it and if its busy it would throw some sort
of thrift exception that indicates its busy. When the client gets the busy
exception it will try another scan server, similar to the reservation failing
here. However this does not require going to ZK and also the busy check on scan
servers could probably be orders of magnitude faster than ZK as it would not
require persisting data in the ZK edit log and getting ZK server side quorom
for a write.
One way a busy check could work on a scan server is that it has a fixed
number of threads in a thread pool using a
[SynchronousQueue](https://docs.oracle.com/javase/8/docs/api/java/util/concurrent/SynchronousQueue.html).
When a scan task is submitted to this thread pool and all threads are busy
the thread pool should reject it. When the thread pool rejection happens then
we can turn around and throw the busy thrift exception. All of this should be
really quick on the scan server.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
