keith-turner commented on code in PR #2665:
URL: https://github.com/apache/accumulo/pull/2665#discussion_r884057335
##########
core/src/main/java/org/apache/accumulo/core/conf/Property.java:
##########
@@ -395,6 +395,74 @@ public enum Property {
+ "indefinitely. Default is 0 to block indefinitely. Only valid when
tserver available "
+ "threshold is set greater than 0. Added with version 1.10",
"1.10.0"),
+ // properties that are specific to scan server behavior
+ @Experimental
+ SSERV_PREFIX("sserver.", null, PropertyType.PREFIX,
+ "Properties in this category affect the behavior of the scan servers",
"2.1.0"),
+ @Experimental
+ SSERV_DATACACHE_SIZE("sserver.cache.data.size", "10%", PropertyType.MEMORY,
+ "Specifies the size of the cache for RFile data blocks on each scan
server.", "2.1.0"),
+ @Experimental
+ SSERV_INDEXCACHE_SIZE("sserver.cache.index.size", "25%", PropertyType.MEMORY,
+ "Specifies the size of the cache for RFile index blocks on each scan
server.", "2.1.0"),
+ @Experimental
+ SSERV_SUMMARYCACHE_SIZE("sserver.cache.summary.size", "10%",
PropertyType.MEMORY,
+ "Specifies the size of the cache for summary data on each scan server.",
"2.1.0"),
+ @Experimental
+ SSERV_DEFAULT_BLOCKSIZE("sserver.default.blocksize", "1M",
PropertyType.BYTES,
+ "Specifies a default blocksize for the scan server caches", "2.1.0"),
+ @Experimental
+ SSERV_CACHED_TABLET_METADATA_EXPIRATION("sserver.cache.metadata.expiration",
"5m",
+ PropertyType.TIMEDURATION, "The time after which cached tablet metadata
will be refreshed.",
+ "2.1.0"),
+ @Experimental
+ SSERV_PORTSEARCH("sserver.port.search", "true", PropertyType.BOOLEAN,
+ "if the ports above are in use, search higher ports until one is
available", "2.1.0"),
+ @Experimental
+ SSERV_CLIENTPORT("sserver.port.client", "9996", PropertyType.PORT,
+ "The port used for handling client connections on the tablet servers",
"2.1.0"),
+ @Experimental
+ SSERV_MAX_MESSAGE_SIZE("sserver.server.message.size.max", "1G",
PropertyType.BYTES,
+ "The maximum size of a message that can be sent to a scan server.",
"2.1.0"),
+ @Experimental
+ SSERV_MINTHREADS("sserver.server.threads.minimum", "2", PropertyType.COUNT,
+ "The minimum number of threads to use to handle incoming requests.",
"2.1.0"),
+ @Experimental
+ SSERV_MINTHREADS_TIMEOUT("sserver.server.threads.timeout", "10s",
PropertyType.TIMEDURATION,
Review Comment:
Based on the problems I saw in testing, I think this default should be
different. But not sure what it should be yet.
##########
core/src/main/java/org/apache/accumulo/core/util/threads/ThreadPools.java:
##########
@@ -474,6 +474,8 @@ public ThreadPoolExecutor createThreadPool(int coreThreads,
int maxThreads, long
public ThreadPoolExecutor createThreadPool(int coreThreads, int maxThreads,
long timeOut,
TimeUnit units, final String name, BlockingQueue<Runnable> queue,
OptionalInt priority,
boolean emitThreadPoolMetrics) {
+ LOG.debug("Creating ThreadPoolExecutor for {} with {} core threads and {}
max threads", name,
+ coreThreads, maxThreads);
Review Comment:
```suggestion
LOG.debug(
"Creating ThreadPoolExecutor for {} with {} core threads and {} max
threads {} {} timeout",
name, coreThreads, maxThreads, timeOut, units);
```
During testing of scan servers under high load I was seeing really erratic
behavior. I noticed that the thread pool that serviced RPC request was cycling
through lots of threads (like 1,000's of thread being created and destroyed in
the pool per minute). Turns out the thread pool timeout defaults to 10s. I
changed it to zero in my config for no thread pool timeout and the performance
problems and erratic behavior went away. I don't fully understand the problem
at this point, but adding the timeout to this debug is useful.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
