yashmayya commented on code in PR #14946:
URL: https://github.com/apache/pinot/pull/14946#discussion_r1936991491
##########
pinot-broker/src/main/java/org/apache/pinot/broker/requesthandler/MultiStageBrokerRequestHandler.java:
##########
@@ -112,6 +118,7 @@ public MultiStageBrokerRequestHandler(PinotConfiguration
config, String brokerId
CommonConstants.MultiStageQueryRunner.KEY_OF_MULTISTAGE_EXPLAIN_INCLUDE_SEGMENT_PLAN,
CommonConstants.MultiStageQueryRunner.DEFAULT_OF_MULTISTAGE_EXPLAIN_INCLUDE_SEGMENT_PLAN);
_queryThrottler = queryThrottler;
+ _queryCompileExecutor = Executors.newCachedThreadPool(new
NamedThreadFactory("query-compile-executor"));
Review Comment:
> Imagine a situation where users are running tons of queries that are
expensive to parse. In that case, having a large number of threads running
optimizations would be expensive and make the system unresponsive. In the same
case, using a fixed thread pool smaller than the number of CPUs would queue up
queries, but other parts of the system shouldn't be affected.
My main concern is potential increase in query latency due to this queueing
but I think the only scenario where that would occur is the one you said with
tons of queries that are super expensive to compile / optimize - and in this
scenario, I think stability is anyway going to be more important to maintain
than query latency.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
