clintropolis commented on issue #8578: parallel broker merges on fork join pool URL: https://github.com/apache/incubator-druid/pull/8578#issuecomment-548137628 Ok, apologies for going dark while I was running benchmarks and making minor adjustments over the last couple weeks; I have some good findings and I'll try to capture everything here across a few comments to follow. **tl;dr** though is that I have changed the defaults to `100ms` run time (using CPU time as the basis of measurement), batch sizes of `4096` results, and `16384` as the number of results to process (4 batches) for the initial task. Based on my observations, this configuration seems to perform the best across different result set sizes, blocking patterns, and concurrency loads. I've also gained more confidence that this will be an acceptable default for most workloads where the broker is not expected to be constantly overloaded with significantly more active requests than the number of cores it has, though this approach can even be tuned to run reasonably well in this case. Next we can see if all of you will come to the same conclusion :+1:
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@druid.apache.org For additional commands, e-mail: commits-h...@druid.apache.org
