pchang388 commented on issue #12701: URL: https://github.com/apache/druid/issues/12701#issuecomment-1183616550
Just in case - including for reference, this is our relevant configuration options for Overlord and MM/Peons: 1. Overlord * runtime.properties ``` druid.service=druid/overlord druid.plaintextPort=8090 # runner druid.indexer.queue.startDelay=PT1M druid.indexer.runner.type=remote druid.indexer.storage.type=metadata druid.indexer.runner.taskAssignmentTimeout=PT10M druid.indexer.runner.taskShutdownLinkTimeout=PT3M ``` * jvm.config ``` -server -Xms15g -Xmx15g -XX:MaxDirectMemorySize=10500m -XX:+ExitOnOutOfMemoryError -XX:+UseG1GC -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Djava.io.tmpdir=var/tmp -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -Dservice=overlord ``` 2. MM/Peon * runtime.properties ``` druid.service=druid/middleManager druid.plaintextPort=8091 # Number of tasks per middleManager druid.worker.capacity=10 # Task launch parameters druid.indexer.runner.javaOptsArray=["-server", "-Xmx3g", "-Xms3g", "-XX:MaxDirectMemorySize=10500m", "-Duser.timezone=UTC", "-Dfile.encoding=UTF-8", "-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager", "-Dlog4j.configurationFile=conf/druid/cluster/_common/log4j2-task.xml"] druid.indexer.task.baseTaskDir=/opt/druid/segment-cache0/task # Processing threads and buffers on Peons druid.indexer.fork.property.druid.processing.numMergeBuffers=2 druid.indexer.fork.property.druid.processing.buffer.sizeBytes=100000000 druid.indexer.fork.property.druid.processing.numThreads=4 # Peon http threads - https://druid.apache.org/docs/latest/operations/basic-cluster-tuning.html#sizing-the-connection-pool-for-queries druid.indexer.fork.property.druid.server.http.numThreads=60 # Processing threads and buffers druid.processing.buffer.sizeBytes=500000000 druid.processing.numMergeBuffers=4 druid.processing.numThreads=15 druid.processing.tmpDir=/opt/druid/segment-cache1/processing # cache druid.realtime.cache.useCache=True druid.realtime.cache.populateCache=True druid.cache.sizeInBytes=256000000 druid.cache.type=caffeine ``` * jvm.config ``` -server -XX:+UseG1GC -Xms256m -Xmx256m -XX:MaxDirectMemorySize=10500m -XX:+ExitOnOutOfMemoryError -Duser.timezone=UTC -Dfile.encoding=UTF-8 -Djava.io.tmpdir=var/tmp -Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager -Dservice=middlemanager ``` As mentioned, I will try to get some peon flame graphs for you @abhishekagarwal87 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
