GWphua commented on issue #19505: URL: https://github.com/apache/druid/issues/19505#issuecomment-4516653213
Throwing some suggestions here just to keep the discussion going: 1. Inspiration from #12514: Turn off per-hydrant thread renaming in the realtime path by default. SpecificSegmentQueryRunner renames the processing thread by default via setProcessingThreadNames=true at SpecificSegmentQueryRunner.java (line 71). After #15757, this happens once per hydrant in SinkQuerySegmentWalker.java (line 311). Add a constructor/default flag so the Peon hydrant path defaults this to false, while preserving existing behavior elsewhere. 2. Use flat hydrant merging only for query types that need it. #15757 flattened non-bySegment hydrants at SinkQuerySegmentWalker.java (line 343), mainly to help GroupBy memory behavior. For segmentMetadata, the flat shape routes many cheap hydrant runners through SegmentMetadataQueryRunnerFactory.mergeRunners, which is a ChainedExecutionQueryRunner at SegmentMetadataQueryRunnerFactory.java (line 190). A fix could keep the flat path for GroupBy, but restore per-sink/nested merging for segmentMetadata and maybe other cheap metadata-style queries. 3. Collapse the per-hydrant wrapper stack. Each hydrant currently gets multiple wrappers: segment metrics, cache/wait metrics, CPU metrics, and SpecificSegmentQueryRunner at SinkQuerySegmentWalker.java (line 249). #17170 restored per-sink metric emission, but implementation still pays per-hydrant wrapper overhead. A better implementation could merge metrics/CPU/specific-segment behavior into fewer wrappers or move metrics accumulation around the per-sink merged runner. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
