yashmayya opened a new pull request, #19816: URL: https://github.com/apache/pinot/pull/19816
Multi-stage (MSE) queries report heap allocation only per operator, inside `stageStats`. The top-level `*MemAllocatedBytes` fields are always 0 for MSE, so users have no single number to find the queries that allocate the most memory. This PR adds `totalMemAllocatedBytes` to the MSE broker response. The broker computes it from stats that already exist: - The `allocatedMemoryBytes` of each stage root. A root's value already includes the other operators of its op chain (since #19365). - The `threadMemAllocatedBytes` of each leaf. The single-stage threads of the leaf are not part of the stage root's value. - The `allocatedMemoryBytes` of each pipeline breaker, which runs in its own op chain. Notes: - A sum of `allocatedMemoryBytes` over all operators gives a number that is too large, because each value includes the children in the same stage. - Like the per-operator stats, the value is 0 unless `pinot.server.instance.enableThreadAllocatedBytesMeasurement` and `pinot.broker.instance.enableThreadAllocatedBytesMeasurement` are enabled. - The value is the bytes allocated, which includes short-lived objects. It is not the peak heap that the query used. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
