GWphua commented on code in PR #18731:
URL: https://github.com/apache/druid/pull/18731#discussion_r2712085454
##########
processing/src/main/java/org/apache/druid/query/groupby/epinephelinae/LimitedBufferHashGrouper.java:
##########
@@ -571,7 +585,19 @@ public void adjustTableWhenFull()
size = numCopied;
tableBuffer = newTableBuffer;
+ updateMaxTableBufferUsedBytes();
growthCount++;
}
+
+ @Override
+ protected void updateMaxTableBufferUsedBytes()
+ {
+ long currentBufferUsedBytes = 0;
+ for (ByteBuffer buffer : subHashTableBuffers) {
+ currentBufferUsedBytes += buffer.capacity();
+ }
Review Comment:
Hello, I have added the tests for the groupers.
Maybe you have sent queries that require a large number of bytesUsed?
How I did in my tests is to query with spill to disk enabled:
1. Set druid.processing.buffer.sizeBytes = 1GB
2. Query on a dataset. (Let's say the results for this is 100MB)
3. Set druid.processing.buffer.sizeBytes to a much smaller value ~5MB
4. Query on the same dataset, and watch the usage metrics cap at 5MB, with
spillage to disk ~95MB.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
