Hi all,

1. I have learned that the /metrics endpoint will be requested by more than
one metrics collect system. In the condition, I want to reimplement
`PromethuesMetricsServlet` by sliding window.
PrometheusMetricsGenerator#generate will be invoked once in a period(such
as 1 minute), the result will be cached and returned for every metrics
collect request in the period directly. It could save memory and avoid high
CPU usage.

2. When there are hundreds MB metrics data collected, it causes high heap
memory usage, high CPU usage and GC pressure. In the
`PrometheusMetricsGenerator#generate` method, it uses
`ByteBufAllocator.DEFAULT.heapBuffer()` to allocate memory for writing
metrics data. The default size of `ByteBufAllocator.DEFAULT.heapBuffer()`
is 256 bytes, when the buffer resizes, the new buffer capacity is 512
bytes(power of 2) and with `mem_copy` operation.
If I want to write 100 MB data to the buffer, the current buffer size is
128 MB, and the total memory usage is close to 256 MB (256bytes + 512 bytes
+ 1k + .... + 64MB + 128MB). When the buffer size is greater than netty
buffer chunkSize(16 MB), it will be allocated as UnpooledHeapByteBuf in the
heap. After writing metrics data into the buffer, return it to the client
by jetty, jetty will copy it into jetty's buffer with memory allocation in
the heap, again!
In this condition, for the purpose of saving memory, avoid high CPU
usage(too much memory allocations and `mem_copy` operations) and reducing
GC pressure, I want to change `ByteBufAllocator.DEFAULT.heapBuffer()` to
`ByteBufAllocator.DEFAULT.compositeDirectBuffer()`, it wouldn't cause
`mem_copy` operations and huge memory allocations(CompositeDirectByteBuf is
a bit slowly in read/write, but it's worth). After writing data, I will
call the `HttpOutput#write(ByteBuffer)` method and write it to the client,
the method won't cause `mem_copy` (I have to wrap ByteBuf to ByteBuffer, if
ByteBuf wrapped, there will be zero-copy).
I tested NO.2 in my local, and it turns out performance is better than the
heap buffer(below images).

https://drive.google.com/file/d/1-0drrs9s9kZ2NbbVmzQDwPHdgtpE6QyW/view?usp=sharing
(CompositeDirectByteBuf)
https://drive.google.com/file/d/1-0m15YdsjBudsiweZ4DO7aU3bOFeK17w/view?usp=sharing
(PooledHeapByteBuf)

Thanks,
Tao Jiuming

Reply via email to