A few hundred milliseconds is pretty short. Are you certain you don’t have a
memory leak?
Still, just run N request handler PROCESSES each with 1/N the heap you have
allocated now.
If each of these continue to grow to unmanageable size - simply kill and spawn
a new request process - with a lo
On Monday, October 30, 2023 at 10:12:08 PM UTC-7 Robert Engels wrote:
What is the average wall time of a request?
The average latency is a few hundred milliseconds.
Based on what you wrote it appears that handling a single request generates
a lot of garbage - high allocation rate - and for
What is the average wall time of a request?
Based on what you wrote it appears that handling a single request generates a
lot of garbage - high allocation rate - and for this to be significant I
suspect the runtime is also significant - which implies to me a spawn and
destroy request handler is
Hi Michael, Jason and Robert, thanks a lot for the replies and suggestions!
I did some profiling today, here are some specific findings:
1, CPUs used for GC is around 35% after we enabled soft memory limit, and
it was 45%+ before. I don't have too much experience here on how much CPU
we should s
I second Jason's message, and +1 to off-heap memory as a last resort. Here
are a few more details:
For a starting point on how to reduce memory allocations directly, see
https://go.dev/doc/gc-guide#Optimization_guide. Note that this may require
restructuring your program in places. (e.g. passin
If the objects are created and discarded you might be able to spawn a new
process to handle the request - or maybe a bunch - then just kill it and start
a new process. Possibly use shared memory for constant data.
This is the poor man’s generational garbage collector.
> On Oct 29, 2023, at 9:
Hi there,
We have a large-scale recommendation system serving millions of users which
is built using Golang. It has worked well until recently when we are trying
to enlarge our index or candidate pool by 10X in which case the number of
candidate objects created to serve each user request can al
