On Tuesday, December 5, 2023 at 1:06:19 AM UTC-5 Zhihui Jiang wrote:
Hi there, We are running a large scale recommendation system using Golang and we are working on some GC related improvement recently. One of the changes we are trying to apply is to use soft memory limit with GOGC=off as suggested here: https://github.com/golang/go/issues/48409. But during our testing, the memory usage never reaches the memory limit we set. For example, we have 100GB memory available and we set the memory limit to 90GB, but the actual memory usage is very low like <50GB. We also observed the GC is actually very frequent which cost a lot of CPU time. That's odd. Are you positive GOGC=off is set correctly? GOGC=0 is not the same, and will in fact trigger the garbage collector constantly. Can you collect the stderr of your program running with GODEBUG=gctrace=1 set? That'll help identify what's going on. This behavior isn't what I would expect, so if you suspect a bug, please file an issue on GitHub. We had another tweak to set GOGC to a high value like 200, and the memory usage is quite close to the memory limit. I have two questions here: 1. Is it expected behavior that the memory usage is much lower than memory limit when GOGC is set to off? From the official doc ( https://github.com/golang/go/issues/48409), it claims " by setting GOGC=off, the Go runtime will always grow the heap to the full memory limit"? No, I would expect the runtime's total memory use to always be roughly at the soft limit in that case. 1. How is the GC frequency decided when using soft memory limit + GOGC=off? Is there some internal defalut value for GOGC in this case to decide when to GC? The frequency isn't decided directly. The heap goal (the total target heap size) is determined entirely from the memory limit. Approximately: heap goal = memory limit - all other memory the runtime is using. The frequency then depends on how big your live heap is and how fast your application allocates from the live heap size to the total target heap size. Thanks! -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/db43725e-58d0-437b-ac24-4e7051e8a121n%40googlegroups.com.
