Hi Naveen, Your expectations about the program immediately giving up memory on deleting an object are wrong. If there is a need for you to have very tight memory controls, you could look into turning GC off entirely and managing memory yourself - See https://blog.twitch.tv/en/2019/04/10/go-memory-ballast-how-i-learnt-to-stop-worrying-and-love-the-heap-26c2462549a2/ for example iirc jvm behaves similarly and doesn't return memory to the OS right away for perf reasons as Ian mentioned.
On Thursday, May 7, 2020 at 2:24:40 PM UTC-7, Ian Lance Taylor wrote: > > On Thu, May 7, 2020 at 11:57 AM Naveen Kak <navee...@gmail.com > <javascript:>> wrote: > > > > Ian, > > Any thoughts on this? Appreciate a response. > > I'm sorry, I'm not sure what you want a response on. > > Using top is a fine way to see total memory usage of your application. > It's a terrible way to examine the Go garbage collector. Using > runtime.ReadMemStats will give you a better understanding of the > garbage collector. > > Whether it makes sense to use sync.Pool depends on your application. > The sync.Pool documentation explains when it is useful. > > Ian > > > > On Tue, 5 May, 2020, 8:34 PM Naveen Kak, <navee...@gmail.com > <javascript:>> wrote: > >> > >> Hi Ian, > >> I explored a few things, calling debug.FreeOSMemory periodically. This > does help, I see a definitely a change in the memory being returned to the > OS ( looking at top o/p). > >> I also set the "GODEBUG=madvdonotneed=1", as per go documentation post > 1.12 release, go release it uses "madvfree" option which is basically a > lazy free to the OS. > >> This didn't surprisingly did not have any effect. > >> So one thing for sure that deleting map definitely doesn't have any > bug, its the way GC is working, not releasing everything back to OS ( I > think after running for 12 hours or so, if we leave the system idle i don't > think memory gets released back to OS, GC probably thinks it will be > required to ask for memory so holds on to it unless we call the > debug.FreeOSMemory periodically). > >> > >> What you think about using "sync pools" so that there are no frequent > memory allocations/de-allocations?, haven't explored this much yet. > >> Another thing, calling debug.FreeOSMemory periodically does cause CPU > spikes. > >> > >> Regards > >> Naveen > >> > >> > >> > >> > >> On Wed, Apr 29, 2020 at 2:16 AM Ian Lance Taylor <ia...@golang.org > <javascript:>> wrote: > >>> > >>> On Tue, Apr 28, 2020 at 1:00 PM Naveen Kak <navee...@gmail.com > <javascript:>> wrote: > >>>> > >>>> Basically using the Top command at end of test. > >>> > >>> > >>> The top command will show you the memory that the program has > requested from the operating system and has not returned to the operating > system. The Go memory allocator works by requesting memory from the > operating system as it needs it. The Go garbage collector works by looking > at that memory and marking it as available for future allocations by the Go > memory allocator. The Go garbage collector does not immediately return > memory to the operating system. That is because requesting from and > returning to the operating system are relatively slow operations, and a > program that has needed memory once is likely to need it again. > >>> > >>> So the top program is not a good way to judge what the garbage > collector is doing. It is an OK way to judge the maximum memory use of the > program, which will include memory that has been allocated and memory that > has been garbage collected. > >>> > >>> If a Go program runs for a while with excess memory, it will slowly > return it to the operating system. You can encourage that process by using > runtime/debug.FreeOSMemory. > >>> > >>> In general, though, if you want to examine the garbage collector, I > recommend that you use runtime.ReadMemStats rather than top. > >>> > >>> Ian > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/de8ddca2-45b9-4d3c-9e69-bba6f62140bb%40googlegroups.com.
