On Fri, Nov 1, 2019 at 6:31 AM Tom Payne <twpa...@gmail.com> wrote: > > cgo is often used to provide bindings to C libraries. Any memory allocated in > the C library is not visible to Go, so Go does not have an accurate view of > the program's memory usage and does not run the garbage collector or any > finalizers often enough. Consequently, memory usage for a Go server that uses > cgo heavily can grow very large, with Go itself being utterly unaware of it. > > If the C functions allocate memory, historically you could set a finalizer to > free the memory sometime after there are no remaining references to it in Go, > as, for example, described in this blog post. However, the current Go docs on > runtime.SetFinalizer state: > > > There is no guarantee that finalizers will run before a program exits, so > > typically they are useful only for releasing non-memory resources > > associated with an object during a long-running program. > > Are there any other ways to automatically release memory resources associated > with an object? Or telling Go's memory manager that the small object it sees > in the Go world is but the tip of an iceberg of memory allocated in the C > world and therefore should be finalized? > > Not-good options include: > - Requiring the programmer to make explicit calls to free the memory > resources when they believe the object is no longer needed, but this takes us > back to the painful world of C's manual memory management and is easy to get > wrong. > - Padding Go objects associated with C memory resources with large, unused > fields (e.g. an [1024]byte) in the hope that the Go garbage collector will be > more likely to finalize and free them when they are unused. > - Avoiding cgo in any server code. > > Are there any good options?
If you are using C code, you are using C's memory resource policy. So the best approach is to use explicit calls to free. I agree that that takes you back to the painful world of C's manual memory management, but, after all, if you are using C then you can't pretend that you are not using C. That said, although Go is careful not to promise that it will actually execute finalizers, in practice it does run them. Padding the Go object won't make any difference as to when a finalizer is run. It will be run in the next full GC cycle after the Go value is no longer referenced. If there are times when the program knows that there is a lot of C memory that is no longer required, it can help by calling runtime.GC itself. In practice I think the best approach is a hybrid: free the C memory explicitly, but also add a finalizer. In the finalizer, free the C memory, and log the fact that you are doing so, along with any information that will help identify where the C memory was allocated. Then periodically check your logs for cases where the finalizer ran, and add the necessary calls to explicitly free the C memory manually. Of course, when you free the C memory manually, don't forget to clear the finalizer. Ian -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUpD9d59Hk7CZECwBd87QN5uyyunWxwKGSpMqnrCSAuLA%40mail.gmail.com.
