Re: [go-nuts] Memory usage

2021-12-15 Thread Gareth Owenson
Thanks both, immensely helpful in narrowing the problem. On Tue, Dec 14, 2021 at 7:12 AM Miha Vrhovnik wrote: > And if you are using a cgo and didn't replace the standard malloc with > jemaloc you are probably suffering from memory fragmentation or whatever > the hell happens. > We had to

Re: [go-nuts] Memory usage

2021-12-13 Thread Miha Vrhovnik
And if you are using a cgo and didn't replace the standard malloc with jemaloc you are probably suffering from memory fragmentation or whatever the hell happens. We had to restart the processors every 1000 iterations because of that. There was no memory leak according to valgrind. (about

Re: [go-nuts] Memory usage

2021-12-13 Thread Kurtis Rader
If you're using cgo to link with C/C++ code then a memory leak in that API is the first place to start looking.. On Mon, Dec 13, 2021 at 10:14 AM Gareth Owenson wrote: > Hi all > > We have a go application that processes large numbers of documents. It > behaves like it has a memory leak,

[go-nuts] Memory usage

2021-12-13 Thread Gareth Owenson
Hi all We have a go application that processes large numbers of documents. It behaves like it has a memory leak, eventually exhausting system ram after a few hours. I've setup pprof and looked at the various outputs, and that is adamant that the application is only using some 50MB of ram,

[go-nuts] memory usage through "json.NewDecoder(res.Body).Decode()"

2021-10-26 Thread RS
err = json.NewDecoder(res.Body).Decode() if err != nil { log.Println("ERROR:", err) return } Using json.newdecoder to decode the coming response body from client? Here is also the issue of " loading the entire response in memory and parsing it" is a matter? As I am not using a buffer to copy?

[go-nuts] Memory usage do not reconcile between pprof vs top

2018-03-17 Thread Eddie Chan
hi all, I've been doing some memory profiling on my application, but I see there's a big disparity between the memory usage reported by pprof and what the system think it is using. My application binary (compiled with go1.9.2) is about 22MB running on FreeBSD v11 on AWS (but I see the

[go-nuts] Memory usage and go tool pprof

2016-11-21 Thread Alexander Petrovsky
Hello! I'm a golang newbie, so could anyone help me with strange memory allocation by my program. What I have from "runtime/pprof": > Entering interactive mode (type "help" for commands) > (pprof) top > 25987.89MB of 26181.92MB total (99.26%) > Dropped 59 nodes (cum <= 130.91MB) > flat