[go-nuts] Re: Go 1.17.1 runtime seems retain too much memory from OS

2022-11-23 Thread 'Zhao Weng' via golang-nuts
sorry for being not specific.

my question is:

according to my understanding about memstat ( correct me if I'm wrong)

with GOGC set to 50, I can see from the graph above that GCGoal is ~ 26GB
and heapIdle - heapReleased is how much memory go runtime retain from OS 
for later usage, how can it (heapIdle - heapReleased) be 35GB which is 1.3x 
GCGoal ? 

it seem to me is something stop scavenger from releasing those memory back 
to OS.

and by reading the code:

// HeapIdle is bytes in idle (unused) spans.
//
// Idle spans have no objects in them. These spans could be
// (and may already have been) returned to the OS, or they can
// be reused for heap allocations, or they can be reused as
// stack memory.
//
// HeapIdle minus HeapReleased estimates the amount of memory
// that could be returned to the OS, but is being retained by
// the runtime so it can grow the heap without requesting more
// memory from the OS. If this difference is significantly
// larger than the heap size, it indicates there was a recent
// transient spike in live heap size.
HeapIdle uint64

and from mgcscavenge.go:

// That goal is defined as:
// (retainExtraPercent+100) / 100 * (heapGoal / lastHeapGoal) * 
last_heap_inuse

it seems (heapIdle - heapReleased) should be like 3GB (which is 
retainExtraPercent/100 * heapInUse ~ 10% * 26GB ~ 3GB)

and i'm really confused by this situation that (heapIdle - heapReleased) be 
35GB
On Thursday, November 24, 2022 at 3:09:10 PM UTC+8 Amnon wrote:

> Have a look at https://tip.golang.org/doc/gc-guide
>
> and 
> https://opensourcelive.withgoogle.com/events/go-day-2022/watch?talk=talk4
>
> On Thursday, 24 November 2022 at 05:08:55 UTC Zhao Weng wrote:
>
>> [image: memory.jpg]
>> memstats show heapIdle - heapReleased > heapAlloc or heapInUse
>>
>> I try to search the code and found the below comment:
>>
>> // HeapIdle is bytes in idle (unused) spans.
>> //
>> // Idle spans have no objects in them. These spans could be
>> // (and may already have been) returned to the OS, or they can
>> // be reused for heap allocations, or they can be reused as
>> // stack memory.
>> //
>> // HeapIdle minus HeapReleased estimates the amount of memory
>> // that could be returned to the OS, but is being retained by
>> // the runtime so it can grow the heap without requesting more
>> // memory from the OS. If this difference is significantly
>> // larger than the heap size, it indicates there was a recent
>> // transient spike in live heap size.
>> HeapIdle uint64
>>
>> and from mgcscavenge.go:
>>
>> // That goal is defined as: 
>> // (retainExtraPercent+100) / 100 * (heapGoal / lastHeapGoal) * 
>> last_heap_inuse
>>
>> it seem to me heapIdle - heapReleased should be ~ heapInUse * 10%
>>
>> can anybody help me understand this?
>>
>> many thanks!!
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/08146e99-5ea8-49ed-9a65-4b04c362f4d3n%40googlegroups.com.


[go-nuts] Re: Go 1.17.1 runtime seems retain too much memory from OS

2022-11-23 Thread Amnon
Have a look at https://tip.golang.org/doc/gc-guide

and 
https://opensourcelive.withgoogle.com/events/go-day-2022/watch?talk=talk4

On Thursday, 24 November 2022 at 05:08:55 UTC Zhao Weng wrote:

> [image: memory.jpg]
> memstats show heapIdle - heapReleased > heapAlloc or heapInUse
>
> I try to search the code and found the below comment:
>
> // HeapIdle is bytes in idle (unused) spans.
> //
> // Idle spans have no objects in them. These spans could be
> // (and may already have been) returned to the OS, or they can
> // be reused for heap allocations, or they can be reused as
> // stack memory.
> //
> // HeapIdle minus HeapReleased estimates the amount of memory
> // that could be returned to the OS, but is being retained by
> // the runtime so it can grow the heap without requesting more
> // memory from the OS. If this difference is significantly
> // larger than the heap size, it indicates there was a recent
> // transient spike in live heap size.
> HeapIdle uint64
>
> and from mgcscavenge.go:
>
> // That goal is defined as: 
> // (retainExtraPercent+100) / 100 * (heapGoal / lastHeapGoal) * 
> last_heap_inuse
>
> it seem to me heapIdle - heapReleased should be ~ heapInUse * 10%
>
> can anybody help me understand this?
>
> many thanks!!
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b63e9b4b-efb4-4209-aaf1-fb4783b41fbdn%40googlegroups.com.