Re: [go-nuts] Slice reuse + GC

[Leszek Kubik]

Actually *T was not a good example to make my point, as we can start 
debating about the GC walking the references and freeing the data pointed 
to. However when you have []T, and you refer to some continuous buffer of 
memory with some chunks [0:5] (gap) [50:100], etc there's not much benefit 
of "freeing" the elements in-between. Anyway, I assume slices are meant to 
be an abstract pointers to arrays and thus they should only change the 
visibility not the array


On Thursday, March 26, 2020 at 7:19:16 PM UTC+1, Leszek Kubik wrote:
>
> I let you consider an example:
>
> s := make([]*T, 100)
> s1, s2, s3 := s[:50], s[50:], s[:]
> ( x lines of code)
> s1 = s1[:5]
>
> Would you like the GC to free the elements past the last s1 slice len? 
> What if s2, s3 are still used somewhere...
>
>
> On Thursday, March 26, 2020 at 7:01:34 PM UTC+1, robfig wrote:
>>
>> I see, thank you.
>>
>> RE reducing the capacity, I want to distinguish freeing the memory (1) 
>> used for the slice and (2) referred to by the slice elements. I can easily 
>> see that freeing (1) is hard and not so beneficial, but I can't see why (2) 
>> would be difficult, and the benefit seems potentially much larger. In our 
>> case, each slice element has a handle to a sizable []byte, so that's why I 
>> was interested to know if they would remain live. 
>>
>> As an aside, (1) was addressed by Ian here:
>> https://groups.google.com/d/msg/golang-nuts/sVhjhiYLXNg/YNZ_H-JsBAAJ
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/57d4b27d-c650-492c-b931-aab0fbf39c68%40googlegroups.com.

Re: [go-nuts] Slice reuse + GC

[Ian Lance Taylor]

On Thu, Mar 26, 2020 at 11:02 AM robfig  wrote:
>
> RE reducing the capacity, I want to distinguish freeing the memory (1) used 
> for the slice and (2) referred to by the slice elements. I can easily see 
> that freeing (1) is hard and not so beneficial, but I can't see why (2) would 
> be difficult, and the benefit seems potentially much larger. In our case, 
> each slice element has a handle to a sizable []byte, so that's why I was 
> interested to know if they would remain live.
>
> As an aside, (1) was addressed by Ian here:
> https://groups.google.com/d/msg/golang-nuts/sVhjhiYLXNg/YNZ_H-JsBAAJ

It's kind of the same problem for (2).  The garbage collector would
have to find every slice that referred to the memory in question and
check the capacity.  This is hard because the GC has no natural way to
go from the backing array pointer to the capacity associated with that
pointer.  In particular when slices are stored on the stack, there is
no requirement that the capacity be near the backing array pointer or
even to exist at all.  I suppose the GC could simply assume that all
backing array pointers on the stack referred to the entire array, but
then compiler optimizations would affect which elements were freed,
and the behavior would be somewhat unintuitive.

All in all, if you want to make sure that the slice elements are
cleared, you'll have to clear them yourself.  Or use a slice copy
rather than a simple slice expression.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUhApbqw_Zcsa0cJ2ZjGz8Ov4gz0Zx-2BjqNXxgd5JjHQ%40mail.gmail.com.

Re: [go-nuts] Slice reuse + GC

[Leszek Kubik]

I let you consider an example:

s := make([]*T, 100)
s1, s2, s3 := s[:50], s[50:], s[:]
( x lines of code)
s1 = s1[:5]

Would you like the GC to free the elements past the last s1 slice len? What 
if s2, s3 are still used somewhere...


On Thursday, March 26, 2020 at 7:01:34 PM UTC+1, robfig wrote:
>
> I see, thank you.
>
> RE reducing the capacity, I want to distinguish freeing the memory (1) 
> used for the slice and (2) referred to by the slice elements. I can easily 
> see that freeing (1) is hard and not so beneficial, but I can't see why (2) 
> would be difficult, and the benefit seems potentially much larger. In our 
> case, each slice element has a handle to a sizable []byte, so that's why I 
> was interested to know if they would remain live. 
>
> As an aside, (1) was addressed by Ian here:
> https://groups.google.com/d/msg/golang-nuts/sVhjhiYLXNg/YNZ_H-JsBAAJ
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/179c7a55-80a6-497c-9a82-5e2c6db628af%40googlegroups.com.

Re: [go-nuts] Slice reuse + GC

[robfig]

I see, thank you.

RE reducing the capacity, I want to distinguish freeing the memory (1) used 
for the slice and (2) referred to by the slice elements. I can easily see 
that freeing (1) is hard and not so beneficial, but I can't see why (2) 
would be difficult, and the benefit seems potentially much larger. In our 
case, each slice element has a handle to a sizable []byte, so that's why I 
was interested to know if they would remain live. 

As an aside, (1) was addressed by Ian here:
https://groups.google.com/d/msg/golang-nuts/sVhjhiYLXNg/YNZ_H-JsBAAJ

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ef48dc4d-ff17-4648-8fb9-4302f43ff5ae%40googlegroups.com.

Re: [go-nuts] Slice reuse + GC

['Axel Wagner' via golang-nuts]

By the way, I don't think your snippet shows what you think it's showing.
In particular, the output stays the same, even if you reduce the capacity
to 0: https://play.golang.org/p/r2YvnDNsoBg
I also always assumed the GC wouldn't do this optimization (throwing away
memory if it's past the capacity of a slice) anyway. Not because it can't
(I think), but just because I didn't think anyone built it to.

On Thu, Mar 26, 2020 at 6:47 PM Axel Wagner 
wrote:

>
>
> On Thu, Mar 26, 2020 at 6:41 PM robfig  wrote:
>
>> Reducing a slice's length makes the elements unreachable, but the GC
>> appears to still treat them as live, based on this snippet:
>> https://play.golang.org/p/SvsE-nXi-JA
>>
>> I would have expected the HeapInUse to go down in the second measurement.
>>
>> Why is that?
>>
>> I presume that the GC is traversing the slice's capacity rather than
>> length, but I'm not sure why this is necessary. Is there a way (without
>> using unsafe) to access slice elements past the length?
>>
>
> Yes. https://play.golang.org/p/HhZWkGvR37L
>
>
>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/c2bea525-0636-4365-b551-59d1fceb5959%40googlegroups.com
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEkBMfEWDfT8syfWM_n-9%2Be9tcs61xUgQoJ8EJzisUZCJEqLNA%40mail.gmail.com.

Re: [go-nuts] Slice reuse + GC

['Axel Wagner' via golang-nuts]

On Thu, Mar 26, 2020 at 6:41 PM robfig  wrote:

> Reducing a slice's length makes the elements unreachable, but the GC
> appears to still treat them as live, based on this snippet:
> https://play.golang.org/p/SvsE-nXi-JA
>
> I would have expected the HeapInUse to go down in the second measurement.
>
> Why is that?
>
> I presume that the GC is traversing the slice's capacity rather than
> length, but I'm not sure why this is necessary. Is there a way (without
> using unsafe) to access slice elements past the length?
>

Yes. https://play.golang.org/p/HhZWkGvR37L



> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/c2bea525-0636-4365-b551-59d1fceb5959%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEkBMfF-GzA%2BEOEANFZTpAZ_CXTNJhLTyY%2B4xZs%2BVa505CGYoQ%40mail.gmail.com.

[go-nuts] Slice reuse + GC

[robfig]

Reducing a slice's length makes the elements unreachable, but the GC 
appears to still treat them as live, based on this snippet:
https://play.golang.org/p/SvsE-nXi-JA

I would have expected the HeapInUse to go down in the second measurement. 

Why is that? 

I presume that the GC is traversing the slice's capacity rather than 
length, but I'm not sure why this is necessary. Is there a way (without 
using unsafe) to access slice elements past the length? 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/c2bea525-0636-4365-b551-59d1fceb5959%40googlegroups.com.