Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[Wojciech S. Czarnecki]

On Sun, 27 Oct 2019 20:20:07 +
roger peppe  wrote:

> If two things aren't writing concurrently to the same address, what's the
> problem?

I've got an impression that OP is concerned about arriving at
inconsistent/"impossible" shared state (due to poorly thought architecture).

30-20yrs ago this kind of bug have had haunted codebases still using global
state, where procedure X set some state in S and precomputed S' that was then
being used by procedure Z; but in meantime the S, and only S, had got modified
(by some procedure Y in the middle). So, from the pov of Z (that assumed
S' ← S) we have had got to an "impossible state".
In event-loop driven programs often this happened in seemingly random ways.

@Gert, 
This indeed can easily can be "achieved" in a single threading Go code that
uses slices (pointers) to the shared state. It was, it is, and it always will
be a programmer's error. More, no known to me code analysis tool could
comprehend changes to your shared state in full and then flag only the wrongs.
An AI capable of that would also be capable of replacing us at our desks ;)


Hope this helps,

-- 
Wojciech S. Czarnecki
 << ^oo^ >> OHIR-RIPE

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/20191028112516.750f1649%40zuzia.

Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[Dan Kortschak]

You are absolutely right. Apologies.


On Mon, 2019-10-28 at 08:43 +, roger peppe wrote:
> On Sun, 27 Oct 2019, 21:31 Dan Kortschak,  wrote:
> 
> > This is not necessarily due to races. You can have this exact
> > situation
> > occurring in single threaded code. I don't think any mention of
> > race
> > issues was made here.
> > 
> 
> The thread subject mentions goroutines and the original message talks
> specifically about concurrent access, which sounds very much like
> it's
> race-related to me.
> 
> > 
> > On Sun, 2019-10-27 at 08:53 +, roger peppe wrote:
> > > On Sun, 27 Oct 2019, 00:04 Gert,  wrote:
> > > 
> > > > I believe it's going to be to hard to write a go vet analyser
> > > > that
> > > > could
> > > > warn to detect unintended state change on a array from two
> > > > different
> > > > pointers.
> > > > 
> > > 
> > > Isn't that why we have the race detector?
> > > 
> > > --
> > > > You received this message because you are subscribed to the
> > > > Google
> > > > Groups
> > > > "golang-nuts" group.
> > > > To unsubscribe from this group and stop receiving emails from
> > > > it,
> > > > send an
> > > > email to golang-nuts+unsubscr...@googlegroups.com.
> > > > To view this discussion on the web visit
> > > > 
> > 
> > 
https://groups.google.com/d/msgid/golang-nuts/30ac16bc-bc70-40a6-bc79-f41e3536d144%40googlegroups.com
> > > > <
> > > > 
> > 
> > 
https://groups.google.com/d/msgid/golang-nuts/30ac16bc-bc70-40a6-bc79-f41e3536d144%40googlegroups.com?utm_medium=email&utm_source=footer
> > > > > 
> > > > 
> > > > .
> > > > 
> > > 
> > > 
> > 
> > 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/fcb7fdf90addf5562cceb4051970c110cc883a9d.camel%40kortschak.io.

Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[atd...@gmail.com]

Yes, the issue is twofold.

Because everything is passed by "copy", it might feel safe to pass around 
structs with slice fields, especially and perhaps unknowingly as "interior 
fat pointers" in structs. 
But, one should remember that these objects are always shallow copies in 
the sense that map/slices/functions will refer to the same underlying 
elements and not be dissociated. 
Thus it could be nice to have a way to enforce deeper copies at the 
language level as well.

The second issue that is related to the first one is that in the case of 
mutliple access by different goroutines, 
this problem can be compounded since the objects can seem to be naturally 
thread safe but the slice operations for instance are not. 
Perhaps is it even the same issue if we send slices through channels if for 
some reason, the backing arrays are shared, I don't know. 

Main thing is that I don't trust the fact that many beginners will have 
thought about this (or perhaps it is just me who didn't). 
Especially when passing objects to different functions that may mutate an 
object internals thinking it is just a copy.

On Monday, October 28, 2019 at 9:44:24 AM UTC+1, rog wrote:
>
>
>
> On Sun, 27 Oct 2019, 21:31 Dan Kortschak, > 
> wrote:
>
>> This is not necessarily due to races. You can have this exact situation
>> occurring in single threaded code. I don't think any mention of race
>> issues was made here.
>>
>
> The thread subject mentions goroutines and the original message talks 
> specifically about concurrent access, which sounds very much like it's 
> race-related to me.
>
>>
>> On Sun, 2019-10-27 at 08:53 +, roger peppe wrote:
>> > On Sun, 27 Oct 2019, 00:04 Gert, > 
>> wrote:
>> > 
>> > > I believe it's going to be to hard to write a go vet analyser that
>> > > could
>> > > warn to detect unintended state change on a array from two
>> > > different
>> > > pointers.
>> > > 
>> > 
>> > Isn't that why we have the race detector?
>> > 
>> > -- 
>> > > You received this message because you are subscribed to the Google
>> > > Groups
>> > > "golang-nuts" group.
>> > > To unsubscribe from this group and stop receiving emails from it,
>> > > send an
>> > > email to golan...@googlegroups.com .
>> > > To view this discussion on the web visit
>> > > 
>>
>> https://groups.google.com/d/msgid/golang-nuts/30ac16bc-bc70-40a6-bc79-f41e3536d144%40googlegroups.com
>> > > <
>> > > 
>> https://groups.google.com/d/msgid/golang-nuts/30ac16bc-bc70-40a6-bc79-f41e3536d144%40googlegroups.com?utm_medium=email&utm_source=footer
>> > > >
>> > > .
>> > > 
>> > 
>> > 
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/7560f6d2-ca55-48ea-a874-5b972ddf40a6%40googlegroups.com.

Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[roger peppe]

On Sun, 27 Oct 2019, 21:31 Dan Kortschak,  wrote:

> This is not necessarily due to races. You can have this exact situation
> occurring in single threaded code. I don't think any mention of race
> issues was made here.
>

The thread subject mentions goroutines and the original message talks
specifically about concurrent access, which sounds very much like it's
race-related to me.

>
> On Sun, 2019-10-27 at 08:53 +, roger peppe wrote:
> > On Sun, 27 Oct 2019, 00:04 Gert,  wrote:
> >
> > > I believe it's going to be to hard to write a go vet analyser that
> > > could
> > > warn to detect unintended state change on a array from two
> > > different
> > > pointers.
> > >
> >
> > Isn't that why we have the race detector?
> >
> > --
> > > You received this message because you are subscribed to the Google
> > > Groups
> > > "golang-nuts" group.
> > > To unsubscribe from this group and stop receiving emails from it,
> > > send an
> > > email to golang-nuts+unsubscr...@googlegroups.com.
> > > To view this discussion on the web visit
> > >
>
> https://groups.google.com/d/msgid/golang-nuts/30ac16bc-bc70-40a6-bc79-f41e3536d144%40googlegroups.com
> > > <
> > >
> https://groups.google.com/d/msgid/golang-nuts/30ac16bc-bc70-40a6-bc79-f41e3536d144%40googlegroups.com?utm_medium=email&utm_source=footer
> > > >
> > > .
> > >
> >
> >
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJhgacjs72m7Ozb3ePFJgLUXBAc30mz8FXaVPxP9xLgfPznmfw%40mail.gmail.com.

Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[Dan Kortschak]

This is not necessarily due to races. You can have this exact situation
occurring in single threaded code. I don't think any mention of race
issues was made here.

On Sun, 2019-10-27 at 08:53 +, roger peppe wrote:
> On Sun, 27 Oct 2019, 00:04 Gert,  wrote:
> 
> > I believe it's going to be to hard to write a go vet analyser that
> > could
> > warn to detect unintended state change on a array from two
> > different
> > pointers.
> > 
> 
> Isn't that why we have the race detector?
> 
> -- 
> > You received this message because you are subscribed to the Google
> > Groups
> > "golang-nuts" group.
> > To unsubscribe from this group and stop receiving emails from it,
> > send an
> > email to golang-nuts+unsubscr...@googlegroups.com.
> > To view this discussion on the web visit
> > 
https://groups.google.com/d/msgid/golang-nuts/30ac16bc-bc70-40a6-bc79-f41e3536d144%40googlegroups.com
> > <
> > https://groups.google.com/d/msgid/golang-nuts/30ac16bc-bc70-40a6-bc79-f41e3536d144%40googlegroups.com?utm_medium=email&utm_source=footer
> > >
> > .
> > 
> 
> 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/fd35e7a25f8a7e0ba57b68ceaf9fc95cece08fa8.camel%40kortschak.io.

Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[andrey mirtchovski]

>> I think it only catches concurrent access to the same address, so not sure 
>> if it catches everything.
>
>
> If two things aren't writing concurrently to the same address, what's the 
> problem?

I took that to mean 'things may be updating concurrently at different
addresses inside an array backing the same slice'. But the race
detector will flag as soon as a read is attempted by another goroutine
on any of those.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAK4xykU5nKhU0BFocR0XP-mYD0rY246Hu6Aq9AG95t-Dh7n-PA%40mail.gmail.com.

Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[roger peppe]

On Sun, 27 Oct 2019, 12:33 Gert,  wrote:

>
> On Sunday, October 27, 2019 at 9:54:29 AM UTC+1, rog wrote:
>>
>>
>> On Sun, 27 Oct 2019, 00:04 Gert,  wrote:
>>
>>> I believe it's going to be to hard to write a go vet analyser that could
>>> warn to detect unintended state change on a array from two different
>>> pointers.
>>>
>>
>> Isn't that why we have the race detector?
>>
>
> I think it only catches concurrent access to the same address, so not sure
> if it catches everything.
>

If two things aren't writing concurrently to the same address, what's the
problem?

> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/39c04bde-2c9c-4396-84c1-a71143de5e39%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJhgacj1P6bzWDMA77F1zE7A-2zzeGqFYhFfG95%2Beu9ZMfa4hQ%40mail.gmail.com.

Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[Gert]

On Sunday, October 27, 2019 at 9:54:29 AM UTC+1, rog wrote:
>
>
> On Sun, 27 Oct 2019, 00:04 Gert, > wrote:
>
>> I believe it's going to be to hard to write a go vet analyser that could 
>> warn to detect unintended state change on a array from two different 
>> pointers.
>>
>
> Isn't that why we have the race detector?
>

I think it only catches concurrent access to the same address, so not sure 
if it catches everything.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/39c04bde-2c9c-4396-84c1-a71143de5e39%40googlegroups.com.

Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[roger peppe]

On Sun, 27 Oct 2019, 00:04 Gert,  wrote:

> I believe it's going to be to hard to write a go vet analyser that could
> warn to detect unintended state change on a array from two different
> pointers.
>

Isn't that why we have the race detector?

-- 
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/30ac16bc-bc70-40a6-bc79-f41e3536d144%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJhgachBNf8S_NMDZkq7AOjDsTvZ1pZy_Veo_S-XwgauZ0h_Ww%40mail.gmail.com.

Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[Gert]

On Saturday, October 26, 2019 at 10:22:42 PM UTC+2, atd...@gmail.com wrote:
>
>
> On Saturday, October 26, 2019 at 9:47:39 PM UTC+2, Ian Lance Taylor wrote:
>>
>> On Sat, Oct 26, 2019 at 10:50 AM atd...@gmail.com  
>> wrote: 
>> > 
>> > I get that we can do that but it's not strictly about slices operations 
>> such as appending or copying. 
>> > Of course, we could copy slices all the time but then there is no 
>> benefit in having them and it is not ergonomic as your example shows, nor 
>> does it fit your performance cost requirements. 
>> > 
>> > If you look at the code snippet I posted, I am merely assigning a value 
>> of type slice to a struct field. Now, that struct can be copied a number of 
>> times without its actual value changing. It looks immutable in some sense. 
>>  BUT, it's actually implicitly tied to an object that is mutable (backing 
>> array) and nowhere have we made sure that the mutation of this backing 
>> array is safe. 
>>
>> For better or for worse, the Go language has shared state.  This is 
>> most obvious when using pointers: if you copy a pointer, you do not 
>> copy the memory to which that pointer points.  If that is clear, then 
>> the thing to remember about slices is that they are pointers. 
>> Pointers in Go do not permit arithmetic.  
>
>  
>
>> Slices in Go are pointers 
>> that permit bounded arithmetic. 
>>
>> Oh, I hadn't thought about it that way! Interesting.
> I still feel like there should be a way to reign in the amount of shared 
> state in the cases of these "fat" pointers a little bit. Or at least, 
> inspect it, warn about it more because most people seem to think that 
> slices are just infinite arrays (hence the need for the extra-level of 
> indirection/pointer). 
> But perhaps, the coding patterns/idioms make it a non issue.  For future 
> considerations I guess.
>

I believe it's going to be to hard to write a go vet analyser that could 
warn to detect unintended state change on a array from two different 
pointers. Go made it this far without many programmers throwing monitors 
out of the window so I think will be fine by making sure we document slices 
pointers and interfaces in detail. Yes does 3 pages going to be heavy to 
understand but if you just remember when you encounter weird stuff to just 
look again at does 3 pages.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/30ac16bc-bc70-40a6-bc79-f41e3536d144%40googlegroups.com.

Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[atd...@gmail.com]

On Saturday, October 26, 2019 at 9:47:39 PM UTC+2, Ian Lance Taylor wrote:
>
> On Sat, Oct 26, 2019 at 10:50 AM atd...@gmail.com  > wrote: 
> > 
> > I get that we can do that but it's not strictly about slices operations 
> such as appending or copying. 
> > Of course, we could copy slices all the time but then there is no 
> benefit in having them and it is not ergonomic as your example shows, nor 
> does it fit your performance cost requirements. 
> > 
> > If you look at the code snippet I posted, I am merely assigning a value 
> of type slice to a struct field. Now, that struct can be copied a number of 
> times without its actual value changing. It looks immutable in some sense. 
>  BUT, it's actually implicitly tied to an object that is mutable (backing 
> array) and nowhere have we made sure that the mutation of this backing 
> array is safe. 
>
> For better or for worse, the Go language has shared state.  This is 
> most obvious when using pointers: if you copy a pointer, you do not 
> copy the memory to which that pointer points.  If that is clear, then 
> the thing to remember about slices is that they are pointers. 
> Pointers in Go do not permit arithmetic.  

 

> Slices in Go are pointers 
> that permit bounded arithmetic. 
>
> Oh, I hadn't thought about it that way! Interesting.
I still feel like there should be a way to reign in the amount of shared 
state in the cases of these "fat" pointers a little bit. Or at least, 
inspect it, warn about it more bceause most people seem to think that 
slices are just infinite arrays (hence the need for the extra-level of 
indirection/pointer). 
But perhaps, the coding patterns/idioms make it a non issue.  For future 
considerations I guess.

> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/be142bd1-cbfc-4652-9381-c05f19f5ce81%40googlegroups.com.

Re: [go-nuts] Re: Slices, backing array, goroutines, perhaps Considerations for Go2

[Ian Lance Taylor]

On Sat, Oct 26, 2019 at 10:50 AM atd...@gmail.com  wrote:
>
> I get that we can do that but it's not strictly about slices operations such 
> as appending or copying.
> Of course, we could copy slices all the time but then there is no benefit in 
> having them and it is not ergonomic as your example shows, nor does it fit 
> your performance cost requirements.
>
> If you look at the code snippet I posted, I am merely assigning a value of 
> type slice to a struct field. Now, that struct can be copied a number of 
> times without its actual value changing. It looks immutable in some sense.  
> BUT, it's actually implicitly tied to an object that is mutable (backing 
> array) and nowhere have we made sure that the mutation of this backing array 
> is safe.

For better or for worse, the Go language has shared state.  This is
most obvious when using pointers: if you copy a pointer, you do not
copy the memory to which that pointer points.  If that is clear, then
the thing to remember about slices is that they are pointers.
Pointers in Go do not permit arithmetic.  Slices in Go are pointers
that permit bounded arithmetic.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXPmriK_C9w7GhwyzOyJzzZT8DJ7uvSa-Uq%3DQwBTKJNtg%40mail.gmail.com.