[go-nuts] Re: “Normal” vs. “error” (control flow) is a fundamental semantic distinction

[Zhaoxun Yan]

Try & Catch sounds great!

在2022年5月19日星期四 UTC+8 00:15:09 写道：

> Hi all,
>
> I thought now was the time to go public. The automatic error propagation 
> is possible with the help of a simple Go package, and it has been about 
> three years now:
>
> func CopyFile(src, dst string) (err error) {
> defer err2.Returnf(, "copy %s %s", src, dst)
>
> r := try.To1(os.Open(src))
> defer r.Close()
>
> w := try.To1(os.Create(dst))
> defer err2.Handle(, func() {
> _ = os.Remove(dst)
> })
> defer w.Close()
>
> try.To1(io.Copy(w, r))
> return nil
> }
>
> The playground .
>
> A couple of blog posts about the subject:
>
>- Error Propagation 
>
> 
>- Errors as Discriminated Unions 
>
>
> I'm the author of those posts and the err2 package 
> . Additional sources for original ideas 
> are mentioned in the blogs and the package documentation. And, of course, 
> pure error values are needed when transporting errors through channels.
>
> I hope that any gopher who tries the err2 package finds it as valuable and 
> productive
> as we have. Indeed, Go is one of the most pragmatic and refactor-friendly 
> native languages.
>
> Best regards,
> -Harri
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/27a9b63c-ad67-475d-851a-97fadb94b3dfn%40googlegroups.com.

Fwd: [go-nuts] Improving safe global variables with generic or inheritance

[Zhaoxun Yan]

-- Forwarded message -
From: Zhaoxun Yan 
Date: Mon, May 23, 2022 at 6:05 PM
Subject: Re: [go-nuts] Improving safe global variables with generic or
inheritance
To: Brian Candler 

*That's really essential.  For example, using your library, the following
code is most definitely *not* race-free:*

tmp := v.Load()
tmp = tmp + 1
v.Save(tmp)

It is race free as long as `tmp` is a local variable.
Although you may argue that another goroutine may have already altered v,
that is an atomicity issue but won't cause a `panic` or crash. My
resolution is to add a "Tick/Increase" method to substitute the above
procedure.

Also the `value` entry in the struct is in small cases, prohibiting getting
accessed when imported, so the side-attack to directly call `v.value` won't
pass the compile phase, which is much safer than the conventional `atomic`
package.

On Mon, May 23, 2022 at 3:11 PM Brian Candler  wrote:

> > It's best to be intentional about it and explicitly acquire and release
> a mutex around the critical sections of your code - because ultimately,
> only your code knows which sections are critical.
>
> That's really essential.  For example, using your library, the following
> code is most definitely *not* race-free:
>
> tmp := v.Load()
> tmp = tmp + 1
> v.Save(tmp)
>
> The mutex has to protect the entire sequence, not the individual load and
> save operations.
>
> On Monday, 23 May 2022 at 07:57:02 UTC+1 axel.wa...@googlemail.com wrote:
>
>> Just to be clear, are you aware of the sync/atomic package?
>> https://pkg.go.dev/sync/atomic
>> There are also some changes in there for Go 1.18, specifically the
>> addition of some types, so that only atomic operations can be done:
>> https://pkg.go.dev/sync/atomic@master
>> I mention this because atomic.Value and atomic.Pointer[T] are essentially
>> what you are suggesting here.
>>
>> On Mon, May 23, 2022 at 8:04 AM Zhaoxun Yan  wrote:
>>
>>> However, as I want to narrow the scope of this type down to generate
>>> integer types (as in the commented code), it encountered two obstacles:
>>>
>>> 1) It is not legal to embed a generic inside a struct, nor can it make
>>> generalized computation  =+1
>>>
>>> 2) Inheritance is not available in golang, so type "Counter" cannot
>>> inherit type "Global" and get its methods automatically. I need to repeat
>>> Save and Load methods to "Counter".
>>>
>>> Am I correct? Or can you improve it?
>>>
>>
>> You are correct that there is no way in Go to write a type which gets all
>> methods from an embedded field using generics. However, that is a good
>> thing. For example, say you could write
>>
>> type Locked[T any] struct {
>> sync.Mutex
>> T
>> }
>>
>> And this would get the methods of T and the methods of sync.Mutex. Then a
>> user could do
>>
>> type Counter int64
>> func (c *Counter) Increment() { *c += 1 }
>>
>> func main() {
>> var c Locked[Counter]
>> go c.Increment()
>> go c.Increment()
>> }
>>
>> And get a data race. That is, a method can modify a value in a way that
>> is incompatible with what your wrapper type is trying to do.
>>
>> That's really the crux of both of the obstacles you mention. You can't
>> run arbitrary computations and you can't promote methods, because *not all
>> computation and not all methods can be made concurrency safe this way*.
>>
>> It's best to be intentional about it and explicitly acquire and release a
>> mutex around the critical sections of your code - because ultimately, only
>> your code knows which sections are critical.
>>
>>
>>>
>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to golang-nuts...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/golang-nuts/baa74bff-6688-4d39-843b-c99a4fea2d1an%40googlegroups.com
>>> <https://groups.google.com/d/msgid/golang-nuts/baa74bff-6688-4d39-843b-c99a4fea2d1an%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/3javezRHm98/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
>

Re: [go-nuts] Improving safe global variables with generic or inheritance

[Zhaoxun Yan]

`&` won't make any difference:

undefined: atomic.Int32



On Mon, May 23, 2022 at 4:31 PM Axel Wagner 
wrote:

> On Mon, May 23, 2022 at 10:19 AM Zhaoxun Yan 
> wrote:
>
>> Yes but the syntax is confusing, e.g. the code below fails:
>>
>
> That's because you can't take the address of a call expression. That is,
> your code is interpreted as &(a.Load()), while you might have intended
> ().Load() <https://go.dev/play/p/tClXJjahCHG?v=gotip>, which you can
> also just write as a.Load() <https://go.dev/play/p/4HGAscVglX2?v=gotip>.
> That's not really about the API, it's how the language works. Note that
> with your original `Global` type, you get exactly the same error if you
> write ().
>
>
>> package main
>>
>> import (
>> "fmt"
>> "sync/atomic"
>> )
>>
>> var a atomic.Int32
>>
>> func main() {
>> fmt.Println(())
>> }
>>
>> although the manual says:
>> type Int32
>> <https://cs.opensource.google/go/go/+/master:src/sync/atomic/type.go;l=59>
>> ¶ <https://pkg.go.dev/sync/atomic@master#Int32>
>>
>> type Int32 struct {
>>  // contains filtered or unexported fields
>> }
>>
>> An Int32 is an atomic int32. The zero value is zero.
>> func (*Int32) Load
>> <https://cs.opensource.google/go/go/+/master:src/sync/atomic/type.go;l=65>
>> ¶ <https://pkg.go.dev/sync/atomic@master#Int32.Load>
>>
>> func (x *Int32 <https://pkg.go.dev/sync/atomic@master#Int32>) Load() int32 
>> <https://pkg.go.dev/builtin#int32>
>>
>> Load atomically loads and returns the value stored in x.
>>
>> That's really essential.  For example, using your library, the following
>> code is most definitely *not* race-free:
>>
>> tmp := v.Load()
>> tmp = tmp + 1
>> v.Save(tmp)
>>
>> Of course not. Neither can misuse of atomic or sync.Map prevent race.
>> I have already shown how simple `=` and `==` caused race on reading
>> occasions in the last post.
>> Strict prevention should have been enforced at the syntax level just like
>> GIL in Python or ownership in Rust,
>> and adding a patch cannot fix loop-holes or side-attacks whatsoever.
>>
>> On Mon, May 23, 2022 at 2:56 PM Axel Wagner <
>> axel.wagner...@googlemail.com> wrote:
>>
>>> Just to be clear, are you aware of the sync/atomic package?
>>> https://pkg.go.dev/sync/atomic
>>> There are also some changes in there for Go 1.18, specifically the
>>> addition of some types, so that only atomic operations can be done:
>>> https://pkg.go.dev/sync/atomic@master
>>> I mention this because atomic.Value and atomic.Pointer[T] are
>>> essentially what you are suggesting here.
>>>
>>> On Mon, May 23, 2022 at 8:04 AM Zhaoxun Yan 
>>> wrote:
>>>
>>>> However, as I want to narrow the scope of this type down to generate
>>>> integer types (as in the commented code), it encountered two obstacles:
>>>>
>>>> 1) It is not legal to embed a generic inside a struct, nor can it make
>>>> generalized computation  =+1
>>>>
>>>> 2) Inheritance is not available in golang, so type "Counter" cannot
>>>> inherit type "Global" and get its methods automatically. I need to repeat
>>>> Save and Load methods to "Counter".
>>>>
>>>> Am I correct? Or can you improve it?
>>>>
>>>
>>> You are correct that there is no way in Go to write a type which gets
>>> all methods from an embedded field using generics. However, that is a good
>>> thing. For example, say you could write
>>>
>>> type Locked[T any] struct {
>>> sync.Mutex
>>> T
>>> }
>>>
>>> And this would get the methods of T and the methods of sync.Mutex. Then
>>> a user could do
>>>
>>> type Counter int64
>>> func (c *Counter) Increment() { *c += 1 }
>>>
>>> func main() {
>>> var c Locked[Counter]
>>> go c.Increment()
>>> go c.Increment()
>>> }
>>>
>>> And get a data race. That is, a method can modify a value in a way that
>>> is incompatible with what your wrapper type is trying to do.
>>>
>>> That's really the crux of both of the obstacles you mention. You can't
>>> run arbitrary computations and you can't promote methods, because *not all
>>> computation and not all methods can be made concurrency safe th

Re: [go-nuts] Improving safe global variables with generic or inheritance

[Zhaoxun Yan]

type Locked[T any] struct {
mu sync.Mutex
val T
}

func (l *Locked[T]) Do(f func(T) T) {
l.mu.Lock()
defer l.mu.Unlock()
l.val = f(l.val)
}

Could you elaborate how 'Save/Store' , 'Load' and 'Tick / Increase '  get
applied to the code above?

Thanks.

On Mon, May 23, 2022 at 3:24 PM 'Axel Wagner' via golang-nuts <
golang-nuts@googlegroups.com> wrote:

> FWIW a compromise is to have
>
> type Locked[T any] struct {
> mu sync.Mutex
> val T
> }
>
> func (l *Locked[T]) Do(f func(T) T) {
> l.mu.Lock()
> defer l.mu.Unlock()
> l.val = f(l.val)
> }
>
> This forces modifications to be under the protection of a mutex while also
> allowing those modifications to do arbitrary things.
>
> The downside is that this might do more locking than is strictly needed.
> So, under contention, it might perform significantly worse than "manually"
> managing the critical section.
>
> On Mon, May 23, 2022 at 9:11 AM Brian Candler  wrote:
>
>> > It's best to be intentional about it and explicitly acquire and release
>> a mutex around the critical sections of your code - because ultimately,
>> only your code knows which sections are critical.
>>
>> That's really essential.  For example, using your library, the following
>> code is most definitely *not* race-free:
>>
>> tmp := v.Load()
>> tmp = tmp + 1
>> v.Save(tmp)
>>
>> The mutex has to protect the entire sequence, not the individual load and
>> save operations.
>>
>> On Monday, 23 May 2022 at 07:57:02 UTC+1 axel.wa...@googlemail.com wrote:
>>
>>> Just to be clear, are you aware of the sync/atomic package?
>>> https://pkg.go.dev/sync/atomic
>>> There are also some changes in there for Go 1.18, specifically the
>>> addition of some types, so that only atomic operations can be done:
>>> https://pkg.go.dev/sync/atomic@master
>>> I mention this because atomic.Value and atomic.Pointer[T] are
>>> essentially what you are suggesting here.
>>>
>>> On Mon, May 23, 2022 at 8:04 AM Zhaoxun Yan  wrote:
>>>
>>>> However, as I want to narrow the scope of this type down to generate
>>>> integer types (as in the commented code), it encountered two obstacles:
>>>>
>>>> 1) It is not legal to embed a generic inside a struct, nor can it make
>>>> generalized computation  =+1
>>>>
>>>> 2) Inheritance is not available in golang, so type "Counter" cannot
>>>> inherit type "Global" and get its methods automatically. I need to repeat
>>>> Save and Load methods to "Counter".
>>>>
>>>> Am I correct? Or can you improve it?
>>>>
>>>
>>> You are correct that there is no way in Go to write a type which gets
>>> all methods from an embedded field using generics. However, that is a good
>>> thing. For example, say you could write
>>>
>>> type Locked[T any] struct {
>>> sync.Mutex
>>> T
>>> }
>>>
>>> And this would get the methods of T and the methods of sync.Mutex. Then
>>> a user could do
>>>
>>> type Counter int64
>>> func (c *Counter) Increment() { *c += 1 }
>>>
>>> func main() {
>>> var c Locked[Counter]
>>> go c.Increment()
>>> go c.Increment()
>>> }
>>>
>>> And get a data race. That is, a method can modify a value in a way that
>>> is incompatible with what your wrapper type is trying to do.
>>>
>>> That's really the crux of both of the obstacles you mention. You can't
>>> run arbitrary computations and you can't promote methods, because *not all
>>> computation and not all methods can be made concurrency safe this way*.
>>>
>>> It's best to be intentional about it and explicitly acquire and release
>>> a mutex around the critical sections of your code - because ultimately,
>>> only your code knows which sections are critical.
>>>
>>>
>>>>
>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "golang-nuts" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to golang-nuts...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/golang-nuts/baa74bff-6688-4d39-843b-c99a4fea2d1an%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/golang-nuts/baa74bff-6688-4d3

Re: [go-nuts] Improving safe global variables with generic or inheritance

[Zhaoxun Yan]

 There are also some changes in there for Go 1.18, specifically the
addition of some types, so that only atomic operations can be done:
https://pkg.go.dev/sync/atomic@master

Yes but the syntax is confusing, e.g. the code below fails:

package main

import (
"fmt"
"sync/atomic"
)

var a atomic.Int32

func main() {
fmt.Println(())
}

although the manual says:
type Int32
<https://cs.opensource.google/go/go/+/master:src/sync/atomic/type.go;l=59> ¶
<https://pkg.go.dev/sync/atomic@master#Int32>

type Int32 struct {
// contains filtered or unexported fields
}

An Int32 is an atomic int32. The zero value is zero.
func (*Int32) Load
<https://cs.opensource.google/go/go/+/master:src/sync/atomic/type.go;l=65> ¶
<https://pkg.go.dev/sync/atomic@master#Int32.Load>

func (x *Int32 <https://pkg.go.dev/sync/atomic@master#Int32>) Load()
int32 <https://pkg.go.dev/builtin#int32>

Load atomically loads and returns the value stored in x.

That's really essential.  For example, using your library, the following
code is most definitely *not* race-free:

tmp := v.Load()
tmp = tmp + 1
v.Save(tmp)

Of course not. Neither can misuse of atomic or sync.Map prevent race.
I have already shown how simple `=` and `==` caused race on reading
occasions in the last post.
Strict prevention should have been enforced at the syntax level just like
GIL in Python or ownership in Rust,
and adding a patch cannot fix loop-holes or side-attacks whatsoever.

On Mon, May 23, 2022 at 2:56 PM Axel Wagner 
wrote:

> Just to be clear, are you aware of the sync/atomic package?
> https://pkg.go.dev/sync/atomic
> There are also some changes in there for Go 1.18, specifically the
> addition of some types, so that only atomic operations can be done:
> https://pkg.go.dev/sync/atomic@master
> I mention this because atomic.Value and atomic.Pointer[T] are essentially
> what you are suggesting here.
>
> On Mon, May 23, 2022 at 8:04 AM Zhaoxun Yan  wrote:
>
>> However, as I want to narrow the scope of this type down to generate
>> integer types (as in the commented code), it encountered two obstacles:
>>
>> 1) It is not legal to embed a generic inside a struct, nor can it make
>> generalized computation  =+1
>>
>> 2) Inheritance is not available in golang, so type "Counter" cannot
>> inherit type "Global" and get its methods automatically. I need to repeat
>> Save and Load methods to "Counter".
>>
>> Am I correct? Or can you improve it?
>>
>
> You are correct that there is no way in Go to write a type which gets all
> methods from an embedded field using generics. However, that is a good
> thing. For example, say you could write
>
> type Locked[T any] struct {
> sync.Mutex
> T
> }
>
> And this would get the methods of T and the methods of sync.Mutex. Then a
> user could do
>
> type Counter int64
> func (c *Counter) Increment() { *c += 1 }
>
> func main() {
> var c Locked[Counter]
> go c.Increment()
> go c.Increment()
> }
>
> And get a data race. That is, a method can modify a value in a way that is
> incompatible with what your wrapper type is trying to do.
>
> That's really the crux of both of the obstacles you mention. You can't run
> arbitrary computations and you can't promote methods, because *not all
> computation and not all methods can be made concurrency safe this way*.
>
> It's best to be intentional about it and explicitly acquire and release a
> mutex around the critical sections of your code - because ultimately, only
> your code knows which sections are critical.
>
>
>>
> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/baa74bff-6688-4d39-843b-c99a4fea2d1an%40googlegroups.com
>> <https://groups.google.com/d/msgid/golang-nuts/baa74bff-6688-4d39-843b-c99a4fea2d1an%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CADEX6_Undz4th%2BTRCUZUyPZ47tf%2BQzdzF_TzM84EXfp7FPZ3Vw%40mail.gmail.com.

[go-nuts] Improving safe global variables with generic or inheritance

[Zhaoxun Yan]

Hi gophers! In the last post I proposed a way to avoid I/O race on any 
global variable:
 https://groups.google.com/g/golang-nuts/c/4AW1Ss9Tjp0

However, the code does not seem tidy, since each type needs to repeat 
defining the two methods Save and Load, though using this methods is easy.

So I did a little play around, but got stuck if I want to add in new 
methods to specific global types, such as tick to all integer types, or 
push to all slice types:
-
package main

import (
"fmt"
"sync"
)

type Global struct {
mutex sync.Mutex
value any
}

func (g *Global) Save(v any) {
g.mutex.Lock()
g.value = v
g.mutex.Unlock()
}

func (g *Global) Load() any {
g.mutex.Lock()
defer g.mutex.Unlock()
return g.value
}

var S = {}

func main() {

S.Save(5)
fmt.Printf("S=%v, S.mutex=%v, S.value=%d \n", S, S.mutex, S.value)
fmt.Printf("S=%v, S type as %[1]T", S.Load())
}

//New global types that has methods beyond Save and Load:
/*
type countable interface {
~int | ~int8 | ~int16 | ~int32 | ~int64
}

type Counter struct {
mutex sync.Mutex
value countable
}

func (c *Counter) Tick() {
c.mutex.Lock()
c.value += 1
c.mutex.Unlock()
}
*/

=

It worked well with the "Global" struct type.
However, as I want to narrow the scope of this type down to generate 
integer types (as in the commented code), it encountered two obstacles:

1) It is not legal to embed a generic inside a struct, nor can it make 
generalized computation  =+1

2) Inheritance is not available in golang, so type "Counter" cannot inherit 
type "Global" and get its methods automatically. I need to repeat Save and 
Load methods to "Counter".

Am I correct? Or can you improve it?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/baa74bff-6688-4d39-843b-c99a4fea2d1an%40googlegroups.com.

Re: [go-nuts] Re: DeepCloning a datastructure

[Zhaoxun Yan]

Yes there are two open source packages regarding "DeepCopy"

https://github.com/mohae/deepcopy/
https://github.com/barkimedes/go-deepcopy/

Has anyone tried them yet?

在2020年4月19日星期日 UTC+8 01:26:11 写道：

> How about doing it manually? You can define a DeepCopy method on the type 
> and manually allocate a new struct and initialize the maps and slices by 
> looping over them. That way you get to have more control over all the 
> fields and problem of private fields is solved as well. I would guess this 
> would be faster as well because no reflection is required.
>
> As far as I know Kubernetes uses this approach and they have a 
> code-generator which can conveniently generate the code for the structs.
>
> https://github.com/kubernetes/code-generator/tree/master/cmd/deepcopy-gen
>
>
>
> On Fri, Apr 17, 2020 at 2:07 PM Carsten Orthbandt  
> wrote:
>
>> The easiest way is to marshal everything into some sort of byte slice, 
>> then unmarshal back. JSON doesn't work for this if there are private 
>> fields. But gotiny (https://github.com/niubaoshu/gotiny) does. If you 
>> use interfaces in your data, you have to register their types, everthing 
>> else is automatic.
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/17810cd4-dab5-4583-8944-aed38b773232%40googlegroups.com
>>  
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/573fca81-61b0-4e32-b051-4a7bcdb2f6f1n%40googlegroups.com.

Re: [go-nuts] Re: A patch to avoid point-time race coincidence like Rust

[Zhaoxun Yan]

Thanks Kurtis and Brian for your discussion.
I have already decided to rewrite globals in my sync.Mutex pattern.
Unprotected global variables are threats, although at a very low
possibility.
Here is the way I protect them, beyond sync.Map


package main

import (
"fmt"
"sync"
)

type Int struct{
mutex   sync.Mutex
value   int
}

func (I *Int) Save(n int){
I.mutex.Lock()
I.value = n
I.mutex.Unlock()
}

func (I *Int) Load() int{
I.mutex.Lock()
defer I.mutex.Unlock()
return I.value
}

/*
type String struct{
...
*/

var S = {}

func main() {

   S.Save(5)
   fmt.Printf("S=%v, S.mutex=%v, S.value=%d \n", S, S.mutex, S.value)
   fmt.Println("S:", S.Load())
}

S=&{{0 0} 5}, S.mutex={0 0}, S.value=5
S: 5

Good night!
  Zhaoxun

On Fri, May 6, 2022 at 12:14 PM Kurtis Rader  wrote:

> On Thu, May 5, 2022 at 8:50 PM Zhaoxun Yan  wrote:
>
>> Hi Kurtis!
>> Thanks for your reply.
>>
>> I am just stating the fact that atomic module did not solve the reading
>> writing conflict as it may sound so, or Brian perceived so.
>>
>
> Brian Candler wrote "If the value you are trying to read and update is
> int32 or int64, you can use the sync.Atomic
> <https://pkg.go.dev/sync/atomic> package". You misunderstood what Brian
> wrote. They were not suggesting that using atomic updates were appropriate
> for modifying a map. They were pointing out that if you need atomic updates
> of ints and pointers then the `sync/atomic` package is the usual solution.
> In hindsight I bet Brian regrets suggesting the use of `sync/atomic`.
>
> I definitely follow the syntax of atomic while writing inside the
>> atomic.StoreInt64 function.
>> But as for reading I used a simple `=`, and it obviously failed.
>>
>
> Yes, that will fail because you cannot do a simple read of a value when it
> might be concurrently modified; even when modified by the `sync/atomic`
> package. You have to also use the `sync/atomic` package to atomically read
> the value.
>
>
>> It seems that both reading and writing need special functions?
>> In that way I do not believe the atomic module makes things simpler than
>> my proposal.
>>
>> I did use sync.Map, and it did not raise such warnings. Maybe it is
>> because I followed the strict reading and writing function syntax.
>>
>
> Okay, if using `sync.Map` solved your problem what else do you need
> help with?
>
>
>> The log4go module raised lots of such race warnings, while in practice it
>> endured high frequently logging and did not crash once.
>>
>
> A lot of code written in Go will emit race warnings when run with the race
> detector enabled. If that code explicitly captures panics then it won't
> crash. That is not considered a best practice. It is better to not write
> code that causes panics than rely on `recover()`.
>
> --
> Kurtis Rader
> Caretaker of the exceptional canines Junior and Hank
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CADEX6_UdU_XoDOyiNfciqntRpZES1fm_P75P8WOALVpC6kASFA%40mail.gmail.com.

Re: [go-nuts] Re: A patch to avoid point-time race coincidence like Rust

[Zhaoxun Yan]

Hi Kurtis!
Thanks for your reply.

I am just stating the fact that atomic module did not solve the reading
writing conflict as it may sound so, or Brian perceived so.
Here is the log analysis out of print out by a race build:
===
WARNING: DATA RACE
Write at 0x016a4808 by goroutine 91:
  sync/atomic.StoreInt64()
  /usr/go15/src/runtime/race_amd64.s:234 +0xb
  quotelive/util.Zero()
  /home/zxun/src/quotelive/util/util.go:89 +0xbe
【atomic.StoreInt64(a, 0)】
  main.startCTP()
  /home/zxun/src/quotelive/ctpschedule.go:279 +0xa5
【util.Zero()】

  github.com/robfig/cron.FuncJob.Run()
  /home/zxun/src/github.com/robfig/cron/cron.go:92 +0x34
  github.com/robfig/cron.(*Cron).runWithRecovery()
  /home/zxun/src/github.com/robfig/cron/cron.go:165 +0x72

Previous read at 0x016a4808 by goroutine 31:
  main.ctpDaemon()
  /home/zxun/src/quotelive/ctp.go:455 +0x257
【oldcount = util.CountQ】

Goroutine 91 (running) created at:
  github.com/robfig/cron.(*Cron).run()
  /home/zxun/src/github.com/robfig/cron/cron.go:199 +0xa90

Goroutine 31 (running) created at:
  main.initQuote()
  /home/zxun/src/quotelive/ctpschedule.go:428 +0xe4
  main.main()
  /home/zxun/src/quotelive/main.go:56 +0x251
===

I definitely follow the syntax of atomic while writing inside the
atomic.StoreInt64 function.
But as for reading I used a simple `=`, and it obviously failed.
It seems that both reading and writing need special functions?
In that way I do not believe the atomic module makes things simpler than my
proposal.

I did use sync.Map, and it did not raise such warnings. Maybe it is because
I followed the strict reading and writing function syntax.

The log4go module raised lots of such race warnings, while in practice it
endured high frequently logging and did not crash once.

==
WARNING: DATA RACE
Write at 0x00cdf7b8 by goroutine 27:
  github.com/jeanphorn/log4go.LOGGER()
  /home/zxun/src/github.com/jeanphorn/log4go/category.go:16 +0xb4
  main.pingSign()
  /home/zxun/src/quotelive/signclient.go:102 +0xc8
【log4go.LOGGER("syslog").Error("Started heartbeat to sign server")】

Previous read at 0x00cdf7b8 by goroutine 19:
  github.com/jeanphorn/log4go.(*Filter).intLogf()
  /home/zxun/src/github.com/jeanphorn/log4go/category.go:51 +0x20f
  github.com/jeanphorn/log4go.(*Filter).Info()
  /home/zxun/src/github.com/jeanphorn/log4go/category.go:252 +0x218
  main.loginSign()
  /home/zxun/src/quotelive/signclient.go:87 +0x45e
【log4go.LOGGER("syslog").Info("Sent login message to sign server!")】
  main.connectSign()
  /home/zxun/src/quotelive/signclient.go:62 +0x544

Goroutine 27 (running) created at:
  main.loginSign()
  /home/zxun/src/quotelive/signclient.go:75 +0x78
  main.connectSign()
  /home/zxun/src/quotelive/signclient.go:62 +0x544

Goroutine 19 (finished) created at:
  main.signStart()
  /home/zxun/src/quotelive/signclient.go:30 +0x116
  main.main()
  /home/zxun/src/quotelive/main.go:53 +0x24c
==

And here is writing vs writing race:
==
WARNING: DATA RACE
Write at 0x00cdf7e8 by goroutine 2:
  github.com/jeanphorn/log4go.LOGGER()
  /home/zxun/src/github.com/jeanphorn/log4go/category.go:16 +0xb4
  quotelive/myredis.Upfutureprice()
  /home/zxun/src/quotelive/myredis/connecttoredis.go:227 +0x48f
【log4go.LOGGER("marketlog").Debug("\n  " + future_id + " : " + data)】

  quotelive/ctp_market.OutputNormal.func1()
  /home/zxun/src/quotelive/ctp_market/outputQuote.go:38 +0xc4
  quotelive/ctp_market.OnQuoteGen.func1()
  /home/zxun/src/quotelive/ctp_market/main_ctp.go:213 +0x70d
  quotelive/ctp_market.OnQuote_2b()
  /home/zxun/src/quotelive/ctp_market/main_ctp.go:233 +0x50
  quotelive/ctp_market._cgoexpwrap_7d5ac618abbe_OnQuote_2b()
  _cgo_gotypes.go:321 +0x2b

Previous write at 0x00cdf7e8 by goroutine 24:
  github.com/jeanphorn/log4go.LOGGER()
  /home/zxun/src/github.com/jeanphorn/log4go/category.go:16 +0xb4
  main.endDaemon()
  /home/zxun/src/quotelive/ctpschedule.go:314 +0xce
【log4go.LOGGER("marketlog").Debug("Snapshot time from quotation " +
timestr)】

Goroutine 2 (running) created at:
  runtime.newextram()
  /usr/go15/src/runtime/proc.go:1596 +0x0

Goroutine 24 (running) created at:
  main.initQuote()
  /home/zxun/src/quotelive/ctpschedule.go:424 +0xbc
  main.main()
  /home/zxun/src/quotelive/main.go:56 +0x251
==

On Fri, May 6, 2022 at 11:25 AM Kurtis Rader  wrote:

> On Thu, May 5, 2022 at 7:50 PM Zhaoxun Yan  wrote:
>
>> I already know that a copy of map does not make a difference. That is why
>> I put it inside a struct, but magic did not happen.
>>
>
> What "magic" did you expect from simply embedding a reference to a map
> inside a struct? I don't ask this rhetorically sinc

Re: [go-nuts] Re: A patch to avoid point-time race coincidence like Rust

[Zhaoxun Yan]

I already know that a copy of map does not make a difference. That is why I
put it inside a struct, but magic did not happen.

Does the sync.Map solves reading and writing conflict? For I checked the
print out log and discovered that even `atomic` module did not prevent such
race warnings.

There are also huge amount of warnings related to log4go module. But the
author seemed to put a recovery patch to it,
is it another way to solve it?
https://github.com/jeanphorn/log4go/commit/7dbb8deb94686ee4c744290168390869ac014312

Although every logging of `log4go` might trigger race panic, it survived a
pressure test as each incoming quotation got logged and got logged again
after reading it back from redis for 8 hours, which resulted a quotation
log as large as tens of MB.

On Fri, May 6, 2022 at 12:55 AM Brian Candler  wrote:

> Correct.
>
> a := make(map[int]string)
> b := a
>
> means that b and a point to the same map.
>
> > Do you know any way in go to copy a map? Or the only way to do it is to
> use json Marshal and Unmarshal?
>
> That is called a "deep copy" and there are third party libraries for
> this.  But if you think you need it - just to access a map safely - then
> almost certainly you are Doing It Wrong™.
>
> One option is to use sync.Map, which is safe for concurrent access.
>
> Better: protect your map properly against concurrent access. See
> "Rethinking Classical Concurrency Patterns" by Bryan C. Mills - although if
> you're not familiar with traditional approaches to concurrency, you may
> find this hard to understand (it's worth the effort though)
> https://www.youtube.com/watch?v=5zXAHh5tJqQ
>
> One of the patterns which comes out of this video is to use a 1-element
> channel as a way to hold a protected value.  Every time you want to use
> this value, you pop it out of the channel, use it, and then push it back in
> (without keeping a copy).  This gives you the same sort of protection as a
> mutex, but cleaner and simpler.
>
> Here is an example: two goroutines both concurrently updating the same map.
> https://go.dev/play/p/nY3ujWh3Mfj
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/4AW1Ss9Tjp0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/38239c7c-2c62-45ef-85ad-3edeb26bda8bn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CADEX6_V-p3Z_nJUzeNLXJbE98Wo6y1NEOVe8D9C5MKgntesrhA%40mail.gmail.com.

Re: [go-nuts] Re: A patch to avoid point-time race coincidence like Rust

[Zhaoxun Yan]

Hi Brian!

I just checked my previous code for race errors. It turned out that a `map`
is very hard to avoid the race condition.

I have already put reading and writing into different functions with a
global lock.

The reading basically sends the struct wrapped map to a channel. Then there
is another goroutine processes the struct which contains the map.

The race condition did not happen between the writing function and reading
function, but instead the writing function and the processing goroutine.

So that indicates neither wrapping the map inside a struct nor passing it
through a channel did not make a copy of the map at all. It is the very
same map that go runtime manages. It is like only the memory address of the
map gets embedded in the struct and then passed over.

Do you know any way in go to copy a map? Or the only way to do it is to use
json Marshal and Unmarshal?

Thanks,
   Zhaoxun

On Sun, May 1, 2022 at 6:45 PM Brian Candler  wrote:

> On Sunday, 1 May 2022 at 07:04:34 UTC+1 yan.z...@gmail.com wrote:
>
>> Any advice?
>>
>
> If the value you are trying to read and update is int32 or int64, you can
> use the sync.Atomic  package.
>
> However, if you feel you need to do this all the time, I can't help but
> wonder that you're approaching Go from the wrong angle.  The solution is
> not to protect concurrent accesses to global variables; it is to get rid of
> the global variables.  Use channels instead.  Share memory by
> communicating; don't communicate by sharing memory.
>
> https://www.youtube.com/watch?v=5zXAHh5tJqQ
>
> > I am not sure if map type needs such special care since I heard map is
> okay as one goroutine reads it while another writes it at the same time.
>
> No, quite the opposite: multiple goroutines accessing the same map
> concurrently are very likely to cause a panic.  There is sync.Map
>  you can use safely from multiple goroutines.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/4AW1Ss9Tjp0/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/7b197eb6-775e-456b-abf9-30ab00becec9n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CADEX6_UWwdqtuQc9Tbo7BgtvCP_CNE-NjbrZ%3Drvp2WBY8%2B8bkg%40mail.gmail.com.

[go-nuts] A patch to avoid point-time race coincidence like Rust

[Zhaoxun Yan]

A global variable has a plausible chance to race as one goroutine reads it 
while another writes it, as shown in previous post: 
https://groups.google.com/g/golang-nuts/c/PHw_zU6Ayfo

So I am trying to enforce a lock on each global variable to avoid such 
accident just as Rust does on every variable.

Here is one implementation where 'S' is the global variable. You can create 
Int, String, Bool, ArrayInt, ArrayStr, etc in a special package, and then 
call its type and function/methods from anywhere. I am not sure if map type 
needs such special care since I heard map is okay as one goroutine reads it 
while another writes it at the same time.

package main

import (
"fmt"
"sync"
)

type Int struct{
mutex   sync.Mutex
value   int
}

func (I *Int) Save(n int){
I.mutex.Lock()
I.value = n
I.mutex.Unlock()
}

func (I *Int) Load() int{
I.mutex.Lock()
defer I.mutex.Unlock()
return I.value
}

var S = {}

func main() {   
   S.Save(5)
   fmt.Printf("S=%v, S.value=%d \n", S,  S.value)
   fmt.Println("S:", S.Load())
}

Any advice?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/967069a9-8ba8-4d20-b02b-0399e31f3fffn%40googlegroups.com.

Re: [go-nuts] go fails to detect a point-time race condition

[Zhaoxun Yan]

Hi Marvin!

  I am sure it did not detect race immediately at least in my project, 
which has similar global variable race conditions, but in a more subtle way 
.

  For example, the checking of one global variable is from an incoming 
message from a remote host, while the changing of the global variable is 
from a crontask. They have a possibility to collide, but my race build did 
not crash because of it yet.

Zhaoxun

在2022年5月1日星期日 UTC+8 00:18:11 写道：

> * Zhaoxun Yan  [220430 02:29]:
> > Hi Dan! 
> > 
> > I did as you told, but go build -race still not functions:
>
> No, Dan said you must build with -race and then execute the output from
> the build:
>
> $ go build race race2
> $ ./race2
>
> What Dan was saying is that «go build -race» does _not_ detect races
> during the build process, but produces an executable that contains the
> race detector built-in, so that when you invoke the executable, any
> races will be detected.
>
> > zxun@zxun-virtual:~/src/race2$ go build 
> > zxun@zxun-virtual:~/src/race2$ ls
> > race2 race.go
> > zxun@zxun-virtual:~/src/race2$ go build -race race2
> > zxun@zxun-virtual:~/src/race2$ go run -race race.go
> > ==
> > WARNING: DATA RACE
> [snip]
> > Found 1 data race(s)
> > exit status 66
> > 
> > 在2022年4月30日星期六 UTC+8 14:22:26 写道：
> > 
> > > On Fri, 2022-04-29 at 23:18 -0700, Zhaoxun Yan wrote:
> > > > And then in that folder you run:
> > > > # go build -race
> > > > Nothing happens, at least in my go1.15
> > >
> > > The race detector needs to run to detect data races; it's not a static
> > > analysis tool.
> > >
> > > So if you execute the binary that you built with `go build -race` you
> > > should see the race report.
> > >
> > > Dan
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/77150944-12e5-441d-8fe0-430f39a2b5cfn%40googlegroups.com.

Re: [go-nuts] go fails to detect a point-time race condition

[Zhaoxun Yan]

Hi Dan! 

I did as you told, but go build -race still not functions:
zxun@zxun-virtual:~/src/race2$ go build 
zxun@zxun-virtual:~/src/race2$ ls
race2  race.go
zxun@zxun-virtual:~/src/race2$ go build -race race2
zxun@zxun-virtual:~/src/race2$ go run -race race.go
==
WARNING: DATA RACE
Read at 0x006489c0 by main goroutine:
  main.read()
  /home/zxun/src/race2/race.go:13 +0x6d
  main.main()
  /home/zxun/src/race2/race.go:24 +0x5d

Previous write at 0x006489c0 by goroutine 7:
  main.increase()
  /home/zxun/src/race2/race.go:9 +0x64
  main.main.func1()
  /home/zxun/src/race2/race.go:19 +0x38

Goroutine 7 (running) created at:
  main.main()
  /home/zxun/src/race2/race.go:17 +0x4f
==
5
Found 1 data race(s)
exit status 66

在2022年4月30日星期六 UTC+8 14:22:26 写道：

> On Fri, 2022-04-29 at 23:18 -0700, Zhaoxun Yan wrote:
> > And then in that folder you run:
> > # go build -race
> > Nothing happens, at least in my go1.15
>
> The race detector needs to run to detect data races; it's not a static
> analysis tool.
>
> So if you execute the binary that you built with `go build -race` you
> should see the race report.
>
> Dan
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0236f4a2-80d7-4a6b-927b-0024029b5b74n%40googlegroups.com.

Re: [go-nuts] go fails to detect a point-time race condition

[Zhaoxun Yan]

go build -race
go build -race .

Either would not detect race condition for the whole project (folder).

在2022年4月30日星期六 UTC+8 14:18:55 写道：

> Hi Axel!
>
> My project is in a folder.
> So actually the best way to mimic the situation is to put this file in a 
> folder. 
> And then in that folder you run:
> # go build -race
> Nothing happens, at least in my go1.15
> It did mention the race condition with `go run -race [filename]`
> $ go run -race race.go
> ==
> WARNING: DATA RACE
> Read at 0x006489c0 by main goroutine:
>   main.read()
>   /home/zxun/src/race.go:13 +0x6d
>   main.main()
>   /home/zxun/src/race.go:24 +0x5d
>
> Previous write at 0x006489c0 by goroutine 7:
>   main.increase()
>   /home/zxun/src/race.go:9 +0x64
>   main.main.func1()
>   /home/zxun/src/race.go:19 +0x38
>
>
> Goroutine 7 (running) created at:
>   main.main()
>   /home/zxun/src/race.go:17 +0x4f
> ==
> 5
> Found 1 data race(s)
>
> Do you know the syntax to detect race condition for a whole folder of 
> codes, instead of one file?
> Thanks.
>   Zhaoxun
>
> 在2022年4月30日星期六 UTC+8 14:05:45 写道：
>
>> On Sat, Apr 30, 2022 at 7:49 AM Zhaoxun Yan  wrote:
>>
>>> Point-time race condition is undetectable by `go build race`.
>>>
>>
>> When I run your code using `go run -race`, it reports a data race:
>>
>>>
>>> ==
>>> WARNING: DATA RACE
>>> Read at 0x005cb580 by main goroutine:
>>>   main.read()
>>>   /home/mero/tmp/x/x.go:15 +0xab
>>>   main.main()
>>>   /home/mero/tmp/x/x.go:26 +0x99
>>
>>
>>> Previous write at 0x005cb580 by goroutine 7:
>>>   main.increase()
>>>   /home/mero/tmp/x/x.go:11 +0x54
>>>   main.main.func1()
>>>   /home/mero/tmp/x/x.go:21 +0x30
>>>   main.main.func2()
>>>   /home/mero/tmp/x/x.go:24 +0x3e
>>
>>
>>> Goroutine 7 (running) created at:
>>>   main.main()
>>>   /home/mero/tmp/x/x.go:19 +0x8e
>>> ==
>>> 4
>>> Found 1 data race(s)
>>> exit status 66
>>
>>  
>> Go version is
>> go version go1.18 linux/amd64
>>
>> So, I don't really understand why you think the race detector can't 
>> detect that race.
>>
>>
>> Thanks to Rust on emphasizing the single entrance to any variable,  I 
>>> realized this possibility and have to add sync.Mutex to both functions. 
>>>
>>> What is your opinion?
>>>  
>>> Best Regards,
>>>   Zhaoxun
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to golang-nuts...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/5c6ae730-36a7-46ce-abb9-ea8d24da601bn%40googlegroups.com
>>>  
>>> <https://groups.google.com/d/msgid/golang-nuts/5c6ae730-36a7-46ce-abb9-ea8d24da601bn%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/97623349-0132-4fa3-b608-ff1342b02372n%40googlegroups.com.

Re: [go-nuts] go fails to detect a point-time race condition

[Zhaoxun Yan]

Hi Axel!

My project is in a folder.
So actually the best way to mimic the situation is to put this file in a 
folder. 
And then in that folder you run:
# go build -race
Nothing happens, at least in my go1.15
It did mention the race condition with `go run -race [filename]`
$ go run -race race.go
==
WARNING: DATA RACE
Read at 0x006489c0 by main goroutine:
  main.read()
  /home/zxun/src/race.go:13 +0x6d
  main.main()
  /home/zxun/src/race.go:24 +0x5d

Previous write at 0x006489c0 by goroutine 7:
  main.increase()
  /home/zxun/src/race.go:9 +0x64
  main.main.func1()
  /home/zxun/src/race.go:19 +0x38

Goroutine 7 (running) created at:
  main.main()
  /home/zxun/src/race.go:17 +0x4f
==
5
Found 1 data race(s)

Do you know the syntax to detect race condition for a whole folder of 
codes, instead of one file?
Thanks.
  Zhaoxun

在2022年4月30日星期六 UTC+8 14:05:45 写道：

> On Sat, Apr 30, 2022 at 7:49 AM Zhaoxun Yan  wrote:
>
>> Point-time race condition is undetectable by `go build race`.
>>
>
> When I run your code using `go run -race`, it reports a data race:
>
>>
>> ==
>> WARNING: DATA RACE
>> Read at 0x005cb580 by main goroutine:
>>   main.read()
>>   /home/mero/tmp/x/x.go:15 +0xab
>>   main.main()
>>   /home/mero/tmp/x/x.go:26 +0x99
>
>
>> Previous write at 0x005cb580 by goroutine 7:
>>   main.increase()
>>   /home/mero/tmp/x/x.go:11 +0x54
>>   main.main.func1()
>>   /home/mero/tmp/x/x.go:21 +0x30
>>   main.main.func2()
>>   /home/mero/tmp/x/x.go:24 +0x3e
>
>
>> Goroutine 7 (running) created at:
>>   main.main()
>>   /home/mero/tmp/x/x.go:19 +0x8e
>> ==
>> 4
>> Found 1 data race(s)
>> exit status 66
>
>  
> Go version is
> go version go1.18 linux/amd64
>
> So, I don't really understand why you think the race detector can't detect 
> that race.
>
>
> Thanks to Rust on emphasizing the single entrance to any variable,  I 
>> realized this possibility and have to add sync.Mutex to both functions. 
>>
>> What is your opinion?
>>  
>> Best Regards,
>>   Zhaoxun
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/5c6ae730-36a7-46ce-abb9-ea8d24da601bn%40googlegroups.com
>>  
>> <https://groups.google.com/d/msgid/golang-nuts/5c6ae730-36a7-46ce-abb9-ea8d24da601bn%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/98c29132-d7d2-42f1-a219-e5bb558b67ban%40googlegroups.com.

[go-nuts] Re: go fails to detect a point-time race condition

[Zhaoxun Yan]

 package main

import "time"
import "fmt"

var index int64

func increase(){
  index++
}

func read(){
  fmt.Println(index)
}

func main(){
   go func(N int){
 for i:=0; i < N; i++{
increase()
time.Sleep(500 * time.Millisecond)
 }
 }(5)
   time.Sleep(2 * time.Second)
   read()
}


在2022年4月30日星期六 UTC+8 13:49:11 写道：

> point race means I/O on a global might clash at certain point of time.
>
> consider this example:
>
> package main
>
> import "time"
> import "fmt"
>
> var index int64
>
> func increase(){
>   index++
> }
>
> func read(){
>   fmt.Println(index)
> }
>
> func main(){
>go func(N){
>  for i:=0; i < N; i++{
> increase()
> time.Sleep(500 * time.Millisecond)
>  }
>  }(5)
>time.Sleep(2 * time.Second)
>read()
> }
>
> Obviously, the increase function is writing the global variable 'index' 
> while the read is reading it. It is highly unlikely that they could 
> function at the same time, but not impossible.  Further, if the increase 
> function takes longer time to manipulate the global, such as an array which 
> might enlarge itself, and reading during the changing of the global is 
> dangerous. 
>
> The code in my project has a little variation -  the writing of the 
> variable is controlled by crontask of "github.com/robfig/cron". However, 
> the essence is the same. Point-time race condition is undetectable by `go 
> build race`. Thanks to Rust on emphasizing the single entrance to any 
> variable,  I realized this possibility and have to add sync.Mutex to both 
> functions. 
>
> What is your opinion?
>  
> Best Regards,
>   Zhaoxun
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d6667b87-3ef9-4cff-b63d-aaef8b035ea3n%40googlegroups.com.

[go-nuts] go fails to detect a point-time race condition

[Zhaoxun Yan]

point race means I/O on a global might clash at certain point of time.

consider this example:

package main

import "time"
import "fmt"

var index int64

func increase(){
  index++
}

func read(){
  fmt.Println(index)
}

func main(){
   go func(N){
 for i:=0; i < N; i++{
increase()
time.Sleep(500 * time.Millisecond)
 }
 }(5)
   time.Sleep(2 * time.Second)
   read()
}

Obviously, the increase function is writing the global variable 'index' 
while the read is reading it. It is highly unlikely that they could 
function at the same time, but not impossible.  Further, if the increase 
function takes longer time to manipulate the global, such as an array which 
might enlarge itself, and reading during the changing of the global is 
dangerous. 

The code in my project has a little variation -  the writing of the 
variable is controlled by crontask of "github.com/robfig/cron". However, 
the essence is the same. Point-time race condition is undetectable by `go 
build race`. Thanks to Rust on emphasizing the single entrance to any 
variable,  I realized this possibility and have to add sync.Mutex to both 
functions. 

What is your opinion?
 
Best Regards,
  Zhaoxun

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5c6ae730-36a7-46ce-abb9-ea8d24da601bn%40googlegroups.com.

Re: [go-nuts] Re: Protective buffered channel that never triggers deadlock

[Zhaoxun Yan]

" Strangely I found the start testing and disconnect log clustered and the
disconnect  did actually happen. Then I switch back to the sequential case
that the receiving channel gots filled without receiving until
disconnection. It works now."

-- I found the error occurred again. It turned out that it referenced a
global integer variable which might be 0 yet so it called the disconnection
function even before it got connected.  It has nothing to do with first to
receive or to send.

On Fri, Apr 15, 2022 at 3:44 PM Brian Candler  wrote:

> On Friday, 15 April 2022 at 08:31:17 UTC+1 yan.z...@gmail.com wrote:
>
>> Thanks for your demonstration, Brian.
>> Actually my code involving cgo is quite like your first case, like this
>> code:
>>
>> *log("start testing")*
>> *go func{*
>> *  for*
>> *  select*
>> *  case a: <=receiving*
>> *  case <- killsig*
>> *...*
>> *}()*
>>
>> *subscribequotations*
>> *( meanwhile the cgo of the dll will call back go functions to fill
>> receiving channel)*
>> *time.Sleep(3 * time.Seconds)*
>>
>> *disconnect and log*
>> *killsig <- true*
>>
>
> That code is obviously incomplete, but I'm guessing when you receive a
> value on killsig you return from the function.
>
> A better Go idiom for that is to close a channel.  All the receivers on
> that channel will see that the channel is closed:
>
> case msg, ok <- killsig:
> // msg is the zero value and ok is false if the channel is closed
>
> The same channel can be shared between many receivers, so it becomes a way
> of broadcasting a kill message.  See https://go.dev/tour/concurrency/4
>
> You should never *send* on a closed channel though - that will cause a
> panic.
>
>
>
>>
>> I am really curious about your example on select -  is it a try... except
>> alternative?
>>
>
> It's not really anything like "try... except".
>
> The "select" statement is specifically for channel communication.  See the
> description in the Go Tour starting here:
> https://go.dev/tour/concurrency/5  # pages 5 and 6
> It picks one branch where a channel is ready to send or receive, or the
> "default" branch if none are.
>
> If you have a series of general conditions that you want to try one by
> one, then there is the "switch" statement:
> https://go.dev/tour/flowcontrol/9   # pages 9 to 11
>
> In most languages "try... except" is an error recovery mechanism which can
> handle unwinding the stack (returning from multiple levels of nested
> function calls).  The nearest equivalent to that in go is panic/recover,
> but that should only be used in very special circumstances.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/6ExktXrF5Xc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/452795b8-4817-4161-84e7-02665e24ae06n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CADEX6_UE4VkNc3gEPRfB046vWYU_%2BcYwUHfSPOCYfrJWJCdypA%40mail.gmail.com.

Re: [go-nuts] Any recipe to stop a goroutine of a function other than of a channel?

[Zhaoxun Yan]

Thanks rog!

  I already dealt with it in my project since the Listen of gorilla
websocket did as you just mentioned - close and even throw out an error.
But I am surprised at why golang still has not provided a general feature
on that.

Zhaoxun

On Sat, Apr 16, 2022 at 10:14 PM roger peppe  wrote:

> Most network functions provide a way to provide a timeout or cancellation
> explicitly. Listen is one such: see this method - if the context that it's
> passed is cancelled, the Listen call will return.
> https://pkg.go.dev/net#ListenConfig.Listen
>
> The most general way to stop waiting on timeout or cancellation in the
> absence of explicitly provided functionality is to start the function in a
> separate goroutine and send notification when the function completes. This
> is non-ideal because the goroutine will remain around even when the waiter
> has given up, but can still be a useful technique in some circumstances.
>
> Hope this helps,
>   rog.
>
> On Sat, 16 Apr 2022, 14:30 Zhaoxun Yan,  wrote:
>
>> Timeout  is quite common practice in programming. For example, Listen
>> function in internet connection with a timeout may just close the
>> connection and Listen thread as silence persists for some period, like 60
>> seconds.
>>
>> I found it very hard to implement such a general shutdown feature of a
>> thread/goroutine on such I/O functions, here is one unsuccessful try, which
>> I want to stop the whole in 9 seconds but it does not function as I wanted:
>>
>> package main
>>
>> import (
>> "fmt"
>> "time"
>> )
>>
>> func wait(){
>> time.Sleep(5 * time.Second)
>> fmt.Println("wait 1st signal")
>> time.Sleep(5 * time.Second)
>> fmt.Println("wait 2nd signal")
>> }
>>
>>
>> func main() {
>>ticker := time.NewTicker(3 * time.Second)
>>i := 0
>>for{
>>select{
>> case <- ticker.C:
>> i++
>> fmt.Printf("ticker rings %d\n", i)
>> if i==3{
>>return
>> }
>>   default:
>>   wait()
>>   }
>>}
>>
>> }
>>
>> The result is to wait whole 30 seconds:
>> wait 1st signal
>> wait 2nd signal
>> ticker rings 1
>> wait 1st signal
>> wait 2nd signal
>> ticker rings 2
>> wait 1st signal
>> wait 2nd signal
>> ticker rings 3
>>
>> An online course suggests to wrap up the wait/Listen function with a
>> channel (which would return something instead of nothing above)
>>
>> go func(){
>>   resultChan <- Listen()
>> }()
>>
>> select{
>> case a := <- resultChan:
>>//analyze a
>> case <-ticker.C:
>>//might break or return
>> }
>>
>> But this recipe obviously only kill the select goroutine rather than the
>> anonymous goroutine that actually runs Listen function. My question is -
>> how to kill the waiting or listening goroutine from outside?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/e774c5a4-ed53-43da-a1fe-0d617603e223n%40googlegroups.com
>> <https://groups.google.com/d/msgid/golang-nuts/e774c5a4-ed53-43da-a1fe-0d617603e223n%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CADEX6_WAwhVnFmEavAThJ5uQN%3DjVK0uuh2pFSxt%3DczjgLFbNPA%40mail.gmail.com.

[go-nuts] Any recipe to stop a goroutine of a function other than of a channel?

[Zhaoxun Yan]

 Timeout  is quite common practice in programming. For example, Listen 
function in internet connection with a timeout may just close the 
connection and Listen thread as silence persists for some period, like 60 
seconds.

I found it very hard to implement such a general shutdown feature of a 
thread/goroutine on such I/O functions, here is one unsuccessful try, which 
I want to stop the whole in 9 seconds but it does not function as I wanted:

package main

import (
"fmt"
"time"
)

func wait(){
time.Sleep(5 * time.Second)
fmt.Println("wait 1st signal")
time.Sleep(5 * time.Second)
fmt.Println("wait 2nd signal")
}


func main() {
   ticker := time.NewTicker(3 * time.Second)
   i := 0
   for{
   select{
case <- ticker.C:
i++
fmt.Printf("ticker rings %d\n", i)
if i==3{
   return 
}
  default:
  wait()
  }   
   }

}

The result is to wait whole 30 seconds:
wait 1st signal
wait 2nd signal
ticker rings 1
wait 1st signal
wait 2nd signal
ticker rings 2
wait 1st signal
wait 2nd signal
ticker rings 3

An online course suggests to wrap up the wait/Listen function with a 
channel (which would return something instead of nothing above)

go func(){
  resultChan <- Listen()
}()

select{
case a := <- resultChan:
   //analyze a
case <-ticker.C:
   //might break or return
}

But this recipe obviously only kill the select goroutine rather than the 
anonymous goroutine that actually runs Listen function. My question is - 
how to kill the waiting or listening goroutine from outside?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e774c5a4-ed53-43da-a1fe-0d617603e223n%40googlegroups.com.

[go-nuts] Re: MatrixOne: a hyperconverged and one-size-fits-most database in Go

[Zhaoxun Yan]

Does it support sequence and upsert in PostgreSQL?

在2022年4月16日星期六 UTC+8 00:47:48 写道：

> Hi,
>
> I’d like to introduce a hyperconverged and one-size-fits-most database 
> named MatrixOne, written in Go. 
>
> Github: https://github.com/matrixorigin/matrixone
>
> It’s a redesigned database with a combined architecture of CockroachDB, 
> Clickhouse and Apache Flink. Our motivation is to bring a simple and 
> one-size-fits-most data processing framework and significantly reduce the 
> complexity of building modern data platform.
>
>
> Key Objectives:
>
>- Simple usage by standardized SQL with support of multiple dialects. 
>- Supports transactional, analytical, streaming workloads in one 
>database.  
>- Blazing fast query performance with strong consistency and ACID 
>transactional property. 
>- Large scalability across different infrastructures, including 
>public/private cloud, on-premise and edge devices.  
>
>
> It’s still a very early stage project, the latest 0.3.0 release version is 
> mainly a MPP OLAP engine, with distributed strong consistency based on Raft.
>
> A SSB benchmark test is run over MatrixOne and it outperforms Clickhouse 
> at 50%, with some constraints on. 
>
>
> Feel free to test MatrixOne and any feedback is welcome. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/fd2cbd01-26b6-4983-bc96-4986d0bf6b31n%40googlegroups.com.

Re: [go-nuts] Re: Protective buffered channel that never triggers deadlock

[Zhaoxun Yan]

Thanks for your demonstration, Brian.
Actually my code involving cgo is quite like your first case, like this
code:

*log("start testing")*
*go func{*
*  for*
*  select*
*  case a: <=receiving*
*  case <- killsig*
*...*
*}()*

*subscribequotations*
*( meanwhile the cgo of the dll will call back go functions to fill
receiving channel)*
*time.Sleep(3 * time.Seconds)*

*disconnect and log*
*killsig <- true*

Strangely I found the start testing and disconnect log clustered and the
disconnect  did actually happen. Then I switch back to the sequential case
that the receiving channel gots filled without receiving until
disconnection. It works now.

I am really curious about your example on select -  is it a try... except
alternative? Can it be used in other scenarios?
func Fill(ch chan int, k int) {
select {
case ch <- k:
// sent
default:
// discard
}
}

On Fri, Apr 15, 2022 at 3:12 PM Brian Candler  wrote:

> On Friday, 15 April 2022 at 06:47:09 UTC+1 yan.z...@gmail.com wrote:
>
>> When one server finds something wrong, it sends out a request to another
>> server for help.
>> Then it makes a log and then a ticker to recheck the situation again and
>> again and in the meantime, to receive a response from that server that
>> indicates everything is fine.
>>
>> So here is a gap between sending help and getting ready to receive the
>> "ok" response. It is a tiny gap but it is not possible that the other
>> server sends the "ok" message back through TCP connection just in this gap.
>> Yes my program handles the TCP message as always, in this case sends back
>> through the channel to the goroutine that has not prepared the select for
>> ticker channel and  ok-receiving channel. Uh-oh, deadlock, crash! What is
>> your suggestion? buffered channel?
>>
>
> TCP sockets and Go channels are two completely different things.
>
> But in either case, if a receiver becomes ready before the sender, it's
> fine.  The receiver just waits for data to arrive.
>
> And if the receiver is not ready first, it's also fine.  The sender will
> send up to the buffer size (in the case of a buffered channel), and after
> that it will pause until the receiver is ready.  (In the case of TCP you
> have quite a lot of buffering in network sockets; typically it could send a
> megabyte or more before it pauses).
>
> A pause is not a "deadlock".  A pause is just waiting until it's OK to
> proceed.
>
> In this case you already admit that opportunity that the sending happens
>> when the receiver is not ready.
>>
>
> And there is no problem at all. It really doesn't matter if the sender
> becomes ready first, or the receiver becomes ready first.  Here are two
> concrete examples, using an unbuffered channel:
>
> https://go.dev/play/p/r5eptx4dE7T
> https://go.dev/play/p/sTB5lio6Rhx
>
> Aside: in those examples I have used a sync.WaitGroup to ensure that the
> main program doesn't terminate until both the goroutines have completed.
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/6ExktXrF5Xc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/9a0a275e-54cf-40da-b0ac-8ce724fc3d49n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CADEX6_X%3Dje43R%3DRQt8PWz64-r1VkDaoB8Z7F3b%3D7Y0byB___1g%40mail.gmail.com.

[go-nuts] Re: Protective buffered channel that never triggers deadlock

[Zhaoxun Yan]

This code really blows my mind, Brian! I didn't know select can be flexible 
as this:

package main

import (
"fmt"
)

var Ch = make(chan int, 1)

func Fill(ch chan int, k int) {
select {
case ch <- k:
// sent
default:
// discard
}
}

func main() {

fmt.Println("Channel Status: ", len(Ch), cap(Ch))
Fill(Ch, 5)
fmt.Println("Channel Status: ", len(Ch), cap(Ch))
fmt.Println("get one out of Channel:", <-Ch)
fmt.Println("Channel Status: ", len(Ch), cap(Ch))

Fill(Ch, 6)
// Ch <- 7 //fatal error: all goroutines are asleep - deadlock!
Fill(Ch, 7)
fmt.Println("Channel Status: ", len(Ch), cap(Ch))
fmt.Println("get one out of Channel:", <-Ch)
}

在2022年4月13日星期三 UTC+8 17:49:58 写道：

> > I made up a work around that just dumps the new message if the buffer is 
> full:
>
> You don't need to do that: use select { ... } to check if the channel is 
> ready to receive or not.
> https://go.dev/play/p/Mb7VVCTvnfk
>
> But in practice, this is almost never needed, because one goroutine is 
> doing the sending and a different goroutine is doing the receiving.  The 
> sending goroutine blocks until the receiver is ready.
>
> On Wednesday, 13 April 2022 at 09:40:22 UTC+1 yan.z...@gmail.com wrote:
>
>> Since I found if inserting into a buffered channel could cause a crash if 
>> it is full already
>> ( now I only use unbuffered channels in work)
>> https://groups.google.com/g/golang-nuts/c/U8lz6noKkuA
>>
>> package main
>>
>> var c = make(chan int, 1)
>>
>> func main() {
>>
>> c <- 1
>> c <- 2 //fatal error: all goroutines are asleep - deadlock!
>> c <- 3
>>
>> } 
>>
>> I made up a work around that just dumps the new message if the buffer is 
>> full:
>>
>> package main
>>
>> import (
>> "fmt"
>> "sync"
>> )
>>
>> type ChanInt struct{
>> Ch chan int
>> Mu *sync.Mutex
>> }
>>
>> var c = ChanInt{make(chan int, 1), new(sync.Mutex)}
>>
>> func (ch ChanInt) Fill(k int){
>> ch.Mu.Lock()
>> defer ch.Mu.Unlock()
>> 
>> if len(ch.Ch) < cap(ch.Ch){
>> ch.Ch <- k
>> }
>> }
>>
>> func main() {
>> 
>> fmt.Println("Channel Status: ",len(c.Ch), cap(c.Ch))
>> c.Fill(5)
>> fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))
>> fmt.Println("get one out of Channel:", <-c.Ch)
>> fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))
>> 
>> c.Fill(6)
>> // c.Ch <- 7 //fatal error: all goroutines are asleep - deadlock!
>> c.Fill(7)
>> fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))
>> fmt.Println("get one out of Channel:", <-c.Ch)
>> }
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/223ebc9a-d7ac-4ef7-baea-f2dedbc7c787n%40googlegroups.com.

Re: [go-nuts] Tool to check binaries for vulnerabilities

[Zhaoxun Yan]

That sounds great! Thanks.

在2022年4月15日星期五 UTC+8 05:55:27 写道：

> On Thu, 2022-04-14 at 03:05 -0700, Michel Casabianca wrote:
> > Any comment and contribution welcome.
>
> Can I suggest that you use golang.org/x/sys/execabs rather than os/exec
> in ExecCommand?
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/454f35cf-6599-4fc1-9dd2-602137d58cb6n%40googlegroups.com.

[go-nuts] Re: generics: parametric types unmarshaling

[Zhaoxun Yan]

Here is my note on json & struct, run it somewhere and it may give you a 
hint.

package main

import "fmt"
import "encoding/json"

type prices struct{
Price   float64 `json:"price"`
floor   float64
Ceiling float64
settle  float64
timeint64
}


func main() {
content := prices{
Price:  6.4 , 
floor:  3.5, 
Ceiling:7, 
settle: 8.5,
time:   1,
}

js,_ := json.Marshal( content)
fmt.Println(string(js))

var back prices
json.Unmarshal(js, )
fmt.Println(back)
fmt.Println(back.Price)
}

在2022年4月14日星期四 UTC+8 16:51:22 写道：

> hi there,
>
> I am playing a bit with generics.
> I am trying to implement a parametrized "Read" function that unmarshals
> bytes into some type value that implements an interface:
>
> type T1 struct {
> v string
> }
>
> func (t *T1) UnmarshalBinary(p []byte) error {
> t.v = string(p)
> return nil
> }
>
> type Unmarshaler interface {
> UnmarshalBinary(p []byte) error
> }
>
> func f1[T Unmarshaler](p []byte) (T, error) {
> var t T
> err := t.UnmarshalBinary(p)
> return t, err
> }
>
> func main() {
> p := []byte("hello")
> t1, err := f1[T1](p)
> if err != nil {
> log.Panic(err)
> }
>
> log.Printf("t1: %+v", t1)
> }
>
> my naive attempt failed like so:
>
> ./prog.go:26:16: T1 does not implement Unmarshaler (UnmarshalBinary method 
> has pointer receiver)
>
> it's only when I rewrite my f1 function like so that it "works":
>
> func f2[T any](p []byte) (t T, err error) {
> switch t := any().(type) {
> case Unmarshaler:
> err = t.UnmarshalBinary(p)
> default:
> panic("boo")
> }
> return t, err
> }
>
> but it doesn't take advantage of the nice type constraints Go's generics
> provide.
>
> of course, replacing:
> t1, err := f1[T1](p)
>
> with:
> t1, err := f1[*T1](p)
> compiles. but fails at runtime.
>
> what am I missing?
> how do I convey the constraint "unmarshaling usually imply passing a
> pointer to some value" ?
>
> -s
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1679908f-db42-4eb7-88d9-1efc95ef7bbbn%40googlegroups.com.

Re: [go-nuts] Re: Protective buffered channel that never triggers deadlock

[Zhaoxun Yan]

Something bizarre happened in CGO. My new code started the anonymous
goroutine to receive first but it does not sleep below the goroutine and
jumps to end the function. So I  returned to the sending before receiving
way, which guaranteed the program's safety.

There ain't always one way that fits all.  Sometimes a public variable is
better than a channel.
Knowing the boundaries of your tool is always better than believing it will
always prevail.

*Blocking on send until the channel is ready is exactly what is required
normally.*
* " Are you suggesting something else should happen? If so, what exactly?"*

Beyond the cgo accident, Here is one scenario:

When one server finds something wrong, it sends out a request to another
server for help.
Then it makes a log and then a ticker to recheck the situation again and
again and in the meantime, to receive a response from that server that
indicates everything is fine.

So here is a gap between sending help and getting ready to receive the "ok"
response. It is a tiny gap but it is not possible that the other server
sends the "ok" message back through TCP connection just in this gap. Yes my
program handles the TCP message as always, in this case sends back through
the channel to the goroutine that has not prepared the select for ticker
channel and  ok-receiving channel. Uh-oh, deadlock, crash! What is your
suggestion? buffered channel? In this case you already admit that
opportunity that the sending happens when the receiver is not ready.

On Thu, Apr 14, 2022 at 3:40 PM Brian Candler  wrote:

> The fact that the sender may have to wait does not by itself make a
> "deadlock".
>
> It's a deadlock if the sender blocks and, due to poor programming, the
> receiver is *never* ready to receive. That's a problem with your software,
> not with Go's concept of channels.
>
> Blocking on send until the channel is ready is exactly what is required
> normally. Are you suggesting something else should happen? If so, what
> exactly?
>
> This is well worth watching:
> Rethinking Classical Concurrency Patterns (Bryan C. Mills)
> https://www.youtube.com/watch?v=5zXAHh5tJqQ
>
> On Thursday, 14 April 2022 at 02:55:56 UTC+1 yan.z...@gmail.com wrote:
>
>> You are right, Brian.
>> The difference is not whether the channel is buffered or unbuffered -
>> while the receiver is present, none will encounter the deadlock.
>>
>> package main
>>
>> import (
>> "fmt"
>> "time"
>> )
>>
>> //var c = make(chan int)
>> var c = make(chan int, 2)
>>
>> func report(){
>> for{
>> select{
>> case n:= <- c:
>> fmt.Println(n, time.Now().Format("2006-01-02 15:04:05"))
>> default:
>> time.Sleep(time.Second)
>> }
>> }
>> }
>>
>> func main() {
>>
>> go report()
>>
>> for i:=1; i<=5; i++{
>> c <- i
>> }
>>
>> }
>>
>> But obviously, the buffered channel provides another layer of support in
>> case the receiver is temporarily absent:
>>
>> package main
>>
>> import (
>> "fmt"
>> "time"
>> )
>>
>> //var c = make(chan int)
>> var c = make(chan int, 5)
>>
>> func report(){
>> for{
>> select{
>> case n:= <- c:
>> fmt.Println(n, time.Now().Format("2006-01-02 15:04:05"))
>> default:
>> time.Sleep(time.Second)
>> }
>> }
>> }
>>
>> func main() {
>>
>> for i:=1; i<=5; i++{
>> c <- i
>> }
>>
>> report()
>>
>> }
>>
>>
>> > I made up a work around that just dumps the new message if the buffer
>>> is full:
>>>
>>> You don't need to do that: use select { ... } to check if the channel is
>>> ready to receive or not.
>>> https://go.dev/play/p/Mb7VVCTvnfk
>>>
>>> But in practice, this is almost never needed, because one goroutine is
>>> doing the sending and a different goroutine is doing the receiving.  The
>>> sending goroutine blocks until the receiver is ready.
>>>
>>> On Wednesday, 13 April 2022 at 09:40:22 UTC+1 yan.z...@gmail.com wrote:
>>>
 Since I found if inserting into a buffered channel could cause a crash
 if it is full already
 ( now I only use unbuffered channels in work)
 https://groups.google.com/g/golang-nuts/c/U8lz6noKkuA

 package main

 var c = make(chan int, 1)

 func main() {

 c <- 1
 c <- 2 //fatal error: all goroutines are asleep - deadlock!
 c <- 3

 }

 I made up a work around that just dumps the new message if the buffer
 is full:

 package main

 import (
 "fmt"
 "sync"
 )

 type ChanInt struct{
 Ch chan int
 Mu *sync.Mutex
 }

 var c = ChanInt{make(chan int, 1), new(sync.Mutex)}

 func (ch ChanInt) Fill(k int){
 ch.Mu.Lock()
 defer ch.Mu.Unlock()

 if len(ch.Ch) < cap(ch.Ch){
 ch.Ch <- k
 }
 }

 func main() {

 fmt.Println("Channel Status: ",len(c.Ch), cap(c.Ch))
 

Re: [go-nuts] Re: Protective buffered channel that never triggers deadlock

[Zhaoxun Yan]

You are right, Brian.
The difference is not whether the channel is buffered or unbuffered - while
the receiver is present, none will encounter the deadlock.

package main

import (
"fmt"
"time"
)

//var c = make(chan int)
var c = make(chan int, 2)

func report(){
for{
select{
case n:= <- c:
fmt.Println(n, time.Now().Format("2006-01-02 15:04:05"))
default:
time.Sleep(time.Second)
}
}
}

func main() {

go report()

for i:=1; i<=5; i++{
c <- i
}

}

But obviously, the buffered channel provides another layer of support in
case the receiver is temporarily absent:

package main

import (
"fmt"
"time"
)

//var c = make(chan int)
var c = make(chan int, 5)

func report(){
for{
select{
case n:= <- c:
fmt.Println(n, time.Now().Format("2006-01-02 15:04:05"))
default:
time.Sleep(time.Second)
}
}
}

func main() {

for i:=1; i<=5; i++{
c <- i
}

report()

}


> I made up a work around that just dumps the new message if the buffer is
> full:
>
> You don't need to do that: use select { ... } to check if the channel is
> ready to receive or not.
> https://go.dev/play/p/Mb7VVCTvnfk
>
> But in practice, this is almost never needed, because one goroutine is
> doing the sending and a different goroutine is doing the receiving.  The
> sending goroutine blocks until the receiver is ready.
>
> On Wednesday, 13 April 2022 at 09:40:22 UTC+1 yan.z...@gmail.com wrote:
>
>> Since I found if inserting into a buffered channel could cause a crash if
>> it is full already
>> ( now I only use unbuffered channels in work)
>> https://groups.google.com/g/golang-nuts/c/U8lz6noKkuA
>>
>> package main
>>
>> var c = make(chan int, 1)
>>
>> func main() {
>>
>> c <- 1
>> c <- 2 //fatal error: all goroutines are asleep - deadlock!
>> c <- 3
>>
>> }
>>
>> I made up a work around that just dumps the new message if the buffer is
>> full:
>>
>> package main
>>
>> import (
>> "fmt"
>> "sync"
>> )
>>
>> type ChanInt struct{
>> Ch chan int
>> Mu *sync.Mutex
>> }
>>
>> var c = ChanInt{make(chan int, 1), new(sync.Mutex)}
>>
>> func (ch ChanInt) Fill(k int){
>> ch.Mu.Lock()
>> defer ch.Mu.Unlock()
>>
>> if len(ch.Ch) < cap(ch.Ch){
>> ch.Ch <- k
>> }
>> }
>>
>> func main() {
>>
>> fmt.Println("Channel Status: ",len(c.Ch), cap(c.Ch))
>> c.Fill(5)
>> fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))
>> fmt.Println("get one out of Channel:", <-c.Ch)
>> fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))
>>
>> c.Fill(6)
>> // c.Ch <- 7 //fatal error: all goroutines are asleep - deadlock!
>> c.Fill(7)
>> fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))
>> fmt.Println("get one out of Channel:", <-c.Ch)
>> }
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/6ExktXrF5Xc/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/7df07f32-d7aa-45db-91d6-de75e18ad8d8n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CADEX6_VU3CcG2jcaXiCdVPTja4TEzDiwHsNhx3g4aGE7ypBpog%40mail.gmail.com.

[go-nuts] Protective buffered channel that never triggers deadlock

[Zhaoxun Yan]

Since I found if inserting into a buffered channel could cause a crash if 
it is full already
( now I only use unbuffered channels in work)
https://groups.google.com/g/golang-nuts/c/U8lz6noKkuA

package main

var c = make(chan int, 1)

func main() {
   
c <- 1
c <- 2 //fatal error: all goroutines are asleep - deadlock!
c <- 3

} 

I made up a work around that just dumps the new message if the buffer is 
full:

package main

import (
"fmt"
"sync"
)

type ChanInt struct{
Ch chan int
Mu *sync.Mutex
}

var c = ChanInt{make(chan int, 1), new(sync.Mutex)}

func (ch ChanInt) Fill(k int){
ch.Mu.Lock()
defer ch.Mu.Unlock()

if len(ch.Ch) < cap(ch.Ch){
ch.Ch <- k
}
}

func main() {

fmt.Println("Channel Status: ",len(c.Ch), cap(c.Ch))
c.Fill(5)
fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))
fmt.Println("get one out of Channel:", <-c.Ch)
fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))

c.Fill(6)
// c.Ch <- 7 //fatal error: all goroutines are asleep - deadlock!
c.Fill(7)
fmt.Println("Channel Status: ", len(c.Ch), cap(c.Ch))
fmt.Println("get one out of Channel:", <-c.Ch)
}

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9ca5c13b-7797-4d64-bc2c-1f1ff7281168n%40googlegroups.com.

[go-nuts] Re: Why Go has only slice and map

[Zhaoxun Yan]

 https://github.com/Workiva/go-datastructures 
Are they available in the current version of go?

在2022年4月12日星期二 UTC+8 23:21:20 写道：

> I'm certainly not privy to the nitty-gritty, but I'd encourage you to skim 
> over the Go Dev team meeting notes. It's really cool to see what people 
> have proposed over the years, how the team interacted with it, and 
> ultimately why yes, why no. It's been really educational.
>
> The short of it is Java. That's a joke, because Java isn't short. We don't 
> need another Java. The result is that an idea has to be very, very, very 
> good to be accepted and developed internally. In the meantime, you should 
> check out this repo: https://github.com/Workiva/go-datastructures It 
> hasn't changed much lately, but that's because, for its API, it's maybe 
> mostly done. The issues are generally from bad expectations, and there 
> isn't much to add. Generics means that some of them could be implemented 
> differently, but that's more an indicator that better projects might exist 
> nowadays.
>
> On Tuesday, April 12, 2022 at 1:19:22 AM UTC-5 Jack Li wrote:
>
>> Hi group,
>>
>> Why Go provides only 2 built-in data structures, slice and map. It has 
>> just more than C, but less than all other programming languages I've heard 
>> of, C++, Python, Swift, Rust. 
>>
>> I think this simplicity attracts me very much. Sometimes I can use only 
>> one data structures for a task. Because there is no more similar data 
>> structures for choice. It's really "one way to do one thing". This also 
>> makes code like familiar and consistent.
>>
>> I want to know more about the mind behind this design of built-in data 
>> structures.
>>
>> Thanks
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/993ffa15-0ee8-4d61-b016-795d7a0068f0n%40googlegroups.com.

Re: [go-nuts] encoding/json mistakenly transfer int64 format to string

[Zhaoxun Yan]

Thank you Steven.
I am a little bewildered by  the new mod configuration. Will it compile If
I download the source file from the new github source to src directory
without further setting up a mod in the old fashioned way? I am using
go1.15.

And today I ran a test logging all the sendings to Redis and readings from
Redis. It is fine - nothing went wrong. But AnotherRedisDesktopManager
still shows the same string format - which I guess is due to it is 32bit
and fail to transfer a long decimal to float32.

So nothing is wrong in golang or its open-source package.

On Tue, Apr 12, 2022 at 9:56 PM Steven Hartland 
wrote:

> First off, the package you're using for redis isn't maintained; you should
> switch to github.com/gomodule/redigo/redis instead, which will allow you
> to remove the c == nil check as that doesn't happen.
>
> In your example you're ignoring error from json.Marshal which could be
> hiding a problem, so I would recommend you handle that.
>
> encoding/json should represent float as a json number so I would never
> expect what you're seeing but its not clear to me if that is down to how
> you are viewing it.
>
> On Tue, 12 Apr 2022 at 04:02, Zhaoxun Yan  wrote:
>
>> The scenario is upon receiving an incoming financial quotation, save it
>> as a string of json into a Redis service. Sorry but I cannot provide the
>> whole code of quotation receiving here, which is very complex with cgo. But
>> the code below will help you get a glimpse on what should be going on:
>>
>> import (
>> "encoding/json"
>> //"errors"
>> "fmt"
>> "time"
>>
>> "github.com/garyburd/redigo/redis"
>> )
>>
>> var pool *redis.Pool
>>
>> type Fvprices struct {
>> P float64 `json:"price"`
>> F float64 `json:"floor"`
>> C float64 `json:"ceiling"`
>> S float64 `json:"settle"`
>> T int64   `json:"time"`
>> }
>>
>> func init() {
>> pool = newPool()
>> }
>>
>> var redisport = "6379"
>> var redisip = "127.0.0.1"
>> var password = ""
>>
>> func newPool() *redis.Pool {
>>
>> fmt.Println("redis @", redisport, redisip, password)
>> return {
>> MaxIdle: 4,
>> MaxActive:   50, // max number of connections
>> IdleTimeout: 30 * time.Second,
>>
>> Dial: func() (redis.Conn, error) {
>> c, err := redis.DialURL("redis://" + redisip + ":" +
>> redisport)
>> if err != nil {
>> ErrMsg = fmt.Sprintf("redis connection error: %s", err.
>> Error())
>> fmt.Println(time.Now().Format("2006-01-02 15:04:05"),
>> ErrMsg)
>> return nil, err
>> }
>> if _, autherr := c.Do("AUTH", password); autherr != nil {
>> ErrMsg = fmt.Sprintf("redis password error: %s", err.
>> Error())
>> fmt.Println(time.Now().Format("2006-01-02 15:04:05"),
>> ErrMsg)
>> return nil, autherr
>> }
>> return c, nil
>> },
>> }
>> }
>>
>> func Upfutureprice(future_id string,
>> future_price, lowerLimitPrice, upperLimitPrice, preSettlementPrice
>> float64,
>> updatetime time.Time) {
>>
>> c := pool.Get()
>> if c == nil {
>> return
>> }
>> defer c.Close()
>>
>> content := Fvprices{
>> P: future_price,
>> F: lowerLimitPrice,
>> C: upperLimitPrice,
>> S: preSettlementPrice,
>> T: updatetime.UnixNano() / 1e6,
>> }
>>
>> js, _ := json.Marshal(content)
>>
>> if _, err := c.Do("SET", future_id, js); err != nil {
>> fmt.Println("cannot save to redis：", err)
>> }
>> }
>>
>> So obviously until the function "Upfutureprice" everything is correct,
>> for all  four prices it receives are in float64 format. After running this
>> program for one day, I just browse the redis using
>> AnotherRedisDesktopManager via ssh port forwarding, and something strange
>> happens as I clicking on various future_id key strings:
>>
>> {
>> price:807
>> floor:720.6
>> ceiling:881
>> settle:"800.80001"
>> time:1649726499000
>> }
>>
>> {
>> price:"369

Re: [go-nuts] Hesitating on using cached / un-cached channels

[Zhaoxun Yan]

Yes you are right! Unbuffered channel is solid!

package main

import (
"fmt"
"time"
)

var c = make(chan int)

func report(){
for{
select{
case n:= <- c:
fmt.Println(n, time.Now().Format("2006-01-02 15:04:05"))
default:
time.Sleep(time.Second)
}
}
}

func main() {

go report()

for i:=1; i<=5; i++{
c <- i
}

}



On Tue, Apr 12, 2022 at 12:31 PM burak serdar  wrote:

> An unbuffered channel is a synchronization mechanism. A send on an
> unbuffered channel will block until there is a concurrent receive from
> another goroutine.
>
> Your program has only one goroutine. A send to an unbuffered channel will
> always deadlock, so will a receive from it. So the problem you are
> describing is not a problem related to the way unbuffered channels work,
> but it is due to the way the program is organized: it is written to
> deadlock. With a buffered channel, it can run. This program does not need a
> buffer more than 1. Once the send runs, the channel will be full, and the
> second case will be activated. The first case cannot run until the second
> case is done, which will terminate the loop.
>
> On Mon, Apr 11, 2022 at 10:08 PM robert engels 
> wrote:
>
>> There are numerous ways to create a “dead lock” in any program (this may
>> actually be a “live lock” but I didn’t fully understand your statement -
>> this is just one of them.
>>
>>
>> On Apr 11, 2022, at 9:29 PM, Zhaoxun Yan  wrote:
>>
>> Hi guys, I have a great demonstration on why an un-cached channel might
>> malfunction while receiving:
>>
>> package main
>> import(
>> "fmt"
>> "time"
>> )
>>
>> func main(){
>> t := time.NewTicker(time.Second * 3)
>> stopnow := make(chan bool)
>> //stopnow := make(chan bool, 1) //cached channel instead
>> var n int
>>
>> for{
>> select{
>> case <-t.C:
>> n++
>> fmt.Printf("%d\n", n)
>>
>> if n==3{
>> stopnow <- true //The Only Sending!!!
>> t.Stop()
>> }
>>
>> case a:=<-stopnow: //The Only Receiving!!!
>> if a{
>> goto END
>> }
>> }
>> }
>> END:
>> fmt.Println("out of select")
>> }
>>
>> In the code above, you will never see the printout "out of select" ,
>> because the for-select receiver is not working while the code is run inside
>> timer receiver in the line " stopnow <- true".
>>
>> So an un-cached channel like " stopnow := make(chan bool)" will
>> occasionally but inevitably miss receiving while the code is busy
>> processing a competing receiving.
>>
>> That is why I use cached channel instead, like " stopnow := make(chan
>> bool, 1)", which un-received message(s) will be cached until gotten
>> received.
>>
>> However there comes another vulnerability - How large should the cache be?
>>
>> In a simple scenario like this, I am very confident there will be 1
>> message to receive only. But on a complex server, it is very common that a
>> goroutine will receive unexpected many messages from other goroutine or
>> functions. And it is not safe for the programmer to just guess a ceiling
>> for the number of unprocessed messages on any time - just as to guess the
>> maximum on how many cars can line up after a red light. If you guess wrong,
>> only one additional message will breach the cache and cause a crash:
>>
>> package main
>>
>> var c = make(chan int, 1)
>>
>> func main() {
>>
>> c <- 1
>> c <- 2 //fatal error: all goroutines are asleep - deadlock!
>> c <- 3
>>
>> }
>>
>> The code above crashes at the point that the cache of channel c is full,
>> but the sender still puts another message into it.
>>
>> What is your thought on this?
>> Regards,
>> Zhaoxun
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/fefc350d-3423-4fb6-b844-703ce03c7e5fn%40googlegroups.com
>> <https://groups.google

[go-nuts] Re: Is it possible to change the existing printline dynamically in golang?

[Zhaoxun Yan]

Thanks a log Amnon! :D

在2022年4月1日星期五 UTC+8 21:25:14 写道：

> Yes, this is the murky world of ANSI escape codes.
> Fortunately there are a whole load of libraries which do this for you...
> Try https://github.com/cheggaaa/pb
> or https://github.com/schollz/progressbar
> or github.com/vardius/progress-go
>
>
> On Friday, 1 April 2022 at 13:12:11 UTC+1 yan.z...@gmail.com wrote:
>
>> Got it:
>>
>> package main
>>
>> import(
>> "fmt"
>> "time"
>> )
>>
>> func main() {
>>   fmt.Printf("Hello")
>>   time.Sleep(time.Second)
>>   time.Sleep(time.Second)
>>   fmt.Printf("\r")
>>   fmt.Printf("World\n")
>> }
>>
>> 在2022年4月1日星期五 UTC+8 15:34:08 写道：
>>
>>> You can use the ansi escape code if the target terminal supports it. 
>>> Alternatively, you can use the carriage return '\r' and reprint the line. 
>>> Note you may need to overwrite with empty space to delete the line before 
>>> rewriting it.
>>>
>>> On Friday, April 1, 2022 at 12:38:37 PM UTC+7 yan.z...@gmail.com wrote:
>>>
 I just noticed how python pip upgraded from printing numerous process 
 bars like this:
 ■■■   30% completed
  40% completed
 ■■60% completed
    80% completed
 ■■  100% completed

 to a single line of a growing bar and changing declaration.

 It is definitely a functionality that  prompts of both Linux and 
 Windows allows -
 To change the previous print line.
 Is it possible to realize this in golang?

   Zhaoxun

>>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/cb9fdc0b-fc25-4b7b-b53b-1377f906829dn%40googlegroups.com.

[go-nuts] encoding/json mistakenly transfer int64 format to string

[Zhaoxun Yan]

The scenario is upon receiving an incoming financial quotation, save it as 
a string of json into a Redis service. Sorry but I cannot provide the whole 
code of quotation receiving here, which is very complex with cgo. But the 
code below will help you get a glimpse on what should be going on:

import (
"encoding/json"
//"errors"
"fmt"
"time"

"github.com/garyburd/redigo/redis"
)

var pool *redis.Pool

type Fvprices struct {
P float64 `json:"price"`
F float64 `json:"floor"`
C float64 `json:"ceiling"`
S float64 `json:"settle"`
T int64   `json:"time"`
}

func init() {
pool = newPool()
}

var redisport = "6379"
var redisip = "127.0.0.1"
var password = ""

func newPool() *redis.Pool {

fmt.Println("redis @", redisport, redisip, password)
return { 
MaxIdle: 4,
MaxActive:   50, // max number of connections
IdleTimeout: 30 * time.Second,

Dial: func() (redis.Conn, error) {
c, err := redis.DialURL("redis://" + redisip + ":" + redisport)
if err != nil {
ErrMsg = fmt.Sprintf("redis connection error: %s", err.Error
())
fmt.Println(time.Now().Format("2006-01-02 15:04:05"), 
ErrMsg)
return nil, err
}
if _, autherr := c.Do("AUTH", password); autherr != nil {
ErrMsg = fmt.Sprintf("redis password error: %s", err.Error
())
fmt.Println(time.Now().Format("2006-01-02 15:04:05"), 
ErrMsg)
return nil, autherr
}
return c, nil
},
}
}

func Upfutureprice(future_id string,
future_price, lowerLimitPrice, upperLimitPrice, preSettlementPrice 
float64,
updatetime time.Time) {

c := pool.Get()
if c == nil {
return
}
defer c.Close()

content := Fvprices{
P: future_price,
F: lowerLimitPrice,
C: upperLimitPrice,
S: preSettlementPrice,
T: updatetime.UnixNano() / 1e6,
}

js, _ := json.Marshal(content)

if _, err := c.Do("SET", future_id, js); err != nil {
fmt.Println("cannot save to redis：", err)
}
}

So obviously until the function "Upfutureprice" everything is correct, for 
all  four prices it receives are in float64 format. After running this 
program for one day, I just browse the redis using 
AnotherRedisDesktopManager via ssh port forwarding, and something strange 
happens as I clicking on various future_id key strings:

{
price:807
floor:720.6
ceiling:881
settle:"800.80001"
time:1649726499000
}

{
price:"3691.5"
floor:3237
ceiling:4204
settle:3721
time:1649726910500
}

{
price:"15405.0004"
floor:13625
ceiling:17340
settle:15485
time:1649728303500
}

{
price:"800.40001"
floor:720.6
ceiling:881
settle:"800.80001"
time:1649728048000
}

Note quotations above. I wonder how encoding/json can made transformation 
from a float64 inside struct Fvprices  to a string instead? It seems that 
only long decimals would trigger such an error while short decimals won't:

{
price:2910
floor:2443.5
ceiling:3305.5
settle:2874.5
time:1649728261026
}

How could that happen? I am really puzzled.

Regards,
Zhaoxun

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f42b29b3-de17-48c6-9f71-1176f1288396n%40googlegroups.com.

[go-nuts] Hesitating on using cached / un-cached channels

[Zhaoxun Yan]

Hi guys, I have a great demonstration on why an un-cached channel might 
malfunction while receiving:

package main
import(
"fmt"
"time"
) 

func main(){
t := time.NewTicker(time.Second * 3)
stopnow := make(chan bool) 
//stopnow := make(chan bool, 1) //cached channel instead
var n int

for{
select{
case <-t.C:
n++
fmt.Printf("%d\n", n)

if n==3{
stopnow <- true //The Only Sending!!!
t.Stop()
}

case a:=<-stopnow: //The Only Receiving!!!
if a{
goto END
}
}
}
END:
fmt.Println("out of select")
}

In the code above, you will never see the printout "out of select" , 
because the for-select receiver is not working while the code is run inside 
timer receiver in the line " stopnow <- true".

So an un-cached channel like " stopnow := make(chan bool)" will 
occasionally but inevitably miss receiving while the code is busy 
processing a competing receiving.

That is why I use cached channel instead, like " stopnow := make(chan bool, 
1)", which un-received message(s) will be cached until gotten received.

However there comes another vulnerability - How large should the cache be?

In a simple scenario like this, I am very confident there will be 1 message 
to receive only. But on a complex server, it is very common that a 
goroutine will receive unexpected many messages from other goroutine or 
functions. And it is not safe for the programmer to just guess a ceiling 
for the number of unprocessed messages on any time - just as to guess the 
maximum on how many cars can line up after a red light. If you guess wrong, 
only one additional message will breach the cache and cause a crash:

package main

var c = make(chan int, 1)

func main() {

c <- 1
c <- 2 //fatal error: all goroutines are asleep - deadlock!
c <- 3

} 

The code above crashes at the point that the cache of channel c is full, 
but the sender still puts another message into it.

What is your thought on this?
Regards,
Zhaoxun

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/fefc350d-3423-4fb6-b844-703ce03c7e5fn%40googlegroups.com.

[go-nuts] Re: Is it possible to change the existing printline dynamically in golang?

[Zhaoxun Yan]

Got it:

package main

import(
"fmt"
"time"
)

func main() {
  fmt.Printf("Hello")
  time.Sleep(time.Second)
  time.Sleep(time.Second)
  fmt.Printf("\r")
  fmt.Printf("World\n")
}

在2022年4月1日星期五 UTC+8 15:34:08 写道：

> You can use the ansi escape code if the target terminal supports it. 
> Alternatively, you can use the carriage return '\r' and reprint the line. 
> Note you may need to overwrite with empty space to delete the line before 
> rewriting it.
>
> On Friday, April 1, 2022 at 12:38:37 PM UTC+7 yan.z...@gmail.com wrote:
>
>> I just noticed how python pip upgraded from printing numerous process 
>> bars like this:
>> ■■■   30% completed
>>  40% completed
>> ■■60% completed
>>    80% completed
>> ■■  100% completed
>>
>> to a single line of a growing bar and changing declaration.
>>
>> It is definitely a functionality that  prompts of both Linux and Windows 
>> allows -
>> To change the previous print line.
>> Is it possible to realize this in golang?
>>
>>   Zhaoxun
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/702dad6d-006f-47b5-94d5-ca231b0245fcn%40googlegroups.com.

Re: [go-nuts] Why is it forbidden to add methods to an existing type?

[Zhaoxun Yan]

Hi Sam! Your solution does not seem to work:

package main

import(
"fmt"
"strconv"
)

type  String struct{string}

func (s String) print(){
fmt.Println(s)
}

func main() {
   var a String ="hello, world\n"
   a.print()
   
   fmt.Println(strconv.ParseInt("78",10, 64))
   
   var x String ="452"
   n, _ := strconv.ParseInt(x, 10, 64)
}

在2022年3月26日星期六 UTC+8 06:41:15 写道：

>
> My workaround like is something like `type String struct{string}. 
> It can be reasonably treated as a string for most cases in which as string 
> is needed, and it lets you convert back conveniently from any scope in 
> which it's reasonable for your program to know the difference.
> On Friday, March 18, 2022 at 12:46:34 AM UTC-5 Henry wrote:
>
>> My own preference is to have a small number of methods and put the 
>> general functionalities into functions. By putting the general 
>> functionalities into functions, you allow code reuse. In object-oriented 
>> programming, you normally attach as many functionalities as possible to 
>> their corresponding types and achieve code reuse via inheritance. Since Go 
>> does not have inheritance, you can achieve a similar effect with standalone 
>> functions. 
>>
>> On Friday, March 18, 2022 at 11:26:51 AM UTC+7 Ian Lance Taylor wrote:
>>
>>> On Thu, Mar 17, 2022 at 7:17 PM Zhaoxun Yan  wrote: 
>>> > 
>>> > I just came across this taboo in golang - new methods cannot be added 
>>> to an existing type: 
>>>
>>> Yes. If we didn't have this prohibition, then the set of interfaces 
>>> satisfied by a type would depend on which package was using the type. 
>>>
>>> See the discussion at https://go.dev/issue/21401. 
>>>
>>> Ian 
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5ef48e3d-e921-46cc-8bf8-a1c91c429845n%40googlegroups.com.

[go-nuts] Is it possible to change the existing printline dynamically in golang?

[Zhaoxun Yan]

I just noticed how python pip upgraded from printing numerous process bars 
like this:
■■■   30% completed
 40% completed
■■60% completed
   80% completed
■■  100% completed

to a single line of a growing bar and changing declaration.

It is definitely a functionality that  prompts of both Linux and Windows 
allows -
To change the previous print line.
Is it possible to realize this in golang?

  Zhaoxun

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/544cba84-31c8-46a7-a9ed-eea65c5c4927n%40googlegroups.com.

Re: [go-nuts] how to protect source code

[Zhaoxun Yan]

I think it is best to run your code on an encrypted disk. So it cannot be 
stolen by taking away the hard-drive.
Furthermore if the config and log files are in other folders, it is okay to 
shut down the encrypted drive I guess.
In that case even it is running in memory, the hacker cannot get access to 
the program file on removed encrypted disk.

Beyond that, there is  few software in the world up till now I believe, 
could decode your program back to go source code.
And it is a better practice to put all the codes together rather then 
dividing it to smaller pieces to make more complexity.

Zhaoxun

在2022年3月18日星期五 UTC+8 09:30:21 写道：

> On Thu, Mar 17, 2022 at 6:46 AM bbb tt  wrote:
> >
> > I want to encrypt my algorithm library, is there any good way in Go. My 
> library is used in both Windows and Linux
>
> For a security issue like this it's essential to define the attack you
> want to defend against.
>
> The nature of Go is such that against a sophisticated attacker the
> best you can do in practice is to hide source code comments, or to run
> your program only as a secure service that is only accessible over a
> network. And to do that you don't need to do anything special at all.
>
> Ian
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5050ddc7-6a56-415a-923a-58caa6dac54dn%40googlegroups.com.

[go-nuts] Why is it forbidden to add methods to an existing type?

[Zhaoxun Yan]

Hi everyone!
I just came across this taboo in golang - new methods cannot be added to an 
existing type:

package main
import "fmt"

func (s string) print(){
fmt.Println(s)
}

func main() {
   "hello, world\n".print()
}
--Error
./main.go:5: cannot define new methods on non-local type string

So I try to go around it by declaring an inheritance type. But it will thus 
be forbidden from functions already defined for the original type:

package main

import(
"fmt"
"strconv"
) 

type mystr string

func (s mystr) print(){
fmt.Println(s)
}

func main() {
   var a mystr ="hello, world\n"
   a.print()
   
   fmt.Println(strconv.ParseInt("78",10, 64))
   
   var x mystr ="452"
   n, _ := strconv.ParseInt(x, 10, 64)
}

Error--
./main.go:21: cannot use x (type mystr) as type string in argument to 
strconv.ParseInt

What made things worse is it is universal, including any type in 
open-source packages from github, to which I cannot add in a new method in 
main.

Is there a way to add methods to an existing type while maintaining all the 
functionalities it already has?

Thanks,
  Zhaoxun

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/ad04a0d8-38e2-4a10-bf35-90698627f170n%40googlegroups.com.

[go-nuts] Is the disconnection synchronization solid in gorilla/websocket?

[Zhaoxun Yan]

Since read and send messages are in different functions/goroutines, when an 
error occurs, how can they be synchronized? 

Suppose I set up a listen goroutine by this function below, with `conn` as 
a global variable of the connection:

func listen(){
  for{
  
_, message, err := conn.ReadMessage()
if err != nil{
break

}else{
//analyze the message
fmt.Println(string(message)) 
}
conn.close()
}

Of course, I have the close function and send function:
func (c *Conn) close(){
if c != nil{
c.Close()
c = nil
}
}

func (c *Conn) send(){
err=c.WriteMessage(websocket.TextMessage, data)
if err != nil{
c.close()
}
}

This brings up the synchronization problem:

An connection error could occur in either occasion, and a new connection 
could be coming in.
Therefore is there an implied synchronization in the connection?

For example, as the err by `c.WriteMessage` occurs, 
the `conn.ReadMessage()` would raise out an error, 
so the goroutine on `listen` would close by itself.

Or, if the `conn.close()` gets called, and `c.close()` gets executed, the `
conn.ReadMessage()` would raise out an error, 
so the goroutine on `listen` would close by itself.

If either of the above is true, I call the disconnection synchronization 
solid.

Otherwise, I am worried that `conn.ReadMessage()` would wait for too long 
(even indefinitely), 
or `conn=nil` could raise panic?

Thanks for your read, and what is your thought?

Zhaoxun

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/60678153-8b44-436c-a5c5-a1cb58bdc2c6n%40googlegroups.com.

Re: [go-nuts] Is there a way to syncronise map reads and writes?

[Zhaoxun Yan]

Hi Brian!
Your code with a map of channels is quite impressive! How can I test if the
code has no races in case I have many go files inside a folder?
`go build -race`?

package main

import "fmt"

type Market struct {
Price float64 `json:"price"`
Timestamp string  `json:"timestamp"`
}

type Command int

const (
SetPrice Command = iota
ReadLatest
)

type CmdSetPrice struct {
Price float64
}

type CmdReadLatest struct {
Response chan RespReadLatest
}

type RespReadLatest struct {
Price float64
}

func MarketManager(c chan interface{}) {
var current Market // hidden internal state
for msg := range c {
switch data := msg.(type) {
case CmdSetPrice:
current.Price = data.Price
//current.Timestamp = ...
case CmdReadLatest:
data.Response <- RespReadLatest{Price: current.Price} // returns copy of
data, not pointer
}
}
}

func RunMarketManager() chan interface{} {
c := make(chan interface{})
go MarketManager(c)
return c
}

func main() {
markets := make(map[string]chan interface{}) // use sync.Map if you
add/remove markets while it's running
markets["Dow"] = RunMarketManager()
markets["Nikkei"] = RunMarketManager()
fmt.Println("Market is open")

// Now all reads and writes are via channels, whenever you like
res := make(chan RespReadLatest)

markets["Dow"] <- CmdSetPrice{Price: 123.0}
markets["Dow"] <- CmdSetPrice{Price: 125.0}
markets["Dow"] <- CmdReadLatest{Response: res}
fmt.Println(<-res)

markets["Dow"] <- CmdSetPrice{Price: 126.0}
markets["Dow"] <- CmdSetPrice{Price: 122.0}
markets["Dow"] <- CmdReadLatest{Response: res}
fmt.Println(<-res)

//Clean shutdown of goroutines
close(markets["Dow"])
close(markets["Nikkei"])
}

On Wed, Feb 23, 2022 at 8:43 PM robert engels  wrote:

> That is not correct.
>
> See https://go.dev/play/p/aZ7LostRuKR
>
> You can run this under the race detector without failure.
>
> Still, the race detector cannot detect all races so better to look at the
> source for sync.Map at 105
> <https://cs.opensource.google/go/go/+/master:src/sync/map.go;drc=2580d0e08d5e9f979b943758d3c49877fb2324cb;l=105>
>
> You can see that it reads the underlying shared maps while not under
> lock/exclusive access.
>
> Starting a go routine creates a “happens before” relationship which allows
> this to work.
>
> On Feb 23, 2022, at 12:10 AM, Jason E. Aten  wrote:
>
> Unfortunately, that's not how Go maps work. You'll still get races unless
> you synchronize even "read-only" maps.
>
> On Tuesday, February 22, 2022 at 11:59:37 PM UTC-6 ren...@ix.netcom.com
> wrote:
>
>> The top level map locks doesn’t matter - it is write once at init and
>> read only after.
>>
>> On Feb 22, 2022, at 11:51 PM, Jason E. Aten  wrote:
>>
>> You have not synchronized access to the top level mapLocks map itself.
>> Thus you will get races. You need to have a mutex that you lock before you
>> access mapLocks; not just the maps that are inside it.
>>
>>
>> I would also recommend being a little object oriented about this design,
>> putting each map inside its own struct, and make a set of accessor methods
>> that Lock() and defer Unlock() on a mutex that protects the maps inside.
>> Read code like this for an idea of what I mean.
>> https://github.com/glycerine/atomicmap/blob/master/amap.go
>>
>> On Tuesday, February 22, 2022 at 9:55:04 PM UTC-6 ren...@ix.netcom.com
>> wrote:
>>
>>> Something else is wrong - because marketMaps is read-only after init… so
>>> unless you have some other code - that is not the map causing the panic.
>>>
>>> My guess is because you are returning the map from ReadMaps (or RWMaps)
>>> that you are using THAT map by multiple threads outside the lock (the map
>>> is a reference not a copy) and that is causing the panic.
>>>
>>> On Feb 22, 2022, at 9:39 PM, Zhaoxun Yan  wrote:
>>>
>>> Hi guys!
>>> I know this is quite challenging, but it makes sense in io-heavy
>>> applications.
>>>
>>> I need to test the responsiveness of different financial future
>>> quotation sources. They return the quotations spontaneously, and my program
>>> respond to each one of the CGO call backs from the quotation library and
>>> save the current prices. Meanwhile, a timer ensures to check the current
>>> quotations of each available sources. And obviously, it raised up a panic
>>> due to synchronization of maps on the call of ReadMaps function.
>>> Unfortunately this came up occasionally and `go build -race` cannot raise
>>> the problem.
>>>
>>> Her

Re: [go-nuts] Is there a way to syncronise map reads and writes?

[Zhaoxun Yan]

Hi Brian, thanks for replying!

So you mean that I have to use 5 Mutexes for Market data of each one source
? That is exactly what I want to do. Every action of RW marketMap[i] must
comes in one line, but RW marketMap[j] concurrently is allowed. And mapLocks
now is fine in golang?

The reason why I do not use channel is that not each one market updates is
needed, but I need to take samples from time to time.

On Wed, Feb 23, 2022 at 6:06 PM Brian Candler  wrote:

> On Wednesday, 23 February 2022 at 08:49:26 UTC yan.z...@gmail.com wrote:
>
>> I have heard that sync.Map allows for reading simultaneously among
>> multiple goroutines.
>>
>
> It does.  But it doesn't synchronize what you do with the values returned
> from the map.  So if you retrieve a *copy* of a value, or replace the
> value, it's fine.  But if you retrieve a pointer, and then either read via
> that pointer or update the object via that pointer, you'll be back to a
> race condition.  You should note that various Go data types include
> pointers internally, especially slices.  However in your case, you're
> proposing that a sync.Map (safe) contains pointers to *Market data (unsafe
> to access concurrently).  So you would still need a mutex to protect each
> Market object, and every read and write would have to apply that mutex.
>
> Personally, I would avoid locks and use channels instead.  It's much
> easier to reason about.  Have one goroutine per market, and have it accept
> 'update' and 'read' messages over a channel (or two different channels).
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/6I_kS6Pya0M/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/1f5beaae-e5d0-4e13-bbe1-28d28ee47e9dn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CADEX6_VXBtWuVCKCZziejjhEf8PWmEjzQhmW4NyAZkXdqJGWJw%40mail.gmail.com.

Re: [go-nuts] Is there a way to syncronise map reads and writes?

[Zhaoxun Yan]

Hi guys!

Thanks for the reply. The scenario is the program booked live financial
quotation feeds by a dynamic library (.so in Linux and .dll in Windows)
from up to 5 different live data sources. It is like testing which CDN
provides faster feeds or faster live streaming videos. It is like booking
several live football matches from 5 different online stations and check
from time to time to compare each match at different data sources.

If I check 10 times that source 1 always has the latest scene of all
matches, I judge it as the best one.

My goal is for the different data sources to write in current messages
simultaneously -  that *source 1* writing on match 1 will queue with the
source 1 writing on match 3, and queue with reading all current match
messages by source 1 if needed, BUT not queue with  *source2* writing on
match 1.

 The conservative way to do it is to declare data1,... data5 and
lock1,...,lock5. But as the code shows, I chose to organize the 5 data
structs and 5 mutex locks into two maps. It saves codes indeed and allows
for scaling up, e.g. 20 data sources. But it caused panic - golang map is
unsafe among goroutines - if a map is being read or written one key-value,
it cannot accept a call to read or write even another key-value otherwise a
panic would occur.

I have heard that sync.Map allows for reading simultaneously among multiple
goroutines. My proposal is to substitute the map of Market data to a map of
*Market data. In that way the map only stores the where the real data is in
memory, which would keep unchanged after initialization and will not cause
trouble even if two cores are changing the real data of source 1 and of
source 2 at the same time.

Is it valid? How to test it without the dynamic libraries?

On Wed, Feb 23, 2022 at 11:52 AM robert engels 
wrote:

> Something else is wrong - because marketMaps is read-only after init… so
> unless you have some other code - that is not the map causing the panic.
>
> My guess is because you are returning the map from ReadMaps (or RWMaps)
> that you are using THAT map by multiple threads outside the lock (the map
> is a reference not a copy) and that is causing the panic.
>
> On Feb 22, 2022, at 9:39 PM, Zhaoxun Yan  wrote:
>
> Hi guys!
> I know this is quite challenging, but it makes sense in io-heavy
> applications.
>
> I need to test the responsiveness of different financial future quotation
> sources. They return the quotations spontaneously, and my program respond
> to each one of the CGO call backs from the quotation library and save the
> current prices. Meanwhile, a timer ensures to check the current quotations
> of each available sources. And obviously, it raised up a panic due to
> synchronization of maps on the call of ReadMaps function. Unfortunately
> this came up occasionally and `go build -race` cannot raise the problem.
>
> Here is a piece of the code, only the read/write part:
>
>
> --
>
> // Please Ignore SyChan to alert the timer that all available sources are
> collected
> var SyChan chan int = make(chan int, 3)
>
> // Make up a map of 5 mutex locks and a map of 5 data structs, indexed by
> 1-5
> var marketMaps = make(map[int]map[string]Market)
> var mapLocks = make(map[int]*sync.Mutex)
>
> // quotation data
> type Market struct {
> Price float64 `json:"price"`
> Timestamp string  `json:"timestamp"`
> }
>
> func init() {
> for i := 1; i <= 5; i++ {
> mapLocks[i] = new(sync.Mutex)
> marketMaps[i] = make(map[string]Market)
> }
> }
>
> //Make sure that for each source, R/W has no race, only took place as
> acquiring the particular Mutex inside the map of Mutex.
> func RWMaps(targetnum int, purpose, future_id string, market Market)
> map[string]Market {
>
> mapLocks[targetnum].Lock()
> defer mapLocks[targetnum].Unlock()
>
> if purpose == "update" {//The original Write part
> marketMaps[targetnum][future_id] = market
>return nil
> } else { //The read part, has been extracted to ReadMaps
> SyChan <- 1
> return marketMaps[targetnum]
> }
> }
>
> //Here is why I use map: not all 5 sources must be available, some may
> have connection failure and would be marked as false in Usable[i] , i being
> its source No.
> func ReadMaps(targetnum, checkTime int) map[string]Market {
> mapLocks[targetnum].Lock()
> defer mapLocks[targetnum].Unlock()
> if Usable[targetnum] {
> fmt.Printf("%d-th time to read source %d \n", checkTime,
> targetnum)
> }
>  

[go-nuts] Is there a way to syncronise map reads and writes?

[Zhaoxun Yan]

Hi guys!
I know this is quite challenging, but it makes sense in io-heavy 
applications.

I need to test the responsiveness of different financial future quotation 
sources. They return the quotations spontaneously, and my program respond 
to each one of the CGO call backs from the quotation library and save the 
current prices. Meanwhile, a timer ensures to check the current quotations 
of each available sources. And obviously, it raised up a panic due to 
synchronization of maps on the call of ReadMaps function. Unfortunately 
this came up occasionally and `go build -race` cannot raise the problem.

Here is a piece of the code, only the read/write part:

--

// Please Ignore SyChan to alert the timer that all available sources are 
collected 
var SyChan chan int = make(chan int, 3)

// Make up a map of 5 mutex locks and a map of 5 data structs, indexed by 
1-5
var marketMaps = make(map[int]map[string]Market)
var mapLocks = make(map[int]*sync.Mutex)

// quotation data
type Market struct {
Price float64 `json:"price"`
Timestamp string  `json:"timestamp"`
}

func init() {
for i := 1; i <= 5; i++ {
mapLocks[i] = new(sync.Mutex)
marketMaps[i] = make(map[string]Market)
}
}

//Make sure that for each source, R/W has no race, only took place as 
acquiring the particular Mutex inside the map of Mutex.
func RWMaps(targetnum int, purpose, future_id string, market Market) 
map[string]Market {

mapLocks[targetnum].Lock()
defer mapLocks[targetnum].Unlock()

if purpose == "update" {//The original Write part
marketMaps[targetnum][future_id] = market
   return nil
} else { //The read part, has been extracted to ReadMaps
SyChan <- 1
return marketMaps[targetnum]
}
}

//Here is why I use map: not all 5 sources must be available, some may have 
connection failure and would be marked as false in Usable[i] , i being its 
source No.
func ReadMaps(targetnum, checkTime int) map[string]Market {
mapLocks[targetnum].Lock()
defer mapLocks[targetnum].Unlock()
if Usable[targetnum] {
fmt.Printf("%d-th time to read source %d \n", checkTime, 
targetnum)
}
SyChan <- 1
return marketMaps[targetnum]
}

--

My problem is :

I still want to keep the map structure, rather than naming mutex1, mutex2, 
mutex3,... , marketmsg1, marketmsg2,  And obviously if the map 
prohibits reading or writing spontaneously, the writing by each usable 
sources is not fair - They must line up as one queue (like using one Mutex 
for all instead) hence the checking of quotation snap on a time-spot is 
defected.  I have also checked the sync.Map and it seems to allow 
spontaneous reading but still prohibits spontaneous writing.

My instinct is to make map on the pointer of structs. On that way, if I 
could use sync.Map, both the read and write function only read the map to 
find the address of data structs, then writing several data structs on the 
same time won't violate the rule on sync.Map.

Is the proposal above legitimate? And anyone could come up a test case to 
mimic the quotation sources? Because with CGO to complicate the program, go 
cannot raise the race problem on -race check, and the successful build can 
panic occasionally.

Thanks in advance! 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9a981200-23f8-4dae-8c20-7acfdcd3f2fcn%40googlegroups.com.

[go-nuts] What goes wrong when embedding goroutines inside a for loop?

[Zhaoxun Yan]

package main
import "fmt"

func main() {
targetIndice := make([]int, 3)
targetIndice[0] = 5
targetIndice[2] = 4

for i,n := range targetIndice{
fmt.Printf("%d : %d \n", i, n)
}

c := make(chan int)
for i, n:= range targetIndice{
go func(){
fmt.Printf("targetIndice[%d]=%d \n", i, n)
c <- 1
}()
}

for i:=0; i<3; i++{
<- c
}

}

-go run main.go-
0 : 5 
1 : 0 
2 : 4 
targetIndice[2]=4 
targetIndice[2]=4 
targetIndice[2]=4 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/03ebf86c-5ec9-486e-81c1-c8126b83deddn%40googlegroups.com.