Re: [go-nuts] tip: data race when running test with coverage

[Shulhan]

On Sun, 2 Oct 2022 20:45:08 +1100
Rob Pike  wrote:

> This is a race in new code, then, so please file an issue at
> https://go.dev/issue/new
> 
> -rob
> 

Done, https://github.com/golang/go/issues/56006

Thank you for walking me through this.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/20221003112532.0fea60ab%40inspiro.localdomain.


pgpCkobBNoM9t.pgp
Description: OpenPGP digital signature

Re: [go-nuts] Re: Go implementation of CLP

[Chris Lu]

Thanks! CLP is great!

I am working on a distributed file system, SeaweedFS,
https://github.com/seaweedfs/seaweedfs
I am interested in a pure-go implementation to store the log files more
efficiently.

Chris

On Sun, Oct 2, 2022 at 6:45 PM david lion  wrote:

> Hi Chris,
>
> I'm one of the CLP developers. We'd be interested in hearing your use case
> and any features you'd like implemented in CLP.
> As for a Go implementation, are you asking about bindings to directly call
> CLP functionality from a Go program or are you interested in a complete
> re-write in Go?
>
> -david
>
> On Friday, 30 September 2022 at 03:58:44 UTC-4 ChrisLu wrote:
>
>> Seems there are no Go implementation for Compressed Log Processor (CLP)
>> yet?
>>
>> CLP is a tool capable of losslessly compressing text logs and searching
>> the compressed logs without decompression.
>>
>> https://github.com/y-scope/clp
>>
>> Chris
>>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/XeDIZfTMlX8/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/0c57905a-1096-4688-b268-93742c48316fn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CACJc9-0gggBXKL0at9k7zKi-KOkXrp5psjQ-W-c-9gdr5QiBAg%40mail.gmail.com.

Re: [go-nuts] Patching Go sources - which files end up in the build?

[Ian Lance Taylor]

On Sun, Oct 2, 2022 at 8:45 AM cpu...@gmail.com  wrote:
>
> For various reasons we need to patch cryptobytes for our build 
> (https://github.com/evcc-io/eebus/issues/1#issuecomment-1146843187).
>
> We've tried to do this in a consistent way for local, Docker and Github CI 
> build by vendoring our modules (go mod vendor) and patching the vendored 
> copy. Test confirms the vendored copy is successfully patched 
> (https://github.com/evcc-io/evcc/blob/master/main.go#L37).
>
> However, and that part is harder to explain, our application- when receiving 
> TLS connections- still seems to act as if the patch is not in place. Since 
> cryptobytes is also part of GOROOT I'm wondering is there is any chance of 
> the original file ending up in our build, too?
>
> What is the role of GOROOT for the final binary and which files (vendored vs. 
> mod cache vs. GOROOT) really end up the build in the end?

When files are vendored into the standard library, such as the
golang.org/x/crypto/cryptobyte package, all references to that package
from the standard library will refer to the vendored copy.  If you
want to change references from the standard library, you will need to
modify the standard library sources.  This of course means that you
must be using a custom Go distribution.  There is currently no way to
tell the standard library to use a different version of a vendored
package.  A mechanism for supporting that is
https://go.dev/issue/30241.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcU4nyLe-N3OJt6ZCZLw1r3qGUjx84e0Q6O3_ERwFBP%2B%3Dg%40mail.gmail.com.

Re: [go-nuts] Easily viewing cross-compiled asm output?

[Ian Lance Taylor]

On Sun, Oct 2, 2022 at 6:45 PM Eli Lindsey  wrote:
>
> I’m on arm64 Darwin and wanted to view the asm output for amd64 Darwin.
>
> Using a simple hello world example, this worked and output arm asm:
>
> $ go tool compile -S hello.go
>
> This worked and built a full x86 binary:
>
> $ GOARCH=amd64 go build hello.go
>
> But this failed:
>
> $ GOARCH=amd64 go tool compile -S hello.go
> hello.go:4:5: could not import fmt (file not found)
>
> To fix it, I downloaded a binary release for amd64 and pointed go tool to its 
> pkg directory. This now works:
>
> $ GOARCH=amd64 go tool compile -I /path/to/go/release/pkg/darwin_amd64 -S 
> hello.go
>
> My question: is there an easier way that I’ve missed? Presumably go build had 
> built fmt _somehow_ since it successfully made an x86 binary. I read the docs 
> for go build and cmd/compile to see if I could get build to output the 
> packagefiles it produces in a way that cmd/compile can point to or some such, 
> but I didn’t see an obvious way.

GOARCH=amd64 go build -gcflags=-S hello.go

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcV4Ndk-VRUuA-KiobXVXHoXb8FUwe5%3DHkZJZ9vBGVnBRQ%40mail.gmail.com.

[go-nuts] Easily viewing cross-compiled asm output?

[Eli Lindsey]

I’m on arm64 Darwin and wanted to view the asm output for amd64 Darwin.

Using a simple hello world example, this worked and output arm asm:

$ go tool compile -S hello.go

This worked and built a full x86 binary:

$ GOARCH=amd64 go build hello.go

But this failed:

$ GOARCH=amd64 go tool compile -S hello.go
hello.go:4:5: could not import fmt (file not found)

To fix it, I downloaded a binary release for amd64 and pointed go tool to its 
pkg directory. This now works:

$ GOARCH=amd64 go tool compile -I /path/to/go/release/pkg/darwin_amd64 -S 
hello.go

My question: is there an easier way that I’ve missed? Presumably go build had 
built fmt _somehow_ since it successfully made an x86 binary. I read the docs 
for go build and cmd/compile to see if I could get build to output the 
packagefiles it produces in a way that cmd/compile can point to or some such, 
but I didn’t see an obvious way.

-eli

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/6B6A9EC0-EEBD-460F-A4D1-A485FBCF80B3%40siliconsprawl.com.

[go-nuts] Re: Go implementation of CLP

[david lion]

Hi Chris,

I'm one of the CLP developers. We'd be interested in hearing your use case 
and any features you'd like implemented in CLP.
As for a Go implementation, are you asking about bindings to directly call 
CLP functionality from a Go program or are you interested in a complete 
re-write in Go?

-david

On Friday, 30 September 2022 at 03:58:44 UTC-4 ChrisLu wrote:

> Seems there are no Go implementation for Compressed Log Processor (CLP) 
> yet? 
>
> CLP is a tool capable of losslessly compressing text logs and searching 
> the compressed logs without decompression.
>
> https://github.com/y-scope/clp
>
> Chris
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0c57905a-1096-4688-b268-93742c48316fn%40googlegroups.com.

[go-nuts] Re: pprof : how to dynamically collect profile data from live API server ?

[Theivaraj S]

how to profiling for post method for rest api in golang 


On Thursday, January 14, 2016 at 7:26:42 PM UTC+5:30 Tamás Gulácsi wrote:

> You can try https://github.com/rakyll/gom which is almost that: a 
> continuous pprof. Maybe some persistence shall be added to gom.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4f9848a0-e742-4bc5-8ec7-9364dd3fb3b7n%40googlegroups.com.

Re: [go-nuts] Struggling w/ use of a waitgroup

[Robert Solomon]

I don't know how to change that.  
I had an issue w/ the main routine ending before some of the worker 
routines were done so I saw incomplete results.
How do I tweak my logic so that I don't need a WaitGroup?  

I forgot about my platform specific code.  Here is the rest of that, that 
is in a file to be compiled on windows.

package main

import (
"fmt"
"os"
"path/filepath"
"strings"
) // Glob is case sensitive.  I want case insensitive.

const estimatedNumberOfFiles = 100

func globCommandLineFiles(patterns []string) []string {
matchingNames := make([]string, 0, estimatedNumberOfFiles)
for _, pattern := range patterns {
matches, err := filepath.Glob(pattern) // Glob returns names of all 
files matching the case-sensitive pattern.
if err != nil {
fmt.Fprintln(os.Stderr, " Error from filepath.Glob is", err)
os.Exit(1)
} else if matches != nil { // At least one match
matchingNames = append(matchingNames, matches...)
}
}
return matchingNames
}

func commandLineFiles(patterns []string) []string {
workingDirname, err := os.Getwd()
if err != nil {
fmt.Fprintln(os.Stderr, "from commandlinefiles:", err)
return nil
}
dirEntries, e := os.ReadDir(workingDirname) // became available as of 
Go 1.16
if e != nil {
return nil
}

matchingNames := make([]string, 0, len(dirEntries))

for _, pattern := range patterns { // outer loop to test against 
multiple patterns.
for _, d := range dirEntries { // inner loop to test each pattern 
against the filenames.
if d.IsDir() {
continue // skip a subdirectory name
}
boolean, er := filepath.Match(pattern, 
strings.ToLower(d.Name()))
if er != nil {
fmt.Fprintln(os.Stderr, er)
continue
}
if boolean {
matchingNames = append(matchingNames, d.Name())
}
}
}
return matchingNames
}


On Sunday, October 2, 2022 at 3:20:22 PM UTC-4 harr...@spu.edu wrote:

> I think Matthew is correct about the immediate source of the deadlock - 
> because the defer is placed too late in the body of grepFile(), the 
> deferred decrement of the waitGroup isn't run on an os.Open() error.
>
> I had the same impression as Jan, I think there is a concern here: the 
> condition for closing grepChan could be, when all files have been sent into 
> the channel. Currently, it's when all work is done. The difference is that 
> one pattern is to increment/decrement waitGroup per unit of work, but 
> another one is to increment/decrement just workers. For the latter, the 
> worker decrements when it infers there is no more work to be done, i.e. it 
> reads that grepChan has closed. I think changing the pattern would avoid 
> the deadlock because, then, there's no reason to interact with the 
> waitGroup per file.
>
> On Sunday, October 2, 2022 at 11:59:26 AM UTC-7 Jan Mercl wrote:
>
>> On Sun, Oct 2, 2022 at 8:52 PM rob  wrote: 
>>
>> > When I do that, I get this error: 
>> > 
>> > panic: sync: negative WaitGroup number 
>>
>> I tried to investigate, but the code does not build. 
>>
>> ./main.go:99:11: undefined: globCommandLineFiles 
>> ./main.go:101:11: undefined: commandLineFiles 
>>
>> Can you please make it a self-contained, minimal program that 
>> reproduces the failure? 
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1208a4ff-64c2-42bb-975d-e83dea933f0an%40googlegroups.com.

Re: [go-nuts] Go routine context

[Ian Lance Taylor]

On Sun, Oct 2, 2022, 4:46 PM Sven Anderson  wrote:

> Ian Lance Taylor  schrieb am Sa. 1. Okt. 2022 um 17:12:
>
>>
>> Another example of goroutine-local storage that we currently support
>> is runtime/pprof.Do, which adds labels to the current goroutine.  This
>> seems OK as the labels are readonly and are inherited by goroutines
>> started with a go statement.  The labels are recorded in profiles.
>
>
> Aren’t they rather write-only? It would be great, if they are
> write-once-read-many, so that you can legally use them in logs and debug
> output at any place.
>

True, write only is a better description.

Ian



I think it would be great to allow storing immutable context.Context for
> such cases.
>
> As it happens, I wrote a small package, that does that „almost“ legally:
> https://pkg.go.dev/github.com/ansiwen/gctx
>
> Cheers
>
> Sven
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcWf%3D4Kc8HmjmWDOiM56opNJFPmKcsNK%2BehWWHOJrtLqRQ%40mail.gmail.com.

Re: [go-nuts] Go routine context

['Dan Kortschak' via golang-nuts]

On Mon, 2022-10-03 at 00:46 +0200, Sven Anderson wrote:
> As it happens, I wrote a small package, that does that „almost“
> legally: 
> https://pkg.go.dev/github.com/ansiwen/gctx

There is also github.com/kortschak/goroutine for getting goroutine IDs,
which can be used as the primitive for constructing your own
personalised foot gun.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/912630d021fb1fcedd90a0467ed19914693ce601.camel%40kortschak.io.

Re: [go-nuts] Go routine context

[Sven Anderson]

Ian Lance Taylor  schrieb am Sa. 1. Okt. 2022 um 17:12:

>
> Another example of goroutine-local storage that we currently support
> is runtime/pprof.Do, which adds labels to the current goroutine.  This
> seems OK as the labels are readonly and are inherited by goroutines
> started with a go statement.  The labels are recorded in profiles.


Aren’t they rather write-only? It would be great, if they are
write-once-read-many, so that you can legally use them in logs and debug
output at any place.

I think it would be great to allow storing immutable context.Context for
such cases.

As it happens, I wrote a small package, that does that „almost“ legally:
https://pkg.go.dev/github.com/ansiwen/gctx

Cheers

Sven

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAFwXxZTHt16ptn1xMGA8a6Qyas4EkE6DAB3gj%2BS7h9FdOeLuZQ%40mail.gmail.com.

Re: [go-nuts] Struggling w/ use of a waitgroup

[rob]

Looks like you're right.

I changed the order of the defer statement and now I'm not getting that 
error.


Interesting that I never saw any file errors.

Thanks

--rob solomon


On 10/2/22 14:46, Matthew Zimmerman wrote:
First reason I notice, if there's an error opening your file, 
wg.Done() is never called.


On Sun, Oct 2, 2022, 1:36 PM Robert Solomon  wrote:

https://go.dev/play/p/gIVVLsiTqod

I'm trying to understand concurrency, so I modified a small
routine I came across quite a while ago.  It's a grep command, but
since I have its source, I am trying to understand its concurrency.
My problem is that when there are more than about 1800 files to be
processed, the go routines all deadlock.
This code works fine as long as I have fewer than about 1800 files
to process.
I don't understand why this happens.

Windows 10.
Tried w/ go1.17, go1.18 and go1.19

--Rob Solomon
-- 
You received this message because you are subscribed to the Google

Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit

https://groups.google.com/d/msgid/golang-nuts/3799e520-9a98-4117-b407-f6aea24995ccn%40googlegroups.com

.



--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/a0e1530f-619f-deaa-585d-66f968b89fa9%40gmail.com.

Re: [go-nuts] Struggling w/ use of a waitgroup

[Andrew Harris]

I think Matthew is correct about the immediate source of the deadlock - 
because the defer is placed too late in the body of grepFile(), the 
deferred decrement of the waitGroup isn't run on an os.Open() error.

I had the same impression as Jan, I think there is a concern here: the 
condition for closing grepChan could be, when all files have been sent into 
the channel. Currently, it's when all work is done. The difference is that 
one pattern is to increment/decrement waitGroup per unit of work, but 
another one is to increment/decrement just workers. For the latter, the 
worker decrements when it infers there is no more work to be done, i.e. it 
reads that grepChan has closed. I think changing the pattern would avoid 
the deadlock because, then, there's no reason to interact with the 
waitGroup per file.

On Sunday, October 2, 2022 at 11:59:26 AM UTC-7 Jan Mercl wrote:

> On Sun, Oct 2, 2022 at 8:52 PM rob  wrote:
>
> > When I do that, I get this error:
> >
> > panic: sync: negative WaitGroup number
>
> I tried to investigate, but the code does not build.
>
> ./main.go:99:11: undefined: globCommandLineFiles
> ./main.go:101:11: undefined: commandLineFiles
>
> Can you please make it a self-contained, minimal program that
> reproduces the failure?
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/efec7a69-0e00-4bc5-a2aa-5c03e3009f05n%40googlegroups.com.

Re: [go-nuts] Struggling w/ use of a waitgroup

[rob]

When I do that, I get this error:

panic: sync: negative WaitGroup number



On 10/2/22 14:39, Jan Mercl wrote:

On Sun, Oct 2, 2022 at 7:36 PM Robert Solomon  wrote:


https://go.dev/play/p/gIVVLsiTqod

I believe wg.Add on line 125 is too late. I think it needs to be moved
before the go statement on line 108. Not tested.


--
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4cf0d245-664e-5571-9617-c1d0d28a580c%40gmail.com.

Re: [go-nuts] Struggling w/ use of a waitgroup

[Matthew Zimmerman]

First reason I notice, if there's an error opening your file, wg.Done() is
never called.

On Sun, Oct 2, 2022, 1:36 PM Robert Solomon  wrote:

> https://go.dev/play/p/gIVVLsiTqod
>
> I'm trying to understand concurrency, so I modified a small routine I came
> across quite a while ago.  It's a grep command, but since I have its
> source, I am trying to understand its concurrency.
> My problem is that when there are more than about 1800 files to be
> processed, the go routines all deadlock.
> This code works fine as long as I have fewer than about 1800 files to
> process.
> I don't understand why this happens.
>
> Windows 10.
> Tried w/ go1.17, go1.18 and go1.19
>
> --Rob Solomon
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/3799e520-9a98-4117-b407-f6aea24995ccn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAD53Lr5MW-pkU1V_a1Pa_1GhxBtikZC%2BXxKq34dbUhxFPMNaTA%40mail.gmail.com.

Re: [go-nuts] Struggling w/ use of a waitgroup

[Jan Mercl]

On Sun, Oct 2, 2022 at 7:36 PM Robert Solomon  wrote:

> https://go.dev/play/p/gIVVLsiTqod

I believe wg.Add on line 125 is too late. I think it needs to be moved
before the go statement on line 108. Not tested.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA40n-WBNtF6mi4kG%2BCqJCvFp-MOf3i7Y4%2BRsSmk9SooWyfXxg%40mail.gmail.com.

[go-nuts] Struggling w/ use of a waitgroup

[Robert Solomon]

https://go.dev/play/p/gIVVLsiTqod

I'm trying to understand concurrency, so I modified a small routine I came 
across quite a while ago.  It's a grep command, but since I have its 
source, I am trying to understand its concurrency.
My problem is that when there are more than about 1800 files to be 
processed, the go routines all deadlock.
This code works fine as long as I have fewer than about 1800 files to 
process.
I don't understand why this happens.

Windows 10.
Tried w/ go1.17, go1.18 and go1.19

--Rob Solomon

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3799e520-9a98-4117-b407-f6aea24995ccn%40googlegroups.com.

[go-nuts] Patching Go sources - which files end up in the build?

[cpu...@gmail.com]

For various reasons we need to patch cryptobytes for our build 
(https://github.com/evcc-io/eebus/issues/1#issuecomment-1146843187). 

We've tried to do this in a consistent way for local, Docker and Github CI 
build by vendoring our modules (go mod vendor) and patching the vendored 
copy. Test confirms the vendored copy is successfully patched 
(https://github.com/evcc-io/evcc/blob/master/main.go#L37).

However, and that part is harder to explain, our application- when 
receiving TLS connections- still seems to act as if the patch is not in 
place. Since cryptobytes is also part of GOROOT I'm wondering is there is 
any chance of the original file ending up in our build, too?

What is the role of GOROOT for the final binary and which files (vendored 
vs. mod cache vs. GOROOT) really end up the build in the end?

Much appreciated,
Andreas (andig)

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8ecc71ca-4d44-4be4-a4ad-401d4687e40cn%40googlegroups.com.

Re: [go-nuts] Go routine context

['Axel Wagner' via golang-nuts]

Can we collectively stay calm and within the bounds of polite discourse?
The overall tone of this thread is pretty aggressive.

I would also like to point out that this particular topic of discussion -
the credentials of the author of the original article - seem fallacious to
me from either side. They can neither be used to elevate arguments - that'd
be an appeal to authority - nor can they be used to dismiss them - that'd
be an ad-hominem.

The original article and the related design documents contain more than
enough material to have a reasoned, substantive discussion, without
litigating the character and credentials of their authors. I find them all
pretty well-written and reasoned TBQH.

---

As far as that quote goes: I haven't read the full interview, but I tend to
agree with the substance of that quote, personally. I myself have noticed
that I rarely use channels and "real" CSP. I've also noticed especially
novices to the language "overusing" channels, creating hard to understand
and often buggy code. I think with discipline and adhering to a couple of
rules can make them safe and lead to easy to understand code - but I find
it often easier to "just slap a mutex on it", if you will.

I don't think it can be argued that Go's concurrency model is particularly
safe on its own. Especially not with a language like Rust existing and
gaining mainstream attention. I think that's fine and I still think
goroutines and channels are a net benefit (it does make it significantly
easier to add concurrency). think Go's overall simplicity still makes it
easier to write understandable, mostly bug-free software, than many other
languages I've tried.

I don't think any of this is an attack on the design of Go. I obviously
love the language, despite these concerns. I just don't advertise its
concurrency features as much to novices as I used to.

On Sun, Oct 2, 2022 at 3:13 PM Jan Mercl <0xj...@gmail.com> wrote:

> On Sun, Oct 2, 2022 at 3:04 PM Robert Engels 
> wrote:
>
> > One other thing, if you don’t think he knows exactly how Go routines are
> implemented you are delusional.
>
> Maybe he should then fix the Wikipedia article I linked before. Good
> luck with that.
>
> PS: I assume you meant "goroutines" instead of "Go routines".
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAA40n-WXjyeW6uHkWvXJyRKxdSmvgbRObkS_%2BxTENSOCRW770A%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAEkBMfE-eExBacz2c1OXtzAPU9NUsNzBBa135yiJFubgvE9bJA%40mail.gmail.com.

Re: [go-nuts] Go routine context

[Jan Mercl]

On Sun, Oct 2, 2022 at 3:00 PM Robert Engels  wrote:

> They are still non-preemptive. Even the way preemption is implemented is 
> cooperative.

Goroutines are preempted in a thread signal handler that invokes the
Go runtime scheduler that in turn saves the required goroutine state
and switches the execution context to a different goroutine, if
available.

Which part of that do you call "cooperative"?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA40n-Vw8YixifiETd2VuAMZyGDfJbr7NeuBtU78BHgPzCxRrQ%40mail.gmail.com.

Re: [go-nuts] Go routine context

[Jan Mercl]

On Sun, Oct 2, 2022 at 3:04 PM Robert Engels  wrote:

> One other thing, if you don’t think he knows exactly how Go routines are 
> implemented you are delusional.

Maybe he should then fix the Wikipedia article I linked before. Good
luck with that.

PS: I assume you meant "goroutines" instead of "Go routines".

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA40n-WXjyeW6uHkWvXJyRKxdSmvgbRObkS_%2BxTENSOCRW770A%40mail.gmail.com.

Re: [go-nuts] Go routine context

[Jan Mercl]

On Sun, Oct 2, 2022 at 2:47 PM Robert Engels  wrote:

> I already pointed that out.Go routines were non preemptive until recently. 
> They are also still non preemptive when calling a system call without direct 
> support, or CGo - it spins up another platform thread.

I'm talking about a principal difference that your "god" seems to not
understand. You are pointing out a particular past implementation
deficiency that has nothing to do with the principle.



-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA40n-UnoN4tkSpz%3Dc-1igE3mqpQUbss3HPm7v0MVkt4LAYrWw%40mail.gmail.com.

Re: [go-nuts] Go routine context

[Robert Engels]

Thank you. Great interview. As expected, everything he said was true - 
especially when you read the full segment - one in about 50 presented. No 
wonder the original poster did not want to link to the full article. I’ll 
reiterate - “CS god”. 

I did a quick review of k8s - and it is almost equal usage of sync.Mutex and 
channels throughout the code. So in practicality - Go “do not share data” 
mantra is not easy to achieve in practice. 

Go channels are simply two versions of a queue - a handoff or a buffered 
blocking queue. Java has many others. 

Similarly, as Brian points out - Go has non-reentrant sync.Mutex. Java has 
multiple synchronization primitives to support a wider variety of use cases. 

I encourage everyone to read the article - it’s an excellent interview (even if 
you disagree on the small reference made to Gos concurrency model). 

> On Oct 2, 2022, at 6:21 AM, 'Dan Kortschak' via golang-nuts 
>  wrote:
> 
> On Sun, 2022-10-02 at 06:15 -0500, Robert Engels wrote:
>> Again, I would like a link to the source of statement to evaluate it
>> in context. 
> 
> https://manningbooks.medium.com/interview-with-brian-goetz-7d6c47d05d63
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/3d6ff1822f402e745cdc51776fcf6bfd0a112d75.camel%40kortschak.io.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1911A3CC-0C01-494F-94BA-CF00FE98AE1F%40ix.netcom.com.

Re: [go-nuts] Go routine context

[Jan Mercl]

On Sun, Oct 2, 2022 at 1:16 PM Robert Engels  wrote:

> By many definitions Go routines and virtual threads are technically 
> coroutines - versus a platform/OS thread.

Show one please.

Coroutines are normally a subject of non-preemptive multitasking,
threads are normally just the opposite. [0]


  [0]: https://en.wikipedia.org/wiki/Coroutine

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA40n-XF7mF0--NbisAQoUHXEj-F1O_VUgxZ_nf1dJmipZTz3g%40mail.gmail.com.

Re: [go-nuts] Go routine context

['Dan Kortschak' via golang-nuts]

On Sun, 2022-10-02 at 06:15 -0500, Robert Engels wrote:
> Again, I would like a link to the source of statement to evaluate it
> in context. 

https://manningbooks.medium.com/interview-with-brian-goetz-7d6c47d05d63

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3d6ff1822f402e745cdc51776fcf6bfd0a112d75.camel%40kortschak.io.

Re: [go-nuts] Go routine context

[Robert Engels]

Also, until fairly recently Go routines were non preemptive- making them even 
closer to a coroutine. I have no idea when this statement was made to judge 
that aspect as well. 

> On Oct 2, 2022, at 6:15 AM, Robert Engels  wrote:
> 
> 
> By many definitions Go routines and virtual threads are technically 
> coroutines - versus a platform/OS thread. 
> 
> Again, I would like a link to the source of statement to evaluate it in 
> context. 
> 
>>> On Oct 2, 2022, at 5:32 AM, w54n  wrote:
>>> 
>> Not an entirely true statement, and a fairly misunderstanding of the 
>> subject.
>> 
>>> On Saturday, October 1, 2022 at 11:13:19 PM UTC+2 al...@pbrane.org wrote:
>>> Robert Engels  once said: 
>>> > I think you’ll find the article interesting. It is certainly written 
>>> > by a CS “god” that knows what he’s talking about. 
>>> 
>>> This is the same "god" that said: 
>>> 
>>> "Everyone thinks that the concurrency model is Go’s secret 
>>> weapon, but I think their concurrency model is actually quite 
>>> error prone. That is, you have coroutines with a very basic 
>>> message-passing mechanism (channels). But in almost all cases, 
>>> the things on one side or the other of the channel are going to 
>>> have some shared mutable state guarded with locks." 
>>> 
>>> Just another heliocentric distortion. Ra blah blah. 
>>> 
>>> Cheers, 
>>> Anthony 
>> 
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/fbd63d49-df17-4150-9d48-6cad39a06381n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/4B6A8E88-9333-4C8A-B403-BE98931E5D48%40ix.netcom.com.

Re: [go-nuts] Go routine context

[Robert Engels]

By many definitions Go routines and virtual threads are technically coroutines 
- versus a platform/OS thread. 

Again, I would like a link to the source of statement to evaluate it in 
context. 

> On Oct 2, 2022, at 5:32 AM, w54n  wrote:
> 
> Not an entirely true statement, and a fairly misunderstanding of the subject.
> 
>> On Saturday, October 1, 2022 at 11:13:19 PM UTC+2 al...@pbrane.org wrote:
>> Robert Engels  once said: 
>> > I think you’ll find the article interesting. It is certainly written 
>> > by a CS “god” that knows what he’s talking about. 
>> 
>> This is the same "god" that said: 
>> 
>> "Everyone thinks that the concurrency model is Go’s secret 
>> weapon, but I think their concurrency model is actually quite 
>> error prone. That is, you have coroutines with a very basic 
>> message-passing mechanism (channels). But in almost all cases, 
>> the things on one side or the other of the channel are going to 
>> have some shared mutable state guarded with locks." 
>> 
>> Just another heliocentric distortion. Ra blah blah. 
>> 
>> Cheers, 
>> Anthony 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/fbd63d49-df17-4150-9d48-6cad39a06381n%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/075DE517-9419-48C8-83DC-547A05F0056F%40ix.netcom.com.

Re: [go-nuts] Go routine context

[w54n]

Not an entirely true statement, and a fairly misunderstanding of the 
subject.

On Saturday, October 1, 2022 at 11:13:19 PM UTC+2 al...@pbrane.org wrote:

> Robert Engels  once said: 
> > I think you’ll find the article interesting. It is certainly written 
> > by a CS “god” that knows what he’s talking about. 
>
> This is the same "god" that said: 
>
> "Everyone thinks that the concurrency model is Go’s secret 
> weapon, but I think their concurrency model is actually quite 
> error prone. That is, you have coroutines with a very basic 
> message-passing mechanism (channels). But in almost all cases, 
> the things on one side or the other of the channel are going to 
> have some shared mutable state guarded with locks." 
>
> Just another heliocentric distortion. Ra blah blah. 
>
> Cheers, 
> Anthony 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/fbd63d49-df17-4150-9d48-6cad39a06381n%40googlegroups.com.

Re: [go-nuts] Go routine context

[Jan Mercl]

On Sun, Oct 2, 2022 at 1:37 AM Robert Engels  wrote:

> I don’t see any arguments refuting the claim? Can you include a link so the 
> analysis can be read in context?

The word "coroutines" does not ring a bell?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA40n-VVTEdU0KczveQgqQy%3DaBtoBTVP%2B6yq7gbRR53Vh1HYmA%40mail.gmail.com.

Re: [go-nuts] tip: data race when running test with coverage

[Rob Pike]

This is a race in new code, then, so please file an issue at
https://go.dev/issue/new

-rob

On Sun, Oct 2, 2022 at 7:40 PM Shulhan  wrote:
>
> On Sun, 2 Oct 2022 19:22:21 +1100
> Rob Pike  wrote:
>
> > Apologies, the cover tool calls it -mode but the go command calls it
> > -covermode.
> >
> > go test -covermode=atomic
> >
> > -rob
> >
> >
>
> Got it, thanks, but it still fail with the data race.
>
> 
> (ins) 1 $ GOEXPERIMENT=coverageredesign CGO_ENABLED=1 go test 
> -covermode=atomic -race -coverprofile=cover.out ./lib/dns
> dns.Server: listening for DNS over UDP at 127.0.0.1:5300
> dns.Server: listening for DNS over TCP at 127.0.0.1:5300
> dns.Server: listening for DNS over TLS at 127.0.0.1:18053
> dns.Server: listening for DNS over HTTPS at 127.0.0.1:8443
> dns: invalid IP address "localhost"
> dns: invalid name server URI "://127.0.0.1"
> dns: invalid IP address "localhost:53"
> dns: invalid IP address "localhost:53"
> PASS
> ==
> WARNING: DATA RACE
> Read at 0x01e5f04c by main goroutine:
>   internal/coverage/encodecounter.(*CoverageDataWriter).writeCounters.func2()
>   /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:261 +0x11c
>   runtime/coverage.(*emitState).VisitFuncs()
>   /home/ms/opt/go/src/runtime/coverage/emit.go:539 +0x6bc
>   internal/coverage/encodecounter.(*CoverageDataWriter).writeCounters()
>   /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:268 +0x16f
> ...
> Found 12 data race(s)
> FAILgithub.com/shuLhan/share/lib/dns0.754s
> FAIL
> 
>
> BTW, from documentation, the -race flag automatically set the
> -covermode=atomic.
>
>-covermode set,count,atomic
> ...
> being tested. The default is "set" unless -race is enabled,
> in which case it is "atomic".
> ...
>
> So I think its redundant to set -race and -covermode=atomic at the same
> time.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOXNBZSaMV83i22F%3Dhu51NEZCZgSVABgcvuUMW3CSZNN92d_tA%40mail.gmail.com.

Re: [go-nuts] tip: data race when running test with coverage

[Shulhan]

On Sun, 2 Oct 2022 19:22:21 +1100
Rob Pike  wrote:

> Apologies, the cover tool calls it -mode but the go command calls it
> -covermode.
> 
> go test -covermode=atomic
> 
> -rob
> 
> 

Got it, thanks, but it still fail with the data race.


(ins) 1 $ GOEXPERIMENT=coverageredesign CGO_ENABLED=1 go test -covermode=atomic 
-race -coverprofile=cover.out ./lib/dns
dns.Server: listening for DNS over UDP at 127.0.0.1:5300
dns.Server: listening for DNS over TCP at 127.0.0.1:5300
dns.Server: listening for DNS over TLS at 127.0.0.1:18053
dns.Server: listening for DNS over HTTPS at 127.0.0.1:8443
dns: invalid IP address "localhost"
dns: invalid name server URI "://127.0.0.1"
dns: invalid IP address "localhost:53"
dns: invalid IP address "localhost:53"
PASS
==
WARNING: DATA RACE
Read at 0x01e5f04c by main goroutine:
  internal/coverage/encodecounter.(*CoverageDataWriter).writeCounters.func2()
  /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:261 +0x11c
  runtime/coverage.(*emitState).VisitFuncs()
  /home/ms/opt/go/src/runtime/coverage/emit.go:539 +0x6bc
  internal/coverage/encodecounter.(*CoverageDataWriter).writeCounters()
  /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:268 +0x16f
...
Found 12 data race(s)
FAILgithub.com/shuLhan/share/lib/dns0.754s
FAIL


BTW, from documentation, the -race flag automatically set the
-covermode=atomic.

   -covermode set,count,atomic
...
being tested. The default is "set" unless -race is enabled,
in which case it is "atomic".
...

So I think its redundant to set -race and -covermode=atomic at the same
time.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/20221002154034.3ea7e088%40inspiro.localdomain.


pgpnCXCcVut4T.pgp
Description: OpenPGP digital signature

Re: [go-nuts] tip: data race when running test with coverage

[Rob Pike]

Also, it is documented:

% go help testflag | grep mode

-covermode set,count,atomic

Set the mode for coverage analysis for the package[s]

When 'go test' runs in package list mode, 'go test' caches successful

%

On Sun, Oct 2, 2022 at 7:22 PM Rob Pike  wrote:

> Apologies, the cover tool calls it -mode but the go command calls it
> -covermode.
>
> go test -covermode=atomic
>
> -rob
>
>
> On Sun, Oct 2, 2022 at 3:48 PM Shulhan  wrote:
>
>> On Sun, 2 Oct 2022 10:41:17 +1100
>> Rob Pike  wrote:
>>
>> > When running coverage in a concurrent program, use the -mode=atomic
>> > flag to avoid data races in the counters. This unavoidably has a
>> > significant performance hit, but it should resolve this race.
>> >
>> > -rob
>> >
>>
>> This flag print "no test files" when running go test,
>>
>> $ GOEXPERIMENT=coverageredesign CGO_ENABLED=1 go test -mode=atomic
>> -race -coverprofile=cover.out ./lib/dns
>> ?   github.com/shuLhan/share[no test files]
>>
>> Not sure I am using the flag correctly or not since I cannot find it in
>> the current documentation [1].
>>
>> [1] https://pkg.go.dev/cmd/go
>>
>> >
>> > On Sun, Oct 2, 2022 at 5:10 AM Shulhan  wrote:
>> >
>> > > Hi gophers,
>> > >
>> > > The latest Go tip always fail with data race when running with -race
>> > > and -coverprofile options.
>> > >
>> > > Here is an example of data race output,
>> > >
>> > > 
>> > > $ CGO_ENABLED=1 go test -failfast -race -count=1
>> > > -coverprofile=cover.out ./...
>> > > ...
>> > > ==
>> > > WARNING: DATA RACE
>> > > Read at 0x01e5f04c by main goroutine:
>> > >
>> > >
>> internal/coverage/encodecounter.(*CoverageDataWriter).writeCounters.func2()
>> > >
>>  /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:261
>> > > +0x11c
>> > >   runtime/coverage.(*emitState).VisitFuncs()
>> > >   /home/ms/opt/go/src/runtime/coverage/emit.go:539 +0x6bc
>> > >
>>  internal/coverage/encodecounter.(*CoverageDataWriter).writeCounters()
>> > >
>>  /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:268
>> > > +0x16f
>> > >
>>  internal/coverage/encodecounter.(*CoverageDataWriter).AppendSegment()
>> > >
>>  /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:175
>> > > +0x8ea
>> > >   internal/coverage/encodecounter.(*CoverageDataWriter).Write()
>> > >   /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:71
>> > > +0x97
>> > >   runtime/coverage.(*emitState).emitCounterDataFile()
>> > >   /home/ms/opt/go/src/runtime/coverage/emit.go:573 +0x91
>> > >   runtime/coverage.emitCounterDataToDirectory()
>> > >   /home/ms/opt/go/src/runtime/coverage/emit.go:322 +0x310
>> > >   runtime/coverage.processCoverTestDir()
>> > >   /home/ms/opt/go/src/runtime/coverage/testsupport.go:39 +0x1c4
>> > >   main.coverTearDown()
>> > >   _testmain.go:179 +0x159
>> > >   testing.coverReport2()
>> > >   /home/ms/opt/go/src/testing/newcover.go:37 +0xcb
>> > >   testing.coverReport()
>> > >   /home/ms/opt/go/src/testing/cover.go:83 +0xc74
>> > >   testing.(*M).writeProfiles()
>> > >   /home/ms/opt/go/src/testing/testing.go:2053 +0xc6f
>> > >   testing.(*M).after.func1()
>> > >   /home/ms/opt/go/src/testing/testing.go:1987 +0x30
>> > >   sync.(*Once).doSlow()
>> > >   /home/ms/opt/go/src/sync/once.go:74 +0x101
>> > >   sync.(*Once).Do()
>> > >   /home/ms/opt/go/src/sync/once.go:65 +0x46
>> > >   testing.(*M).after()
>> > >   /home/ms/opt/go/src/testing/testing.go:1986 +0x55
>> > >   testing.(*M).Run.func4()
>> > >   /home/ms/opt/go/src/testing/testing.go:1761 +0x39
>> > >   runtime.deferreturn()
>> > >   /home/ms/opt/go/src/runtime/panic.go:476 +0x32
>> > >   testing.(*M).Run()
>> > >   /home/ms/opt/go/src/testing/testing.go:1771 +0xbb3
>> > >   github.com/shuLhan/share/lib/dns.TestMain()
>> > >   /home/ms/go/src/github.com/shuLhan/share/lib/dns/dns_test.go:63
>> > > +0x5db
>> > >   main.main()
>> > >   _testmain.go:192 +0x33d
>> > >
>> > > Previous write at 0x01e5f04c by goroutine 9:
>> > >   sync/atomic.AddInt32()
>> > >   /home/ms/opt/go/src/runtime/race_amd64.s:281 +0xb
>> > >   sync/atomic.AddUint32()
>> > >   :1 +0x1a
>> > >   github.com/shuLhan/share/lib/dns.(*Server).processRequest()
>> > >   /home/ms/go/src/github.com/shuLhan/share/lib/dns/server.go:593
>> > > +0xc67
>> > >   github.com/shuLhan/share/lib/dns.(*Server).ListenAndServe.func1()
>> > >   /home/ms/go/src/github.com/shuLhan/share/lib/dns/server.go:187
>> > > +0x39
>> > >
>> > > Goroutine 9 (running) created at:
>> > >   github.com/shuLhan/share/lib/dns.(*Server).ListenAndServe()
>> > >   /home/ms/go/src/github.com/shuLhan/share/lib/dns/server.go:187
>> > > +0xe6 github.com/shuLhan/share/lib/dns.TestMain.func1()
>> > >   /home/ms/go/src/github.com/shuLhan/share/lib/dns/dns_test.go:54
>> > > +0x44
>> > > ==
>> > > ...
>> > > 
>> > >
>> > > There are many lines like that with the same pattern.
>> > >

Re: [go-nuts] tip: data race when running test with coverage

[Rob Pike]

Apologies, the cover tool calls it -mode but the go command calls it
-covermode.

go test -covermode=atomic

-rob


On Sun, Oct 2, 2022 at 3:48 PM Shulhan  wrote:

> On Sun, 2 Oct 2022 10:41:17 +1100
> Rob Pike  wrote:
>
> > When running coverage in a concurrent program, use the -mode=atomic
> > flag to avoid data races in the counters. This unavoidably has a
> > significant performance hit, but it should resolve this race.
> >
> > -rob
> >
>
> This flag print "no test files" when running go test,
>
> $ GOEXPERIMENT=coverageredesign CGO_ENABLED=1 go test -mode=atomic
> -race -coverprofile=cover.out ./lib/dns
> ?   github.com/shuLhan/share[no test files]
>
> Not sure I am using the flag correctly or not since I cannot find it in
> the current documentation [1].
>
> [1] https://pkg.go.dev/cmd/go
>
> >
> > On Sun, Oct 2, 2022 at 5:10 AM Shulhan  wrote:
> >
> > > Hi gophers,
> > >
> > > The latest Go tip always fail with data race when running with -race
> > > and -coverprofile options.
> > >
> > > Here is an example of data race output,
> > >
> > > 
> > > $ CGO_ENABLED=1 go test -failfast -race -count=1
> > > -coverprofile=cover.out ./...
> > > ...
> > > ==
> > > WARNING: DATA RACE
> > > Read at 0x01e5f04c by main goroutine:
> > >
> > >
> internal/coverage/encodecounter.(*CoverageDataWriter).writeCounters.func2()
> > >   /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:261
> > > +0x11c
> > >   runtime/coverage.(*emitState).VisitFuncs()
> > >   /home/ms/opt/go/src/runtime/coverage/emit.go:539 +0x6bc
> > >   internal/coverage/encodecounter.(*CoverageDataWriter).writeCounters()
> > >   /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:268
> > > +0x16f
> > >   internal/coverage/encodecounter.(*CoverageDataWriter).AppendSegment()
> > >   /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:175
> > > +0x8ea
> > >   internal/coverage/encodecounter.(*CoverageDataWriter).Write()
> > >   /home/ms/opt/go/src/internal/coverage/encodecounter/encode.go:71
> > > +0x97
> > >   runtime/coverage.(*emitState).emitCounterDataFile()
> > >   /home/ms/opt/go/src/runtime/coverage/emit.go:573 +0x91
> > >   runtime/coverage.emitCounterDataToDirectory()
> > >   /home/ms/opt/go/src/runtime/coverage/emit.go:322 +0x310
> > >   runtime/coverage.processCoverTestDir()
> > >   /home/ms/opt/go/src/runtime/coverage/testsupport.go:39 +0x1c4
> > >   main.coverTearDown()
> > >   _testmain.go:179 +0x159
> > >   testing.coverReport2()
> > >   /home/ms/opt/go/src/testing/newcover.go:37 +0xcb
> > >   testing.coverReport()
> > >   /home/ms/opt/go/src/testing/cover.go:83 +0xc74
> > >   testing.(*M).writeProfiles()
> > >   /home/ms/opt/go/src/testing/testing.go:2053 +0xc6f
> > >   testing.(*M).after.func1()
> > >   /home/ms/opt/go/src/testing/testing.go:1987 +0x30
> > >   sync.(*Once).doSlow()
> > >   /home/ms/opt/go/src/sync/once.go:74 +0x101
> > >   sync.(*Once).Do()
> > >   /home/ms/opt/go/src/sync/once.go:65 +0x46
> > >   testing.(*M).after()
> > >   /home/ms/opt/go/src/testing/testing.go:1986 +0x55
> > >   testing.(*M).Run.func4()
> > >   /home/ms/opt/go/src/testing/testing.go:1761 +0x39
> > >   runtime.deferreturn()
> > >   /home/ms/opt/go/src/runtime/panic.go:476 +0x32
> > >   testing.(*M).Run()
> > >   /home/ms/opt/go/src/testing/testing.go:1771 +0xbb3
> > >   github.com/shuLhan/share/lib/dns.TestMain()
> > >   /home/ms/go/src/github.com/shuLhan/share/lib/dns/dns_test.go:63
> > > +0x5db
> > >   main.main()
> > >   _testmain.go:192 +0x33d
> > >
> > > Previous write at 0x01e5f04c by goroutine 9:
> > >   sync/atomic.AddInt32()
> > >   /home/ms/opt/go/src/runtime/race_amd64.s:281 +0xb
> > >   sync/atomic.AddUint32()
> > >   :1 +0x1a
> > >   github.com/shuLhan/share/lib/dns.(*Server).processRequest()
> > >   /home/ms/go/src/github.com/shuLhan/share/lib/dns/server.go:593
> > > +0xc67
> > >   github.com/shuLhan/share/lib/dns.(*Server).ListenAndServe.func1()
> > >   /home/ms/go/src/github.com/shuLhan/share/lib/dns/server.go:187
> > > +0x39
> > >
> > > Goroutine 9 (running) created at:
> > >   github.com/shuLhan/share/lib/dns.(*Server).ListenAndServe()
> > >   /home/ms/go/src/github.com/shuLhan/share/lib/dns/server.go:187
> > > +0xe6 github.com/shuLhan/share/lib/dns.TestMain.func1()
> > >   /home/ms/go/src/github.com/shuLhan/share/lib/dns/dns_test.go:54
> > > +0x44
> > > ==
> > > ...
> > > 
> > >
> > > There are many lines like that with the same pattern.
> > >
> > > In the above snippet, the lib/dns/dns_test.go:63 point this code
> > > [1],
> > >
> > >   os.Exit(m.Run())
> > >
> > > So it does not make sense if the data race is in my code.
> > >
> > > A quick bisect point to this commit [2].
> > >
>
> Just want to be clear, I run the test using
> GOEXPERIMENT=coverageredesign flag using Go tip commit 53773a5d08
>
> $ go version
> go