Re: [go-nuts] sync.Mutex encounter large performance drop when goroutine contention more than 3400

[Robert Engels]

I don't think you've posted code for the atomic version...Each Go routine has its own stack. So when you cycle through many Go routines you will be destroying the cache as each touches N memory addresses (that are obviously not shared).That's my guess anyway - the performance profile certainly looks like a cache issue to me. Once the cache is exhausted - the kernel based scheduler is more efficient - so it does suggest to me that there are some optimizations that can be done in the Go scheduler.I will look at a few things this evening.-Original Message-
From: changkun 
Sent: Aug 21, 2019 4:51 PM
To: golang-nuts 
Subject: Re: [go-nuts] sync.Mutex encounter large performance drop when goroutine contention more than 3400

"less than N Go routines it fits in the L1 CPU cache," I am guessing that you are thinking of local queues on each M, the scheduler's local queue size is strict to 256 goroutines. However, in our case, all blocking goroutines don't go the run queue but blocked and stored on semtable, which is a forest and each tree is an unlimited balanced tree. When a lock is released, only a single goroutine will be detached and put into the local queue (so scheduler only schedules runq with a single goroutine without content to globalq). How could an L1/L2 problem appear here? Do you think this is still some kind of "limited L1 cache to store large mount of goroutines" ?What interests me is a newly created issue, I am not sure if this question is relevant to https://github.com/golang/go/issues/33747The issue talked about small contention on large Ps, but a full scale of my benchmark is shown as follows:On Tuesday, August 20, 2019 at 6:10:32 PM UTC+2, Robert Engels wrote:I am assuming that there is an internal Go structure/process that when there is less than N Go routines it fits in the L1 CPU cache, and beyond a certain point it spills to the L2 or higher - thus the nearly order of magnitude performance decrease, yet consistent times within a range.Since the worker code is so trivial, you are seeing this. Most worker code is not as trivial so the overhead of the locking/scheduler constructs have far less effect (or the worker is causing L1 evictions anyway - so you never see the optimum performance possible of the scheduler).-Original Message-
From: changkun 
Sent: Aug 20, 2019 3:33 AM
To: golang-nuts 
Subject: Re: [go-nuts] sync.Mutex encounter large performance drop when goroutine contention more than 3400

Hi Robert,Thanks for your explanation. But how could I "logged the number of operations done per Go routine", which particular debug settings you referring to?It is reasonable that sync.Mutex rely on runtime scheduler but channels do not. However, it is unclear why a significant performance drop appears. Is it possible to determine when the performance will appear?Best,ChangkunOn Monday, August 19, 2019 at 10:27:19 PM UTC+2, Robert Engels wrote:I think you'll find the reason that the Mutex uses the Go scheduler. The chan is controlled by a 'mutex' which eventually defers to the OS futex - and the OS futex is probably more efficient at scheduling in the face of large contention - although you would think it should be the other way around.I am guessing that if you logged the number of operations done per Go routine, you will see that the Mutex version is very fair, and the chan/futex version is unfair - meaning many are starved.-Original Message-
From: changkun 
Sent: Aug 19, 2019 12:50 PM
To: golang-nuts 
Subject: [go-nuts] sync.Mutex encounter large performance drop when goroutine contention more than 3400

I am comparing the performance regarding sync.Mutex and Go channels. Here is my benchmark: https://play.golang.org/p/zLjVtsSx9gdThe performance comparison visualization is as follows:What are the reasons that 1. sync.Mutex encounter a large performance drop when the number of goroutines goes higher than roughly 3400?2. Go channels are pretty stable but slower than sync.Mutex before?Raw bench data by benchstat (go test -bench=. -count=5):MutexWrite/goroutines-2400-8  48.6ns ± 1%MutexWrite/goroutines-2480-8  49.1ns ± 0%MutexWrite/goroutines-2560-8  49.7ns ± 1%MutexWrite/goroutines-2640-8  50.5ns ± 3%MutexWrite/goroutines-2720-8  50.9ns ± 2%MutexWrite/goroutines-2800-8  51.8ns ± 3%MutexWrite/goroutines-2880-8  52.5ns ± 2%MutexWrite/goroutines-2960-8  54.1ns ± 4%MutexWrite/goroutines-3040-8  54.5ns ± 2%MutexWrite/goroutines-3120-8  56.1ns ± 3%MutexWrite/goroutines-3200-8  63.2ns ± 5%MutexWrite/goroutines-3280-8  77.5ns ± 6%MutexWrite/goroutines-3360-8   141ns ± 6%MutexWrite/goroutines-3440-8   239ns ± 8%MutexWrite/goroutines-3520-8   248ns ± 3%MutexWrite/goroutines-3600-8   254ns ± 2%MutexWrite/goroutines-3680-8   256ns ± 1%MutexWrite/goroutines-3760-8   261ns ± 2%MutexWrite/goroutines-3840-8   266ns ± 3%MutexWrite/goroutines-3920-8   276ns ± 3%MutexWrite/goroutines-4000-8   278ns ± 3%MutexWrite/goroutines-4080-8   286ns ± 5%MutexWrite/goroutines-4160-8   293ns ± 

Re: [go-nuts] sync.Mutex encounter large performance drop when goroutine contention more than 3400

[changkun]

"less than N Go routines it fits in the L1 CPU cache," I am guessing that 
you are thinking of local queues on each M, the scheduler's local queue 
size is strict to 256 goroutines. However, in our case, all blocking 
goroutines don't go the run queue but blocked and stored on semtable, which 
is a forest and each tree is an unlimited balanced tree. When a lock is 
released, only a single goroutine will be detached and put into the local 
queue (so scheduler only schedules runq with a single goroutine without 
content to globalq). 
How could an L1/L2 problem appear here? Do you think this is still some 
kind of "limited L1 cache to store large mount of goroutines" ?

What interests me is a newly created issue, I am not sure if this question 
is relevant to https://github.com/golang/go/issues/33747
The issue talked about small contention on large Ps, but a full scale of my 
benchmark is shown as follows:

[image: performance_ channel v.s. sync.Mutex v.s. atomic.png]



On Tuesday, August 20, 2019 at 6:10:32 PM UTC+2, Robert Engels wrote:
>
> I am assuming that there is an internal Go structure/process that when 
> there is less than N Go routines it fits in the L1 CPU cache, and beyond a 
> certain point it spills to the L2 or higher - thus the nearly order of 
> magnitude performance decrease, yet consistent times within a range.
>
> Since the worker code is so trivial, you are seeing this. Most worker code 
> is not as trivial so the overhead of the locking/scheduler constructs have 
> far less effect (or the worker is causing L1 evictions anyway - so you 
> never see the optimum performance possible of the scheduler).
>
> -Original Message- 
> From: changkun 
> Sent: Aug 20, 2019 3:33 AM 
> To: golang-nuts 
> Subject: Re: [go-nuts] sync.Mutex encounter large performance drop when 
> goroutine contention more than 3400 
>
> Hi Robert,
>
> Thanks for your explanation. But how could I "logged the number of 
> operations done per Go routine", which particular debug settings you 
> referring to?
> It is reasonable that sync.Mutex rely on runtime scheduler but channels do 
> not. However, it is unclear why a significant performance drop appears. Is 
> it possible to determine when the performance will appear?
>
> Best,
> Changkun
>
> On Monday, August 19, 2019 at 10:27:19 PM UTC+2, Robert Engels wrote:
>>
>> I think you'll find the reason that the Mutex uses the Go scheduler. The 
>> chan is controlled by a 'mutex' which eventually defers to the OS futex - 
>> and the OS futex is probably more efficient at scheduling in the face of 
>> large contention - although you would think it should be the other way 
>> around.
>>
>> I am guessing that if you logged the number of operations done per Go 
>> routine, you will see that the Mutex version is very fair, and the 
>> chan/futex version is unfair - meaning many are starved.
>>
>> -Original Message- 
>> From: changkun 
>> Sent: Aug 19, 2019 12:50 PM 
>> To: golang-nuts 
>> Subject: [go-nuts] sync.Mutex encounter large performance drop when 
>> goroutine contention more than 3400 
>>
>> I am comparing the performance regarding sync.Mutex and Go channels. Here 
>> is my benchmark: https://play.golang.org/p/zLjVtsSx9gd
>>
>> The performance comparison visualization is as follows:
>>
>> [image: sync.Mutex performance (1).png]
>> What are the reasons that 
>>
>> 1. sync.Mutex encounter a large performance drop when the number of 
>> goroutines goes higher than roughly 3400?
>> 2. Go channels are pretty stable but slower than sync.Mutex before?
>>
>>
>>
>> Raw bench data by benchstat (go test -bench=. -count=5):
>>
>> MutexWrite/goroutines-2400-8  48.6ns ± 1%
>> MutexWrite/goroutines-2480-8  49.1ns ± 0%
>> MutexWrite/goroutines-2560-8  49.7ns ± 1%
>> MutexWrite/goroutines-2640-8  50.5ns ± 3%
>> MutexWrite/goroutines-2720-8  50.9ns ± 2%
>> MutexWrite/goroutines-2800-8  51.8ns ± 3%
>> MutexWrite/goroutines-2880-8  52.5ns ± 2%
>> MutexWrite/goroutines-2960-8  54.1ns ± 4%
>> MutexWrite/goroutines-3040-8  54.5ns ± 2%
>> MutexWrite/goroutines-3120-8  56.1ns ± 3%
>> MutexWrite/goroutines-3200-8  63.2ns ± 5%
>> MutexWrite/goroutines-3280-8  77.5ns ± 6%
>> MutexWrite/goroutines-3360-8   141ns ± 6%
>> MutexWrite/goroutines-3440-8   239ns ± 8%
>> MutexWrite/goroutines-3520-8   248ns ± 3%
>> MutexWrite/goroutines-3600-8   254ns ± 2%
>> MutexWrite/goroutines-3680-8   256ns ± 1%
>> MutexWrite/goroutines-3760-8   261ns ± 2%
>> MutexWrite/goroutines-3840-8   266ns ± 3%
>> MutexWrite/goroutines-3920-8   276ns ± 3%
>> MutexWrite/goroutines-4000-8   278ns ± 3%
>> MutexWrite/goroutines-4080-8   286ns ± 5%
>> MutexWrite/goroutines-4160-8   293ns ± 4%
>> MutexWrite/goroutines-4240-8   295ns ± 2%
>> MutexWrite/goroutines-4320-8   280ns ± 8%
>> MutexWrite/goroutines-4400-8   294ns ± 9%
>> MutexWrite/goroutines-4480-8   285ns ±10%
>> MutexWrite/goroutines-4560-8   290ns ± 8%
>> MutexWrite/goroutines-4640-8   271ns ± 3%
>> MutexWrite/goroutines-4720-8   271ns ± 4%
>>
>> 

[go-nuts] Go 1.13 Release Candidate 1 is released

[Andrew Bonventre]

Hello gophers,

We have just released go1.13rc1, a release candidate version of Go 1.13.
It is cut from release-branch.go1.13 at the revision tagged go1.13rc1.

Please try your production load tests and unit tests with the new version.
Your help testing these pre-release versions is invaluable.

Report any problems using the issue tracker:
https://golang.org/issue/new

If you have Go installed already, the easiest way to try go1.13rc1
is by using the go command:
$ go get golang.org/dl/go1.13rc1
$ go1.13rc1 download

You can download binary and source distributions from the usual place:
https://golang.org/dl/#go1.13rc1

To find out what has changed in Go 1.13, read the draft release notes:
https://tip.golang.org/doc/go1.13

Cheers,
The Go Team

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAFPZzeRDmp9JStzxmL4Q9xfK9tRj0AO_Bc3wMKAnvnbq1EwRog%40mail.gmail.com.

Re: [go-nuts] what do init functions like "math ~math" in .gox refer to?

[Ian Lance Taylor]

On Tue, Aug 20, 2019 at 9:53 PM Xiangdong JI  wrote:
>
> Thanks Ian.
> Curious to know the purpose of those dummy functions, and for the 'import' 
> functions do they have real function bodies?

The dummy functions are there mainly so that the "init" line can list
all imported packages without having an even weirder syntax.  For the
real functions, yes, they have real function bodies defined in the
appropriate package.

Ian


> On Wednesday, August 21, 2019 at 12:40:44 PM UTC+8, Ian Lance Taylor wrote:
>>
>> On Tue, Aug 20, 2019 at 9:12 PM Xiangdong JI  wrote:
>> >
>> > In 'init' section of .gox files, there might be functions like the 
>> > followings besides those 'import',
>> >
>> > math ~math bits ~math..z2fbits atomic 
>> > ~runtime..z2finternal..z2fatomic
>> >
>> > where and how are those functions generated, any source code or doc I can 
>> > refer to? Thanks.
>>
>> These are pairs of package name and init function name.  If the init
>> function name starts with "~" then it is a dummy name and there is no
>> actual function.  Otherwise it is a real function.  The function is
>> generated by the compiler and handles initialization of package scope
>> variables that cannot be statically initialized, and calls all the
>> init functions in the package.
>>
>> Ian
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/322e7407-bc21-44ea-8ca3-4ad07a7851b7%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcWne8igK9-v2pBbkPqCqARASXWoLRnxJwk%3D-BqLpr_qWQ%40mail.gmail.com.

Re: [go-nuts] asm: FUNCDATA PCDATA

[Ian Lance Taylor]

On Tue, Aug 20, 2019 at 6:42 PM Gert  wrote:
>
> https://golang.org/doc/asm
>
> $ cat x.go
> package main
>
> func main() {
> println(3)
> }
> $ GOOS=linux GOARCH=amd64 go tool compile -S x.go# or: go build 
> -gcflags -S x.go
>
> --- prog list "main" ---
>  (x.go:3) TEXTmain+0(SB),$8-0
> 0001 (x.go:3) FUNCDATA $0,gcargs·0+0(SB)
> 0002 (x.go:3) FUNCDATA $1,gclocals·0+0(SB)
> 0003 (x.go:4) MOVQ$3,(SP)
> 0004 (x.go:4) PCDATA  $0,$8
> 0005 (x.go:4) CALL,runtime.printint+0(SB)
> 0006 (x.go:4) PCDATA  $0,$-1
> 0007 (x.go:4) PCDATA  $0,$0
> 0008 (x.go:4) CALL,runtime.printnl+0(SB)
> 0009 (x.go:4) PCDATA  $0,$-1
> 0010 (x.go:5) RET ,
> ...
>
>
> The FUNCDATA and PCDATA directives contain information for use by the garbage 
> collector; they are introduced by the compiler.
>
>
> 1) funcdata What is this .0 which is not a dot :)

Note that the current compiler emits different data.

I think the ·0 was just used to distinguish the GC information for
different functions.  The second function would use ·1, and so forth.

> 2) pcdata what does $-1 mean here

I'm not sure.

> 3) Is there some more information on funcdata and pcdata? Doesn't have to be 
> in detail, because i know its a can of worms, but just enough so I can tell 
> what the garbage collector will be doing :)

I don't know of any real documentation.  You can see some basic stuff
in cmd/internal/objabi and runtime/symtab.go.  You can see the garbage
collector using the information in the runtime function getStackMap
and its callers.


> When defining a TEXT, specifying frame size $-4 tells the linker that this is 
> a leaf function that does not need to save LR on entry.
>
>
> 4) What is LR?

The Link Register used on architectures like ARM.

> 5) Leaf as in closure function?

No, leaf as in a function that does not call any other functions.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcW9m9FokcK1V2tdu%2B1g2nTw%2BwawZp%2BtcybUhKPBZMuAiA%40mail.gmail.com.

Re: [go-nuts] A problem about runtime.SetFinalizer

[Ian Lance Taylor]

On Wed, Aug 21, 2019 at 10:18 AM Robert Engels  wrote:
>
> I understand the first part, but the second? Why is the finalizer function 
> passed a reference to the object being finalized then?

There are cases where the finalizer function does want to refer to the
object, such as the finalizer on os.File which closes the descriptor.
And there are cases where the finalizer does not need to refer to the
object, such as a finalizer that just tracks the number of live
objects of some type.

Ian


> -Original Message-
> >From: Ian Lance Taylor 
> >Sent: Aug 21, 2019 12:06 PM
> >To: Robert Engels 
> >Cc: zct , golang-nuts 
> >
> >Subject: Re: [go-nuts] A problem about runtime.SetFinalizer
> >
> >On Wed, Aug 21, 2019 at 9:14 AM Robert Engels  wrote:
> >>
> >> Seems like GoLint should emit an 'unused parameter' in this case.
> >
> >It's normal for a function to have an unused parameter.  And in
> >particular it's normal for a finalizer to not refer to the object
> >being finalized.
> >
> >Ian
> >
> >
> >> -Original Message-
> >> From: zct
> >> Sent: Aug 21, 2019 10:50 AM
> >> To: golang-nuts
> >> Subject: Re: [go-nuts] A problem about runtime.SetFinalizer
> >>
> >> I got it.Thank you for your help
> >>
> >> 在 2019年8月21日星期三 UTC+8下午11:00:26，Ian Lance Taylor写道：
> >>>
> >>> On Wed, Aug 21, 2019 at 5:55 AM zct  wrote:
> >>> >
> >>> > I recently had  a goroutine leak problem, the code i reduced is like 
> >>> > this: https://play.golang.org/p/YW4hWoZZ7CD.
> >>> >
> >>> > The program is long-running and the finalizer is not called
> >>> >
> >>> > The RoomObj is deleted from map， why was it not released？ Is it 
> >>> > refereed by the asyncChan object？
> >>> >
> >>> >  I don't understand here, can somebody explain the reason
> >>>
> >>> Your finalizer itself is keeping the value alive.
> >>>
> >>> runtime.SetFinalizer(a, func(r *RoomTest) {
> >>> fmt.Println("SetFinalizer")
> >>> close(a.asyncChan.a)
> >>> })
> >>>
> >>> You need to write the finalizer function to refer to r, not a.  The
> >>> reference to a in the finalizer function ensures that a is always
> >>> live.
> >>>
> >>> Ian
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups 
> >> "golang-nuts" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an 
> >> email to golang-nuts+unsubscr...@googlegroups.com.
> >> To view this discussion on the web visit 
> >> https://groups.google.com/d/msgid/golang-nuts/906c3d64-9211-4206-a2ee-1a0d881a35b7%40googlegroups.com.
> >>
> >>
> >>
> >>
> >> --
> >> You received this message because you are subscribed to the Google Groups 
> >> "golang-nuts" group.
> >> To unsubscribe from this group and stop receiving emails from it, send an 
> >> email to golang-nuts+unsubscr...@googlegroups.com.
> >> To view this discussion on the web visit 
> >> https://groups.google.com/d/msgid/golang-nuts/821137931.4341.1566404048080%40wamui-kitty.atl.sa.earthlink.net.
> >
> >--
> >You received this message because you are subscribed to the Google Groups 
> >"golang-nuts" group.
> >To unsubscribe from this group and stop receiving emails from it, send an 
> >email to golang-nuts+unsubscr...@googlegroups.com.
> >To view this discussion on the web visit 
> >https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUjmy8%2BMPageq73sjWOR792mkXtLbGtwg6jfB_76R2R1Q%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcVSUCXfXFdw3zDUxEksKWX%3DOS9-CPbQsoXN2e3R4dY%2BfA%40mail.gmail.com.

Re: [go-nuts] A problem about runtime.SetFinalizer

[Robert Engels]

I understand the first part, but the second? Why is the finalizer function 
passed a reference to the object being finalized then?





-Original Message-
>From: Ian Lance Taylor 
>Sent: Aug 21, 2019 12:06 PM
>To: Robert Engels 
>Cc: zct , golang-nuts 
>
>Subject: Re: [go-nuts] A problem about runtime.SetFinalizer
>
>On Wed, Aug 21, 2019 at 9:14 AM Robert Engels  wrote:
>>
>> Seems like GoLint should emit an 'unused parameter' in this case.
>
>It's normal for a function to have an unused parameter.  And in
>particular it's normal for a finalizer to not refer to the object
>being finalized.
>
>Ian
>
>
>> -Original Message-
>> From: zct
>> Sent: Aug 21, 2019 10:50 AM
>> To: golang-nuts
>> Subject: Re: [go-nuts] A problem about runtime.SetFinalizer
>>
>> I got it.Thank you for your help
>>
>> 在 2019年8月21日星期三 UTC+8下午11:00:26，Ian Lance Taylor写道：
>>>
>>> On Wed, Aug 21, 2019 at 5:55 AM zct  wrote:
>>> >
>>> > I recently had  a goroutine leak problem, the code i reduced is like 
>>> > this: https://play.golang.org/p/YW4hWoZZ7CD.
>>> >
>>> > The program is long-running and the finalizer is not called
>>> >
>>> > The RoomObj is deleted from map， why was it not released？ Is it refereed 
>>> > by the asyncChan object？
>>> >
>>> >  I don't understand here, can somebody explain the reason
>>>
>>> Your finalizer itself is keeping the value alive.
>>>
>>> runtime.SetFinalizer(a, func(r *RoomTest) {
>>> fmt.Println("SetFinalizer")
>>> close(a.asyncChan.a)
>>> })
>>>
>>> You need to write the finalizer function to refer to r, not a.  The
>>> reference to a in the finalizer function ensures that a is always
>>> live.
>>>
>>> Ian
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/906c3d64-9211-4206-a2ee-1a0d881a35b7%40googlegroups.com.
>>
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/821137931.4341.1566404048080%40wamui-kitty.atl.sa.earthlink.net.
>
>-- 
>You received this message because you are subscribed to the Google Groups 
>"golang-nuts" group.
>To unsubscribe from this group and stop receiving emails from it, send an 
>email to golang-nuts+unsubscr...@googlegroups.com.
>To view this discussion on the web visit 
>https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUjmy8%2BMPageq73sjWOR792mkXtLbGtwg6jfB_76R2R1Q%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2086058808.5386.1566407889031%40wamui-kitty.atl.sa.earthlink.net.

Re: [go-nuts] A problem about runtime.SetFinalizer

[Ian Lance Taylor]

On Wed, Aug 21, 2019 at 9:14 AM Robert Engels  wrote:
>
> Seems like GoLint should emit an 'unused parameter' in this case.

It's normal for a function to have an unused parameter.  And in
particular it's normal for a finalizer to not refer to the object
being finalized.

Ian


> -Original Message-
> From: zct
> Sent: Aug 21, 2019 10:50 AM
> To: golang-nuts
> Subject: Re: [go-nuts] A problem about runtime.SetFinalizer
>
> I got it.Thank you for your help
>
> 在 2019年8月21日星期三 UTC+8下午11:00:26，Ian Lance Taylor写道：
>>
>> On Wed, Aug 21, 2019 at 5:55 AM zct  wrote:
>> >
>> > I recently had  a goroutine leak problem, the code i reduced is like this: 
>> > https://play.golang.org/p/YW4hWoZZ7CD.
>> >
>> > The program is long-running and the finalizer is not called
>> >
>> > The RoomObj is deleted from map， why was it not released？ Is it refereed 
>> > by the asyncChan object？
>> >
>> >  I don't understand here, can somebody explain the reason
>>
>> Your finalizer itself is keeping the value alive.
>>
>> runtime.SetFinalizer(a, func(r *RoomTest) {
>> fmt.Println("SetFinalizer")
>> close(a.asyncChan.a)
>> })
>>
>> You need to write the finalizer function to refer to r, not a.  The
>> reference to a in the finalizer function ensures that a is always
>> live.
>>
>> Ian
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/906c3d64-9211-4206-a2ee-1a0d881a35b7%40googlegroups.com.
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/821137931.4341.1566404048080%40wamui-kitty.atl.sa.earthlink.net.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUjmy8%2BMPageq73sjWOR792mkXtLbGtwg6jfB_76R2R1Q%40mail.gmail.com.

[go-nuts] Re: Empty cpu profile file?

[jake6502]

On Tuesday, August 20, 2019 at 7:58:29 PM UTC-4, joe mcguckin wrote:
>
> Using Dave Cheney's profile package to figure out the execution path of a 
> library I'm trying to understand.
>
> The application starts, (it's a server), I hit ^c after a couple of 
> minutes, and the console says
>
> Stopping Profiling
> Stopping
>
> Main() catches the console interrupt:
> signal.Notify(c, os.Interrupt)
> log.Info("Stopping")
>
> But the cpu profile file is empty. Is there a way to force is to sync 
> before exiting?
>

Have you looked at this thread? 
https://groups.google.com/d/topic/golang-nuts/YhnyJDI3IG0/discussion 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2aca2638-cc06-4c9a-9eee-88efca612862%40googlegroups.com.

Re: [go-nuts] A problem about runtime.SetFinalizer

[Robert Engels]

Seems like GoLint should emit an 'unused parameter' in this case.-Original Message-
From: zct 
Sent: Aug 21, 2019 10:50 AM
To: golang-nuts 
Subject: Re: [go-nuts] A problem about runtime.SetFinalizer

I got it.Thank you for your help在 2019年8月21日星期三 UTC+8下午11:00:26，Ian Lance Taylor写道：On Wed, Aug 21, 2019 at 5:55 AM zct  wrote:
>
> I recently had  a goroutine leak problem, the code i reduced is like this: https://play.golang.org/p/YW4hWoZZ7CD.
>
> The program is long-running and the finalizer is not called
>
> The RoomObj is deleted from map， why was it not released？ Is it refereed by the asyncChan object？
>
>  I don't understand here, can somebody explain the reason

Your finalizer itself is keeping the value alive.

runtime.SetFinalizer(a, func(r *RoomTest) {
    fmt.Println("SetFinalizer")
    close(a.asyncChan.a)
})

You need to write the finalizer function to refer to r, not a.  The
reference to a in the finalizer function ensures that a is always
live.

Ian




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/906c3d64-9211-4206-a2ee-1a0d881a35b7%40googlegroups.com.




-- 
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/821137931.4341.1566404048080%40wamui-kitty.atl.sa.earthlink.net.

Re: [go-nuts] A problem about runtime.SetFinalizer

[zct]

I got it.Thank you for your help

在 2019年8月21日星期三 UTC+8下午11:00:26，Ian Lance Taylor写道：
>
> On Wed, Aug 21, 2019 at 5:55 AM zct  > wrote: 
> > 
> > I recently had  a goroutine leak problem, the code i reduced is like 
> this: https://play.golang.org/p/YW4hWoZZ7CD. 
> > 
> > The program is long-running and the finalizer is not called 
> > 
> > The RoomObj is deleted from map， why was it not released？ Is it refereed 
> by the asyncChan object？ 
> > 
> >  I don't understand here, can somebody explain the reason 
>
> Your finalizer itself is keeping the value alive. 
>
> runtime.SetFinalizer(a, func(r *RoomTest) { 
> fmt.Println("SetFinalizer") 
> close(a.asyncChan.a) 
> }) 
>
> You need to write the finalizer function to refer to r, not a.  The 
> reference to a in the finalizer function ensures that a is always 
> live. 
>
> Ian 
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/906c3d64-9211-4206-a2ee-1a0d881a35b7%40googlegroups.com.

Re: [go-nuts] A problem about runtime.SetFinalizer

[Ian Lance Taylor]

On Wed, Aug 21, 2019 at 5:55 AM zct  wrote:
>
> I recently had  a goroutine leak problem, the code i reduced is like this: 
> https://play.golang.org/p/YW4hWoZZ7CD.
>
> The program is long-running and the finalizer is not called
>
> The RoomObj is deleted from map， why was it not released？ Is it refereed by 
> the asyncChan object？
>
>  I don't understand here, can somebody explain the reason

Your finalizer itself is keeping the value alive.

runtime.SetFinalizer(a, func(r *RoomTest) {
fmt.Println("SetFinalizer")
close(a.asyncChan.a)
})

You need to write the finalizer function to refer to r, not a.  The
reference to a in the finalizer function ensures that a is always
live.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcWwYTGGULQnhBeG804Om33PoppxsVFC%2B8ZuAqL78uXfOw%40mail.gmail.com.

[go-nuts] syscall.GetQueuedCompletionStatus returning qty as 0

[gaurav]

I am using a library which is watching a NAS directory for changes using 
golang syscall on windows platform.

while making syscall.GetQueuedCompletionStatus I am getting value of qty as 
0, I tried to check the syscall source code but not able to debug it.

syscall source code is one of the complex golang code I have seen and need 
help to understand it.

Library is - 
https://github.com/howeyc/fsnotify/blob/master/fsnotify_windows.go

Error on Line  479 w.Error <- errors.New("short read in readEvents()")

Can anyone who haas worked on syscall help here?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/51d749ad-0730-439d-8f6d-ecb801a4c5ac%40googlegroups.com.

[go-nuts] A problem about runtime.SetFinalizer

[zct]

I recently had  a goroutine leak problem, the code i reduced is like this: 
https://play.golang.org/p/YW4hWoZZ7CD.

The program is long-running and the finalizer is not called

The RoomObj is deleted from map， why was it not released？ Is it refereed by 
the asyncChan object？

 I don't understand here, can somebody explain the reason 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/1e06bb6a-ba93-452e-b954-161adc45b191%40googlegroups.com.

Re: [go-nuts] Existing production-ready libraries for parallelizing HTTP requests?

[roger peppe]

For just the exponential backoff part, you might want to take a look at
https://godoc.org/gopkg.in/retry.v1 which provides easily pluggable retry
strategies, including exponential backoff with jitter, and you can write
your code as a normal for loop - no awkward callback to use.

  cheers,
rog.

On Mon, 19 Aug 2019, 17:20 tom via golang-nuts, <
golang-nuts@googlegroups.com> wrote:

> tl;dr Do you of any libraries for parallelizing HTTP requests with
> per-server concurrency control and handling of retries?
>
> I'm writing a service that fetches many independent small binary blobs
> (map tiles) over HTTP from a several upstream servers and package them
> together in to a single archive. I want to parallelize the fetching of the
> small binary blobs. Currently there are O(10) upstream servers and O(1000)
> small binary blobs fetched from each.
>
> Making parallel HTTP requests in Go is trivially easy and is demonstrated
> in many Go tutorials and blog posts. However, I'm looking for a "production
> ready" library that supports:
> * Per upstream server concurrency limits.
> * Overall (across all upstream servers) concurrency limits.
> * Controllable retries with exponential backoff in the case of upstream
> server errors.
> * Timeouts for upstream requests.
> * context.Context support.
>
> This would seem to be a common enough task that I would expect to find an
> existing library that does all of the above. Existing Go web scrapers, e.g.
> colly , likely have this functionality internally
> but do not expose it in their API and are instead focused on crawling web
> pages.
>
> Do you know of any such library?
>
> Many thanks,
> Tom
>
> Confidentiality Notice:
> This electronic message and any attached documents contain confidential
> and privileged information and is for the sole use of the individual or
> entity to whom it is addressed. If you are not the addressee of this email,
> or the employee or agent responsible for delivering it to the addressee,
> you are hereby notified that any dissemination, distribution or copying of
> this transmission is strictly prohibited. If you receive this message in
> error, please notify the sender immediately by return e-mail or telephone
> and destroy the attached message (and all attached documents) immediately.
> Thank you for your cooperation.
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/cfb138b2-88d4-46ee-9315-996389718bad%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAJhgacj4OGP-xOYUimUZtQawJtQ2BDcKEvQOOaU2VDuAngCh3w%40mail.gmail.com.