Re: [go-nuts] Get tmp stuff in $GOTMPDIR instead of /tmp?

[Ian Lance Taylor]

On Thu, Aug 22, 2019 at 7:55 AM  wrote:
>
> I get a bloated /tmp-dir (in just a few days I get tens of thousands of 
> folders named /tmp/go-build828718408 etc.)

That is odd in itself.  The go-buildN directories should normally
be deleted when the go command completes.  Are you routinely killing
your go commands before they complete and have a chance to remove the
temporary directory?


> so I tried to set $GOTMPDIR, but nothing ends up there. (From what I 
> understand of the documentation the purpose of GOTMPDIR is to move tmp stuff, 
> see e.g. line 1471 of https://golang.org/src/cmd/go/alldocs.go?h=GOTMPDIR, or 
> release notes of 1.10 https://golang.org/doc/go1.10).
>
> Am I using things wrong?
>
> $ go version
> go version go1.12.6 linux/amd64
>
> $ echo $GOTMPDIR
> /tmp/go-builds
>
> $ ls $GOTMPDIR
> please-end-up-here.txt
>
> $ go build/test/this/that/etc
> [...]
>
> $ ls $GOTMPDIR
> please-end-up-here.txt

Setting GOTMPDIR should work.  What shell are you using?  Did you
"export GOTMPDIR"?

Note that, as mentioned above, an ordinary go build will create a
temporary directory in GOTMPDIR and then delete it.  So it's normal to
not see anything there after the go command completes.  You can verify
the directory it uses by running "go build -work" (in which case it
will not delete the temporary directory).

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcU-U_2Rz6UB1TUijdVyRSR-BJxxQdXgGVbJz4dX2p3vaA%40mail.gmail.com.

Re: [go-nuts] Get tmp stuff in $GOTMPDIR instead of /tmp?

[温博格]

This is routinely happening to me too, on linux. My go commands are go
build *.go
The left-over directories are empty
I'm on go1.12.5 linux/amd64

On Fri, Aug 23, 2019 at 10:04 AM Ian Lance Taylor  wrote:

> On Thu, Aug 22, 2019 at 7:55 AM  wrote:
> >
> > I get a bloated /tmp-dir (in just a few days I get tens of thousands of
> folders named /tmp/go-build828718408 etc.)
>
> That is odd in itself.  The go-buildN directories should normally
> be deleted when the go command completes.  Are you routinely killing
> your go commands before they complete and have a chance to remove the
> temporary directory?
>
>
> > so I tried to set $GOTMPDIR, but nothing ends up there. (From what I
> understand of the documentation the purpose of GOTMPDIR is to move tmp
> stuff, see e.g. line 1471 of
> https://golang.org/src/cmd/go/alldocs.go?h=GOTMPDIR, or release notes of
> 1.10 https://golang.org/doc/go1.10).
> >
> > Am I using things wrong?
> >
> > $ go version
> > go version go1.12.6 linux/amd64
> >
> > $ echo $GOTMPDIR
> > /tmp/go-builds
> >
> > $ ls $GOTMPDIR
> > please-end-up-here.txt
> >
> > $ go build/test/this/that/etc
> > [...]
> >
> > $ ls $GOTMPDIR
> > please-end-up-here.txt
>
> Setting GOTMPDIR should work.  What shell are you using?  Did you
> "export GOTMPDIR"?
>
> Note that, as mentioned above, an ordinary go build will create a
> temporary directory in GOTMPDIR and then delete it.  So it's normal to
> not see anything there after the go command completes.  You can verify
> the directory it uses by running "go build -work" (in which case it
> will not delete the temporary directory).
>
> Ian
>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcU-U_2Rz6UB1TUijdVyRSR-BJxxQdXgGVbJz4dX2p3vaA%40mail.gmail.com
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOUkXSpJ-qDqR15JcQETYFnR9FJGGbiji-iRz_qJsAQyevN14w%40mail.gmail.com.

Re: [go-nuts] Get tmp stuff in $GOTMPDIR instead of /tmp?

[Ian Lance Taylor]

On Fri, Aug 23, 2019 at 7:34 AM Peter Weinberger (温博格)  wrote:
>
> This is routinely happening to me too, on linux. My go commands are go build 
> *.go
> The left-over directories are empty
> I'm on go1.12.5 linux/amd64

Well, that is ungood.  And I don't see it myself.  Can you figure out
a way to reproduce the problem?

Ian

> On Fri, Aug 23, 2019 at 10:04 AM Ian Lance Taylor  wrote:
>>
>> On Thu, Aug 22, 2019 at 7:55 AM  wrote:
>> >
>> > I get a bloated /tmp-dir (in just a few days I get tens of thousands of 
>> > folders named /tmp/go-build828718408 etc.)
>>
>> That is odd in itself.  The go-buildN directories should normally
>> be deleted when the go command completes.  Are you routinely killing
>> your go commands before they complete and have a chance to remove the
>> temporary directory?
>>
>>
>> > so I tried to set $GOTMPDIR, but nothing ends up there. (From what I 
>> > understand of the documentation the purpose of GOTMPDIR is to move tmp 
>> > stuff, see e.g. line 1471 of 
>> > https://golang.org/src/cmd/go/alldocs.go?h=GOTMPDIR, or release notes of 
>> > 1.10 https://golang.org/doc/go1.10).
>> >
>> > Am I using things wrong?
>> >
>> > $ go version
>> > go version go1.12.6 linux/amd64
>> >
>> > $ echo $GOTMPDIR
>> > /tmp/go-builds
>> >
>> > $ ls $GOTMPDIR
>> > please-end-up-here.txt
>> >
>> > $ go build/test/this/that/etc
>> > [...]
>> >
>> > $ ls $GOTMPDIR
>> > please-end-up-here.txt
>>
>> Setting GOTMPDIR should work.  What shell are you using?  Did you
>> "export GOTMPDIR"?
>>
>> Note that, as mentioned above, an ordinary go build will create a
>> temporary directory in GOTMPDIR and then delete it.  So it's normal to
>> not see anything there after the go command completes.  You can verify
>> the directory it uses by running "go build -work" (in which case it
>> will not delete the temporary directory).
>>
>> Ian
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/CAOyqgcU-U_2Rz6UB1TUijdVyRSR-BJxxQdXgGVbJz4dX2p3vaA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXhD%2BFG7K30GsH_TNTGoijQwZdgJ4f%2BQLXcgJzcfL5xLQ%40mail.gmail.com.

Re: [go-nuts] Goroutine scheduled 10 seconds too late

[Michael Andersen]

Thanks, that's a good idea. I'll add that in. It can take a while to
reproduce, so it might be a week or so before I have the results.

On Thu, Aug 22, 2019 at 10:23 PM Ian Lance Taylor  wrote:

> On Thu, Aug 22, 2019 at 5:09 PM  wrote:
> >
> > I have a fairly complex program that has about 150 goroutines servicing
> requests. Some requests go out to cgo, which does some network IO. There is
> one goroutine that is responsible for a heartbeat (etcd keepalive) and
> sometimes (~every two weeks) this goroutine doesn't get scheduled for long
> periods of time (more than ten seconds), so it misses the heartbeat.
> >
> > I've been debugging this on and off for several months, and am
> completely sure it relates to scheduling because there is code that looks
> similar to this this:
> >
> > fmt.Printf("before %v\n",time.Now())
> > <-time.After(500*time.Millisecond)
> > fmt.Printf("after %v\n",time.Now())
> >
> > And the print statements show times more than ten seconds apart.
> >
> > I considered it might be a very long GC stop the world (maybe some rogue
> unpreemptible goroutine) but I don't see any output from gctrace=1 that
> indicates that (I could be wrong)
> >
> > The goroutines in cgo can take some time to complete their operations
> (normally tens of milliseconds, sometimes tens of seconds) but I have never
> observed more than 200 processes being created in response to the blocking
> M being moved off the P and a new M being created.
> >
> > The total CPU usage doesn't exceed 60%, so there are spare CPU cycles.
> >
> > I turned on scheduling trace, and here is the output from a problematic
> event:
> > # this is where we select on <-time.After(500ms)
> > [etcddebug] start of select 2019-08-01 06:29:55.873680674 + UTC
> m=+132838.232219823
> > SCHED 132838304ms: gomaxprocs=8 idleprocs=0 threads=123
> spinningthreads=0 idlethreads=102 runqueue=2 [0 0 3 0 0 0 0 0]
> > [INFO] 2019-08-01T06:29:56.084Z 
> > SCHED 132839312ms: gomaxprocs=8 idleprocs=0 threads=123
> spinningthreads=0 idlethreads=105 runqueue=9 [0 0 0 0 5 0 0 0]
> > 
> > SCHED 132840318ms: gomaxprocs=8 idleprocs=0 threads=123
> spinningthreads=0 idlethreads=106 runqueue=12 [0 0 0 0 5 0 0 0]
> > SCHED 132841325ms: gomaxprocs=8 idleprocs=0 threads=123
> spinningthreads=0 idlethreads=106 runqueue=12 [0 0 0 0 5 0 0 0]
> > SCHED 132842331ms: gomaxprocs=8 idleprocs=0 threads=123
> spinningthreads=0 idlethreads=107 runqueue=13 [0 0 0 0 5 0 0 0]
> > SCHED 132843338ms: gomaxprocs=8 idleprocs=0 threads=123
> spinningthreads=0 idlethreads=107 runqueue=13 [0 0 0 0 5 0 0 0]
> > SCHED 132844344ms: gomaxprocs=8 idleprocs=0 threads=123
> spinningthreads=0 idlethreads=107 runqueue=13 [0 0 0 0 5 0 0 0]
> > SCHED 132845351ms: gomaxprocs=8 idleprocs=0 threads=123
> spinningthreads=0 idlethreads=107 runqueue=14 [0 0 0 0 5 0 0 0]
> > SCHED 132846357ms: gomaxprocs=8 idleprocs=0 threads=123
> spinningthreads=0 idlethreads=107 runqueue=14 [0 0 0 0 5 0 0 0]
> > SCHED 132847364ms: gomaxprocs=8 idleprocs=0 threads=123
> spinningthreads=0 idlethreads=107 runqueue=15 [0 0 0 0 5 0 0 0]
> > # This is where the program gets killed because the <-time.After
> goroutine never ran
> > [ERROR] 2019-08-01T06:30:05.472Z clustering.go:259 ETCD LEASE CHANNEL
> CLOSED
> >
> >
> > The program panics soon after that message (on purpose) and I get this
> printout from sched trace. Not sure what triggers it.
> >
> > SCHED 132847848ms: gomaxprocs=8 idleprocs=0 threads=123
> spinningthreads=0 idlethreads=97 runqueue=2 gcwaiting=0 nmidlelocked=1
> stopwait=0 sysmonwait=0
> > P0: status=2 schedtick=714575223 syscalltick=286704429 m=-1 runqsize=0
> gfreecnt=41
> > #this is the <-time.After goroutine finally running
> > [etcddebug] time.After 2019-08-01 06:30:05.473046143 + UTC
> m=+132847.831585336 (227)
> > P1: status=1 schedtick=714230235 syscalltick=286072819 m=36 runqsize=0
> gfreecnt=11
> > P2: status=2 schedtick=710948022 syscalltick=284960027 m=-1 runqsize=0
> gfreecnt=53
> > P3: status=1 schedtick=707003392 syscalltick=283311908 m=88 runqsize=1
> gfreecnt=53
> > P4: status=2 schedtick=698758214 syscalltick=280825500 m=-1 runqsize=0
> gfreecnt=23
> > P5: status=2 schedtick=690236090 syscalltick=278745170 m=-1 runqsize=0
> gfreecnt=38
> > P6: status=1 schedtick=685940734 syscalltick=276797704 m=77 runqsize=0
> gfreecnt=51
> > P7: status=2 schedtick=681779831 syscalltick=275239769 m=-1 runqsize=0
> gfreecnt=50
> >
> >
> >
> > I've tried several times to create a simple reproducing program, and
> have failed. I'm reaching out to ask if anyone could help me interpret the
> output from the scheduling trace, or perhaps even suggest what might be
> causing this, so I can create a standalone reproducing program.
>
>
> One possibility is an unpremptible loop.  Can you get a stack trace at
> the point where the timeout fails?  Perhaps where you print ERROR
> above you could call runtime.Stack(buf, true) and print the output.
> Look at the code that you see from that output and see if it any 

Re: [go-nuts] Goroutine scheduled 10 seconds too late

[Ronny Bangsund]

On Friday, August 23, 2019 at 5:58:44 PM UTC+2, Michael Andersen wrote:
>
> It can take a while to reproduce, so it might be a week or so before I 
> have the results.
>
Can you think of ways to cause the same memory pressure quicker? Mocking 
data, running the functions the number of times they would have run in a 
two-week period etc.
 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d3d080e0-30d5-4a5c-b5fd-5014071f3bfc%40googlegroups.com.

Re: [go-nuts] Goroutine scheduled 10 seconds too late

[Michael Andersen]

On Fri, Aug 23, 2019 at 1:23 PM Ronny Bangsund 
wrote:

> On Friday, August 23, 2019 at 5:58:44 PM UTC+2, Michael Andersen wrote:
>>
>> It can take a while to reproduce, so it might be a week or so before I
>> have the results.
>>
> Can you think of ways to cause the same memory pressure quicker? Mocking
> data, running the functions the number of times they would have run in a
> two-week period etc.
>
>

It's not memory pressure, there's a ton of free memory and I'm not GC'ing
particularly often. I've tried to create a smaller reproducing example but
haven't had any luck so far. I have the servers hooked up to a synthetic
load, so there are always exactly the same number of requests being made,
at the same rate, and it sometimes happens in a day, sometimes in two
weeks. Fortunately this is in a development cluster, not production.
Incidentally, it's happened in several clusters, some in the cloud some on
bare metal, so it's unlikely to be a hardware problem.

I've added the stack print, as well as a runtime.schedtrace(true) at the
point of the error, so hopefully I will have something to work off the next
time it happens.

-- 
> You received this message because you are subscribed to a topic in the
> Google Groups "golang-nuts" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/golang-nuts/RT0vrWEc5fw/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/d3d080e0-30d5-4a5c-b5fd-5014071f3bfc%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAPLpPrtFXzN9itXStxCW-j8N172h1LrGsZxOqU9G5QwQsOZJyQ%40mail.gmail.com.

Re: [go-nuts] Goroutine scheduled 10 seconds too late

[Robert Engels]

If you write an long running non pre-emptable all go routines are effected. 

> On Aug 23, 2019, at 4:03 PM, Michael Andersen  wrote:
> 
> 
> 
>> On Fri, Aug 23, 2019 at 1:23 PM Ronny Bangsund  
>> wrote:
>>> On Friday, August 23, 2019 at 5:58:44 PM UTC+2, Michael Andersen wrote:
>>> It can take a while to reproduce, so it might be a week or so before I have 
>>> the results.
>> Can you think of ways to cause the same memory pressure quicker? Mocking 
>> data, running the functions the number of times they would have run in a 
>> two-week period etc.
>>  
> 
> It's not memory pressure, there's a ton of free memory and I'm not GC'ing 
> particularly often. I've tried to create a smaller reproducing example but 
> haven't had any luck so far. I have the servers hooked up to a synthetic 
> load, so there are always exactly the same number of requests being made, at 
> the same rate, and it sometimes happens in a day, sometimes in two weeks. 
> Fortunately this is in a development cluster, not production. Incidentally, 
> it's happened in several clusters, some in the cloud some on bare metal, so 
> it's unlikely to be a hardware problem.
> 
> I've added the stack print, as well as a runtime.schedtrace(true) at the 
> point of the error, so hopefully I will have something to work off the next 
> time it happens.
> 
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "golang-nuts" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/golang-nuts/RT0vrWEc5fw/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to 
>> golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/d3d080e0-30d5-4a5c-b5fd-5014071f3bfc%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/CAPLpPrtFXzN9itXStxCW-j8N172h1LrGsZxOqU9G5QwQsOZJyQ%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5B760743-D64C-4EB7-925B-FEB2594F888C%40ix.netcom.com.

Re: [go-nuts] Goroutine scheduled 10 seconds too late

[Michael Andersen]

On Fri, Aug 23, 2019 at 2:12 PM Robert Engels  wrote:

> If you write an long running non pre-emptable all go routines are
> effected.
>
>
Yeah, I've come across the case before where GC would begin a stop the
world but get stuck waiting for a non-preemtable goroutine. I don't think
that's happening here because when the program gets "unstuck" as the error
gets printed, there is no gctrace output that indicated a GC happened. I
can make sure by adding a dummy goroutine that just logs output every
second. That would make it more obvious that more than one goroutine is
being affected. Thanks for the suggestion.

Are you suggesting that there might be enough unpreemtable goroutines to
fill all the P's? I do have several cgo goroutines sitting in syscalls, but
my understanding was that the scheduler would "preempt" that by moving the
M off the P and creating a new M.

Thanks


> On Aug 23, 2019, at 4:03 PM, Michael Andersen 
> wrote:
>
>
>
> On Fri, Aug 23, 2019 at 1:23 PM Ronny Bangsund 
> wrote:
>
>> On Friday, August 23, 2019 at 5:58:44 PM UTC+2, Michael Andersen wrote:
>>>
>>> It can take a while to reproduce, so it might be a week or so before I
>>> have the results.
>>>
>> Can you think of ways to cause the same memory pressure quicker? Mocking
>> data, running the functions the number of times they would have run in a
>> two-week period etc.
>>
>>
>
> It's not memory pressure, there's a ton of free memory and I'm not GC'ing
> particularly often. I've tried to create a smaller reproducing example but
> haven't had any luck so far. I have the servers hooked up to a synthetic
> load, so there are always exactly the same number of requests being made,
> at the same rate, and it sometimes happens in a day, sometimes in two
> weeks. Fortunately this is in a development cluster, not production.
> Incidentally, it's happened in several clusters, some in the cloud some on
> bare metal, so it's unlikely to be a hardware problem.
>
> I've added the stack print, as well as a runtime.schedtrace(true) at the
> point of the error, so hopefully I will have something to work off the next
> time it happens.
>
> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "golang-nuts" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/golang-nuts/RT0vrWEc5fw/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/golang-nuts/d3d080e0-30d5-4a5c-b5fd-5014071f3bfc%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/golang-nuts/CAPLpPrtFXzN9itXStxCW-j8N172h1LrGsZxOqU9G5QwQsOZJyQ%40mail.gmail.com
> 
> .
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAPLpPruKr0C3122ymyV1VCdJYiZDiNQzRqX-R3oeUpqOrKmMvw%40mail.gmail.com.

Re: [go-nuts] Goroutine scheduled 10 seconds too late

[Ian Lance Taylor]

On Fri, Aug 23, 2019 at 2:30 PM Michael Andersen  wrote:
>
> Are you suggesting that there might be enough unpreemtable goroutines to fill 
> all the P's? I do have several cgo goroutines sitting in syscalls, but my 
> understanding was that the scheduler would "preempt" that by moving the M off 
> the P and creating a new M.

That is how it is supposed to work, yes.  That process can take up to
10ms, but that doesn't explain what you are seeing.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcUZo%3DCeqNW2szzmy8XynVpiGX9AcvNGshs-UKWu1%2Bqgog%40mail.gmail.com.

Re: [go-nuts] Goroutine scheduled 10 seconds too late

[Michael Andersen]

Ok, so I have more information, and it's not what I would expect.

I added scheddetail=1,schedtrace=2000 so that I had a list of which M's and
G's were on the P's during the 5 seconds that scheduling stalled. I added a
sentinel goroutine that sleeps 1 second in a loop and panics if the sleep
takes more than 4 seconds. As soon as the sentinel wakes up after the
stall, it dumps all the stacks for all the goroutines under  the assumption
that at least some of them might be in the same place as they were during
the stall. Thus, I know for sure which goroutines caused the problem and
maybe which piece of the code they were in.

During the stall, all of the P's had the same M's and the same G's, so
nothing was changing except the idlethreads and runqueues (as I would
expect because my sentinel goroutine didn't get run). Here is a compilation
of the information captured during the stall.

SCHED 3425490ms: gomaxprocs=8 idleprocs=0 threads=52 spinningthreads=0
idlethreads=35 runqueue=8 gcwaiting=0 nmidlelocked=1 stopwait=0 sysmonwait=0
 < same M's on the same P's with the same curg's as below >
SCHED 3427497ms: gomaxprocs=8 idleprocs=0 threads=52 spinningthreads=0
idlethreads=36 runqueue=9 gcwaiting=0 nmidlelocked=1 stopwait=0 sysmonwait=0
  P0: status=1 schedtick=22978494 syscalltick=6811602 m=44 runqsize=0
gfreecnt=2
  M44: p=0 curg=12934173 mallocing=0 throwing=0 preemptoff= locks=0 dying=0
spinning=false blocked=false lockedg=-1
  G12934173: status=2(chan receive) m=44 lockedm=-1

  P1: status=1 schedtick=22848756 syscalltick=6851102 m=34 runqsize=0
gfreecnt=32
  M34: p=1 curg=12934084 mallocing=0 throwing=0 preemptoff= locks=0 dying=0
spinning=false blocked=false lockedg=-1
  G12934084: status=2(chan receive) m=34 lockedm=-1

  P2: status=1 schedtick=22743131 syscalltick=6730185 m=17 runqsize=0
gfreecnt=37
  M17: p=2 curg=12934032 mallocing=0 throwing=0 preemptoff= locks=0 dying=0
spinning=false blocked=false lockedg=-1
  G12934032: status=2(chan receive) m=17 lockedm=-1

  P3: status=1 schedtick=22674653 syscalltick=6516472 m=11 runqsize=0
gfreecnt=20
  M11: p=3 curg=12934147 mallocing=0 throwing=0 preemptoff= locks=0 dying=0
spinning=false blocked=false lockedg=-1
  G12934147: status=2(chan receive) m=11 lockedm=-1

  P4: status=1 schedtick=22693327 syscalltick=6322345 m=33 runqsize=0
gfreecnt=12
  M33: p=4 curg=12934059 mallocing=0 throwing=0 preemptoff= locks=0 dying=0
spinning=false blocked=false lockedg=-1
  G12934059: status=2(chan receive) m=33 lockedm=-1

  P5: status=1 schedtick=22370152 syscalltick=6219903 m=0 runqsize=0
gfreecnt=39
  M0: p=5 curg=12934131 mallocing=0 throwing=0 preemptoff= locks=0 dying=0
spinning=false blocked=false lockedg=-1
  G12934131: status=2(chan receive) m=0 lockedm=-1

  P6: status=1 schedtick=21910259 syscalltick=6171748 m=51 runqsize=0
gfreecnt=39
  M51: p=6 curg=12934126 mallocing=0 throwing=0 preemptoff= locks=0 dying=0
spinning=false blocked=false lockedg=-1
  G12934126: status=2(chan receive) m=51 lockedm=-1

  P7: status=1 schedtick=22019793 syscalltick=6091894 m=32 runqsize=1
gfreecnt=30
  M32: p=7 curg=12934191 mallocing=0 throwing=0 preemptoff= locks=0 dying=0
spinning=false blocked=false lockedg=-1
  G12934191: status=2(chan receive) m=32 lockedm=-1

After the sentinel goroutine was scheduled (~1 second after this schedtrace
output) I captured the stacks and crossreferenced the G's. All of them are
in time.Now()

  goroutine 12934173 [runnable]:
  time.Now(0x7f453a441dc0, 0xc0169fe000, 0x5a0)
  /usr/local/go/src/time/time.go:1087 +0xb2
  -- line in request handler in my app--

  goroutine 12934084 [runnable]:
  time.Now(0x7f4614a65400, 0xc0155a2000, 0x1680)
  /usr/local/go/src/time/time.go:1087 +0xb2
  -- line in request handler in my app--

  goroutine 12934032 [runnable]:
  time.Now(0x7f457d74ae20, 0xc023179400, 0x3c0)
  /usr/local/go/src/time/time.go:1087 +0xb2
  -- line in request handler in my app--

  goroutine 12934147 [runnable]:
  time.Now(0x7f461fcca4c0, 0xc00cf38400, 0x3c0)
  /usr/local/go/src/time/time.go:1087 +0xb2
  -- line in request handler in my app--

  goroutine 12934059 [runnable]:
  time.Now(0x7f45ff9dcd00, 0xc0245b1000, 0xf00)
  /usr/local/go/src/time/time.go:1087 +0xb2
  -- line in request handler in my app--

  goroutine 12934131 [runnable]:
  time.Now(0x7f4513453e80, 0xc02a96c000, 0x5a0)
  /usr/local/go/src/time/time.go:1087 +0xb2
  -- line in request handler in my app--

  goroutine 12934126 [runnable]:
  time.Now(0x7f457fd46e60, 0xc016889200, 0x5a0)
  /usr/local/go/src/time/time.go:1087 +0xb2
  -- line in request handler in my app--

  goroutine 12934191 [runnable]:
  time.Now(0x7f45088d6b80, 0xc01770ec00, 0x5a0)
  /usr/local/go/src/time/time.go:1087 +0xb2
  -- line in request handler in my app--

I went through the path that a goroutine takes to get to that time.Now()
line and it flows from GRPC through to some simple sanitization. There are
function calls sprinkled everywhere and no loops that would be
unpreemtible, so I don't think

Re: [go-nuts] Goroutine scheduled 10 seconds too late

[Michael Andersen]

This is built with go 1.12.5, incidentally, but I have seen this on several
go versions spanning several months.

On Fri, Aug 23, 2019 at 7:36 PM Michael Andersen 
wrote:

> Ok, so I have more information, and it's not what I would expect.
>
> I added scheddetail=1,schedtrace=2000 so that I had a list of which M's
> and G's were on the P's during the 5 seconds that scheduling stalled. I
> added a sentinel goroutine that sleeps 1 second in a loop and panics if the
> sleep takes more than 4 seconds. As soon as the sentinel wakes up after the
> stall, it dumps all the stacks for all the goroutines under  the assumption
> that at least some of them might be in the same place as they were during
> the stall. Thus, I know for sure which goroutines caused the problem and
> maybe which piece of the code they were in.
>
> During the stall, all of the P's had the same M's and the same G's, so
> nothing was changing except the idlethreads and runqueues (as I would
> expect because my sentinel goroutine didn't get run). Here is a compilation
> of the information captured during the stall.
>
> SCHED 3425490ms: gomaxprocs=8 idleprocs=0 threads=52 spinningthreads=0
> idlethreads=35 runqueue=8 gcwaiting=0 nmidlelocked=1 stopwait=0 sysmonwait=0
>  < same M's on the same P's with the same curg's as below >
> SCHED 3427497ms: gomaxprocs=8 idleprocs=0 threads=52 spinningthreads=0
> idlethreads=36 runqueue=9 gcwaiting=0 nmidlelocked=1 stopwait=0 sysmonwait=0
>   P0: status=1 schedtick=22978494 syscalltick=6811602 m=44 runqsize=0
> gfreecnt=2
>   M44: p=0 curg=12934173 mallocing=0 throwing=0 preemptoff= locks=0
> dying=0 spinning=false blocked=false lockedg=-1
>   G12934173: status=2(chan receive) m=44 lockedm=-1
>
>   P1: status=1 schedtick=22848756 syscalltick=6851102 m=34 runqsize=0
> gfreecnt=32
>   M34: p=1 curg=12934084 mallocing=0 throwing=0 preemptoff= locks=0
> dying=0 spinning=false blocked=false lockedg=-1
>   G12934084: status=2(chan receive) m=34 lockedm=-1
>
>   P2: status=1 schedtick=22743131 syscalltick=6730185 m=17 runqsize=0
> gfreecnt=37
>   M17: p=2 curg=12934032 mallocing=0 throwing=0 preemptoff= locks=0
> dying=0 spinning=false blocked=false lockedg=-1
>   G12934032: status=2(chan receive) m=17 lockedm=-1
>
>   P3: status=1 schedtick=22674653 syscalltick=6516472 m=11 runqsize=0
> gfreecnt=20
>   M11: p=3 curg=12934147 mallocing=0 throwing=0 preemptoff= locks=0
> dying=0 spinning=false blocked=false lockedg=-1
>   G12934147: status=2(chan receive) m=11 lockedm=-1
>
>   P4: status=1 schedtick=22693327 syscalltick=6322345 m=33 runqsize=0
> gfreecnt=12
>   M33: p=4 curg=12934059 mallocing=0 throwing=0 preemptoff= locks=0
> dying=0 spinning=false blocked=false lockedg=-1
>   G12934059: status=2(chan receive) m=33 lockedm=-1
>
>   P5: status=1 schedtick=22370152 syscalltick=6219903 m=0 runqsize=0
> gfreecnt=39
>   M0: p=5 curg=12934131 mallocing=0 throwing=0 preemptoff= locks=0 dying=0
> spinning=false blocked=false lockedg=-1
>   G12934131: status=2(chan receive) m=0 lockedm=-1
>
>   P6: status=1 schedtick=21910259 syscalltick=6171748 m=51 runqsize=0
> gfreecnt=39
>   M51: p=6 curg=12934126 mallocing=0 throwing=0 preemptoff= locks=0
> dying=0 spinning=false blocked=false lockedg=-1
>   G12934126: status=2(chan receive) m=51 lockedm=-1
>
>   P7: status=1 schedtick=22019793 syscalltick=6091894 m=32 runqsize=1
> gfreecnt=30
>   M32: p=7 curg=12934191 mallocing=0 throwing=0 preemptoff= locks=0
> dying=0 spinning=false blocked=false lockedg=-1
>   G12934191: status=2(chan receive) m=32 lockedm=-1
>
> After the sentinel goroutine was scheduled (~1 second after this
> schedtrace output) I captured the stacks and crossreferenced the G's. All
> of them are in time.Now()
>
>   goroutine 12934173 [runnable]:
>   time.Now(0x7f453a441dc0, 0xc0169fe000, 0x5a0)
>   /usr/local/go/src/time/time.go:1087 +0xb2
>   -- line in request handler in my app--
>
>   goroutine 12934084 [runnable]:
>   time.Now(0x7f4614a65400, 0xc0155a2000, 0x1680)
>   /usr/local/go/src/time/time.go:1087 +0xb2
>   -- line in request handler in my app--
>
>   goroutine 12934032 [runnable]:
>   time.Now(0x7f457d74ae20, 0xc023179400, 0x3c0)
>   /usr/local/go/src/time/time.go:1087 +0xb2
>   -- line in request handler in my app--
>
>   goroutine 12934147 [runnable]:
>   time.Now(0x7f461fcca4c0, 0xc00cf38400, 0x3c0)
>   /usr/local/go/src/time/time.go:1087 +0xb2
>   -- line in request handler in my app--
>
>   goroutine 12934059 [runnable]:
>   time.Now(0x7f45ff9dcd00, 0xc0245b1000, 0xf00)
>   /usr/local/go/src/time/time.go:1087 +0xb2
>   -- line in request handler in my app--
>
>   goroutine 12934131 [runnable]:
>   time.Now(0x7f4513453e80, 0xc02a96c000, 0x5a0)
>   /usr/local/go/src/time/time.go:1087 +0xb2
>   -- line in request handler in my app--
>
>   goroutine 12934126 [runnable]:
>   time.Now(0x7f457fd46e60, 0xc016889200, 0x5a0)
>   /usr/local/go/src/time/time.go:1087 +0xb2
>   -- line in request handler in my app--
>
>   goroutine 12934191 [runnable]:
>   

[go-nuts] how to merge different struct to one struct

[Lee Rick]

i want to do 
type A struct{ Name string}
type B struct{Age int}
type C struct{Address string}

a, b, c := A{}, B{},C{}
have a function
d1 := merge(a,b)  //d1 like struct{Name string, Age int}
d2 := merge(a,c)  //d2 like struct{Name string, Address string}
d3 := merge(b,c) //d3 like struct{Age int, Address string}


how to realize merge function? anyone can give me a demo

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/9d0f4159-146b-4d5b-91c0-8846445a84d4%40googlegroups.com.

Re: [go-nuts] how to merge different struct to one struct

[burak serdar]

On Fri, Aug 23, 2019 at 9:25 PM Lee Rick  wrote:
>
> i want to do
> type A struct{ Name string}
> type B struct{Age int}
> type C struct{Address string}
>
> a, b, c := A{}, B{},C{}
> have a function
> d1 := merge(a,b)  //d1 like struct{Name string, Age int}
> d2 := merge(a,c)  //d2 like struct{Name string, Address string}
> d3 := merge(b,c) //d3 like struct{Age int, Address string}

You cannot have a function that merges two structs and returns a new
one, but you can have structs that are compositions of other structs.

type AB struct {
  A
  B
}

Struct AB is a struct containing both A and B. You can access the
members of A and B as though they're members of AB.

ab:=AB{}
ab.Name="a"
ab.Age=1

This is also valid:

ab.A.Name="a"
ab.B.Age=1

Initialization has to be explicit though:

ab:=AB{A:A{Name:"name"}, B:B{Age:1}}




>
>
> how to realize merge function? anyone can give me a demo
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/9d0f4159-146b-4d5b-91c0-8846445a84d4%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAMV2Rqrw7t59H7B8d3bsbskNBhwBXMyR6tMar9Q_xBAZCbsSOA%40mail.gmail.com.

Re: [go-nuts] how to merge different struct to one struct

[Lee Rick]

it's not my need, hope other methods

在 2019年8月24日星期六 UTC+8上午11:43:32，burak serdar写道：
>
> On Fri, Aug 23, 2019 at 9:25 PM Lee Rick > 
> wrote: 
> > 
> > i want to do 
> > type A struct{ Name string} 
> > type B struct{Age int} 
> > type C struct{Address string} 
> > 
> > a, b, c := A{}, B{},C{} 
> > have a function 
> > d1 := merge(a,b)  //d1 like struct{Name string, Age int} 
> > d2 := merge(a,c)  //d2 like struct{Name string, Address string} 
> > d3 := merge(b,c) //d3 like struct{Age int, Address string} 
>
> You cannot have a function that merges two structs and returns a new 
> one, but you can have structs that are compositions of other structs. 
>
> type AB struct { 
>   A 
>   B 
> } 
>
> Struct AB is a struct containing both A and B. You can access the 
> members of A and B as though they're members of AB. 
>
> ab:=AB{} 
> ab.Name="a" 
> ab.Age=1 
>
> This is also valid: 
>
> ab.A.Name="a" 
> ab.B.Age=1 
>
> Initialization has to be explicit though: 
>
> ab:=AB{A:A{Name:"name"}, B:B{Age:1}} 
>
>
>
>
> > 
> > 
> > how to realize merge function? anyone can give me a demo 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an email to golan...@googlegroups.com . 
> > To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/9d0f4159-146b-4d5b-91c0-8846445a84d4%40googlegroups.com.
>  
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/f7e61c2d-1aef-4160-b525-7d9c2488f9e1%40googlegroups.com.

Re: [go-nuts] how to merge different struct to one struct

[burak serdar]

On Fri, Aug 23, 2019 at 11:11 PM Lee Rick  wrote:
>
> it's not my need, hope other methods

What is your need? What are the types of the variables d1, d2, d3 in
your example?

>
> 在 2019年8月24日星期六 UTC+8上午11:43:32，burak serdar写道：
>>
>> On Fri, Aug 23, 2019 at 9:25 PM Lee Rick  wrote:
>> >
>> > i want to do
>> > type A struct{ Name string}
>> > type B struct{Age int}
>> > type C struct{Address string}
>> >
>> > a, b, c := A{}, B{},C{}
>> > have a function
>> > d1 := merge(a,b)  //d1 like struct{Name string, Age int}
>> > d2 := merge(a,c)  //d2 like struct{Name string, Address string}
>> > d3 := merge(b,c) //d3 like struct{Age int, Address string}
>>
>> You cannot have a function that merges two structs and returns a new
>> one, but you can have structs that are compositions of other structs.
>>
>> type AB struct {
>>   A
>>   B
>> }
>>
>> Struct AB is a struct containing both A and B. You can access the
>> members of A and B as though they're members of AB.
>>
>> ab:=AB{}
>> ab.Name="a"
>> ab.Age=1
>>
>> This is also valid:
>>
>> ab.A.Name="a"
>> ab.B.Age=1
>>
>> Initialization has to be explicit though:
>>
>> ab:=AB{A:A{Name:"name"}, B:B{Age:1}}
>>
>>
>>
>>
>> >
>> >
>> > how to realize merge function? anyone can give me a demo
>> >
>> > --
>> > You received this message because you are subscribed to the Google Groups 
>> > "golang-nuts" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an 
>> > email to golan...@googlegroups.com.
>> > To view this discussion on the web visit 
>> > https://groups.google.com/d/msgid/golang-nuts/9d0f4159-146b-4d5b-91c0-8846445a84d4%40googlegroups.com.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/f7e61c2d-1aef-4160-b525-7d9c2488f9e1%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAMV2RqrTX01gUEHfimvH3rLgLW6h0bRRRi0mK41vf-fK9wBWoQ%40mail.gmail.com.

Re: [go-nuts] how to merge different struct to one struct

[Kurtis Rader]

On Fri, Aug 23, 2019 at 10:11 PM Lee Rick  wrote:

> it's not my need, hope other methods
>

Then you need to explain why that answer is not satisfactory and otherwise
better explain your requirements. In your hypothetical example the objects
such as `d1` would not be usable except via reflection (assuming that
approach was even viable). What exactly would be the point? If you need a
data structure that can be composed at run time there are many options. Why
does it need to be a Go struct?

-- 
Kurtis Rader
Caretaker of the exceptional canines Junior and Hank

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CABx2%3DD8T1%2BO7mf6OZH0A23miR9cidgzCUsqUM5zRnjS6e_YarA%40mail.gmail.com.

Re: [go-nuts] how to merge different struct to one struct

[Jan Mercl]

On Sat, Aug 24, 2019 at 5:25 AM Lee Rick  wrote:
>
> i want to do
> type A struct{ Name string}
> type B struct{Age int}
> type C struct{Address string}
>
> a, b, c := A{}, B{},C{}
> have a function
> d1 := merge(a,b)  //d1 like struct{Name string, Age int}
> d2 := merge(a,c)  //d2 like struct{Name string, Address string}
> d3 := merge(b,c) //d3 like struct{Age int, Address string}
>
>
> how to realize merge function? anyone can give me a demo

Is this the XY problem? What is the task/goal?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA40n-U%3DrJD8%3D_adUnK0hXjHzw23DLG5XS6LKYsh0PGSYEvcoQ%40mail.gmail.com.