Re: [go-nuts] Compilation fails when adding option "-trimpath", because change a dependency to require gcc

[Jérôme LAFORGE]

> I haven't verified this, so this is just a guess, but perhaps without 
> -trimpath the go tool is using the prebuilt runtime/cgo.a, which was 
> built on a system with a C compiler.  When you add -trimpath, the 
> runtime/cgo package needs to be rebuilt in order to trim the paths. 
> Unfortunately, that rebuild fails, as the system you are building on 
> does not have a C compiler. 
>

It would be great to prebuilt runtime/cgo.a also with `-trimpath` option in 
order  to avoid unobvious dependency with gcc when you have just added 
`-trimpath` option or at least mention it in `go help build` doc.  

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e7617baf-d989-4777-b940-913c34078b27%40googlegroups.com.

[go-nuts] Re: Go 1.14 Release Candidate 1 is released

[Manlio Perillo]

Is there some overhead due to preemption?

About EINTR, I suspect that some programs will have problems.
As an example
https://github.com/golang/go/issues/22838

The issue was reported for MacOS, but technically every signal, including 
SIGSTOP, can  interrupt a syscall.

Thanks
Manlio

On Wednesday, February 5, 2020 at 8:43:47 PM UTC+1, Carlos Amedee wrote:
>
> Hello gophers,
>
> We have just released go1.14rc1, a release candidate version of Go 1.14.
> It is cut from release-branch.go1.14 at the revision tagged go1.14rc1.
>
> Please try your production load tests and unit tests with the new version.
> Your help testing these pre-release versions is invaluable.
>
> Report any problems using the issue tracker:
> https://golang.org/issue/new
>
> If you have Go installed already, the easiest way to try go1.14rc1
> is by using the go command:
> $ go get golang.org/dl/go1.14rc1
> $ go1.14rc1 download
>
> You can download binary and source distributions from the usual place:
> https://golang.org/dl/#go1.14rc1
>
> To find out what has changed in Go 1.14, read the draft release notes:
> https://tip.golang.org/doc/go1.14
>
> Cheers,
> Carlos and Dmitri on behalf of the Go team
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/e8db33b3-097a-42ff-a486-49733cbd8cf9%40googlegroups.com.

Re: [go-nuts] runtime/cgo: pthread_create failed: Resource temporarily unavailable

[Robert Engels]

And I’ll repeat, you probably need a pool to limit the number of concurrent 
processes/connections. 
If you had that you would probably of detected the problem sooner. 

> On Feb 5, 2020, at 6:32 PM, Robert Engels  wrote:
> 
> 
> I am not familiar with YouTube-dl but if you can pass a timeout on the 
> command line that would be safest. 
> 
> Using CommandContext will forcibly kill the process. 
> 
>>> On Feb 5, 2020, at 4:13 PM, jnaderl...@gmail.com wrote:
>>> 
>> 
>> So someone answered on the SO question I had posted, explaining how to check 
>> the number of processes, they suggest I add a timeout, since it seems 
>> youtube-dl, after succesffully running, sometimes hangs, so it seems golang 
>> isn't closing it sometimes.
>> 
>> Should I do a timeout, orinstead the CommandContext?
>> 
>>> On Wednesday, February 5, 2020 at 2:20:25 PM UTC-7, Johnathan Nader wrote:
>>> How can I debug the number of threads and go routines running and checking 
>>> if the Wait()'s finish? Because I believe that may be problem, that they 
>>> hang.
>>> 
>>> And when you say append the output, are you saying make a go routine to 
>>> write to the headers?  If you have an example I would appreciate it
>>> 
 On Wednesday, February 5, 2020 at 8:47:05 AM UTC-7, Robert Engels wrote:
 I think your problem may be 
 
 "Depending on the HTTP protocol version and the client, calling 
 // Write or WriteHeader may prevent future reads on the 
 // Request.Body. For HTTP/1.x requests, handlers should read any 
 // needed request body data before writing the response. Once the 
 // headers have been flushed (due to either an explicit Flusher.Flush 
 // call or writing enough data to trigger a flush), the request body 
 // may be unavailable. For HTTP/2 requests, the Go HTTP server permits 
 // handlers to continue to read the request body while concurrently 
 // writing the response. However, such behavior may not be supported 
 // by all HTTP/2 clients. Handlers should read before writing if 
 // possible to maximize compatibility." 
 
 You may need to write the ResponseHeader as a final stage and append the 
 output - if you write the header you may be hanging the input stages. If 
 the input stage hangs (you tube download hangs, etc.), the whole process 
 is going to hang. 
 
 Did you debug the number of threads and go routines the process has while 
 running? I am betting these are continually increasing. (Another check 
 would be that all Waits() complete). 
 
 Finally, I would use a CommandContext with a Deadline to ensure stragglers 
 are cleaned-up. 
 
 
 
 -Original Message- 
 >From: Ian Lance Taylor  
 >Sent: Feb 5, 2020 8:32 AM 
 >To: jnade...@gmail.com 
 >Cc: golang-nuts  
 >Subject: Re: [go-nuts] runtime/cgo: pthread_create failed: Resource 
 >temporarily unavailable 
 > 
 >On Tue, Feb 4, 2020 at 11:22 PM  wrote: 
 >> 
 >> I don't think that is the issue.  I have tried it on a few different 
 >> servers.  Most recent one with 100 gig's of ram and 50 cores.  The load 
 >> average never goes above 9, but the ram slowly but surely on htop 
 >> starts to go up.  The go binary ends up climbing slowly in it's ram use 
 >> over time, then after a hour or so, it reaches around 30 gigs of ram 
 >> and then crashes, and restarts. 
 >> 
 >> I have it under supervisor. 
 > 
 >That is not inconsistent with Robert's suggestion.  If you are 
 >starting C threads that don't do any work but never exit, that is 
 >exactly what you would see. 
 > 
 >It's not the only possible cause of this.  There could also be a space 
 >leak, either in C code with memory that is malloced but never freed, 
 >or in Go code with memory that something keeps a permanent reference 
 >to. 
 > 
 >Ian 
 > 
 > 
 >> On Tuesday, February 4, 2020 at 2:00:55 PM UTC-7, Robert Engels wrote: 
 >>> 
 >>> Are you certain you are not just starting too many processes? Ie use a 
 >>> “worker pool” so you have at most N conversions happening at the same 
 >>> time. 
 >>> 
 >>> On Feb 4, 2020, at 2:34 PM, Robert Engels  
 >>> wrote: 
 >>> 
 >>>  
 >>> I will take a more in-depth look this evening. 
 >>> 
 >>> On Feb 4, 2020, at 2:19 PM, jnade...@gmail.com wrote: 
 >>> 
 >>>  
 >>> I also thought the Wait() took care of closing the file descriptors? 
 >>> Are you saying I should add a pipe3.Close()? Or a youtube.Close()? 
 >>> 
 >>> -- 
 >>> You received this message because you are subscribed to the Google 
 >>> Groups "golang-nuts" group. 
 >>> To unsubscribe from this group and stop receiving emails from it, send 
 >>> an email to golan...@googlegroups.com. 
 >>> To view this discussion on the web vi

Re: [go-nuts] runtime/cgo: pthread_create failed: Resource temporarily unavailable

[Robert Engels]

I am not familiar with YouTube-dl but if you can pass a timeout on the command 
line that would be safest. 

Using CommandContext will forcibly kill the process. 

> On Feb 5, 2020, at 4:13 PM, jnaderl...@gmail.com wrote:
> 
> 
> So someone answered on the SO question I had posted, explaining how to check 
> the number of processes, they suggest I add a timeout, since it seems 
> youtube-dl, after succesffully running, sometimes hangs, so it seems golang 
> isn't closing it sometimes.
> 
> Should I do a timeout, orinstead the CommandContext?
> 
>> On Wednesday, February 5, 2020 at 2:20:25 PM UTC-7, Johnathan Nader wrote:
>> How can I debug the number of threads and go routines running and checking 
>> if the Wait()'s finish? Because I believe that may be problem, that they 
>> hang.
>> 
>> And when you say append the output, are you saying make a go routine to 
>> write to the headers?  If you have an example I would appreciate it
>> 
>>> On Wednesday, February 5, 2020 at 8:47:05 AM UTC-7, Robert Engels wrote:
>>> I think your problem may be 
>>> 
>>> "Depending on the HTTP protocol version and the client, calling 
>>> // Write or WriteHeader may prevent future reads on the 
>>> // Request.Body. For HTTP/1.x requests, handlers should read any 
>>> // needed request body data before writing the response. Once the 
>>> // headers have been flushed (due to either an explicit Flusher.Flush 
>>> // call or writing enough data to trigger a flush), the request body 
>>> // may be unavailable. For HTTP/2 requests, the Go HTTP server permits 
>>> // handlers to continue to read the request body while concurrently 
>>> // writing the response. However, such behavior may not be supported 
>>> // by all HTTP/2 clients. Handlers should read before writing if 
>>> // possible to maximize compatibility." 
>>> 
>>> You may need to write the ResponseHeader as a final stage and append the 
>>> output - if you write the header you may be hanging the input stages. If 
>>> the input stage hangs (you tube download hangs, etc.), the whole process is 
>>> going to hang. 
>>> 
>>> Did you debug the number of threads and go routines the process has while 
>>> running? I am betting these are continually increasing. (Another check 
>>> would be that all Waits() complete). 
>>> 
>>> Finally, I would use a CommandContext with a Deadline to ensure stragglers 
>>> are cleaned-up. 
>>> 
>>> 
>>> 
>>> -Original Message- 
>>> >From: Ian Lance Taylor  
>>> >Sent: Feb 5, 2020 8:32 AM 
>>> >To: jnade...@gmail.com 
>>> >Cc: golang-nuts  
>>> >Subject: Re: [go-nuts] runtime/cgo: pthread_create failed: Resource 
>>> >temporarily unavailable 
>>> > 
>>> >On Tue, Feb 4, 2020 at 11:22 PM  wrote: 
>>> >> 
>>> >> I don't think that is the issue.  I have tried it on a few different 
>>> >> servers.  Most recent one with 100 gig's of ram and 50 cores.  The load 
>>> >> average never goes above 9, but the ram slowly but surely on htop starts 
>>> >> to go up.  The go binary ends up climbing slowly in it's ram use over 
>>> >> time, then after a hour or so, it reaches around 30 gigs of ram and then 
>>> >> crashes, and restarts. 
>>> >> 
>>> >> I have it under supervisor. 
>>> > 
>>> >That is not inconsistent with Robert's suggestion.  If you are 
>>> >starting C threads that don't do any work but never exit, that is 
>>> >exactly what you would see. 
>>> > 
>>> >It's not the only possible cause of this.  There could also be a space 
>>> >leak, either in C code with memory that is malloced but never freed, 
>>> >or in Go code with memory that something keeps a permanent reference 
>>> >to. 
>>> > 
>>> >Ian 
>>> > 
>>> > 
>>> >> On Tuesday, February 4, 2020 at 2:00:55 PM UTC-7, Robert Engels wrote: 
>>> >>> 
>>> >>> Are you certain you are not just starting too many processes? Ie use a 
>>> >>> “worker pool” so you have at most N conversions happening at the same 
>>> >>> time. 
>>> >>> 
>>> >>> On Feb 4, 2020, at 2:34 PM, Robert Engels  wrote: 
>>> >>> 
>>> >>>  
>>> >>> I will take a more in-depth look this evening. 
>>> >>> 
>>> >>> On Feb 4, 2020, at 2:19 PM, jnade...@gmail.com wrote: 
>>> >>> 
>>> >>>  
>>> >>> I also thought the Wait() took care of closing the file descriptors? 
>>> >>> Are you saying I should add a pipe3.Close()? Or a youtube.Close()? 
>>> >>> 
>>> >>> -- 
>>> >>> You received this message because you are subscribed to the Google 
>>> >>> Groups "golang-nuts" group. 
>>> >>> To unsubscribe from this group and stop receiving emails from it, send 
>>> >>> an email to golan...@googlegroups.com. 
>>> >>> To view this discussion on the web visit 
>>> >>> https://groups.google.com/d/msgid/golang-nuts/2cbf0d51-0b36-4c37-a324-8a343193769a%40googlegroups.com.
>>> >>> 
>>> >>> -- 
>>> >>> You received this message because you are subscribed to the Google 
>>> >>> Groups "golang-nuts" group. 
>>> >>> To unsubscribe from this group and stop receiving emails from it, send 
>>> >>> an email to go

Re: [go-nuts] Re: Go 1.14 Release Candidate 1 is released

[Ian Lance Taylor]

On Wed, Feb 5, 2020 at 3:40 PM amr  wrote:
>
> I'm seeing the same problem, also.

Sorry about that.  It's been fixed.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcVMTuyf7RE6TnGegwCopUPeGR5mto5%2B-xQHr5fPBaz2Ag%40mail.gmail.com.

[go-nuts] Re: Go 1.14 Release Candidate 1 is released

[amr]

I'm seeing the same problem, also.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3ef3143b-15d4-4ae1-902b-c9d860ebc1d4%40googlegroups.com.

[go-nuts] Re: Go 1.14 Release Candidate 1 is released

[Michael Ellis]

On Wednesday, February 5, 2020 at 6:12:21 PM UTC-5, Andrey Tcherepanov 
wrote:
>
> Sorry to mention, but for me https://tip.golang.org/doc/go1.14 returns 
> "template: 
> main:846: unexpected EOF"
>
> Not sure if this is because it is work in progress, or too many people 
> trying to hit this site after announcement.
>

I'm seeing the same problem. 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/b534f383-9421-4504-8816-df496a2ecdc7%40googlegroups.com.

[go-nuts] Re: Go 1.14 Release Candidate 1 is released

[Andrey Tcherepanov]

Sorry to mention, but for me https://tip.golang.org/doc/go1.14 returns 
"template: 
main:846: unexpected EOF"

Not sure if this is because it is work in progress, or too many people 
trying to hit this site after announcement.

Andrey


On Wednesday, February 5, 2020 at 12:43:47 PM UTC-7, Carlos Amedee wrote:
>
> Hello gophers,
>
> We have just released go1.14rc1, a release candidate version of Go 1.14.
> It is cut from release-branch.go1.14 at the revision tagged go1.14rc1.
>
> Please try your production load tests and unit tests with the new version.
> Your help testing these pre-release versions is invaluable.
>
> Report any problems using the issue tracker:
> https://golang.org/issue/new
>
> If you have Go installed already, the easiest way to try go1.14rc1
> is by using the go command:
> $ go get golang.org/dl/go1.14rc1
> $ go1.14rc1 download
>
> You can download binary and source distributions from the usual place:
> https://golang.org/dl/#go1.14rc1
>
> To find out what has changed in Go 1.14, read the draft release notes:
> https://tip.golang.org/doc/go1.14
>
> Cheers,
> Carlos and Dmitri on behalf of the Go team
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/5807e8c6-e48d-433e-9642-f1fa86cf427a%40googlegroups.com.

Re: [go-nuts] Seeking new custodian for godoc.org

[Ian Lance Taylor]

On Wed, Feb 5, 2020 at 2:29 PM Liam  wrote:
>
> Russ, I apologize for any misunderstanding, but below is the relevant quote 
> from your post. When I suggested in response that godoc.org be returned to 
> the community, Ian asked if anyone would be interested to take it on. My note 
> here was prompted by Ian's query. (Appended is my post to which Ian 
> responded, posing Q's which have not yet been answered.)

That's not quite what I meant, and I apologize for muddying the
already very murky waters.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXojbKZwv0w53J1aqHJHbytt5pHTjCh96jzxqJ6rW4%2B4g%40mail.gmail.com.

Re: [go-nuts] Seeking new custodian for godoc.org

[Liam]

Russ, I apologize for any misunderstanding, but below is the relevant quote 
from your post. When I suggested in response that godoc.org be returned to 
the community, Ian asked if anyone would be interested to take it on. My 
note here was prompted by Ian's query. (Appended is my post to which Ian 
responded, posing Q's which have not yet been answered.)

It was an oversimplification to say, "Legal advice will ... close 
godoc.org," but hardly a "grotesque misrepresentation". Redirecting links 
from godoc.org to another site amounts to closure, IMO. 

I assume good faith on the part of the Go team in its efforts and 
communications; please do the same for mine.

Most sincerely,
Liam


*# Why does pkg.go.dev  require a detected license to 
show docs? *
*Why doesn't godoc.org ?*
The teams working on the proxy and on pkg.go.dev have spent a lot of
time talking to Google's lawyers about what we can and can't do with
Go source code downloaded from the internet. The rule we've been given
to follow is that serving a pretty HTML version of the docs is
displaying a modified version of the original, and we can only do that
if there's a recognized known-good license that gives us that
permission.

When we adopted godoc.org from Gary Burd back in 2014, it did not
occur to any of us to put it through that kind of review. If we had,
maybe the community would have gone through this licensing pain
earlier. For now we are focusing on making changes to pkg.go.dev
rather than correcting past mistakes on godoc.org. (At this point,
more scrutiny of what godoc.org does is not likely to have an outcome
that anyone likes.)


*# What fraction of popular packages don't display on pkg.go.dev 
?*
Right now it looks like pkg.go.dev sees 1,200 modules imported by at
least 100 other modules. Of those, it looks like 82 are flagged as not
redistributable, so that we can't show their docs. That's under 7%,
and we're working to understand that better. If any of those are
mistakes on our end, we'll fix them.

Another thing that was suggested that I think is a great idea is to
change the “no docs available” page to have a command-line to bring up
the docs in your own local godoc command.


On Tuesday, February 4, 2020 at 11:38:22 AM UTC-8, Liam Breck wrote:
>
> Many search services do what godoc.org does, print part of a published 
> document. It's not generally considered illegal.
>
> The normal means to protest such use is a DMCA takedown request. Google 
> publishes data on those it receives at transparencyreport.google.com. 
> Have you ever received a takedown request re godoc.org?
>
> Have you heard of any search service which refuses to print full results 
> for open source projects that don't use one of a certain set of licenses?
>
> There is no need for the anxious policy re open source docs which you plan 
> to adopt. If your lawyers are convinced there is, and you can't seek a 
> second opinion, a Go foundation is a reasonable solution.
>
> At the very least, hand godoc.org off to a third party.
>
 

On Wednesday, February 5, 2020 at 1:04:05 PM UTC-8, Russ Cox wrote:
>
> > [go-nuts] Seeking new custodian for godoc.org
>
> The Go team is the custodian for godoc.org and we are not seeking a new 
> one.
>
> > Legal advice will apparently force the Go team to close godoc.org [1]
>
> This is a grotesque misrepresentation of what I wrote. This is what I 
> wrote:
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/77796d78-e134-49d6-947e-bbe2471db8f8%40googlegroups.com.

Re: [go-nuts] Re: Go 1.14 Release Candidate 1 is released

[Ian Lance Taylor]

On Wed, Feb 5, 2020 at 2:05 PM Kaveh Shahbazian
 wrote:
>>
>> "Goroutines are now asynchronously preemptible."
>
> Very nice!
>
>> "A consequence of the implementation of preemption is that on Unix systems, 
>> including Linux and macOS systems, programs built with Go 1.14 will receive 
>> more signals than programs built with earlier releases. This means that 
>> programs that use packages like syscall or golang.org/x/sys/unix will see 
>> more slow system calls fail with EINTR errors. Those programs will have to 
>> handle those errors in some way, most likely looping to try the system call 
>> again."
>
> How does this affects projects like Docker?

I expect that Docker already checks for EINTR where required, so there
shouldn't be much effect at all.  Note that checking for EINTR was
always required; what is new in 1.14 is that failing to check for
EINTR is more likely to appear as a problem.

But please do let us know if Docker doesn't work well with 1.14.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcX7WhnuQQW2AeaQcGNbx9guaYdGS%2BDdJvM5_V81XBUg0Q%40mail.gmail.com.

Re: [go-nuts] runtime/cgo: pthread_create failed: Resource temporarily unavailable

[jnaderlowe]

So someone answered on the SO question I had posted, explaining how to 
check the number of processes, they suggest I add a timeout, since it seems 
youtube-dl, after succesffully running, sometimes hangs, so it seems golang 
isn't closing it sometimes.

Should I do a timeout, orinstead the CommandContext?

On Wednesday, February 5, 2020 at 2:20:25 PM UTC-7, Johnathan Nader wrote:
>
> How can I debug the number of threads and go routines running and checking 
> if the Wait()'s finish? Because I believe that may be problem, that they 
> hang.
>
> And when you say append the output, are you saying make a go routine to 
> write to the headers?  If you have an example I would appreciate it
>
> On Wednesday, February 5, 2020 at 8:47:05 AM UTC-7, Robert Engels wrote:
>>
>> I think your problem may be 
>>
>> "Depending on the HTTP protocol version and the client, calling 
>> // Write or WriteHeader may prevent future reads on the 
>> // Request.Body. For HTTP/1.x requests, handlers should read any 
>> // needed request body data before writing the response. Once the 
>> // headers have been flushed (due to either an explicit Flusher.Flush 
>> // call or writing enough data to trigger a flush), the request body 
>> // may be unavailable. For HTTP/2 requests, the Go HTTP server 
>> permits 
>> // handlers to continue to read the request body while concurrently 
>> // writing the response. However, such behavior may not be supported 
>> // by all HTTP/2 clients. Handlers should read before writing if 
>> // possible to maximize compatibility." 
>>
>> You may need to write the ResponseHeader as a final stage and append the 
>> output - if you write the header you may be hanging the input stages. If 
>> the input stage hangs (you tube download hangs, etc.), the whole process is 
>> going to hang. 
>>
>> Did you debug the number of threads and go routines the process has while 
>> running? I am betting these are continually increasing. (Another check 
>> would be that all Waits() complete). 
>>
>> Finally, I would use a CommandContext with a Deadline to ensure 
>> stragglers are cleaned-up. 
>>
>>
>>
>> -Original Message- 
>> >From: Ian Lance Taylor  
>> >Sent: Feb 5, 2020 8:32 AM 
>> >To: jnade...@gmail.com 
>> >Cc: golang-nuts  
>> >Subject: Re: [go-nuts] runtime/cgo: pthread_create failed: Resource 
>> temporarily unavailable 
>> > 
>> >On Tue, Feb 4, 2020 at 11:22 PM  wrote: 
>> >> 
>> >> I don't think that is the issue.  I have tried it on a few different 
>> servers.  Most recent one with 100 gig's of ram and 50 cores.  The load 
>> average never goes above 9, but the ram slowly but surely on htop starts to 
>> go up.  The go binary ends up climbing slowly in it's ram use over time, 
>> then after a hour or so, it reaches around 30 gigs of ram and then crashes, 
>> and restarts. 
>> >> 
>> >> I have it under supervisor. 
>> > 
>> >That is not inconsistent with Robert's suggestion.  If you are 
>> >starting C threads that don't do any work but never exit, that is 
>> >exactly what you would see. 
>> > 
>> >It's not the only possible cause of this.  There could also be a space 
>> >leak, either in C code with memory that is malloced but never freed, 
>> >or in Go code with memory that something keeps a permanent reference 
>> >to. 
>> > 
>> >Ian 
>> > 
>> > 
>> >> On Tuesday, February 4, 2020 at 2:00:55 PM UTC-7, Robert Engels wrote: 
>> >>> 
>> >>> Are you certain you are not just starting too many processes? Ie use 
>> a “worker pool” so you have at most N conversions happening at the same 
>> time. 
>> >>> 
>> >>> On Feb 4, 2020, at 2:34 PM, Robert Engels  
>> wrote: 
>> >>> 
>> >>>  
>> >>> I will take a more in-depth look this evening. 
>> >>> 
>> >>> On Feb 4, 2020, at 2:19 PM, jnade...@gmail.com wrote: 
>> >>> 
>> >>>  
>> >>> I also thought the Wait() took care of closing the file descriptors? 
>> Are you saying I should add a pipe3.Close()? Or a youtube.Close()? 
>> >>> 
>> >>> -- 
>> >>> You received this message because you are subscribed to the Google 
>> Groups "golang-nuts" group. 
>> >>> To unsubscribe from this group and stop receiving emails from it, 
>> send an email to golan...@googlegroups.com. 
>> >>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/2cbf0d51-0b36-4c37-a324-8a343193769a%40googlegroups.com.
>>  
>>
>> >>> 
>> >>> -- 
>> >>> You received this message because you are subscribed to the Google 
>> Groups "golang-nuts" group. 
>> >>> To unsubscribe from this group and stop receiving emails from it, 
>> send an email to golan...@googlegroups.com. 
>> >>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/A56FB7F5-95D2-4B0E-B90C-B335B8714009%40ix.netcom.com.
>>  
>>
>> >> 
>> >> -- 
>> >> You received this message because you are subscribed to the Google 
>> Groups "golang-nuts" group. 
>> >> To unsubscribe from this group and stop receiving emails from it,

[go-nuts] Re: Go 1.14 Release Candidate 1 is released

[Kaveh Shahbazian]

>
> "Goroutines are now asynchronously preemptible."

Very nice!

"A consequence of the implementation of preemption is that on Unix systems, 
> including Linux and macOS systems, programs built with Go 1.14 will receive 
> more signals than programs built with earlier releases. This means that 
> programs that use packages like syscall 
>  or golang.org/x/sys/unix 
>  will see more slow system calls 
> fail with EINTR errors. Those programs will have to handle those errors 
> in some way, most likely looping to try the system call again."
>
How does this affects projects like Docker? 

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/d3b3104e-42fa-4ab6-abc6-23b21fd3a333%40googlegroups.com.

Re: [go-nuts] runtime/cgo: pthread_create failed: Resource temporarily unavailable

[Robert Engels]

For threads just use the os command top or ps. 

For go routines I would use the net based pprof

> On Feb 5, 2020, at 3:20 PM, jnaderl...@gmail.com wrote:
> 
> 
> How can I debug the number of threads and go routines running and checking if 
> the Wait()'s finish? Because I believe that may be problem, that they hang.
> 
> And when you say append the output, are you saying make a go routine to write 
> to the headers?  If you have an example I would appreciate it
> 
>> On Wednesday, February 5, 2020 at 8:47:05 AM UTC-7, Robert Engels wrote:
>> I think your problem may be 
>> 
>> "Depending on the HTTP protocol version and the client, calling 
>> // Write or WriteHeader may prevent future reads on the 
>> // Request.Body. For HTTP/1.x requests, handlers should read any 
>> // needed request body data before writing the response. Once the 
>> // headers have been flushed (due to either an explicit Flusher.Flush 
>> // call or writing enough data to trigger a flush), the request body 
>> // may be unavailable. For HTTP/2 requests, the Go HTTP server permits 
>> // handlers to continue to read the request body while concurrently 
>> // writing the response. However, such behavior may not be supported 
>> // by all HTTP/2 clients. Handlers should read before writing if 
>> // possible to maximize compatibility." 
>> 
>> You may need to write the ResponseHeader as a final stage and append the 
>> output - if you write the header you may be hanging the input stages. If the 
>> input stage hangs (you tube download hangs, etc.), the whole process is 
>> going to hang. 
>> 
>> Did you debug the number of threads and go routines the process has while 
>> running? I am betting these are continually increasing. (Another check would 
>> be that all Waits() complete). 
>> 
>> Finally, I would use a CommandContext with a Deadline to ensure stragglers 
>> are cleaned-up. 
>> 
>> 
>> 
>> -Original Message- 
>> >From: Ian Lance Taylor  
>> >Sent: Feb 5, 2020 8:32 AM 
>> >To: jnade...@gmail.com 
>> >Cc: golang-nuts  
>> >Subject: Re: [go-nuts] runtime/cgo: pthread_create failed: Resource 
>> >temporarily unavailable 
>> > 
>> >On Tue, Feb 4, 2020 at 11:22 PM  wrote: 
>> >> 
>> >> I don't think that is the issue.  I have tried it on a few different 
>> >> servers.  Most recent one with 100 gig's of ram and 50 cores.  The load 
>> >> average never goes above 9, but the ram slowly but surely on htop starts 
>> >> to go up.  The go binary ends up climbing slowly in it's ram use over 
>> >> time, then after a hour or so, it reaches around 30 gigs of ram and then 
>> >> crashes, and restarts. 
>> >> 
>> >> I have it under supervisor. 
>> > 
>> >That is not inconsistent with Robert's suggestion.  If you are 
>> >starting C threads that don't do any work but never exit, that is 
>> >exactly what you would see. 
>> > 
>> >It's not the only possible cause of this.  There could also be a space 
>> >leak, either in C code with memory that is malloced but never freed, 
>> >or in Go code with memory that something keeps a permanent reference 
>> >to. 
>> > 
>> >Ian 
>> > 
>> > 
>> >> On Tuesday, February 4, 2020 at 2:00:55 PM UTC-7, Robert Engels wrote: 
>> >>> 
>> >>> Are you certain you are not just starting too many processes? Ie use a 
>> >>> “worker pool” so you have at most N conversions happening at the same 
>> >>> time. 
>> >>> 
>> >>> On Feb 4, 2020, at 2:34 PM, Robert Engels  wrote: 
>> >>> 
>> >>>  
>> >>> I will take a more in-depth look this evening. 
>> >>> 
>> >>> On Feb 4, 2020, at 2:19 PM, jnade...@gmail.com wrote: 
>> >>> 
>> >>>  
>> >>> I also thought the Wait() took care of closing the file descriptors? Are 
>> >>> you saying I should add a pipe3.Close()? Or a youtube.Close()? 
>> >>> 
>> >>> -- 
>> >>> You received this message because you are subscribed to the Google 
>> >>> Groups "golang-nuts" group. 
>> >>> To unsubscribe from this group and stop receiving emails from it, send 
>> >>> an email to golan...@googlegroups.com. 
>> >>> To view this discussion on the web visit 
>> >>> https://groups.google.com/d/msgid/golang-nuts/2cbf0d51-0b36-4c37-a324-8a343193769a%40googlegroups.com.
>> >>>  
>> >>> 
>> >>> -- 
>> >>> You received this message because you are subscribed to the Google 
>> >>> Groups "golang-nuts" group. 
>> >>> To unsubscribe from this group and stop receiving emails from it, send 
>> >>> an email to golan...@googlegroups.com. 
>> >>> To view this discussion on the web visit 
>> >>> https://groups.google.com/d/msgid/golang-nuts/A56FB7F5-95D2-4B0E-B90C-B335B8714009%40ix.netcom.com.
>> >>>  
>> >> 
>> >> -- 
>> >> You received this message because you are subscribed to the Google Groups 
>> >> "golang-nuts" group. 
>> >> To unsubscribe from this group and stop receiving emails from it, send an 
>> >> email to golan...@googlegroups.com. 
>> >> To view this discussion on the web visit 
>> >> https://groups.google.com/d/msgid/golang-nuts/0e7e67a6-6ad4-418

Re: [go-nuts] runtime/cgo: pthread_create failed: Resource temporarily unavailable

[jnaderlowe]

How can I debug the number of threads and go routines running and checking 
if the Wait()'s finish? Because I believe that may be problem, that they 
hang.

And when you say append the output, are you saying make a go routine to 
write to the headers?  If you have an example I would appreciate it

On Wednesday, February 5, 2020 at 8:47:05 AM UTC-7, Robert Engels wrote:
>
> I think your problem may be 
>
> "Depending on the HTTP protocol version and the client, calling 
> // Write or WriteHeader may prevent future reads on the 
> // Request.Body. For HTTP/1.x requests, handlers should read any 
> // needed request body data before writing the response. Once the 
> // headers have been flushed (due to either an explicit Flusher.Flush 
> // call or writing enough data to trigger a flush), the request body 
> // may be unavailable. For HTTP/2 requests, the Go HTTP server permits 
> // handlers to continue to read the request body while concurrently 
> // writing the response. However, such behavior may not be supported 
> // by all HTTP/2 clients. Handlers should read before writing if 
> // possible to maximize compatibility." 
>
> You may need to write the ResponseHeader as a final stage and append the 
> output - if you write the header you may be hanging the input stages. If 
> the input stage hangs (you tube download hangs, etc.), the whole process is 
> going to hang. 
>
> Did you debug the number of threads and go routines the process has while 
> running? I am betting these are continually increasing. (Another check 
> would be that all Waits() complete). 
>
> Finally, I would use a CommandContext with a Deadline to ensure stragglers 
> are cleaned-up. 
>
>
>
> -Original Message- 
> >From: Ian Lance Taylor > 
> >Sent: Feb 5, 2020 8:32 AM 
> >To: jnade...@gmail.com  
> >Cc: golang-nuts > 
> >Subject: Re: [go-nuts] runtime/cgo: pthread_create failed: Resource 
> temporarily unavailable 
> > 
> >On Tue, Feb 4, 2020 at 11:22 PM > 
> wrote: 
> >> 
> >> I don't think that is the issue.  I have tried it on a few different 
> servers.  Most recent one with 100 gig's of ram and 50 cores.  The load 
> average never goes above 9, but the ram slowly but surely on htop starts to 
> go up.  The go binary ends up climbing slowly in it's ram use over time, 
> then after a hour or so, it reaches around 30 gigs of ram and then crashes, 
> and restarts. 
> >> 
> >> I have it under supervisor. 
> > 
> >That is not inconsistent with Robert's suggestion.  If you are 
> >starting C threads that don't do any work but never exit, that is 
> >exactly what you would see. 
> > 
> >It's not the only possible cause of this.  There could also be a space 
> >leak, either in C code with memory that is malloced but never freed, 
> >or in Go code with memory that something keeps a permanent reference 
> >to. 
> > 
> >Ian 
> > 
> > 
> >> On Tuesday, February 4, 2020 at 2:00:55 PM UTC-7, Robert Engels wrote: 
> >>> 
> >>> Are you certain you are not just starting too many processes? Ie use a 
> “worker pool” so you have at most N conversions happening at the same time. 
> >>> 
> >>> On Feb 4, 2020, at 2:34 PM, Robert Engels  
> wrote: 
> >>> 
> >>>  
> >>> I will take a more in-depth look this evening. 
> >>> 
> >>> On Feb 4, 2020, at 2:19 PM, jnade...@gmail.com wrote: 
> >>> 
> >>>  
> >>> I also thought the Wait() took care of closing the file descriptors? 
> Are you saying I should add a pipe3.Close()? Or a youtube.Close()? 
> >>> 
> >>> -- 
> >>> You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group. 
> >>> To unsubscribe from this group and stop receiving emails from it, send 
> an email to golan...@googlegroups.com. 
> >>> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/2cbf0d51-0b36-4c37-a324-8a343193769a%40googlegroups.com.
>  
>
> >>> 
> >>> -- 
> >>> You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group. 
> >>> To unsubscribe from this group and stop receiving emails from it, send 
> an email to golan...@googlegroups.com. 
> >>> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/A56FB7F5-95D2-4B0E-B90C-B335B8714009%40ix.netcom.com.
>  
>
> >> 
> >> -- 
> >> You received this message because you are subscribed to the Google 
> Groups "golang-nuts" group. 
> >> To unsubscribe from this group and stop receiving emails from it, send 
> an email to golan...@googlegroups.com . 
> >> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/0e7e67a6-6ad4-4187-9f15-0d9f278b55a2%40googlegroups.com.
>  
>
> > 
> >-- 
> >You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group. 
> >To unsubscribe from this group and stop receiving emails from it, send an 
> email to golan...@googlegroups.com . 
> >To view this discussion on the web visit 
> https://groups.

Re: [go-nuts] Seeking new custodian for godoc.org

[Russ Cox]

> [go-nuts] Seeking new custodian for godoc.org

The Go team is the custodian for godoc.org and we are not seeking a new one.

> Legal advice will apparently force the Go team to close godoc.org [1]

This is a grotesque misrepresentation of what I wrote. This is what I wrote:

*# Why is there a new package docs site at all? Why not update*
*godoc.org  in place?*

With the introduction of modules and the notion of multiple versions
of a package, we knew we had to update the godoc.org experience. After
a hard look, it seemed worth starting anew, especially since the
godoc.org server design, with its single-VM database, had been
starting to show its age. In addition to the modules work, there are
other things we're addressing, such as accessibility and overall
scalability of the service.

As a side note, there's almost nothing in the Go distribution that has
survived eight years without being redone. The compiler, the
assembler, the linker, the go command itself, most of the standard
library: all of them have been massively overhauled one or more times
since the start of Go. That's how we take what we learn and make
things better.

This kind of rewrite always involves a transition period in which the
old version is still the workhorse that most people use and the new
version has a new name for early adopters to test and find bugs in.


The license detection discussion is a sideshow.

Please stop spreading FUD.

Best,
Russ

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAA8EjDQk09JhsDeY-eVrTWE9dGShGYk_3aMX6zQnq9%2Bskc2Vaw%40mail.gmail.com.

[go-nuts] Go 1.14 Release Candidate 1 is released

[Carlos Amedee]

Hello gophers,

We have just released go1.14rc1, a release candidate version of Go 1.14.
It is cut from release-branch.go1.14 at the revision tagged go1.14rc1.

Please try your production load tests and unit tests with the new version.
Your help testing these pre-release versions is invaluable.

Report any problems using the issue tracker:
https://golang.org/issue/new

If you have Go installed already, the easiest way to try go1.14rc1
is by using the go command:
$ go get golang.org/dl/go1.14rc1
$ go1.14rc1 download

You can download binary and source distributions from the usual place:
https://golang.org/dl/#go1.14rc1

To find out what has changed in Go 1.14, read the draft release notes:
https://tip.golang.org/doc/go1.14

Cheers,
Carlos and Dmitri on behalf of the Go team

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAD7Aq3QARXvqyTtDZHtMM3%2B7YqrRzwNSvYa4VupM%3Dh-SpgNdzA%40mail.gmail.com.

Re: [go-nuts] Compilation fails when adding option "-trimpath", because change a dependency to require gcc

[Ian Lance Taylor]

On Wed, Feb 5, 2020 at 9:33 AM Florent Bonneau
 wrote:
>
> Using docker image golang:1.13-alpine to build our code :
> - Compiling with "go install -v oursoftware/..." our build is sucessfull
> - Compiling with "go install -v -trimpath oursoftware/..." fails with 
> following error :
>
>
> ...
>
> runtime
> runtime/cgo
> vendor/golang.org/x/crypto/curve25519
> # runtime/cgo
> exec: "gcc": executable file not found in $PATH
> encoding
> ...
>
> We can for sure install the build tools upon the base image to solve the 
> issue, but it doesn't seem normal that the "-trimpath" option had this 
> side-effect ?
>
> Can someone explain why it happens ?

I haven't verified this, so this is just a guess, but perhaps without
-trimpath the go tool is using the prebuilt runtime/cgo.a, which was
built on a system with a C compiler.  When you add -trimpath, the
runtime/cgo package needs to be rebuilt in order to trim the paths.
Unfortunately, that rebuild fails, as the system you are building on
does not have a C compiler.

Ian

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcXzreOoLtEa5bwky_zq3wza45iSZxYbcD9X5GN%3DYBxsug%40mail.gmail.com.

[go-nuts] Compilation fails when adding option "-trimpath", because change a dependency to require gcc

[Florent Bonneau]

Using docker image golang:1.13-alpine to build our code :
- Compiling with "go install -v oursoftware/..." our build is *sucessfull*
- Compiling with "go install -v *-trimpath* oursoftware/..." *fails* with 
following error :


...

runtime
runtime/cgo
vendor/golang.org/x/crypto/curve25519
# runtime/cgo
exec: "gcc": executable file not found in $PATH
encoding
...

We can for sure install the build tools upon the base image to solve the issue, 
but it doesn't seem normal that the "-trimpath" option had this side-effect ? 

Can someone explain why it happens ?

The content of our go.mod if it helps : 

go 1.13

require (
   github.com/BurntSushi/toml v0.3.1
   github.com/davecgh/go-spew v1.1.1
   github.com/gin-gonic/gin v1.4.0
   github.com/go-openapi/errors v0.19.2
   github.com/go-openapi/strfmt v0.19.3
   github.com/go-openapi/swag v0.19.5
   github.com/go-openapi/validate v0.19.4
   github.com/gofrs/uuid v3.2.0+incompatible
   github.com/google/uuid v1.1.1
   github.com/nicksnyder/go-i18n/v2 v2.0.3
   github.com/prometheus/client_golang v1.2.1
   github.com/rogpeppe/go-charset v0.0.0-20190617161244-0dc95cdf6f31
   github.com/spf13/viper v1.4.0
   github.com/stretchr/testify v1.4.0
   go.mongodb.org/mongo-driver v1.1.2 // indirect
   go.uber.org/atomic v1.4.0
   go.uber.org/zap v1.11.0
   golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be
   golang.org/x/text v0.3.2
   gopkg.in/yaml.v2 v2.2.4
)


Thanks !

Florent Bonneau


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2df7307a-233c-42ab-80c8-08a006707418%40googlegroups.com.

Re: [go-nuts] runtime/cgo: pthread_create failed: Resource temporarily unavailable

[Robert Engels]

I think your problem may be 

"Depending on the HTTP protocol version and the client, calling
// Write or WriteHeader may prevent future reads on the
// Request.Body. For HTTP/1.x requests, handlers should read any
// needed request body data before writing the response. Once the
// headers have been flushed (due to either an explicit Flusher.Flush
// call or writing enough data to trigger a flush), the request body
// may be unavailable. For HTTP/2 requests, the Go HTTP server permits
// handlers to continue to read the request body while concurrently
// writing the response. However, such behavior may not be supported
// by all HTTP/2 clients. Handlers should read before writing if
// possible to maximize compatibility."

You may need to write the ResponseHeader as a final stage and append the output 
- if you write the header you may be hanging the input stages. If the input 
stage hangs (you tube download hangs, etc.), the whole process is going to hang.

Did you debug the number of threads and go routines the process has while 
running? I am betting these are continually increasing. (Another check would be 
that all Waits() complete).

Finally, I would use a CommandContext with a Deadline to ensure stragglers are 
cleaned-up.



-Original Message-
>From: Ian Lance Taylor 
>Sent: Feb 5, 2020 8:32 AM
>To: jnaderl...@gmail.com
>Cc: golang-nuts 
>Subject: Re: [go-nuts] runtime/cgo: pthread_create failed: Resource 
>temporarily unavailable
>
>On Tue, Feb 4, 2020 at 11:22 PM  wrote:
>>
>> I don't think that is the issue.  I have tried it on a few different 
>> servers.  Most recent one with 100 gig's of ram and 50 cores.  The load 
>> average never goes above 9, but the ram slowly but surely on htop starts to 
>> go up.  The go binary ends up climbing slowly in it's ram use over time, 
>> then after a hour or so, it reaches around 30 gigs of ram and then crashes, 
>> and restarts.
>>
>> I have it under supervisor.
>
>That is not inconsistent with Robert's suggestion.  If you are
>starting C threads that don't do any work but never exit, that is
>exactly what you would see.
>
>It's not the only possible cause of this.  There could also be a space
>leak, either in C code with memory that is malloced but never freed,
>or in Go code with memory that something keeps a permanent reference
>to.
>
>Ian
>
>
>> On Tuesday, February 4, 2020 at 2:00:55 PM UTC-7, Robert Engels wrote:
>>>
>>> Are you certain you are not just starting too many processes? Ie use a 
>>> “worker pool” so you have at most N conversions happening at the same time.
>>>
>>> On Feb 4, 2020, at 2:34 PM, Robert Engels  wrote:
>>>
>>> 
>>> I will take a more in-depth look this evening.
>>>
>>> On Feb 4, 2020, at 2:19 PM, jnade...@gmail.com wrote:
>>>
>>> 
>>> I also thought the Wait() took care of closing the file descriptors? Are 
>>> you saying I should add a pipe3.Close()? Or a youtube.Close()?
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golan...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/2cbf0d51-0b36-4c37-a324-8a343193769a%40googlegroups.com.
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golan...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/A56FB7F5-95D2-4B0E-B90C-B335B8714009%40ix.netcom.com.
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golang-nuts+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/0e7e67a6-6ad4-4187-9f15-0d9f278b55a2%40googlegroups.com.
>
>-- 
>You received this message because you are subscribed to the Google Groups 
>"golang-nuts" group.
>To unsubscribe from this group and stop receiving emails from it, send an 
>email to golang-nuts+unsubscr...@googlegroups.com.
>To view this discussion on the web visit 
>https://groups.google.com/d/msgid/golang-nuts/CAOyqgcVeSsLHe92m8eQue4Lfyk3uf%2B%3D94Qy7ZapHdjsgA1RjhA%40mail.gmail.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/2131046265.665.1580917592645%40wamui-scooby.atl.sa.earthlink.net.

Re: [go-nuts] runtime/cgo: pthread_create failed: Resource temporarily unavailable

[Ian Lance Taylor]

On Tue, Feb 4, 2020 at 11:22 PM  wrote:
>
> I don't think that is the issue.  I have tried it on a few different servers. 
>  Most recent one with 100 gig's of ram and 50 cores.  The load average never 
> goes above 9, but the ram slowly but surely on htop starts to go up.  The go 
> binary ends up climbing slowly in it's ram use over time, then after a hour 
> or so, it reaches around 30 gigs of ram and then crashes, and restarts.
>
> I have it under supervisor.

That is not inconsistent with Robert's suggestion.  If you are
starting C threads that don't do any work but never exit, that is
exactly what you would see.

It's not the only possible cause of this.  There could also be a space
leak, either in C code with memory that is malloced but never freed,
or in Go code with memory that something keeps a permanent reference
to.

Ian


> On Tuesday, February 4, 2020 at 2:00:55 PM UTC-7, Robert Engels wrote:
>>
>> Are you certain you are not just starting too many processes? Ie use a 
>> “worker pool” so you have at most N conversions happening at the same time.
>>
>> On Feb 4, 2020, at 2:34 PM, Robert Engels  wrote:
>>
>> 
>> I will take a more in-depth look this evening.
>>
>> On Feb 4, 2020, at 2:19 PM, jnade...@gmail.com wrote:
>>
>> 
>> I also thought the Wait() took care of closing the file descriptors? Are you 
>> saying I should add a pipe3.Close()? Or a youtube.Close()?
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golan...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/2cbf0d51-0b36-4c37-a324-8a343193769a%40googlegroups.com.
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "golang-nuts" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to golan...@googlegroups.com.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/golang-nuts/A56FB7F5-95D2-4B0E-B90C-B335B8714009%40ix.netcom.com.
>
> --
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/0e7e67a6-6ad4-4187-9f15-0d9f278b55a2%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/CAOyqgcVeSsLHe92m8eQue4Lfyk3uf%2B%3D94Qy7ZapHdjsgA1RjhA%40mail.gmail.com.

[go-nuts] How to push/append object into mongodb array

[abraham . afriyie]

Hi,


I have the JSON below in mongodb collection

{
  "Id":"3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "allowedNssaiList": [
{
  "allowedSnssaiList": [
{
  "allowedSnssai": {
"sst": 1,
"sd": "2"
  },
  "IMSI": "2443401",
  "tac": "3022"
}
  ],
  "accessType": "3GPP_ACCESS"
}
  ]}
 

I would like to append to the sub array allowedSnssaiList with the object


{
  "allowedSnssai": {
"sst": 1,
"sd": "2"
  },
  "IMSI": "2443401",
  "tac": "3022"
}

I have tried with the following


selector := bson.M{"_id": "3fa85f64-5717-4562-b3fc-2c963f66afa6"}PushToArray := 
bson.M{"$addToSet": bson.M{"allowedSnssaiList": bson.M{"allowedSnssai": 
bson.M{"sst": 3,"sd": "4"}, "IMSI": "2445104","tac": "3022"}}}

err := db.C(COLLECTION).Update(selector, PushToArray)

but when i push, it does not append well, i get 

{
  "_id":"3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "allowedNssaiList": [
{
  "allowedSnssaiList": [
{
  "allowedSnssai": {
"sst": 1,
"sd": "2"
  },
  "IMSI": "2443401",
  "tac": "3022"
}
  ],
  "accessType": "3GPP_ACCESS"
}
  ],
  "allowedSnssaiList" : [
{
"allowedSnssai" : {
"sst" : 1,
"sd" : "5"
},
"IMSI" : "2443405",
"tac" : "3022"
}
]}


but i want the result or append as



{
  "_id":"3fa85f64-5717-4562-b3fc-2c963f66afa6",
  "allowedNssaiList": [
{
  "allowedSnssaiList": [
{
  "allowedSnssai": {
"sst": 1,
"sd": "2"
  },
  "IMSI": "2443401",
  "tac": "3022"
},
{
"allowedSnssai" : {
"sst" : 1,
"sd" : "5"
},
"IMSI" : "2443405",
"tac" : "3022"
}
  ],
  "accessType": "3GPP_ACCESS"
}
  ]}

changing to

PushToArray := bson.M{"$addToSet": 
bson.M{"allowedNssaiList[0].allowedSnssaiList[0]": bson.M{"allowedSnssai": 
bson.M{"sst": 3,"sd": "4"}, "IMSI": "2445104","tac": "3022"}}}

still did not work, any help?

Abraham


-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/3f8eb588-59a4-4501-bce0-3bca0bfb3795%40googlegroups.com.

[go-nuts] Correct way to handle errors from net.http?

[Rio]

Background: I was writing an HTTP proxy using net.http which supports both 
HTTP/1 and HTTP/2. I need to detect errors and do correct things according 
to RFC (https://http2.github.io/http2-spec/#CONNECT). 

It turns out that net.http has its own bundled copy of x/net/http2 
(https://golang.org/src/net/http/h2_bundle.go) with nothing exported. So I 
cannot do things like

var h2err http2.StreamError
if errors.As(err, &h2err) { // dealing with errors

it will never match because the actual error is `http.http2StreamError` 
which is not exported. In this case, what's the correct way to detect http2 
specific errors?

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/46a9e5fb-4902-4c36-8dc7-4efb069b0515%40googlegroups.com.

Re: [go-nuts] go test -c in go 1.13 does not include -test.timeout flag in binary?

[Craig Rodrigues]

On Monday, February 3, 2020 at 11:37:08 AM UTC-8, Ian Lance Taylor wrote:
>
> On Mon, Feb 3, 2020 at 8:13 AM Craig Rodrigues  > wrote: 
> > 
> > Regarding the fact that I should not call flag.Parse() from inside an 
> init() method in my_test.go file, 
> > if a note for this is not added to the documentation at 
> https://golang.org/pkg/flag/#Parse , 
> > how would an end user know about this?  This detail is buried deep, and 
> the only way to 
> > understand this problem was to read the release notes for golang 1.13 
> and to ask on this mailing list. 
>
> The docs for flag.Parse already say "Must be called after all flags 
> are defined and before flags are accessed by the program."  I think 
> that is precise and accurate.  The testing package defines flags, so 
> you have to call flag.Parse after the testing package defines those 
> flags.  When you call flag.Parse in an init function, you are making 
> an assumption about how the testing package defines the flags, an 
> assumption that happened to work before the 1.13 release and happened 
> to fail after the 1.13 release. 
>
> I personally don't think it is appropriate to add docs for flag.Parse 
> that say "Note that the testing package defines flags after init 
> functions are run, so in a test don't call flag.Parse in an init 
> function."  That makes the docs longer and somewhat harder to 
> understand for an unusual situation that I hope is already somewhat 
> clear. 
>


I did some more web searching and found that someone else ran across a 
similar issue:
https://www.bountysource.com/issues/79987300-flag-provided-but-not-defined-test-timeout-go-1-13
 

where antoineco recommended replacing:

func init() {
flag.Parse()
}

TestXYZ(t *testing.T) {}


with


func TestMain(m *testing.M) {
flag.Parse()
os.Exit(m.Run())
}

TestXYZ(t *testing.T) {}





I searched further and saw that the documentation for TestMain at 
https://golang.org/pkg/testing/#hdr-Main
explicitly mentions that calling flag.Parse() should be done in TestMain.  
So I think that is a fair place to

document this.

I guess I was unlucky enough to inherit a codebase where calling 
flag.Parse() inside init() "just worked" before. :(

--

Craig

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/0b64e4ff-1f8e-47d6-acae-421c71f62da7%40googlegroups.com.