I can't see how a large number of waiting goroutines would cause an increase in the number of OS threads, which was the OP's original problem (hitting the 10,000 thread limit)
What the OP is implying - but we have not seen good evidence for yet - is that in some circumstances a non-blocking connect() becomes blocking. But even then, how would Go know to allocate an OS thread for it *before* calling it?? I suspect something else is going on. Does the original program with the 10,000 thread problem make any use of external C code, directly or indirectly? On Saturday 10 February 2024 at 05:17:21 UTC Kurtis Rader wrote: > On Fri, Feb 9, 2024 at 2:10 PM Ian Lance Taylor <ia...@golang.org> wrote: > >> On Fri, Feb 9, 2024 at 11:57 AM Kurtis Rader <kra...@skepticism.us> >> wrote: >> > >> > The connect() syscall is normally blocking. It doesn't return until the >> connection is established or an error occurs. It can be made non-blocking >> by putting the file-descriptor into non-blocking mode before the connect() >> call. However, that then requires either an async callback or another >> syscall to check whether the connection was established or an error >> occurred. Neither approach is idiomatic Go. >> >> That is true, but the Go standard library's net package does use >> non-blocking calls to connect internally when implementing net.Dial >> and friends. >> >> I don't have any problem running the original program on Linux 6.5.13. >> > > Yes, I realized after my previous reply that Go could obviously use > non-blocking connect() calls coupled with select(), poll(), or similar > mechanisms to wakeup a goroutine blocked waiting for a net.Dial() (or > equivalent) connection to complete in order to minimize the number of OS > threads required to handle in-flight network connections. Without requiring > exposing an async callback or a mechanism to explicitly start a connection > and at a later time test whether it has been established or failed. > > What might be happening for the O.P. is that the systems they are > connecting to are not explicitly accepting or rejecting the connections in > a timely manner. Thus causing a huge number of goroutines blocked waiting > for the net.Dial() to complete. The systems they are connecting to may be > simply discarding the TCP SYN packets due to firewall rules or something > similar. This is something that is going to be hard for the Go community to > provide help since it is fundamentally not an issue with Go itself. > > -- > Kurtis Rader > Caretaker of the exceptional canines Junior and Hank > -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/7a585302-ba38-4822-bdd7-76f9076ccd91n%40googlegroups.com.
