Spinning up a Go routine when for each piece of work may be idiomatic but it is 
far from the best practice for many situations - mainly because of cpu cache 
invalidation. Most HPC systems create worker pools by type and then isolate 
them to cpus/cores - something you can’t do in Go.

I believe there are outstanding proposals for grouping routines and core 
affinity. 

> On Dec 28, 2019, at 7:02 AM, Agniva De Sarker <agniva.quicksil...@gmail.com> 
> wrote:
> 
> 
> > (the task was to limit the whole thing to about 10% of cores)
> 
> I still don't think you needed a worker pool here. Like OP mentioned above, 
> you could just limit the number of goroutines executed to 10% of total cores.
> 
> 
>> On Saturday, 28 December 2019 18:02:08 UTC+5:30, Chris Burkert wrote:
>> There are Pros and Cons for everything in life. Some time ago I wrote a 
>> database tool which does something per table where the runtime largely 
>> depends on the table size. I started with one goroutine per table because it 
>> was easy. But that put a high load on the database (the task was to limit 
>> the whole thing to about 10% of cores) with out of memory situations. So I 
>> switched to a worker pool which solved that. However now the overall runtime 
>> was unpredictable. When the large tables were handled at the end their 
>> runtime defined the overall runtime. So I to feed the pool with large tables 
>> first. This again lead to out of memory situations so I reordered the tables 
>> such that large tables are mixed with smaller tables.
>> 
>> Brian Candler <b.ca...@pobox.com> schrieb am Sa. 28. Dez. 2019 um 11:09:
>>>> On Friday, 27 December 2019 16:30:48 UTC, Bruno Albuquerque wrote:
>>>> This might be useful too you, in any case:
>>>> 
>>>> https://git.bug-br.org.br/bga/workerpool
>>>> 
>>> 
>>> I think the point from Bryan Mills' video is, "worker pool" is something of 
>>> an anti-pattern in go.  goroutines are so cheap that you might as well 
>>> start a goroutine for each piece of work you have to do, and let it 
>>> terminate when that piece of work is done.
>>> 
>>> Apart from the startup cost, the other reason for having a "worker pool" is 
>>> to limit the number of concurrent tasks being executed, and there are 
>>> better ways of doing that in go (also shown in the video).
>>> -- 
>>> You received this message because you are subscribed to the Google Groups 
>>> "golang-nuts" group.
>>> To unsubscribe from this group and stop receiving emails from it, send an 
>>> email to golan...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/golang-nuts/f840beee-748f-42b6-809f-4c7505208aee%40googlegroups.com.
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "golang-nuts" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to golang-nuts+unsubscr...@googlegroups.com.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/golang-nuts/ecddafbc-eb73-45e1-8a5a-f738e88c6821%40googlegroups.com.

-- 
You received this message because you are subscribed to the Google Groups 
"golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to golang-nuts+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/golang-nuts/8C1153D5-25E7-4A06-8626-1FE61D9015D5%40ix.netcom.com.

Reply via email to