I think the reason you haven’t found a library, is that given the specifications, it is probably less than 100 lines of Go code to accomplish.
The important elements lacking in your specification though are “how are the tasks submitted” (e.g. http? in code ?, etc.) , , and “how is the output presented” (e.g. to the console, each task decides, etc .) > On Jan 25, 2019, at 1:04 PM, twpa...@gmail.com wrote: > > Hi, > > I have a number of slow tasks that I want to run concurrently across > runtime.NumCPU() workers in a single process. The tasks have a specific input > order, but they are completely independent of each other and can execute in > any order. I would like to print the output of each task in the same order as > the input order of tasks. > > This can be implemented by including each task's index in the input order as > it is distributed via a channel to the workers, and the final collection of > results assembled using these task indexes before the results are printed. > > Assumptions: > - Small number of tasks (~10,000 max), i.e. this easily fits in memory. > - Single Go process, i.e. I don't want/need a distributed system. > > This feels like it should be common problem and there's probably either a > library or a standard Go pattern out there which can do it. My web search > skills didn't find such a library though. Do you know of one? > > Cheers, > Tom > > > Background info to avoid the XY problem <http://xyproblem.info/>: this is to > make chezmoi <https://github.com/twpayne/chezmoi> run faster. I want to run > the doctor checks > <https://github.com/twpayne/chezmoi/blob/ed27b49f9ca4cd3662e6a59908dee24b0d295b79/cmd/doctor.go#L102-L163> > (basically os.Exec'ing a whole load of binaries to get their versions) > concurrently in the short term. In the long term I want to make chezmoi's > apply concurrent, so it runs faster too. In the first case, the order > requirement is because I want all users to see the output in the same order > so that it's easy to compare. In the second case, the order requirement comes > because I need to ensure that parent directories are in the correct state > before checking their children. > > -- > You received this message because you are subscribed to the Google Groups > "golang-nuts" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to golang-nuts+unsubscr...@googlegroups.com > <mailto:golang-nuts+unsubscr...@googlegroups.com>. > For more options, visit https://groups.google.com/d/optout > <https://groups.google.com/d/optout>. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
