Hi, I have a number of slow tasks that I want to run concurrently across runtime.NumCPU() workers in a single process. The tasks have a specific input order, but they are completely independent of each other and can execute in any order. I would like to print the output of each task in the same order as the input order of tasks.
This can be implemented by including each task's index in the input order as it is distributed via a channel to the workers, and the final collection of results assembled using these task indexes before the results are printed. Assumptions: - Small number of tasks (~10,000 max), i.e. this easily fits in memory. - Single Go process, i.e. I don't want/need a distributed system. This feels like it should be common problem and there's probably either a library or a standard Go pattern out there which can do it. My web search skills didn't find such a library though. Do you know of one? Cheers, Tom Background info to avoid the XY problem <http://xyproblem.info/>: this is to make chezmoi <https://github.com/twpayne/chezmoi> run faster. I want to run the doctor checks <https://github.com/twpayne/chezmoi/blob/ed27b49f9ca4cd3662e6a59908dee24b0d295b79/cmd/doctor.go#L102-L163> (basically os.Exec'ing a whole load of binaries to get their versions) concurrently in the short term. In the long term I want to make chezmoi's apply concurrent, so it runs faster too. In the first case, the order requirement is because I want all users to see the output in the same order so that it's easy to compare. In the second case, the order requirement comes because I need to ensure that parent directories are in the correct state before checking their children. -- You received this message because you are subscribed to the Google Groups "golang-nuts" group. To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
